Very late to this thread. Welcome Kennedyland. You've received some great advice from some of our very wise members.
My take is this:
You've taken a stand. It's a very difficult thing to do, especially if you're the first one in your family who's ever stood up to your son. If you back down now, it will be MUCH harder to take a similar stand later. You will teach your son that, if he escalates far enough you too will back down. That's not a lesson you want to teach. You need to be true to yourself. Your daughter and other family members may come around, or they may not. But you're doing what's best for you and your wife.
I come from a family riddled with mental illness, and some very desctructive behaviour patterns. For my own self-preservation, I had to cut ties with the lot of them several years ago. I have had minimal contact with them since then, and my life is the better for it. Those family members are pushing hard for a reconciliation these days, telling me that I should "put the past behind me". What that tells me is that nothing has changed, and I would be walking right back into the dragon's lair I walked out of all those years ago. So, I hold my ground. And I like it up here on this patch of ground, where I have a clear view of the chaos I left behind me, and the green pastures I'm standing in these days.
Hang in there.
Trinity