I'd say you're taking the problem backwards. Your friends and family will follow you probably because you ask them, rather than because of any intrinstic feature of the protocol.

So the real question is which one do you want to use?

In terms of server software, XMPP is lighter, easier to manage and scales better.

In terms of protocol, the Matrix protocol has an immense metadata problem. While encryption is sound, the metadata carried with every message is as good as plaintext. XMPP meanwhile provides pseudonymous rooms and minimal metadata leakage.

In terms of direction, I trust the XMPP foundation more. The Vector foundation comes from a multinational and is clearly a startup. I don't think anything good can come from the tight coupling between Vector and Matrix, no matter how much Matthew tries.

Vector also relicensed Element under a CLU (Contributor License Agreement). For me, it's a sword of Damocles.

In terms of community, it's pretty personal but I don't like the people on Matrix at all. The XMPP users seem more friendly. Your mileage may vary though. It may also not matter if you only want to talk to your already existing friends.

However, the best XMPP client is beautiful like the face of a dying man, and just as easy to use. Matrix is way ahead on client design.