Signal is only easier because it entirely ignores logging in on multiple devices. Maybe for some it is ok, but for me this is a huge dealbreaker, not an advantage.
If you dont set up key backups (an optional feature), its the same thing: with Signal, if you delete the app or lose your phone, all your messages are gone, along with your contacts that werent saved in your phone contacts and uploaded to a cloud service. If you use Matrix as you do with Signal, it works the same: you delete it, messages are gone. This is the default. But, you have the option to keep your messages.

Identity server? You dont have to use that, and I don't either. You are not obliged to set up being discovered by outside identifiers. Like I don't want people to find me by my phone number, as I don't want to use my phone number, for anything, at all, and so I didn't do that.
I see that on Signal, you always find people by their phone number, which you are required to hand in. On Matrix, you find people either by their handles (~username), or their phone number or email address if they have handed those in, voluntarily.
So with an indentity server you can make yourself discoverable by your phone number, and you must use one if you want that.
But I think there is a better solution (on the long term, at least): to forget about phone numbers altogether, when possible. Why would this be feasible? It is possible to store the handle in your phones contacts, with the standard "instant messenger" field. Contacts then are usually sharable in messaging apps, or with a QR code, and a lot of software generally understands this format, so you could use this to make your handle known.
By the way, identity servers and discovery is the same step, not 2 different one.