Group DMs should change

For all my time on discord, group DM's have always annoyed me slightly. But now i'm done with it.

My main problem with group DM's is that you have no choice but to join it. Recently my friends have made a group DM and because they know it annoys me they continue to add me back to it whenever I leave and there is no way out of it. I've even tried to block all of them but they can still add me to it. I don't understand why it is so flawed. 

I believe a good change to the group dm system is for the user to be invited to the DM so that they can accept or decline it the invite and not be forced in. Kind of like how server invites work. This is just an idea, but I believe something along these lines should happen.