Removing or make setting to turn off the rightclick user > Invite to Server option

When I was clicking on this one guy the "invite to server" button make me accidentally send an invite link without realise it.