There are several other posts that suggest partial implementations or express general frustration with accidental actions. I simply attempt to generalize those.
Add a lock toggle that a user with channel or server management rights must pass through before making changes to a server, with a special nod to user and channel dragging and context menu options. Perhaps a clear icon producing a modal confirmation that enables actions like kick, ban, channel and user drag-drop, etc.
Make it revert on a timer, or on server focus change, or both, or provide a preference, including disabling this gating entirely, to determine the behavior.
If there's space for nitro gift icons, there's space for confirming that a user intends to carry out administrative actions.