Server (un)Mute/Deafen

Hey there! So as a discord server moderator I had a problem with the server muting/deafening system and here's what:

-So when a user is not in the voice channel you can not give them a server (un)mute/deafen and this can be kind of frustrating. 

Would it be possible if you could maybe do this by right clicking the user in the server member list?