Make Apps Context Menus not rely on Send Messages permission

Currenty, disabling Send Message permission will prevent members from using Apps (Context Menus) too. I don't really think there is reason for that but there should at least be a way to make Context Menus work even if Send Messages permission is disabled.

Having Send Messages disabled and Use Application Commands perm enabled (which includes Context Menus) doesn't change anything and context menus still cannot be used.

For my specific case, I have a suggestion channel and while users cannot directly talk in the channel they can make suggestion in another bot-commands channel that will appear here. I'd like them to be able to remove their suggesiton by using context menus but I cannot do that without enabling Send Messages.