Favorite Emotes Being Frozen

I’ve found that if I’ve favorited an emote in a server that is a level 1 or higher and that server then loses their level, I obviously lose access to use that emote. But I also don’t regain that spot in my favorites list. So I now am still stuck with a full favorites list and am unable to unfavorite the emote that is unusable to me. I can’t seem to find any solution at this time.