Discord Overlay massively drops FPS

I get 40 - 50 FPS when enabling discord overlay, but when I disable it I get 120+ FPS when playing a game.