Leave servers that are temporarily unavailable

One of the servers I am in is most of the time unavailable as it is experiencing temporary outage and it is not possible to leave the server while it is unavailable. Having the option to leave them while they are still unavailable would be great instead of waiting them to be available again just to leave them.