How do you unfollow a channel?

I have a real problem, in some channels it says "Follow to get this channel's updates in your own server." So I do this for a couple of servers' channels so I can keep up to date. But I've left a server but I keep getting the updates from that other server's channel. How do I stop it? Going back to the channel, it just says “Follow.” I can't see the option to “unfollow”. Help is appreciated :)