Show game activity to friends only

I like the idea of having people I've added as friends see what game I'm playing and for how long. I don't like the idea of letting everyone in any server I've joined being able to see and track what game I'm playing and for how long.