Server Template carries users from old server into new server

I created a template from a server I manage and used it to create another server.

Overall the process was simple and effective.

The only problem I have is that the new server has users from the old server listed as members, and I would like to delete these users (without blocking them) since they should not be there in the first place.

Is there a way of transparently deleting a user?
(Meaning the user does not get a message that they've been removed, and the user does not show in any user lists for this server.)