Mic doesn't work while having it on speaker.

I have had this trouble on my new phone. Everytime I put on speaker mode my mic stops working. I don't get mute because that symbol isn't up. But people can't here me. But if I'm not on speaker my mic works fine. Why is that and how can I fix it.