Get banned from the guild when change phone number

After I change the phone number in my account, I got banned from a server without no reason.
Then I changed to another phone number, I can access that server again.
I asked the admin of the server and they said there are no alts
Can anyone help?