If I understand your post correctly ...
That's like when the democrats in the U.S. of A. and elsewhere, called other people racist just because they did not agree with obama, when he was president, when the people that did not like him were not racists!
Deliberately using an incorrect, baseless, prejudicial term should be abhorred. It appears that a person using the term .........phobia is attempting their own kind of negative denigration in a malicious way. That or the person using the .......phobia word does not understand what that word means.
If you have prejudices about a group or culture, that is not in itself, a phobia!!
Call it what it is, prejudice.
Whew!!! I guess a nerve was hit. Rant over.