Why do we pay more attention to someone’s English accent rather than what they are saying?

When an Indian speaks English with a heavy Indian accent, it becomes a marker of a lack of sophistication, and a put-on accent is not tolerated either