You don’t have to. I mean, just ignoring might not help. And does it matter what people say about you?
You need to learn how to value yourself as a unique person and how to not depend on what others say or think about you. But it is tricky: you don’t want to be an outlaw, but free.
The first step is… learning more about you. What I mean, try to understand those things that make you worry — are they real? Do they matter anything to you? Are they are valuable to you?
Usually the first step is in taking or learning control of your mind: thoughts, motivation, desires — you don’t have to eliminate them, but try to see are they part of you? When you get some experience in thoughts control you will discover an interesting thing: you are not necessarily your thoughts, so there is no need for you to worry about them, their nature and their origin; and you don’t have to worry about what do people say about you, and what words caused some of your thoughts…
