Everyone has their skeletons, everyone has their pain and suffering. Some groups have privileges some don't, the thing is if you are a white male (as I am) we have certain extra luck that others don't have.
That doesn't mean karma and life will heap us with great situations. If we take everything that happened to you and change your identity demographics, your situation would be different.
What you can be irritated at is the gross labeling that happens and generalizations; however, active listening will help shield your inner shatter being.
So if one takes a buddha view on life (work with me on this), good and bad happens to a person. We can't control it; however, we can control how we react to it. Let the sensation flow into you and out, and observe your reactions.
So let's stop and think about this:
You were grossly assaulted, you know what it feels like to have zero control and zero chance of feeling safe.
This is a feeling some minorities feel "every" day, because of how life is for their entire life with no break. Capture that feeling and think about that.
So someone unknown to you grossly labeled you, that happens to those same folks all the time to the point one would have to ask WTF.
You know what, it says more about them as a person than you because they are the one projecting their trauma onto another to hurt another person.
Instead of feeling hate for you or anger for them, you can also look at several ways to deal with it.
Pity: This person is damaged beyond all hope. You know the look of disgust with an anti-vaxxer, anti--masker, covidiot, or using politics Hillary/Trump lover?
Laughter: This person's stupidity rivals the person of a court jester because to grossly label must be a joke.
Acceptance: Agree that you have had extra chances in life due to your identity, and then respond with gee it must suck to be me since that privilege train dropped a bunch of shit on my life and let me tell you why.
You can process the rest in your own way.
But if you have the experiences you have and you have processed the shock and WTF, you should not let it fester in your soul because as you said the person doesn't know you from Adam.
Shame on them for being prejudiced (which is worse than bigotry), but dwell on the negative to the point it causes an issue no?
Take the angst and pain and transform it, and if you meet this person again, you can bring it up.
This time bring some research showing how they were right and yes you have privilege, but then share your story of being abused as a child so you can empathize with their pain. since you know how it is to be out of control.
Then ask them how would you fix the world where people are judged and treated fairly by their actions and not generalized and labeled.
Let them speak first and process it, and step back.
Less is more.
The key line you need to remember is that the discomfort and feeling you feel is that ... a bad feeling, quite a few folks that are disadvantaged actually have been impacted hardcore and labeled and raped of their civility every day.
If you are strong and a survivor, then use your ability to process shit and help others grow and be safe.