Societal stereotypes are always interesting, and there is a fine-line between sexual preference and overt racism.
It does happen both ways, a beautiful woman (who was African American was surprised that biology knows no gender) had some fun comments to me.
She laughed as she saw the "light" and it was "all right" :)
But as you called out ... it is all about perspective and privilege and what not.
If a person has preference due to the lovely contrast or differences that make us more awesome. Game on. After that it takes consent and time with talking about desires so folks don't get objectified non-consensually.