I would say no, and in fact even having a preference for something like skin complexion isn't enough to make one racist. What is racist is stating the preference in terms of race and categorically denying that one would ever be attractive to a person of <race> or <skin complexion> or whatever. I think it would be pretty ugly to say that you would never date a short person, regardless of any implications for one's ideas about race, too. Like, it's different to say "i find tall people attractive" versus "i find short people unattractive".
Personal preferences aside, the way society signals which physical features are desirable and undesirable is racist. This is largely commercialized and driven by marketing departments these days but it is rooted in all sorts of institutions that govern the desirability and undesirability of certain bodies. So, as an individual, signaling one's preferences can feed into this structural power that decides who is pretty and who is not. Individual preferences are almost certainly partly influenced by messages society sends us and it's important to keep that in mind.
It is fine, there is nothing wrong with it, as you should do whatever you want with your romantic life, but it is racist as you are excluding a group of people from your dating pool just because of their race.
I think this whole idea of race should be put down, euthanised. I like to mock it by comparing people to dog breeds. We don't call the English Pointer and the Miniature Shar Pei different races, but they have distinctly stronger difference than English compared Chinese people.
In an effort to reclaim my preferences I can often be hearding saying things like "I don't mean to sound racists, but could you pass the salt" at the dinner table.