Its's more than a perception, they've tacitly admited it. Wikipedia:
As a result of the public backlash it received from the United States for naming the Khomeini as Man of the Year in 1979, Time has shied away from using figures that are controversial in the United States due to commercial reasons. Time's Person of the Year 2001, immediately following the September 11, 2001 attacks, was New York City mayor Rudolph Giuliani, although the stated rules of selection, the individual or group of individuals who have had the biggest effect on the year's news, made Osama bin Laden a more likely choice. The issue that declared Giuliani the Person of the Year included an article that mentioned Time's earlier decision to elect the Ayatollah Khomeini and the 1999 rejection of Hitler as "Person of the Century". The article seemed to imply that Osama bin Laden was a stronger candidate than Giuliani, as Adolf Hitler was a stronger candidate than Albert Einstein.
And if you attribute the existence of nuclear weapons to Einstein (a legacy he'd have been pretty unhappy about), that changed the world much more lastingly than Hitler did. Hell, it almost destroyed the world entirely. How's that for a legacy.
There is a perception among some people that the selection process is biased away from controversial figures. Last year's Person of the Year was Pope Francis; some people thought it should have been (e.g.) Edward Snowden. He was the runner-up.