"Independent" is more important than "opposite". I don't know that promptfoo would be overtly biased. Granted they might have unconscious bias or sensitivities about offending paying customers. I do note that they present all their evidence with methods and an invitation for others to replicate or extend their results, which would go someway towards countering bias. I wouldn't trust the neutrality of someone under the influence of the CCP over promptfoo.

We’ll see soon enough, no use debating now. But I’d put money on them not showing any examples that might get them caught up in a media frenzy regarding whether they’re x-ist or anti-x-ic or anything of the sort, regardless of what the underlying ground truth in their specific questions might be.

You’ll note even on this platform, generally regarded as open and pseudo-anonymous, only a single relevant example has been put forward.

Somethings never change. Reminds me of this joke from Regan:

Two men, an American and a Russian were arguing. One said,

“in my country I can go to the white house walk to the president's office and pound the desk and say "Mr president! I don't like how you're running things in this country!"

"I can do that too!"

"Really?"

"Yes! I can go to the Kremlin, walk into the general secretary's office and pound the desk and say, Mr. secretary, I don't like how Reagan is running his country!"

Not funny anymore, after 1/20/2025