folktheory

Author here. What if you could see the bias of different academic researchers? Where their rankings overlap and differ?

We built a trust graph out of citations from Semantic Scholar, with an LLM classifying positive, negative, and neutral citations. We applied it to 176,234 researchers and 856,433 papers in 302 subfields.

Why might this matter to you? We think that in a lot of scientific fields there’s less consensus than it seems, and this basically starts to show where those gaps are the biggest. Citation count pretends to be a global ranking but ends up promoting citation-maxing, instead of good science. We think trust-maxing would be a move in the right direction. (if all you do is Goodhart being more trustworthy, you’re actually more trustworthy, so it’s prosocial.

What you can do: Try switching the viewer on any ranking, or compare two researchers side by side to see where they line up and differ.

Differences from a personalized PageRank:

- Includes positive and negative evidence - we treat distrust and uncertainty as first-class evidence.

- Asymmetric propagation of distrust - trust flows transitively, but distrust doesn't. Your friend's friend is your friend, but your enemy's enemy isn't your friend.

- New rank requires new independent trusted sources, not more citations

Disclosures:

- Citations are an imperfect proxy for trust

- LLM classification is lossy

- This is a self-run showcase of the engine behind UpTrust (a product we're building)

If you're a researcher whose field this covers, we’d love to hear from you. How well does this represent your actual view? How much better (or worse) is it than citation-count list, or whatever else is available?

If you have a dataset you’re curious to run this over, also let us know!

if it's a well understood fact, then how is it a new treatment?

According to US government, women spend 16 minutes a day, on average, doing laundry, and men spend 4 minutes on average. That's over a 100 hours a year for women, so yes, there is probably a lot of demand for a technology that makes this faster.

I'm guessing it generates the tree (2 levels deep) and then uses an evaluation function to choose the best branch.

Not too far off from a modern algorithm, except that it lacks even basic optimizations like alpha-beta pruning, etc. and an incredibly fine-tuned evaluation function and a huge library of opening and closing moves.

Nooooo!

I was going to start looking at places in Padmapper next month, for my upcoming move!

Is there any other similar service?

I'm surprised someone as skilled as you hasn't been able to find a job. Have you considered moving out of Mozambique to a greener pasture?

They "monetize" it by charging money for use of the service.

It's a video game, not a movie.

Welcome to MIT...

As someone who already has a bachelors in CS, I'm still looking forward to fill in some gaps in my education with these. For example, I never took a Theory of Computation class.