News for hackers. Stuff that matters.
Re:Yandex ‘leak’ reveals 1,922 search ranking factors by yodsanklai on Monday January 30, 2023 @05:46AM (#34578106)
Could someone explain to me how you go from these ranking facts to an actual sorted list of results? and doing it with an acceptable latency. Also, what happened to Google page rank, is is still relevant today?
Re:Yandex ‘leak’ reveals 1,922 search ranking factors by WinLychee on Monday January 30, 2023 @06:37AM (#34578457)
All of the listed "facts" or "features" can be incorporated into an index (e.g. with Lucene) or fed into ML-models (logistic regression, trees, neural nets, etc).

Without going into specifics, I've seen ranking treated as a multi-stage algorithmic problem. Initially, you rank results with a lower-quality but low-latency ranker at the first stage (inverted index, tf-idf, knn, etc), and subsequent stages rerank the top-K results with higher-latency ML models outputting a relevance score.

I believe progress is currently being made to combine everything into one giant neural model that just ranks everything from the get-go rather than pass in multiple stages.

Re:Yandex ‘leak’ reveals 1,922 search ranking factors by miohtama on Monday January 30, 2023 @06:07AM (#34578257)
For an exhaustive answer, you can check from Yandex source code that was posted on another thread on HN

https://news.ycombinator.com/item?id=34525936

Re:Yandex ‘leak’ reveals 1,922 search ranking factors by ad404b8a372f2b9 on Monday January 30, 2023 @06:11AM (#34578290)
"SEOs have already started analyzing Yandex's search ranking factors, which include PageRank and several other link-related factors"

PageRank is factor 0.

Re:Yandex ‘leak’ reveals 1,922 search ranking factors by fsflover on Monday January 30, 2023 @05:52AM (#34578140)
I can't answer, but you should be able to find the answer on https://yacy.net -- free web search. By the way, it can probably tremendously benefit from the leak.
Re:Yandex ‘leak’ reveals 1,922 search ranking factors by ValleZ on Monday January 30, 2023 @06:45AM (#34578510)
Pages are sorted by page rank, you pick first 10-100k pages and sort using other factors, then take first 10-100 results and sort using most costly factors.
Re:Yandex ‘leak’ reveals 1,922 search ranking factors by thrwyndx on Monday January 30, 2023 @06:47AM (#34578523)
You throw them into a machine learning model together with a big dataset of queries/urls annotated by humans for relevancy. Catboost is yandex's choice of model here.

> and doing it with an acceptable latency

Lots of interesting optimizations possible here, but the big obvious one is multiple level models: score documents with a cheap model (FastRank in yandex lingvo) first using a subset of the fastest available features, then rescore top docs with your best slow expensive model. Perhaps rescore multiple times at different points in the stack with models of varying complexity, at each index shard and after aggregating the results from subset/all shards. Also sort documents in each index shard by some other ML model with query-independent features to push all the junk to the end of the index where you'd likely skip it when running out of time budget to process a query.

> Also, what happened to Google page rank, is is still relevant today?

Vanilla 1990s' pagerank obviously not, but the idea of such graph-based calculations is still very useful yes.

Re:Yandex ‘leak’ reveals 1,922 search ranking factors by swyx on Monday January 30, 2023 @11:35PM (#34591180)
> Vanilla 1990s' pagerank obviously not,

what did we learn about the flaws?