Understanding BERT and Search Relevance (2019)

Re:Understanding BERT and Search Relevance (2019) by binarymax on Wednesday April 22, 2020 @12:18PM (#22947204)

Building upon @m_ke's note, when you have the sentence embeddings indexed, you use query similarity with those sentences as an additional signal that can be used further. For example, you can weight sentences that appear earlier in the document higher, you can take the top N sentences and use those, you can apply clustering to find hotspots, etc.

I'd treat it as a base signal like BM25 for a field - which is frequently tuned and used as one of many features for relevance.

[ Parent | on HN ]

Re:Understanding BERT and Search Relevance (2019) by petulla on Wednesday April 22, 2020 @01:11PM (#22947826)

Sort of makes sense. You'd need a ground-truth search->document dataset, I think, to test on, though. Seems a bit hand-tuned.

[ Parent | on HN ]

Re:Understanding BERT and Search Relevance (2019) by binarymax on Wednesday April 22, 2020 @01:33PM (#22948061)

Absolutely! We call it a "judgement list" in search relevance land, and one is required and used for training and testing of a relevance config/model...though the training is often manual, until your configuration and infrastructure are mature enough for learning-to-rank.

[ Parent | on HN ]

Re:Understanding BERT and Search Relevance (2019) by binarymax on Wednesday April 22, 2020 @02:08PM (#22948445)

@petulla we've reached the thread depth limit so I can't reply to your last question, but typically MSMarco is used for SOTA testing of deep learning for search: https://microsoft.github.io/msmarco/

[ Parent | on HN ]

Re:Understanding BERT and Search Relevance (2019) by petulla on Wednesday April 22, 2020 @01:57PM (#22948318)

Last q: which datasets of this kind are used for SOTA testing or document clustering?

[ Parent | on HN ]