Show HN: I made a free course, 'NLP for Semantic Search'

Re:Show HN: I made a free course, 'NLP for Semantic Search' by rolisz on Tuesday December 14, 2021 @03:43PM (#29557344)

Does this semantic search work for longer documents? I know BERT&Co models are limited to around 500-1000 tokens.

[ Parent | on HN ]

Re:Show HN: I made a free course, 'NLP for Semantic Search' by crucialfelix on Wednesday December 15, 2021 @01:20AM (#29562712)

You can split the document by paragraph or sentence. Search results will then match a sentence in a document. You display with a link to the document with a reference to the span.

[ Parent | on HN ]

Re:Show HN: I made a free course, 'NLP for Semantic Search' by jamesbriggs on Wednesday December 15, 2021 @03:02AM (#29563204)

I don't remember the paper, but the authors working on this same problem suggested to do this + also include an overlap between spans.

[ Parent | on HN ]

Re:Show HN: I made a free course, 'NLP for Semantic Search' by crucialfelix on Wednesday December 15, 2021 @04:27AM (#29563724)

Would you include the overlap in the text for training, or just for the displayed result? I assume the later.

[ Parent | on HN ]