News for hackers. Stuff that matters.

A comment on Creating an advanced search engine with PostgreSQL. Parent comment

Re:Creating an advanced search engine with PostgreSQL by SpriglyElixir12 on Wednesday July 12, 2023 @03:19PM (#36700267)
How would something like this work in practice? Would you generate any tags or summaries per site when inserting it into the db?
[ Parent | on HN ]
Re:Creating an advanced search engine with PostgreSQL by janalsncm on Wednesday July 12, 2023 @05:35PM (#36702008)
You could run a full text search or search against an auto-generated summary. Or if you want to be fancy, use semantic search like in Retrieval Augmented Generation.
[ Parent | on HN ]
Re:Creating an advanced search engine with PostgreSQL by sudobash1 on Wednesday July 12, 2023 @07:34PM (#36703143)
ArchiveBox can extract text from HTML (and possibly PDFs too). I think it can be configured to extract subtitles from YouTube videos as well. So it can do full text searches. Basically you could have your own, offline & curated search-engine.
[ Parent | on HN ]