News for hackers. Stuff that matters.
Re:Show HN: Turning books into chatbots with GPT-3 by fzliu on Tuesday January 24, 2023 @06:04PM (#34511913)
If you sprinkle in a bit of infrastructure, I think we're already there. The ability to distill a variety of content into vectors and perform approximate nearest neighbor search (shameless plug: https://milvus.io) across all of them can really help power a lot of these applications. With the retrieved vectors, you could match questions with answers or create a reverse index to the original content to perform summarization.

With that being said, one of the main challenges ahead will be multimodal learning. We're sort-of there combining text with visual data, but there are many other modalities out there as well.

Re:Show HN: Turning books into chatbots with GPT-3 by williamcotton on Tuesday January 24, 2023 @07:57PM (#34512894)
How usable would the document embedding be for a nearest neighbor search if the dimensions were reduced to three?
Re:Show HN: Turning books into chatbots with GPT-3 by fzliu on Tuesday January 24, 2023 @08:38PM (#34513248)
Might be usable for low column count tabular data, but it would be pretty terrible for any other semantically dense modality e.g. video, molecules, geospatial, etc.
Re:Show HN: Turning books into chatbots with GPT-3 by hansvm on Tuesday January 24, 2023 @08:27PM (#34513143)
Nearly useless for most applications unless there's been a major improvement in the SOTA that I missed.
Re:Show HN: Turning books into chatbots with GPT-3 by sdenton4 on Tuesday January 24, 2023 @08:58PM (#34513415)
can i use the three dimensions to encode a space-filling curve over a 1000-dimensional embedding?
Re:Show HN: Turning books into chatbots with GPT-3 by williamcotton on Wednesday January 25, 2023 @07:43AM (#34517573)
Not precisely, but if you had 50 documents in that 1000-dimensional embedding and you reduced the dimensions to three and still got at least the exact same nearest neighbor ordering then it would at least still function, right?

I guess the problem is taking a new document (like a search term) in the higher dimensional embedding and reducing it to three dimensions for searching in that reduced space and expecting that to also maintain the same nearest neighbor ordering.