Re:Show HN: Turning books into chatbots with GPT-3
by williamcotton on Tuesday January 24, 2023 @07:57PM
(#34512894)
How usable would the document embedding be for a nearest neighbor search if the dimensions were reduced to three?
Re:Show HN: Turning books into chatbots with GPT-3
by fzliu on Tuesday January 24, 2023 @08:38PM
(#34513248)
Might be usable for low column count tabular data, but it would be pretty terrible for any other semantically dense modality e.g. video, molecules, geospatial, etc.
Re:Show HN: Turning books into chatbots with GPT-3
by hansvm on Tuesday January 24, 2023 @08:27PM
(#34513143)
Nearly useless for most applications unless there's been a major improvement in the SOTA that I missed.
Re:Show HN: Turning books into chatbots with GPT-3
by sdenton4 on Tuesday January 24, 2023 @08:58PM
(#34513415)
can i use the three dimensions to encode a space-filling curve over a 1000-dimensional embedding?
Re:Show HN: Turning books into chatbots with GPT-3
by williamcotton on Wednesday January 25, 2023 @07:43AM
(#34517573)
Not precisely, but if you had 50 documents in that 1000-dimensional embedding and you reduced the dimensions to three and still got at least the exact same nearest neighbor ordering then it would at least still function, right?
I guess the problem is taking a new document (like a search term) in the higher dimensional embedding and reducing it to three dimensions for searching in that reduced space and expecting that to also maintain the same nearest neighbor ordering.
With that being said, one of the main challenges ahead will be multimodal learning. We're sort-of there combining text with visual data, but there are many other modalities out there as well.