Native JSON Output from GPT-4

Re:Native JSON Output from GPT-4 by edwin on Wednesday June 14, 2023 @05:27PM (#36332971)

New techniques like semantic caching will help. This is the modern era's version of building a performant social graph.

[ Parent | on HN ]

Re:Native JSON Output from GPT-4 by daralthus on Wednesday June 14, 2023 @05:36PM (#36333102)

What's semantic caching?

[ Parent | on HN ]

Re:Native JSON Output from GPT-4 by edwin on Wednesday June 14, 2023 @05:50PM (#36333247)

With LLMs, the inputs are highly variable so exact match caching is generally less useful. Semantic caching groups similar inputs and returns relevant results accordingly. So {"dish":"spaghetti bolognese"} and {"dish":"spaghetti with meat sauce"} could return the same cached result.

[ Parent | on HN ]

Re:Native JSON Output from GPT-4 by m3kw9 on Wednesday June 14, 2023 @05:58PM (#36333352)

Or store as sentence embedding and calculate the vector distance, but creates many edge cases

[ Parent | on HN ]