Show HN: I built a LLM-powered Ask HN: like Perplexity, but for HN comments

Re:Show HN: I built a LLM-powered Ask HN: like Perplexity, b... by jnnnthnn on Thursday May 16, 2024 @06:43PM (#40384218)

I got some feedback that GPT-3.5 was letting people down so improved my caching strategy and defaulted everything to GPT-4o!

[ Parent | on HN ]

Re:Show HN: I built a LLM-powered Ask HN: like Perplexity, b... by sebastiennight on Friday May 17, 2024 @04:34AM (#40387694)

Great to hear! Would you be willing to share a bit on what changes you made to your caching strategy to optimize costs?

[ Parent | on HN ]

Re:Show HN: I built a LLM-powered Ask HN: like Perplexity, b... by jnnnthnn on Friday May 17, 2024 @09:14AM (#40389525)

I made the caching window substantially longer (2 days instead of a few minutes) upon noticing most queries weren't about current events, and fixed a bug in the cache resolution logic that'd lead to a substantial number of cache misses.

[ Parent | on HN ]

Re:Show HN: I built a LLM-powered Ask HN: like Perplexity, b... by sebastiennight on Saturday May 18, 2024 @11:28AM (#40399809)

Cool, thanks!

[ Parent | on HN ]