News for hackers. Stuff that matters.

A comment on Show HN: ChatGPT UI for Rabbit Holes. Parent comment

Re:Show HN: ChatGPT UI for Rabbit Holes by Falimonda on Thursday May 30, 2024 @10:47AM (#40524413)
It seems like they're caching information/replies on key words which is a good optimization.
[ Parent | on HN ]
Re:Show HN: ChatGPT UI for Rabbit Holes by iknownthing on Thursday May 30, 2024 @11:00AM (#40524574)
I assumed it was coming straight from the API because of the token-by-token generation effect but maybe you're right.
[ Parent | on HN ]
Re:Show HN: ChatGPT UI for Rabbit Holes by rodiger on Thursday May 30, 2024 @12:56PM (#40525925)
The token-by-token responses are probably API, while the "instant" loads seem to be cached.

Probably using groq based on speed of response

[ Parent | on HN ]