Re:Show HN: ChatGPT UI for Rabbit Holes
by iknownthing on Thursday May 30, 2024 @10:40AM
(#40524332)
Are you paying for the API calls yourself here?
[ Parent
| on HN ]
It seems like they're caching information/replies on key words which is a good optimization.
Re:Show HN: ChatGPT UI for Rabbit Holes
by iknownthing on Thursday May 30, 2024 @11:00AM
(#40524574)
I assumed it was coming straight from the API because of the token-by-token generation effect but maybe you're right.
The token-by-token responses are probably API, while the "instant" loads seem to be cached.
Probably using groq based on speed of response
Re:Show HN: ChatGPT UI for Rabbit Holes
by maxkrieger on Thursday May 30, 2024 @01:12PM
(#40526144)
I have a lot of OpenAI API credits to use by end of the month, so I’m using 4o. I’ll probably switch to a more sustainable model afterwards, consider this a request for API credits, all! Email’s on website :)