News for hackers. Stuff that matters.

A comment on Show HN: I built a better UI for ChatGPT. Parent comment

Re:Show HN: I built a better UI for ChatGPT by jw1224 on Tuesday March 07, 2023 @08:04AM (#35054853)
> I was annoyed by the default OpenAI ChatGPT slow-typing animation

I’d assumed ChatGPT’s slow-typing was actually the model’s real-time output, streamed to the browser token-by-token. When I’ve made GPT3 API calls it’s worked like this, with streaming switched off I might need to wait ~20 seconds for a long response back.

I just tried your UI, and it’s so much faster! Was my assumption about real-time output wrong?

Great project, looking forward to using it more :)

[ Parent | on HN ]
Re:Show HN: I built a better UI for ChatGPT by IanCal on Tuesday March 07, 2023 @08:23AM (#35055014)
ChatGPTs paid version is extremely fast, and this is using the paid API. The paid API feels slower than the paid UI, but this may be the difference between streaming results and getting everything in one payload.
[ Parent | on HN ]
Re:Show HN: I built a better UI for ChatGPT by trungdq88 on Tuesday March 07, 2023 @08:07AM (#35054888)
Yep, I was surprised I about the response time too. I'm not sure what's going on under the hood, maybe OpenAI did something with the stream output that causes the slowness.
[ Parent | on HN ]
Re:Show HN: I built a better UI for ChatGPT by lordnacho on Tuesday March 07, 2023 @08:39AM (#35055182)
Perhaps it is simply a way to limit the number of queries? If you force people to wait 10s for an answer they aren't writing new questions all the time.
[ Parent | on HN ]
Re:Show HN: I built a better UI for ChatGPT by TeMPOraL on Tuesday March 07, 2023 @09:08AM (#35055481)
For me it's been consistently the case that after 2-3 messages, the animation would start to visibly lag, and eventually (on 3rd or 4th reply) slow to a halt, sometimes mid-sentence, completely bricking the session. It's been like this since ChatGPT was first opened to public. I initially assumed it's just the usual case of a bloated SPA, but now I think maybe I'm being throttled on purpose?
[ Parent | on HN ]
Re:Show HN: I built a better UI for ChatGPT by weird-eye-issue on Tuesday March 07, 2023 @10:35AM (#35056463)
It uses a different model. ChatGPT Plus is fast too.
[ Parent | on HN ]
Re:Show HN: I built a better UI for ChatGPT by SomewhatLikely on Tuesday March 07, 2023 @10:41AM (#35056551)
I think the slow responses are the real generation rate when you time share the GPU. If you are willing to let GPUs occasionally sit idle though, you can probably get a faster generation rate by dedicating more of the GPU time to that one client.
[ Parent | on HN ]