Re:Numbers every LLM Developer should know
by armchairhacker on Wednesday May 17, 2023 @04:10PM
(#35980604)
But I'd say LLMs produce content faster than I can read or write it, because they can produce content which is really dense.
Ask GPT-4 a question and then answer it yourself. Maybe your answer will be as good or better than GPT-4's but GPT-4 writes its answer a lot faster.
Re:Numbers every LLM Developer should know
by furyofantares on Wednesday May 17, 2023 @07:16PM
(#35982442)
It certainly doesn't produce content as fast as I can read it.
Only if you use gpt-4. gpt-3.5-turbo is much faster, and gpt-4 is only going to get faster as GPUs get faster.
Bing also uses GPT-4 and it is very fast. Microsoft spends more ok compute.
Re:Numbers every LLM Developer should know
by furyofantares on Thursday May 18, 2023 @10:38AM
(#35988613)
It doesn't exclusively use GPT-4, you might be right anyway that their GPT-4 is much faster but you're also not always seeing GPT-4 with them.
I'm pretty sure it at least mostly uses GPT-4.
Re:Numbers every LLM Developer should know
by darknoon on Thursday May 18, 2023 @08:31PM
(#35996306)
afaict OpenAI's instance is massively overloaded, you can see with the 32k context model actually being faster in practice rather than slower
Re:Numbers every LLM Developer should know
by furyofantares on Thursday May 18, 2023 @10:37AM
(#35988595)
Yep. I use GPT-4 extensively and exclusively, and the comment I was replying to mentioned GPT-4. I can't wait for it to get faster.
dense content? Not in my experience. It seems to be really overly verbose for me.
Prompt it to be information dense in its response then.
So I must specifically ask for it, but it's not at all the default.
I get it, but it is just about infinitely configurable to your specific needs so it doesn't bother me too much what the default response is.
Re:Numbers every LLM Developer should know
by skykooler on Wednesday May 17, 2023 @07:31PM
(#35982546)
Human reading speed varies by a factor of 10 or more between individuals, while speaking speed is much more consistent.
Even my own reading speed even varies by a factor of 5 day to day, depending on how much reading I've been doing, sleep I've gotten, etc.
Re:Numbers every LLM Developer should know
by irrational on Thursday May 18, 2023 @12:30AM
(#35984137)
Plus, whether I am reading light fiction versus technical documentation.
- Human Reading Speed (English): ~250 words per minute
- Human Speaking Speed (English): ~150 words per minute
Should be treated like the Doherty Threshold [1] for generative content.
[1] https://lawsofux.com/doherty-threshold/