News for hackers. Stuff that matters.
Re:Numbers every LLM Developer should know by abetlen on Wednesday May 17, 2023 @03:39PM (#35980190)
I would add the following two numbers if you're generating realtime text or speech for human consumption:

- Human Reading Speed (English): ~250 words per minute

- Human Speaking Speed (English): ~150 words per minute

Should be treated like the Doherty Threshold [1] for generative content.

[1] https://lawsofux.com/doherty-threshold/

Re:Numbers every LLM Developer should know by armchairhacker on Wednesday May 17, 2023 @04:10PM (#35980604)
But I'd say LLMs produce content faster than I can read or write it, because they can produce content which is really dense.

Ask GPT-4 a question and then answer it yourself. Maybe your answer will be as good or better than GPT-4's but GPT-4 writes its answer a lot faster.

Re:Numbers every LLM Developer should know by furyofantares on Wednesday May 17, 2023 @07:16PM (#35982442)
It certainly doesn't produce content as fast as I can read it.
Re:Numbers every LLM Developer should know by renonce on Thursday May 18, 2023 @01:24AM (#35984394)
Only if you use gpt-4. gpt-3.5-turbo is much faster, and gpt-4 is only going to get faster as GPUs get faster.
Re:Numbers every LLM Developer should know by cubefox on Thursday May 18, 2023 @04:15AM (#35985259)
Bing also uses GPT-4 and it is very fast. Microsoft spends more ok compute.
Re:Numbers every LLM Developer should know by furyofantares on Thursday May 18, 2023 @10:38AM (#35988613)
It doesn't exclusively use GPT-4, you might be right anyway that their GPT-4 is much faster but you're also not always seeing GPT-4 with them.
Re:Numbers every LLM Developer should know by cubefox on Thursday May 18, 2023 @01:31PM (#35991350)
I'm pretty sure it at least mostly uses GPT-4.
Re:Numbers every LLM Developer should know by darknoon on Thursday May 18, 2023 @08:31PM (#35996306)
afaict OpenAI's instance is massively overloaded, you can see with the 32k context model actually being faster in practice rather than slower
Re:Numbers every LLM Developer should know by furyofantares on Thursday May 18, 2023 @10:37AM (#35988595)
Yep. I use GPT-4 extensively and exclusively, and the comment I was replying to mentioned GPT-4. I can't wait for it to get faster.
Re:Numbers every LLM Developer should know by bombolo on Thursday May 18, 2023 @02:31AM (#35984694)
dense content? Not in my experience. It seems to be really overly verbose for me.
Re:Numbers every LLM Developer should know by Tostino on Thursday May 18, 2023 @08:48AM (#35987119)
Prompt it to be information dense in its response then.
Re:Numbers every LLM Developer should know by bombolo on Thursday May 18, 2023 @02:53PM (#35992440)
So I must specifically ask for it, but it's not at all the default.
Re:Numbers every LLM Developer should know by Tostino on Thursday May 18, 2023 @03:00PM (#35992530)
I get it, but it is just about infinitely configurable to your specific needs so it doesn't bother me too much what the default response is.
Re:Numbers every LLM Developer should know by skykooler on Wednesday May 17, 2023 @07:31PM (#35982546)
Human reading speed varies by a factor of 10 or more between individuals, while speaking speed is much more consistent.
Re:Numbers every LLM Developer should know by fritzo on Wednesday May 17, 2023 @07:46PM (#35982645)
Even my own reading speed even varies by a factor of 5 day to day, depending on how much reading I've been doing, sleep I've gotten, etc.
Re:Numbers every LLM Developer should know by irrational on Thursday May 18, 2023 @12:30AM (#35984137)
Plus, whether I am reading light fiction versus technical documentation.
Re:Numbers every LLM Developer should know by telesilla on Thursday May 18, 2023 @08:41AM (#35987065)
Exceptions for languages where rapidity of speech really varies according to context, such as in Spanish.
Re:Numbers every LLM Developer should know by JohnFen on Thursday May 18, 2023 @11:36AM (#35989518)
> speaking speed is much more consistent.

Is it? I've noticed a huge variance in speaking speed in the US, but it tends to vary more between regions rather than individuals.