Re:Anthropic - Introducing 100K Token Context Windows, Aroun...
by dragonwriter on Friday May 12, 2023 @12:13AM
(#35911815)
> It's the throughput compared to humans that is the true differentiator. If hooking up the API in a loop up ends up costing more than a human, then it's not worth it.
If its better in another dimension (e.g., calendar elapsed time), it may well be worth being more expensive.
Re:Anthropic - Introducing 100K Token Context Windows, Aroun...
by koboll on Friday May 12, 2023 @12:40AM
(#35911991)
Two dollars spent over and over would still take a while to equal a software engineer's salary, and if the end result is "produce a fully functional codebase in minutes" the premium might be worth it even if it exceeds that much.
Re:Anthropic - Introducing 100K Token Context Windows, Aroun...
by gmerc on Friday May 12, 2023 @03:45AM
(#35913210)
We all know the game Sam Altman is playing. Eventually you can squeeze the full salary in cost out of companies who have laid off their workers and it would still be a good deal because AI does not need health care, pay taxes, hardware, sick days and so on.
The real utility of LLMs is that they can be called in a loop to scan through many web pages, many code files, issue tickets, emails, etc...
There are already demos and experiments out there that for every input, 4x outputs are generated, then those are fed back into the LLM 4x for "review", then the best variant is then used to generate code which then automatically tested, errors are fed back in a loop, also with 4x parallel tries, etc...
It's the throughput compared to humans that is the true differentiator. If hooking up the API in a loop up ends up costing more than a human, then it's not worth it.