News for hackers. Stuff that matters.

A comment on GPT-4 is coming next week – and it will be multimodal, says Microsoft Germany. Parent comment

Re:GPT-4 is coming next week – and it will be multimodal, sa... by dmix on Thursday March 09, 2023 @10:12PM (#35089964)
Any chance this could improve coding LLMs like Co-pilot? Or would that sort of thing be limited to source code feeds (not that Github has a shortage).
[ Parent | on HN ]
Re:GPT-4 is coming next week – and it will be multimodal, sa... by refulgentis on Thursday March 09, 2023 @11:09PM (#35090262)
The next big step for coding LLMs will be context window increases, leaked docs have OpenAI pricing for up to 16K I believe, 4x the current maximum. Now you're talking "write a class" instead of this line and maybe sometimes a method
[ Parent | on HN ]
Re:GPT-4 is coming next week – and it will be multimodal, sa... by williamcotton on Thursday March 09, 2023 @11:29PM (#35090399)
It can already reliably write a todo web browser application.

With 16k and some other techniques, I’m guessing it could write a custom CMS database backed web application.

[ Parent | on HN ]
Re:GPT-4 is coming next week – and it will be multimodal, sa... by flangola7 on Thursday March 09, 2023 @11:48PM (#35090506)
Not 16k, 32k. 8x the current window.
[ Parent | on HN ]
Re:GPT-4 is coming next week – and it will be multimodal, sa... by refulgentis on Friday March 10, 2023 @12:16AM (#35090662)
Nice
[ Parent | on HN ]
Re:GPT-4 is coming next week – and it will be multimodal, sa... by dmix on Friday March 10, 2023 @12:09AM (#35090617)
What is 16k referring to here
[ Parent | on HN ]
Re:GPT-4 is coming next week – and it will be multimodal, sa... by refulgentis on Friday March 10, 2023 @12:19AM (#35090677)
I’ve begun to grok it as “the amount of ram I have to play in before I have to start sharding work”

more literally and correctly, it’s the maximum number of tokens in the input and output, combined, where a token is 4/3 of a word

So we’re shifting from 5K words maximum to 40K (per sibling comment, who pointed out 32K context leaked as well)

[ Parent | on HN ]
Re:GPT-4 is coming next week – and it will be multimodal, sa... by MattRix on Friday March 10, 2023 @01:08AM (#35090985)
A minor correction: a token is 3/4 of a word. ie. it’s slightly smaller than a word, not larger.
[ Parent | on HN ]