News for hackers. Stuff that matters.
Re:John Carmack: From a DM, just in case anyone else needs t... by rocho on Sunday March 19, 2023 @09:22AM (#35218663)
In that case, how is the AI going to keep tens or hundreds of thousand of lines in memory to produce cohesive code that works with the rest of the codebase?

It seems prohibitely expensive to build and run transformer models with that much capacity.

Re:John Carmack: From a DM, just in case anyone else needs t... by anon7725 on Sunday March 19, 2023 @01:27PM (#35221366)
GPT 4 already has 32k tokens of context for prompts. Once we’re making arguments about scale only a few orders of magnitude larger than the current state of the art, it seems similar to arguments 10-15 years ago that real-time ray tracing is not feasible.