News for hackers. Stuff that matters.
Re:HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends... by marcosdumay on Friday March 31, 2023 @02:31PM (#35391284)
I believe you will quickly run into a barrier due to the small size of the context window. And increasing the context window gets harder and harder the larger it is.

There's a sibling talking about internal memory. That's how our brains solve the issue. AFAIK, nobody knows how to train something like it.

Re:HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends... by wyager on Friday March 31, 2023 @02:36PM (#35391361)
I wouldn't be surprised if AI research moves towards training models to use bolt-on peripherals like scratchpad memory. Transformers showed us an interface for addressing, so I wouldn't be surprised if someone figures out a way to use a similar addressing scheme and made it read/write.
Re:HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends... by UniverseHacker on Friday March 31, 2023 @02:45PM (#35391477)
You are right, but it is unclear to me where this limitation will fall in terms of actual abilities.

GPT-4 can divide large tasks up into logical steps and instructions that can be worked on independently by other instances, and it can create "compressed" condensed explanations of it's own state that can be stored externally and repeated back to it, or passed back and forth between instances. With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on.

Re:HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends... by IanCal on Friday March 31, 2023 @03:18PM (#35392019)
> With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on.

Also that means you can just iterate with different contexts until you can deal with the problem. How many problems need 50 pages of context on top of what gpt4 knows for solving the next step?

Re:HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends... by Workaccount2 on Friday March 31, 2023 @05:29PM (#35393935)
>How many problems need 50 pages of context on top of what gpt4 knows for solving the next step?

Perhaps the unknown unkowns?

I envision a scenario where a super intelligent AI seemingly runs off the rails and becomes obsessed with overtly complex problems that are totally intractable to humans. Where we'd be like ants trying to make sense of cell phones.

Re:HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends... by jules on Saturday April 01, 2023 @02:25AM (#35397703)
Maybe it can do this with sleep, where it gets further training on the new input and output it produced that day.
Re:HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends... by Sinidir on Saturday April 01, 2023 @03:54AM (#35398204)
That just gave me the idea that there could be a main instance that keeps a summary of the problem (List of paragraphs for each subproblem) and other instances that hold the larger text that these summaries are based on. Then on demand the main instance could get needed extra information for a subpoint back. Kinda like having forgein keys in a databas with which you can join in the relevant row!
Re:HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends... by nomel on Friday March 31, 2023 @03:05PM (#35391802)
I think an attention/focus system might be applicable. I think humans have the same limitations, with tricks to work around it. As Jim Keller once said on an Lex Fridman interview, a good engineer is one that can jump around between the different levels of abstraction. We think in a high level, then "focus" and jump down when needed.

I think something like this could be hacked together, with summarization and parallel "focus threads", containing prompts that focus on details. These could be pruned/merged back together, to add to the "summary" higher level abstractions.

I use this approach already, to some extent, when a conversation gets too long and I need specific details. I'll start a new prompt, include a high level summary, and then "focus" on specific ideas to get more specific answers about a particular detail.

Re:HuggingGPT: Solving AI Tasks with ChatGPT and Its Friends... by sdenton4 on Friday March 31, 2023 @04:06PM (#35392793)
Approaches like RETRO tack vector search over a fixed database onto an attention based model - that's how you get long term memory. People are already working on it for the current crop of LLMs.