News for hackers. Stuff that matters.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by fnordpiglet on Thursday August 10, 2023 @07:31AM (#37074746)
Would be better to extend to use Llama 2 and break itself away from an open ai dependency. However I think this is striking in the right direction with LLM, embed them in a goal based agent framework with a variety of abilities and available actions that are mediated by planners, optimizers, constraint systems, etc, using the abductive capabilities of LLM for providing the abstract semantic glue and delegating other functions to classical AI and algorithms.

I always find it weird people are obsessed with LLMs being unable to solve quadratic equations or recite pi to some digit or play chess. That’s not the interesting abilities they demonstrate, and we already have techniques that do those tasks as good as you would ever need or want. But we have never had something that can operate in a natural language space and “reason” in that abstract space so effectively. That language alone is sufficient to bring out such amazing capabilities is cool, and mixing models and techniques using LLM as a glue between them will be (IMO) where they really change things.

Re:Generative Agents: Interactive Simulacra of Human Behavio... by HanClinto on Thursday August 10, 2023 @08:25AM (#37075312)
Agreed -- now that the repository is open-source, it feels ripe for adapting to a smaller local model. Is it too much dreaming to imagine that if we can perhaps fine-tune a small LLaMa2 model to perform well in this context, we might even be able to get this small enough to run on consumer hardware in an actual "Sims" type game...?
Re:Generative Agents: Interactive Simulacra of Human Behavio... by krautt on Thursday August 10, 2023 @10:29AM (#37076609)
there is no technical reason why you couldn't unbolt openai interface and bolt in llama. Moreover, once you have this, you need only load the model into memory once. emulating different agents would be handled exclusively through the context window sizes that llm's expect. each agent would just have its own evolving context window. roundrobin the submissions. repeat

what's crazy to think about is what new things will become possible as the context window sizes creep up.

Re:Generative Agents: Interactive Simulacra of Human Behavio... by dogcomplex on Thursday August 10, 2023 @04:27PM (#37081312)
Or as the model is able to be trained (integrated with new information) on the fly, making a somewhat limitless context window
Re:Generative Agents: Interactive Simulacra of Human Behavio... by __loam on Thursday August 10, 2023 @05:11PM (#37081857)
I think there are actual technical problems in doing this.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by dogcomplex on Saturday August 12, 2023 @05:25AM (#37098465)
Well, cost prohibitive on big models obviously. But even just things like DreamBooth training extra associations to sets of new data images is tuning the model as you go
Re:Generative Agents: Interactive Simulacra of Human Behavio... by gabereiser on Thursday August 10, 2023 @03:03PM (#37080361)
My dwarves in DF would be much more predictable. I’m all for it. So long as the model is safely trained and constraints are in place to prevent Actor A from doing something shockingly bad to Actor B in a way that wasn’t designed.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by HanClinto on Thursday August 10, 2023 @08:31AM (#37075371)
Side note -- discussion on adding support for local models is here, along with a preliminary fork that adds support: https://github.com/joonspk-research/generative_agents/issues...
Re:Generative Agents: Interactive Simulacra of Human Behavio... by taneq on Thursday August 10, 2023 @09:14AM (#37075743)
Plenty of people I know can't solve quadratic equations or more than 4-5 digits of pi, and nobody claims they're not intelligent.

Once again it's a kind of backhanded milestone for AI that in less than a year the goalposts have moved from "can it even once hold a half-coherent conversation" to "if anyone can think of a question it can't answer then it's dumb and not AI".

Re:Generative Agents: Interactive Simulacra of Human Behavio... by bee_rider on Thursday August 10, 2023 @09:59AM (#37076246)
It is definitely a nice milestone, for researchers.

I do think, though, that one of the rewards of making LLMs promising was the discovery that the bar for “what is useful” for any task is higher than what the average person can do. That could be seen as moving the goalposts, but another way of looking at it is: we’ve finally bothered to install a scoreboard.

Like, if a person can’t solve a quadratic equation, they probably just aren’t useful for manipulating equations. Which is fine, of course, the average person is also not a very good hammer or screwdriver.

We’re all specialized, the bar for usefulness is probably something more like “how would a person who’s taken an intro college course on this do it.”

Re:Generative Agents: Interactive Simulacra of Human Behavio... by fnordpiglet on Thursday August 10, 2023 @10:34AM (#37076687)
The goal isn’t to be more or less useful than a person. It’s to expand the capabilities of computers. And computers are exceptionally good at mathematics tasks already. Why do we care if LLMs are good at things that are already exceptionally well done with computers? The point is they’re exceptionally good at things computers to date have been exceptionally bad at. The rest is weird goalpost moving noise - I wouldn’t use ChatGPT to solve a quadratic equation, I would use Mathematica, or any number of numeric libraries and runtimes. But I challenge you to get Mathematica to summarize a book, carry on a conversation, or even call an api based on natural human language without a rigorous specification apriori. These are holy grails of NLP, which is a capability that is enhancing of the existing capabilities offered by computing technologies today.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by cmpalmer52 on Thursday August 10, 2023 @06:59PM (#37082788)
What we need are LLMs that can use the tools for mathematics (and other disciplines) that already exist and to write and execute new programs to hopefully solve novel problems, or at least to be able to glue the existing tools together with shells.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by fnordpiglet on Thursday August 10, 2023 @08:03PM (#37083366)
Well, this project can use LLMs do decide routes to take then delegates to a path finding algorithm. That’s why it’s interesting IMO. There are increasing number of things that do that. But I would note the LLMs don’t have to use the tools - a new tool can use the LLM, and be the coordinator between tasks to solve. A lot of the challenges folks are struggling with is making a LLM reliably invoke APIs and stuff. I suspect doing that lobotomizes the LLMs just as guard rails and safety does. Better, IMO, is to manage the LLM in an extrinsic framework like this.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by glial on Thursday August 10, 2023 @11:22AM (#37077320)
Agreed, it's apologetics for human intelligence.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by __loam on Thursday August 10, 2023 @05:11PM (#37081862)
More like a reasonable response to VC funded hysterics.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by glial on Thursday August 10, 2023 @10:28PM (#37084384)
That too, but this trend has been going on for decades in the field of linguistics. "Language is X," people say. Oh, it turns out that prairie dogs can do X. "Oh, well then language is X-prime!" Turns out parrots can do X-prime. "Oh, umm, maybe language is X-prime-prime! Whatever it is, only humans can do it!"
Re:Generative Agents: Interactive Simulacra of Human Behavio... by __loam on Friday August 11, 2023 @03:48AM (#37086174)
I find it pretty insane to argue that the human mind isn't somehow exceptional considering we're currently communicating through devices and massive infrastructure require centuries of research produced by those minds.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by taneq on Friday August 11, 2023 @09:18AM (#37088368)
I don't think anyone's arguing that we're not exceptional at all. After all, those prairie dogs aren't sitting around arguing about whether humans are actually on their level. Really what we're wondering when discussing AI is whether we're exceptional enough to build machines smarter than ourselves?

Has the Singularity already happened, and was it us?

Re:Generative Agents: Interactive Simulacra of Human Behavio... by glial on Friday August 11, 2023 @10:56AM (#37089406)
It's not the status of the human mind we're talking about here, it's definitions of 'language' and 'intelligence.' It seems offensive to some people that those labels might be applied to entities other than humans.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by __loam on Friday August 11, 2023 @12:43PM (#37090749)
It makes sense that animals are smart in similar ways to how we are smart since they're using the same meat we are. Comparing us to computers is a bit offensive in my view.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by Davidzheng on Thursday August 10, 2023 @09:35AM (#37075988)
Not your point but I'm pretty sure gpt4 is more than capable of solving quadratic equations and playing chess and probably knows pi to quite a few digits... also it knows many great algorithms for computing pi
Re:Generative Agents: Interactive Simulacra of Human Behavio... by fnordpiglet on Thursday August 10, 2023 @12:29PM (#37078274)
Yeah, but it’s not nearly as good at playing chess as a basic chess algorithm. Why would I use GPT4 for chess when Chessmaster 2000 can beat it 100/100 times?
Re:Generative Agents: Interactive Simulacra of Human Behavio... by earthboundkid on Thursday August 10, 2023 @03:48PM (#37080845)
I would be shocked if GPT4 could reliably solve novel quadratic equations when it struggles with simple arithmetic.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by flangola7 on Thursday August 10, 2023 @04:42PM (#37081508)
Why would you be shocked? GPT-4 scores in the 89th percentile on SAT Math.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by earthboundkid on Thursday August 10, 2023 @09:05PM (#37083882)
Because SAT math is all multiple choice, which can be reverse engineered. Ask it to multiply numbers too large to be in the memorization set.
Re:Generative Agents: Interactive Simulacra of Human Behavio... by earthboundkid on Friday August 11, 2023 @03:41PM (#37093045)
I don't pay for GPT4, but here's Claude falling on its face:

Q:

Provide the quadratic roots of the following equation. Work out the answer step by step.

21 - 135 x + 19 x^2 = 0

A:

Okay, here are the step-by-step workings:

1) Start with the quadratic equation: 21 - 135x + 19x^2 = 0

2) Group the x^2 and x terms together: 19x^2 - 135x + 21 = 0

3) Use the quadratic formula to find the roots: x = (-b ± sqrt(b^2 - 4ac)) / 2a Where: a = 19 b = -135 c = 21

4) Substitute the values into the formula: x = (-(-135) ± sqrt((-135)^2 - 4(19)(21))) / 2(19)

5) Simplify: x = (135 ± sqrt(18225 - 1596)) / 38

6) Evaluate the square root: x = (135 ± 135) / 38

7) The two roots are: x = 5 x = 3

Therefore, the roots of the quadratic equation 21 - 135x + 19x^2 = 0 are x = 5 and x = 3.

Re:Generative Agents: Interactive Simulacra of Human Behavio... by lusus_naturae on Thursday August 10, 2023 @08:41PM (#37083662)
Wolfram alpha was doing those things and more way before gpt was a thing. I don't think any of the attributes you listed make intelligent machines interesting..
Re:Generative Agents: Interactive Simulacra of Human Behavio... by avereveard on Thursday August 10, 2023 @06:45PM (#37082673)
We need a llama2 shim that runs locally and understand openai calls, then we can run all these funny software just changing our host file at no additional complexity for the implementer