News for hackers. Stuff that matters.
Re:Building Effective "Agents" by jascha_eng on Friday December 20, 2024 @07:31AM (#42470556)
I put the agents in quotes because anthropic actually talks more about what they call "workflows". And imo this is where the real value of LLMs currently lies, workflow automation.

They also say that using LangChain and other frameworks is mostly unnecessary and does more harm than good. They instead argue to use some simple patterns, directly on the API level. Not dis-similar to the old-school Gang of Four software engineering patterns.

Really like this post as a guidance for how to actually build useful tools with LLMs. Keep it simple, stupid.

Re:Building Effective "Agents" by curious_cat_163 on Friday December 20, 2024 @12:05PM (#42472766)
Indeed. Very clarifying.

I would just posit that they do make a distinction between workflows and agents

Re:Building Effective "Agents" by Philpax on Friday December 20, 2024 @05:15PM (#42475610)
Aren't you editorialising by doing so?
Re:Building Effective "Agents" by jascha_eng on Friday December 20, 2024 @05:29PM (#42475730)
I guess a little. I really liked the read though, it put in words what I couldn't and I was curious if others felt the same.

However the post was posted here yesterday and didn't really have a lot of traction. I thought this was partially because of the term agentic, which the community seems a bit fatigued by. So I put it in quotes to highlight that Anthropic themselves deems it a little vague and hopefully spark more interest. I don't think it messes with their message too much?

Honestly it didn't matter anyways, without second chance pooling this post would have been lost again (so thanks Daniel!)

Re:Building Effective "Agents" by rybosome on Friday December 20, 2024 @06:04PM (#42475996)
I felt deeply vindicated by their assessment of these frameworks, in particular LangChain.

I've built and/or worked on a few different LLM-based workflows, and LangChain definitely makes things worse in my opinion.

What it boils down to is that we are still coming to understand the right patterns of development for how to develop agents and agentic workflows. LangChain made choices about how to abstract things that are not general or universal enough to be useful.

Re:Building Effective "Agents" by wahnfrieden on Saturday December 21, 2024 @10:23AM (#42480156)
The article does not mention the LangChain framework. LangGraph is a different framework, have you tried it?
Re:Building Effective "Agents" by rybosome on Saturday December 21, 2024 @01:00PM (#42481099)
Yes, our previous lead dev built a lot of our infra using LangGraph. I've been slowly ripping it out since assuming ownership of this part of the codebase.

I've been replacing LangGraph with simple primitives, relying on native Python constructs, etc. For example, instead of building this verbose graph of computation with LangGraph, you can just...call functions in the order you want them. Or declare them async, add them to a list, then await the resolution of all of them.

For a time I was maintaining a spreadsheet of all the refactor PRs, and I had a cumulative reduction of over 1,000 lines of code from these changes. Eventually I stopped keeping track.

^ That's a 1K LOC reduction with no functionality changes. I feel pretty strongly that LangChain/LangGraph are a net negative for our use case.

Re:Building Effective "Agents" by temp_praneshp on Saturday December 21, 2024 @01:44PM (#42481371)
So all you achieved from a spreadsheet's worth of PRs was a 1k LoC reduction?
Re:Building Effective "Agents" by herecomethefuzz on Saturday December 21, 2024 @02:28PM (#42481636)
If you do any software engineering all, you would know that a 1k LoC reduction to achieve the same functionality at the same/better performance is non-trivial.
Re:Building Effective "Agents" by wahnfrieden on Saturday December 21, 2024 @05:05PM (#42482726)
They’re questioning whether it was a valuable use of time, not whether a spreadsheet of PRs was time-consuming which is apparent
Re:Building Effective "Agents" by ahmadawais on Monday December 23, 2024 @12:54AM (#42492155)
Couldn’t agree more. Btw what’s your use case?
Re:Building Effective "Agents" by ankit219 on Saturday December 21, 2024 @04:55AM (#42478651)
Deploying in production, the current agentic systems do not really work well. Workflow automation does. The reason is very native to LLMs, but also incredibly basic. Every agentic system starts with planning and reasoning module, where an LLM evaluates the task given and plans about how to accomplish that task, before moving on to next steps.

When an agent is given a task, they inevitably come up with different plans on different tries due to inherent nature of LLMs. Most companies like this step to be predictable, and they end up removing it from the system and doing it manually. Thus turning it into a workflow automation vs an agentic system. I think this is what people actually mean when they want to deploy agents in production. LLMs are great at automation*, not great at problem solving. Examples I have seen - customer support (you want predictability), lead mining, marketing copy generation, code flows and architecture, product specs generation, etc.

The next leap for AI systems is going to be whether they can solve challenging problems at companies - being the experts vs the doing the task they are assigned. They should really be called agents, not the current ones.

Re:Building Effective "Agents" by Kydlaw on Saturday December 21, 2024 @06:04AM (#42478930)
In fact they are mentioning LangGraph (the agent framework from the LangChain company). Imo LangGraph is a much more thoughtful and better built piece of software than the LangChain framework.

As I said, they already mention LangGraph in the article, so the Anthropic's conclusions still hold (i.e. KISS).

But this thread is going in the wrong direction when talking about LangChain

Re:Building Effective "Agents" by jascha_eng on Saturday December 21, 2024 @12:19PM (#42480880)
I'm lumping them all in the same category tbh. They say to just use the model libraries directly or a thin abstraction layer (like litellm maybe?) if you want to keep flexibility to change models easily.