News for hackers. Stuff that matters.
Re:What Makes 5% of AI Agents Work in Production? by EdwardDiego on Tuesday October 07, 2025 @12:55AM (#45499498)
> One team suggested that instead of text-to-SQL, we should build semantic business logic layers, “show me Q4 revenue” should map to a verified calculation, not raw SQL generation.

Okay, how would that work though? Verified by who and calculated by what?

I need deets.

Re:What Makes 5% of AI Agents Work in Production? by dchftcs on Tuesday October 07, 2025 @01:34AM (#45499682)
On one side, you have an agent calculating the revenue.

On the other side, you have an SQL that calculates the revenue

Compare the two. If the two disagree, get the AI to try again. If the AI is still wrong after 10 tries, just use the SQL output.

Re:What Makes 5% of AI Agents Work in Production? by mnky9800n on Tuesday October 07, 2025 @10:15AM (#45503319)
so you have an answer and then you throw compute at trying to produce the answer in a different way.

What I hear is a billion dollar AI startup in the making!

Re:What Makes 5% of AI Agents Work in Production? by encyclopedism on Wednesday October 08, 2025 @09:30AM (#45515968)
Well put...lol :-)
Re:What Makes 5% of AI Agents Work in Production? by tirumaraiselvan on Tuesday October 07, 2025 @01:04AM (#45499545)
A simple way is perhaps implement a text-to-metrics system where metrics could be defined as SQL functions.
Re:What Makes 5% of AI Agents Work in Production? by moomoo11 on Tuesday October 07, 2025 @01:07AM (#45499562)
psychedelics
Re:What Makes 5% of AI Agents Work in Production? by meheleventyone on Tuesday October 07, 2025 @10:16AM (#45503341)
They're saying that someone should implement the CalculateQuarterRevenue(year, quarter) function somewhere in a manner that has been verified (e.g. run it against previous quarters to make sure it works correctly) then rather than using the LLM to generate SQL you use it to decide what domain function should be called. Which to me seems to mean that someone on the panel was gently taking the piss out of the idea. Since if you've done all the hardwork anyway presenting this in a deterministic way with a nice UX is straightforward bit of front end work.
Re:What Makes 5% of AI Agents Work in Production? by moduspol on Tuesday October 07, 2025 @12:48PM (#45505528)
It also removes a lot of the value of the LLM. They're perceived as being smart, and the interface (open-ended text) implies they are capable of more than executing pre-defined functions.

So if you have a "CalculateQuarterRevenue(year, quarter)" function, you'll soon find your users asking for the data per-month. Or just for the last six weeks. Or just for a specific client. And they'll be confused when it doesn't work.

Re:What Makes 5% of AI Agents Work in Production? by esafak on Tuesday October 07, 2025 @01:57PM (#45506452)
The conversational user interface is misleading then, isn't it? It can't make you a sandwich either, though it allows you to submit this request.
Re:What Makes 5% of AI Agents Work in Production? by moduspol on Tuesday October 07, 2025 @02:35PM (#45506945)
Yes. The sandwich example is contrived, but the basis is "discoverability." It's very opaque to the user what actually can be done and how reliable the result is.

Compare this to basically any website you've ever been to. It's the "GUIs vs. CLIs" discussion all over again, except even CLIs had man pages for discoverability.

Re:What Makes 5% of AI Agents Work in Production? by esafak on Tuesday October 07, 2025 @10:31AM (#45503536)
In other words, there should be a list of predefined queries, or possibly subqueries, that the user can request. This is basically how products used to work before AI. The difference is now you can request which query you want verbally.
Re:What Makes 5% of AI Agents Work in Production? by lesuorac on Tuesday October 07, 2025 @10:52AM (#45503824)
I assume you're being tongue in check but I've watched a lot of people use software and they really just don't know anything about it. Being able to verbally request something is something they can learn to do while googling how do I normalize the scores in my rubric to add up to 100 is something they couldn't.

Verbal queries is the solution for the world we have even if it's not optimal.

Re:What Makes 5% of AI Agents Work in Production? by slfnflctd on Tuesday October 07, 2025 @11:34AM (#45504450)
Your last sentence sums it up. This is what users want.

The main killer app, I think, boils down really expensive speech-to-text (and vice versa) with a reasonable number of seemingly authoritative querying details in fairly plain language. It's a new, 'better' search engine, just with different pitfalls people need to get up to speed on. And that may be enough, because employing humans to fill the same role as effectively is more expensive still.

Re:What Makes 5% of AI Agents Work in Production? by thr0w on Tuesday October 07, 2025 @10:45AM (#45503716)
So simple classification problem. Big deal.