News for hackers. Stuff that matters.
Re:Do AI Companies Work? by AdieuToLogic on Monday September 30, 2024 @11:52PM (#41704489)
> Yes. Numbers / math is pretty much instant hallucination.

Programming is codified applied mathematics and involves numbers in all but the most trivial programs.

> LLMs are pretty human-like in their mental abilities.

LLM's are algorithms. Algorithms do not have "mental abilities", people do. Anthropomorphizing algorithms only serves to impair objective analysis of when they are, or are not, applicable.

> In the products we've built, there is an AI feature that generates aggregations of spreadsheet data. We have a dual unittest & aggregator loop to generate correct values.

> The first step is to generate some unittests. And in order to generate correct numerical data for unittests, we ask it to write some code with math expressions first. We interpret the expressions, and paste it back into the unittest generator - which then writes the unittests with the correct inputs / outputs.

> Then the aggregation generator then generates code until the generated unittests pass completely. Then we have the code for the aggregator function that we can run against the spreadsheet.

How is this not a classic definition of overfitting[0]?

Or is the generated code intentionally specific to, and only applicable for, a single spreadsheet?

0 - https://en.wikipedia.org/wiki/Overfitting

Re:Do AI Companies Work? by unoti on Tuesday October 01, 2024 @12:09AM (#41704545)
> Anthropomorphizing algorithms only serves to impair objective analysis of when they are, or are not, applicable.

Actually, in this case, comparing how we as humans think to how LLM's work is in fact useful. It's hard for us to eyeball a word and say how many consonants are in it, we need to count. I wouldn't ask a human to eyeball a tax return and tell me what the totals are reliably without giving them the tools to add things up. LLM's are the same way.

It's true that anthropomorphizing in general can be a trap, but when working with LLM's it can be a useful guide in pointing the way towards workable solutions.

Re:Do AI Companies Work? by AdieuToLogic on Tuesday October 01, 2024 @12:47AM (#41704681)
> Actually, in this case, comparing how we as humans think to how LLM's work is in fact useful.

Agreed. Contemplating the difference between what people and LLM's are is very useful IMHO. Understanding this is key to making informed decisions as to when LLM's can provide real value.

The assertion originally proffered, however, is quite different than your nuanced perspective:

> > LLMs are pretty human-like in their mental abilities.

It is this to which I object.

Re:Do AI Companies Work? by Ironchefpython on Tuesday October 01, 2024 @10:30AM (#41708938)
I think it would be more accurate to say that LLMs are pretty human-like in their limitations.
Re:Do AI Companies Work? by Mashimo on Tuesday October 01, 2024 @02:41AM (#41705260)
> Programming is codified applied mathematics and involves numbers in all but the most trivial programs.

I don't understand that. Can you expand on it?

When does the AI or the programmer need to do math calculations?

Re:Do AI Companies Work? by hsjdhdvsk on Tuesday October 01, 2024 @11:03PM (#41716782)
Everything a computer does is math...
Re:Do AI Companies Work? by Mashimo on Wednesday October 02, 2024 @03:01AM (#41717938)
Yes, but the programmer or the AI don't need to do the calculations in order to program.