News for hackers. Stuff that matters.
Re:My AI skeptic friends are all nuts by ofjcihen on Monday June 02, 2025 @05:18PM (#44163127)
I feel like we get one of these articles that addresses valid AI criticisms with poor arguments every week and at this point I’m ready to write a boilerplate response because I already know what they’re going to say.

Interns don’t cost 20 bucks a month but training users in the specifics of your org is important.

Knowing what is important or pointless comes with understanding the skill set.

Re:My AI skeptic friends are all nuts by mountainriver on Monday June 02, 2025 @05:33PM (#44163303)
I feel the opposite, and pretty much every metric we have shows basically linear improvement of these models over time.

The criticisms I hear are almost always gotchas, and when confronted with the benchmarks they either don’t actually know how they are built or don’t want to contribute to them. They just want to complain or seem like a contrarian from what I can tell.

Are LLMs perfect? Absolutely not. Do we have metrics to tell us how good they are? Yes

I’ve found very few critics that actually understand ML on a deep level. For instance Gary Marcus didn’t know what a test train split was. Unfortunately, rage bait like this makes money

Re:My AI skeptic friends are all nuts by Night_Thastus on Monday June 02, 2025 @06:05PM (#44163640)
Models are absolutely not improving linearly. They improve logarithmically with size, and we've already just about hit the limits of compute without becoming totally unreasonable from a space/money/power/etc standpoint.

We can use little tricks here and there to try to make them better, but fundamentally they're about as good as they're ever going to get. And none of their shortcomings are growing pains - they're fundamental to the way an LLM operates.

Re:My AI skeptic friends are all nuts by mountainriver on Monday June 02, 2025 @07:17PM (#44164369)
Most of the benchmarks are in fact improving linearly, we often don't even know the size. You can find this out but just looking at the scores over time.

And yes, it often is small things that make models better. It always has been, bit by slow they get more powerful, this has been happening since the dawn of machine learning

Re:My AI skeptic friends are all nuts by _dain_ on Monday June 02, 2025 @07:04PM (#44164239)
remember in 2022 when we "hit a wall"? everyone said that back then. turned out we didn't.

and in 2023 and 2024 and january 2025 and ...

all those "walls" collapsed like paper. they were phantoms; ppl literally thinking the gaps between releases were permanent flatlines.

money obviously isn't an issue here, VCs are pouring in billions upon billions. they're building whole new data centres and whole fucking power plants for these things; electricity and compute aren't limits. neither is data, since increasingly the models get better through self-play.

>fundamentally they're about as good as they're ever going to get

one trillion percent cope and denial

Re:My AI skeptic friends are all nuts by jhonof on Monday June 02, 2025 @10:13PM (#44165578)
The difference in quality between model versions has slowed down imo, I know the benchmarks don't say that but as a person who uses LLMs everyday, the difference between Claude 3.5 and the cutting edge today is not very large at all, and that model came out a year ago. The jumps are getting smaller I think, unless the stuff in house is just way ahead of what is public at the moment.
Re:My AI skeptic friends are all nuts by yahoozoo on Tuesday June 03, 2025 @07:10AM (#44168669)
Yet we are still at the “treat it like a junior” level
Re:My AI skeptic friends are all nuts by attemptone on Monday June 02, 2025 @06:05PM (#44163637)
>I feel the opposite, and pretty much every metric we have shows basically linear improvement of these models over time.

Wait, what kind of metric are you talking about? When I did my masters in 2023 SOTA models where trying to push the boundaries by minuscule amounts. And sometimes blatantly changing the way they measure "success" to beat the previous SOTA

Re:My AI skeptic friends are all nuts by mountainriver on Monday June 02, 2025 @07:18PM (#44164377)
Almost every single major benchmark, and yes progress is incremental but it adds up, this has always been the case
Re:My AI skeptic friends are all nuts by attemptone on Tuesday June 03, 2025 @02:35AM (#44167036)
We were talking about linear improvements and I have yet to see it
Re:My AI skeptic friends are all nuts by mountainriver on Tuesday June 03, 2025 @01:04PM (#44172179)
check the benchmarks or make one of your own
Re:My AI skeptic friends are all nuts by attemptone on Tuesday June 03, 2025 @04:20PM (#44174261)
I checked the BlEU-Score and Perplexity of popular models and both have stagnated around 2021. As a disclaimer this was a cursory check and I didn't dive into the details of how individuals scores were evaluated.
Re:My AI skeptic friends are all nuts by mountainriver on Wednesday June 04, 2025 @12:37PM (#44182596)
on what benchmarks? pretty much every major one is linear improvement
Re:My AI skeptic friends are all nuts by nickpsecurity on Monday June 02, 2025 @11:06PM (#44165879)
"pretty much every metric we have shows basically linear improvement of these models over time."

They're also trained on random data scraped off the Internet which might include benchmarks, code that looks like them, and AI articles with things like chain of thought. There's been some effort to filter obvious benchmarks but is that enough? I cant know if the AI's are getting smarter on their own or more cheat sheets are in the training data.

Just brainstorming, one thing I came up with is training them on datasets from before the benchmarks or much AI-generated material existed. Keep testing algorithmic improvements on that in addition to models trained on up to date data. That might be a more accurate assessment.

Re:My AI skeptic friends are all nuts by mountainriver on Tuesday June 03, 2025 @01:06PM (#44172202)
thats not a bad idea, very expensive though, and you end up with a pretty useless model in most regards.

A lot of the trusted benchmarks today are somewhat dynamic or have a hidden set.

Re:My AI skeptic friends are all nuts by nickpsecurity on Tuesday June 03, 2025 @07:59PM (#44176019)
That could happen. One would need to risk it to take the approach. However, if it was trained on legal data, then there might be a market for it among those not risking copyright infringement. Think FairlyTrained.org.

"somewhat dynamic or have a hidden set"

Are there example inputs and outputs for the dynamic ones online? And are the hidden sets online? (I haven't looked at benchmark internals in a while.)

Re:My AI skeptic friends are all nuts by mrkurt on Monday June 02, 2025 @05:19PM (#44163141)
PLEASE write your response. We'll publish it on the Fly.io blog. Unedited. If you want.
Re:My AI skeptic friends are all nuts by ofjcihen on Monday June 02, 2025 @05:28PM (#44163241)
I’m uninterested in giving you content. In particular because of your past behavior.

Thanks for the offer though.

Re:My AI skeptic friends are all nuts by throwaway314155 on Monday June 02, 2025 @05:57PM (#44163560)
> past behavior

Do go on.

Re:My AI skeptic friends are all nuts by tptacek on Monday June 02, 2025 @05:33PM (#44163301)
Kurt, how dare you.
Re:My AI skeptic friends are all nuts by ofjcihen on Monday June 02, 2025 @06:25PM (#44163841)
You wouldn’t happen to work for fly.io as well, would you?

Edit: Nm, thought I remembered your UN and see on your profile that you do.

Re:My AI skeptic friends are all nuts by grzm on Monday June 02, 2025 @06:25PM (#44163845)
Yes. And the author of the submission.
Re:My AI skeptic friends are all nuts by Mofpofjis on Monday June 02, 2025 @07:50PM (#44164642)
OMG I didn't notice. Way to burn down a huge amount of respect stemming from past cryptography work in just one blog post.
Re:My AI skeptic friends are all nuts by dogecoinbase on Monday June 02, 2025 @11:17PM (#44165924)
it's irksome to see tptacek having his patio11 moment for sure.
Re:My AI skeptic friends are all nuts by tptacek on Tuesday June 03, 2025 @12:03AM (#44166147)
What does this even mean? (I should be so lucky.)
Re:My AI skeptic friends are all nuts by kubb on Monday June 02, 2025 @05:26PM (#44163225)
Maybe make a video of how you're vibecoding a valuable project in an existing codebase, and how agents are saving you time by running your tools in a loop.
Re:My AI skeptic friends are all nuts by metaltyphoon on Monday June 02, 2025 @05:52PM (#44163514)
Seriously… thats the one thing I never see being posted? Is it because Agent mode will take 30-40 minutes to just bookstrap a project and create some file?
Re:My AI skeptic friends are all nuts by csallen on Monday June 02, 2025 @06:12PM (#44163710)
It takes like 2-6 minutes to do that, depending on the scope of the project
Re:My AI skeptic friends are all nuts by sensanaty on Monday June 02, 2025 @08:53PM (#44165085)
Well no, the reality of this workflow is the farcica, abject failures unleashed on the Dotnet codebase a week ago.

https://news.ycombinator.com/item?id=44050152

Re:My AI skeptic friends are all nuts by andrepd on Monday June 02, 2025 @06:14PM (#44163732)
So they can cherry pick the 1 out of 10 times that it actually performs in an impressive manner? That's the essence of most AI demos/"benchmarks" I've seen.

Testing for myself has always yielded unimpressive results. Maybe I'm just unlucky?

Re:My AI skeptic friends are all nuts by kubb on Tuesday June 03, 2025 @02:18AM (#44166923)
Livestream would be fair.
Re:My AI skeptic friends are all nuts by briandrupieski on Monday June 02, 2025 @05:26PM (#44163219)
> with poor arguments every week

This roughly matches my experience too, but I don't think it applies to this one. It has a few novel things that were new ideas to me and I'm glad I read it.

> I’m ready to write a boilerplate response because I already know what they’re going to say

If you have one that addresses what this one talks about I'd be interested in reading it.

Re:My AI skeptic friends are all nuts by slg on Monday June 02, 2025 @06:01PM (#44163606)
>> with poor arguments every week

>This roughly matches my experience too, but I don't think it applies to this one.

I'm not so sure. The argument that any good programming language would inherently eliminate the concern for hallucinations seems like a pretty weak argument to me.

Re:My AI skeptic friends are all nuts by simonw on Monday June 02, 2025 @06:10PM (#44163688)
Why does that seem weak to you?

It seems obviously true to me: code hallucinations are where the LLM outputs code with incorrect details - syntax errors, incorrect class methods, invalid imports etc.

If you have a strong linter in a loop those mistakes can be automatically detected and passed back into the LLM to get fixed.

Surely that's a solution to hallucinations?

It won't catch other types of logic error, but I would classify those as bugs, not hallucinations.

Re:My AI skeptic friends are all nuts by slg on Monday June 02, 2025 @06:23PM (#44163812)
>It won't catch other types of logic error, but I would classify those as bugs, not hallucinations.

Let's go a step further, the LLM can produce bug free code too if we just call the bugs "glitches".

You are making a purely arbitrary decision on how to classify an LLM's mistakes based on how easy it is to catch them, regardless of their severity or cause. But simply categorizing the mistakes in a different bucket doesn't make them any less of a problem.

Re:My AI skeptic friends are all nuts by layer8 on Monday June 02, 2025 @06:52PM (#44164128)
I don’t see why an LLM wouldn’t hallucinate project requirements or semantic interface contracts. The only way you could escape that is by full-blown formal verification and specification.
Re:My AI skeptic friends are all nuts by ofjcihen on Monday June 02, 2025 @06:16PM (#44163753)
A good example of where a linter wouldn’t work is when the LLM has you import a package that doesn’t exist.
Re:My AI skeptic friends are all nuts by jonas21 on Tuesday June 03, 2025 @12:06AM (#44166158)
It'll run it through the type checker, it will fail, and the LLM will go off and fix it (all without user intervention).
Re:My AI skeptic friends are all nuts by ofjcihen on Monday June 02, 2025 @06:08PM (#44163673)
It’s a confusing one for sure.

To be honest I’m not sure where the logic for that claim comes from. Maybe an abundance of documentation is the assumption?

Either way, being dismissive of one of LLMs major flaws and blaming it on the language doesn’t seem like the way to make that argument.

Re:My AI skeptic friends are all nuts by kubb on Monday June 02, 2025 @05:31PM (#44163280)
There's also the reverse genre: valid criticism of absolutely strawman arguments that nobody makes.
Re:My AI skeptic friends are all nuts by tptacek on Monday June 02, 2025 @05:34PM (#44163311)
Which of the arguments in this post hasn't occurred on HN in the past month or so?
Re:My AI skeptic friends are all nuts by sethev on Monday June 02, 2025 @07:56PM (#44164687)
Or in this very thread, for that matter.
Re:My AI skeptic friends are all nuts by kubb on Tuesday June 03, 2025 @07:00AM (#44168604)
"I tried copilot 2 years ago and I didn’t like it."

Great article BTW, it’s amazing that you’re now blaming developers smarter than you for lack of LLM adoption, as if it weren’t enough for the technology to be useful to become widespread.

Try to deal with „an agent takes 3 minutes to make a small transformation to my codebase and it takes me another 5 to figure out why it changed what it did only to realize that it was the wrong approach and redo it by hand, which took another 7 minutes” in your next one.

Re:My AI skeptic friends are all nuts by csallen on Monday June 02, 2025 @05:22PM (#44163167)
Can you direct me somewhere with superior counterarguments? I'm quite curious
Re:My AI skeptic friends are all nuts by therealpygon on Monday June 02, 2025 @06:59PM (#44164194)
“Valid” criticism rarely come from the people barely capable of understanding the difference between AI and LLMs, and using them interchangeably.
Re:My AI skeptic friends are all nuts by calf on Monday June 02, 2025 @06:09PM (#44163682)
What valid AI criticisms? Most criticisms of AI are not very deep nor founded in complexity theoretic arguments, whereas Yann LeCun himself gave an excellent 1 slide explanation of the limits of LLMs. Most AI criticisms are low quality arguments.