The problem with vibe coding is more behavioral I think: the person more likely to jump in the bandwagon to avoid writing some code themselves is probably not the one thinking about long term architecture and craftsmanship. It’s a laziness enhancer.
Trying to get some third party hardware working with raspi
The hardware provider provides 2 separate code bases with separate documentation but only supports the latest one.
I literally had to force feed the newer code base into ChatGPT, and then feed in working example code to get it going, else it constantly reference the wrong methods.
If I just kept going Code / output / repeat it would maybe have stumbled on the answer but it was way off.
It only looks effective if you remove learning from the equation.
It's the wrong tool for the job, that's what it is.
I kind of view this use case as enhanced code linters
I quit my last very good job because I became so fed up with this situation. It was bad enough before the CTO started using LLMs. It was ABSURD after.
(This was a YC company that sold quickly after I quit, at a loss, presumably because they didn't know what else to do)
someone holding a hammer by the head and failing at getting the nail in doesnt mean a hammer is a bad tool for nailing
No part of what I said suggested the tool wasn't capable of being a useful tool.
It's effective on things that would take months/years to learn, if someone could reasonably learn it on their own at all. I tried vibe coding a Java program as if I was pair programming with an AI, and I encountered some very "Java" issues that I would not have even had the opportunity to get experience in unless I was lucky enough to work on a Fortune 500 Java codebase.
AI doesn't work in a waterfall environment. You have to be able to rapidly iterate, sometimes in a matter of hours, without bias and/or emotional attachment.
Throw a big context window model like Gemini at it to document the architecture unless good documentation exists. Then use modify that document to drive development of new or modified code.
Many big waterfall projects already have a process for this - use the AI instead of marginally capable offshore developers.
What do you mean? There is no difference between waterfall or agile in what you do during a few hours.
Are you being ironic?
Do you expect an incoming collapse of modern society?
That's the only case where LLM would be "not there anymore." Even if this current hype train dies completely, there will still businesses providing LLM interference, just far less new models. Thinking LLM would be "not there anymore" is even more delusional than thinking programmer as a job would cease to exist due to LLM.
And it’s better in the long run.
Part of the reason is that the reporters are themselves not in the trenches coding to be skeptical of the claims they hear.
"Technology acts as an amplifier of human intentions"
...so, if someone is just doing a sloppy job, AI-assisted vibe coding will enable them to do it faster.
Reading "couldn't" as, you would technically not be able to do it because of the complexity or intricacy of the problem, how did you guarantee that the change offered by the AI made proper sense and didn't leave out critical patterns that were too complex for you to detect ?
Your comment makes it sound like you're now dependent on AI to refactor again if dire consequences are detected way down the line (in a few months for instance), and the problem space is already just not graspable by a mere human. Which sounds really bad if that's the case.
If you can do it using an LLM in a few hours however, suddenly making your life, and the lives of everyone that comes after you, easier becomes a pretty simple decision.
AI is a sharp tool, use it well and it cuts. Use it poorly and it'll cut you.
Helping you overcome the activation barrier to make that redactor is great if that truly is what it is. That is probably still worth billions in the aggregate given git is considered billion dollar software.
But slop piled on top of slop piled on top of slop is only going to compound all the bad things we already knew about bad software. I have always enjoyed the anecdote that in China, Tencent had over 6k mediocre engineers servicing QQ then hired fewer than 30 great ones to build the core of WeChat...
AI isn't exactly free and software maintenance doesn't scale linearly
While that is true, AI isn’t going to make the big difference here. Whether the slop is written by AI or 6000 mediocre engineers is of no matter to the end result. One might argue that if it were written by AI at least those engineers could do something useful with their lives.
But if someone else were to do it for me I would gratefully review the merge request.
For me personally, the activation energy is higher when reviewing: it’s fun to come up with the solution that ends up being used, not so fun to come up with a solution that just serves as a reference point for evaluation and then gets immediately thrown away. Plus, I know in advance that a lot of cycles will be wasted on trying to understand how someone else’s vision maps onto my solution, especially when that vision is muddy.
Submitting LLM barf for review and not reviewing it should be grounds for termination. The only way I can envision LLM barf being sustainable, or plausible, is if you removed code review altogether.
What does it mean to have to review your own code as a separate activity? Do many people contribute code that they wrote but… never read?
> Submitting LLM barf
Oh right…
The concept and design were by that point iterated on, so it doesn’t happen that I need to rewrite a significant amount of code.
If you need an example, it's easy to add a debugging/logging statement like `console.log`, but if the coder committed and submitted the log statement, then they clearly didn't review the code at all, and there are probably much bigger code issues at stake. This is a problem even without LLMs.
If person A committed code that looks bad to person B, it just means person A commits bad code by the standard of person B, not that person A “does not review own code”.
Maybe it’s a subjective difference, same as you could call someone “rude” or you could say the same person “didn’t think before saying”.
There's a difference between not intellectually understanding something and not being able to refactor something because if you start pulling on a thread, you are not sure what will unravel!
And often there just isn't time allocated in a budget to begin an unlimited game of bug testing whack-a-mole!
My confidence in LLMs is not that high and I use Claude a lot. The limitations are very apparent very quickly. They're great for simple refactors and doing some busy work, but if you're refactoring something you're too afraid to do by hand then I fear you've simply deferred responsibility to the LLM - assuming it will understand the code better than you do, which seems foolhardy.
A lot of repetitive slight variations on the same easy to describe change sounds pretty good to ask an LLM to do quickly.
I’m very much able to understand the result and test for consequences, I wouldn’t think of putting code I don’t understand in production.
Not necessarily. It may have refactored the codebase in a way that is more organized and easier to follow.
> how did you guarantee that the change offered by the AI made proper sense and didn't leave out critical patterns that were too complex for you to detect ?
Perhaps extensive testing? Or a prayer.
Theres some fidelity loss but it works for text, because there's quite often so much redundancy.
However, I'm not sure this technique could work on code.
> Counterpoint: AI has help me refactor things where I normally couldn’t. Things like extracting some common structure that’s present in a slightly different way in 30 places, where cursor detects it, or suggesting potential for a certain pattern.
I have a feeling this post is going to get a lot of backlash, but I think this is a very good counterpoint. To be clear: I am not here to shill for LLMs nor vibe coding. This is a good example where an "all seeing" LLM can be helpful.
Whether or not you choose to accept the recommendation from the LLM isn't my main point. The LLM making you aware is the key value here.Recently, I was listening to a podcast about realistic real world uses for an LLM. One of them was a law firm trying to review details of a case to determine a strategy. One of podcasters (sp?) recoiled in horror: "An LLM is writing your briefs?" They replied: "No, no. We use it generate ideas. Then, we select best." It was experts (lawyers, in this case) using an LLM as a tool.
You can have AI almost generated anything, but even AI has limited to understanding requirements, if you cannot articulate what you want very precisely, it's difficult to get "AI" to help you with that.
In any case a very rare and specific corner case you mention, a dev can go on a decade or two (or lifetime or two) without ever experiencing similar requirement. If it should be a convincing argument for almighty llm it certainly isnt.
It’s like comparing a machine gun to a matchlock pistol and saying they’re the same thing.
AI is extremely new, impressive and is already changing things.
How can that be in any way be similar to crypto and VR?
Facebook of AI is coming and it's going to be much, much worse. and it'll still contain as you describe.
Nonetheless it's different things.
Just because they struggle with that makes all the other things wrong? Underwhelming?
I don't think so.
> Just because they struggle with that makes all the other things wrong?
No, it makes their grandiose claims very tenuous. But for me, yes I'm very underwhelmed by what AI is capable of. I think it's a useful tool, like a search box, but that's it at this point. That they are branding it as a Ph.D. researcher is just blowing smoke.
Not only did Claude respond correctly, I also wrote it in German.
And I often switch between English and German for single words just because ai is already that good.
This wasn't even imaginable a few years back. We had 0 technology for that.
I created single page html Javascript pages for small prototypes withhin 10 minutes with very little repromting.
I can literally generate an image of whatever I want with just text or voice.
I can have a discussion with my smartphone with voice.
Jules detected my programming language, how to build and running it by me writing 'generaten GitHub action for my project and add a basic test to it's
I don't get how people can't be amazed by these results.
What are you normally do when you try it out?
You're missing the point though. If it worked for you when you tried it, that's great, but we know these tools are stochastic, so that's not enough to call it "solved". That I tried it and it didn't work tells me it's not. And that's actually worse than it not working at all, because it leads to the false confidence you are expressing.
The strawberry example highlights like all abstractions, AI is a leaky one. It's not the oracle they're selling it as, it's just another interface you have to know the internal details of to properly use it beyond the basics. Which means that ultimately the only people who are really going to be able to wield AI are those who understand what I mean when I say "leaky abstraction". And those people can already program anyway, so what are we even solving by going full natural language?
> I created single page html Javascript pages for small prototypes withhin 10 minutes with very little repromting.
You can achieve similar results with better language design and tooling. Really this is more of an indictment of Javascript as a language. Why couldn't you write the small prototype in 10 minutes in Javascript?
> I can literally generate an image of whatever I want with just text or voice.
Don't get me wrong, I enjoy doing this all the time. But AI images are kind of a different breed because art doesn't ever have to be "right". But still, just like generative text, AI images are impressive only to a point where details start to matter. Like hands. Or character consistency. Multiple characters in a scene. This is still a problem in latest models of AI not doing what you tell it to do.
> I can have a discussion with my smartphone with voice.
Yes, talking to your phone is a neat trick, but is it a discussion? Interacting with AI often feels more like an improv session, where it just makes things up and rolls with vibes. Every time (and yes I mean every) I ask it about topics I'm an expert about, it gets things wrong. Often subtly, but importantly still wrong. It feels like less of a discussion and more like I'm being slowly gaslight. This is not impressive, it's frustrating.
I think you get my point. Yes, AI can be impressive at first, but when you start using it more and more you start to see all the cracks. And it would be fine if those cracks were being patched, or even if they just acknowledged them and the limitations of LLMs. But they won't do that; instead what they are doing is pretending they don't exist, pretending these LLMs are actually good, and pushing AI into literally everything and anything. They've been promising exponential improvements for years now, and still the same problems from day 1 persist.
And we live now in such a fast pace that it feels like Ai should be perfect already and it's of course not.
I also see that an expert can leverage Ai a lot better because you still need to know enough to make good things without.
But Ai progresses very fast and still has achieved things were we have not had any answer than before.
What it can already do is still very aligned of what I assume/expect it from the current hype.
Llama made our ocr 20% better just by using it. I prompted plenty of code snippets and stuff which saved me time and was fun using it. Including python scripts for a small ml pipeline and I normally don't write python.
It's the first tech demo ever which people around me just 'got' after I showed it to them.
It's the first chatbot I have seen which doesn't flake out after my second question.
Chatgpt pushed billions into new computer. Blackwell is the first chip to hit the lower estimation for brain compute performance.
It changed the research field of computer linguistics.
I believe it's fundamental to keep a very close eye on it and trying things out regular otherwise it will over roll us suddenly.
I really enjoy using Claude.
Edit: and eli5 on research paper. That's so so good
They would point to the increasing model sizes and the ability to solve various benchmarks as proof of that exponential rise, but things have really tapered off since then in terms of how rapidly things are changing. I was promised a Ph.D. level researcher. A 20% better result on your OCR is not that. That's not to say it's a good thing and an improvement, but it's not what they are selling.
> Blackwell is the first chip to hit the lower estimation for brain compute performance.
What does that even mean? That's just more hype and marketing.
But hey it seems I can't convince you of my enthusiasm regarding ai. It's fine I still will play around with it often and looking forward to it's progress.
Regarding your researcher: NotebookLM is great and you might need to invest a few hundred bucks to really try out more.
We will see anyway we're it's going
It isn't true.
The tool he's bragging about most went about changing JDK8 to JDK17 in a build config file, and if you're lucky tweaking log4j versions. 4,500 years my ass. It was more regex and AI
It doesn't matter if it's not working perfect yet. It's only a few years got 3 came out.
Having a reasonable chat with a computer was also not possible at all. AI right now already feels like another person.
You bought gasoline for the first cars in a pharmacy.
Indeed, which is exactly what's happening with AI.
Exactly. And this is why I feel like we are going to go full circle on this. We've seen this cycle in our industry a couple times now:
"Formal languages are hard, wouldn't it be great if we could just talk in English and the computer would understand what we mean?" -> "Natural languages are ambiguous and not precise, wouldn't it be great if we could use a formal langue so that the computer can understand precisely what we mean?"
The eternal hope is that someday, somehow, we will be able to invent a natural language way of communicating something precise like a program, and it's just not going to happen.
Why do I think this? Because we can't even use natural language to communicate unambiguously between intelligent people. Our most earnest attempt at this, the law, is so fraught with ambiguity there's an entire profession dedicated to arguing in the gray area. So what hope do we have controlling machines precisely in this way? Are future developers destined to be equivalent to lawyers, who have to essentially debate the meaning of a program before it's compiled, just to resolve the ambiguities? If that's where this ends up, I will be very sad indeed.
"Forty-two!" yelled Loonquawl. "Is that all you've got to show for seven and a half million years' work?"
"I checked it very thoroughly," said the computer, "and that quite definitely is the answer. I think the problem, to be quite honest with you, is that you've never actually known what the question is."
"But it was the Great Question! The Ultimate Question of Life, the Universe and Everything!" howled Loonquawl.
"Yes," said Deep Thought with the air of one who suffers fools gladly, "but what actually is it?"
A slow, stupefied silence crept over the men as they stared at the computer and then at each other.
"Well, you know, it's just everything... everything..." offered Phouchg weakly.
"Exactly!" said Deep Thought. "So once you do know what the question actually is, you'll know what the answer means."I think the closest might be the constructed language "Ithkuil". Learning it is....difficult, to put it mildly.
Future developers? You sound like you've never programmed in C++.
My take is more nuanced.
First, there is some evidence [1] that human language is neither necessary nor sufficient to enable what we experience as "thinking".
Second, our intuition, thinking etc are communicated via natural languages and imagery which form the basis for topics in the humanities.
Third, from communication via natural language - slowly emerges symbolism, and formalism which codifies intuitions in a manner which is operational, and useful.
As an example, socratic dialog was a precursor to euclidean geometry which operationally codifies our intuitions of space around us in a manner which becomes useful.
However, formalism is stale as there are always new worlds we experience which cannot be captured by any formalism. The genius of the human brain which is not yet captured in LLMs is to be able create symbolisms of these worlds almost on demand.
ie, if we were to order in terms of expressive power, it would be something like:
1) perception, cognition, thinking, imagination 2) human language 3) formal languages and computers codifying worlds experienced via 1) and 2)
Meanwhile, there is a provocative hypothesis [2] which argues that our "thinking" process lies outside computation as we know it.
[1] https://www.nature.com/articles/s41586-024-07522-w [2] https://www.amazon.com/Emperors-New-Mind-Concerning-Computer... [3] https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...
what we operationally mean by "precise" involves formalism. ie, there is an inherent contradiction between precise, and natural languages.
this matches the description of every codebase (except one) I came across in my 30-year career
LLMs will produce code with good structure, as long as you provide that architecture before hand.
I can only accept that as true if I also accept the fact that the vast majority of jobs won’t be on a good team
One thing especially, is the loss of knowledge about the codebase. While there was always some stackoverflow-coding, when seeing a weird / complicated piece of code, I used to be able to ask the author why it was like that. Now, I sometimes get the answer "idk, its what chatgpt gave me".
> except one
Please share more!"Code increases in complication to the first level where it is too complicated to understand. It then hovers around this level of complexity as developers fear to touch it, pecking away here and there to add needed features."
http://h2.jaguarpaw.co.uk/posts/peter-principle/
Seems like I'm not the first one to notice this either:
https://nigeltao.github.io/blog/2021/json-with-commas-commen...
I've noticed this over the last decade where tech people (of which I am one) have considered themselves above the problems of ordinary workers such as just affording to live. I really started to notice this in the lead up to the 2016 election where many privileged people did not recognize or just immediately dismissed the genuine anger and plight of working people.
This dovetails into the myth of meritocracy and the view that not having enough money or lacking basic necessities like food or shelter or a personal, moral failure and not a systemic problem.
Tech people in the 2010s were incredibly privileged. Earnings kept going up. There was seemingly infinite demand for our services. Life was in many ways great. The pandemic was the opportunity for employers to reign in runaway (from their perspective) labor costs.
Permanent layoff culture is nothing more than suppressesing wages. The facade of the warm, fuzzy Big Tech employer is long gone. They are defense contractors now. Google, Microsoft or Amazon are indistinguishable from Boeing, Lockheed Martin and Northrop Grumman.
So AI won't immediately replace you. It'll start by 4 engineers with AI being able to do the job that was previously done by 5. Laying off that one person saves that money directly but also suppresses the wages of the other 4 who won't be asking for raises. They're too afraid of losing their jobs. Then it'll be 3. Then 2.
A lot of people, particularly here on HN, are going to find out just how replaceable they are and how aligning with the interests of the very wealthiest was a huge mistake. You might get paid $500K+ a year but you are still a worker. Your interests align with nurses, teachers, baristas, fast food workers and truck drivers, not the Peter Thiels of the world.
And it's an understandable impulse, but at some point you'd think people would learn instead of being mesmerized by the promise of slightly better treatment by the higher classes in exchange for pushing down the rest of the working class.
Now it's our turn as software engineers to swallow that bitter pill.
They have already learned, fortunately the entire 20th century was devoted to this. That is why any worker with even a little bit of brain and ability to learn perceives socialists as the main threat to his well-being.
Of course, it's not like one needs to be a Marxist or even any other sort of a socialist to see the whole "employers are screwing their employees", since I do doubt that many employees working in Amazon's tech like AWS and whatnot would be in the ideology. In fact, that has become a fairly popular position even outside of traditionally leftist politics.
Only leftist politics will actually be able to address the issues. Of course people will just mindlessly scream "sOcIaLiSm!" at anything and everyone, but it ultimately doesn't matter. We still have to be optimistic, once enough people have the vocabulary to think about their increasingly parlous circumstance, things will change in our direction inevitably.
But these groups also don't have strong unions and generally don't have the class consciousness you are talking about, especially as the pay increases.
The "middle class" is propaganda we've been fed for decades by our governments, the media and the very wealthy. It's just another way to pit workers against one another, like how the flames of white supremacy were fanned after the slaves were freed so poor whites wouldn't socially align with freed slaves. It's why politics now tends to focus on socially divisive issues rather than economics, like abortion, LGBTQIA+ people, migrants, Islamophobia, etc.
Doctors are workers. Lawyers are workers. Professional athletes are workers.
They could buy/build a decent home in a safe neighborhood, had decent health care, good schools for their kids, disposable income for leisure.
Now, every single fucking productivity gain is going for the finance overlords.
In the future no one will have to code. Well compile the business case from UML diagrams!
My boss wanted me to put blockchain in everything (so he could market this to our clients). I printed a small sign and left on my desk. Everytime someone asked me about blockchain, I would point the sign "We don't need blockchain!"
I recently had the (dis)pleasure of fixing a bug in a codebase that was vibe coded.
It ends up being a collection of disorganized business problems converted into code, without any kind of structure.
Refinements are implemented as super-narrow patches, resulting in complex and unorganized code, whereas a human developer might take a step back to try and extract more common patterns.
And once you reach the limit of the context window you're essentially suck, as the LLM can no longer keep track of its patches.
English (or all spoken human language) is not precise enough to articulate what you want your code to do, and more importantly, a lot of time and experience precedes code that a senior developer writes.
If you want to have this senior developer 'vibe' code, then you'll need to have a way to be more precise in your prompts, and be able to articulate all learnings from your past mistakes and experience.
And that is incredibly heavy. Remember, this is opposite from answering 'why did you write it like this'. This is an endless list of items that say 'don't do this, but this, in this highly specific context'.