> not much worse than a junior dev and 100x faster.
Is there a greater hell than this!?
If you actually take the time to tell it “hey, don’t do it this way,” it can definitely do it differently the next time.
On top of that, is anyone training models on their own codebase, and noting to AI which patterns are best practice and which aren’t?
There are a ton of ways to make it better than the baseline copilot experience
Especially given that you can ask an LLM to optimise code and on multiple runs it can not tell if it's is improving or degenerating.
Yes — junior management using LLMs and 100x more cocksure.
But like you said, in a few more years we'll see! It does feel like there's some missing pieces yet to be figured out to truly "reason" and generalize.
This makes me think of a quirk I discovered recently which is that ChatGPT simply won't generate a picture of a 'full glass of wine'. It generates pictures with all sorts of crazy waves/splashes in the glass but the glass is always half full no matter how you prompt it.
I'm not enough of an expert to make any deductions from this, but I think it hints at what the limitations of the currently models are.
So now my company makes more money, and the work gets done faster, but I can't say I feel appreciative. I'm sure it's great for founders though, for whom doing the work is merely an obstacle to having the finished product. For me, the work is the end goal, because I'm not hired to own the result.
I do the fun bit: having creative ideas and trying them out.
For an incremental improvement...not great, not terrible.
It does what the other poster said: it automates the boring parts of "this db model has eight fields that are mostly what you expect" and it autocompletes them mostly accurately.
That's not something an IDE does.
You're either completely misremembering what IDEs have been able to do up until 3 years ago, or completely misunderstanding what is available now. Even the very basic "autocomplete" functionality of IDEs is meaningfully better now.
It would be so nice to have a productivity Linux OS that just works on all my devices without tinkering. I want to stop supporting the closed source monopolies, but the alternatives aren't up to par yet. I am extremely hopeful that they will be once mega corps inevitably decay and people tire of the boom-bust cycle.
As technologists, we all want beautifully designed tools, and I'm increasingly seeing that these are only created by passionate and talented people who truly care about tech, unlike megacorps that only care about enriching their board and elite shareholders.
I'm able to do a so much more using LLMs (Mistral-Large, Qwen2.5 and R1 locally, Claude via API) than without them.
I have to get the IDE setup properly now.
I had a complex finance situation that I was struggling with, both from a mathematical/taxation perspective and a personal psychological finance hangup. I spent a few good hours talking to it through everything and had a mental breakthrough. To get the same kind of insight, I would have to pay a financial advisor AND a psychologist for several hours.
That all of this was free while someone calls it a "con" seems completely wrong
(I got my CFA cousin to look over the numbers and he agreed with R1's advice, fwiw)
Hopefully Mistral can copy their technique and give us a 123b reasoning model.
For medium complexity things, I can get them done quickly without manual coding if I have a clear understanding in mind of what the implementation should look like. I supply the requirements, design and strategy and it's fairly easy to "keep things on the rails". The "write a PRD first" hack (https://www.aiagentshub.net/blog/how-to-10x-your-development...) works pretty well. Agent with YOLO mode and terminal access rips, particularly if you have good tests.
For tasks where I know the spec of the feature but don't clearly understand how I would design / implement the feature myself, it's hit-and-miss. Mostly miss for me.
I also haven't had much success with niche libraries, have to stick to the most popular library/tool/framework choices or it will quickly get stuck.
They need trillions of dollars in returns. VC's won't finance tech startups for decades.
I use Cursor sometimes, and VSCode + Continue with llama.cpp, and it's great. That's not worth billions. It's definitely not worth trillions.
Now someone will respond about how it's just a stepping stone, and how the billions are justified by _something completely imaginary, and not invented yet, and maybe not ever_ e.g. agents.
The BigTech companies have been flush with liquidity and poured those hundreds of billions into the promising tech, and as result we got a wonderful new technology. There is not much need for those trillions in return - just look at liquidity positions of those companies, they are just fine. If those trillions come in eventually - even better.
Whilst you are correct that big tech cos do not need the return to survive, that's not how public markets work at all, and thus not how the incentives for those in charge of the companies work, and so making you actually wrong.
If investment in AI don't pan out (i do think that it will pan out, and those trillions will come) then those companies would just pour even more billions into whatever big thing/promise would come next. Rinse and repeat. Because some of those things do generate tremendous returns, and thus not playing that game is what really constitute true loss of money.
US right now is run by someone whose explicit promises, if actually implemented, have an obvious immedidiate 13-14% reduction in GDP — literally, never mind side effects, I'm not counting any businesses losing confidence in the idea that America is a place to invest, this is just direct impact.
DOGE + deportation by themselves do most of that percentage. The tariffs are a rounding error in comparison, but still bad on the kind of scale that gets normal politicians kicked out.
And yet, the markets are up.
I just want to know so that I can set a reminder and check back on your comment when the time arrives.
Just as they were convinced after Covid that they needed to put hiring into overdrive.
Tech management has the collective IQ of a flock of sheep.
The whole thing is like bitcoin. There’s too many people that benefit from maintaining the collective illusion.
I think if they could find a way to make their software good, instead of bad, like it increasingly is, that would be a good use of that money.
But I do think (and better understand) there is a failure to understand this at a higher abstraction. One part is simply "money is a proxy." This is an uncontestable fact. But one must ask "proxy for what?" and I think people only accept the naive simple answer. Unfortunately, this "is a proxy" concept is extremely generalization. Everything is an estimation, everything is an approximation, and most things are realistically intractable. We use sibling problems or similar problems to work with that are concrete, but there are always assumptions made and ignoring these can have disastrous consequences. Approximations are good (they're necessary even) but the more advanced a {topic,field,civilization,etc} gets, the more important it is to include higher order terms. Frankly, I don't think humans were built for that (though by some miracle we have the capacity to deal with it).
My partner and her dad are both economists, and one thing I've learned is that what many people think are "economics questions" are actually "business questions". I think a story from her dad makes this extremely clear. A government agency hired him to look at the cost benefit analysis of some stuff (like building a few hospitals and some other unambiguously beneficial institutions), and when he presented everyone was happy but had a final question "should we build them?" The answer? "That's not the role of an economist." The reason for this is because money can't actually be accurately attributed to these things. You can project monetary costs for construction, staffing, and bills, and you can make projections about how many people this will benefit, how it can reduce burdens elsewhere, and as well as make /some/ projections about potential cost savings. But you can't answer "should you." Because the weight of these values is not something that can be codified with any data. It is an importance determined by the public and more realistically their representatives. Very few times can you give a strong answer to a question like "should we build a new hospital" and essentially in only the extreme cases. I'll give another example. In my town there was an ER that was closed due to budget constraints. This ER was across the street to the local university, which students represent ~15% of the population. The next nearest ER? A 15 minute ambulance ride away and in the next town over. Did the city save money? Yes. Did the sister city's ER become even busier? Also yes. Did people lose access to medicine? Yes. Did people die? Also yes. Have economists put a price on human life? Also yes, but they are very clear that this is not a real life and a very naive assumptions[1]. It is helpful in the same way drawing random squiggles on a board can help a conversation. Any squiggles can really be drawn but the existence of _something_ helps create some point to start from.
[0] okay crypto bros, you're not wrong but low volatility is critical as well as some other aspects. Let's not get off topic
[1] https://www.npr.org/2020/04/23/843310123/how-government-agen...
That seems like a suspect claim. If you're saying that you, personally, cannot create billions of dollars in value with Cursor & friends that is certainly true - but you are in no position to make a judgement call about where the cap on value creation is for the LLM market is worth based on your personal use cases. LLMs don't just do code completion. We really can't estimate how much potential value is being created without doing some serious data diving and studying of cases.
A better argument would be that the DeepSeek experience suggests these companies have no moat and therefore no way to earn a return on capital. But LLMs are probably going to generate at least trillions of dollars in value because they're on par or ahead of Wikipedia and Google for answering many queries then they also have hundreds of ancillary uses like answering medical questions at weird hours or creative/professional writing.
Consider that Wikipedia is much bigger than Encyclopedia Britanica, but because it is given away to everyone for free, it is not counted as E.B.'s max sale price ($2900 in 1989?) times the world's internet connected population (5.6e9?) — $16 trillion.
AI, regardless of value, are priced at the marginal cost to reproduce weights or run inference depending on which you care about.
But I do mean "reproduce" not "invent" — it doesn't matter if DeepSeek's "a few million" was only possible because they benefited from published research, it just matters that they could.
And if the hardware is the bottleneck for inference, that profit goes to the hardware manufacturer, not to the top ten companies who made models.
That is a problem for the VC’s that bet wrong, not for the world at large.
The models exist now and they’ll keep being used, regardless of whether a bunch of rich guys lost a bunch of money.
These companies are heavily subsidized by investors and their cloud service providers (like Microsoft and Google) in an attempt to gain market share. It might actually work - but this situation, where a product is sold under cost to drum up usage and build market share, with the intent to gain a monopoly and raise prices later on - is sort of the definition of a bubble, and is exactly how the mobile app bubble, the dot-com bubble, and previous AI bubbles have played out.
How? I get that many devs like using them for writing code. Personally I don't, but maybe someday someone will invent a UX for this that I don't despise, and I could be convinced.
So what? That's a tiny market. Where in the landscape of b2b and b2c software do LLMs actually find market fit? Do you have even one example? All the ideas I've heard so far are either science fiction (just wait any day now we'll be able to...) or just garbage (natural language queries instead of SQL). What is this shit for?
Not since the advent of Google have I heard people rave so much about the usefulness of a new technology.
To make money though it just needs to have a large or important audience and a means of convincing people to think, want, or do things that people with money will pay to make people think, want or do.
Ads, in other words
A related question: has anyone figured out how to monetize LLM input? When a user issues a Google search query they're donating extremely valuable data to Google that can be used to target relevant ads to that user. Is anyone doing this successfully with LLM prompt text?
Various minor thing so far. For example I heard about ChatGPT being evaluated as a tool for providing answers for patients in therapy. ChatGPT answers were evaluated as more empathetic, more human and more aligned with guidelines of therapy than answers given by human therapists.
Providing companionship to lonely people is another potential market.
It's not as good as people at solving problems yet but it's already better than humans at bullshiting them.
I could see this being useful in a "dark pattern" sense, but only if it's incredibly cheap, to increase the cost to the user of engaging with customer support. If you have to argue with the LLM for an hour before being connected to an actual person who can help you, then very few calls will make it to the support staff and you can therefore have a much smaller team. But that only works if you hate your users.
Microsoft turned itself into a trillion dollar company off the back of enterprise SAAS products and LLMs are among the most useful.
My company uses them for a fuckton of things that were previously too intractable for static logic to work (because humans are involved).
This is mostly in the realm of augmented customer support (e.g. customer says something, and the support agent immediately gets the summarized answer on their screen)
It’s nothing that can’t be done without, but when the whole problem can be simplified to “write a good prompt” a lot of use cases are suddenly within reach.
It’s a question if they’ll keep it around when they realize it doesn’t always quite work, but at least right now MS is making good money off of it.
No, it's not. The first half of the article talks about how useless the actual product is, how the only reason we hear about it is because the media loves to talk about it.
What it goes into is how over hyped and over valued these companies are. They've blown through $5bn of compute each in a year and their revenue is abysmal. Microsoft won't report on ai separately, probably because it's abysmal.
I'm positive on LLMs for coding. But I think I have to agree with their assessment. Coding seems like the best area for these tools and what we see now is great. It's probably even worth $10b to the IT industry maybe eventually. But they're not paying for it yet, clearly. And I also think it's just not going to have huge significance outside our industry. The people I rub shoulders with outside of work have not mentioned or asked about it once, which is not necessarily meaningful but it does reveal the limits of hype too.
While the timeline is unclear; it seems likely that LLMs will obsolete precisely the skills that developers use to earn their income. I imagine a lot of them feel rather threatened by the rapid rate of progress.
Pointing out that it is already operating at junior dev quality and rapidly improving is unlikely to quiet the discontent.
If you think LLMs operate at "junior dev" capacity you either don't work with junior devs and is just bullshitting your way around here, or you just pick pretty awful junior devs.
LLMs are alright. An okay productivity tool, although its inconsistencies many times nullify productivity gains - By design they often spit out wrong results that look and sound very plausible. A productivity blackhole. Its mistakes are sometimes hard to spot, but pervasive.
Beyond that, if your think that all a dev does is spit out code, and since LLMs can spit out code it can replace devs in some imaginary timeline, you are sorely mistaken. The least part of my work is actually spitting out code, although it is the part I enjoy the most.
I honestly feel way nore threatened by economic downturns and the looming threat of recession. The only way LLMs threaten me is by being a wasteful technology that may precipitate a downturn in tech companies, causing more layoffs, etc nd so forth.
If you base it on ability, then an LLM can be be more useful to a good developer than 1 or more less competent "junior" team members (regardless of their age).
Not because it can do all the things like any "junior" can (like make coffee), but because the things it can do on top of what a "junior" can do, more than makes up for it.
Code is liability. The knowledge inside developers' heads is the corresponding asset. If you just produce code without the mental models being developed and refined, you're just increasing liability without the counterpart increase in assets.
I’ve hired lots of junior devs, some of them very capable. I’ve been in this industry for more than 15 years. LLMs operate at junior dev capacity, that’s pretty clear to me at this moment.
I'm in a middle. I enjoy Zed and its predictions, I utilize R1 to help me to reason. I do _not_ ever want to stop programming. And I see so often whenever somebody less experienced than me shows me look how Cursor did this with three prompts, can we merge? And the solution is just wrong and doesn't solve the hard issues.
For me the biggest issues are the people who want to see the craft of programming gone. But I do enjoy the tooling.
These types of articles are just catching the next meme wave, which will be hating on and making fun of "AI" of all sorts.
I’m not particularly worried. I think it’s obvious that software engineering is definitely an “intelligence complete” problem. Any system that can do software engineering can solve any problem that requires intelligence. So, either my job is safe or I get to live through the fall of almost all white collar disciplines. There’s not a huge middle ground.
Although perhaps this is just the programmer stereotype of thinking that if someone can code, they can do anything.
How about the middle ground where a human using AI replaces you?
The human job is (maybe) safe, but your job?
Developer productivity has gone up immensely in the last 50 years and the industry is larger than ever.
Disclaimer: I’m the developer behind CodeBeaver
The author spends a good amount of bytes telling us that they don't want to hear this argument even though they expect it.
Even the US Government is getting involved in subsidizing these companies and all of the infrastructure and resources needed to keep it expanding. We can look forward to even more methane power plants, more drilling, more fracking, more noisy data-centres sucking up fresh water from local reserves and increased damage to the environment that will come out of the pocket books of... ?
Update: And for what? "Deep Research"? Apparently it's not that great or world-changing for the costs involved. It seems that the author is tired of the yearly promise that everything is just a year or two away as long as we keep shovelling more money and resources into the furnace.
Unless closed models have significant advantage AI inference will be a commodity business - like server hosting.
I'm not sure that closed models will maintain an advantage.
Personally, I find that waiting for the code to generate, then reviewing the code carefully, then deciding if I need to rewrite it to be more painful, more error prone, and much slower than writing the code correctly.
Especially since this AI junior never learn from it’s mistakes.
I think it speaks to different approaches to how individuals write code.
> How great is it gonna be in a year or two?
I would bet that it’s about the same (not great code, generally), but the tools fail to generate responses less often and likely would have more context.
Hopefully they become fast enough to run offline or at least feel more instantaneous.
I happen to value human creativity.
Who? How? This is not what I've seen where I work. There's a bunch of hubbub and generalized excitement, and lots of talk about what could be done, or what might be done, but not very much actual doing. I must just be a clueless "mid".
Guido van Rossum - "I use it every day. My biggest adjustment with using Copilot was that instead of writing code, my posture shifted to reviewing code." https://www.youtube.com/watch?v=-DVyjdw4t9I
Here's Jeff Dean saying 25% of the characters in new PRs at Google are AI Generated. https://www.dwarkeshpatel.com/p/jeff-dean-and-noam-shazeer
Andrej Karpathy - "I basically can't imagine going back to "unassisted" coding at this point" https://www.reddit.com/r/singularity/comments/1ezssll/andrej...
Andrew Ng - "I run multiple models on my laptop — Mistral, Llama, Zefa. And I use ChatGPT quite often. " https://www.ft.com/content/2dc07f9e-d2a9-4d98-b746-b051f9352...
Simon Willison https://simonwillison.net/2023/Mar/27/ai-enhanced-developmen...
I mean I can keep going. I doubt you would compare yourself to these people.
Anyways, here goes.
1. Guido uses Copilot like I do - as a StackOverflow replacement to write the dumb boilerplate code. A much less flattering quote is "It doesn't save me much thinking, but [it helps] because I'm a poor typist". Also it's literally a minute or two of a three hour podcast.
2. A lot of code is autogenerated lol. Again, it's all the boring boilerplate stuff.
3. The cofounder of OpenAI is a biased source lol
4. He's an AI researcher, of course he runs that stuff.
5. Again, similar to Guido. He's using it for the boilerplate. Nothing wrong with enjoying using it as a toy, as he is here. But he's not doing serious work with it.
There's no virtue in hyping this stuff like a HODL bitcoin cultist.
>here's a bunch of hubbub and generalized excitement, and lots of talk about what could be done, or what might be done, but not very much actual doing
I am showing that indeed many top people use the tools to make themselves more productive, in direct contradiction to the comment above.
Listen: it's a fun toy. Engineers love toys and shiny distractions.
Don't confuse shiny rocks for gold.
Keep thinking that and don't feel too bad when 21 year old zoomers are 10x more impactful than you are at work.
1- he states that the generated code is most likely wrong. He is appreciative of it though because he is a very poor typer so he doesn't have to do that part as much
2- so that's not supporting your argument that the 'top' devs are using it. Besides it doesn't say how it's counted, nor how much time is spent reviewing and correcting it
3- actually okay. But is he using it for production code? Doesn't say
4- he definitely doesn't talk about coding, only brainstorming and writing text.
5- your best one. Still, the use case here is side projects not production
You might still be right, I definitely do not compare myself to these people, but trying to glue some sources together makes a poor argument.
And the subject on hand is more that just using LLMs, it's the role of LLMs in the dev work environment
It makes a better argument then the bold and plainly wrong claim that no one is using them and its all just "a bunch of hubbub".
None of the 5 sources say that AI code generation is really making them more productive
There's a Quentin Tarantino quote where he says there are 2 kinds of film critics. There are those who love movies and there are those that love the movies they love.
A lot of developers really seem to love the technology they love.
These people are where most of the negativity is coming from. And my guess is that the people who are encouraged by LLMs and not negative (mostly) aren't taking time out of their days to write long blog posts or argue about it online.
It's not the technology, it's the stupid overhype. It really feels like all the HODL bitcoin cultists have finally gotten over their lost apes and found a new technogod.
So many people in these threads are convinced it's about to gain sentience. That's not going to happen. You get the people outing themselves by saying "it does my job better than me!"
If you say something honest and direct like "their output is mediocre and unreliable" or "the RNG text generator is not capable of thinking, you're just Clever Hans-ing yourself" or "if it does your job better than you, that says more about you than about it", you get people clutching their pearls like a Stanford parent whose kid got a D.
arXiv has turned into a paper mill of AI startups uploading marketing hype cosplaying as "research".
So it's from middle management levels riding the hype train, and possibly trying to save money and getting bonuses for it at the expense of other people.
Just like when offshoring was in its same point on Gartner's hype curve.
"Everyone has a model, but no one has a business".