I have no interest in learning the horrible unintuitive UX of every CLI I interact with, I'd much rather just describe in English what I want and have the computer figure it out for me. It has practically never failed me, and if it does I'll know right away and I can fall back to the old method of doing it manually. For now it's saving me so much time with menial, time-wasting day-to-day tasks.
Most of those people are a bit bad at making their case. What they mean but don't convey well is that AI is useless for it's proclaimed uses.
You are correct that LLMs are pretty good at guessing this kind of well-documented & easily verifiable but hard to find information. That is a valid use. (Though, woe betide the fool who uses LLMs for irreversible destructive actions.)
The thing is though, this isn't enough. There just aren't that many questions out there that match those criteria. Generative AI is too expensive to serve that small a task. Charging a buck a question won't earn the $100 billion OpenAI needs to balance the books.
Your use case gets dismissed because on it's own, it doesn't sustain AI.
But it can be simultaneously true that LLMs add a lot of value to some tasks and less to others —- and less to some people. It’s a bit tautological, but in order to benefit from LLMs, you have to be in a context where you stand to most benefit from LLMs. These are people who need to generate ideas, are expert enough to spot consequential mistakes, know when to use LLMs and when not to. They have to be in a domain where the occasional mistake generated costs less than the new ideas generated, so they still come out ahead. It’s a bit paradoxical.
LLMs are good for: (1) bite-sized chunks of code; (2) ideating; (3) writing once-off code in tedious syntax that I don’t really care to learn (like making complex plots in seaborn or matplotllib); (4) adding docstrings and documentation to code; (5) figuring out console error messages, with suggestions as to causes (I’ve debugged a ton of errors this way — and have arrived at the answer faster than wading through Stackoverflow); (6) figuring out what algorithm to use in a particular situation; etc.
They’re not yet good at: (1) understanding complex codebases in their entirety (this is one of the overpromises; even Aider Chat’s docs tell you not to ingest the whole codebase); (2) any kind of fully automated task that needs to be 100% deterministic and correct (they’re assistants); (3) getting math reasoning 100% correct (but they can still open up new avenues for exploration that you’ve never even thought about);
It takes practice to know what LLMs are good at and what they’re not. If the initial stance is negativity rather than a growth mindset, then that practice never comes.
But it’s ok. The rest of us will keep on using LLMs and move on.
And it just isn't that thing. Or, rather, it is super intelligent but lacks any wisdom at all; thus rendering it useless for how it's being sold to me.
>which is at the early adoption stage
I've said this in other places here. LLM's simply aren't at early adoption stage anymore. They're being packaged into literally every saas you can buy. They're a main selling point for things like website builders and other direct to business software platforms.
I don’t use anything other than ChatGPT 4o and Claude Sonnet 3.5v2. That’s it. I’ve derived great value from just these two.
I even get wisdom from them too. I use them to analyze news, geopolitics, arguments around power structures, urban planning issues, privatization pros and cons, and Claude especially is able to give me the lay of the land which I am usually able to follow up on. This use case is more of the “better Google” variety rather than task-completion, and it does pretty well for the most part. Unlike ChatGPT, Claude will even push back when I make factually incorrect assertions. It will say “Let me correct you on that…”. Which I appreciate.
As long as I keep my critical thinking hat on, I am able to make good use of the lines of inquiry that they produce.
Same caveat applies even to human-produced content. I read the NYTimes and I know that it’s wrong a lot, so I have to trust but verify.
And it's just not.
We made a scavenger hunt full of puzzles and riddles for our neighbor's kids to find their Christmas gifts from us (we don't have kids at home anymore, so they fill that niche and are glad to because we go ballistic at Christmas and birthdays). The youngest of the group is the tech kid.
He thought he fixed us when he realized he could use chatgpt to solve the riddles and cyphers. It recognized the Caesar letter shift to negative 3, but then made up a random phrase with words the same length to solve it. So the process was right, but the outcome was just outlandishly incorrect. It wasted about a half hour of his day. . .
Now apply that to complex systems or just a simple large database, hell, even just a spreadsheet. You check the process, and it's correct. You don't know the outcome, so you can't verify unless you do it yourself. So what's the point?
For context, I absolutely use LLM's for things that I know roughly, but don't want to spend the time to do. They're useful for that.
They're simply not useful for how they're being marketed, which is too solve problems you don't already know.
Where are you being told all of these things? I haven't heard anything like it.
But my counterargument is that what I find to be so powerful about the LLMs is the ability to refine my question, narrow in on a tangent and then pull back out, etc. And *then* I can take its final outcome and cross reference it. With the old way of doing things, I often felt like I was stumbling in the dark trying to find the right search string. Instead I can use the LLM to do the heavy lifting for me in that regard.
It's well and ok with things you can botch with no consequence other than some time wasted. But I've bricked enough VMs trying commands I did not understand to know that if you need to not fuck up something you'll have to read those docs and understand them. And hope they're not out of date / wrong.
Pick a hammer, not a shitty hammer factory to assemble bits of hammer.
The best ffmpeg and regex command generators
It's useful for that yes, but I'd rather just live in a world where we didn't have such disasters of CLI that are git and ffmpeg.
LLMs are very useful for generating the obscure boilerplate needed because the underlying design is horrible. Relying on it means acquiescing to those terrible designs rather than figuring out redesigns that don't need the LLMs. For comparison, IntelliJ is very good at automating all the boilerplate generation that Java imposes on me, but I'd rather we didn't have boilerplate languages like Java, and I'd rather that IntelliJ's boilerplate generation didn't exist.
I fear in many cases that if an LLM is solving your problem, you are solving the wrong problem.
These days, I'm more likely to read the manual pages and take notes on interesting bits. If I'm going to rely on some tooling for some time, dedicating a few hours of reading is a good trade-off for me. No need to even remember everything, just the general way it solves the problem. Anything more precise is captured in notes, scripts, shell history,... I dare anyone to comes out with an essay like this from LLMs: https://karthinks.com/software/avy-can-do-anything/
I'm asking not for snark, but because when AI gives me something not _quite_ working, it requires much more time than what a "every 6 minutes in 10 hour work day" frame would allow to investigate. I just wonder if maybe you're pasting it as is and don't care about correctness if the happy path sort of works. Speaking of subsets, coders who did that before AI were also quite a group.
There must be something that explains the difference in our experiences. Apologies for the fact that my only idea is kinda negative. I understand the potential hyperbola here, but it doesn't explain much. I can stand AI BS once a day, maybe twice, before uncontrollably cursing into the chat.
Don't start when you're already in a buggy dead-end. Test-driven development with LLMs should be done right from the start.
Also keep the code modular so it is easy to include the correct context. Fine-grained git commits. Feature-branches.
All the tools that help teams of humans of varying levels of expertise work together.
I'd also do code reviews on the code AI produces.
In my experience, Claude Sonnet 3.5 (3.6?) has been unbeatable. I use it for Rust. Making sense of compiler errors, rubberducking, finding more efficient ways to write some function and, truth be told, some times just plain old debugging. More than once, I've been able to dump a massive module onto the chat context and say "look, I'm experiencing this weird behavior but it's really hard to pin down what's causing it in this code" and it pointed to the exact issue in a second. That alone is worth the price of admission.
Way better than ChatGPT 4o and o-1, in my experience, despite me saying the exact opposite a few months ago.
It helps me get to an answer a little bit quicker, but it doesn't perform any absolutely groundbreaking work for me.
I don't know how useful it would be for my job, where I do maintenance on a pretty big app, and develop features on this pretty big app. But it could be great, I just don't know because work only allows Copilot. And Copilot is somewhere between annoying and novelty in my opinion.
It's a valid conversation after ~3 years of anticipating the world to be disrupted by this tech. So far it has not delivered.
Wikipedia did not change the world either, it's just a great tool that I use all the time
As for software, it performs ok. I give up on it most of the time if I am trying to write a whole application. You have to acquire a new skill, prompt engineering, and feverish iteration. It's a frustrating game of whack-a-mole and I find it quicker to write the code myself and just have the LLM help me with architecture ideas, bug bashing, and it's also quite good at writing tests.
I'd rather know the code intimately so I can more quickly debug it than have an LLM write it and just trust it did it well.
Generally people are resistant to change and the average person will typically insist new technologies are pointless.
Electricity and the airplane were supposed to be useless and dangerous dead ends according to the common person: https://pessimistsarchive.org/
But we all like to think we have super unique opinions and personalities, so "this time it's different."
When the change finally happens, people go about their lives as if they were right all along and the new technology is simply a mysterious and immutable fixture of reality that was always there.
That aside optimist are also not always right, otherwise we would have cold fusion already and have a base on mars.
Are you suggesting that anything which is hyped is the future? Like, for every ten heavily-hyped things, _maybe_ one has some sort of post-hype existence.
A world where Segway never happened would be nearly indistinguishable from our own.
Not the most popular, especially these days, but they are very much descended from Segways and have their own fans.
Similarly, being optimistic doesn't mean you have to believe every single early-stage invention will work out no matter how unpromising - I've been enthusiastic about deep learning for the past decade (for its successes in language translation, audio transcription, material/product defect detection, weather forecasting/early warning systems, OCR, spam filtering, protein folding, tumor segmentation, spam filtering, drug discovery and interaction prediction, etc.) but never saw the appeal of NFTs.
Additionally worth considering that the cost of trying something is often lower than the reward of it working out. Even if you were wrong 80% of the time about where to dig for gold, that 20% may well be worth it; reducing merely the frequency of errors is often not logically correct. It's useful in a society to have people believe in and push forward certain inventions and lines of research even if most do not work out.
I think xvector's point is about people rehashing the same denunciations that failed to matter for previous successful technologies - the idea that something is useless because it's not (or perhaps will never be) 100.0% accurate, or the "Until it can do dishes, home computer remains of little value to families"[0] which I've seen pretty much ad verbatim for AI many times (extra silly now that we have dishwashers).
Given in real life things have generally improved (standard of living, etc.), I think it has typically been more correct to be optimistic, and hopefully will be into the future.
I'd love to hear more about how you utilised AI for this.
Personally I'm struggling to find it more useful than a slightly fancy code completion tool
Does this alone not increase your productivity exponentially? It does mine. I personally read code faster than I write it so it is an undeniable boon.
For example, personal projects that are small and where copilot has access to all the context it needs to make a suggestion - such as a script or small game - it has been really useful.
But in a real world large project for my day job, where it would need access to almost the entire code base to make any kind of useful suggestion that could help me build a feature, it's useless! And I'd argue this is when I need it.
Check out these tips for using Aider (a CLI based LLM code assisntant) https://aider.chat/docs/usage/tips.html
It can ingest the entire codebase (up to its context length), but for some reason, I’ve always had much higher quality chats with smaller bite-sized pieces of code.
There was a different thread on this site i read where a journalist used the wrong units of measurement (kilowatts instead of killowatt-hours for energy storage). You could paste the entire article into chatGPT with a prompt "spot mistakes in the following; [text]" and get an appropriate correction for this and similar mistakes the author made.
As in there's journalists right now posting articles with clear mistakes that could have been proof read more accurately than they were if they were willing to use AI. The only excuse i could think of is resistance to change. A lot of professions right now could do their job better if they leant on the current generation of AI.
It's still useful to a small subset of all those professions - the early adopters. Same way computers were useful to many professionals before the UI (but only a small fraction of them had the skillset to use terminals)
How so? How are you using LLMs to practice law? Genuinely curious.
There's even reported cases of entire legislations being written with LLMs already [1]. I'm sure there's thousands more we haven't heard about - the same way researchers are writing papers w/ LLMs w/o disclosing it
[1] https://olhardigital.com.br/2023/12/05/pro/lei-escrita-pelo-...
Like, this is malpractice, surely?
Some people expect AI to never make mistakes when doing jobs where people routinely make all kinds of mistakes of varying severity.
It’s the same as how people expect self-driving cars to be flawless when they think nothing of a pileup caused by a human watching a reel while behind the wheel.
So, the mechanic who maintenanced it last?
...
We don't fault our tools, legally. We usually also don't fault the manufacturer, or the maintenance guy. We fault the people using them.
If you’re not using ai in your practice, you’re doing a disservice to your clients.
I use a custom prompt to adjust the tone, but that’s about it.
imo while many here on HN debate the future of SWEs in the era of LLMs, there is no debate about future of many legal jobs - they will disappear
Interesting, I'm the opposite now. Why would I click a couple links to read a couple (verbose) blog posts when I can read a succinct LLM response. If I have low confidence in the quality of the response then I supplement with Google search.
I feel near certain that I am saving time with this method. And the output is much more tuned to the context and framing of my question.
Hah, take for example my last query in ChatGPT:
> Are there any ancient technologies that when discovered furthered modern understanding of its field?
ChatGPT gave some great responses, super fast. Google also provides some great results (though some miss the mark), but I would need to parse at least three different articles and condense the results.
To be fair, ChatGPT gives some bad responses too.. But both an LLM and Google search should be used in conjunction to perform a search at the same time.
Use LLMs as breadth-first search, and Google as depth-first search.
I could have google it myself in the evenings and had the answer in a few days of research, but with o1 in a 15min session my wife had had a solid weekly routine, the reasoning about those choices and academic papers with research about those products. (Obviously she knows a lot about skincare in general, so she had the capacity to recognize any wrong recommendation).
Nothing game changer, but is great to save lots of time in this kind of tasks.
If you're relying on AI to replace a human doctor trained in skin care or alternatively, your Google skills; please consider consulting an actual doctor.
If she "knows a lot about skincare in general, so she had the capacity to recognize any wrong recommendation", then what did AI actually accomplish in the end.
No worries, I can tell you what to expect: nothing. No effect. Zilch. Nada. Zero. Those beauty creams are just a total scam and that's obvious from the fact they're targetted just as well to women who don't need them (young, good skin) as to ones who do (older, bad skin).
About the only thing the beauty industry has figured out really works in the last five or six decades is Tretinoin, but you can use that on its own. Yet it's sold as one component in creams with a dozen others, that do nothing. Except make you spend money.
_____________
[1] Topical tretinoin for treating photoaging: A systematic review of randomized controlled trials (2022)
From Wikipedia[2]: "Isotretinoin is a teratogen; there is about a 20–35% risk for congenital defects in infants exposed to the drug in utero, and about 30–60% of children exposed to isotretinoin prenatally have been reported to show neurocognitive impairment".
See also pages like r/AccutaneRecovery[3] for people harmed by using it for acne, reporting systemic damage, perhaps permanent damage.
Scroll down[1] for the picture of some of the possible side effects of oral Accutane/Isotretinoin on the mother[3] and note that Wikipedia says "the most common adverse effects are dry lips (cheilitis), dry and fragile skin (xeroderma), dry eyes[8] and an increased susceptibility to sunburn" and wonder how a beauty treatment which improves skin condition has most common side effects which ruin skin condition.
This line of inquiry leads to a fun conspiracy/woo hypothesis; Grant Genereux[5]'s claims that what it does is trigger stem cells to differentiate in the epithelial layers of the skin, which makes thicker skin in the short term (wrinkle free) and worn out stem cells and thick skin in the longer term - and that many small vessels in the body have an epithelial lining of 'internal skin' and that thickens by the same mechanism leading to narrowing and closing of all kinds of internal vessels - tear ducts and sweat glands and blood vessels and inside the kidneys and liver and inner ear, etc. which cause the dry skin and dry eyes "side effects" (direct effects really) seen outside, and the organ damage/dizziness/etc. seen inside. And that it's a teratogen by getting inside cells, damaging them, damaging the DNA/protein building mechanisms causing wider systemic damage which can be long term and is not cleared up by stopping taking Accutane, this is misunderstood as retinoids "mediating hundreds of gene expressions" but is really shotgun chaotic damage and that's why there isn't a single symptom to look for and how it gets diagnosed as many different organ-specific diseases instead of retinoid toxicity damage. And/or causing cellular apoptosis with immune system response to a percieved 'attack', which is then seen as organ damage with immune system activity present, and misdiagnosed as "autoimmune" where the immune system has decided to attack an organ for no reason, which is why autoimmune disorders never have treatments or cures and why they cluster (people with one often get more) despite no good reason that should happen.
And that this whole collection of behaviours is triggered by food with Vitamin A (retinol in the tretinoin family) in it such as dairy and meat fat and Cod Liver oil, and foods with Beta Carotene (retinoids in the same family) such as orange/yellow/dark green coloured fruits and vegetables, and fortified Vitamin A in low-fat dairy and flours and other products through the USA/Europe. And it doesn't take much more than the RDA of Vitamin A to become problematic, and once it builds up in the body beyond the level the body can handle over a few decades, it's like blue touch paper waiting to be lit. Which, he suggests, is why auto-immune disorders cluster together (if you get one, you likely get more), why Eastern Canada Prince Edward Island near a Cod Liver Oil refinery was the highest incidence of Alzheimers in the world and that has been dropping since the refinery closed, and many more connection-between-retinoids-and-disease-states including claims by other people[6].
(I called it a 'fun' idea - it is at least fun along the lines of Tyler Vigen's spurious correlation noticer. https://www.tylervigen.com/spurious-correlations even if the main idea is not true).
[1] https://www.acne.org/accutane-in-pregnancy
[2] https://en.wikipedia.org/wiki/Isotretinoin
[3] https://www.acne.org/wp-content/uploads/2021/12/04-accutane-...
[4] https://www.reddit.com/r/AccutaneRecovery/comments/1c28jgg/a...
[5] https://ggenereux.blog/my-ebooks/
[6] https://nutritionrestored.com/blog-forum/topic/the-known-his... - blog post about paper observing hypervitaminosis-A bone damage in fossil skeletons, articles observing hysteria in Eskimo women, speculated to be caused by Vitamin A toxicity from Atlantic fish liver (callback to Atlantic coast cod-liver oil refinery), connection between hypervitaminosis A and calcified arteries, connection between hypervitaminosis A and scoliosis, hypervitaminosis A in small animals causes bone growth problems and symptoms of depression.
LLMs as a popularized thing is just about 2 years old. It is still mainly early adopters.
For smartphones it might have taken 10 to 15 years to gain widespread traction.
I think it is safe to say that we are only scratching the surface.
I just disagree with this. Every b2b or saas is marketing itself as using hallucination free AI.
We're waaaaayyyyyy past the early adoption stage, and the product hasn't meaningfully improved.
Sounds like early criticisms of the internet. I assume you mean he should be doing those things with a search engine, but maybe we shouldn't allow that either. Force him to use a book! It may be slightly less convenient, and could still be wrong, but...
I’ll ask her what her use cases are and reply here later if I don’t forget.
When preparing for a summit, she gave it a list of broad topics she wanted to cover. Gpt generated a list of specific titles and descriptions for her talks. This in turn gave her specific ideas to write talks about instead of just the broad topic.
When she wasn’t sure about the sequence of her talks, she asked GPT for advice on the order. GPT suggested an arrangement that created a logical flow and the reasoning for that flow, which ended up being pretty good.
She often uses gpt as a sounding board for ideas. She said she likes having an always-available colleague to bounce thoughts off of.
The argument I make is why aren’t more people finding ways to code with AI?
I work in a leadership role at a marketing agency and am a passable coder for scripts using Python and/or Google Apps Scripts. In the past year, I’ve built more useful and valuable tools with the help of AI than I had in the 3 or so years before.
We’re automating more boring stuff than ever before. It boggles my mind that everybody isn’t doing this.
In the past, I was limited by technical ability because my knowledge of our business and processes was very high. Now I’m finding that technical ability isn’t my limitation, it’s how well I can explain our processes to AI.
I know tons of people in my social groups that love AI and use it every day in it’s current form.
I'd call it a working google search, unlike, you know, google these days.
Actually google's LLM-based search results have been getting better, so maybe this isn't the end of the line for them. But for sophisticated questions (on noncoding topics!) I still always go to chatgpt or claude.
don't worry, Google WILL change this because they don't make money when people find the answer right away. They want people to see multiple ads before leaving the site.
And HN, it isn't just a few odd balls on HN championing it. I wish there was way to get a sentiment analysis of HN, it seems there are lot more people using it than not using it.
And, what about the silent majority, the programmers that don't hang out on HN? I hear colleagues talk about it all the time.
The impact is here, whether they are self directed or not, or whether there are still a few people not using it.
When I use LLMs, I use it exactly as Google search on steroids. It's great for providing a summary on some unknown topic. It doesn't matter if it gets it wrong - the main value is in keywords and project names, and one can use the real Google search from there.
And it isn't expensive if you are using the free version
AI is a type of outsourcing, you became a customer.
The same type of argument has been made for decades -- when coders wrote in ASM, folks would ask "are you still a coder when you use that fancy C to make all that low-level ASM obsolete?". Etc etc.
Do consider script kiddies hackers?
Something/someone other writes the code, that's outsourcing.
I wouldn't consider myself an artist if I create a picture per midjourney.
Do you design all the NAND gates in your processor to get the exact program you want out of it or use a general purpose processor?
Current "coding" is just a detail of what you want to do: solve problems. Which can require making a machine do what you want it to.
A coder writes code in a programming language, that what distinguishes them from the customers who use natural language. The coder is the translator between the customer and the machine. If the machine does that, the machine is the coder.
If you think you are obsolete or faster than anyone else with these tools then you only naive enough to have lost your objectivity to the marketing. I deal with real risk and failures from the output of ChatGPT which have serious financial consequences. The first victim is always the developer, then the tester.
At best, it is very good at ousting people who shouldn't be allowed anywhere near a damn computer.
Citation needed.
LLMs obviously have use cases but the market has practically priced in "AGI".
The danger is not that LLMs take jobs. The danger is that we are in a massive bubble and while these are nice tools they are not worth anything close to the trillions of dollars bet on them.
IMO the psychology at work here is basically denial that we can both be in the biggest bubble of all time in terms of dollars and LLMs are useful. Just not THAT useful.
(cue rebuttal based on systemic consequences / financial bailouts etc, but you know what I mean; also, the dotcom bubble deflation didn't require a bailout)
I use AI 100 times a day as a coder and 10,000 times a day in scripts. It’s enabled two specific applications I’ve built which wouldn’t be possible at single-person scale.
There’s something about the psychology of some subset of the population that insists something isn’t working when it isn’t _quite_ working. They did this with Wikipedia. It was evident that Wikipedia was 99% great for years before this social contingent was ready to accept it.