You'll have to elaborate on that. How much trust was there in electricity, flight and radioactivity when we discovered them?

In science, you build trust as you go.

> As the use of AC spread rapidly with other companies deploying their own systems, the Edison Electric Light Company claimed in early 1888 that high voltages used in an alternating current system were hazardous, and that the design was inferior to, and infringed on the patents behind, their direct current system.

> In the spring of 1888, a media furor arose over electrical fatalities caused by pole-mounted high-voltage AC lines, attributed to the greed and callousness of the arc lighting companies that operated them.

https://en.wikipedia.org/wiki/War_of_the_currents

Most of what people think they know about Tesla is not actually true if you examine the historical record. But software engineering as a discipline demands business villains and craftsman heroes, and so Edison and Tesla were warped to fit those roles even though in real life there is only evidence of cordial interactions.

The moment a technology reaches trust at scale, it becomes a non-innovation in people's mind.

Happened for TVs, electrical light in homes, AI for chess, and Google. Will happen with LLM-based assistants.

LLM leads to distrust between people. From TFA, That concept is Trust - It underpins everything about how a group of engineers function and interact with each other in all technical contexts. When you discuss a project architecture you are trusting your team has experience and viewpoints to back up their assertions.

True not only in innovation, but in business settings.

I don't think there's anyone who works in any business long enough who doesn't have problems getting their job done simply because someone else with a key part of the project doesn't trust that you know what you're doing.

Humans communicate using symbols. That could be patterns of sound waves, gestures, or written characters.

If we can't trust that the communicated symbols match their agreed meaning, what is the use of them? Communication breaks down quickly, and being social individuals which inter-depend on others, we can't live without communication. Nobody likes a liar, and the reason is, the words they give us do not match what they mean. (Perhaps not only humans. I read that dolphins readily come and rescue individuals which are drowning - including humans - but they punish individuals which fake drowning.)

And that goes from every stratum of social interaction, from big treaties to selling a bagel. Would you sell a bagel for a fake dollar bill? The number on it is a symbol as well, it has a meaning.

So it is right down one of the very bases of human cooperation.

Not much.

Plenty of people were against electricity when it started becoming common. They were terrified of lamps, doorbells, telephones, or anything else with an electric wire. If they were compelled to use these things (like for their job) they would often wear heavy gloves to protect themselves. It is very occasionally mentioned in novels from the late 1800's.

(Edit: If you'd like to see this played out visually, watch the early episodes of Miss Fisher's Murder Mysteries on ABC [.oz])

There are still people afraid of electricity today. There is no shortage of information on the (ironically enough) internet about how to shield your home from the harmful effects of electrical wires, both in the house and utility lines.

Flight? I dunno about back then, but today there's plenty of people who are afraid to fly. If you live in Las Vegas for a while, you start to notice private train cars occasionally parked on the siding near the north outlet mall. These belong to celebrities who are afraid to fly, but have to go to Vegas for work.

Radioactivity? There was a plethora of radioactive hysteria in books, magazines, comics, television, movies, and radio. It's not hard to find.

It's just too useful to ignore, and trust is always built, brick by brick. Let's not forget humans are far from reliable anyway. Just like with driving cars, I imagine producing less buggy code (along predefined roads) will soon outpace humans. Then it is just blocking and tackling to improve complexity.

Can we really do this reliably? LLMs are non-deterministic, right, so how do we validate the output in a deterministic way?

We can validate things like shape of data being returned, but how do we validate correctness without an independent human in the loop to verify?

I check AI output for hallucinations and issues as I don’t fully trust it to work, but we also do PRs with humans to have another set of eyes check because humans also make mistakes.

For the soft sciences and arts I’m not sure how to validate anything from AI but for software and hard sciences I don’t see why test suites wouldn’t continue serving their same purpose

If we need a human in the loop to check every row of code for the deep logic errors... then we could just get the human to write it no?

I find its a slightly improved google for vague questions. Or a doxygen writer.

Its all use I've found for any ai model since i first started playing with github copilot beta.

Ive been trying the newer models as they arrived, and found they're getting more verbose, more prone to hallucinating functions that dont exist, and more prone to praise me as a god when trying to ask about basic assumptions. (you're cutting to the heart of the matter)

What kind of code do you write where its somehow replacing coding itself? I spent 30 minutes trying to get mistral to write a basic bash script yesterday.

So for me, the future will tend towards this ... Currently the tech is early days, we have no way to steer thought.. We have no way to align it to our thought processes... But eventually we will get to I want x pls make and it will be able to do it well.

If no? Then congrats, you are now in a position where your software development lifecycle needs to handle non-determinism.

This fanatical vibing movement is ridicolous, but this luddite stance that LLMs cannot contribute to software dev because they are «non deterministic» is almost as ludicrus.

Sure, except the Juniors producing wrong solutions is the Juniors problem, not mine

If I give four LLM agents tasks and they call come back with slightly wrong solutions, that's me adding four problems to my own workload

I'm not sure how I'm supposed to keep up with that. I'm definitely not sure it makes me overall more productive

This new world of having to verify every single thing at all points is quite exhausting and frankly pretty slow.

I don't agree with this at all. Writing new code is trivially easy, to do a full in depth review takes significantly more brain power. You have to fully ascertain and insert yourself into someone elses thought process. Thats way more work than utilizing your own thought process.

They basically achieve over 80% agreement with human evaluators [1]. This level of agreement is similar to the consensus rate between two human evaluators, making LLM-as-a-judge a scalable and reliable proxy for human judgment.

[1] https://arxiv.org/abs/2306.05685 (2023)

It sounds nice but it means at least 1 in 5 are bad. That's worse odds than rolling 1 on a d6. You'll be tripping over mistakes constantly.

  > 80% is a pretty abysmal success rate and means it's very unreliable.

So if 80% is abysmal, you are getting the same abysmal level from human code reviewers.

The whole point of a tool is to reduce workload. If you have to frequently correct it, then it's not worthwhile to use.

Sure there's no bug with how the logic is defined in the CR or even in the context of the project, ti maybe won't throw an exception.

But the LLM won't know that the query is iterating over an unindexed field in the DB with the table in prod having 10s of millions of rows. The LLM won't know that even though the code says the button should be red and the comments say the button should be red, the corporate style guide says red should be a very specific hex code that it isn't.

Oh goodness that's like trusting one kid to tell you whether or not his friend lied.

In matters where trust matters, it's a recipe for disaster.

Give it another year and HN comments will be very different.

Writing tests already works now. It's usually easier to read tests than to read convoluted logic.

Mmmhmm. And you think this "growing up" doesn't have biases to lie in circumstances where it matters? Consider politics. Politics matter. It's inconceivable that a magic algorithm would lie to us about various political concerns, right? Right...?

A magic algorithm lying to us about anything would be extremely valuable to liars. Do you think it's possible that liars are guiding the direction of these magic algorithms?

I notice a distinct lack of blockchain hegemony.

The classic on the subject.