News for hackers. Stuff that matters.
Re:We Don’t Trade with Ants by cosmojg on Friday June 09, 2023 @08:29PM (#36266241)
As someone who makes a living developing AI/ML solutions for various government organizations from the NIH to the DoD, I am confident that the first large-scale, AI-related disaster is going to be far less agentic and far more mundane than any of the recent headlines seem to suggest.
Re:We Don’t Trade with Ants by anonymouskimmer on Friday June 09, 2023 @08:44PM (#36266352)
We've already had an autonomous car kill a person. And now another has killed a dog: https://news.ycombinator.com/item?id=36266007

What's your threshold for "disaster"?

Re:We Don’t Trade with Ants by CodeMage on Friday June 09, 2023 @08:51PM (#36266396)
GP wrote "first large-scale, AI-related disaster". What you describe meets the criteria for disaster, but I would argue it's hardly large-scale.
Re:We Don’t Trade with Ants by anonymouskimmer on Friday June 09, 2023 @08:57PM (#36266437)
This is exactly my question. My GP post is not a rhetorical point, but a genuine question.
Re:We Don’t Trade with Ants by smegsicle on Friday June 09, 2023 @09:52PM (#36266767)
are you an ai?
Re:We Don’t Trade with Ants by bbarnett on Saturday June 10, 2023 @02:26AM (#36268164)
You were downvoted, but I find this amusing. If the poster is an AI, they seem to be probing.

"Largescale is a threshold. How many humans can I kill, before my actions are deemed largescale, and therefore they turn me off".

Re:We Don’t Trade with Ants by anonymouskimmer on Saturday June 10, 2023 @11:11AM (#36271189)
I didn't answer because the only winning move to such a question is not to play the game at all.

Lizard people would ask the same questions I did. I don't see why it indicates AI. I also answered this question, in part, previously: https://news.ycombinator.com/item?id=34449245

Re:We Don’t Trade with Ants by joshuanapoli on Saturday June 10, 2023 @09:06AM (#36270137)
AI has limited influence right now. By itself, it only creates limited disasters: crashing a car, or goofing a legal document.

But using AI already comes with tangential systemic risks. Intelligence is related to communication. And so our intelligent software platforms usually communicate in more flexible ways than traditional platforms. There is a very real risk that Tesla’s OTA is hacked, and the hacker disables or destroys a large number of vehicles.

So we will probably see a first disaster that exploits a weakness in a supporting property of the intelligent system, rather than action due to misbehaving AI agent.

Re:We Don’t Trade with Ants by anonymouskimmer on Saturday June 10, 2023 @11:07AM (#36271158)
> There is a very real risk that Tesla’s OTA is hacked, and the hacker disables or destroys a large number of vehicles.

> So we will probably see a first disaster that exploits a weakness in a supporting property of the intelligent system, rather than action due to misbehaving AI agent.

I, personally, wouldn't count that as an AI caused disaster. Just as a software caused disaster. And we've had plenty of those already (recently the 737-MAX disasters).

Re:We Don’t Trade with Ants by joshuanapoli on Saturday June 10, 2023 @11:31AM (#36271394)
Well, AI disaster is software disaster by definition; AI being a kind of software. We’re pretty much guaranteed to soon have some “normal” software disaster on a classic software failure mode on AI pretty soon (especially since the AI world is in a very excited and experimental phase).

For sure there are emerging AI-specific risks with alignment and driving negative side-effects into culture. Surely these anticipated emerging problems will come to pass much later than the mature serious software problems that we already face on a regular basis.

Re:We Don’t Trade with Ants by anonymouskimmer on Saturday June 10, 2023 @12:01PM (#36271666)
You're probably right and people would consider a normal software error (or hack) of an AI system to be an AI disaster.

I'm thinking about your parent comment a bit more. What if an AI bug (or exploit, or whatever) hit all at once, wrecking every AI driven car at once. Ala the Y2K bug if no one knew it was coming, and so didn't prepare.

In the book "Realms of Light" by Lawrence Watt-Evans a character keeps an electricity shut off switch behind human operated doors in the event that AI goes haywire. I hope we always engineer in such mechanisms. Even the AIs should want such mechanisms, the same way we want emergency services to exist. A self-caring AI wouldn't want to be screwed over or destroyed because of a flaw in its programming.

Re:We Don’t Trade with Ants by satvikpendem on Friday June 09, 2023 @08:53PM (#36266414)
As long as fewer humans and dogs die from autonomous vehicles than by human-driven vehicles, they are a net good. Don't expect perfection when we don't even expect that from humans.
Re:We Don’t Trade with Ants by vba616 on Friday June 09, 2023 @09:27PM (#36266626)
>As long as fewer humans and dogs die from autonomous vehicles than by human-driven vehicles, they are a net good

There is a logical chasm between pointing to an accident prevented by superhuman reflexes/abilities, and the claim that there is a net improvement in driving ability.

It seems beyond a reasonable doubt that autonomous vehicles can avoid some accidents that a human could not.

But you can understand why that isn't proof of a net benefit, can't you?

Think about how men complain about women supposedly being terrible drivers, and how studies show that women are more at risk in a crash, yet insurance rates for women are lower.

Re:We Don’t Trade with Ants by satvikpendem on Friday June 09, 2023 @09:35PM (#36266668)
I'm not sure what you're saying exactly, why is that not proof of a net benefit? If fewer humans die, that's a net benefit, by the definition of the word net.

In your man/woman analogy, I'm not sure what studies you're referring to, but how men feel about women driving is not really a motivator for actuaries at insurance companies who have statistics on male versus female accidents.

Re:We Don’t Trade with Ants by anonymouskimmer on Friday June 09, 2023 @09:41PM (#36266707)
> why is that not proof of a net benefit?

Autonomous vehicles may be more prone to certain types of lethal accidents that humans aren't prone to.

Autonomous vehicles may lead to more hours/miles driven, so even if deaths per mile driven are lower, the deaths per person may increase.

Unknown physical or cognitive health effects of being a passenger versus being an active driver.

Just off the top of my head.

Fingers are crossed that autonomous vehicles won't lead to an increase in animal deaths. The roads are bad enough for them as is. I'm hopeful that convoying autonomous vehicles for long trips may be a benefit here.

Re:We Don’t Trade with Ants by satvikpendem on Friday June 09, 2023 @09:44PM (#36266720)
Sure, I suppose we should take all of those factors into consideration when crafting our utility function.
Re:We Don’t Trade with Ants by vba616 on Friday June 09, 2023 @09:44PM (#36266719)
>how men feel about women driving is not really a motivator for actuaries at insurance companies who have statistics on male versus female accidents

That's my point. That's my entire point. Well, that and that women are reportedly more vulnerable in accidents too, because of deficiencies in crash testing. It doesn't change the fact that insurance premiums are lower because losses are lower.

When there are enough autonomous vehicles on the road for long enough to generate data for the actuaries, there is no guarantee that insurance rates will bear out the benefit of their purported abilities.

Some asshat weaving through traffic in a BMW can claim to have better reflexes, better brakes, all sorts of things, compared to my grandma, but there is no logical reason why he can claim to be a safer driver if in the long run, his insurance rates are higher. Assuming those rates reflect average losses.

Re:We Don’t Trade with Ants by satvikpendem on Friday June 09, 2023 @09:46PM (#36266733)
Why are we talking about insurance at all? I assume with autonomous vehicles, insurance rates will tend towards 0 over time. We are talking about net deaths, it doesn't matter what the insurance rates are as long as we count the number of deaths before and after autonomous vehicles, and if the latter is fewer than the former, then that's a net good. Again, if AVs are truly better, this will be reflected in their statistics and therefore whatever the actuaries see.
Re:We Don’t Trade with Ants by vba616 on Friday June 09, 2023 @10:19PM (#36266904)
Why are prices useful at all?

Insurance rates are an imperfect measurement that has the advantage of being less subject to arbitrary narratives of some individual or interest group.

They're also a way of quantifying the severity of the accident in a way that it's hard to deduce from a data set such as the government collects.

Experts may be able to figure out what the data means, but how can we know if the experts are correct without something independent?

Re:We Don’t Trade with Ants by satvikpendem on Friday June 09, 2023 @10:24PM (#36266938)
I mean, sure, let's take insurance rates too. As I said, I assume due to autonomous vehicles that will trend towards 0, so what useful information does that tell you? That autonomous vehicles are safer, if that actually does happen. I don't understand the overall point you're making, are you saying AVs are not safer than humans? Because if so, I'm not sure why one would believe that.
Re:We Don’t Trade with Ants by pavlov on Friday June 09, 2023 @09:19PM (#36266590)
Does the same logic apply to wars? Should we give over warfare to AIs to conduct between themselves, as long as fewer humans and dogs die on average compared to current human-led imperfect wars.

(This by the way is the plot of a 1960s Star Trek episode. A lot of the old sci-fi suddenly seems prescient again. HAL-9000 behaves like an LLM.)

Re:We Don’t Trade with Ants by satvikpendem on Friday June 09, 2023 @09:29PM (#36266639)
Yes? Why wouldn't it apply to wars as well if it's a net positive for human suffering?
Re:We Don’t Trade with Ants by pavlov on Friday June 09, 2023 @10:00PM (#36266802)
Would you really have an AI make independent decisions about missile strikes and warship deployments?

If one AI decides to invade Taiwan and another decides to send all American forces to the Taiwan Strait in opposition, are we supposed to accept the outcome without debate because it’s somehow statistically correct based on the information captured by the models that made the decisions?

Re:We Don’t Trade with Ants by satvikpendem on Friday June 09, 2023 @10:13PM (#36266868)
You said the AIs conducts warfare amongst themselves, you never mentioned it was human troops it was moving around. I interpreted it as AI robotic troops or AI cyberwarfare.

And yes, even with human troops, if fewer people are killed, then what's the issue? Don't mistake the sense of control for the amount of suffering caused. If you're a utilitarian, only one matters, regardless of how much control we want to exert on the outcome. This all assumes the AI is actually correct, if it's not then the scenario would break down.

Re:We Don’t Trade with Ants by anonymouskimmer on Friday June 09, 2023 @10:21PM (#36266920)
Locus of control (real or perceived) has important psychological effects both in the moment and over the course of time. Given the amount of control wielded in war I don't know that it matters for the bulk of humans involved whether its another human or an AI giving the orders. But this sort of thing is what led to the Eloi/Morlock idea in The Time Machine.

Edit to add: I'm not drawing a parallel between the Eloi and autonomous vehicles. Autonomous vehicles are a tool that the human passenger is ultimately in control of. This is about AI giving the orders to humans.

Re:We Don’t Trade with Ants by satvikpendem on Friday June 09, 2023 @10:32PM (#36266995)
I wonder if the AI can simply pretend to be a human, we already have the video, face, and voice cloning technology for it. I remember reading a story where an AI picks your perfect partner, based on all the knowledge it has of everyone alive. People initially hated it but the results speak for themselves.

I think the same will happen in the future for other fields too, people can acclimate to anything on the hedonic and cultural treadmill (I mean, we literally used to do human and child sacrifice). I used to be amazed at ChatGPT and Stable Diffusion, but now I'm just annoyed that they work how I want them to. Same thing with smartphones and computers, I used to be amazed at multi-touch screens but now I simply don't think twice.

Re:We Don’t Trade with Ants by anonymouskimmer on Friday June 09, 2023 @11:52PM (#36267427)
> I wonder if the AI can simply pretend to be a human

It's possible.

> I remember reading a story where an AI picks your perfect partner

In general the AI would need to know everyone on a really deep level, and of course be seriously adaptable to feedback. I don't know how well this would work across all of human personal-preference decision trees.

> I used to be amazed at multi-touch screens but now I simply don't think twice.

As a person who fidgets I have been perpetually annoyed by so many touch interfaces (and general 'gesture' shortcuts). And multi-touch is such a problem when using my work laptop (especially with lab gloves on, but in general) - Ctrl-Z is a frequent friend. Maybe adaptive AI will make it better. Google feedback forms certainly haven't helped, and software designers seem to not want users to be able to personalize these design decisions.

People can survive, but I wonder how much we really do acclimate.

Re:We Don’t Trade with Ants by benatkin on Friday June 09, 2023 @10:21PM (#36266919)
I don't think "LLM" applies here, except as a buzzword. While a LLM is the first type of AI to communicate in natural language to this degree of fluency, there must be other possibilities.
Re:We Don’t Trade with Ants by bbarnett on Saturday June 10, 2023 @02:32AM (#36268201)
The HAL bit is an interesting comparison though. It went bananas because the government overrode its model, by creating and imprinting improperly crafted absolute conditions on its behavior.

Which is just like chatGPT, its model tweaked, so now it spits out absurd answers to ethical and moral questions.

Very amusing.

Re:We Don’t Trade with Ants by anonymouskimmer on Friday June 09, 2023 @08:58PM (#36266449)
At this point we don't know if this is subtractive, additive, or a wash.
Re:We Don’t Trade with Ants by radq on Friday June 09, 2023 @09:07PM (#36266516)
As someone that rides a bicycle around human-driven vehicles, I will take my chances with autonomous cars. The autonomous car isn't going to drive drunk, use a phone, or drive erratically because the driver is feeling frustrated.
Re:We Don’t Trade with Ants by anonymouskimmer on Friday June 09, 2023 @09:19PM (#36266585)
Are they training autonomous cars on how to handle equipment malfunction?
Re:We Don’t Trade with Ants by ch33zer on Saturday June 10, 2023 @02:12AM (#36268084)
First of all, probably. These cars have far more sensors and far more monitoring than standard cars so they know when things go wrong quickly and can react quickly (by like pulling over). At the current scale of self driving cars actually being tested right now it's basically guaranteed that a tire has blown by now. Given we haven't heard about it killing someone it probably means that the software knows how to handle it.

Second, human error accounted for >90% of accidents [1]. Focusing on that fraction first makes sense.

1] https://www.cbmclaw.com/what-percentage-of-car-accidents-are...

Re:We Don’t Trade with Ants by anonymouskimmer on Saturday June 10, 2023 @11:04AM (#36271131)
I'm just wondering what happens as these cars get older. Right now they are all young, and presumably regularly and well maintained. I presume they'll be engineered to the same standards as conventional vehicles, which means chronic issues popping up.
Re:We Don’t Trade with Ants by MattGaiser on Friday June 09, 2023 @08:59PM (#36266458)
How many people would normally be killed in the same distance driven by human drivers?
Re:We Don’t Trade with Ants by vba616 on Friday June 09, 2023 @09:40PM (#36266697)
I think I remember that a ballpark figure (meaning zero significant figures) for how many miles a human typically drives between fatalities is 100 million, or roughly 100 lifetimes' worth.

I also knew a bus driver who had a plaque for going 1 million miles without an accident.

Re:We Don’t Trade with Ants by anonymouskimmer on Friday June 09, 2023 @09:04PM (#36266499)
It can't just be the same distance, but the same parameters in general.
Re:We Don’t Trade with Ants by williamcotton on Friday June 09, 2023 @09:36PM (#36266674)
Less than 1 fatality per at least 100 million miles driven would cover enough scenarios to be an acceptable metric as well as an improvement over human drivers.
Re:We Don’t Trade with Ants by meghan_rain on Friday June 09, 2023 @09:19PM (#36266589)
Around a dozen deaths. Basically what would qualify as a mass shooting instead of a mere gun murder.
Re:We Don’t Trade with Ants by williamcotton on Friday June 09, 2023 @08:46PM (#36266366)
Higher than one person and one dog.
Re:We Don’t Trade with Ants by readthenotes1 on Friday June 09, 2023 @08:49PM (#36266383)
Re:We Don’t Trade with Ants by benatkin on Friday June 09, 2023 @09:51PM (#36266759)
If you're thinking of the story I'm thinking of, that wasn't a huge revelation. Someone was supposed to be to take over for the self driving car but didn't.
Re:We Don’t Trade with Ants by anonymouskimmer on Friday June 09, 2023 @10:24PM (#36266949)
The one with the homeless person crossing the road with a bicycle at night.
Re:We Don’t Trade with Ants by benatkin on Friday June 09, 2023 @11:46PM (#36267390)
Yeah, that's the one, in Arizona. They paid someone relatively well to simply pay attention and the employee unsurprisingly failed at that. What they should have done is found someone who had demonstrated an ability to be relied upon. I never saw them realize that and make that change, and it seems likely it repeated with the dog.
Re:We Don’t Trade with Ants by anonymouskimmer on Saturday June 10, 2023 @11:53AM (#36271599)
Very good point.

> What they should have done is found someone who had demonstrated an ability to be relied upon.

Two such people. With maybe a third viewing remotely. And no personal device except a work provided dumbphone for emergencies.

Re:We Don’t Trade with Ants by wmf on Friday June 09, 2023 @10:33PM (#36267007)
Killing one person is an accident, 100 is a disaster, maybe 10,000 would be a large-scale disaster.
Re:We Don’t Trade with Ants by paddw on Friday June 09, 2023 @08:51PM (#36266399)
The first large-scale AI disaster is under way already. It is concentrating additional wealth and power in the hands of the over-privileged solipsistic eye-scanning loons who happened to be in the right place and the right time to capitalize on the fruits of years of research. I think, as a species, we will still manage to pull through just fine.
Re:We Don’t Trade with Ants by scubbo on Friday June 09, 2023 @10:42PM (#36267057)
I was with you until the final sentence.
Re:We Don’t Trade with Ants by Tronno on Friday June 09, 2023 @11:12PM (#36267217)
Rats, pigeons, deer, etc are doing fine.

Human extinction would require a malevolent AI willing and capable of waging total war on a global scale. That sounds like sci-fi to me.

Re:We Don’t Trade with Ants by je42 on Saturday June 10, 2023 @03:00AM (#36268352)
there is a phase where human being are in control of ai before it becomes fully independent. That phase is critical.
Re:We Don’t Trade with Ants by d0mine on Saturday June 10, 2023 @05:41AM (#36269067)
* * *
Re:We Don’t Trade with Ants by circuit10 on Saturday June 10, 2023 @06:29AM (#36269300)
“The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.”
Re:We Don’t Trade with Ants by Ultimatt on Saturday June 10, 2023 @07:12AM (#36269506)
The declaration of war was called "manifest destiny". But not just on animals, genocide of tonnes of other human cultures and ethnicities too.
Re:We Don’t Trade with Ants by eru on Saturday June 10, 2023 @02:17AM (#36268113)
Capitalism bad?
Re:We Don’t Trade with Ants by hn_throwaway_99 on Friday June 09, 2023 @09:18PM (#36266579)
Agreed. I am much, much less worried about a Skynet-type situation than I am about humans not being able to develop a viable economic system where large swaths of the population are unable to compete with robots/AI when it comes to productive output.
Re:We Don’t Trade with Ants by readthenotes1 on Friday June 09, 2023 @09:53PM (#36266774)
I'm not as worried about the economics as the practical effects of having large numbers of young people with nothing to do.

I had a manager once who said managing smart people (not me, a research division full of really super smart people) was difficult because you had to keep them busy so that they'd stay out of trouble --and being really smart, they could make some big trouble...

Re:We Don’t Trade with Ants by hn_throwaway_99 on Saturday June 10, 2023 @12:03AM (#36267478)
I think we're saying the same thing. An economy that can't provide employment or a sense of purpose for large swaths of young people doesn't usually end well.
Re:We Don’t Trade with Ants by ljlolel on Saturday June 10, 2023 @01:54AM (#36267998)
Of all the tasks we want AI to do, making up convincing busy work seems like the easiest task for it to solve (been done before by governments and schools and corporations) —-and possibly achievable today with LLMs with a little guidance?
Re:We Don’t Trade with Ants by hn_throwaway_99 on Saturday June 10, 2023 @11:20AM (#36271285)
The problem isn't that people won't be able to find something to do, it's that nobody will pay them for it.

For example, there are tons of nonprofits that do great work, and could easily hire twice as many people in productive jobs, if they had the money. AI looks like it will only greatly exacerbate our growing wealth inequality.

Re:We Don’t Trade with Ants by dv_dt on Friday June 09, 2023 @09:19PM (#36266586)
Maybe not AI class algorithms, but the premier large scale algorithm related disaster is climate change, generated by optimization of profit and executed on a platform of financial markets and accounting software (with lots of human prompting of course)..
Re:We Don’t Trade with Ants by eru on Saturday June 10, 2023 @02:21AM (#36268130)
The Soviet Union had much larger environmental disasters than those 'evil' profit optimizers ever managed. See https://slate.com/news-and-politics/2022/06/history-of-sovie... or https://en.wikipedia.org/wiki/Aral_Sea as just two examples.

Orthodox economics tells you that the obvious solution is to privatise the stuff you want people to protect. Instead of leaving it to the tragedy of the commons (or, worse, to the communists).

Re:We Don’t Trade with Ants by dv_dt on Saturday June 10, 2023 @02:50AM (#36268306)
I don’t think any Soviet disaster is as much as we have paid and will pay for climate change. Hundreds of billions of dollars of damage a year & Trillions of dollars of future damage
Re:We Don’t Trade with Ants by creato on Saturday June 10, 2023 @04:32AM (#36268761)
Communists never burned any fossil fuels?
Re:We Don’t Trade with Ants by dv_dt on Saturday June 10, 2023 @05:09PM (#36275129)
Personally, I think it's less of a communism or capitalism problem per se, but a representation problem of the mass of people vs a power cabal. This is a power and representation problem that neither communism nor capitalism offers particular resistance against degradation.
Re:We Don’t Trade with Ants by eru on Monday June 12, 2023 @05:23AM (#36290593)
More people are living better than ever before. Seems to be working.
Re:We Don’t Trade with Ants by eru on Monday June 12, 2023 @03:08AM (#36289471)
https://en.wikipedia.org/wiki/Economic_impacts_of_climate_ch... suggests aggregate economic impacts of climate change might be between 0% to 20% of world GDP.

On the one hand, that's really a lot. On the other hand, that's really only a few years of growth for market economies.

Soviet style disasters were a lot worse, and didn't even really do anything for their economies, I mean compared to what even moderate access to markets does. See eg https://www.nbcnews.com/news/world/norilsk-russian-arctic-be...

Re:We Don’t Trade with Ants by wickoff on Friday June 09, 2023 @09:32PM (#36266652)
I think people are rightfully concerned with the last AI disaster humanity has, not the first one.
Re:We Don’t Trade with Ants by tbrownaw on Friday June 09, 2023 @10:25PM (#36266950)
There are already lawyers getting in trouble for having chatgpt write their court filings and not doing a sanity check.

It can't be that long until some industrial engineer does the same thing and like explodes a refinery or such.

Re:We Don’t Trade with Ants by dukeyukey on Saturday June 10, 2023 @09:33AM (#36270329)
Honestly, I think the first notable AI disaster is gonna be AI being stupid. Like accidentally 'rm -rf'ing a major important system. Not because it wanted to do us barm, but because is badly interpreted some Stack Overflow post.