News for hackers. Stuff that matters.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by userbinator on Tuesday September 21, 2021 @01:22AM (#28602271)
If I look into my mirrors to make sure I am safe to change lanes, it dings me for distraction because my face is turned to look into my mirror.

WTF.

This seems like the real-world analogy of a similar trend I've noticed in software development where, to "improve code quality", humans are replaced with stupid tools that give false positives all the time, and to placate their warnings and satisfy the metrics of the management imposing them, writing code becomes perverted into trying to figure out how to stop the tool from complaining instead of focusing on the actual problem being solved.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by hughrr on Tuesday September 21, 2021 @02:39AM (#28602548)
I am living in this hell at the moment. This situation originated from compliance box ticking and trickled down to a kafkaesque nightmare where you are punished for ancillary problems. This is of course while being hammered because delivery has declined to a snail’s pace because the numerous hoops that need to be jumped through and general cruddiness of the process get in the way.

But someone somewhere high up in the ivory tower ticked a box. I suspect the same here happened with Amazon and an insurer somewhere. Push the hell down to the bottom of the stack.

The funny thing in this case is the process holes are still so enormous that you can drive a tank on fire through it and straight into production. The net gain is zero and the outcome is overconfidence.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by sdedovic on Tuesday September 21, 2021 @05:32AM (#28603359)
Reminds me of something that happened a few years ago to me.

Company was primarily using Java for the backend. The Application Security team decided to start scanning all our deployable JARs and WARs with a static analysis tool. Well, immediately the tool started complaining about our libraries. For example, the official Elasticsearch library, for the version of the database we used, had multiple System.out.println calls. Well those were all flagged and the static analysis tool deemed our code unacceptable. We needed to use a logger, not stdout. So we had to go and take it up with AppSec. And explain, no thats not our code. Yes, it does run in production. No we can't use a different library.

The whole thing was exactly as you put it, a kafkaesque nightmare where I'm being punished for something I don't understand by people who don't understand. An no-one can tell me why.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by piokoch on Tuesday September 21, 2021 @08:31AM (#28604374)
Phew, this at least spot some real issue (in some cases System.out.println can make multi-threaded program to get "synchronized" on console acces).

At some point I came across a tool (I think it was white source) that was going through Github page of each library used and was checking if there are no bugs reported.

Obviously very often there were totally bogus tickets, support requests created as bugs, rants as bugs, duplicates of already fixed tickets, etc. so investigating all that crap was really hard.

Another problem was licencing. There are a lot of libraries that have dual licence, e.g. GPL and CDDL, obviously the tool was panicking that GPL is used, not noticing that there is a second "business-friendly" license.

Those static analysis tools are not bad as a concept, however ratio of false positives to real issues found is too large to make them truly useful unless one can invest a lot of resources into tuning them.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by version_five on Tuesday September 21, 2021 @08:22AM (#28604305)
Just want to point out that Kafka based his stories in part on his experience working in the (Hapsburg i think?) bureaucracy, in particular the idea of a faceless and impenetrable organization that can't be reasoned with an is full of catch-22s (which I haven't read). So I find it funny we call it Kafkaesque when Kafka was just referring to how bureaucracy is. We should call it bureaucratic and admit that is what all big orgs end up becoming.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by robbedpeter on Tuesday September 21, 2021 @03:42AM (#28602859)
Taser the camera. "Dunno, hoss, didn't notice it wasn't working. "
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by mikro2nd on Tuesday September 21, 2021 @07:50AM (#28604090)
Speak softly and carry a can of spray-paint. (Hairspray might work just as well and be more defensible.)
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by bryanrasmussen on Tuesday September 21, 2021 @08:09AM (#28604213)
sure, so the AI knows you're driving and it can NEVER see your face.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by lm28469 on Tuesday September 21, 2021 @06:41AM (#28603693)
I think we're in a huge bubble in term of automation/AI these days, people are convinced they're very close to being able to automate most things while in reality we're faaaar away from even simple tasks. We're high on tech and the afterglow will hit hard

The seemingly very smart people designing these systems don't understand that their 1 in a billion edge case happens hundred of times every single day when you scale it to the world

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by bsenftner on Tuesday September 21, 2021 @08:19AM (#28604277)
We're in for a few decades of poorly implemented automation, to the degree many will feel they live in a dystopian nightmare. The arrogance of our management class to think society can be automated without a society wide analysis of what that automation's unexpected side-effects are going to be is equal to the colossal disaster we're going to see. I serious feel sorry for everyone that is going to live through this period. We're gonna be creating misery for a few decades, until the assholes on top figure out this needs massive pre-planning and the seat-of-the-pants methods being used now is just setting up for a planet-wide disaster.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by trhway on Tuesday September 21, 2021 @02:58AM (#28602642)
can the driver just put a photo of himself in front of the camera and score total bonus for being always on attention. Or (using the Moscow street surveillance contractor recipe) if the system is smart to not be fooled by a photo - record one ideal day (or even just an hour) and put the smartphone with the video in front of the camera on repeated play.

Of course the most interesting is to use counter-AI to generate video of right behavior without repetition and with matching location/street view in case if the Amazon/Netradyne AI is smart enough to have geolocation based street view integration.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by a012 on Tuesday September 21, 2021 @05:43AM (#28603419)
> can the driver just put a photo of himself in front of the camera and score total bonus for being always on attention. Or (using the Moscow street surveillance contractor recipe) if the system is smart to not be fooled by a photo -

Only if you know what the AI cam sees, if it's like a blackbox to you then I doubt there's way that you could do to fool it.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by trhway on Tuesday September 21, 2021 @04:00PM (#28609234)
After partially obscuring the lenses by say an accidental drop of ketchup :

"Your camera doesn't get correct image"

"Why?! What is wrong?"

"See for yourself"

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by ben_w on Tuesday September 21, 2021 @03:27AM (#28602801)
I suspect the people who understand AI well enough to fool it also know they can get more money not being a driver.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by apexalpha on Tuesday September 21, 2021 @05:12AM (#28603281)
You would be surprised how quickly people can learn how to fool something, without knowing exactly how it works.

Uber drivers used to look up time slots for big planes arriving at the airport. They would then all go offline on the app to trigger "surge pricing" and then all benefit from it.

No one knows exactly how the algorithm works, but they know how to fool it.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by ben_w on Thursday September 23, 2021 @08:45AM (#28628329)
Yeah, that’s fair. Cargo cults got famous because they worked, despite the mocking way they are often reported.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by trhway on Tuesday September 21, 2021 @05:24AM (#28603325)
interesting aspect here is that otherwise honest people at some point of unjust pressure put on them start to allow themselves to cross into that gray zone, and without feeling shame it leads to that being done in the open inside those specific social groups affected and thus becomes the acceptable social norm there.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by oarsinsync on Tuesday September 21, 2021 @09:01AM (#28604625)
> interesting aspect here is that otherwise honest people at some point of unjust pressure put on them start to allow themselves to cross into that gray zone, and without feeling shame it leads to that being done in the open inside those specific social groups affected and thus becomes the acceptable social norm there.

The thing to consider is that no actions are taking place in a vaccuum. The drivers gaming the system is not an isolated input that is occurring. Instead, there are a number of different inputs that are putting downwards pressure on the rates a driver gets to charge / how much money the driver actually takes home after costs, taxes, etc.

While it's not incorrect (or even unjust) to describe the behaviour as being potentially 'gray' (although the use of colour is a different matter, I understand the sentiment being expressed), by only highlighting and focusing on the least powerful members in the system, it unconsciously paints a picture that draws focus away from the other inputs into the system that have resulted in this situation occurring in the first place.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by bigfudge on Tuesday September 21, 2021 @08:08AM (#28604199)
I think people are able to segment moral behaviours very effectively. Uber’S ride allocation system would be very far down the list of things I would feel bad about gaming.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by trhway on Tuesday September 21, 2021 @04:18AM (#28603020)
I think that deepfake tools will soon be available to everyday user just like filters in Instagram or basic video editing tools.

And even if we just limit ourselves to tech users - some companies have long been logging coders' Eclipse (mostly not in the 1st rate countries of course), and I suspect these days that may get fed to AI. And some day it will make into US and West Europe too.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by fortran77 on Tuesday September 21, 2021 @09:29AM (#28604825)
What makes you think people who take a job driving are incapable of understanding things as well as you?
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by ben_w on Tuesday September 21, 2021 @02:40PM (#28608254)
As soon as someone has developed that understanding, they get a chance for a massive pay rise and better working conditions.

While I acknowledge that some people will prefer to be a driver than to work in any of the places whose doors would open with this skill, I suspect this is a minority.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by exporectomy on Tuesday September 21, 2021 @02:23AM (#28602481)
It's hard to believe or every single driver would get the same consequences. This reminds me of drivers who complain about speeding tickets right after the speed limit changed. Well, slow down before the speed limit changes and you'll be fine. Some people just have bad habits without realising.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by mcv on Tuesday September 21, 2021 @04:28AM (#28603074)
Automatically analysing safe traffic behaviour from a camera is a hard problem. From the article, it sounds like this system is incredibly naive and doesn't really understand real world traffic situations, so it ends up punishing people for doing the right thing while demanding they do the wrong thing (like not looking in the mirror), and on top of that, distracting them and adding extra stress.

It sounds like a system that needs to be banned or thoroughly regulated. Naive driver distraction systems like this do not make traffic safer. The company should prove that this system is safe just like autopilot manufacturers have to.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by exporectomy on Tuesday September 21, 2021 @05:07PM (#28609970)
I'm talking about the unfairness, not safety. The article doesn't say that some drivers are allowed to do it while others aren't but still presents the whole thing as unfair.
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by mcv on Wednesday September 22, 2021 @06:51PM (#28623068)
Sure, but I'm talking about safety. And safety is also an important issue. And one that's at least as much subject to regulation. And regulating this infringement on driving safety would also help with the unfairness, even if it doesn't address it directly.

Of course the unfairness is also a valid concern, and I'd love to see them both addressed.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by Cipater on Tuesday September 21, 2021 @03:37AM (#28602845)
“When I get my score each week, I ask my company to tell me what I did wrong,” the driver told Motherboard. “My [delivery company] will email Amazon and cc' me, and say, ‘Hey we have [drivers] who'd like to see the photos flagged as events, but they don't respond. There's no room for discussion around the possibility that maybe the camera's data isn't clean.”

You should interview at Netradyne. From your comment you seem like a good fit.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by laumars on Tuesday September 21, 2021 @02:44AM (#28602577)
This comment reminds me of a, thankfully small, subset of developers who believe their code is infallible and it must be the users fault if the application has bugs.

What the AI is attempting to solve is a complex problem. I’m willing to give the drivers here the benefit of the doubt ;)

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by exporectomy on Tuesday September 21, 2021 @05:12PM (#28610025)
This comment reminds me of a, thankfully small, subset of drivers who believe their driving is infallible and it must be the other drivers' fault if they're penalized. What the driver is attempting to do is a complex task. I’m willing to give the AI here the benefit of the doubt ;)
Re:Amazon’s AI cameras are punishing drivers for mistakes th... by laumars on Tuesday September 21, 2021 @06:56PM (#28611061)
You clearly didn’t read the article then. The drivers are open to dialog. It’s the maintainers of the software and/or Amazon who are assuming the system is perfect.

I’ve worked on smart interfaces for cars by the way. My experience not only as a driver but also as a developer in this space tells me that AI described in the article needs further development and your presumptions in this discussion are BS.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by exporectomy on Tuesday September 21, 2021 @07:44PM (#28611436)
You're missing the point. If the AI is consistently penalizing drivers for looking in the mirror, then it'll affect them all equally and not be unfair. If it treats some drivers worse than others, why didn't they mention that instead of leaving us to assume that there really are drivers who are never penalized for looking in the mirror?

TFA even points out drivers making excuses for not wearing a seatbelt. If they won't even take personal responsibility for their own illegal actions, of course their opinions on more subtle things like following distances aren't reliable.

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by laumars on Wednesday September 22, 2021 @02:20AM (#28613588)
> You're missing the point. If the AI is consistently penalizing drivers for looking in the mirror, then it'll affect them all equally and not be unfair. If it treats some drivers worse than others, why didn't they mention that instead of leaving us to assume that there really are drivers who are never penalized for looking in the mirror?

They did mention it: not every driver has the AI installed.

Also you’re assuming the bonuses are awarded like prizes to the top performers. So if everyone is penalised equally then the same people still get awards. My impression was those bonuses were awarded to anyone who passes specific milestones. So even if everyone is penalised equally it’s still unfair to everyone.

> TFA even points out drivers making excuses for not wearing a seatbelt. If they won't even take personal responsibility for their own illegal actions, of course their opinions on more subtle things like following distances aren't reliable.

Yes, that is a fair argument. I interpreted that in a more charitable way saying “they’re acknowledging some bad practices on their side so at least trying to meet the AI developers half way”. But I will admit your interpretation is just as, if not more so, plausible.

Even so, that doesn’t justify being ignored to by Amazon and the AI developers. Any new tech needs debugging in the field. And this is more so true with AI and complex problems like driving assistants than it is in most other software specialties.

That’s the part of the article that stinks the most and the original point I was making yesterday. I couldn’t give a rats ass who is to blame because the biggest fault lies with the company for not working with the users to ensure the software is performing correctly. And this is especially important when people’s income are being directly effected by the software too!

Re:Amazon’s AI cameras are punishing drivers for mistakes th... by exporectomy on Wednesday September 22, 2021 @04:30AM (#28614177)
Yea not excusing Amazon ignoring their requests or blindly using AI results to decide compensation, or that it can encourage dangerous driving to get points, or that it's probably inaccurate. Overall awful, but might be a fun game if you didn't depend on the job for your livelihood.