News for hackers. Stuff that matters.
Re:Google terminated our Developer Account, says it is “asso... by matsemann on Wednesday March 30, 2022 @10:16AM (#30855533)
I wonder if the devs behind these kind of automations ever stop and think about the ethical sides of it.
Re:Google terminated our Developer Account, says it is “asso... by jrockway on Wednesday March 30, 2022 @10:58AM (#30856079)
I've spent an inordinate amount of time banning fraudsters from my own cloud platform. The bots that companies write to do this are designed to fight off other bots. If you have an appeals process, the humans that control the malicious bots will show up to abuse the appeals process. One of my favorite cases was a user that signed up with hundreds of stolen credit card numbers (all from the same IP address, as they are apt to do), got banned, and then opened an issue in our open source repo complaining of trouble logging in. Yeah, you're having trouble because I banned you and your entire network of bots.

There is a balance, of course, but if you haven't seen how much automated abuse there is on the Internet, be careful of what you wish for. Even the tiniest of the tiny services suffer from massive amount of automated abuse. At that tiny scale, it was nearly impossible for me to keep up with the abuse without the help of automation, very broad bans, and deleting related accounts by walking a reputation graph (like Google is being criticized for doing here). At Google's scale, I don't think there are enough humans on Earth to deal with the abuse. As a result, there are going to be some innocent casualties.

I don't think it's an ethics thing, it's simply not possible to run a business without some sort of process like this. There are laws, rules, and processes that could cut all this down to levels that could be managed by humans, but the cure might be worse than the disease. (For example, it would be great if there was a 1:1 mapping between your national ID card and your IP address. All of these stolen-credit-card users could then be imprisoned. But, you know that that's a terrible thing, because it will also be used against anyone criticizing governments or large corporations.)

Re:Google terminated our Developer Account, says it is “asso... by nkrisc on Wednesday March 30, 2022 @11:13AM (#30856280)
> At Google's scale, I don't think there are enough humans on Earth to deal with the abuse. As a result, there are going to be some innocent casualties.

Perhaps they've scaled too large then. I don't think "we're too big to be held accountable for screwing innocent people" is a valid excuse.

Re:Google terminated our Developer Account, says it is “asso... by spookthesunset on Wednesday March 30, 2022 @12:41PM (#30857595)
One thing that never gets talked about is how do we go after the actual fraudsters and bring them to justice? How can private industry work with law enforcement across the globe and get some of these dudes behind bars?

I’m not saying it would be easy. Doing so would require insane amounts of coordinated with basically every country on the planet. But damn would it be nice if fraudsters couldn’t easily hide behind internet anonymity.

Re:Google terminated our Developer Account, says it is “asso... by geophertz on Wednesday March 30, 2022 @12:24PM (#30857369)
> At Google's scale, I don't think there are enough humans on Earth to deal with the abuse.

I disagree.

Some time ago, a person (on HN, although it may have been somewhere else) did an estimate of what it would take for Google to review all 500 hrs of video uploaded to YouTube every minute.

The result was that Google could more than afford it.

Based on that I don't see any reason Google couldn't add more humans to deal with these ban appeals.

Re:Google terminated our Developer Account, says it is “asso... by matsemann on Wednesday March 30, 2022 @11:51AM (#30856847)
While I see your point, when it's accounts with years of history, millions of downloads, thousands of dollars paid in sales etc., it should be easy to discern those from spammers creating thousands of new accounts with no standing.
Re:Google terminated our Developer Account, says it is “asso... by spookthesunset on Wednesday March 30, 2022 @12:37PM (#30857550)
Maybe. Or maybe the account in question has a long history but got taken over by a scammer. It’s the oldest trick in the book. Find long standing accounts and take them over. Boom, now they look totally legit!

Shit is hard. If it was easy to tell fraudsters from real people we’d never be discussing this. The fraudsters are willing to invest unimaginable amounts of time and effort to get into your systems and do their dirty work. Every fix you make will eventually be routed around. Always.

Re:Google terminated our Developer Account, says it is “asso... by gambler on Wednesday March 30, 2022 @10:26AM (#30855649)
People who consider this an issue either wouldn't work for Google to begin with or end up butting heads with management and leave the company. The benefits of self-selection.

There is also a lot of internal corporate training and team-building designed to redefine "ethics" to make this sort of stuff acceptable.

Re:Google terminated our Developer Account, says it is “asso... by leros on Wednesday March 30, 2022 @10:35AM (#30855760)
It's probably something they're not really aware of. My company does something kinda similar. We're an aggregator of sorts. We have all sorts of quality and safety rules and we de-list things all the time for violations. As long as we aggregate 95% of what's out there, our users are happy, so we don't have massive incentive to spend lots of time manually working with rule violators, especially when a good chunk of them are actually scammers manipulating the system and harming our users.
Re:Google terminated our Developer Account, says it is “asso... by hbn on Wednesday March 30, 2022 @10:55AM (#30856039)
I'd be willing to bet that there was no one developer or team who was given a task such as, "run our audit AI on accounts and permanently blacklist them forever if that's what the algorithm wants"

It was more likely years worth of minor changes that got it into this state. Teams adding different algorithmic checks for various things, where the output is just setting a flag on the account. And other teams adding account termination logic for certain flags or a certain number/combination of flags, not knowing exactly how the flags are set. Though maybe that would be the bad task. I'm just spitballing.

Re:Google terminated our Developer Account, says it is “asso... by inafewwords on Wednesday March 30, 2022 @10:22AM (#30855604)
Nope. You can tell by how it is implemented.
Re:Google terminated our Developer Account, says it is “asso... by rexreed on Wednesday March 30, 2022 @10:43AM (#30855892)
Exactly. You think that developers are paid to stop and think about the outcomes of what they do? They're all too busy running sprints and meeting OKRs and performance metrics and hustling design and code reviews and playing internal politics to worry about the effect of what they are doing on their users and ordinary people. Besides, even if they did have an end user in mind, it's not the "users". They're motivated by the customers - advertisers.

I really found this video to be enlightening on our current techno-dystopian state: https://yewtu.be/watch?v=GWvFZ99s558

Re:Google terminated our Developer Account, says it is “asso... by bogwog on Wednesday March 30, 2022 @10:25AM (#30855642)
Even if they do, it’s not their responsibility to fix those problems. It’s government’s responsibility to step in and fix bad behavior that market forces can’t fix (like when it’s more economically viable to destroy thousands of livelihoods per year than to invest in human review processes)
Re:Google terminated our Developer Account, says it is “asso... by matsemann on Wednesday March 30, 2022 @10:34AM (#30855755)
I do feel it's my responsibility to think of the side-effects of what I do and the consequences that bear on others.

But as a sibling to you say, maybe I therefore self-select away from those kind of companies, and those not thinking or caring about those things end up taking those jobs.

Re:Google terminated our Developer Account, says it is “asso... by bogwog on Wednesday March 30, 2022 @10:42AM (#30855877)
That’s because you probably have a moral compass, but that still isn’t enough. If you were a decision maker at Google, those morals would probably get you fired at some point.

The people running a company have a legal responsibility to act in the interests of their shareholders. Even if making a morals>economics decision doesn’t get them into legal trouble, it can get them into trouble with shareholders, and they may be fired and replaced with someone who doesn’t have a moral compass.

So if the infrastructure doesn’t have room for morally correct decisions, no amount of martyrdom from executives is going to eliminate bad behavior in the long run.

Re:Google terminated our Developer Account, says it is “asso... by hwers on Wednesday March 30, 2022 @11:11AM (#30856263)
They select for sociopaths coding these things. If someone is able to be bothered by these actions, they won't last long or will quit voluntarily and someone more sociopathic will take their place. It's a problem higher up the chain of management.