Ahem.

> "The Dragonfly project was an Internet search engine prototype created by Google that was designed to be compatible with China's state censorship provisions. The public learned of Dragonfly's existence in August 2018, when The Intercept leaked an internal memo written by a Google employee about the project."

https://en.wikipedia.org/wiki/Dragonfly_(search_engine)

At least the project was leaked by a MAMAA employee (it's no longer FAANG), but there might be a kind of 'internal social credit score program' being applied in the Big Tech world to get rid of those with pesky moral standards...

Incidentally this is a major reason why democratization of corporations is important - I tend to believe the regular rank-and-file would be less willing to implement something like this than the top shareholders would.

Yeah that's not a social credit system. It's a censored search engine. I'm fine with that, as a Google employee.

What if it also reports your 'bad searches' back to central control and uses that as a factor in calculating your social credit score?

Yeah maybe, but this is absolutely nothing like what I was responding to. Maybe Google will build tools for authoritarian regimes, maybe they'll sell them to US gov, who knows. None of that is anything like what I was responding to, a system of forced labor camps and heritable servitude. That's just not the world we live in.

Hint: They won't say "develop software to oppress the people". It will say "develop software to track child abuse and criminal activity" and "develop software to track people's preferences to be better able to help them find the products they need" and very slowly devolve from there, one incident at a time.

> None of that is anything like what I was responding to, a system of forced labor camps

I saw stories from the US about debtor prisons and about for-pay prisons with very questionable tactics used to fill them, people who cannot fight back because they have no money, often don't ever get an actual date in court but are forced to submit to accepting a sentence because the prosecutor threatens them with trying to get an outrageous sentence should they actually want a real judgement.

https://www.aclu.org/news/criminal-law-reform/coercive-plea-...

It's old: https://elibrary.law.psu.edu/cgi/viewcontent.cgi?referer=&ht...

I think your opinion on what is possible in the US justice system depends on where you are, i.e. location, as well as in the social hierarchy.

This is a slippery slope argument, and by itself is just a restatement of your belief that ads are bad. I don't think there's any plausible path from building ads and product recommendations to gulags. Maybe if we go another 20 years without anything like that happening anywhere in the world you'll be convinced?

See also: newspaper, radio, television, cable television, internet, mobile

There's been an unceasing procession of potentially-abused technologies in modern times, and we've more or less managed to thread the needle.

Instead of pessimism, I'd encourage people to engage in advocacy groups for their desired outcome!

The road to hell is paved with good intentions.

It's not that a company like Google is going to release a Google Gulag product after years of working on it in secret.

What people are worried about is that the various pieces of technology that Google is developing can be used to make a horrible dystopian future and that one day, even if Google as a corporation, or the individuals who make up Google as employees decide to fight against it, they will be unable to stop a fascist government from using that technology to do exactly as is described by OP, or even worse.

There are unintended consequences to everything, and you may think that America is the freest nation on Earth, and that Google is the greatest company ever, but that could quickly change.

It could happen here, and we must be eternally vigilant.

Sure, and internal combustion engines were critical in allowing the Nazis to overwhelm their European neighbors. But like ads and product recommendations, internal combustion engines are overwhelmingly used for good so we accept that bad actors can sometimes misuse them.

The prolific use of internal combustion engines is a large part of why our planet is warming which is causing the impending collapse of food chains and societies across the world.

I don't accept the premise that advertisements and product recommendations are even comparable to either themselves, or the internal combustion engine, nor that what Google produces is even comparable to traditional advertisements.

I only accept begrudgingly what bad actors like Google do with technology, and only because I don't really have any other choice, do I?

> But like ads and product recommendations, internal combustion engines are overwhelmingly used for good

Without internal combustion engines (at least until electric replaces them), billions will starve as tractors can't plow fields, and trucks can't deliver food.

Without online ads and spying...?

See you have to insert the word "spying" to make your case, because by itself there's clearly just nothing wrong with personalized ads. Online ads are good because they make tons of business and economic activity possible and connect people to products that they want. People are better on the whole with online ads.

There are ways to misuse ads, but it's hard to make the case that they are inherently bad without inserting weasel words like "surveillance" and "spying".

What if every time you use it someone dies?

“When it comes time to hang the capitalists, they will vie with each other for the rope contract.” - attributed to Lenin, probably apocryphal, but accurate.

It really depends on the government(s) in which the FAANGs exist. They're not quite powerful enough to escape governments.

If the USA goes all the way towards fascism, the FAANGs will go along for the ride and implement all sorts of terrible things.

Sure I believe that, but I think because of the way these companies operate internally and because of the extraordinary amount of transparency and voice given to employees employees (particularly at Google and Facebook where I have worked), these companies would be amongst the most difficult for a hypothetical fascist US gov to use for nefarious purposes. I've worked at 3 other large US tech companies (Qualcomm, Intel, APC) besides these, and they are radically more ethically malleable and able to conceal bad things from their employees.

All of them have done and are doing ethically questionable things already, and they really haven't had much trouble keeping people working there. Google and Facebook would be the first two I'd expect to go along with whatever awfulness, if they're not driving it themselves.

Perhaps this is cultural difference at play here, before you downvote this comment to oblivion I'd like you to hear me out.

In India, we were expected to create value for our stakeholders, and bringing personal political views into office is deemed highly unprofessional. This doesn't stop the junior talents that are highly influenced by foreign media to have a go at it, but they're often silenced with righteous resolve.

Tldr: Employees that openly engages in activism are often seen as problematic elements, which severely degrade the cohesion and unity of a company, therefore they're often the first on the chopping block. Are things different in the states?

> This doesn't stop the junior talents that are highly influenced by foreign media

I don't understand. They are not influenced by the local media, somehow? Only "foreign" one? What is the magical mechanism behind that?

And of course, it's only "activism" when it does against the accepted order. The activism supporting current powers flows much more smoothly and does not cause any such ripples, so you conclude it does not exist?

> bringing personal political views into office is deemed highly unprofessional

Ethics are not politics. Engineers should be required to refuse unethical orders and work.

This isnt cultural. This "deemed unprofessional" and "silenced with righteous resolve" is just reflective of the relative power differential between employee and employer. It's an artefact of capitalism.

It also doesnt happen when the employee has more power.

It already exists in the US. Every person is scored by data brokers based on their address and income.

That's a credit score, not a social credit score.

That's not really true either. The US credit score is a type of social credit score.

Case in point: You generally cannot be a responsible adult, living within your means, and expect to have a credit score. Most utilities won't report on-time payments (and usually won't report until you are a few months late, sometimes only if you have a late final bill). Most landlords don't report. Saving for a used car doesn't earn you credit, and credit generally doesn't care if you have enough money in the bank or if you earn enough, at least if you aren't rich, anyway.

Basically, you need to have the right sort of credit history and borrow in the correct fashion: For most folks, this means getting a credit card or paying too much for a used car just to prove that one day, they might be reliable enough for a new car loan or a house loan. In the meantime: Don't go into medical debt. Don't avoid most debt and simply live within your means. Don't check your credit too much (This is around 10% of your score).

And not doing things correctly affects where you'll be allowed to live, where you can work, how much you pay for car insurance, and things like that.

Just because it isn't just like another countries doesn't mean it isn't a social score with a different name.

None of the behaviors you describe are social behaviors, they are financial behaviors. Social factors are why they have to add the word "social" to the term "credit score".

Framing zero-debt as being a "responsible adult" and "living within your means" doesn't make sense as a counter-argument. It's not social as it doesn't involve other people. For eg, your credit score isn't affected by the social associations you have with other people (with maybe the exception of marriage because that combines non-social assets). Your credit score isn't affected by your social interactions with other people. For eg, sharing your anarchist views with other people or maybe a bank teller doesn't affect your score.

It's also just untrue, leveraging zero-interest loans and loan perks like purchase protection can be very responsible and still living within your means.

It was explained to me like this: the credit score is not a credit score in the sense you might imagine. It doesn't track your likelihood to repay. It's a score indicating your likely aggregate profitability to banks as a borrower.

That does make sense, though it makes it that much more of a horror that places are using it to determine whether or not you'll make a good employee or a good renter. "You won't make money for the bank, so pay more for insurance and don't rent here". Those places definitely use it differently than the folks providing the scores.

It isn't credit score. That isn't easy to peddle in because it requires authorization to access. Data brokers have their own scoring system based on all the other unregulated data they have on every person.

The fundamental difference is not perfectly clear to me.

So it's a social credit score, but not their social credits score, its our social credit score, and that makes it okay :)

I wonder if workers at IBM prior to World War II would say the same about the holocaust. Something as simple as period tracking apps can be turned against people. Sure it starts off as a marketing department, then it moves on to a domestic surveillance program for terrorism, and then you are rounding up people who get abortions and miscarry. After that, who knows? Gays? Atheists?

Agreed, and it doesn't just go one direction politically. Totalitarian governments hunted and killed Christians and capitalists with tremendous zeal throughout the 20th century.

This isn't really about left/right, it's about power. Governments simply can't be trusted with the power to track and target dissidents at scale.

We nerds are to a great extent the modern gatekeepers of that power. If people ask you to track and suppress free speech don't do it. It starts there.

You don't need a FAANG compaby to do this - just a sqlite database and sufficient motivation for prosecuting some cyberbullying campaigns

I mean, hell, how many facebook groups are out there that feed on persecuting outgroups? You dont need a central state to do this stuff. A state can provide the NKVD/Stasi but its peoples neighbours who use it to screw one another. Deliver them Facebook and they'll do most of a social credit system for you

> Not sure what your professional experience or knowledge of politics or technology is

I’m not sure how far back in time your experience or knowledge of politics or technology goes, but there are plenty of examples of the US government using technology of all levels to target and control individuals deemed “troublesome”.

Yeah for sure, but Google wasn't doing it.

Oh c'mon, have some imagination. Have you not seen Black Mirror?

S3E1, specifically.

That's a TV show

One that shows a very plausible reality where FAANG comes out with a rating system, and how that might potentially go down.

Agreed, as a thrice-faanger.