As someone who’s also worked in this area, I disagree with this take.

There is typically clear objective function of a recommendation system.

What Twitter is optimizing for is what’s of interest here. And some of the hidden business rules. It’s likely these are specified in the code in an obvious way.

How exactly they achieve that is the part that is complex and relatively indecipherable.

It’s possible that it’s designed in such a way the optimization objectives are also unclear, but that would indicate a bad design and be to the detriment of the company and users.

Yeah seriously. If they are not able to put into words or diagrams what the algorithm is doing then the company itself has no idea how it works. And that to me would suggest it’s far from optimal.

Many complicated research papers have had no issues describing their models at a high level. This should be no different.

Twitter PMs would argue they've already communicated the objective function: relevance and engagement.

My point is that the devil is in the details and implementation. These details are likely something that no one person understands and no one person is able to fix. The concept of being able to extract "the algorithm", factor it out from the codebase and share it with the public doesn't make sense to me. It won't be possible to fully understand how Twitter serves recommendations and ranks posts without understanding how all the different services at Twitter interact. Are they planning on open sourcing all of Twitter? Highly doubtful.

Yeah, this makes no sense. There is no golden algorithm that Facebook, Tiktok or Twitter has figured out.

All these feed rankings are complex combinations of features, models coupled with weights and filters. On top of this abuse detection layers are added.

Unless Musk is planning to open source user data to show what all the "scores" and "features" for all the entities are and how they were reached to, this will make no sense. The whole argument against some people being downranked has been, why me? Just writing a whitepaper to tell the general methodology, is not going to make that go away.

On top of that, exposing every vector through which you measure and stop abuse, will just allow for more sophisticated abuse.

Can you mention the source of your information?

The openness for the algorithm is to make public how the algorithm works internally. It has nothing to do with how "novel" or "good" or "bad" the algorithm is. It is just a way to check if there is anything fishy going on in there. I don't see what the fuss is about if it only makes the defacto town square of the internet more transparent and open.

Twitter isn't the town square of the internet. The internet is the town square of the internet and you aren't owed a fair ranking in relation to a certain search term user or combination thereof because no singular definition of fair.

> Twitter isn't the town square of the internet

Maybe to you. You are entitled to your opinion. I won't censor you for having a contrary opinion.

> owed a fair ranking

Well we are getting it whether you like it or not.

> because no singular definition of fair

Of course there is. If New York Post's tweet about Hunter Biden's story gets blocked, I would like to see Taylor Lorenz's story about LibsofTikTok get blocked in a "fair" system. Since both are doxxing after all.

I don't believe the point was ever to convey an complete understanding of the system, but to allow for the identification of "red flags": Specific functions that would be deemed controversial or counter to equal opportunity speech.

Is the idea somewhere in the-algorithm there's a function called `derankGOPMembers()`?

Twitter's been pretty transparent in how it "deranks" certain accounts [1]. What more would come from opening the code that certainly not include the actual database of "no no terms" (if you were to believe that exists)?

[1] https://blog.twitter.com/en_us/topics/company/2022/our-ongoi...

The idea is that people should be able to find out if there is such a function. It's taking a stance that Twitter has nothing to hide. In my opinion, it's a great way to build trust with users, something that all of the popular social media platforms lack.

Twitter is transparent in how they do shadow bans? They still deny they are a thing but it's clear to see the network effects of them. Twitter is transparent - that's the most hilarious thing I have read today!

Has there ever been any actual indication of twitter shadow bans being a real thing, or are we putting it in the same "weird conspiracy theory" category as "facebook listens using microphone to target ads"?

I follow a few people in "gamer/twitch twitter" and every now and then this meme pops up of twitter secretly deranking "go live" twitch.tv tweets, which is much more palatable to these people than the reality of just no one caring about their boring tweets.

> Has there ever been any actual indication of twitter shadow bans being a real thing

Please stop living in this bubble. There have been plenty of reports of Twitter shadow banning speech it doesn't like. It has even suspended accounts multiple times only to reinstate them by saying it was an "algorithm error" or it being "wrongly flagged". In India we regularly see Twitter suspending accounts that have not violated any Indian law.

All because it doesn't align with the left-leaning ideology of Twitter. Even trends are boosted to favor left-leaning news portals over right-leaning news portals (which is visible in the trends section).

An example: https://www.freepressjournal.in/viral/bringbacktrueindology-...

^ This account specifically was suspended a total of 3 times. Ultimately the anonymous account owner decided to come out in the open and open a Twitter account in his real name. He got banned again very recently. In all those instances, there was not even one tweet of his that "crossed the line" when it came to anything illegal. Be it speech, instigation of violence or even a curse word. Nothing at all. The suspension was purely ideological.

What you linked to is not shadow banning, and was explicit banning/suspension that happens transparently (in the sense that anyone can see that it happened).

> What you linked to is not shadow banning

Of course this is evident too. And let me quote directly from an Opposition leader in India (whose party/ideology I completely oppose):

"“I have been reliably, albeit discreetly, informed by people at Twitter India that they are under immense pressure by the government to silence my voice. My account was even blocked for a few days for no legitimate reason,” Gandhi said in his letter."

"“For example, in May 2021, my account gained roughly 640,000 new followers. This had been the case for several years until July 2021. Then something strange happened. Since August 2021, the average number of my new monthly Twitter followers has fallen to nearly zero,” he claimed."

Article in question: https://www.businesstoday.in/latest/in-focus/story/rahul-gan...

Now just because I am opposed to his ideology (which is Left-Centrist) doesn't mean I want him banned/shadow-banned from the platform. He has every right to continue to voice his opinions. Now he claims that it was the Government (whose ideology I support) which supposedly interfered and pressurized Twitter to manipulate follower count. I would like transparency on this if my Government, which I support, did indeed do that. Or did Twitter itself decide to shadow ban his account. Or was Twitter doing it legitimately because he had a lot of bots following him and when they banned those bots his follower count decreased. Either ways, I want the facts in the open.

As far as shadow banning content goes. It happens on a fairly regular basis. Open any Tweet that has quite a few comments right now and scroll to the end. You typically have a "Show additional replies, including those that may contain offensive content". Take this tweet as an example: https://twitter.com/Bob_Mayo/status/1518679097672617990

When you click more replies it shows "Show additional replies". Clicking on it is an innocuous tweet with laughing emoji. But the tweet above it isn't put behind a collapsible card. How many would click on the "Show" button and read those replies that are hidden by default? This is throttling reach a.k.a shadow banning. It is not like there is some abusive word being used in the tweet. It is just laughing emojis.

Yes, there is. I had a friend accidently shadow ban himself by posting nsfw art without marking it as nsfw. After that his future posts wouldn't show up when searching hashtags he had tweeted on. He had to make a new account until the older one was good again.

take a look at the Twitch leak sometime. There was a hard-coded list of bad words (largely racial epithets & other 4chan constructions). Presumably there is a way to create some kind of config file to penalize "bad users" of Twitter, I know these things existed in other orgs I've worked at

This is likely in response to upcoming EU legislation on algorithm transparency[0]. It's not useful, but they'll need to do it eventually.

[0]: https://mashable.com/article/eu-digital-services-act-big-tec...

It's so difficult as someone who works in technology to tolerate this idea that there is "an algorithm". Maybe this will get flagged on HN but the overwhelming feeling is to just scream "don't be such a fucking idiot, this is a $43Bn company it doesn't boil down to 50 lines of code" and hell, there are plenty of examples of 50 lines of code at my company I could spend a month really understanding and even then not understand the full ramifications. It's a stupid persons idea of how tech works.

I don't think that most people actually believe "the algorithm" is like a Python script or something.

The average person on twitter isn't a software engineer. They won't know what python is. They don't know what an algorithm is. So let's be clear about what the baseline is here. There's no point talking to them about how you distribute queries using micro-services or whaever bullshit twitter's engineering team bought into this season.

This idea that "the algorithm" is something you can just "publish" is a pernicious lie told by people like Musk - who knows it isn't true - to the general public who don't know better. The "algorithm" in reality is probably farcical calls to cusotm APIs that no current employees understand well enough to modify which is why Twitter hasn't changed anything in coming up to a decade - which is when all the engineering talent left.

I believe all the public wants is a report with an overview of the mechanisms that dictate how and in which order tweets are shown to different users. Even if no current Twitter employee knows the system well enough to write it, they can still create a taskforce to do so. Sure, 3rd party APIs and machine learning stuff may obfuscate part of the system, but I'm sure the best they can do is good enough for starters.

This is a bad take. Just because it is a stupid unaccountable heap of shit now doesn't mean it has to remain that way in the future.

I'm not talking about whether it's accountable or not, I'm saying that to actually share the algorithm you're basically saying that you're going to open source Twitter's entire code base. Oh and when you do that the average person will be no better off because they don't read code anyway. And when engineers read through the code it's going to like "Where did you get all these different variable values from?" and the answer isn't "We came up with a method for valuing tweets from first principles", it's going to be "We showed 7 billion people tweet X and 7 billion people tweet Y and tweet X caused 5% more people to engage so we tweaked this value".

And sure, you can say "Well that's a bad way of designing the algorithm" but then what you're really saying is that you don't want to open source the algorithm at all, you want to re-write the algorithm to satisfy your sense of how the world should work with no evidence it'll actually work.

> , it's going to be "We showed 7 billion people tweet X and 7 billion people tweet Y and tweet X caused 5% more people to engage so we tweaked this value".

There is an entire new subfield of ML that is tackling this problem. There are now conferences dedicated to this topic. It is not an easy problem, but it is not impossible.

There are hundreds of researchers working on fairness, interpretability, trust and explainability in ML and a lot of them are working on models much much bigger than what Twitter's feed might involve.

This is a good starting point:

[1] https://www.fatml.org

> And sure, you can say "Well that's a bad way of designing the algorithm" but then what you're really saying is that you don't want to open source the algorithm at all, you want to re-write the algorithm to satisfy your sense of how the world should work with no evidence it'll actually work.

You can still open source multiple steaming piles of shit and then let the community improve that so that it is more widely understandable and trusted. See [1] again.

No, it's much worse. The general public thinks it's a one line mathematical formula. The secret sauce.

agreed, I see this more as a simplification of how a network of interdependent systems operate, just because its not simple doesn't mean there isn't a fundamental structure the farther and farther you zoom out.

> it doesn't boil down to 50 lines of code

No one said that. You created a straw man and are arguing with it.

This comment says more about you than you think.

Go on then, what do you estimate the complexity of the algorithm to be. Because I'm going to ball park it and say to fully understand the algorithm behind twitter's feed, you probably need to open source twitter's entire production code base and it's research framework.

> source twitter's entire production code base and it's research framework.

So why is that impossible? Mastodon exists. Is Twitter engineering so horrible that they are orders of magnitude worse than Mastodon's engineer quality?

Understood and several of us have dealt with large complex systems.

Open sourcing algorithm or code is not about everyone go and analyze the same, instead when controversy or issues arise it'll be readily available for independent experts to review it.

My point isn't that it's impossible to analyze. It _is_ impossible to analyze without the context of all of Twitter's other services though. Twitter are not going to open source all of their services.

I agree. I doubt the Twitter moderation system is more complicated than the Linux kernel or any of the open source compilers. Sufficiently motivated individuals contribute, find a fix bugs all the time.

Even just releasing the audit trail of shadowbans, upranking, downranking would be meaningful. Full transparency of actions taken in the past and going forward.

This will never happen, not even under Elon. It makes them wide open to litigation.

> It makes them wide open to litigation

How? What specifically opens them up to litigation? It is a private company. It can do anything it wants as long as it is within the bounds of law. They have every right to ban anyone. Even on flimsy grounds. The idea here is to expose all the moral wrongs and bring it to the fore. Not that they are legally in the wrong (most cases they aren't). An audit is a good place to start.

Not every country follows section 230, actually only one does.

I don't think transparency of internal Company controls/audits has anything to do with Section 230.

Section 230, in brief, only provides immunity for social media providers from being responsible for the content that is posted by users on their platform. It has nothing to do with internal company policies. Rather, Section 230 actually enables internet companies to moderate content through the Good Samaritan protection.

So, it is actually worse that Twitter was moderating content in majority countries (except USA) where Section 230 wasn't even recognized. Revealing what exactly happened behind the scenes would not be a sufficient reason for litigation.

When Twitter was pulled up in India for its opaque moderation policies, it tried to quote American laws for its defense. In fact, when Jan 6th happened in USA, Twitter was quick to ban accounts of those who took part in the protests. 70,000 accounts were purged from Twitter. However, Twitter refused to ban accounts of those Khalistani terrorists who vandalized the Red Fort on Jan 26th in India and only partially complied with Government of India's orders. This double standard was visible to majority of Indians. So it is not like Twitter actually follows the laws set by the Country it is operating in either. There are multiple instances where Twitter has refused to follow directions by Government of India or by the Courts in India. You can read more about it here: https://www.hindustantimes.com/india-news/have-to-follow-ind...

So yes, moderation policies cannot be opaque. It has to be transparent. The reason for transparency is so that we know exactly why and for what reason was an account banned/shadow banned. If the Government of India sends a legal request for take down of accounts, it has to be complied with. Twitter cannot decide to invoke US laws in India.

Also, read this to understand more about this issue: https://techcrunch.com/2021/08/10/twitter-now-in-compliance-...

I guess I kind of think about these things not as algorithms, but as a collection of frontends and backends. Collectively, any human request (typing [ ramen shops near me ] as an example) will be handled by a bunch of different code, typically that code is structured as RPCs.

We can think of the main interaction as being a query which is an RPC payload. The contents contain the user request and a wide amount of other context (either referenced by a collection of keys like cookies, or materialized like fields that specify the user's age) and the response is a web page which contains sections (the web search response to the query, as well as the ads; either these could be rendered to two different frames, or interspersed, by the result presentation engine).

That query -> frontend translates into a tree or a graph of requests which collect up various bits of contextual data required to satisfy the query. For example, the query terms might be rewritten slightly and then sent to a web search backend which searches/ranks documents and returns the top matching documents on the organic web, or sent to an ads backend that returns the top matching bidders for those query terms. Again, just RPC/responce, although the actual context that the frontend and backend systems are dealing with, and use to modify the result, are truly enormous.

Each of those backend systems itself was produced with an enormous amount of data processing and contextual data that is available at serving time. All of this is implemented using various algorithms; everything from the TCP algorithms that manage bandwidth to the neural networks doing inference on the joint product of the user context and the query context and the ad context, and the logging system that writes the queries and their clicks to centralized storage for more ML training.

In theory though you could set up a system that compiled the full web stack, and ran the end to end of a user query, dumping all the intermediate RPCs, etc, from a modestly sized instantiation of the production system. and people could sit down and inspection what terms affected query result order, or which pages were omitted at which part of the filtering, or what data was logged.

It would be hell for a team to maintain and keep up to date wrt the production system, but many folks do this any way to have a simple version of the system around so they can make quick changes and see if it breaks part of the complex system without doing a full deployment.

Even if publishing the underlying algorithm isn't useful, they should be able to do research on what the end result of endless a/b testing has created in practice. I'm sure you could find the highest level heuristics and that would be extremely valuable to both the public and decision makers internally.

You won’t know what they were A/B testing though.

I get what you're saying. I'd imagine this repo will be closer to pseudo code than "complete" code for all systems that a tweet flows through. For example, the system that flags/remove picture of hotdogs would likely be represented as "if image contains hotdog: weight: -500"

That could be a starting point for a DSL that realizes the algorithm.

As you probably know, the algorithm is NOT the code, never has been.

It can be pseudo-code or diagram or whatever that can be used to understand what logic lies behind decision making.

Just because it has evolved to be a complex monster that is unnaccountable doesn't mean it has to be that way in the future.

There are ways to translating trained ML models and associated systems into understandable hierarchical rules.

Twitter's timeline is NOT AGI.

it’s almost like this thread is nothing but people being pedantic and saying, there isn’t just one algorithm, it’s multiple algorithms. yes do you really think elon doesn’t know that? it’s almost like he’s just trying to get the point across in the most simple and basic way possible. pointing out that the recommendation algorithm isn’t just one algorithm isn’t profound at all. this entire conversation is mostly just people who want to argue and point out that they know something about how large tech companies work. congrats, why don’t you tell elon since you’re obviously so much smarter

For someone who worked on recommendation systems, you really don't seem to understand the concept of an "algorithm" across the abstraction of multiple systems and at different layers of the stack you worked on.

In fact, a lot of people here really think what people are talking about is the equivalent of what is handled in a subroutine.

No, what people are talking about when they talk about "the algorithm" is anything affecting the result set they're reading. Concepts like eventual consistency and edge computing are... well... a part of a model which laypeople, and even reasonably technical people call an "algorithm."

Being pedantic about whether or not this happens in an SQL query, or across multiple codebases, or by region, doesn't escape the question.

Absolutely agree with what you say, but I wanted to pick a nit here:

> Being pedantic about whether or not this happens in an SQL query, or across multiple codebases, or by region, doesn't escape the question.

Actually, epistemic ~"muddying of the waters" is a well proven technique to control perceptions and public discourse. If it works on HN folks, I expect it would work much more easily on amateurs.

Did you just tell me to go write a map/reduce function in Erlang?

How did you work with a system that couldn't be understood even with access to full source and documentation? Surely your engineering process wasn't purely stochastic.

Let alone those 10s of GB of embeddings which might be user identifiable, without those, how do they claim to 'open source' the algorithm?

It would be gamed 100%.

theres gonna be a few crazy guys who devote their entire time to understanding some obscure part of this algorithm

i'm looking forward to see what's going to be on it.

I don’t understand people like you. Just because Elon Musk used the term “the algorithm” in conversation, it must be interpreted literally? And for someone like Elon Musk, he doesn’t know the complexity and need you to point out?

It's so weird. We get it, data scientists, your implementations are a garbage mess of last minute paper submission levels and you yourself are blind to half the things happening. That doesn't mean an implicit algorithm doesn't still exist. The term is perfectly applicable and it would probably do you a world of good to abstract it to the point where you can legitimately put it into a single repo.

Of course it doesn't make sense. I think it's just a dog whistle to the people who believe google have a guy in a room somewhere turning the "conservative search results" lever down a notch during elections.

Not saying Google is turning down conservative search results, but they absolutely can. Using the same cycle of human raters and tweaking weights they used to push down comparison shopping sites.

See http://graphics.wsj.com/google-ftc-report/

It's the man in a room part I was emphasising. It's simultaneously massively incorrect about how software actually works, misunderstands social dynamics inside companies, and massively stokes the fire when it comes to conspiracy theories along these lines (in this case at least)

There absolutely are explicit lists of controversial words that get weights on the models for up/down ranking or hiding content or flagging for manual review.

It's also true that these lists are only a small part of what affects your fringe website or twitter accounts.

Are you claiming that you have comprehensive objective knowledge of what did and did not happen in privileged Twitter operations across time, or is this more like your opinion on the matter? And if the latter, how is it that some people's opinions "don't make sense" or "are dog whistles", but yours are epistemically flawless (assuming you actually believe them to be true, of course)?

So you're saying google doesn't have infrastructure to allow a human specified list of keywords or domain names to "Twiddle" the results returned in varrious services such as news, search, and youtube for the purpose of artificial promotion or de-boosting?

"Among the elite, and within Twitter specifically, there is much more inclination to ban the right.

I say this as someone whose political views, if you force them onto the left-right spectrum, probably end up about 80% toward the left. E.g. I've spent millions over the past several elections supporting the Democrats.

It used to be that censorship was something the right did, and free speech was something the left were in favor of. But over the last few decades, banning "problematic" ideas has become a huge component of left culture (http://paulgraham.com/heresy.html).

Plus tech companies in general, and especially Twitter, lean to the left. Imagine walking around Twitter pre-Covid. You'd find plenty of openly far-left employees. How many openly far-right employees would you find? I don't think you'd find any.

The combination of (a) the left's recent focus on banning heretical ideas, (b) the leftward lean of tech companies generally, and (c) the leftward lean of Twitter even among tech companies, means that right-wing speech is much more likely to get banned on Twitter than left.

That's why people on the far right keep starting lame Twitter alternatives. You don't see people on the far left doing that. They don't need to. They have Twitter."

https://twitter.com/paulg/status/1515235822890532864

I’m pro censorship. We have and need speech norms.

The left and right are not equal. The left does not rely as much on lies to advocate its positions, and the left is not as oriented around destruction and regression as the right.

We need a more robust understanding of speech than “allowed or not”. Emphasis & volume matters.

I'm someone on the left and I fundamentally disagree with literally every point you made.

Good for you.

I'm a firm believer in the horseshoe theory.

If you listen to the intolerant wing of the right, the parts that would legitimately support overturning elections and arresting political opponents, they sound eerily similar to your sentiment.

The horse shoe theory makes a lot of sense until you actually research the right and left and discover that they are in fact very different.

It’s a thought stopping theory that allows you to remain smug and correct despite being deeply ignorant.

those modern lefts.. just look at russia. censorship is how it all started there.

The fact is that humans are not smart enough to reliably detect fact from fiction. Half of this website would just as soon look the other way at trumps track record.

Something needs to be done. Free speech maximalism is a threat.

"does not rely as much on lies to advocate its positions"

To be honest, that's hilariously false. The left suppressed Hunter Biden's laptop story for two years, saying it was misinformation, before admitting it was true. The IRS targeted right-wing and Christian non-profit organizations deliberately for auditing. Obama claimed that forcing nuns to buy health insurance covering birth control was necessary, and claimed you could keep your doctor under Obamacare, which even PolitiFact called a Pants-on-Fire Lie. The left claimed that a baker needed to make a personalized cake supporting gay rights, or it would open the waves to discrimination, even though the baker was willing to sell any other cake to them that wasn't customized with that particular label.

I can go on and on.

Please, do go on and on. You can stop when you get to the left attempting to overturn the American presidential election.

Edit: are our neoliberal overlords a bunch of cold hearted shit heads? Yes. Should we elect and promote barely disguised fascists in response? No.

Huh. I remember when the left had Maxine Waters saying to publicly harass voters of the opposite political party, and during the 2016 election when they tried to harass and dox the electors in hope that would prevent Trump from being elected.

Do you remember when the sitting democratic president of the United States claimed to have won the election and attempted to persuade his colleagues to overturn it?

Who the fuck cares about Maxine waters?

They framed Trump for treason and thereby completely undermined him before he even formally took office. Are we going to memoryhole that? That’s the same thing as stealing the election.

One is an indisputable coup attempt and the other is exaggerated.

> it’s not the one you think

Yes it is.

> That's why people on the far right keep starting lame Twitter alternatives. You don't see people on the far left doing that.

They don't need to because they have cool Twitter alternatives. Like Mastodon.

Which is also the Trump alternative, iirc. Just the far left seems to respect the license.

No, it's not THE trump alternative.

It existed before.

Open source, such that anyone can use it

I meant used by the Trump alternative.

That's fair, although part of the reason I call Mastodon the "cool" alternative is because of its federation. AFAIK the right wing forks don't generally have that.

Which side is on a book banning tear again?

Both! The difference is in what they target.

"On the right, banning books means trying to prevent their kids from reading them. But on the left it often means trying to prevent anyone from reading them." https://twitter.com/paulg/status/1515563419386191880

The example he cites is from a tweet made by a guild of writers for the Oxford University Press.

> We, the members of the OUP USA Guild, are calling upon our colleagues and authors to take a stand against the upcoming publication of “Gender-Critical Feminism”. Sign the petition below (OUP employees and authors only).

Yeah, that's not a "ban", and "gender critical feminism" = TERFs. Ie, "people who think trans women are faking it so they can rape women and children in bathrooms or 'cheat' 'the system'"

Remind me which side thinks businesses have a constitutional right to refuse to serve people if it violates their "beliefs"...yet finds it unacceptable that unionized workers are objecting to publishing of research on a subject?

Also, remind me which side is boycotting a major entertainment company...because said company's CEO voiced displeasure...at a bill that bans any language referencing non-straight gender orientations?

That reads like those "UI vs UX" images where the creator just takes 2 random photos and sticks them next to each other—while simultaneously being deeply ignorant of what’s going on in the country right now.

Paul Graham's tweet with zero corroborating evidence other than "believe me I'm paul graham" is not a source.

Mastodon is from a far right?

> I think it's just a dog whistle to the people who believe google have a guy in a room somewhere turning the "conservative search results" lever down a notch during elections.

Even if that's precisely true, is it not good to be creating a more trusted space for everyone? The grievances, regardless of merit, are mostly coming from the right. If you want to create a service that caters to all you're going to have to address their concerns. If he can do that in a way that is fair to all, it sounds like a win to me.

Perception is reality and right now the perception is a clear bias toward the left on Twitter. Anything that can be done to combat this fairly is a good thing.

Everyone seems to think that accept for the left on Twitter who believe there's a clear bias towards the right.

My whole point is that the task is sufficiently impossible (either technically or in terms of satisfying people) that aiming to do it feels like pure bluster. It will never be fair - you can interpret that sentence however you see fit.

> It will never be fair

But couldn't it at least be better?

And that hasn’t happened? Google got fined by the EU for manipulating search results. It definitely happens.

https://amp.usatoday.com/amp/1248099002

https://www.vox.com/2017/6/27/15878980/europe-fine-google-an...

That EU fine is a totally different issue, about defining the barrier between Google Search and other Google products.

A comparison I did from four months ago across google, ddg, yandex, and search.brave, searching for "are conservatives being silenced by big tech"

https://i.imgur.com/MVlshAT.png

You don't have to be conservative to see there's a pretty significant bias, just in the headlines. I'm a Pacific Green and I can still see it.

I think HN, as a largely technical audience, should see the clear problem with that query. Of course search engines certain results when you use terms like "big tech". It's a term that is largely used by conservative groups in criticism of the tech industry and for little else, and so the bias there is hardly surprising.

Like, say I drum up a conspiracy about how the government is putting Glupkleins in the water. Since it's not a real word, the only results that'll show up will be the conspiracy nonsense that I myself am peddling. People who "do their own research" on Glupkleins by punching it into a search engine will come away with the impression that the entire community is unified on whatever stance I want purely because nobody else knows or cares enough about it enough to write their own articles debunking it.

This is the same thing here, just at a murkier scale. Nobody uses the term big tech except in this context, and so using that term to find something which disproves the context around the term is a losing battle

Top result from google for me is: "Big Tech’s Conservative Censorship Inescapable and Irrefutable" from the Heritage foundation.

that's kind of a silly meta search query, not checking for bias in coverage of climate or race or religion or whatever.

Oh the racial bias is much easier to find, just start typing for "are white people". The suggestion list is perfectly happy with "are white p" and "are white pe", but literally vanishes as you type "peo". It does not do the same thing as you type "are black peo" (...ple more athletic, "what is a blue black person", "is wearing black bad", "how does wearing black affect you"). It's like they don't think white people are people, or don't want you talking about them as a group.

If it were objective, you'd think it would try and suggest something for all queries just to be maximally useful. But it doesn't. I mean surely people have asked questions that start with "are white people" why not show the most common one, or the most controversial, or the "highest quality" whatever that is. But no, absolutely nothing.

Someone clearly has their thumb on the scale.

>religion

Nah, this one is super obvious in the lack of an Easter doodle, but every other religion gets a shoutout on their important days. Just like the lack of one for International Men's day. Google's bias is as predictable as every other (D-CA).

Was it the algorithm or was it a "guy in a room" which/who decided to block New York Post's article on Hunter Biden's laptop scandal?

To those that are probably triggered by this and likely even uninformed, the "left" view has been that the laptop story was fake, but it turns out it was real.

Yep! But irrespective of whether the story was real/fake, the GP comment said that people "believe" that `Google have a guy in a room somewhere turning the "conservative search results" lever down a notch during elections`.

Now if you truly want to address this concern of the people, the question obviously would be if it was the algorithm that blocked the article or was it "a guy in a room"? I don't see what is the difficulty in admitting that it was "a guy in a room" because our values/ideology does not align with ideology of New York Post.

When you start to say that it was the algorithm that did it, and that there was zero human interference, without any proof to back up those assertions, then it is much the same as saying that it was a "popup" that triggered when one is caught watching porn. "I did not do it, it was the machine."

Then it becomes even more important to open source the algorithm so everyone can see what is happening internally. It at least puts some doubts to rest. It is better than saying "believe me it was the algorithm not me".

Oh, was this finally confirmed? I stopped following that story a long time ago.

What was “confirmed” was that many emails placed onto the hard drive were, in fact, real emails. (The content of those emails is fairly innocuous, neither illegal nor even particularly corrupt. But a year of continuous drumbeat spun GOP partisans into a frothy outrage about it.)

Those who investigated thoroughly found that the chain of custody of the hard drive was incredibly sloppy and the content of the hard drive was changed multiple times, making it impossible to confirm where the hard drive came from or who put which content onto it.

The speculation is that someone obtained Hunter Biden’s emails in some unrelated way (e.g. from other hacks) and then placed them onto the hard drive in question as a way to obscure the source/method.

Here’s the clearest story explaining the details: https://www.washingtonpost.com/technology/2022/03/30/hunter-...

Washington Post was one of the news outlets that denied Hunter Biden's laptop story and cast shadow on its authenticity until it had to do a major U-turn one year later. I wouldn't touch that publication with a 6-foot pole when it comes to unearthing Truth about Hunter Biden laptop case at the very least.

The Washington Post (among others) were rightly skeptical of a tabloid fluff story with no details, corroboration, or expert analysis about a mysterious harddrive with an implausible backstory somehow in the custody of Steve Bannon and Rudy Giuliani, dropping immediately before a presidential election.

Their skepticism turned out to be well founded. There’s really no story there. (This was similar to the non-story of Hillary Clinton’s email server, Russian-hacked DNC emails, etc. of 2016 which turned out to be completely anodyne and routine, but became the intense focus of months of news coverage and whipped partisans into a frothy frenzy based on wild lies/speculation.)

> There’s really no story there

That's your opinion and you are entitled to it. But it doesn't turn away from the fact that the story was suppressed. Real or not.

> This was similar to the non-story of Hillary Clinton’s email server, Russian-hacked DNC emails, etc. of 2016 which turned out to be completely anodyne and routine

At least it got proper coverage. There was no suppression of either the email gate or the Russia gate. Whether it was fake, true, hoax doesn't matter. It got the coverage it was due.

> were rightly skeptical of a tabloid fluff story with no details, corroboration, or expert analysis about a mysterious harddrive

Joe Biden said it was based on a "bunch of garbage". That it was "Russian disinformation". All news media outlets, including Washington Post, carried that forward in all their news headlines and editorial posts (except right wing news media outlets of course). What were the details, corroboration or expert analysis that they went through before labelling the Laptop as "Russian disinformation"?

> Their skepticism turned out to be well founded

I missed the part of their skepticism where they called it "Russian disinformation" and it turned out to be true. Can you highlight to me where it was proved that the Laptop was part of a smear campaign against Joe Biden initiated through "Russian disinformation"?

What Washington Post has finally done is admitted that the Laptop and its contents are real. Now when it comes to the meat of the matter, the actual contents of the Laptop, it hasn't gone through sufficient scrutiny yet. It also hasn't taken into account the whistleblower's account of what dealings the Biden family had with the Chinese and Ukrainians. On who the "Big Guy" was who received the payments; which is mentioned in a March 2017 email conversation between James Gilliar (Hunter's associate) and a Chinese energy firm where he says: "10 held by H for the big guy?". Who is the "Big Guy" here? Who is "H" here? I am assuming H is Hunter and "Big Guy" is Joe Biden. It is still an assumption and can only be proved in some Court of law in USA. But if you aren't even going to give this evidence a chance to see the light of the day you'll never be able to get to the bottom of the Truth.

Now I am not saying any of it is 100% true as it is still under scrutiny by a Grand Jury (though a lot of it is coming out to be true now). I am just saying that the media actively suppressed this information from the general public out of "fears" of it being "Russian disinformation" which still hasn't been proved. Or maybe, just maybe, they did not want their favorite candidate to lose.

The claim that it was Russian disinformation (Russian-hacked emails placed on a random hard drive along with gigabytes of other tampered-with content, then passed along to friendly partisans who had literally distributed Russian-intelligence produced disinfo multiple times in the recent past) is entirely plausible, indeed more plausible than the story about the abandoned laptop.

There is unfortunately not enough verifiable information about the sources and chain of custody of the hard drive to distinguish between these two alternatives.

Various media sources (and the American public in general) were burned multiple times by credulously repeating Bannon- and Giuliani-pitched (completely made up) bullshit, some of which actually was later demonstrated to be Russian government produced disinformation. Even if the hard drive turned out to be what they said (again, this is still dubious), it would be a good example of a “boy who cried wolf” situation.

Yep it was!

"it" is carrying a lot of weight.

What do you mean? The laptop has been proven to be hunter's, and most sources agree that the contents are real (especially the emails, which have been proven to be true). So what gives?

I have seen the videos of Hunter nude and engaging in sexual intercourse. You are telling me that what I saw is fake? The emails are legit as well. What exactly is "not true" except for your assertion?

Even the laptop has been proven to belong to him.

The onus is on you to prove it isn't Hunters laptop.

Twitter has a clear policy about posting and linking to doxxing / stolen private files / revenge porn. They never blocked based on commentary about the the laptop or the Burisma allegations, and only blocked NY Post account until they stopped trying to post the doxxing / revenge porn content.

> Twitter has a clear policy about posting and linking to doxxing

Excellent. So when is Twitter going to ban Taylor Lorenz for doxxing an anonymous Twitter user LibsofTikTok? This is her tweet right here: https://twitter.com/TaylorLorenz/status/1516399663305297920

It has been up since April 19th, 2022. Twitter immediately blocked New York Post's tweet.

I wonder why the algorithm is so slow when it comes to Taylor Lorenz? Maybe we'll find out when the code is open sourced. Then we can all legitimately blame "the algorithm" for causing all this divide between the left and right wing in a neutral, faultless, perfect digital town square called Twitter.

It's not a dog whistle, it's just a simplification because it's easier to say "the algorithm" than describing the system. Moreover, this nomenclature was mainstreamed by left-wing concern about "racist algorithms", especially in law enforcement, so it's not a partisan phrase at all.

That isn't very fair though, because its very different concern isn't it?

The racist algorithms critique isn't that their is a shadowy conspiracy of people pushing buttons behind the curtain, getting the results they want with conscious decisions, it is a concern about datasets mainly, as well as sociological questions about unconscious bias in testing and verifying the correctness of programs, which can cascade into social effects.

It is a completely different thing, and one grounded in actual research.

My point wasn’t that these theories are equal, but that “the algorithm” isn’t a right-wing dog whistle because it’s used in left-wing contexts as well.

The Epoch Times is not a credible source.

“The Epoch Times is a far-right[16] international multi-language newspaper and media company affiliated with the Falun Gong new religious movement”

https://en.m.wikipedia.org/wiki/The_Epoch_Times

How is epoch times as a news source relevant to the credibility of Professor Robert Epstein in his research area?

Got a link to some actual peer reviews of his research? I've seen him trotted out before, but at most the only data people seem to be able to produce related to his work are interviews like the one you linked, and his website (which is just request for donation after request for donation).

The article summarize his research. He had published many articles on the topic, and most have linked pdfs you can download for free in his bibliography here:

https://drrobertepstein.com/pdf/vita.pdf

A news article is not a peer review.

A link to his own website and his bibliography is not a peer review.

You said his work has been peer-reviewed, but when asked to provide such peer-reviews, you only cited a news article about him, and his self-published bibliography. Interestingly enough, his own bibliography doesn't even link to peer reviews of the Google/election-focused research.

Just his foundational study, where he discuss methodology, is cited by 470 others. Double check for “The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections” here

https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=robe...

In his bibliography you can see many follow up papers. So you are just wrong saying his methodology and approach is the work of a lone warrior.

If you are asking me to summarize his research body that’s unreasonable to show my claim that it’s peer reviewed, which it as shown above demonstrably is.

In other news, accurate information shifts voters to Democrats. Did Robert Epstein analyze how many Fox News viewers shifted to Trump? My own experience in watching relatives slowly radicalized by Fox News hints this may be a fertile ground for some peer-reviewed research. :)

Also, The EpochTimes is not a good source, it's a far-right site run by IMHO a religious movement that borders on a cult. It publishes Q-Anon theories on its front page uncritically.

And "peer-reviewed" by whom?

https://slate.com/technology/2019/08/robert-epstein-google-b...

"Epstein is a senior research psychologist at the American Institute for Behavioral Research, but other than a technologist who doesn’t appear to help with the studies, Epstein is the organization’s only staff member, according to the AIBR website. Epstein is the co-author of a study published by his employer, only the summary of which is available, that claims that biases in Google’s search results may have tilted millions of undecided voters to vote for Clinton in the 2016 election. That was the study Epstein described in his Senate committee testimony, where he didn’t mention that his huge claim is based on monitoring the search results of just 21 undecided voters out of 95 voters for a 2017 white paper. In his submitted testimony, Epstein did provide seven pages of citations—but all of them are papers or op-eds he wrote or co-wrote himself. Only one of them—a 2015 paper in the Proceedings of the National Academy of Sciences, about how biased results produced by search engines could have the ability to sway undecided voters—was peer-reviewed. Even that study didn’t demonstrate that this has actually happened."

There you go, n=21. This guy fails high school stats. Fails 30% large counts condition, fails 10% sample condition, probably isn't a random sample either.

Starting out with smearing as qanon, far right etc demonstrate that you are an adherent of repressive tolerance. This is a labeling of viewpoint opponents as beyond the pale using appeals to emotion (pathos) without evidence, instead of or in addition to engaging in appeals to reason (logos). It’s a cheap trick when used like this.

With regards to this specific paper, you say yourself the methods paper is peer reviewed. It’s also been cited 470 times according to Google scholar, which is an academic home run.

Will a methods paper apply the methodology at scale? Look at his follow up papers for that.

The point on scale is fair if true, but the paper Slate criticize is a methodology paper which is not where you would expect a large scale study. I haven’t read his body of work to get a full picture of that question, and it’s not clear to me that Slate did so either. No matter what Slate is an activist publication that at times misrepresent to support a predetermined conclusion.

I'm simply stating facts, Epoch Times is not a reliable source. And as the Slate article pointed out, his 7 pages of citations were all his own articles and only one citation was Peer Reviewed in a journal with any credibility.

Slate's criticisms were factual, they are of course biased, but so what?

This study's statistical parameters are really shoddy, and just straight up bad data. Garbage in, garbage out. His PNAS original study had a larger sample, but again, the conclusions don't really follow, and there are many more explanatory effects for how search behaves (including his own point about how searches can create band-wagoning and reinforcing behavior), and the claims that it shift votes are not established.

One only has to imagine that if more voters are searching for Clinton and clicking Ads or the first few SERP results, then the search engine might take that as a signal to return MORE of those results, which causes more people to do follow up searches for Clinton info, establishing a positive-feedback loop without any BS about tech companies putting their foot on the balance. We've in fact seen this behavior before, famously when Microsoft's Bing started returning search results for documents it couldn't possibly have crawled, simply due to click-tracking.

Plus, the sheer fact that a majority of world wide media were against Trump, Google's crawler is going to end up with a lot of pro-Clinton content. I'm not just talking about US mainstream media and newspapers, but UK, French, German, Canadian, Australian, you name it, a majority of the media did not like Trump. The search engines can only deal with the data they're crawling and the user behavior they're receiving.

I know it comes as a shock, but a majority of not just Americans, but the Western world did not have a high opinion of Trump in 2016. You don't need to invent elaborate tech conspiracies to end up with pro-Clinton or Trump-critical articles getting surfaced since that was the actual Zeitgeist.

It's amazing to me that even after winning in 2016, Trump and his cadre were still sore that Clinton won the popular vote. Well, newsflash, but the Democrats have been winning the popular vote pretty much every election. Add up all of the votes House & Senate members got, and you'll see far more Democratic votes, because the GOP generally represents lower population density districts. Again, not surprising.

The GOP could use that information to decide they need to change policies to attract different demographics, and widen their audience, but it seems like in the post-Trump era, it's down to claiming vote fraud, media bias, and doubling down on populism and weird stuff about their opponents being Groomers or CRT theorists. It's honesty sad, because the GOP used to run some campaigns on issues that had cross cultural appeal, like crime reduction, but now it's all weird culture war stuff.

> The GOP could use that information to decide they need to change policies to attract different demographics, and widen their audience

I agree the GOP fails at providing a policy alternative. That is undoubtedly true, but it’s not clear it will remain so.

That said look at how Hispanics in polls are moving heavily towards GOP. But gaining votes by just not being democrat and putting on a pretense of not supporting the woke policies is not a stable way to shift coalitions.

> weird stuff about their opponents being Groomers or CRT theorists.

More than 50% of the second biggest teacher union voted to teach CRT even in states where it’s illegal. I’ll summarize why a few popular activist teaching tools accurately fit the term grooming.

Queer theory as praxised in schools seek to teach the student to deconstruct their sexual identity and interpret it in an oppressor/oppresssed dynamic. Goal isn’t to acknowledge and stabilize a sexual identity, but to destabilize the kid sexually into constantly engaging in a critical introspection of their sexuality as a Marxist activist. Queer theory seek to destabilize in the vulnerable stage when the kid is learning that they are a sexual being. This was piloted by Lukacs in the Hungarian communist revolution.

CRT does the same as queer theory, but for race.

Critical pedagogy (SEL, the other CRT etc) that is applied across almost all schools instead of teaching skill, teach the student to interpret the words and math through their personal emotion applying the Marxist process to find the oppressor/oppressed dynamic instead of skill.

Paulo Freire that invented critical pedagogy, like all Marxists, view skills teaching as evil because it at best train people to take on roles in the meritocratic system they deem evil. In Marxist theology skilled people manifest the status quo reality through false consciousness. They believe a skilled person act like a robot based upon instructions of dead people.

Critical pedagogy results in a huge expected drop in functional literacy in high school graduates, that means they can not read words on a page and understand what the sentences mean, and the math skills are dropping in the same way. That’s not surprising when you spend school talking about how you feel about a subject instead of how to establish a shared interpretation and practicing skill.

SEL is a super evil corruption of group psychology techniques by teachers unlicensed in psychology to break kids resistance to the teachers activism.

Why should we allow Marxist teachers to indoctrinate in Marxist process instead of teaching skill?

> More than 50% of the second biggest teacher union voted to teach CRT even in states where it’s illegal.

What's really happening is the "CRT" in high schools being taught was already being taught before it got labeled CRT. Here's examples of what it boils down to: 3/5ths clause, red lining, crack vs cocaine sentencing, etc. What's controversial about this? Even conservatives admit for example, that the criminal justice system is inherently racist due to sentencing differentials.

>"Queer"..."Marxist"..."Communist"...

Do you realize how crazy this sounds? Maybe in college if you take a course in Queer Theory, you might get this, but K-12 are not teaching kids to "deconstruct sexual identity and interpret it in an oppressor/oppresssed dynamic". Do you even have kids in school? My kids are in hyper-liberal California school system and they teach ZERO of anything you're said. It's nonsense.

> teach the student to interpret the words and math through their personal emotion applying the Marxist process to find the oppressor/oppressed dynamic instead of skill.

Reality: https://www.youtube.com/watch?v=ikehX9o1JbI Actual SEL in classroom: https://www.youtube.com/watch?v=DaCVfeWKTp0

Here is the most famous teacher on TikTok that uses SEL techniques, are you really going to claim this woman is teaching a super evil marixst corruption of psych? Good lord: https://www.youtube.com/watch?v=Fc_RnQvaqoY

Where dafuq is the Marxism? They're teaching the kids self control, to regulate their emotions. I have a daughter, whenever she gets something wrong, or doesn't understand something, she gets frustrating and emotionally upset, and while she is upset, nothing else the teacher says is heard. At home I have to reassure her that failure is the process of learning, that its ok to get something wrong, or not understand it, that's a step along the way to understanding. Only after I manage to calm her, can she return to doing her work. She's too much of a self-critic/perfectionist.

What I see is a bunch of stodgy conservatives who often have no clue about what is actually going on in their schools, because they don't really spend hours a day with their kids (I spend 2hrs a day helping my kids with school every night), and freaking out because things aren't done the way they learned them, and therefore, since it is different, it is bad. I know exactly what is being taught in my kid's middle school and high school history class. No marxism, no "CRT", but yes, guess what, they do teach the factual horrors of colonialism, of slavery, Jim Crow, and the REAL causes of the civil war, not the "Lost Cause" BS.

Our high school has on campus counselors, psychologists (you can get therapy), and they specifically teach conflict resolution and empathy. So unlike my high school experience which was full of bullies and fights and cliques, the kids are remarkably decent towards one another.

And ironically, none of this is really any different than what my Catholic middle school taught me growing up. the "SEL" techniques were all over my Catholic middle school. If two people were in conflict, the nuns would talk to you about empathy for the other person and how you made them feel. They literally preached recognition of other kids feelings, with a religious overtone (because that's how Jesus would act towards others), and this was 40 years ago. They just didn't call it "SEL".

What I see is the conservative movement demonizing pretty milquetoast things, and most people just mindlessly repeating CRT CRT CRT or SEL SEL, and never having actually visited a classroom or read their kid's textbooks.

Prove me wrong: Cite a K-12 textbook that "deconstructs sexual identity", or teaches "Marxism"

Let me give you an example of NOT CRT. One of the Math textbooks banned in Florida had a statistics question about racism in a Implicit Association Test (https://www.fldoe.org/academics/standards/instructional-mate...). Are you telling me that using an actual test questionnaire as a data to construct a model on is CRT just because it's subject matter is race? it's not even a systems view (which is what CRT is), but an individual bias dataset. Explicitly NOT CRT, but LABELED CRT and banned.

What we're seeing is CRT being used as a swearword to ban any discussion of America's racist history in Southern/Conservative school districts because they don't like it. Just as "Voter ID" under the guide of securing elections (which have statistically insignificant fraudulent votes) is being used to roll back decades of progress in civil rights after the Voting Rights Act provisions expired and failed to get renewed.

It's fooling no one.

Interesting. I reckon Google wasn't causal here, but rather access to unfiltered information was. A goodly portion of conservatives/conservative-leaners I know and love are fairly gullible to Fox News and other bastions of ten-truths-with-a-lie approaches. Unfiltered information and data literacy sometimes helps to overcome that problem -- 2% - 5% of the population sounds about right for that shift to my prior.

Your mileage may vary.

Also, The Epoch Times is a far right source -- not trustworthy generally.

Yes, and people tend to downplay the human influence on algorithms. Its fashionable to pretend that algorithms don’t carry the bias of their maintainers, while “the emperor is clearly naked”.

Humans write and train algorithms, and the quality of a search engine stems from biasing the result. Human moderation is also done by people with bias.

So tech workers that tilt heavily woke, not even just democrat, are likely to both intentionally and unintentionally bias results.

Tech oligarchs of cause will pursue their own politics, but their use of repressive tolerance to censor true unfavorable stories to their politics (eg Hunter Biden laptop showing curruption) and pushing dubious stories favorable to their politics (eg russia collusion) should give them editorial liability as a publication not a section 230 company.

[0] The laptop controversy gives some false flag smell, does it not? https://en.wikipedia.org/wiki/Hunter_Biden_laptop_controvers...

[1] Russia interfered with the 2016 election (though "collusion" seems a few steps too far into "they changed people's votes", which to my understanding did not happen with any high frequency, they just advertised and propagandized illegally) https://en.wikipedia.org/wiki/Russian_interference_in_the_20...

People at tech companies have political persuasions across the board. It's naive to think the companies are any sort of concerted bloc of voting power, or that the company leadership is the new room of smoking men around a conference table deciding elections.

Political party leadership, on the otherhand....

We are in agreement that politics is kayfabe, it’s like wrestling the moves of a fight playing out a scripted story to manufacture consent around what they already wanted to do, and it’s a uniparty.

The laptop is real as verified by the FBI that has opened a grand jury on Hunter Biden due to this and other evidence.

Likewise, FBI verified that the Hailey Biden diary was real in its illegal raid of Project Veritas.

Despite the hoax Mueller found nothing despite spending immense resources. With regards to the Russia collusion hoax, follow the Durham indictments around it that show it was a Hillary political ploy all along. Example indictments is Fusion GPS fabricating the Steele report for Hillary, a tech executive thar tried to plant evidence on Trump computers, a Hillary lawyer that lied to the FBI about not acting on behalf of Hillary etc

How does one “tilt heavily woke”?

By it being common to do some activism as part of work activities, such as activism for the woke DEI doctrine. Also, by people in the valley commonly cutting off friends and acquaintances if they speak critically of woke activist measures (or fail to post the required flags-of-day on social media). 80%+ of political donations are democrat.

Wokism, like all forms of Marxism, demand praxis (“do the work”) from its adherents which is why it’s so visible and progressively take over organizations until all they do is praxis.

Side note: Activism through praxis is not optional, because Marxist theology has this idea that their adherents have a special gnostic knowledge where they like a God create reality. The idea is that once all alive are Marxist utopia will be spontaneously created by the collective.

Why?

Ok cool

> turning the "conservative search results" lever

It may not have been algorithmic, but it definitely happened.

You know this how? Because you were that guy?

A quick Google (ironically) confirms this has happened, beyond the anecdotal evidence of people (including me) who remember it on several occasions.

https://www.mediaite.com/news/ex-google-engineer-says-glitch...

I'm not making any comment on the theory in the article, but it has certainly happened.

Edit: more context in thread here:

https://news.ycombinator.com/item?id=31161790

Mellosouls posted a link to a toxic source, and the article doesn’t even have anything in it except empty speculation based on nonsense. Brutal. Either the psyops have reached HN or the level of intellectual honesty about these topics is trash. Or both.

I lifted the source indirectly from Google's own results (it was linked to from the Mail online), I have no idea of the general toxicity or otherwise of the website itself.

Update: here's a Twitter thread linked from Google's Tweet denying any bias in the incident. Make of it what you will.

https://twitter.com/CharlieNash/status/1285612797028700160

Source needed, plenty of liberal disinformation gets into search results as well and all kind of spammy SEO bullshit. Google search results have been getting shittier across the board is my intuition on this, censorship is less relevant.

IMO they just stopped caring about accuracy along time ago and have shifted into "engagement" and recommendation based results instead of accurate ones. Now when you search for something and very few results come back or they think your search terms should be something else, your given a shitty recommendation instead of what you actually searched for.

> the concept of publishing or open sourcing "the algorithm" doesn't make sense

Whatever. He paid for it. Private company. Do what it wants.

> it would be very difficult to make sense of the entire system

No. Not buying that. Difficult isn't the same as impossible and, if only to game the system (harder,) people will figure it out. And even if it isn't 100% possible to reproduce the results based on what is released significant insights will still emerge.

Further, there is some ceiling on the complexity. Twitter operates at scale and that means they can't actually burn 52kWh of power for every tweet or store TiBs of metadata for every user to do the analysis or take 30 minutes to publish. Likely it's a pretty efficient system and, therefore, limited in complexity.