Which means that (in turn) you can’t just “log off” and be rid of the problem since it predates... logging off.

Neo thought he could just “log off” in The Matrix. But then then it turned out that the illusions ran deeper than first thought.

[1] Filter Bubble quacks et al

"Everywhere the world movement seems to be in the direction of centralised economies which can be made to ‘work’ in an economic sense but which are not democratically organised and which tend to establish a caste system. With this go the horrors of emotional nationalism and a tendency to disbelieve in the existence of objective truth because all the facts have to fit in with the words and prophecies of some infallible fuhrer. Already history has in a sense ceased to exist, ie. there is no such thing as a history of our own times which could be universally accepted, and the exact sciences are endangered as soon as military necessity ceases to keep people up to the mark. Hitler can say that the Jews started the war, and if he survives that will become official history. He can’t say that two and two are five, because for the purposes of, say, ballistics they have to make four. But if the sort of world that I am afraid of arrives, a world of two or three great superstates which are unable to conquer one another, two and two could become five if the fuhrer wished it. That, so far as I can see, is the direction in which we are actually moving, though, of course, the process is reversible."

But these issues have become internalized, not merely something that the Ministry of Truth or whatever says. I think the introduction to Eclipse of Reason explains it better (published 1947).

In general I think “Truth” died due to Hyperreality. (If I could read French. Or understand Postmodernism.) Orwell’s vision has become too simplistic. I don’t really understand why his memes are so popular. Although perhaps on a meta-level it could be understood.

[1] The propaganda in Animal Farm was mostly delivered through the henchmen who personally berated the animals and gaslit them. It’s harder to resist even that.

If you alone feel you also need to duplicate their efforts, you may just have control issues, but if the need is genuine and shared by many then you need to rally the people to get the problem solved. Paying an employee to do nothing while you pick up the slack is a plain waste of resources.

Ideally, the press would actually report on what is happening, but the incentives just aren't there for thorough and accurate reporting. You either get highly partisan press that lies or just omits important parts of what is happening because the ones paying the bills want them to, or you get an anemic press that can't do more than trawl FB or, worse, official declarations for info.

Although that is not nearly as silly as hiring an employee and then doing their job for them. There is no point in hiring anyone if you want to do the job yourself. You go through the employee election process because you recognize that it is more efficient for one person to do the work on your, and your neighbour's, behalf. You are choosing to hand off the work to someone else precisely so you do not have to do it.

2. If you cannot establish enough trust with the employee you are choosing to hire, why are you selecting anyone? If you are going to do the work yourself, why not do the work yourself?

3. As before, there is nothing special about cities and countries when as compared to any other organization. If you have a widget factory, how do you ensure that your hired engineers are designing widgets that meet your expectations and requirements? Do you simply push them aside and design the widgets yourself in case what they designed wasn't up to snuff?

2. Managers typically have a handful of direct reports that they can check and trust. Also, managers can fire their employees on the spot, reduce their pay etc. You have nowhere near that level of control over your representatives, and you have many more representatives than any normal manager has direct reports.

3. While I trust my direct reports and employees, I very much occasionally verify that their work is being done. I don't need to design a widget myself: I can find out if the widget works or not by checking how well it holds up, what the QA department finds and so on.

Then have your worker collect video evidence and information from multiple sources to have on hand when you need greater assurances. You're the boss. They're simply there to relieve you from having to do the job yourself. It is reasonable to expect them to do the job like you would.

> you have many more representatives than any normal manager has direct reports.

Government systems can differ, but typically you hire one employee for any one government that you oversee. It is likely that you are in charge of a few governments (e.g. federal, state, municipal), but unlikely more than a small handful. It would be quite unusual for a manager to have fewer direct reports in an organization of any meaningful size.

> You have nowhere near that level of control over your representatives

You most certainly do as long as you have the rest of the board on your side. Finding consensus with the board may be a challenge, but that's true of any organization. The sitting board at the widget factory isn't bowing to your every wish either.

> While I trust my direct reports and employees, I very much occasionally verify that their work is being done.

So why not do the same with the employee you hire to watch over political matters?

Plus, while it's true that in some government systems you only vote for a specific person, in many others you vote for a political party, and they allocate people as needed and according to the total votes they got. I think this system is far more common than the other one, and it is the system I live in (for parliament and town councils - it's true that for presidential and mayoral elections you vote for a single person).

Finally, having "a board" of some 2 million or 20 million or 200 million people is entirely different from having a board of 10. And then, social media is exactly one of the ways I can collaborate with my other "board members" at this scale.

Already if I'm a board member and not a manager, then much of your metaphors die away: active board members absolutely do more than read reports from their CEO, they read newspapers and watch analysts talk about their business. They don't rely entirely on the CEO for info on how the business is doing. And I have yet to see a company led by 200 co-CEOs, each chosen by a different board - and yet this is how must countries operate. It's almost like democratic countries are a totally different kind of organization than autocratic companies, and require totally different means of interaction.

In addition, CEOs are legally liable if they lie to their boards about the state of the business. Politicians are very much entirely protected for any and all things they say to the public about their activity. Your mayor could publicly tell you that they have paved your own driveway with gold and that they are increasing taxes to pay for the maintenance of that gold that doesn't exist, and it would be completely legal for them to do so.

This year, my town had a referendum on whether to raise taxes to fund a public pool to replace a 70-year-old one that is falling apart. The public conversation was only focused on actual pros and cons of that particular project rather briefly, quickly devolving first into a general fight over development vs lack thereof, and from there to national economic policy, crime etc. And I have a distinct feeling that most people who actually voted ended up being swayed (one way or the other) by those arguments - the partisan alignment of support & opposition at the end was very blatant.

How would you feel if you were now getting stressed at a chatbot?

Instead I’m baking cookies for friends and family today.

The real goal is to get people to give up on finding the truth. At that point, they become easy to manipulate (because there is no outside standard of facts to work around).

While far from perfect, I'm not sure a deluge of generated AI nonsense outside of the vetted channels, complementing people already stirring up nonsense, changes anything.

Rather strained definition of truth there. The truth that matters is a basic not-very-wrong version of what is happening.

Examples: 1) The earth is roughly spheroidal. 2) NASA is a real space agency and landed people on the moon in 1969-1973. 3) The trails behind airplanes are condensation particles, not sprayed chemicals. 4) Vaccines are safe and effective, and the ratio of issues to prevented disease is on the order of millions-to-one, a vast benefit to society, especially those who medically cannot get vaccines. 5) Hijacked airlines and not pre-set explosives brought town the WTC towers in NYC on 9/11/2001. Etc. Etc. Etc....

We do NOT need a doctorate in Geology, selection into NASA Astronaut Corps, graduate astronomic physics course, PhD-level immunology, or a structural engineer's license to understand any of the above to a useful level.

Yet, there is sufficient human-generated noise to convince millions that none of these are true.

And you are claiming that adding industrial-scale AI agents to the public 'conversation' at a scale never seen in history will have zero effect?

Sure, it might completely fizzle. But the normal and prudent expectation when something comes in and "floods the zone with shit" [0], especially when that shit has the potential to be at a volume that is far beyond anything experienced, is to expect that it will have some effect.

What evidence do you have that it will have no effect? In answering, consider another top HN story at the moment "Facebook Is Being Overrun with Stolen, AI-Gen Images That People Think Are Real". [1]

[0] Steve Bananon's description of a disinformation tactic

[1] https://news.ycombinator.com/item?id=38684309

The standard model for electrical current is not true, but it is still useful in the vast, vast, vast majority of cases. True and useful are very different things. The conflation here is erroneous.

The average person is never going to understand what is true in any meaningful way, and even those who focus on certain truths are going to be focused on very specific truths, not all truths. There is only so much time in the day. For everything else they have to work with simplified models that may be quite disconnected from reality, but as long as they work for all practical purposes, who cares?

> And you are claiming that adding industrial-scale AI agents to the public 'conversation' at a scale never seen in history will have zero effect?

Conversation only happens when people realize they are working with a faulty model, so if conversation keeps happening then that means people are realizing that the model they currently have doesn't work. What do you think is going to happen with more noise? If people can't find a usable model in a not-so-noisy environment, do you think they will suddenly be able to find one in a noisier environment? That seems quite unlikely. In reality, the outcome is the same either way.

Maybe increased noise will turn more interest to the vetted environments and ignoring the noisy conversations, but if people are already not finding the vetted, the reality is that we are not likely to see change there either.

Agree, but this means we are talking about two different topics. I'm talking at a functional level of truth; a base level of truths commonly understood in a society such that it can maintain cohesion. You're talking about deeply understood truths, like the 2+-hour discussion of quantum effects on the question of "why is the sky blue?" at a PhD's defense presentation.

>> Conversation only happens when people realize they are working with a faulty model...

what about when people actually have a good model, and a disinformation stream replaces that with a bad model, or causes them to just say 'nothing is true'?

For your purposes of deep truth, sure, everyone knows almost nothing except maybe in their own tight discipline, so one noise level or another will make no difference.

But at the level I'm talking about, common understanding of basic truths (the sky is blue, scattering, not a clue abt quantum effects), we are already seeing how increasing the noise level only by removing the Fairness Doctrine in broadcast media — society is significantly more fragmented. Automating and scaling those drivers of untruths is likely to accelerate the fragmentation.

Do they actually have a good model? Or a flawed model with good intentions?

Global warming stands out as a prime example where many accepted the general idea in theory back when it was a novel concept, but under a flawed model, and when the flawed model didn't stand up to practical use the conversations began... There was an attempt to rebrand the idea as climate change to try and reset the clock, so to speak, but in many ways that only confused the issue further.

Separating truth from lies is something every one of us as a human have to do, and that we can't outsource with complete confidence to anybody else. Credentials are one of the last factors that should be weighed when determining truth vs lies.

What will probably (and already is starting to) happen is that people will revert to deriving truth through tribalism rather than evidence.

On a more serious note haven't societies long been based upon common brainwashing? The Bronze Age and before are fuzzier on the details but propaganda certainly existed then as well at a smaller scale.

Technology advancements raise the danger potential almost exponentially when compared to late history, or even recent history.

It doesn't take much to tip the course of history. The bronze age didn't require the same type of risk assessment that we have the burden of managing.