This is my big fear about pervasive use of AI. I'm afraid that companies, policy makers, regulators, etc. all start letting AI make important decisions without any human understanding of the reasoning behind the decision and the human puppets hiding behind it with no accountability.

I imagine scenarios where AI could be given complete authority to decide who is hired/fired, who gets medical care, who gets food, who gets utilities (water/electricity/natural gas) to their homes, who gets disaster relief, etc. Quite frightening when you think about it. If AI decided to cancel you (and it had this level of authority) your very existence would be in danger.

Haven’t books been written about this exact scenario?

I have no idea, but it wouldn't surprise me if there were.

I think the scale of applying AI to generate contents will cripple search engine even before your scenarios become true.

I have a hobby with natural photos, and in the last 2 years, I have stopped spending time browsing pages as most of them are garbages generated by AI.

One thing many people hype about generative AI as they believe human also make mistake the same way as AI do, so at worst you have something similar to human mistake. Yes, but the volumes generated by AI are at magnitude bigger, and only a limited amount of human can validate and filter it out. If there is nothing change, this volumes of garbages will surely overflow and there is no way we can differentiate bad and good, fake and authentic contents.

Sounds about right. Let's throw more money, power and natural resources at it and see if the scale tips the other way! If not well nothing of value will have been lost right? What's shortening humanity's lifetime in comparison to the potential productivity gains!

Aren't you at least a little bit excited for the potential benefits?

My spouse and I both work on different ends of AI work (spouse, content production, editing, and prompt-library building; me, feeding the things with data).

No, not really. They're useful but not revolutionary for actual productive work, and I'm being generous in dubbing some AI-based products themselves "productive" (I eagerly await the studies that I'm sure the companies building these things will not bother to do, proving that these are cost/benefit better than other approaches they're replacing—I definitely don't consider it certain that they are).

They shine when the fewest shits are given, which is mostly work that didn't need to be done in the first place, and... mass scams/astroturfing/spam. Hooray.

I don't really see this balance tipping much with the general approach the field's pursuing now.

I am not. If AI delivers on its promises, an unprecedented dumbing down of humanity is to follow. If it doesn't, it was all a waste. Even if we win, we lose.

Is there anything AI (or more like AGI) could do for you that would change your opinion? Eternal life, interstellar travel, curing diseases, educating people with superhuman patience and competence would not be worth it?

I’m excited about AI technologies that supplement human capability, like driving autonomy and document search.

I’m not excited about AI technologies that replace human capabilities, like pretty much everything that seems to be getting investment dollars these days.

AI-generated art aside (which is mostly terrible), I’m already having a tough time telling what’s real and what’s not online. The thought of this bleeding into daily life in big ways is depressing.

I’m also not thrilled about a generation of people that struggle to write a paragraph of critical or original thought because of AI dependency.

The last act of Up was supposed to be a warning, not an end-goal.

Online comments are about to die. First, we won't be able to tell who is real anymore. People will profit by running huge botnets for advertising and political manipulation. (Inb4 someone says this already happens: AI will keep lowering the bar to enter and evade detection). But a knock on effect is there will be a market for farming fake accounts for later sale to said manipulators.

> But a knock on effect is there will be a market for farming fake accounts for later sale to said manipulators.

We'll lift millions in the "global south" out of poverty by providing the tools to criminals and foreign adversaries that drive demand for cheaply-staffed high-rep social media account farms.

What a time to be alive. What frontiers we are exploring.

It'll be sad but the brain rot from pervasive social media is getting severe, especially in older generations. If this pushes more people to touch grass, it's a net positive.

I totally agree that many online comments are toxic and many people should go touch grass. However I also see this as the next step in a series of humans getting their communications cut off by computers.

Talking to people online is in fact a way that some people get socialization and develop writing skills. If my prognostication is correct and people give up even trying to interact on most websites because "it's not worth starting a conversation only to find out 10 minutes later that they're a bot" or "no one listens to my opinion because that's what all the bots say so they don't think I'm real", the effect is to make the web even more "read-only" where people are discouraged from sharing themselves.

Whether the "sharing themselves" age was a peak or a low is still up for debate.

There's something very "late stage capitalism" about pouring torrents of capital into tools that can replace human ingenuity, artistry and creativity while starving stuff like infrastructure, space travel, etc.

The only times I've use copilot is when I want to execute my creative goals more easily rather than waste mental energy on the boring parts. It's weird and sad that you see it the other way around

Capitalism is about profit. Infrastructure is financed when it brings profit, and ignored when it stops doing that. E.g. railroad boom and the current state of railroads.

When someone figured that space can bring profit too, we got some developments in space travel as well.

That’s just because people have been so indoctrinated by capitalism that they only find joy if they’re being worked to the bone. They have forgotten the joy of not doing anything productive.

Why learn to paint or draw if generative ML can create a picture for you?

Why practice writing if generative ML can create a poem or short story for you?

In that regard you're best off just sitting down in front of a screen and consuming content generated by ML.

And from the example in the original blog post the author had the generative ML do the fun stuff in solving the problem and all they did was drudge work cleaning it up and submitting it. Very productive from the company perspective but reminds me a lot of low thought factory processes.

Many many people still play violin, piano, trumpet, guitar even though recordings (in various forms) have been around for more than 100 years.

You didn’t answer your questions.

To me the joy of not doing anything productive means just sitting/laying on the couch consuming content in one form or another. I'm sure it can also mean just sitting and enjoying a coffee, which is fine but I don't find that necessarily enjoyable by itself.

Making art, music, painting, are all creative and productive endevours, so when you're saying to be okay not being productive to me it means to be okay being a couch vegetable. We need the occasional rest but I don't want generative ML to do all the creative and rewarding things.

Consuming content is really just another capitalist activity, you’re producing views, impressions, you’re trading away attention in exchange for some entertainment.

Instead, go outside and consume nature. Consume a sunset, not a TV show.

Strongly disagree. There is an intrinsic pleasure for many people in feeling that they can make a contribution - that their skills, knowledge and ability is valuable and valued

But their contributions are not valued. You can spend a year working on something cool and when you finally show it to the world the most you get is a couple upvotes on Hackernews and people using it for maybe 10-15 minutes and then never again. That’s the reality for many people.

Exactly, that exact drive to feel helpful and needed is why dogs are trainable and men are depressed. It's why the old wolf who can't catch their own food anymore leaves the pack to die alone, cold, and hungry.

That craving is a deeper more primalistic emotion than hunger or thirst for many people (and animals) - boiling it down to "muh capitalism" is just disingenuous.

There are many people who can afford to do nothing. They rarely do, and those who do, are not necessarily happy.

Those who do nothing tend to die early/young. Those who get a hobby of some sort tend to live a normal lifespan.

I know full well the joy of being unproductive because capitalism affords me the ability to do it. It's expensive and requires an amount of agency only afforded by a high income or outside wealth.

Do you really think the authoritarian elites will let the unwashed masses with no income do whatever the hell they want? Can you really say that after COVID lockdowns demonstrated their true colors?

None of the other capitalism-alternatives have historically afforded the kind of luxury you're suggesting either. Quite the opposite, in fact.

Boohoo. Stop complaining and don't let it. Why do I feel like one of the few people who knows they will win against AI? Is your self worth really that low that you think an LLM can beat you?

A lot of people don't/didn't set out in life to be a hyper-adaptiver worker who puts 110% into the latest thing they can be the best at all the time. They set out to be an average worker who learns a skill and lives a life focused on things outside of being the maximal worker. For these people it's even less about if an LLM can beat them (which, to be honest, for most all of us is or will be true at least for certain tasks) and more about how much an LLM may upset the life path they were in.

> A lot of people don't/didn't set out in life to be a hyper-adaptiver worker who puts 110% into the latest thing they can be the best at all the time

This used to be life for 99% of human history. Don't take the past 30 years lifestyle for granted just because we created government, free market or whatever. Human nature is to be a hyper-adaptiver worker who puts 110% into the latest thing they can be the best at all the time otherwise they die.

Humans never needed to be hyper-adaptive. 80+% of people in the far past were farmers their whole life, just like their parents, and their children, and everyone they associated with. Prior to farming, dumber means of food gathering. You worked your ass off or died but it was never about the latest thing, that's a purely modern invention. Hardly anybody went to "school" or learned a trade, nobody expected job changes every 5 years. You were a dirt poor, uneducated, worn down farmer. The modern invention was the idea we'd have time to be something else and the very recent invention is that you'd need to be multiple something elses over time.

any worn down farmer you are describing as having it easier would change lives with the modern human. They would not complain as much, that's for sure. At least not until they get used to it.

this is absolutely not true. Human history is not the story of hyper adaptive silicon valley disruptors, in fact your completely backwards. Most of human history growth was enormously slow, and only in the past few decades has it really skyrocketed to the point where people become obsolete this quickly. If you were born in 1008 you could be a blacksmith for the next 600 years at least without too much issue.

its not about whether an LLM can do better work than you, its about if it can do cheaper work than you, and it can.

It’ll do both soon.

Your last sentence crosses the line into personal attack.