Reminds me of the parrot https://www.sfsfss.com/stories2/BLIT.htm

As others on Telegram have said: automated search for visual superstimuli likely leads to bad outcomes.

https://en.wikipedia.org/wiki/Supernormal_stimulus

https://en.wikipedia.org/wiki/BLIT_(short_story)

Also: one of the V3A animations reminds me loosely of things I saw when I was a kid, at night, shortly before I slept (though my experience then was more circular).

Breaking news: TikTok is bad for you

On telegram?

Isn't there that one Harry Potter warning. I think it was the potion guy who said too much luck is dangerous. I guess that is somewhat of a parallel to this. Too much positive visual stimuli is dangerous or bad.

Others on Telegram? Some sort of a HN channel?

How fried does someone's brain have to be to say "others on telegram", as if everyone is in the same chatrooms together? If they're on HN, they should at least know why that doesn't make sense, so I have to assume they're in a deteriorated mental state.

Friendly reminder to touch grass on a regular basis... Even virtual grass in a video game is a good pressure valve for excessive social media use.

Or as SCP calls them, cognitohazards.

Also relevant: <https://www.quantamagazine.org/a-math-theory-for-why-people-...>

My understanding is that those who work with the mentally handicapped use bright lights and other stimuli to soothe and control them. It is also my understanding that the autistic are stimulated by vibrant colors (coughcoughMy Little Ponycoughcough).

Who is to say that the rest of us are not also vulnerable to such controlling stimuli?

Bright lights in particular, I'm thinking: yes, normal people do find sunbathing relaxing.

There are undoubtedly 'adversarial images' that will induce certain effects in particular biological minds. It seems like this is neither good or bad inherently, just another tool.

So are opioids.

Or how a very useful tool can become a public health catastrophe.

> one of the V3A animations reminds me loosely of things I saw when I was a kid, at night, shortly before I slept

Hypnagogic hallucinations: https://en.wikipedia.org/wiki/Hypnagogia

I can't wait until I see AI-generated gambling ads that are specifically created to stimulate my brain the most

My brain never liked vertical video, shortform content and AI slop.

Is my brain different or am I just a grumpy millenial hipster?

My current theory is that these are similar to cigarettes. Nobody likes the first draft, it burns your lungs, your entire body wants to reject it. But the nicotine stimulates just the right receptors so that if you keep at it for just long enough, you'll be hooked and start disregarding the terrible taste, smell, tar in your lungs, and yellowing of your teeth.

All of this to say, if you subjected yourself to just enough TikTok scrolling on just the right topic, you might find yourself using it occasionally after that initial hump, then slightly more frequently, then daily.

You might still not "like" it, but the habit is what matters.

I have the opposite theory. I burned my self out on cheap and bad image gen meme sites like 15 years ago until the point I hated memes.

Prior exposure to worse feeds gives like an analytical look on the vids rather than emotional. I am fast scanning for the joke. Or something.

As a teenager, I tried to get addicted to cigarettes so I could stand by the cool kids and smoke with them. I started smoking 4-6 cigarettes a day but hated it so much, I couldn't continue after a week or two...

I guess there's about a billion smokers who would envy you for that.

You're not alone!

I hope my brain is also different. I also have never spent hours scrolling through short-form videos on Instagram, TikTok, Facebok, etc. I never ever walk outside with my phone in my hand, instead enjoying the view.

I do enjoy watching YouTube videos at home, on the living-room flatscreen, on a variety of topics, but I select them manually, one at a time, from the vast selection The Algorithm(TM) offers me, plus my own searches.

That's fascinating. I wish the demo videos were longer.

Am I the only one who is avoiding even clicking the link just in case?

It is like screen saver moving patterns or corridors with a strange field of view zooming effect.

Nothing special compared to purpose made screen savers.

That’s what someone would say if they were brainwashed by them!!!

Half joking, half paranoid.

Really, I don't think you need to worry. A prevailing latent attitude in the comments is that the brain is somehow super malleable and subject to external control through some incantations. Like subliminal messaging, it will at most be moral panic that fades, and then people will act like they never believed in it.

I'd argue that the guys in the chair 'optimizing' those vids might be the sleeper agemt you need to look out for. It is probably somewhat personal?

No, you are not alone.

All glory to the Hypnotoad!

Straight out of Echopraxia.

Will be interesting to see how strong the controlling forces can be - enough to make you miss things in direct perception like in the book, or only softer effects further up the cognition layer stack

We are really getting to the point where the tech industry must be stopped if humanity is to continue at all, let alone thrive.

This does indeed seem comically evil. While surely this may provide somewhat interesting insights in how our brain processes things, this seems squarely past the "should" part of "you scientists were so obsessed with whether you could you failed to consider whether you should"

> This does indeed seem comically evil.

And I have yet to see a single paper like this where a researcher bails out and publicly says they refuse to work on such projects. Not one.

The most benign interpretation of this observation is that science is filled with spineless opportunists who don’t care who they hurt with what they create. A slightly less benign interpretation might be that many of these people are doing this deliberately, and getting off on the sense of power it gives them.

I was asked to train a neural network do detect how much pain a mouse was in- our partner company would be responsible for hurting and filming the mice. I refused and subsequently quit- this produced no paper and I don’t know if they got someone else to do it. I probably should have done something stronger.

When skip level bosses on my last job wanted to do boneheaded things in automotive design they usually had to keep asking different engineers until they got a yay.

When it is pushed from the top it is hard to stop at ground level.

In their defence, don't shoot the messenger. Just because they published it doesn't mean that others haven't already discovered it. Better to know its possible than be completely ignorant.

Then maybe publish the results, but don't publish the "how to".

> Just because they published it doesn't mean that others haven't already discovered it.

That’s irrelevant for the moral evaluation. Everyone is responsible for their own actions. If you choose not to kill anyone it won’t stop others from killing. But the fact that this is so doesn’t give you a license to kill.

Would it be better if this was done on monkeys? Because people did that before this in silico digital brain stuff.

We don't hear from researchers who just decide not to work on something without making a fuss.

These aren't scientists. They are techbros. That's why it comes out like this.

I'd say we're already well past that point. Short-form "content" already exists and is messing with people's brains, this is the same thing just taken a few steps further. By the time the tech companies start using it, it will already be too late and we'll be left discussing whether the next man-made nightmare they come up with is the point where the tech industry must be stopped.

I share your concern but the generalization is improper (that as solution would be infinitely far worst than the problem)

I get that people see this and think: ads and social media. My first thought was cognitive neurorehabilitation and brain stimulation.

Realistically, probably ads, but maybe not only that?

(AI start-up idea: one of our ads a day keeps dementia away! /s)

You can't say things like this on this website. On here, every new tech thing is a "progress" /s

Progress, but towards what?

Ads efficacy, of course.

Advertisements actually run the world. Imagine a world without ads. The economy would grind to a halt. It is exploitation and manipulation, but that enables creation of large capital, which leads to great things. In the past it was pyramids and temples, and in modern times it is space exploration, scientific research and other things that require huge capital. Even the current advances in AI/LLMs are made possible by mass exploitation.

Without ads and exploitation of the masses, none of these would not be possible.

Even if that is true, what you are saying is, essentially, that the ads-driven exploitative economy is a very expensive innovation engine.

It is then useful to ask: if innovation is what we want, do there exist engines that aren't quite as expensive?

If the market economy has more of a human face than outright slavery, there's no reason to believe that it's impossible to do better: just like there would have been no reason for an ancient Egyptian to believe it's impossible to do better than state slavery.

> do better than state slavery.

The point is the exploitation of the masses and the ensuing concentration of wealth is the enabler. Sure we might find a way to do that without Ads...

Please tell me how ads help science research or why should I care about space exploration or LLMs that threaten me and hundreds of thousands of other peoples jobs?

Can’t tell if this is satirical or serious. Those ancient wonders were enabled by slave labor, treating people as “capital” is the ultimate climax of capitalism.

Not sure what you find funny about it.

>Those ancient wonders were enabled by slave labor

That is exactly the point.

Sorry, should have said parody. Don’t find it funny. I guess Poe’s law applies.

> Those ancient wonders were enabled by slave labor

This is not true of the Pyramids

https://www.sciencefocus.com/science/were-the-egyptian-pyram...

This reads like the mental hoops someone who works for or invests into Meta comes up with to resolve their cognitive dissonance.

You might as well have said that without advertising, we wouldn't have charities, freedom of thought, and literature: the argument is as strong and might make you feel even better about yourself.

Oh, turns out that you did say that advertising solved slavery in another comment. Carry on, mate.

I hate ads of all kinds and I hate targeted manipulations of all kinds.

But that does not stop me from recognizing the part it plays in the world. That shouldn't stop you either.

> we wouldn't have charities, freedom of thought, and literature:

Does great literature require huge capital?

> which leads to great things

Slavery (or getting poorer people to do the work for you) is the central concept of civilization. It cannot be done without it. It's not capital or advertising that makes all this work, it's forcing the poor to work and pay tax that makes the world go round d

> pyramids

The pyramids were not built by slaves, and while we can waffle around about "all labor is exploitative to some extent", it doesn't take exploitation-maxxing to drive great achievements. Most great modern achievements are driven by a desire for self-actualization or recognition, not survival.

https://www.sciencefocus.com/science/were-the-egyptian-pyram...

Think of the shareholders and Capital. Money matters more than human, commie. /s

The problem is the incentive structure. We could be doing good things with tech. The incentive is to use it to addict and destroy. Addiction and war is where all the money is, not education or health care or positive forms of entertainment and art. No money in those. Why?

I’m sure there was an early hominid version of this discourse. “Maybe bad to make sharp rock and sharp stick if this what we do with it…” “Mmm yes someday we make sharp rock big enough smash world.”

One might even come to the conclusion that the Industrial Revolution and its consequences have been a disaster for the human race.

Wait, this is with a digital twin only? Not fMRI or webcam based?

"Prime Intellect, I would like you to begin stimulating the neurons of the pleasure center of my brain, one at a time, and remember the ones I report to you as being favorable."

I wonder what Meta could do with a similar technology…

But here we can start also the usual discussion about technology research for the sake of it vs calibration of possible side effects of new research

Personally i think we haven’t solve this problem and thus it’s just a matter of time until we’ll get in a non-going-back point

these videos were disappointing and underwhelming

If you read the techbique it reads like something far less remarkable being PR’d to sound like a big deal.

The fact it’s bucketing by making images of lighting and facial expressions, the fact it doesn’t natively do the video it does an image then video generates from it.

The results look really bad and samey. Doubt this would work for the actual thing they’re pitching it for.

This is already happening at scale by the social media feed algorithms. We don't need generated content to accomplish this. In a sea of user created content, plenty of it is already at peak activation.

The plan is to get content producers out of the loop to reduce revenue share and boost profits.

Optimizing for engagement is a categorically different goal than optimizing for lighting up a specific part of someone's brain.

There's algorithmic ways to accomplish the same thing without requiring monitoring of the brain, if there's enough data to pull from. Semantic analysis + engagement monitoring is more than enough. Peoples emotions expressed via comments is a reflection of brain region activation.

The end result of this will be that creating viral or maximally converting content will not be restricted to the top content creators. It will also result in down regulation of those brain regions due to constant stimulation, so the bar will increase for what classifies as attention grabbing and the brain will filter most of it out as noise.

All in all, I don't really see this being a big difference to the outcomes we're already seeing.

WOW! Cant wait to tell to future generations that we had voluntarily made these algorithms to manipulate and influence our own brains

How can one work on that and not consider that they're pure evil?

For one, it's bound to teach us something about our brains. And while I'm not usually an optimist, I actually find it more likely that we'll take active steps to stop the evil applications of this if research like this is out in the open.

Think of the shareholders (it's time to start using physical force to stop people)

To flip the question on it's head, what exactly do you see as the "evil" potential? The most prominent comments to this effect are just scifi handwaving.

What are these videos supposed to do? I watched few of them and it does nothing for me?

if it is targetting visual regions of brain and I have aphantasia (I cannot visualize anything in my mind) is that connected?

> I watched few of them and it does nothing for me?

It's so cool that you have an MRI handy to check such things.

Apart from ethically bad and evil use cases of this application, can we use it to massage the parts of brain like we do it to our bones and muscles with the help of physiotherapists?

reason I am asking it could be some relief to our brains after tedious working day, especially after heavy AI usage

Its also an interesting way to discover what that part of the brain is for, right.

what the brain needs is Default Mode Network, not more stimuli

Sure. The relief is actually free ; though once every 10 uses we'll spike your dopamine-related brain centers to 1,100% activation for a quick second while flashing a peptide sponsor ad, then back to normal.

You can of course, also not get the spike ads, by subscribing to Premium. Just have a quick look at this animated QR code, it will explain better than I can in writing.

Edit: there used to be an OSS app you could use where you actually pick what you want activated in your brain and when, but it's been banned after one of the commercial producers's investors convinced the government that this technology was too powerful, so only government-approved apps are allowed now, and an emergency vote passed the BrainControl law authorizing logging of all visual stimuli you look at, in order to protect your children.

What in the zuck is this?

I wonder if the end-game of this field of research will be to run these simulations at scale using neuron-on-a-chip services such as [0] Cortical Cloud.

I don't think it's a matter of if but when. Grim.

[0] https://corticallabs.com/cloud

Very interesting. We have an organic experiment converging to maximum stimulation in short form videos (which will become the majority of training data for future video gen models) Already approaching the capabilities of a “mood organ” from blade runner. Except usually most people don’t even make the choice to change their mood anymore. https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_...

This is very similar to last week with that mind reading startup thing. Please read the paper before commenting.

This is a tool to help researchers in figuring out what different parts of the brain are actually for with less experimenter bias contamination of “well we think maybe it’s about this so let’s show it video of x to see”.

The essence runs on having someone sit in a scanner for a couple hours watching all sorts of things, and then feeding that to a model that will then build its own representation of said data and try different things on it until it’s found what makes a certain part sing in the model.

The purpose is a generalized understanding of brain function, more or less the same way we’ve been doing it all these years. Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.

What is a "purpose"? Something people wish something would only be used for, right? How does it relate to, what influence does it have on what something will end up being used for?

> The synthesized clips line up with what each region is known to care about, faces for FFA, places for PPA, bodies for EBA, motion for MT, patterns for V1 / V3A, and lively social scenes for pSTS / aSTS

(STS seems to be https://en.wikipedia.org/wiki/Superior_temporal_sulcus in the temporal lobe, so I guess it's the "I sense a presence" region.)

Explain the potential to exploit strong stimulation of specific visual regions for evil. "Oh, I very much detect a face/place/body/motion/pattern/human", says the subject. What are you going to do with that, startle them?

Apply the same technology several layers deeper. Stimulate the novelty or sexual attraction or general reward neurons. Stimulate the thirst center in a Pepsi ad.

Do you know that strongly triggering these regions at the same time/repeatedly/in various patterns doesn't have any effect on your mood, wants, needs and so on due to experience in the field? Or why are you just discounting the idea?

IMO it's terrifying to imagine that adtech companies could plug this stuff into their optimizers and just run an experiment on broad swaths of the population to figure out if there's any effects.

Oh you are right. Let's just end neuroscience altogether. Also computer science, aerospace, and biology in general. In fact, let's go full Amish. Wait, no. Someone might use a buggy cart to run people over.

What exactly is your new fear here?

This isn't about "might", this is about "obviously will", and the sheer fact that questions that were 101 of scientific discussion now lead to instant freaking out and accusations, i.e. stupid cultish bullshit, show me we need to be more serious about it, not less. If that's not to your liking, is it okay for you if other people ponder such questions? Can you maybe stim your brain into believing they don't?

>Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.

It also helps companies like Moonbug Entertainment (Candle Media) understand how to build better Distractatrons.

    It’s a small TV screen, placed a few feet from the larger one, that plays a continuous loop of banal, real-world scenes — a guy pouring a cup of coffee, someone getting a haircut — each lasting about 20 seconds. Whenever a youngster looks away from the Moonbug show to glimpse the Distractatron, a note is jotted down.

    “It’s not mega-interesting, what’s on the Distractatron,” said Maurice Wheeler, who runs the research group. “But if they aren’t fully focused, they might go, ‘Oh, what’s that?’ and kind of drift over. We can see what they’re looking at and the exact moment when they got distracted.”

https://www.nytimes.com/2022/05/05/arts/television/cocomelon...

https://www.newyorker.com/magazine/2024/06/17/cocomelon-chil...

What a world.

I know a technology like that was used ~20 years ago for ADHD. EEG feedback, as soon as the kid looks away or zones out, the movie stops playing.

> ...the Distractatron

I can see it now.. The Distractatrons: a new chapter of protagonists in Transformers! The modern equivalent of evil in this day and age of ADHD and low attention span!

I recently rewatched the movie Looker which was vastly ahead of its time in 1981

https://en.wikipedia.org/wiki/Looker

the same year as Baudrillard's Simulacra and Simulation, featuring computer generated characters, brainwashing television commercials, a light pulse gun that causes absence seizures, gun battles inside a plastic surgery clinic and an AR simulation environment, a sadistic computer, a physician who doubles as an action hero, and James Coburn giving a lecture explaining the enemy's evil plans in the opposite role that he played in the (excellent) The President's Analyst.

This time I was not so dazzled and saw it for as atrocious everyone else things it is. The minions of "Digital Matrix, Inc." manage several assassinations with the light-pulse L.O.O.K.E.R. gun but when they use real firearms they outdo Vader's stormtroopers by shooting each other. (Want to see the scene where somebody from E.Y. tells them to stick to the L.O.O.K.E.R. gun) The bad guys explain the penultimate secret to the protagonist early on but the ultimate secret is revealed in the L.O.O.K.E.R. lab which doesn't feel like a lab at all but rather a rather good room in a theme park experience where you're supposed to uncover the secret. (Contrast that to the lab Doug Trumbull outfitted in the Brainstorms movie a few years later which is packed with real surplus equipment... I've been to that lab!)

https://www.youtube.com/watch?v=cBlWxXqH8vA

Sounds like they took a cue from Sesame Street:

> p8: "A child watching television under normal conditions is subject to frequent interruptions and distractions. The TV must vie for his attention. In order to simulate this condition, we decided to program distractions into the laboratory situation... Slides could be used to fill the slide tray and they could be projected automatically, at regular intervals, onto a screen similar to that of the television set. The carousel projector allows the viewer to choose three exposure times. The 7.45 second interval proved most satisfactory with the preschool children.

THE FIRST YEAR OF SESAME STREET: THE FORMATIVE RESEARCH https://files.eric.ed.gov/fulltext/ED047822.pdf

And nowadays we think of Sesame Street as wholesome compared to today's content. They may have optimised attention but they were delivering positive-value content. That's not the case any more.

Anyone have a link or more info on the "mind reading startup" being referred to?

This specific research may not be morally corrupt in itself, nor may his intentions be bad, but it could absolutely lead to something horrific. Nuclear technology was also initially developed with good intentions and provided much good for humanity. But of course it ended with biggest bad actors having a lot of nuclear weapons.

The title reads a lot like the lab logs you'd find in a horror game.

But for the paper itself, it seems they're using genetic optimization over predefined keywords. Wonder what would happen if they did gradient descent on the latent space directly. Is brain stimulation just not a good domain for GD?

> Is brain stimulation just not a good domain for GD?

The largest LLMs right now are at best 1% the number of parameters of a human brain.

"At best" if synapses are one parameter each, real ones are probably more than that, but nobody's entirely sure yet.

The whole site looks AI generated. Those 3D brains are... smooth.

Future looks bright....

Inventing Snow Crash. Neat.

See Zuck, the metaverse failed, but who said the mind control part couldn't work?

Fuck this and whoever is driving this type of research. We’re increasingly living in a world designed by sociopaths. What do these people think this research will be used for?

I wish we had the Hippocratic oath for STEM, or at least that they would take ethics seriously rather than an afterthought against the god of Progress at all costs.

There are ethical use cases for this research. The first that comes to mind is mapping the brain before a surgery. Right now some surgeries are made with the patient concious to verify that relevant brain damages are not being done. Having a deeper knowledge of the patient's brain before the surgery may decrease risks further. The bad scenario (using targeted content to hook the user to a screen) already exists, it's not like it can get that worse (or at least so I hope)

> There are ethical use cases for this research.

Of course there are. Given what you know about the society we live in, do you think those will be the primary use cases that will be pursued?

So, AI images to maximally drive reward centers in the brain?

What could possibly go wrong?

This is the absolutely horrific next stage for social media platforms:

- They're already well able to surface the most addictive short video for a specific user out of millions of real videos.

- But these millions of real videos are just darts thrown into the space of "videos that could hook the user", in the end even the best-selected of them is not perfect.

- Now, behold! AI allows to generate the perfect video to surgically hit all the switches in the viewer's brain and turn it into a zombie hooked for days on end.

Let's hope our regulations hit these "social networks" hard enough so that never dare deploy this kind of technology.

Social media is not even the worst threat vector, in my opinion. It’s Anthropic/OpenAI/Google, whom people are giving an unprecedented level of access to their innermost thoughts.

We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver. And I have no doubt the likes of Dario Amodei and Sam Altman will show us, if we give them enough time.

* I suppose the GPT-4o sycophancy/AI psychosis crisis was a preview, but that was just blunt “engagement” tuning.

The largest social media company in the world by several orders of magnitude is Meta, who are both using data collected from their social media platforms to train their AI models and improve ad targeting. Meta also offers AI chat bots on their platforms (FB Messenger and WhatsApp), to capture those innermost thoughts.

Don’t count out Zuckerberg, he’s very good at being a villain.

The mother of social media is free TV.

We tend to forget that both, the social and the media part, input and output, are interpersonal, first.

Ad targeting doesn't need improvement. The incentives offered to people with an adverse attitude towards Ads are shit. Think of incentives as having effects and side effects. It's about intent. Free TV had that down ages ago. It was awesome. But they fucked market entry and whole population segments, more or less conservative and/or liberal as well as more or less authentic sub-cultural patchworks were left unserved.

That wasn't even nonsense! MTV, suddenly going away? Uhm, guys, I only get Peeeeeeeeeeeeeeeeeeeeee... beep boop bop beep boop bop beep... BEEEEEEEEEEEEEEEEEEEEP... (pause) ...brr-ding... brr-ding... Krrrrrrrrrrr-shhhhhhhhhhhh-eeeeeeeeeee-awwwwwwwww-eeeeeeeeeee... Bong-bong-bong... KSHHHHHHHHHHHHHHHHH (click).

Silence ...

around here, and then a lot of popups, ads, .... you know, the wild wild west of internet history ...

We talked TV on the street and in school. And Video games. Now Netflix et al and your feed, too. Who's roofs is most of that media industrial complex under?

Thinking Meta et al and their ads infrastructure and UX weighs as much as that of Free TV is falling for a rather simple Kansas City Shuffle.

"New", only now native media still doesn't interface as well with the brain and our emotional processing as TV and "The News" do. And won't until Neuralink et al get their interfaces all the way up your ... lobes.

Forget Zuckerberg et al and let them do their jobs. They don't need publicity, they need to filter for suitable test subjects, edge case pop segments whose needs & desires are yet to be fulfilled.

There's still a lot of people out there with something to put on the market but they can't because a little something is missing and nobody seems to know why ... something double long running on the news maybe?

I’m not sure I understood more than half of your comment, but there is a significant, massive difference between social media and TV.

TV was filtered and averaged to appeal to a wide audience. By design, it could not target small niches like social media does. Or amplify them - only a few, selected people, with hard edges removed, became part of a broadcast. Same for ads, we did not have 1% of the current targeting and retargeting capabilities.

> to appeal to a wide audience.

... to appeal to the widest of audience(s) at the same time.

> filtered and averaged

... feeds, websites, news, ... everything is filtered and averaged, ... we have a lot of city part of town "content" because there are audiences--already there--who are drawn towards city parts of town content and produce ...

> we did not have 1% of the current targeting and retargeting capabilities.

You and I didn't. Some folks did. Some had access to cables. Others to satellites. Some were bosses and managers and fancied their power and enough of all these people had no issue with a lack of boundaries and nothing to stay busy enough with ( or the intensity of their predilections was a little too much ) ...

Convenience and accessibility are not capabilities/skill. To some people in every city part of town, things just don't look like anything to them. Monkey see, monkey do.

> You and I didn't. Some folks did.

No they didn’t. I’m sorry but this seems to be pure waffle.

Social media is also AI now, so I don't really see the difference.

Sadly, this is the reality.

> 59% of videos served to a new TikTok account’s For You page were AI slop

97% of videos under the #cartoonkids hashtag were AI slop

57.4% of TikToks in the Kids category were AI slop

Source: https://www.kapwing.com/resources/the-tiktok-ai-slop-report

depends. I, for one, take control of what I consume on social media. Bluesky's design allows this. YouTube also, if you turn off recommendations, lose watch history, and vet your subscriptions

What makes you think they’d restrict this to logged in accounts? Not so long ago, Meta got caught building “ghost” social media profiles for people who didn’t have accounts on their platforms. They essentially constructed a “missing puzzle piece” based on people this person knew who did have profiles in publicly available information. Building a psychosocial profile on a user who has not set up an account yet using AI would be trivial.

excuse me? I didn't say that anyone restricts anything, I only said that I take effort to curate what I see

Has been going on for a long time in the adtech space.

Ah, the difference is in the richness of the signal sent back to the company. TikTok knows what content you’ll engage with and what demographic/social clusters you belong to. ChatGPT knows what you think, how you reason, your personal history, your hopes, your worries.

These AI companies, if they don’t implode, will bring campaigns that make Cambridge Analytica look like someone handing out flyers at the mall.

Is the scale there though? I know some people use AI like a companion or mentor, but how common is that really?

I didn't realize that we'd start creating Westworld quite so quickly.

What crisis and engagement tuning are you referring to?

> We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver.

We did. Pre-WW II ..., and afterwards, the old, everlasting Cold War (vs the new one). And there are self-emerging/self-organizing buckets of people who all pledged their allegiance to ideas and plans, works forever in progress and aggregating jobs and expanding industries. People with value and virtue systems, radicalized or called to some need-to-know duty. And the personal level was always covered by applied and philosophical psychology, cultural discourses, however progressively or decadently lead by public figures while peoples' desires were and still are shaped by interactions, social AND parasocial.

Shouldn't we stop wondering whether they have enough time, now that several hundreds of thousands of hours of work already done and work yet to get done are neatly compressed into a few instants?

AI will always be "two" things:

- a search engine that misses less and less and is just waiting for you to increase your processing power - a tool to cope with your laziness or disability by deferring or skipping intervals of learning curves and/or biomechanics - an evergrowing compendium of abstractions and intents, formalized or not, by humans, for humans, off humans (even when it's AI)

but that's irrelevant ... because humans will always be only "two" things, as well.

So, ... the threat vector remains unchanged, however AI, accessibility and skill will amplify or change the landscape: casually, no, leisurely ignorant bystanders.

Look at who is in the board seats to see how crucial 1:1 steering + aggregated sentiment analysis and coersion at scale is to these stakeholders.

We are doing to our brains and pur children the same thing we are doing to pigs in cages, squezeeing every last cent until the planet burns down.

> until the planet burns down.

Common misconception. We won't kill the planet even if we tried to. Wed make it too hard to sustain billions of humans, sure. But they are very different things

The planet will still be here even if it burns down. After London burned down, there was still London. It was just burned down.

Sure, and the planet survived the Permian-Triassic extinction event. Cold comfort to 70% of the terrestrial vertebrate species at the time.

This is a good distinction because its a common idea but its based on a conflation: human civilizationhealth and planet health. Planet health is hard to even think about apart from humans but the planet in no way “needs” us.

It's a silly distinction. Unless we're wiped out by a virus or something similar, whatever ends us is likely to also take down a huge part of the biosphere as well - as is already happening.

"Burning something down" doesn't mean literally killing all life. But the unnecessary deaths of likely billions of animals in the long run should weigh enough to stop this nitpicking.

Same could be said about any of the many mass extinction events in earth history. Not that I’m in favor of setting the planet on fire, but if you zoom out far enough it’s clear earth would bounce back from even a nuclear war coupled with climate change and microplastics.

And that makes it somehow okay? You just don't care about billions of deaths?

Like, do you think I'd argue that any of the previous mass extinctions were good?

We might not be here without those mass extinction events. So yeah, they were good, fck the dinos.

I think the mere fact that they resulted in comments like yours proves they were objectively not good.

So why are you upset that it might disappear? Make your mind up

Not good but definitely inevitable. It’s sort of like a cache clearing mechanism for the sim.

The current mass extinction was in no way inevitable, it is caused by humans. We can still have an unknowably large impact on the number of lives lost. I don't know why we should pretend otherwise.

Sorry but how will a return to a hot climate kill all life? +2c and the earth dies? It's apocalyptic hysterical thinking

Why are you thoughtlessly spamming climate denier talking points? Nobody said anything about all life being killed.

> Like, do you think I'd argue that any of the previous mass extinctions were good

Without the most recent one, mammals and thus humans wouldn't have flourished... So yes. If you conversely care about those people dying, then you should be celebrating the comet that killed the dinosaurs

Could you explain why I "should" be celebrating that? There's no logical sequence leading from "caring about people & animals dying" to "one should think that previous mass extinctions were good".

You say that, but run away greenhouse gases can sterilize the entire planet, like Venus.

Fortunately that particular extreme scenario doesn't seem likely. Radiance scales with the fourth power of temperature, and we're not filling the atmosphere with Venus levels of sulfuric acid. Our CO2 output will be self-limiting when it kills us and we stop outputting it.

Sure, it will "only" look like another one of the Big Five extinction events.

We've already caused one of those and it's ongoing. Wiping ourselves out would actually end it.

Are you sure it would end it?

We won't kill it, but we can try to burn it. Conflagration comes to mind.

Like a burned out brick building, a burned down planet is still there, but still also burned down. Of course it doesn't 'disappear'

Not even close chief.

Earth has been a rich, life sustaining planet in far hotter conditions then anything we are projected to take it to.

We can kill billions of ourselves, but short of ww3 earth will crack on just fine.

Social media emotional hot-takes dont match the numbers.

We are still screwed though.

There's always the Death Star angle.

>We won't kill the planet even if we tried to.

Common misconception. The planet isn't alive. We can't kill it. It's a rock.

Carfentanyl potentcy is roughly 10,000 times morphine. The pursuit of optimized technology usually far outpaces regulation or any kind of control. As technologies are controlled, even the lessened availability of more potent types causes an enormous overall issue. Humans are obsessive-compulsive optimizers, for good and for ill.

I am surprised that no comment has yet mentioned Infinite Jest (which I'm still yet to complete - it's been years since I started).

Care to elaborate for those that haven't started tackling this tome yet?

I think parent is referring to the film in the novel, also named Infinite Jest. The film is so compelling that its viewers lose all interest in anything other than repeatedly viewing it, and thus eventually die.

This occurred to rats who had levers wired directly to their reward centers.

if you're interested in completing it, check out the 30th anniversary audiobook, it just came out a few months ago, very accessible

Curious how the audiobook deals with the end notes? They are crucial imo.

it interleaves the end note in the text at the point where the endnote is referenced (by number), reads the endnote, and then sounds a bell to indicate that we've returned to the main text.

Sometimes endnotes reference other endnotes and you have to wait until the point in the text, e.g. endnote 304 is referenced from earlier endnotes than it occurs in the text (rather late in the book) and you just have to wait until endnote 304 is read to find out what it is. that's a bit of a disadvantage, but infinite jest is the kind of book that you really need to read multiple times in order to get it anyways.

That seems reasonable! I’ll check that out.

And yes, I agree. Im on my second read through now. The first time I skipped the first ~50 end notes. I thought the creator of infinite jest was a mystery, same with Joelles deformity (“was she really mangled or not— maybe we find out at the end!”). Oh, those two things are spelled out clearly in early end notes..

Any DFW fans here want to conjecture as to what “atemporal” jazz Pemulis listens to? I was thinking maybe some albert manglesdorf or perhaps ornette coleman..

Don't forget it's not "the platforms" doing this, it's people like you and me, maybe even people reading this message. We, they, are building this, and don't care about anything as long the paycheck comes. This is the reality.

Speak for yourself, I would rather clean toilets for a living than work on these mind-destroying technologies

I suspect OP was referring to not only working for these platforms but also creating content or even consuming it.

Moloch wins again

https://medium.com/@darth.mark.23/the-moloch-dilemma-how-gam...

Stop calling them “social” media. They have not been that since algorithmic feeds driving infinite scroll replaced updates from your friends. That was more than a decade ago.

There is nothing at all social about the latest generation like TikTok and YouTube Shorts.

If YouTube is social media, anything with a comments section is social media, even the old "guestbooks" of web 1.0 days were social media. It's a pointless term whoever uses it.

Youtube is very obviously social media. It is a platform that serves algorithmically sorted third party content based on user preferences and a network of accounts that a user follows, and it allows communication between users through comments. It has communities. It has memes. It is certainly addictive in all of the ways social media can be.

Forums are social media. Irc is social media. BBS is social media. They all qualify under your definition.

Neither IRC nor BBS* use algorithms to create feeds of third party content based on user preference, history and a network of subscribed accounts, but Youtube does. Forums can count, depending on their implementation - I don't think many people would argue that Reddit is social media, but it definitely suggests related content.

More to the point, "social media" is not as strictly defined as many people on HN want it to be, because social media features have crept into numerous platforms and applications over the years, so it really can be subjective.

* maybe some have, IDK.

> Neither IRC nor BBS* use algorithms to create feeds of third party content based on user preference,

Both bbs and irc had top user features that encouraged users to post, bbs certainly had third party content sorted and presented to the user by algorithm.

The reason the definition is so vague is because it's largely driven by parental hysteria and can be ignored as such

Algorithmic media

There's actual real people working on this bullshit. Horrifying doesn't even come close.

They have information barriers to keep the non-sociopaths from understanding the horror they're working on. Military forces do the same thing. Even the German train drivers didn't know what kind of facilities their trains were going to or why.

I think you're giving this a bit too much credence. And I think you also think there's something special about sorting videos by how often they're viewed in a recent timeframe (which is all you need to recreate evil social media algorithm)

Feed algorithms are far from sorting videos only by strict decreasing order of trending factor. One example of that is that feeds are randomized: refresh the page, it changes completely. Another example is how it optimizes for bait content.

If it's randomised that's not malignant. Optimizing for bait content is just showing what's popular recently. The system only knows it's bait because so many people clicked it (and took the bait)

> Let's hope our regulations hit these "social networks" hard enough so that never dare deploy this kind of technology.

You'd do as well to hope that people wake up and stop opening the damned sites

you really have to catch the moment to do this, and there must be someone else who is supportive of this idea.

my mother recently got served a quite... disturbing video on Facebook. I won't get into details of the vid. what I can tell is that she was so dazed that she didn't even immediately understand me when I asked her, "delete Facebook and Instagram immediately or else they will think you like it and will serve you more." she kept replaying this video in her head. in a moment, I thought I wouldn't make her snap out.

for context, she is a damned Reels addict with war-induced traumas. (Ukrainian here.) she agreed to delete Facebook but decided to keep Instagram. we frequently pick bones about her sending stupid Reels and me hating them with passion. and every time I said "delete Instagram and your life will go up," she brought up scandals until she talked to a therapist.

EDIT: to quote a sibling comment, this is, indeed, like dealing with a drug addict.

Ah, just like drug addicts put down their poppies...

If you're relying on government regulation to solve this, I guess the obvious bigger question would be "Quis custodiet ipsos custodes?

Now apply it to porn

Sounds like some people are trying to develop Snow Crash.

there's not gonna be a regulation and even if there is, the damage would already be done. If you want to protect yourself, then best thing is to just delete the account and apps.

All glory to hypnotoad!

That would be ideal, but I think we're heading straight for Idiocracy's 'Ow! My Balls'.

Yeah, I was surprised that they didn't list dlPFC or some other reward-associated area as one of their targets.

I am not sure if we should want to understand how our brain works. This is why I don't like libertarian free will people. We need to accept that we don't have free will. Otherwise we will study the brain thinking that we are above it. No. We are soon going to find out exactly how the brain works and I am not sure people understand what that means.

Among my secret research ideas, this is the most dangerous and morally wrong one.

If future generations of researchers will wonder why IRB reviews became mandatory for computer science, studies like this will be the answer.

Seriously, some people don't seem to realize the point at which they are becoming Fritz Haber.

In general, technology has always decreased our liberty though

Are you serious?

It's hard to enslave your fellow man without a bigger stick

You're talking about superiority of technology, not technology itself.

Describing some of our colleagues as "becoming Fritz Haber" is a highly deep and accurate cut that I will 100% be stealing from you.

Customized propaganda made for the individual is the only thing I'm afraid of. Right now its easy to avoid the farm trolls that harves boomers minds on facebook, but given the costs go down and tech improves, we each of us have a trollfarm working specifically for us and that would make it extremely hard to not get brainwashed.

Aee they trying to create snow crash for real??? [1] https://scifi.stackexchange.com/questions/184341/in-the-nove...

or perhaps infinite jest once it hits tiktok

https://en.wikipedia.org/wiki/Infinite_Jest

> a film, Infinite Jest, also called "the Entertainment" or "the samizdat". The film is so compelling that its viewers lose all interest in anything other than repeatedly viewing it, and thus eventually die.

For a video to literally maximally drive a target brain region might actually lead to fatal consequences - e.g. the plot of David Foster Wallace's Infinite Jest (1996).

I can imagine truly horrifying images coming out of this. Many people who have spent a lot of time online have come across unfortunate gore images that they may have seen for only a fraction of a second that they wish they could unsee. One can imagine the brain is able to see much worse.

June 9, 2032: ReMind, a new biotech startup, has opened to a recordbreaking IPO after a demo and a series of small-scale trials that stunned the world. The company is pioneering a new way to ingrain memories into the human brain - each ReMind screen is equipped with a special camera that can quickly recognize anyone who walks up to it, and ReMind's secret sauce algorithm is able to use past data of each user to generate imagery that will have the most profound impact on their brains. They claim that the algorithm is highly effective, being able to burn in imagery into the viewer's brains with an average accuracy of 83% per viewing. The CEO has promised to launch two services to start: ReMind Long-term, which focuses on creating imagery that simulates a 'traumatic' impression for creating long-term memories, which will be immensely useful for advertisers; as well as ReMind Short-term, for milder experiences that resemble a catchy song in a visual format for a quick boost in sales for a limited-time promotion. Advertising companies around the world have already placed tens of thousands of sales and deployment of ReMind-equipped screens is planned to begin immediately. Look forward to spotting one in your grocery store, in an elevator, on gas stations, on in-flight entertainment systems and in place of billboards on public spaces. The future is looking bright for ReMind!

We're edging ever closer to Langford's Basilisk.

the logical endpoint of this research is an AI that generates videos so compelling your brain forgets it was supposed to be working

An anecdote, but some reason to think that overworking part of your brain is a bad idea:

I had an Aunt who had dementia. This is obviously a terrible outlook. But her and my Uncle seemed to be doing okay. My Uncle was an ultra-competent guy, highly stable, the person you could rely on; and had been his whole adult life. So it was shocking when he had a mental breakdown and became manic. There was probably something physical going on, but what was also going on was that he had wanted my aunt to be able to live normally for as long as possible, so had been covering everything - for a year or more he'd had to be alert 7 days a week in case my aunt tried to cook on the gas stove or something like that, at which she was no longer safe. So the risk-alert part of his brain had been constantly overworked.

I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.

I'm not sure it's about overworking as much as stress - stress is a known potential trigger for manic episodes. Typically in people who already have some biological potential for experiencing it. And the stress of having your partner come down with dementia and suddenly becoming their full-time caregiver must be absolutely massive.

But re the underlying physical thing: You might look back and be able to spot some episodes of hypomania in your uncle's life. Perhaps just reduced sleep, or twinges of paranoia, or periods of on-again/off-again hyperproductivity, etc.

Hope that if this story is recent he's doing well. The recent generations of medications to prevent and treat mania are a huge improvement on the ones available in the past, so fingers crossed.

This was a while back, they have both now passed; but thanks for the good wishes.

> I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.

What's to gain in doing so?

I will state plainly that I do not think you are asking this maliciously, and that instead you are asking from curiosity. From that perspective, though, I am a bit flabbergasted.

Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent, to such an extent as to drive motivations and bias neuronal activation that leads to what they think is a choice?

Said another way: Are you really curious as to how allowing a company additional tools to bias human interactions may not be A Good Thing?

If you need an example as to how this is clearly not A Good Thing, and that at least a portion of the world population agrees with that characteristic, that there are currently companies that have tried to actively do this already, and attempts are being made at least to fine them for their behavior as an attempt to curb it. Whether or not it succeeds is a question of governance at the moment since the US is currently under plutocratic control.

However, I would really urge you to explore this topic and really engage with what's happening due to the plutocratic class' capabilities over technologic change and societally enforced or biased adoption, and how they, in many extreme cases that are continuing to become more extreme, are not A Good Thing.

Simplest, most readily available example: https://ground.news/article/meta-says-us-states-are-seeking-...

> Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent, to such an extent as to drive motivations and bias neuronal activation that leads to what they think is a choice?

I guess I just don't think this is the topic at hand. What you're proposing is mind control. That's not what's at issue. What's at issue is scientifically optimized superstimuli that can predictably activate certain brain regions. So when I read "rowhammer" I'm thinking about it in this context, the ability of a video to consistently activate a targeted section of your brain.

As you should know, and IIRC, cognition does not cleanly or consistently map geographically in the brain. To go from the ability to activate a subset of brain regions to influencing behavior is not a neglible step. The only real potential I imagine is in emotional activation, which is fairly well-localized.

So really what we're talking about is being able to evoke ideas and feelings with more consistency and precision than before. This doesn't seem new. And it can't be that intense either—I have the sense that your hypothetical mind control video would be extremely annoying or unpleasant before becoming influential. And for now we have the freedom to choose what we watch and turn the video off.

And it might be reasonable to say that you were referring to future possibilities, and that I'm focusing too much on what's currently available. But I am not just reacting to the first quoted clause of parent, but to their [vomits] policy recommendations. "This sort of thing", if not heavily controlled, etc. Maybe you see no value in this research, maybe there isn't any. Nonetheless the paranoid and the scifi-as-nonfiction readers will throw up the Bat signal for the idiots and moral panic orchestrators in power to wrap us all in comforting red tape for our own good.

I exaggerate. And I am talking about many more things than you were, as if the volume of text strengthens any rebuttal. All I'm saying is, this tech has little power over human behavior, in my opinion, and we should be cautious before bringing the hammer down before we know if we've got a bumblebee or a wasp.

This just strikes me as the false dichotomy that there are two option; complete freedom of choice, and complete mind control that you are physically incapable of resisting. That as long as you are not physically incapable of, say turning off a video, that there is no problem. Just because you can stop using crack doesn't mean it isn't dangerously addictive. Just because you can say no to someone holding a gun to your head does not mean it isn't incredibly coercive.

Also this seems to ignore that people can be influenced or manipulated without being consciously aware of it. Like the people that say that marketing doesn’t work on them, failing to see that every interaction with the products and services they buy and use are curated in some way to influence and manipulate them.

I'm not certain I took that dichotomy as a premise, but regardless I agree there is a middle ground of many patches occupied by real events characterized by mixtures of intentionality and submission. That's the easy part. What I think is the harder question posed by my original parent is when is it worth wielding the state at the issue.

For example, I'm in theory opposed to the prohibition on crack cocaine, despite its strong consistent effect on personal agency, because of two things: basic moral principles (my body my choice), and the massive (hopefully unintended) side effects of prohibition. Namely, mass incarceration, black markets, decreased drug quality, gang violence, exhaustion of policing resources, the public perception of systemic racism, and tax losses. My guess is the downsides of a hypothetical ban on algorithmically optimized advertisements would be of a very different kind but similar quantity. And again, the other issue is basic moral principles. I'm not comfortable with bracketing free expression with numerous exceptions, rate-limits, inspectors, or whatever apparatchiks would be employed to enforce such a restriction.

Its like we don't already show faces in fMRI experiments because we know that it will often activate the fusiform face area. Its just light years away from anything dangerous, that I am compelled, as a neuroscientist, to try to shake some sense into people. This is HN, but it feels like twitter on some topics.

You just reminded me why I unsubscribed from ground news: you can't share a link without them adding their own paywall to it, even if it's a free article. Fuck that

I bet it wasn't overwork, it was the fight or flight response being constantly activated for your uncle when he's caring for your aunt who could do anything unpredictably.

A completely off-topic anecdote, then a completely unfounded fear based on a clear misunderstanding of this research. If companies ever start making us live in a MRI machine so they can reliably activate brain areas with certain stimuli, I think there is a bigger problem we are having.

Here is how I see it. This type of research helps us understand the brain, helps us do things like model potential surgery sites better (e.g. for seizure activity interventions). What it does not do is become the basis for mind reading.

Just so that you know that I am not totally unsympathetic to your apparent worldview, let me tell you how I think your concern might actually play out:

AI continues to get more powerful, and computer brain interfaces begin to move beyond EEG scalp electrodes, but begins to take the form or brain augmentation via integrating networked AI compute into implanted chips with electrode neural interfaces, where our brain's neurons and the neural interfaces learn to speak each other's language (i.e. integrate). I can actually see this happening within 20 years. At first, it would be our brain using this augmentation for greater intelligence, etc. However, remote manipulation of the interface could reverse control of brain activity, leading exactly to your fear.

But this research? This is light years from that,,,so much so that it is not even relevant to bring up unless you are just against all technology.

What's interesting is that the process of identifying what gets human attention is nothing new.

e.g. cartoons from the 1950s were designed to have:

- cute animals

- short and fast scene cuts

- catchy music

b/c all three of the above were known to trick the human brain into "sit still and observe" mode.

Focus groups did the same thing for advertising which then morphed into A/B testing to determine revealed preferences.

Same thing with the whole fat/salt/sugar research that led to near addictive fast food.

This is just the next logical step.

Books are designed to be addictive. People said the same things about books as they say about screens nowadays. I think it's more important to understand the emotional hysteria people get themselves into

My favorite addiction epidemic was when apparently the "catch the ball on the string in the cup" took over parts of Europe to the point that people were ignoring their work.

https://en.wikipedia.org/wiki/Cup-and-ball

One difference to consider: books require active participation on the part of the consumer. Whereas digital/screens can just happen to you whether you participate or not.

Books are not nearly as addictive as social media though.

A portion of young men already are more or less addicted to pornographic content.

Tolls like this have the potential to make this so much worse.

Could you elaborate? Esp. on the "more or less" bit. E.g. I can't imagine you take away porn and more than 100 people experience withdrawal.

Ludovico Technique as a service

I’ll leave it to other comments to discuss the societal and moral implications of being able to do this (which, I agree, ick…). On a practical level:

We train an encoding model, a “digital twin”, that predicts how each visual region responds to any video. Now we can ask: which video would make a chosen region light up the most? NEvo searches for that video automatically, using the twin’s prediction as its reward.

I only scanned the paper, so maybe I missed it, but is there any confirmation that this ‘digital twin’ works? Like, do the generated videos actually cause the same patterns as in the ‘digital twin’ brain model in real humans in an MRI machine? My instinct is to be skeptical that it’s possible to reliably create a video -> brain activation prediction model.

... and shouldn't this backfire hard?

All the signals that are missed by the (f)MRI are never "mirrored" in the digital twin even though on screen it will look like it. Bam, Experimenter Bias in the machine and I don't know how to phrase it but ... does this method/experiment leave any wiggle room for Falsifiability?

Even if the Hawthorne Effect does not apply to humans, it most certainly translates onto the brain and sensory (post)-processing-- live and remembered (vs in-memory, stored), real, virtual or imagined.

... I just realized how vastly different peripherals are when real, remembered, virtual or imagined sensory input is (post-)processed ...

It feels more like the machine will reproduce a superficial pattern. And at least some signal streams echoing in the brain on input from the digital twin won't emerge because the digital twin has only part of the data. And that's a premise for an unnoticeable rewiring of what once used to fire together, ... given a hell of a lot of exposure, of course, ... or not ... we'll see

yeah. This is one of those things where they used previously collected data, built a model that has validity on modeling validity between simulation and actual brain (encoding model for digital twin), then built a new model using the encoding model that tries to reliably activate particular brain regions. The next obvious step is to validate this back to the original person, but they will need to collect new data, and so likely, will need to get research funding. One step at a time.

The digital twin types of models will become more and more useful, but I don't think we will be at the point where all research will be done entirely within these models.

They can be fun to play with btw: https://www.thevirtualbrain.org/tvb/zwei/ and they don't require a beast of a machine to run.

You would like to buy some paperclips.

A. Unless you think that brain science is immoral science, then I do not see any problem with this kind of research. As a neuroscientist, I strongly object to the insinuation as unfounded.

B. "Digital twins" e.g. [1] are a growing class of brain simulations that can successfully approximate brain activity patterns at large scale. I think these can be very useful, but we shouldn't think that they are at the level of actually simulating a brain. They are usually made of model neuronal approximative simulations (e.g. integrate and fire, balancing excitatory and inhibitatory neural populations within units), then using diffusion imaging to estimate white matter axonal wiring between those populations from the subject to increase the accuracy of the simulation. These are increasingly being used to, for example, model how a surgical intervention would effect seizure propagation prior to actual surgery. Here is a nice episode of Theoretical Neuroscience podcast [2] on the Virtual Brain [3], one of the available models for this kind of work.

C. In terms of validation. Only partly. From my quick read, this NEVO model optimized neural response only in the digital twin encoding model. While the digital twin model reportedly has solid predictive validity [4], which by the way was not the Virtual Brain model I mentioned in point B. Moreover, the outputs looked neurobiologically plausible, but at this point, there is no independent model or new fMRI showing the optimized stimuli actually drive the target regions. This was performed using previously collected fMRI data, and full validation of this model *IS* the obvious next step, but the money to collect such data does not come from nowhere: funding will be needed, and such a paper as this can help them get it.

D. A final point I'd make. We have long been able to create static stimuli that we can be fairly certain will activate above baseline certain brain regions, on average. Certain stimuli-region pairs ar emore homogenous between people, others e.g. the fusiform face area (FFA), are small enough that individual differences prevent a simple ROI approach, and identification depends on using face stimuli to identify at the individual level, but for the most part, it is reliably locatable. Brain activations are very coarse things. In fMRI, you are talking about ~3x3x3mm voxels (27mm^3) where the hemodynamic responses have a ton of spatial autocorrelation, or in EEG, where the surface spatial area of the reeptive fields are very large(~400 mm^2). These virtual twin models already do a decent job of modeling dynamics of the brain there parameters are tuned to *at this scale*..but this scale does not have a ton of information content. Automating this with video content is not that much a reach.

[1] https://spj.science.org/doi/10.34133/icomputing.0055 [2] https://theoreticalneuroscience.no/thn23/ [3] https://www.thevirtualbrain.org/tvb/zwei/ [4] https://www.biorxiv.org/content/10.1101/2025.07.22.664908v2....

I guess people aren't over the dead salmon study.

On (A), what I - and I am sure most of the commenters here (epically the ones that did not read the article :-) - am uneasy about is that, intentions of these researchers notwithstanding, if it’s possible to generate videos to optimize for brain activation patterns, it feels inevitable that the Facebooks and Kalshis of this world will use this to generate videos that are engineered to be maximally addictive.

Perhaps we’re naive and this is a long way from being possible - but it seems like the way this is heading eventually and that seems obviously bad for society.

Yeah, this smells very fishy to me. It's almost trivially easy to gather a small validation dataset in humans for the paper. At my institution, it's about $800 an hour to scan someone. You can probably get enough data to validate the model with a half hour scan. Surely the group has enough grant funding to pop a few healthy controls in the scanner.

I haven't looked in super close detail to the paper, but their methods section says that they fit a video model (V-JEPA2) to the fMRI dataset in a voxelwise ridge regression, meaning that the baked in assumption is that the visual response affects each voxel independently. Voxelwise models are very nice for making statistical inferences, but are less good for prediction and modelling tasks, because our brains certainly do not work as collections of independent regions.

BOLD is intensely messy data, and their design is far too simple IMO to reflect anything of reality.

What's next, a movie you watch and die in 7 days?

Ah great, work on the Torment Nexus is progressing well. I can head into the weekend with peace of mind.

I actually had a similar idea that I cam across in my own research in brain-to-video applications. You could theoretically train the model in real-time by using brain data as a reward signal, such that the model would find the image or video that most strongly produces the desired response (e.g. an increase in certain brainwave bands associated with pleasure, or alertness, or fear).

I can imagine this has some very undesirable usecases if used in the wrong way.

Goal state lol:

https://www.afternow.io/content/images/2016/10/virtual_reali...

I'm pretty sure by blindly following this path to commercial success in marketing; we're just gonna end up with content that is as implicitly adjacent to porn as possible without being explicit.

The sci-fi hand wringing is embarrassing. A huge chunk of the population is afflicted with mental health issues and the best psychitrists can seem to do is "Oh bro, wanna try some ketamine? Anti-psychotics?? Maybe some SSRIs, how about psychdelics?!" - this is a joke. Im all for more neuroscience research in every form because our contemporary understanding is embarassingly poor. This is a research tool for understanding what visual brain regions encode, not a mind-control device. Better models of brain function may eventually improve diagnosis and treatment of mental illness. Calling every neuroscience advance "Black Mirror" is not ethical insight. It saddens me the reaction so many commentors have here, it is just lazy technological illiteracy imo.

In a society that wasn't visibly hell-bent on monetizing every discovery in a cartoonishly evil way, I'd probably agree with you.

I can see lot of potential in this, but I somehow doubt it's going to be used for good.

Absolutely horifying.

Let's stop pretending that we can separate the good from the bad here. This industry and everyone in it is morally bankrupt.

I’m skeptical.

I’m always skeptical of these papers that claim to read minds, because my understanding is current brain reading technology is very coarse (Neuralink is SOTA, very invasive, and only 1-3 thousand electrodes for 55-70 billion of neurons; fMRI is much worse) and neural networks generate plausible images and videos from noise. Indeed, the example videos don’t seem particularly stimulating, just random.

Someone should use this to make the "Infinite Jest" film from the book "Infinite Jest"

The trope of vibe coded pages adding this slide-up transition on every paragraph needs to die yesterday.

Could be great for therapy, but who am I kidding, this shit's going to be abused for profit and addiction.

This… wow… should be tightly regulated and illegal to use on the general population.

Literally hijacking people’s neural activation pathways on a mass scale that can’t be easily blocked, unless you look away or start meditating.

Are there any other precedents for this kind of neural-subterfuge being released into the wild, effectively lowering the barrier to mass manipulation?

is the target activation actually measured against per-subject fMRI, or optimized only against the encoder model? the landing page is ambiguous and it changes what the whole result means.