Wouldn't it be a bad thing, even if it didn't require any privacy invasion?

If it matched human intellectual productivity capacity, that ensures that human intelligence will no longer get you more money than it takes to run some GPUs, so it would presumably become optional.

kind of too close to reality more than anyone knows :)

tbh human rights are all an illusion especially if you are at the bottom of society like me. no way I will survive so if a part of me survives as training data I guess better than nothing?

imo the only way this could happen is a global collaboration without telling anyone. the AGI would know everything about all humans but its existence has to be kept a secret at least for the first n generations so it will lead to life being gameified without anyone knowing it will be eugenics but on a global scale

so many will be culled but the AGI would know how to make it look normal to prevent resistance from forming a war here a war there, law passed here etc so copyright being ignored kind of makes sense

Jesus Christ

sadly he supports the AGI, eugenics and human sacrifice lol my pastor told me he gave him 6 real estate holdings

Given how much copyrighted content I can remember? To the extent that what AI do is *inherently* piracy (and not just *also* piracy as an unforced error, as this case apparently is), a brain scan would also be piracy.

Could this agi cure cancer, and would it be in the hands of the public? Then sure, otherwise nah.

> in the hand of the public

Would you trust a businessman on that?

Nope, they haven’t earned an ounce.

How about a politician?

Eh, a transparent organization of elected officials with short term limits and strong public oversight, a little bit. A very smart ai would represent power, and so every mechanism we’ve used as humans to guard against power misuse would be the ones I’d want to see here.

at least I can fire my politicians.

no

Fuck no

Ah geeze, I come to this site to see the horrors of the sociopaths at the root of the terrible technologies that are destroying the planet I live on.

The fact that this is an active question is depressing.

The suspicion that, if it were possible, some tech bro would absolutely do it (and smugly justify it to themselves using Rokkos Basalisk or something) makes me actually angry.

I get that you're just asking a hypothetical. If I asked "Hypothetical: what if we just killed all the technologists" you'd rightly see me as a horrible person.

Damn. This site and its people. What an experience.

I read that as a (possibly sarcastic) rhetorical and cautionary hypothetical used to demonstrate the absurdity of ignoring copyright.

You seem like you set aside any critical thinking to come to "this website" looking for a reason to seethe over complete strangers about whom you know very little and whose motives you belligerently misrepresent all the while making exaggerated and extremist statements, and no doubt embracing worse thoughts.

You're the type of person destroying the planet _I_ live on.

This isn't a defense of technologists, it's a plea to stop tripping over yourself to see the worst in everyone.

I agree that was the intent of the analogy but it’s not a great one. The idea that Disney, who has perverted IP laws globally for almost a century, should have equivalent ownership over their over-extracted copyrighted works to the same degree I have privacy for the thoughts in my own head? Really?

What's with the unnecessary straw man? Who said any of that?

Would the average person even be against it? I am the most passionately pro-privacy person that i know, but i think it is a good question because society at large seems to not value privacy in the slightest. I think your outrage is probably unusual on a population level

The don’t value it because they think companies are not abusing this power too much. Little do they know…

When i talk to people it seems like they know but they just dont care. They even think their phones are listening to their conversations to target ads.

It is well known that people change how they act when they know they are being watched. Even if they can't see it, just the threat of surveillance is enough to make people change their behavior.

I say it is no different than the people who are claiming they don't care. They absolutely do care, but at this point, saying "no" makes you the odd one with obviously something to hide, so they do this from a place of duress.

Unfortunately, I feel we are not too far from people finally snapping and going off the deep end because it's so pervasive and in-your-face that there is seemingly no escape left.

It’s a fun hypothetical and not an obvious answer, to me at least.

But it’s not at all a similar dilemma to “should we allow the IP empire-building of the 1900’s to claim ownership over the concept of learning from copyrighted material”.

> It’s a fun hypothetical and not an obvious answer, to me at least.

As I wrote it out, I didn't know what I thought either.

But now some sleep later, I feel like the answer is pretty clearly "No, not worth it", at least from myself.

Our exclusive control over access to our mind, is our essential form of self-determination, and what it means to be an individual in society. Cross that boundary (forcefully no less) and it's probably one of the worst ways you could violate a human.

Besides, I'm personally not hugely into the whole "aggregate benefits could outweigh individual harms" mindset utilitarians tend to employ, and feels like it misses thinking about the humans involved.

Anyways, sorry if the question upset some people, it wasn't meant to paint any specific picture but a thought experiment more or less, as we inch closer to scarier stuff being possible.

Even most morally inclined people tend to overestimate the value of immediate benefits, and underestimate the eventual (especially delayed, unknown) harms.

> would it be worth just ignoring everyone's wish regarding privacy one time to suck up this data and have AGI moving forward?

Sure, if we all get a stake of ownership in it.

If some private company is going to be the main beneficiary, no, and hell no.

> Sure, if we all get a stake of ownership in it.

But we do, in the sense that benefits flow to the prompter, not the AI developers. The person comes with a problem, AI generates responses, they stand to benefit because it was their problem, the AI provider makes cents per million tokens.

AI benefits follow the owners of problems. That person might have started a projct or taken a turn in their life as a result, the benefit is unquantifiable.

LLMs are like Linux, they empower everyone, and benefits are tied to usage not development.

We've seen this kind of system before. It was called sharecropping, and it was terrible.

The price will be ratcheted up, such that the majority of the economic surplus will go to the owner of AGI.

As Are you claiming that the right to use a product or service implies a sort of ownership of it? If it’s free to use, I suppose that makes some sense. If you’re saying that the right to purchase use of it implies a level of ownership, that’s just prima facie absurd.

If you don't, your geopolitical adversary might be the first to build AGI.

So in this scenario I could see it become necessary from a military perspective.