This has cosmological significance if it leads to superintelligence

Don't worry, it'll only lead to superstupidity.

And superplagiarism of human-created content

I'm sure this will age well.

Is that the prequel to Idiocracy?

It won't unless there's another (r)evolution in the underlying technology / science / algorithms, at this point scaling up just means they use bigger datasets or more iterations, but it's more finetuning and improving the existing output then coming up with a next generation / superintelligence.

Okay, but let’s be pessimistic for a moment. What can we do if that revolution does happen, and they’re close to AGI?

I don’t believe the control problem is solved, but I’m not sure it would matter if it is.

Being pessimistic, how come no human supergeniuses ever took over the world? Why didn't Leibniz make everyone else into his slaves?

I don't even understand what the proposed mechanism for "rouge AI enslaves humanity" is. It's scifi (and not hard scifi) as far as I can see.

> Being pessimistic, how come no human supergeniuses ever took over the world? Why didn't Leibniz make everyone else into his slaves?

We already did. Look at the state of animals today vs <1 mya. Bovines grown in unprecedented mass numbers to live short lives before slaughter. Wolves bred into an all new animal, friendly and helpful to the dominate species. Previously apex predators with claws, teeth, speed and strength, rendered extinct.

Sometimes I wonder if we are going to be the unkillable plague that takes over the universe. Or maybe we will dissappear in a blink. It's hard to know, we don't have any reference point except ourselves.

Destroying human life in Earth (the only habitable place in the solar system) is far far easier than reaching something outside the solar system.

Once you have one AGI, you can scale it to many AGI as long as you have the necessary compute. An AGI never needs to take breaks, can work non-stop on a problem, has access to all of the world's information simultaneously, and can interact with any system it's connected to.

To put it simply, it could outcompete humanity on every metric that matters, especially given recent advancements in robotics.

...so it can think really hard all the time and come up with lots of great, devious evil ideas?

Again, I wonder why no group of smart people with brilliant ideas has unilaterally imposed those ideas on the rest of humanity through sheer force of genius.

An equivalent advance in autonomous robotics would solve the force projection issue, if that's what you're getting at.

I don't know if this will happen with any certainty, but the general idea of commoditising intelligence very much has the ability to tip the world order: every problem that can be tackled by throwing brainpower at it will be, and those advances will compound.

Also, the question you're posing did happen: it was called the Manhattan Project.

And if this whole exercise turns out to be a flop and gets us absolutely nowhere closer to AGI?

“AGI” has proven to be today’s hot marketing stunt for when you need to raise another round of cash and your only viable product is optimism.

Flying cars were just around the corner in the 60s, too.

This thread started from a deliberately pessimistic hypothetical of what happens if AGI actually manifests, so your comment is misplaced.

You really haven't used any LLM seriously eh

So don't plug the smart evil computer into the strong robots? Great, AI apocalypse averted.

The Manhattan Project would be a cute example if the Los Alamos scientists had gone rogue and declared themselves emperors of mankind, but no, in fact the people in charge remained the people in charge - mostly not supergeniuses.

Look at any corporation or government to understand how a large group of humans can be driven to do specific things none of them individually want.

Quite a few have succeeded in conquering large fractions of the Earth's population: Napoleon, Hitler, Genghis Khan, the Roman emperors, Alexander the Great, Mao Zedong. America and Britain as systems did so for long periods of time.

All of these entities would have been enormously more powerful with access to an AGI's immortality, sleeplessness, and ability to clone itself.

And of course the more society is wired up and controlled by computer systems, the more the AGI could directly manage it.

I can see what you're trying to say, but I cannot for the life of me figure out how an AGI would have helped Alexander the Great.

Alexander the Great made his conquests by building a really good reputation for war, then leveraging it to get tribute agreements while leaving the local governments intact. This is a good way to do it when communication lines are slow and unreliable, because the emperor just needs to check tribute once a year to enforce the agreements, but it's weak control.

If Alexander could have left perfectly aligned copies of himself in every city he passed, he could have gotten much more control and authority, and still avoided a fight by agreeing to maintain the local power structure with himself as the new head of state.

Oh, you're assuming an entire networking infrastructure as well. That makes way more sense, but the miracle there isn't AGI - without networking they'd lose alignment over time. Honestly, I feel like it would devolve in a patchwork of different kingdoms run by an Alexander figurehead... where have I seen this before?

The problem you're proposing could be solved via a high quality cellular network.

I consider many successful military leaders and politicians to be geniuses as well. In my books, Caesar is as genius as Newton!

Having said that, we do not to understand the world to exploit it for ourselves. And what better way to understand and exploit the universe than science? Its an endearment.

This is profoundly and disturbingly bad argument.

1)Leibniz wasn't superhuman 2) Leibniz couldn't work 24/7 3) he could not self increase the speed of his own hardware (body) 4) he could not spawn 1 trillion copies of him to work 24/7

Like how much time did you think before writing this

Again, my reaction is... so what?

A trillion hyperintelligent demons might be cogitating right now on the head of a pin. You can't prove they aren't thinking up all sorts of genius evil schemes. My point is that "intelligence" has never been a sufficient - or even necessary - component of imposing ones will on humans.

I feel like HN/EA/"Grey Tribe" people fail to see this because they so worship intellect. I'm much more likely to fall victim to a big dumb man than smart computers.

huh what ? this is a whole new level of flat earther thinking. you actually believe that apex predators are not the most intelligent? Like humans became the apex because of something else than intelligence? AI and covid showed humanity what levels of wacky stuff people believe. I am not trying to convince you, thank you for showing me this perspective :)

> bigger datasets

Not even, they already ran out of data.

I am sure that the M.I.C. have a ton of classified data that could be used to train a military AI.

> It won't unless there's another (r)evolution in the underlying technology / science

I think reinforcement learning with little to no human feedback, O-1 / R-1 style, might be that revolution.

There is lots of human feedback. This isn’t a game with an end state that it can easily play against itself. It needs problems with known solutions, or realistic simulations. This is why people wonder if our own universe is a simulation for training an asi.

I think gluing wings to a pig will make it fly. Show me examples or stop the conjecture.

"this generation shall not pass"... to me that's about as credible as wanting to "preserve human consciousness" by going to Mars.

Setting the world on fire and disrupting societies gleefully, while basically building bunkers (figuratively more than literally) and consolidating surveillance and propaganda to ride out the cataclysm, that's what I'm seeing.

And the stories to sell people on continuing to put up with that are not even good IMO. Just because the people who use the story to consolidate wealth and control are excited about that, we're somehow expected to be excited about the promise of a pair of socks made from barbed wire they gave us for Christmas. It's the narcissistic experience: "this is shit. this benefits you, not me. this hurts me."

One thing is sure, actual intelligence, regardless of how you may define it, something that is able to reason and speak freely, is NOT what people who fire engineers for correcting them want. It's not about a sort of oracle for humanity to enjoy and benefit from, that just speaks "truth".

Wealth residistribution scheme. Your tax dollars into their pockets.

As far as I can tell, this will be financed by private money. Can you elaborate?

Tax breaks, government forced to become a customer etc. the usual. Just like the astronauts to Mars thing will just shovel your money that might have gone to NASA into Musk's pocket.

Tax breaks, i.e. my money not being in your pocket means that they are stolen?

Assuming the tax money has to come from somewhere at some point, those who pay taxes have to make up the shortfall from those who have tax breaks. So far the US just kicks that can down the road so...

That is a big assumption. Tax money need not be a constant. But for the sake of following the same logic: if companies pay bigger taxes, they also have to make up the shortfall. Actually, this last one is much more accurate statement. Companies do not pay taxes, PEOPLE pay taxes. So taxes are paid either by the employees, the clients or by the owners (which in case of the big tech are generally common people). With high taxation you are hurting: the customers, the workers and the middle class saving for their retirement. Who is winning the tax money: state bureaucracy, corrupt politicians and the business around them, people who live like parasites (or rather are forced to live like that, because they are electoral power).

Tax breaks, i.e. a company extracting wealth from a community without paying into the systems that keep all the parts of that community running, forcing the community to ultimate subsidize that business's weath extraction from them.

Companies do not extract value, they create value which is then transferred to the people via the market through voluntary exchange (ideally). Where have you learned about those things? Oh, yeah, “community” , i.e. Marx.

>Companies do not extract value,

Oil and minning companies too?

Yes, before the resource is taken out of the Earth it doesn’t exist, it is created in a sense by them. Look at Venezuela - they are dying of hunger with all the oil in the world (Russia, too) But socialist ideas prevailed there and the bad companies are banned.

Tax breaks have basically the same effect as the government writing a check, increases inflation.

This is utter nonsense. If 1000 people go to a deserted island with no government and taxation would that mean the inflation will be plus infinity or at least very high??? Inflation is monetary phenomenon, it happens when money is being printed.

In that case there would be no inflation or deflation, assuming a fixed money supply and no economic growth. However, the the key here is that the government, the federal government anyways, is spending money regardless of the tax break. Anytime the government writes a check, that's a little bit more money floating around; anytime the government collects some money, such as taxes, there's that much less money to be had. Every tax break causes the money supply to increase more relative to if the tax break did not exist, causing more inflation (or less deflation, if that were the case). If the government spent exactly as much as it taxed, then there would be... actually deflation, because the economy is growing. This is the basics of fiscal policy.

There's also the monetary policy, which is when the federal reserve does this on purpose. The general principle is the same, but instead it spends its money buying bonds and gets its money selling those bonds, and creates a bunch of rules about where banks keep their money so it always has some money on hand.

So, in this desert there would be no inflation or deflation, you say. Let’s say we use gold coins there. Wouldn’t we have an inflation if we find a gold mine there and everybody start digging up gold? You are missing the fact that the money printing is not driven only by government spending. It is driven primarily by the monetary policy (in the hands of the FED) and to some extent by the government debt. You have knowledge gaps on a very basic level. The idea that taxation stops inflation is absolutely ridiculous. It would mean that countries with low taxes have very high inflation and this is not the case. It would also means that the inflation should be constant and in struct correlation with the taxes. Both statements are completely false and very easily provable by quick fact check. The only things taxes do are: misplacing capital and stopping economic growth, which may be the same thing arguably

> It is driven primarily by the monetary policy

Yeah, that's why I mentioned the fed.

> It would mean that countries with low taxes have very high inflation and this is not the case.

It's about the total balance of government spending and taxes. The point being made is that tax breaks have the same effect as government spending. Recall that I was replying to

> Tax breaks, i.e. my money not being in your pocket means that they are stolen?

The government writing someone a million dollar check and the government giving someone a million dollar tax break (assuming they pay at least a million in taxes), contribute to inflation by increasing the money supply by a million dollars than it would be otherwise. Yes, this federal reserve is by far a larger driver of inflation, but the government giving this tax break still degrades the value of your money, same as if they wrote a check.

Of course, it is easy to view a tax break as a non-action, but that's exactly why the government gives so many tax breaks. Once you're taxing everyone, you can hand out tax breaks that's the same as handing out money only you can pretend that it's doing nothing.

Think of it as 3 Scenarios:

1) The island government writes a check to everyone except you, increasing their wealth by 50%.

2) The island government taxes just you for 50% of your wealth.

3) The island government taxes everyone 75% of their wealth, grants everyone but you a total tax-break, and you 25 percentage point tax break.

Basically the same result, only in one they say "It was fair, and we handed out a few tax-breaks, what's wrong with letting people keep their money?"

1) wealth is not increased by tax break, only current income is increased

2) if government gives everyone tax break but not me, it means only that the government taxes only me

3) if everyone has 50% more money, there is very high probability that my business will go up A LOT

Seriously, dude, it’s not worthy anymore to try and explain to you very basic stuff. Inflation is not a balance between taxation and spending. All Middle Eastern countries are having huge spending and almost zero taxes. I asked you very simple question and you couldn’t answer.

What bothers me most is why people write about things they have no clue about and clearly haven’t even put a decent thought into it.

Basically what you believe in is that the thieves are controlling the inflation because they get some of the citizens wealth.

> only current income is increased

There's actually lots of taxes that aren't income or sales tax

> if everyone has 50% more money, there is very high probability that my business will go up A LOT

No, you'll be getting twice the money, but the money is worth half as much.

> Inflation is not a balance between taxation and spending.

It is for the US federal government.

> All Middle Eastern countries are having huge spending and almost zero taxes.

Those countries peg their currency to the dollar. Their money doesn't come from taxes, but instead from state oil companies. These countries aren't as free to hand out money like the US. If enough people tried to exchange their Saudi Riyals for dollars quick enough, and the Saudi government couldn't gather US dollars quick enough, their currency would very quickly collapse.

The Mars walk is just 3 years away baby!

The best part about this answer is it's always true.

Related: GenAI, Cold fusion

Yup, and FSD.

3 months maybe, 6 months definitely.

What do you think NASA does with the money? Is doesn't build a NASA house for its NASA babies.

> the usual. Just like the astronauts to Mars thing will just shovel your money that might have gone to NASA into Musk's pocket.

The difference is that Musk can do twice as much for 1/10 what Nasa thinks the program will cost, which is never what the program will actually cost, and Musk will do it in half that time to boot.

The guy is an unhinged manchild, but if what you care about is having your money well spend and getting to Mars as cheaply as possible, he's exactly who you're looking for.

I think you meant to type SpaceX. Which works as well as it does partly because Musk is kept at a careful length from the controls...

Do you have inside knowledge or a reputable source that he is kept at a distance from the controls? How much control does he have as the CEO?

I'm sure you are claiming that the founder, CEO, and controlling shareholder is "kept a careful length from the controls" because you have detailed first hand knowledge of the internal operations of Space, right?

> if what you care about is having your money well spend and getting to Mars as cheaply as possible, he's exactly who you're looking for.

I do find impressive that SpaceX engineers figured out reusable rockets and now we can send things more cheaply out to orbit. But in all seriousness, should we care about getting to Mars cheaply? Or do people care because Musk came along to convince them (and the US government) to invest in this venture of his?

Your tax dollars are the customer.

what's the difference

Not all money making schemes involve the military.

Mostly benefiting the fossil fuel industry. How are they going to power this? Gas is the only option that can be implemented within single years. And this is going to need a lot of power.

Who cares about the planet, anyway.

For $500bn they can build a nuclear power plant dedicated to these data centres

They can build a couple. With nuclear money is rarely the issue. It is that it takes forever because reasons.

It's not like the current admin respects the rule of law anyways...

Ah so it's commissioned in 2040 and renewables already made it obsolete.

Trump just rescinded licenses for offshore wind farms via an EO. We're fucking cooked (and I mean this literally)

Before downvoting the OP and, for more information, see:

https://apnews.com/article/wind-energy-offshore-turbines-tru...

https://www.utilitydive.com/news/trump-offshore-wind-leasing...

> Offshore wind is among the sources of new power generation that will cost the most, at about $100 per megawatt hour for new projects connecting to the grid in 2028, according to estimates from the Energy Information Administration. That includes tax credits under the Inflation Reduction Act, which reduces the cost of renewable technologies. But onshore wind is one of the cheapest sources, at about $31 on average for new projects.

I mean, I can see how numbers wise this decision makes sense.

You need to stop this nonsense. Pollution is a long term problem, but it does not mean it is productive to do what Germany has done and cease development.

You need to stop this nonsense. The path we were on, that Trump has already overthrown, was nothing like Germany's.

Pollution and climate change are two separate problems (both linked to the fossil fuel industry).

There probably will be a clause of mandatory consumption of a given percentage of power generated from coal ensuring continued coal generation of a given minimum providing excellent talking-points for broadcasting to the incumbent's base.

> Who cares about the planet, anyway.

Maybe at some point they are going to AI themselves out of climate change. Well.. except for the part where they don’t believe in man-escalated climate change.