qayxc

> That sounds ridiculous though, no? Look at how much forest the country has.

The British isles where once covered in forest, too. Then they decided to build a big fleet. Then came the industrial revolution. Now they're down to some 13% of forest...

Yes, Finland might be one of most forested countries in the world (by percentage of land area) now, but history shows that can change quickly if one is not careful about it. Death of a country's forest by a thousand cuts, if you will :)

To an outsider this might sound like an overreaction, but it has to be taken seriously - keep in mind that Finland is one of Europe's premiere locations for grand scale datacentre buildout. Cool climate, lots of available land, politically stable, easy access to cheap and clean power; many very good reasons to assume that this will not be the last datacentre being built there. So yeah, stories like this concern people.

Very good question!

The problem assembly programmers had with compilers wasn't that they wouldn't understand the generated machine code - it was that the generated code was often bad (as in unoptimized and bloated) and didn't justify the resources required to run the compiler. As hardware got more complex and powerful, compiler got much better, and operating systems became the primary layer between hardware and applications, assembly became more niche. Nowadays it's pretty challenging to beat the compiler at generating efficient assembly and writing a program in assembly would consist primarily of OS and library API calls anyway.

There are definitely parallels there. The only obvious difference I can see is that both high level language code and assembly are unequivocal in their meaning (as in you can determine what gets executed), whereas natural language is not.

You vastly underestimate the capabilities of a bored kid. The barrier of entry for setting up OpenClaw is not reasonably lower than reading a tutorial and running malware builders. But yes, I agree that scale is an issue.

As for your question - the same way you apply restrictions to any external system or user. If you don't do that - that's on you. Same category as driving drunk, playing with guns, making explosives in your garage, you name it. The danger is still not the model itself - it's the access to systems that you provide it without any checks or safety barriers.

Sure, but that's a problem of regulating and controlling the production and rollout of superpowered exoskeletons, not an issue of of terrorising limbless villains. Because, guess what, the same dangerous tech can be used to turn squirrels into dangerous monsters, too. So you see where the actual problem is. Giving full unchecked access to systems to any random person would be considered reckless and foolish. Replace person with AI model and it's called the future...

And therein lies the problem. I listed all the things that differentiate cars from current AI models: cars require a license to drive, they are subject to heavy regulation (insurance, registration, inspections, etc.), there's road signs, police, incident statistics, recalls in case of defects etc. etc. NONE of that is currently in place for AI models and the systems surrounding them. So the analogy is flawed on every level. I don't buy the convenience is just too appealing narrative when your own analogy clearly demonstrates what is required in order to roll out dangerous technology to the masses, while NONE of that is in place in the context of AI models.

I'm sick of this strawman nonsense. You are attacking a false analogy. Discuss the actual meat of the argument instead.

The analogy would be more fitting if you had said "cars don't kill people if every drivers has proven skills, never drives impaired, keeps the speed in line with weather and road conditions and stays on actual roadways". You know, like everyone should, regardless of whether they're driving a high performance sports car.

To keep with the analogy: cars have seatbelts, airbags, ABS, ESP, lights, horns, crumple zones, emergency braking systems, roads have speed limits, there are traffic stops, insurance, regular inspections (not in all countries), etc. etc.

So what's unsafe here? The car or roads without speed limits, complete lack of safety measures (both active and passive), absence of any supervision and no insurance? That's the problem. It's not the models themselves - they can spit out tokens by the billions, there's no risk there.

You wouldn't give full access to your phone, your computers, your house keys and your credit cards to any stranger on the street now, would you? How is it then, that people act all surprised when a non-deterministic machine that's optimised to achieve goals while taking all the shortcuts it can, suddenly uses the tools handed to it in unexpected ways? That's a failure on the operator's side, not an inherent danger within of the model.

When put to the test in real-world environment, the capabilities don't look as impressive as benchmarks and synthetic tests might indicate. So doubts about actual spatial reasoning capabilities remain.

> If you give the model access to filesystem and bash that technically unlocks all computer use, so how are you going to control that?

The same way we've done it since machines became multi-user: boring old system access restrictions. Nothing fancy, nothing radical, just good old minimal access rights required to perform a defined set of whitelisted operations.

> It's a ridiculous premise that just because the harness is "normal code" that means we can control the AI.

What is it then? Is not just a program that takes model output, parses it and performs tool calls from the text it receives and then feeds the result back into the model and calls it again with those results? It is normal boring old deterministic code. Many are open source. Look at them. Understand what they do and the apparent "magic" goes away real quick. Harnesses are nothing special.

> AFAIK there isn't that much stopping anyone from spawning an AI swarm and telling it to "spread and go hack everything for the lulz."

Aside from lower cost and possibly greater scale, there's literally NO difference between that and (state sponsored) hacking that has been going on for decades. First it was script kiddies, now it's ML models. The threat model remains the same and so do the counter measures. The real danger is still the harness (and its access to external systems), not the model itself. Restrict the access of the harness and the model can't do anything harmful, see above.

I think we're talking about fundamentally different perspectives here.

I'm not talking about what the LLM does internally. If a metaphor helps, here's one to help you understand what I was trying to get at:

Imagine an evil genius that has no eyes and no limbs. Everything they could learn about the world is presented to them by means of some person describing it to them through words. They have no way of directly interacting with the world and rely on someone executing any action they want to take and describe the outcome to them. Now how dangerous would you say such person would be? How dangerous could they become?

That's what I was getting at. Replace person with LLM (or any other AI system). Replace the person that communicates with an external interface (the harness) and I hope you understand. It doesn't matter whether the LLM could generate the harness by itself - it still is just a bunch of weights sitting in memory being run by an execution engine. That's what it fundamentally is, whether you like it or not. It cannot do anything on its own - and no, not even writing files. It's the execution engine that translates the numeric output into words (or images or video or audio) and the layer above (the harness) that takes that output and interprets it to execute actual actions.

This is not about what you or I think about the internal capabilities of the model - that's irrelevant to the conversation and you can replace LLM with a random token generator and the point still stands. The model itself is incapable of performing actions - from reading files to writing files, to controlling physical machines. All that is and HAS to be done by external interfaces outside the control of the model.

He has a point, though. The LLMs (or any AI for that matter) can't do anything. They can't. It's a function call that ingests symbols and spits out symbols and that's it.

100% of its actual capabilities are tied to harnesses (the actual "agent"), i.e. ordinary deterministic programs that are connected to networks or machines and enable interaction with the outside world. This part (the part that can do harmful things) is fully under human control and all the recent headlines about "agents going rogue" are - as someone (forgot who) put it - akin to strapping a weedwhacker onto a dog and letting it run wild.

The tech itself is safe as far as real-world interactions go - the weakness lies in unchecked access to systems surrounding it. It's not safe at all when it comes to human interaction (lots of ongoing lawsuits demonstrate that), though. There is real danger here, but it has nothing to do with doomsday scenarios ala Terminator or I,Robot and more with total corporate control over the lives, perception of reality, and abilities (like critical thinking) of people.

...poorly? https://youtu.be/ENWVpqtOdRI?t=867

I don't know - real-world tests leave me unconvinced: https://youtu.be/ENWVpqtOdRI?t=867

I've worked in wholesale logistics (backend for inventory management) for many years way back when and let me tell you, cost of ink vs cost of a chip the least of the problems. Barcodes (and QR codes as used here) have two major flaws: reading them is challenging (works fine if the package is undamaged and you TRUST that it contains what it says on the box) and they're still just a number. RFID can help with both these issues and the savings from that can outweigh the additional cost by a lot.

Utility is key and RFID has a lot of advantages over bar codes (and QR codes).

100% agreed, but if you truly want to go down that route, you'll quickly find that basically NOTHING has externalities priced in. If you truly were to do that (which I'm personally not opposed to), prices of pretty much all goods and services would need to be adjusted.

Passive ones can be less than a cent per unit.

The chips are sub-cents per unit in terms of cost and weigh milligrams. RFID chips come in many flavours the ones most used are passive, i.e. basically just a bunch of wires printed on a tiny piece of carrier material. The label on a shirt contains more resources by weight.

I hear this a lot without actual calculations backing up the claim.

First of all, orbital data centre doesn't mean a monolithic 500MW monstrosity orbiting Earth. That's not what the companies actually investigating the idea (Google, Amazon, SpaceX, ...) have in mind.

The real-world implementation would use constellations of Starlink-sized satellites, so we're talking about a ~10kW power envelope. Under ideal orbital conditions (sun synchronous near polar orbit, ~250-450km), you can get away with as little as 40m² of radiator area depending on engineering margins, provided you allow your radiator/coolant temps to be 70°C and above (70°C is what I used for my estimation plus an ideal 400km SSO and a reasonably efficient radiator similar to the improved ETCS on the ISS - no ammonia for the coolant, though see below as to why).

People often cite the ISS and its ~100m² radiators for comparison. The problem with that is that for one, the design operating temperature of the ISS' EATCS is 4°C on the low temp loop and 17°C on the moderate temp loop. That little known fact aside, the current improved ATCS is capable of rejecting 70kW of waste heat - way more than the thermal output a realistic satellite would produce. The active temperature control system (ATCS) of the ISS is designed around humans and science experiments in a 400km 52°N to 52°S orbit, not silicon chips that are allowed to run much hotter than 25°C and sit in SSO.

In short the *actual* thermodynamics work out just fine if you consider real satellite designs instead of sci-fi monstrosities, use realistic operating temperatures for the coolant loop designed around silicon chips, and the desired target orbit.

So when you say "unless we learn how to change those rules the problem will be unsolvable" I'd honestly like to know why my 25+ year old literature on satellite mission and hardware design, Boeing's numbers from "Active Thermal Control System (ATCS) Overview" document, the formulas from NASAs "Guidelines for thermal analysis of spacecraft hardware", this gem from DLR https://doi.org/10.1016/j.asr.2024.11.024 and my calculator are all wrong at the same time.

Sure, I only did some napkin math (still took nearly 45min) using some simplified formulas and crude estimates for the required parameters (I was curious, but not THAT curious), but I always rounded up and used a healthy 2x margin on top of the estimated result.

So please tell me why I'm wrong.

The physics aren't the problem, though. You CAN cool the processors provided you have large enough radiators, a really good system for the coolant flow, and hardware that's fine operating continuously at around 100C or above.

That's basically an engineering challenge more than anything else. Also keep in mind that one of the reasons the ISS for example requires rather large radiators, is that humans typically don't enjoy temperatures above 40C for prolonged periods of time.

Silicon chips on the other can tolerate much higher operating temperatures just fine and even short spikes around 100C can be acceptable for some hardware.

The real challenge is getting launch cost down and securing the rights to pollute polar orbits and frequency bands. Sure, there's plenty of room up there in orbit, but much of that room is not fit for purpose in the context of orbital data centres. Frequency bands for up- and downlinks are also often ignored in the discussion.

Basically I don't see the current issues with the concept in the required engineering or physics - that's solvable in the near term.

I find it much more difficult to believe that there's a way hundreds of thousands of satellites can share the same orbit (i.e. polar orbit in a quite "narrow" band of distances from the surface) without causing major issues.

Same goes for frequency bands, as it wouldn't be just one US company doing this, but dozens from around the world, each with hundreds or thousands of sats and everyone needs a share of the available communication frequencies.

Then there's the logistics of getting the satellites into orbit in the first place. Not just the launch cost, the logistics! Fuel, NOTAMS, maritime exclusion zones, launch licences, etc. The concept would necessitate increasing current launch rates by at least an order of magnitude and cost aside, the logistics and regulatory frameworks aren't ready for this.

That's nit-picking. It's one of the first steps towards this goal. Why would any sane engineer put a complete orbital data centre prototype into space without having any data on how most crucial piece of equipment would perform in the target environment?

First they figure out whether they could get away with using unmodified "off-the-shelf" hardware they already have and how to cool it.

By the time they have a working design, it'll already be a small group of such satellites basically being a fully operational orbital data centre, albeit a rather small one. So yeah, I think there's a point to calling this a test for an "orbital data centre", just like Apollo 1-A was a test for a manned Moon mission despite being suborbital and uncrewed.

Debatable. It's not economically viable under current economics. There's an engineering problem that's solvable in the short term, e.g. the cooling issue.

The economics of mass production of satellites has been demonstrated to work out just fine (see Starlink) and what's missing is the same shift when it comes to space launches. Now I'm not saying that this is going to happen, but that's what the whole purpose of Musk's Starship/Superheavy rocket is.

If - and that's a big if! - the whole Starship concept works out, it'd drastically change the economics of putting satellites in LEO and the whole idea instantly becomes much less stupid.

So basically it's a bet on Starship working as advertised in the very near future. If it does, data centres in space (e.g. in the form of laser-linked satellite swarms) become a compelling alternative to terrestrial data centres. No regulatory issues, no infrastructure problems (power, water, ...), no risk of sabotage or drone strikes (see AWS in Middle East), and no land leases required.

Once you add up all these risks and costs and put them next to the launch cost that SpaceX envisions for Starship, the concept suddenly sounds quite plausible.

I'm sceptical myself - though for very different reasons - but I wouldn't call it an outright scam. It all hinges on one thing: Starship has to work. Cheaply, reliably, and within the next couple of years. If it turns out to be less capable, significantly more expensive, less reliable than F9 and F9 Heavy, or not ready until the mid-2030s, the concept is dead.

True. Physical capabilities can be added with ease. The actual problem isn't the actuators anymore, though. Very useful for a remote controlled puppet ala "Surrogates" (the film), still not quite there yet when it comes to performing even simple tasks autonomously.

Emptying a washing machine located in the cellar/basement, taking the clothes to a room on the ground floor, folding and sorting them before putting them in the closet still is quite the challenge.

Wirelessly hooking such robot up to a data centre running heavy machine learning models isn't a proper solution either, because in contrast to digital agents, latency matters quite a bit in physical interactions.

The thing has no real arms, hands or carry capacity, so what would be the point?

Running up the stairs is the easy part after all. The difficult part in such scenario is lack of orientation due to smoke, obstacles like locked doors or fallen debris, unstable walls and ceilings, locating people and getting them out safely. All of those things require a good amount of autonomy, on-the-spot physical problem solving and judgement.

To this effect, the recent DARPA Triage Challenge, which has its final taking place in November this year, seems like a much more thorough and practical demonstration of current capabilities. Something to look forward to if you're interested in the current state-of-the-art w.r.t. real-world applications of robotics.

> If you were ever trapped on the top floor of a burning building, should we just send a car up there?

Riddle me this then: what role does a robot that has been built to run real fast play in that scenario?

Sure, a good deal of engineering went into this, but the hard part - solving bipedal locomotion - had been done long before that. Apart from showing off it's not really advancing the field and so there's not too much left to be impressed by (honestly, the battery and actuator efficiency that allowed the thing to do a half marathon was MUCH more impressive, at least to me). It seems to me to be more analogous to car tuning.

Absolutely. I was impressed by ASIMO and Atlas. Running fast, though, doesn't seem all that impressive by comparison, but yeah YMMV.

My contacts at various factories and my own eyes after touring said factories tell me that the assembly lines have been mostly run by programmed (no quotes required, just SPS) robots for the past 50 years or so. Welcome back to the 1970s I guess?

The ones I see deployed in construction do stuff like drilling holes in hard to reach places, or laying bricks. None of them look like people, though. Much like the typical household dishwasher hasn't been named Alice or Doris for the past 50+ years, but rather Bosch or Samsung and doesn't use actual hands.

But yeah, humanoid robots do look cool and many movies and books told us to be fearful of them.

It would've been impressive 15 years ago. People seem to forget that first humanoid robot capable of competently walking was showcased by Honda 26 years ago. over 20 years ago, the updated version was capable of running and climbing stairs.

So a purpose-built (look at the thing - running is all it can actually do) two legged fast running robot doesn't seem all that impressive today. Cool? Sure! Useful? Not so much. Impressive? Whatever... it's just showing off at this point, much like Boston Dynamic's Atlas series that went nowhere (in contrast to this machine: this one did half marathon distance:)

Nah. What's often overlooked when seeing these (mostly) propaganda events is the fact that the actual autonomy of these machines is still lacking.

We've had purpose-built machines that outperform every human at specific tasks for over a century now. Just because a subset that's popular these days (the "humanoid" and dog-like variants) invoke some psychological and cultural (we've been primed by media for over 150 years at this point) fears, doesn't make them more capable than they seem.

Sure, the thing can run real fast, but looking at it, that's pretty much all it actually can do. In contrast to a boring old industrial robot, though, being able to win a half-marathon or a 100m dash against any hypothetical human isn't all that useful, no?

I don't see the difference. Is a humanoid robot not a machine? What surprises is that it took so long to achieve this. It took Honda 20 years to get to ASIMO in 2000. I'm not surprised by the fact that 26(!) years later, a humanoid robot finally exceeds human capabilities in basic tasks like running.

Such robot would easily smash the current marathon record as well and the only thing I'd be impressed by, would be the battery capacity tbh.

Developing coding skills is both easier and more difficult than ever at the same time.

50 years ago the main barrier was access to hardware - computers were big, expensive, and not easy to come by for average people.

30 years ago, the biggest hurdle was access to resources: you had to spend a fortune on books that quickly became obsolete or had to wait for your local library to get them for you.

Today, you basically have everything - up-to-date free reference material on pretty much everything in an instant, cheap and accessible hardware (no need for high-end stuff), great online tutorials and interactive courses, message boards, the works.

But then you also have a machine that completely removes the need to put in the effort required to actually learn and hone your skills. Human psychology always seeks shortcuts and more discipline than ever is needed to not give up and have the machine do the thinking for you instead.

Proficient developers are getting pressured into outsourcing their craft and skills to the machine, thus deskilling quickly. Juniors never get the chance to become proficient in the first place - either because they're not getting hired to begin with, or because they succumb to the siren song of the machine...