See also:

"Twitter will be the public town square."

"Tax cuts for the wealthy will trickle down to everyone."

"Uber and Lyft will undercut taxis and make rides cheaper for everyone."

etc.

https://news.ycombinator.com/newsguidelines.html

Flipping a pancake is extremely difficult because each pancake is different. I know that these videos must be cherry-picked but to be able to train a Robot to do this just by demonstrating feels like a massive leap.

If instead your choice is your high school bully or a robot, well for now pick the bully. Because that robot isn’t even being vicious and will hurt more.

Rodney Brooks at the MIT AI Lab was a big advocate of something called "series elastic actuators." The idea was was that you didn't allow motors to directly turn robot joints. Instead, all motors acted through some kind of elastic. And the robots could also measure how much resistance they encountered and back off.

MIT had a number of demos of robots that played nicely around fragile humans. I remember video of a grad student stopping a robot arm with their head.

Now, using series elastic actuators will sacrifice some amount of speed or precision. You wouldn't want to do it for industrial robots. And of course, robots also tend to be heavy and made of metal, so even if they move gently, they still pose real risks.

But real progress has been made on these problems.

I'll take the robot, thanks.

So, the moment your system needs this kind of data and that kind of data, oh and btw it needs a few hundreds of thousands of examples of all those kinds of data, that's very clear to me that it's far away from being capable of any kind of generalisation, any kind of learning general behaviour.

So that's "60 difficult, dexterous skills" today, "1,000 by the end of 2024", and when the aliens land to explore the ruins of our civilisation your robot will still be training on the 100,000'th "skill". And still will fall over and die when the aliens greet it.

Solving any one problem with robotic manipulation isn’t all that hard. It takes a lot of trial and error, but in general if the task is constrained you can solve it reliably. The trick is to solve *new* tasks without resorting to all that fine tuning every time. Which is what Russ is claiming here. He’s training an LLM with a corpus of one-off policies for solving specific manipulation tasks, and claiming to get robust ad hoc policies from it for previously unsolved tasks.

If this actually works, it’s pretty important. But that’s the core claim: that he can solve ad hoc tasks without training or hand tuning.

I know it isn’t as open ended as plenty of more important problems in robotics, but this doesn’t strike me as easy at all.

I’ve only dabbled in robotics as an entry level hobbiest, so I really don’t know the answer.

It's hard to point to a single thing that would make "flipping pancakes" intractable, it's sort of the other way around, to usefully flip pancakes in the same way as a person takes a lot of skills chained together.

The "door problem" is a sort of compendium of many real-world skills, identifying the door, understanding its affordances and whether to push or pull it, predicting the trajectory of the door when opened, estimating the mass of the door and applying the right amount of force, understanding if there any springs or pulls on the door and how it must be held to traverse through it. Etc. There are also a ton of things I'm probably missing that are so fundamental one tends to take them for granted, like knowing your own size and that you can't fit through a tiny doorway.

I think you can ramp towards the "door problem" in difficulty by slowly relaxing constraints. A video linked above shows "can flip a pancake successfully with a particular pan (you are already holding) and pancake with a fixed camera and visual markers". Ok, now do it in varying lighting conditions. With no visual markers. With different camera views. Different pancakes. Real pancakes (which are not rigid, and sometimes stick to the pan). Different pans. Now you have to pick up the pan. Use a stove. Different stoves. Identify griddle vs pan and use the right flipping technique. Find everything and do it all in a messy kitchen... eventually you're getting to same ballpark as the "door problem".

(I also ignored that door opening is generally done by mobile robots of a certain weight class which tend to be more expensive than a stationary arm with enough strength to pick up a spatula or hold a pan).

There is a steep difficulty gradient from "works in the lab" to "works under semi-controlled real world conditions" to "works in uncontrolled real-world situations".

The way things are going is sensors (cameras, force, etc) and neural networks. You let the robot try a bunch of ways of opening the door, sometimes it doors itself in the face, eventually it'll figure out good places to stand based on what the door looks like. The more doors you make it try to open hopefully the better it gets at generalising over the task of opening doors. The hacks/heuristics are really still there but the robot is supposed to learn them.

> Surely people don’t stop in front of a door and begin planning things out, rather they seem to go for it and adjust on the fly, is this an approach that won’t work in robotics? Why not?

Yeah, figuring out how to do this is basically "the problem". Most people don't have a sense or feeling of "planning things out" as they open a door because we reached "unconscious competence" at that task. We definitely have predictions of what is going to happen as we start opening the door based on prior experience and our observations so far. If reality diverges from our expectations we will experience surprise, and make some new predictions, etc.

Not sure that anyone has ever studied how people open doors in detail, it'd be interesting. I bet there are a ton of subtle micro behaviours. One that I know is, if you hear kids running in the house it is a good idea to keep a foot planted in front of you as you approach the door, because those guys will absolutely fling or crash doors open right into your face.

  > He’s training an LLM with a corpus of one-off policies for solving specific manipulation tasks, and claiming to get robust ad hoc policies from it for previously unsolved tasks.

In my opinion that really is a good definition of intelligence, and puts this technique at the forefront of machine intelligence.

Noble, but the reality is that once the genie is out of the bottle it will be used by many MBAs to replace people.

Not gonna happen.

Try economically impossible for starters.

Do some napkin math on how much an individual needs to live each month, and then multiply that by how many people you need to support nation-wide. The dollar amounts are staggering, and make our current annual budget (all of it!) look like mere child's play.

We're talking hundreds of billions of dollars every month. It's simply not possible.

Lets say that robots cut US 'labor' costs in half -- from about 10 trillion to 5 trillion. Add in a 5 trillion dollar tax on robot labor and you've got about 14,700 per capita to spend. And costs for businesses don't change.

Someone working an entry level fast food job earned vastly more than that per year.

Let's redo your math using realistic numbers and then we can see what's possible or not.

You have to, otherwise you assemble a perverse incentive to not be productive or work. We want less of that as it is, not more.

Working doesn't just mean "working for the man", it can be anything productive that results in income. Such as making and selling paintings, music, whatever.

However, there cannot be a reality where choosing to not work rewards you with as much or more than those who actually work. We also cannot have a system where people choose to peruse fruitless endeavors simply because they enjoy them, and then still get government payments. Yet, the system you propose will be just that - "I need it more because I'm poor - I'm poor because I choose not to work".

The money used to pay these people is complex, but it is not "free" and is largely supported by the working class. We cannot build incentives for the working class to stop working and subsist entirely off government payments (which come from the rest of the working class, leading to a downward spiral for any such program in terms of costs to the nation).

So realistically, the numbers for some sort of UBI are far, far greater than most people admit in these debates (as all-things government tend to be).

Additionally, if the tax equals the original labor, then there is now a negative incentive for businesses to adopt technology and replace employees as well. Employees are more flexible than a robot, for instance, so if costs are equal the human is the better value from the perspective of most businesses (some excluded such as maybe manufacturing).

Your system falls apart if more and more people decide not doing anything at all is a worthy trade off for a reduction in annual salary.

But that's not the scenario that is being entertained here.

We're talking about a hypothetical future world in which robots with AGI are capable of performing basic labor. Incentivize a human all you want, they will never be able to compete in a labor pool where their competition has no rights and will work 24/7.

That being said, a handout is not the best way to use that money anyway. What it should be used for are stronger safety nets and public services, along the lines of what already exists in western nations.

I'm actually pretty sure if you raised the income tax rate for the top 0.1% and closed corporate tax loopholes, you could get that level of money.

I know you're trying to make a point, but let's be realistic. That's never going to fly... and it's wrong to even think people are going to be ok submitting to technology and the government overlords for a life filed with poverty.

And as long as regulators maintain a competitive marketplace, the plummeting in costs to operate a business will result in lower prices. Labor is the largest cost for most businesses, and businesses that take advantage of new automation typically use it to undercut their competitors prices.

It's simply very possible.

https://www.ers.usda.gov/data-products/ag-and-food-statistic...

They'll get pie in the sky when they die

Very exciting times in robotics!

In the same way, letting an AI actually touch and interact with the world would do wonders in grounding it and making sure it understand concepts beyond just the words or the bytes it was fed during training. GPT4 can already understand images and text, it should not be long until it takes care of videos and we can say AI has vision. This robot from toyota would have touch. We need hearing and smelling and then maybe we will get something resembling a true AGI.

I do think that this is an impressive accomplishment and will lead to valuable commercial products regardless of the AGI issues.

What is that? Most humans have general intelligence, but do other apes? Do dogs? A quick google search suggests that the answer is yes.

If that’s the case, then this approach may indeed yield an AGI but maybe it’s not going to be much better than a golden retriever. Truly excellent at some things, intensely loyal (I hope), but goofy at most other things.

Or maybe just as dogs will never understand calculus, maybe our future AGI will understand things that we will not be able to. It seems to me that there’s a good chance that AGI (and intelligence in general) is a spectrum.

I think it would be surprising if it did. Just as our morality is shaped by our understanding of the world and our capabilities, a future AGI's morality would be shaped by its understanding of the world and capabilities. It might do something that we think is terrible but isn't because we lack the capacity to understand why it's doing what it's doing. I'm thinking about how a dog might think going to the vet is punishment but we are actually doing it out of love.

At the point we're describing "touching" massless particles, we might as well say that's what our retinas do. In terms of novel senses, some kind of gravitometric sense would be neat. LIGO-on-a-chip and all.

See: Pieter Abbeel & Jitendra Malik

https://www.therobotbrains.ai/copy-of-jitendra-malik

"Siri, make me a sandwich"

"You're out of bread, would you like me to bake some bread and then make your sandwich?"

isn't going to happen by 2025.

There will be a gap between “can be done reliably in testing” (probably doable by 2030 even in a randomly chosen house that’s not part of the test set) and “safe enough for consumers, children, and pets to not get injured by the movements of a robot who is strong enough to lift and carry a 40lb bag of flour or 40lb laundry basket”.

That safety gap for “co-working” robots will be very difficult to close enough for the CPSC to be satisfied and also avoid expensive class-action or individual lawsuits.

Ah wait, this isn't Reddit. Well anyhow we'll see I suppose. My money would be more on unfreezing and reheating bread instead, dough rising takes too long for practical sandwich creation.

So maybe it’s not one robot, but 3-4 robots that interface with each other.

The first pancake sequence starts with the pancake half off the baking surface and in danger of falling to the floor. So the device pushes the pancake back onto the baking surface. Good enuf if the pancake is "stiff" (already cooked and rigid). So hope you like your pancakes well-done!

And the pancake-flipper doesn't flip the pancake - it slides the spatula under the pancake and then, just like a robot, rotates the spatula until the pancake drops. In any case, fuggeddabout "eggs over easy".

The first jaunt starts with the vehicle half off the pathway and in danger of colliding with a fence. So the operator manipulates the wheel to guide the contraption back onto the path. Fine enough if the path is "level" (already flattened and firm). So hope you enjoy your rides on well-paved roads!

And the operator doesn't truly control the vehicle - they merely grasp the steering device and then, just like a carriage driver, rotate the wheel until the vehicle alters its direction. In any case, fuggeddabout "tranquil and fragrant travels".

It's exciting to see someone with a bit more deeper knowledge than "Flex tape slap LLM on robotics" featured here, which is majority of Robot Learning work upvoted on HN.

There's more to it than just language learning to be solved before we can have proper embodied agents in the chaotic real world.

Also:

> can make plans and execute those plans

https://github.com/antony0596/auto-gpt

> cannot reason about the world, it does not know or understand the world it just assembles words

They can reason a surprising amount given that they only work with text. With vision/actuation encoding there's potential for far more. Remember, it doesn't have to be smart or conscious as long as it gets the job done with cold hard statistics while just appearing as such. A submarine does not swim but crosses the ocean just the same.

To be pedantic, language is only half the job, as a protocol droid it's C3POs job to understand social protocols, ie etiquette, and knowing what one culture might misunderstand about another and smooth over any faux pas, a task that requires considerable empathy and attention to subtle emotional cues.

i'm very curious to know what it would take to turn a language model designed to respond to prompts, and create something that can proactively interrupt a situation - to realize when it has something to contribute, and keeping its virtual mouth shut otherwise.

So yes, I think he seems a reasonable target.

- You have your Boston Dynamics style humanoid robot at the job site, lets say it's a bricklayer for the purposes of this example.

- You have a human somewhere offsite in an open room with an omnidirectional treadmill floor, and cameras and depth sensors positioned all around the room. They're wearing a Hollywood style motion capture suit and have a VR headset on so they can see what the humanoid robot sees through their cameras.

- The human then acts as they would on site, walking up to the pile of bricks, picking them up, placing them etc. The robot moves in real time on the job site, mimicking whatever action the human performs. I don't know if you'll need props to do this properly or if the muscle memory from years on the job will be enough for the humans to get the motions right.

- You log all the data. You then have someone watch through the video stream, labelling each action that is being performed.

- You run it all through a machine learning algorithm, until you get to the point where you can just send the architectural plan to the robot and essentially say "Build this wall for me".

I believe they will send out a team to digitize the job site.

The team will then create a digital twin.

The architect will map everything into this twin.

The computer system will simulate the build steps.

Robots will be brought onto the job site and get a finetuned model (if necessary) and will build it automatically.

Often nothing as cheap as humans in terms of energy consumption. I doubt LLMs will beat that energy consumption.

Anywhere a human gets involved shit gets expensive. Interestingly robots are pretty human-intensive, from design to programming and maintenance. Only at scale does the “human factor” go down and things become affordable.

Once those two layers interact, you get a whole new dimension of problems you get to deal with. The creation of yet another layer of "management" is inevitable. It's basically management all the way up.

But man, once you get a bunch them aligned and motivated the sky is the limit.

https://www.youtube.com/watch?v=w-CGSQAO5-Q

Thanks, for anyone wondering, it’s totally worth watching.

In software engineering terms, you would gladly pay a full good salary and give a good role to John Carmack to work on your projects 6/7 days per month anyway, because he's John Carmack.

that's the neat part. you don't

This is especially true in the United States or Switzerland from my experience but it applies to virtually any country where the budget that goes in public research is not really enough.

Their first job is really fundraising, and their second job is actually academical.

I've worked for some time with scientists like Michael Graetzel[1], he was very important in the lab, don't get me wrong, he had a brilliant mind and he would still take decisions and give terrific feedback when he was in the labs, but the primary role was fundraising.

You can only have the best lab if you have the most money and can hire more and the best people, that's more important than teaching or directing a lab, sadly.

[1] https://en.wikipedia.org/wiki/Michael_Gr%C3%A4tzel

    - 6.4210 (2023) Robotics Manipulation
    - 6.8210 (2023) Underactuated Robotics

Now the can learn, quickly, perhaps a more dexterous robot with flexible digits etc will become the norm.

Are there any cars out there that have something like a sense of touch and with it can sense the road or things that they crash into?

For everything else isn't touch too late, especially at high speeds? the point is to avoid the crash.

it's (generally) done via comparison of RPM, but one can imagine some future tire that could realistically give feedback about how the weight is actually sitting on the tire, which portions are heating disproportionately, and which parts are experiencing friction anomalies versus the rest of the tire.

that's the direction it would have to kind of go towards to reach the dimensionality and resolution of human proprioception.

that kind of stuff is approached on F1 test rigs and cars. multiple IR laser therms and lidar to compare temperature across the tire at speed as well as judging deformation.

https://wellsve.com/products/electrical-lighting-body-system....

Likewise, maybe self-driving cars need to get a few bumps and scratches to learn not to crash

"American manufacturers use far fewer robots than their competitors, in particular in Taiwan, South Korea and China" [1]. And specialized manufacturing is in a permanent skills shortage. A bit more automation may boost employment and wages for blue-collar workers. Particularly if such kit enables new entrants to better challenge incumbents.

[1] https://www.wsj.com/economy/american-labors-real-problem-it-...

/s But seriously, while I think humans will always find some meaning in work, there will come a day when that work is no longer required. Or at the very least, work will look so different that it will be unrecognizable to us currently. I think "work" will look more and more like "art" in the year 4000 for example.

The idea of a person sacrificing some of their time to enrich the person at the top of a company already sounds bad, but we all accept it must happen so we can afford to live. But when robots and AI takes over labor in a significant way, what will be left for those humans that remain? And let's say we converge on something like a universal basic income to fix this, where will humans find meaning?

I'm leaning more towards socialized housing, food, health care, and public transporation now, at least for the basics.

Everyone should be able to at least meet their minimal basic needs without depending on their economic relevance (which isn't a given for many people over the next 100 years). People who want more than the essentials (a house with larger bedrooms, equipment for their hobbies, travel accommodations) can find paid work in positions that still require a human touch.

And if there aren't enough jobs available for everyone who wants one, then start reducing the standard work week from 40 hours, until everyone who wants can find work.

What's the difference between that and UBI? The state giving the people a magical ticket to redeem for food and shelter or simply giving them food and shelter is the same thing at the end of the day.

Both are equally prone to be inflationary as they require workers to provide the food and shelter without receiving anything real in return. In both cases you need to seize real value from the wealthy population if you want to avoid the inflationary effects.

It's the same thing, just different accounting.

> Both are equally prone to be inflationary as they require workers to provide the food and shelter without receiving anything real in return.

I'm not sure I follow why this would be inflationary, but it's also not what I was suggesting.

Even if this does lead to more inflation though, it ensures the government is responsible for securing food and housing for everyone, rather than something like UBI which might result in people who can't find work not actually being able to afford the essentials.

There's also the tough tradeoff that money is useful for more than just food and shelter, and so there's high incentive for unscrupulous people to try to get you to transfer it to them instead of covering your basic needs: for example, by spending millions on advertising campaigns to get you to stretch yourself to buy things you don't need; or just regular, transparently illegal things like extortion, selling addictive drugs, etc. It's not that people are dumb, it's that other people have incentive to try to get you to do things that are bad for you but good for them, if you have money. If you have shelter that can't be transferred, there's just a lot less incentive to try to abuse you.

Not that money is bad, just if we're trying to run an efficient program that covers basic needs (but not much else), directly providing the things we want to provide is IMO likely to be more efficient than cash transfers and unaccountably hoping that the cash is spent on what the package is aiming to provide.

Straight up giving people money is different from living "on stamps". I know it's fashionable these days to call this "shaming" and I guess it is. My question is, is that a bad thing?

If I wasn't shamed/bullied/pestered into being a proper citizen I don't think I'd become one. Maybe I'm just an inferior breed or something, but my guess is this is true for a lot of folks.

It’s not about paychecks, it’s about being (semi)productive or at least making it seem that you are. It’s easy for humans to slide into idleness no matter how much we deny it and hope somehow everyone will chase their dreams if released from money issues.

If released from money pressure, something else entirely will replace it to excuse us from being productive members of society. (“The system is holding me back!”)

The problem with this is that these are all limited resources. That's the whole reason we invented money: to control access to limited resources. If you have free housing for everyone, who decides who gets a luxury penthouse apartment, and who lives on a 20 m^2 apartment on the 1st floor? Not everyone can live in the penthouse, and there's not enough room for everyone to have a 300 m^2 apartment without making the city sprawling. Same with food: some food costs a lot more to make than others. Same with healthcare: while arguably everyone should get basic care and emergency care, should we give free plastic surgery to everyone who wants it? Money lets people decide what's more important to them.

Subsistence housing and food is by definition not luxury. No one who receives social housing is getting the penthouse suite. Luxury is still pay to play. The government would just provide modest housing and food. Maybe the food is Soylent. Maybe the housing is a studio apartment or a tiny home.

I agree that these are limited resources, but in theory there should be enough housing and food for everyone to survive, and if there isn't, making it a social responsibility can hopefully catalyze action to change that. The government has a lot more sway in getting more housing built (and in fact, are often responsible for policies that prevent enough housing being built) than someone who can't afford market rent for an unglamorous studio.

If AI gets advanced enough that it can make art better than humans (not just technically superior, but artistically superior, more inspired/innovative etc.), then humans will be truly obsolete and doomed, because AIs will be just as sentient as humans, but more capable.

If AIs don't quite reach this point, but still do all the other drudge-work, humans are going to have a hard time still. Some humans will do great, because they'll be doing things that AIs can't, and working by telling AIs what to do, but not-so-smart humans will find themselves without any purpose in life, and only UBI to keep them from rioting.

Without labour? None. Without human workers? All of them. That said, everything we label the humanities has plenty of runway apart from automation.

Humans only own and consume while robots (functionally, the capital of this economy) provide the labour. Everyone is a founder, but instead of co-founders and employees, you just command a team of AIs and robots. You're still trying to innovate and provide a product, as are others to you. But nobody is selling labour per se.

One, I don't assume this will happen. I was responding to the hypothetical of an AGI and economic model without workers. Two, I outright assume AI will take the job of CEO. Otherwise, we're still selling labour. What we can provide AI, novelly, is our preferences. In that hypothetical world, most people would presumably let their AI(s) manage their capital. The same way many aristocrats couldn't tell you anything about how their estates generated income.

One, sure, we can expand the hypothetical envelope to infinity. That wasn't the question.

Two, I'm not sure. Human preferences need not be rational. Given the choice, many would choose the flawed work of a human versus the synthesized product of an AGI.

Three, if we have an AGI that can do everything humans can do we've rendered the question irrelevant. There is no "economy" anymore because everything can be centrally measured, produced and dispatched. By the AGI. (Or it can destroy us.) Either way, we return to production and consumption of non-AI work products being purely voluntary.

Surely you mean the top 1%, who have the capital to invest into robots?

Whomever we empower. The people left out of the loop would die, suffer in quiet subsistence or be folded into the society. The first two are the "people are pests" solution. You see it in resource-rich countries where the population isn't part of the economic machinery.

The last is precedented; see how ancient Sparta dealt its public allotments of land and slaves. Which way various societies go will depend, in part, on decisions we take today. (Should such a future come to pass, which, again, is predicated on massive leaps in robotics and AI.)

Also, a totally fictional one.

That said, as limited as AI and robotics are today, they are nevertheless already sufficient to be extremely dangerous.

Everything will become dirt cheap.

People will play with robots for things like space travel, habitat restoration etc, but it will be more like a passion job.

There will be no more self important rich founders.

The issue is training humans fast enough to stay one step ahead of the robots and the LLMs.

Making someone work a job that could be done cheaper and faster by a robot doesn’t benefit anyone. You’re destroying economic value and wasting the worker’s time.

I always found it weird that outsiders need to dictate what dignity is, since it is an internal state/feeling about own actions.

I’m not against automating high toil (the definition from the Google SRE book) jobs. But people will find dignity in their hobbies if they can’t find it at work. If they can’t find dignity there, they have been failed by society.

I say this as someone who knows he has been directly responsible for eliminating dozens of jobs through automation. Not all the people affected had lives were improved by the job elimination, even if I truly believe our solution made far more peoples lives better and was a win overall.

—- Edits for typos

There’s already value in the human made and hand crafted. Maybe our society just becomes one where we’re left to the retirement of a civilization.

It makes me think of this series: https://www.sbnation.com/a/17776-football

Post-scarcity, post capitalism, post everything.

A lot of these jobs also make people happy, though. That is why the loss of manufacturing in particular was such a blow to Americans. People love manufacturing – to the point that having small-scale manufacturing facilities in one's garage so that one can keep on manufacturing things on the weekend is a dream of many.

Moreover I and they alike would argue it tends to produce better outcomes than the frantic, often brutal thrust of the greater industrial revolution

When first tractor arrived in my village, when grandfather joked that all the landless labourers will dies of hunger now since there won't be work for them. Manual ploughing reduced but a number of other work became routine. These days it's hard to find labour in my village (western UP).

Really - start thinking carefully about what you’re working on

When this video refers to "large behavioral model", where's the "large" part? Where are they getting a similarly "large" amount of behavioral input data? It looks like they have a big lab with a few dozen people modeling behaviors. that's great but it's not like this number of people could produce as much content as all of digital written content.

The core of these advancements are powered by Diffusion Policy [1], which Prof. Shuran Song's lab at Columbia (before she moved recently to Stanford) developed and pioneered. I'd suggest everyone to view the original project website [2], it has a ton of amazing real world challenging experiments.

It was a community favorite for the Best Paper Award at the R:SS conference [3], this year. I remember our lab (and all other learning labs in our robotics department), absolutely dissecting this paper. I know of people who've entirely pivoted away from their projects involving behavior cloning/imitation learning, to this approach, which deals with multi-modal action spaces much more naturally than the aforementioned approaches.

Prof. Song is an absolute rockstar in robotics right now, with several wonderful approaches that scale elegantly to the real world, including IRP [4] (which won Best Paper at R:SS 2022), FlingBot [5], Scaling Up Distilling Down [6] and much more. I recommend checking out her lab website too.

[1] - https://arxiv.org/abs/2303.04137

[2] - https://diffusion-policy.cs.columbia.edu/

[3] - https://roboticsconference.org/program/awards/

[4] - https://irp.cs.columbia.edu/

[5] - https://flingbot.cs.columbia.edu/

[6] - https://www.cs.columbia.edu/~huy/scalingup/

Which lab are you referring to?

Btw, I was at R:SS 2022 and meeting the Skydio autonomy team was one of the highlights of my career as a robotics engineer!

They died before hardware achieved their decades old visions. Not much of this work is net new description, moreso normalizing old descriptions with observation now that we can actually build the old ideas.

> Diffusion Policy: TRI and our collaborators in Professor Song’s group at Columbia University developed a new, powerful generative-AI approach to behavior learning. This approach, called Diffusion Policy, enables easy and rapid behavior teaching from demonstration.

Or so he said…

What makes it work so much better than alternatives mentioned above?

Real world adoption will catalyze with the company brave enough to train these behaviors in.

There was once a cartoon where a robot is instructed to protect it's "primary" from rain and proceeds to swat away all the individual drops of rain from it. A ridiculous parody, but... think about it.

But yeah ... sex robots will absolutely be a thing. Might HAVE to be the thing for the female clientele with insemination abilities to continue the existence of the human race.

Why do CEOs make public statements such as this when the goal is humanoid robots to replace human labor, particularly in countries with declining birthrates?

I’m not too worried about the current generation, but my kids. Don’t know what to tell them TBH.

I guess it’ll all be fine though. We techies tend to have a paranoid streak which isn’t becoming.

I wonder how much force feedback they have. Is that big round squishy thing in the videos sort of like a big finger, with lots of pressure sensors? People have built area pressure sensors before, as far back as the 1980s, but nobody knew what to do with all that data back then. Today, too much sensor data is far less of a problem.

I once took a crack at this problem by equipping a robot arm with an end wrench. The idea was that it would feel around for a bolt head, get the wrench onto it, and turn. A 6 DOF force sensor is enough for that. But this was pre deep learning, and I didn't get very far. although I did build the wrench robot setup.

[1] https://news.ycombinator.com/item?id=37167698