Object manipulation has potential products in dishwashing and vegetable chopping - sufficiently large markets, potential billion $ outcomes for a startup which takes the early mover lead. Two robotic hands that can work in co-ordination just as human hands do. Extremely difficult to solve, but money is there.
1. The cost & technical limitation. The cost, as some of the solution per problem OP suggest include "Lidar, visions and sensor" and all of those thing cost a lot by themselves. On top of that you need good actuators (Harmonic drives etc) for precision, and then good computing unit to handle all of it. Now you also need more power supply, and if it is mobile it needs a battery. Now your robot have a price tag only Arabic princes and well funded academic labs are interested in. And we haven't even touched the cost of try&fail iterative engineering process to make these work. In that process most companies/labs realizes the problem/condition have to be severely limited (run time, indoor/outdoor, general applicability vs made for one & one application only).
2. Human resource. Really, there's not much money in robotics, as a robotics engineer. Software robotics engineer can get a better working condition, security, salary & fulfillment in a software company. Mechanical, EE, embedded, etc is in a similar situation. Most robotics people I know are in for the passion. Application specific development (that is the norm right now) also require very niche knowledge that is hard to find.
The problem is of course until we can solve the problem of making it work at all price isn't a consideration. You can go bankrupt advancing the state of the art which then is enough for someone else to take your work and (repeat the bankruptcy part many times until someone works it out), and finally someone makes a ton of money with their useful robot.
You're right, but also software is weird. The world is absolutely littered with good computers that other people have already paid for, just waiting to be put to use for your project to expand their capabilities. If every new web app required its own little pocket computer to run, that you had to convince your customers to buy and keep in their home, it would be much less profitable to develop web apps.
The world is not littered with idle, high-performance robots waiting around for your motion planning algorithm to kick them into action. That makes it a lot more expensive and a lot less profitable.
You will find the most automated Mc Donalds in the world in Switzerland. Mc Donalds over there has a lot more automation there than it does in the US, both in the cooking and in the order taking.
Either this is because the automation is extremely difficult to do, but is simpler in Switzerland. Or alternatively, this is because in most countries in the world humans are cheaper than the automated system, even after it has already been developed.
As someone who does research on robotics for a living. The problem is not the former, but the latter.
For a consumer, I don't know if the equation 'human labor cost << robots' holds true. It is a hassle to get human labor and there is no scope for time arbitrage. A robot can do the dishwashing job at night for example. Unlike a traditional dishwasher, you don't have to load the utensils. Just leave the utensils in a sink and you are done.
Rethink Robotics, a Rod Brooks startup, tried that, with "Baxter". Company went bust.
There's good, simple commercial hardware for high-volume vegetable chopping and dishwashing, and it can be found in most large commercial kitchens.
There's a huge amount of special purpose machinery in the world. Most of it is pretty dumb. Newer machinery tends to have more sensors and a bit of self-adjustment. Vision systems have become parts of much industrial machinery - they're cheap now. Mostly they're looking for things aligning or looking like a known good object.
ABB's YuMi is perhaps a better example.
Unless real human hands continue to cost less.
If robotics and machine learning can progress to the point where household robotics or general-purpose robots become a thing, there will suddenly be an explosion in demand. Salaries for robotics experts will go through the roof, and all of a sudden, everyone will want to study robotics in school. This could take another 10-20 years to happen though!
Source: I work in an ADV company.
That said it's a problem that steers awfully close to needing a full real AI and that's been a showstopper for loads of potential solutions for decades now.
The ability to manipulate destandardized sizes would be useful but precision manufacturing ate most of the lunch long ago which reduces it to more of a "last mile" task.
Also I'd like to point out that you incorrectly stated it was an "old life pro tip" instead of an "internet law". Anyways, here is the correct information for you.
I worked on this a bit a number of years ago and I thought I had scene matching working pretty well. The problem is I was trying to make it work without actually having two cameras (ie. on a smartphone where binocular cameras were not available at the time and for the most part still aren't). I was hoping to use the accelerometers and other sensors for dead reckoning the camera's position as the user swept it over a scene, but that turned out to be too hard.
[1] https://docs.opencv.org/4.4.0/d2/d85/classcv_1_1StereoSGBM.h... [2] https://www.intelrealsense.com/stereo-depth
People tend to think about machine vision in perfect conditions. This is the problem, me thinks. In practice, you will get a lens flare in most important case, or lose a sensor or get some rain.
Instead of designing solutions starting from most hardcore edgecases (which are common with humans), robotics researchers tend to provide MVP that works in best situation. That is far from end-user environmnent.
This needs global fixes, if you (I don't care) want robots working in real world.
That's a "we need an expert in filter theory" problem. I had four people crash and burn on that problem when we were building a DARPA Grand Challenge vehicle. Combining GPS, accelerometer, gyro, compass, and odometer data to get position is a hard problem. All those sensors are noisy, but in quite different ways. There are off the shelf solutions now, but there were not in 2004. We could not get below 3 degrees of heading noise, and had trouble keeping the sensor map aligned to the real world.
I'm certainly not claiming that I had a fully robust solution but I'm pretty sure lens flares wouldn't be too hard to filter out and there are lots of useful things that need doing in rain-free environments.
The problem with it is that humans don't just use binocular vision - we have a whole model of the world. So for instance, if I see an object, I usually know roughly how big it is supposed to be because I have a conception of "object". I also know that the straight line on both sides of an object is a wall, and that the wall continues behind the object and therefore the object is in front of the wall and the object is closer than the wall.
That means I'm not just working with my binocular vision and it can be deceptive to think that because it works for human vision it will also work well for computer vision.
You get around just fine without it. Got it back after some speciality glasses (PRISM).
Was a complete shock see depth again. Didn’t seem to help anything getting it back. Mostly just trippy. Chairs were amazing to stare at.
But unlike Quake, Tetris isn't providing you with a 3D experience. That makes me wonder how much overlap there is between the two experiences.
Walking outside you see lines and shadows everywhere. Entire perspective changes.
My vision would switch from one eye to the other roughly every 30 seconds. Mostly seamlessly. Took ages to figure this out. Had some minor left brain issues. So writing would go gibberish every 30 seconds. Keeping a patch on right eye meant my vision went black every 30 seconds but my writing was fine when I could see.
Left eye being patched meant writing mostly gibberish. Still had blackness every 30 seconds.
Going on blood thinners resolved the vision going black and gibberish problems.
Prism glasses fixed double vision problems.
Medication for cranial pressure removes the need for prism glasses for about 10 hours. I have several extremely different pairs of glasses depending on what my brain is currently doing.
Inhaled some caustic gas a few years ago. My blood went “sticky” from it. Apparently it activated a Latent blood disorder.
Would take memory tests daily or more to see how “stupid” I was.
Graphing my own cognitive decline and eventual resurgence was “interesting”
Reducing pressure resolves the issue.
Both eyes are very healthy.
At worse it’s +24 diapors, -1 power.
When pressure is good it’s 0-2 diapors +3power.
For awhile it was -6 diapors.
At this point I have glasses for much of the range.
Temple pain tells me when it’s time to switch glasses.
Avoiding caffeine and other stimulants helps.
Taking a Diamox will drop pressure in a couple of hours.
We can see a forest, then a tree, then a leaf. All different "objects", yet related. The ability to not distinguish each blade of grass in my yard and just see it all as "grass" is very useful for noticing and not stepping on what the dog left behind.
Downsides include being less precise at longer distances (an object 1.5 meters away becomes a lot larger if it gets 1 meter closer, an object 150 meters away barely changes) and poor performance on surfaces with fewer features, or really dense features that all look alike (e.g. running across a field while spotting bumps and dips in the grass, or measuring whether sheets of steel are flat or not)
In some cases this doesn't matter - a Roomba doesn't care if it's hard to see things 150 meters away, as rooms are rarely that big.
- Multiaxis singularities: much less of a problem than it used to be. We don't need closed-form solutions any more; we have enough CPU power at the robot to deal with this. You need some additional constraint, like "minimize jerk" when you have too many degrees of freedom.
- Simultaneous Location and Mapping. SLAM for short: Getting much better. Things which explore and return a map are fairly common now. LIDAR helps. So does having a heading gyro with a low drift rate. Available commercially on vacuum cleaners.
- Lost Robot Problem: hard, but in practical situations, markers of some kind, visual or RF, help.
- Object manipulation and haptic feedback: Look up the DARPA manipulation challenge. Getting a key into a lock is still a hard problem. It's embarrassing how bad this is. Part of the problem is that force-sensing wrists are still far too expensive for no good reason. I once built one out of a 6DOF mouse, which is just a spring-loaded thing with optical sensors. Something I was fooling around with before TechShop went down. I have a little robot arm with an end wrench and a force sensing wrist. The idea was to get it to put the wrench around the bolt head by feel. Nice problem because a 6DOF force sensor gives you all the info you can get while holding an end wrench.
- Depth estimation: LIDAR helps. The second Kinect was a boon to robotics. Low-cost 3D LIDAR units are still rare. Amusingly, depth from movement progressed because of the desire to make 3D movies from 2D movies. (Are there still 3D movies being made?)
- Position estimation of moving objects: the military spends a lot of time on this, but doesn't publish much. Behind the staring sensor of a modern all-aspect air to air missile is - something that solves that problem.
- Affordance discovery: Very little progress.
The real problem: solve any of these problems, make very little money. If you're qualified to work on any of these problems, you can go to Google, Apple, etc. and make a lot of money solving easier problems.