I imagine an excavator, meant to touch and dig through things, and lift things, benefits from force feedback for the same reason VR would.
Have you played those VR sword games? BeatSaber works great because you're cutting through abstract blobs that offer no resistance. But the medieval sword-slashing games feel weird because your sword can't impact your opponent.
I saw a video recently of a quadcopter lifting heavy objects. When it's overloaded, it can't maneuver because all its spare power is spent generating lift to maintain altitude. If the controls had force feedback, the copter's computer could tell you "I'm overloaded, I can't move" by putting maximum resistance on the sticks.
https://x.com/jash_mota/status/1969091992140304703
I think force feedback is key for small excavators, but not really true for 25+ tons excavators. Hence how easy it is for operators to accidentally kill someone with it.
So even if the electric system fails completely, you can still actuate the brakes.
When the car is off or the system loses power, the master cut valves return to their open position, and the brakes are pressurized manually, like an old car without power brakes.
This is "brake-by-wire" to everyone, except apparently you.
If there's a failure of the electrical supply to the brake ECU, or another fault condition occurs, various valves then revert to their normally-open or normally-closed positions to allow hydraulic pressure from the pedal through to the brake cylinders, and isolate the stroke simulator.
Because the engine isn't constantly running and providing a vacuum that can be used to assist with brake force, the system also includes a 'brake accumulator' and pump to boost the brake pressure.
Reference: https://pmmonline.co.uk/technical/blue-prints-insight-into-t...
I don't know for certain, but I would assume that other hybrids and EVs have similar systems to maximise regenerative braking.
One big advantage would be cameras mounted on the boom and rear view cameras, as many machines have obstructed views.
Also, teleoperation is likely to produce lower-quality operation data than hooking up to locally operated excavators. Just a thought.
Do you want to sell Microsoft Teams to the executives or do you want to give joy to the people who actually use the product?
I'd like to get a chance to talk to you and your Dad to get feedback. How do I reach you? My email is contact at useflywheel dot ai
I am curious if something like this is an opportunity for a whole new type of controls and feedback. Since the operator doesn’t have to be in the excavator physically they could take on any position: standing, sitting, lying down, etc. Instead of sending haptic feedback to the joystick it could be sent to a vibrating wrist band. You could hook up the equivalent of a Nintendo Power Glove to have the scoop operated by the operator simulating scooping action. Turning the excavator can be controlled by the operator turning their head and moving it around can be done by the operator walking on an infinite treadmill. Motor strain can be done via color of light or temperature rather than sound. You could have a VR helmet that can also show you a birds eye view from a companion drone, overlay blueprints, show power and water lines, measure depth, etc. I don’t know if it is possible but maybe you could even measure soil composition somehow to show large rocks, soft soil that is dangerous to drive over, inclination angles where the excavator is about to drive, etc.
I imagine skilled operators prefer familiar controls but perhaps there are interesting improvements unlocked by exploring alternatives. It might also fundamentally change how accessible it is for non-professionals to use these machines. I rented an excavator from Home Depot a few years ago to dig a foundation and the learning curve was not shallow. I wonder if a more “natural” interface would help keep people safer.
I'd like to have a chat with you if you're up for it: contact at useflywheel dot ai
I cannot imagine this being useful to me unless the virtual operators cab closely mimicked an actual machine. It would have to have audio from the machine and be on a platform that tilted relative to the real thing. It would also need 270 degrees of monitors with a virtual mirror to see behind. On the front monitor, minimally, would need the to see vertically up and down too.
I also imagine all of this would be more useful to seasoned operators who can do most things on excavators in their sleep (definitely not me lol)
So if we are able to have really good autonomous safety layers to ensure safe movements, and dynamically resize remote teleop windows, you'd make the operator more efficient. So while we stream 360 degree view, we get creative in how we show it.
That's on the vision side. We also stream engine audio, and do haptic feedback.
Takeuchi are interesting! Rare ones to have blades even on bigger sizes - is that why you got one?
Skilled humans have a tendency to fully engage all of their senses during a task. For example, human drivers use their entire field of vision at night even though headlights only illuminate tiny portions of their FoV. I've never operated an excavator, but I would be very surprised if skilled operators only used the portion of their vision immediately around the bucket and not the rest of it for situational awareness.
That said, UI design is a tradeoff. There's a paper that has a nice list of teleoperation design principles [0], which does talk about single windows as a positive. On the other hand, a common principle in older aviation HCI literature is the idea that nothing about the system UI should surprise the human. It's hard to maintain a good idea of the system state when you have it resizing windows automatically.
The hardest thing is going to be making really good autonomous safety layers. It's the most difficult part of building a fully autonomous system anyway. The main advantage of teleop is that you can [supposedly] sidestep having one.
It's like driving a Ford F150 without backup camera. You'd add the backup camera upfront, and not display the back view at the back window.
It's definitely challenging and we're far from something that's perfect. We're iterating towards something that's better everyday.
Yes that is precisely why - makes for a much more versatile machine. TB180FR - it’s med-small, about 10 ton.
Those workflow numbers come from multiple observations at different sites, one of the examples is this: https://www.youtube.com/watch?v=orEsvu1CS64
I wish to talk to you because it's rare sight someone has a Takeuchi - is there a way to connect? My email is contact at useflywheel dot ai
I disagree and here's a couple of reasons why I say that:
1. What am I going to do with the time between releasing control and regaining it from the autonomous control?
2. In that break of workflow my first thought is it will cause a break in my concentration.
3. When I am swinging back from the truck to the trench the bucket is naturally in my control. It seems that in autonomy mode the transition from autonomous to my control would be very unnatural and choppy. I suppose with time it would be okay but man seems to violate the whole smooth is fast concept.
I'll shoot you an email.
It's been over a decade since I last operated an excavator, so grains of salt as usual - but I'd say it should be manual, or at least semi-automated. You need to take care where you unload the bucket on a truck, to avoid its weight distribution being off-center, or to keep various kinds of soil separated on the bed (e.g. we'd load the front with topsoil and fill the rear with the gravel or whatever else was below.
I've actually spent a decent amount of time running an excavator, as my Dad owns a construction / road building company. It was a great summer job!
An important note about the pilot hydraulics is that they _provide feedback to the operator_. I would encourage any system that moves these controls on behalf of a remote human operator or AI to add strain gauges or some other way to measure this force feedback so that this data isn't lost.
The handful of "drive by wire" pieces of equipment that my Dad or other skilled operators in my family have ran were universally panned, because the operators are isolated from this feedback and have a harder time telling when the machine is struggling or when their inputs are not sufficiently smooth. In the automotive world, skilled drivers have similar complaints about fully electronic steering or braking systems, as opposed to traditional vacuum or hydraulic boosting approaches where your foot still has a direct hydraulic connection to the brake pads.