klowrey
klowrey on Hacker News
Here's a (hacky) demo of MPC using MuJoCo in the browser:
https://klowrey.github.io/mujoco_wasm/I hacked it using MPPI and it only works on the cartpole model so as to not have to dwell in Javascript too long; just click the 'MPPI Controller' button and you can perturb the model and see it recover.
Vyvanse is dextroamphetamine, while Adderall and Adzenys are mixed dextro- and Levo-.
While there are now generic versions of Vyvanse, they’re unlikely to be as effective until other developers work out the manufacturing kinks.
However, generic extended release dextroamphetamine is generally not in short supply and 10x cheaper than Vyvanse. Might be worth a shot if that’s what worked best for you.
Combined with broadcasting its fast.
Under the hood it’s doing the same thing with a vector of ints (64 bits for bitvectors) and all the bulk manipulation is handled that way so SIMD in inherent as well. Worth a shot.
Woah we can’t ask for the impossible. We just want the robots to _make_ all the things that end up as garbage in the average American home.
https://www.youtube.com/watch?v=xNeZWP5Mx9sdifferent configuration, but electric motors are fine if you get momentum on your side. Humans use their entire range of motion get build up velocity to jump; this is motion control thing.
Static holds. Once you pressurize the cylinder to make it move to a certain position, it can hold that position without using more energy.
This makes sense for quasi-static systems but obviously is a limiting factor for dynamic robots.
Hydraulics shouldn't have any give, as the working fluid is considered "incompressible". Of course in the real world the tubing can expand slightly and there are friction losses, but the reason they went with hydraulics in the first place is they can set a position and not have to use more energy to hold it there (since the cylinders are pressurized).
If the gear ratio on these motors is high, then there can only be faked compliance in the tuned force-torque controllers you mentioned. MIT's little cheetah robot, on the other hand, deliberately used low-gear ratios to keep things naturally squishy if needed. This is the way to go; putting elastic tendons or spring elements seems like a good idea but then you can't actually model the non-linearity well (the 1st order motor becomes a 2nd or higher order system).
We're building software for neuromorphic cameras specifically for robotics. If robots could actually understand motion in completely unconstrained situations, then both optimal control and modern ML techniques would easily see uplift in capability (i.e. things work great in simulation, but you can't get good positions and velocities accurately and at high enough rate in the real world).
Robots already have fast, accurate motors, but their vision systems are like seeing the world through a strobe light.
The detail about end-effector frame is pretty critical as doing this BC with joint angles would not be tractable. You can tell there was a big shift from the RL approaches trying to do very generalizing algorithms to more recent works that are heavily focused on this arms/manipulators because end-effector control enables more flashy results.
Another limiting factor is that data collection is a big problem: not only will you never be sure you've collected enough data, they're collecting data of a human trying to do this work through a janky teleoperation rig. The behavior they're trying to clone is of a human working poorly, which isn't a great source of data! Furthermore limiting the data collection to (typically) 10Hz means that the scene will always have to be quasi-static, and I'm not sure these huge models will speed up enough to actually understand velocity as a 'sufficient statistic' of the underlying dynamics.
Ultimately, it's been frustrating to see so much money dumped into the recent humanoid push using teleop / BC. It's going to hamper the folks actually pursing first-principles thinking.
Have any links you would recommend?
I manage the MuJoCo.jl wrapper, and having the source code wont (naively) help with AD. Internally, there's a number of iterative algorithms that you wouldn't want to automatically differentiate, and if analytical derivatives were apparent they would be in there (the MuJoCo creator was my advisor). As it stands, finite differencing is the best way to extract derivative/gradient information from the underlying physics model of MuJoCo, and is the suggested method.
We had a paper that included MuJoCo (with custom defined adjoints) within Julia's DiffEq framework to learn continuous control policies (https://arxiv.org/abs/2012.06684) which revealed a whole host of other issues, namely that gradients are not all that helpful when your optimization landscape is insanely non-linear. But that's a different problem.
Open to chatting about; Julia is definitely the tool for this kind of work.
There was a free license for individual users, with the non-free license applicable to those who were receiving financial support (i.e. academic / industry researchers, etc.). The individual license was very popular and heavily utilized.
It may be frowned upon, but we communicated to our cats that meowing at the door was not acceptable by discouraging them with a vacuum cleaner. Flip the switch on the vacuum and run the cord under the door to your bed side. Upon meow,
quietly plug the vacuum in to activate -- if the cat senses that you activated the device, the process doesn't work. If "the environment" doesn't want them to meow at the door, they learn quickly.
Of course give them lots of love in the morning and before bed.
For folks interested in Julia and RL, I've been involved in
https://www.lyceum.ml/ a set of tools for continuous control problems like robotics.
It's pretty quick.
After wrapping the Mujoco physics engine in Julia, I did a few reinforcement learning / robotics control projects, combining it with the Flux.jl library for neural nets.
While I never made my research code public, https://openreview.net/forum?id=SyxytxBFDr documents a similar setup to what I use.
Full stack Julia for RL (and trajectory optimization)
https://arxiv.org/abs/1703.02660Disclosure: I contributed to the linked work.
To do simple tasks, you don’t even need neural networks, in short. If you want to do complex tasks, training the neural net isn’t the problem, but a NN can indeed represent more than say a linear policy. The issue is getting the NN to do the complex thing correctly: it’s more of an exploration problem than function approximation problem.