News for hackers. Stuff that matters.
Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @10:14AM (#13652518)
I'm a member of this team. Please feel free to ask any questions!
Re:Microsoft lets you crash drones and robots in its new rea... by cr0sh on Wednesday February 15, 2017 @11:54AM (#13653377)
Can this be used for more than just multirotor drones? IE - what about other drone aircraft, as well as ground and water-based craft?

I'm currently working thru the Udacity Self-Driving Car Engineer nanodegree; as a part of the first term one of the projects used a Unity3D-based simulator to train a CNN to drive a vehicle.

It would be nice to have a more generalized simulator for this kind of work, to be able to simulate potentially more accurate environments (off road, especially). I could also see interesting work being allowed for underwater ROV environments as well...

As it is, I have considered (for an off-road and more general use) re-tasking TORCS or Trigger Rally (though after just doing some research, it seems like there are ton of these simulators - so maybe one more isn't necessary).

Still - I'm curious whether other environments and vehicles are being considered...

EDIT: It looks like Udacity recently open-sourced their simulator (one thing I have noticed about this course is that sometimes they do things, but don't let the students know about it - that, or maybe I've just been too focused on my work to notice, which is a very real possibility!)

Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @12:12PM (#13653517)
You are absolutely at the right place. We have gone to extra length to design our code so that vehicle specifics are abstracted away behind interfaces. Also things are easily switchable. For example, don't like our physics engine, bring your own! As mentioned in release notes, we are working on other vehicle types. However water based crafts are not on our radar. My guess would be that physics would be more expensive and less accurate in general for water based vehicles.
Re:Microsoft lets you crash drones and robots in its new rea... by LoSboccacc on Wednesday February 15, 2017 @02:23PM (#13654636)
Re:Microsoft lets you crash drones and robots in its new rea... by croon on Wednesday February 15, 2017 @10:24AM (#13652591)
Is the benefit of making it a separate tool related to making better/more APIs or avoiding graphical shortcuts made in video games? Or other benefits?

I realize that Microsoft wouldn't have access to GTA5 source code, but Microsoft Studios could have other games, that have already made substantial work in terms of graphical representation and performance, thus saving time for more API work, if the platform could be reused.

Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @11:28AM (#13653153)
One of our major focus is to use AI in real world. A lot has been done in applying AI to video games but we are still taking baby steps in applying it in real world. The question that we want to answer is this: can we train and test algorithms in simulated world and then use them in real world? This is known as transfer learning problem and its a huge area of research where people are investigating, for example, how can we use 80% simulated data and 20% real world data? The games typically found on commercial market are created for entertainment purposes. For example, game creator might decide to cheat on physics to optimize performance and may configure things so its easy but not too easy. Our focus was rather to pursue good physics, sensor models, vehicle dynamics, hardware-in-loop and so on. Also, we wanted to make this project as open source. Most commercial products typically involve complex chain of 3rd party licenses that almost make it impossible for them to be open source.
Re:Microsoft lets you crash drones and robots in its new rea... by chromanoid on Thursday February 16, 2017 @05:38AM (#13658914)
This is a plugin for the Unreal Engine.
Re:Microsoft lets you crash drones and robots in its new rea... by npsimons on Wednesday February 15, 2017 @10:51AM (#13652819)
No questions, just a big thanks for doing this, and even more thanks for releasing the source code. Also, thanks for doing it in UE4. This could be very useful to me where I work.
Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @11:45AM (#13653310)
We would love to connect and see how community is using this. Join our FB group and we can take it from there: https://www.facebook.com/groups/1225832467530667
Re:Microsoft lets you crash drones and robots in its new rea... by raghavkhanna on Wednesday February 15, 2017 @06:19PM (#13656446)
As someone who works with autonomy for UAVs, previous simulation tools such as gazebo etc. are great for developing and testing control and path planning algos. However, developing, testing and tuning perception techniques such as visual inertial odometry, vision based slam remains and challenge since it typically requires getting your hands on real world data. Photorealistic rendering envs should make it a lot easier for developing good perception algorithms faster. The tools looks very promising, thanks for sharing it and keep up the good work.
Re:Microsoft lets you crash drones and robots in its new rea... by 6stringmerc on Wednesday February 15, 2017 @12:28PM (#13653658)
1. Do you have plans to adapt this to Autonomous Cars anytime in the near future as a potential temporary work-around for navigation / technology shortcomings?

2. Why not port this in partnership with Forza to make race tracks out of real world environments? I had an idea along this line, though it might be way out of scope heh:

https://hackernoon.com/dashcam-google-maps-dev-kit-custom-ne...

Re:Microsoft lets you crash drones and robots in its new rea... by eriknstr on Wednesday February 15, 2017 @10:58AM (#13652883)
When the readme says that the official builds for Linux will arrive in a few weeks, does that mean that it is not yet possible to compile from source also on Linux or is that already possible?
Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @11:37AM (#13653243)
We are internally able to compile the core (AirLib) however there are few rough edges and also we want to make sure Unreal also works as expected on Linux with our plugin.
Re:Microsoft lets you crash drones and robots in its new rea... by eduren on Wednesday February 15, 2017 @10:32AM (#13652657)
Are there plans for implementing weather simulation? Things that might force course adjustments or reduce visibility would be a useful addon to the simulation. I know one team when I was in school would have loved something like that when they were making a drone meant to scout forest fires and navigate through smoke.
Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @11:32AM (#13653180)
Yes, that's on our plan. Stay tuned. Please also feel free to join our FB group if you had like notifications: https://www.facebook.com/groups/1225832467530667
Re:Microsoft lets you crash drones and robots in its new rea... by zardo on Wednesday February 15, 2017 @12:21PM (#13653611)
This would be very important for lighter than air drones. Which... might have much more commerical application anyways due to their long loiter capability.
Re:Microsoft lets you crash drones and robots in its new rea... by jipot on Wednesday February 15, 2017 @12:21PM (#13653607)
I have been scanning through the repo for the past hour and I am blown away. Amazing work! As a recent CS graduate who's dream is to work on drones, virtualization and autonomy, is there any place I can shoot you a resume?
Re:Microsoft lets you crash drones and robots in its new rea... by verelo on Wednesday February 15, 2017 @10:41AM (#13652724)
Can i build the project on a Mac? I only see instructions for Linux and Windows.

Super interested in all things flight (I'm a private pilot and software developer), so I'd just love to get my hands on this.

Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @11:34AM (#13653207)
We are not officially supporting Mac yet, but contributions are welcome :). Our code is cross platform and given enough efforts, I think it can be made available on pretty much any platform. One thing to think about is GPU requirements. Even moderate environments in Unreal can easily require GPUs with at least 4GB RAM.
Re:Microsoft lets you crash drones and robots in its new rea... by cr0sh on Wednesday February 15, 2017 @12:17PM (#13653563)
Many students of the Udacity Nanodegree have found out the hard way that using a stock MBP for CNN training isn't exactly great. Even MBPs with the NVidia 750m struggle (my home workstation is Ubuntu 14.04 LTS with a 750ti SC - and it is only barely adequate).

Honestly, I can't see how you'd be able to run a high-quality 3D sim while training a CNN or other ML model at the same time without something beefier (I haven't tried it yet with my 750ti - but with only 2 gig I have a feeling it wouldn't like it much; it's ok running with the trained model, though).

What students usually (I think) do when all they have is an MBP (especially one with only intel graphics) is train on an AWS GPU instance, then run the trained model locally. This class has been instructive for me in showing that for anything beyond the toy stuff we're playing around with, you really need a beefy ($$$) system. I wish I had the kind of money needed in order to do so (an 8-way 1080 or TitanX box would be sweet - fat luck with me getting it past my wife, though!).

Re:Microsoft lets you crash drones and robots in its new rea... by om42 on Wednesday February 15, 2017 @12:55PM (#13653887)
Awesome work! Wondering if it is possible to get data (telemetry, IMU, etc) from a drone during simulation? And is it possible to have multiple drones in the simulation?
Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Friday February 17, 2017 @04:50AM (#13667132)
Yes, we are supporting to get telemetry as well as ground truth. We also support multiple vehicles but you will need to then build your controller for them or set up multiple HITL/SITL flight controllers.
Re:Microsoft lets you crash drones and robots in its new rea... by om42 on Friday February 17, 2017 @11:56AM (#13669719)
That's awesome! Can I shoot you an email for more information? I'm doing some research where we're trying to improve flight time with IMUs (GPS-denied environments) and AirSim looks like a great environment for us to get more data and train our algorithms!
Re:Microsoft lets you crash drones and robots in its new rea... by kawsper on Wednesday February 15, 2017 @11:45AM (#13653313)
Have you plans of emulating real hardware and running something like Betaflight[0] on top of the emulated hardware?

Betaflight is the most popular firmware for racing drones, so it could be cool to run that, and test different configuration changes.

[0] https://github.com/betaflight/betaflight

Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @12:19PM (#13653592)
Yes! We have been fortunate to find few in-house drone racers and learning the grit behind their skills. Getting SITL/HITL with Betaflight and/or Cleanflight is definitely one of our important goal in coming months.
Re:Microsoft lets you crash drones and robots in its new rea... by nojvek on Thursday February 16, 2017 @01:02AM (#13658046)
First I want to say, "you folks are awesome".

Suppose someone found a great algorithm for drone control based on your simulator and decided to mount a gun on the drone and kill a whole bunch of people.

With all the hotness in AI, rich simulations and algorithms, what steps are you taking to ensure power doesn't fall into the wrong hands?

Re:Microsoft lets you crash drones and robots in its new rea... by state_less on Wednesday February 15, 2017 @11:30AM (#13653163)
Very cool, have you measured the model using real flight data and seeing the discrepancy between the model generated photo/sensor data and the real photo/sensor data? And is their a plan for laser tag mode where multiple vehicles compete with each other for air dominance :)
Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @11:42AM (#13653291)
We are conducting experiments for transfer learning. Stay tuned! For laser tag mode, we will leave it to the community :).
Re:Microsoft lets you crash drones and robots in its new rea... by bootloop on Wednesday February 15, 2017 @03:01PM (#13654925)
Will there be ROS support? So I can use it as a drop-in replacement for Gazebo?
Re:Microsoft lets you crash drones and robots in its new rea... by vowelless on Wednesday February 15, 2017 @03:50PM (#13655236)
From the actual release page :https://www.microsoft.com/en-us/research/project/aerial-info...

Looks like it already does support it.

Re:Microsoft lets you crash drones and robots in its new rea... by NicoJuicy on Wednesday February 15, 2017 @04:30PM (#13655516)
So i build / compiled the software, if i understand i need a PixHawk to make it work.

Is there a setup the run the entire project without hardware?

Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Friday February 17, 2017 @04:52AM (#13667137)
You can and I'd set that up at least once successfully but it isn't easy. I've created a doc for this here: https://github.com/Microsoft/AirSim/blob/master/docs/sitl.md
Re:Microsoft lets you crash drones and robots in its new rea... by wilhil on Wednesday February 15, 2017 @11:34AM (#13653209)
Is this a whole new thing or the evolution of Robotics Studio and the CCR? It looks strangely familiar!
Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @11:44AM (#13653301)
This is not Robotics Studio. It's a simulator for training and testing deep learning/reinforcement learning algorithms for autonomous vehicles. However we are sure community will find other uses :).
Re:Microsoft lets you crash drones and robots in its new rea... by kretash on Wednesday February 15, 2017 @11:36AM (#13653226)
What depth camera are you guys using? and at what framerate and resolution?
Re:Microsoft lets you crash drones and robots in its new rea... by eli_gottlieb on Wednesday February 15, 2017 @11:08AM (#13652980)
Is this Microsoft's answer to OpenAI Universe?
Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @11:15AM (#13653036)
No, we are not OpenAI competitors! In fact, we had very much like to work with OpenAI and think about how do we integrate together to advance the research.
Re:Microsoft lets you crash drones and robots in its new rea... by lawless123 on Wednesday February 15, 2017 @11:36AM (#13653223)
Was about to ask, do think this could be use as one of the open AI environments?
Re:Microsoft lets you crash drones and robots in its new rea... by baq on Wednesday February 15, 2017 @10:57AM (#13652875)
are those 4 views on a single view in the video real time?
Re:Microsoft lets you crash drones and robots in its new rea... by shitals on Wednesday February 15, 2017 @11:38AM (#13653252)
That's correct :). The NVidia 1080 yields FPS of ~23 with all 3 views in real time!