The early results are good. There will be an improvement [1]. How much of an improvement? We don't know yet. Cliff thinks it could be a revolution for some types of forecasts, but we don't have the density of sensors yet to know for sure.
For some comparisons, we sent cliff about 20,000 measurements per hour for his experiment in that paper. We're now delivering about 200,000 per hour to researchers, and that's not nearly enough. Our aim is for 2M per hour and I hope to reach that in the next 2-3 months. Around 1-2M per hour is probably sufficient to provide the "revolution" in accuracy that Cliff predicting.
I should also note that 1-2M per hour is small. We should be able to get closer to 1B per hour, but it'll take a while to ramp up to that kind of scale (that'll be like, every smartphone + watch + car that has a barometer).
The improvements will be slow and steady until we get massive scale and are able to run our models in real time. Until then, it's tough to guess how good the improvements will be.
[1] In the linked paper above, I believe the results were a reduction in root mean square error of about 1deg C for a 3-hour temperature forecast in the Pacific Northwest.
http://weather.rap.ucar.edu/model/?model=gfs
I never tried it on my phone until a couple seconds ago. It looks pretty good!
You can create a shortcut to that URL on android, presumably legacy phones can do the same.
In an unexpected move, The Weather Channel was much more accurate than the US NWS / AccuWeather / etc because of their use of the GFS as a trusted model. More info in this article: http://www.alternet.org/environment/why-almost-everyone-got-...
Edit: And of course, it's my opinion that all the models are currently suffering from a lack of input data. The "...and what should be done" part of the headline, in my opinion, should be answered with, "everyone download PressureNet and contribute your phone's sensor data to weather models", so that we can better predict difficult quantities like heavy snowfall and severe thunderstorms: https://play.google.com/store/apps/details?id=ca.cumulonimbu... ;)