News for hackers. Stuff that matters.
Re:IBM to Acquire the Weather Company by myth_buster on Wednesday October 28, 2015 @03:50PM (#10466882)

  The company also has a popular mobile app, which Mr. Kenny said 
  collected barometric pressure from its users’ phone sensors. 
  Millions of locations are turning in information 96 times a day.
Well, this is news to me.
Re:IBM to Acquire the Weather Company by cryptoz on Wednesday October 28, 2015 @04:17PM (#10467086)
Yes, I think that news is about as big as the news of the acquisition itself. Very exciting news! I have been trying to get the leadership of WU and TWC to do this for the last 3-4 years, and I'm very happy they are finally doing it. In the beginning[1], they made comments suggesting that they thought it was a dumb idea or a waste of time, and that the barometric pressure data from smartphones could never be used in a weather forecast model. I was flabbergasted and could not understand why they wouldn't even consider trying it.

Well, four years later here we are! Finally. We are at the beginning of a revolution in how we understand our atmosphere. There are now billions of internet-connected barometers that can be used in weather models - this should provide a revolution in our ability to forecast short-term mesoscale events like thunderstorms and tornadoes, among other things [2].

[1]: http://gizmodo.com/5851288/why-the-barometer-is-androids-new...

[2]: http://cliffmass.blogspot.com/2015/10/apple-computer-could-r...

Some startups working on this are Sunshine, PressureNet, Weathermob, WeatherSignal:

https://thesunshine.co

https://pressurenet.io

http://weathermob.me

http://weathersignal.com

Re:IBM to Acquire the Weather Company by InvisibleCities on Wednesday October 28, 2015 @05:54PM (#10467707)
Maybe this is a dumb question, but how does the weather app know if the user is outside? Many buildings have HVAC systems that are designed to have positive pressure relative to the general environment, and most phone GPS systems are not accurate enough to tell the difference between being inside a building and outside of it on the sidewalk.
Re:IBM to Acquire the Weather Company by cryptoz on Wednesday October 28, 2015 @05:59PM (#10467732)
That's definitely not a dumb question, it's actually quite nuanced. First I will stress that it's not necessary to know if the phone is inside or outside, because the trends in pressure will be the same. Over a period of 6 hours, the pressure will drop inside if it is dropping outside, and knowing the rate of that drop and the timing of it is key for the weather forecasts.

In the work I've done on this problem we've never been too concerned about inside vs. outside. A much bigger problem is small altitude changes that mess with the exact readings by huge margins, but again, if we're focusing on trends instead of the immediate exact value, we can smooth out the noise relatively easily.

However, to actually answer your question, an app can make a very good guess about whether it is inside or outside. Key sensors and metric for this are: ambient light sensor, humidity sensor, GPS location / barometer. During the day it is normally brighter outside than inside, and during the night is normally brighter inside than outside. So that's clue #1. Clue #2 is to check GPS to see if the user is moving - if you are moving rapidly then you are probably not inside. And if the barometer is rapidly going up and down, but GPS is not changing, then the user is probably inside (elevator, stairs, etc) and that's clue #3.

Again, none of that is necessary to solve for this particular problem, but it's interesting.

Re:IBM to Acquire the Weather Company by kobayashi on Thursday October 29, 2015 @01:26AM (#10469444)
I didn't even know that modern smartphones included humidity sensors and/or barometric sensors.
Re:IBM to Acquire the Weather Company by robzyb on Wednesday October 28, 2015 @09:49PM (#10468712)
That's essentially a statistics / data analytics question.

Statisticians (et. al) are well versed at receiving a whole bunch of messy data and figuring out the value that they want to observe.

From a rough perspective, I think it would look something like this:

1) There will be a large concentration of people reporting pressures that are on the low side, this is the true outside pressure.

2) There will be clusters of people reporting high pressures, and that will be inside buildings.

3) To ignore funny circumstances where large amounts of people are reporting lower pressures than the true outside pressure (I'm not sure why this might be, but this is not my domain of expertise) you can compare it to the pressure that people are seeing around that area, and to official pressure readings.

Re:IBM to Acquire the Weather Company by skimpycompiler on Wednesday October 28, 2015 @05:58PM (#10467729)
Little bit of machine learning and here you go.

The biggest deal is to collect the data. After data arrives, they can then look for any patterns they need. Filter anything that seems wrong.

Re:IBM to Acquire the Weather Company by joe5150 on Wednesday October 28, 2015 @07:54PM (#10468291)
When will phones begin to also incorporate thermometers? And don't most already include hygrometers?
Re:IBM to Acquire the Weather Company by arjunrc on Wednesday October 28, 2015 @08:38PM (#10468474)
I'm also interested in this question. Can someone with this expertise answer?

Is there a reason why we haven't seen a thermometer in a phone so far?

Re:IBM to Acquire the Weather Company by cryptoz on Wednesday October 28, 2015 @08:55PM (#10468538)
The Samsung Galaxy S4 has both a thermometer and a hygrometer (as well as a barometer). Samsung removed both of those sensors for the S5 in order to waterproof the phone - they were able to keep the barometer while still waterproofing, which is neat. Someone else might have to answer how that part of it works (something about watertight and airtight having different physical requirements..?)

In fact, every phone does have a thermometer, but that is not usually accessible by developers and it does not measure the actual air temperature - it's used by the phone to make sure it doesn't get too hot from the battery or CPU. We might not see many app-level thermometers get added to phones because the data is extremely difficult to use. For example, most of the time it will measure the temperature in your pocket or in the room where the phone is - and even then, the reading will be radically affected by what the phone is doing (browsing the web is going to cause a different temperature reading compared to watching a video, compared to stand-by mode).

I do expect that we'll see more hygrometers and thermometers and other sensors make their way into phones as time goes on, but I think the primary reason for the slow take-off is because the data is really really hard to use. That's the thing with the barometer - it's very resilient to noise and even when the data has problems, they are fixable.

Re:IBM to Acquire the Weather Company by arjunrc on Monday November 02, 2015 @07:22AM (#10491458)
Appreciate the detailed response.
Re:IBM to Acquire the Weather Company by jes5199 on Wednesday October 28, 2015 @04:43PM (#10467260)
Installed!
Re:IBM to Acquire the Weather Company by nosnos on Wednesday October 28, 2015 @04:21PM (#10467113)
Uninstalled.
Re:IBM to Acquire the Weather Company by badloginagain on Wednesday October 28, 2015 @04:27PM (#10467145)
Why? Do you find your barometric data to be particularly sensitive?
Re:IBM to Acquire the Weather Company by cryptoz on Wednesday October 28, 2015 @04:30PM (#10467176)
(I'm not the person you are asking, but): People often don't really know how sensitive the barometric pressure data could be, if used for tracking purposes. The barometer in your smartphone can help provide your physical location to within 1 meter, if used for location tracking instead of weather forecasting. If used for weather forecasting, the altitude data is treated as noise and will not be used to find your location.

However, it is now mandated by the FCC that all barometer-carrying smartphones will report your accurate altitude information to 911 emergency response if you make a 911 call from your smartphone. So the location tracking element of the barometer is a very real concern for a lot of people, as it can tell a tracking company what floor of a building you are on at any given moment.

However I think it's quite clear that weather apps, and The Weather Company, are much more interested in the local atmospheric trends than they are about your current floor.

Edit: FCC citation: https://www.fcc.gov/document/fcc-adopts-new-wireless-indoor-...

Re:IBM to Acquire the Weather Company by badloginagain on Wednesday October 28, 2015 @04:41PM (#10467248)
That's a good response to an admittedly pithy comment. I suppose the point I was trying to make was that most people have never considered their devices barometric capabilities. And that the OC was throwing a kneejerk "data-collection is bad" response, and deciding to uninstall because data-collection of any kind is bad.
Re:IBM to Acquire the Weather Company by TeMPOraL on Thursday October 29, 2015 @12:08AM (#10469229)
That's why I'm still inclined to believe "privacy vs. progress of mankind, pick one" soundbite. It's a perfect example of data that's really useful if collected this way at scale. Destroying large-scale data collection because advertisers/Evil Future Government is throwing the baby out with the bathwater.
Re:IBM to Acquire the Weather Company by logicallee on Wednesday October 28, 2015 @08:40PM (#10468479)
> The barometer in your smartphone can help provide your physical location to within 1 meter, if used for location tracking instead of weather forecasting.

How can this possibly be true? Wouldn't the pressure around a couple of square meters simply equalize instantly? You're not saying "within a mile" you're saying "to within 1 meter", and I don't get it.

Re:IBM to Acquire the Weather Company by cryptoz on Wednesday October 28, 2015 @09:00PM (#10468552)
Well first, I'm in the weather industry, not the location-tracking industry, but I will share some of the insight for how this could be done. The barometer won't help with your latitude or longitude usually, because you are correct: the pressure will be approximately equal in those dimensions when you are looking at such a small area.

The barometer is sensitive to altitude changes much much smaller than a meter, probably something like 0.1 meters. So if you are a person, with a smartphone, and you are in a tall building, the barometer can assist with determining which floor you're on. Other sensors would need to be used for calibration (since the changes weather will affect the floor-level) and it doesn't help with the x,y coordinates, just the z. But other location systems are now very very good at x,y and they mostly lack the z.

So by adding the high-accuracy relative altitude capabilities of the barometer to the existing x,y systems, you can get extremely precise locations.

Re:IBM to Acquire the Weather Company by Terribledactyl on Wednesday October 28, 2015 @05:45PM (#10467661)
They should already have GPS info, but I am always weary of unintentional data leakage through presumed sanity of data.
Re:IBM to Acquire the Weather Company by badloginagain on Wednesday October 28, 2015 @07:15PM (#10468097)
If anything the deal with IBM should make you more worried. It's more likely that intelligence agencies have hooks in IBM than the Weather Network, and barometric data can be another heuristic to define who/where you are.
Re:IBM to Acquire the Weather Company by donutz on Wednesday October 28, 2015 @05:02PM (#10467382)
The barometric data is only useful if you know where the reading was taken, right? So you're sending (at least!) pressure + location to their servers. Probably some identifying information as well.
Re:IBM to Acquire the Weather Company by nilved on Wednesday October 28, 2015 @07:50PM (#10468269)
Maybe he just doesn't like his data being stolen regardless of its content
[dead]
Re:IBM to Acquire the Weather Company by nosnos on Wednesday October 28, 2015 @04:29PM (#10467162)
lol
Re:IBM to Acquire the Weather Company by theseatoms on Wednesday October 28, 2015 @04:15PM (#10467073)
Does anyone know if it asks for these permissions on install?
Re:IBM to Acquire the Weather Company by cryptoz on Wednesday October 28, 2015 @04:20PM (#10467104)
Specific permission to access the barometer is not required by Apple or Google. The weather app already does ask permission for location services in general, so yes it is already there. Since they have already been tracking your latitude and longitude, the barometer gives altitude as well - along with current atmospheric pressure trends.
Re:IBM to Acquire the Weather Company by myth_buster on Wednesday October 28, 2015 @04:40PM (#10467241)

  permission for location services in general
This is interesting primarily because for most part I switch-off my location services as I think it's a drain on my battery. The only time it's enabled is when I need the GPS which is quite seldom. So I would assume that it would affect the accuracy of the model if say I live quite a way off from where I work.

Also if I'm spending most of my time indoors in buildings, how much of it would affect the readings as compared to when I'm out in the open?

Re:IBM to Acquire the Weather Company by cryptoz on Wednesday October 28, 2015 @04:44PM (#10467265)
> Also if I'm spending most of my time indoors in buildings, how much of it would affect the readings as compared to when I'm out in the open?

Indoor vs. outdoor isn't really a problem most of the time - the pressure is basically the same inside and outside. A bigger problem is fast altitude changes, which make the absolute reading from the barometer very difficult to interpret safely.

One of the reasons that it took so long for TWC to finally do this is because of the concerns you just voiced: that having this data might actually make the models worse, if they don't take enough care to quality-control it first.

A lot of the research done by scientists (like Cliff Mass at the University of Washington) is how to assimilate this data in a reliable and trusted way. It is likely that The Weather Company does not know how to use the exact pressure readings from the barometer yet (nobody does). Instead, they will probably use the trends over time (say 6 hours), because those are more resilient to noise.