News for hackers. Stuff that matters.
Re:The July Galileo Outage: What happened and why by outworlder on Thursday November 07, 2019 @07:43PM (#21479050)
The most interesting part of the whole write up is that it sheds light on how dependent of the base stations these constellations are. I used to think that, other than sending orbital corrections and the occasional fixes, the constellation would run itself. Apparently this is not the case.

So a localized outage can in fact affect the entire planet. This has also implications in disaster scenarios.

Not sure how applicable this is for GPS.

Re:The July Galileo Outage: What happened and why by nwallin on Thursday November 07, 2019 @10:26PM (#21479754)
The GPS constellation is considerably more robust to a ground station outage. If the ground station goes out, the satellites themselves are a distributed network and they use each others' data to correct themselves. It can operate in this mode for something like 60 days before it's unable to continue correcting itself.

Of course, if the ground station failure was a soft failure, and instead of ceasing to upload, it began uploading incorrect data, the GPS location results would be arbitrarily bad to unusable. It's not clear to me whether the Galileo outage was because of a hard ground station failure or a soft one. But given that one of the contributing factors was that the backup system was not online, it would indicate to me that this was a hard failure that GPS would have been able to correct.

Re:The July Galileo Outage: What happened and why by bradknowles on Monday November 11, 2019 @11:41PM (#21511576)
The other factor for the US GPS system is that there are actually over 24 ground stations, one for each time zone plus at least a couple of spares. A local failure in one time zone doesn’t take out the whole fleet — they can fall over to a backup ground station, and backups for the backups, etc....
Re:The July Galileo Outage: What happened and why by walrus01 on Thursday November 07, 2019 @07:53PM (#21479102)
This is one of the reasons for the ridiculously high perimeter security around Schriever AFB in Colorado, which runs the GPS network, and a number of other critical DoD satellite programs. It's guarded as thoroughly as the Pantex plant in Texas.
Re:The July Galileo Outage: What happened and why by guykk22 on Friday November 08, 2019 @02:02AM (#21480441)
Just wanted to give you a little bit more info. I work for the Spanish company that has developed the OSPF, GMV, and although I'm not currently involved at all in this project, from what i know, the following statement: "around 5% of the Galileo capacity is lost to software problems likely in the Orbit Synchronization Processing Facility (OSPF), run by GMV." is not fully true. The OSPF receives data from sources (From my limited knowledge, time stations in order to generate the ephemeris), and it looks like the problem was not in the OSPF code but in one of these time stations sending the data.
Re:The July Galileo Outage: What happened and why by ahubert on Friday November 08, 2019 @03:03AM (#21480632)
Hi - thanks for this (author here). What I mean is, currently, around 5% of the satellites on average broadcast NAPA status. I did not refer to the big outage with this statement, I mean to ongoing developments. Or perhaps you are referring to the same thing & in that case it is entirely possible that the NAPA flag gets raised because the ephemeris is being generated from stale data. Do you know? Thanks!
Re:The July Galileo Outage: What happened and why by zmk_ on Friday November 08, 2019 @06:10AM (#21481272)
So that's 1 satellite on average. Even though 5% means the same it makes you think the number is much larger if you do not constantly think of the total available.
Re:The July Galileo Outage: What happened and why by ahubert on Thursday November 07, 2019 @03:37PM (#21477011)
author here - if anyone has any questions or remarks, let me know!
Re:The July Galileo Outage: What happened and why by cyanoacry on Thursday November 07, 2019 @04:59PM (#21477766)
Nice article! I used to work in this space (was formerly in charge of GNSS hardware at my company), and it's neat to see Galileo being put through its paces.

I see now that the US gets to take GPS (and the fact that the Air Force pretty much runs it single-handedly) for granted. We are seeing _some_ problems (like the GPS III block upgrade and OCX, ahah), but the org chart seems to be considerably simpler than what Galileo's got to deal with.

Re:The July Galileo Outage: What happened and why by Aloha on Thursday November 07, 2019 @07:35PM (#21479007)
The operational and management aspects strike me as what happens when you have too many cooks in the kitchen - if everyone is responsible for everything, no one is responsible for anything
Re:The July Galileo Outage: What happened and why by michaelt on Thursday November 07, 2019 @04:23PM (#21477439)
I gather the US GPS system has "Autonav mode" where each satellite has long-term ephemerides, allowing 60 days of reduced-precision operation if the ground segment fails. Does Galileo have no such thing, or had the errors propagated into such data by the time someone sounded the alarm?
Re:The July Galileo Outage: What happened and why by makomk on Thursday November 07, 2019 @04:40PM (#21477595)
The 60 day long-term ephemerides were the older sytem. Autonav on the current GPS satellites is considerably fancier - basically, the satellites carry out ranging measurements between each other and compute their own updated clock and ephemeris solutions on the satellites themselves in a distributed fashion. Specced to provide essentially full accuracy for something like six months, minus ionosphere estimates which can only be measured from the ground.

As far as I can tell, Galileo has nothing like this. They have a small buffer of future ephemeris data, but the ground segment failure here was so long that it completely ran out. Their approach seems to be full accuracy or why even bother.

Re:The July Galileo Outage: What happened and why by DuskStar on Friday November 08, 2019 @02:21AM (#21480504)
> As far as I can tell, Galileo has nothing like this. They have a small buffer of future ephemeris data, but the ground segment failure here was so long that it completely ran out. Their approach seems to be full accuracy or why even bother.

It's probably easier to say "our system being down won't be catastrophic" when it's one of 4+ systems rather than one of one, like GPS was.

Also probably not inaccurate to say that the US military cares more about what happens after the nukes start falling than the ESA does.

Re:The July Galileo Outage: What happened and why by ahubert on Friday November 08, 2019 @03:09AM (#21480650)
Hi makomk, (author here)

Indeed, Galileo has nothing like this. The GPS solution sounds exceptionally fancy, and it may go a long way to explaining where all those billions of USD/year go :-) Do you know if it has ever been tried for real? Thanks!

Re:The July Galileo Outage: What happened and why by londons_explore on Friday November 08, 2019 @04:19AM (#21480888)
Doesn't sound that fancy... Orbital calculations are hard maths, but when you've written code to do it, that code can run anywhere, including on the satellite. Very little compute power is required.

Also, satellites are in direct view of one another and can easily receive each others signals. A rather simple software radio could receive the signals from a basic omnidirectional antenna. The only challenge is subtracting out the very strong local signal before digitising, but considering the very strong coding gain in GPS, it should be doable.

Re:The July Galileo Outage: What happened and why by guenthert on Friday November 08, 2019 @09:08PM (#21489075)
> A rather simple software radio

I'd think that would be rather a phase accurate software radio.

Re:The July Galileo Outage: What happened and why by londons_explore on Saturday November 09, 2019 @05:06PM (#21494732)
Aren't all software radios phase accurate...?
Re:The July Galileo Outage: What happened and why by makomk on Saturday November 09, 2019 @07:48PM (#21495626)
I'm not actually sure if it's been used for real, and I don't think it's really been needed - apparently it's rare for GPS ephemeris data uploads to be late by even an hour or two. In principle the idea is that it would be operating all the time and give slight accuracy and integrity monitoring improvements, but that doesn't seem to be a priority.

Incidentally, it seems like the EU is looking at developing their own version of Autonav for the next generation of Galileo, quite possibly with optical rather than radio links between the satellites.

Re:The July Galileo Outage: What happened and why by Aloha on Friday November 08, 2019 @02:39PM (#21485916)
I mean the system was designed as a military tool, with the primary scenario of surviveability being nuclear war, so the level of redundancy seen makes sense to me.
Re:The July Galileo Outage: What happened and why by toomuchtodo on Thursday November 07, 2019 @05:32PM (#21478016)
Any chance we'll see WAAS reference stations with more autonomy for uploading ionsphere correction data directly to the constellation, instead of having to aggregate at master stations, which then forwards to ground uplinks?
Re:The July Galileo Outage: What happened and why by londons_explore on Friday November 08, 2019 @04:14AM (#21480875)
Why would you want that? For slightly more accuracy in a severe ground station outage?

Seems only marginally worth it...

Re:The July Galileo Outage: What happened and why by toomuchtodo on Friday November 08, 2019 @09:40AM (#21482529)
More redundancy, less single points of failure.
Re:The July Galileo Outage: What happened and why by bobviolier on Thursday November 07, 2019 @03:55PM (#21477153)
No question, but thanks for the research and article!
Re:The July Galileo Outage: What happened and why by ahubert on Friday November 08, 2019 @03:04AM (#21480637)
thanks!
Re:The July Galileo Outage: What happened and why by efm on Thursday November 07, 2019 @03:59PM (#21477191)
Thanks for writing this.

The title doesn't mention that Galileo is a Global Navigation System (GPS).

Re:The July Galileo Outage: What happened and why by heydabop on Thursday November 07, 2019 @11:06PM (#21479888)
That's because Galileo is a Global Navigation Satellite System (GNSS).

Saying Galileo is a GPS is like saying Target is a Walmart.

Re:The July Galileo Outage: What happened and why by 9nGQluzmnq3M on Thursday November 07, 2019 @04:31PM (#21477519)
GPS refers specifically to the US system.
Re:The July Galileo Outage: What happened and why by myself248 on Thursday November 07, 2019 @05:13PM (#21477882)
Which is funny, because I've heard it both ways.

For a while, as new systems were coming online, GPS was being used as a generic term, and the US system was being referred to by its original name, NAVSTAR.

But only purists would do that, and everyone else kept saying GPS to refer to the US system, so the new term GNSS was invented as the generic.

I try to say NAVSTAR and GNSS to avoid the ambiguous GPS, just like I say "gridiron" and "soccer" to avoid "football".

Re:The July Galileo Outage: What happened and why by reaperducer on Thursday November 07, 2019 @05:52PM (#21478146)
But only purists would do that

Perhaps the same sort of space nerds who still call Dish Network "Echostar."

/Waves hand.

Re:The July Galileo Outage: What happened and why by myself248 on Thursday November 07, 2019 @06:03PM (#21478238)
Or track the ISS as Zarya.

/waves a trusty Omnipoint phone

Re:The July Galileo Outage: What happened and why by michaelt on Thursday November 07, 2019 @04:47PM (#21477660)
Unfortunately, anyone who knows what the acronym GNSS stands for probably also already knows Galileo is such a system. So changing the title to 'Galileo GNSS outage' probably wouldn't clarify matters :(
Re:The July Galileo Outage: What happened and why by marcosdumay on Thursday November 07, 2019 @06:19PM (#21478391)
"Galileo" can stand for a lot of things.
Re:The July Galileo Outage: What happened and why by pacaro on Thursday November 07, 2019 @04:33PM (#21477533)
Indeed, the second paragraph spells out that this is a Global Navigation Satellite System (GNSS) and places it in context with GPS, GLONASS, and BeiDou
Re:The July Galileo Outage: What happened and why by dzhiurgis on Thursday November 07, 2019 @06:28PM (#21478481)
Is 1cm accuracy part of Galileo system already operational? If so, why our phones and other devices cannot utilise it yet?
Re:The July Galileo Outage: What happened and why by drmpeg on Thursday November 07, 2019 @10:16PM (#21479709)
It's a problem in the US. The Galileo high accuracy service (HAS) is broadcast on 1278.75 MHz (the E6 signal). But the 1240 to 1300 MHz band is not allocated to satellite radionavigation in the US. It's allocated to radar with a secondary allocation for amateur radio.

Because of that allocation, the FCC cannot guarantee that Galileo HAS receivers will not be interfered with.

https://docs.fcc.gov/public/attachments/FCC-18-158A1.pdf

Re:The July Galileo Outage: What happened and why by Rebelgecko on Thursday November 07, 2019 @07:05PM (#21478806)
Galileo has like a dozen different signals and I believe that most of them aren't broadcasting yet. I'm not sure which of those signals is supposed to give 1cm accuracy, but it could be one of the encrypted signals. Another consideration is that the 1cm accuracy would only be possible with fancy DGPS or RTK setups. 1cm accuracy is doable with GPS today, but unless you're a farmer or land surveyor you probably don't own any hardware capable of realizing that accuracy.
Re:The July Galileo Outage: What happened and why by mehrdadn on Thursday November 07, 2019 @07:13PM (#21478877)
I suspect this is an extremely naive question, but why would one signal give a 1cm accuracy and another one not? Shouldn't the satellite just tell you its trajectory + identification info so you can calculate where you are based on that? Or perhaps by signal do you mean the physical electromagnetic properties (frequency etc.) are unsuitable for high accuracy, rather than the actual information content it's supposed to carry?
Re:The July Galileo Outage: What happened and why by petschge on Thursday November 07, 2019 @07:52PM (#21479099)
To get good precision you need to remove the effect of electrons in the ionosphere. These basically change the index of refraction and therefore the propagation speed of the radio waves. The delayed signal makes your receiver think it is further away from the satellite than it actually is. One way to estimate this delay (instead of relying e.g. on the correction signal of a differential-GPS base station close to you) is to receive signals from the satellites at two different frequencies. The time delay is proportional to f^-2. Knowing the frequencies and the delay between the two signals allows you to estimate the electron content and and correct both signal to the undelayed "infinite frequency" arrival time.
Re:The July Galileo Outage: What happened and why by mehrdadn on Thursday November 07, 2019 @08:00PM (#21479123)
Ahh I see! So they're referring to signals of different frequencies? Makes sense, thanks :)
Re:The July Galileo Outage: What happened and why by dzhiurgis on Thursday November 07, 2019 @07:19PM (#21478926)
AFAIK each satellite runs two atomic clocks one being much more accurate. Completely different from GPS and way more advanced.

Unsure how it works singal-wise. I assume not only you need different receiver, but also its hardware has to have much higher clock rate.

Re:The July Galileo Outage: What happened and why by mehrdadn on Thursday November 07, 2019 @07:23PM (#21478944)
Interesting, thanks!
Re:The July Galileo Outage: What happened and why by mistrial9 on Monday November 11, 2019 @11:06AM (#21505908)
orbits in space are geometrically perfect, you can reasonably talk about 1cm accuracy there.. On earth, variations in a long list of things, including physically moving land (due to seismic activity), make "1cm accuracy" something that does not actually hold.. no doubt it has a lot of positive measuring ability .. but the real truth gets messy..
Re:The July Galileo Outage: What happened and why by ken on Thursday November 07, 2019 @05:03PM (#21477802)
The "instantaneous violations" link is dead -- I was hoping to learn what that means!
Re:The July Galileo Outage: What happened and why by ahubert on Friday November 08, 2019 @03:06AM (#21480643)
back up, thanks! It is on our experimental node. What is means is: the Galileo service definition regards a 'dilution of position of >6' to be problematic, but only if this happens for more than 77% of the time. The instantaneous bit is "well it is >6 NOW".
Re:The July Galileo Outage: What happened and why by madengr on Thursday November 07, 2019 @07:04PM (#21478801)
Maybe instead of 24*7 work, some manager thought it was 24/7 work, and sent everyone home.
Re:The July Galileo Outage: What happened and why by jepler on Thursday November 07, 2019 @09:44PM (#21479584)
we shoot for π uptime, or around 22/7...
Re:The July Galileo Outage: What happened and why by toast0 on Thursday November 07, 2019 @09:59PM (#21479642)
Eight 8s
Re:The July Galileo Outage: What happened and why by jtwaleson on Thursday November 07, 2019 @04:06PM (#21477247)
When was the last time GPS had issues like this? I can imagine that during the early years this happened there as well? Or maybe not, as military quality engineering requirements were stricter.
Re:The July Galileo Outage: What happened and why by toyg on Thursday November 07, 2019 @04:20PM (#21477400)
Or rather silence requirements were stricter. The military is not in the business of publicising failure and weaknesses.
Re:The July Galileo Outage: What happened and why by michaelt on Thursday November 07, 2019 @06:44PM (#21478643)
Perhaps in the cold war era.

In the current age, where GPS can be post-processed precisely enough to measure things like land subsidence, there are networks of base stations at fixed locations recording GPS observables and providing historical archives [1] - some operated by US government agencies, some by private companies, some by universities and suchlike.

Due to the use of GPS for things like aircraft navigation, the government also issues 'NANUs' ('Notice Advisory to Navstar Users') and provides an online archive of them [2]. So we can say with some certainty that GPS SATELLITE SVN31 (PRN31) WAS UNUSABLE ON JDAY 349 (15 DEC 97) BEGINNING 1300 ZULU UNTIL JDAY 349 (15 DEC 97) ENDING 2200 ZULU.

Someone who wanted to find the largest recent GPS outage could download the archived NANUs and parse them.

[1] https://cddis.nasa.gov/Data_and_Derived_Products/GNSS/RINEX_... [2] https://navcen.uscg.gov/?Do=gpsArchives&exten=txt

Re:The July Galileo Outage: What happened and why by vel0city on Thursday November 07, 2019 @04:33PM (#21477536)
This is a bit of the interesting aspect to this failure. Official channels have overall been pretty weak on details on what all happened in this error. Most public knowledge of what happened has been gleaned from the signals coming off the satellites themselves which anyone can see. This live data combined with understandings of how the system should work is most of the public knowledge of what has been going on.

I don't have a ton of knowledge into the deep technical aspects to GPS, but I imagine we would probably have some similar clues of an outage of this scale in GPS. Maybe a little less technical details of what is happening behind the scenes, but knowing a high percentage of satellites entering a no guarantee precision mode should be possible.

Re:The July Galileo Outage: What happened and why by myself248 on Thursday November 07, 2019 @07:33PM (#21479002)
Exactly. And the galmon network already does that. (It should really be renamed gnssmon at this point, it's far outgrown its Galileo-watching roots.)

You can run a station yourself if you have a Ublox 8- or 9-series receiver.

Re:The July Galileo Outage: What happened and why by vel0city on Thursday November 07, 2019 @10:10PM (#21479690)
As an amateur radio operator interested in space signals, it definitely intrigues me. Does this galmon project collect data collectively as a wider analysis of the satellite network, or does operating a node only really benefit the group operating a node? I'd love to operate a station to analyze GNSS traffic.
Re:The July Galileo Outage: What happened and why by FeepingCreature on Friday November 08, 2019 @02:05AM (#21480458)
My current impression from reading this article is that SpaceX could create a better GNSS than Galileo in a year as a side project - not because of their cheap launches, but because of their diy development culture. I am stunned by the level of organizational inadequacy of allowing such critical parts of the system to be developed by not just a contractor but a chain of subsidiary contractors, with feedback cycles that move at the speed of corporate communication. My expectation now is that Galileo will never catch up to GPS and will eventually be forgotten.
Re:The July Galileo Outage: What happened and why by blub on Friday November 08, 2019 @03:44AM (#21480774)
I think you're being downvoted because you made a pretty amateurish mistake: on HN you're supposed to say that you yourself can create a better GNSS over the week-end.
Re:The July Galileo Outage: What happened and why by abdulhaq on Friday November 08, 2019 @05:19AM (#21481086)
Galileo is a strategic asset in case of dispute with the US. It is not in a competition with GPS, it simply has to function as designed and continue to operate independently of US desires. The US objected to the construction of the Galileo system.
Re:The July Galileo Outage: What happened and why by creato on Friday November 08, 2019 @07:06AM (#21481521)
The US did not object to the concept of Galileo, the US objected to the EU's choice of frequency, which made it impossible to jam Galileo without jamming GPS. The EU changed the design to avoid this, and I am not aware of any follow-up objections.
Re:The July Galileo Outage: What happened and why by LeoPanthera on Friday November 08, 2019 @02:17AM (#21480490)
Perhaps a good lesson that the opposite of “move fast and break things” is “move slowly - but still break things”.
Re:The July Galileo Outage: What happened and why by abra0 on Friday November 08, 2019 @02:48AM (#21480593)
Move slowly and be unable to fix things fast.