News for hackers. Stuff that matters.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by nodesocket on Sunday January 22, 2017 @09:11PM (#13459163)
I'm not advocating that it is ok, but I am sure these airline systems are super old and wouldn't be surprised if they use IBM DB2 or similar ancient database technology. Moving to the cloud is not a trivial task for these sorts of mission critical antiquated systems.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by eplanit on Sunday January 22, 2017 @10:21PM (#13459490)
It's a legacy story like none other, in fact. The predecessor/origins of SABRE is IBM's Airline Control Program (ACP). When I worked for IBM years ago I heard many stories of how difficult it was to try an modernize to a newer system because of absurd complexity, but just as much because the whole airline industry became so inextricably bound to the legacy:

https://en.wikipedia.org/wiki/IBM_Airline_Control_Program

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by madeofpalk on Sunday January 22, 2017 @11:17PM (#13459715)
I'm currently doing consulting work for That Major Australian Airline and, while I'm rather high up the stack, you do get a sense of the amount of legacy that's built into everything and the monumental effort it would be to migrate all this old stuff to newer infra. I mean, AFAICT there's no database/service to _query_ flights - you have to register a web hook to receive flight data and store it yourself.

New projects are cloud-first, and more and more stuff gets migrated or replaced with equivalents which run in whatever cloud provider. But I can't even imagine how the replacements for all these old legacy services would go down.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by pbarnes_1 on Sunday January 22, 2017 @11:33PM (#13459781)
They used to run on Tandem NonStops with a dual DC setup in Sydney and Newcastle. Those were the days.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by hourislate on Sunday January 22, 2017 @10:45PM (#13459597)
You would be surprised how much has already been migrated away from the old IBM systems (TPF). The big players these days are Lufthansa, Jeppesen, Navitaire, Appolo, Sabre, etc.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by tyingq on Sunday January 22, 2017 @11:51PM (#13459854)
Apollo is now Galileo. Galileo, Sabre, HP Shares, and Amadeus (the actual "Big 4" in this space) all still use the TPF operating system on IBM mainframes. They all are offloading functionality piecemeal, to more modern systems. But TPF is still at the core.

You mentioned Navitaire. They are not using TPF...they use COBOL on Windows (not kidding). They do have a large list of airline customers, but none with a big fleet. Reportedly, it doesn't scale up well enough to serve a large airline.

TPF also lives on in the financial world as well, like at Visa, for example.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by chx on Sunday January 22, 2017 @09:36PM (#13459263)
IBM DB2 is downright futuristic compared to what some of these systems are. SABRE, for example, is probably the granddaddy and horror textbook example of what's commonly referred as "legacy codebase" (although the IRS Master Files written in S/370 assembly could give it a run).
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by gravypod on Sunday January 22, 2017 @10:35PM (#13459565)
There's no way the IRS still uses an S/370 system. Please give me a link so I can have a laugh.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by wpietri on Monday January 23, 2017 @12:07AM (#13459928)
Both the individual and business master files are still written in IBM mainframe assembly language, and are circa 56 years old. See the table on page 4 of this PDF for a list of the oldest systems in operation:

https://oversight.house.gov/wp-content/uploads/2016/05/2016-...

Number three on the the list is The DoD's Strategic Automated Command and Control System, which runs on "an IBM Series/1 Computer—a 1970s computing system—and uses 8-inch floppy disks". No biggie; it just "coordinates the operational functions of the United States’ nuclear forces, such as intercontinental ballistic missiles, nuclear bombers, and tanker support aircrafts."

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by chx on Monday January 23, 2017 @05:34PM (#13466790)
I am wrong then! 56 years? That's the sixties. Yes, https://www.cnet.com/news/irs-trudges-on-with-aging-computer... it's from 1962 according to this 2008 article and the S/370 was introduced in 1970. It's actually an IBM 7074. God above. http://webcache.googleusercontent.com/search?q=cache:J3hXqKq...

This is a PDF from 2016 https://www.irs.gov/pub/irs-utl/scap-pia.pdf

> Standard CFOL Access Protocol (SCAP) is written in COBOL/Customer Information Control System (CICS). SCAP downloads Corporate Files On-Line (CFOL) data from the IBM mainframe at the Enterprise Computing Center, Martinsburg. The CFOL data resides in a variety of formats (packed decimal, 7074, DB2, etc.)

7074 format. weeps

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by raverbashing on Monday January 23, 2017 @05:07AM (#13461106)
Yeah, but it's better for the ICBM control systems to run on 8'floppies, it's much safer (also because of the added friction)
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by 100ideas on Monday January 23, 2017 @03:06AM (#13460647)
Distorts the attack surface these legacy systems present.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by hindsightbias on Monday January 23, 2017 @01:33AM (#13460296)
Code that works, and has for 40 years.

You'll never write something that lasts that long.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by GFischer on Monday January 23, 2017 @03:24PM (#13465646)
I feel really old when I realize some of my code is almost halfway there, and the odds are it WILL reach 40 years :(

You seriously underestimate inertia at financial institutions.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by ams6110 on Sunday January 22, 2017 @10:49PM (#13459620)
New IBM z-series mainframes will still run most System 370 assembly language programs. So it's possible.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by Shivetya on Monday January 23, 2017 @09:47AM (#13462479)
never ever under estimate the government and military's ability to keep old systems operational well past what others consider reasonable.

anecdotal, back in the late 80s I was in the USAF. Our secure communication center was running the first model Burroughs machine to not use tubes. It was that old. It could boot from cards, paper tape, or switches. The machine was older than many people who would be assigned to it. This was closely repeated in the main data center (personnel records, inventory, and such) which had a decade old system that migrated off physical cards by 89 but still took them as images off 5.25 floppy uploaded by PC)

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by nodesocket on Sunday January 22, 2017 @09:37PM (#13459273)
You're right, DB2 is way too hipster and modern for airlines.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by nradov on Monday January 23, 2017 @12:16AM (#13459960)
DB2 isn't "ancient" in any meaningful sense. The first release shipped long ago but IBM has kept it fully up to date since then and it's still competitive with any other relational database for high-volume OLTP.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by mbesto on Sunday January 22, 2017 @09:44PM (#13459308)
> but I am sure these airline systems are super old and wouldn't be surprised if they use IBM DB2 or similar ancient database technology.

Not even. They're still on mainframes.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by sithadmin on Sunday January 22, 2017 @10:30PM (#13459542)
I don't get what you're trying to get at here. DB2 started its life on IBM mainframes (MVS), and is still often run on z/OS or i.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by tyingq on Monday January 23, 2017 @10:37AM (#13462874)
They are still on mainframes running TPF, where DB2 is not an option. The database is TPFDF, a KV datastore.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by compuguy on Monday January 23, 2017 @01:07PM (#13464279)
The financial industry still has software running on mainframes. That some are still writing LOB software in VB6.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by GFischer on Monday January 23, 2017 @03:32PM (#13465718)
Can confirm both - I've been writing LOB software for the financial industry in VB6 up until very recently, and stopped because I switched jobs, not because they've stopped writing them :P

To be honest, VB6 is much better than some of the other stuff they have around.

I now switched to a travel agency and am interacting with the Sabre blue screen systems that are similarly old.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by jimktrains2 on Sunday January 22, 2017 @09:23PM (#13459209)
> Moving to the cloud is not a trivial task for these sorts of mission critical antiquated systems

The question is if moving to _anything_ more modern is less trivial/costly than keeping the current systems which appear to have many single points of failure/

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by shakna on Sunday January 22, 2017 @09:30PM (#13459241)
Often I've seen businesses reason that the failures are cheaper than the upgrades.

That metric is often miscalculated as the initial cost is more of a deal than future revenue.

So they stay with antiquated mainframes and DOS interfaces.

Providing jobs for people like myself who can code in PL1, though I'd much rather take a brick to the face than do it again.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by jimktrains2 on Sunday January 22, 2017 @09:44PM (#13459310)
> Often I've seen businesses reason that the failures are cheaper than the upgrades.

I would love to know what the cost of today's outage in terms of overtime, gate fees, fuel, additional crew, &c.

Delta's cost ~$150MM [1]. That's something on the order of a thousand mid- to senior- level programmers for a year in my area. Even if you allocate a quarter of that cost to computer costs (which I'm betting is a fairly large over estimate), that still leaves a sizable team.

> DOS interfaces.

TUI interfaces can be really, really efficient in terms of navigation and getting around. I rarely see GUI apps that function anywhere near as smoothly or keyboard only.

[1] http://www.datacenterknowledge.com/archives/2016/09/08/delta...

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by et-al on Sunday January 22, 2017 @10:33PM (#13459558)
> TUI interfaces can be really, really efficient in terms of navigation and getting around. I rarely see GUI apps that function anywhere near as smoothly or keyboard only.

TUIs have a steeper learner curve, but I agree that once someone masters the hotkeys for that particular interface, they're much quicker. In addition, depending how old the computer is running the TUI, the staff may not be able to use the computer to browse anything else. ;)

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by shakna on Sunday January 22, 2017 @11:24PM (#13459745)
I absolutely agree that TUIs can be great, especially at user congestion sites, like POS and similar points.

I meant DOS when I said DOS. Where you run into program/system memory split problems when you keep piling on features.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by vacri on Monday January 23, 2017 @01:49AM (#13460355)
> That's something on the order of a thousand mid- to senior- level programmers for a year in my area

I shudder to think of a scheduling program as complex as an airline's ticketing system being run on a bit of software written by a thousand programmers in only one year.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by jimktrains2 on Monday January 23, 2017 @04:20PM (#13466152)
I didn't mean to imply it'd take a single year, but just trying to think about the cost of manpower.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by blhack on Sunday January 22, 2017 @10:59PM (#13459661)
>Delta's cost ~$150MM [1]

Okay, I admit that I have absolutely no idea what software like that costs to build, but surely they could have rebuilt their entire software stack for that, couldn't they?

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by slededit on Sunday January 22, 2017 @11:37PM (#13459801)
You are excluding the costs of bugs such a rewrite would inevitably produce. Normally that is drastically more expensive then Dev time.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by GFischer on Monday January 23, 2017 @03:44PM (#13465815)
No. Software at that scale is humungously expensive.

I worked on quoting an insurance policy core (just the core mind you, not the extras), and for a medium-sized insurance company it would reach that amount of money.

I suspect a complete rewrite would go into the billions.

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by jimktrains2 on Monday January 23, 2017 @04:23PM (#13466179)
Just curious, where did all that money go? How many people for how long?
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by GFischer on Monday January 23, 2017 @05:42PM (#13466851)
The company I worked for ended up not doing it (and regretting it).

It ends up not being that many people, but extremely high paid consultants (200 to 400 dollars an hour), and extremely high licensing costs. Some projects can go on for years, should be 1 to 2 years.

It's extremely profitable and very well paid, one such company, Guidewire, is one of the top 10 best paying employers in the U.S.

http://www.cio.com/article/3064769/careers-staffing/10-best-...

Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by gaius on Monday January 23, 2017 @11:10AM (#13463139)
What? You can trivially run DB2 "in teh cloudz" http://www.ibm.com/analytics/us/en/technology/cloud-data-ser...
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by roscoebeezie on Sunday January 22, 2017 @09:39PM (#13459287)
Yeah a lot of this stuff is on old school mainframes written in assembly. I work in the airline/travel industry and I've seen 28 year old code still running in production.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by DrJokepu on Monday January 23, 2017 @06:29AM (#13461429)
DB2 is actually a fairly decent database. I would be really surprised if DB2 was the weakest link in their technology stack.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by ams6110 on Sunday January 22, 2017 @10:47PM (#13459605)
I wonder if they are still interconnected with 56Kb leased lines?
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by jwilliams on Sunday January 22, 2017 @09:33PM (#13459250)
DB2 isn't that old, relatively. It's in the same league as Oracle or many other mature relational databases. You might be thinking of IMS.
Re:United Airlines Domestic Flights Grounded for 2 Hours by ... by gaius on Monday January 23, 2017 @11:13AM (#13463166)
The funny thing is if you take about 5% of IMS's features and reliability, you get super-trendy MongoDB.