News for hackers. Stuff that matters.
Re:Western civilization runs on the mainframe by mey on Monday August 02, 2010 @11:36AM (#1567900)
These corporations may not see it, because they do not understand it, but having such legacy systems a risk to their operations. The longer they wait to migrate, the more expensive it will be, and the harder it will be to attract talent to the project. There may come a point where they can no longer find anyone to support their hardware or software, and it'll force them to stagnate against their competition who may be able to provide better or different offerings because they are not tied to a system they can't modify.

My understanding of business is that you should never stagnate, and always keep advancing.

People that I've talked to who have worked with such systems indicated that they built giant shims around the systems in other languages to allow them to be adaptable. In theory this gives them a clear interface definition (it may be huge, thousands of method signatures, and possible direct schema access, but it can still be a clear interface) to boot strap a core system replacement in another language and hardware system.

Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @11:40AM (#1567914)
The laws of supply and demand will make sure that there will always be people available to maintain those systems. The price will go up until some 'modern' programmer is tempted enough to open a 3 decades old book on COBOL and JCL.

Your understanding of business is flawed, businesses are advancing by marketing, not by shacking up with every new hot tech that comes along in favour of their old, trusted and reliable tech.

The larger, older and more profitable a business is the more likely that they are going to be ultra-conservative about using new technology at the core of their business.

Eventually we'll get rid of this stuff but I wouldn't bet on it happening in the next two decades, unless someone finds a way to emulate a mainframe on a cluster with similar reliability. And then we still need to get rid of the software.

Re:Western civilization runs on the mainframe by logicalmind on Monday August 02, 2010 @12:03PM (#1567976)
You don't think that there is room for a new high-tech bank to step in and eat the lunch of the big banks? If you started a bank from scratch, built with modern technology, you could provide all kinds of valuable services and adapt to change much more quickly. Pretty much every service provided, particularly on the web, is some application that screenscrapes CICS screens or reads data from VSAM. These applications a tricky to build and take much more time than they should.
Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @12:10PM (#1568001)
Starting a bank from scratch will have you worry about other things than the technology of your IT department, mostly to do with solvency and staying clear of SO.

Sure, at some level the technology will come in to play but nobody will 'start a high tech bank'. For instance, ING direct (one of the efforts of the banking industry to leverage the use of technology) was jump started from the legacy ING systems and as far as I know still ties in to them for all of their back-end stuff.

When I was the 'systems administrator' for the corporate division of a bank (in 1986 or so) we had daily rounds of getting the signatures of the various people responsible that the totals for the night batch runs were correct. That's the sort of responsibility you're talking about, I can't imagine that an MD5 or some digital signature would be sufficient today, it's all about procedures and accountability, emphatically not about technology.

I got one of the first PCs the bank bought on account of our project manager being very forward thinking but nobody ever thought of it as more than a curiosity and something that might be a handy replacement for the terminals.

That attitude has changed a bit but not by much and that's 25 years.

Re:Western civilization runs on the mainframe by logicalmind on Monday August 02, 2010 @12:20PM (#1568043)
Well, nobody would have built another search engine either, but google did it. Part of the problem is the legacy mindset of the higher ups at financial institutions combined with their extremely conservative nature.

I currently work in a fairly large financial institution, giving them a web presence for various services and the backend system is no longer a mainframe but an AS/400. Though it still runs COBOL-based apps, so it's really no different than the standard mainframe.

Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @12:24PM (#1568052)
I'm not sure I follow you on the google link here.

Google built another search engine, which ran on a cluster of PCs and did so more efficiently than the big competitor at the time, AltaVista, which ran (afaik) on a cluster of Dec Alphas. Because google needed lot of horsepower and had a tight budget they came up with some pretty clever packaging ideas for their cluster.

In that sense both these companies took the best route to a solution. Now a search engine is materially different from a bank in that if something goes wrong in a bank the potential damage (not just to the bank, but also to the customers of the bank) is enormous.

A search engine that spits out 'wrong' results is still a search engine. A bank that doesn't know where your money went is no longer a bank (at least, not for very long).

Financial institutions rely heavily on their image of reliability, much more so than any web company.

Re:Western civilization runs on the mainframe by logicalmind on Monday August 02, 2010 @12:43PM (#1568118)
I was speaking more about the mentality that led to the business opportunity that google exploited. More so than the technology. The mentality at that time was that "nobody would build another search engine". Any time I hear someone say "nobody would..." I think about how that mentality allows you to exploit that opportunity.

In this case, I'm referring to banking. You've got an industry that is based almost entirely on legacy systems that has not evolved with the progress of technology. As correctly pointed out, you would not want to modify an existing legacy system to move it to a new technology, which is why nobody has done so. What I'm referring to is a new, from scratch, bank with new technology. One that would allow you to disconnect from the current banking model and allow you to offer services that nobody could provide with today's legacy systems.

This combined with the fact that if you ask any bank customer they will tell you how much they hate their bank and hate the service. Sounds like it's ripe for the pickings with the right investment, investors and people.

Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @12:56PM (#1568159)
I don't think the mentality was that 'nobody would build another search engine', it may have been around where you were at the time but I know plenty of people - myself included - that were wondering about building another search engine. Also, altavista definitely wasn't the only player. Even today, with google as established as it is I'm pretty sure there are at least several hundred people involved in building new search engines. Whether one of them replaces google at any point in time remains to be seen, but it's definitely not classed as a 'hopeless problem'.

As for the banking bit. You are missing my point, nobody builds a bank because of the technology, people found new banks because of the money. If that means adapting old, tried and true technology to jump trough new hoops then that's a cost of business. Bankers are conservative by nature, not because they like it but because it makes very good business sense when you're a bank. It's no coincidence that banks, insurance companies and governments are the areas where the mainframe is still dominant.

I'm really not sure what kind of services you could offer with a 'new' software stack that you could not offer with the old one. Bank transfers are routinely done in seconds between banks, you can pay with everything from credit cards to mobile phones and computers, they seem to have done a remarkable job at adapting to new tech while remaining conservative at heart.

People hate their banks mostly for the costs and for the lack of human interest, rarely if ever because of the lack of technology. In fact, I could do with less technology and better human interaction.

Re:Western civilization runs on the mainframe by logicalmind on Monday August 02, 2010 @02:51PM (#1568591)
I can think of a lot of new products and services a bank could offer. You would have the opportunity to rethink the standard checking/savings account world and hybridize cd's and savings accounts. There has been so little innovation in banking (some would say that it's because of how banks are tied to legacy infrastructure) that a re-think of what exists and what customers want would be very refreshing.

I think the best part would be that because of the legacy technology in place at existing banks, they would have a very hard time making changes to adapt to anything new that would be offered. Just imagine adding a new account type that isn't checking or savings and has new rules. How long would it take for a bank to re-write their backend system to incorporate that change? A long time.

If a disruptor entered banking and did offer this kind of change, I think you'd have banks banging down your door to buy you out, if nothing else.

Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @03:02PM (#1568630)
Interesting. My bank has lots of different account types that aren't checking or savings, and all of them have different rules (and I use a few of those).

There is a big temptation to assume that banks don't do stuff because they're technologically inept, but I don't think that's the case. The bank I worked for was a subsidiary of the Chase Manhattan bank and as far as the hardware went they were cutting edge at the time.

Software driving products that were the banks income was extremely well documented and extremely well protected in terms of changes to the code and who could do what with it. Security was tighter than a gnats ass, both physical and network wise, with round the clock monitoring and so on.

A banks idea of disruption is not the fear of some new tech, banks know that technology only appears to move fast, they can afford the wait until something has proven itself.

A bank fears things like paypal, that obviate the need for banks and their 'products which are the privilege of accessing your own hard earned money'.

Banks have tons of special products in their inventory, the fact that you don't actually see them in the wild is that you are mostly looking at the consumer side of the operation. On the business side the special products are the majority. Then there is all the regulatory business, that too is responsible for an enormous amount of code.

For a newcomer the barrier to entry is not technology, it is simply the vast headstart that existing banks have in terms of knowledge, marketing power and financial resources.

I'm only aware of one bank that started in recent times that was not tied to another bank, and unfortunately it went bust.

Thanks for the exchange by the way, that was interesting. Who knows, maybe one day you'll found that disruptive bank based on high-technology!

Re:Western civilization runs on the mainframe by lkozma on Monday August 02, 2010 @02:09PM (#1568455)
I think there are new banks starting up and some probably see themselves as high tech operations. But in that sector that means J2EE and not Ruby on Rails.
Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @02:19PM (#1568489)
Personally I think that any new banks should use NoSql to store their data and use haskell for their core code.
Re:Western civilization runs on the mainframe by lkozma on Monday August 02, 2010 @03:10PM (#1568667)
I don't disagree, I just meant it from a realistic/pessimistic point of view.
Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @03:46PM (#1568786)
It was in jest, a serious answer would have been to use Erlang, which I think is 'hot' new tech (for those that don't know how old it really is, it's 'new' to them so it must be new to everybody) and a suitable replacement because of the design rather than j2ee.
Re:Western civilization runs on the mainframe by lkozma on Monday August 02, 2010 @04:08PM (#1568871)
I imagined reading that aloud in thick Swedish accent to get in the mood for how serious Erlang is :) (just joking: I'm refering to the famous Erlang demo video)
Re:Western civilization runs on the mainframe by randomtask on Monday August 02, 2010 @03:13PM (#1568675)
Why so?
Re:Western civilization runs on the mainframe by patrickk on Monday August 02, 2010 @05:16PM (#1569121)
Starting a new bank - if only it were that simple.

I actually looked into the requirements for establishing a new bank (not in US) and the requirements were so prohibitive (tens of millions in reserve from day one, virtually obliged that no single shareholder hold more than 5-10% stake), not to mention meeting regulatory requirements like BASEL in Europe and SOX in the US mean that the big banks own the market in many countries, with no fear of swifter moving newcomers "eating their lunch". Not to mention people's resistance to change, especially with something as important as their money. And that is only on the retail banking side. It's even worse in i-banking, with the bigger investment banks making more money than ever on Wall Street, as if the crash of 2008 never happened. They may be crooks, but they have the money and influence to make sure the laws get written in their favour - see https://www.banksimple.net/blog/2010/07/21/banks-strike-back... (there was an article on HN recently around this post.)

Also looked into providing the "valuable services" you mention too, a la Mint.com (no equivalent where I live) and this is something I believe there is certain potential (you will have to build a service that taps into banks online portals and offers more useful features than the banks themselves, e.g. account aggregation, financial product comparisons, tailored financial advice etc.) I think the future is one where virtually ordinary transactions will be done online (paying bills, transferring funds), possibly on smartphones, and where you only go into a branch to get a mortgage or something major - and possibly not even then. Of course you can do all this already online, but the world is still very cash and cheque based outside of the US. It will be a while yet until the majority of people are willing to do 100% of their banking with an institution that is 100% web-based.

Re:Western civilization runs on the mainframe by srveit on Monday August 02, 2010 @04:05PM (#1568863)
It looks like at least one company is trying this: https://www.banksimple.net/about/
Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @04:11PM (#1568878)
That doesn't say anything about their tech, and they're not open for business (yet). It will be interesting to see how they fare.
Re:Western civilization runs on the mainframe by Hoff on Monday August 02, 2010 @11:55AM (#1567951)
Don't presume that you're operating with sufficient domain knowledge to make those judgements, and don't presume that the folks that are making those judgements are ignorant of the options and alternatives.

When you're operating at the juggernaught-scale of business or with industrial-scale production, the rules and the requirements and the competitive landscapes are different.

And the risks can come not from the existing environments - which generally work - but from making the changes to the environment, and from the risks of outages and downtime.

Every vendor that can make even a tangental case for a platform or application or process migration or update or new hardware is also certainly pressing that, too; that's an automatic education for the folks involved.

If you can't process customer orders for a week or a month or can't ship orders or your outage slags a whole production line, you could well end up out of business. I know of places that have had similar meltdowns during updates and during migrations, and it's Not Pretty.

Re:Western civilization runs on the mainframe by fmora on Monday August 02, 2010 @12:15PM (#1568024)
You have to remember that they use these systems to run their business and they've been tested for thousands or millions of hours so the system is extremely reliable. It is naive to suggest to switch to a newer platform or language. Those things are always changing. Also, once you are a seasoned software developer in multiple languages how long do you really think it will take you to learn cobol? Not much I can assure you. You'll probably be familiar with the basics in a couple of days, and maybe even enough to be dangerous. Switching to a platform just for the sake of switching is extremely dangerous and potentially quite costly. Especially when from the customers point of view it really makes no difference whether behind the scenes the business is using cobol, fortran, java, ruby or whatever new fad of the day. If the system is going to behave exactly the same whether you use java or cobol then what is the point of switching?
Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @04:07PM (#1568865)
A million hours is more than a century, I think that is somewhat unlikely.
Re:Western civilization runs on the mainframe by fmora on Monday August 02, 2010 @06:12PM (#1569299)
No is not. Thousands if not millions of people use their systems every year. Every person that uses it counts as time testing the system. Notice how quickly the hours add up if you look at it this way.
Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @10:26PM (#1569884)
Ok, I was thinking in terms of the run-time of the software, not in terms of the time people spent using the system. But that's a valid way to look at it.
Re:Western civilization runs on the mainframe by whatusername on Tuesday August 03, 2010 @01:17AM (#1570208)
You could say the same about Windows XP.
Re:Western civilization runs on the mainframe by rbranson on Monday August 02, 2010 @11:46AM (#1567930)
He's talking about FedEx. They understand that it's a risk to their business continuity. The guy who hijacked a FedEx plane planned to fly it into their datacenter near the airport, which houses the primary mainframe that runs FedEx's most critical business software. He wasn't an IT employee either, so it's fairly common knowledge. AFAIK, they are now running in a hybrid mainframe / client+server model. Most of the package transactions are processed simultaneously. Apparently all of the tracking on FedEx.com uses the client+server system to back it. Obviously they are taking it slowly and carefully (it's like a decade-long project) to avoid any screwups.
Re:Western civilization runs on the mainframe by Hoff on Monday August 02, 2010 @12:08PM (#1567995)
When you're working at this scale, geographic redundancy of DCs is commonplace.

I know of several cases where entire DCs were destroyed, and production continued unabated.

There are commercial platforms that are explicitly designed for this sort of DC redundancy over 800+ km spans. Where you have 400 km between various volumes in your RAIDset.

Re:Western civilization runs on the mainframe by mkr-hn on Monday August 02, 2010 @12:39PM (#1568104)
Do you have links? That's a scenario I've always been curious about, and would like to read about situations where it actually happened.
Re:Western civilization runs on the mainframe by Hoff on Monday August 02, 2010 @01:36PM (#1568324)
http://h71000.www7.hp.com/openvms/brochures/commerzbank/comm...

A moderately technical article on some of the issues that arise, and (for your question) see the end of the following:

http://h71000.www7.hp.com/openvms/journal/v1/disastertol.pdf

Re:Western civilization runs on the mainframe by wallflower on Monday August 02, 2010 @07:47PM (#1569551)
Fascinating.

"At 2 a.m. on Dec. 29, 1999, an active stock market trading day, the audio alert on a UPS system alarmed a security guard on his first day on the job. He pressed the emergency power-off switch, taking down the entire datacenter. Because a disaster-tolerant OpenVMS cluster was in place, the cluster continued to run at the opposite site, with no disruption. The brokerage ran through that stock-trading day on the one site alone, and performed a shadow full-copy operation to restore redundancy in the evening, after trading hours. They procured a replacement for the failed security guard by the next day."

The last sentence.

Re:Western civilization runs on the mainframe by mkr-hn on Monday August 02, 2010 @08:03PM (#1569579)
It's a little disturbing that a security guard had the ability to shut the whole thing down. Is that normal?

I would think they'd keep the power systems under lock and key, or at least under the control of someone who knew what to do when a UPS starts making noise.

Re:Western civilization runs on the mainframe by throw_away on Monday August 02, 2010 @10:03PM (#1569826)
The big red buttons are often there as a safety measure in case someone starts getting electrocuted. At very least, though, these very tempting buttons should be labeled in very dire language that they're not to be pressed unless either a) someone is about to die or b) you actually know what you're doing.
Re:Western civilization runs on the mainframe by jauer on Monday August 02, 2010 @09:58PM (#1569821)
Nope. Data centers are really easy to shut off.

Emergency Power Off buttons are often required by code to shut down breakers and UPS.

I've heard that this is to prevent loss of life if a tech grabs a livewire and/or let the fire department know that power is removed before they start spraying water.

Re:Western civilization runs on the mainframe by dfox on Monday August 02, 2010 @12:18PM (#1568033)
I believe that almost every "Major Package Shipping Company" uses mainframes in their operations, not just FedEx. My friend works as System/z administrator for DHL.