I know it's silly to think that banks would be better than anyone else, but good lord, malware running on machines capable of transferring millions of dollars that's able to send out video feeds from the network without anyone noticing?! Your various IT/Security teams should be absolutely ashamed. And then the banks don't even have to stand up and admit their incompetence publicly; that's a total disgrace.

That's the state of corporate security I guess. I've dealt with corporate IT departments over the years where they put these "processes" in place to mitigate these security issues but it's all a load of rubbish. Filling in forms to tick boxes so that everyone can go home happy pretending there's security going on, when really their network is a leaky sieve.

At one point I saw a release by a 3rd party supplier to a large corporate system that included privilege escalation, blatantly, at the start of a T-SQL script. It was done because the IT department refused to carry out the action on request via the official channel but it was work that needed to be done to complete a project. The 3rd party knew the admins would just be running scripts as SA so they escalated their own account to do what they needed to do later.

I know it's silly to be so frustrated about it, but we've all dealt with crappy banking systems for years, with totally insane security measures; meanwhile hackers can just walk away with millions using a bit of malware.

So where a bank has a risk of an unauthorized financial transaction, there are multiple options to claw that back (or to shift the risk to other parties, notably merchants).

A disclosure, though, of account information is a different case, and here the results can be damaging to the banks and their customers. One instance I'm generally aware of is an increasing number of disclosures pertaining to offshore banking, many uncovered by the the ICIJ (International Consortium of Investigative Journalists: http://www.icij.org/) and the Guardian. Again, the case involves banks, but it's rather more difficult to reverse transactions when it's your client list and balances, or communications, which have spilled.

Many revealed by insiders, as it turns out.

Do you think they'll be getting back the money in this case? Presumably the people involved know enough about the operations to have moved the cash to somewhere out of reach before being exposed.

Not this time: hackers withdrew some of the money from ATMs.

I stand corrected. That's quite impressive and amazing that they had that many people involved and nobody tipped it off.

I would honestly expect it to be a bit better than average. I suppose there are many different types of banks and they all vary. Let's just consider your chain banks like Wells Fargo or BoA, since I'm sure somebody around here has worked at one of those places.

Across the industry it's generally better than corporate IT. A lot better. However, it varies widely by sector.

Companies with trading floors or that interact regularly with traders have the best IT practices in the industry. Banking conglomerates are kind of messy - they combine IT operations for each business and never change anything and the systems don't cooperate.

I remember one such company's backup procedures. At that point they were made up of 13 separate large (regional/national) banks. They were trying to standardize the backup procedures between all the banks and run them from a centralized system. At the time that I got there, the nightly backup process failed every single day for over a year and a half. I didn't even get a computer or working logins to be able to do any work for nearly a month. Anyway, getting it to work involved getting the people responsible for the backups at each individual bank's IT group to get their system to cooperate. All of them knew that this would be putting them out of a job at the completion of the project, so there was tons of resistance and it usually took a week of calling peoples' bosses to get the work done. This was also in the middle of forced relocations for most of them. Most of the folks responsible for the work quit. It was really ugly.

"The cybercriminals sent their victims infected emails — a news clip or message that appeared to come from a colleague — as bait. When the bank employees clicked on the email, they inadvertently downloaded malicious code. That allowed the hackers to crawl across a bank’s network"

There must be plenty of people on HN with experience in this field, so it'll be interesting to hear their take on it.

My (very ranting/rambling) point was that I've seen other large organisations pretending to do security (and probably believing it themselves), where it's really just security theatre.

I saw a DefCon video where the guys were talking about something similar. Lots of small banks in the US use 3rd party services for their banking software. One of them had horrendous security and so some hackers made off with several million dollars before anyone found out.

The IT department seemed to use the following logic to justify it: the system served its purpose, the legacy employees already knew how to use it and the developers who made it were long gone thus it was cheaper and easier to just leave it be. While my firm had plenty of developers that could rewrite the system from scratch, their attention was devoted solely to money making endeavors like trading platforms and client facing projects.

As for the "Enterprise Architecture Group" (ie the developer department) that I interned in, the big problem was the heavy reliance on third party development companies. While the firm wanted to hire more developers, simply put very few developers want to work for banks (it's funny though that people in finance would have killed to work at the firm). It would take 6 months to a year on average to fill a developer position and they would have to pay a big premium over the average dev salary with a large yearly bonus.

In order to keep up with all the various projects, they would pay third party development/consulting companies millions to come in and create apps. While this allowed the firm to get the necessary apps "done", it created the most crazy spaghetti architecture you could ever imagine. All these different apps were built using different companies/languages/platforms/technologies then thrown together in a big mish mash of iframes and duct tape. The fact that any of them were able to communicate with each other at all was a miracle. I don't actually blame the developers themselves for this, they would constantly voice their concerns while the completely clueless department head/"architects"/project managers/business analysts would shoot them down. They would say things like "I understand your concerns but Super Consultancy X says that they would do it and it will only take 12 to 18 months!! They are even available to help support the app once its finished!!". Security and use experience were not even on the company's radar, only making money.

Infrastructure architect at a major Bitcoin exchange here.

It's about defense in depth. Processes. An architecture level stance like "do not trust the client, the server, the network, the data center, the hardware provider, or any particular stage within those three elements". Each element validates the other. An alarm raised by inappropriate behavior at any point will shut down an entire instance, cell, or data center before allowing an attacker a foothold.

The only way to realistically take such a stance without going broke or becoming functionally paralyzed is infrastructure level automation beyond what is common in the industry. Hence, cue for meaningful cloud infrastructure management systems spanning private and arbitrary third party infrastructure. Docker-level stuff is about 1/2 way, what we really need is a few degrees of abstraction beyond that.

In that case, it doesn't matter how much security you have in your data center. Employees need access to central systems to do their job, so client security is paramount.

For instance, for a bank or Bitcoin exchange, I think it would relevant how many client operating systems can access your crown jewels. I think if you're just using Mac or Windows with antivirus or whatever, there's already a pretty low upper bound on your client security and thus your overall system security.

What I'm wondering if anybody is deploying some kind of custom client OS similar in spirit to Qubes OS, or a build of Chromium OS or Android, which have application sandboxing beyond what stock Linux, Mac or Windows have.

Also, I would imagine that each teller has their own credentials, and the bank should have policies about the transaction rate / total for a single teller. It sounds like the attackers would have to compromise multiple employee accounts to steal that much money. So you also want to protect employees machines from each other as much as possible (not just "outside" attackers).

I'm guessing that a Bitcoin Exchange doesn't have that many employees, since the whole industry is new. You probably have people just accessing stuff with their personal MacBooks or whatever, and that's fine for now (there are bigger risks). But when you start to have 100, 1000, 10,000 employees capable of doing financial damage, then I think this type of thing will start to matter more.

EDIT: Actually I remember one large deposit I made required three people at a bank to approve it. The teller said, "Wait my boss has to approve this." Then the boss said, "Wait my boss has to approve this". So they are probably using the presence of three credentials and credentials at a sufficient employee level to authorize large transactions. So I take it the attackers would have to target employees with those credentials.

But that can cause problems for customers -- e.g. if the branch manager isn't around, you might not be able to do what you wanted. To some degree, they are using meat space protocols to mitigate risk that their software systems can't handle.

If you have two factor auth, the employee will go through the process since they need it to do their job for 8 hours a day. Then they will have credentials on their machine (in memory or wherever).

Any attacker sitting on the machine can use those same credentials. Whether you have two factor auth or not doesn't matter.

The point is that you need to prevent the client from getting infected in the first place (which isn't easy if you have 10,000+ employees). As mentioned, if the state of the art is Windows or Mac + antivirus, then your upper bound on security is pretty low.

I recommend reading "Kingpin", a recent book about Max Butler. There's a nice story where he is hired for a penetration test. He guarantees 100% success rate, since he's always been able to get in.

He was coming out of jail and his skills were perhaps rusty, and he couldn't get into this particular server.

So what he did is hack an employee's home computer, steal their VPN credentials, and hack the company server with internal access. Apparently the company was agnry that he did this, but it pretty vividly illustrates the point.

I recall that Kevin Mitnick also used employee VPN attacks. Just because you have hardened Linux, regular updates, jailed processes, etc. on your server doesn't mean it's secure. Employees have to access systems to work, so that is often the weakest link. It's not surprising that this is how major banks got hacked and relieved of millions of dollars.

Defence in depth is a placebo. Separation of concerns, principle of least privilege, honeypots, SIEM, file integrity monitoring, host intrusion detection, IDS/IPS on all your ingress and egress points, WAF, content filtering and a responsive and empowered SOC capable of acting on auditing events will get you half way to not showing up on the front cover of NY Times.

The problem is that it takes money to keep money safe and too much security is often not secure at all, so putting everything together in a way that you doesn't motivate your users to find new and exciting ways to bypass your controls is an art in itself.

Would love to discuss some of these things with you, any chance I can interview you for my blog?

No organization like being attacked, but any defensive measure that cost money will always be balanced to the potential loss, risk, and convenience of employees. If the risk feels low, the potential loss minimum (worst case, government will intervene), and employees inconvenience high from employing effective security schemes, then no such efforts tend to be used.

Until you have similar standards for software development, I cannot see how such liability shift could work. This is one of the reasons I tend to avoid using the phrase software engineering. It's so different from traditional engineering that it feels incorrect to put it in the same category.

You don't need Qubes to secure this situation. You could use an iPad/Chromebook or a filtering proxy (whitelisted websites) and either would be sufficient.

Don't run Outlook and don't autorun USB. That should stop most automated attacks, including all the big known ones that breached large companies such as RSA and Google.

To stop the rest, don't surf from sensitive machines, and require two factor auth such as Yubikeys or RSA dongles to log in to them.

Compartmentalize sensitive information on separate machines and networks, and externalise sanity checks of data transactions where possible.

Not only that, Qubes uses a vulnerable git version from several years ago so practically anyone could go backdoor it if they cared.

[1] http://en.wikipedia.org/wiki/Blue_Pill_%28software%29