News for hackers. Stuff that matters.
Re:2.5M Medical Records Leaked by AI Company by maybelsyrup on Tuesday August 18, 2020 @01:13AM (#24195505)
I'm a medical researcher, and I've worked with exactly this sort of data. The number of hoops I had to jump through to even be allowed on the project was insane. Background checks, legal forms of all kinds, trainings on how to keep the data secure. Fingerprint locked rooms. Access control. And most importantly: the data themselves were air-gapped.

If I fucked up on this scale, jail would be the least of my worries. I'd never work again. I'd be professionally shunned forever, with no hope of redemption once my name is attached to the incident in some Google search. No more grants, no more collaborations, no nothing. I'd be ruined. Frankly I'd be begging to be let into prison; where else am I gonna be able to eat?

Nothing will happen to these people.

Re:2.5M Medical Records Leaked by AI Company by sillysaurusx on Tuesday August 18, 2020 @07:20AM (#24197380)
This is a wild exaggeration. First of all, everyone is insecure. Work as a pentester, and you'll see that the whole world is insecure. After the 30th project with an SQL injection vuln or misconfiguration, you'll never be phased again.

Secondly, this seems to be a straightforward database misconfiguration. The database was set to public with no restrictions. It likely wasn't accessible to anyone who wasn't explicitly looking for it, i.e. security researchers or black hats. Yes, this was horrible, but nothing bad happened here. The database was simply set to private.

Third, my friend, chillax a bit. Yes, it's extraordinarily important to protect people's medical data. But no, you're not personally liable if a fuckup happens. Everyone fucks up. That's why the fuckups are covered by legal protections. You take your job seriously. Good! But you also take your job way too seriously. Bad. You're gonna burn yourself out within a decade with this mindset.

None of this is to say that what happened didn't matter. Of course it matters. But you are literally saying you'd be begging to be let into prison. That's not proportionate, and feels like a reflection of the current social climate of retribution-as-forgiveness.

Re:2.5M Medical Records Leaked by AI Company by mirashii on Tuesday August 18, 2020 @10:52AM (#24199315)
> This is a wild exaggeration.

It isn't. I get that experience as a pentester can make you jaded to security practices, but the healthcare space is extremely regulated and much of what the grandparent said is a direct result of that.

> Secondly, this seems to be a straightforward database misconfiguration. The database was set to public with no restrictions. It likely wasn't accessible to anyone who wasn't explicitly looking for it, i.e. security researchers or black hats. Yes, this was horrible, but nothing bad happened here. The database was simply set to private.

This isn't just a minor misconfiguration. Again, this is a regulated space, with rules. There should be multiple layers in place to make sure that this is not opened to the public. There's risk assessments that needed to be carried out when the database was launched, and on a regular cadence afterwards. "It wasn't accessible to anyone who wasn't explicitly looking for it" isn't reassuring. Look at access logs on any device with a public IP address and it's clear that people are constantly looking.

> But no, you're not personally liable if a fuckup happens.

Under HIPAA, you can be held personally liable. This includes jail time for certain classes of violations.

> But you also take your job way too seriously. Bad.

People working in this space are required to have regular training and undergo audits of their activity, what information they access, risk assessments on how it is stored, and long, drawn out compliance activities just to get access.

Re:2.5M Medical Records Leaked by AI Company by kspacewalk2 on Tuesday August 18, 2020 @10:59AM (#24199426)
>Under HIPAA, you can be held personally liable. This includes jail time for certain classes of violations.

Has anyone ever been actually sent to jail under HIPAA for something comparable to an honest DB config mistake? Has anyone been credibly threatened with it in the court of law? Has anyone come remotely close?

Re:2.5M Medical Records Leaked by AI Company by parliament32 on Tuesday August 18, 2020 @12:47PM (#24200957)
Not even close. The only case I could see it happening was if you purposefully exfiltrated data and sold it, or got some sort of personal gain out of it.

It's the same thing with PCI. They say you can go to jail for violations but exactly 0 people have ever been prosecuted, despite constant negligence in the payment industry.

Re:2.5M Medical Records Leaked by AI Company by sillysaurusx on Tuesday August 18, 2020 @11:07AM (#24199546)
Nope. Definitely not rank-and-file engineers.

Suppose you fucked up, and the worst happened: You were the reason the database was misconfigured. What then?

Then you live your life. You think future employers are going to pass on you because of it? Maybe some would. Most don't give a shit. They care about one thing and one thing only: whether you can help them achieve their objectives.

Morals are a fine thing, but the reality is that the original comment was correct: Nothing will happen to these people. As it should be.

They are required under law to notify the affected people. THat's the extent of what will happen. There may be legal repercussions for the company. And I'm sure the engineers will feel pretty awful at this mistake. I know I would. Everyone does.

But that doesn't change the fact that the world is insecure.

Re:2.5M Medical Records Leaked by AI Company by cutemonster on Tuesday August 18, 2020 @02:29PM (#24202334)
> Look at access logs on any device with a public IP address and it's clear that people are constantly looking

Yes, I'd be a bit surprised if this database hadn't already been detected by some automatic IP and port scanning scripts, and the wrong people found out about it

(I don't with exactly these things though do I might be mistaken)

Re:2.5M Medical Records Leaked by AI Company by stjohnswarts on Tuesday August 18, 2020 @12:22PM (#24200620)
Sorry you're the one wildly exaggerating about its importance. 2.5 million people wouldn't agree with your take on this. This stuff should never be public facing. It should be behind several layers of security not sitting on a public facing server without any protection. The people screwing this should be heavily fined and not be able to work on anything internet facing for several years while they have the time to learn a bit of network security if it does turn out to be gross incompetence/negligence.
Re:2.5M Medical Records Leaked by AI Company by cb504 on Tuesday August 18, 2020 @08:03AM (#24197638)
If I understood correctly, the database was on a publicly accessible network. Wouldn't the problem be averted by putting the database in a separate VLAN, where only specific IP's from the public VLAN could access it, and not IP's from WWW??
Re:2.5M Medical Records Leaked by AI Company by francis-io on Tuesday August 18, 2020 @09:58AM (#24198593)
You could secure it in a number of ways, including the way you mention. Unfortunately, it seems like security is no ones top concern unless they are forced to handle it correctly.
Re:2.5M Medical Records Leaked by AI Company by parliament32 on Tuesday August 18, 2020 @12:48PM (#24200974)
That's not really what VLANs are for, you'd probably just use firewall rules to restrict access to certain IPs. VLANs are more for creating virtual-networks-inside-networks, they're more organization-based than security-based.
Re:2.5M Medical Records Leaked by AI Company by pc86 on Tuesday August 18, 2020 @07:32AM (#24197456)
What's more, it's important to take in the context of the data. It's not mental health, or HIV status, or sexual history, or really of any nature that can do damage other than the privacy aspect. It's people who hurt their neck after a car accident. Not the most sensitive stuff in the world.

Just to be explicit about it, I'm not suggesting the release of this data isn't a problem. Just that as medical data goes this is about an innocuous as it could have been. Even day-to-day visits to your primary care doc would probably have references to much more sensitive health data.

Re:2.5M Medical Records Leaked by AI Company by nitrogen on Tuesday August 18, 2020 @10:05AM (#24198681)
It's more sensitive than you'd think if any claims or settlements were involved. Or employers might want to snoop on their injured employees to find a reason to deny leave.
Re:2.5M Medical Records Leaked by AI Company by g_p on Tuesday August 18, 2020 @03:06AM (#24195994)
In contrast here, this data likely came from an insurance company, which got access to the data from their claims. They don't inherently care about these people, because they're all seen as "costs" to the balance sheet profits for the year. Sending their data out for "AI" to try and learn how to predict which claims can be rejected is a rational thing to do, in their eyes, whatever the cost (to those concerned and their data), since it might help improve their bottom line by rejecting claims faster.

At least there's a cynical take on this. They have no inherent interest in protecting this data, absent external pressures or legislation that imposes penalties.

It strikes me the real issue is the lack of negative inventive for people and businesses who screw up with data. If you build something physical that falls down and hurts people, your investor money ends up paying compensation. In tech though, the goal seems to be to make the revenues without ever touching the responsibility.

The story over the weekend of Amazon trying to avoid being deemed liable as a seller for goods they ship, (often) deliver, warehouse, market, dispatch and receive seems to sum this up. A race to the bottom when it comes to protections - Amazon effectively is a shop selling goods from others, as it all happens on their terms. We see similar in physical retail where the manufacturer or reseller has a retention of title over goods in the store, and that doesn't affect the store being responsible. It seems like the "being responsible for what you build" is very much lacking in tech companies, due to lack of external regulation to achieve it. Same here in this case.

Re:2.5M Medical Records Leaked by AI Company by harha on Tuesday August 18, 2020 @06:24AM (#24197069)
We've tried this for a while (companies using customers data) and I see a strong case for governments to start protecting users more strongly, because the current way this is being treated isn't working (medical data needs to be handled with greatest care everywhere else, why not here?).

The user already pays for the insurance (or hardware or service in other cases), so the data collected should only be used for those purposes, e.g., handling the claim and not building AI models without the necessary care. The customer should have the right to sell their own data to a company they see fit to take care of it appropriately and that compensates them for it.

Re:2.5M Medical Records Leaked by AI Company by pc86 on Tuesday August 18, 2020 @07:37AM (#24197489)
The problem is with ownership I think. I'm not German so I'm really only speaking from what I've seen online (mostly here), but my understanding is that for Germans in particular (maybe EU writ large), you own data about you regardless of how it is collected or who has custody of it. In the US, it seems like the data about people is owned by whoever creates that data (in this case, an insurance company), so they are free to do with that data whatever they like provided it doesn't violate some other non-data-specific law.
Re:2.5M Medical Records Leaked by AI Company by mola on Tuesday August 18, 2020 @04:13AM (#24196328)
When the mantra of a huge swathe of business is "disruption" "move fast and break things" etc, well, we end up with a broken disrupted system. Innovation is good, but stability is also good. Instead of optimizing only for dynamism we need to optimize for both.

The reason you'll get yourself in trouble and these people won't is because you are operating in different moral systems. Where they operate, the moral thing to do is whatever it takes to succeed, it's entirely moral to "fake it until you make it" to buy media articles promoting your product, to sell media space your readers believe is actually curated, to the highest bidder.

You operate in the "ivory tower" they hate, because it's slow and puts up boundaries that limit people from succeeding at all cost, that makes you think about the consequences of your actions. They call it slow and inefficient, I call efficient in the long run.

Re:2.5M Medical Records Leaked by AI Company by unishark on Tuesday August 18, 2020 @09:39AM (#24198384)
Universities can be punished by NIH in addition to the feds if they violate privacy rules in clinical studies. Columbia was threatened with losing all their research funding if they didn't get their act together after a big breach a few years ago.

But I know plenty academic researchers with a "move fast and break things" and "ignore all rules" attitude. In any highly-competitive system there will be those that gain an advantage by being amoral while everyone else follows the rules. Healthcare is probably especially vulnerable to this as the rules (and legitimate companies following them) has resulted in healthcare lagging far behind other industries in terms of new technologies.

Re:2.5M Medical Records Leaked by AI Company by telchar on Tuesday August 18, 2020 @10:28AM (#24198953)
"The principle with which we are now dealing is that one which is called Expediency. The usage of this word has been corrupted and perverted and has gradually come to the point where, separating moral rectitude from expediency, it is accepted that a thing may be morally right without being expedient, and expedient without being morally right. No more pernicious doctrine than this could be introduced into human life."

-M. Tullius Cicero. De Officiis. Book II.

Re:2.5M Medical Records Leaked by AI Company by thinkharderdev on Tuesday August 18, 2020 @09:29AM (#24198274)
I'm not sure that framing makes a lot of sense here. This was a SaaS product for process automation which seems like a perfectly reasonable product. I would argue in fact that having tech companies (meaning companies which have specific expertise in building software products as opposed to companies that have a expertise in something else and also have an "IT department" bolted on) manage sensitive data is likely to make it more secure rather than less. Companies and other institutions which don't have any particular expertise in software engineering and security aren't likely to do a good job. Of course when those companies do screw up they should be held liable though, up to an including criminal liability is the leak is bad enough and the exploited vulnerability is egregious enough.
Re:2.5M Medical Records Leaked by AI Company by dekhn on Tuesday August 18, 2020 @10:21AM (#24198862)
I was a medical researcher and also a data engineer for a biotech and I've never seen a dataset that required background checks, or "fingerprint locked rooms" (fingerprints are not biometric security). Instead, a partner company (a well-known, multibillion dollar biotech) handed us a pile of hard drives that were encrypted by trivial keys.

I sat down with the CEO and discussed our plans (to upload the data to our S3 buckets with ACLs) along with all our other security plans, pointed out the two of us were professionally and legally on the hook (her more than me, as I was not an executive and had limited liability), and what I was doing to protect the clinical data we had. We looked at our plans, and ultimatley concluded we could both sleep happily at night with what we were doing.

It was definitely a lot of work to maintain the ACLs and monitoring and checking to make sure we weren't leaking data, and a lot of education for our scientists. Even if something negative had happened, we would have been able to foresically detect it, and work with legal authorities with confidence we had done a good-faith, better-than-average practices implementation.

Re:2.5M Medical Records Leaked by AI Company by ryandrake on Tuesday August 18, 2020 @02:16PM (#24202161)
> Nothing will happen to these people.

Not true. They may end up putting out a press release explaining how "Here at Company X, we take security very seriously. Securing data is very important, and we will investigate ourselves to understand if we did anything wrong." So yea, consequences!

Re:2.5M Medical Records Leaked by AI Company by mvn9 on Tuesday August 18, 2020 @01:56AM (#24195706)
Somebody has to work for those who collect all the leaked data and use it for whatever is possible. You wouldn't earn fame but money for being a data pirate. Or you create a company and start with a fresh name. On the internet, nobody knows that you are a dog.
Re:2.5M Medical Records Leaked by AI Company by thecleaner on Tuesday August 18, 2020 @07:24AM (#24197401)
Maybe that is why these kinds of projects are picked up by private companies ? I believe currently in the USA a pursuit of profit is allows you make far less mistakes than a pursuit of public good. Sometimes the two overlap but when they dont we frequently see the former being allowed far more mistakes.