> US Gov should make it clear, if you are a critical service and if your service drops due to items being on the internet, for each occurances 10% of your total revenue (including your parent companies) are forfeited

This sounds good in theory but suffers from the cobra effect [1]; you think you’re incentivising security. You’re actually pushing obscurity. Colonial preëmptively shut down its pipe to prevent physical damage. Attach a fine to the discovery and disclosure and you disincentivise that prudence.

Better: make it easier for industry to build securely and incentivise redundancy.

[1] https://en.m.wikipedia.org/wiki/Perverse_incentive#The_origi...

I reckon air-gapped networks are a valid defense. If something needn't be connected, why let it? It mitigates so many threats.

Pipelines run for thousands of miles and operate 24/7. What do you imagine? Keeping a fleet of helicopters on standby to pick up a technician at home, and drop him wherever the equipment is, in case something needs to be adjusted at night?

You could have an air-gapped system and still have remote access. Just not external access. I don't think it's unreasonable to have a couple of people in a control booth monitoring a computer that regulates the pipeline 24/7. The recommendation is, however, that we should not have that monitoring computer connected to any other network besides the internal one. If you're running pipeline, surely you can run some data cables with it?

This basically describes the Iranian nuclear system that was hit by stuxnet, which, as I recall, was spread via USB drive.

The airgap certainly reduces the chances of getting hit with a joe-random ransomware attack, though. Defense in depth...

Yes - and consider how many people have the resources needed to run Stuxnet versus ordinary ransomware. That seems like a pretty clear win if you can shrink the pool of likely attackers down to “top-tier nation-state”.

Let's not let perfect be enemy of good, especially since we can't never be perfect.

Stuxnet was built by a very well funded organization and was not targeting monetary gains. Getting ransomware spreading via USB drive is insanely expensive and complicated. You won't make money on it, whatever they are extorting will not cover development expenses

"and was not targeting monetary gains"

But in the end was it really though?

This is what the financial industry does. There's a secured "extranet" that everyone runs FIX on top of. For the most part, it works.

There's an entire parallel internet for the DoD and it's contractors. It's not like there isn't precedent that critical industries can emulate if they ever became concerned about the common good.

That's interesting. Is it physically separate? I would assume so but don't know.

I'd love to read an article on it. Maybe there are others that are geographically dispersed.

> You could have an air-gapped system and still have remote access

You're suggesting the gas company run their own network, and then you assume no employee will connect that network to the general internet for their own convenience? Not happening.

That would be a fireable offense. Even criminal.

Again, stuff like that just don't happen. It's too inconvenient. How many major utility companies run such airgapped networks?

https://gca.isa.org/blog/common-ics-cybersecurity-myth-1-the...

Are pipelines something we invented after the internet? Have we lost the knowledge we used to support infrastructure before the 90s?

Before the 90s? Hmm. Well, if you believe Thomas C. Reed's account, the US was using trojans to sabotage Soviet oil pipelines in 1982.

Is that the one when the CIA got wind of a Soviet industrial espionage operation, and seeded it with a legit-looking but subtly flawed schematic, which the Soviets ended up actually building to spec, and it exploded shortly after?

That wasn't an internet trojan, fyi.

According to the story, this was some Canadian pipeline control equipment or software which the USSR purchased from some Canadians, but the CIA modified the software somehow before it was delivered. A supply-chain attack on a computer system.

I understand there's nonzero doubt as to the credibility of this story.

I'm currently reading The Dead Hand, and I think this matches what you're talking about:

> Rather than roll up the Line X officers and expel them, Reagan approved a secret plan to exploit the Farewell dossier for economic warfare against the Soviet Union. The plan was to secretly feed the Line X officers with technology rigged to self-destruct after a certain interval. The idea came from Weiss, who approached Casey, who took it to Reagan. The CIA worked with American industry to alter products to be slipped to the KGB, matching the KGB’s shopping list. “Contrived computer chips found their way into Soviet military equipment, flawed turbines were installed on a gas pipeline, and defective plans disturbed the output of chemical plants and a tractor factory,” Weiss said. “The Pentagon introduced misleading information pertinent to stealth aircraft, space defense, and tactical aircraft.”

> Oil and gas equipment was at the top of the Soviet wish list, and the Soviets needed sophisticated control systems to automate the valves, compressors and storage facilities for a huge new pipeline to Europe. When the pipeline technology could not be purchased in the United States, the KGB shopped it from a Canadian firm. However, tipped by Vetrov, the CIA rigged the software sold from Canada to go haywire after a while, to reset pump speeds and valve settings to create pressures far beyond those acceptable to the pipeline joints and welds. One day, the system exploded. “The result was the most monumental non-nuclear explosion and fire ever seen from space,” Reed recalled. The blast was starting to trigger worried looks in the U.S. government that day, he recalled, when, at the National Security Council, “Gus Weiss came down the hall to tell his fellow NSC staffers not to worry.” The explosion had been one of the first fruits of the Reagan confrontation.

Interesting, it's not unlike the Crypto AG story.

It's funny how upset the US public can get about any perceived incursions by the Russians (sometimes true, often not) when you consider the country's own history.

There's a big difference between the capabilities of a criminal organization (like the one involved here) and that of a nation state. Such attacks are also responded to differently, and it's not going to be send some Bitcoin to this address.

Couldn't the pipeline have it's own network connected to a monitoring station. At the station employees could access the pipeline network but never connect it to the network from which they could communicate with the people who would be dispatched to make repairs or adjustments?

Sounds looks a solved problem, but it isn’t. The electrical grid isolates SCADA networks from the internet, so substations are interconnected via dedicated networks. But then at the command centers you have the entire monitoring and control systems with on site operators. Then, inevitably you will have some employees with vpn access, and now you have 2 vectors: remote admins getting hacked and local admins plugging in external devices. You’d think it’s easy to get rid of vpns, but things like the pandemic brought them all back to full force.

Is this even safe enough, though? They had one of those in the Iranian uranium enrichment facilities, and it still didn't work out for them.

I remember they got past the Iranian air gap using a USB stick. It’s a mistake to think air gaps are safe but they are certainly better than having your network open to the internet.

Well, whatever they used to do before they connected everything to the internet. What about just do that?

You can have a maintenance network that is disconnected from the internet.

Can an individual not accidentally or intentionally infect a computer not connected to the internet?

It's called Stuxnet, and it ruined 20% of Iran's uranium centrifuges. Which were air-gapped.

Air gaps didn’t save Iran. Air gaps are just one layer in the onion.

Is the argument that Iran was attacked on air-gapped network, so its not worth doing?

Do the opposite of last time logic is surprisingly popular, despite leading to some odd conclusions.

Tradeoffs. Air gaps come with other costs of their own.

Allowing remote monitoring and troubleshooting is too convenient

Iran would like a word!

> Attach a fine to the discovery and disclosure and you disincentivise that prudence.

Sue them. Failure to disclose key documents in the discovery phase of a trial carries hefty fines and jailtime. And quadruple the fine for misrepresenting the cause.

People act like the government doesn't have the power of subpoena. They can absolutely compel you to tell the truth.

Yes. I've worked in a very regulated industry before and it ended up being less secure than any other I've worked in. Checking your secure email required so many hoops people would just text each others personal phones instead, etc.

I've actually worked in govt systems. If you think the whole endless threats of jail make for more secure systems you are truly clueless.

These systems are RIDDLED with the WORST outdated crap you can imagine. Absolute insane hoop jumping so plenty of pressure to work around security just to get jobs done (seriously - start with the help desk if you want access - they are so used to password resets the procedures become a joke - literally - what's the username and that's it, because if you have thousands of folks on 30 day password rotations with insane complexity all you do is password resets endlessly). Password sharing can also be crazy so passwords float all over.

The govt has had it's top stuff leaked. Office of personnel management leaked insanely sensitive stuff. They contract with the WORST folks in security. It's really crazy.

Google has never asked me to rotate my password. I have non-SMS two factor authentication options, they do pretty sophisticated rate and geo monitoring so you are not annoyed but pretty secure.

Cyberattacks, mechanical failures, weather disasters, meteor strikes, terrorist bombs, stupid construction workers ALL could affect this pipeline. People on HN have no risk perspective. Make the system resilient to a proactive few day outage. Why does this system have to run 365 / 24 / 7? Have you mitigated EVERY possible issue - including disgruntled employees? No - then instead of over doing one corner, design some give in the system.

I want to add that the physical limits of how the design is done is as much as corruption/stupidity.

By physical limits I mean us, the wet ware in the middle of all this. These systems can be designed years if not decades before they are actually brought online. By simple temporal placement they get the materials and techniques of that time span. By the time these things are ageing out of the system they will have some old tech on them.

> Failure to disclose key documents in the discovery phase of a trial carries hefty fines and jailtime

When you do this, the documents never get created. Not due to nefarious cover-ups. But because if little incentivises the creation of documentation, and everything penalises it in the edge case, you get rubber stamped compliance stacks for decades until a crash.

If one has massive downside for reporting a potential risk, one better be 100% sure that risk is manifest before pulling the trigger. That delay and omission is the cost of such draconianism.

All internal communications, unless they are with an attorney and are clearly marked "privileged" and pertain to actual legal advice, are discoverable.

> All internal communications, unless they are with an attorney and are clearly marked "privileged" and pertain to actual legal advice, are discoverable

If a communication doesn't exist, it's not discoverable. If you legislate penalties for a certain type of communication, it shouldn't be surprising when it ceases to exist. This isn't the product of cover ups. It's the long-term effect of penalties dissuading the looking into of certain things. If discovering a breach is penalized, nobody competent will look for breaches--that leaves no discoverable liability.

You're imagining that every company is going to transition some kind of nefarious mob-style wink-wink, nudge-nudge, no-written records speakeasy, or what? That's really taking things far.

This whole line of reasoning is specious, anyway. It's based on a fallacy that enforcing penalties is just going to make everyone lie their asses off to get out scott free. Is society so broken that they can get away with this? Come on.

Security through obscurity is almost always a bad idea. There are exceptions: namely nuclear missiles.

There are counterarguments to this, but they're mostly academic: https://core.ac.uk/download/pdf/228618432.pdf.

Stuxnet, by contrast is very real

> companies are chasing profits at any cost

Government systems get hacked all the time, too. Just because the government doesn't have a profit motive doesn't change a long list of human motivations that can be counterproductive.

The profit motive also incentivizes improved quality. If the product is bungled, the company is not likely to get the next contract. If the government agency bungles the product, they'll get a budget increase next time.

"If the product is bungled, the company is not likely to get the next contract."

> The US Gov should make it clear, if you are a critical service and if your service drops due to items being on the internet, for each occurances 10% of your total revenue (including your parent companies) are forfeited.

I would love to see this but somehow I doubt it will happen any more than my pipe dream of holding the CXO and the board criminally liable for the criminal actions of management/employees/contractors/agents of a corporation during the course of their work for the corporation.

It is nice to dream though. I would certainly welcome any kind of accountability.

They aren't 'on the internet'. They are connected via several layers of networks with firewalls etc in between to a system which has direct access to the internet. There often is no practical way around this - data from these networks needs to be shared with business users, other companies, regulators and so on. Data diodes can be applicable in some situations, but I've never worked with a company that uses one. I don't know the details of this situation, but demanding that it be impossible to compromise infrastructure networks is ridiculous. If you throw enough money and resources at it, no network is secure.

Indeed the best evidence supporting this view is that Natanz was fully air-gapped and still got destroyed by the Stuxnet hack.

That said I have heard of customers expressing desire to control valves and pumps using iPhones, and believe there are several initiatives at SCADA/PLC/DCS/System Integrator companies to provide this.

However I've seen as many of those in practice as I have data-diodes, which is to say, none/never.

It's multi-dimensional. SCADA itself being networked, and it reaching other systems that may be internet-enabled.

  * What systems are affected by the hack?
  * Could the shutdown be needed because of critical data the ICS gets from business?
  * Or is it shut down because business needs real-time data from ICS it can't ingest?

In general, the idea of completely isolating an ICS from any other network is a tough one.

My question is, how often are these critical suppliers audited by the federal government? I have worked in banking cybersecurity and the amount of auditing from federal and state regulators is mind boggling. If a single company controls 45% of fuel transport to the east coast, it should carry some designation as a quasi-state entity subject to federal cybersecurity audits like banks.

This question of missing cyber security audits came up for me in discussion of the Verkada hack. That’s the startup providing security cameras inside hospitals, prisons and schools.

It seems like cybersecurity and audits of security readiness need to be demanded from any authority over companies operating in sensitive areas.

>if you are a critical service and if your service drops due to items being on the internet, for each occurances 10% of your total revenue (including your parent companies) are forfeited.

The problem is that it matters less than we'd like to think how "serious they are about security".

I'm seeing a lot of discussion about the responsibility of the victims to secure their networks, and it's mostly valid.

But it's strange that we're talking about how to punish the victims versus the criminal conspirators. We virtually let the malicious, overtly criminal party off the hook. It's almost like we're saying we expect criminals to be criminals and these guys are so hard to catch that the onus is entirely on the targets to repel their incessant attacks, else they're negligently malicious themselves. Sure, most of these victims can do better in just about every case. But, people here know better than most how difficult it is to 100% secure every layer of the stack from software to firmware to hardware, with multiple vendors and vectors, OSS, zero days, etc. And the bad guys only have to be right once across this broad attack surface. It's impossible to defend completely. There will always be breaches.

So, there's another element of this that has to be addressed, and that's getting serious about punishing these people. As it is, there's zero disincentive for them to just keep trying until they get through, but the upside is massive.

Most if not all of this activity originates from nations that are adversarial towards the US. So, we need to start treating these instances as official sovereign actions, especially when they originate from nations wherein the government and their intelligence services exert control over (and outright sponsor) such criminal schemes, and wield these attacks as a projection of national power.

These regimes also tend to feature oppressive criminal justice systems and harsh reprisals for even political dissidents. So, the message is, "we're not going to argue whether you're sanctioning these acts, but we're also not buying that you can't stop them, so we'll treat each incident as an official act of the state. We're holding you responsible for your criminals when they attack us and we will respond accordingly".

Detailed discussion around exactly how the most recent exploit might be mitigated is interesting and useful. But the balance in these discussions between mitigation and reprisals for the perpetrators needs to be shifted much more towards the latter. Otherwise, we can expect these discussions ad infinitum.

> That will get them serious about security.

My guess is that they only get serious about security after a breach occurs.

You can view it all as strengthening an immune system. Without attacks, and the occasional successful ones, nobody is going to bother to harden anything.

Is that like how the banks all got serious about evaluating their risk carefully after the first time [1] they saw their models, and consequently their liquidity, evaporate?

Obviously I agree about your dissatisfaction with the other proposed solution: that just lets corporate entities put a low (10%) ceiling on what should be unlimited liability, allowing them to say that failing catastrophically by utter neglect to security is reliably a survivable offense (I recognize that in reality the liability of course ends at the dissolution of the corporation.)

I don't know what the actual answer is.

[1] https://en.m.wikipedia.org/wiki/List_of_economic_crises

I don’t think the analogy applies. We are humans, capable of observing the failures of others and making rational decisions to avoid those same failures. The immune system is programmed to react one specific way. For example, we can choose or choose not to go out of our way to get vaccinated. If someone doesn’t get vaccinated and infects a dozen other people and then get really sick themselves, it’s hard for me to have sympathy. Unfortunately here, when there’s an illness, millions of people have private information leaked. This should simply be an unacceptable.

> companies are chasing profits at any cost

What does that mean?

This was addressed in the article. Critical services are on the internet because remote workers need access to them. I don't see how profits factor into it.

Without the overhead of reaching many locations, fewer engineers/technicians can more efficiently operate the whole thing remotely.

Those remote workers wouldn’t have to be ‘remote’ if there were other workers hired on site.

What "on site"? The various valves that need to be controlled remotely often are just a box, perhaps even underground, in a place where's no buildings for people to stay - a mechanical team can access the hardware on-site, but building and maintaining an actual office on each site is not practical. The same applies for power grids - you can't staff every substation with people.

A remote operator can manage dozens of such sites, a single "local" person might be close to one point, but the next control point is going to be miles away, so you either need much, much more people to station one at every valve, or have a situation where flipping a switch in all the "sites" is very slow because requires the "local" person to drive many miles visiting each location.

No, there's a reasonable objective need for this management to be actually remote - there's a discussion on how this should be implemented in a secure way, but it does have to be remote.

Many sites are in the middle of nowhere so it is inconvenient to go to them, so accessing them over a network saves a lot of travel time and cost.

This is true for nuclear weapons infrastructure. Paying for commutes and lodging are used there, might be useful to consider?

At one extreme a Nuclear weapons accident can kill millions of people and destroy the environment of a large area for a long time.

And then a sliding scale of risk and cost of getting hacked vs savings and increases in efficiency resulting from remote access.

Are you for or against Tesla having remote access to all of the Tesla vehicles? Are their OTA updates innovative or reckless?

Lost revenue would be justified by execs as cost of doing business. And they'd probably buy hacker-insurance to cover some/all of that risk. They'd pay for the hacker-insurance by getting a shittier employee benefit plan, and cutting the employee 401k match by 2%. For each quarter they aren't hacked, they'd probably receive a credit from the hacker-insurance company, which would be distributed to execs via performance bonuses.

> It needs to be asked again, why are critical services on the Internet?

From TFA:

Digital Shadows thinks the Colonial Pipeline cyber-attack has come about due to the coronavirus pandemic - the rise of engineers remotely accessing control systems for the pipeline from home... believe DarkSide bought account login details relating to remote desktop software like TeamViewer and Microsoft Remote Desktop.

So fast food workers were “essential workers” but oil pipeline control system engineers aren’t?

Projects that require a security clearance have to be done from inside of a secure facility. This clearly needs to be the same level.

>We all know why, companies are chasing profits at any cost,

I wonder if that's even true 100% of the time.

It could simply be programmers + feature creep, bolting great wads of software to systems with small value on the margin.

It would be interesting to talk to software people in the gambling business, you'd think that would be Ground Zero for nefarious attacks.

Couldn't a service provider use an internet backbone with a one time pad comms protocol to essentially have a secure controls channel?

Then once a year a rep could fly out a few terabytes of OTP to each location and all comms would be impenetrable.

Sure beats a parallel physical comms system cost wise.

Maybe that'd make sense if it was the encryption that was broken. I wouldn't suggest a one-time pad for anything where modern encryption works just fine.

People whine about infrastructure being accessible over the internet, but in the old days it was just accessible over the phone network instead of the internet. It's not realistic for every bit of infrastructure to have the security protocols of military systems.

> The US Gov should make it clear, if you are a critical service and if your service drops due to items being on the internet, for each occurances 10% of your total revenue (including your parent companies) are forfeited.

Make it a crime to pay the ransom in a ransomware attack.

Make it a crime to fail to report a ransomware attack in a timely manner.

Ransomware attacks (and companies with poor security practices) will go away.

Make it a crime to possess, consume, or distribute substances that are bad for people.

Make it a crime for people to knowingly withhold information about such activities from the authorities.

Abuse of substances, trafficking, and associated criminality will go away.

But seriously...

> Make it a crime to pay the ransom in a ransomware attack.

Ok, so ... what should a company do? File a report with some government agency or with an insurance company and wait until the bureaucratic process maybe results in being able to pay the ransom to resume business operations? Punishing the victim of a crime? Really?

> Make it a crime to fail to report a ransomware attack in a timely manner.

Further punishing the victim of a crime? Really?

> Ransomware attacks (and companies with poor security practices) will go away.

lol

I don’t see an analogy to drug laws. Is your point simply that such a law is impractical to enforce?

Paying extortionists begets more extortion. The only argument I’m seeing is a bunch of hand waving and people telling me I’m being hopelessly naive. I don’t think I am.

There is already a law. It's called misprison of felony.

>Make it a crime to fail to report a ransomware attack in a timely manner.

You mean like "18 U.S. Code § 4 - Misprision of felony" [0]?

[0] https://www.law.cornell.edu/uscode/text/18/4

Just like all other laws that companies break, they'll just get a slap on the wrist and continue their ways

Also, why do critical services run Microsoft systems?

The real question right here. When will the US government finally take Linux seriously and invest heavily in it instead of relying on Microsoft solutions?

so then we rely on linux solutions instead? I've seen lots of control systems running linux and windows, personally I'd rather keep a mix of the two. Something about eggs and baskets.

> Digital Shadows thinks the Colonial Pipeline cyber-attack has come about due to the coronavirus pandemic - the rise of engineers remotely accessing control systems for the pipeline from home.

I would disagree with this. No petrochemical company I've worked with had OT/PLC/SCADA layers enabled for remote login. Other middle layers, like the historian, potentially. But not the control system itself.

It's possible to enable this but knowing the culture at Colonial Pipeline first-hand, I strongly doubt they did this during COVID.

Agreed. Our companies are driven to increase profit at all cost. Even cost to their function and utility.

Our over financialization is squeezing everyone and everything.

To clarify: this is not from "financialization" generally speaking. This is specifically from consolidation for the sake of increasing efficiency and thus margins (as you pointed out).

This is opposed to increased competition (which also increases volatility) in the markets.

The reason why the markets are so consolidated is because it removes short-term risk. If there is an obvious market and only one (or a few) companies involved then everyone makes money (magic, I know). This is why Wall Street lobbies for regulations so hard. They have ownership in all the existing major companies that can afford those regulations and it consolidates the profits (and thus returns). De facto Crony Capitalism at its finest. Your aristocratic oligarchs.

So, stop giving your money to large index funds. (And every time there is a comment on HN telling you to, and there are plenty, downvote them and tell them a proper F-off).

The idea that no one can/should ever lose is what is killing the economy. It got its birth in the boomer-retirement-fund markets of the past few decades.

To recap, it is consolidation for margins that is the problem. That is not the same as general "financialization" (increased trading) that helps increase volatility in the markets and actually increases the size of the economy.

We desperately need real markets and not this crony capitalism that seems self-persistent.

The current system is such a marriage between corrupt politicians and wall street (which is now heavily extended into SV, btw) that it is absolutely disgusting.

Critical services are on the Internet because they are provided by the lowest bidder.

They are not always the lowest bidder. Colonial Pipeline was known for selecting the higher bids sometimes in order to ensure quality of work. They'd generally evaluate risk vs. quality vs. price.

Colonial also kept multiple overlapping vendors for their last SCADA upgrade in order to make sure that no contractor was too overloaded during a "boom time". They'd generally stagger the work between locations (they had to upgrade dozens of stations along the pipeline) and keep track of the performance of everyone they hired and try to keep a steady workflow for everyone over a multi-year period.

It was generally not about the lowest bid.

> if you are a critical service ... for each occurances 10% of your total revenue ... are forfeited

You can achieve the same effect without all the arbitrary political decision-making inherent in this proposal by requiring these companies to buy delivery insurance or something. The insurance company will charge them proportionally to the risk of attack, which will internalize the cost.

Like AIG insuring investments during the 2008 crisis? I think for critical infra you want some extra care and redundancy (not arbitrary though of course).

We should also shut down Russian infrastructure through cyberattacks. The Russian government supports DarkSide.

Totally, take out all the hospitals, education and basic needs for the rest of the innocent people. Or just nuke them, that will teach um.

On a serious note, sure retaliate, probably don't hurt innocent people.

or maybe the notion that russia is even behind it is a lie aimed at stoking tensions between the united states and russia, probably for the sake of profitting companies like digital shadows--or wharever it was called.