News for hackers. Stuff that matters.
Re:Maersk reinstalled thousands of servers and workstations ... by scribu on Sunday January 28, 2018 @04:31PM (#16253653)
For me, the most surprising thing is that Maersk was able to carry on during those 10 days without any IT:

> "We only had a 20% drop in volume, so we managed 80% of that volume manually. [...] Customers were great contributors to overcoming that."

That makes me think that either a) IT isn’t quite as essential as we software people like to think, or b) the full potential of automation hasn’t been achieved yet.

Re:Maersk reinstalled thousands of servers and workstations ... by Animats on Monday January 29, 2018 @01:09AM (#16255637)
It was much worse than Maersk's PR indicated. Their big automated ports, LA, NY, and Rotterdam, were down for most of a week. Maersk operates in about 160 ports, and about 17 were affected by the computer attack.

The Port Authority of New York and New Jersey has a message page for announcements to truckers. They're very real-world: "Gate 6 is closed due to flooding from a broken water main" kind of info. They were saying Maersk was totally down, then only handling inbound refrigerated containers, then inbound only. Later announcements mentioned manual processing, few gates open, long wait times, and suggested delaying shipments. It was about two weeks before they were fully back up and accepting everything, including empties.

LA was down for four days, but Maersk also has a terminal at Long Beach which was unaffected, so they diverted one container ship there. They weren't accepting online orders for a week or two, the tracking system was down, email was down, and they were trying to operate by voice phone only. Many customers had containers stuck in the container stacks at the port.

Rotterdam's robotic container handling was down for a while, but details are scanty. Most of the problems were apparently with the truck-side systems, the ones which get the right container onto the right truck.

This was a big deal.

Re:Maersk reinstalled thousands of servers and workstations ... by jsiepkes on Sunday January 28, 2018 @05:35PM (#16253994)
They just started loading containers onto ships based on the best available information. Information ports had, customers, etc. A lot of containers ended up in the wrong place or simply went missing. It became a mess that will take them at least a year to fully cleanup.
Re:Maersk reinstalled thousands of servers and workstations ... by erk__ on Sunday January 28, 2018 @05:07PM (#16253870)
They still had a loss on $200-300 million, after everything. When it happend there was stories about how workers switched to using WhatsApp on their own phone to communicate.
Re:Maersk reinstalled thousands of servers and workstations ... by koalalorenzo on Monday January 29, 2018 @09:37AM (#16257705)
...and forced to close down Slack. That is painful! :(
Re:Maersk reinstalled thousands of servers and workstations ... by malux85 on Sunday January 28, 2018 @05:09PM (#16253878)
Some of it must have been sympathy by the customers too - "We got hacked and and to re-install thousands of machines" as a customer I'd let some late shipping slide. But could the company operate at a permanent performance penalty - i.e. without IT systems. They probably couldn't stay competitive
Re:Maersk reinstalled thousands of servers and workstations ... by tjoff on Sunday January 28, 2018 @04:49PM (#16253772)
I guess there are some factors that helps in this particular case. These are large container ships, the schedule is known very long in advance (the trips themselves take a long time too). After the initial hit the ships would have plenty of time to prepare before hitting port. Not trying to diminish the effort though! It is hard to imagine how they managed to hold everything together.

As mentioned in the embedded clip, it has the potential to get much worse in the future where even the boats are completely autonomous.

Also mentioned was that 90% of the orders came from internet, so 10% is not? Surprised me a little. Got the feeling that they have lots of manual paperwork even for internet orders, which probably helped a lot.

Re:Maersk reinstalled thousands of servers and workstations ... by andy_ppp on Sunday January 28, 2018 @05:25PM (#16253948)
From what I understand shipping is still an industry that operates between a few connected people. You still can barely go online and book a set of containers onto a ship, you call someone you have a relationship with. I assume that’s a big part of it.
Re:Maersk reinstalled thousands of servers and workstations ... by tinymollusk on Sunday January 28, 2018 @08:26PM (#16254719)
They had years of organization using IT that built up a lot of inertia. As another commenter said, 10 days is probably well below the threshold of their typical shipment turnover.

My guess is they'd quickly suffer potentially company-ending losses if they were to stop using modern IT altogether.

Re:Maersk reinstalled thousands of servers and workstations ... by bkor on Monday January 29, 2018 @05:35AM (#16256677)
> For me, the most surprising thing is that Maersk was able to carry on

It takes a while for one vessel to reach the next port/terminal. Once it left the previous terminal you'll get a stowage layout. Basically showing all the containers on a vessel and where they're supposed to unload/discharge.

You can use that file to tell the next terminal to unload whatever the file mentions. Then load whatever customers handed into the terminal and have the terminal handle the new stowage layout.

Thanks to loads of practical problems terminals are pretty capable to handle loads of things for you.

Obviously if the terminal itself is also affected it's a way bigger problem.

Re:Maersk reinstalled thousands of servers and workstations ... by jlgaddis on Sunday January 28, 2018 @04:22PM (#16253592)
I can't imagine the stress and working conditions those "in the trenches" IT folks dealt with over those ten days.

Hopefully they all got rewarded with, at minimum, a few days off to relax and recover after everything got sorted out.

Re:Maersk reinstalled thousands of servers and workstations ... by post_break on Sunday January 28, 2018 @04:25PM (#16253614)
As someone in IT, they probably didn't get anything, maybe a friendly smile and a handshake.
Re:Maersk reinstalled thousands of servers and workstations ... by bkor on Monday January 29, 2018 @05:38AM (#16256688)
> As someone in IT, they probably didn't get anything, maybe a friendly smile and a handshake.

At least in one country they held a proper thank you party plus thank you gift certificates. That party was for everyone, not just IT.

Re:Maersk reinstalled thousands of servers and workstations ... by blibble on Sunday January 28, 2018 @05:31PM (#16253976)
now the company has realised it can essentially work without IT, I suspect their reward might be a severance package a few months down the line
Re:Maersk reinstalled thousands of servers and workstations ... by lfam on Sunday January 28, 2018 @06:06PM (#16254143)
The article says that Maersk is taking the opposite approach. Quoting the CEO, Jim Hagemann Snabe:

"And this was a wake-up call to become not just good —we actually have a plan to come in a situation where our ability to manage cyber-security becomes a competitive advantage."

Re:Maersk reinstalled thousands of servers and workstations ... by s0rce on Sunday January 28, 2018 @11:35PM (#16255359)
With a $300M loss...
Re:Maersk reinstalled thousands of servers and workstations ... by erk__ on Sunday January 28, 2018 @05:08PM (#16253877)
I am pretty sure That Mærsk also invested a lot of money to prepere better for this in the future.
Re:Maersk reinstalled thousands of servers and workstations ... by koalalorenzo on Monday January 29, 2018 @05:53AM (#16256743)
In the same old fashion, but yes: IMHO the people mindset is the same. Nothing changed really.
Re:Maersk reinstalled thousands of servers and workstations ... by erikbern on Sunday January 28, 2018 @10:58PM (#16255242)
They should have used containers.
Re:Maersk reinstalled thousands of servers and workstations ... by koalalorenzo on Monday January 29, 2018 @05:52AM (#16256736)
Since I am getting sick of this joke: no we don't! :P
Re:Maersk reinstalled thousands of servers and workstations ... by canada_dry on Sunday January 28, 2018 @06:21PM (#16254213)
Being a (former/retired) IT Exec this kinda thing used to keep me up at night!

Sadly, most C-level folks don't understand that even gold-standard cyber security is far from perfect, and IT can't do much against dumb, careless users.

Re:Maersk reinstalled thousands of servers and workstations ... by cm2187 on Monday January 29, 2018 @03:52AM (#16256267)
But in this case they had not patched their windows machines for a vulnerability that Microsoft fixed months before. It was really a case of dumb, careless IT.
Re:Maersk reinstalled thousands of servers and workstations ... by toomuchtodo on Sunday January 28, 2018 @09:59PM (#16255040)
Disaster recovery/business continuity is like insurance: you grumble about paying it (or don’t pay it at all) and then you’re happy you did pay for it when disaster strikes.
Re:Maersk reinstalled thousands of servers and workstations ... by Simulacra on Sunday January 28, 2018 @03:29PM (#16253286)
Wow, that is a monumental effort. We can't even get a critical software patch in less than three months. I would be very curious to see what their new infrastructure defense plan looks like now compared to the old
Re:Maersk reinstalled thousands of servers and workstations ... by sneak on Sunday January 28, 2018 @05:18PM (#16253919)
The real cost of continuing to run Windows.
Re:Maersk reinstalled thousands of servers and workstations ... by wglb on Sunday January 28, 2018 @06:04PM (#16254130)
Re:Maersk reinstalled thousands of servers and workstations ... by yjftsjthsd-h on Sunday January 28, 2018 @06:22PM (#16254216)
In an article from late 2017:

> kernel 2.6.24.2, which was compiled back in 2008.

Next you'll tell us XP is vulnerable. Like, Linux isn't 100% immune, but it has a head start.

Re:Maersk reinstalled thousands of servers and workstations ... by TeMPOraL on Monday January 29, 2018 @01:29AM (#16255704)
Well, wasn't this attack all about unpatched XPs? :P.
Re:Maersk reinstalled thousands of servers and workstations ... by cm2187 on Monday January 29, 2018 @03:54AM (#16256272)
Linux had its fair share of major vulnerabilities. Starting with heartbleed.
Re:Maersk reinstalled thousands of servers and workstations ... by hawski on Monday January 29, 2018 @05:56AM (#16256754)
Heartbleed was not a Linux issue per se. It was OpenSSL issue, which is used also on Windows.
Re:Maersk reinstalled thousands of servers and workstations ... by lmm on Monday January 29, 2018 @05:22AM (#16256620)
Heartbleed was a bug in OpenSSL code running on any platform, including Windows.
Re:Maersk reinstalled thousands of servers and workstations ... by cm2187 on Monday January 29, 2018 @05:30AM (#16256647)
IIS doesn't use OpenSSL. It mostly affected linux boxes.
Re:Maersk reinstalled thousands of servers and workstations ... by lmm on Monday January 29, 2018 @05:51AM (#16256730)
Sure, because most people using e.g. Apache run Linux, but it doesn't make sense to count it as a "linux bug" any more than it makes sense to count e.g. Ruby bugs as Linux bugs, or .net CLR bugs as Windows bugs.
Re:Maersk reinstalled thousands of servers and workstations ... by v_lisivka on Sunday January 28, 2018 @08:07PM (#16254643)
Rule of thumb says that it's ok to have 20 windows servers per 1 admin, or 2000 Linux servers per 1 admin. Linux is cheaper.
Re:Maersk reinstalled thousands of servers and workstations ... by sneak on Sunday January 28, 2018 @09:47PM (#16254988)
Nothing’s immune. That doesn’t mean it’s smart running the #1 (by a huge margin) malware target OS.
Re:Maersk reinstalled thousands of servers and workstations ... by colejohnson66 on Monday January 29, 2018 @01:26AM (#16255692)
Saying you should use Linux because it’s not the #1 malware target sounds like security through obscurity to me.
Re:Maersk reinstalled thousands of servers and workstations ... by kelnos on Monday January 29, 2018 @01:52AM (#16255790)
There's nothing wrong with security through obscurity, as long as it is merely one part in a defense-in-depth strategy, and can help you on occasion when other defenses fail in unexpected ways.
Re:Maersk reinstalled thousands of servers and workstations ... by sneak on Wednesday January 31, 2018 @02:49PM (#16276944)
Who said that?
Re:Maersk reinstalled thousands of servers and workstations ... by rainer_zufall on Monday January 29, 2018 @12:34PM (#16259042)
Does anyone know which software they use to roll out so many systems?