News for hackers. Stuff that matters.
Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by dekhn on Thursday July 12, 2018 @05:04PM (#17518602)
Yes, that's why he was sent. In case something went wrong, he was qualified to fix it in production. I've come to an appreciation for people who have jobs where they don't do much most of the time, but when the shit hits the fan, they do exactly what was required.
Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by ghaff on Thursday July 12, 2018 @07:00PM (#17519329)
Pilots being another good example.

Years ago, I interviewed for a couple of positions with Schlumberger which, at the time, basically went out to oil wells and made various measurements. In a lot of ways, it was essentially a tech job but the reason you had engineers (assisted by techs) was that you had various training and, presumably, other background for when things went sideways.

Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by skrebbel on Friday July 13, 2018 @02:37AM (#17520933)
Fun story, though off topic: Schlumberger, having people all over the world on dangerous places such as oil platforms (but also in offices), thinks that physical safety for their employees is very important. They iterated on this so well for so many decades, that the primary cause of employee death is now traffic accidents, involving employees thousands of miles away from oil wells, on their way to the office.

Problem solved? Of course not. Employee safety is still a core KPI after all. So to address this problem, they installed a device in every company issued car which makes a BEEP! sound every time you drive aggressively (as judged by the device). After three BEEP!s you have to explain yourself to your boss.

Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by Ntrails on Friday July 13, 2018 @04:11AM (#17521271)
I do not want a company car that much...
Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by Cyph0n on Friday July 13, 2018 @03:47PM (#17526089)
Not only that, but not wearing a seatbelt (even as a passenger) can get you fired on the spot!

I was extremely impressed with the safety-first culture when I interned at Schlumberger a few years back. Your coworkers will yell at you if you simply step into a work area with proper PPE (hardhat, safety boots, etc.).

Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by raverbashing on Friday July 13, 2018 @08:56AM (#17522509)
Seems like your standard tachometer with an alerting system, a lot of companies have company cars with something similar
Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by drumttocs8 on Thursday July 12, 2018 @08:14PM (#17519645)
Power utility system operators, and other operators, are prime examples. Usually they just sit around and play farmville. But when it hits, they're the guys making complex decisions that affect when your power turns back on, and therefore how much money the company is losing until that happens.
Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by Cthulhu_ on Friday July 13, 2018 @03:44AM (#17521169)
Sometimes I wonder about an operator-type job like that, or security, or stuff like that - you know, sit around all day / night, browse the interwebs, play farmville or what have you, or do remote contract work to maintain skills and make a lot of money on the side. I had a neighbour who worked as a roadworks driver (like, the car that puts down cones and such), his job involved a lot of night shifts (good money) and then waiting for hours, occasionally moving - that gave him a lot of time to study.
Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by baxtr on Friday July 13, 2018 @04:23AM (#17521338)
Isn’t that exactly how firemen operate? In the business world these crucial roles are neglected and operations is driven towards maximum efficiencies. On the other hand, no one sane would suggest to utilize firemen better during their idle time.

EDIT: I guess that is the reason why consultants come into the picture

Re:The IceCube Neutrino Detector at the South Pole Hits Paydirt by munificent on Thursday July 12, 2018 @06:44PM (#17519244)
You can think about it much like load managing servers:

A job has a task load that varies over time. Those tasks also have some maximum latency before Bad Things happen. Thus, you need enough worker capacity to handle the peak load. You can allocate new workers on demand, but that also has a certain warm-up time which can be problematic if it's lower than the maximum latency.

For most white-collar service-oriented jobs, the variance of that task load is quite smooth. If you're a software dev not working directly on a production service, there aren't that many urgent surprises. Also, the max latency is really high — many tasks can be put off till tomorrow, next sprint, etc.

That makes it easy to allocate just enough workers to handle your average task load, keep them busy all day, and rarely need to spin up or spin down new ones.

But some jobs just have really high variance or really low latency. No one knows when a patient is going to show up at a trauma hospital but when one does, you need the surgeon on site right now and not somewhere stuck in traffic.

For that kind of work, the logical thing to do then is to have spare idle worker capacity. It seems inefficient, but it's cheaper than the cost of missing your latency targets when a spike happens.

You can reduce the idle capacity needed by reducing variance. For example if you flew all your trauma patients to one hospital, that actually lets you allocate surgeons more efficiently. Because then at that point you have spikes frequently enough that the aggregate number of patients in a given day becomes more consistently predictable. Instead of one surgeon each sitting on their thumbs most of the day at ten city hospitals, you get three surgeons that reliably have a couple of patients a day at the regional hospital. (But of course there is the overhead of getting all the patients there.)

Also, you can reduce idle capacity by lowering warm-up time. That's why doctors have pagers — it reduces the time between a patient coming in and the doctor being ready to help.

Or you can increase the max latency, though this can be hard based on the nature of the work. For example, if you're better able to stabilize patients, you may be able to wait longer until the doctor is on-site.

In this case, the "install Debian" job has really high variance. 99% of the time it's a click, 1% of the time, you've gotta know some Linux internals. The max latency is low because you don't want the IceCube array non-operational for long. And the warm-up time is really high — fly someone out to the South Pole.

So the reasonable response is to throw money at idle capacity and send someone out pre-emptively.