News for hackers. Stuff that matters.
Re:Three people in critical condition from Google data cente... by belkarx on Tuesday August 09, 2022 @12:27AM (#32394145)
Arc flashes, for the uninitiated, are short-circuits caused by electricity flowing through through air (partially ionized) between two live conductors. The arc of electricity causes an arc blast, which is a huge fiery shockwave that can (and does) blast through metal and all surroundings
Re:Three people in critical condition from Google data cente... by jfoutz on Tuesday August 09, 2022 @02:19AM (#32394781)
Yes. Like this, but not intentional - https://www.youtube.com/watch?v=GMbN9nb3qyk That alone looks very scary to me, indoors? accidentally? gah.
Re:Three people in critical condition from Google data cente... by moring on Tuesday August 09, 2022 @02:32AM (#32394859)
I don't get it. There is a huge arc in that video, but no shockwave.

edit: It's explained further down the thread.

Re:Three people in critical condition from Google data cente... by mc32 on Tuesday August 09, 2022 @11:32AM (#32399346)
The question is what caused it?

One scenario is equipment failure. On-site person was not familiar, called for back-up, back ups did not don proper PPE nor follow proper protocol. Another scenario is the equipment failed in a manner not envisioned by the designers of the protocols and were really unlucky.

In any case, this is really sad and bad and hope they recover.

Re:Three people in critical condition from Google data cente... by gaudat on Tuesday August 09, 2022 @08:27PM (#32406475)
It may be something simple as a shor tbetween the phases. In power engineering these are called fault conditions and the currents can run up to tens of thousands of amps before the circuit breaker acts. Not sure if you can imagine the magnitude but that is why the standard PPE in working with on-line low voltage (<1 kV) switchgear is a full body silver arc flash suit.

Now imagine if the datacenter guys need to wear that suit before going down the racks...

Re:Three people in critical condition from Google data cente... by mc32 on Tuesday August 09, 2022 @08:52PM (#32406649)
There are unknowns to us. But, no I don't imagine regular detacenter folks walking around in PPE --unless they are going in to these segregated areas to address a known issue. Maybe it was totally unpredictable --like someone just walking in proximity to said equipment just as it was about to catastrophically fail, but in most accidents there was a known bad condition, techs go in to check things out and then go back in to bring in the certified electrician or whatever and in those instances people are supposed to don proper equipment and follow certain procedure typically involving cutting power away from the faulty equipment, though sometimes people cut the wrong power or altogether forget that step in the SOP. Sometimes the diagrams are outdated --CBF is old now, I think?
Re:Three people in critical condition from Google data cente... by HKH2 on Tuesday August 09, 2022 @12:38AM (#32394209)
Like a giant welder?
Re:Three people in critical condition from Google data cente... by atoav on Tuesday August 09, 2022 @01:38AM (#32394528)
You are forbidden by law (and mandated to report) such arcs in Europe. In Germany people who work in such environments have to have a few special certificates and PPE that looks like they are trying to defuse a bomb.

Arc flashes like that can burn with 10000 to 15000 degrees celsius and if you are close by not only will they send you flying, you are probably dead before you hit the ground.

And trying to extinguish such an arc can be quite a challenge.

Re:Three people in critical condition from Google data cente... by Semaphor on Tuesday August 09, 2022 @02:24AM (#32394806)
> You are forbidden by law (and mandated to report) such arcs in Europe.

You are forbidden by law to have an accident? This misunderstanding is probably because of my lack of knowledge, but what do you mean?

> And trying to extinguish such an arc can be quite a challenge.

I only read about them in this thread, so I’m probably missing something again, but isn’t it an instantaneous thing and not something you need to extinguish?

Re:Three people in critical condition from Google data cente... by Doxin on Tuesday August 09, 2022 @02:49AM (#32394943)
You are forbidden by law to have an accident of this nature. Yes. If you do cause an accident of this nature you are criminally negligent no matter the circumstances leading up to the accident because it's your duty to ensure it doesn't.

Of course you need to keep in mind that courts in Europe tend to not hold to the letter of the law quite as strongly as US courts seem to do. Laws like these are mostly so the judicial system gets booted up when something awful happens. Might still turn out that no one was responsible for the accident. Might turn out the boss of the guy throwing the switch was responsible. Or the equipment manufacturer.

Re:Three people in critical condition from Google data cente... by chrisseaton on Tuesday August 09, 2022 @05:53AM (#32395915)
Doesn’t make having an accident illegal make it more likely that people won’t report the accidents and won’t learn and improve?
Re:Three people in critical condition from Google data cente... by smcl on Tuesday August 09, 2022 @06:27AM (#32396065)
It sounds like an accident of this nature would be quite difficult to cover-up, and given that doing so would be a criminal offense I imagine it's an easy decision for those involved to just go ahead and file the appropriate reports.
Re:Three people in critical condition from Google data cente... by Doxin on Tuesday August 09, 2022 @06:49AM (#32396164)
Making this sort of accident legal makes it definitive that people won't report it. Because there's no requirement to.
Re:Three people in critical condition from Google data cente... by chrisseaton on Tuesday August 09, 2022 @06:56AM (#32396214)
> Making this sort of accident legal makes it definitive that people won't report it. Because there's no requirement to.

Huh? No. Make it a legal requirement to report it. Then there is all the incentive to report it and no incentive not to.

If it's illegal to have the accident, then attempting to cover it up is the most rational course of action, because if you can not report and get away with it then that's the only way you aren't punished.

If instead it's just a legal requirement to report it, then reporting it is the most rational course of action, because you can report and then aren't punished.

Re:Three people in critical condition from Google data cente... by Macha on Tuesday August 09, 2022 @08:02AM (#32396675)
Right, but inserting a legal incentive to do everything to prevent the accident is considered more important than inserting a legal incentive to report when it happens. If there was no consequences to having the accident, sure, you'd know all about the ones that happen, but there wouldn't be much incentive for investing in prevention
Re:Three people in critical condition from Google data cente... by atoav on Tuesday August 09, 2022 @03:45PM (#32403196)
We are talking about accidents that mostly happen on the ownership border between house owners and grid maintainers. If you can cover up an arc flash there in such a way they will not notice on their next visit without dying, you might deserve to get away with it.
Re:Three people in critical condition from Google data cente... by c_o_n_v_e_x on Tuesday August 09, 2022 @02:45AM (#32394926)
>You are forbidden by law to have an accident? This misunderstanding is probably because of my lack of knowledge, but what do you mean?

There are systems (permits to work, lock out tag out, risk assessments, etc.) in place to maintain worker safety mandated by law. An incident of this kind typically means that there was a failure with one of more levels of the system... Someone did something they weren't supposed to, they turned off the wrong equipment, they didn't turn off the equipment before working on it, etc.

> And trying to extinguish such an arc can be quite a challenge.

Once established, an arc can continue as long as current is still flowing. It's possible for arcs to continue and still be below the trip current of the upstream circuit protection. In non-catastrophic scenarios, you have mechanisms to "quench" an arc... one method is to use compressed air.. like blowing out a very hot, angry candle on a birthday cake.

Re:Three people in critical condition from Google data cente... by bcbrown on Tuesday August 09, 2022 @04:09AM (#32395364)
> isn’t it an instantaneous thing and not something you need to extinguish?

I'm not familiar with the details of this accident, but in general, not necessarily. Air is normally an excellent insulator. But when voltages are high enough, the molecules ionize, become conductive and transmit current. This is a path-dependent process, meaning that once it begins, the voltage can drop below what is required to begin the process while maintaining current.

So an arc caused by a system with low capacitance (or total count of free electrons on the negative side) may quickly transfer all the free electrons, lowering the voltage enough to close the circuit. But anything connected to mains power will have an essentially infinite source of more electrons, meaning there won't be a voltage drop big enough to close the circuit. I'm being a little handwavy with the physics concepts, but hopefully the point is clear.

This can be thought of as a sort of hysteresis: the dependence of the state of a system on its history.

Edit: If you want to go down a rabbit hole, check out the solutions used in power substations as circuit breakers. It's not enough just to mechanically separate the connection, so they use systems like these:

https://en.wikipedia.org/wiki/Sulfur_hexafluoride_circuit_br...

https://www.codrey.com/electrical/air-blast-circuit-breaker/

Re:Three people in critical condition from Google data cente... by MayeulC on Tuesday August 09, 2022 @04:49AM (#32395594)
Maybe not only accidental arcs? If only to prevent accidental EMI and jamming of frequencies. There used to be spark transmitters: https://en.wikipedia.org/wiki/Spark-gap_transmitter
Re:Three people in critical condition from Google data cente... by c_o_n_v_e_x on Tuesday August 09, 2022 @02:25AM (#32394809)
Arcs are rated by calories per square centimeter. There's a lot factors that go into arc ratings, more than just voltage. To give a frame of reference, a flame from a lighter on the tip of your finger for 1 second is considered 1 calorie. Arc flash PPE ratings start at 4 calories and can go above 100.

The high cal rated PPE does indeed look like bomb squad gear.

Re:Three people in critical condition from Google data cente... by MayeulC on Tuesday August 09, 2022 @04:17AM (#32395416)
It's weird to rate it in calories (∝ joules, W.s) instead of Watts.
Re:Three people in critical condition from Google data cente... by closewith on Tuesday August 09, 2022 @05:12AM (#32395698)
Energy is used rather than power as it's the cumulative effect in energy incidents that causes damage. In most of the world, the unit is J/cm2, though.
Re:Three people in critical condition from Google data cente... by MayeulC on Saturday August 13, 2022 @07:53AM (#32449042)
Ah, all right, I understand if it's used to rate accumulated damage for incidents. For ongoing arcs, or comparing them however, I think it's a weird.

A small arc, not even dangerous to touch, could have the same energy rating as a gigantic one if you measure the first over a year and the second over a millisecond.

Re:Three people in critical condition from Google data cente... by closewith on Tuesday August 09, 2022 @05:15AM (#32395712)
> You are forbidden by law (and mandated to report) such arcs in Europe.

When you say in Europe, normally that refers to an EU Regulation or Directive (since other legislation varies wildly in member states). What are you referring to here?

Re:Three people in critical condition from Google data cente... by lima on Tuesday August 09, 2022 @09:55AM (#32397951)
Not sure why this comment was downvoted, I'm curious too.
Re:Three people in critical condition from Google data cente... by vsnf on Tuesday August 09, 2022 @01:57AM (#32394642)
> will they send you flying

What is the mechanism by which an object would be propelled when hit by such an arc?

Re:Three people in critical condition from Google data cente... by schoen on Tuesday August 09, 2022 @02:04AM (#32394690)
Apparently mainly air pressure:

> The pressure wave created by an arc flash explosion has the force of thousands of pounds per square inch. This is powerful enough to knock down or throw nearby workers and at the same time cause damage to the eardrums, lungs, brain and other vital organs.

https://www.electricityforum.com/iep/arc-flash/electrical-ar...

(I didn't know that, and just found that explanation while looking for an answer to your question.)

A different risk is that electric current passing through the body could cause muscle convulsions, which could also cause someone to move or fall suddenly or erratically.

Re:Three people in critical condition from Google data cente... by ars on Tuesday August 09, 2022 @02:00AM (#32394660)
The heat of the arc rapidly expands air (like a lighting bolt), the heat of the arc also vaporizes the metal conductor, and the vaporized metal expands rapidly, and enormously.

It's basically a small explosion.

The hot metal then ignites in air adding even more heat to the explosion.

Re:Three people in critical condition from Google data cente... by mediaman on Tuesday August 09, 2022 @02:01AM (#32394667)
There is an explosive heating of air that’s like a bomb going off.
Re:Three people in critical condition from Google data cente... by ta2234234242 on Tuesday August 09, 2022 @12:52AM (#32394283)
More powerful than that. Think molten metal going everywhere or vaporizing into the air.
Re:Three people in critical condition from Google data cente... by changoplatanero on Tuesday August 09, 2022 @01:39AM (#32394532)
What happens to the vaporized metal after things cool down? Does it become dust?
Re:Three people in critical condition from Google data cente... by hexane360 on Tuesday August 09, 2022 @01:49AM (#32394597)
To be more specific, when GP says 'vaporized', they mean that part of it turns into plasma, and a lot more is sprayed out as molten droplets.

All of it freezes on contact with solid surfaces, often the eyes and exposed body parts of the victims.

Here's a good (SFW) video demonstrating an arc flash: https://youtu.be/bPifdBpcMRY

Re:Three people in critical condition from Google data cente... by ars on Tuesday August 09, 2022 @01:44AM (#32394565)
It becomes a metal oxide - don't breathe it. Basically it's powered rock, rocks are made of metal oxides.
Re:Three people in critical condition from Google data cente... by sebazzz on Tuesday August 09, 2022 @01:48AM (#32394591)
So basically the metal is going back to its original form. You might also call it accelerated rust.
Re:Three people in critical condition from Google data cente... by mlindner on Tuesday August 09, 2022 @02:30AM (#32394839)
Re:Three people in critical condition from Google data cente... by noja on Tuesday August 09, 2022 @07:16AM (#32396346)
Is this at all related to use of DC-power in datacentres?
Re:Three people in critical condition from Google data cente... by andruby on Tuesday August 09, 2022 @08:04AM (#32396687)
Afaik DC or AC can cause similar arc flashes. Typically you do need high voltages to bridge larger gaps.
Re:Three people in critical condition from Google data cente... by fzzzy on Tuesday August 09, 2022 @08:28PM (#32406480)
But ac will break the current at the alternating frequency and dc won't, so it is much harder to break a circuit with a high direct current. That's why monster movies have those huge knife switches.
Re:Three people in critical condition from Google data cente... by n0mn on Tuesday August 09, 2022 @07:47AM (#32396581)
What conditions can cause this to happen, and how does one prevent it?
Re:Three people in critical condition from Google data cente... by albrewer on Tuesday August 09, 2022 @12:11PM (#32399869)
Basically, the dielectric constant of air is not infinite. If the electrical potential is high enough between two points, air can conduct electricity.

If you put a small bit of conductor near two legs of a high voltage system, and that conductor has a path to ground, you can get now get an arc between a voltage source and the conductor (e.g., someone with a screwdriver loosening a screw is too close to a high voltage leg). This initial arc ionizes the air, which makes a pretty great conductor, so the amount of current flowing through the air increases really quickly as if it's flowing through an open circuit. The arc is superheated, can burn through steel (or flesh) quickly, and expands.

One of the hazards of working on high voltage is gasping in shock after an electrical arc - you inhale superheated plasma and it cooks your lungs instantly, so you suffocate.

Anyway, to prevent this, you don't cut corners as an electrician - you have proper insulation on all your tools and body parts to prevent this sort thing. You isolate legs of high voltage circuits, or de-energize the circuit you're working on with proper lock-out tag-out procedures.