News for hackers. Stuff that matters.
Re:Texas was minutes away from months-long power outages, of... by defen on Thursday February 18, 2021 @04:34PM (#26185952)
I know nothing about utility-scale electricity distribution ... can someone explain how "demand > supply" leads to things (apparently literally) blowing up?
Re:Texas was minutes away from months-long power outages, of... by rhodozelia on Thursday February 18, 2021 @04:46PM (#26186110)
Generators are spinning machines with mechanical power applied to the shaft via some type of prime mover such as a gas turbine. The gas turbine has a throttle which controls how much mechanical power it makes, and how much electrical power the generator makes.

When the power applied to the generator shaft by the turbine is greater than the load placed on the generator by people’s lights and heaters, the turbine and generator speed up until the gas turbine backs off the throttle a bit to balance power out of the turbine in to the generator and power out of the generator.

If people turn on more lights and heaters and power out of the generator is greater than power in, the imbalance is made up from the rotating machinery’s inertia and it slows down. If the imbalance isn’t corrected by opening the throttle on the gas turbine it slows down until it stops rotating.

If the throttle is already wide open and demand exceeds supply the machines slow down until they stop and nobody gets any electricity, so they have to have some blackouts to reduce the load so that demand is not greater than supply.

Nothing should blow up though.

Re:Texas was minutes away from months-long power outages, of... by itronitron on Thursday February 18, 2021 @05:15PM (#26186468)
A fun science demonstration is to take an old-timey hand crank electrical generator, crank it by hand with nothing attached to the power output, then attach a light bulb and crank it a second time. The amount of resistance resulting from the light bulb using the generated electricity is remarkable. It is much harder to turn the crank with the light bulb attached and that really drives home that electrical devices really do use a lot of energy.
Re:Texas was minutes away from months-long power outages, of... by WJW on Thursday February 18, 2021 @05:10PM (#26186388)
This is entirely correct, I would like to add that every single grid-attached power generator I have ever seen has had ample safety devices that would safely disconnect and shut off the generator well before physical damage could occur.

Something "blowing up" is only possible if every possible safety device failed.

Re:Texas was minutes away from months-long power outages, of... by rhodozelia on Thursday February 18, 2021 @05:33PM (#26186716)
true. generally if frequency is greater than 62 Hz or less than 57 Hz generators will be allowed to disconnect immediately to avoid damaging themselves.

Offhand I am not quite sure what the damage from off-nominal frequency operation is though - inductance is jwl - any impedances with a lagging reactive component will vary proportionally with frequency. a decrease in frequency will cause an increase in current. I imagine off-nominal frequency operation causes something to get hot.

Re:Texas was minutes away from months-long power outages, of... by mschaef on Thursday February 18, 2021 @05:43PM (#26186846)
> Offhand I am not quite sure what the damage from off-nominal frequency operation

I can think of at least a couple potential issues:

* If the grid frequency is lower than the frequency the machine is trying to turn, it's possible for the generator to jump forward and mechanically slam forward to the next pole.

* Some turbines have problems running too far off rated speed. (IIRC, torsional resonance issues.)

Re:Texas was minutes away from months-long power outages, of... by rhodozelia on Thursday February 18, 2021 @06:04PM (#26187084)
Resonance slightly above rated speed I have heard of.

Pole slipping, a loss of synchronism, i would associate more with transient events such as large losses of load or generation that cause big changes in power flows as opposed to a sustained operation at 58 Hz. Loss of synchronism or pole Slipping should be protected against by out of step protection elements?

Re:Texas was minutes away from months-long power outages, of... by thu2111 on Friday February 19, 2021 @10:14AM (#26194015)
I think their assumption is that if the grid collapses completely it will take place in a short period of wild frequency fluctuations as generators trip out, demand swings wildly and so on, so you could get very large frequency distortions very quickly, perhaps too quickly for some safety equipment to kick in? Or some equipment can disconnect itself "safely" in the sense of not putting humans in danger but may still sustain damage to itself.

I guess it's also quite hard to regularly test this sort of safety mechanism end to end, so there's surely a fear that it wouldn't work.

Re:Texas was minutes away from months-long power outages, of... by mschaef on Friday February 19, 2021 @10:43AM (#26194307)
> if the grid collapses completely it will take place in a short period of wild frequency fluctuations as generators trip out, demand swings wildly and so on, so you could get very large frequency distortions very quickly, perhaps too quickly for some safety equipment to kick in? Or some equipment can disconnect itself "safely" in the sense of not putting humans in danger but may still sustain damage to itself.

My general read on this is that generation can trip off more easily than most kinds of load, so when reserves are low, they need to be more explicitly proactive about taking load offline. That In case a generator trips offline that consumes all the remaining responsiveness and takes the grid to a collapse. I assume what a total collapse looks like is every generator tripping offline in a matter of a few seconds or minutes.

Hopefully, though, there'd be enough ability to disconnect transmission lines to isolate the failure to a specific part of the grid. Back in the 2003 NY blackout, the failure was isolated to the northeast because NJ disconnected its grid. The last couple steps of this timeline are illustrative:

https://en.wikipedia.org/wiki/Northeast_blackout_of_2003#Seq...

* ..,

* 4:11:57 p.m. The Keith-Waterman, Bunce Creek-Scott 230 kV lines and the St. Clair–Lambton #1 230 kV line and #2 345 kV line between Michigan and Ontario fail.

* 4:12:03 p.m. Windsor, Ontario, and surrounding areas drop off the grid.

* 4:12:58 p.m. Northern New Jersey separates its power-grids from New York and the Philadelphia area, causing a cascade of failing secondary generator plants along the New Jersey coast and throughout the inland regions west.

* 4:13 p.m. End of cascading failure. 256 power plants are off-line, 85% of which went offline after the grid separations occurred, most due to the action of automatic protective controls.

I know people that were working at PJM when that happened, and they tell stories of essentially continual announcements on the PA system announcing various emergency states they needed to respond to. Which they did.

> I guess it's also quite hard to regularly test this sort of safety mechanism end to end, so there's surely a fear that it wouldn't work.

Generators trip all the time. In fact, one of my first jobs was working at a TX based utility company analyzing the frequency disturbances that occur when generators trip offline. You can learn a lot about the way the grid responds by looking at the plot of frequency over time right around a trip.

Re:Texas was minutes away from months-long power outages, of... by amluto on Thursday February 18, 2021 @05:19PM (#26186529)
It’s a bit more complicated than this. A lot of big generators are induction generators, and the associated electronics can make a tradeoff of voltage and frequency. In a grid setting, the frequency will be maintained at 60 Hz plus or minus not very much. Everything is, hopefully, arranged to maintain the correct voltage and frequency. The actual mechanism behind this have always struck me as arcane and somewhat fragile, especially in light of modern loads that don’t act at all like resistors under varying voltage conditions.
Re:Texas was minutes away from months-long power outages, of... by rhodozelia on Thursday February 18, 2021 @05:31PM (#26186698)
are you referring to modern wind turbines and solar farms which have their output converted through power electronics?

I have literally never seen a large induction machine used as a generator, every machine larger than 300 kW I have seen has been a synchronous machine. If I've read about 600 different power plants, 598 of them have had synchronous machines.

Where in the world, what application, and in what age of installation are engineers choosing induction machines + power electronics over synchronous machines?

Re:Texas was minutes away from months-long power outages, of... by thu2111 on Friday February 19, 2021 @10:17AM (#26194045)
I think it's indeed a reference to wind and solar.

There's a discussion of the problems posed by loss of system 'inertia' here:

https://www.drax.com/technology/shock-absorbers-keeping-grid...

Re:Texas was minutes away from months-long power outages, of... by jasonwatkinspdx on Thursday February 18, 2021 @05:26PM (#26186640)
An important detail here is that as imbalances happen it pulls the grid frequency around. Bad things can happen to various systems if the frequency goes too far out of range. Yes in theory all equipment should be able to trip fail safe, but in practice a power grid has a ton of inertia and there are limits to how you can engineer around that. Even if the grid blacks itself off safely with minimal equipment damage, bringing it back up is hugely more complex than just resetting the breakers on your house/apartment.
Re:Texas was minutes away from months-long power outages, of... by floxy on Thursday February 18, 2021 @07:41PM (#26187959)
Re:Texas was minutes away from months-long power outages, of... by mschaef on Thursday February 18, 2021 @05:20PM (#26186556)
Given that there are no great ways to store large amounts of power at utility scale, the first thing to know is that demand and supply have to always match. (There's mechanical inertia in the grid, pumped storage, and batteries, but those are modest in capacity and mostly short-term storage.)

So, absent ways to support the full demand, the grid essentially slows down. You can see this if you look at the frequency of the current at the wall outlet - the nominal 60Hz will dip, and it will also dip across the entire rest of the grid. (In this case, across the entire state of Texas.)

The next thing you need to know is that the rotational speed of the generators on the grid are bound to the grid's operating frequency. The frequency lags, and so does the speed of all the generators and sychronous motors connected to the grid. (In this case, across the entire state of Texas.)

What happens then is that each generator with the capacity to do so will throttle up, try to hold the 60Hz, fail to do so, and potentially do things like jump poles. This is where a rotating assembly trying to maintain 60Hz on, say, a 58Hz grid, mechanically skips cycles, and jumps forward suddenly, and potentially catastrophically. It can literally physically destroy equipment.

So what happens instead is that the generator trips, goes offline to protect itself, and the overall load imbalance problem gets worse. This is why grid operators pay a great deal of attention to the amount of reserve generating capacity they have online at any moment. Some of this means the ability to turn on a generator that's completely off, but it's also very important that there be enough generation with the capability of throttling up quickly to deal with short term transient failures (like other generation tripping offline, etc.).

Re:Texas was minutes away from months-long power outages, of... by jarpadat on Thursday February 18, 2021 @04:54PM (#26186196)
I’m affected and have been stress-reading every report ever about this since it began.

The short version is, to a first approximation power generation requires power first. For many of the same reasons your house with solar panels is itself on the grid - your one source is not a reliable way to operate all the equipment on your home.

Another issue is that on a grid everything has to run on similar clock. The process of aligning the clock frequency takes time and you have to do it for every node.

There are special plans to “black start” a power grid but if you didn’t prepare adequately for cold which happens every decade, you definitely didn’t plan adequately for black start which happens theoretically never.

Re:Texas was minutes away from months-long power outages, of... by rhodozelia on Thursday February 18, 2021 @06:31PM (#26187333)
"black start" of a power plant, the ability for it to start the generator rotation, provide current to the field, and energize a power line is something that costs more to provide when building a plant. Utilities that have old school focus on reliability will generally have that capability. Private companies who just generate power to make money don't have any incentive to pay for that function. If the grid is dead there is nobody to use that power.

Another cool thing that utility power plants can do is feed themselves - keep the pumps running, lights on, heaters running, coffee hot, etc in the power plant using the power from the generator in the plant even if it is not connected to the transmission system. Most privately owned power plants forego this ability along with black start and will have a diesel generator instead.

Re:Texas was minutes away from months-long power outages, of... by LatteLazy on Thursday February 18, 2021 @05:11PM (#26186418)
I am also curious, I can't think how it would happen either. Here are 2 guesses that are probably wrong that people can correct:

Maybe, when demand>supply they have to cut demand, but if you cut a whole chunk of grid demand is suddenly a lot less than supply and there is a spike elsewhere causing issues?

I know the frequency of the power supply fluctuates when the there is too much/little power. So if there is too much demand, maybe electronics that work at the correct frequency break at the new bad frequency? Or maybe power stations get out of sink and one pushes positive voltage when the other pushes negative?

Maybe it's a logistical thing: imagine if you have 3 neighbourhoods in a row ABC, and you supply power to A and C from high capacity systems, and power B indirectly from each side. Now, if you want to turn off A (rolling blackout), the operator might not realise that doing so means B has to pull all its power from Cs supply. So Cs supply suddenly sees a 30% jump and maybe the transformers etc can't handle that and catch fire?

Those are my guesses.

Re:Texas was minutes away from months-long power outages, of... by ceejayoz on Thursday February 18, 2021 @04:42PM (#26186056)
From the article:

> The worst case scenario: Demand for power overwhelms the supply of power generation available on the grid, causing equipment to catch fire, substations to blow and power lines to go down.

Re:Texas was minutes away from months-long power outages, of... by defen on Thursday February 18, 2021 @04:46PM (#26186106)
Right, I mean physically what is happening that causes that.
Re:Texas was minutes away from months-long power outages, of... by tibbydudeza on Thursday February 18, 2021 @04:56PM (#26186222)
My understanding is the entire grid needs to be at a stable frequency of either 50 or 60 Hz (affected by demand) , if a generator goes out of sync/phase you are going have major trouble.

Bringing up a grid from scratch from a total blackout scenario is such a time consuming process so you rather throw consumers off with no warning even in the coldest winter imaginable.

Re:Texas was minutes away from months-long power outages, of... by ldbooth on Thursday February 18, 2021 @05:16PM (#26186493)
I was scratching my head too. It's also a broad statement to cover who knows what but I read "ground faults" while getting our tinker on. Agree the frequency and voltage would drop out of phase tolerance. That shouldnt cause fires unless there's a spinning resource that fails. The sudden availability or lack of power would create imbalances across the grid, frankly I don't think they know what would break. Might also drop one circuit and accidentally overload another, there's lots of failure points so they just broad stroke it.
Re:Texas was minutes away from months-long power outages, of... by tibbydudeza on Thursday February 18, 2021 @05:52PM (#26186945)
Yep lot's of failure points due to the cold weather that affected a nuclear power plant (faulty sensors in the inflow tripped it into safe mode and took it off the grid) , the natural gas wells (water vapour in the gas freezes) and the infamous wind turbines (no heating/anti-freeze).

The problem was they could not import power from the Western grid (little interconnect capability for political reasons) and meanwhile El Paso and some border counties is humming right along because they are tied to the Western grid and not ERCOT.

Re:Texas was minutes away from months-long power outages, of... by yabones on Thursday February 18, 2021 @05:11PM (#26186415)
There's a few factors that could lead to widespread damage. The other commenter addressed grid frequency which is also important, but the real issue is over drawing current. (Disclaimer - I am not an electrician).

In most houses you have a fuse box or breaker panel, this is to prevent the appliances from drawing too much current. Conductors heat up according to their resistance and the current applied to them, so if a circuit with 14 gauge wires suddenly has a load of 2000 watts applied to it, it will begin to heat up. If this load is applied for too long it can get hot enough to start a fire inside the walls of a house. This is why we have breakers - they will cut the circuit if it exceeds a safe level for too long.

Power grids don't really have breakers. They have complex control systems and highly trained operators to serve the same purpose. If transmission lines have too much current applied to them they can expand and sag, catching on other wires or structures causing damage and outages. Overloads can also damage substations and transformers for similar reasons (heat, thermal expansion). In this case, they did exactly what they had to. They cut the power to millions of people for a few days and saved their infrastructure from months of repairs if things went catastrophically wrong.

Re:Texas was minutes away from months-long power outages, of... by caddywompus on Thursday February 18, 2021 @05:20PM (#26186552)
I think you're right. The system expects a certain voltage, if the load increases too much, the generators would struggle to keep up, and the voltage would begin to drop.

However, the total amount of power would remain the same, which means that the electrical current would increase to compensate.

The problem is that the amount of current increase is a square law increase compared to the voltage,

V = I * R

P = V * I

P = I^2 * R

So now you have equipment having to deal with the heat created by resistance (since nothing is 100% efficient), but way more heat than it can handle.

I'm not an electrician either, and my knowledge of AC isn't great, so I'm sure there are other reasons too, like the frequency is also effected if the generator can't keep up with the load, and that causes other problems

Re:Texas was minutes away from months-long power outages, of... by morio on Thursday February 18, 2021 @05:26PM (#26186637)
There are many failure points but one are the high voltage carrying power lines themselves.

As they reach their capacity they start to droop quite a bit, which in itself can cause issues like touching something they usually would not.

Because HV lines are made from aluminum they loose some of their strength as they are heated up. Above 100C or so the aluminum anneals which means strength is lost. Now you just need some wind shaking those lines and the whole thing breaks. If you have reached that point you likely have to replace the entire length of the HV line.

Of course the cold weather counters some of this. But only to a certain point.

If just one HV line fails you now have to deal with the cascading effect as the current is load balanced on the remaining HV lines.

Re:Texas was minutes away from months-long power outages, of... by ChrisLomont on Thursday February 18, 2021 @05:08PM (#26186368)
>an someone explain how "demand > supply" leads to things (apparently literally) blowing up?

Imagine hitting your car accelerator to the floor in neutral - things blow up when spun too fast.

This was what was going to happen to parts of the grid, and then the now higher per unit demand would blow another item, etc...

They had to disconnect things so this didn't happen.

Re:Texas was minutes away from months-long power outages, of... by mschaef on Thursday February 18, 2021 @05:22PM (#26186571)
> Imagine hitting your car accelerator to the floor in neutral - things blow up when spun too fast.

The problem here is the opposite - put your car in top gear, drive up a steep hill, and watch the thing lag and lag behind as it tries to keep up with a load its unable to support.

Re:Texas was minutes away from months-long power outages, of... by Retric on Thursday February 18, 2021 @04:42PM (#26186062)
Demand > supply means stuff is operating outside of how it’s designed which can cause a lot of rather uninteresting problems relating to impedance mismatch etc. IMO, the reverse is more interesting.

Let’s suppose your adding 200GW to an electric grid, lose 10GW to transmission losses and consume 190GW. Everything is nice and balanced.

Now suppose someone adds another 50GW to the grid, where does that energy go? It ends up as waste heat, but while a few KW here and there is meaningless at that scale 50GW will melt or burn some important stuff. That’s a rather extreme example, but generators are only designed to spin so fast etc.

Re:Texas was minutes away from months-long power outages, of... by jeffbee on Thursday February 18, 2021 @04:44PM (#26186082)
Electricity doesn't work like that. You don't "add energy" to wires. It's the loads that draw the current.
Re:Texas was minutes away from months-long power outages, of... by Retric on Thursday February 18, 2021 @04:47PM (#26186126)
Wires behave differently when supplying AC power over very long distances from multiple power stations to many different loads. You have a given force from your steam turbine or whatever turning a generator which is frequency matched to the grid. If the grid doesn’t use as much power the turbine is still supply the same force so the generator spins faster and the grid frequency increases.

Impedance losses for example are proportional to frequency, and a generator spinning faster also produces more waste heat etc.

PS: Many systems are designed to avoid this situation, but a prior assumption is this stuff has failed.

Re:Texas was minutes away from months-long power outages, of... by _Microft on Thursday February 18, 2021 @04:52PM (#26186177)
Line frequency increases if the supply is larger than the demand.