US - Canada blackout report

formatting link
I'm not an electrical engineeer, but, makes interesting reading.

Reply to
Richard
Loading thread data ...

I am an electrical engineer...in the power industry. I read the entire report last night and found it very interesting. I am dissappointed (but not surprised) that they did not include any discussion on how the deregulation of the power industry has helped create the situation that led to the blackout. Of course I never expected FERC to point a finger at itself.

Charles Perry P.E.

Reply to
Charles Perry

I have not read the complete report yet. So far I have not seen any comments about the level of staffing. Staffing has changed substantially due in part to de regulation.

former electrical operator

Reply to
anon

There was actually quite a bit about how deregulation changed things.

But, in this case anyway, it seems like the main issues were failure to trim trees, operators who did not think, operators who did not communciate effectively, a failure of a critical system (the alarm program) without anyone realizing it had failed, and a failure of the modeling software to function properly. Couple that with a poorly designed control room.

None of these things have anything to do with deregulation. They point to management failures.

Its also clear from the report that even though the operators were getting telephone reports from other people telling them something was seriously wrong, they took no action, instead believing their non-functioning alarm system.

Reply to
bob peterson

I'll have to ask my brother what he thinks. He was the one setting the switches on the AEP end of the debacle and saved their system from the rolling disaster.

Reply to
Keith R. Williams

Well, it takes money to trim trees. The distribution system seems to be forgotten in the de(or not)regulation scheme.

My older brother (I'm a simple low-voltage engineer type) claims the *real* problem is PF in the transmission system. The currents are so high that the system simply isn't stable. However, the politicians can't be bothered with "imaginary" power. I suggested the synchronous capacitor made from a retired coal plant (IIRC someone here talked about it). His response was, "who would fund such a folly 'wasting' power".

Sounds like the blame-game after TMI. Everyone pointing fingers, but no one fixing the problem.

Duh! Is there a difference? Deregulation left the transmission piece out of the puzzle. ...not that I like regulation, but...

My source tells me otherwise. They full-well knew *hours* ahead that they were in trouble, but pressed on thinking they were in control. Alarms? Maybe, but when was the last time anyone took a car alarm seriously?

We'll investigate this thing to death and learn nothing, as usual. The system is taxed to the limit and MIMBYs won't allow any margin to be built. Expect more of the same.

Reply to
Keith R. Williams

Sorry, but you are missing the point. A system that can take 5% of the world economy off line is not acceptable (regardless of the blaming or chaotic stimulus). There should not be a large grid. This was created to allow sharing. The failure domain should be smaller!

This can only be achieved by breaking the grid into smaller domains. This will limit selling of power from state to state, but insure that OH can't take the east coast down.

Solve the problem, don't play games blaming nature or management. If all the planes fell out of the sky based on one person's failure, no one would fly. Failure group size is too large, divide the system.

Rob

Reply to
Rob

--------------- Actually, a well run large grid will be more economical and also more reliable than a bunch of small ones. In the case of smaller, isolated systems, the impact would be greater, in the form of a thousand small cuts which would bleed the economy even more. Note: "well run" does not mean maximising profits. It does mean that the people at the top should understand power systems and not have "business "experience that considers utilities, grocery stores, department stores, etc as interchangable. Deregulation, has, in many places, put short term profits ahead of long term planning for reliability and adequacy of supply.

-- Don Kelly snipped-for-privacy@peeshaw.ca remove the urine to answer

Reply to
Don Kelly

Nonsense!

MOST of the time, the large grid (aside from "wheeling" excess power to where is it needed from where it is not) increases reliability. This is expecially true for customers at the "edge" of a service area.

This effect is so important that utilities at the "edge" of the major grids often establish DC links to the adjacent grid.

Well, the "east coast" (actually only a part of it) went down because the local utilities were unable to maintain operations when the grid had problems. Each utility has a obligation to its customers to maintain the ability (within its own resources) to determine if a major problem with the "grid" exists and temporarily and without damage to its backbone distribution system and power plants shed or adjust load and production as necessary.

IOW: The systems in NY and Canader should have been able to get back on line for most of its customers within minutes.

NYC, for example, was not even able to restore power to its traffic signals. It's irresponsible to blame this failure on a utility in the midwest.

Reply to
John Gilmer

You continue to use terms like "well run" and "people should understand". They aren't and they don't. Good system design takes the fool into account.

You can't claim "it was a great design except the user screwed it up" -- It was a bad design.

The root cause was not just a failed alarm, it was a system design that let a failed alarm create a massive catastrophic failure that in many cases took over a week to bring back up.

This is an engineering group is it not? Let's face the facts: BAD DESIGN!!

Rob

I agree that the power traders stole money from us after deregulation. They squeezed money from a system that we all paid for and failed to invest in it. Now they want to raise our rates to fix it.

Reply to
tom smith

Agreed.

This is not that simple. The CNY area *had* sufficient local power available to sustain itself. In fact, some portions of CNY didn't even go dark. With some of the 'islands' that resulted, we were able to bring several units on-line without having to wait for power from outside our region.

This is not true. To think that a plant that is tripped off-line can be restored in minutes shows a lack of understanding of plant operations. Many plants are not even capable of doing a 'black' startup. The cost of such a capability is high, and the need for it is *very* rare. Most plants are still recovering from the trip for some time, they can't even begin to think about coming back on line 'within minutes', they are still stabilizing from the trip.

A real issue is that plant trips occurred with a transient that did not isolate regions. Seems like the line settings should have isolated regions

*before* plants begin to trip. So a severe transient would isolate regions, not cause a wide-spread generation trip. But the 'isolation zones' have to be large enough to include a number of base-load *and* regulating units so that each region can reasonably survive such an 'isolation event'.

There is obviously a tradeoff here. A larger region is more likely to survive isolation provided it encompasses enough regulating units, spinning reserves, MVAR capacity and base load. The region has to be able to survive when some plants (including regulating) are down for maintenance. Today, the utility saves the ratepayer money by 'borrowing' the services of regulating and base load from nearby regions through interties. If not for this ability to rely on neighboring regions for such services, each region would have to overbuild equipment that would be underutilized except for the 'once every 30 years transient isolation event'.

But wait, we've just 'designed' a system remarkably similar to today's. Moral is, if you want more reliability than we have, build more, pay more. Granted, the 8/14 blackout is still 'fresh' in everyone's mind, but is it

*really* a warning of an unreliable system, signaling a future of frequent blackouts?

------ A second issue is ratepayers paying for equipment upgrades so that more power can be carried *through* their region. Should ratepayers in rural NY pay to upgrade equipment such that customers in NYC can buy cheap power from the Midwest? The large amount of power flow *through* a region is a burden on the equipment, while the local users see little benefit.

Restructuring 'carrying charges' so that the NYC consumer pays a more realistic fee for the use of the equipment used to transmit the power from the Midwest through the rural NY utility seems in order. Then the rural utility can justify equipment upgrades without burdening the local ratepayer that receives little from such an upgrade. Rural NY is facing rate increases so that NYC and mid-west can buy/sell more power from each other. Louisiana is in a similar situation between Texas and Florida. They are being asked to upgrade their east-west transmission capabilities at the expense of LA ratepayers, so Texas and Florida can trade more power.

----- Thirdly, some consider our limited transmission capability as our only way to keep Midwest coal plants from making more acid rain for NY and east. If older OH coal generation units (exempt from anti-pollution upgrades) are given more access to distant markets, they may operate full time, exacerbating the acid rain problems in NY and New England.

daestrom

Reply to
daestrom

________ If you had read my comments -I am not claiming the system was well run although parts of it were. . Good system design is foolproof but not damfoolproof, particularly where the top management falls into the latter category and good design is compromised because decisions are made by those whose only concern is the immediate bottom line and engineers are just hired help who "don't see the big picture".

-- Don Kelly snipped-for-privacy@peeshaw.ca remove the urine to answer

Reply to
Don Kelly

You have the advantage over me in that I have never participated in day to day power plant management.

I do, nontheless, understand that a "Panic Button" shut down (with complete shutdown, steam dumping, turbines coasting to a stop, etc,) might well require inspections and a SLOW re-start.

It is my understanding that the FEDs require that nuke plants have enough diesel generator capacity to do a "black" startup. (Even if it isn't a formal requirement, safety requirements providing for operation of safety related sysems (in a nuke plant, that covers a lot of territory).

Well, nothing is "free." But that's not the same a throwing money at the problem as was proposed in the weeks following the blackout.

Again, you have the advantage but it seems to me that you are talking about a variation of the "Interstate Highway Problem" whereby the interstates become the main rural routes and all the stores and new construction happens within a few miles of interstate exits.

A rural utility has the option of builing lines to its neighbors on its own or just using a line constructed (say) to bring power from Niagra Falls to NYC. Maybe things are different in NYS, but in Virginia rural folks pay a fraction of the rates that suburban or urban folks pay.

Well, I would be interesting a hearing about specific examples.

Well, don't the owners of transmission lines get paid for carrying power?

Interesting issue.

I recall reading that there is a question as to whether "improvements" to existing coal fired plants should require that the entire plant meet "modern" anti-polution standards.

Maybe the best solution is a "polution" tax. But the utilities don't like the tax and the "enviromental wackos" do like that idea of "approved" polution.

Reply to
John Gilmer

telling them something was seriously wrong, they took no action,

Short version: It's FirstEnergy's fault. Surprize!

Public utitities which do sloppy work should be seized by government. Where's Harry Truman when we need him?

Reply to
JeffM

Not true. Only power to support the residual heat removal function and control the accident is required IIRC (it's been 25 years since I designed one of these things). Basically this includes residual heat removal pumps, injection of water into the reactor, control of containment building temperature/pressure and making safe the control room and switchgear areas which support these functions. Also included are places like fuel storage, low level nuclear waste, offgas, etc.

The rest of the plant (turbine/generator and associated auxiliaries, anything beyond the reactor/containment isolation not required to control the accident) can disappear. The goal is to keep the nasty stuff inside the containment; preferably inside the fuel rods if possible.

Also, you're not allowed to connect the different safety related diesel generators (or any other duplicate safety related equipment) together so as to prevent any common failure mode. This prevents getting enough combined capacity on site to black start the typical nuclear plant.

Reply to
Fred Lotte

---------- I have just had a chance to glance at the report but, in light of your comments, Tabel 6.1 is of interest. I have not had a chance to see whether the points listed were discussed.

-- Don Kelly snipped-for-privacy@peeshaw.ca remove the urine to answer

Reply to
Don Kelly

One of the major issues an operator always faces is 100 percent hindsight in a quiet back office a week, a month or a year later.

"operators who did not think" ? The alarm system had failed! What actual issues/events/communications were occurring in the control room at the time of the alarm system failure?

From Computerworld (November 24) - as taken from the task force report

"... a software program that gives operators visual and audible indications of events occurring on their portion of the grid, began to malfunction. As a result, ' key personnel may not have been aware of the need to take preventive measures ..." ...

The statement below indicates the system board that was promised in planning the control room design was deleted to save $'s. Definitely a typical management shortcoming taking away the operators tools.

'this may have been, in part, the result of a failure to use modern dynamic mapping and data sharing systems'

The operators were always convenient targets.

Re failure to trim trees - are there any forestry personnel who are utility staff (not contractors) left at the utilities?

former Electrical Operator

Reply to
anon

They did have these systems. The systems failed (or were inadvertently disabled). I don't know how 'modern' the systems were. The report didn't go into details on the SCADA, EMS and other systems architectures, s/w versions, etc.

As to the lack of a 'system board', how would system status get displayed on that board? It might be possible that the failure of the status/alarm system may have rendered the state of this board invalid, in which case it wouldn't have helped.

One thing I did note is that there was no mention of a monitor on the status of the state estimator and alarm applications. Something like a watchdog function that would alert operators to the difference between a quiet alarm panel and a 'frozen' one.

It would also be interesting to see the design methodology behind the various supervisory systems, including an FMEA (failure modes and effects analysis). Everything fails. Its just a matter of ensuring that nothing does so in a passive manner (unnoticed).

The report stated that the operators were unaware of the current system state. I didn't see where the blame for that situation was placed upon them. Other than a suggestion that they may have placed too much trust in a single source of system information which failed.

Reply to
Paul Hovnanian P.E.

Exactly. Returning a unit to service can take quite some time. Tripping the boiler off and blowing the safety's for several minutes is pretty severe. Getting the turbine back is also a slow process. Massive machinery requires one to adhere to heat-up /cool-down limitations to avoid damage. And of course, you have to have a 'grid' to synch back on to unless you are a regulating unit.

Very WRONG!! The NRC requires taht nuke plants have emergency generators to put the plant in a safe condition and maintain it in that safe condition. That is to say, the diesels are used to provide power necessary for cooling and monitoring the reactor, and ensuring the safety of the reactor. This is a far cry from that needed to perform a startup.

The question is one of simply "Who pays for the transmission line, vs. who benefits from its construction?" If the rural community 'pays' but recieves no benefit, while the city 'benefits' but does not pay, then we have a problem. With the interstates, everyone 'paid' while not everyone benefited equally. But imagine if only the rural community paid and the urban folks did not?

When crossing territories and state lines, this *could* be the result. Users in FL getting the benefit of larger transmission across LA from TX (mandated by congress), yet only the LA ratepayers footing the bill?

Very true. The lawyers/lawmakers get to argue about just what constitutes an 'improvement'. Replace an old worn out motor/pump for the condensate system with a newer one. Substantially the same overall, but made with newer materials, lower maintenance shaft-seals, higher-quality, longer-lasting bearings. Lowers maintenance costs, replaces an obsolete component, but is it an 'improvement' that warrants upgrading the pollution equipment? Probably not, at least the owners think so.

But replace the turbine rotor with a new one. Old rotors *do* wear out, and erode. New one made from better materials, less bypass losses, more MWe output. Is *this* an improvement? Owners claim 'like component replacement', but opponents argue 'improved plant output'.

As with many things, compromise will probably rule.

daestrom

Reply to
daestrom

PolyTech Forum website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.