After three harrowing days during which the Texas grid veered precipitously toward system collapse, the Electric Reliability Council of Texas (ERCOT) has begun directing transmission operators to cease rotating blackouts.
The grid recorded a gradual increase of restored generation capacity through Feb. 17 and overnight on Feb. 18. Early on Thursday, a little more than 40 GW was still offline, but about 57 GW of the grid’s available capacity was back online.
However, though reserves had risen above 3 GW as of Thursday morning, ERCOT sustained its Energy Emergency Alert (EEA) 3, a measure that cautions that power supplies cannot keep up with demand, forcing transmission companies to reduce demand on the system via controlled interruptions.
Early on Feb. 18, Oncor reported ERCOT had directed the power delivery firm and other utilities to begin restoring power previously dropped from the grid during the unprecedented winter storm. CenterPoint Energy, too, said it was making quick progress to restore power
EMERGENCY UPDATE: Yesterday, @ERCOT_ISO directed Oncor and other utilities to begin restoring power previously dropped from the electric grid. As a result of the increased generation and stable demand, Oncor has since been able to cease controlled, or rotating, outages.— Oncor (@oncor) February 18, 2021
As more generation becomes available, we are working as quickly and safely as we can to restore your power. Conservation is still needed if your electricity has been restored. Currently under 115,000 customers remain without power: https://t.co/V2CD1rxovM pic.twitter.com/2Vu8DLXrya— CenterPoint Energy (@energyinsights) February 18, 2021
ERCOT CEO Bill Magness told reporters on a call on Thursday morning that the grid operator has begun to see “steady increases and return of a lot of generation” that had been unavailable. “As that grew, we were—as quickly, as practicable, and safe on the system—reducing the number of outages that we were requesting from our transmission operators.” By the end of last night, ERCOT was able to allow transmission owners to bring back “any of the [load] that was part of this original event,” he said.
However, Magness cautioned the emergency is continuing, owing to continued frigid conditions expected on Feb. 19. “We’re still seeing much higher-than-normal winter demand,” he noted. “If we do hit a bump and have some generation have to come back off, we may have to ask for outages—but if we do, we believe they’ll be at the level where they could be rotating outages, not the larger numbers that we faced earlier this week.”
An Unprecedented Crisis
The grid began grappling with the unprecedented event late on Sunday night, when an Arctic “outbreak” swooped over much of Texas, blanketing it with snow and ice, and sending temperatures plunging to historic, frigid lows in some areas.
Anticipating a widespread critical impact from the weather, ERCOT on Feb. 14 sought and obtained a Department of Energy (DOE) Section 202(c) emergency order, allowing a lengthy list of coal and gas plants to operate at maximum output notwithstanding air quality or other permit limitations. In granting the order, the DOE directed ERCOT to submit a report by March 1 specifying how the resources listed were operating and tallying their permitting exceedances.
As ERCOT’s Feb. 14 letter to the DOE shows, the grid entity was aware that it was preparing to shed as much as 4 GW of load starting on Sunday and extending through Feb. 16, and “perhaps even later in the week.” In the letter, notably, ERCOT cited a meteorologist it hired after a previous 2011 cold snap load-shedding event, who warned: “This period will go down in Texas weather history as one of the most extreme events to ever impact the state.”
Shortly after ERCOT set an all-time winter peak demand record Sunday, Feb. 14, between 6 p.m. and 7 p.m.—reaching 69,150 MW—beginning at around 11 p.m., multiple generating units began tripping offline in rapid progression due to the severe cold weather, said ERCOT Senior Director of System Operations Dan Woodfin on Feb. 17. Some generation was already out when the grid hit the demand record owing to gas restrictions and wind turbine blade icing, he said.
“Starting at 11 p.m. on Sunday night, that’s when we started seeing a massive amount of these generators tripping offline,” Woodfin said. “There was a little over 10,000 MW that tripped offline in a very short period of time—a matter of an hour or two,” he said. “That’s really the thing that caused this to be a much more severe event than what we were expecting going into it: the outages, all those generators, one after the other, as the storm came through, with very cold temperatures, very high wind speeds.”
At 1:25 a.m. CST on Feb. 15., ERCOT made the dire call to initiate rolling blackouts. On Monday, ERCOT initially reported 30 GW was offline, but then revised that number to 34 GW. On Tuesday, as frigid conditions wore on, ERCOT suffered another debacle, losing 600 MW of imported power from the east DC-tie owing to a power emergency in the Midwest. Imports from ERCOT’s other ties—another that transfers 200 MW from the eastern U.S. and two others that transfer 400 MW from Mexico also diminished owing to heightened demand in those regions stemming from the severe weather.
By Wednesday morning at 9 a.m., forced outages surged to 46 GW—nearly 60% of installed capacity at the end of 2020. About 28 GW was thermal capacity, and 18 GW was wind and solar, ERCOT said. It estimated 185 generating units had tripped offline during the event. By 7 p.m. on Wednesday, generators provided more relief, though ERCOT still reported that 43 GW of generation—26 GW of thermal capacity and 17 GW of wind and solar—was still offline.
At 8 a.m. on Thursday, 40 GW remained offline: 23.5 GW of thermal and about 16.5 GW of wind and solar. As of 6:30 p.m., that figure had fallen to 36 GW: 21.4 GW of thermal, and the rest, wind and solar.
As Woodfin suggested to reporters on Thursday, the speed at which some generation began coming back online was based on improved conditions. “During the afternoon yesterday, we had quite a bit of solar generation online. Once the solar generation was online, we started trying to bring back a lot of the load,” he said. After asking transmission owners to restore portions of load, the grid entity recognized that it had “more generation coming online and the load that we were restoring per feeder was far less than what we expected,” he said. “And so we started telling the transmission operators, ‘restore pretty much as fast as you can,’ but we could keep up with it in terms of generation.”
Harrowing Conditions and Close Calls
As soon as it was initiated, the mounting supply gap quickly morphed into a system crisis, suggested Jeff Dagle, chief electrical engineer and head of transmission reliability subsector for Pacific Northwest National Laboratory’s (PNNL’s) Electricity Infrastructure team.
“When demand exceeds supply, the frequency of the grid goes down from the 60 Hz. If the mismatch will continue, the frequency will drop to the point where the generation that is online can’t sustain itself anymore,” Dagle told POWER on Feb. 17. “And I’ve seen some data where that frequency has gotten down as low as 59.3 Hz from time to time during the emergency this week. That’s significantly low.”
Normal variations for a 60-Hz grid range from 59.8 Hz to 60.2 Hz. “So when you get down to 59.3 Hz, that’s extreme. So some operators have definitely had their hands full trying to maintain the integrity of that grid,” Dagle said. “The only way to keep the grid together is curtailing demand and [implementing] rolling blackouts.”
Extreme Conditions Crippled Power Generators
Generators have told POWER their forced outages were multifaceted and complex. According to Texas Competitive Power Advocates (TCPA), a trade association whose members provide more than 43 GW of installed capacity to the ERCOT market—70% of total net operable capacity—the record cold presented an unprecedented crisis.
“There is not a singular cause but many inter-related and complex factors contributing to the rapidly developing situation. Texas is heavily dependent on wind and gas-fired generation for electric reliability. The gas supply and infrastructure system were severely impacted and wind generation was all but shut down by this unprecedented extreme weather,” the trade organization said on Tuesday.
Among factors TCPA cited are that the demand for natural gas to meet heating needs is critically high, while demand for natural gas-produced power is also high. But, “Natural gas production has been frozen out like many other industrial processes limiting its availability,” it said. “Higher demands and frozen wells have limited the availability and pressure of the natural gas system in turn limiting the ability for plants to produce energy.”
Several generators also suffered frozen components. Though plants winterized in preparation for winter storms in accordance with general, voluntary best practices following the 2011 cold snap, “Texas is seeing unprecedented extreme weather,” TCPA said. Adding to these calamities, hazardous road conditions further interrupted the supply chain for fuel, water, and other products necessary for power production. And finally, intermittent communication channels, owing to telecom and internet interruptions, created additional challenges with coordinated response, it said.
These photos show what our crews at generating plants are facing as they work to return our power plants back to service. We are asking our customers to voluntarily reduce their electricity usage where possible, especially between 5- 9 pm. Learn more ➡️ https://t.co/AZIXIS4BDt. pic.twitter.com/qwa0CVzvAb— Entergy Texas (@EntergyTX) February 16, 2021
Among critical units that went offline was the 1.3-GW South Texas Unit 1, 90 miles south of Houston, which tripped at around 5:37 a.m. on Feb. 15. The unit returned to service on Feb. 17, and it was ascending to 100% power late on Feb. 18. According to a plant spokesperson, the automatic trip resulted when a small diameter sensing line “apparently froze and created a false signal to [STP 1’s] feed pumps that caused the pumps to trip.” This situation “caused a loss of feedwater to the steam generator that created a condition that caused the turbine generator and reactor to trip.” The event occurred in the non-nuclear “secondary” side of the plant and the primary side remains safe and secured, the spokesperson said. The trip was STP’s first since May 2016, when it tripped due to a main generator lock.
Woodfin pointed to a number of issues that crippled generation, though he said ERCOT would seek to provide more clarity over the coming months in a report dissecting the debacle. He confirmed, however, that gas supply issues exacerbated the power shortages. On Monday, “we started having problems with having enough gas to get to some generators,” he said. “Fundamentally, it seems like the problem goes all the way back to the wellhead. We had a lot of wellheads freezing up so that the gas wasn’t able to come out of the ground and get into the pipeline and make it to generation.”
NERC, FERC Launch an Inquiry
On a bigger plane, while the event prompted fierce and justifiable outrage from electric consumers—some enduring more than continuous 30 hours without electricity and water—the events morphed into a political firestorm. Texas Gov. Greg Abbott on Tuesday said reforming ERCOT is an emergency item for state legislators in the current session, and he urged lawmakers to investigate the beleaguered grid operator that manages more than 90% of the Texas grid.
The Federal Energy Regulatory Commission (FERC) and the North American Electric Reliability Corp. (NERC)—the entity tasked with ensuring bulk power system reliability—meanwhile, launched a joint inquiry into the events in ERCOT, as well as in the Midcontinent Independent System Operator (MISO) and Southwest Power Pool (SPP) grids.
Stricken by the severe weather, MISO on Feb. 15 directed Entergy to shed load in Texas. Shortly after noon on Feb. 15, SPP also began directing its member utilities to implement controlled outages to prevent more widespread uncontrolled outages within its 14-state balancing authority area. While it restored load later that day, SPP again issued an EEA 3 alert on Tuesday morning after its system-wide capacity dropped below its current load of about 42 GW. The events marked the first time SPP was forced to shed load, it noted.
NERC Is Developing Mandatory Winterization Standards
For ERCOT, however, the events constitute the first major winter emergency since February 2011, when another unusually bitter cold snap forced 7 GW of generation offline, prompting the entity to initiate rolling blackouts. As a result of the February 2011 event, a FERC/NERC task force recommended states mandate that generators winterize their power plants. NERC also said it planned to work with the industry to develop changes to NERC Reliability Standards to properly address winterization needs.
Responding in detail to POWER’s question about what the nation’s certified electricity reliability organization (ERO) had accomplished so far, NERC said on Feb. 17 that “after significant consideration of the issue,” NERC and the power industry pursued and published a Reliability Guideline “with the intent to help industry develop their own readiness program for generating units throughout North America.” NERC has also published specific lessons learned in a document library that is dedicated to record and convey key takeaways from debilitating events.
However, NERC is now also working on “development of a mandatory standard on winterization and cold weather preparation, which is more comprehensive than a focus of ‘generating unit winterization,’ ” it said. That effort, which began in early 2020, is in direct response to the Jan. 17, 2018, cold weather event, when MISO and SPP urged voluntary reductions in power owing to abnormally cold temperatures and higher than forecast demand.
For now, NERC said it is “actively monitoring” the conditions and was “in constant contact” with the regional entities in the affected areas. “NERC, FERC and the Electricity Subsector Coordinating Council are working together as industry continues to work diligently to restore power as safely and quickly as possible,” it said.
But ERCOT, too, plans to glean lessons from the event. Woodfin conceded that while ERCOT cleared generation based on its seasonal assessment of resource adequacy (SARA), the SARA on which ERCOT bases its winter preparedness assessment uses the 2011 event as a type of “historic event” benchmark. (ERCOT has, however, also worked to align its practices to NERC/FERC’s findings from the Jan. 17, 2018, event).
As Woodfin noted on Thursday, the number of outages assumed in the SARA “are based on a 95th percentile.” He added: “I think, because of the really unprecedented nature of this weather event, the load levels we saw were beyond the kind of 2011-level of load—for one thing, by several thousand megawatts,” he said. “And on the outage side, we’re probably well-beyond the 95th percentile of outage of generating plants that [has been experienced] during a winter peak type situation.”
Magness said the event will no doubt figure into its future assessments. “What we saw with this storm—we haven’t seen anything like this—that certainly goes into our consideration of what the risks are,” he said.
Updated (Feb. 18): Adds details from ERCOT’s latest media call at 10 a.m. CST on Feb. 18.