A nearly two-day-long boil water order that shuttered schools and businesses and affected as many as 2.2 million customers in Houston—the nation’s fourth-largest city—was caused by a power outage stemming from the failure of two city-owned transformers.
The incident began on Nov. 27 at 10:30 a.m. when “a ground trip and current overload” tripped the main transformer as well as a backup transformer at the 1954 East Water Purification Plant, one of Houston’s oldest water treatment facilities, said Houston Public Works.
The critical infrastructure facility is connected to the Texas grid, and the City of Houston has a 20-year contract with NRG Energy for 32 heavy-duty diesel generators at the purification plant. However, as Houston Public Works spokesperson Erin Jones told POWER, the issue was not an external power outage. “This was a power interruption by transformers not being able to connect to power, so generators would not have helped,” she noted.
A Cascading Event
A typical plant electrical system operates on its own transformers directly from the utility high-voltage line. At a water purification plant, each transformer can feed dozens of small pumps, or two or three large ones, large compressors, some mixers, two or three ozone generators, a few banks of UV lamps, dozens of electrically operated valves, various control panels and automation systems, as well as lighting and air conditioning.
When the transformers tripped at the East Water Purification Plant on Sunday at 10:30 a.m., they interrupted power at two of the plant’s three units (Plant 1 and 2). “Plant 3 lost power at 10:50 a.m. City electricians were on site at 11 a.m., identified the cause of the interruption, and worked to bring the plants back online. Plant 2 and parts of Plant 1 were back online at 12:15, and Plant 3 was back online at 12:30 p.m.,” Jones said.
At around 10:58 a.m., however, “21 pressure sensors in the transmission system reported pressures below 35 psi. By 11 a.m., 16 of these 21 sensors reported pressures below 20 psi—the emergency regulatory threshold for water systems,” Jones said. “By 11:02, 14 of the 16 sensors were reading above 20 psi, and the final two sensors showed readings above 20 psi by 11:30 a.m.” Crews were dispatched to obtain manual field readings to confirm if the sensors’ readings were accurate or could be false readings, she added.
While power had been restored to the water treatment plant’s units by 12:30 p.m., and full water pressure was restored to the distribution system by 3:30 p.m., the city reached out to the Texas Commission on Environmental Quality (TCEQ) at 2:45 p.m. to alert the agency to the power failure and sensor readings. “All water pressure monitoring sites were reporting at or above 35 psi,” Jones said. “There was preliminary discussion on the need for a boil water notice if the field measurements confirmed the sensor data.”
After five more calls with TCEQ to review the data, a call at 6:40 p.m. confirmed with TCEQ staff that, based on the available data, a boil water notice would be required. It was issued to the public at 6:44 p.m.
The boil water notice was eventually lifted at 6:40 a.m. on Nov. 29 after water quality testing submitted to the TCEQ confirmed that “tap water meets all regulatory standards and is safe to drink.”
The boil water notice prompted six school districts across Houston—including 276 schools—to announce closure on Nov. 28. Schools on Monday announced they would remain closed on Tuesday owing to “logistical challenges” caused by the notice. “Those challenges prevent the district from being able to provide meals for its students and ensure safe water is available for students and staff,” said the Houston Independent School District, Houston’s largest school district.
Houston Public Works told KHOU 11, a local television station, that the city “has a contract with a vendor that performs regular maintenance on the transformers,” which are about 24 years old. “There was nothing in their history to indicate Sunday’s malfunction,” the station reported.
Jones told POWER on Tuesday that Houston Mayor Sylvester Turner “has ordered a diagnostic review to ensure this doesn’t happen again.”
Turner, at a news conference this week, noted that Houston has only issued three boil-water orders in recent years. During Winter Storm Uri in February 2021—when a four-day freeze prompted perhaps the most sobering U.S. electric reliability crisis in recent decades—Texas’ power outage disaster transitioned into a water crisis.
More than 1,180 public water systems in 160 counties statewide reported disruptions from the 2021 winter storm, affecting 14.6 million people. Reduced water pressure—mainly owing to pump failures—was a primary cause of the problem, but power outages also prevented treatment centers from properly treating water. In September 2021, TCEQ and the Public Utilities Commission of Texas (PUCT) released guidance for water systems on compliance with legislation that responded to the Winter Storm Uri disasters. As of November 2021, water utilities must submit emergency preparedness plans to TCEQ, as well as submit their critical infrastructure information to their retail electric providers. That includes notifying all required entities about critical load infrastructure incidents as they occur.
A Spotlight on Transformer Reliability
While Houston’s boil-water incident does not constitute an immediate grid-related electric reliability concern, it puts a spotlight on equipment reliability—specifically on transformers. While the power industry has grappled with general vulnerabilities relating to large power transformers for years, concerns are emerging about the availability and reliability of distribution transformers.
Power transformers, typically used in transmission networks of higher voltages, are used for step-up and step-down applications and are generally rated above 200 MVA. Distribution transformers are used for lower-voltage distribution networks “as a means to end-user connectivity,” explains the Electrical Engineering Portal.
The North American Electric Reliability Corp. (NERC), in its latest Winter Reliability Assessment issued on Nov. 17, specifically highlighted reliability concerns stemming from an inadequate supply of distribution transformers. The nation’s designated Electric Reliability Organization (ERO) said that the electricity industry is facing a shortage of distribution transformers “as a result of production not keeping pace with demand.”
A survey by the American Public Power Association (APPA) revealed that many utilities have low levels of emergency stocks that are used for responding to natural disasters and catastrophic events. “Between 2020 and 2022, average lead times to procure distribution transformers for all voltage classes rose 429% for public power respondents—from about two to three months pre-2021 to about 12 months in 2022. Some utilities reported being quoted lead times of more than three years,” APPA said.
Supply issues are prompting many utilities to defer or cancel infrastructure projects because they are unable to procure the additional distribution transformers required for these projects. “Among public power utilities, one in five projects were deferred or canceled,” the trade group said.
NERC noted that “Severe winter storms often include high winds, icing, and precipitation that damage distribution power lines and transformers.” Utilities typically alleviate supply disruptions through asset-sharing programs, which serve to provide visibility and voluntary equipment sharing to maximize resources. “However, electricity customers may experience delayed restoration of power following storms as crews must work to obtain new equipment,” the ERO warned.
The issue has grown into such a prominent vulnerability for the electric sector, several trade groups have urged the Department of Energy (DOE) to prioritize Defense Production Act (DPA) authority from the Inflation Reduction Act (IRA) for transformers. On Nov. 30, APPA, the National Rural Electric Cooperative Association (NRECA), a national trade association representing nearly 900 local electric cooperatives, and the Edison Electric Institute, which represents investor-owned utilities, underscored the transformer shortage’s implications for reliability in comments filed in response to a DOE request for information (RFI).
While the groups urged the DOE also to prioritize large power transformers and other critical grid components ahead of the other technologies considered in the RFI, they described the distribution transformer shortages as “the most acute supply chain challenge the electric industry is facing.”
NRECA said the risk was unacceptable. “If the U.S. cannot address the shortage and growing backlog associated with transformers and other grid components, we may not be able to serve the new load that would be added to the electric distribution system by the heat pumps DOE is trying to spur on as described in this [notice of intent] and RFI,” it told POWER in a statement. “In addition, the goals for decarbonization and electrification envisioned by the Biden administration will likely be unattainable without sufficient supply of distribution transformers.”
Updated Dec. 1 to add comments from trade groups about transformer shortage implications on reliability.