Catch of the day
Entergy has made a significant investment in establishing and operating its PM&DC and has an enviable track record of early identification of impending failures. Spotting the steam turbine bearing damage on Sabine Unit 3 and the broken forced draft fan vane regulating arm on Independence Unit 1 are just two examples. These and many other early alerts allow PM&DC specialists to identify problems before they became catastrophes, saving potentially hundreds of thousands of dollars with each early alert and preventive repair.
When asked about the most spectacular success of the PM&DC, Barnes was quick to discuss what has become known inside Entergy as “the catch.” The Waterford Unit 2 generator failure intercept is perhaps the most profound example of why a monitoring and diagnostic center is vital for any utility focused on reliable and safe operation of its power plant fleet. The catch undoubtedly not only saved the plant from catastrophic damage that would surely have required a months-long unplanned repair outage, but it also could well have saved lives.
The catch also demonstrates that there are possible failure modes that plant supervisory instrumentation cannot identify and alarm in sufficient time to prevent damage.
Anything but routine
Routine monitoring of Waterford Unit 2 at the PM&DC, including an evaluation of the turbine generator system using SmartSignal’s EPI*Center software, was completed on December 12, 2007. No abnormalities were found at the time (Figure 2).
2. Routine operation. PM&DC staff evaluated the Waterford Unit 2 generator operating data the afternoon of December 12, 2007, and found nothing out of spec. Courtesy: Entergy
The next morning, bearing number 8 vibration (horizontal plane, exciter bearing) levels began to rise, and the EPI*Center placed this reading on the watch list for the first time (Figure 3) at 12:59 p.m. Note that the vibration levels were still lower than the alarm limits at the plant, yet the rate of change trend away from the expected level of vibration was of concern even though all readings were still below the plant’s 5 mil alarm setting. By 1:39 p.m. a second alert and watch list incident was posted on bearing #7 at the PM&DC.
3. Vibration levels rising. On the morning of December 13, slowly rising vibration levels in the Waterford Unit 2 generator were visible. The vibration levels triggered additional scrutiny by PM&DC staff in the early afternoon. Courtesy: Entergy
The PM&DC specialist on duty contacted Waterford’s control room by phone to alert operations staff about a developing vibration issue that was “uncharacteristic” on the Unit 2 generator. Specifically, number 6, 7, and 8 bearing vibration readings were increasing, while number 5 was decreasing. The specialist recommended to plant operations staff that vibration readings be taken as soon as possible, because the unit was schedule to come off-line after the evening load peak.
At 1:56 p.m. the PM&DC specialist also contacted the Plant Support vibration specialist for consultation and followed up with the plant production superintendent on the potential seriousness of the situation. By 3:20 p.m. the vibration specialist was en route to the plant to take vibration data. The highest readings were still approximately 5 mils (the alarm setpoint) at this time.
The vibration specialist arrived at the plant and took additional readings as the PM&DC shift changed at 5 p.m. The plant shutdown began at 5:48 p.m. The PM&DC specialist again contacted the control room to advise operations staff to expect elevated vibration levels to appear on their supervisory instruments during the coast-down period, when the rotating speed passed through critical frequencies. The specialist also recommended that operations staff break condenser vacuum as soon as possible following shutdown to shorten the coast-down period and to reduce the turning gear speed to extend the cool-down period (Figure 4). Shutdown was completed without further incident.
4. Break vacuum early. The PM&DC staff recommended an accelerated shutdown of the steam turbine to minimize the time the turbine spent at critical speeds, where vibration levels peak. Courtesy: Entergy
Entergy engineers first expected the problem to be exciter-related, but an inspection found no problems there. The decision was then made to pull the generator rotor for inspection. A circumferential crack in the shaft at the “J” section was found that ran 180 degrees around the shaft (Figure 5) and at least 1.5 inches deep (Figure 6).
5. Cracked shaft. A shaft crack was found 180 degrees around the shaft when the generator rotor was removed. Courtesy: Entergy
6. Money shot. The location, length, and depth of the generator shaft crack meant a catastrophic failure was imminent. The depth of the crack was estimated as 1.5 inches. Courtesy: Entergy
The staff’s consensus opinion was that the crack was propagating while the generator was operating and was sure to have catastrophically failed if it had remained on-line to the normally scheduled shutdown later that night.
Repairs were later estimated at approximately $5 million. Costs for picking up the pieces and putting out lube oil and hydrogen fires—which would have resulted if the shaft actually had failed—were estimated at between $20 and $40 million, not including personnel injuries.
Email
Comments
Print
Reuse
