What You Need to Know Before Replacing Your Control System

Many power plants have a distributed control system nearing the end of its useful life. Here’s how to determine when to replace it—and how to plan the upgrade project. 

Power plants were among the leaders in implementing distributed control systems (DCSs), starting in the 1980s with systems based on mini-computers and continuing through the turn of the century. Although many of these older plants are sound mechanically, in many cases their control systems are now reaching the end of their useful life, necessitating migration to a new automation system.

But before such a project can begin, the business case for a migration must be made so that funding can be secured. Unfortunately, stakeholders often struggle to fund projects that replace something that’s working, albeit poorly, as is often the case with an older DCS.

Another issue germane to automation system upgrades is perception. It’s often easy to see a railcar unloading station is in desperate need of replacement by just looking at it. By contrast, an existing DCS will usually be tucked away in a climate-controlled room, apparently doing its duty, at least to an untrained eye. However, experienced automation engineers know that a new automation system will improve efficiency and productivity while reducing downtime. The challenge is to prove to others that these benefits will justify the cost and risk of a migration.

This article details the best approach for DCS migration, starting with justification and ending with commissioning and startup.

Do You Need a New Automation System?

At some point, an older DCS must be replaced with a newer automation system, but the challenge is determining exactly when the replacement should be made. A number of factors will drive this decision, and judgment must be used, as there’s no single, quantitative formula for making this determination.

The following list includes some of the factors driving the decision:

  • Discontinued technical support and spare parts service
  • Reliability issues
  • Hard-to-procure replacement parts and accompanying services
  • Sharp increases in vendor support costs
  • Difficult-to-provide internal and third-party support
  • Major plant expansion that can’t be accommodated by the existing DCS
  • DCS can’t connect with new or existing third-party applications
  • Desire to standardize automation systems among plants
  • Better plant performance needed
  • Improved cybersecurity required

The first five factors are all related to obsolescence of the DCS. Vendors tend to encourage their customers to buy new automation systems by reducing support for older DCSs, and they also face their own internal issues when trying to support designs that are decades old.

As a result, vendor support will sharply decline at some point in terms of services, parts, and patches. This will result in significant price increases to keep a DCS running and will push many issues to internal support personnel, who will face many of the same challenges. Both DCS vendors and power plants will find it harder with each passing year to find personnel familiar with older DCS hardware, operating systems, and application software.

Another factor driving DCS replacement is a plant expansion or a major upgrade to the facility’s mechanical infrastructure, often in response to a regulatory issue. It can be very expensive, if not impossible, to expand or modify the DCS, so in this case replacement can be the lowest cost option, particularly when other advantages of a new automation system are taken into account.

Nearly every power plant has auxiliary processes with their own controls, such as water treatment, fuel delivery, and ash removal systems, and interfacing these control systems with an old DCS can be very problematic. Regulatory and other issues may drive the addition of continuous emissions monitoring or other systems, which ideally would also interface with the DCS—a difficult task if the new control system is PC-based and the old DCS is driven by a mini-computer, for example. These subsystems often have critical inputs and outputs into the primary process, giving rise to not only compatibility issues but also maintainability concerns.

Power companies are continually optimizing their mix of generation sources, often through acquisitions and divestitures, and this can create a hodgepodge of automation systems, making support difficult. Replacing a DCS with a modern automation system can promote standardization, easing internal support.

In almost all cases, a new automation system will provide superior performance as compared with an older DCS. This performance can result in improvements that will create an attractive return on investment for the new automation system investment. For example, improving a plant’s heat rate by just one or two percentage points can result in fuel savings sufficient to pay for the DCS replacement in short order. An improved operator interface can enable quicker response to incidents, preventing a small problem from escalating into expensive downtime.

Finally, a new automation system will have superior cybersecurity (see sidebar and “EPRI and Luminant Collaborate to Create Common Understanding of Cybersecurity Requirements” in this issue).

Cybersecurity Concerns Encourage Upgrades

For regulatory and practical reasons, compliance with North American Electric Reliability Corp. Critical Infrastructure Protection (NERC-CIP) and other cybersecurity guidelines is now more critical than ever. In addition to complying with current NERC-CIP requirements, a new automation system must also be flexible enough to take advantage of new security measures and patches.

An older DCS will have little or no built-in cybersecurity, making compliance with current standards extremely difficult. In some cases, the only way to make an older DCS secure is to completely isolate it from other systems, which isn’t an attractive option, as it greatly interferes with optimal power plant operation.

By contrast, a modern automation system will have a host of built-in functions that ease implementation and ongoing maintenance of cybersecurity measures, including a high level of application security, access control, and auditing.

Application security ensures that critical software such as human machine interface (HMI) programs executing on the underlying operating system will have priority access to key computing resources such as processing time, memory, and bandwidth. It also prevents interference among applications by limiting communications to noncritical times.

Access control features help plant personnel enforce policies concerning who is permitted to interact with the system and what they are permitted to do. For example, an operator may have full access to the HMI parameters, including the ability to change setpoints, whereas maintenance personnel might be limited to view-only privileges.

The new automation system should have sophisticated user account management systems to facilitate access control such that each user can be assigned a password that gives only the access each individual needs to perform his or her job.

Auditing provides a centralized log of interactions with the automation system so that all actions with the automation system can be tracked. Auditing provides a way of detecting security breaches and can act as a deterrent to inappropriate actions if plant personnel are aware that their activities are being tracked. In addition, auditing can be used to track root causes of incidents, providing benefits over and above increased security.

Another cybersecurity feature to look for in a new automation system is a network security perimeter system, used to restrict access points where foreign software can enter the automation system. A related feature is built-in hardening of the automation system workstations, cutting off another access point for malicious software to find its way into the system.

No matter how well the automation system is designed in terms of cybersecurity, there will be an ongoing need for updates and patches as new threats emerge. The right automation system supplier will provide a means to test these changes before they are implemented—a necessity, as some updates and patches can cause problems with the operating system, the application software, or between the two.

The complexity of implementing and maintaining cybersecurity compliance will be greatly eased by selecting an automation system that contains built-in security features and by selecting a supplier that will work with customers to provide short- and long-term cybersecurity support.

Taking all of these factors into account and quantifying the items listed above will show if and when it’s time to replace your plant’s DCS. Once the decision is made to replace an existing DCS, the next step is implementation, which requires careful planning.

Proper Planning Predicts Success

When the decision is made to replace a DCS, the new automation system will be a modern DCS, PLC-based system, or PAC-based system. Whatever option is chosen, it’s critical to follow a project plan, which should start with a front end loading (FEL) evaluation or similar type of study.

The FEL evaluation is a multistep process and will include a comprehensive evaluation of the facility. It will yield a comprehensive plan to achieve success, a list of project tasks with budget data, durations, and an overall project schedule.

If a plant owner typically undertakes major capital projects internally, then the FEL may be performed by staff, assuming there’s sufficient automation expertise. But because major automation system upgrades aren’t performed regularly by most power companies, it’s often more cost effective to engage an outside service provider to produce the FEL evaluation. That provider should be independent of any particular vendor’s automation system to ensure that the best new automation system for a particular power plant application is recommended.

The right provider will have produced many FEL evaluations for automation system upgrade projects and will have extensive power industry experience, enabling the provider to work with internal staff to produce a thorough study. The FEL evaluation will produce a budget, show potential problems, and provide a clear route to project success.

A DCS replacement is a major undertaking requiring significant expenditures and staff training. There are risks involved, and in most cases there will be some downtime, although it can be minimized through careful planning. In some cases, the FEL will show that a DCS replacement should be postponed, maybe until the end of the plant’s useful life, in which case no replacement is required.

In other cases, where replacement is justified, the effort invested in the FEL evaluation will often be directly proportional to overall project success. Skipping the FEL evaluation or performing it in a perfunctory manner will increase risks and can lead to missed opportunities for plant improvements.

The main items that should be addressed in a DCS replacement FEL are:

  • Server-level applications such as human-machine interfaces (HMIs), historians, and the like
  • Controllers
  • I/O
  • Networks among servers, HMIs, controllers, and I/O
  • Field wiring, networks, and components
  • Interfaces to ancillary systems
  • Space requirements for new automation system components
  • Demolition, installation, and commissioning plans
  • Compliance with safety standards and regulatory requirements
  • Alarm management

For each of these items, decisions must be made including what, when, and how to replace. For example, it may be best to replace the HMIs first, often by installing the new HMIs in parallel with the old ones, to ease operator anxiety and reduce risk (Figure 1).

1. A fresh look. A new automation system provides power plant operators with a better view into the process via PC-based human-machine interfaces and should also include an improved alarm management regime. Courtesy: ControlsPR

New automation system hardware will need to be tied together with a variety of digital interfaces such as multiple variants of Ethernet and one or more permutations of fieldbus networks. Interfaces to ancillary systems will be required, some via networks and others via simple hardwired I/O. All of these networks and interfaces must be evaluated for compatibility with existing systems.

New automation systems may require different types of climate-controlled spaces, which will affect demolition, installation, and commissioning plans. Compliance with regulatory requirements will probably require enhanced cybersecurity, and a new alarm management regime will be desirable in many cases.

When a proper FEL evaluation is conducted, the result is a successful DCS replacement, as illustrated in the following examples.

Ripon Finds Reasons to Upgrade

Maverick Technologies frequently performs DCS upgrades for clients in various industries, including the power industry. A recent project was a DCS replacement for Ripon Cogen LLC, a firm that owns and operates gas-fired cogeneration plants in California (Figure 2). The main equipment that needed to be migrated to a new automation system was the heat recovery steam generator, deaerator and feedwater system, gas compressor system, compressed air system, and water treatment plant.

2. Upgrade time. The existing DCS at this cogen plant needed to be replaced to improve operations, cut downtime, and minimize maintenance. Courtesy: Maverick Technologies

In the initial phases of the project, Maverick worked with Ripon to determine its goals from a technical and business perspective, a process that allowed the client to objectively look at the technology available to best achieve its objectives.

Ripon’s goals were to take advantage of its existing HART smart instrumentation and to standardize on a new automation platform that would allow for increased productivity of its maintenance teams. The company also wanted a system that included a central and actionable alarm system to reduce required operator action by giving the operators a means to quickly react and provide a solution to any process upset. Another area of concern was compliance reporting and ease of access to plant operating and performance data.

As with most power generators undertaking a major automation upgrade, Ripon was concerned with the impact to operations due to possible downtime, and managers wanted to take advantage of existing field wiring and infrastructure. In addition, they wanted an open architecture automation system that would allow seamless interface with their critical subsystems. Finally, they wanted a system that they could support primarily with internal resources.

These were the reasons Ripon needed a new automation system. Those reasons framed the case for the DCS replacement and indicated what type of new system should be selected—in this case a Rockwell Automation PlantPAx.

As part of the FEL evaluation, Maverick worked with Ripon to develop a cutover plan that would minimize downtime and risk. Existing processor and I/O drops were replaced, but all existing field terminations were left intact, a course of action that greatly reduced required new wiring and associated cost and downtime issues (Figure 3).

3. Time management. Upfront planning can reveal many ways to minimize downtime when migrating from an old DCS to a new automation system, as with this new system, which allowed existing field terminations to remain intact. Courtesy: Maverick Technologies

The plant experienced no unscheduled outages during the migration process, and the facility started up on schedule with output ramped up to full power on the first day of operation.

As confirmation that an upgrade was the right decision, the plant has experienced fewer outages, due in large part to the improved automation system. Performance has also improved, and all of the other goals outlined above were attained.

Upgrading from the 1960s to the 21st Century

A municipal utility in Louisiana needed to replace the existing pneumatic and relay-controlled combustion, steam, and burner management systems on two of its units. Each of these units uses a natural gas–fired boiler to supply steam to a turbine generator. The capacity of each unit is in the 50-MW to 100-MW range, and the two units were commissioned decades ago.

Although the existing automation system was not a DCS, the challenges were similar in that the reasons for the upgrade had to be determined to ascertain the goals, and then a plan had to be made to perform the upgrade and meet those goals.

The city engaged Maverick Technologies for the upgrade project, and Maverick found that the initial challenges were to develop conceptual designs with basically no data from the original configuration and with minimal information on any subsequent upgrades and/or changes to the facility. The first steps were to identify the customer’s goals and develop a baseline for the project.

During that process, it was determined that the reasons for replacing the existing automation systems were to stabilize critical control variables (such as steam pressure, temperature, and drum level) in order to produce power based on varying dynamic load changes. The utility also needed to ensure it could maintain load during peak operating periods.

Plant operators needed an efficient operational management system that would enable them to quickly address system upsets, and the facility needed to generate regulatory reports from historical trend data. To achieve these goals, it was important to design a system offering seamless control while providing operators with centrally located operator consoles. As with most upgrades, it was also critical to minimize downtime during system cutover and to provide the operations staff with an automation system that they could maintain with existing internal resources.

The evaluation study showed that the best way to satisfy operator demands was with a centralized HMI console that would include a number of PC-based operator interface terminals. This was a big step for the plant, as the console replaced antiquated operator interface panel boards (Figure 4), so extensive training would be required prior to commissioning and startup.

4. Old school. When operator interface panel boards are replaced with modern PC-based HMIs, extensive training is needed to bring plant personnel up to speed. Courtesy: ControlsPR

Study and planning also revealed that maintaining unit uptime was essential, as these units provide baseload to the city. To address this issue, a redundant automation system was selected. New coordinated combustion control and burner management systems were provided for each unit, and each of these systems was provided with triple-redundant controllers and dual-redundant power supplies.

The new automation system is a GE Mark VI Integrated Control System with Cimplicity HMI, as this combination of hardware and software was found to be best for this particular application. To meet the customer’s reporting and compliance goals, an OSIsoft PI data historian was purchased, installed, and integrated with the new automation system.

The data historian allows plant personnel to quickly generate reports to ensure regulatory compliance and to identify operational performance trends. Identification and analysis of these trends enables performance improvements and also can pinpoint problem issues before they escalate. ■

Tyrone Bowman (tyrone.bowman@mavtechglobal.com) is industry manager, Pulp & Paper, Power & Energy Solutions, Field Services for Maverick Technologies.