AI-Powered Computer Vision Is Transforming Utility Inspection and Maintenance

Employing artificial intelligence (AI) in the power industry may seem like a futuristic concept, but the truth is that it’s available in several innovative applications today. Power companies that tap into the latest computer vision technology stand to benefit greatly.

Infrastructure maintenance has proved to be one of the greatest challenges—and most costly expenses—for the utility industry. As risks grow, utilities companies increasingly seek out more efficient solutions for assessing and mitigating those risks. In particular, the emerging technology of computer vision—an application of artificial intelligence (AI)—is rising in prominence. It’s become of interest because it is such an efficient and cost-effective means for managing the growing burdens of asset inspection and asset maintenance.

Interconnected Challenges Bring Increased Industry Risks

Recent events have shown that insufficient maintenance of utility infrastructure can cause catastrophic damage to human lives, communities, and the environment. This, in turn, leads to costly consequences for those utility companies that are found liable for such losses.

The prime example, of course, is Pacific Gas & Electric’s (PG&E’s) liabilities following a series of deadly California wildfires in 2017 and 2018 that killed 104 people. California government authorities found the fires were sparked by downed power lines and faulty electrical equipment. The findings concluded that these fires were fueled by improperly maintained vegetation around electrical equipment. PG&E has thus far agreed to settle multiple lawsuits for more than $14 billion in total, which prompted a Chapter 11 bankruptcy filing.

The industry is facing a rising set of risks to utility infrastructure, and these will become more common with the mounting effects of climate change especially. Planning for extreme weather events is now commonplace, whether it be California droughts that fuel catastrophic wildfires, or hurricane or deep-freeze impacts on Texas power lines. It’s no longer just a matter of “if” they’ll occur, but a question of when, and how often.

In addition, risks associated with the industry’s aging assets carry real costs. For example, California’s Camp Fire was sparked by nearly 100-year-old power lines that PG&E had acquired in 1930. The U.S. Department of Energy reported that 70% of power transformers and transmission lines are more than 25 years old. Companies are faced with tough choices: which assets to maintain, upgrade, or replace, based on their risk profiles.

In the face of these challenges, inspection and maintenance become key priorities. However, these are traditionally labor-intensive tasks, and that is not good news when the industry faces historic labor shortages—another fundamental challenge. In the U.S., the Bureau of Labor Statistics predicts that the employment of electricians and other skilled laborers will need to increase by 10% between 2020 and 2030. This comes at a time when the current workforce is rapidly approaching retirement age, while younger generations are reluctant to enter the field to replace them. Labor shortages lead to delays in new construction projects, and a lack of resources and necessary experience to conduct critical inspection and maintenance work.

As these and other interconnected risks and challenges increase, so do their associated costs. In the wake of the California wildfires and the resultant PG&E bankruptcy, other utility companies are shoring up their mitigation efforts. For example, since 2019, Xcel Energy in Colorado—another prime wildfire area—has invested nearly $500 million in mitigation costs alone. These efforts come at a time when utility companies have reported skyrocketing costs for liability insurance, along with a decline in the number of insurers and the addition of built-in wildfire exclusions. Meanwhile, on the labor front, companies are coming to grips with the fact that they must invest significantly in recruitment and training efforts. This has to happen along with offering competitive compensation packages and advancement prospects to attract and retain workers.

A Key Technological Advancement Steps In: Computer Vision

As such risks and challenges increase, the good news is that new technological advancements have also stepped in to help solve them. Recognizing the dire need for new solutions, the utility industry is increasingly investing in new technology. Executives see this as a way to assess and mitigate some of the utility’s major risks.

One prime area of innovation is ripe with applications for the utility industry: the field of computer vision. As a key part of the fast-growing AI field, computer vision involves building systems that quickly obtain, analyze, and learn from information gathered from images, videos, or multi-dimensional data. It is the technology at the core of facial recognition, for example—the ability of your phone to take a photo, “see” a pattern of pixels, recognize that pattern as one that matches your own unique face, and then unlock your phone.

How does it work? Imagine a photo taken of a power line in a forest (Figure 1). When humans look at the photo, our eyes take in shapes, colors, and countless other details, and send this information to our brains. Our brains have evolved over millennia to instantly recognize patterns, turning them into meaning without having to think about it. We look at the photo and see every tree, branch, bush, and the power line. We might even focus in on a particular tree branch to see if looks to be hanging low to the power line.

1. A view of Stanislaus National Forest from the “Rim of the World” turnout area on the side of California State Route 120. Source: Creative Commons / King of Hearts

For a computer, though, the photo begins as just a series of pixels—assorted strings of data. To “see” an image, computer vision relies on machine learning and neural networks. Machine learning (or deep learning) involves building algorithms and models that allow a computer to teach itself to recognize and see what it needs to see.

Over time, the network’s predictions—that is, the prediction that a certain shape corresponds with an object it wants to identify—become increasingly accurate. Massive amounts of data are needed for this kind of learning, however. The computer must be fed image after image of forests and power lines in order for it to eventually be able to distinguish between a power line and a tree (let alone to “see” and assess small details like a single branch’s proximity to a line).

These feats, while powerful, were not widely available in the commercial arena until recently. Roadblocks to their adoption included the computing power needed to store and analyze the vast quantity of data collected from video and images, as well as the need for engineers with the expertise to create, train, and deploy the machine learning models and neural networks involved.

However, thanks to cloud-related cost savings, the technology is now entirely cost-effective. Rather than hosting the supercomputers themselves, companies can partner with cloud-based computing services to store their data, along with taking advantage of an increasing number of pre-built deep-learning vision models and platforms available from providers.

Furthermore, computer vision’s cost-effectiveness increases profoundly when its applications can mitigate the costs of the challenges it is helping to solve. We’re seeing this now in the utility industry in a number of applications, including those described below.

Computer Vision in the Utility Industry: Drones That Can See

Computer vision can be harnessed through various means in the utility industry. It’s already being used in interesting ways, especially when combined with other rapidly advancing technologies, such as autonomous robots and unmanned aircraft systems—that is, drones.

One example is the deployment of drones to aid with nuclear power inspections. Nuclear utilities have used drones to access and inspect areas where safety may be a risk for a human inspector, or that are otherwise hard to reach. Drones can inspect construction activities, concrete structures, hard-to-reach components, storage systems, radiation mapping, and more.

Computer vision enters the picture to analyze the images collected; otherwise, a human would have to manually review every minute of footage collected. One technology company developed machine learning models to assist utilities in automatically detecting defects in their concrete containment structures. Using this tool, nuclear utilities can deploy drones to conduct inspections of the structures, upload their image datasets, and then use machine learning models to evaluate and review the results.

Another example of the technology’s use can be found in the solar power industry. For instance, Genium, a Silicon Valley–based software and engineering service company, has developed a solution called SolarScan. It combines data collection, processing, and analysis to streamline and improve the solar panel inspection process.

For example, one large European utility company expanded its renewable energy sources to include a large solar field in Africa. In order to achieve optimal performance at these solar panel assets, the field must be regularly inspected. Many kinds of obstructions (such as sand, dirt, bird droppings, and debris) can prevent optimal performance or even cause lasting damage. However, such inspections are labor- and time-intensive, all of which is complicated by their geographic distance from the parent company.

Genium’s solution involves deploying drones, such as Parrot’s Anafi model, to collect RGB (red-green-blue) and radiometric images of solar panels. When combined with information from a solar utility’s existing voltage and amperage sensors, the data collected is aggregated and processed through the cloud.

Then, using computer vision learning models that are able to learn and recognize anomalies, the system efficiently detects equipment, system, environmental, and other issues. These range from shorted circuits to soiled panels to vegetation overgrowth and more (Figure 2). Finally, the system generates detailed reports and notifies a technician—who then uses customized maintenance tools and dashboards to determine exactly where the problem lies, and how and when to prioritize it.

2. Example of a solar panel field image captured by a drone and analyzed with computer vision to detect potential anomalies (captured in red and yellow). Courtesy: Genium

The result is significant gains in efficiency and cost savings. “By identifying anomalies and issues in real-time, SolarScan allows for proactive maintenance that is targeted to areas of high risk,” said Alex Iceman, founder and CEO of Genium. “This approach helps prevent potential downtime, reduces the amount of inspection resources necessary, lowers repair costs, and extends the lifetime of solar panel assets.”

Looking Ahead: Barriers and Solutions to Widespread Industry Adoption

There are clearly many concrete benefits for using computer vision applications to assess and mitigate risks in utility infrastructure. However, some areas of the industry are moving slower than others to adopt these solutions, for a variety of reasons.

First, these approaches are not yet mandated by government regulators or insurance companies. Thus, utility companies must pursue them of their own accord. While major well-established utility companies may not be ready to scale a computer vision-based inspection solution across their entire system, smaller pilot programs help to demonstrate their effectiveness in the shorter term.

Further, there are preconceived notions of the costs, risks, and benefits to such approaches, but these barriers may no longer apply. It is important to note that technology is advancing rapidly, which means the landscape has changed significantly in a short amount of time. For example, compared to just a few years ago, the costs of drones have plummeted while their capabilities (including range, speed, communications, and more) have swiftly improved. Similarly, with the use of cost-efficient cloud-based solutions, the analytical capabilities of AI and computer vision technology are increasing exponentially. This, in turn, can save millions of dollars in human inspection costs.

Genium’s Iceman put it this way: “Today, we are standing on a precipice of unprecedented technological innovation. AI and computer vision tech is poised to take us in directions we can barely imagine. The utility industry is in the early stages of using AI and computer vision solutions and already finding it’s helping to mitigate some big risks.”

—Gordon Feller (@GordonFeller on X) is a leading advisor on energy futures and provides insights to top executives at many of the world’s leading organizations. From his Silicon Valley base, he’s worked during 40 years with large companies, associations, universities, governments, and international organizations, and he’s published more than 400 articles.