Will AMI 2.0 Drown Utilities with Data?

The digital revolution that kicked off in the 1970s had a massive influence on the energy and utility industry as I discussed in my previous blog.

After all, this was the era of the first mass-produced CPUs, the earliest home computers and the nascent internet. 

I still remember the now-ancient-seeming electricity meters of that time, with rotating discs and dials that represented tens, hundreds or thousands of kilowatt-hours.

These early residential electricity meters were simple electromechanical devices that generated only a few hundred bytes of data per year. It took 30 years for these meters to take advantage of the cheap CPUs that came to market around the same time, largely because metrology algorithms are very computationally intensive, and our early CPUs were simply not up to the task.

Of course, there have been many improvements to the humble electricity meter since then. Radio was added to meters in the late 1970s, enabling remote reading, later, digitalization.

Advanced metering infrastructure (AMI)

Digital technology finally caught up when the first smart meters were deployed in the early 2000s. Of course, backend systems needed to be updated to cope with the influx of data, and Endava was leading on these implementations from very early on. Our technical consultants were on-site in 2009 for one of the first mass deployments at the California energy utility firm SCE and then supported many others across the globe.

The first smart meters were met with mixed expectations and results. However, from a technological point of view, the first wave was successful and provided utilities with much more granular information and the ability to operate their grid more efficiently.

If we compare just one metric, the amount of data generated by a electricity meter, the increases are mind-boggling. Current meters can generate up to 400mb of data per year. With 135 million smart meters in the US alone, that amounts to 54 petabytes of data annually.

In the eyes of consumers, not much had changed. Electricity meters (or gas and water meters, for that matter) were simple boxes sitting on the wall. In fact, most customers probably didn’t care if their meter was ‘smart’ or ‘dumb, as long as they didn’t have to pay extra for the upgrade.

Clearly, the industry as a whole needs to do a much better job of educating users about the positive benefits of the technology and find more ways to provide tangible benefits in the future.

Next generation smart metering (AMI 2.0)

The opportunity to do that is just around the corner. The industry continues to add more computing power to smart meters, and the entire technology chain that sits above them is becoming more sophisticated. The timing could not have been better.

As more and more renewables come online, energy storage is being deployed at the consumer, commercial and utility levels, both in front of and behind the meter. Long gone are the days of energy flowing in one direction from power plants, through transmission and distribution to users.

Nowadays anyone can generate electricity for their own use, store excess in home batteries or in their electric vehicle. They can even sell the excess back to the utility provider, or directly to neighbors… at least in principle.

These technological changes are incremental but staggering. According to some estimates, the second generation of smart meters can generate up to 50 million times (!) more data than previous generations.

Even if this number is highly exaggerated, it should still serve as a warning. There will clearly be technical challenges related to the sheer volume of data, requiring different approaches than those that have been successful in the past.

Technical challenges

Given the vast amount of data involved, we can group the challenges into three main categories.

First, storage. It will be impossible to transfer all the details to the backend and store them for extended periods.

Second, security. The granularity of the data will make it much more sensitive, as it can reveal much about users and their behaviors.

Third, speed. The data is generated in near real time, and it is simply impossible to perform manual processing, or write algorithms for every possible situation.

To tackle the above challenges, we must tap into some of the most advanced technologies in our toolbox.

At Endava we have been solving these challenges for the utilities industry and vendors of AMI solutions for almost 20 years, in addition to building hands-on experience in other industries that have undergone similar transitions.

It’s hard to overstate how important machine learning and artificial intelligence will be in overcoming these challenges. We must intercept data and perform intelligent actions in real time right at the edge.

Communication to the backend can be further improved using AI as well. Things like detecting anomalies or fraud and improving encryption will continually evolve using machine learning.

Since operations will be run in cloud environments, solutions will need to be cloud-vendor agnostic.

They will even need to support private clouds, where regulations demand. But I strongly believe that the days of utilities firms buying disk drives for server farms are numbered. In fact, the challenges are reversed as data centers, especially those running AI processes, are some of the largest energy consumers.

Business view

Engineers often focus too much on technical problems and solutions.

I know, I am one.

But technology only becomes useful when it helps to solve real-life problems or enables us to do things that would otherwise be impossible. In the business world, that means unlocking new business opportunities.

In future blog posts I will dive into all kinds of advanced use cases, but I believe that by far the most important business case of the next-generation grid is to provide what we have already had for the last 60 years: reliability.

 It seems counterintuitive, but the grid of the past was extremely reliable, probably the most reliable infrastructure system of all, with 99.95% uptime or better.

However, the intermittent nature of renewable power generation, the rapid introduction of energy storage, microgrids, the rapid increase in energy needs, climate change, and a volatile geopolitical environment all add to a grid that is starting to show its age.

After all, the grid was built for very simple one-way generation, transmission and usage. Keeping it running smoothly in the modern era will be no easy task.

Endava has been supporting the energy industry through this transition for nearly 20 years, and we would be happy to speak with you about how we can contribute to your innovations.