Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Linux / Open Source

Fight Climate Change with Open Source

Why Alliander moved its proprietary project, GXF (formerly known as Open Smart Grid Platform) to LF Energy’s open source community.
Jul 3rd, 2020 12:00pm by
Featued image for: Fight Climate Change with Open Source

The Linux Foundation sponsored this post.

Arjan Stam
Arjan Stam is Director of System Operations at Alliander, an energy network company, and a member of the LF Energy governing board.

The EU’s goal to achieve carbon neutrality — net-zero carbon dioxide emissions by balancing carbon emissions with carbon removal — is only 30 years away. While 2050 seems like a long way off, the consequences of climate change will only become more catastrophic if we don’t make massive changes.

The good news is that there’s still hope. Using renewable energy can decrease carbon emissions, but only if our power grid is flexible enough to store its sporadic supply. To do it, the entire energy industry must commit to updating our outdated power grid infrastructure — together. Alliander’s project Grid eXchange Fabric (GXF), run in conjunction with the open source Linux Foundation community LF Energy, demonstrates how the industry can use open source technology to help fight climate change and reach our carbon neutrality goals.

The Challenges: An Outdated Power Grid and Industry Deregulation 

A carbon-neutral world ensures a safe, habitable planet for generations to come and eliminates our reliance on harmful fossil fuels. For the energy sector, this means transitioning from a world powered by coal, oil and gas to one powered by green sources like wind and solar energy.

While renewable energy sources are readily available and make sense from an economic standpoint, a major issue lies in the power grid’s infrastructure. The grid currently operates on a centralized, point-to-point framework; meaning it requires a source like an electrical power plant to distribute electricity in real-time to homes and businesses. Since electricity cannot be stored economically yet, it usually must be produced as it is used.

This makes it difficult to integrate renewable energy into the grid, because renewable energy is highly variable. Without the ability to store energy efficiently — you can only harness its power when the sun is shining or the wind is blowing — it is difficult to respond to peaks and valleys in renewable energy supply. Given the grid’s outdated infrastructure, developing a green grid could take a long time, straining our goal to reach carbon neutrality by 2050.

In addition to time constraints, rapid deregulation in the energy industry (especially in the United States) has increased competition between utility companies over the years. Without industry-wide mandates by the government, the pressure is on individual utility providers to improve the grid’s infrastructure and integrate renewable energy by themselves.

Open Source in the Energy Industry

One leading effort to retool the power grid involves open source technology, following a model similar to the one we’ve seen used in the telecommunication industry.

At Alliander, we moved our proprietary project, GXF (formerly known as Open Smart Grid Platform) to LF Energy’s open source community to spur the required innovation and broader use in the industry. GXF is a scalable and technology-agnostic industrial Internet of Things (IoT) platform that allows grid operators to securely collect data and monitor, control and manage smart devices on the grid.

Before our open source work with LF Energy, constantly innovating GXF was an arduous process. We were limited to internal resources and relied on bespoke software from suppliers to update our energy transition systems. Because of the complex structures of these technologies, finding and implementing a common solution often took several years. Additionally, the security offered by proprietary suppliers and contractors usually didn’t meet the standards for the grid’s critical infrastructure. But once we moved GXF to LF Energy, we saw several benefits, including:

  • Greater insight into infrastructure performance: As our open source partners update and integrate GXF, we can more quickly assess how our systems can be used effectively and efficiently to optimize the grid.
  • More choices for solutions: With access to a variety of partner projects, our team can now research multiple solutions thoroughly and assess how they can be used for our renewable energy work on the grid.
  • Quicker processes: Partnering with a widespread community of innovators allows for more comprehensive, reliable and quick innovations on the GXF project and on our partners’ projects. This factor of speed is vital in building solutions for the grid that further help us reach our worldwide goal of complete carbon neutrality.
  • Security transparency: Through open source, we have a clear view of how security systems operate. With full transparency, we know our work on the grid is guarded from cybersecurity issues by trusted sources. Total security ensures the grid is protected not only for our company, but also for the people we ultimately deliver sustainable energy to.

These production improvements highlight the beauty of open source. Alone, you can create software that benefits your company but might take years to perfect. But together, you can quickly create a piece of software with innumerable application possibilities that benefit the entire industry.

The Importance of Accurate Data Measurement in Combating Climate Change

Alliander uses GXF today to manage public street lights in the Netherlands, while other grid operators have since used it as a head-end system to allow for maximum data flexibility between smart meters and network operators. We’ve also seen operators use GXF to manage microgrids. The data flexibility between smart meters and network operators on the grid that the application programming interface (API) of GXF provides helps utility companies accurately measure energy efficiency. Accurate measurement on the grid is vital for fighting climate change — the more efficient we are with our electricity usage, the less carbon dioxide emissions we emit.

The need for a platform like GXF was born out of interoperability challenges on the grid. In recent years, utility companies have incorporated IoT sensors onto the grid’s infrastructure to collect data and gain better insight into the grid’s performance. But with millions of different energy devices feeding into the grid from sources like resident homes and office buildings, each device requires unique systems to gather its data readings. These differences of standardization in data access protocol make it difficult to accurately measure energy efficiency on the grid.

The API Alliander developed with our open source partners for GXF, in effect, enables utility companies to gather data from multiple devices on the grid by establishing a generic communication system between the grid operator and the individual device. To ensure accuracy, the platform contains a built-in verification mechanism that validates the data measurement of each device. At Alliander, GXF helps us pull daily energy measurements from smart meters across the country that we share with customers on their electricity bill. With this visible, easily interpreted data, our customers can clearly understand the effects of their own electricity consumption and practice more efficient energy usage to reduce emissions.

On the utility provider side, GXF also helps mitigate grid loss — the energy lost in electricity transmission and distribution. With GXF, we can see where energy is being lost on the grid and redirect electrical loads to areas in need of energy to reduce losses, which is necessary for us to effectively manage the country’s public street lights.

As we continue to develop GXF with our open source partners, the end goal is to establish the platform as an industry-wide standard for data abstraction on the grid. Ultimately, a set industry standard ensures continued interoperability between devices, which will be key as we continue incorporating renewable energy source device systems into the grid from wind and solar farms. With accurate data in hand, individual energy consumers and grid operators can more efficiently consume and distribute energy to fight climate change.

Use Open Source to Fight Climate Change 

Our work with LF Energy demonstrates how any developer in the energy industry or with a technical background can join the fight against climate change.

If you’re interested, determine if working with an open source partner would positively impact your time spent building your product, and then select a partner whose open source culture aligns with your organization’s goals. If membership cost is a factor, consider the possible benefits of starting as a non-member — many organizations, including LF Energy, allow contributions or accreditation opportunities through this type of role.

The days of companies in the energy industry operating as all-out competitors are over, because we know that we can’t achieve complete carbon neutrality without working together. Through open source, we can collaborate to develop the innovative technology needed for a greener future and establish the open standards the energy sector desperately needs.

Feature image via Pixabay.

At this time, The New Stack does not allow comments directly on this website. We invite all readers who wish to discuss a story to visit us on Twitter or Facebook. We also welcome your news tips and feedback via email:

Group Created with Sketch.
TNS owner Insight Partners is an investor in: The New Stack.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.