“GreenOps, Dark Data, and a Sustainable Approach to Cloud Migration” (Guest blog from KPMG and GoCodeGreen)

Authors: Eric Zie, CEO, GoCodeGreen and Daryl Elfield, Partner, KPMG

As sustainability and net zero targets become ever more important, organisations are looking for solutions and best practices that can help them achieve these goals. Often, these solutions can be found within their own Enterprise IT (EIT) department – and CIOs therefore can play a key part in helping achieve these ESG commitments.

While migration to cloud, data centre and data warehouse optimisation have been in practice for several years now, it is critical that these solutions are implemented with a sustainable mindset that not only helps the IT function become more efficient but also reduces its impact on the environment.

In this blog, we focus on how GreenOps practices can enable a sustainable cloud migration strategy and the role of dark data optimisation techniques in further helping organisations to achieve their sustainability targets.  

Sustainable Cloud Migration Strategy - Cloud Migration is often the “go-to” solution in many sustainable IT discussions. But the cloud does not miraculously eliminate all the environmental effects of IT, and as recently reported in the media, vendors claim of Net-Zero are not always as they seem.

“In 2019, Amazon committed to achieving ‘Net-Zero carbon’ across its businesses by 2040. The online retail behemoth’s company-wide emissions have soared by 40 percent since then, topping 70 million metric tons of carbon dioxide last year. It is a glaring example of the gulf between corporate pledges and climate progress.” MIT Technology Review 2022

Numerous industry observers assert that the major cloud service providers rely too heavily on carbon offsets and credits and do not provide enough transparency as each provider may have their own method of calculating carbon footprint. This makes it difficult for customers who are using multicloud to have a standard view across their infrastructure.

GreenOps plays an important role for a successful and sustainable cloud migration. GreenOps is a combination of DevOps, FinOps, and environmental sustainability that provides an integrated approach to achieving operational excellence with ecological responsibility. Mentioned below are some key GreenOps considerations during each phase of a cloud migration strategy:

Infrastructure Implementation

1. Choosing Renewable Energy Regions: Greenpeace have reported that while several companies list renewables energy supply as a consideration, there are relatively few companies that clearly make it an investment priority. The carbon impact and emissions directly related to a digital product can be significantly reduced if hyperscale data centres are used in suitable geographic locations that are powered by a high percentage of renewable energy. For instance, the Nordic countries are known for their reliance on clean energy. By strategically placing workloads in such regions, organizations contribute to a greener cloud ecosystem.
2. Energy-Efficient Architectures: GreenOps practices involve designing energy-efficient cloud architectures. This includes leveraging serverless technologies, event-driven solutions, and optimizing resource usage. Technology leaders must manage their data centres and network service consumption by modernising their applications and adopting sustainable software engineering and green design practices. By doing so, organizations not only enhance cost efficiency but also reduce energy consumption and carbon emissions.
3. Integration of DevOps - An efficient way of implementing a cloud migration strategy is to integrate DevOps practices in a sustainable manner. DevOps empowers organizations to adapt to customer demands and innovate rapidly. As per DevOps trends survey by Atlassian and CITE research 2020, 83% of organizations have embraced DevOps practices and only 18% have fully integrated approaches. The awareness of environmental impact of digital technology has prompted the need for DevGreenOps - a culture that aims to meet this challenge by continually measuring, controlling, and reducing the environmental impact of digital product development and release.
   
   Some good practices to consider when implementing DevGreenOps - 
   
   Continuous Monitoring: Implement tools to track energy consumption and carbon footprint.
   Resource Efficiency: Avoid resource waste by dynamically scaling infrastructure based on usage.

Managing Operations

Waste and resource management are key towards ensuring that move to cloud remains a sustainable option in the long run.

1. Heat and Water Reuse: Organisations should explore ways to reuse heat and water within their data Centers. “According to an article in Time Magazine, in 2020, Google needed more than 2.3 billion gallons of water to cool its data centres in three US states, with severe environmental and nature-based implications for local natural ecosystems.” Hence it is important to recycle the water which is used for cooling the data centers. Plus, organisation should also look at alternative to reroute the heat produced from data centers for other purposes such as heating of the other areas in their facility.
2. Resource Management: Once workloads are in the cloud, effective resource management becomes crucial. GreenOps helps identify underutilized resources to reduce the carbon footprint. FinOps encourages accountability across teams, ensuring optimal cloud resource usage by tagging resources by cost center and right-size deployments. By tracking and optimizing cloud spending, organizations achieve not only financial efficiency but also energy savings and environmental benefits.

Reporting

To measure the success of migration to cloud, accurate reporting of carbon footprint is critical. It is imperative for organisation to establish an approach of regular monitoring and reporting to mitigate any risks of greenwashing. Here are few considerations to be -

1. Understanding the country or region-specific compliance, guidelines, regulation on sustainability is the first step towards reporting emissions in the right context.
2. Utilise tools, frameworks and algorithms that not only calculate the emissions accurately but also are tailored to measure against the country or region-specific requirements.
3. Prior to migration to cloud, organisation should ensure they have created a baseline of their on prem carbon emissions and then continue to track their emission post migration on a regular basis.

In summary, GreenOps and FinOps are brothers-in-arms, forming a powerful strategy for organizations seeking to optimize their cloud usage while minimizing their environmental impact. By embracing these methodologies, businesses can move toward a more sustainable and responsible digital future.

The Role of Dark Data

As per Gartner, dark data refers to the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes (for example, analytics, business relationships and direct monetizing). Innovations have made storage devices cheap, making it easy to justify storing data in the assumption that one day it may become valuable. We are constantly collecting & storing data – Call centre logs, Server/System logs, Financial Records, Bank Statements, Video surveillance, Past Student/Employee records, Raw Data (Surveys, CDR, polls) emails, photos etc to name a few.

This data is not carbon neutral and the storage & processing of digital data is already comparable to the automotive, energy & aviation sectors and is still growing. As per WEF (World Economic Forum)  Production of digital data is increasing fast – this year the world is expected to generate 97 zettabytes (97 trillion gigabytes) of data. By 2025, it could almost double to 181 zettabytes. And by 2040, world’s carbon emissions from storing digital data is set to reach to 14%.

Recent studies indicate that up to 75% of this data is “dark” – which means optimising even a fraction of dark data can help organisation towards their sustainability targets and net zero goals a long way.

Few techniques to optimise dark data are:

1. Data Management – Organization of data into distinct categories (correct meta data labels) makes it easy to find & gives an accurate & complete view.
2. Data Sharing Strategy – A uniform data sharing strategy across departments within the organisation will ensure prevention of data redundancy.
3. Parsing of Data – Usage of proper tools AI/ML to sift through the vast data storage & retain only the relevant information & get rid of the fillers.
4. Data Governance – Setting up of processes & updating data retention policies to ensure compliance with regulatory requirements.

Conclusion

Cloud computing if implemented correctly in combination with GreenOps principles such as utilising renewable energy sources or low-carbon intensity grid locations can have positive environmental impact along with delivering economies of scale, improved workload, and efficient compute utilisation. In addition, taking a sustainable approach towards management & collection of data can lead to a marked decrease in the creation of dark data lakes, which in effect would reduce the costs in managing, storing & processing of large swathes of data.

In a perfect world, all digital products and services would be able to run in these efficient, clean computing environments, and all consumers would be able to weigh their options alongside other factors such as proximity to customers, regulatory requirements, risk, resilience, and stability, and, of course, cost. However, we do not live in a perfect world, and many larger organisations will face a multiyear journey to achieve this goal.

Hence for CIOs and other technology leaders, accurate and context-based carbon measurements, integrated with a sustainable IT strategy specific to the product and organisation, are key towards achieving sustainability goals and avoiding the risks of making claims that could otherwise be considered inaccurate, or worse, as greenwashing.