Data sharing: the foundational success of shared intelligence
Sharing is one of the many principles that we teach our children from an early age. It is a value that is part of the foundations of human social development. Once we become adults, this value follows us. However, sometimes we do not want to share everything. So when we start reading that data sharing is becoming a business imperative for organisations, we think that sharing data is not as easy as sharing a toy. While the Covid-19 pandemic has highlighted the benefits of large-scale data sharing to accelerate innovation, data sharing projects in enterprises are still struggling to emerge.
In any data project, organisations are torn between the return on investment on one hand and the ability of a team to ensure proper and necessary governance on the other. Data sharing is no exception.
When the public sector leads the way
A few years ago, Western European countries did not have government ministries in charge of technology issues. Today, technology and data are at the heart of political agendas and concerns. This turn of events coincides with the emergence of Web 2.0 and, in particular social media platforms, where internet users have become actors — a society within the society who had to adopt legal, ethical, and moral rules.
The introduction of the General Data Protection Regulation (GDPR) has marked a new turn in the digital world. The European Union has also opened the way to a more responsible and transparent data industry. To this end, new initiatives and proposals are emerging, such as the Data Governance Act. We can see the same trend in the UK; in its 2020 National Data Strategy, the UK Government outlined plans to increase the societal and economic benefits of data, including improved data sharing.
These proposals aim to provide a framework for organizing data sharing to foster innovation and collective intelligence for the good of society. The COVID-19 applications deployed in different countries illustrate this phenomenon; citizens have access to data on the evolution of the pandemic and are encouraged to declare themselves in case of a positive test to alert people who may have been in contact. Serving the common interest is also the goal of the public sector’s open data initiatives; data is made available free of charge to foster innovation. The applications are numerous, such as in transportation or urban planning. Traffic applications such as Waze use published data on road works integrated into the application to provide users with a real-time overview of road conditions and the other way around in the UK Waze also shares their crowd-sourced data with transport authorities to help them analyse and map data.
Democratizing access to data
While organisations in the private sector understand the value of a data sharing strategy, existing barriers make it difficult to operationalise. The problems encountered are common to every data project: the technology and its deployment, trust and data governance, and organisational culture. From a technology perspective, APIs are the foundation upon which every successful data sharing strategy will be built.
While public sector organisations typically use and publish open APIs that are available without restriction as part of open data projects, private sector organisations will mostly use private (internal to the organization) or "partner" APIs with specific access rules. API development is the first barrier to implementing a data sharing strategy. Developing APIs requires resources — developers — and time, as developing an API can take between 3 and 5 days. However, we are now seeing the emergence of technologies and practices to simplify this process. API consumers like data analysts can use solutions that provide standard API libraries, such as an API to share data between SAP and Salesforce. In a matter of an hour, a data analyst can publish an API and have access to the information he needs. Access to the data is then managed automatically from the tool that provides the APIs.
For data sharing to be valuable and monetised, data must be reliable and governed. According to a Talend survey conducted in 2021, ensuring data quality was a challenge for 50% of business leaders. Organisations can't take the next step in the data lifecycle without ensuring their data health. Much like the French adage "tell me what you eat, and I'll tell you who you are," the data health of an organisation must be measurable from creation to use. Data quality must be consistently monitored, access to data must be defined, and data must be part of a governance cycle through data quality and data cataloguing technologies.
If the future UK and European regulations propose a framework that favours secure data sharing, certain industries such as the retail and distribution sector have been precursors in this field. Some players have set up centralised data sharing systems such as "data clean rooms". Data is encrypted and anonymised, then made accessible from a secure space such as a cloud or on-prem data lake with access management and governance rules applied. Fintech companies have also developed powerful data sharing models such as the French payment app Lydia which partners with Bitpanda to allows its users to invest in cryptocurrency in just a few clicks.
But in a world where the notion of data "ownership" is omnipresent, an organisation's data culture remains perhaps the most critical barrier. Today, data sharing projects are mainly driven by business departments — marketing, sales, or finance — while IT must provide a security framework and the technological levers. But business units still need to understand each other and unlock silos. Is it the role of the Chief Data Officer to lead this new data change, or should it come directly from the executive management? Will we ever see a Chief Data Sharing Officer? There are no solid answers yet to these questions, but what is certain is that data sharing can help promote data literacy within an organisation as technologies are often more agile and the business use cases much more palpable by non-specialists; a way to involve employees, federate them and train them a little more about data.
Although data sharing has become a matter of course for many organisations and projects are emerging, the ultimate success of these projects will depend on the transparency of the data shared with customers and users — and a common understanding about what that data means.
Felipe Henao Brand
Felipe Henao Brand has joined Talend in June 2021 and leads Product Marketing for EMEA, supporting innovation and business development across the region. Prior to Talend, Felipe led the product strategy for the Marketing Services and Data Quality propositions at Experian where he was responsible for the product roadmap and GTM definition, collaborating with local teams and supporting global strategic clients to drive revenue growth.