The Way Forward for Technology and Data in Financial Markets
Digiterre considers how data management challenges are driving financial services executives to think more strategically about their data engineering approach.
In twenty years of working with financial institutions, we have noticed that the most successful firms were quick to recognise the importance of technology and data to competitive advantage in the digital age, and acted accordingly. On the buyside, investment managers with the strongest growth relative to their peers all invested early in institutional class technology and data management in the areas of trading, valuation, risk and operations. By contrast, those who didn’t invest early, and relied for too long on manual processes, typically experienced investment and operational challenges down the line. On the sell side, a striking geographic disparity has emerged with European investment banks under-investing in technology compared to their US peers. The US banks have transformed themselves into technology companies, in some cases employing more developers than Google, and continue to invest heavily in data and technology engineering and adopt workable technologies early on. For example, using the cloud earlier in the cycle compared to European banks or investing in more speculative technology such as blockchain.
In data-rich banking, investment and trading businesses, technology enables traders and analysts to stay ahead of ever-changing markets. Organisations need to successfully integrate, manage and analyse large and complex data sets rapidly and sometimes in near real-time. Data analytics is crucial to achieve their ambitions, but we regularly see the obstacles that arise, including issues with data silos, volume, value, fidelity and timeliness.
Data challenges in wholesale financial services often fall into three broad categories: market data, such as time-series and non-time-series data orchestration and analysis; operational data, including data ingestion, validation, enrichment, analysis and reporting; and network and relationship data, for example global transactional look-throughs, rebates management and pre- and post-trade analytics.
All financial markets participants recognise that tackling data management is key to operational efficiency and competitive advantage. However, the cost and complexity of data management is driving many executives to think more strategically about their approach.
Financial institutions are increasingly adopting technology approaches that are streamlined, flexible and customisable. Greater flexibility and ease of customisation facilitates high levels of user adoption, among investment managers, traders and analysts, which is often the single most important determinant of project success. Achieving this approach can be challenging, especially as the amount of data being handled, manipulated and acted on has grown exponentially in recent years. For example, we have noticed that the majority of a data scientist’s time can be consumed in data wrangling, which encompasses data selection, ingestion, validation, visualisation and schema creation. Businesses need to focus on this process so it’s streamlined and effective. This will ensure the subsequent analytics and modelling phases can occur faster. Organisations are spending vast sums on data scientists, but not getting the value and output they expect because they have failed to take into account data wrangling.
An effective data management approach must also include Agile, which is a powerful way of working in high-demand, high-risk, high-profile, and data-rich environments. By operating in small, self-sufficient teams, technologists focus on providing useful outputs as they go, in a pragmatic, iterative way, rather than waiting until the end to deliver something which may not be the right solution. For example, by using multiple entry points into an organisation, technology teams can spot and resolve day-to-day pain points while also identifying and working on longer-term strategic goals.
By applying these learnings, technologists can simultaneously plan and implement data transformation, and work pragmatically to achieve both small and large goals, in an increasingly complex, dynamic and data-rich business environment.
To find out more about best practice in technology and data in financial services please contact here.