How Do Mature Organisations Co-Exist Digital Technologies and Legacy Components (Guest blog from Fujitsu)

Author: Peter Wilson, Public Sector Industry Advisor, Fujitsu 

There are many cloud native technologies, but it is arguably the way they enable the Microservice Architecture approach to Digital Transformation that is most significant and which of course emerged from rapid innovation by companies that didn’t exist 10-15 years ago, for instance Netflix and Amazon, who have built their infrastructure and architecture around this paradigm. However, the problem for established enterprises following this approach, especially in the Public Sector, is how to integrate this with legacy estates for the foreseeable future.

In terms of the Microservice style of Application Development, whilst there is no consensus on a single definition for the term, there is some consensus on the characteristics that apply, the most frequently cited of which include: -

• Independently deployed, and capable of running unique and complete business processes.
• Expressed through business-oriented APIs.
• Able to communicate via well-defined, lightweight integration mechanisms.
• Able to scale cleanly across servers, replicating as needed.

The problem with adopting these characteristics in a mature organisation where end-to-end customer journeys are often supported by multiple (old and new) technology components, is being able to deal with errors, exceptions and edge cases in a way that doesn’t collapse the journey in unexpected ways. So, thinking about how to maintain the fidelity of the Customer Experience, in sometimes complex circumstances, is crucial to ensure contributing services both old and new can be sequenced, prioritised, and successfully completed.

This is a complicated proposition, because each microservice is a unique system with its own view of Scope, Process and State pertinent only to its context, with no awareness of other services and so, the problem of maintaining an end to end Customer Experience across processes that are vested in both these and more traditional Applications, when process tasks might be co-dependant and, conditional and where error conditions need to be handled gracefully, calls for a combination of integration approaches: -

a) Choreography: when there is no need for central coordination, each service produces and listens to other service events and decides if an action should be taken or not. It is the natural way to implement interaction between microservices, it is simple, easy to understand, relatively simple to build, and all parts remain loosely coupled. However, in more complex user journeys, it can add a cyclic dependency between services as they subscribe to each other events, to surmount the need for sequencing.

b) Orchestration: when there is complexity requiring a coordinator service for centralising the sequencing of business logic, this provides a good way for controlling the flow of the application when there is synchronous processing, for example, if Service A needs to complete successfully before Service B is invoked. However, it more closely couples the services creating dependencies, for example if service A is down, service B and C will never be called.

So, co-existing Event Driven Microservices and Legacy Architectures, while ensuring the fidelity of the end-to-end customer experience and the accurate recording of the outcome in a system of record, benefits from a hybrid of these two integration patterns, either or both of: -

1. Choreography Between and Orchestration Within

In this example, Services A, B, and C are reactive to one another, there is no central coordinator that controls the interactions between them, each determining its own behaviour based on events received from other services. However, Service C consumes an event that triggers it to orchestrate calls to additional Services D and, E (which could be in the legacy estate), in effect acting as a central controller that manages and coordinates the calls to these additional services. Service C then produces an event with the result from those two service calls, which could represent additional edge case actions.

2. Choreography Between and an Orchestration Coordinator to Drive Flow

The second hybrid pattern uses reactive between services and a central coordinator to drive the overall flow, acting in a reactive mode. It uses the concept of commands and events; commands being things that need to be done, and events being things that have been done. The coordinator produces commands to the event stream and the relevant microservice(s) consume them, performing some processing, and then publishing outcomes to the event stream. In this example, the coordinator consumes an event from service A and invokes B and C in parallel. The coordinator then consumes B and C responses and publishes an event that D and/or E consumes from the event stream, in effect reacting only to those events it cares about as necessary, this could represent a conditional error/exception route.

In summary, as Micro-Service Architecture becomes more widely adopted, to build flexible, scalable, and resilient enterprise applications it is inevitable that Event Driven Integration will play its part. However, for mature organisations surfacing a multiplicity of complex processes to Customers as part of their digital transformation, it’s important to ensure there are affordable, scalable, easy to deploy measures to deal with ambiguity, inter-dependency, and exception. This can even be thought of as giving both humans and algorithms a connected end to end digital experience.

So, while I would always advocate looking at micro-applications to respond to discrete events, building complex application processes to deal with more substantial Customer Journeys will always be necessary in big organisations. However, both approaches can live co-operatively within an event driven architecture, processing anything from single events to managing a series of tasks delivering complex outcomes.