Synchronous Integration Pattern

Tags: #integrationPatterns #synchronous #integrationArchitecture #realTime #event #soa
Published: 18.04.2021
Last Updated: 18.04.2021

Disclaimer

Naming conventions in the following text and its diagrams derive from Mulesoft, since this is the platform I currently work with in SOA model the most and can be replaced with different names as needed or as used in an organization implementing a SOA Integration Platform.
All information presented here is based on my own research and professional experience. I welcome all comments, constructive criticism and I'm open for discussion.

Intro

Synchronous communication is the most basic form of communication between IT systems/software. It is utilised by web browsers, applications, mobile apps, IoT, etc.. It's no surprise that it is the most basic pattern for an Integration Platform. It differs a bit from a simple peer to peer communication, as platforms designed in the philosophy of Service Oriented Architecture (SOA) have some principles to follow adding some extra steps on the way, that may not seem necessary from the perspective of this single communication, but are extremely important in the context of the whole platform.

Characteristics

A synchronous integration pattern in SOA must consist of at least of two technical applications. For the most simple of cases, where a client is requesting or putting data with a single system we need a means of exposing the client API and similarly a way to expose the backend systems model reducing the technical complexity of how to access it.

The most simple synchronous point to point communication is shown in the diagram below.

Let's dive into what's happening here. A client (System A) wishes to get data from a provider (System B). Now lets assume a hypothetical scenario, that System A is a warehouse system pulling data on a customer from a CRM system (System B) to process an order and send it to the right recipient.

To do this System A must invoke a service exposed and dedicated to it on the integration platform. This Experience API (EAPI) can be tailored to System A communication capabilities, so it could be a RESTful Service, a SOAP WebService or simply some crude XML/HTTP.

Experience API has a few responsibilities:

Validation of input data - it is a common good practice to validate data against a schema (e.g. openApi, WSDL, XSD, etc.). This enables the integration application to reject the data early on and return an appropriate error response before the data is passed on and other processes are invoked. This supports two key factors of successful integration:
- maintainability - error is found fast and it triggers the error handling path,
- resource saving - applications down the line are not triggered, so they don't need additional resources to handle bad data quality cases and the resources can be reassigned to other applications.
Translation the event data to CDM/BCM - it is crucial that all integration applications understand data in a context of a model the same way, so it is a common good practice to use a CDM or a BCM in a given business domain, to support a common and shared understanding which includes:
- data types,
- data restrictions,
- data enumerations,
- grouping into objects and arrays,
- cardinality of elements, objects and arrays.
Sending the translated message to the next integration component in line (Process or System API, for this simplified case this will be a System API) and awaiting response.

The EAPI invokes a System API (SAPI) to access data (in our scenario stored in a CRM system). The System API serves as a way to expose a system functionality or objects in an organized and consistent manner so it can be used to form services like the one in our scenario where we retrieve customer data to process an order. This application is always tailored tightly to the system.

System API is responsible for:

Exposing a standardized API using a CDM/BCM,
Translating the event data to the system data model,
Hiding the complexity of a system call:
- Amount of calls needed to fulfill service,
- System security,
Composing and sending a response (sometimes an empty body, just a HTTP status),

Higher complexity

The synchronous integration pattern is used commonly to compose more complex patterns as a means to expose APIs to client systems or communicate between layers. So if we'd like to step up a notch, we got the Synchronous with Enrichment pattern. It does not differ much from a regular synchronous communication, we just add a single step. An application in the Process Layer.

The Process Layer and its applications act as a logic center for integrations. It can be responsible for data transformation, orchestration, merging or business logic. The latter should be avoided if possible. Processing business logic on the integration platform would imply having some sort of a business state and the platform to be providing more than integration. I'd say, let's leave business logic to systems handling business, not integration. This is a principle rule to avoid having a stateful or business logic over the integration platform, since this could severely cripple reusability of services created this way.

Example use cases for the pattern shown in the diagram above would be:

System A is missing key data to invoke System C and get required resource data. In this case the Process API invokes a data service of System B to get additional data and then to compose a full and precise request for data to System C. The response is then sent to System A to fulfill the business needs.
System A requires specific data, a large object composed of a few smaller resources mastered by different systems. The Process API orchestrates the calls to query all needed resource data from System B and C and then composes the complex object by merging and transforming the data. The response is then sent to System A to fulfill the business needs.

Process API is responsible for:

Transforming events data,
Orchestrating data between different calls,
Aggregating data from calls,
Merging event data into service responses,

This concludes the basic description of the Synchronous Integration Pattern.

Glossary

SOA, Service Oriented Architecture - a style of Integration Platform Architecture focused on providing reusability and defined services for the IP client systems in an organised and manageable fashion. See: https://en.wikipedia.org/wiki/Service-oriented_architecture
CDM, Canonical Data Model - https://en.wikipedia.org/wiki/Canonical_model
BCM, Bounded Context Model - https://en.wikipedia.org/wiki/Domain-driven_design#Bounded_context
Cardinality - a numerical relationship between elements, objects and arrays, very important in database modeling, can be a bit simplified for data modeling in an Event-driven Integration Platform, https://en.wikipedia.org/wiki/Cardinality_(data_modeling)

Notable sources

Evans, Eric (2004). Domain-Driven Design: Tackling Complexity in the Heart of Software. Addison-Wesley. ISBN 978-032-112521-7

Credits

Beata Skrzymowska - my wife, for being patient with me and giving me space to write, :-)
Dominik Kruszewski - a long time college, specialising in DevOps in the Integration world, for proof reading all that I write, asking the right questions and supporting my technical OCDs with some nice discussions.

Comments

Dear Reader!

Please leave a comment under the LinkedIn post for this article! I would really love to hear your thoughts on the topic and get some constructive feedback! I also welcome any ideas for future articles!

Page updated

Report abuse