Wednesday, July 27, 2011

Significance of Interfaces in Enterprise Integration Architecture

By Saleh Najar

The trend in large enterprises nowadays seems to be in creating enterprise data integration architectures. Business leaders are becoming more aware of the huge costs associated with having duplicate, erroneous and inconsistent data scattered all over the enterprise in spread sheets and little Access databases, more referred to as the 'data swamp'.

In this short article, I would like to shed the light on the role of interfaces and what they really mean when creating the foundation of integration efforts. We hear interfaces to mean flat files, views, web services, database layer, xml files. These are all valid implementations or artifacts of interfaces, however, these are not all what an interface is or should be.

Just because a department exports their data to a delimited flat file for example so that other systems can pick up in an ETL process, this doesn't constitute an interface in the architectural sense

What happens if this flat file is in a different environment then its clients? In that case, the clients would need to know the access details and protocols to access that file. Multiply this by ten different client systems trying to integrate or ETL this data and you have a fragile system. Why fragile? for one thing, this breaks the abstraction rule in good architecture. Now all ten clients must know the internal details of the source environment in order to access it.

What happens if the access details of the source environment change? Now we have a cascading problem. All client systems would have to update their access methods to ETL data out of this source.

Another problem will be encountered if the source system decided to change the schema of data. Now suddenly, all ten client systems will break and all data warehouse reports and front ends will break too.

So from the above examples, it becomes obvious that an interface is not simply equal to its implementation format. It is more than that. At least an interface must include the following attributes to serve as a true interface that provides stability in an integration architecture:

1. The interface should be understood by teams on both sides to mean a real contract. This means that the source system must promise the target client system that the schema will not change without a proper change management process and notification. Guaranteeing backward compatibility or versioning the interface is a big plus (web services are ideal for this).

2. There must be loose coupling and dependency inversion. This means that the client systems must not depend on knowing the details of the source systems. This dependency should be inverted and both system should depend on a third system accessible by both to retrieve data from. In our example, the source system should place the delimited file on a universally accessible drop location using standard protocols, instead of requiring client systems to depend on its internal details.

3. The format of the interface must be standard and not internal to either the source or the destination systems.

The above list is not comprehensive, but using those quick core rules for understanding what an interface means will help create a stable and reusable enterprise integration architecture implementation. Above all, it will create a common understanding among team members as to what an interface means when they use that word.