Perfect Integration 8 - history of the last decades

In the first seven posts, the approaches to Integration have been shown. The architectural top-down approach, the common subset theme, and the central integration methods: messaging, transformation and transportation.
Now the approach to successful integration has been set out, a brief story about integration in the real IT world, over the last decades, is in its place

Point to point interfacing
Point to point connections, is a term that describes how applications can be connected. Point to point interfacing was commonly used to link applications together "in the early days", and was a reasonable solution as long as the scale of integration stayed small

Point-to-point interfacing involved applications having to speak each other's language; the resulting IT-landscape was a tightly interwoven one, where a lot of effort and time was spent in information exchange. Replacing an application by another one meant that all connected applications had to change their interface to it at the same time.
Point-to-point interfacing has proven to be complex, costly and unreliable for any mid-sized enterprise or up.
Point-to-point interfacing lacked centralisation and standards, making it very hard to trace and solve issues

There is a formula for point-to-point interfacing: it is n * (n - 1)
That means that for every connected / interfacing application in an enterprise, there are n * (n - 1) connections: each application (n) needs to support an interface to all other applications (n - 1)
However, the enterprise formula is not that interesting: the formula that is valid for each connected application, has far more impact: 2 * (n - 1)
That means that every connected / interfacing application has to support 2 * (n - 1) interfaces: 1 * (n - 1) to give information to all other applications connected (as an answer to their request), and 1 * (n - 1) to do so the other way around (requesting an answer from them)

Point-to-point connections thus posed a considerable problem to a company: the more widespread its usage became, the harder it was to maintain and monitor. It in fact reduced all flexibility with regards to upgrading or replacing applications to almost zero

Moreover, point-to-point connections are hard to build, as they form a direct link between 2 applications: the combined knowledge of both applications is needed to support them

Point-to-point connections are hard to support, as they're usually built as 'temporary' interfaces without much documentation, although that is just a historical note and has little to do with the (lack of) success of point-to-point interfaces themselves.
Point-to-point connections are hard to control, as they're mostly built on top of an existing application, meaning that a proprietary application has to bend and stretch itself in order to comply with another application, and vice versa: and that repeated many times.
As an answer to the problems point-to-point interfacing caused, EAI was developed

Enterprise Application Integration (EAI) is a term that describes how applications can be connected. EAI is commonly used to link applications together, and is an answer to the disadvantages of point-to-point interfacing.
EAI introduces a centralised hub, that takes over the interfacing effort from the surrounding (i.e. connected) applications. Not only is the translation performed by the hub, but centralisation also manages to bring a great span of control to an enterprise.
Within the hub, there usually still is point-to-point interfacing involved. However, EAI-tools are very well suited to do so, thus lifting the burden from applications to the right place.
With EAI, replacing an application by another one means that all connected applications don't have to do anything: only within the hub an effort would have to be made.
EAI interfacing has proven to be complex but very cost-efficient. However, it didn't solve the diversity-problem

There is a formula for EAI: it is n²
That means that for every connected / interfacing application in an enterprise, there are n² connections: each application (n) needs to have an interface translated by the hub for each application (n)
The enterprise formula for EAI is thus greater than that of point-to-point interfacing; on the other hand the formula that is valid for each connected application, has become smaller: n
That means that every connected / interfacing application has to support n interfaces: 1 to give information to all other applications connected (as an answer to their request), and (n - 1) to do so the other way around (requesting an answer from them)

EAI connections are hard to build, as they form a direct link between 2 applications: the combined knowledge of both applications is needed to support them.
EAI connections are easy to control, as they're centralised in one tool and place.
As an answer to the problems EAI interfacing didn't solve, ESB developed

An Enterprise Service Bus (ESB) is a virtual bus through which services travel within an enterprise.

• An ESB's primary goal is to offer a very high degree of standardisation
• An ESB encompasses services, which are usually, but not exclusively, Web services

Those services are wrapped in envelopes which are usually, but not exclusively, designed according to SOAP.
There's no consensus about whether an ESB does, or doesn't, transform messages. Some say it's not part of it, others say it is but that it should happen "outside the bus".
The strict version of an ESB is that the connected applications all comply with the standard message and protocol used in the bus, and the ESB itself only routes the messages - establishing a clear difference between itself and Enterprise Application Integration (EAI)

There is a formula for ESB: it is n

That means that for every connected / interfacing application in an enterprise, there are n connections: each application (n) needs to port their own interface to the bus.
On the application level, the formula is also n.
That means that every connected / interfacing application has to support n interfaces: 1 to give information to all other applications connected, and (n - 1) to do so the other way around.
Formula-wise, ESB is the best of all solutions so far

ESB connections are very hard to build, as they require the ability to support highly customized messages from just day-to-day applications. ESB connections are hard to control, as they're mostly built on top of an existing application.
ESB services involve applications having to speak the language of the bus; with this approach, IT is in fact back to the point-to-point interfacing of some decades ago, although the diversity in messages exchanged has now become slightly less. On the application level, a lot of effort and time is spent in information exchange.
Replacing an application by another one means that the new application has to support all the existing interfaces in the IT-landscape before it can be connected.
ESB is a very nice thought on the IT enterprise level, were not much has to be done: on the application level however, ESB interfacing has proven to be very complex, costly and time-consuming. Most applications, that aren't specialised in creating highly customised interfaces, produce unstable and unreliable interfaces.

From a business and ROI point of view, pure ESB's are financial disasters


In the next post, which will be slightly shorter, the differences between all these approaches, and of course the pros and cons