SAP gets the Future of Integration

OK, I'll admit it: this title is heavily (heavenly?) influenced by the previous Easter weekend - yet has no relation to it whatsoever. Or has it?

Let's skip the usual introduction, here is the message from Vishal Sikka that absolutely thrilled me

@MartijnLinssen @steinermatt we do.The Gateway. It will simply be services in HANA.Also PI, NW rules engine, MDM, EP, all in or on HANA
— Vishal Sikka (@vsikka) april 12, 2012

I have never been a big fan of SAP. I presented my Enterprise Integration 101 at Sap Inside Tech NL last year, and the Borg picture of poor old Jean-Luc Picard is some representation of my feelings regarding any (ERP) monolith

Yet, after this tweet, I have been turned. Into a Borg? Maybe - I just couldn't care less at the moment

For your information, I visited Wikimedia for a nice picture. This is today's: A storm of rapid star births in Galaxy Centaurus A

Apt. Very apt. Vishal's words tell me a tale of what's to come for SAP, and I won't mention the word disruptive but if SAP follows up on this, well, I might need to modify some posts.
Let me walk you through the history of Enterprise Integration, where and how that went completely wrong, my vision on the ideal world, and how SAP will enable my vision. Say what? Yes

How the IT world came to its current form

In the beginning, there was a business application. It was whole, and it ran somewhere in the bellies of an enterprise, on mainframe. It would solely rely on data entry by living human beings, aka users, and check and verify, i.e validate, the data entered before swallowing it whole and storing it onto tape or disc, becoming the Single Source of Truth (needless to say, there wasn't an issue around sources yet, let alone truth, so that phrase didn't exist back then, but let's just use SSoT from now on).

Then, information interchange entered the scene: user-less data coming in sideways, from one machine to the other. That too had to be validated before being able to enter the SSoT, so another validation layer got introduced

Then, databases replaced file systems. And front ends were introduced. And B2B arose, and information interchange between enterprise applications grew out of hand. Point to point database connections ran into limitations of uptime, validation, and general control. B2C complicated matters even more, and Architecture saw the light, dividing everything neatly into tiers:

1. Presentation
2. Business logic
3. Data

Architecture still didn't solve the integration issue: data has no users, so a separate layer mimicking the business logic layer had to be introduced in order to make sure that no invalid data entered the database

B2C created new front ends facing new users, implementing an extra presentation layer. Still, like in the old, data validation was done in the presentation layer as well, leading to extra costs that theoretically could have been avoided - but certainly not in practice. ESB's were used to some extend, but mostly as dumb transportation systems for custom-made XML, wrapped or not in a bilaterally agreed-upon SOAP header

B2B avoided all of that, as its data came from backends. As such, it put business applications to the test and put the spotlight in some cases on IT fatigue: imperfect backend systems that were perfectly intelligible to its users but had outrun the initial data model, freely abusing unstructured text fields to provide information - useless to machines, of course

A2A, application to application information interchange, ran into issues of ownership: there was no clear audience or consumer, nor an identifiable owner. Most applications were shared in at least a few ways

Then, the concept of services was discovered in IT: business services. Hyped as Services Oriented Architecture having to rely on an Enterprise Service Bus, the combination of which being widely misunderstood as having to transmit data in the form of XML via SOAP, it cost many enterprises dearly. Yet, the idea itself was a sane one

What it should have come to

The main error committed during the ESB and SOA hype was the fact that Architects viewed the IT landscape from above, high-level, and judged: all this diversity ain't good. These applications need to comply with our standards.

Needless to say, these standards architects referred to, had been invented by themselves just a few minutes earlier - in the best-case scenario, that is...

That's like inventing Esperanto and demanding that everyone in your club or group masters it. Just checking: how is your Esperanto?

Right. The simplified version is that an ESB should facilitate, not dictate. An ESB (and the same goes for SOA, by the way) should add value to your company by doing something for your applications, and not the other way around. An Enterprise needs a professional interpreter who translates all the technical diversity back to the business uniformity, serving both business as well as applications. Applications exist in your landscape because they're the best of the best you could buy for money, solely, or at least mostly, because of their functionality.
Problems arising with regards to information interchange into, and out of those same applications? IT problems, to be solved by a central interpreter/translator: not by making every application speaking the same technical language, that's an enormous waste of money

The ideal world

In the ideal and perfect Enterprise Integration situation, and Lawd knows I have architected, designed, built, tested and seen countless of those, there is one backend (layer) and it contains the full business logic that the Enterprise needs.
It consists of best-of-breed applications of all sorts of platforms, vendors, programming languages and databases, and talks to anyone else through the ESB layer. It does so by complying with the functional needs for the specific service at hand (i.e. semantics defined by the business), in a technical syntax that suits each application best

The ESB takes this syntax and translates it into a simple, uniform syntax that can carry along the full business process definitions: old-fashioned flat-file or new-ish JSON would suit fine.
That's on the message level: on the transportation level the same happens. If you have an old mainframe, MQ series would be best to carry information to and fro, yet web users will (want to) use HTTP - no problem, similar translation there by the ESB, from one transportation protocol to the other

So, you have one backend, Lawd knows how many different frontends (Desktop, laptop, mobile, web user, B2B, etc), and one ESB that decouples it all, receiving, translating and sending service requests and replies

That's my vision on Enterprise Integration, and it always has been - and it will never change. It will give you access to this

and I call it Adaptive Integrated Enterprise. Look at the colours, and you'll see how each speaks its native tongue to the ESB hub, which takes care of all the nitty-gritty tech stuff, working hard as it should be.
It will e.g. link the following tech diversities:

This is a mere example. SMTP for email is missing, so is SMS, MMS, X400, X.25, JMS, industry messaging standards such as HL7, Swift, X12, and this picture is just a snapshot anyway - in 5 years it will be invalidated for at least 1/3rd.
But, I hope it shows my point: many points of entry, all totally different, yet if relatable to the business in a profitable way, drag them in there and prepare them a warm welcome

What the naysayers (would) say

Oh my, that's quite a bottleneck in there, isn't it? Not only will it take quite a performance drain, but imagine how long everything will take, making it from A to B and back, through all those layers of translation!
Probably. While the TIBCO's of this world are used to handle millions of messages per second on a single box, people who live in the Java world live by different standards: a 30-second timeout for one single request-reply is considered tight

Hence, within SAP, among others, continuous attempts to bypass any piece of middleware and fiddle a solution together yourself. Sometimes excuses are used that (draft) tech innovations aren't yet supported, most of the time performance and "ease-of-use" is used. "Much quicker to build!", "Great debug facilities!", and "All you need for REST is a webserver and a consumer that understands HTTP!" are some of the arguments made by coders

Integration middleware is a means to a goal: the goal is sustaining the business in a cost-efficient way.
You don't do that with dozens of disparate home-made geekified tech wonders: you do that with a centralised solution that stands in the middle between your world and everyone else's, giving you full control over incoming and outgoing data- and information flow, who used what when and most, and build a BPM layer on top of that, or BI, or Big Data, or whatever you want.
You need a uniform business information flow on top of your so very diverse IT landscape, regardless of its brand, make or colour, and for that you need an ESB that speaks all syntaxes, all protocols, and can support (or simulate) all new tech "innovations"

SAP to the rescue

And then HANA enters the scene, running middleware on top of it. This would solve any and all issues! You'd simply hurl out your native request to the HANA-ised Message Broker, which translates it to your client's uniform business language, looks up which application(s) will make up the answer, translate it to the language syntax those expect and send it off, await the response(s), again translate those to the uniform business language, and into the initial native language again

That does sound complicated and a lot of work, doesn't it? Yet, it's the smartest solution to the problem, and proven technology since decades. Performance has always been the biggest objection, and the irony of it all is that the introduction of XML as an integration syntax actually did cause an issue.
Yet, the P-word can't be used anymore, can it now? Not as far as I'm concerned, nor Vishal it seems. The new bottle-neck will be front-end and back-end performance, and everything in between will simply fly

This approach will turn all of SAP into simple back-end services.

• Java or ABAP? Who cares - whatever makes you feel good, just point and shoot
• Mobile? Build a front-end in HTML5, fire off mean and lean, slim services, and anyone can develop mobile SAP solutions
• REST? Just take the unsupported HTTP extensions, their required parameters, transform them into the uniform business request actually meant, and on the way out simulate the REST service again - the bandwagoneers will love you for it

The HANA-ised Service Broker will simply wrap SAP functionality into any service you want, and with the word "performance" and "overhead" out of the way, the uniform business language (yes, the canonical) will make it back into play and enable cost-efficient BPM, EDA, CEP, BI - whatever you want

This will finally elevate Integration as a profession back to where it belongs: in the centre of the IT landscape. Application rationalisation? I've advised on Enterprise Integration before (where there are more than a few Integration products providing similar functionality) but this will take that a whole new step forward I suspect

SAP to point out as the ones who raised Integration from the deep sleep? Who would have thought...