How and why common sense will beat REST

In my previous post I described how REST would replace SOAP. If you paid close attention you will have noticed that I actually didn't say anything in favour of REST, but everything at the expense of SOAP.
Because it indeed seems like REST will be the new SOAP - which is in contradiction with the idea that today's Enterprises that have any form of Service Oriented Architecture will replace their current implementations by those fit for the future

Because "REST" just doesn't make any sense in that context. Mind you, I'm talking about the REST that the low-level techies hijack; exactly what I described, i.e. JSON with the four HTTP verbs. Not the REST as Roy Fielding intended, i.e. a verb-independent style. Apart from all the heavy caching on every side of any connection, which really enabled the scale he was looking for. Without cache, there would be no Internet. Period.
And in case you want to know what Roy thinks of the current hijacking of REST, just read this and this

Enterprise Integration interview by Richard Seroter

I got the chance to participate in Richard's Interview Series - I was number 40 and you might know what that number means to me
Richard is a principal architect and Microsoft MVP, and well-versed in integration on a whole, especially Microsoft / BizTalk. He's been blogging since 2007, authored / contributed to three books and written an extensive series on BizTalk

You can read the interview on Richard's site, or right here:

Q: You've been writing a series of provocative articles that take a bit of a contrarian view of REST as a viable enterprise (integration) mechanism. You seem pretty sceptical that REST/JSON is a practical service strategy for most enterprises. Given that an earlier post of yours also expresses doubt that XML/SOAP/WSDL is the answer, What types of services SHOULD enterprises be embracing and investing in so that they have a maintainable and usable ecosystem?

A: Tools and techniques aren't the answer to the Integration issue, and certainly not one single tool and technique. But first you'd have to know what the Integration issue actually is, before trying to formulate an answer to it

The Integration issue is that in IT there's an evolutionary, ever-changing diversity in platforms, operating systems, programming languages, applications - and now also devices and locations. Will there ever be a one-size-fits-all for even any of those? No.
I compare this diversity to human languages: they are extremely diverse, and then you have dialects and accents, and those also evolve, and the persons that speak them also get better or sometimes even worse at speaking them

So, we have to tackle that diversity - we can do that in two ways

1) We can make everyone speak the same language, e.g. English.
What's the ROI of that? It takes years, and the majority of people will never get fluent at any language. A huge investment in time and money, and what is the result?
Take American English, English English, Dutch English, but especially German English, French English and (my favourite) Indian English: very hard to understand.
What's the spin-off of that, the result? Well nothing really, given the bare fact that people speak the same language: you need to understand each other. Does you and your partner speaking the same language prevent arguments, misunderstandings? No.

You first need to find a common ground in the actual topics you want to discuss. You ask me a question, I give you an answer, and / or vice versa: we hold entire conversations by firing off requests and responses. I myself usually switch languages when I speak to e.g. Germans; when it gets hard, I switch back from German to English which is neither my native tongue but still a lot more often used than German.
Does that change the conversation? No - it just serves me better. For me there's no difference between speaking English or Dutch, but for a lot of people it would be a whole lot easier to speak just their native tongue

Take this back to Enterprise IT: you bought, built or made all those applications exactly because they play their role so very well. Each of them are Olympic athletes, perfectly apt to do what you want them to do, specialised in one thing only, well maybe 1.5. Now spend the time and money to teach them a different language - ouch! that will cost you dearly, and probably give you Frenglish or Indienglish at best.
[On a side-note, I am not making any statement about nationality or race here, I am just taking an example everyone can relate to. To me, all people are equal regardless of their physical attributes]
Now, let's see how this can be handled in a professional, business-efficient way: the European Parliament. With currently 23 languages in the EP, there are 506 (23 x 22) possible combinations of spoken languages. 750 members serve for 5 years, which means that on average 12.5 people per month get replaced

How much time and money would it cost to teach each of those e.g. English? Could that even be worthwhile? Of course not, and it would seriously hamper the content of messages sent and received across. So, they don't make all these people speak one and the same language, because the diversity and dynamics are so great, that it is simply not an option.
Remember that these 12.5 people per month getting replaced represents 1.5% of total: could you handle 1.5% of your IT landscape being replaced every month?

2) We can hire interpreters. People specialised in translating languages on the fly in mid-air, face-to-face, real-time. That exactly is what happens at the European Parliament.
Now, we run into another problem: you'd need at least 506 interpreters to handle all the diversity (= variations in language combinations). This is commonly known as the N2 (N to the power of 2) problem where (back to IT!) N2 possible combinations arise for N applications / languages.
The solution to that? Still using one common language, but this time it's used by the translators / interpreters to translate any language into, and from. The result? One fluid, fluent common language hanging in mid-air above all the awesome diversity of all languages spoken. The effort for the participants? Null, zilch. Nada. Niente. Niks. Nichts. Rien
[On a side note, the EP uses three middle languages: English, French and German. That's linguistically but also politically determined]

So, I believe in one common language so that the business is not bothered with the evolutionary IT diversity - after all, that diversity is not a goal, nor even a means; it's an unwanted side-effect that will never go away and has to be dealt with.
Do I think the business should be burdened with that diversity? Absolutely not.
Do I think the participants in the Enterprise conversations should be burdened with it? Most certainly not either


Back to your question, the answer to which will now be easy to understand. Did SOAP solve the Integration issue? No. XML? No. WSDL? No. Will REST? No. Will JSON? No. All those imposed, and all these will impose, the Integration issue onto the participants in the conversation, and the Business.
But let's turn that around: where do I see good application for either? In some places, mainly B2C. Not in A2A, and certainly not in B2B. If your customers or service consumers demand any of the above, or if you can profitably maintain or extend market share by translating from your common business language into those, and back again, please be my guest - you'd be a fool if you wouldn't.
But hold a knife to everyone's throat and force them to change their existing SOAP/XML/WSDL to REST/JSON? Good luck with that

Why do you think Google, Twitter and Facebook never used SOAP? It's too undefined a standard, even after more than a decade - and no one asks for it. I've witnessed its use and implementation in Enterprises, and it only resulted in long, heated debates about whose perception of it was right, ending up in yet another bilateral agreement that didn't result in any interoperability whatsoever.
Why do you think they booted or even refrained from using XML? It's too bloated of a syntax, doesn't add anything but overhead. I've witnessed the use and implementation of it in Enterprises, and it only resulted in long, heated debates about whose perception of it was right, ending up in yet another bilateral agreement that didn't result in any interoperability whatsoever. (sic)
Why do Twitter and Facebook now support JSON? Easy, it dramatically decreases overhead compared to XML. You'll notice that the implementation of JavaScript Object Notation has come to be extremely loosely coupled from Javascript (pun intended) and that it is only used as a flat-file syntax for exchanging information regardless of platform, operating system, etc etc etc. To no surprise, as it's ye good old fashioned CSV with a twist

So, what type of services should Enterprises embrace? Simply extending their existing back-office functionality outside the Enterprise is all.
In what form? Whichever form is best suited. Speak Chinese in China, Greek in Greece, and certainly not vice versa.
The location (= bandwidth) impacts the form because the services need to be exposed and thus transported from the back-end to somewhere else on this earth, and vice versa: the further away from the office and civilised world you get, the smaller the bandwidth.
Fit impacts the form, because most programming languages and platforms have a predefined taste, and even ready-built building blocks or components. The older the platforms and programming languages, the more old-fashioned that taste is and the higher the chance that building blocks are present, and fixed. The older the platforms and programming languages, the smaller the variety as well as the chance that building blocks are present: old will tell you: "Listen we only support format XYZ" whereas new will ask you "Well what do you have to choose from and we'll just pick one" - this is presuming that old is on the supply side, and new on the demand side

It all is a question of supply and demand. If you have ample of supply but little demand, you'll be inclined to adopt your consumers' format and transport protocols. If vice versa, you'll wave your existing format(s) across the consumers' faces and say "my way or the highway". It is as simple as that

Q: What are some the positive trends you see in enterprise integration? What are integrators doing now that they weren't doing 5 or 10 years ago?

A: Well, if my answer to the previous question was long, this one might be even longer - but it ain't. To be concise: we have to travel back to the previous century to answer this.
Back in the 80's Integration was confined to database point-to-point connections. All was batch, mostly focused at database replication when there weren't any tools for that, and the database market was still very diverse and far from mature / settled.
A decade later (I'm being very rough with regards to timelines here), Enterprise Integration moved up the stack and targeted applications itself, directly addressing the business logic layer. It was at that point that the canonical model was invented because diversity dramatically increased

In fact, the invention of the canonical model was the solution to the Integration issue

Yes it added overhead because messages had to be translated more than once, but with the batch schedule and low-frequency near-time Integration back then it was heaven on earth. It also enabled BIM and BAM although those two acronyms never made it out into the world because of the fact that the Integration filed got extremely disrupted by Web.
Then, 10 years plus a few years ago, B2C entered the arena, along with Web. Client-server happened along, and along with all that was the cheapification (some poetic freedom here) of servers and clients. Microsoft invaded the Enterprise and pushed aside the costly main- and midframes. Along with that, VB and Javascript put themselves on the stage

The result? Anyone who was handy could sit next to the business and script them through their solution - it was the point where we as an IT industry went from the old ways to the new ways. The old ways? 80% of code was meant to prevent the system from doing what it was not supposed to do. The new ways? 80% of code was directed at having the system do what it was supposed to do.
Anyone with even a faint memory can tell you that this resulted in unintelligible error messages and program dumps - yet that was beyond the scope of the initial key user.
The effects for Enterprise Integration? It put the profession back for a decade and more, reintroducing siloed point-to-point integrations

And here we are now. Over the last decade, we've tried ESB and SOA, focusing on XML and WSDL to make those happen, forcing all consumers to speak that one single language. And it failed, as I have been saying since last century that it would. W3C has become an authority, Oasis has, and countless others try to become yet another purely technical institution that is sponsored by vendors. It resulted in "standards" that are compromised to death: the standards support what their constituents support.
Will REST make up for that? Absolutely not, it is as undefined a "standard" as SOAP was, and will be. 5 Years from now a new tech discovery (no, not invention) will see the light or some old paradigm will get hijacked the way REST currently is, and the world will try to force it onto Enterprise Integration in exactly the same way. Will I stand at the front lines then? Yes, just like now

So, what are the positive trends I see? Well, not much really. I really like how XSLT enables vendor-independent XML-based mappings, yet every vendor has their own implementation of it, so there goes that win. The vendors have to uphold their lock-in and they do it very well, alas.
Yet I see some positive spin-off from SOAP with companies thinking about an envelope to accompany their messages - they're getting closer to the proven concept of old-fashioned snail mail for routing information exchange.
Gateways are still there, functioning as good old post offices, whether they are VANs or not. It depends on industry really, the financial world has remained almost untouched by the craze of the last decade (they can't afford experimenting) as have most if not all logistic and retail platforms. It is governments and semi-governments (e.g. insurance companies) that still hold the deep pockets of Mickey Mouse money with which they can finance early adoption of a tech solution to a business issue (with the likely outcome) - although that will be changed in the future too, given the current crisis

What are integrators doing now that they weren't doing 5 or 10 years ago? They just try to offer New Blacks as much as they can, regardless of their business value. Integration has become a predominantly tech-ruled field, and I despise that.
System integrators are still partnering with vendors and get a cut of the pie for every vendor product they sell to the customer. On the other hand, there are new kids on the block like tibbr, who handle Integration from a customer-friendly and even neutral perspective.
Apart from that, there are Social Integration tools flooding the world, all of them lightweight and inside-out focused, providing their customers with a few basic Integrations. All these will have to learn the hard way that there is no Integration but any-to-any, and who ever learns that quickest and best will lead that pack. But it will be 2-5 years.
A positive side-effect is that Integration has been put onto the agenda of the Social world - I can't complain about that nor would I want to

Q: What, if any, new challenges arise from integrating off-premises/SaaS applications with on-premises systems? Have you seen what decisions makes these scenarios successful, and unsuccessful?

A: Ah. Now that deserves a really long answer (just kidding). Off-premise poses exciting problems to real-time Integration - bandwidth is the new bottle-neck. Regarding successful or not scenarios, there is no choice really. Salesforce.com does a very nice job integrating real-time and batch, limiting each of those with regards to message size depending on what you pay for. So pay-per-Integration is the new mind boggling topic for Enterprises, and speaking of which, yes JSON in stead of XML will absolutely make a difference there - I bet some sweet money on compressing data before it gets interchanged, and back again, at least for the batch variant

The big question of on-premise versus off-premise is out of the question for Integration there, as a fun side-effect: whether you Cloud your Integration solution or keep it on-premise has become irrelevant from a single CIO decision-point, as performance latency is a given now. Having your own Integration solution and hauling in off-premise data or information versus hosting it in the Cloud (right next to your SaaS) is becoming a very interesting decision matrix, highly dependent on what you SaaS where.
The speed of light doesn't help much either, although any request-response still remains sub-second in theory. A round-trip request-reply over 20,000 km will take at least 0.3 seconds, and I predict that Cloud will follow the same pattern that physical distribution of logistics warehouses whave: some centralised, some decentralised.
I expect SSD to be a best solution for making up the increased latency as Integration is all about I/O, as it always has been. Of course it won't overcome the physical barriers of speed, and if it does, let's excavate Einstein please - he wouldn't want to miss that.

The real issue, however, will be that SaaS will just tell you "hey, here's my integration syntax and transport protocol, happy now?" and eliminate the option of customising-to-death, and lest not forget, the practice of pure ESB: forcing all applications to speak the language of the Bus, reducing the Bus to an architect's wet dream that doesn't add any value whatsoever to the Business.
Of course you will be offered a choice between one or two, maybe even three, but that's it. Cloud will greatly drive standardisation, it's even one of my blog post titles I believe

New challenges in a nut shell then, wrapping this one up? Changing the supply-demand paradigm for most Enterprises into demand-supply. I really would like to see how e.g. SAP handles that, but I'm not putting any money on it any time soon. Off-premise SaaS (that's a pleonasm but hey) will confront all Integration participants with the simple fact I described above: the Integration issue is that there's an evolutionary, ever-changing diversity in the IT components that make up or affect your landscape, and the only solution to that is adapt, not adopt

Q: [stupid question]: I don't think I use more than 20% of the features of any single software product. Microsoft Office? Maybe 15%. Sparx Enterprise Architect? 10%, at best. Microsoft Visual Studio? Probably 2%. What software do you use every day, but rarely stray beyond a core set of capabilities? What software do you think you take the MOST advantage of?

A: Not a stupid question really, it's the package paradigm: you pay for 100% and never use more than 10-20%. Then you have to put up with 100% of upgrades and pay even more for functionality you don't use in terms of time and effort.
I use Notepad for the full 100%, primarily to cut and paste between applications, even if those are Microsoft Word and Microsoft Word. I use that, and PowerPoint for fancy forms / images - my world is limited to content and fancy images really.
I use plenty of programming languages to do whatever I need to do, if that gets complicated I prefer using Ultra Edit over Visual Studio. Why? Because I don't like being confronted with change. I prefer growth over change

Thank you Richard for this interview, and keep it up!

REST definition and its place within Enterprise Integration

In a previous post I explained why REST is useless when it comes to Enterprise Integration. Even though at the very beginning I explicitly stated that

Roy Fielding wrote his dissertation entirely in the context of Web

and that

REST has absolutely no business benefits whatsoever with regards to Enterprise Integration

I got surprised to say the least by comments in various channels, most from professionals, even vendor representatives. Tech people, but nonetheless

Simple Service Enterprise - part 6

In my latest post, I recapped on the previous posts and started to take Integration from a business point of view. I'll continue to do that here, and try to mix in technical details without it getting too confusing. Wish me luck!

Here's the conversation again:

Simple Service Enterprise - part 5

In my first post on SSE I explained why and how I want, and can achieve, and have achieved, an Enterprise Integration paradigm that will give you a device-agnostic, platform-agnostic, tool-agnostic architecture that will free you from being crushed by the two tectonic plates in IT at the moment: diversity in devices, platforms and tools on the inside, and diversity in devices, platforms and tools on the outside

In my second SSE post, I explained why the approaches of the last decade and more (XML, ESB, SOA and SOAP) have failed, and in my third post I dared to state that REST is never ever going to be a solution to solve the issues either.
The reactions I got to that mainly showed me that myopic perceptions persist across techniques - yet replacing SOAP by REST and XML by JSON without questioning the business value of any of those is now being undertaken by the most zealous, and I simply won't have it. Not on my watch

Simple Service Enterprise - part 4

Today we'll take a REST from REST and I'll touch upon one of the issues I ran into today: the two types of data there are. REST assured however that at least a few of the next posts will be about yesterday's topic, as it has led to fierce debates here and there over the course of the day. Yes, pun intended

There are two types of main data: Master Data and Transactional Data. And both have very different CRUD models, requirements and needs

Simple Service Enterprise - part 3

My previous post showed the fundamentals of information interchange: exposing business functionality, currently encapsulated in the back-end, to the outside world via services. These services are a one-to-one translation to back-end functions, which are one-to-one translations to business process steps themselves: the smallest level of business transaction.
I also showed that the How of exposing these services, e.g. in which format, largely if not solely depends on a healthy amount of opportunism

This post will explain why REST is a bad idea, while taking the previous one a level deeper

Simple Service Enterprise - part 2

Yesterday's post was about Simple Service Enterprise, and showed the basics: to keep up with the growing diversity inside and outside your enterprise for getting the same functionality on different devices and platforms, you need an Integration layer (the red in the middle). Can't argue with that, point-to-point integration is a neat quick and dirty solution for very small IT landscapes and doesn't scale cost effectively

SOA attempted to do so via ESB, but there a few reasons why that failed:

Simple Service Enterprise - part 1

I plead for a Simple Service Enterprise.
One that is ruled by Business, not IT.
One that is interoperable with any other business, customer or consumer, regardless of the platforms they operate on.
Regardless of the vendors that dominate those platforms.
Regardless of the programming languages used on those platforms.
Regardless of the devices used.
Regardless of the operating systems running on those devices.
Regardless of the programming languages used on those devices

I wanna have it all, for free - or almost free. I want a fuss-free, cost-efficient, business that scales horizontally, vertically, diagonally even if that's what I need; because either my customers demand it, or it allows me to gain market share - be it regaining lost share, cannibalising existing services that will be cannibalised by others if I don't do it myself, or simply gain market share in new markets

And I need IT to follow me where ever I go, sustain me all the way, not in the future, not in the near future, not when ever their partners decide to, I want it now. NOW!

I sympathise with you - and offer the solution right here

SAP gets the Future of Integration

OK, I'll admit it: this title is heavily (heavenly?) influenced by the previous Easter weekend - yet has no relation to it whatsoever. Or has it?

Let's skip the usual introduction, here is the message from Vishal Sikka that absolutely thrilled me

@MartijnLinssen @steinermatt we do.The Gateway. It will simply be services in HANA.Also PI, NW rules engine, MDM, EP, all in or on HANA
— Vishal Sikka (@vsikka) april 12, 2012

I have never been a big fan of SAP. I presented my Enterprise Integration 101 at Sap Inside Tech NL last year, and the Borg picture of poor old Jean-Luc Picard is some representation of my feelings regarding any (ERP) monolith

Yet, after this tweet, I have been turned. Into a Borg? Maybe - I just couldn't care less at the moment

SAP, Integration and Star Trek: the future is now

I commented ranted on an SDN post yesterday. Submitting it failed, and I lost the +/- 500 words. A bit more miffed after that, I wrote the comment anew in Notepad, and copy/pasted that - it worked.
I got a few reactions, some of which inviting me to post on the topic on SDN via a blog post in stead of just a (lengthy) comment. While I appreciate the invite, I'll just do it here for now

How Klout could make Twitter a better place

I've written my fair share of posts on Klout. 1.5 years ago I started off with a mild post called "Why I have doubt about @Klout"

At the beginning of that I stated

First of all, I highly appreciate the service

- and I ended with

11 extra Klout points in 12 days on the one hand comforts me, and on the other makes me think about the stability and value of it all...

Since then, I've become increasingly weary of the product, especially because of the consistently incosnsistent quality of the product, its lack of service, and its continuous drumming on the marketing machine nonetheless - no wonder 2.5 million people instantly killed their Klout profile as soon as they finally got the chance to do so

Today, I present a way for Klout to actually create some goodwill on Twitter

tibbr 3.5 turns the world into interactive post-its

Tibbr released version 3.5 to the public today in Palo Alto California, 9 AM Pacific time. I got a solo preview yesterday and I was impressed by it - as usual I'd say.
"In twelve months since launch, tibbr has been deployed to hundreds of thousands of employees across global enterprises, who can now use tibbr to unify people, data and businesses processes to get work done"

A clear message: ...to get work done. In my opinion, tibbr dramatically reduces unnecessary human intervention in the workplace, thereby making work less unpleasant while freeing up resources for the really interesting work.
Not the greatest nor shortest sales pitch - but then I'm not selling anything here

Integration is the new Operation - this decade and next

I gave a presentation the other day that is a very short version of my Integration book. As usual, that forced me to compact thoughts and ideas, and craft a new visual - see above.
I've used that already in a post the other day, but that didn't pay proper attention to it

I'm a bit tired of all the use of the word integrated and integration over the last few weeks and months. I would like to say: "You keep using that word. I think it does not mean what you think it means"

My first anniversary

Today, it's been exactly one year since I became self-employed.
I've loved almost every second of it - while the start was hard, the middle and end were absolutely great

The market is still going up and down so I've had quite a few days without paid work, but used those to work on my business, network, future clients and assignments, and the apps and websites I'm working on

After one year, I've made a few times what I used to earn so the future looks bright. I treated our family to our most expensive vacation ever as successes have got to be celebrated, and conveniently that concluded my first year

How to queue - that is the question

The other day my attention got drawn by a very large national company that claimed to have a performance problem: sometimes it would take ages for messages to reach their destination, and entire applications would come to a screeching halt.
After a few questions and answers, it was clear that they didn't have a performance problem: they had an architectural problem or, in a nutshell, a very unfortunate design

Queueing has become a much-appreciated message-transport system in the last decades.

Selling licenses to bureaucracies is embarrasingly easy

This is a fictitious post. It's all based on nothing, if any, maybe my dreams or nightmares or who knows what. This isn't real - it's just a dream. Somehow my memory got enriched with this information, and whether it actually did or did not happen, I really can't tell.
Anyway, it's such a bizarre story that I want to share it with you. Here goes

Social silos adding to enterprise silos? Not with proper Integration

Laurie Buzcek called out for Integration as a solution for the failure of Enterprise 2.0 and Social Business - which she equates to each other - and I couldn't help but think of Tibbr when reading her post
Dion Hinchcliffe responded with a post in which he also stresses the integration of social media with enterprise tools, albeit he's careful to stress that pure technology can't be the answer - apparently we're really beyond E2.0 now

Dion claims OpenSocial 2.0 is the answer but I fail to see how that will help us further: although an impressive amount of work, it is purely technical and relying on the fact that

Developers can create applications, using standard JavaScript and HTML, that run on social websites that have implemented the OpenSocial APIs

and I don't see that happen any time soon - the only successful 2.0 Social Tools are those that are 1.0 in nature: confined

Apples and oranges: Socialcast and tibbr

The other day one of my commenters asked:

What's the difference between Socialcast and tibbr?

I had a rough idea but not much of a clue, and decided to do a quick one. I commented back but there has been some considerable dustblowing going on, so I'll repeat that here and elaborate on it

1.0, 2.0, 3.0 - Tibbr shows the way

For those who are familiar with me and my posts, you'll know that I'm passionate about Integration. My Integration eBook, my posts - even though I cover a wide range of topics - all are about diversity, evolutionary growth and change, while standing with both feet on the ground and keeping a pragmatic view

I love social - wrote an eBook about that too and ended with a balanced conclusion. I don't like 1.0, nor the 1.0 I encounter in this 2.0 world - search my blog for evangelist or Klout and you'll get my idea

I got a lot of reactions to yesterday's post on tibbr 3.0 - a good lot of people still don't get my idea, so let me explain why I think Tibco's tibbr is redefining the enterprise business model