80-20: the deadly cause of IT project failure

There seems to be a rush of IT failure topics these days, all trying to find the Holy Grail of project failure. While I hold that this is a world of AND and AND, not OR and OR, I do see a major cause for project failure for the last decade: shifting from serial processing (waterfall) to parallel (Agile, Scrum, and so on)

Apart from the fact that Agile is solely focused on delivering a project rather than a product, this is not an Agile-bashing post: it is intended to show a very weak spot of contemporary IT projects

In the 80's and 90's, IT projects would take a sequential approach: first administrative design, then functional design, after that technical design, then build, then test, then go-live, and then maintenance and governance (the latter two hadn't been invented back then as IT themes, but existed nonetheless as separate fields of attention)

Then, IT became somewhat of a big business (cough) and projects grew longer, and longer, sometimes taking up up to 3 years - not unusual these days for some really big enterprises that get a huge overhaul, but that's food for another post.
Around the year 2000, a major shift occurred in IT: the web became a primal focus point. Up to that point, the end-user was someone who got the main attention in the administrative design phase, and a good part in the functional design one as well - to be completely ignored when the techies made their entry in the relay race and picked up the baton from the functional folks

In a nutshell, the whole model collapsed: in the web era, the programmer was basically sitting next to the end-user, slapping one screen onto another and quickly modelling and pre-building the end product.
Largely following the Pareto principle of 80-20: "This is what we have in mind, more or less"

Suddenly, waterfall was out the door and predictions were made in the line of "9 months, 9 weeks, 9 days" - referring to the total time it would take to come from the initial end-user business requirements to the final product. Much to the delight of the entire IT industry, those 9 days were never achieved - yet those 9 weeks have come dangerously close.
Scrum, Agile and XP saw the light - all focused at shortening the project duration and reaching go-live far earlier than before.
The trade-off for all these was requiring highly skilled designers and / or developers, and spending considerably less time on documentation - there's always a trade-off of course

Yet, IT projects somewhat adopted to the changed ways: in stead of sticking to the strictly sequential ways of conducting a project, they went parallel: work streams were introduced where focal points were addressed by specialists. You'd have a work stream directed towards the business, one towards tech, maybe one or two directed towards business or tech specialties, one for test, and that's about it

These streams worked in parallel, where they used to work in series. Of course they didn't follow the adagium "9 women can deliver a baby in 1 month" but they did infringe on the old ways: they simultaneously relied on each other

Enter the Pareto principle, the 80-20 rule: naturally you can't expect either one of these streams to be fully perfect, they're all working towards the same project goal in the same time, each in their own fashion. So, the best you can expect is that they've got their stuff together for 80%, and will deal with the remaining 20% in due time.
Which they do. So, if you're the tech stream, you'll build upon the 80% finished product of the functional stream. Of course you also can only output a maximum of 80% yourself. When the test stream approaches you, you give them whatever you have, and expect them to do an 80-20 job as well - which they do.
Math, however, is a stubborn outsider. Math teaches us, that 80% (the tech stream best effort) of 80% (the functional stream best effort) is 64% at best. When the test stream comes to the tech stream for advice on what to test, 80% of that returns a result of 51% at best

So, on a running project, parallel execution of serial activities results in poor coverage - very poor. You relying on me and me relying on them: 50% outcome at best.
This gets worse when the streams are separated by location, time, language, culture or company; resulting in resistance rather than the will to achieve a joint solution.
Streaming activities into certain work streams might give you the feeling of control, but overall you'll lose out on quality. When the right hand relies on the left hand, it rightfully assumes it has 5 fingers - not just 4

You can't hide a project's complexity by turning serial phases into parallel ones: that's only paralising the entire project. If you do, then make sure you have a daily progress meeting across all streams (which would turn it into a proper project again)