It’s a puzzle that perplexes time after time – why, with the full might and mind of experts, corporations and governments focused on the task at hand, do efforts to produce large scale IT systems, more often than not, lead to abject failure?
For UK government departments the list is a very long one, the Passport Office, DSS, Home Office, countless NHS systems, the Home Office, Ministry of Defense to name but a few. Each has poured billions of pounds and years of work into systems, only to see all that effort and money disappear into a black hole.
It’s not solely a UK problem. The US have their own ever growing list of IT disasters which dwarf those of the UK in size and cost. Nor is the problem limited to government. According to a McKinsey report more than half of all large IT projects go way over budget in the corporate sector.
So why do large IT systems fail so often?
Citing a recent Public Accounts Committee report, “Delivering successful IT-enabled business change” Computer Weekly concluded “most UK central government systems are doomed before they start because no one will be responsible for ensuring that they succeed. … It is the churn of ministers and officials that occurs between concept and implementation, and a lack of clarity over roles and responsibilities, that ensure failure.” This is certainly true, but there must be more factors involved in such repetitive and massive failure. Here are just a few ways large IT systems fail:
Budget
The budget is too small. This is invariably true because all participants in the deal have a vested interest in making it so. The customer wants their system at the lowest possible cost, and can only gauge costs by comparing quotes. System providers want to win business, so keep their quotes as low as possible. Both have a huge disincentive to be realistic about the true time and costs involved.
Aims
Not everyone’s aims are the same. IT systems often promise, in the minds of the consumer, and through the sweet patter of the salesman, to solve all woes, now, and on into the future. As such they promise different things to different people – all things to all men – and they promise the impossible.
Scale
Information Technology is complicated. Great tomes of thought have been penned in an effort to simplify what is, and will always be, a complex field. And the complexity of systems rises exponentially with their scale, so when systems are very large, their complexity extends beyond the means of any individual or group to understand them.
Politics
Something must be done, this is something, so we must do this. Big IT systems generally involve governments and large corporations. To be in power in such institutions you must be politically astute, asking few questions, answering fewer still. People who are politically naive in contrast ask and answer questions freely, just the skill you need to develop a successful system. So the people most able to develop systems successfully are the least likely to have control over production.
Momentum
It’s too late to turn back now. This happens at every level of systems development, from the individual developer crafting a new solution to a logical dilemma, to a project manager changing the plan, to a director presenting a new strategy. To find a way through a forest you must be prepared to turn around and take a different path. Knowing when to change direction is an art, not a science, and there are not many great artists about.
Planning
Early plan – late build. It’s natural to expect a great deal of planning before a major system build. There is a balance to be struck though, and the temptation is to over-plan early and build late. Successful builds run in step with plans. It’s imperative to build early, allowing plans and builds to co-develop. Changes of direction arrive earlier, and problems are solved before they’re planned (when they become much more difficult to avoid).
Change
It seemed like a good idea at the time. Mission creep, technology
Legacy
Ideally we wouldn’t start from here. We’ve always done it this way.
What to do about it?
The critical thing is to accept big IT systems will go wrong throughout development. Be honest about it and factor it in. A good rule of thumb is, work out how long you think things should take, then multiply it by three, at least, to get a more accurate assessment of how long things will actually take. Similarly a budget holder should be prepared to pay three times the amount quoted to have the job done fully and properly. Using the Rule of Three for IT will enable you to go in with your eyes open, or back out with your balls intact.