Accusations fly in advance of hearing on £126m waste on BBC Digital Media Initiative Project
At 3:15pm the Parliamentary Accounts Committee will take evidence about the BBC’s failed £126m Digital Media Initiative (DMI) project.
(Note: Just one hour later another committee will grill Iain Duncan Smith about the failing £500m Universal Credit Project).
Accusations are flying.
In the blue corner will be John Linwood, the former £287,000 a year Chief Technology Officer (CTO). He claims he was scapegoated, and he is now taking legal action over his sacking after the DMI Project was cancelled.
In the red corner, Mark Thompson, the £900,000 a year former BBC Director General from before the project started in 2007, through to when it was canned in 2012.
A dossier with incendiary claims was published last week in advance of tomorrow’s hearing. It contains two witness statements.
Firstly, a 30 page statement from whistle-blower, Bill Garrett, a former BBC IT boss, who in 2012 told BBC Trust Chair Lord Patten privately about problems with DMI. Garrett’s background places him perfectly to comment. He was Head of Technology of the BBC Vision Productions arm until 2010, and had previously been an investigative journalist specialising in “commercial misconduct, corporate finance and technology transformation”.
Garrett’s claims are dynamite. He says:
- “Staff falsified estimates of financial benefits to ensure project approval.”
- “The National Audit Office (NAO) was misled by the BBC in 2011”
- “The evidence given by the BBC at the PAC hearing of the 15th February 2011 was misleading”
Secondly, a 10 page statement from John Linwood objects to the BBC’s statement that the IT was “worth nothing” and “didn’t work”. Far from it, he asserts: the project delivered what it was asked to do, but the BBC management changed its mind on what was needed, and used him as a scapegoat.
The project started in 2007 as a grand scheme to unify the BBC’s broadcast and archival operations with a single, integrated fully digital system. The BBC says that it singularly failed to deliver any useful IT in the six years of its existence despite having up to 200 people working full-time on the IT at its peak.
Ironically, the only significant part of the DMI system delivered was a library system used to catalogue the physical tape library which, due to the failure of the digital project, is still in constant use.
Thompson will have to explain why the BBC did not appoint a Senior Responsible Owner (SRO) to act as a single point of accountability. Appointing an SRO is an integral principle of the Cabinet Office’s standard Programme Management approach.
And why was Linwood sacked, if accountability was so dispersed?
Major reports have been published by both the NAO and also Price Waterhouse Coopers (PwC) which typically (from auditors) blame lack of detailed planning and insufficient oversight.
But in my view these official reports miss the point. Over bloated, overpromised and overlong IT projects are almost always going to fail.
I will explain why, and what can be done about it in the future.
The three crucial faults that wasted £126m at the BBC
Firstly, a big promise was made to the BBC’s Board that the cost would be contained at £82m. This was wildly over-optimistic. The ‘fixed-price’ contract to deliver the system unravelled. In 2011 after severe technical difficulties, the BBC wrote off £38m.
Although the BBC estimated that the DMI would cost £134m but only create benefits of £98m, incredibly the death march to deliver the project continued. The Executive Board delayed telling Lord Patten, the Chair of the BBC Trust, the bad news for over three months, and then promised that although there would be a delay, it would be worth the wait, as additional, unspecified benefits were expected, if the Trust would only be patient. BBC Trust allowed the project to proceed, despite the risk status having been increased to amber‑red.
Secondly, the Board assumed that the use of a prime contractor, Siemens, would unload all of the risk of mismanagement of the project from the BBC. In the event it just made monitoring real progress of Siemens’s work impossibility. The Board’s confidence that the project was progressing well was based on trust, rather than solid evidence. The project was actually plagued by technical difficulties, and in the end Siemens had to admit failure and paid a penalty of £27m, leaving the BBC with a loss of £11m, and a slip of three years behind other major broadcasters.
Finally, the expected savings were forecast to be £18m per year – but only if management were willing to wait until the whole integrated system was ready to use. The BBC took the project in house and re-planned the timescales. No usable system would be ready for years until all was ready. The system would ‘go-live’ in a ‘big-bang’. Photographs, audio and video for all divisions would be digitised all at the same time. That moment, of course, never came.
‘Emperor’s New Clothes’ Syndrome
It took two incidents for the DMI Project’s bubble to burst.
In May 2012, a whistle-blower, now revealed as Bill Garrett, contacted Lord Patten. He believed that “a number of staff knowingly falsified estimates of financial benefits” to get the project started and to keep it going, even as the assumptions it was based on were falling apart.
Garrett alleged to Lord Patten that progress reporting had been economical with the truth and that benefits had been.
The second incident was in April 2013. The truth became inescapable. The Guardian made public the problems encountered by editing staff in preparing archive footage of Margaret Thatcher for obituary items and her subsequent funeral. Production staff had to use motorcycle couriers to deliver tapes because the digital editing suites were not suitable. In his statement to PAC John Linwood maintains that such archives were never part of the remit of the DMI Project. Perhaps they should have been?
Cracks in the plans became evident in 2011, when the BBC relocated some departments to Salford. The DMI system was still not ready to use, and the Sport Division had to hurriedly buy a standard digital storage package. The fact that this cost just £0.8m and was adequate for their needs should have sounded warning bells about the overblown budget for the DMI Project.
The DMI Project had left the BBC as naked, technologically speaking, as the Emperor’s new clothes had done in the cautionary children’s tale. And it was the news media that revealed the problem, not the people closest to the project.
Typical Symptoms of a Runaway Project
John Linwood, the sacked CTO, was a first-hand witness to the car crash that DMI became. He says that even though part of the system was ready in 2012, the business refused to implement it. In the four years since the start of the project, priorities had changed, he was told. Technology had changed, and the business realised that it actually had “different requirements in different departments”. The last thing it needed was a “single standardised production process across the BBC”.
DMI production tools were planned for five areas
In the end they were only used in one BBC series
appropriately called ‘Bang goes the theory’.
When the BBC took the project back in-house from Siemens, it said that it would deliver the IT in phases. But excuses were made, and deliveries were cancelled rather than holding the project team to account to deliver.
Is it really surprising that requirements change over a period of 4 years?
And what sane person would continue an IT project that had not delivered any outputs for 4 years?
The sensible approach is obvious, but not an appetising one perhaps for management bent on implementing a splashy ‘big-bang’ change to the business.
Do it bit-by-bit – the Agile Approach
Why agree a business plan that is predicated on waiting for ages for a monolithic ‘one-size fits all’ perfect solution to an uncertain problem?
The Agile approach is to plan an incremental implementation focussed on early ‘quick wins’ that can test the various ideas, and then move step-by-step towards a solution. An overarching Programme steering board ensures coherency. Sub-projects pilot new technology, and when the IT is shown to work, then a roll-out takes place, nuanced by the lessons learned by the piloting.
Starting in 2007 a series of mini-projects should have been initiated – each one no more than 6 months long. Each sub-project just one step towards an evolving solution.
Maybe first a pilot media ingest system just for audio for non-copyright talk radio assets perhaps. Then maybe just photographic archival. Then maybe more indexation of assets with complex copyright restrictions. Then, maybe, having learned from that work video assets could be tackled – perhaps just for off-line production work without the pressures of tight deadlines that the news divisions have.
Although the recent PwC report says that this was the BBC’s plan when they took the DMI Project back in house, a second delivery never came, let alone the five more that were promised.
Bit by bit, the whole solution could have built up in an Agile way. In small, effective chunks that were easy to build, and easy to put to use. Changing user perceptions as to the ‘real’ requirements would have informed each step of the process. Existing technologies are superseded with frightening pace. 6 month long sub-projects would have matched that pace with quick and effective harnessing of new technologies as they become available.
Is the Advice in the Official Reports ‘Agile’?
Unfortunately both of the recent PwC and NAO reports reflect the corporate love of expensive, long, big-bang projects. Their advice is that such projects are essentially sound, but just need tighter management to succeed. Their recommendations are that requirements needed to be better documents. More independent desk-checks of the theoretical designs would have avoided problems, they say.
However, in their own report the NAO admit that there were several internal BBC reports that warned both the Executive Board and the Trust that things were going awry.
The DMI Project’s own ‘Programme Management Office’ had warned management of ‘Amber/Red’ risk status. An Internal Audit report on raised major issues on misreporting of progress, and Accenture consultants had also warned that the already ageing meta-data archive technology chosen three years earlier as the basis for the project had been badly designed and inadequately tested.
The NAO note six other major reviews that raised ‘red’ warnings during the life of the project. None of them resulted in a major rethink of the strategy.
Not surprisingly perhaps, the NAO feels that more audit and assurance reports are the answer to such runaway projects. Rather than measure progress against actual delivery their advice is to use theoretical ‘performance milestones’ on a traditional project plan since, they say, the Agile approach “does not lend itself to straightforward reporting against performance milestones”.
The pointlessness of creating additional levels of assurance is highlighted by the recent PwC report which found that recommendations made by the NAO two years previously were “were mostly implemented by the BBC”. The project still failed.
The PwC report argues for better audit plans, but admits that “Given the change in status from Amber to Amber-Red, it is not clear why the Executive Audit Committee chose to accept a delay in a review of DMI by Internal Audit.”
Human Behaviour Will Always Trump Process
In other words, human behaviour will always trump process. Given bad news, committees will often delay and prevaricate.
Repeated extensions to timetables are a symptom of runaway projects. The Agile way is to cut scope severely to meet deadlines – not battle on building a monolith and putting back the judgement day of implementation as long as possible. It is certainly not Agile to create a fog of consultants’ assurance and review reports with theoretical recommendations that, even if implemented, have no practical impact.
It is true that the BBC’s governance arrangements for the DMI were inadequate for its scale, complexity and risk. But there is not mention in the NAO’s recommendations of the need for an overarching Programme, and modular sub-projects as I have argued previously should have been used on the Universal Credit project (more on that also tomorrow afternoon in another committee room).
Will this problem be repeated?
The BBC says that lessons have been learned. That new procedures have been adopted to manage big projects.
The big question that is being avoided is: Should big projects be started, and allowed to run for years without any delivery?
Or should an Agile approach be adopted by breaking large projects into Programmes of incremental delivery?
Review of the BBC’s management of DMI, December 17, 2013
Memorandum on the Digital Media Initiative, NAO, January 2014
Also: Various news sources, including The Guardian, The Telegraph, YouTube and LinkedIn.