In a previous VANGUARD article, I explained my perspective on why risk aversion is prevalent in Canada and other democracies, and the damage it does to weapons systems platform acquisitions.
I asked whether a general contractor could give you a believable firm price to build a yet-to-be-developed fulsome set of requirements, for a house on an identified Rocky Mountain hilltop, and to be delivered 10 years from today. I then said this was analogous to what we do when buying a weapons systems platform and much riskier than for the house price quote.
Please be patient as I remind of the basics around project costing. Budgets essentially reflect the aggregate cost of work to be done as planned – in a specific order and at specific planned dates/times – in terms of salaries, time and material, and overhead. When plan implementation is delayed and deviations occur due to errors in the timely conduct of the work as a result of emerging risks, additional time is required to resolve them. Time costs money, so project overall efficiency often suffers in terms of delivery schedule slippage and cost. This is the reality of complex weapons systems platform acquisitions, but why is it so prevalent an issue?
Consider for a moment the first time you tackled a complicated task: preparing a soufflé, completing a DIY brake job on your car or producing your first urgent one-page briefing note for a CEO or Minister. Typically, the first time takes longer than expected due to additional care and a few wrong turns. By the time you repeat any of these, you are much more efficient.
The same is true of Canada’s weapons systems platforms, uniquely designed to address our climate, operational doctrine, national standards, specific choice of weapons systems and so forth. The design is routinely novel to some extent and the company constructing the platform will require a degree of new production techniques, workforce learning and suppliers to some extent – in other words, there will be many ‘first timers’. The reality is that designing, constructing and testing new platforms unavoidably involves a complex – even monumental – set of novel challenges, even for those in the business for decades.
As we launch these acquisition projects, taxpayers and thus government officials insist on being told what the cost will be and when deliveries will occur, as well as the through-life price tag. While this works for simple purchases of off the shelf items, such pronouncements by our government do not work for complex acquisitions characterized with novel technologies, scores of internal and external stakeholders, a project life from five to 20 years in length and an expected price tag of a billion dollars or more – almost all that are unknown when a project is launched.
Consider how risky it is to publish the cost and delivery schedule of the desired artefacts when a platform acquisition is first announced:
- Beyond scant operational requirements, no technical specifications exist and no prime contractor or design has been selected.
- The government’s complicated approval process involves a myriad of stakeholders who do not commit to schedules for their services.
- Once selected, the prime contractor will face unpredictable risks with workforce availability, subcontractors, national security, technologies, relationships, extreme climate events, wars and more.
- A through-life cost estimate includes hundreds of assumptions in place of unavailable certainties for a 20-year platform’s life expectancy.
Why then would the government announce such risky schedules and budgets with confidence at the commencement of projects? In doing so, projects are set up at the starting gate for the eventual perception of project failure.
You will say that it is politically too risky to announce a weapons systems platform acquisition project without a committed project budget and schedule – there can be no blank cheques. But probity – the truth – is also a requirement. Hence a dilemma presents itself: does our government continue to announce a budget and schedule estimate that is essentially an educated guess (being generous) and embrace great risk to credibility, or should the Ministers avoid doing so because no one in government yet knows and the admission of same could be political suicide?
Treasury Board (TB) policies appear to require Class D estimates based on uncertainty margins of 10% at the first Project Approval stage. The Parliamentary Budget Officer’s job is to produce independent assessments of the validity of cost estimates when requested by Parliamentarians. And the Auditor-General doubles down on the need for better costing capabilities in the Department of National Defence (DND) when projects significantly exceed initially announced cost estimates.
Typically, the methods sanctioned by the International Cost Estimating and Analysis Association are employed within DND. As well, sophisticated commercial models are based on databases of past international examples. And yet, many projects exceed these estimates by 25% and much more.
So what can be done?
From a risk-smart perspective, budgets and scheduled could be created for multiple project phases, with the first meaningful publishing of the project’s full budget and schedule delayed until well into the project. In that vein, I recommend moving from the five project phases now employed to nine as per the diagram below, and that the project’s first unqualified and evidence-based cost and delivery schedule provided publicly only at the end of the fifth phase.
The risk-smart approach is to adopt wave planning, with the following general comments being germane:
- Complete transparency with stakeholders (including the public) would occur at the initial announcement of projects, explaining the challenges of producing reliable cost estimates so early in the project. Hence, the first reliable project cost and schedule will only be available after the fifth phase of the project which is years away. By developing understanding of the issues and the rationale for the approach going forward, credibility is possible. Comments such as “We don’t know yet” and “We will tell you when mistakes are made” are necessary, despite the political challenge.
- A phased detailed planning, risk review and budgeting approach would then be pursued for each phase’s budget and submitted to TB for approval – thus the wave approach. It is essential that TB facilitate the timely approval of phase budgets to prevent significant disruptions to the work.
- The first two phases (Initiation and Definition) would be based on Service Level Agreements (SLAs) for all government stakeholders, along with the use of ‘swarming’ as led by TB to gather all stakeholders for day-long meetings to expedite submissions to the TB-ready standard.
- The follow-on implementation phases would be planned with budgets and schedules developed by the select prime contractor, in collaboration with the Project Office.
Exploring the notional phases, efficiency measure announcements (cost and schedule) and risk treatment considerations, we would see the following:
- The Initiation phase budget would be part of the project’s capital cost, this to enable a fully staffed Project Office to be stood up to conduct (or contract for) analyses and early formal engagement with industry. Project Initiation should include a simple TB submission to achieve ample capital funding to properly plan, schedule and deliver a budge for the Project Definition phase.
- With TB approval of funding for Project Definition, the usual activities would occur in terms of finalizing requirements, industry engagement, sourcing and evaluating bids and negotiation with the selected/preferred prime contractor to achieve clarity and to agree on an start-up contract and budget to facilitate early action
- The winning prime contractor would be announced, along with an explanation that the contractor would lead on planning, scheduling and budgeting the follow-on Implementation phases, and the reason for generous contingencies for some phase budgets of the to deal with ‘first time’ activities.
- Within months or the award of the start-up contract, the plan and budget estimate through to production design completion would be developed, approved and announced, this critical stage including generous time and funding for multiple preliminary and functional design options or spirals to reduce risk during construction.
- As the design was finalized, the plan and budget to achieve first platform construction, trials and commissioning would be provided and approved.
- As the first artefact approached trials and commissioning, the full project cost estimate would be produced to complete all construction and deliveries until project completion, while recognizing expected learning curves as production picks up. If deliveries were ‘batched’ (common with ship production), each batch would then receive phase status for planning and budgeting.
In terms of the through-life cost estimates for new platforms:
- This phase must be delayed until all major equipment suppliers are on contract with the prime contractor and have provided estimates for maintenance and anticipated through-life upgrades, for the expected and various potential operational profiles (e.g. high contingency operational tempo, alongside mothball status due to funding shortages). The rationale for this delay also needs explaining to the public.
- Assumptions must be created for personnel and consumables (especially fuel) where there is considerable volatility beyond anyone’s control out 20 years.
- Such estimates should be accompanied with a commitment to updates at specific periods of time and to track reality and adapt life cycle costs against the original assumption set.
The eighth phase requires life extension as necessary for some of the in-service platforms to be replaced, to maintain operational capability until the acquired new equipment is delivered. This is an exceptionally complex planning activity that is beyond the scope of this paper. However, it will be considerably facilitated by embracing the Continuous Capability Sustainment proposal now being trialed within the government.
In closing, AI offers the ability to significantly assist in much of this work, with commercial interests racing to perfect timesaving products to facilitate decision-making. What is essential now is the definition of the information set required to enable AI to reach its full potential.
And finally, all of this raises two key questions. Will the stand-up of a Defence Procurement Agency embrace similar smart and appropriate risk treatment as demonstrated in this note, in a courageously transparent and meaningful treatment of budgeting and scheduling? And might this evolved and somewhat de-risked approach first be trialled for the Canadian Patrol Submarine acquisition project?
Time will tell and the clock is ticking.