I choose effectiveness over efficiency – Peter Drucker
Anyone who has been reading my recently published notes and papers would have noticed a focus on risk. But the obvious question is risk to what? In this article, the answer is to the government’s competence and credibility in acquiring weapons systems platforms.
To state the obvious, virtually everyone who works for someone else or is personally pursuing just about any endeavour with a goal is expected 0r expects to reduce risks to the desired outcome. Whether a delivery agent, parent, teacher, receptionist or engineer, the job is to follow agreed procedures to avoid risk or develop guardrails amidst novelty. And the desired outcome is success in reaching the defined goal(s), assuming success was clearly understood.
For decades, the project management professionals described success in terms of three factors related to when projects were entering implementation. These were described as the ‘iron triangle’ of constraints to be controlled to achieve project success:
- Scope – the work required to produce the artifact and/or service defined by the end users in terms of features and functionality.
- Time – the period required to complete the project scope.
- Cost – the aggregated dollar value for each element of the work, also known as the project budget and the price that the user would pay for the artifact or service defined by the client.
Simply put, the scope defined the work planned to meet the customer’s product or service requirements, which would then be delivered at the time and within the budget agreed to achieve success.
However, we all know that there are a myriad of things that can go wrong in any commercial endeavour. Personnel availability, material transportation, certification and systems supplier performance can all occur. As we have recently seen, climate extremes and pandemics can create disruptions and project chaos. Stakeholders can cease to be aligned and withdraw critical support. These take time to address such that the schedule slips and project costs increase. If the triangle is iron, this puts pressure on the prime contractor’s profit margin – whether he is to blame or not – and this can lead to a flawed product. As well, warranties are routinely short, especially when military clients take possession of weapon systems platforms – pampering equipment is never part of combat readiness training.
And as is the case across democracies, such complex acquisition projects are assessed by observers if they are significantly late and/or over-budget as failures.
At this point, it is useful to revisit the terms efficiency and effectiveness, based on a random review of definitions which specifically included those of PROJECTRON, a project software company:
- Efficiency is defined as the ability to achieve desired results with the least possible effort, in terms of the degree of optimization of resources (such as time, money and materials) to achieve a clear and specific goal by completing tasks quickly. Many definitions suggest moving quickly to complete work ‘without sacrificing effectiveness’.
- Effectiveness is defined as the ability to achieve the desired results corresponding to a previously defined goal. It is about what you have done and whether the result meets each project’s expected outcomes.
Governments prioritize the efficiency measures of project schedule and cost when defining and discussing project success, thereby unintentionally prioritizing these as critical project outcomes. They also unintentionally seem to assume that the delivered weapon systems platforms will effectively provide military members with what they asked for to go into harm’s way. Not surprisingly, soldiers, sailors and air men and women care most about effectiveness – that for them is success.
There are two areas where effectiveness matters: the enabling capabilities of appropriate project enterprise management, and the platform/equipment that is delivered.
Effectiveness in Executing Complex Acquisitions
It is important to understand what you are signing up for when you make a decision and to be willing to commit to everything required to make your chosen course successful – Robert Glazer
Peter Drucker summed up effectiveness as “doing the right things right”.
I argued in a previous VANGUARD article that the statements of requirements of all nations’ military organizations are typically unique to meet their assigned roles by government, civil and military standards, operating climates, operational doctrine, the selected weapons on the platform and much more. In finalizing the design and construction plans, considerable novelty will translate during implementation into emerging risks. And when these are numerous and significant, you have complexity in execution that must be catered for.
A research paper by Mikkelsen in 2018 offered a fresh perspective on the subject of complex projects, his main thesis captured in the diagram below.
The primary conclusion was that as projects become more complex, the importance of cost and schedule are of less relevance to successful outcomes than the effectiveness of tailoring the methods employed to the degree of project complexity as you plan and execute such projects. And yet, ensuring projects are equipped for the expected project challenges and supporting them so they have the appropriate capability and especially capacity are based on what is available. When a new weapons systems acquisition project’s political stars finally line up in Canada – and usually very late to need – we always go with who we have and the processes and practices in place.
Below is a schematic that captures the majority of the components required for all projects to be effectively planned and implemented. But complex weapons systems acquisitions need such components to be tailored, going well beyond the famous iron triangle and the demands of simple or complicated projects.
The message is simple. Complex projects require project officials with an open mind and additional skill sets in managing complexity to plan and execute effectively. I have elsewhere enunciated the details of various techniques to be considered in navigating complexity, based on my involvement with the International Centre for Complex Project Management (ICCPM) headquartered in Australia. I would add that the risk treatment policy and practices during my tenure – as highlighted at the top of this star diagram – were in my view the most inadequately addressed of all these requirements for navigating complexity.
Those involved in shaping the Defence Procurement Agency would do well to connect with ICCPM to optimize the potential benefits of the intended reforms.
This leads to two critical takeaways:
- If the client and prime contractor do not have the appropriate capabilities in skills and knowledge (including access to comprehensive commercial acumen relevant to the related project), they are likely to be ill-prepared to recognize the early signals of emerging risks before significant project delays occur and costs increase dramatically.
- Even when employing techniques with a track record of taming complexity, the best likely result will be the minimization of the harm to project outcomes.
Effectiveness of the Product Delivered
Effectiveness of the product can only be confirmed long after deliveries are completed, contracts are defunct and project offices have been closed. More specifically, it is very risky to declare “Mission Accomplished’ prematurely.
Briefly, key effectiveness considerations must ignore the early equipment and man-machine interface teething problems, often lasting beyond the warranty periods and into years. It is only then that the user community can assess the initial operational capability acquired and its availability, either of which could fail to meet the user’s needs:
- Safety is always the number one priority, especially with novel weapon systems. Dangerous consequences have occasionally resulted when weapons were first being employed, requiring prolonged investigations, procedural changes and the related delays to effective equipment employment.
- Once users are trained and experienced with the equipment, platform, weapon and sensor capabilities can be assessed in realistic combat environments for different climates and contingency scenarios.
- Ease of in-theater and home base maintenance is crucial for the technicians in uniform, many more complicated maintenance procedures not required or pursued until years after commissioning.
- Reliability is critical for the in-service period to confirm the original equipment manufacturers’ (OEMs’) reliability claims, routinely a decade or more after initial light-off.
- Timely post-warranty customer support from the OEMs is also part of project success, including in-theater field service representative assistance, depot-level maintenance and necessary design modifications, upgrades and software loads.
Two examples from history are germane:
- The USAF’s Globemaster transport aircraft program under McDonnell Douglas was launched in the ’70s and ‘80s, bur significant development issues plagued the program such that the OEM by 1993 was out of pocket $1.5B (US) and well behind schedule. During this long period, the original order for 220 aircraft was reduced to 10 as a result of these challenges, and the project was on the cusp of being shutdown as a failure. But by 1997, the company started selling the aircraft which was subsequently purchased by 7 other nations. After 20 years, 274 aircraft had been delivered.
- The famous Sydney Opera House was originally scheduled as a project to take four years to deliver with a budget of $7 million (AUS). When delivered, after 14 years, the cost was $102 million (AUS). Many declared the Opera House a failed project when it opened, but today it is heralded as an international icon of Australia.
As these examples demonstrate, risk aversion in response to media inquiries in democracies encourage the government to focus too much on the efficiency of the execution of weapons systems platform acquisitions, by which such projects are deemed to be failures. But what truly matters with such complex endeavours – along with unavoidable concerns about schedule and budget – is the effectiveness of the project execution aspects being appropriate for the project, and then the equipment delivered and its support. The latter – the true purpose of acquiring weapons – is what really counts, and what the project execution team must ensure they also focus significantly on.
For the record and contrary to the standard refrain that ‘military procurement is broken’, my observation is that once these systems reach full operational capability, Canadian Armed Forces members generally believe that they have effective platforms to go into harm’s way.
As the Defence Procurement Agency stands up, explaining this to government stakeholders and the public in transparent briefings could go a long way to refocusing as more on the effectiveness of the execution team and the product delivered when evaluating the success or failure of weapons systems platform acquisition projects.
