Engineering Defence Enterprises and IT Systems in a complex, turbulent and fast-changing world

Defence Enterprise Systems (DESs) must be continually reengineered and improved to achieve excellence under stress in the increasingly complex, turbulent and fast-changing world in which we live. A DES is a generic appellation used here to designate any sociotechnical system of interest within national defence organizations, ranging from large strategic units to much more focused cross-functional services. DESs must accomplish a mission in an evolving environment. 

In order to succeed, they use human, material and informational resources to provide services with the support of Information Technology Systems (ITSs). ITSs are technical systems that, taken in isolation, add no value. They add value only when used in DESs to provide capabilities and improve service recipients’ experiences in ways not otherwise possible. The use of ITSs must thus be justified by demonstrating that they help the DES in which they are embedded add value. They should not be engineered in isolation based on fixed rapidly outdated requirement statements. They must be engineered as integral parts of a living DES providing evolving services.

This is challenging, particularly in large interconnected organizations like DND-CAF. Although one would like to adapt DESs/ITSs to perceived political, economic, social, technological, legal and ecological (PESTLE) changes as quickly as possible, this is not without drawback. It can be difficult to determine if a major unexpected event or change of state is a temporary disruption (e.g. a service demand swing) or the symptom of a permanent evolutionary shift (e.g. the emergence of the digital society). 

The systems developed must be agile and resilient to cope with disruptions, but changing their design in response to temporary events is counterproductive. Empowering agile multifunctional teams to adapt quickly to local conditions is an approach favored by many, but if applied blindly, without strategic directions and a whole-system value driven architecture, it may destroy system synergy and even lead to a loss of purpose (c.f. Australia’s special forces’ abuse scandal – Foreign Policy, August 14, 2018). 

In addition, to collaborate efficiently with internal and external partners, preestablished interoperability standards must be followed, which requires some stability. Also, human beings have difficulty coping with complexity and change. A minimum of stability is required to allow internal/external actors to learn and adapt to new DES processes and ITSs. Because of this, and because internal struggles induce myopia and hesitation, PESTLE changes tend to outpace organizational adaptation capacity. An approach to try to develop synergic, interoperable and adaptable DESs has been to base their implementation on a shared enterprise architecture (EA). However, specifying detailed all-encompassing EAs introduces rigidities and impedes imagination and adaptiveness. Producing them is costly and time consuming, and it does not guarantee superior results. 

Being adaptive in a turbulent and fast-changing world and, concurrently, preserving interoperability and organizational concinnity seems an unsurmountable enterprise development challenge. To develop robust value-creating DESs in this context, discerning methodological trade-offs must be reached, based on the proven strategy “Think Globally, Act Locally.” Complexity reduction and rapid intervention methods must be found while shaping whole-system synergies and preserving interoperability. This relates to how:

Making educated methodological trade-offs and adopting an insightful enterprise development posture under evolving stakeholders’ needs, finite resources and progressing ED-EA maturity levels, is an important top-management responsibility. In large organizations such as DND-CAF, this responsibility is typically partially delegated to strategically located Development Methodology Authorities (DMAs). Our aim in this short article is to examine some of the issues to consider when elaborating a strategic ED-EA posture and to outline an approach to develop robust value-creating DES/ITSs for a complex, turbulent and fast-changing world.

DES/ITS Architecture

DESs, and their embedded ITSs, evolve continually, and during their lifecycle, one needs to study them periodically As-Is, to evaluate their performance and identify capability gaps. At times, one also needs to redesign/revise their configuration To-Be able to close capability gaps and to ensure that the DES is sufficiently robust to accomplish its mission under any foreseeable future. 

In order to implement, audit or improve a DES, its design (the way in which it is, or needs to be, constructed) must be described in a more-or-less detailed EA. The emerging design of a DES can never be observed/described exactly and exhaustively. An EA is a rigorous description, at a point in time, of some essential elements of a DES design sufficient for action, i.e. focusing on what is useful and nothing else. EAs are shared understanding (synthesis) of an As-Is DES or shared vision of a To-Be DES.

DESs are complex systems of systems and they cannot be described as a whole. EAs are therefore an integrated and coherent set of complementary views (facets of a design) reflecting stakeholders’ concerns. To capture the essence of a DES, four fundamental viewpoints must be taken. They boil down to the following questions about the DES:

1. What is it? – the DES as perceived (human, material, informational and financial resources, as well as external entities and forces)

2. What is it supposed to do? (mission, strategic directions, value creation measures and targets)

3. How is it functioning? – functional views (capabilities, services, activity networks, user journeys, processes, methods, controls)

4. How are resources configured? –structural views (ontology, authority structure, role matrix, IT network, data model, application structure tree, facility layout, bill of material)

These fundamental viewpoints are summarized in the generic high-level EA-cube in Figure 1. A DES can be examined from different viewpoints, but it is indivisible. The functional and structural views are two sides of the same coin. The evolutionary perspective arrow indicates that DESs are living and adaptive organisms in a changing environment.

In an EA, one does not necessarily provide detailed views of all DES elements. The purpose of an EA is to effectively and rigorously convey information about a DES design to facilitate its synergic development and adaptiveness, and to nurture organizational learning. The DES is typically represented at different abstraction levels using selected modeling formalisms. A modeling formalism is a visual (e.g. diagram, plot), tabular (e.g. spreadsheet, database), textual (e.g. pseudocode, form) and/or mathematical notation and syntax used to describe DES views. 

To be useful, a descriptive model must be synthetic, rigorous, easy to produce and manipulate, and easy to understand. Abstraction refers to the extraction of the essence of a DES to make its understanding or reengineering more insightful and substantial. Modeling is usually performed at three abstraction levels. At the top, conceptual models shape concepts and ideas; at the bottom, physical models display specific objects. In between, logical models define the logic of the methods used to perform processes (services) and their use of resource types (no attention is paid to specific resources). A modeling formalism can be telescopic, to provide a holistic view without details, or microscopic, to describe a component in detail. Business shaping views tend to be more holistic conceptual or logical representations. System component specifications are more detailed logical or physical representations.

When studying an existing DES, one proceeds bottom-up, from the physical to the conceptual. When designing a To-Be DES, one proceeds top-down, from the conceptual to the physical. The essence of reinforced DESs emerges from strategic reorientations and generalizations at the conceptual level where the vulnerabilities, plausible future scenarios and strategic directions identified shed light on what must be done to create a design assuring sustained value-creation. 

Determining what to represent formally at different levels and how is the responsibility of the DMA. It involves the adoption of development policies and of an EA-Framework, a formal specification of what an EA should contain. It usually includes a concept’s metamodel (ontology), crucial viewpoint and formalism descriptions, and rules to document EAs. Since DESs are living organisms, in time, during development/improvement initiatives, several fit-for-purpose EA versions are elaborated (As-Is EA, To-Be EA candidates, Transition EAs, etc.). These EAs must be memorized to enable rigorous interventions, knowledge transfer and maintenance. A formal EA-Repository (EAR) congruent with the EA-Framework must thus be implemented to retain evolving DES design descriptions.

Adaptability and Adaptiveness

DESs must perform their mission in complex evolving environments characterized by increasing PESTLE constraints, disturbances and threats, by turbulent and shifting service recipient needs, by dynamic adversary and ally strategies, and by imperfect enabler services. These are the source of incessant unexpected events taking the form of temporary disruptions (e.g. a burst in service demand), or of permanent evolutionary shifts (e.g. a new type of threat made possible by cutting edge technologies). 

Take the recent waves of illegal migrant crossings into Canada as an example. Addressing such a burst in demand is not possible with standard processes and resources. However, a well-designed system can scale its resources up temporarily, reassign existing resources quickly, and have access to external recourses, such as CAF assets, to maintain reasonable service times until the situation comes back to normal. A system capable of coping efficiently with major disruptions is said to be adaptable. This requires readiness, agility and resilience. 

On the other hand, many believe that migrant flows will continue to increase as people flee from failed states to seek a better life. Such an evolutionary shift cannot be addressed with temporary recourses. It requires lasting changes in resources and processes (e.g. building permanent migrant residences near borders and changing processes to accelerate the dispatching of admitted migrants to host cities). However, mistakenly interpreting this increased demand as a permanent shift would be a substantial error. DESs capable of detecting definite shifts and performing required changes quickly and efficiently are said to be adaptive. This necessitates foresight and resolve. Since efficient recourses are needed until the shift is definitively acknowledged, adaptability is a prerequisite to adaptiveness. 

Time is an essential element here. Disruptions and evolutionary shifts arise in continuous time. PESTLE problems and opportunities continually emerge, and successive governments adjust laws, policies, programs and services to cope with them. Thus, to fulfill their mission, DESs need to adapt continuously. Unfortunately, in real life, DES can only adapt in discrete time, and they often exhibit inertia (incapacity to move quickly) and sclerosis (resistance to change). Reengineering and realizing DES capabilities and ITSs takes time, and reacting too fast to apparent contextual changes can lead to confusion and chaos. 

Moreover, imposed antecedents (regulations, standards, budgets, labor agreements, legacy systems, etc.) limit freedom of action, and actors often display behaviors impairing foresight and resolve. In this context, lead times tend to belong, and at a given point in time, there is always a gap between what is needed to cope perfectly with incoming events and the designed (To-Be) or implemented (As-Is) capabilities of a DES. Also, because of the lags involved, a DES/ITS engineered for today’s requirements is likely to be inadequate and misaligned when implemented, which perpetuates large gaps instead of eliminating them. Adaptive DESs can avoid these mishaps and keep lead times and capability gaps relatively small.

Robust Value-Creating Designs

Engineering choices can be made to elude complexity while preserving sustained value creation. A good DES/ITS design can cope efficiently with temporary disruptions and even with some evolutionary shifts without alterations, and if required, it can be changed easily and rapidly. Engineering is mainly about finding (analysis) and specifying (design) patterns: discernable contextual factor and business practice regularities. 

Rooted patterns are discovered by looking at DES to detect recurring behaviors, repeating structures and causal relationships. Design patterns are archetypal building blocs specified to solve recurring DES or ITS engineering problems. They are often based on observed patterns and best practices, but they may also be original innovations. They can take the form of a reference architecture (RA), an application component, etc. Note, however, that bad habits are also patterns. It is not because something was always done one way that it adds value! Hence, before adopting a design pattern, engineers must prove that it adds value in its application context.

Engineering for mission assurance requires the generalization and optimization of design patterns and their compliance with interoperability standards so that they can be easily attuned to varied contexts and conditions, as well as customized for special needs, often unpredictable at the design time. This is the secret of robust long-lasting designs. When generalizing, one typically replaces particular constructs with personalization data (or particular settings) within more generic constructs. 

For example, the mathematical function:

 is a particular case of the general function:

Implementing the latter allows a great variety of functions to be used without changing the design. To be reusable, generic patterns are documented and memorized in a Building Blocks Repository (BBR). Setting-up and managing this repository is an important function of the DMA. 

Generalization and reuse help greatly, but in a fast-changing world, if development lags are substantial, it may not be sufficient to ensure that capability gaps remain small. To achieve this, lead times must be shortened, and it must be possible to postpone the adoption of a final design as far as possible in the development cycle. Delays can be reduced by assigning activities related to the engineering and transition of service value chains to agile multifunctional teams, and through strong leadership to grow a collaborative team attitude. Design decisions can be postponed by defining real options (i.e. EAs embedding services, components or technologies) that are desirable but that could eventually be dropped, deferred, expanded or altered, depending on how the future unfolds. The lifecycle of a DES/ITS contains gestation periods (see Figure 2) followed by much longer operation periods. 

Since the life of a DES can span several decades, during the steering and design phases of reengineering initiatives, it is not possible to anticipate needs reliably for far-away years. The best that can be done is to elaborate plausible future scenarios for a foreseeable moving planning horizon (say 5-10 years). To postpone design decisions as far as possible and base them on fresh information, the scenarios and real options elaborated are revisited as new data becomes available, and final decisions are made at the last minute to ensure that capability gaps are small. At the limit, all the design is optional and EA elements are revised as new information becomes available.

Adaptive Enterprise Development

Knowing how to describe successive DES/ITS designs with EAs is important, but determining how to evolve DES/ITS designs to adapt to environmental changes and keep capability gaps small is essential. Depending on which aspect of a DES one concentrates (its organizational structure, material structures, service delivery, business processes, or ITSs), several methodologies were proposed in different fields (organizational design, business strategy, system/industrial engineering, business process reengineering, software engineering) to create or evolve enterprise systems. 

These methodologies tend to be complementary, and they are mostly based on a sequence of analysis, steering, design and transition phases as illustrated in Figure 2. The design phase produces a To-Be EA which becomes the blueprint for the construction/procurement, testing and deployment of a reengineered DES in the transition phase. Once the new system is born, continual local improvements are made to try to adapt the DES/ITS to evolving needs. However, with time, because of technological advances, completely new needs, new threats, etc., legacy systems become obsolete and one must engage in new major reengineering and transition initiatives. These initiatives often give rise to large and costly projects lasting several years (above a moderate size threshold, larger projects often display diseconomies of scale). The resulting delays and rigidities are counterproductive and preclude adaptiveness. This explains why these classical approaches are severely criticized and organizations are seeking more agile development methodologies.

The rigidity and inertia of conventional approaches come mainly from the fact that phases are performed sequentially, often by different actors, so it is difficult to go back and change direction when new information about needs, ally/enemy strategies and PESTLE trends become available. This is exacerbated when the DES-of-interest is sizeable and when documentation standards and preservation means are deficient or fuzzy, which hinders fast EA changes and reuse. 

To be able to adapt quickly to emerging evolutionary shifts, ED phases must be decoupled. This is possible only if the monitoring, analysis, steering, design and transition information produced is shared in a common formal memory, as shown in Figure 3. This memory takes the form of an ED-Repository (EDR) that contains the previously defined EAR and BBR, as well as any other ED deliverables necessary to empower evolved designs in time (factor trends, plausible future scenarios, diagnosis, strategic directions, cost-benefit-risk evaluations, transition plans, knowledge transfer material). The EDR must be congruent with the EA-Framework, but it does not have to be monolithic. It can be split and distributed in selected strategic units. These strategic units then become the depository of the ED information of the DESs under their authority.

DESs are part of larger systems (alliances, GC, DND, etc.) and they embed smaller systems related to an enterprise unit or cutting across several units. A reengineered DES-of-interest can be at any level in this continuum, and it may focus on specific capabilities. When adopting an agile ED approach, DESs-of-interest are smaller and directly related to what needs to be changed to adapt. Any facet of their positioning, orientation or EA can be remodeled by collaborative multifunctional teams as new substantiated information becomes available. The coherence and integrity of the design elements modeled is enforced by adhering to the EA-Framework and by preserving successive changes in the EDR to allow the reconstruction of views at selected points in time, and not by producing EA documents sequentially as imposed by a compulsory engineering process. 

The transition can then be made gradually for small DES-of-interest and even allow changes in design during construction. If the EA is well generalized at the conceptual and logical level and real options are available, physical level design and personalization decisions can be postponed and finalized by multifunctional development teams during the transition. Note, however, that the approach does not preclude the assignment of several adjacent development activities to a larger project if necessary.

The EA models produced at the conceptual and/or logical level do not have to be very detailed. They aim mainly to provide a common language (ontology), to identify and shape reusable patterns, and to give a holistic picture of the functionalities and structures required to achieve value-creation, mission assurance, interoperability and generality. The optimization of material and data structures is also a concern at this level. 

By thinking globally, analysts, strategists and engineers working as a collaborative team on cross-functional service value chains (and not on siloed activities) reduce the design space and provide guidelines for physical level design decisions. At the physical level, teams of engineers, builders and operators act locally, under the directions of higher-level EA-views and with the help of proven building blocks, to quickly implement services, applications and structures that are simultaneously synergic, interoperable, user-friendly and conducive to superior service recipients’ experiences. 

This short article sketches the elements of an agile strategic planning and enterprise engineering approach to adapt DESs and ITSs as needed in our complex, turbulent and fast-changing world. We hope the ideas introduced provide a lens through which this challenging competency can be mastered.

The views expressed in this text are those of the authors and not that of DND-CAF or the Government of Canada (GC).  

Alain Martel is Professor Emeritus at Université Laval in Quebec City and he is a former CAF officer.

Sophie Martel is a Master of Engineering and she is a former CAF officer (CD).