Improving Future Systems and Capabilities through Analysis and (Warfighting) Experimentation Lessons learned from Niteworks experience of Live, Virtual, Constructive Experimentation 1
Scope of Presentation What do we mean by Warfighting Experimentation? The contribution from Warfighting Experimentation What do we mean by Constructive Modelling? The contribution from Constructive Modelling Application of the Live, Virtual and Constructive Towards an Integrated Analysis and Experimentation Approach 2
Recording Subjects Experimenters UK UNCLASSIFIED Warfighting Experimentation (WFE) A subset of OA-Experimentation that involves the participation of military personnel, in an appropriate role, in an environment which allows all Lines of Development to be considered. EXCON Exp. Director C4I systems Military experts HQ Military staff C4I systems HQ Military staff HQ Military staff C4I systems C4I systems Data Collection Observers Simulation of the real world 3
WFE Example 1: Land Experimental Environment (V/C) War Fighting Experiment over 10 days Virtual World Environment New TTPs developed with customer Synthetic platforms Communications Network Emulation Battle Management System Joint Fires/Air/Avn Workload assessment tools 35 warfighters from Army recce units Appropriate scenario & vignettes 4
WFE Example 2: Warfighting Expt: Medium Weight Capability (V/C) 300 400Km Log Bde Corps Main/ C/JFLCHQ Main HQ Moving Battlespace Command and Control IMPLIES Context: NEC FRES Medium Weight Ops 150 250Km 50 150Km Fwd HQ Fwd HQ Situational Awareness Addressed: HQ Structures Comms Situational Awareness Sensor to Shooter 0Km IMPLIES AMG Comd Mobile AMG Manoeuvre Battlespace Experiment: Live players 10 runs (1/2 day per run) 5 treatments, 2 runs/treatment 5
WFE Example 3: Maritime ISTAR (V/C) Context: Force Protection Littoral Projection Addressed: Eqpt options C4ISTAR Visualisation: Live players Simulated cockpits set 2 Scenarios 4 vignettes 4 treatments 32 unique runs 6
Contribution of Warfighting experimentation Maritime-ISTAR: Provided warfighting experimental evidence for an Equipment Review Note. Contributed to analytical framework and identification of MOPs/MOEs to support capability view of acceptance criteria for URD/SRD shaping. WFE specifically contributed Workload, Picture Quality and Situational Awareness evidence Evidence used within the Assessment Phases 1 and 2 to determine how candidate solutions are discriminated. No project-level IAEP existed at the start 7
Dstl Inputs/Outputs OA 8
Other Dstl OA Studies and M-ISTAR Expt Study 1 Study 2 Study 3 Study 4 9
What we have learnt about the role of Warfighting Experimentation WFE provides evidence but at a significant cost WFE often answers a small part of the problem space due to the requirements for control and availability of participants to contribute Experimentation is one of the tools that can be used to evaluate alternative candidate concepts of equipment and/or processes Experimentation sits within a wider operational analysis context There must be an Integrated Analysis and Experimentation Campaign Plan (IAECP) to ensure appropriate exploitation and co-ordination of co-dependent or related studies Consistent with extant OA guidance (2001): multi-methodology likely the right approach for most assessments...consistent with the heritage of OR/OA... likely to result in well-structured, rigorous, and quantitative analysis, of maximum utility to the decision maker 10
Constructive Modelling/Simulation Virtual Modelling and Simulation involving real people operating simulated systems. Virtual simulations inject a Human-in-the-Loop into a central role by exercising motor control skills (e.g., flying a simulated jet), decision making skills (e.g., committing fire control resources to action), or communication skills (e.g., as members of a C4I team). Constructive - M&S involving simulated people operating simulated systems. Real people stimulate (provide inputs) to such simulations, but are not involved in determining the outcomes. TTCP GUIDEx suggests that: Experiments conducted using Constructive simulations allow repeated replay of the same battle under identical conditions while systematically varying capabilities, tactics employed, or levels of threat. Experiments using Constructive simulations with multiple runs are ideal for detecting change and isolating the cause of that change. Because modelling complex events requires many assumptions, such as valid models of human behaviour, critics may question the applicability of results to operational situations. Some examples follow: Source: Wikipedia 11
Analysis/Modelling: Cross-Theme Programme (C) Purpose: Animated Architectures Calibrated by experimental measurements Derive MOEs Shape future Themes (M-E-M) Achievements Synthesis of architectures Working model + analysis Commentary on past Themes Challenges Complexity/Opacity Tempo Credibility 12
Analysis/Modelling: Information Exchange Requirements (C) A FOB 1 To Bde HQ Ptl Reaper PMT ANP H450 FOB 1 ANA OMLT BG Operational Area (Representative) As Is Ptl Ptl Logistics Supply Base C FOB 3 FOB 3 Ptl Ptl OMLT ANA FOB 2 B ANA OMLT CLP FOB 2 Mob Ptl Ptl OMLT ANA Air Tac Recce FOB 4 Key: FOB Ptl Mob Ptl CLP ANP PMT D DH3 FOB 4 (-) Open Area/Desert Green Zone High Population Density Compounds Forward Operating Base FOB Sensor Coverage Patrol Base UK or ISAF Afghan National Army Afghan National Police Foot Patrol Mounted Patrol Combat Logistics Patrol Routes (incl. MSR) Purpose Beauty contest for comms options Option evaluation in context of a scenario Achievements Comms/SA representation Contextual Visual Intuitive Challenges Specifying the scenario Generalisation of results Incorporating human behaviour Data intensive Expectations of fidelity 13
Application of the Live, Virtual and Constructive Model Test Model Constructive Model runs? Live Virtual Identify required constructive enhancements? Develop modelling capability Provide base-case for model capability Understand model shortfalls & direct enhancements Re-run developed model Understand model shortfalls & direct enhancements Assess fitness for purpose Roll-out as an Operational Effectiveness tool? 14
URBAN WARRIOR 5 Concept 2012 2013 Method Baseline Study Constructive Experiment Field Experiment Virtual Experiment Validate Findings & Generate Constructive Model for Future MQ Feb - Apr Oct Dec Jan Mar Aims. Methodology - Complementary analysis of Live, Virtual and Constructive experimentation to support 5f in a Model-Test-Model Paradigm Military Question (MQ) Army experiment in challenging environment 15
Challenges encountered with a M-T-M approach Operational Tasks Areas not addressed by experiment Operational Context Terrain, RoE, Op Type Specific Tasks Environment operations Focus of M-T-M Broader task set Conclusion: While operations within a particular scenario could be considered as a generic vignette across multiple scenarios; it will always be a special case, and will highlight only a small proportion of the capability requirements demanded by Policy. 16
Towards an Integrated Analysis and Experimentation approach? Multiple forms of evidence are required to support decisions and inform requirements. Evidence generated can support capability requirements, development of System Requirement Documents, trade space for solutions and business space. Integrated and coherent across DLoDs and project/programme activities. Appropriate techniques need to be applied. Identify evidence required. Exploit individual strengths of each technique to generate evidence Conduct Analysis and Experimentation to collate or generate new data. Testing validity of hypotheses. Apply quantitative and qualitative methods. Employ Constructive Modelling and Warfighting experiments Live or Virtual. 17
Questions? Chris Jordan Chief Analyst, Niteworks chris.jordan@niteworks.net 18