Verification and Validation of Mission Data Libraries for Electronic Warfare Operational Support

Verification and Validation of Mission Data Libraries for Electronic Warfare Operational Support Flight Lieutenant Charles Winsor Royal Australian Air Force Dr Thomas Millhouse Nova Systems, Australia

Introduction Electronic Support systems depend on libraries of accurate, current and relevant mission data to identify emitters and provide tactical threat warning to aircraft and ships Electronic Warfare Operational Support is the process of updating these mission data libraries to adapt to changes in the operational environment

Introduction Library reprogramming has historically been treated like software development Software engineering standards demand time-consuming Verification and Validation for mission critical systems Verification and Validation provide assurance that an Electronic Support system will perform in its intended operational environment

Introduction Excessive Verification and Validation may compromise the essential responsiveness of Electronic Warfare Operational Support We propose a new approach to balance the rigour of software engineering with the responsiveness of Electronic Warfare Operational Support

Acknowledgement

Outline Electronic Support Mission Data Libraries Electronic Warfare Operational Support Verification and Validation Finding the Balance A New Approach

Electronic Support Electronic Support is the branch of Electronic Warfare that deals with the passive intercept, analysis and exploitation of electromagnetic radiation Electronic Support is used in modern warfare for surveillance, intelligence collection and tactical threat warning of radar-guided weapons

Electronic Support Electronic Support systems aim to: intercept and detect signals locate, identify and report emitters INTERCEPT Receiver DETECT LOCATE Signal Processor Mission Data Library REPORT Operator Display IDENTIFY

Mission Data Libraries Most Electronic Support systems are loaded pre-mission with reprogrammable mission data libraries These libraries enable automatic identification of detected emitters High quality libraries are essential to achieve timely threat warning: manual emitter identification by an operator is slow fast reactions needed against threats

Mission Data Libraries Emitter parameters form the basis of a mission data library: enables identification of emitters from detected signals Emitter-to-platform associations improve situational awareness: identified emitters may be correlated to known platforms or locations Additional information can be associated to emitters to improve situational understanding

Mission Data Libraries Mission data quality characteristics: Accuracy Currency Relevance All share a common dependence on time All are interdependent

Mission Data Libraries Mission data libraries are typically specified to suit an Area of Operations: libraries should be updated as emitters enter or leave the specified region Mission data is typically based on observed emitter parameters: libraries should be updated as emitter parameters change particularly important as we enter the era of software-defined radio

Consequences No identification Misidentification Ambiguous identification

Consequences No identification: no threat warning countermeasures not initiated platform loss Misidentification or ambiguous identification: false alarm wrong countermeasures initiated mission abort

EW Operational Support Aim To update libraries before they become ineffective or unsuitable for operational use Problem The electromagnetic environment may change significantly on a daily basis Challenge To complete post-mission analysis, update library requirements, develop an updated library, and verify and validate that library to a suitable extent all before the current library becomes an operational liability!

EW Operational Support Pre-Mission Upload Post-Mission Download OPERATE Mission Replay ANALYSE Resolve Anomalies Recommend Changes Validate Verify REPROGRAM Develop Design

Verification & Validation Australian Technical Airworthiness regulations treat mission data libraries as software (not as data): similar regulations are applied in the Australian maritime and land domains Software engineering standards like DO-178B are applied to routine library reprogramming: threat warning sensors are mission critical systems libraries may be assessed to need high Software Assurance Levels Assurance of mission critical software demands time-consuming Verification and Validation

Verification & Validation Verification The evaluation of whether or not a product complies with a requirement specification The library was developed correctly Validation The assurance that a product meets the needs of the customer The correct library was developed [IEEE 1490-2011]

Verification & Validation Verification and Validation are important to provide assurance that mission data requirements are satisfied by a library......but those mission data requirements change whilst library Verification and Validation is being conducted An optimal balance must be found between rigour and responsiveness

Finding the Balance 1. Programming errors found and resolved 2. System performance optimisation 3. Mission data currency degradation 4. Mission data relevance degradation

Verification Software-only testing: basic error checking basic performance estimation Was the library implemented as designed? Hardware-in-the-loop testing: advanced error checking advanced performance estimation assess emitter interactions Does the library design satisfy its requirements?

Limitations of Verification The same mission data is used to develop and verify the library The performance of a library is fundamentally limited by the quality of the available mission data Verification predicts library performance assuming mission data is correct Library verification provides no assurance of underlying mission data quality

Validation Field testing: Were the library requirements fit for purpose? Options: controlled or mission conditions? generic test scenario or mission rehearsal? emitters of interest only, or background emissions as well?

Limitations of Validation High cost If emitter emulators are used, same fundamental limitation as verification: predicts library performance assuming emitter emulation is correct If authentic emitters are used, still not a true representation of the operational environment: no two radars emit exactly the same signal when does close enough become good enough?

A New Approach Verification and Validation are activities performed as part of the library reprogramming process Library reprogramming is an activity performed within the EWOS cycle What if we treated the entire EWOS cycle as a holistic Validation activity? Can rapid, continuous execution of the EWOS cycle fulfil the aim of Validation?

EWOS as Validation Collect test data from operations: true mission conditions emitters of interest background emitters Assess library performance in the operational environment through post-mission analysis Assess the ongoing suitability of library requirements: Were yesterday's library requirements still valid during today's mission? If not, what needs to change? And how else do we expect tomorrow's mission to change those library requirements?

EWOS as Validation Does rapid, continuous execution of the EWOS cycle mitigate the need for a formal Verification and Validation process? A low quality library can still cause mission failure or the loss of life and materiel: focussed testing of high risk emitters Potential to compound errors: initial library version must be of reasonable quality Similar principles to the Spiral Model for software development [Boehm, 1988]

EWOS as Validation OPERATE ANALYSE REPROGRAM [Boehm, 1988]

Improving Library Performance

Implementation Holistic consideration of Verification, Validation and the entire EWOS cycle Organisations, personnel and infrastructure capable of sustaining highly responsive EWOS Expertise to identify mission risks within libraries and tailor Verification and Validation processes to specifically address those risks: instead of blanket-application of generic software engineering standards to EWOS

Into the Future Mission data library development processes for Electronic Support systems are currently similar to the Waterfall Model [Royce, 1970] Rapid, continuous execution of the EWOS cycle could be considered similar to the Spiral Model [Boehm, 1988] Many new software development models have emerged in the last few decades Is it time for EWOS to move into the twenty-first century?

Conclusion Libraries of accurate, current and relevant mission data are essential to enable the performance of Electronic Support systems Low quality mission data libraries can result in mission failure and other serious consequences Assurance of library quality is conventionally achieved through application of software engineering rigour

Conclusion Conducting extensive Verification and Validation processes to assure library quality can actually reduce performance The EWOS cycle must be responsive to keep pace with an evolving electromagnetic environment A better balance must be found between rigour and responsiveness

References D.C. Schleher (1999), Electronic Warfare in the Information Age R.G. Wiley (2006), Electronic Intelligence: The Interception and Analysis of Radar Signals DO-178B, Software Considerations in Airborne Systems and Equipment Certification IEEE 1490-2011, A Guide to the Project Management Body of Knowledge W. Royce (1970), Managing the Development of Large Software Systems B. Boehm (1988), A Spiral Model of Software Development and Enhancement

Verification and Validation of Mission Data Libraries for Electronic Warfare Operational Support Flight Lieutenant Charles Winsor Royal Australian Air Force Dr Thomas Millhouse Nova Systems, Australia