Combat System Application of Change-Tolerant Technology: Using Rules Engine for Decision Automation

Transcription

1 Catherine L. Payne, Mark E. Schmid, Barbara A. Shapter, and Brian T. Taylor Combat System Application of Change-Tolerant Technology: Using Rules Engine for Decision Automation ABSTRACT Advances in threats, geopolitical developments, and commercial technologies continually challenge combat system stability because they constantly create new demands and therefore requirements for the system. Because the lifecycle of major Department of Defense (DoD) systems can last decades, long-fielded systems in particular are vulnerable to requirements creep. One technique to mitigate the impact requirements volatility can have on a system is through the implementation of technologies that have been developed to allow the system to adapt while minimizing the effect of change on the system as a whole. This paper investigates the application of one such technology Rules Engine to the Decision Automation domain of two Navy combat systems. This paper also proposes modifications to the systems engineering process given that using change-tolerant technologies affects the way a system can be developed and maintained. INTRODUCTION Development and maintenance of major DoD systems face unique challenges because of the length of their expected lifecycles. Because major defense systems can be in service for decades, they are vulnerable to changes in requirements and technology. Vulnerability to technological changes is highlighted by the military sector s los of DoD influence in critical technological development areas. According to the Defense Microelectronics Activity, which operates under the authority of the Undersecretary of Defense for Acquisition, Technology, and Logistics, the military sector has significantly lost its market share in key technologies. [DoD programs held 17% of microcircuit market share in 1977, <0.1% in 2002 (DMSMS 2002).] Changes in the international political landscape, coupled with technological advances, led to the emergence of new threats that fielded systems, though not originally intended to address, must contend with. Recognizing that major DoD programs are susceptible to changes in both the technologies used by these systems and the emergent requirements due to new threats, increasing the long-term responsiveness in DoD acquisition approaches is critical to mitigate disruptions to these fielded systems. To date, the acquisition community s answer to these challenges has been to reduce the development cycle of large programs through approaches such as capabilities-based development and to emphasize software decoupling from hardware in Open Architecture (PEO IWS 2004). An additional approach to improving responsiveness in system capabilities is inclusion of technologies that allow for configurability and change. Technologies that facilitate changes in functionality at runtime (vice compile time) allow new development to be more easily supported throughout the lifecycle of long-fielded systems. This paper explores the application of Rules Engine technology as it can be applied to a common application found in most tactical systems (decision automation). Because of initiatives such as reduced staffing (CDRL 2007), as well as increased information integration and system complexity, the Decision Automation domain is an increasingly critical one for combat systems development. CHANGE-TOLERANT TECH- NOLOGY APPLICATION RESEARCH In FY07, the Air and Missile Defense Department of The Johns Hopkins University Applied Physics Laboratory conducted an Internal Research and Development (IRAD) project that explored the application of change-tolerant technologies in combat systems. Change-tolerant technologies are designed to accommodate volatility introduced into systems by changes in requirements. Many changetolerant technologies have been developed by or in conjunction with commercial industries as mechanisms to quickly adapt products for meeting

2 new demands, which is essential for a company to remain competitive. Examples include COM (Microsoft), and various open standards technologies (XML and SOAP/web services) which are then implemented in core technologies in companies like Sun, Microsoft, and Google. Open source projects (Python, Ruby) are also significant sources of change-tolerant technologies. Forms of change-tolerant technologies include data models, extensibility, scripting, and scalability. These features allow for a higher tolerance of change because theyintroduce plasticity into a system s architecture in a manner that does not require the mechanics of a traditional software build process. This reduces the effort needed to implement a requirement post-build and reduces the overall lifecycle cost and development time for a system. Different technologies have been developed to address different aspects of system change (Table 1). TABLE 1 Examples of Change and Corresponding Adaptive Technology Forms of Change Data Functionality Integration Controls Adaptive Technology XML, HDF Python, Ruby, Pearl, VisualBasic CORBA, COM, Service-oriented Architecture Rules Engine The problem space selected to explore changetolerant technologies was Decision Automation. Several domains were evaluated as candidates for exploration based on a set of properties that predict reuse potential. These predictors include abstraction (can the problem or solution be generalized?), granularity (how complex are the functionalities and dependencies?), and coherence (are the problem or solution boundaries well-defined?). Filter, Track Manager, and Decision Automation were the domains initially considered, and Decision Automation ranked the highest in terms of abstraction, granularity, and coherence (Figure 1). This high potential for reuse was a strong motivator for selection because of the broad applicability of the lessons learned to other Decision Automation applications. availability, decision criteria, and desired responses can change dramatically during the life of a longfielded system, investigating how different technological solutions can help absorb the impact of requirements volatility on the system s lifecycle will hopefully yield techniques and approaches that can improve the DoD acquisition process. Options Filter Why Decision Automation A: Application can be limited by sensor/weapon C: Functionality easily defined G: Solution is highly granular Track Manager Optimal Reuse A: Functionality can be generally defined C: Problem space hard to definitively bound G: Implementation is complex Decision Automation A A: Doctrine application is consistent C: Problem space can be tightly bound G: Implementation has medium to high granularity Abstraction Coherence Granularity Suitability A C G FIGURE 1 Evaluation of Problem Space Domain Candidates Decision Automation A C C G G Selected Domain In its simplest form, Decision Automation allows the a priori coupling of an action with a condition or policy. Therefore, when the specified conditions or policies are met, the prescribed actions would then automatically occur without any kind of intervention required by the user. Decision Automation has many benefits, including quicker system reaction, increased responsiveness to system environment, and the ability to process increasingly complex information. The technology can also be implemented in a variety of ways in terms of architecture, interface, and data model. Figure 2 illustrates the different types of combat system Decision Automation. Additionally, the Decision Automation problem space would benefit from the features offered by change-tolerant technologies. Because information

3 Combat System Decision Automation Examples Engagement Track Management Resource Management Weapons Selection Engage Identification Drop Track Discrimination Classification Cued Acquisition Search Sector FIGURE 2 Examples of Combat System Decision Automation One specific area where Decision Automation is used in combat systems is in support of a system s engagement process. Traditionally, the decision to engage is made in real time by an operator (who is likely following well-established procedures, and often, direction, from another individual higher up in the chain of command). Although Decision Automation is not a perfect surrogate for human judgment, automation can help streamline the engagement process in a way that improves reaction time but still includes final approval by a human in the loop. In limited cases where a person might not be able to process the situation quickly enough, a completely automated process can react within the time constraint. The mechanism that current combat systems use to accomplish operator-managed partial or full automation is referred to as doctrine. Doctrine in this context is a set of conditions with values that can typically be changed by the operator based on the situational requirements. If a target object within the system qualifies based on the values of the conditions, a pre-specified action is taken. This action can impact anything from database management, to identification, to weapons employment. Most modern weapons systems have the capability to partially or fully automate the engagement process via weapons or engagement doctrine. Changes in threat space, sensors, weapons, and availability of different information sources can cause significant alterations in requirements, which in turn create a challenge for combat system stability. Rules Engine Technology The IRAD project selected a Rules Engine technology as a possible solution to the functional and development challenges presented by the Decision Automation problem space. Rules Engine technologies are software systems that originated in commercial business systems. A rule in the business or enterprise context is guidance that there is an obligation concerning conduct, action, practice, or procedurewithin a particular activity or sphere. From an information systems perspective, a rule pertains to the facts of the system that are recorded as data and to the constraints on changes to the values of the facts. ( The rules are managed within a Rules Engine, which is a system in which behaviors are specified in the form of if/then statements that in turn operate on datasets. Because the language specifying the rules is restricted to only statements of this form, the prescribed behaviors governed by the rules have the potential to be verifiable as consistent (and complete). Rules Engines are one form of an expert system, which is a class of knowledge-based software systems originally developed by artificial intelligence researchers but are now more broadly used. The term Rules Engine used in this paper refers to a software component implementing an inference engine. Note that in Rules Engine parlance, firing is defined as when the logic of a rule is exercised. The inference engine (mechanism) is that part of the expert system kernel which supports reasoning about the environment by proper manipulation of its rule and fact bases. It establishes the current state of the environment from its fact base and uses that state information to identify the set of rules whose conditional parts are satisfied by the environment s state. It determines which rules in the rule base are possible candidates for firing based on the circumstance that the conditional part of the rules are satisfied by facts in the fact base. These facts provide an up to date picture of the environment for the expert system. (Pomykalski, Truszkowsi, and Brown 1999) Rules Engines have been widely deployed to automate complex business logic by separating welldefined portions of business logic from frequently updated details of the application.

4 In most Rules Engine implementations, rules are specified in a language that must be interpreted at runtime by the software that is executing them. Runtime interpretation requires that the system information and interfaces be explicitly defined and exported to the interpreting environment. The separation is useful for enforcing system boundaries and for protecting against catastrophic failures due to faulty interpreted code. A Rules Engine implementation called Drools ( was selected for exploring combat system applications. Drools is an open source Rules Engine written in Java that uses the standard Rete algorithm (Forgy 1982) to evaluate the rules. The rule architecture captures the logic in a text file using XML syntax, making the rule definition separate from the inference engine itself. Drools is created and supported by an open source community and is provided freely to use and distribute under the Apache license ( 2.0.html). Prototyping Approach To explore the benefits of using a Rules Engine in a combat system decision automation context, a prototyping approach was used. Prototyping was selected because many of the research themes (reuse, improvement in decision automation) could best be explored through actual implementation and testing of the technology. By taking an empirical approach, the effort was able to collect lessons learned on how difficult the technology was to work with, the extent of integration complexities, whether it could support the functionality demanded by the combat systems, whether there were software language limitations, and whether the technology could meet the requirements for individual combat system decision automation. Two combat systems were addressed in the prototyping: Aegis and Ship Self-Defense System (SSDS). The IRAD effort had access to tactical SSDS code (Mk 2, v5.04.3) and used that for the SSDS application, but used a high-fidelity model of Aegis (Baseline 7.1) to test the application for that combat system. In each prototype, weapons doctrine was incorporated into the architecture via the Rules Engine component. The same software component was interfaced to both SSDS and the Aegis highfidelity model. The rules repository, however, was different, reflecting the specific doctrine used by the given system. Both prototypes implemented weapons doctrine because of the importance of automation in improving reaction time. Note that SSDS and Aegis both define weapons doctrine and integrate it uniquely, so by implementing the same doctrine type in two different combat systems, the prototyping was not redundant but instead gave the opportunity to explore the flexibility of the system for broader application across combat systems in general. Figure 3 provides a high-level view of the prototype architecture and the interactions between the Rules Engine and other combat system components. Event Doctrine Configuration Request Doctrine Manager Operator Combat System Component Action Rules Engine Combat System Responder Doctrine (Rules) Repository Administrator Doctrine Doctrine Repository Repository FIGURE 3 Prototype Architecture Generates an event that prompts the application of doctrine to some data set. Data sent to process against doctrine statements that exist In the Rules Engine as rules. Depending on the result, a Notification might be sent to Another combat system Component for action Receives notification generated by doctrine processing Managing component for Doctrine logic. Allows for the creation, editing and deletion of automated doctrine/rules.. Metadata about the doctrine operational context, the data such as fields ( and formats and special default parameters Doctrine logic captured as Rules The Aegis and SSDS prototypes were evaluated based on whether functional equivalency to the tactical systems could be achieved. Functional equivalency indicates that processing the same inputs provided to a tactical system result in the same outputs. Functional equivalency was used as the first standard to assess suitability of the Rules Engine technology to the specific domain of Decision Automation in the combat system context. Without first establishing confidence that the Rules Engine technology can produce the same results as the existing systems, further research would be

5 premature. Real-time evaluation, which takes into consideration constraints such as throughput and latency, was not conducted during this study. After being developed, the prototypes were used to process track state data from recorded live-fire events. The Rules Engine in each prototype was configured to duplicate the weapons doctrine used in the respective combat system s events. The prototype output was then compared with the result found within the tactical data. For the Aegis and SSDS prototypes, the same track became engaged on the same update as it did in the actual tactical system. The results of the prototype testing supported the conclusion that functional equivalency can be atained for existing combat systems decision automation. IMPLICATIONS AND FUTURE WORK Because change-tolerant technologies can allow functional development but short circuit the traditional software development cycle by not requiring source code changing and software rebuilding, the authors recognize that use of these types of technologies also has an impact on the systems engineering process. This paper also identifies significant issues that are introduced to the DoD Systems Engineering process through application of these technologies. Discussion of System Engineering Implications As depicted in Figure 4, the Rules Engine implementation, and change-tolerant technologies in general, provides a significant opportunity for establishment of a much abbreviated evolution process to functions in which change-tolerant technology is employed. In addition to rapid adaptation allowed by field-changeable parameters, and the long-term response of the acquisition development process, change-tolerant components introduce an opportunity for making adjustments to system behavior that may not require the full acquisition development process. Conventional Feedback to Lifecycle Evolution Concept System Requirements Operation Operational Verification System Verification Alternative Evolution Path Field Parameter Adjustment Modification of Change-tolerant Components Requirements Subsystem Requirements Component Requirements Subsystem Verification Component Verification Appropriate testing selected based on change Build: Code and Test FIGURE 4 Modified Systems Engineering Process

6 This process is illustrated here with an example use case from the rule engine implementation of decision automation: A new and difficult threat is discovered that exhibits a characteristic kinematic and RF profile. Engineers are able to define new automation logic that will review threat profiles, identify this specific threat case, and provide operator recommendations for the most effective weapons employment strategy. The rules are tested against recorded datasets to assure that the rules function as intended, and delivered to the fielded combat systems for employment. This alternative evolution path has some significant advantages. From an operational perspective, the approach offers an approach for greater responsiveness to changing threats, environments, and mission needs. With the prospect of much more immediate impact, it might also foster more direct involvement of the operational community involvement in that aspect of product evolution. Close association between the ultimate customer/user and the organization evolving the product is always of benefit. From an acquisition perspective, a new role has been created: the modifier of the change-tolerant components. The organization responsible for evolving these components could be either an industry partner or a Navy laboratory. In either case, the organization needs only to have deep domain understanding and knowledge of the specific change-tolerant technology (e.g., Rules Engine). Commercial off-the-shelf (COTS) and open source change-tolerant components excel in providing opportunity for competition by incurring low entry cost by the participants. In the Rules Engine example, the task of the rule developer should be constrained by a well-defined set of information: the information set that the Rules Engine may access, the actions it is allowed to initiate, and the Rules Engine itself. The rule developer can focus on the problems of the domain, the creation of rules that wil automate the system s best response, and methods to rapidly achieve testing and certification requirements. This new role seems to fit well with current Navy initiatives (Guertin 2007) to continue to pursue surface Navy Open Architecture as a method to expedite the fielding of improved capability and to incorporate more opportunity for innovative newcomers in the development of combat systems. Inthis paper srules Engine example of changetolerant technology, the use of an open source COTS product as the foundation, coupled with the isolation of rule development from the complexities of the system construction, provides an excellent opportunity to establish a stream of creative solutions from a much broader community. Although the time to effect significant change can be reduced and the potential contributor base broadened, the issue of appropriate testing is still quite significant. Testing and certification add substantial cost to the acquisition process and are just as relevant to changes introduced by interpretive technologies as they are to the mainstream lifecycle build. A key aspect of a change-tolerant technology in combat system applications will be the degree to which the system design can establish isolation of the alterable behaviors. The ideal would be that testing would only need to address the system functionality undergoing improvement, with high confidence that other aspects of system functionality would be unaffected. Future Work The authors initial investigation of change-tolerant technology is both motivating and promising. However, there are many issues that would need to be considered more fully to establish viable and fruitful application. These include: Impact on testing and certification It is imperative that any approach with an objective of expediting system change address the requirements for testing and certification (especially if the approach to doing so is to shortcut portions of the development cycle). Two areas of investigation seem of great interest in this regard: functional isolation and analytic evaluation. Functional isolation was discussed previously and, at least in the Rules Engine case, seems to have some potential for natural bounds through the constraints placed on a given rule engine s posible actions (perhaps inputs as well). Analytic evaluation recognizes that some change-tolerant technologies have functionality expressed in a rigorous form that can be

7 analyzed to guarantee the presence or absence of behavior. Additionally, support capabilities in the form of targeted testing apparatus and datasets could add consistency and rigor to the testing and certification process. Real-time characteristics Many of the changetolerant technologies rely on interpretive techniques that can have poor or unpredictable real-time performance.the authors research to date has not examined this broad issue or the real-time characteristics of Rules Engines as a class. A simple benchmark run on the rules engine used in the study suggests that response times might be suitable for many functions if design techniques can constrain the number of rules. Figure 5 shows a plot of the benchmark data. It exhibits basically a linear relationship between the number of rules and the response time for a rule to fire. These results are preliminary, have not been optimized in any manner, etc., but were included to suggest that although there is some promise, there is also the need to explore this aspect of the approach in greater detail. ms per event Rules Activated FIGURE 5 Minimal Benchmarking of Rules Engine Performance Exploration of other change-tolerant technologies This IRAD activity addressed only one form of change-tolerant technology, a rule engine, and its application to one particular functional need decision automation. Before attempting to generalize too much on strategies and benefits that might be realized, it would be prudent to examine other forms of changetolerant technologies and their application to combat systems. There are very likely significant lessons to be learned before generalizing to the whole class of changetolerant technology. CONCLUSION The authors experimentation in applying Rules Engine technology to combat system decision automation demonstrated functional equivalency for two major combat systems, thus supporting the possibility that Rules Engine technology could be successfully used in sophisticated combat systems without degrading existing functionality. Because Rules Engine technology is built to accept new logic and behavior during runtime, new functionality could be added with significantly less disruption than making source code changes and having to rebuild software. Using change-tolerant technologies like Rules Engine to introduce new system capability can therefore be supported by a more flexible systems engineering process because a practical distinction can now exist between new systems development and development introducing changes to existing systems. Questions about realtime performance of Rules Engine technology and how the acquisition community would actually implement the modified systems engineering process remain open and are subject to future investigation. REFERENCES CDRL B, Manning Analysis Report (MAR) for AEGIS Modernization (AMOD) Commercial Off the Shelf (COTS) Refresh 3 (CR3), Revision 1, September 28, 2007 DMSMS Tutorial, DMSMS Conference 2002, New Orleans, Louisiana, March 25, 2002 Forgy, C. L., Rete: A Fast Algorithm for the Many Patern/Many Object Patern Match Problem, Artificial Intelligence, V19, pp 17 37, Guertin, N., Adopting Open Architecture Practices in Military Systems, AFEI Conference, December 11, 2007 PEO IWS, Open Architecture (OA) Computing Environment Design Guidance V1.0, August23, 2004 Pomykalski, J. J., W. F. Truszkowsi, and D. E. Brown, Expert Systems, Wiley Encyclopedia

8 of Electrical and Electronics Engineering, December 1999 Catherine L. Payne is a Sr. Software Engineer with a background in Command and Decision at The Johns Hopkins University Applied Physics Laboratory (JHU/APL). She is the point of contact for this paper ( ). Mark E. Schmid is Supervisor of the Force Level Computing and Systems Development Group at JHU/APL. He has over 25 years of experience supporting the development and transition of new capabilities into Surface Navy combat systems. Barbara A. Shapter is a project manager for the Enterprise Knowledge Systems Group at JHU/APL. She has over 15 years of experience developing software for models and capabilities in support of Navy systems. Brian T. Taylor is a Software Engineer at JHU/APL. He is a recent graduate of LeTourneau University with a strong interest in the design and development of extensible software solutions to Navy combat system challenges.