The Tactical Tomahawk Weapons Control System: Interface Design Project

Transcription

1 The Tactical Tomahawk Weapons Control System: Interface Design Project Student Team: Julie Besselman, Meredith Logsdon, Brian Whisnant, Timothy Yewcic Faculty Advisor: Dr. Stephanie Guerlain, Department of Systems and Information Engineering Graduate Student Advisor: Robert Willis, M.S. Student, Department of Systems and Information Engineering Client Advisors: Bruce Copeland, Wayne Harman, Bob Athay, and Alan Thomas Naval Surface Warfare Center Dahlgren, VA KEYWORDS: Cruise missile, interface design, situational awareness, cognitive task analysis, and usability testing. ABSTRACT Scheduled for deployment in 2003, the Tactical Tomahawk cruise missile represents a qualitative leap forward in long-range smart weapon technology. The weapons control officer will be able to retarget these missiles during flight by directing the missiles to strike time-critical, emergent targets within minutes of their appearance. Other new capabilities include battle damage assessment and the capacity to plan missions onboard the launching ship or submarine, further increasing the flexibility of this new weapons system. The objective of our Capstone team was to design a prototype of the computer interface that an operator of such a system would use to monitor and control the cruise missiles. We conducted extensive research into situational awareness and interface design techniques as we formulated our list of functional requirements and developed our interface. Based on our design, we created an interactive computer program that we used to conduct usability testing on 20 subjects from a systems engineering graduate class. The program simulates a battle situation, and the subjects interacted with the interface to gather information about what was happening and to make decisions based on that information. We modified several variables during each hourlong test, including changing the total number of missiles and targets on the screen at once as well as turning on and off a feature that graphically displayed the range of each missile. Though the focus of our project was the development of the interface and the design of our usability test, we were able to draw some valuable conclusions from our test data. Our work is intended to be a first iteration, and the test results should be used to make improvements to both the interface and the testing method itself. INTRODUCTION The goal of this Capstone project was to develop and test an interactive prototype of a weapon control system interface for the new Tactical Tomahawk cruise missile. Unlike the current missiles, which operators program before launch, the next generation Tactical Tomahawk will have retargeting capability, giving it the ability to loiter in enemy territory and then quickly strike an emergent target wherever it may appear. The implementation of this tactical retargeting feature will not be complete until the interface for operating its control system has been established, designed, and built. The Navy has contracted Lockheed Martin to build the software applications that will connect the missiles to the control system guidance system, and other subsystems (Willis, 2000). As a separate venture, the task of this Capstone team was to determine how to best design a weapons control system interface for the Tactical Tomahawk cruise missile. The goals of this project were to: Design, develop and test interface concepts for the US Navy s Tactical Tomahawk cruise missile operator interface. Provide specific display recommendations to the Naval Surface Warfare Center Dahlgren Division (the client) for use in future contracted TTWCS user-interface development.

2 Recommend the appropriate level of automation to minimize mission execution time while maximizing operator situational awareness BACKGROUND The Tomahawk cruise missile saw action for the first time in 1991 in Operation Desert Storm. It immediately proved itself as an effective tool for precise, deep strike missions and has since seen action in regional conflicts in Afghanistan, Sudan, and Serbia (Sanders, 2000; U.S. Navy, 2000). Fig. 1 A Tomahawk in flight ( Current Tomahawk missiles are programmed and launched with a predetermined flight path and target. The missile can be launched from either a ship or a submarine and has a range of over 800 nautical miles ( U.S. Navy, 2000). Once over land, its guidance system computer compares the surrounding terrain with the maps in its memory to keep the missile on the correct course (See Fig. 1). After launch, the operator cannot control the missile and only receives information via the Tomahawk In-flight Position Reporting System (ORD, 1998). The new Tactical Tomahawk Weapons Control System (TTWCS), which is not yet operational, will improve upon this successful design by adding some powerful capabilities. The new missile will have the ability to be retargeted while in flight, making it capable of striking an emergent target of opportunity that may not even appear until the missile is actually airborne. The operator will be able to keep missiles in loiter patterns over enemy territory until emergent targets appear, giving the Navy the power to strike an emergent target within minutes of its initial appearance. It will be possible for the operator to plan missions from the launch platform. Presently, mission plans are created at headquarters in Norfolk, VA or Pearl Harbor, HI, and then transmitted to the ship or submarine. In addition, the new missile will periodically relay missile status details to the weapons systems operator, giving him or her the information required to effectively control the missiles in real time (Sanders, 2000). With its increased capabilities, the proposed system places much greater responsibility on the weapons systems officer controlling the missiles. The system must provide the operator with a wide range of information, including information on the position, range, and warhead of each missile as well as current and projected positions for the different types of targets. These will all be factors needed to decide which missile should be assigned to which target and when. The system creates the need for a weapons control interface that will allow the weapons officer to comprehend the complex arena of combat and make decisions quickly and effectively. The existing hardware, a workstation with a keyboard, trackball, and two 19 monitors will be used with the new missile, and we have designed our interface accordingly. Our Capstone team has done the necessary research, design, and testing to develop an interactive interface prototype that answers the question: how can a single interface be created that allows the weapons control officer to operate these missiles effectively? RELEVANT THEORY Although no standard guideline exists for interface design, methodologies for humancomputer interface design are often based on principles from several fields of study including situational awareness and cognitive science. These general principles are helpful for understanding the basis of many of the TTWCS interface design decisions. Situational Awareness Situational awareness (SA) is a term originally used only by the aircraft community to represent a pilot s ability to correctly perceive the state of his environment while utilizing flight automation tools. However, SA has become a field of study that pertains to many different systems with varying levels of automation. Situational awareness is defined as the perception of elements in a [dynamic environment], the comprehension of their meaning, and the projection of their status in the near future (Endsley, 1995). This is of

3 particular concern to automated, dynamic systems because the removal of operators from physical contact with the process may diminish the accuracy of their perceived system-state [Moray in Endsley article]. In other words, as the system automates or internalizes some of the operator s tasks, the operator loses touch with what is happening within the system. Endsley believes that at a minimum, the [interface] design process should include steps to ensure that needed information is always present regarding the state of automation and the state of the parameters being monitored in a clear, easily interpreted format. It is important to note that the automation of some tasks can benefit system performance by decreasing the mental workload on the human operator. The key issue (as it pertains to interface design) is to make sensible choices regarding the allocation of functions between the human operator and computer automation. Cognitive Task Analysis Cognitive Task Analysis is a relatively new branch of applied psychology that has recently been utilized in the development of expert systems, system training, expert decision-making, and policymaking (Chipman, 2000). It is a broad area of study consisting of tools and techniques for describing the knowledge and strategies required for task performance. This is extremely pertinent to the design of this interface because new TTWCS operators will have to perform tasks (involving knowledge and strategy) not familiar to the old system. While the system must provide the functionality for performing these new tasks, it must also support/aid the cognitive or mental work that the operator will undertake while interacting with the system. The field of Cognitive Task Analysis focuses on issues such as identifying and supporting these responsibilities. APPROACH Our approach to this project had four distinct phases (see Figure 2). Domain Analysis Scenario Development Fig. 2 - Approach Phases The first phase was a Domain Analysis of three primary domains: interface design principles, the TTWCS system properties, and the larger context of air campaign command and control. Second, potential scenarios were developed to identify functional requirements. Next, interface components were developed and synthesized into a prototype. Finally, Subject Testing and Analysis were conducted. Domain Analysis Interface Design and Prototype Development Subject Testing and Analysis There are three basic types of Tactical Tomahawk missions. The class of the target to which the missile is assigned defines the missions. Target classes (and thus mission types) include default, flex, and emergent (or d-, f-, and e-targets). A missile is launched along a path initially toward its d-target. While en-route, the operator can command the missile to abort its default mission, and execute one of several preprogrammed flex missions. The operator can also respond to the appearance of an e-target by rapidly programming and transmitting a completely new route and aimpoint. As a retargetable system, TTWCS brings the cruise missile into the realm of on-call weapons available to an air campaign or ground commander for destruction of timecritical targets (Scott, 2000). In the context of the doctrinal Decide, Detect, Deliver targeting process, the TTWCS display development must therefore consider from whom a call for a fire or attack order is received, and in what format. A central assumption for this project is that an attack order from a targeting center specifies the what, where, when, while the TTWCS operator and/or system decides how (with what missile(s)). There must be a two-way communication link to provide feedback to the targeting center specifying any targets that were given up as an opportunity

4 cost for servicing an unplanned emergent target. Further, the development of the displays also addresses the likelihood of the targeting cell asking if a vessel could accomplish a certain mission. The displays must therefore facilitate rapid analysis of what-if scenarios to determine the acceptance/rejection of a possible time-critical mission. Initial products from the Domain Analysis phase include a formal list of objects, properties, and values that are inherent in the system. Objects and properties include Targets, Missiles, Airspace Constraints and Threats, Vessels, Missions, Campaigns, and BattleGroup. Scenario Development Following the object definition, a description of seven possible scenarios led to the development of a formal set of interface functional requirements. The scenarios also facilitated the identification of operator information requirements. A sample scenario was: Three inflight missiles are along separate routes prior to the branch points for a common flex target. The operator receives the command to service the flex target as well as a new emergent target that is within range of two missiles. Priorities of all targets are known, and the expected move time of the e- target is given. Select the best course of action. Other scenarios included interaction with no-fly zones, targets requiring multiple missiles, multiple simultaneous emergent targets, and targets closer to a vessel than an in-flight missile. Interface Design and Prototype Development A systems approach as described in Gibson (1991) was applied to the development of several levels of prototypes. The system was first reduced to its components, and then was built up iteratively into system prototypes of increasing fidelity. Static, non-interactive Power Point slides were used for cognitive walkthrough purposes (Kellmeyer & Osga, 2000). The rapid prototyping iterative process progressed through scripted movies, and into static yet interactive slides to demonstrate basic system display functions. A design document was submitted to several professional graphic designers and multimedia producers, and Creative Perspectives (Charlottesville, Virginia) was selected to code a dynamic and interactive Version 1 prototype suitable for subject testing in the near-term, as well as a Version 2 incorporating subject testing results and a more deliberate graphic design effort. Macromedia Director 8 was selected as the prototyping software. Components The previously defined objects and functional requirements were combined to create component prototypes. Initially, prototype concepts were developed that addressed component processes of the total system. The most basic types of components are the symbols used to represent the individual missiles and targets. To the greatest extent possible, the symbol development is in accordance with Department of Defense common interface symbology (Mil Std-2525B, 1999). Examples of several target and missile symbols are shown in Figure 3. Fig. 3 - Example component prototypes: icons Requirements for subsequent components were made clear in the analysis of other scenarios and their functional requirements. For example, the previously described scenario requires an interpretation of the missiles with respect to time, and requires the operator to rapidly evaluate the problem, reduce and compare options, and make a decision. The missile-time component prototype shown in Figure 4 maps object properties into a graphical feature for a missile assigned to an emergent target. This missile-time component would be multiplied for each candidate missile. (See Figure 4) Timebar for Missile #20 Launched about 11:35 Current time Straight line time to impact Fig. 4 - Missile Timebars Missile will run out of fuel about 1330 (1:30 pm) Scheduled time of impact

5 Coverage Zones are a third key component in the prototype. Coverage zones are missile-time-space representations that simply show a radius of action on the map for each missile in a specified time factor. The time factor is selected by the operator, and can vary at 30, 20, and 10 minutes, as well as any specified numerical value. Other interface components include missile routes, missiletarget assignments, airspace control measures, and threats. System Prototype While the design document describes a twomonitor interface consisting of a predominantly text data display separate from a predominantly graphical situation display, the Version 1 Director prototype which was complete for our usability test consists of one monitor with a small message window, as shown in Figure 5. The prototype architecture enables the programming of new scenarios (number of missiles & targets, waypoints, messages, etc.), and continuously updates all the geometric, trigonometric, and physics expressions inherent in the system, with the exception of missile altitude and dive angle. Interface settings Tactical Display Coverage Zone Missile Timebar Display Message Display Area Launching submarine Default Target Fig. 5 Dynamic and interactive prototype screenshot

6 USABILITY TESTING The final phase of the project was to conduct usability testing of the interface design. The subjects were systems engineering graduate students, and received two 75-minute training sessions to familiarize themselves with the system. During testing, the subjects monitored and redirected the airborne missiles, responding to message requests and changes in the tactical environment. We varied two major aspects of the scenario with different subjects: the number of objects on the screen at once (missiles and targets) and the presence or absence of coverage zones (a circle that was attached to the onscreen missile as it flew, representing the area that it could cover in a given period of time). When an emergent target appeared, we also varied the number of missiles that were candidates for attacking the new target. Each scenario had either 10 or 20 objects, and only half included the coverage zone feature. A spilt plot experimental design was followed, such that different combinations of these variables were assembled and given to each subject in such a way that statistical conclusions could be drawn from the results. The hypotheses for the tests were that response time and accuracy would suffer when more objects were present, and situational awareness would increase with the presence of the coverage zones due to the utility of the function. There were eight different possible test scenarios, each incorporating a different combination of the experimental variables. There was a separate Multimedia Director movie for each scenario, and each subject ran through three of these movies. A testing block was approximately one hour long, including a 10-minute review period, three 12-minute scenarios (the first one was practice, though the subject did not know this) and a 15-minute post-test survey. A total of 20 subjects were tested over three days, and a test administrator and assistant were present to record answers and response times. RESULTS Objective findings A statistical evaluation was performed on the data gathered through testing. This data included all of the subjects response times for each question. Through this evaluation, it was determined that the number of objects present on the screen and the absence or presence of coverage zones were significant factors in response times for some questions and not for others. Further investigation into this finding revealed that the number of objects on the screen seemed to affect the response times for questions pertaining to the evaluation of individual objects. These questions included: Say aloud the number of targets that require or prefer more than one missile. (R 2 = , P-value = ) Say aloud the Missile ID number of soft warhead missiles hitting targets with a priority of six or more. (R 2 = , P-value = 0.008) Questions whose response times were not affected by the number of objects present on the screen tended to be questions requiring the comparison between objects or could be answered by utilizing the timebar section. These questions included: Say aloud the Missile ID number of the missile that would reach its default target last if all missiles were commanded to go directly to their default target. (R 2 = , P-value = Insignificant) Say aloud the Missile ID numbers of all the missiles that will impact before 12:30. (R 2 = , P-value = Insignificant) The fact that the number of objects is not a significant factor affecting response times for the latter questions, lends some credibility to the design of our interface. This indicates that the complex comparison of time attributes for missiles does not get significantly harder with more missiles, likely due to the design of the timebar display that allows easy comparison of time features versus missiles. The absence or presence of coverage zones during testing scenarios was found to affect the response time for only one question. This question was: Describe or show the general areas in which you could NOT service and e- target whose expected movetime is 12 minutes from now. (R 2 = , P-value < ) With the utilization of coverage zones, a subject could very accurately answer this question with a high degree of confidence. This made it clear why coverage zones significantly affect response time for this question. Subjective findings In addition to our empirical data, subjective data was gathered via comments

7 during testing and a post-test survey. Several interesting patterns emerged through these comments. One of our survey questions addressed the issue of monitoring many objects at a time. Many of the subjects felt that the scenarios with the most objects (20) were too much for them to handle, and several others believed that they could handle 20 objects with some practice but no more than that. Only a couple of the subjects predicted that they would be able to handle more than 20 objects simultaneously. These responses indicated that our choice of scenarios with 10 and 20 objects was appropriate. When answering questions concerning coverage of the target area, the subjects found the coverage zones to be useful. However, several of the subjects felt that they cluttered the screen and were more trouble than they worth. Nearly every subject wanted the ability to turn the coverage zones on and off (a feature not included in the test scenarios). CONCLUSIONS The work of our Capstone team was a continuation of a preliminary effort by Rob Willis and David Sanders during the previous year; and we intend for others to continue our work in the future. Our interface prototype was designed as a first iteration and served its purpose of demonstrating our design recommendations and allowing us to conduct extensive usability tests. However, neither our design nor the prototype itself is intended to be a final product. Our test results provide valuable information concerning the strengths and weaknesses of our interface, and they bear further examination, as they will serve as the foundation for the next iteration. Dr. Guerlain, our technical advisor, has sought funding to continue the project and maintain the partnership with the software firm for later versions of the prototype. While our effort this year comprises a complete design project by itself, there is more work to be done. REFERENCES Chipman, Susan F., Jan Schraagen, Valerie L. Shalin. Cognitive Work Analysis. New Jersey: Lawrence Erlbaum Associates, Inc., Department of Defense. Interface Standard. MIL-STD-2525B. 15 December Endsley, Mica R., Esin O. Kiris, The Out-ofthe-Loop Performance Problem and Level of Control in Automation, Human Factors June 1995: Gibson, J. (1991). How to do systems analysis. Charlottesville, VA. PS Publishing. Kellmeyer, D. & Osga, G.A. (2000). Usability testing & analysis of advanced multimodal watchstation functions. Proceedings of the Human Factors and Ergonomics Society 44 th Annual Meeting, San Diego, California Sanders, D. & Willis, R. (2000). Tactical tomahawk missile user interface analysis. School of Engineering and Applied Science, University of Virginia. Scott, W. B. (2000). Experimental center nails time-critical targets. Aviation Week and Space Technology. 2 Oct., United States Department of Defense. (1998). Operational Requirements Document (ORD) for Tomahawk Weapon System Baseline IV (U). Washington, D.C. The United States Navy Navy Fact File Web Site. Tomahawk Cruise Missile. Retrieved Sept. 16, 2000 from the World Wide Web: missiles/wep-toma.html Willis, Rob. Proposal to Integrate the Systems Engineering Capstone Team. University of Virginia, Charlottesville. 8 September 2000 BIOGRAPHIES Julie Besselman is a fourth-year Systems Engineering major from Pittsburgh, PA, concentrating in Management systems. Her principal contributions to the project were in defining the functional requirements of the system and in developing the experimental design and usability test plan. After graduation, Ms. Besselman will be working as a consultant for Booz-Allen & Hamilton in McLean, VA. Meredith Logsdon is a fourth-year Systems Engineering major from Reading, MA, concentrating in Management systems. Her contribution to this project included extensive research, design of prototype components, and statistical analysis of testing results. Following graduation, she will be working as an IT

8 consultant for PriceWaterhouseCoopers in Falls Church, VA. Brian Whisnant is a fourth-year Systems Engineering major from Saratoga, CA, concentrating in Management systems. His main contributions were the design of prototype components and statistical analysis of test results. Mr. Whisnant will be working as an analyst at Accenture in Reston, VA following graduation. Timothy Yewcic is a fourth-year Systems Engineering student from Pittsburgh, PA, concentrating in Management systems. His principle contributions to this project were scenario development, definition of system functional requirements, and helping with the usability test plan and administration. Mr. Yewcic will graduate in December 2001, and will seek a profession in either the finance or consulting industry.