Human Performance Issues in Remotely Piloted Aircra6 Systems

Human Performance Issues in Remotely Piloted Aircra6 Systems Jay Shively Human Systems Integra4on, Project Engineer NASA- UAS Integra4on into the United States Na4onal Airspace (NAS) ICAO: Remotely piloted or piloted: sharing one aerospace system March 23 25, 2015

Legacy from Manned AviaHon Good Safety Professionalism Bad Slow, reachve regulatory process Flying economy High entry price Ugly High Human Error rates!! 2

Human Performance In RPAS AviaHon accident stats Although percentages vary, most would agree that somewhere between 60-80% of aviahon accidents are due, at least in part, to human error (Shappell & Wiegmann, 1996). RPAS accident stats About 100 Predator Accidents from 1997 2005 ; 51% mishaps involving crewmember acts (Tvaryanas, 2005) Tvaryanas (2004) on mishap epidemiology states that Human System Interface (HSI) issues are discussed in 89% of the Predator accidents and are cited as a causal factor in 44% of those accidents. Mishaps in the first few years were o6en abributed to equipment failures, while mishaps in the last three years were predominantly operator error (Nullmyer, et al, 2007) As drone use has taken off, human error remains a leading cause of unmanned aerial vehicle crashes. (Popular Science, 2013). Why? How can we do beber?

Current UAV Operator Interface Issues Lengthy process to handoff vehicle control Multiple screens require significant mental integration to obtain mission/ battlespace awareness Too many inputs required to implement commands (e.g., 22 key strokes to turn on the auto pilot ) Poor ergonomics Highly loaded visual channel Add-on systems (i.e., Falconview) provide needed functionality but cannot interface directly with core GCS Varying methods to input data Limited alert cues to warnings where messages can be hidden Visualization is poor and perpetuates mistakes Lack of system feedback regarding task completion Narrow visual field of view Extra workspace required Multiple keyboard/input devices required Multiple separate comm devices Numerous alphanumeric status displays Non-intuitive multilayered menus No decision aiding /support technology Cleared for Public Release 88ABW- 2012-5552 4 4

Example 5

Courtesy of Former NASA UAS Pilot 6

How can we replace the informahon? 7

Is it Pilot Error? A lible background: April 26, 2006 (NTSB, 2006) Predator B with two side- by- side consoles Power lever operates in two modes Events: Sensor controls Iris, middle posihon locks Pilot engine control, middle posihon shuts off fuel Pilot s console locked- up Pilot took over sensor console Did NOT perform transfer checklist Power lever was in middle posihon (now fuel shut off) Warning tone (but all the same; thought to be temp lost link) 3:50 a.m., A/C impacts the ground near Nogales, AZ 8

UAS Pilot Quotes This thing s kind of climbing like a pig. Climb, you pig.... Boy, this is going to be <ght.... Okay, interes<ng. We are falling out of the sky. UnidenHfied pilot of a Predator that crashed near Creech Air Force Base in Nevada on May 13, 2013. Drone just pitched up. Drone s pitching over. Drone is uh, crashed and destructed, at uh, the end of the runway. UnidenHfied pilot of a chase plane that was following a QF- 4E target drone before it crashed at Tyndall Air Force Base in Florida on July 17, 2013. We re in the soup here.... Dude, uh, we re not sure what the aircrae is doing.... Yeah, we crashed. UnidenHfied pilot of an Air Force Reaper as it crashed in Douglas County, Nev., on Dec. 5, 2012. Whoa.... I don t know what the hell just happened. Where the hell is where is the runway? It s all the way over here. I overshot. Oh, shit. I think we lost the engine. Oh, shit, oh damn, oh my God, what is that?... What was all that stuff I just hit? Air Force Capt. Mabhew Scardaci as his Predator crashed into a row of empty shipping containers at Kandahar Air Base on May 5, 2011. I couldn t tell which way it was turning, or if it was straight, if it was upside down, or if it was right- side up.... I couldn t grasp what was happening with the aircrae. And he said he thought it was upside down. UnidenHfied Predator pilot to inveshgators trying to determine how and why she flew the drone upside down before it crashed near Kandahar Air Base on Jan. 15, 2010. Um, I guess I ll just be blunt and say not well. The Predator pilot who flew upside down, when asked by inveshgators how well her training had prepared her for such an incident. Whitlock (2014) Washington Post. 9

A VERY ParHal List of Issues Pilot selechon/ cerhficahon/ training GCS database compahbility Lack of GCS Standards Pre- flight planning Lack of sensory cues GCS Human Factors Best PracHces Clumsy automahon Detect and Avoid Well Clear Collision Avoidance Lost Link Visual clearances Surface operahons 10

How can the industry reduce human performance errors? Introduce Human Factors early on in a human- centered design process Design to let people perform the tasks where they are superior Novel condihons Pabern recognihon Decision making Let Computers do what they are good at Number crunching Vigilance IntegraHng diverse sources and large amounts of informahon Don t automate with le6over principle Design robust, flexible automahon that is mission/scenario driven 11

A Few Current Efforts NASA UAS IntegraHon into the NAS Human Systems Displays, alerts, Ground Control StaHon CommunicaHons Detect and Avoid Department of Defense Air Force Research Laboratory Airborne Sense and Avoid Army Ground Based Sense and Avoid FAA Technical Center Minimum InformaHon displays 12

UAS IntegraHon in the NAS OrganizaHonal Structure Host Center Program Office Project Support Lead Resource Analyst Cindy Brandvig - AFRC Lead Procurement Officer R. Toberman - AFRC Lead Scheduler John Percy - AFRC Mgmt Support Specialist Jamie Turner - AFRC Administra4ve Support Giovanna Bowen - AFRC Bus. Sys. Coordinator Stacey Mulligan - AFRC AFRC Director of Programs Dennis Hines Deputy Director: Joel Sitz Project Office ISRP Program Director Dr. Ed Waggoner Deputy PD: Cathy Bahm Project Manager - Laurie Grindle - AFRC Deputy Project Manager Robert Sakahara AFRC Deputy Project Manager, Integra4on Davis Hackenberg - AFRC Chief Systems Engineer Debra Randall AFRC Staff Systems Engineer Dan Roth - AFRC ExCom, RTCA Steering CommiSee, UAS Avia<on Rulemaking CommiSee External Interfaces FAA, DoD, RTCA SC- 228, Industry, etc. Senior Advisor: VACANT DPMf AFRC Heather Maliska DPMf ARC Ma_ Knudson DPMf GRC Amy Jankovsky DPMf LaRC Vince Schultz AFRC ARD ARC ARD GRC ARD LaRC ARD Subprojects/Technical Challenges (TC) TC- SAA: SAA Performance Standards Separa4on Assurance/ Sense and Avoid Interoperability (SSI) Co- PEs Confesor San4ago - ARC Maria Consiglio - LaRC TC- C2: C2 Performance Standards Communica4ons PE Jim Griner - GRC TC- HSI: Human Systems Integra4on (HSI) HSI PE Jay Shively - ARC TC- ITE: Integrated Test and Evalua4on (IT&E) IT&E Co- PEs Sam Kim - AFRC Jim Murphy - ARC Cer4fica4on PE Kelly Hayhurst LaRC PE: Project Engineer, DPMf: Deputy Project Manager for 13

Subproject Technical Challenge Alignment Airspace Integra4on Validate technologies and procedures for unmanned aircra6 systems to remain an appropriate distance from other aircra6, and to safely and rouhnely interoperate with NAS and NextGen Air Traffic Services Standards/Regulations! Validate minimum system and operational performance standards and certification requirements and procedures for unmanned aircraft systems to safely operate in the NAS" Relevant Test Environment! Develop an adaptable, scalable, and schedulable relevant test environment for validating concepts and technologies for unmanned aircraft systems to safely operate in the NAS" CommunicaHons PE Jim Griner - GRC SeparaHon Assurance/Sense and Avoid Interoperability (SSI) Co- PEs Eric Mueller - ARC Maria Consiglio - LaRC Human Systems IntegraHon (HSI) PE Jay Shively - ARC CerHficaHon PE Kelly Hayhurst - LaRC Integrated Test and EvaluaHon Co- PEs Jim Murphy - ARC Sam Kim - DFRC PE Project Engineer" 14

Stakeholders Partnerships and CollaboraHons Aviation Safety Program" Airspace Systems Program! Science Mission Directorate! Other Government Organizations and FFRDCs! UAS Integration in the NAS Project! Academia! Foreign Organizations! Standards Organizations! Industry! 15

FAA Test Site Visits In order to properly understand the skills and core capabilihes of the FAA Test Sites the project visited all six FAA test sites At each FAA Test Site there was a series of briefing exchanges that generally included: FAA Test Site briefing UAS- NAS project overview briefing UAS Traffic Management (UTM) overview briefing Tours of the Test Site and FaciliHes The Project created a FAA Test Site Catalog for each locahon Path Forward: Project will conhnue an open dialog with the FAA Test Sites for potenhal collaborahon opportunihes (i.e. Capstone, CerHficaHon, and suas on- ramps) 16

UAS Integra4on in the NAS Project Value Proposi<on Flow Diagram NASA UAS- NAS Project Ac.vi.es Key Products Resultant Outcomes TC! SAA! SAA Performance Standards Develop SAA Performance Testbed Develop SAA Interoperability Testbed Conduct SAA Flight Test and MS&A Performance Trade- offs CONOPs Interoperability Well Clear Self Separa<on Collision Avoidance Develop SAA Performance & Interoperability Requirements SAA Performance Requirements to inform DAA MOPS DAA MOPS SAA Technical Standard Order (TSO) TC! C2! C2 Performance Standards Develop C2 Prototype System Conduct C2 Flight Test and MS&A Data Link CNPC Spectrum CNPC Security LOS BLOS ATC Interoperability Develop C2 Requirements C2 Performance Requirements to inform C2 MOPS C2 MOPS C2 Technical Standard Order (TSO) TC! HSI! Human Systems Integra.on Develop Prototype GCS Conduct Human Factors (HF) Flight Test and MS&A Con<ngency Management Pilot Response Autonomy SAA C2 Displays Develop HF Guidelines for SAA, C2 & GCS HF Performance Requirements to inform DAA & C2 MOPs, HF Guidelines TC! ITE! Integrated Test & Evalua.on Develop LVC Test Infrastructure Conduct TC Specific Tes4ng Re- usable Test Infrastructure Conduct IHITL Conduct SAA Ini4al Flight Test Scenarios Conduct FT3 Test Scenarios Conduct FT4 Test Scenarios & Capstone 17

HSI Subproject! Efficiently manage conhngency operahons w/o disruphon of the NAS Coordinate with ATC - respond w/o increase to ATC workload Ensure operator knowledge of complex airspace and rules Seamlessly interact with SSI Research test- bed and database to provide data and proof of concept for GCS opera4ons in the NAS Standard aeronauhcal database for compahbility Human factors guidelines for GCS operahon in the NAS Traffic informahon for situahon awareness and separahon (NextGen) 18

QuesHons? NASA s UAS cockpit situa<on display with TCAS advisories