AIAA 3rd "Unmanned Unlimited" Technical Conference, Workshop and Exhibit 20-23 September 2004, Chicago, Illinois AIAA 2004-6417 Nighttime UAV Vineyard Mission: Challenges of See-and-Avoid in the NAS Stanley R. Herwitz * UAV Applications Center, NASA Research Park, Moffett Field, CA, 94035 Karl Allmendinger UAV Collaborative, NASA Research Park, Moffett Field, CA, 94035 and Robert Slye **, Steve Dunagan **, Brad Lobitz **, Lee Johnson **, and James A. Brass NASA Ames Research Center, Moffett Field, CA, 94035 A Nighttime UAV Vineyard Mission will demonstrate the use of a UAV-based thermal infrared imaging system for improved direction of frost damage mitigation efforts in agricultural crops. The UAV selected for this April 2005 mission is the APV-3. A flight height of 8,000 ft is planned, enabling thermal mapping coverage of the largest vineyard in California on an hourly basis. To accomplish the Nighttime Mission, it is necessary to demonstrate that the ground-based autopilot has the capability to see-and-avoid potentially conflicting aircraft in the National Airspace System (NAS). This paper provides a review of a daytime UAV test flight conducted in-visual range over the vineyard in August 2003 and describes additional tests being conducted to satisfy FAA see-and-avoid requirements for the planned out-of-visual range nighttime mission. I. Introduction he UAV Applications Center is pursuing a proof-of-concept commercial UAV demonstration in partnership T with Monterra Delicato s 5,000 acre San Bernabe Vineyard south of King City in Monterey County, California. Radiative heat transfer during cold, clear nights poses a risk to California winegrape industry. Following bud break, the threat of nighttime frost damage is highest during the months of March-May and can result in a 50% decrease in annual yields. For example, the San Bernabe Vineyard, which has an annual revenue of approximately $25,000,000 from grapes alone (not including wine value), could sustain an annual revenue loss exceeding $12,000,000 from a severe frost event. The objective of the Nighttime UAV Vineyard Mission planned for April 2005 is to demonstrate how a small UAV (12 ft wingspan) equipped with a thermal imaging system can be used to reduce the risk of frost damage by optimizing frost mitigation treatments. This vineyard mission is a follow-on to the proof-of-concept coffee harvest optimization mission conducted in Hawaii over the 3,600 acre Kauai Coffee Plantation in September 2002 using the solar-powered UAV Pathfinder-Plus. 1 This paper highlights the progress made in preparing for the nighttime mission. Details are provided on the UAV platform, its payload, and the strategy being undertaken to address the see-and-avoid challenge of operating out of visual range to a flight height of 8,000 ft. * Professor of Earth Science, Clark University, Worcester, MA 01610 and Director of the UAV Applications Center, NASA Research Park, MS 18-3, Moffett Field, CA, 94035. Flight Director, UAV Collaborative, NASA Research Park, MS 18-2, Moffett Field, CA, 94035. ** Research Scientist, Earth Science Division, NASA Ames Research Center, MS 242-4, Moffett Field, CA 94035. Branch Chief, ECOSAT Branch, Earth Science Division, NASA Ames Research Center, MS 242-4, Moffett Field, CA 94035. 1 Copyright 2004 by the, Inc. All rights reserved.
II. The Challenges of Nighttime Temperature Assessment The challenges in assessing nighttime temperatures are significant because of the wide range of spatial and temporal variation in large vineyards. Low-lying areas may have a higher incidence of frost on some nights, but, on other nights, the more elevated areas may have a higher incidence of frost. Of the California grape growers that have a frost protection system, three primary methods are currently used to monitor nighttime temperatures: (i) handheld or vehicle-mounted systems; (ii) automated electronic frost alarm networks that rely on stationary ground-based point samples; and (iii) weather information from fee-based proprietary websites. (i) Handheld or vehicle-mounted systems are labor intensive. It is not possible to have a work force distributed over a 27-square mile stretch of vineyard, such as San Bernabe, throughout the night. Work fatigue at night is a compounding issue. With the sampling often restricted to road networks, the procedure often fails to sample high-risk zones in the field interiors. (ii) Stationary ground-based sensors, which comprise frost alarm networks, are limited in number and require labor-intensive maintenance. Geographic coverage is generally lacking, especially when the topography is highly variable. In addition, the sensors are vulnerable to disturbance and degradation by farm operations, wild animals (e.g., rodents) and exposure to the weather. (iii) Commercial fee-based website weather data available 24 hours a day are not specific to the geographic units for which frost control measures may be activated. These temperature readings may not readily provide the broader temporal trend analysis needed for more accurate predictions of the microclimatic variation between the vineyard s fields. At the San Bernabe Vineyard, nighttime temperature assessment currently involves manual sampling of air temperatures throughout the night using truck-mounted systems limited to the vineyard s roads. The frost mitigation treatment is an irrigation pump control system that involves discretionary water spraying over the threatened fields (Figure 1). Phase change latent heat helps stabilize the leaf temperature above the freezing point, with the ice layer acting as an insulator. Improving the timeliness and spatial coverage of nighttime temperature data could have significant economic benefits. Maps showing temperature variation within individual irrigation sections would enable the vineyard managers to better coordinate the frost mitigation effort. (a) (b) Figure 1. (a) Ripening grapes at the San Bernabe Vineyard, and (b) the irrigation pumping system. III. Plans for Nighttime UAV Vineyard Mission During the planned 6-hour nighttime mission (11pm to 5am), fixed flightlines will be flown repeatedly using a miniaturized thermal imager. The UAV will takeoff from the vineyard s centrally-located private airstrip. The thermal imagery will be downlinked to a payload ground station that will be established in the vineyard data management office. The thermal imagery will be georectified using GPS/INS information, mosaicked into a base map, and overlaid by the vineyard s irrigation section boundary map. The product will then be delivered to the vineyard manager for timely decision support that will enable more spatially-controlled, water-conserving irrigation treatments. 2
Thermal image indications of potential freezing temperatures will be used to determine when and where the irrigation treatments should be applied. With full airborne coverage of the vineyard planned on an hourly basis, a series of thermal maps showing trends and rates of change will form the basis for the key decision of timing of irrigation treatments. Pixels will integrate radiation from not only the discontinuous overstory canopy surface of the grape vine rows, but also the more continuous grass, weed and bare ground understory. Field measurements will be made throughout the UAV flight to establish the relationship between the remotely sensed and the actual ground temperatures. IV. Materials and Methods A. UAV Platform The UAV to be used for the Nighttime Mission is the APV-3 (Figure 2) built by RnR Products (Milpitas, CA) and equipped with a Piccolo autopilot system made by Cloud Cap Technology, Inc. (Hood River, OR). The APV-3 can remain aloft for up to 8 hours at 45 mph, with an altitude ceiling of 10,000 ft. The autopilot system enables the aircraft to automatically negotiate a course defined by a number of pre-specified geographic waypoints, which can be updated on-the-fly by ground personnel. Minimal runway clearance is required for takeoff and landing operations, and the aircraft can be partially disassembled for transport by small pickup truck. (a) (b) Figure 2. (a) APV-3 airframe with payload pod attached on San Bernabe runway, and (b) APV-3 on ascent over the San Bernabe Vineyard. B. Payload Recent advances in microbolometer thermal infrared camera designs are an enabling technology for this mission. The selected sensor is the Indigo Omega camera, a microbolometer array with sensitivity in the 7.5 to 13.5 micron range. This sensor is small (32 x 40 x 45 mm), lightweight (120 g), and has low power requirements (1.5 W). The imaging system is interfaced to an on-board flight computer system through a multi-channel IEEE 1394 controller. Image meta-data from an on-board GPS engine and 3-axis integrating attitude sensor are encoded into the image file headers. The payload package includes a wireless communication system designed to control the camera and downlink imagery directly to the vineyard within moments of collection. The payload package is contained in a detachable pod on the underside of the fuselage (Figure 2). Also included in the pod is a mode C transponder. V. Daytime UAV Vineyard Test Flight In the spring of 2003, APV-3 operation and payload integration tests were conducted at the Moffett Federal Airfield (MFA). During these initial test flights, the APV-3 was equipped with an MLB, Co. (Palo Alto, CA) autopilot system. The UAV team s on-site safety officer monitored the position of the APV-3 by visual observation of strobe lights on the wingtips, while the MFA Air Traffic Control tower received returns from the mode C transponder. In July 2003, a Certificate of Authorization (COA) was issued by the FAA Western-Pacific Region Air Traffic Division, enabling APV-3 flights within visual range in the National Airspace System (NAS) over the San Bernabe Vineyard. On August 20, 2003, a daytime UAV vineyard flight was conducted to test the performance of the telemetry system and some additional payload components. These components included a Bayer array (280x1024) RGB camera, and a monochromatic camera fitted to a miniature imaging spectrograph. The monochromatic camera operated in push-broom (linear array) fashion to collect high spectral resolution (580 band) data in the 0.4 to 0.9 micron region. Both of these imaging systems were interfaced to the same common data system as the thermal IR camera through the multi-channel IEEE 1394 controller. Two wireless links were used to provide reliable control of 3
the payload at low bandwidth (19.2 Kbaud at 900 MHz carrier frequency) and rapid download of image data at high bandwidth (11 Mbaud via IEEE 802.11b WLAN at 2.4 GHz carrier frequency). Imagery was collected from a flight height of 2,000 feet. A total of 165 RGB images were collected at a high spatial resolution of approximately 8 inches. Several images were registered to the grower s georeferenced base map. Post-processing was performed to separate the images into vegetation and soil components, and to calculate percent vegetation cover. Data from the imaging spectrograph were used for more detailed examination of canopy reflectance differences as related to crop vigor. 2 A wireless local area network was used to transmit data from both imaging systems to a ground receiving station in near-real-time. In preparation for the nighttime mission, a wireless network of ground-based air temperature sensors was installed for the daytime test flight. The sensors were positioned at several key locations throughout the vineyard, monitoring air temperature data every 10 sec during the flight. Temperature data were collected from the top of the canopy and from below the canopy in the fruit zone. VI. The Challenge of See-and-Avoid in the NAS At the present time, UAV operations in the National Airspace System (NAS) require a COA from the FAA, even when the UAV mission is conducted within visual range. The certification process requires that the UAV pilot can reliably see-and-avoid other air traffic in the area of operation. 3 Significant progress has been made with technologies that can provide information on potentially conflicting aircraft for UAVs operating under Instrument Flight Rules (IFR). However, the FAA requirement to see-and-avoid aircraft flying at altitudes below 12,500 feet under Visual Flight Rules (VFR), including those aircraft not equipped with transponders, continues to be a limiting factor to low altitude UAV operations. In order to proceed with the Nighttime UAV Vineyard Mission out-of-visual range under VFR conditions, we have developed and are in the process of validating the requisite see-and-avoid system. In our earlier UAV Coffee Mission in the NAS, the solar-powered transponder-equipped PF+ was treated like a conventionally piloted aircraft supervised by Honolulu air traffic controllers. 1 For the upcoming Nighttime Mission, the APV-3 will be controlled by a ground-based autopilot having the capability to view other conflicting aircraft using a portable ground-based short range radar system. The NASA Safety Review Board has required tests and verification of the system s performance and capability. This requirement is being fulfilled in three phases. In the Phase 1 test, the performance of the short range radar tracking system was measured directly at the MFA by flying an OH-58 helicopter through the radar envelope at different altitudes. Documentation for the short range radar tracker, from the manufacturer Lockheed Martin, indicates that the radar can detect targets within 28 degrees above the horizon to a range of 20 km. This range has been empirically verified in radar data log files by our monitoring of numerous outbound targets. The region of detectability below 28 degrees was verified by flying the OH-58 over the radar at 500-foot altitudinal increments up to 7,000 feet, and by recording the radar returns. Phase 2 involved the refinement of see-and-avoid training. With the radar operational at the MFA, computerbased simulations of APV-3 flights were conducted using the Cloud Cap autopilot. Displayed on the same georeferenced screen were the positions of all aircraft in the South San Francisco Bay airspace and the simulated position of the APV-3. Avoidance responses to the radar detection of air traffic in the South San Francisco Bay airspace were conducted. Carefully monitored was the time between conflicting air traffic detection and the avoidance response. The success of these Phase 2 see-and-avoid training activities has now led to the preparation for Phase 3. Phase 3 will involve the use of the OH-58 helicopter and the APV-3 in flight in the airspace at Crows Landing in Central California. The objective is to demonstrate the see-and-avoid capability of the upgraded display system with both the APV-3 and OH-58 in flight. The OH-58 will generate radar returns to simulate air traffic profiles based on ground references at a constant altitude. The avoidance waypoints for the APV-3 will be predefined, and a vertical separation of 800 feet will be maintained. The display showing the UAV position in relation to the OH-58 detected by the radar will be co-located on the same tabletop with the UAV autopilot ground station, thus enabling the UAV autopilot to evaluate and perform the recommended avoidance maneuver. VII. Conclusion Nighttime frost damage is capable of decreasing grape yields with devastating economic results to vineyard growers. A system is being developed using UAV technology to improve the timeliness and spatial coverage of temperature information for this high value crop. From the see-and-avoid perspective, the challenge of our Nighttime UAV Vineyard Mission is that it will be conducted out-of-visual-range in the NAS below 12,500 feet. Most see-and-avoid technologies are not sufficient to support our planned UAV mission. The phased series of tests 4
are being completed in order to serve as technology-proving milestones leading to our nighttime mission. It is our contention that the see-and-avoid technology that we have developed will advance the progress of UAV operations conducted at low altitudes out-of-visual-range in the NAS. Acknowledgments The authors wish to acknowledge the assistance of the following individuals: R. Higgins, D. Allen, D. Sullivan, K. Weinstock, M. Fladeland, G. Witt, R. Spicer, K. Carter, S. Morris, D. Steele and J. Chovan. The APV-3 UAV was provided by RnR Products (Milpitas, CA) and Lockheed Martin Maritime Systems and Sensors (Syracuse, NY). Thanks to C. Hoover and D. Rosenthal of Monterra Delicato s San Bernabe Vineyard for their support. References 1 Herwitz, S.R., Johnson, L.F., Dunagan, S.E., Higgins, R.G., Sullivan, D.V., Zheng, J., Lobitz, B.M., Leung, J.G., Gallmeyer, B.A., Aoyagi, M., Slye, R.E., and Brass, J., Demonstration of UAV-based Imaging for Agricultural Surveillance and Decision Support, Computers & Electronics in Agriculture, Vol. 44, 2004, pp. 49-61. 2 Johnson, L.F., Herwitz, S.R., Dunagan, S.E., Lobitz, B.M., Sullivan, D.V., and R.E. Slye, R.E., Collection of Ultra High Spatial and Spectral Resolution Image Data over California Vineyards with a Small UAV, Proceedings of the 30th International Symposium on Remote Sensing of Environment, Honolulu, Hawaii, 2004. 3 Dornheim, M.A., Flying Well with Others, Aviation Week & Space Technology, Vol. 161, No. 5, 2004, pp. 54-56. 5