Saint Louis University Rogue Squadron

Transcription

1 Saint Louis University Rogue Squadron 2015 AUVSI Student UAS Competition Abstract Saint Louis University s Rogue Squadron has created an unmanned aerial system (UAS) with three main components: an aircraft, payload, and ground control station. The aircraft is a Senior Telemaster from Hobby Lobby. This airframe has great stability, a big fuselage, and a longstanding history of being a reliable airframe. The payload consists of a gimbal, gimbal controller, two cameras, four batteries, one Raspberry Pi 2 model B, and an autopilot. The ground control station is composed of a Dell Inspiron 15 equipped with Mission Planner, Google Earth, and SimpleBGC_GUI, two remote control transmitters at 2.4Ghz, one transmitter/receiver at 915Mhz, and one video receiver at 5.8Ghz. This UAS will be capable of autonomous takeoff, waypoint navigation, and landing. Along with autonomy, the UAS will also have a live video feed that is inline with its primary camera s line of sight. When an area of interest or target is in sight, a command will be given for the primary camera to take a picture. This picture will then be processed in the Raspberry Pi 2. 1 SLU Rogue Squadron

2 Table Of Contents 1 Introduction 1.1 Rogue Squadron & Design Methodology 1.2 Mission Requirements 1.3 System Overview 1.4 Mission Preview 1.5 Expected Task Performance 1.6 Programmatic risks and mitigation methods 2 Aircraft 2.1 Overview 2.2 Propulsion 2.3 Payload Layout 3 Payload 3.1 Cameras 3.2 Gimbal 3.3 Pixhawk Autopilot Requirements Autopilot Peripherals 3.4 Data Processing Target Detection and Classification Letter Recognition 4 Ground Control Station 4.1 Overview 4.2 Mission Planning 5 Communications 5.1 MAVLink 5.2 FPV 5.3 R/C Manual Control 6 Safety 6.1 Specific Safety Criteria for Both Operations and Design Safety Switch, Shunt Plug, Buzzer, and Multicolor LED Flight Modes and GeoFence 6.2 Safety Risks and Mitigation Methods 7 System Testing 7.1 Mission Task Performance Aircraft Flight Performance 7.2 Payload System Performance Gimbal Performance Target Classification Testing Character Recognition 7.3 Autopilot System Performance 7.4 Evaluation Results Supporting Evidence of likely Mission Accomplishment 8 Conclusion 9 Acknowledgements 2 SLU Rogue Squadron

3 1 Introduction 1.1 Rogue Squadron & Design Methodology Rogue Squadron is a team composed of six undergraduates and one graduate student from the department of Aerospace and Mechanical engineering as well as one undergraduate student from the department of electrical engineering at Saint Louis University (SLU). Our design methodology focused on three main concepts: ease of integration, ease of use, and low cost. When examining various components for the UAS, it was desired that there was documentation on previous integration of the two components. For example, when choosing a LiDAR system for accurate altitude readings, it was found that PulseLight LiDAR Lite in particular had a great amount of documentation on how to integrate it with the Pixhawk autopilot. This documentation and its low cost led its use as our LiDAR system. Even the decision of our airframe followed this methodology. The Senior Telemaster is a well known airframe in the realm of R/C aircraft. It has a big empty fuselage that can easily house our payload. 1.2 Mission Requirements There are two primary goals in this year s AUVSI competition: autonomous flight and target classification. The UAS is required to achieve autonomous takeoff and landing as well as waypoint navigation. The UAS must be capable of capturing images of targets on the ground and decipher what is in the image. In addition to these primary objectives, there are also many secondary goals that Rogue Squadron will be attempting this year: Automatic Detection, Localization, and Classification (ADLC), Actionable Intelligence, Off-Axis Standard Target, and Emergent Target Task. These four secondary goals all require image recognition as well as a gimbal to control the camera s orientation. 1.3 System Overview Rogue Squadron s UAS consists of four main elements: airframe, payload, telemetry, and ground control. A Senior Telemaster airframe with an eight-foot wingspan was chosen for its high lift capabilities, high payload capacity, and long-standing history of being a successful airframe. The payload component on the UAS consists of various subsystems; including the Pixhawk autopilot, four transmitters/receivers, the Raspberry Pi 2 single board computer for onboard processing, batteries, a gimbal, and two cameras. The telemetry is based on a pair of 915MHz radios, interfaced with the ground control station. The Ground Control Station (GCS) hardware consists of a Dell Inspiron 15 and two transmitter/receivers. Figure 1: Conceptual representation of the avionics and payload integration on Rogue Squadron UAS 3 SLU Rogue Squadron

4 1.4 Mission Preview At the flight line, the following sequence of steps will be executed. First, all transmitters are to be turned on at the same time Mission Planner is opened on the GCS. Once the GCS is ready, a shunt plug will be used to arm the power module on the UAS. Once the power is on, the Mission Planner will connect to the onboard autopilot. After the connection is made and the mission is uploaded to the autopilot, the R/C Pilot will be able to initiate take off by switching the aircraft s transmitter into auto mode. At this point, the aircraft will perform its autonomous takeoff and reach an altitude preprogrammed into the software. Next, the aircraft will follow the waypoints programmed into the Pixhawk after takeoff to validate autonomous waypoint navigation. The aircraft will then proceed to the search area task. During this task, the aircraft will use the auto-grid pattern from mission planner to search the area. Concurrently, the gimbal and gimbal transmitter will be utilized to find targets of interest. Once these targets are found, the Canon PowerShot will be given the command to capture and save the image to the onboard flight computer where the data processing will occur. After the primary objectives have been met, besides autonomous landing, the secondary missions will be attempted. 1.5 Expected Task Performance Rogue Squadron is expecting to complete both primary missions as well as four secondary missions. The primary missions requires the UAS be capable of autonomous takeoff, landing, and navigation, as well as be capable to execute a target classification system. The secondary missions are as follows: Auto Detect, Localize, and Classify, Actionable Intelligence, Off-Axis Target, and Emergent Target. Off-Axis Target requires the capabilities for examining targets that the UAS cannot directly go over. Emergent Target task will require visual capabilities to help understand what activity is occurring at a given waypoint. The Actionable Intelligence mission will require automatically detecting and localizing a target. Finally, Actionable Intelligence requires immediate information of the target when the image is taken. Through the use of Rogue Squadron s UAS these tasks are expected to be accomplished. 1.6 Pragmatic Risk and Mitigation Methods The UAS can be considered to be a system of systems, with complex interactions and dependencies between the various components. As can be expected with such systems, there are many different risks, of various levels associated with our UAS, which could affect the performance of the missions. The main sources of risk are associated with the autopilot module, and the imaging system The first and foremost source of problems is a failure in communication with the onboard autopilot (Pixhawk). Throughout our testing, there have been a few occasions where in flight the Pixhawk has disconnected from the ground control station. We have seen that when this occurs, the Pixhawk can still continue to execute the current mission and uploaded data point. Once the current mission ends, the pilot will take control over aircraft and bring it back to a location where communication can be reestablished. The next most likely error that could occur is with the imaging subsystem. This is based on the Canon Hacker Development Kit (CHDK). Through testing, it was found that the camera can shut down due to inactivity. There are two methods to mitigate this issue from occurring during a mission. First, the camera is consistently taking photographs. This was done both to prevent the camera from shutting down as well as to quicken capture time. Finally, on the controller for the 4 SLU Rogue Squadron

5 gimbal, there is a reboot switch. This switch will restart the program on the Raspberry Pi 2 Model B and by doing so reestablish the communication between the camera and the Raspberry Pi (RPi) onboard image processing computer. 2 Aircraft 2.1 Overview For this competition it was decided that the airframe should have several characteristics which included high lift capability, endurance, and stability. Taking all these into consideration, it was decided that an eight foot Senior Telemaster would be well suited for the task. After this decision, a further analysis was done on the aerodynamic capabilities of the Senior Telemaster to confirm it would work with the design payload of electronics and whether it would be possible to integrate it with the Pixhawk autopilot. Using software designed for R/C aircraft such as XFLR5 and ecalc, it was Figure 2: Internal layout of avionics and Pixhawk confirmed theoretically that the Senior Telemaster would carry the estimated 10 pounds of payload. The plane had already been wired for R/C when it was acquired so the conversion to integrate the Pixhawk was quite simple from a hardware standpoint. 2.2 Propulsion The Senior Telemaster was designed to run on an internal combustion engine but for this competition it was decided that it would be possible to increase reliability and simplify the mechanical complexities of the aircraft by converting it to an electric motor. Much of this team s experience was with electric systems, so the decision to convert to electric was also based on prior expertise in this area. The combination of motor, battery and ESC, was decided based upon input from senior team members, faculty advisors, and the computer program, ecalc. The objective of this analysis was to determine the combination which would provide maximum Figure 3: Internal layout of the nose of the aircraft endurance with minimum cost of weight and space. After taking many combinations into consideration, it was decided that a Hacker A50-12LV3, in conjunction with an 8000mAh LiPo 6S battery and a Castle Creations Phoenix Ice 75 ESC, would provide enough endurance for about 20 minutes of safe flight. When performing flight testing in a fully loaded aircraft, it was found that the plane stayed aloft for 42 minutes until the ESC detected low voltage and cut the throttle. This test was done with dummy weights in place of the cameras and gimbal. The large difference in flight times was attributed to the 5 SLU Rogue Squadron

6 approximate nature of ecalc and the fact that during the endurance flight test the throttle was maintained at about 40%. 2.3 Layout The orientation of each component was chosen so that the aircraft s center of mass was at the quarter chord of the wing. Therefore, the batteries had to be placed at the front of the fuselage while the electronics such as the Pixhawk and the receiver had to be placed in the back of the fuselage near the trailing edge of the wing. The heaviest component, the gimbal and cameras combination, was placed as close to the quarter chord as could be done within the constraints of the landing gear, while allowing for necessary gimbal movement. With the batteries located on the opposite side of the fuselage from the components, the wiring to reach them became an issue and it was necessary to create wiring harnesses on the inside of the fuselage to contain the wires and secure them during flight. If some of the wires were to come loose during flight that could spell the end of the mission and possibly the aircraft. The Pixhawk, the main flight computer, was mounted on a separable board with all its components except the GPS receiver, and was placed high up in the fuselage near the center of mass. This was done to ensure the accelerometers measurements would represent the accelerations of the aircraft C.G. and that no corrections would need to be applied to account for the sensor displacement. The First Person View (FPV) camera transmitter was placed directly below that, and connected to the FPV camera mounted on the gimbal. Right forward of the transmitter is the mount for the 3-axis gimbal with both cameras attached. Finally, the batteries powering all the components in the UAS and the ESC were mounted in the nose of the plane, directly aft of the motor. 3 Payload 3.1 Camera In order for the aircraft to take pictures of ground targets, the team opted for a dual camera configuration. The first camera being a miniature camera is simply used for ground monitoring. It sends a continuous live video feed to the ground control station via radio transmission. However, for high quality images and image processing, higher resolution images are needed. For this task the Canon Powershot G9 was used. This camera offers auto focusing, continuous image capture, and 12.1 megapixel resolution. Typical computer interfacing only goes as far as image transferring through USB. In order to automate the camera as well a couple of programs were needed. The first program is called CHDK or Canon Hacker Development Kit. It is a firmware kit that can be installed on Canon cameras for functions beyond off the shelf usage. The firmware is downloadable online and then can be installed on an SD memory card. The card then installs the firmware onto the PowerShot camera. The makers of the CHDK software package also have platforms for CHDK interfacing including Windows and Linux. By downloading the software and installing it onto the raspberry pi computer, the canon PowerShot could then be controlled from a command line interface on the raspberry pi via USB connection. Typing commands such as shoot will cause the camera to autofocus, capture and image, and store it to the SD card. At this point of the process, the camera could be controlled remotely from a computer, but still required user inputs to run commands. Since the camera and Raspberry Pi would be in the air during missions, another component is needed for full autonomy. In this case, MATLAB was chosen to run the control program. 6 SLU Rogue Squadron

7 The program is simple. There are two threads each under an index value of 0 or 1.The initial thread under index of 0 runs a continuous loop of capture and dump commands for a variable interval of time. This gives the team control over how many and how often images are taken. The loop must be continuous because the canon camera needs some type of input or else it will automatically shut down to conserve battery power. The loop will capture images and then immediately delete the image files so memory space in conserved. The second thread of the program has the same structure, but is under an index of 1. The other difference is that instead of deleting the files, a transfer command is ran instead to move the captured image files to storage for processing. At the end of this loop, the index is reset to 0. The way the entire system integrates is as follows. The live video and image capture cameras are mounted together. The image capture canon is repeatedly capturing images. Once a target is found on the feed, a small switch is engaged on the gimbal controller. This switch send a ping to the Raspberry Pi onboard that is running the MATLAB script. The ping sets the index value to 1, which calls the second thread to run, which stores the images captured and runs the image processing thread. Since the second thread resets the index value, only the images captured during the loop are stored, and the switch can be flipped for any object suspected of being a valuable ground target. The Canon PowerShot runs on its own power supply and is mounted in line with the webcam so the video feed directly corresponds to the images seen by the Canon. Once the images have been saved, the camera capture step is complete. 3.2 Gimbal The UAS is equipped with a 3-axis gimbal controlled by an R/C Timer Basecam SimpleBGC 32 bit AlexMos gimbal controller. With the combination of the two, a state of the art camera stabilizing system was created. The system is capable of carrying two cameras, a Canon PowerShot G9 and miniature camera. The controller allows us to program in profiles for the gimbal. In our case, we will be having 3 different modes: takeoff/landing, stabilized flying and R/C Controlled. Once the gimbal is assembled. The controller was connected to the gimbal motors along with two different IMU sensors; one Figure 4: Graphical User Interface used to tune the Gimbal Controller connected above, the motor controlling the roll of the gimbal and the second under the motor controlling the pitch of the gimbal. Once all of the components were connected, the Basecam controller had to be connected to a computer via USB. The Basecam Controller needs a program, SimpleBGC_GUI, for set up. Here, we can adjust the power a motor receives, the speed of the motor as well as profile set-ups. The first step in setting up the gimbal controller is Gyro Sensor calibration. This is done by choosing IMU Calibration Help then holding the gimbal in the position indicated by the program for a few seconds and then moving on to the next position. Once they Gyro calibration is complete, Motor Configuration needed to be performed. By choosing Auto under the Motor Configuration, the controller tests each motor and learns its movements, deciding whether a motor needed to be 7 SLU Rogue Squadron

8 inverted or not as well as assigning Power and Poles to each Motor. The Poles of each motor had to be fixed at 22 Poles based on our motor specifications. Once the motors were calibrated, PID Controller had to be tuned. The PID (proportional-integral-derivative) controller allows us to control the power, reaction speed as well as movement speed of the gimbal. P describes the power of disturbance response; the higher the value, the stronger the response. D normalizes the frequency and dampens the vibrations of the gimbal. I controls the speed at which the gimbal moves. For each axis, PID values were set to 10, 0.4, 10 respectively. Then the power was set to zero for two axis, leaving one having power and then the PID values were tuned until the gimbal behaved the expected way in that specific direction. Each axis was tuned the same way. The goal was to reduce the vibrations as much as possible while keeping the reaction time at a speed that was acceptable for our application. Once the gimbal was tuned, the next step was to program different profile. For our Take-off/landing profile, we were required to have the gimbal stationary keeping pitch at a -90 angle so the plane had enough ground clearance. The stabilized mode would only be used as a transition mode between take-off and R/C Controlled. 3.3 Pixhawk Autopilot Autopilot Requirements The following are the requirements that were considered prior to choosing an autopilot system: Autonomous Takeoff and Landing GPS Waypoint Navigation Switch between flight modes when necessary Monitor flight properties such as position, altitude, and velocity Receive and send telemetry data to the GCS Autopilot System The team chose the Pixhawk autopilot system to fulfill the tasks of the competition. Other systems were considered, like the APM 2.5 autopilot system, but the Pixhawk was the best match for the mission tasks. It is more advanced than the APM as it has more RAM, a built-in gyro and magnetometer, as well as other enhanced capabilities Peripherals A 915 MHz telemetry radio,made by3drobotics (3DR), was used to communicate between the Pixhawk autopilot and the GCS. One of the antennas is mounted on the side of the aircraft and is connected to the Pixhawk, and a similar antenna is connected to the GCS, to create a wireless telemetry transmission between the autopilot and the Mission Planner GCS. The antenna is mounted on the side of the aircraft. A pitot tube, mounted on the left wing and protruding out of the leading edge, is connected to a digital airspeed sensor. While in flight, it provides indicated airspeed and, using the static pressure sensor on the Pixhawk, the true airspeed. For accurate data, the pitot tube is placed well in front of the wing's leading edge to avoid any unfavorable interference due to the wing, on the on-coming air flow around the pitot tube. To reduce the weight of the aircraft, the main battery, which is a 6S lipo battery, is used to power the autopilot, along with other components in the aircraft. The Pixhawk comes with a power module that accepts a maximum input voltage and current of 18V and 60A, respectively, while supplying the autopilot with 5.3V and 2.25A of power. The Pixhawk cannot not support 8 SLU Rogue Squadron

9 anything higher than a 4S lipo battery, so the power module was replaced with one that would accept an input voltage of 30V and current of 90A, which is compatible with our selected 6S battery. To enable a more accurate, autonomous landing, a LiDAR is also connected to the Pixhawk as a powerful proximity sensor. It has an accuracy of +/ m. Further tests are underway to validate the flight performance with LiDAR. An FPV camera is used as an on-board camera that takes a continuous video. This video feed will be inline with the sight of the Powershot G9. To create a connection between the on-board camera and the Pixhawk, a MinimOSD is used. Originally, a 1.1 GHz Vee Antenna was connected to the MinimOSD to transport the feed to the ground control station. This signal was found to be too close in frequency with the GPS. To fix this issue, it was replaced with a 5.8Ghz video transmitter. 3.4 Data Processing After the images are taken by the Canon, they are sent to the onboard flight computer where image processing for the target recognition is performed. The onboard computer used is a Raspberry Pi 2 Model B. This single board computer (SBC) released earlier this year was selected because it is small, lightweight, very inexpensive, and provides sufficient processing power due to its quad-core processor and 1GB of RAM. The operating system running on the Pi 2 is Raspbian, which is based on Debian GNU/Linux and designed specifically for the Raspberry Pi. A partitioned microsd card runs Raspbian on one partition, while the images and associated data are stored on another partition which acts as a hard drive and will retain this information even in the event of a loss of power Target Detection and Classification The images are processed by a script written in the Python programming language. OpenCV (Open Source Computer Vision Library) is used for the processing of the images. The script first takes in the photos taken by the Canon and performs a series of operations on them to identify the location of potential targets in the image. Although various techniques and manipulations of the images have been explored and tested, work is ongoing to determine a method or combination of methods that can be deemed reliable, as the script currently sometimes identifies hundreds of "targets" that are in actuality slight variances in the grass surrounding the targets. Attempts to identify a reliable method have included the following manipulations to the testing images: blurring, grayscaling, quantizing, k-means clustering, downscaling then upscaling using pyramids, various types of thresholds, and Canny edge detection. Currently, the most promising technique involves converting the image from RGB (Red Green Blue) color space to HSV (Hue Saturation Value) color space and then performing both a binary and an inverted binary threshold on the hue channel to identify contrasting colors, which is then considered to be a potential target. When a potential target in a photo is located, a cropped selection of the image that contains the potential target plus a little extra surrounding area is saved as a new file. Another script takes in these cropped images and performs similar operations as before to determine if the potential target is in fact a target. If so, the target's geometric shape and color is then determined. Again, work is currently being done to identify a reliable method, but currently, the most promising method is to use a combination of quantization, Gaussian blur, grayscale, and then binary thresholds to identify the potential target. Using the cropped image has been more reliable in accurately identifying the characteristics of the target as opposed to using the larger, raw image 9 SLU Rogue Squadron

10 taken with the Canon. If another "shape" (the letter) is completely inside the potential target, then the potential target is confirmed as a target. From there, the letter is cropped out and saved in another file to be later processed in order to identify which letter it is. Techniques to do this are explained in the Section When the target is confirmed, the contours of the shape are considered. The number of edges and the angles they make are used to identify the geometric shape. The hue of the pixels inside the object's perimeter, ignoring those of the letter, are used to identify the color of the target. Similarly, the color of the letter is also determined Letter Recognition The files containing the cropped out letters are used for the letter recognition portion of the target classification task. In order to complete the process of letter identification, multiple methods were attempted using various functions built into MATLAB. These functions are all currently used in letter recognition outside of the competition setting; therefore, multiple tests were devised to evaluate the recognition capabilities of each function. Knowing the limits of each function allowed for the selection of the best function of those tested. The classification functions and libraries used were the cascade object detection (COD) function library, the Hugh oriented gradient (HOG) function library, and the optical character recognition (OCR) function. This paper will discuss the criterion used for selecting which set of function or functions to use as well as the methods used for training the function. COD is a method of detecting objects based off of training a function that will recognize specific features of the object. Determining these features is done Figure 5: A sample of images used for testing and validating Letter Recognition Task by user defined regions of interest that are placed in a training function. Also necessary for training is a set of images that do not contain the object of interest. With this positive and negative set of information, the classifier is trained and produced. Similarly, the HOG requires a set of images to train the classifier function. This training set finds the orientation and length of straight lines found in the picture and then saves them for identification. The training process creates a classifier that should identify any letter presented to it as that letter. Finally, the OCR function is pre-trained; therefore, it only requires an input image that it would then identify as the letter or string of letters within the image. To properly test the processes, an image set was compiled from the internet of a variety of letters. This image set consisted of one hundred images of each of the twenty-six letters producing a total set of two thousand six hundred images. Once all the images were collected, the COD and HOG were trained leaving out five of the images in the image set out from each letter for testing. The OCR did not require training since it was a pre-trained function. All three classifiers were then used to identify the remaining five images from each set. False alarm rate (FAR) and positive identification rate (PIR) were the evaluation criterion for the classifiers. FAR and PIR respectively determine the percentage of images wrongly identified as being a specific 10 SLU Rogue Squadron

11 letter and the percentage of images correctly identified as a specific letter. When the evaluation was complete the COD was determined to be the top contender. With the COD selected as the classifier for the UAS, the problem became determining if the classifier could handle images taken from a moving aircraft. To test how well the classifier could handle these potentially blurred images, multiple images were taken at ground level of each letter used in classification. This process involved taking images at various angles and lighting, so that anything that could potentially cause the classifier problems could be eliminated by image processing prior to being fed into the classifier. For instance, handling multiple angles was done by rotating the captured image in five-degree increments for three hundred and sixty degrees. Completing all these processes at ground level was then repeated by capturing multiple images of the targets from the UAS. A similar process of evaluation was used, including taking images from multiple altitudes in various conditions. This process of testing the classifier allowed ample preparation for the unknown competition conditions. Ideally, more images would be used for training in order to improve the accuracy of the system. For future improvements, more images would be included in the training set. Also more potential classifiers would be tested outside of MATLAB. Finally, a full test mission would be run in order to evaluate the capabilities of the classifier in flight. After the image is processed onboard and the target's shape, color, letter, and letter's shape have been determined, the information is stored and sent down to the ground computer for data retrieval. 4 Ground Control Station 4.1 Overview The ground control station consists of a Dell Inspiron 15 laptop computer, two R/C transmitters, one 3DRobotics transmitter/receiver, and one video receiver. The GCS allows a user to plan and analyze flight missions, as well as calibrate actuators and sensors for the UAS. There are two programs used consistently with the UAS: Mission Planner and SimpleBGC_GUI. Mission Planner is open source software that allows users to download flight data, create flight plans, and configure the Pixhawk autopilot. SimpleBGC_GUI allows a user to calibrate gimbals and configure gimbal controllers. One of the R/C transmitters controls the aircraft Figure 6: Ground Control Station and Peripheral controllers while it is in flight. This transmitter has a threeposition switch that controls the mode of the autopilot. When it is in manual mode, the transmitter has full control over the aircraft. The other R/C transmitter controls the gimbal and camera. This transmitter will be able to change where the camera is looking and, when a target is in sight, take a picture. 4.2 Mission Planning All of our mission planning is conducted by using Mission Planner created by Michael Oborne. This program allows for autonomous events, autogrid, and waypoint navigation. 11 SLU Rogue Squadron

12 Autonomous events consist of loiter, takeoff, landing, and return-to-launch. By utilizing autonomous events, waypoints, and autogrid features, flight plans can be created easily. All one has to do is go into Mission Planner and change to the Flight Planner tab. This tab displays a Google Map of the Earth. Through the use of longitude and latitude a home waypoint is set. This is used for the return to launch feature. Once a home point is created, the Google Map will zoom into its position and events and waypoints can be added. After the flight plan is devised, a GeoFence is set. The GeoFence prevents an aircraft from exiting an area. This is done in three dimensions. First, a set of points are set around the flight plan. These points are then connected to create a fence. After the fence is created Mission Planner requests a minimum and maximum altitude. These two values create an upper and lower boundary that the aircraft cannot pass through unless takeoff and Figure 7: Image of Mission Planner from plane.ardupilot.com landing are being conducted. The final result from the GeoFence is an imaginary boundary that the aircraft will bounce off of to prevent it from going into undesirable airspace. If there is a GeoFence breach and the aircraft is completely out of the GeoFence, the aircraft will return to a predescribed waypoint and loiter until it is told to do otherwise. 5 Communications 5.1 MAVLink To facilitate communication between the ground station and the Pixhawk flight controller, the MAVLink (Micro Air Vehicle Link) communication protocol is utilized over the 915 MHz radio link. This protocol is built-in to both the autopilot firmware running on the Pixhawk and the Mission Planner software used on the ground computer. Various built-in interfaces allow for two-way parameter and waypoint data transmission. A heartbeat message also allows for checking if the connection is still active. Mission Planner retrieves the necessary device parameters on initial connection to the Pixhawk. The Mission Planner software then handles sending and receiving data, such as GPS location, waypoint locations, and air speed. MAVLink also supports sending and receiving control data from the serial pins of the Pixhawk. This allows for control of external devices, such as the Raspberry Pi. Within Mission Planner, additional MAVLink messages are sent and received through a command window. 5.2 FPV The 5.8 GHz FPV video feed is fed to the ground station through a receiver that provides composite video output. This allows the FPV to be either displayed on an external monitor or in Mission Planner through a video capture card input. 12 SLU Rogue Squadron

13 5.3 R/C Manual Control As with many UASs, there needs to be a connection for R/C. This connection is created by utilizing a Futaba six-channel receiver and twelve-channel transmitter. This system allows for manual control of both the aircraft s attitude and throttle as well as which flight mode the Pixhawk autopilot is in. 6 Safety 6.1 Specific Safety Criteria for Both Operations and Design Safety Switch, Shunt Plug, Buzzer, and Multicolor LED The Pixhawk autopilot system comes with a safety switch (push button), buzzer, as well as integrated LED lights at the lower portion of the Pixhawk board. The light and buzzer are used to indicate the state of the system with a sequence of blink patterns, beeps, and lights. Some of the states indicated are arming/disarming, low battery, and command acceptance/rejection. This allows a user to know what the autopilot is doing and thinking without needing to connect the Pixhawk to the ground control station. The safety switch is used for arming/disarming the system. When the system is disarmed, the throttle and actuators are held inactive regardless of the R/C transmitter input. To arm the Pixhawk, the switch, located below the left wing, needs to be held for about three seconds. This prevents the system from being accidentally activated. The shunt plug is located in the nose of the aircraft. This plug allows for power to go from the battery to the Pixhawk, motor, and actuators. Since the battery is accessed by taking the wing on and off, this shunt plug allows for quick access and immediate shutdown of the UAS on the ground Flight Modes and GeoFence During a flight session, the R/C transmitter can be used to switch between three flight modes: stabilize, manual, and auto. The aircraft will be in auto mode during the primary tasks. Knowing this, the autopilot system has been programmed to allow for some user input in this mode. This allows the pilot to make minor adjustments without switch to the manual mode. Also, in the case that the aircraft will drift away from the set flight zone demarcations, a geofence is set during the creation of the waypoints. A return point is set within the geofence. When the geofence is breached, Mission Planner will warn the user and the aircraft will return to the location, and altitude, indicated by the user. 6.2 Safety Risks and Mitigation Methods The main concern for safety is associated with the aircraft itself. Potential problems include a loss of power from the onboard battery, damage to the airframe, and a loss of R/C radio link to the aircraft. To mitigate these potential problems, prior to each flight, the team captain will ensure that the battery is fully charged, and is damage free. In addition, a thorough visual inspection of all the aircraft components will be performed. To mitigate the issue of R/C link loss, a failsafe feature will be programmed into the onboard autopilot to bring the aircraft back to its launch position. Also, if the communication link between the GCS and the aircraft is still 13 SLU Rogue Squadron

14 healthy, then appropriate commands will be issued through the ground control station to the autopilot to safely land the aircraft. 7 System Testing 7.1 Mission Task Performance Aircraft Flight Time Performance To analyze and validate the aircraft s performance, multiple flight tests were conducted. The first flight test examined the airframe worthiness. The airplane had the Pixhawk equipped but was held in manual mode so that the R/C pilot had full control over the airframe. The camera and gimbal had dummy weights in its place to protect them from potential damage. This flight had a successful takeoff, flight, and landing. The next flight test was used to look at the endurance of the aircraft and whether it could operate for the full forty-minute mission time. The aircraft took off under R/C control and then the Pixhawk was told to hold altitude and loiter around a waypoint. The aircraft was 100 feet off the ground with a 40% throttle setting. While holding altitude, the pilot had minimal control over the aircraft's heading. When the battery reached low voltage the ESC cut-off the motor. This occurred after 43 minutes of flight. 7.2 Payload System Performance Figure 9: Rogue Squadron s UAS taking off during a flight test mission Gimbal and Camera Testing The gimbal and camera combination make up the Telemasters payload. Testing the gimbal proved to be a tedious process that involve a great deal of power changes to get the correct configuration. The first step in testing the gimbal was to establish the correct power to each motor. The Auto motor configuration was run to find out whether any of the motors had to be inverted; it was found that the Roll motor did need to be inverted. The power was manually set to 150; a value where the motors had enough power to function, but low enough not to over-heat the motors. Once the motors were fully configured, they had to be tuned to perform to the desired expectations. 14 SLU Rogue Squadron

15 (a) (b) Figure 10: (a) shows disturbances in the electric input in the motor, (b) shows the motors in a stable mode The input power in the previous step controls the necessary PID Controller configuration needed to satisfy the mission. Each individual axis had to configured one by one. This was done by disabling the other two axis. The P value is the first one that needs to be raised by just onestep increment at a time until the motor starts knocking or vibrating, as can be seen in Figure 10a. Raising the D value will dampen the vibrations and settle the motors into a steady operating state, as seen in Figure 10b. Once the vibrations were not visible on the graphs within SimpleBGC_GUI, the test was then stopped and moved on to the next axis. Once all three of the axis were tuned, the gimbal was taken and moved around to be observed how it behaved. If there were any glitches seen in the test, the tuning process had to be resumed. After a significant amount of time spent tuning, it was determined for best behavior: Roll Figure 11: Gimbal configuration during Take-off/Landing setting of P=15, I=0.4 and D =18, Pitch setting of P=8, I=0.4 and D=6, and Yaw setting of P=10, I=0.4 and D=10 gave us the best results with the gimbal staying stable. Once the gimbal was tuned, 3 profiles had to be tested. The full configuration was done one the first gimbal profile, Stable flight. This mode allows the gimbal to stay level when not in use. The second profile, takeoff/landing, requires the gimbal to reconfigure itself so the plane has can take off and land without the length of the gimbal getting in its way. Figure_2 shows the take off/landing configuration. Here the initial pitch angle had to be set to -90. To make sure the gimbal worked properly, the profiles were changed multiple times to test whether the gimbal would return to the profiles initial conditions, with each test being a success. The third profile, R/C controlled, uses the same configuration as stable flight with an added controller that allows a user from the ground to move the gimbal. 15 SLU Rogue Squadron

16 7.2.2 Target Classification Testing Pictures of mock targets that were taken with the Canon were shot and used to test the performance of the target classification script discussed in section As stated earlier, the most promising technique for target classification involves performing both a binary and an inverted binary threshold on the image s hue channel to identify contrasting colors, which is then considered to be a potential target. The current version of the script combines the binary threshold image with the inverted binary threshold image for both Figure 12a and Figure 13a, resulting in Figure 12b and Figure 13b respectively. As seen in the examples, results vary. Generally, when the background is fairly uniform, the code is easily able to identify the targets. However, if there is a lot of other objects in the background, the results are not always accurate. For example, the background of Figure 13a is more variant than that of Figure 12a. This makes it difficult to correctly identify the targets, as variances are picked up in the color differences of the background as well as the texture in sidewalk. Figure 12a s background was slightly less complicated, resulting in both targets being identified. Results also vary depending on the color of the target and the color of the background immediately around it. This is observed in Figure 13a since the green B target is not distinguished from the green grass directly behind it. Work is ongoing to correct these issues and to increase the accuracy of the target recognition.. (a) (b) Figure 12: (a) Unaltered Canon photo (b) Threshold of Figure 12a (a) (b) Figure 13: (a) Unaltered Canon photo (b) Threshold of Figure 13a 16 SLU Rogue Squadron

17 After detecting the objects from the original images taken with the Canon, the target is cropped out and saved to a separate image file. This has been done of the A target in Figure 12a, resulting in Figure 14a, seen on the next page. As stated in section 3.4.1, a different technique of extracting and classifying the target is used in this second stage of target classification than that used in the first. Using a K-means algorithm with four clusters, the image being examined is reduced to four colors, returning Figure 14b. This creates more defined edges, which can easily be determined and drawn using Canny edge detection as seen in Figure 14c. From here, the shape, letter, and colors of each can be deduced Character Recognition (a) (b) (c) Figure 14: (a) Cropped Target (b) K-means reduction (c) Canny edges Evaluation of the character recognition classifier was done using a series of tests designed to push the classifier to the limit of its capabilities. This would help determine how much pre processing would need to go into each image before it was presented to the classifier. Problems began to arise as backgrounds became cluttered or the color of the letters varied. The classifier works by characterizing features of the Figure 15: Example of character recognition program letter in order to distinguish it from the surrounding features. When the background features became blended into the letter itself, the classifier was unable to distinguish if there was an actual letter in the image. In order to address this issue, the letter was pulled out from the background by finding eliminating the background colors from the image leaving a black and white image of the letter. Varying color caused the features of the letters to become skewed; however, since the letters in competition were solid colors, this issue was not investigated further. Figure 15 shows the types of images that were able to be identified by the classifier. The images that were not correctly identified, required more preprocessing to distinguish the letter from the background. Since the images in black and white were easily identified, it was decided that this was the best sort of image to present to the classifier. Another issue faced was object orientation within the image. The classifier was presented with images containing an object at a variety of angles. Once the character of interest was rotated 17 SLU Rogue Squadron

18 beyond a certain angle the classifier was unable to identify the character. In order to ensure the classifier would have the image in proper orientation, the images were rotated three hundred and sixty degrees. Although this would solve the position issue, some letters could be mistaken with others when rotated. This problem was solved by choosing the character that was the most frequently identified. This method of training and preprocessing was successful in creating a character classifier that could identify alphanumeric characters. 7.3 Autopilot System Performance After the airframe was validated, the Pixhawk autopilot was tested. The first element tested was waypoint navigation. This was done by taking off under pilot s control and then switching into auto mode. When the aircraft was being tested, auto mode allowed for some pilot input if it was found necessary. In this flight test, it was found the aircraft was capable of reaching each waypoint but would sway from side to side on the flight path. This showed that further tuning of the autopilot would be needed to keep it on the given flight path. (a) (b) Figure 16: (a) Actual flight path for takeoff and waypoints (b) Planned flight path for takeoff and waypoints Following the navigation test, autonomous takeoff was tested. This was tested by bringing the aircraft to the runway and having the pilot switch from manual mode to auto mode. Once in auto mode, the aircraft revved the engine up to the proper throttle setting and took off to the first waypoint. After the aircraft met all waypoints, the pilot took over control of the plane and landed. This was repeated multiple more times all having successful results. In Figure 16, a flight test is seen. On the right, is the planned mission and on the left is the actual result of the test. This figure validates that the Pixhawk auto piloting system is capable of takeoff and waypoint navigation all in one mission. (a) (b) Figure 17: (a) Actual flight path for landing (b) Planned flight path for landing 18 SLU Rogue Squadron

19 The last component of the autopilot to test was autonomous landing. Currently, only an aerial landing has been completed. In this test, the autopilot was told to land fifteen meters off the ground. As seen in Figure 17,on the previous page, the aircraft had the expected flight path although it landed slightly off the runway. This should not be an issue at the competition since the runway is much larger. Also, the aircraft was told to touchdown at fifteen meters off the ground at waypoint seven. This was validated through flight data. 7.4 Evaluation Results Supporting Evidence of likely Mission Accomplishment Based on the success of flight, programming, and component testing, we hope to accomplish all of the primary missions and four of the secondary missions. As can be seen in section 7.3, extensive testing has been done on the Pixhawk autopilot. This has lead to a great confidence in Rogue Squadron s capabilities of autonomous takeoff, flight, and landing. Section 7.2 discusses both programming and component testing. This testing has provided promising results that the primary search area task will be accomplished. 8 Conclusion Rogue Squadron will attempt six missions in the 2015 AUVSI-SUAS competition. These missions required an autonomous aircraft capable of target classification and detection. To achieve these requirements, a UAS was created with four main systems: payload, telemetry, ground control station, and aircraft. A Senior Telemaster was chosen as the aircraft due to its reliability, stability and storage capacity. The Payload is comprised by two cameras, a miniature camera and a Canon PowerShot G9, a Pixhawk autopilot, a gimbal, a Raspberry Pi 2, video transmitter, a 3drobotics transmitter/receiver, and four batteries. The ground control station consists of a Dell Inspiron 15 equipped with Mission Planner, Google Earth, and SimpleBGC_GUI. The ground control also has a 915MHz transmitter/receiver and a 5.8GHz receiver. There are four components in the telemetry: two 2.4GHz R/C transmitter/receivers, one 5.8GHz transmitter/receiver, and one 915Mhz transmitter/receiver. To analyze targets a two-camera system was developed. These two cameras will have the same line of sight and be mounted on a three-axis gimbal. A 2.4GHz transmitter receiver will control the gimbal on the ground. A live video feed will be gathered from a miniature camera. When a target is seen, a switch on the gimbal controller will be activated. This will inform the Canon PowerShot G9 to store the image it took for analysis. The image will then be passed through various programs on-board the Raspberry Pi 2 to determine the characteristics of the target. Finally, this information is passed to the ground control station. 9 Acknowledgements A special thanks goes to David McQuinn, Matt Trani, and Srikanth Gururajan. We would also like to thank the Saint Louis Aeropilots for facilitating our flight tests. 19 SLU Rogue Squadron