Robotic Apple Harvesting in Washington State

Transcription

1 Robotic Apple Harvesting in Washington State Joe Davidson b & Abhisesh Silwal a IEEE Agricultural Robotics & Automation Webinar December 15 th, 2015 a Center for Precision and Automated Agricultural Systems (CPAAS) & b School of Mechanical and Materials Engineering, Washington State University (WSU) 1

2 Acknowledgements This work was funded by the United States Department of Agriculture (USDA- NIFA) through the National Robotics Initiative (NRI). 2

3 Presentation Overview Motivation Working environment Design objectives Hand picking analysis System design Preliminary field testing results Future work Questions 3

4 Research Motivation Washington State fresh market apple industry in million metric tons of apples valued at $1.84 billion USD 1 Accounted for 70% of U.S. apple production The WA fresh market apple harvest requires Employment of 30,000 additional workers An estimated cost of $1,100 to $2,100 USD per acre per year 2,3 Labor costs are rising and there is increasing uncertainty about the availability of farm labor Lack of mechanical harvesting for fresh market apples is a significant problem 1 USDA National Agricultural Statistics Service. (2014 Washington Agriculture Overview). Retrieved November 2, 2015, from 2 Galinato, S., & Gallardo, R. K. (2011) Estimated Cost of Producing Pears in North Central Washington (FS031E). Retrieved January 13, 2013, from 3 Gallardo, R. K., Taylor, M., & Hinman, H. (2010) Cost Estimates of Establishing and Producing Gala Apples in Washington (FS005E). Retrieved January 7, 2013, from 4

5 Long-term Goal: Reduce dependence on the labor force for fresh market tree fruit harvesting 5

6 Working Environment Commercial apple orchard located in Prosser, WA Highly unstructured environment Modern cultivation systems with formal tree architectures Fruit Wall concept simplifies the task 6

7 Cycle time < 6 sec Design Objectives Detachment success > 90% Fruit damage < 10% Pick multiple apple varieties Modular design that is cost-effective Our Approach: An undersensed design that executes look-and-move fruit picking, is mechanically robust to position error, and replicates the human picking process 7

8 Initial Design Development: Manual Apple Picking Fruit is grasped with a spherical power grasp 4 with the index finger applying pressure against the stem No dexterous manipulation of the fruit with the fingers To separate the apple from the branch, the hand moves the fruit in a pendulum motion 4 Cutkosky, M. R. (1989) On Grasp Choice, Grasp Models, and the Design of Hands for Manufacturing Tasks. IEEE Transactions on Robotics and Automation 5 (3):

9 Undersensed Hand Picking5 Are there effective methods to pick fruit that do not require fruit orientation and stem location? 5 Davidson, J., Silwal, A., Karkee, M., Mo, C., & Zhang, Q. (2015). Hand Picking Dynamic Analysis for Undersensed Robotic Apple Harvesting. Transactions of the ASABE. (Under review) 9

10 Representative Hand Picking Data 10

11 Custom design 7 degrees of freedom Modular configuration (Dynamixel Pro actuators) Mechanical Design 11

12 End-Effector Design Underactuation provides shape-adaptive grasping Passively compliant joints enhance robustness to position error & unplanned collisions 6 Grasping is executed in an open-loop manner Fabricated with additive manufacturing 6 Davidson, J., Silwal, A., Karkee, M., & Mo, C. (2015). Proof-of-Concept of a Robotic Apple Harvester. Robotics and Autonomous Systems. (Under review) 12

13 Vision System 7 1 Color CCD Camera PMD Camcube 3.0 (ToF, 3D camera) 2 Camera Rig Identification 3 Localization Fusion 7 Silwal, A., Gongal, A., & Karkee, M. (2014). Identification of Red Apples in Field Environment with Over-the-Row Machine Vision System. Agricultural Engineering International: Agric Eng Intl (CIGR Journal), 16(4),

14 Experimental Setup 8 8 Silwal, A., Davidson, J., Karkee, M., Mo, C., Zhang, Q., & Lewis, K. (2015). Design, Integration and Testing of a Robotic Apple Harvester. Journal of Field Robotics. (To be submitted) 14

15 Hardware Architecture 15

16 Video 16

17 Vision Performance Actual vs. Recovered Image 17

18 Task Timing & Vision Accuracy Vision Accuracy: Total # of Images = 54 Total Fruit Manual Count: 193 Total Fruit Identified: 193 Identification Accuracy = 100% Total Fruit in Workspace = 150 Average Fruit per Image = 4 Average Vison Time per Image = 6.3 s Average Vision time per Apple = 1.7 s 18

19 Picking Results 127 of 150 fruits attempted were picked (approximately 85%) 8/127 No stems 33/127 Spur attached to fruit 86/127 Stems attached to fruit Misses fall into the following five general categories 1. Poorly thinned branch (aka fruit pendulum ) 7 instances 2. Finger grabbed adjacent obstruction 3 3. Position and/or calibration error 8 4. Fruit slipped from grasp 2 5. Previous fruit stuck in hand - 3 No obvious evidence of bruising Ideal fruit location is 3 6 in away from the trellis wire 19

20 Picking Time Mean picking time 6.01 sec/per fruit 1 st fruit in a cycle: 6.22 sec Remaining fruits in a cycle: 5.84 sec Each task in the picking sequence was segregated into an individual function and timed Motion planning computation: 0.15 sec Approach: 2.14 sec Grasp: 1.5 sec Removal: 1.23 sec Fruit release: 1 sec Time Picking time 0 Motion Planning Approach Grasp Removal Fruit Release 20

21 Future Work Higher level decision making based on detection of trunks and trellis wires Grasp planning based on visual input Tactile sensor integration for detection of stem break, missed fruits, etc. 21

22 Questions??? 22