Pong Bot. Richard Dowdell. Zean Ng. Wesley Vowell. ME 4451: Robotics

Transcription

1 Pong Bot Richard Dowdell Zean Ng Wesley Vowell ME 4451: Robotics Fall 2011

2 Initial Plan The initial plan of our project was actually to program the Epson robot to be able toplay Ping Pong with the user. The ball was to be tracked with two cameras in two different planes to get a real time position of the ball (shown in Fig 1.1 and Fig 1.2) and the Epson moved in 3-D space to the final position of the ball. Figure 1.1: Setup of Cameras Y and Z axis Figure 1.2: Setup of Cameras Y and X axis

3 However, it was pointed out that the Epson robot and the camera s capabilities would not allow for this as the ball would move too fast if it was bounced towards the Epson. Also, there was the issue of safety where the Epson may pose hazards if moving at fast enough speeds to intercept the Ping Pong ball in time. Therefore, the goal of the project was changed toplay a game of Pong with the user where the Ping Pong ball was rolled to the robot instead of bounced. The Epson robot would move towards the ping pong ball which is rolled towards it and knock it back, simplifying the initial plan to just a single plane and reducing the work on the software. The plan also required one camera to be used instead of two as shown in Figure 1.3. The camera is to be placed above the Epson robot to allow a full view of the ball s position and also to reduce obstruction to the movement of the Epson. The bulk of the project was writing code for image acquisition and realtime data processing in Matlab and Epson s software. Figure 1.3: Single Camera Position There were three steps the Pong robot had to carry out to meet the goal. The first step was to detect the Ping Pong ball as it moves across the camera s field of vision and take several

4 points of its centroid in a short period of time. Doing this allows several positions of the ball to be calculated with respect to time. The second step then includes having the software use the previous data of the ball s centroid and extrapolates the data to predict the final position of the ball upon reaching the Epson robot s range of motion. The final step is then having the Epson take in the data of the final position of the ball and move preemptively to the predicted position to hit the ball back and swings at it to hit the ball back. Robot Kinematics The Pong project used an Epson robot to accomplish the task. There was no inverse kinematics done in the project. It was decided that the Epson robot s motion pathway would be moving transversely across a straight line in the x-axis to meet the ping pong ball. 11 ranges of points were first measured along the Epson s motion pathway. The 11 points were divided equally between the maximum positions the Epson could move to the furthermost right and left. The Epson was then taught to move to a specific point for each range.based on the initial points of the ping pong ball when it first comes into the view of the camera, the final position of the ping pong ball would be estimated. This predicted final position of the ping pong ball would fall within one of the 11 ranges of points that were set and the Epson would move to that predetermined position corresponding to that chosen point. If the predicted final position of the ping pong ball fell out of the ranges of motion, the Epson was programmed to not move but instead spit out an error which tells the user that it is out of bounds. A schematic of the range of motions is shown in Figure 2.1. If the predicted final position of the ping pong ball fell within the numerical range of one of the boxes, the Epson would move to that specific box and then swing forward slightly to hit the ball back to the user. Each box is 4.6 cm apart.

5 Figure 2.1: Range of Motion of Epson Robot Challenges and Solutions a. Machine Vision One of the challenges of the Pong project was the area of machine vision. An issue the group faced in the beginning was getting a smooth and fast enough real-time vision capture from the camera. The Trendnet cameras used in the lab did not have enough frames per second required by the project to provide the information needed. Also, the method taught in lab where the image was continuously downloaded from the IP address of the Trendnet was not fast enough for real-time data processing. Therefore, a solution to this was to use a Logitech webcam which had 30 frames per second and using different code to acquire the image. The code was based a premade code taken from Matlab found when using the imaqtool function in the Matlab library. This gave a fast enough real-time image of the camera that could be worked with.

6 Also, a Matlab code initially coded in one of the robotics lab sessions was utilized to accurately find the centroid of the ball as it moves. The code would convert the video input to black and white and let the ping pong ball be the only white object in the camera s view However, the code would pick up many reflections of light and centroids of random smudges or any imperfections on the surface the ball rolls on top of. A solution to this problem was buying a new black board which would reduce surface imperfections and also upping the threshold of the im2bw function to a value of 0.7 to reduce reflections of light. b. Real Life to Matlab Measurement Conversion The next challenge faced was to use the data of the centroid of the ping pong ball acquired from the first steps and converting it into a predicted final position where it would meet the Epson robot. The first step was to relate real-life measurements to Matlab measurements. The dimensions of the vision of the camera was first calculated in Matlab and then measured in real life. It wasdetermined that the relationship of Matlab and real measurements were pixels per inch. The distance of the center of the camera s view and the end effector of the Epson was measured to be 25.5 inches or Using a Matlabpolyfit function and a 1 degree polynomial (the ping pong ball was expected to move in a straight line), the final position of the ping pong ball at the end effector of the Epson could be predicted. c. Real Time Data Processing and Estimation One of the biggest challenges faced in this project was to make sure the processing of the data taken would be fast enough to move the Epson to the final position especially since it had to be relayed from Matlab to the Epson controller software. The first step taken to increase the

7 speed of data processing was to predict an accurate final position of the ball as fast as possible using the minimum amount of points of the ball s centroids. After some trial and error, it was decided that the final position of the ball would be calculated once at least two points of the ball s centroids was taken and a time of 3 iterations in Matlab passed. The way the code is written is that the time of iterations would only start once at least two points of the centroid is obtained. This allows the program to be triggered only once the ball comes into the camera s vision. An example of the extrapolation of the ball is shown in Figure 2.1 below where the red circles are the ball s centroids. Figure 2.1: Extrapolation of ping pong ball based on centroids One problem was getting the information of the final position of the ball from Matlab to Epson s controller. As both were different software, there was no direct link between the two programs. The solution to this was using the fopen function and the fprintf function in Matlab which would write the final position of the ball onto a text file named xy.text. The Epson would then access the text file continuously to obtain the data from it. The main problem with this solution was that sometimes the Matlab code would give an error when the Matlab software and the Epson software both try to access the file at the same time. Experimenting with different

8 pause times between both programs, it was found that the problem was best minimized with a pause of 0.06s in the Epson controller software and a 0.05s pause in the Matlab software. However, the occurrence of the problem was only minimized and not solved entirely as the problem would still happen occasionally. d. Epson Speed and Motion Once the machine vision, estimation and data relay between the programs were solved, there was still a hardware related challenge of moving the Epson fast enough. Experimenting with different speeds and accelerations, it was determined that the best yet safest speed and acceleration was a 75 speed and an acceleration of 25,20. Any higher would be a hazard as the table itself would shake from the torque of the Epson. However, at this maximum speed, it was observed that the Epson would still be unable to intercept the ping pong ball in time. Therefore, the solution to this was to move the camera s field of vision further away from the Epson robot, thus giving it more time to input the final position and move towards it. Doing this helps the Epson robot to be able to intercept the ping pong ball more often but only if it is rolled towards the Epson at a relatively low speed. Also, one of the goals of the project was to make the Epson not just move to the position of the ball but also to hit it back to the user. The problem about this goal is that the ball is always rolled at different speeds towards the Epson. The only way to find out when the ball would reach the Epson would be through a velocity analysis by using the points of the centroids and how much time passes between them to find the velocity. The velocity would then be taken into consideration to find out when the ball would be in its final position and then the Epson could swing at it at the exact moment the ball reaches that point. However, due to limitations of the

9 software, it was decided that even without the velocity analysis included in the code, the Epson was barely reaching the ball in time. Therefore, adding more code especially an intensive one like the velocity analysis would cause the Epson to delay further. The Epson was then programmed to just move forward slightly in a single back and forth swinging motion of its third joint upon reaching the final position in order to hopefully hit the ball back in time. It worked well for the most part but when the rolling of the ball was tested at lower speeds, it was found that the swinging was premature and it would miss the ball completely, ending up just stopping the ball. Achievement/Results The Pong robot achieved all of the tasks it was programmed for consistently when tested in the lab. When both programs are turned on, it would only trigger once the ball appears in the field of vision of the camera. The final estimated position of the ball was consistently accurate for the most part if the ball moves in a straight line; as the final position was estimated using a linear polynomial. Also, when the ball was rolled at a slow enough speed, the Epson robot would always move to the right position, meet it in time and swing at it. However, like mentioned before, the software limitations did not allow for velocity analysis and it would sometimes cause premature swinging of the paddle to the ball if rolled too slowly. Learning Experiences Working through a project such as this allowed members of the group to experiment with the fundamentals taught to us in the classroom and therefore have a firmer grasp on the concepts. It is one thing to work through a set assignment given in lab where there is only one solution and it is completely different when there are infinite solutions when working with a project. The

10 application of data processing in this project definitely allowed the members to learn about how to use the camera for machine vision and how to extract the data wanted from it. As the team worked with real-time data processing, it allowed the members to appreciate the value of efficient coding because a second is all it takes to mess up the workings of the robot. The team also realized that errors may arise from anywhere and the behavior of the code will not always act as predicted. Therefore, iterating and working through the code again and again is the best way to troubleshoot and work through all the problems. With respect to the Epson robot, the team also learned about the syntax of the SPEL+ language used by the Epson controller and how to move the Epson robot through basic commands. On the performance of the Epson, we feel that it is satisfactory and it is performing at its best given the safety constraints and the software limitations. The organization and time allocated for the final project was good. The infrastructure provided was also excellent and the TA was very helpful in guiding us down paths we had not considered before. However, considering the amount of work put in, it would be better if the final project holds more weight in the final grade. We did work many weeks on this and spent many hours on it.

11 Appendix Code for Visual Tracking and Estimation in MatLab function [x y] = Pong vid = videoinput('winvideo',1,'yuy2_160x120'); vid.returnedcolorspace = 'grayscale'; src = getselectedsource(vid); src.framerate = ' '; preview(vid); while 1 == 1 NUM = 0; xy = []; x = []; y = []; p = []; xp = []; c = []; F = []; N = 1; cl = 0; while NUM == 0 [~,NUM] = Pongsnap(vid); end while cl < 3 [F NUM] = Pongsnap(vid); c = regionprops(f,'centroid'); for i = 1:NUM xy(end+1,:) = c.centroid; end [cl,~] = size(xy); end

12 while N < 3 [F NUM] = Pongsnap(vid); c = regionprops(f,'centroid'); for i = 1:NUM xy(end+1,:) = c.centroid; end N = N + 1; end hold on xy = xy(2:end,:); x = xy(:,1)'; y = xy(:,2)'; p = polyfit(y,x,1); % => 28 in, diagonal % => 22 in, x % => 18 in, y => pix/in % 25.5 in to Epson => -(25.5 * )... yep = ; xp = p(1)*yep.^1 + p(2) x = [x xp]; y = [y yep]; plot(x,y,'ro-') fid = fopen('c:\documents and Settings\EPSON RC User\Desktop\Pong\xy.txt','r+'); fprintf(fid,'%f',x(end)); fclose('all'); while NUM == 1 [~,NUM] = Pongsnap(vid); end

13 pause(0.05) clf end function [F NUM] = Pongsnap(vid) pause(0.05) I = getsnapshot(vid); BW = im2bw(i,0.7); imshow(bw) %[B,L] = bwboundaries(bw); [F,NUM] = bwlabel(bw,8);

14 Epson Code for Robot Movement from Given Input EPSON CODE: Function SaveData Integer filenumber Real x Real data(100) filenumber = FreeFile Speed 75 Power High Accel 25, 20 x = data(1) = Do Do While data(1) = x ROpen "C:\Documents and Settings\EPSON RC User\Desktop\Pong\xy.txt" As #filenumber Input #filenumber, data(1) Close #filenumber Wait 0.06 Loop Print "Old value: " Print x x = data(1) Print "New value: " Print x Print "Position: " If data(1) < -50 Then Print "LEFT: OOB"

15 ElseIf data(1) < 14 Then Go P0 Go P11 Go P0 Print 0 ElseIf data(1) < 28 Then Go P1 Go P12 Go P1 Print 1 ElseIf data(1) < 42 Then Go P2 Go P13 Go P2 Print 2 ElseIf data(1) < 56 Then Go P3 Go P14 Go P3 Print 3 ElseIf data(1) < 70 Then Go P4 Go P15 Go P4 Print 4 ElseIf data(1) < 84 Then Go P5 Go P16 Go P5 Print 5

16 ElseIf data(1) < 98 Then Go P6 Go P17 Go P6 Print 6 ElseIf data(1) < 112 Then Go P7 Go P18 Go P7 Print 7 ElseIf data(1) < 126 Then Go P8 Go P19 Go P8 Print 8 ElseIf data(1) < 140 Then Go P9 Go P20 Go P9 Print 9 ElseIf data(1) < 270 Then Go P10 Go P21 Go P10 Print 10 Else Print "RIGHT: OOB" EndIf 'Wait 0.25 'Go P5 Loop Fend