Pattern Recognition Techniques for Cortical Control of a Robotic Arm

Transcription

1 Pattern Recognition Techniques for Cortical Control of a Robotic Arm J. D. Echard 1,M. Burrow 1,J. W. Miao 1,and D. Humphrey 2 1 Georgia Institute of Technology, Bioengineering Research Center Atlanta, Georgia Emory University, Department of Physiology Atlanta, Georgia ABSTRACT This paper describes the pattern recognition techniques used to develop a robotic interface utilizing neural signals recorded from the primary motor cortex area of the brain. The ultimate objective of this research is to provide severely disabled individuals with a natural method of controlling prosthetic devices. The short term goal of this project is to develop a natural neural signal interface that will allow a rhesus monkey to control a robotic arm in real time. INTRODUCTION Many individuals who have been spinalized as the result of an accident retain complete mental functionality. It is hypothesized that the signals for controlling the individual s limbs continue to be generated in the motor cortex areas after spinalization; however, the severed spinal cord prevents motor control signals from reaching the intended muscle group. The goal of this project is to develop an interface to control a robot arm using signals directly from the primary motor cortex (MI) (See Figure 1). Primate Amplifier / Filter 300-5kHz Precentral Motor Cortex Signals Signal Integration (33ms integration) Pattern Recognition Robot Kinematic & Reverse Kinematic Modeling Figure 1. Block Diagram of the Cortical Interface The primate is connected to the interface system via four multi-channel needle electrodes. The multi-channel electrodes have six recording sights per electrode for a total of 24 recording sights. These probes are implanted into the arm/wrist mapped area of the MI and records neural activity while the primate preforms reach and grasp tasks. This information is then used to train

2 and test the cortical interface system. The needle electrodes are used to measure firings from the small group of neurons directly surrounding the recording sites. This population of neuron firings is filtered, full wave rectified, and then integrated over a 33 ms period to correspond with a video camera frame rate. It has been shown in many studies that movement in a direction is determined not by one neuron, but by the summed spike activities of many neural units [1-3]. By accurate placement of electrodes, it is possible to acquire signals that have correlations with muscle groups in the arm/wrist area. Accurate pattern recognition techniques are the key to controlling the robotic arm. The integrated neural data, used as input into the interface, is time variant. Unfortunately, the real time constraint of identifying an arm movement (less than 200 ms) means that only a fraction of the time variant data can be processed before a decision regarding the intended movement must be made. This problem has led to the use of pattern recognition methods using various detection, discrimination and recognition stages. This paper presents methods and results for recognizing simple arm movements. After the neural pattern has been recognized, the data must be further processed to gain information on the approximate velocity of the arm movement. Finally, this information is used in the kinematic model of the robot to produce the desired arm movement. METHODS The Bioengineering Research Center at Georgia Tech is currently using Electromyography (EMG) data to test the accuracy of the pattern recognition system. The discharge of corticospinal and other cells in the MI has been found to be highly correlated (with correlation coefficients up to 94 percent) with the low-passed integrated EMG signals from sets of synergistic muscles [4]. Given these research results, integrated EMG data should have similar characteristics to that of integrated MI data. There are four main parts to the pattern recognition process: These are 1) Detection, 2) Discrimination, 3) Initial Recognition, and 4) Final Recognition. Figure 2 shows a flow diagram of the pattern recognition techniques used. Digital Neurological Input Data DETECTION Threshold Detection INITIAL RECOGNITION Back Propagation Neural Network or Quadratic Distance Decision DISCRIMINATION 5 Previous Outputs FINAL RECOGNITION Compare Results & Solve To Robot Controller Priority Based Figure 2. Signal Processing Flow Diagram Detection is a yes/no decision process; there is either a potential pattern or there is not. For the detection process, a threshold value, which is equal to two times the standard deviation of the background noise, will be used to make a decision. If any one of the recording channels is above the given threshold level, then the detection process indicates that there is a potential pattern. The threshold value will allow the pattern recognition algorithm to reject data when the arm is inactive. The Initial Recognition stage uses either a Back Propagation Neural Network (BPN) or a Quadratic Distance Decision (QDD) program to make a decision regarding the most likely arm

3 movement. The BPN has 8 inputs, 20 hidden layer nodes, and 7 output nodes [5]. The QDD is based on distance calculations using the Bayes Likelihood ratio test assuming Gaussian distribution [6]. These two algorithms were compared because each offers distinct advantages. The BPN does not assume any particular distribution, and therefore is generally more accurate then the QDD approach. However, even if the data is not strictly Gaussian distributed, as long as the classes are fairly well separated in feature space, the QDD can produce a high degree of accuracy. The advantage of the QDD approach is that the computational time required is significantly lower than that for the BPN when it is used in a pattern recognition problem with a large number of inputs or outputs [7]. The discrimination stage uses a channel priority algorithm based on temporal relationships. Since the real time constraints of the system only allow evaluation of about 6 data points before the pattern must be recognized, virtually no temporal relationships are evaluated in the initial recognition stage. The priority based discrimination stage takes this temporal information into account by prioritizing features based on when the feature turns on (goes above the threshold value). Given the current data, this information is then used to reduce the possible arm movements. This discrimination technique was chosen because it uses temporal relationships and it can be computed quickly. The final recognition stage chooses the best arm movement based on the current outputs from the initial recognition stage, the current reduced set from the discrimination stage, and the last five outputs from the final recognition stage. RESULTS The EMG data used for testing has eight channels (recording sites), each corresponding to a differential voltage across major muscles of the arm and wrist. These channels are the pectoral, deltoid, triceps, biceps, wrist extensor (ECU), wrist flexor (FCR), finger extensor (EDC), and finger flexor (FDS). The pattern recognition methods were trained and tested on 7 single joint arm/ wrist movements. These movements are elbow flexion (ef), elbow flexion with co-contraction (ef/co), elbow extension with co-contraction (ee/co), wrist flexion (wf), wrist extension (we), finger flexion (ff), and finger extension (fe). Each one of these arm movements produced values above the threshold level on multiple channels. One set of data was used for training and four sets of data were used to test the two pattern recognition methods (with QDD or BPN). Each data set consisted of samples from all seven movements for a total of 280 time samples of 8 channel data (at 33ms between integrated samples). The BPN was trained for 20,000 iterations using a gradient descent adaptive learning rate with momentum [5]. Data sets 2-5 were used to test the accuracy of the trained pattern recognition methods. The results are show in Figures 3 and 4. The average accuracy for both the BPN and QDD initial pattern recognition stages were approximately 90 percent. On average, the discrimination stage reduced the possible arm movements by 60 percent. When the discrimination and final recognition stages were combined with the initial recognition stage for both the QDD and BPN, the average accuracy was 94 percent. This demonstrates the value of these extra stages in increasing the overall accuracy. Since these extra stages use simple algorithms they consume little computational time. During the course of an arm moment, the active EMG channels start at low values (just above the threshold level), ramp up, and then ramp down. This characteristic trait of integrated EMG data produces substantial error in pattern recognition at the start and ending of each movement. This is due to the fact that when the data point values are small, different arm movement classes are closer together in feature space. The result of this ramp up/down phonomania is that the pattern recognition algorithm s output fluctuates as a particular arm movement begins. As the

4 data values become larger, the output stabilizes to the correct arm movement. Figure 4 shows the average time required for stabilization of different outputs (arm movements). As can be seen in figure 4, all but one of the arm movements is correctly identified under the real time constraint of 200ms. The fluctuation in output at the beginning and end of an arm movent account for approximately 80 percent of the error in the pattern recognition algorithms. Percent Initial Recognition Stage Full Pattern Recognition methods Data2 Data3 Data4 Data5 Time (ms) FE FF WE WF EE/CO EF/CO EF BPN QDD BPN QDD Figure 3. Accuracy BPN QDD Figure 4. Time Required For Correct Identification The results show that these pattern recognition techniques are useful in identifying simple arm movements based on EMG data. Given that integrated EMG data has similar characteristics to that of integrated MI data, these pattern recognition approaches should work equally as well on MI data. Future work on this project will involve the estimation of arm velocity, and relationships between simple and complex arm movements. The exact relationship between the integrated MI data and force exerted by the muscle is not known; therefore, the relationships of arm velocity and combinations of simple movements will have to be evaluated and modeled before full robotic arm control can be achieved. ACNOWLEDGEMENT This project is funded by the National Institute of Health s Neural Prosthesis Program NIH-NINDS-90-17, and was awarded to Dr. Donald R. Humphery of Emory University (as the principal investigator) with Georgia Tech as the subcontractor. REFERENCES [1] Humphrey, D. R., Representation of movements and muscles within the primate precentral motor cortex: Historical and current perspectives. Fed. Proc. 45: [2] Cheney, P.D., Mewes,K. and Fetz, E.E. Encoding of motor parameters by corticomotoneuronal and rubromotoneuronal cells identified by spike triggered averaging in the awake monkey. Behavioral Brain Research 28: (1988) [3] Kandel, E.R., Schwartz, J.H., and Jessell, T.M. Principles of Neural Science, 3rd edt., Elsevier Science Pub. Co., Chapters 1,19 20,35-40., 1991 [4] Fetz, E.E, Finnochio D.V., Baker M.A., and Soso M.J., Sensory comparable passive and active joint movements. J. Neurophysiology 43 (4): [5] Freeman, James, A., and David M. Skapura. Neural Networks - Algorithms, Applications, and Programing Techniques. New York: Addison-Wesley, [6] Fukunaga, Keinosuke. Introduction To Statistical Pattern Recognition. New York: Academic Press, [7] Echard, J., M. Burrow and D. Humphrey, Viability of Quadratic Distance Decisions for use in Pattern Recognition, Proceedings of the 11 th Southern Biomedical Engineering Conference, , Oct

5