Programming Robotic & Automation Applications from Matlab

Programming Robotic & Automation Applications from Matlab Fernando Brandão, João Vaz, J. Norberto Pires Mechanical Engineering Department University of Coimbra Polo II Campus, 33 Coimbra, Portugal brandão, vaz, norberto @robotics.dem.uc.pt Correspondi ng Author: J. Norberto Pires Mechanical Engineering Department University of Coimbra Polo II Campus, 33 Coimbra, Portugal Tel. 239 79712 Fax. 239 7971

Programming Robotic & Automation Applications from Matlab Fernando Brandão, João Vaz, J. Norberto Pires Mechanical Engineering Department University of Coimbra Polo II Campus, 33 Coimbra, Portugal brandão, vaz, norberto @robotics.dem.uc.pt Abstract In this paper an object-oriented and distributed architecture to interface Robotic & Automation (R&A) equipment with a Personal Computer (PC), introduced in [12], is used from Matlab [16]. The above mentioned architecture was designed to work under Win32 environments, using standard visual programming tools (Microsoft Visual C++ and Basic) and mathematical packages (Matlab). Within the paper we focus mainly on using Matlab with the presented architecture. Application examples using an industrial robot are also presented. Keywords : Robotics, Software for Robotics and Automation, Object-oriented programming, Matlab toolbox. 1. Introduction. Current Personal Computers have sufficient computing power that candidate them to be used as host computers in industrial distributed environments, not only for monitoring applications, but also for programming and control applications. Specially they can be used to program and control R&A equipment even when they are organized in Flexible Manufacturing Cells (FMC) [12]. Using a PC is also taking advantage of the enormous variety of applications available on those platforms for programming, data analysis and presentation, etc. Matlab is one of those applications that provide to the user a powerful mathematics environment, very popular for teaching and research tasks on R&A. And that is so because Matlab is a powerful linear algebra tool, with a very good collection of toolboxes that extend its basic functionality. In this paper we present a toolbox that enable access to real R&A equipment from the Matlab shell. If used in conjunction with a robotics toolbox [2][3][4] it will extend significantly their application, i.e., besides robotic simulation and data analysis the user can interact on-line with the equipment he is using. Our personal experience with this tool shows its usefulness for research applications, but also for teaching projects. With students, using Matlab means taking advantage of the less training required to start using it, if we compare with other programming environments and languages we also use (Microsoft Visual C++ or Visual Basic). 2. Software design & interfacing Matlab. Using some equipment from a computer, i.e., exploring the equipment functionality remotely in a distributed way means writing specific code and data structures to handle all its functionality. If we want to be able to distribute the code and use it in several applications in a easy way, the best solution is to build a software component (using ActiveX [5] or JAVA [6]) and integrate it with each project. Fortunately there are many development tools,

Force/Torque Sensor Board most of them visual, that can act as containers of software components built using the above mentio ned technologies. Interfacing Matlab may be done just by using DDE [5]. Since Matlab is an interpreted language and can act as DDE client, it is fairly easy to build a DDE server that can offer to Matlab a collection of services to be acces sed using the Matlab DDE implementation [1][7]. There are a lot of solutions in the market that use DDE to expose services to other applications [14][15]. Another way is by using Matlab MEX functions, embedding into Matlab the software components or libraries used to explore the equipment functionality. Services (client and server sides) can be thought as "options" of a multi-parameter driven function that can be called remotely using some calling and data representation protocols. Implementing service calls using MEX functions is straightforward. In conclusion, the basic idea is simple. For each equipment we need to design and build a server (if it is not yet available) to expose the equipment functionality as remote services. The technology to build the server is highly dependent on the equipment resources and computing facilities, but if possible some kind of RPC (Remote Procedure Calls) [8] mechanism should be used. Software controls that explore these services should then be available as basic tools to develop remote and distributed applications using the selected equipment(s). Finally, the Matlab interface to those equipments is built just by emb edding the above mentioned software controls into MEX functions. That has been done for several type of R&A equipment [12][16]: a robotic manipulator, a force/torque sensor, a microcontroller board, a CCD camera, etc. The all collection of functions constitutes the Matlab toolbox MATROBCOM. Board NT Kernel Driver ActiveX Force/Torque Sensor Object ActiveX Force/Torque Sensor Object Messages OLE OLE DDE Server DDE Callback Function OLE ActiveX Robot Communication Object RPC Server Windows Dialog OLE ActiveX Robot Communication Object Win32 Application Portmaper DDE DDE Services Sync. Answers RPC Calls RPC Calls Async. Msg. Fig. 1 - Basic software architecture [12][16]. Sync. Answers DDE Client Applications Robot Control System RPC Server Programs In this paper we focus only on the interface to an industrial robot (ABB IRB14 using the S4 control system). The interested reader can find the interface to other equipment in [12][16], and download freely this software and other related tools from http://www.dem.uc.pt/norberto/. 3. Interfacing an Industrial Robot. The module briefly presented here (MATABBS4) [16] works with any ABB robot equipped with the S4 controller [9]. It implements calls to all services available from the robot controller (RPC servers). ABB S4 controllers implement RPC server programs [1] with variable access services, file and program management services and system status services. To access those services the host computer (client) must implement RPC calling code through an ethernet or serial connection (both using TCP/IP protocols). ABB also developed an Application Specific Protocol (ASP) called RAP [1] to be used with these services. The RAP messaging protocol works in the same way as MMS specified for MAP networks [11]. To implement the PC-Robot RAPID Aplication

communication code based as mentioned on RPC s using RAP, the facility developed by the Sun Microsystems Open Computing (ONC) Group named SUN RPC 4. ported to Windows NT 4. or later, has been used. User services may be implemented using the basic RPC services, written using the Rapid programming language (available at the robot controller). A simple switch-casedo loop driven by a parameter remotely controlled would do the job. The all list of functions available in the module is presented in Table I. The robot may be connected to the computer using a serial port or preferably an ethernet port, both using TCP/IP protocols. If a local area netwo rk is available, several users/computers may be connected to the robot at the same time (with a line or channel open), and then MATABBS4 keeps track of actual open lines/channels. Opening a line means starting a client connection to the RPC servers running on the robot. In the following we will demonstrate a few capabilities of the module MATABBS4. To open a connection to the robot the command is, >> line = matabbs4 ('open', 'babylon', 'reserved') line = 1 where 'babylon' is the name of the robot as declared in the DNS table or system "hosts" file. Load and start a program, >> matabbs4 ('program_load', 'flp1:\example.prg', line) >> matabbs4 ('motor_on', line) If the robot is not in "AUTO MODE" then, >> matabbs4 ('motor_on', line) Error in 'motor_on' RPC call: Error Code: -1114 ans = -1114. The returned error code means "Invalid System State", and is printed on ABB manuals. >> matabbs4 ('program_prep', line) >> matabbs4 ('program_run', line) To check system state the command is, >> matabbs4 ('sysstate', line)' ans =? Error code ( = OK). 2? Controller State (Stand By). 4? Operational State (Auto Mode). 1? Number of Programs Loaded (1). 4? Program Controller State (Stopped State). 4? Program State (Initiated). 9 3? Space Available on System Memory (%). The user should check system state after any system change state command to confirm system change and to determine when to start commanding the system. For example, if a 'program_run' command is issued the user should wait until the controller state is 'Run State = 4', before starting using the features of the loaded robot program. Checking the system state can be performed in cycle until a condition is met, or by monitoring the RPC messages from the robot. Table I - Functions available in the MATABBS4 module [16]. Function Brief Description open Opens a communication line with a robot (RPC client) close Closes a communicatio n line. motor_on Go to Run State motor_off Go to Standby State prog_stop Stop running program

prog_run prog_load prog_del prog_set_mode prog_get_mode prog_prep pgmstate ctlstate oprstate sysstate ctlvers ctlid robpos read_xxxx read_xdata write_xxx write_xdata digin digout anain anaout Start loaded program Load named program Delete loaded program Set program mode Read actual program mode Prepare Program to Run (Program Counter to begin) Get Program Controller State Get Controller State Get Operational State Get System State Get Controller Version Get Controller ID Get current robot position Read variable of type xxxx (there are calls for each type of variable defined in RAPID [9]) Read user defined variables Write variable of type xxxx (there are calls for each type of variable defined in RAPID [9]) Write user defined variables Read digital input Set digital output Read analog input Set analog output 9.e+9? Position of External Axis 1 9.e+9? Position of External Axis 2 9.e+9? Position of External Axis 3 9.e+9? Position of External Axis 4 9.e+9? Position of External Axis 5 9.e+9? Position of External Axis 6 Note: 9.e+9 means that the external axis is not currently in use. Reading and acting on IO, >> matabbs4 ('digin', 5, line)? to read the value of digital input 5. ans = 1 Signal is ON >> matabbs4 ('digout', 7, 1, line)? to activate the digital output 7. To read current robot position the command is, >> matabbs4 ('robpos', line)' ans =? Error code. 4.6833e+2? Cartesian X position. 2.689e+2? Cartesian Y position. 1.261e+3? Cartesian Z position. 5.2125e -2? Quaternion (q 1 ). -2.35e-1? Quaternion (q2). 4.3565e -1? Quaternion (q 3 ). -8.6733e-1? Quaternion (q 4 ).? Configuration data (ABB CF 1 ).? Configuration data (ABB CF4). 1.e+? Configuration data (ABB CF 6 ).? Configuration data (ABB CF x ). >> matabbs4 ('anain', 2, line)? to read the value of analog input 2. ans = 3453 >> matabbs4 ('anaout', 1, 236, line)? to write the value 236 on analog output 1. Acting on program variables defined in RAPID (see Table I) or user defined variables (XDATA type), >> matabbs4 ('write_num', 'name', value, line)? writing on a type 'num' variable. >> matabbs4 ('read_bool', 'name', line)? reading the state of a 'bool' variable. ans = 1 Variable is TRUE.

Suppose that we have a robot program (written in RAPID [9], a robot programming language from ABB Robotics) running, which is switched by a variable named, let say, 'decision'. The basic structure of the program would be something like (using a C-type definition), while never_end; switch decision case 1: call routine_1; break; case 2: call routine_2; break; case n: call routine_n; break; end_switch; end_while; Suppose again that routine_1 moves the robot from actual position to another one defined by a position variable named 'new_pos' of the type robtarget. Routine_1 should then be something like, PROC routine_1 MoveJ to new_pos; decision = -1; ENDPROC From within Matlab we can then use the MATABBS4 module to command new positions to the robot. Those positions could be defined as a set of joint vectors specifying the joint angles to be achieved. We should then run a forward kinematics function to obtain the positions in cartesian coordinates, with orientation in quaternions and using the ABB configuration definition. For that we can use any forward kinematics function from one of the available robotics toolboxes [2][3][4] and then convert the 4x4 transformation matrices into 'robtarget' structures. In [4] there is a function that does it all, outputting a matrix with one 'robtarget' per matrix line. If we name that matrix 'pos' we could then command it to the robot with the commands, >> matabbs4 ('write_robtarget', 'new_pos', pos[1,:], line) >> matabbs4 ('write_num', 'decision', 1, line) checking between each vector if the last comma nd was already performed, i.e., when the next call outputs -1, >> matabbs4 ('read_num', 'decision', line) The resulting motion would not be continuous but a sequence of "steps". A continuous motion can be easily achieved by redesigning routine_1 to accept a matrix of 'robtargets' and defining zones sufficiently large to avoid stops between each point of the trajectory. We implemented a 'play_trj' service as described to accept trajectories with up to 5 points. Note: With the new versions of RAPID there is also the possibility to command joint vectors directly using the new 'jointtarget' structures. With the new versions, with a slight modification in routine_1 we can start commanding joint vectors directly (the write jointtarget calls were also implemented in MATABBS4). Figure 2 shows a two-finger gripper attached to the robot through a tool changer (from ATI Inc., USA). A force/torque sensor is also included (from JR3 Inc., USA), although it is not used in this example. The gripper is based on a pneumatic module (from Zaytran Inc., USA), having an air pressure regulator to control the grasp force. Currently there is no force feedback, so we use an openloop control approach based on the mapping between the analog input to the air pressure regulator and the force between the fingers. That mapping was obtained by taking a few measures of commanded pressure (digital value to the ADC) and the obtained grasp force, using a F/T sensor inserted between the two fingers. The open loop transfer function was then obtained using the Matlab function 'polyfit'. This means that before commanding the gripper to close, we need to select the required analog

value (actually the digital value to the ADC of the analog board) appropriate to the desired grasp force, Fig. 2 Two finger pneumatic gripper. >>ana_value = pressure_force_map (required_force) ana_value = digital value to command to the ADC A simple pick-and-place operation would be: Open Gripper and approach object, >> matabbs4 ('digout', 1,, line)? OPEN Gripper Move to grasp position and pick object, >>matabbs4 ( digout, 1,, line)? OPEN Gripper >> matabbs4 ( write_robtarget, new_pos, pos_1, line) >>matabbs4 ( write_num, decision, 1, line)? MOVE pos_1 To select the grasp force, >> ana_value = pressure_force_map (5) >> matabbs4 ( anaout, 1, ana_value, line) >> matabbs4 ( digout, 1, 1, line)? PICK object Aproach final position (suppose that from pos_1 to pos_2 there are no obstacles), >> matabbs4 ( write_robtarget, new_pos, pos_2, line) >> matabbs4 ( write_num, decision, 1, line)? MOVE pos_2 Move to final position and place object, >> matabbs4 ( write_robtarget, new_pos, pos_3, line) >> matabbs4 ( write_num, decision, 1, line)? MOVE pos_3 >>matabbs4 ( digout, 1,, line)? PLACE object Move away, >> matabbs4 ( write_robtarget, new_pos, pos_2, line) >> matabbs4 ( write_num, decision, 1, line)? MOVE pos_2. This is a very simple example where we didn't care about selecting the velocity, the position accuracy, the acceleration, etc. All that can be also parameterized remotely before commanding any motion. For example, if we write routine_1 in the form, PROC routine_1 SetAcc n_acc; MoveJ to n_pos, with n_vel & new_prec; SetAcc default_acc; decision = -1; ENDPROC where, 'n_acc', 'n_vel' and 'n_prec' are parameters that we can change remotely and have predefined default values. In this case, the motion to point 3 above, for example, should be commanded as follows, >> matabbs4 ('write_num', 'n_acc', n_acc, line) >> matabbs4 ('write_speeddata', 'n_velc', n_speed, line) >> matabbs4 ('write_zonedata', 'n_prec', n_precision, line)? SET Values >> matabbs4 ('write_robtarget', 'n_pos', pos_3, line) >> matabbs4 ('write_num', 'decision', 1, line)? MOVE pos_3 This pick and place operation was used for playing chess were the robot moves the pieces. The application will be presented in the conference, and uses different grasp forces for each type of chess piece. User plays chess with

the mouse, the robot moves the pieces and confirms moving operations (which updates user screen), and the CCD camera enables final visual verification of players moves. 4. Timing All calls to the robot require between 1 and 15 ms depending on the data structure to write or read, if an ethernet connection is used. If a serial connection is used (192 bps) the access time goes up to around 2 ms. All calls are made with confirmation, i.e., a code is sent back informing about call success or cause of failure. 5. Conclusion. In this short paper we demonstrated one technique to interface R&A equipment with personal computers. The paper focuses on using the Matlab environment to interface the selected equipment. The solution is based on a distributed software architecture that may be used to include support to other type of equipment. In fact other modules were built with it, not presented in the paper, to handle force/torque sensors, CCD cameras, PLCs and microcontrollers. With this type of interface even nontrained users can easily use actual R&A equipment, since Matlab is a fairly simple environment to use. This is particularly true with students, as our personal experience clearly show. The ideas presented here were then applied to an industrial robot and demonstrated with an example were the robot moves the pieces of a chess game, played by two users on different computers. More info about this work including videos may be found in the URL: http://www.dem.uc.pt/norberto/seminarios/ References: [1] Matlab Users Guide, The MathWorks Inc., USA, http://www.mathworks.com. [2] Gourdeau R., "Object Oriented Programming for Robotic Manipulator Simulation", IEEE Robotics and Automation Magazine, 1997;4(3):21-29. [3] Corke PI., "A Robotics Toolbox for Matlab", IEEE Robotics and Automation Magazine, 1996;3(1):24-32. [4] Pires JN. and Sá da Costa JMG., "Ferramentas de Software para Controlar, Programar e Monitorizar um Robô Manipulador Industrial usando Computadores Pessoais", Ibero-American Mechanical Engineering Association Magazine, 1998;2(1):43-67. [5] Box D., "Essential COM", Addison-Wesley, 1998 [6] Camplone M. and Walrath K., The JAVA TM Tutorial, 2 nd Edition, Addison-Wesley, 1998. [7] Matlab API Reference, The MathWorks Inc., USA, http://www.mathworks.com. [8] Bloomer J., "Power Programming with RPC", O'Reilly & Associates, Inc., 1992. [9] ABB IRB14 Users Manual, ABB Flexible Automation, 1997. [1] RAP, Service Protocol Definition, ABB Flexible Automation, 1996. [11] Halsall F., "Data Communications, Computer Networks and Open Systems", Third Edition, Addison-Wesley, 1992. [12] Pires JN, Sá da Costa JMG, Object Oriented and Distributed Approach for Programming Robotic Manufacturing Cells, IFAC Journal on Robotics and Computer Integrated Manufacturing, February 2. [13] Pires JN, Controlo de Força em Robôs Manipuladores Industriais, Ph.D. Thesis, University of Coimbra, June, 1999. [14] Factory Suite 2, Wonderware Inc, USA, http://www.wonderware.com [15] ABB DeskWare, ABB Flexible Automation, http://www.abb.se. [16] Pires JN, Using matlab to interface robotic and automation equipment, IEEE Robotics And Automation Magazine, September 2.