Research Robots for Applications in AI, Teleoperation and Entertainment

Research Robots for Applications in AI, Teleoperation and Entertainment S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin Sarcos Research Corporation (SRC), 360 Wakara Way, SLC, Utah 84108 and, Center for Engineering Design (CED), University of Utah, SLT, Utah 84104 Abstract Sarcos Research Corporation, and the Center for Engineering Design at the University of Utah, have long been interested in both the fundamental and the applied aspects of robots and other computationally driven machines. We have produced substantial numbers of systems that function as products for commercial applications, and as advanced research tools specifically designed for experimental use. This paper reviews various aspects of the design and control of a number of robot-like machines ranging from our first projects, the Utah Arm and the Utah/MIT Dextrous Hand, to present work on humanoid robots and the Wearable Energetically Autonomous Robot (WEAR). Our systems have been used in: entertainment, operator remotization from hazardous environments, R&D, and medicine. In addition to the robots and their subsystems, extensive work has been devoted to command systems that drive the robots. Command systems have been: playback supervisors, teleoperation masters, and various higher level approaches based on work from the AI community. Playback interfaces have included motioncapture mechanisms that provide movement-stream information to storage systems configured for later, repeated and coordinated, operation of many robots and associated mechanisms. Play-back command systems use human commands, from an earlier time, to command motions that are played out, over and over, mindlessly. Teleoperation masters, that operate in real-time with the robot, have ranged from simple motion capture devices, to more complex force reflective exoskeletal masters. Teleoperation interfaces have been composed of complex kinematic structures designed to perform motions compatible with operator movements and are attached via appropriate soft tissue interfaces. The masters emit lower level commands (joint angles) in real-time using the natural intelligence and sensory systems of the operator. AI-based command sources, blend higher level (simple) commands, with system and existing environmental states, to make decisions for the management of the robot. As with the playback systems, AI-based

2 S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin systems are programmed earlier to perform later operations. In the AI case, however, adaptive intelligence and sensory capabilities reside in the robot. Our general design approach has been to begin with the definition of desired objective behaviors, rather than the use of available components with their predefined technical specifications. With the technical specifications of the components necessary to achieve the desired behaviors defined, the components are either acquired, or in most cases, developed and built. The control system, which includes the operation of feedback approaches, acting in collaboration with physical machinery, is then defined and implemented. Control is considered a function of both feedback, and the designed-in performance of the robot s physical machinery. It has not been true that bad performance from physical machine elements can be simply compensated out via innovative control methods and faster computers. After the completion of many projects we believe that the final frontier(s) of robotics reside at both ends of the brain and brawn spectrum. Both frontiers (barriers) are related to autonomy - intelligence/computation and energy/power. Recently, energetic autonomy has become a major interest at Sarcos and projects are underway to develop appropriate fuel-based servo-actuators to satisfy that need. Our objective is to develop power systems that are capable producing high performance servo-quality actuation for extended operating times without reenergizing the system. At the other end of the spectrum, we are working in collaboration with various groups to supply physical robots capable of operation under the control of advanced AI-based systems. SYSTEMS DEVELOPED BY SARCOS AND THE CED As shown in Figures 1 to 9, a variety of systems have been designed and manufactured. Examples of products include: The Utah Artificial Arm, Electromyographically controlled (electric)(4 Degrees of Freedom [DOFs] each)(2000 produced); Life-sized humanoid robots for Disney and other clients (hydraulic) (from 5 to 50 DOFs each)(100 produced); Large robotic dinosaurs for the Universal Studios Jurassic Park ride in Hollywood (from 4 to 18 DOFs ea.) (16 produced); Large programmable robotic fountains at the Bellagio Hotel in Las Vegas;

Research Robots for Applications in AI, Teleoperation and Entertainment 3 Fig. 1. Photographs showing, the Utah arm (left), an electric prosthetic limb commanded by Electro-myographic signals and (right) the Utah MIT Dextrous Hand and its master glove. Fig. 2. Humanoid entertainment robot developed for the Carnegie science museum (left) and the Dextrous Arm (DA) and DA Master - a combined 20 degrees of freedom forcereflecting teleoperated system. Fig. 3. Photographs showing T-Rex, one of the 16 life form robots developed by Sarcos for Universal Studios Jurassic Park The Ride, California, and, on the right, one of the 225 vector fountain installed the Bellagio Hotel in Las Vegas.

4 S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin Fig. 4. Photographs showing two humanoid robot systems developed for research in Artificial Intelligence (AI). Left hand side, 34 DOF hydraulic driven humanoid robot. Right hand side, Primus an electrically driven, biomimetic robotic head. Fig. 5. Concept drawing (left), passive structure (center) and single leg (right) of an exoskeleton system currently being developed by Sarcos Research Corporation. Fig. 6. Photograph showing tactile sensors for the UMDH (left) and BiAST TM a bi-axial, multiplexed, digital extensometer.

Research Robots for Applications in AI, Teleoperation and Entertainment 5 Fig. 7. The Morph Hand can be used to grasp delicate objects as well as standard tool. Fig. 8. The SenSuit (left), the GRLA force reflecting 20 DOFs master-slave (center) and the space suit tester developed for NASA (right) Fig. 9. Photographs showing vector fountains developed by SRC during their installation at the Bellagio Hotel (left). Choreographed light and water ballet at the Bellagio (right).

6 S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin Systems have also been designed and fabricated for research purposes, including the: TOPS, force-reflective master and slave robot with hand, arm, torso, & head (hydraulic)(22 DOFs each), Utah/MIT Dextrous Hand slave and Master for studies in machine dexterity (pneumatic)(16 DOFs each), Dextrous Arm and the larger GRLA slave arm with their force reflective Master (hydraulic)(10 DOFs each, including 3 DOFs in each end-effector), Sarcos humanoid robot for AI research and Sarcos Sensuit Master (ATR, Japan)(hydraulic)(39 DOFs each), and the Primus head and eyes with four cameras & lasers (Riken, Japan and USC)(electric)(7 DOFs). Associated body parts have also been developed including: head, eyes, eyelids, ears, neck, faces, mouths, torsos, tails, wings, arms, hands, fingers, legs, feet, and toes. Components developed have included: controllers, actuators, servo-valves, load cells, position sensors, digital sensor networks, tactile sensors, and complex structures. Present efforts are focused on: WEAR, an exoskeletal robot, which is in development along with actuation systems specialized for this project (hydraulic & combustion) with force-reflective, close proximity teleoperational control, a fully instrumented version of WEAR intended for autonomous operation, and an advanced version of Primus, a 9 DOF high performance electronic robotic head. BEHAVIOR-BASED DESIGN (BBD) As mentioned above, it has not been true that bad performance can be simply compensated out via innovative control methods and faster computers. Past attempts to trick energy with algorithms, or use clever mechanisms to achieve one type of performance at the expense of others, have always produced systems with unacceptable behavioral limitations and behaviors. Our general design approach has been to first define the desired objective behaviors in a global way. We then proceed with the specification of the characteristics of the physical components, and related computational elements necessary to implement control functions. These behavioral features, although less familiar in technical environments, are in fact the objective to a sponsor. An approach for the design of robotic actuation systems based upon global behavior objectives was presented by Jacobsen et al. (1987). This method allows the quantitative selection of system parameters, such as motor parameters, gear reduction ratios, and structural stiffness and damping, that is based upon the overall desired robot behaviors such as: structural smoothness, quickness, static force accuracy, load movement grace, and others. The BBD method was investigated for the simple case briefly discussed below and proved to be one of the reasons for our growing lack of confidence in electrical actuation systems for ambulatory robots. Although we did not pursue the approach for systems of higher complexity, we have confidence in the method. We plan to begin working in this area again, and hope that others will also search for better methods to define the required performance of subsystems based upon the desired robot performance. rather than guessing or using successive simulation.

Research Robots for Applications in AI, Teleoperation and Entertainment 7 The process includes five steps: (1) Identifying the desired system behavioral objectives, (2) defining quantitative performance criteria, (3) establishing a minimum complexity system model, (4) determining parameter constraint equations, and (5) applying parameter constraint equations. The specific example in the paper uses second, third, and fifth order linear models of a system with only one external DOF. The approach, available at the time, used Root Locus plots and certain system simplifications to define the characteristics of the sub-systems required to produce behaviors as defined by eleven Quantitative Performance Criteria (QPCs)., as follows: 1. Structural smoothness - absence of structural oscillations, 2. Structure/actuator stability assurance, 3. Static force accuracy, 4. Quickness - inverse of rise time, 5. Positional accuracy, 6. Load movement grace, 7. Strength, 8. Saturation avoidance, 9. Response of the load to insults - carried load, 10. Response of the structure to insults - passive impedance, and 11. Force generation quickness - application of force. A simple demonstration case: a robot with an electric motor connected to a flexible beam with a mass at its ends is used by Jacobsen et al. (1987). The robot was combined with a controller to form the model shown in Fig. 10. The model shows a controller with position and load feedback, a motor, a racktype gear reducer, a compliant arm, and a mass/spring/damper as a load impedance. Additionally, the system is mounted on a base with a certain mass connected to ground by a damper and spring. The base can shake in response to movement of the actuated systems. The model also includes a supplemental mass (touching) that can be connected to alter the impedance seen by the arm. Using those additions, the approach can include additional QPCs such as: 1. Response to shove, 2. Stability margin when touching external loads, 3. Response to base shaking, 4. Performance in operation with a compliant base, 5. Operation with non-collocated sensors, and 6. Intrinsic qualities.

8 S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin Fig. 10. One-dimensional linear effector model. The analysis defines a number of equations that receive QPCs as inputs and which produce relationships between model parameters necessary to achieve desired behaviors. CONTROL APPROACH High performance systems, especially high performance, force reflective teleoperators with many DOFs, require control approaches with numerous capabilities. The following section is a discussion of specific performance capabilities, but not how they were achieved. Ideally, a synthesis approach, such as BBD, would make the process rigorous. However, in reality, we attempt to define the objectives, and then work iteratively, via experimentation, analysis, and simulation in order to achieve a compromise solution. The compromise produces the best overall robot performance that can be achieved given the realities of the hardware, the controller, and the method(s). Our approach considers seven levels of control subdivided into three categories. The categories are: A) variable control, B) intrinsic control, and C) power systems control. Category A - Variable control is subdivided into two levels. - Command Signal Production, and Variable Autonomic Level 1 Command Signal Production (brain-like). Commands in this group can be generated in three ways; a) Playback - The robot is commanded via signals from a pre-programmed, stored database. Commands, which are defined, are generated at an earlier time by an operator using a motion capture master of some sort. No real-time interaction occurs between the robot and the operator.

Research Robots for Applications in AI, Teleoperation and Entertainment 9 b) Tele-operation - Real-time commands are issued by an operator using a force reflective, motion capture, master. Also included are vision systems and other application-specific subsystems. The master sends out commands (joint angles) with low latencies at high frequencies to insure stability. With the proper communication system, the operator can be great distances from the tele-robot. For example, the Utah Dextrous Arm can operate 20,000 feet below the ocean surface with the operator on a shipboard command station. c) Artificial Intelligence (AI) - The AI controller issues real-time commands to the robot. A typical system includes: (i) arrays of externally-looking sensors which assess the state of the environment; (ii) high level commands from a local operator at the beginning of a mission; and (iii) computational elements that formulate strategies for execution of the task by the robot, operating in its environment, with contingencies to deal with unforeseen circumstances. AI systems are typically thought of as capable of operating intelligently, for substantial duration, without continual commands from a human operator. In order to achieve the desired objectives in the presence of realities imposed by the various states of the machine and its environment, AI commands can be: (i) high level from the past (go get the code book); (ii) Real-time changes in strategy; or (iii) Real-time commands from the AI controller. Level 2, Variable Autonomic Control. The controller can adjust system characteristics, but slower than operational speeds and slower than command signals. Level 2 can include various functions such as: (i) self-modeling along with a display of variations in system properties (slow alterations up to failure of a component); (ii) gravity compensation (during orientation variations); (iii) compensation for system variations; (iv) force gain adjustment in teleoperation; (v) management of mechanical impedance and stability during contact with external objects; (vi) insuring stability during loading insults; (vii) coordination of joints; and (viii) trading off properties to achieve quickness versus smoothness during variable speed movements. Category B - Intrinsic control is subdivided into 3 additional levels - (3) Fixed Autonomic Control, (4) Servo Control, and (5) Passive Intrinsic Properties. Categories B and C are designed for maximum simplicity and autonomy with respect to Category A. The system should be operational with Category A systems in a non-functional state. Level 3 - Fixed Autonomic Control, uses fixed parameter settings (via sensing or physical constraints such as stops) for: (i) strength and work space limiting - structures and controls; (ii) gravity compensation; (iii) control of fixed impedance at joints (first the Utah Arm, then the Disney Animatronics); (iv) achievement of grace in movement while minimizing internal structural loads; (v) insuring stability in force reflective teleoperation, and trajectories; (vi) maintaining stability while coordinating multi-jointed compliant robotic structures. Level 4 - Servo Control manages single-joint actuators using PID-like approaches with certain included non-linearities. Levels 2 and 3 might have hooks into Level 4 in order to adjust characteristics such as joint impedance. Level 5 - Passive Intrinsic Properties are a result of inherent characteristics of the machine being controlled. Note that, although ignored by many, passive in-

10 S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin trinsic properties substantially determine ultimate system performance via features such as; kinematics configuration, mass distribution of moving elements, stiffness and damping in structures, stiction and/or backlash in joints, and other factors. Note that in terms of behaviors, damping produced via velocity feedback or physical viscosity produces the same effect. More importantly, poor, designed-in, physical machine properties, especially nonlinear ones, can permanently compromise machine performance. Very often it is impossible to compensate-out inbred properties such as excessive stiction, excessive backlash, excessive mass, poor mass distribution, and others. Category C - Power Systems - is subdivided into 2 additional levels - (6) Actuation, and (7) Energy Storage Systems. These two levels are fundamental determinants of both performance and the potential for energetic autonomy. Level 6 - Actuation systems produce the speed, strength, bandwidth, impedance, stiction, and backlash characteristics required for effective implementation of Level 1 commands. Actuators are also the major consumers of energy and are typically prime determinants of weight. Furthermore, support systems for actuators, such as control hardware and electronics, along with conduits (wiring and power), add to the weight, cost, and complexity of any robot. Level 7 - Energy Storage Systems are the prime determinants of maximum available power usage rates and fundamentally limit the range/duration of operation of any ambulatory, or otherwise energetically autonomous system. Due to the fundamental limitations of electrical energy storage systems, as well as electronic actuators, these systems are inappropriate for practical autonomous, dynamic operation. Hydraulic systems are more suitable if driven by fuel-based hydraulic pumps coupled to hydraulic valves and actuators - but these are not the long term answer either. Therefore, many groups are investigating alternative systems based upon the use of fuels to drive servo level performance actuators. DISCUSSION A review of a number of specific systems is presented below. Specific characteristics are reviewed, such as: application, motivation for the development, system configuration, DOFs, control approach, actuation and energy storage and modulation systems, speed, strength, grace, sensing systems used, and others. Utah Arm The Utah Arm (Fig. 1) is a high performance, battery powered, electric motordriven and electro-myographic (EMG) signal commanded, 3 DOF (elbow, forearm rotation, terminal device open-and-close) upper extremity prosthesis for above-the-elbow amputees and bilateral amputees. Similar to other systems developed by the team from SRC and the CED, the Utah Arm implements hardware-based low level control functions that provide de-

Research Robots for Applications in AI, Teleoperation and Entertainment 11 sired behaviors including: (i) an actively controlled dangling which permits the arm to swing freely while the wearer is walking; and (ii) an activity-type-based, non-linear gain parameter adjustment that improves the performance of the arm during high speed, low precision movements (low impedance), as well as for operations that require high precision but low speed (high impedance). Utah/MIT Dextrous Hand and Master The multi-fingered, high performance, quasi-anthropomorphic, Utah/MIT Dextrous Hand (UMDH) and the Utah Hand Master (UHM) (Fig. 1), were developed to provide the research community with multi-purpose tools for the study of machine dexterity, ranging from the investigation of concepts in manipulation theory and control strategies, to tactile sensing. Inasmuch as it was conceived as a provocation to facilitate physical experiments of previously untested theories on dexterity, the system was designed to have sufficient functional richness to allow the researchers to work without being unduly encumbered by fundamental performance limitations of the system. For this reason the UMDH has many DOFs (16), very high active and passive performance characteristics (comparable to, and in some cases exceeding, human capabilities), was designed to be a vehicle for a broad variety of tactile sensing element designs, and was made modular and reconfigurable to a limited extent. The UMDH itself is configured to be: (i) tendon-actuated (to remotize the actuators and servo-valves); (ii) electro-pneumatically powered; (iii) equipped with four digits (three fingers and one thumb, each having four DOFs); and (iv) quasianthropomorphic (Jacobsen et. al., 1986). It is designed to achieve the speed, strength, and grace of motion (an intrinsic level of control), required to perform dynamic manipulation on a broad range of human scale objects. For instance, in order to perform manipulation tasks on slippery hard or fragile objects, the individual digits have been configured to achieve operational frequencies larger than 60 Hz for the most distal DOFs. In order to permit handling of human scale objects, such as a styrofoam cup, a metal rod, or lubricated surfaces, the UMDH was designed to generate large finger tip forces up to 27 N. The desirable qualities, outlined above, have been achieved by using antagonistic tendons, pulleys with low friction, actuators with low piston mass and stiction (glass cylinder/graphite piston) supplied by high bandwidth, high flow rate, zerostiction servo-valves that allow the actuators to operate as a force source. The Utah Hand Master (Fig. 1) measures the motion of the thumb, index, middle and ring fingers of the operator s hand by means of Hall Effect sensors mounted on a carbon-fiber composite exoskeleton structure which is attached to the surface of a glove. The UHM linkage is designed such that the exact location of the attachment pads does not affect the joint angle measurement; thereby, accommodating a wide variety of hand sizes.

12 S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin Dextrous Arm and Force Reflective Master The Dextrous Arm (DA) Slave and the fully Force Reflective Dextrous Arm Master (DAM) (Fig. 2) have been developed for a broad range of applications where high fidelity dexterity and proprioception must be achieved for conducting operations in environments that are inhospitable to people. For instance, in its original configuration the DA and DAM was developed to perform underwater tasks at a depth up to 20,000 feet (substantially deeper than humans can dive). The system is also well suited for tasks such as: (i) nuclear plant maintenance; (ii) hazardous waste handling; (iii) high voltage live-power line maintenance; and other applications in harsh environments. In summary, the DA and DAM are able to carry out tasks with little or no operator training, and are therefore able to provide certainty in unstructured environments. When used alone the DAM can command and receive signals from virtual slave manipulators operated in synthetic environments. The DA is a human-scaled, ten DOFs manipulator (with three intersecting axes of rotation at each of the shoulder and the wrist, and one axis at the elbow joint) that is kinematically equivalent to a human being (Jacobsen et. al 1991). Its anthropomorphic configuration makes its use totally intuitive for the operator and ensures that its reach space is not very different from its workspace (close to being the same as that of a human). The arm its the end effector are hydraulically powered using low stiction rotary actuators and a supply pressure of approximately 21 MPa. The hydraulic fluid flow to the actuators is modulated by means of very high bandwidth servo-valves developed by SRC (Smith et. al. 1992). These components, combined with high resolution force and position sensing at each joint, make the arm nimble, easy to control, and precise. For instance the arm can be made to free swing, with low damping under the influence of gravity, and can also be used to thread a needle, put a nut on a bolt, pick up an egg, or screw a light bulb into a socket. Other impressive capabilities include: (i) lifting up to 180 N minimum, anywhere in its workspace: (ii) the operator in the master being able to detect as little as a 0.15 N force applied to the slave (the system is far more sensitive than this, but that s what the human can feel when using it); (iii) applying torque of up to 95 N-m at the end effector; and (iv) achieving unloaded joint speeds up to 600 deg./sec at the wrist joints and 450 deg./sec at the elbow and shoulder joints. In addition, its ranges of motion are comparable, and in some cases exceed, those of a human. Several modular three DOF end effectors have been developed and used on the DA. Each of these end effectors was equipped with thumb up-down and side-toside rotation motions, and two fingers that moved like a two jaw gripper. All three DOFs were equipped with force and position sensors. The most interesting end effector developed for the DA was the Morph Hand (Fig. 7), which has a two DOF thumb (again with up-down and side-to-side rotation) and two fingers that can be operated as a parallel jaw gripper. It possessed an additional DOF in the form of reconfigurable finger tips which could be oriented straight out as in a

Research Robots for Applications in AI, Teleoperation and Entertainment 13 standard two jaw gripper, and could also be bent over towards the thumb at a right angle (a hook-like configuration) to facilitate trapping spherical and cylindrical shapes; thereby, minimizing the grip force necessary to effectively handle/grasp a large class of objects. The combination of a dexterous end effector installed on the end of an anthropomorphic arm makes the DA system inherently adapted to work with standard human tools in environments and workspaces originally configured for use by humans. Viewed in the context of full system costs, an arm able to avoid the requirement of custom-designed specialized tooling can result in significant overall savings, in addition to providing superior capabilities. Different control methods including: (i) position and force feedback with PID servo-control loops; (ii) gravity compensation, so that the operator does not have to carry the weight of the slave and master while performing dextrous tasks; (iii) Tap Response which provides enhanced force feedback to the operator when the slave contacts an object; (iv) on-the-fly force reflection ratio scaling capability to adjust the relative strengths of the master and slave consistent with the application requirements; and (v) a novel control strategy that allows the master-slave system to have significantly higher inter-system stiffness and low slew drag, making the system fast and graceful when moving freely, and have high impedance when in contact with fixed objects. Humanoids for Entertainment Sarcos humanoid robots are the most advanced and life-like anthropomorphic robotic figures in the world. Sarcos involvement with humanoid robots began in the 1980 s when Disney wanted to improve upon their Audio-Animatronic robots by making the motions more graceful and realistic. SRC has supplied its humanoid robot to numerous other customers. For instance, to introduce their newly redesigned Taurus in 1995, the Ford Motor Company sought an innovative way to attract and educate customers using high technology. "Sarcos", as the robot was named, traveled North America and Europe for the major car shows from 1995 to 1997. "Sarcos" was operated by two methods: live, real-time teleoperator control, and the playback of pre-programmed skits. During interactive segments, a stand-up comedian in a Sarcos SenSuit controlled the robot. The SenSuit and the exhibit area were equipped with a series of cameras, monitors, microphones, and speakers that allowed the robot perceive and actively interact with Ford spokes-models and visitors to the Ford display. The Sen- Suit was fitted with special helmet-mounted displays, headphones, and a microphone to provide the operator with a robot view and facilitate communication and interactive body movements. SRC humanoid robots can be programmed to recreate smooth, graceful, fast human actions so effectively that they are frequently mistaken for human actors. Under a contract with the Disney organization, SRC and its affiliates provided approximately 90 anthropomorphic robots for European Disneyland, as well as Disney World and Disneyland. These robots are constructed in a modular fashion,

14 S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin typically have between 12 and 52 DOFs, and are actuated using both pneumatic and hydraulic power. They have moving arms, legs, torsos, necks, eyes, and mouths and are capable of a variety of facial expressions. Life Forms for Entertainment Life Forms and Non-Humanoid (Fountains) for Entertainment In 1994, SRC was asked to provide Universal Studios with a collection of sixteen robotic dinosaur figures for the Jurassic Park ride attraction in Los Angeles, California which opened in May, 1996. These animated dinosaur figures are exceptionally realistic in both their appearance and their movements, and were all the more challenging, since they were viewable from all sides in many cases, and they didn t wear clothing. The development of these figures presented difficult engineering and control problems which involved multiple constraints such as: (i) operational seven days per week, year round, in harsh environments (e.g. water spray, direct sun exposure, partial and total submersion); (ii) stringent safety factors; (iii) lots of variety (of the sixteen figures, thirteen were different from the rest, varying in weight from 30 kg to 35,000 kg); (iv) four levels of control (low level force, position, and velocity, figure level, system level, and ride level); and (v) demanding aesthetic requirements. A simulation-based design methodology was implemented from the outset and was used throughout the design phases of the project. A 3D computer model of each figure was constructed, complete with the DOFs that were anticipated. Then these models were programmed with show programs by the customer s creative team using the SenSuit as the human-dinosaur interface (Fig. 8). This approach provided the customer with a tool to work out the look of the ride in a virtual environment with virtual dinosaurs and obtain management buy-off, and at the same time it provided the engineers a reliable, customer-generated motion database (position, velocity, and acceleration versus time, and ROMs) that could then be used to construct the engineering models of the figure designs and calculate the static and dynamic loads applied to the various structures when performing their show programs. In 1996, SRC began work to develop a robotic fountain for the Bellagio Casino in Las Vegas, NV. Each fountain is 3 m tall, weighs 1,400 kg, has four lights that move with the nozzle, and has five controlled DOFs, including: (i) an adjustable water jet height (0 to 27 meters); (ii) a two-axis vectoring gimbal (electrical motor actuated); and (iv) a two DOF pneumatically actuated lift device that can locate the gimbal in day, night, and fully submerged positions. SRC built 225 of the waving fountains and installed them in the summer of 1998. The fountains are in the water 24 hours a day and operate up to 14 hours per day. Choreographed music, light, and the synchronized water ballet, such as the one illustrated in Fig. 9, blend to create a unique show experience.

Research Robots for Applications in AI, Teleoperation and Entertainment 15 Humanoids for R&D and Primus Head SRC has developed two specialized humanoid figures for research purposes. The first is a 39 DOF, pelvis-mounted, hydraulically actuated figure developed for the Kawato Dynamic Brain Project (KDBP), under the direction of the Japan Science and Technology Corporation (JST). The JST/KDBP figure (Fig. 4) is being used in advanced computational algorithm research related to the adaptive learning of system dynamics for model-based control of vision-based tasks, such as juggling, human movement tracking and emulation, and other high level control algorithms. The JST/KDBP figure is pelvis-mounted on a movable base. As a result, unlike the entertainment humanoid figures which are fixed at the feet, the legs of the JST/KDBP figure can emulate walking and running motions. The second research figure is the NASA Space Suit Tester (Fig. 8). NASA s research interests centered on two main areas: (i) what were the force-versusposition characteristics of each of the spacesuit s joints; and (ii) what were the fatigue capabilities of the spacesuits. Both of these had been explored using human operators in the past, but it was difficult to obtain consistent, and reliable results. There was clearly a need for a less subjective, more repeatable method of obtaining the desired information. To address this problem, SRC designed and fabricated a system that was hydraulically (using water-glycol mixture to avoid oil contamination of the very expensive spacesuit in the event of a leak) actuated figure having greater than human strength, and which could go through the human ranges of motion in a controlled, reproducible manner, while sensing the force and position at each joint. Primus (Fig. 4) is an electrically driven, biomimetic robotic head that has human-like performance characteristics. Primus is designed for AI research and applications involving head motion, vision, and hearing. For this purpose, each independent eye has two DOFs: (i) 75 side-to-side; and (ii) 75 up-down. These allow both conjugate and disjunctive eye movements and can operate at up to 26 Hz for small amplitude motions and up to 3.3 Hz for full range of motion. Each eye is also equipped with: (i) separate wide angle and foveal cameras, the outputs of which are combined to emulate the characteristics of the human eye; and (ii) two lasers which add alignment, aiming, and ranging capabilities. The neck has three actuated DOFs that emulate the motions of the human head with ranges of motion of up to 100 at 2 Hz. The Primus had can have its appearance altered through the use of interchangeable head coverings, which can be used to create different personalities. Moreover, the system has been designed to allow the future implementation of: (i) actuated plate-based shell and soft coverings to permit mouth motion and facial expressions; (ii) inertial sensing modules (multi-axis accelerometers and gyroscopes); and (iii) microphones for binaural hearing.

16 S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin Exoskeletal Robot The ability to augment human performance by means of mechanical systems is an on-going quest. Despite much effort, the development of efficient, powered, legged machines that can augment the capability of a human operator and, at the same time, achieve a large range of autonomy, has remained an elusive target. SRC, in collaboration with the CED at the University of Utah, Boston Dynamics Inc., the University of Southern California and Reaction Engineering International, has recently undertaken the development of a series of human-scale Wearable Energetically Autonomous Robots (WEAR) (Fig. 5). The efficient mobility of legged machines over rugged terrain as well as the ability of WEAR to increase the effective stamina, mobility, speed and strength of the user is the basic motivation for this development. These qualities can in turn open the door to a very broad range of applications, such as: (i) carrying heavy payloads or equipment over extended ranges, or for long periods of time, while reducing operator fatigue and risk of injuries; (ii) increasing the operator s mobility while transporting protective shields for heat, flames, radiation, and chemical and biological contaminants; (iii) providing assistance during physical rehabilitation therapy, as well as a new means of locomotion for people suffering from neuromuscular disorders. The technical challenge is substantial. WEAR systems require: (i) compact, robust, lightweight, kinematically and dynamically compatible structures; (ii) suitable interfaces between the machine and the operator s soft tissues as well as haptic interfaces; (iii) new control strategies that minimize the impact of the system on the user s mobility; (iv) the integration of a large number of sensors; (v) and compact, integrated, high power density actuators, that efficiently convert energy from high energy density portable fuel sources, while delivering a high bandwidth servo-controllable output. The general conceptual configuration is illustrated in Fig. 5. WEAR includes: (1) a Base-Unit (BU); and (2) Application-Specific Packages (ASP). The BU is a self-contained, human-compatible powered robot that includes structures, actuators and the energy reserves required for energetically autonomous operation, as well as sensors, structures, and controllers, which can be combined to form an exoskeletal system with various configurations of legs, torso, and arms. The ASP, that is attached to the BU consists of outer coverings tailored for various applications such as those briefly discussed above. The structure is designed to be kinematically compatible with the human skeleton, and the system s mass properties and passive characteristics are set to ensure that the gait of the user is not significantly altered while using the system. Fig. 5 is a photograph of a prototype structure having the desired kinematics. This structure has so far been: (i) used to investigate actuator and sensor placement and conduits routing; (ii) instrumented with joint position sensors and used as a motion capture system (similar to the SenSuit ); (iii) used to experimentally confirm its compatibility with human kinematics and ROMs; and (iv) to identify suitable soft tissue attachment points.

Research Robots for Applications in AI, Teleoperation and Entertainment 17 Work on the kinematics configuration of Full-Body WEAR is on-going, but the first series of fully powered systems will consist of a Lower Extremity (legs and pelvis) WEAR (referred to as LE-WEAR). Figure 5 shows a photograph of a LE-WEAR experimental leg. The leg is equipped with hydraulically actuated knee and hip joints, which allow a peak speed of approximately 500 deg./sec and torque of up to approximately 230 N-m at the knee joint. Position and torque sensors are also used for control. The first complete LE-WEAR system will have 14 DOFs, and will be hydraulically actuated using an external hydraulic supply (tethered system). The system is being designed to allow an operator to perform tasks such as climbing stairs, as well as to walk at speeds in the range of 1.5 to 1.8 m/s on flat and hard terrain while carrying a load of approximately 60 to 70 kg, and, most of all to perform such tasks while minimizing the forces exerted by the payload and the WEAR structure on the operator. Initial tests will be performed on a large scale treadmill developed by Sarcos as a mobility interface for synthetic environments. The second generation of LE-WEAR will weigh approximately 35 kg (not including the fuel), will have performance characteristics that are comparable to those of the first generation tethered system, and, most importantly, will achieve energetic autonomy by using a new type of chemo-hydraulic supply that is currently under development. The new actuators used in the second generation LE- WEAR will achieve unprecedented performance, including, high power density, servo-controllability, and high conversion efficiency, by using: (i) fast energy extraction mechanisms; (ii) high energy density gas or liquid fuels (e.g. propane); and (iii) new system configurations that minimize the number of inefficient energy conversion steps and also allow high bandwidth operation. Preliminary experimental tests and dynamic simulations indicate that: (i) the overall power density is in the range of 150 to 300 W/kg (i.e. including all energy conversion and modulation steps); (ii) the overall conversion efficiencies are on the order of 10 to 20%; (iii) the system will have low quiescent power consumption; and (iv) servo-quality force control will be obtained. Details of the operation and performance of these new actuators will be presented elsewhere. A hierarchical control architecture similar to that described earlier is being developed and will be implemented on the WEAR system. The most basic level starts with the achievement of desired passive characteristics, supplemented by low level PID position-force control (with non-linear and model-based inclusions) to manage individual joint actuators and supplies. Other layers of control will include: (i) distributed non-linear Get-Out-of-the-Way control aimed at minimizing the contact force between the human and the machine and to produced the desired force amplification; (ii) gravity compensation, and fault recognition; (iii) adaptive control for joint-coordination and gait emulation, as well as for compensation of operator and system variations.

18 S.C. Jacobsen, M. Olivier, F.M. Smith, D.F. Knutti, R.T. Johnson, G.E. Colvin, and W.B. Scroggin Sensor Systems While developing the various sensor-intensive, servo-controlled systems discussed in the previous sections, the SRC team encountered a recurring problem: that sensors and wiring harnesses have high costs and poor reliability. The entertainment robotics projects alone have used over 4,000 sensors and 2,500 actuators. In these systems, the sensors have accounted for up to 30% of the system cost, and up to half of the system failures have occurred in the supporting wiring harnesses and connectors. In order to address these problems, SRC has, since the early 1980 s, been working on the development of a series of high resolution, absolute, digital multiplexed output, compact, robust, high bandwidth, low cost sensors. These efforts have resulted in a new generation of strain, rotation, multi-axis strain, load, shear stress, pressure, and other types of sensors which are ideally suited for high performance robots and other information-driven machines that move. Details have been presented in (Jacobsen et. al. 1998). Acknowledgements Support for the work described in this paper has been provided by, Defense Advanced Research Projects Agency, contracts F33615-82-K5125, DABT63-98- C-0048, and MDA972-01-C-0023, the Office of Naval Research contract N00014-82-K-0367, the Department of the U.S. Navy, the National Institute of Health, the Department of Veterans Affairs Rehabilitation R&D, Westinghouse Idaho Nuclear Company (Department of Energy funding), Bell laboratories, Universal Studios, the Bellagio Hotel and Casino, Carnegie Science Museum, Ford motor Corporation, as well as ATR and Riken Japan. References Jacobsen SC, Iversen EK, Knutti DF, Johnson RT, Biggers KB (1986) Design of the Utah/M.I.T. dextrous hand. In Proceedings of the 1986 IEEE International Conference on Robotics and Automation, April 7-10, San Francisco, California, pp 1520-1534 Jacbosen SC, Smith CC, Biggers KB, Iversen EK (1987) Behavior based design of robot effectors. In Proceedings of the 4 th International Symposium on Robotics Research, Santa Cruz, California, August 10-15, pp 41-55 Jacobsen SC, Smith FM, Backman DK (1991) High performance, dextrous telerobotic manipulator with force reflection. In Proceedings of the Intervention/ROV '91 Conference, Hollywood, Florida, May 21-23, pp 213-218 Jacobsen SC, Olivier M, Maclean BJ, Mladejovsky MG, Whitaker MR (1998) Multiregime integrated transducer networks. In Proceedings of the Solid-State Sensor and Actuator Workshop, Hilton Head Island, South Carolina, June 8-11, pp 136-143

Research Robots for Applications in AI, Teleoperation and Entertainment 19 Smith F, Jacobsen S, Potter D, Davis C (1992) Miniature high performance servo-valves. In Proceedings of the International Fluid Power Exposition and Technical Conference, Chicago, Illinois, March 24-26.