Principles of inertial sensing technology and its applications in IHCI

Transcription

1 Principles of inertial sensing technology and its applications in IHCI Intelligent Human Computer Interaction SS 2011 Gabriele Bleser

2 Motivation I bet you all got in touch with inertial sensors already What can be done with inertial sensors? Think of common devices that you use Smartphones, digital picture frames, mobile PCs: Align image content Fall detection Sports & fitness products: Step count/length, velocity, travelled distance Activity classification, sleep analysis, etc. Game controller: Gesture recognition Motion sensing (control by physical interaction) Pointing Lecture IHCI SS

3 Motivation What do you know about inertial sensors? Accelerometers: Measure linear acceleration (body acceleration + gravitational acceleration) When non accelerating: indicate up down direction Gyroscopes: Measure angular velocity around an instantaneous axis (turning rate) Lecture IHCI SS

4 Motivation: trends in the game industry Yesterday: Classic controller (gamepad, joystick) Button presses, stick control Today: Controller with motion sensing capability (mostly inertial, magnetic and optical sensors) Gestures, motion physical interaction more intuitive Games involving physical interaction (fitness & health) Lecture IHCI SS

5 Motivation: game controllers Wii MotionPlus + Sensor Bar 3D accelerometer: Detect rapid motions Roll and pitch angles 3D gyroscope: 3D turning rate 3D orientation Distinguish body acceleration and acceleration due to gravity Distinguish position and orientation Optical sensor: Detect LED clusters of Sensor Bar pointer, roll angle, distance 3D position (if 3D rotation known) Reminder: insideout tracking Lecture IHCI SS

6 Wii MotionPlus + Sensor Bar Derive roll angle from angle of detected LED clusters Can pitch and yaw be derived from horizontal and vertical shift of the detected LED clusters with respect to principal point alone? Requires information from inertial sensors! Image plane Roll angle Lecture IHCI SS

7 Motivation: game controllers PlayStation Move + Eye Camera 3D accelerometer, 3D gyroscope, 3D compass in the controller: Drift free 3D orientation Compass: drift correction for yaw angle Temporary dead reckoning for position (during occlusion) LED orb + external camera: Detect orb 3D controller position (distance by size of orb in image) Z Reminder: outside in tracking Lecture IHCI SS

8 Outline Until here motivation and a lot of (new) terms: Gyroscope, turning rate Accelerometer, body acceleration vs. gravity Compass, heading direction 3D orientation: yaw, pitch, roll angles Drift (correction) Dead reckoning Now the technologies and principles behind Lecture IHCI SS

9 Outline 1. Inertial sensor basics 2. Inertial measurement units (IMUs) 3. Orientation estimation principles 4. Orientation and position estimation principles 5. Outlook: advanced applications Lecture IHCI SS

10 Inertial sensor basics Why do we call accelerometers and gyroscopes inertial sensors? Their functionalities are based on the principle of inertia, stating the resistance of an object to a change in its state of motion or rest/to be accelerated. Many different types and categories of inertial sensors available Here: micro machined electromechanical systems (MEMS) technology Small size, low weight, low power consumption, etc. But also reduced accuracy and bias stability Lecture IHCI SS

11 Accelerometers Principle (of mechanical type): A spring suspended mass in a housing will be displaced when subjected to a force The displacement is proportional to the specific force and can be measured The output is an electrical signal that by calibration can be related to the physical quantity Lecture IHCI SS

12 Accelerometers Measurement in 1D: Specific force, f, in direction of sensitive axis, n: Sensitive axis Accelerometers measure the difference between body acceleration and gravity acceleration compared to free fall Gravity Acceleration Assuming perfect calibration: What does an accelerometer measure when lying still with the sensitive axis leveled? 9.81 m/s 2 (assuming positive axis points up) What is measured in free fall with sensitive axis leveled? 0 m/s Lecture IHCI SS

13 Linear velocity: Reminder: translational motion Linear acceleration: Position: Holds in 3D with vectors Initial position Lecture IHCI SS

14 Gyroscopes Principle (vibrating mass type): A mass is actuated to vibrate in direction r act and a displacement is measured in perpendicular direction r cor An angular velocity, ω, perpendicular to the plane induces a Coriolis force, which results in a proportional displacement along r cor From this ω can be calculated Measurement in 1D: Angular velocity, ω, around the sensitive axis Lecture IHCI SS

15 Reminder: circular motion Coriolis acceleration: A person moving northward towards the outer edge of a rotating platform must increase the westward speed component (blue arrows) to maintain a northbound course. The acceleration required is the Coriolis acceleration Lecture IHCI SS

16 Angular velocity: Reminder: rotational motion (1D) Rotation: In 3D, a bit more involved (later) Initial orientation Lecture IHCI SS

17 Inertial measurement units (IMUs) Triads of gyroscopes and accelerometers to obtain 3D measurements + compass triad to obtain 3D earth magnetic field Calibrated to provide all measurements in one orthogonal righthanded body coordinate system (typically aligned to housing) in physical units, typically at 100 Hz Commercially available IMUs: Magnetometer Wireless Trivisio Colibri Wireless InertiaCube3 Xsens MTi Lecture IHCI SS

18 Inertial measurement units (IMUs) Typical coordinate system definitions: IMU coordinate system (s) Global reference system (g) Taken from Xsens MTi/MTx User Manual Lecture IHCI SS

19 Inertial measurement units (IMUs) Measurement models in 3D (idealized!): 3D gyroscope [rad/s]: 3D accelerometer [m/s 2 ]: IMU orientation with respect to global frame 3D compass [tesla or gauss]: under no magnetic disturbances, measures magnetic north Lecture IHCI SS

20 Earth magnetic field declination.com Magnetic north Inclination angle Field strength Lecture IHCI SS

21 Accelerometers: acceleration/gravity ambiguity z y y z Ambiguity! Once we know the IMU s rotation, we can separate body acceleration and acceleration due to gravity Lecture IHCI SS

22 Orientation estimation principles Assume perfect measurements and negligible body acceleration: the measurement tells us, where down is: How do we know, whether acceleration is present? The measurement provides the last column of the IMU s rotation matrix roll and pitch angles. The yaw angle (rotation around global z axis) can t be determined Lecture IHCI SS

23 Orientation estimation principles Naive solution under negligible body acceleration: Accelerometer provides negative z axis of global frame in IMU frame yields last column of required IMU rotation matrix Magnetic north How can we use the magnetometer information? Yields y axis of global frame in IMU frame Lecture IHCI SS

24 Orientation estimation principles What about the gyroscopes? Naive solution using gyroscopes: integrate angular velocity measurements to obtain absolute rotation Easy in 1D: integration based on rectangular rule yields: In 3D: Angular velocity vector describing turning rate around instantaneous rotation axis Lecture IHCI SS

25 3D rotational kinematics In 3D: Integration of angular velocity based on rectangular rule: For the derivation of the differential equation and the matrix exponential as required for integration see, e.g., [Woodman 2007, Shuster 1993, ] Relative rotation in axis angle representation resulting from constant angular velocity, ω, over time, δt Rodrigues rotation formula Lecture IHCI SS

26 Orientation estimation principles Bad news: IMU measurements are not perfect! More realistic models including bias and noise terms: Zero mean white noise (typically modelled as Gaussian) Even worse are magnetic disturbances! What does this mean for naive solution based on accelerometers and magnetometers? Jitter and systematic error What does this mean for naive solution based on gyroscopes? Error accumulates over time (drift) Lecture IHCI SS

27 Orientation estimation principles Solution? Sensor fusion! Gyroscopes provide short term indication of rotation (depends on, e.g., bias stability and noise scale, independent of acceleration) Accelerometers provide drift correction for roll and pitch angle during periods of negligible body acceleration Magnetometers provide drift correction for heading direction during periods of no magnetic disturbance Typically, a statistical filter (e.g. extended Kalman filter) is used for fusion [Rehbinder and Hu 2001, Harada et al 2007, ] Bias terms can also be estimated Applications, e.g.: Head tracking for VR (HMD) 3D pointing devices Reminder: improved motion sensing of Wii MotionPlus and PlayStation Move Lecture IHCI SS

28 Orientation and position estimation principles Dead reckoning: Reminder: PlayStation Move What problems do you expect here? Additional references required, e.g. visual information Applications, e.g.: 6 DOF camera tracking for AR Inertial navigation systems (aircrafts, submarines, spacecrafts much better sensors!!!) Lecture IHCI SS

29 Outlook: advanced applications Body motion tracking Pedestrian tracking (NavShoe) X X IMU integrated in shoe to estimate travelled distance Body worn IMU network to capture human motions Lecture IHCI SS

30 References Inertial sensors: O. J. Woodman: An introduction to inertial navigation. Technical Report UCAM CLTR 696, University of Cambridge, Computer Laboratory, Aug D. Titterton and J. Weston: Strapdown Inertial Navigation Technology, American Institute of Aeronautics and Astronautics, 2004 Orientation estimation: T. Harada, T. Mori and T. Sato: Development of a Tiny Orientation Estimation Device to Operate under Motion and Magnetic Disturbance, The International Journal of Robotics Research, 2007, 26, H. Rehbinder and X. Hu: Drift free attitude estimation for accelerated rigid bodies, IEEE International Conference on Robotics and Automation (ICRA), 2001 Rotation representations and rotational kinematics Shuster, M. D.: A Survey of Attitude Representations, The Journal of the Astronautical Sciences, 1993, 41, Lecture IHCI SS

31 We are searching for students in this area! Contact: Gabriele Bleser (Dr. Ing.), Senior Researcher German Research Center for Artificial Intelligence (DFKI) Department Augmented Vision Trippstadter Straße 122, Kaiserslautern E Mail: Gabriele.Bleser@dfki.de Lecture IHCI SS