Sensor Fusion and its Applications in Portable Devices. Jay Esfandyari MEMS Product Marketing Manager STMicroelectronics

Transcription

1 Sensor Fusion and its Applications in Portable Devices Jay Esfandyari MEMS Product Marketing Manager STMicroelectronics

2 Outline What is Sensor Fusion? What Are the Components of Sensor Fusion? How Does Sensor Fusion Work? What Problems Does Sensor Fusion Solve? What are the applications of Sensor Fusion? Summary 2

3 Sensor fusion What is Sensor Fusion? is a set of digital filtering algorithms to make multiple MEMS sensors work together as a team. takes advantage of each sensor s capabilities to optimize the performance and deliver accurate and responsive outputs. can include a 3-axis accelerometer, a 3-axis gyroscope, a 3-axis magnetometer and a pressure sensor. can be simple or complex depending on how many sensors are used and how the sensor fusion algorithm is implemented. 3

4 What are the Components of Sensor Fusion? To satisfy the requirements for most applications, a 9-axis sensor system is required. 9-axis sensor system a 3-axis magnetometer a 3-axis accelerometer a 3-axis gyroscope Outputs Quaternion Heading, Pitch, & Roll Pointer hpos, vpos Linear Acceleration Gravity 4

5 How does Sensor Fusion work? Sensor fusion creates an accurate orientation solution by taking advantage of the benefits of each sensor: The magnetometer tracks the direction of the Earth s magnetic field, The accelerometer tracks acceleration and the direction of gravity, The gyroscope tracks dynamic angular rotation to determine yaw (heading), pitch and roll angles, while eliminating their respective weaknesses. 5

6 Issues with 6-axis Sensor Solutions? Key motion sensing performance issues of current 6-axis systems: 6-axis systems (accelerometer + gyroscope) lose absolute orientation as the gyro drifts over time. 6-axis systems (accelerometer + magnetometer) are prone to data corruption in the presence of external magnetic field sources. 6

7 Problems Solved with the Sensor Fusion? Sensor Fusion solution with accelerometers, gyros and magnetometers (9-axis systems) uses algorithms to fuse the sensor data to: Eliminate the drift that occurs with a stand-alone gyroscope. Eliminate the magnetic interference of the magnetometer, which is susceptible to external magnetic field. 7

8 What Are Some of the Applications of Sensor Fusion? Augmented Reality Pedestrian Dead Reckoning Motion Gaming in 3-D Space TV Remote Control (Air Mouse Pointing Device) Human Body Tracking

9 9 Sensor Fusion Enables New Applications Optical Image Stabilization Point Of Interest Location Based Services Enhanced User Interface & On Line Gaming Augmented Reality

10 Augmented Reality Augmented Reality (AR): Real world environment with superimposed graphics, audio and other sense enhancements in real time. Purpose: LBS, science fiction on smart phones 10

11 Pedestrian Dead Reckoning Pedestrian Dead Reckoning (PDR) needs: Distance travelled: Using pedometer solution Heading information: Using tiltcompensated E- Compass Purpose: pedestrian indoor/outdoor navigation when GPS signal is not available Barometric pressure sensor can be added to detect the floor change inside buildings. 11

12 Motion Gaming in 3D Space Motion gaming requires precise and responsive capture of player s body motion. Sensor Fusion eliminates the need for video camera, etc. Purpose: Natural and responsive feeling while playing games Example: Motion gaming can be used in smart phones, PDAs, PCs and TVs. 12

13 Motion controlled and gesture recognition based TV remote control is a new application for sensor fusion. Sensor fusion algorithms can be embedded into a single IC to reduce the size and cost for TV remotes and air mouse pointing devices. Gesture recognition, which is more natural, can eliminate the need for some of buttons on the remote control. TV Remote Control 13

14 Human Body Tracking Human body tracking using sensor fusion: fall-detection of senior people golf swing motion capture sports collision detection Sensors can be used to track human body motion: head movement arm or leg movement body jump twist motion, etc. Flexible Lens for Eye Pressure Monitoring Insulin Nano Pump Pressure Sensor Electro Cardiogram Temperature Sensor Movement Recognition For example: This can be used to model the gait of a person for dead reckoning development. Step Counter 14

15 Human Body Tracking Human body tracking using sensor fusion: fall-detection of senior people golf swing motion capture sports collision detection Sensors can be used to track human body motion: head movement arm or leg movement body jump twist motion, etc. For example: This can be used to model the gait of a person for dead reckoning development. 15

16 Summary Sensor Fusion consists of multiple sensors and uses filtering algorithms to take advantage of each sensor s benefits and eliminates / reduces the weaknesses of the sensor system. A 9-axis sensor fusion solution can meet the requirements of most applications. Sensor fusion solutions enable new applications in areas of entertainment, healthcare, fitness and sports. 16