Designing An Android Sensor Subsystem Pitfalls and Considerations Jen Costillo jen@rebelbot.com
Simple Choices User experience Battery performance 7/15/2012 Costillo- OSCON 2012 2
Established or Innovative Product? Established Will I be making another new product in 6 months? Is the reference design considered good enough for the application? Innovation-Driven Do I have new sensors types? Are features more important than release date? Are money and resources no problem? 7/15/2012 Costillo- OSCON 2012 3
Forsaking Reference Designs 7/15/2012 Costillo- OSCON 2012 4
Going On Your Own If you make your own, You re on your own Integration pains Test time Gesture testing becomes a challenge Calibration blues Larger mechanical footprint But power Control code size Control mechanical footprint In-house expertise 7/15/2012 Costillo- OSCON 2012 5
Android Universe Android Application SensorManager Sensor JNI Sensor Service Sensor Manager Sensor HAL Application Frameworks Libraries Interface Kernel Driver Sensor Driver Linux Kernel Sensor Hub/ Coprocessor Sensors Hardware 7/15/2012 Costillo- OSCON 2012 6
Application Frameworks Libraries HARDWARE Linux Kernel Hardware 7/15/2012 Costillo- OSCON 2012 7
Hardware Architecture 7/15/2012 Costillo- OSCON 2012 8
Sampling Rates: The 3 Rates Under-sampling Inaccurate, sluggish response Slight power savings Over-sampling Accurate, smooth response Power-hungry Sampling Rate 7/15/2012 Costillo- OSCON 2012 9
Wake up events and power considerations Application Processor only Internal Coprocessor External Processor D C Reference supported Most power hungry Reference supported Most work done for you More processor selection More outcome control Most customized Footprint impact 7/15/2012 Costillo- OSCON 2012 10
Hardware Summary Power Consumption = Σ sensors n + any dedicated processor Latency = Max(sensors n ) + dedicated processing time Sensor Solution Use tie-breaker criteria 7/15/2012 Costillo- OSCON 2012 11
Application Frameworks Libraries KERNEL Linux Kernel Hardware 7/15/2012 Costillo- OSCON 2012 12
Kernel Driver Application Processor Peripheral Interface Shared Memory Microcontroller Sensor Coprocessor 7/15/2012 Costillo- OSCON 2012 13
Application Frameworks Libraries LIBRARIES AND SERVICES Linux Kernel Hardware 7/15/2012 Costillo- OSCON 2012 14
Sensor HAL and Services HAL device/<vendor>/<board name>/libsensors Service frameworks/base/services/sensorservice Manager frameworks/base/libs/gui 7/15/2012 Costillo- OSCON 2012 15
Sensor Fusion Libraries Linux Kernel Sensor Hub Sensors http://en.wikipedia.org/wiki/sensor_fusion https://www.llnl.gov/news/newsreleases/2010/nr-10-01-06.html 7/15/2012 Costillo- OSCON 2012 16
Gesture Detection Algorithm Application Processor Android SensorService Sensor Hub Co- Processor Sensors MPU with Gyro/Accel Barometer Proximity Compass 7/15/2012 Costillo- OSCON 2012 17
Gesture Detection Comparison Make Off-load AppPro Buy Minimal Schedule Impact Application Powerful processor In-house expertise Already Tested &tuned Compact code Complete solution Sensor hub Off-load to cheaper power Wake up Event Handling 7/15/2012 Costillo- OSCON 2012 18
Calibration Use of Calibration Gesture in the Compass App By Catch.com 7/15/2012 19 Costillo- OSCON 2012
Application Frameworks Libraries FRAMEWORKS Linux Kernel Hardware 7/15/2012 Costillo- OSCON 2012 20
Virtual Sensors Leverages 1+ physical or other virtual sensors Multiple Options Google (version 3) Reference Vendor Sensor Vendors 7/15/2012 Costillo- OSCON 2012 21
Android Virtual Sensors Accel Gravity Gravity Linear Accel Accel 7/15/2012 Costillo- OSCON 2012 22
Android Virtual Sensors Gyro Accel Orientation Compass Accel Rotation 7/15/2012 Costillo- OSCON 2012 23
Virtual Sensors Challenges Garbage In- Garbage Out Sensor 1 Latency Non-Synchronized samples Implementation Dependencies Sensor 2 Sensor 1 Virtual 1 Virtual Output Multi-vendor problems, verify Vendor ID Sensor 2 Virtual 2 http://www.invensense.com/midc/presentations/james%20lim.pdf 7/15/2012 Costillo- OSCON 2012 24
Application Frameworks Libraries APPLICATIONS Linux Kernel Hardware 7/15/2012 Costillo- OSCON 2012 25
Using Sensors msensormanager = getsystemservice(context.sensor_service); msensormanager.registerlistener( msensorlistener, msensormanager.getdefaultsensor( Sensor.TYPE_ACCELEROMETER), SensorManager.SENSOR_DELAY_NORMAL); void onsensorchanged(sensorevent event) { } // get sensor data float x = event.values[sensormanager.data_x]; 7/15/2012 Costillo- OSCON 2012 26
Using Sensor (Continued) SensorManager.getRotationMatrix( m_rotationmatrix, null, m_mag, m_accels); SensorManager.getOrientation( m_rotationmatrix, m_orientation); float yaw_deg = m_orientation[0] * 57.2957795f; float pitch_deg = m_orientation[1] * 57.2957795f; float roll_deg = m_orientation[2] * 57.2957795f; 7/15/2012 Costillo- OSCON 2012 27
Types of Sensor Problems Bias Drift Settling Time Jitter/Noise Environmental Interference 7/15/2012 Costillo- OSCON 2012 28
Bias Problem: Data is off by a constant value. Sources: static calibration failure Solutions: Calculate linear offset at start of application Recalibrate locally 7/15/2012 Costillo- OSCON 2012 29
Drift Problem: Shift of data without cause Sources: Magnetic interference Poor HW calibration Solutions: Increase smoothing techniques 7/15/2012 Costillo- OSCON 2012 30
Problem: Extended time before finalized steady data. Sources: Latency Sensitivity Solutions: Limit additional processing Settling Time 7/15/2012 Costillo- OSCON 2012 31
Problem: Data jumps around constantly Sources: Sensor Calibration Poor filtering Solutions: High pass filter Linear averaging FFT Noise 7/15/2012 Costillo- OSCON 2012 32
Environmental Interference Problem: Inconsistent results Sources: Magnetometer EMI Solutions: Reference Device Calibration gesture 7/15/2012 Costillo- OSCON 2012 33
Best Practices in Application Development Select the right sensor for the job. Use the Correct Data Rate. UI or GAMING are the most common. Use Sensor In Context Customize for your hardware and system capabilities Magnetometer-based sensors are the most touchy. Keep the Gesture UI simple. 7/15/2012 Costillo- OSCON 2012 34
QUESTIONS? JEN@REBELBOT.COM Additional resources http://processors.wiki.ti.com/index.php/android_sensor_portingguide http://www.kandroid.org/online-pdk/guide/sensors.html http://invensense.com/midc/ http://developer.android.com/reference/android/hardware/sensorevent.html 7/15/2012 Costillo- OSCON 2012 35