Optical Sensing: 1D to 3D using Time-of-Flight Technology. Shaping the Future of MEMS & Sensors September 10, 2013

Transcription

1 Optical Sensing: 1D to 3D using Time-of-Flight Technology Shaping the Future of MEMS & Sensors September 10, 2013

2 Agenda 2 Presentation Marc Drader Optical Sensors Intro Time-of-Flight Technology ToF for 1D ranging ToF for 3D gestures Next Steps

3 STMicroelectronics Imaging Division 3 Camera Modules Image Sensors Imaging Processors Photonic Sensors Fixed focus camera Wafer Level reflowable camera EDoF camera Auto-focus camera Innovative optics, assembly & test technologies Production from 1.4um to 5.6um pixel 1.1um development From VGA to 24Mpix Stand alone ISP Full ST video pipe IP Integration of third party IP on demand User detection, Proximity, ALS, Optical navigation, Man Machine Interface, Automotive, Medical

4 Brief Overview: Optical Sensors 4 May seem obvious but Optical path considerations Transmission spectrum Transmission path efficiency Field of view Target object characteristics System considerations Optical crosstalk! Ambient or background illumination (noise)

5 Proximity Detection 5 Conventional IR sensor Attempt to detect whether object/user is near or far, based on reflected signal amplitude impossible to know object DISTANCE 2 Unknowns Target Distance Target Reflectance Near 1 Output Amplitude Signal & Noise Far Noise 90% 17% 3% Threshold setting will detect user anywhere from 0.5 to 6cm

6 What is Time-of-Flight Sensing? 6 Active Illumination system: 1. Emit light (photons) towards a target 2. Light (partially) reflects from the target 3. Sensor determines when light (photons) arrive Photon travel time multiplied by speed of light = distance 1cm = 66ps round-trip travel time at the speed of light Target object Photon travel time NOT affected by target reflectance Photon(s) Distance Sensor Emitter Single photon travel time

7 Available on YouTube ToF Illustration (video) 7

8 Motivation for ToF 8 Real-world application/need Best example: Smartphone proximity sensor detects user s head during a phone call; shuts off touchscreen & display But it doesn t work 100% of the time Search: face hang-up + any smartphone brand, to find frustrated users whose touchscreen did not shut off before their cheek pressed a button Time-of-Flight technology adds value by providing true, accurate distance measurements Independent of target object reflectance Immune to ambient illumination & optical path variations (glass, plastic cover)

9 Motivation for ToF 9 Dark hair Small displacements ST ToF distance measurement Threshold Reflected power (Conventional PS (*) ) Far (screen on) Near (screen off) Skin Threshold Near (screen off) Far (screen on) Conventional IR sensor bouncing between far and near states vs robust ToF solution Photon travel time NOT affected by the object reflectance (*) Reflected power information also available on ST ToF Proximity Module

10 Time-of-Flight Physics 10 Emit and receive photons :.follow Poisson distribution may be correlated or uncorrelated (ambient, dark current) to emitter Finally photon arrival rate does depend on object reflectance & distance Delay we want to measure = returned photons (correlated) Emitted pulse = ambient photons (uncorrelated) Received pulse (delay=distance) Many repeated pulses required for correlation

11 Single Photon Avalanche Diode 11 SPAD digital output used to: Count arrival of single photons and/or Time arrival of single photons Unique Properties each photon provides valuable time/distance info Fully Integrated in CMOS

12 More Time-of-Flight Challenges 12 Optical constraints Coverglass contributes optical crosstalk (shortcut from emitter to sensor) Ambient light is main contributor of uncorrelated photons Co-existence of visible & NIR systems for ALS & ranging Conflicting wavelength and field-of-view requirements Ambient light Sensor field of view IR emission Phone Window Airgap

13 Optical Crosstalk 13 Time-of-Flight Sensor always sees two targets: Product-level cover (glass/plastic) fixed distance, and (relatively) fixed optical characteristics distorts reflected signal in both time & amplitude domain Target object varying distance and optical characteristics Target object Photon Sensor Emitter FlightSense technology compensates for optical crosstalk automatically Opens up use cases in very challenging optical environments

14 Crosstalk Compensation 14 Compensation algorithm Firmware uses known crosstalk characteristics to correct the time-domain measure Simple register write (absolute value of photons from emitter to sensor coupled through phone housing) Raw range results (no compensation applied) Crosstalk compensation applied (register setting)

15 Ambient Immunity 15 System performance Keep ambient photons out Optical filtering (notch around 850nm) Reject remaining ambient photons Time-domain rejection System-level noise management SNR limit FlightSense technology will NOT report false distance in high ambient light conditions

16 Ranging Conditions 16 Ranging specifications 0 to 100mm, 3% to 90% reflectance 0 to 250mm for a 45% reflectance target (i.e.. Human hand) Eye-safe, low power IR (850nm) emitter Accuracy: σ = 3mm (resolution = 1mm steps) FlightSense architecture allows zero mm measurement * 0mm defined at the product/system-level There must be an available emittersensor optical path!

17 Simple Optical Module 17 Simple (reflowable) package Small size (2.8 x 4.8 x 1.0mm) Integrated emitter/sensor & optics/filters Opposing requirements Proximity: near infra-red wavelengths, narrow field of view Ambient Light Sensor: visible wavelengths, wide field of view Device delivered full calibrated Simple electrical integration Single power supply (2.8V) I2C & GPIO (1.8V or 2.8V) Programmable I²C address Flexible window & threshold interrupts

18 VL6180X Ranging Performance 18 10x measurements per chart, 10mm step, in the dark, 0.2mm air-gap, no gasket, Oval artwork (75%>800nm) Ranging performance is is independent independent of target reflectance/color of target reflectance/color Distance standard deviation < 3mm Reflective charts (in %):

19 VL6180X Convergence Time Distance (mm) Convergence vs Convergence Time Time (us) Target reflectance from 3% to 88% Convergence Time (us) Target Object Distance (mm) Reflective charts (in %): 3% 5% 17% 88%

20 Low Power Consumption 20 Real-world current consumption Varies with object distance & reflectance Max consumption set by user Peak current consumption Conv. time Conv. time Examples (2.8V supply) 10Hz ranging, object 5cm 88% (white): 40µA 18% (grey): 200µA 5% (black): 550uA 3% (deep black): 760uA 1Hz ALS, 100ms integration ALS: 32uA average Low standby current HW standby <1uA SW standby <7uA Average Current (ma) Average Current Consumption 10Hz repetition rate 88% 17% 5% 3% Target Object Distance (mm)

21 New FlightSense VL6180X Sensor 21 Disruptive Time-of-Flight Technology 6 years of R&D Key patents for innovative sensor/system architecture Differentiating, unique technology Manufactured in ST s custom process Fast, Accurate Distance Ranging Independent of object reflectance (color) Ambient rejection (sunlight, etc) Phone window crosstalk compensation Enable creative use cases (1D gesture application) High-sensitivity ALS Invisible for Industrial Design Ultra-wide dynamic range Calibrated output value in Lux Simplified Integration & Manufacturing Small reflowable module with embedded light emitter No additional optics or gasket No phone-to-phone calibration required Robust to phone glass manufacturing dispersion Robust to phone drop / minimize field return Robust supply chain with dual sourcing strategy Production in H1 2014

22 Product Readiness 22 Mass Production in H1/2014 CP code : VL6180XV0NR/1 Not just for mobile phone applications Robust proximity detection Consumer robotics Gaming And much more! SPAD/ToF potential applications are endless Check out our page on ST.com

23 ToF for Gestures: Motivation 23 Multiple outputs eliminate ambiguity for gesture detection Up/Down Distance Amplitude Swipe Distance Amplitude

24 Multiple ToF sensor 24 Reduce ambiguity with more info: Order Position 1 device gestures capabilities : 2 devices gestures capabilities :

25 3D Gesture Detection 25 High potential for differentiation Setting Expectations Unlike a touchscreen No touch or release Detecting user intention more difficult No physical boundary Live interaction vs post-processed result Optical 3D gestures can complement existing systems Off screen/over-screen sensing volume New uses cases Wakeup or UI response as user approaches Hands-free interaction (many ideas) Gaming controller

26 1D Gesture Detection 26 Time of Flight IR Time domain Distance Amplitude Signal & Noise 2 Outputs Object Properties Distance Reflectance Surface % fill factor Multiple objects 2 Unknowns Distance measurement alone Tap / double-tap Up/down level control IF we assume fixed object reflectance Lateral motion can be estimated from a single pixel!

27 Lateral Motion Estimation 27 Assume Surface properties are stable (same object) within a given time period only change must be due to % filled FOV (represents x-y motion) We can therefore calculate the % of the Field of View filled by the object Independent of object distance! 20% 80% 100% 80% 20% Amplitude can be normalized (using distance info)

28 % Field-of-View Coverage 28 Object reflectance modeled at all distances Model needs to include non-linearities Sensor saturation Emitter blocked

29 3D Gesture Detection 2 ToF Pixels 29 Spatially or angularly separated ToF detectors Linear continuous slider Smooth triangulation of position/speed in X (horizontal) and Z (vertical)

30 Gesture Definitions 4 ToF Pixels 30 Continuous control hover & tilt Hover & tilt Great gameplay Can act as mouse/track pad Post-processed movement examples Swipe Press Wave

31 More Gesture Definitions 31 Movement properties that can be detected Motion lateral/angular speed Object width Closed vs spread fingers Hand-tilt detection Goal is for ROBUST detection of gestures/motion Flat hand swipe Tilted hand swipe 4 fingers swipe Tilted hand swipe

32 What s Next? 32 3D Gestures is a wide field more to come!

33 Q&A 33