Introducing Computer Vision and Assertive technologies from the new Imaging and Vision Group

Transcription

1 Introducing Computer Vision and Assertive technologies from the new Imaging and Vision Group Anand Patel Senior Product Manager, Media Processing Group ARM Tech Symposia India December 7 th 2016

2 ARM acquired Apical a global leader in HDR, Display, Camera and Vision technologies 2

3 Imaging is everywhere Experience Intelligence 3

4 ARM s new additions to the Mali Multimedia Suite Delivering user experience Extracting intelligence from vision Assertive Display ARM Computer Vision Mali Camera 4

5 Assertive Display 5

6 Assertive Display Multimedia can be viewed even in bright sunlight Display power can be reduced by up to 50% Automated re-mastering for HDR content to be viewed on mobile displays 6

7 Assertive Display Saves power and allows you to see screens under bright sunlight Without Assertive Display in bright sunny conditions With Assertive Display under same conditions 7

8 HDR content viewed on new ultra bright HDR TVs HDR content viewed on standard mobile display: With Assertive Display - HDR Without Assertive Display - HDR 8

9 Mali Camera 9

10 What is an ISP (Image Signal Processor) light pixels ISP pixels Still encode Video encode Display preview Inside camera module Inside applications processor (SOC) The Image Signal Processor (ISP) processes raw image data into great looking images The ISP algorithms greatly influence picture quality 10 10

11 ARM Image Signal Processor (Mali Camera) Features Lens shading (vignetting) correction (mesh and radial) and Geometric Distortion Correction engine Advanced spatial (2D) noise reduction (sinter ) Motion-adaptive temporal (3D) noise reduction (temper ) Multi-exposure HDR image fusion Space-variant HDR processing (iridix ) Performance Lens / Image sensor MIPI CSI-2 Receiver Host CPU ARM ISP firmware ARM ISP SRAM: Line mems, LUTs Imaging SOC Mali V-550 Still/Video Encode >500Mpixel/sec 1080p60/4K video 256Mpixel sensor support Deliverables DRAM Controller CoreLink Interconnect Display Controller Hardware IP Firmware Tuning software and support DRAM Display 11 11

12 ARM a world leader in HDR Leading mobile handset with video HDR function is based on ARM s HW technology ARM s HDR solution leads the market by a significant margin Supports leading industry standard HDR formats HDR10 and Hybrid Log-Gamma capture support 12 Camera without ARM HDR Camera with ARM HDR (same image sensor)

13 Complete GDC engine Geometric Distortion Correction engine Full correction for fisheye or lens effects Ideal for automotive applications mirror replacement and bird s-eye view 13

14 ARM Computer Vision 14

15 ARM Computer Vision On-device hardware accelerator of vision tasks Capable of detecting an unlimited number of objects in real-time HD video* High power-efficiency and low silicon area Suitable for battery-powered, portable devices Licensed by multiple SoC vendors for next-generation smart cameras and smart sensors 15 *Subject to the minimum resolvable object size

16 ARM Computer Vision enabling new applications With a low-power object detection engine you can: Smart crop & zoom Automatic capture of still images during video App level (user space) Smart continuous Auto Focus & Auto Exposure 3A FW Real-time video indexing during capture Region-of-Interest based video encoding CMOS sensor ARM CV Core Mali Camera Still capture Video Crop/scaler HEVC encoder MP4 movie file ARM CV metadata 16

17 Automatic scene composition during video capture Enabled by ARM Computer Vision 8MP video capture, rescaled to 1080p for video encode Automatic continuous crop & rescale from 8MP to 1080p for Region of Interest (ROI) 17

18 Smart crop & zoom example Smoothly transition from camera-based stabilization to object-based stabilization Video available at:

19 19

20 Better together 20

21 Cross IP integration Assertive Mali Camera Imaging (ISP) Mali Video Mali Display Assertive Display Vision CoreLink Interconnect Mali Graphics Cortex CPU Cortex CPU 21

22 Combining the best technologies for VR, AR and CV Virtual Reality (VR) Augmented Reality (AR) Computer Vision (CV) ARM Cortex-A processors - Cortex-A73 for premium smartphones - Cortex-A57 and Cortex-A35 for entry-level devices ARM Mali Graphics Processors (GPU) - Ten years of integration with Cortex-A technology Mali Camera + Assertive Camera - Improves images in low-light conditions n/a Mali Video Processors - Encode and decode HD video at up to 120fps Mali Display Technology - Already integrates with Assertive Display n/a Assertive Display - Needed for any display in different lighting conditions n/a ARM Computer Vision - Provides PPA-leading hardware acceleration of vision tasks n/a 22

23 Delivering an end-to-end HDR experience on mobile HDR Capture HDR Encode HDR Decode HDR Display Mali Camera with Mali GPU Mali Video Encode Mali Video Decode Mali Display Processor with Assertive Display 23

24 The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited (or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners. Copyright 2016 ARM Limited