Embedded Vision on FPGAs The MathWorks, Inc. 1

Transcription

1 Embedded Vision on FPGAs 2015 The MathWorks, Inc. 1

2 Enhanced Edge Detection in MATLAB Test bench Read Image from File Add noise Frame To Pixel Median Filter Edge Detect Pixel To Frame Video Display Design 2

3 Video Processing and Computer Vision Video Processing Ø Video in and out Ø Gamma correction Ø Color balancing Ø Noise removal Ø Image sharpening Computer Vision Ø Feature matching, and extraction Ø Object detection and recognition Ø Object Tracking and motion estimation Ø Dynamic resolution scaling Ø Focus assessment 3

4 A Workflow for Video Image Processing Concept Development Algorithm Development Prototyping Architecture design Prototyping Chip design Frame based Pixel based Image/Video Engineer HW Engineer 4

5 Vision HDL Toolbox, R2015a New Product Customer Problems Prototyping video and image processing algorithms on FPGA/SoC Our Solution Pixel-streaming algorithms for the design and implementation of vision systems on FPGAs and ASICs A diverse set of frame sizes and frame rates Product Detail Price: 5000/$5500 Platforms: MATLAB and Simulink Dependency: MATLAB for modeling, HDL Coder for code generation 5

6 Vision HDL Toolbox, Key Benefits Modeling bit-true and cycle-accurate behavior of video image processing algorithms Pixel-based functions and blocks Optimized for HDL code generation Conversion between frame and pixel Standard and custom frame sizes and frame rates Prototyping video and image processing algorithms on FPGA/SoC (With HDL Coder) Efficient and readable HDL code (With HDL Verifier) FPGA-in-the-loop testing and acceleration 6

7 Pixel Based Video Image Algorithms Analysis & Enhancement Edge Detection, Median Filter Conversions Chroma Resampling, Color-Space Converter Demosaic Interpolator, Gamma Corrector, Look-up Table Statistics Histogram Image Statistics I/O Interfaces Frame to Pixels, Pixels to Frame, FIL versions Filter Utilities Image Filter, Median Filter Pixel Control Bus Creator Morphological Operations Pixel Control Bus Selector Dilation, Erosion, Opening, Closing 7

8 A broader solution for embedded vision Product Vision HDL Toolbox ( VHT ) HDL Coder HDL Verifier Computer Vision System Toolbox ( CVST ) Image Processing Toolbox Capabilities Design and simulate image processing, video, and computer vision systems for FPGAs and ASICs Provide code generation capability for the functions and blocks in Vision HDL Toolbox Provide FPGA-in-the-loop capability for Vision HDL Toolbox Provide frame based computer vision functions and blocks as well as image and video I/O capability Provide image processing and analysis functions 8

9 CVST and VHT, complimentary products Scopes Vision algorithms Ø Feature matching, and extraction Ø Object detection and recognition Ø Object Tracking and motion estimation Ø Dynamic resolution scaling Ø Focus assessment CVST 9

10 CVST and VHT, complimentary products Scopes Camera pipeline Ø Video in and out Ø Gamma correction Ø Color balancing Ø Noise removal Ø Image sharpening 10

11 CVST and VHT, complimentary products Scopes image image information Camera Pipeline Vision Algorithm CVST VHT 1.0 VHT future releases 11

12 CVST and VHT, complementary products Product details VHT Image Processing Pixel based Frame based Workflow System prototyping Algorithm design CVST Algorithms Image filtering Color space conversion Edge detection Statistics & histogram Morphological operations (Most of the VHT algorithms, plus) Object detection & tracking Feature extraction and matching Stereo vision Camera calibration Image registration I/O Frame-to-pixel Pixel-to-frame FIL File I/O Video display Code gen HDL (with HDL Coder) C (with ML Coder or SL Coder) 12

13 CVST and VHT, complementary products MBD workflow for embedded vision MBD workflow 1. Build behavior model with CVST blocks and simulate 2. Build prototyping model with VHT blocks and simulate 3. Generate HDL code using HDL Coder 4. Run HDL code on FPGA and run testbench in SL (FIL) 13

14 CVST and VHT, complementary products MBD workflow for embedded vision 14

15 Enhanced Edge Detection: Simulink Demo 15

16 Streaming Pixel Interface Full Frame vs Pixel Stream full-frame source à pixel-stream processing à full-frame sink 16

17 Streaming Pixel Interface Full Frame vs Pixel Stream full-frame source à pixel-stream processing à full-frame sink 17

18 Streaming Pixel Interface Pixel control signals pixelcontrol bus Ø Ø hstart, hend, vstart, vend, and valid Consistent block interface 18

19 Frame To Pixels and Pixels To Frame 19

20 Frame To Pixels and Pixels To Frame 20

21 Examples: Starting Points for Your Models 21

22 Image Filtering Keywords: Median Filter, Image Filter, PSNR 22

23 Gamma Correction Example Keywords: Gamma, PSNR 23

24 Histogram Equalization Keywords: Histogram, Linear Equalization, External Frame Delay

25 Edge Detection and Image Overlay Keywords: Sobel, Align Video, Alpha Mix, PSNR 25

26 Edge Detection with Impaired Frame Keywords: Sobel, Align Video, Alpha Mix, PSNR 26

27 Multi-Zone Metering Keywords: ROI, Image Statistics, Mean Source Mask Selected ROI 27

28 Multi-Zone Metering Keywords: ROI, Image Statistics, Mean Source Mask Selected ROI 28

29 FLIR Accelerates Development of Thermal Imaging FPGA Challenge Accelerate the implementation of advanced thermal imaging filters and algorithms on FPGA hardware Solution Use MATLAB to develop, simulate, and evaluate algorithms, and use HDL Coder to implement the best algorithms on FPGAs Results Time from concept to field-testable prototype reduced by 60% Enhancements completed in hours, not weeks Code reuse increased from zero to 30% Link to user story Raw image (left) and image after applying filter developed with HDL Coder (right). With MATLAB and HDL Coder we are much more responsive to marketplace needs. We now embrace change, because we can take a new idea to a real-time-capable hardware prototype in just a few weeks. There is more joy in engineering, so we ve increased job satisfaction as well as customer satisfaction. Nicholas Hogasten FLIR Systems 29

30 From Algorithm to FPGA Implementation MATLAB and Simulink Algorithm and System Design Model Refinement for Hardware HDL Coder RTL Creation Behavioral Simulation HDL Verifier QuestaSim HDL Co-Simulation Back Annotation Implement Design Synthesis Map Place & Route Verification Functional Simulation Static Timing Analysis Timing Simulation HDL Verifier FPGA-in-the-Loop 30

31 A complete solution Frame based Concept Development Algorithm Development Prototyping Architecture design Prototyping Chip design Pixel based Computer Vision System Toolbox HDL Coder Image Processing Toolbox Vision HDL Toolbox MATLAB Coder Fixed Pt Designer HDL Verifier MATLAB 31

32 ROI: Customer Adoption Of Model-Based Design Time spent on FPGA implementation Shorter implementation time by 48% (total project 33%) Reduced FPGA prototype development schedule by 47% Shorter design iteration cycle by 80% 1 st FPGA Prototype 2 nd FPGA Prototype 1 st FPGA Prototype 32