Photonfocus AG Usability of high quality image compression in machine Vision

Size: px

Start display at page:

Download "Photonfocus AG Usability of high quality image compression in machine Vision"

Harold Greer
7 years ago
Views:

1 Photonfocus AG Usability of high quality image compression in machine Vision Dr. Peter Mario Schwider, CTO

Basic Issue in Transmission Systems Bandwidth Limitation in Data Transmission Systems bearbeiten High-speed transmission of large amounts of data require high bandwidths that are typically not

2 Basic Issue in Transmission Systems Bandwidth Limitation in Data Transmission Systems bearbeiten High-speed transmission of large amounts of data require high bandwidths that are typically not supported by todays standard interfaces. This bandwidth limitation can be eliminated with data compression methods. Questions: Original data Reconstruction Compression Decompression Compressed Data 1. Are compression techniques suitable for standard applications or only fringe applications? 2. Are there compression technologies which are suitable for machine vision applications? Answer: Image processing algorithms can cooperate successfully with compression algorithms which eliminate redundant and irrelevant information. 2

3 Requirements of Machine Vision Requirements for compression method and implementation bearbeiten Consumer chip sets could not be re-used development of FPGA cores needed Real-time compression with optimal balance between image quality and FPGA resources Lossless or visual lossless compression method for monochrome and colour raw Bayer pattern images Adoption for multi-tap data paths, important for implementation of multi-tap CMOS image sensors No external memory for low latency Intra-frame compression (I-Frames) for fast access of single images Compression rate independent of image content Implementation within the GigEVision and GenICam standard Network connectivity without any limitations to enable multi-camera system solutions No Link-Aggregation, one cable solution Compression can be disabled to transmit original image data at slower frame rate 3

4 Compression ABC Overview of Compression Methods Titelmasterformat Entropy durch Klicken Coding bearbeiten Run-Length Coding Huffman Coding Arithmetic Coding Source Coding Prediction Transformation Layered Coding Vector Quantization DPCM DM DCT DWT proprietary Bit Position Subsampling Sub-Band Coding Hybrid Coding JPEG MPEG H.261, H.263, H.264, H.265 proprietary Photonfocus products evaluated and successful implemented 4

5 Approach Double Rate Technology Photonfocus DPCM Compression Algorithm with 1.92:1 compression rate bearbeiten The basic concept of Differential Pulse Code Modulation (DPCM) is coding a difference. The motivation for this method is based on the fact that most source signals show significant correlation between successive samples so encoding uses redundancy in sample values which implies lower bit rate. DPCM is a form of predictive coding. DPCM compression depends on the prediction technique. Well-conducted prediction techniques lead to good compression rates. Photonfocus DPCM implementation has a compression rate of about 1.92 : 1 and a fixed data format Implementation can code monochrome and Bayer pattern raw data The influence of transmission errors is reduced by periodical transmission of sampling points Multi-tap FPGA implementation has a good balance between resources and image quality Decompression is much less complex than the compression, could be realized in software and hardware Decompression in real-time with CPU resources possible 5

6 Photonfocus DR Technology Results of the Photonfocus DPCM compression algorithm bearbeiten Difference image after histogram equilibration (gain around 50) Lena test image 6

7 Approach QuadRate Technology Photonfocus wavelet based transformation codec with 4:1 compression rate bearbeiten Transformation codec with strict compression rate control, patent pending technology Compression strict Rate Control Original data Reconstruction Decorrelation Transformation inverse Decorrelation Transformation Quantisation Dequantisation Entropy Coder Entropy Decoder Compressed Data Decompression 7

8 DR and QR Technology in Comparison Comparison with USB3.0 and CameraLink in different configurations bearbeiten 8

9 Summary Rating of different implementations bearbeiten Assured Compression Rate ~ 2 : 1 4 : 1 Lossless Compression Lossy Compression visual losless No Block artefacts Variable Format No Framedrops Real-time FPGA Implementation JPEG Cores Competitor solution DR Photonfocus QR Photonfocus Dependent on Image Content 9

10 Applications with DR Technology I Simultaneous Colour 2D/3D Inspection with 1.8 khz Scan 896 x 100 Pixels bearbeiten Source M3C Industrial, Automation & Vision, Spain 10

11 Applications with DR Technology II Motion Analysis bearbeiten Source Contemplas GmbH, Germany 11

12 Thank you very much for your attention bearbeiten Photonfocus AG Bahnhofplatz 10 CH-8853 Lachen SZ Switzerland Phone: Date: August 2016 Produced by: Dr. Peter Mario Schwider 12

13 Photonfocus CMOS Cameras with Image Compression Technologies DualRate Technology DR1 Cameras DPCM based compression Visual lossless compression FPGA implementation without external RAM DR technology can process monochrome and Bayer pattern raw images Compression rate fixed at around 1.92 : 1 Compressed image has fixed format Codec data output compatible with GigEVision Decompression effort very low, real-time decompression in CPU possible Decompression of multiple cameras in real-time with Silicon Software GigE frame grabber and Visual Applet realized QuadRate Technology QR1 Cameras Titelmasterformat durch Klicken Wavelet based bearbeiten compression Lossless compression for images with low frequency content, visual lossless for images with high frequency content FPGA implementation without external RAM QR technology can process monochrome and Bayer pattern raw images Strict compression rate control between 2 : 1 and 6 : 1, compression rate control in QR cameras 4 : 1 Compressed image has variable format, no constant number of lines Codec data output compatible with GigEVision Standard, warnings for no constant line length have to be ignored Decompression effort high, no real-time decompression in CPU possible Decompression IP Core for FPGAs available 13

Photonfocus Products DR1 Camera Series QR1 Camera Serie High dynamic range CMOS cameras with LinLog technology DR1-D1312-200-G2-8 DR1-D1312IE-200-G2-8 135 fps at full resolution 180 fps at 1024 x

14 Photonfocus Products DR1 Camera Series QR1 Camera Serie High dynamic range CMOS cameras with LinLog technology DR1-D G2-8 DR1-D1312IE-200-G fps at full resolution 180 fps at 1024 x 1024 pixel resolution LowLight CMOS cameras DR1-D2048x G2-8 DR1-D2048x1088I-192-G2-8 DR1-D2048x1088C-192-G fps at full resolution Titelmasterformat durch LowLight Klicken CMOS bearbeiten cameras QR1-D2048x G2-8 QR1-D2048x1088I-384-G2-8 QR1-D2048x1088C-384-G fps at full resolution 358 fps at 1024 x 1024 pixel resolution 180 fps at 1024 x 1024 pixel resolution NIR cameras are used together with NIR markers in motion analysis systems. Colour cameras enable colour based feature extraction methods. NIR cameras are used together with NIR markers in motion analysis systems. Colour cameras enable colour based feature extraction methods. 14

15 Approach JPEG/JPEG2000 JPEG and JPEG2000 IP Cores bearbeiten JPEG is based on the Discrete Cosinus-Transformation (DCT) JPEG IP cores for FPGAs are available These cores could be adopted to multi-tap data paths Compression rate depends on core settings and image content Medium compression settings deliver compression rates up to 10:1 with good image quality Photonfocus has successfully implemented such a solution for a customer application Solution was not introduced to market because of lack of customer acceptance of block artefacts Starting point for evaluation of compression methods by Photonfocus JPEG2000 is based on the Discrete Wavelet-Transformation (DWT) and shows no block artefacts JPEG2000 FPGA implementations exist but multi-tap solutions limit the possibilities of industrial camera platforms in terms of FPGA/Hardware resources and power consumption. 15

16 Approach JPEG/JPEG2000 JPEG and JPEG2000 IP Cores bearbeiten Source: Internet Original JPEG 1:64 DCT JPEG2000 1:64 DWT 16

17 Approach JPEG-LS JPEG-LS compression based on LOCO-I algorithm bearbeiten The core of JPEG-LS is based on the LOCO-I algorithm, that relies on prediction, residual modelling and context-based coding of the residuals. Low complexity of the algorithm enables FPGA implementations Besides lossless compression, JPEG-LS also provides a lossy mode ("near-lossless") where the maximum absolute error can be controlled by the encoder. For more information see: This idea was picked up by a Photonfocus competitor. A modified predictor was used and the algorithm was successfully implemented in a camera platform. The issue of the variable compression rate was fundamentally solved by the competitor with an averaging of the compression rate by buffering the compressed images before transmission over the GigE interface. Frame drops for image sequences with high frequency image content in general cannot be avoided. Compression rate varies from 1.2 : 1 to 2.35 : 1 depending on image content. 17

Introduction to image coding

Introduction to image coding Image coding aims at reducing amount of data required for image representation, storage or transmission. This is achieved by removing redundant data from an image, i.e. by