Object-based Layered Depth Images for improved virtual view synthesis in rate-constrained context

Size: px
Start display at page:

Download "Object-based Layered Depth Images for improved virtual view synthesis in rate-constrained context"

Transcription

1 Object-based Layered Depth Images for improved virtual view synthesis in rate-constrained context Vincent Jantet 1, Christine Guillemot 2, Luce Morin 3 1 ENS Cachan, Antenne de Bretagne, Campus de Ker Lann, Bruz France 2 INRIA Rennes, Bretagne Atlantique, Campus de Beaulieu, Rennes France 3 IETR - INSA Rennes, 20 avenue des Buttes de Coësmes, Rennes France ICIP 2011 Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

2 Context of multi-view videos Functionalities: 3DTV: Depth feeling by stereo-vision simulation. FVV: Live viewpoint selection. Require a virtual view synthesis method. Fig: 3D rendering Acquisition Compression Rendering Representation Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

3 Table on contents 1 Introduction 2 Object-based classification 3 Rendering results 4 Compression results Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

4 Outline 1 Introduction 2 Object-based classification 3 Rendering results 4 Compression results Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

5 Depth Image-Based Rendering (DIBR) Warping algorithm View i Depth i Input: View and associated depth map Output: New viewpoint (texture & depth) Disocclusions Warping Vincent Jantet (ENS-Cachan France) Obstructed scene information from reference viewpoint They appear along depth discontinuities Solution: Add additional informations (LDI) Object-based LDI Fig: Disocclusion ICIP / 21

6 Layered Depth Image (LDI) [SGHS98] A set of layers, containing depth pixels from a single viewpoint From a reference viewpoint 1 st layer Reference view Visible texture and depth 2 nd layer Residual data Hidden texture and depth Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

7 Layered Depth Image (LDI) [SGHS98] A set of layers, containing depth pixels from a single viewpoint From a reference viewpoint 1 st layer Reference view Visible texture and depth 2 nd layer Residual data Hidden texture and depth 1 st layer Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

8 Layered Depth Image (LDI) [SGHS98] A set of layers, containing depth pixels from a single viewpoint From a reference viewpoint 1 st layer Reference view Visible texture and depth 2 nd layer Residual data Hidden texture and depth 1 st layer 2 nd layer Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

9 LDI from real multi-view plus depth sequence [JMG09] Compressed depth map 1st layer 2nd layer Limitations - Redundant boundaries in both layers - Moving elements in both layers - Layers contain large depth discontinuities (Discontinuities are hard to compress) Vincent Jantet (ENS-Cachan France) Object-based LDI Synthesized virtual view Fig: Rendering impact of depth map compression ICIP / 21

10 Outline 1 Introduction 2 Object-based classification 3 Rendering results 4 Compression results Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

11 Object-based LDI representation This representation organizes pixels into layers to enhance depth continuity 1 st layer 2 nd layer Fig: Classical LDI depth layers Foreground Background Fig: Object-based LDI depth layers Method based on a growing-region algorithm Region R initialized with pixels where Z FG and Z BG are already defined. For each pixel q outside R: - Extrapolate Z FG and Z BG. - Classify q. Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

12 Object-based LDI representation This representation organizes pixels into layers to enhance depth continuity 1 st layer 2 nd layer Fig: Classical LDI depth layers Foreground Background Fig: Object-based LDI depth layers Method based on a growing-region algorithm Region R initialized with pixels where Z FG and Z BG are already defined. For each pixel q outside R: - Extrapolate Z FG and Z BG. - Classify q. Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

13 Classification: Initializing Foreground Unclassified Background Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

14 Classification: Results Foreground Unclassified Background Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

15 Background inpainting [CPT03] Advantages - Remove unnecessary boundaries - Inpainting processed once, before encoding stage - No need of inpainting method at rendering stage Fig: Background inpainting Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

16 Outline 1 Introduction 2 Object-based classification 3 Rendering results 4 Compression results Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

17 Rendering results Classical LDI rendering Virtual view inpainting O-LDI rendering Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

18 Fast mesh-based rendering Fig: Object-based LDI Continuous layers can be rendered as meshes. Foreground mesh is partially transparent. Vincent Jantet (ENS-Cachan France) Object-based LDI Fig: Meshes rendering ICIP / 21

19 Outline 1 Introduction 2 Object-based classification 3 Rendering results 4 Compression results Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

20 LDI and MVD compression schemes MVD compression (MVC) Input views V 1 V 3 V 5 V 7 LDI compression (MVC) LDI 4 Input LDI MVC MVC Compression V 1 V 3 V 5 V 7 Compressed views Rendering Final view VSRS V 6 LDI 4 Compressed LDI DIBR Rendering V 6 Final view Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

21 Rate-distortion curve 90 SSIM (%) Object-based LDI Classical LDI MPEG (MVC/VSRS) Bitrate (Mbit/s) Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

22 Conclusions on Object-Based LDI Advantages - Remove unnecessary boundaries Improve compression - Static background along time - Compatible with fast mesh-based rendering - Depth continuity improves rendering quality Limit - No backward compatibility with 2D decoding scheme Questions? Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

23 [CPT03] [JMG09] A. Criminisi, P. Pérez, and K. Toyama. Object removal by exemplar-based inpainting. In Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society Conference on, volume 2, pages , June Vincent Jantet, Luce Morin, and Christine Guillemot. Incremental-ldi for multi-view coding. In 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, pages 1 4, Potsdam, Germany, May [SGHS98] Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. Layered depth images. In SIGGRAPH 98: Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pages , New York, NY, USA, July ACM. Vincent Jantet (ENS-Cachan France) Object-based LDI ICIP / 21

The Open University s repository of research publications and other research outputs

The Open University s repository of research publications and other research outputs Open Research Online The Open University s repository of research publications and other research outputs Disocclusion Hole-Filling in DIBR-Synthesized Images using Multi-Scale Template Matching Conference

More information

VIEW INTERPOLATION WITH STRUCTURED DEPTH FROM MULTIVIEW VIDEO

VIEW INTERPOLATION WITH STRUCTURED DEPTH FROM MULTIVIEW VIDEO 19th European Signal Processing Conference (EUSIPCO 2011) Barcelona, Spain, August 29 - September 2, 2011 VIEW INTERPOLATION WITH STRUCTURED DEPTH FROM MULTIVIEW VIDEO Pravin Kumar Rana and Markus Flierl

More information

Hole-Filling Method Using Depth Based In-Painting For View Synthesis in Free Viewpoint Television (FTV) and 3D Video

Hole-Filling Method Using Depth Based In-Painting For View Synthesis in Free Viewpoint Television (FTV) and 3D Video MISUBISHI ELECRIC RESEARCH LABORAORIES http://www.merl.com Hole-Filling Method Using Depth Based In-Painting For View Synthesis in Free Viewpoint elevision (FV) and 3D Video Kwan-Jung Oh, Sehoon Yea, Yo-Sung

More information

Livrable D4.2 of the PERSEE project : Représentation et codage 3D - Rapport intermédiaire - Définitions des softs et architecture

Livrable D4.2 of the PERSEE project : Représentation et codage 3D - Rapport intermédiaire - Définitions des softs et architecture Livrable D4.2 of the PERSEE project : Représentation et codage 3D - Rapport intermédiaire - Définitions des softs et architecture Marcus Barkowsky, Emilie Bosc, Marco Cagnazzo, Josselin Gautier, Christine

More information

Quality improving techniques for free-viewpoint DIBR

Quality improving techniques for free-viewpoint DIBR Quality improving techniques for free-viewpoint DIBR Luat Do a, Sveta Zinger a and Peter H.N. de With a,b a Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, Netherlands b Cyclomedia

More information

Disocclusion Handling Using Depth-Based Inpainting

Disocclusion Handling Using Depth-Based Inpainting Disocclusion Handling Using Depth-Based Inpainting Suryanarayana M. Muddala, Roger Olsson and Mårten Sjöström Dept. of Information Technology and Media Mid Sweden University Sundsvall, Sweden suryanarayana.muddala@miun.se,

More information

Compressing Depth Maps using Multiscale Recurrent Pattern Image Coding

Compressing Depth Maps using Multiscale Recurrent Pattern Image Coding Manuscript for Review Compressing Depth Maps using Multiscale Recurrent Pattern Image Coding Journal: Electronics Letters Manuscript ID: ELL-2010-0135 Manuscript Type: Letter Date Submitted by the Author:

More information

Immersive Medien und 3D-Video

Immersive Medien und 3D-Video Fraunhofer-Institut für Nachrichtentechnik Heinrich-Hertz-Institut Ralf Schäfer schaefer@hhi.de http://ip.hhi.de Immersive Medien und 3D-Video page 1 Outline Immersive Media Examples Interactive Media

More information

Kapitel 12. 3D Television Based on a Stereoscopic View Synthesis Approach

Kapitel 12. 3D Television Based on a Stereoscopic View Synthesis Approach Kapitel 12 3D Television Based on a Stereoscopic View Synthesis Approach DIBR (Depth-Image-Based Rendering) approach 3D content generation DIBR from non-video-rate depth stream Autostereoscopic displays

More information

Automatic 2D-to-3D Video Conversion Techniques for 3DTV

Automatic 2D-to-3D Video Conversion Techniques for 3DTV Automatic 2D-to-3D Video Conversion Techniques for 3DTV Dr. Lai-Man Po Email: eelmpo@cityu.edu.hk Department of Electronic Engineering City University of Hong Kong Date: 13 April 2010 Content Why 2D-to-3D

More information

Template-based Eye and Mouth Detection for 3D Video Conferencing

Template-based Eye and Mouth Detection for 3D Video Conferencing Template-based Eye and Mouth Detection for 3D Video Conferencing Jürgen Rurainsky and Peter Eisert Fraunhofer Institute for Telecommunications - Heinrich-Hertz-Institute, Image Processing Department, Einsteinufer

More information

Adaptive Block Truncation Filter for MVC Depth Image Enhancement

Adaptive Block Truncation Filter for MVC Depth Image Enhancement 2014 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) Adaptive Block Truncation Filter for MVC Depth Image Enhancement Xuyuan Xu 1, Lai-Man Po 1, Chun-Ho Cheung 2, Litong

More information

Silhouette completion in DIBR with the help of skeletal tracking

Silhouette completion in DIBR with the help of skeletal tracking KASIMIDOU ELEFTHERIA Silhouette completion in DIBR with the help of skeletal tracking Diploma Thesis Supervision: Prof. Pascal Frossard Prof. Anastasios Delopoulos Autumn 2015 ACKNOWLEDGMENTS Working

More information

REPRESENTATION, CODING AND INTERACTIVE RENDERING OF HIGH- RESOLUTION PANORAMIC IMAGES AND VIDEO USING MPEG-4

REPRESENTATION, CODING AND INTERACTIVE RENDERING OF HIGH- RESOLUTION PANORAMIC IMAGES AND VIDEO USING MPEG-4 REPRESENTATION, CODING AND INTERACTIVE RENDERING OF HIGH- RESOLUTION PANORAMIC IMAGES AND VIDEO USING MPEG-4 S. Heymann, A. Smolic, K. Mueller, Y. Guo, J. Rurainsky, P. Eisert, T. Wiegand Fraunhofer Institute

More information

Color Segmentation Based Depth Image Filtering

Color Segmentation Based Depth Image Filtering Color Segmentation Based Depth Image Filtering Michael Schmeing and Xiaoyi Jiang Department of Computer Science, University of Münster Einsteinstraße 62, 48149 Münster, Germany, {m.schmeing xjiang}@uni-muenster.de

More information

Service-Oriented Visualization of Virtual 3D City Models

Service-Oriented Visualization of Virtual 3D City Models Service-Oriented Visualization of Virtual 3D City Models Authors: Jan Klimke, Jürgen Döllner Computer Graphics Systems Division Hasso-Plattner-Institut, University of Potsdam, Germany http://www.hpi3d.de

More information

Peter Eisert, Thomas Wiegand and Bernd Girod. University of Erlangen-Nuremberg. Cauerstrasse 7, 91058 Erlangen, Germany

Peter Eisert, Thomas Wiegand and Bernd Girod. University of Erlangen-Nuremberg. Cauerstrasse 7, 91058 Erlangen, Germany RATE-DISTORTION-EFFICIENT VIDEO COMPRESSION USING A 3-D HEAD MODEL Peter Eisert, Thomas Wiegand and Bernd Girod Telecommunications Laboratory University of Erlangen-Nuremberg Cauerstrasse 7, 91058 Erlangen,

More information

Key Terms 2D-to-3D-conversion, depth image-based rendering, hole-filling, image warping, optimization.

Key Terms 2D-to-3D-conversion, depth image-based rendering, hole-filling, image warping, optimization. Disparity-Map-Based Rendering for Hole Prevention Mr. S. K. Somasundaram, Department of Computer Science & Engineering, Associate Professor, PSNA College of Engineering & Technology, Anna University,Chennai,

More information

View Sequence Coding using Warping-based Image Alignment for Multi-view Video

View Sequence Coding using Warping-based Image Alignment for Multi-view Video View Sequence Coding using Warping-based mage Alignment for Multi-view Video Yanwei Liu, Qingming Huang,, Wen Gao 3 nstitute of Computing Technology, Chinese Academy of Science, Beijing, China Graduate

More information

Depth Consistency Testing for Improved View Interpolation

Depth Consistency Testing for Improved View Interpolation Depth Consistency Testing for Improved View Interpolation Pravin Kumar Rana and Markus Flierl ACCESS Linnaeus Center, School of Electrical Engineering, KTH Royal Institute of Technology Stockholm, Sweden

More information

POSE-SPACE IMAGE-BASED RENDERING

POSE-SPACE IMAGE-BASED RENDERING POSE-SPACE IMAGE-BASED RENDERING Anna Hilsmann 1,2 Philipp Fechteler 2 Peter Eisert 1,2 1 Humboldt University of Berlin 2 Fraunhofer HHI The work presented in this paper is funded by the German Science

More information

Adaptive Image Warping Using Disparity Map-Based Rendering For Hole Prevention In 3D View Synthesis

Adaptive Image Warping Using Disparity Map-Based Rendering For Hole Prevention In 3D View Synthesis Adaptive Image Warping Using Disparity Map-Based Rendering For Hole Prevention In 3D View Synthesis Yoganandhan.S Lakshmanan.V VinithaSubashini.B Department Of Computer Science and Engineering yoganandhan.cse@gmail.com

More information

Impact of Light Field Compression on Focus Stack and Extended Focus Images

Impact of Light Field Compression on Focus Stack and Extended Focus Images 26 24th European Signal Processing Conference (EUSIPCO) Impact of Light Field Compression on Focus Stack and Extended Focus Images Mira Rizkallah, Thomas Maugey, Charles Yaacoub and Christine Guillemot

More information

Multi-layered Image-Based Rendering

Multi-layered Image-Based Rendering Multi-layered Image-Based Rendering Sing Bing Kang Compaq Computer Corporation Cambridge Research Lab One Kendall Square, Bldg. 700 Cambridge, MA 02139 sbk@crl.dec.com Huong Quynh Dinh College of Computing

More information

The MPEG Standard. MPEG-1 (1992) actually a video player. plays out audio/video streams same type of access as home VCR

The MPEG Standard. MPEG-1 (1992) actually a video player. plays out audio/video streams same type of access as home VCR The MPEG Standard MPEG-1 (1992) actually a video player plays out audio/video streams same type of access as home VCR MPEG-2 (1995) introduced for compression and transmission of digital TV signals still

More information

Reference view selection in DIBR-based multiview coding

Reference view selection in DIBR-based multiview coding Reference view selection in DIBR-based multiview coding Thomas Maugey, Member IEEE, Giovanni Petrazzuoli, Pascal Frossard, Senior Member IEEE and Marco Cagnazzo, Senior Member IEEE and Béatrice Pesquet-Popescu,

More information

Video Quality Assessment: From 2D to 3D - Challenges and Future Trends

Video Quality Assessment: From 2D to 3D - Challenges and Future Trends Video Quality Assessment: From 2D to 3D - Challenges and Future Trends Quan Huynh-Thu, Patrick Le Callet, Marcus Barkowsky To cite this version: Quan Huynh-Thu, Patrick Le Callet, Marcus Barkowsky. Video

More information

Loss-resilient Coding of Texture and Depth for Free-viewpoint Video Conferencing

Loss-resilient Coding of Texture and Depth for Free-viewpoint Video Conferencing IEEE TRANSACTIONS ON MULTIMEDIA, APRIL 2014 1 Loss-resilient Coding of Texture and Depth for Free-viewpoint Video Conferencing Bruno Macchiavello Camilo Dorea Edson M. Hung Gene Cheung Senior Member, IEEE

More information

Quality Estimation for Scalable Video Codec. Presented by Ann Ukhanova (DTU Fotonik, Denmark) Kashaf Mazhar (KTH, Sweden)

Quality Estimation for Scalable Video Codec. Presented by Ann Ukhanova (DTU Fotonik, Denmark) Kashaf Mazhar (KTH, Sweden) Quality Estimation for Scalable Video Codec Presented by Ann Ukhanova (DTU Fotonik, Denmark) Kashaf Mazhar (KTH, Sweden) Purpose of scalable video coding Multiple video streams are needed for heterogeneous

More information

Perceived quality of DIBR-based synthesized views

Perceived quality of DIBR-based synthesized views Perceived quality of DIBR-based synthesized views Emilie Bosc, Romuald Pépion, Patrick Le Callet, Martin Köppel, Patrick Ndjiki-Nya, Luce Morin, Muriel Pressigout To cite this version: Emilie Bosc, Romuald

More information

Detection of the Single Image from DIBR Based on 3D Warping Trace and Edge Matching

Detection of the Single Image from DIBR Based on 3D Warping Trace and Edge Matching Journal of Computer and Communications, 2014, 2, 43-50 Published Online March 2014 in SciRes. http://www.scirp.org/journal/jcc http://dx.doi.org/10.4236/jcc.2014.24007 Detection of the Single Image from

More information

YÉvâá. Image domain warping as an alternative to DIBR. Special points of interest: Position Statement 2nd MUSCADE. Workshop. Publications Corner

YÉvâá. Image domain warping as an alternative to DIBR. Special points of interest: Position Statement 2nd MUSCADE. Workshop. Publications Corner Ref. Ares(2011)767431-14/07/2011 Q u a r t e r l y n e w s l e t t e r o f t h e M U S C A D E c o n s o r t i u m Special points of interest: T h e p o s i t i o n statement is on Image domain warping

More information

AES standard for digital audio - Audio-embedded metadata - Part 2: MPEG-1 Layer II or MPEG-2 LSF Layer II. Preview only

AES standard for digital audio - Audio-embedded metadata - Part 2: MPEG-1 Layer II or MPEG-2 LSF Layer II. Preview only Revision of AES41-2009 AES standard for digital audio - Audio-embedded metadata - Part 2: MPEG-1 Layer II or MPEG-2 LSF Layer II Published by Audio Engineering Society, Inc. Copyright 2012 by the Audio

More information

Introduction to Computer Graphics

Introduction to Computer Graphics Introduction to Computer Graphics Torsten Möller TASC 8021 778-782-2215 torsten@sfu.ca www.cs.sfu.ca/~torsten Today What is computer graphics? Contents of this course Syllabus Overview of course topics

More information

Effective Gradient Domain Object Editing on Mobile Devices

Effective Gradient Domain Object Editing on Mobile Devices Effective Gradient Domain Object Editing on Mobile Devices Yingen Xiong, Dingding Liu, Kari Pulli Nokia Research Center, Palo Alto, CA, USA Email: {yingen.xiong, dingding.liu, kari.pulli}@nokia.com University

More information

Diminished Reality using Multiple Handheld Cameras

Diminished Reality using Multiple Handheld Cameras Diminished Reality using Multiple Handheld Cameras Akihito Enomoto Hideo Saito Graduate School of Science and Technology, Keio University, okohama, Japan Abstract In this paper, we present a system for

More information

Special issue on three-dimensional video and television

Special issue on three-dimensional video and television Signal Processing: Image Communication 22 (2007) 103 107 www.elsevier.com/locate/image Guest Editorial Special issue on three-dimensional video and television The term three-dimensional television (3DTV)

More information

THE emerging 3-D video applications have encouraged. Towards a New Quality Metric for 3-D Synthesized View Assessment

THE emerging 3-D video applications have encouraged. Towards a New Quality Metric for 3-D Synthesized View Assessment Towards a New Quality Metric for 3-D Synthesized View Assessment Emilie Bosc, Student Member, IEEE, Romuald Pépion, Patrick Le Callet, Member, IEEE, Martin Köppel, Patrick Ndjiki-Nya, Muriel Pressigout,

More information

Efficient Video Coding with Fractional Resolution Sprite Prediction Technique

Efficient Video Coding with Fractional Resolution Sprite Prediction Technique Efficient Video Coding with Fractional Resolution Sprite Prediction Technique Yan Lu, Wen Gao and Feng Wu An efficient algorithm for dynamic sprite-based video coding with fractional resolution motion

More information

REMOTE RENDERING OF COMPUTER GAMES

REMOTE RENDERING OF COMPUTER GAMES REMOTE RENDERING OF COMPUTER GAMES Peter Eisert, Philipp Fechteler Fraunhofer Institute for Telecommunications, Einsteinufer 37, D-10587 Berlin, Germany eisert@hhi.fraunhofer.de, philipp.fechteler@hhi.fraunhofer.de

More information

JL-1 3D4YOU. Ralf Tanger Fraunhofer HHI Berlin, Germany

JL-1 3D4YOU. Ralf Tanger Fraunhofer HHI Berlin, Germany JL-1 3D4YOU Fraunhofer HHI Berlin, Germany Introduction JL-2 3D in the Cinema Increasing number of 3D productions First studios start to release all productions also in 3D Technology Stereoscopic 3D Glasses

More information

3D Rendering - Techniques and Challenges

3D Rendering - Techniques and Challenges 3D Rendering - Techniques and Challenges Vishal Verma #1, Ekta Walia *2 # Department of Computer Science, M L N College, Yamuna Nagar, Haryana, INDIA 1 me_vishaal@hotmail.com * IT Department, Maharishi

More information

Wireless Ultrasound Video Transmission for Stroke Risk Assessment: Quality Metrics and System Design

Wireless Ultrasound Video Transmission for Stroke Risk Assessment: Quality Metrics and System Design Wireless Ultrasound Video Transmission for Stroke Risk Assessment: Quality Metrics and System Design A. Panayides 1, M.S. Pattichis 2, C. S. Pattichis 1, C. P. Loizou 3, M. Pantziaris 4 1 A.Panayides and

More information

Edge tracking for motion segmentation and depth ordering

Edge tracking for motion segmentation and depth ordering Edge tracking for motion segmentation and depth ordering P. Smith, T. Drummond and R. Cipolla Department of Engineering University of Cambridge Cambridge CB2 1PZ,UK {pas1001 twd20 cipolla}@eng.cam.ac.uk

More information

Announcements. Active stereo with structured light. Project structured light patterns onto the object

Announcements. Active stereo with structured light. Project structured light patterns onto the object Announcements Active stereo with structured light Project 3 extension: Wednesday at noon Final project proposal extension: Friday at noon > consult with Steve, Rick, and/or Ian now! Project 2 artifact

More information

Wednesdays 09:30-11:30 Room 3305

Wednesdays 09:30-11:30 Room 3305 Wednesdays 09:30-11:30 Room 3305 Ioannis Stamos istamos@hunter.cuny.edu Office: 1090G, Hunter North (Entrance at 69 th bw/ Park and Lexington Avenues) Class website: http://www.cs.hunter.cuny.edu/~ioannis/3d_f11.html

More information

COMP-557: Fundamentals of Computer Graphics McGill University, Fall 2010

COMP-557: Fundamentals of Computer Graphics McGill University, Fall 2010 COMP-557: Fundamentals of Computer Graphics McGill University, Fall 2010 Class times 2:25 PM - 3:55 PM Mondays and Wednesdays Lecture room Trottier Building 2120 Instructor Paul Kry, kry@cs.mcgill.ca Course

More information

Efficient Coding Unit and Prediction Unit Decision Algorithm for Multiview Video Coding

Efficient Coding Unit and Prediction Unit Decision Algorithm for Multiview Video Coding JOURNAL OF ELECTRONIC SCIENCE AND TECHNOLOGY, VOL. 13, NO. 2, JUNE 2015 97 Efficient Coding Unit and Prediction Unit Decision Algorithm for Multiview Video Coding Wei-Hsiang Chang, Mei-Juan Chen, Gwo-Long

More information

Texture Unmapping Combining Parametric and Non- Parametric Techniques for Image Reconstruction

Texture Unmapping Combining Parametric and Non- Parametric Techniques for Image Reconstruction Texture Unmapping Combining Parametric and Non- Parametric Techniques for Image Reconstruction S. Mills and T.P. Pridmore School of Computer Science & Information Technology University of Nottingham Jubilee

More information

Books. CS155b Computer Graphics. Homework. Additional References. Syllabus. Goals

Books. CS155b Computer Graphics. Homework. Additional References. Syllabus. Goals CS155b Computer Graphics Instructor: Giovanni Motta (gim@ieee.org) Volen, Room #255. Phone: x62718 Class: Mon. and Wed. from 5 to 6:30pm Abelson #131 Teaching Assistants: Anthony Bucci (abucci@cs) John

More information

White paper. H.264 video compression standard. New possibilities within video surveillance.

White paper. H.264 video compression standard. New possibilities within video surveillance. White paper H.264 video compression standard. New possibilities within video surveillance. Table of contents 1. Introduction 3 2. Development of H.264 3 3. How video compression works 4 4. H.264 profiles

More information

Visualization and Feature Extraction, FLOW Spring School 2016 Prof. Dr. Tino Weinkauf. Flow Visualization. Image-Based Methods (integration-based)

Visualization and Feature Extraction, FLOW Spring School 2016 Prof. Dr. Tino Weinkauf. Flow Visualization. Image-Based Methods (integration-based) Visualization and Feature Extraction, FLOW Spring School 2016 Prof. Dr. Tino Weinkauf Flow Visualization Image-Based Methods (integration-based) Spot Noise (Jarke van Wijk, Siggraph 1991) Flow Visualization:

More information

9: MPEG Video. MPEG Family. Mark Handley

9: MPEG Video. MPEG Family. Mark Handley 9: MPEG Video Mark Handley MPEG Family MPEG-1 Similar to H.263 CIF in quality MPEG-2 Higher quality: DVD, Digital TV, HDTV MPEG-4/H.264 More modern codec. Aimed at lower bitrates. Works well for HDTV too.

More information

9. Illumination and Shading

9. Illumination and Shading -128-9. Illumination and Shading Approaches for visual realism: 1. Remove hidden surfaces 2. Shade the visible surfaces and reproduce shadows 3. Reproduce surface properties: texture degree of transparency,

More information

Space-Time Video Completion

Space-Time Video Completion Space-Time Video Completion Y. Wexler E. Shechtman M. Irani Dept. of Computer Science and Applied Math The Weizmann Institute of Science Rehovot, 7600 Israel Abstract We present a method for space-time

More information

VIDEO CODING USING TEXTURE ANALYSIS AND SYNTHESIS

VIDEO CODING USING TEXTURE ANALYSIS AND SYNTHESIS VIDEO CODING USING TEXTURE ANALYSIS AND SYNTHESIS Patrick Ndjiki-Nya, Bela Makai, Aljoscha Smolic, Heiko Schwarz, and Thomas Wiegand Fraunhofer Institute for Communications Engineering Heinrich Hertz Institute

More information

ERROR CONCEALMENT TECHNIQUES IN H.264 VIDEO TRANSMISSION OVER WIRELESS NETWORKS

ERROR CONCEALMENT TECHNIQUES IN H.264 VIDEO TRANSMISSION OVER WIRELESS NETWORKS Multimedia Processing Term project on ERROR CONCEALMENT TECHNIQUES IN H.264 VIDEO TRANSMISSION OVER WIRELESS NETWORKS Spring 2016 Under Dr. K. R. Rao by Moiz Mustafa Zaveri (1001115920) moiz.mustafazaveri@mavs.uta.edu

More information

Image Based Rendering With Stable Frame Rates

Image Based Rendering With Stable Frame Rates huamin Department Image Based Rendering With Stable Frame Rates Huamin Qu Ming Wan Jiafa Qin Arie Kaufman Center for Visual Computing (CVC) and Department of Computer Science State University of New York

More information

A Short Introduction to Computer Graphics

A Short Introduction to Computer Graphics A Short Introduction to Computer Graphics Frédo Durand MIT Laboratory for Computer Science 1 Introduction Chapter I: Basics Although computer graphics is a vast field that encompasses almost any graphical

More information

An Image Inpainting Algorithm based on K-means Algorithm

An Image Inpainting Algorithm based on K-means Algorithm Proc. of World Cong. on Multimedia and Computer Science An Image Inpainting Algorithm based on K-means Algorithm Ouafek Naouel 1, Pr Kholladi Mohamed-Khireddine 2 1 Assistant Prof, Department of computer

More information

Model-Based Video Coding Using Colour and Depth Cameras

Model-Based Video Coding Using Colour and Depth Cameras Model-Based Video Coding Using Colour and Depth Cameras David Sandberg Per-Erik Forsse n Jens Ogniewski Computer Vision Laboratory Department of Electrical Engineering Linko ping University SE-581 83 Linko

More information

MULTI-LAYER VISUALIZATION OF MOBILE MAPPING DATA

MULTI-LAYER VISUALIZATION OF MOBILE MAPPING DATA MULTI-LAYER VISUALIZATION OF MOBILE MAPPING DATA D. Eggert, M. Sester Institute of Cartography and Geoinformatics, Leibniz Universität Hannover, Germany - (eggert, sester)@ikg.uni-hannover.de KEY WORDS:

More information

Video compression: Performance of available codec software

Video compression: Performance of available codec software Video compression: Performance of available codec software Introduction. Digital Video A digital video is a collection of images presented sequentially to produce the effect of continuous motion. It takes

More information

IMPACT OF COMPRESSION ON THE VIDEO QUALITY

IMPACT OF COMPRESSION ON THE VIDEO QUALITY IMPACT OF COMPRESSION ON THE VIDEO QUALITY Miroslav UHRINA 1, Jan HLUBIK 1, Martin VACULIK 1 1 Department Department of Telecommunications and Multimedia, Faculty of Electrical Engineering, University

More information

Low power GPUs a view from the industry. Edvard Sørgård

Low power GPUs a view from the industry. Edvard Sørgård Low power GPUs a view from the industry Edvard Sørgård 1 ARM in Trondheim Graphics technology design centre From 2006 acquisition of Falanx Microsystems AS Origin of the ARM Mali GPUs Main activities today

More information

Infrared target simulation environment for pattern recognition applications

Infrared target simulation environment for pattern recognition applications Rochester Institute of Technology RIT Scholar Works Articles 1994 Infrared target simulation environment for pattern recognition applications Andreas Savakis Nicholas George Follow this and additional

More information

http://www.springer.com/0-387-23402-0

http://www.springer.com/0-387-23402-0 http://www.springer.com/0-387-23402-0 Chapter 2 VISUAL DATA FORMATS 1. Image and Video Data Digital visual data is usually organised in rectangular arrays denoted as frames, the elements of these arrays

More information

OPTIMIZING PEER GROUPING FOR LIVE FREE VIEWPOINT VIDEO STREAMING

OPTIMIZING PEER GROUPING FOR LIVE FREE VIEWPOINT VIDEO STREAMING OPTIMIZING PEER GROUPING FOR LIVE FREE VIEWPOINT VIDEO STREAMING Yuan Yuan, Bo Hu, Gene Cheung #, Vicky Zhao University of Alberta, Canada, # National Institute of Informatics, Japan ABSTRACT In free viewpoint

More information

Pattern Recognition. 3D video and free viewpoint video From capture to display. Aljoscha Smolic 1. article info. abstract

Pattern Recognition. 3D video and free viewpoint video From capture to display. Aljoscha Smolic 1. article info. abstract Pattern Recognition 44 (2011) 1958 1968 Contents lists available at ScienceDirect Pattern Recognition journal homepage: www.elsevier.com/locate/pr 3D video and free viewpoint video From capture to display

More information

A flexible video server based on a low complex post-compression rate allocation

A flexible video server based on a low complex post-compression rate allocation A flexible video server based on a low complex post-compression rate allocation François-Olivier Devaux and Christophe De Vleeschouwer Communications and Remote Sensing Laboratory, Université catholique

More information

Cut-Out Animation Using Magnet Motion

Cut-Out Animation Using Magnet Motion Cut-Out Animation Using Magnet Motion Srinivas Anumasa Avinash Singh Rishi Yadav srinu.0922@gmail.com singhavinash66@gmail.com rishiyadavjec@gmail.com Abstract In this paper a new video-based interface

More information

VIDEO-BASED RENDERING

VIDEO-BASED RENDERING VIDEO-BASED RENDERING Richard Szeliski Microsoft Research Abstract Image-based rendering is one of the hottest new areas in computer graphics. Instead of using 3D modeling and painting tools to construct

More information

Very Low Frame-Rate Video Streaming For Face-to-Face Teleconference

Very Low Frame-Rate Video Streaming For Face-to-Face Teleconference Very Low Frame-Rate Video Streaming For Face-to-Face Teleconference Jue Wang, Michael F. Cohen Department of Electrical Engineering, University of Washington Microsoft Research Abstract Providing the best

More information

Parametric Comparison of H.264 with Existing Video Standards

Parametric Comparison of H.264 with Existing Video Standards Parametric Comparison of H.264 with Existing Video Standards Sumit Bhardwaj Department of Electronics and Communication Engineering Amity School of Engineering, Noida, Uttar Pradesh,INDIA Jyoti Bhardwaj

More information

CS229 Project Final Report. Sign Language Gesture Recognition with Unsupervised Feature Learning

CS229 Project Final Report. Sign Language Gesture Recognition with Unsupervised Feature Learning CS229 Project Final Report Sign Language Gesture Recognition with Unsupervised Feature Learning Justin K. Chen, Debabrata Sengupta, Rukmani Ravi Sundaram 1. Introduction The problem we are investigating

More information

Mesh-based Integration of Range and Color Images

Mesh-based Integration of Range and Color Images Mesh-based Integration of Range and Color Images Yiyong Sun *, Christophe Dumont, Mongi A. Abidi Imaging, Robotics, and Intelligent Systems (IRIS) Laboratory Department of Electrical and Computer Engineering

More information

AUDIO CODING: BASICS AND STATE OF THE ART

AUDIO CODING: BASICS AND STATE OF THE ART AUDIO CODING: BASICS AND STATE OF THE ART PACS REFERENCE: 43.75.CD Brandenburg, Karlheinz Fraunhofer Institut Integrierte Schaltungen, Arbeitsgruppe Elektronische Medientechnolgie Am Helmholtzring 1 98603

More information

MPEG Audio and Video Technology for ENG: Parameters and Capabilities

MPEG Audio and Video Technology for ENG: Parameters and Capabilities ITU-R Study Group 6 Seminar - Spectrum Usage and User Requirements of Terrestrial Electronic News Gathering MPEG Audio and Video Technology for ENG: Parameters and Capabilities Nikolaus Rettelbach e for

More information

CHAPTER 6 TEXTURE ANIMATION

CHAPTER 6 TEXTURE ANIMATION CHAPTER 6 TEXTURE ANIMATION 6.1. INTRODUCTION Animation is the creating of a timed sequence or series of graphic images or frames together to give the appearance of continuous movement. A collection of

More information

Depth and Depth-Color Coding using Shape-Adaptive Wavelets

Depth and Depth-Color Coding using Shape-Adaptive Wavelets 1 Depth and Depth-Color Coding using Shape-Adaptive Wavelets Matthieu Maitre and Minh N. Do Abstract We present a novel depth and depth-color codec aimed at free-viewpoint 3D-TV. The proposed codec uses

More information

A ghosting artifact detector for interpolated image quality assessment

A ghosting artifact detector for interpolated image quality assessment A ghosting artifact detector for image quality assessment Kai Berger, Christian Lipski, Christian Linz, Anita Sellent and Marcus Magnor Computer Graphics Lab TU Braunschweig, Germany Email: {berger, lipski,

More information

JOINT GAZE-CORRECTION AND BEAUTIFICATION OF DIBR-SYNTHESIZED HUMAN FACE VIA DUAL SPARSE CODING

JOINT GAZE-CORRECTION AND BEAUTIFICATION OF DIBR-SYNTHESIZED HUMAN FACE VIA DUAL SPARSE CODING JOINT GAZE-CORRECTION AND BEAUTIFICATION OF DIBR-SYNTHESIZED HUMAN FACE VIA DUAL SPARSE CODING Xianming Liu 1,, Gene Cheung, Deming Zhai 1, Debin Zhao 1, Hiroshi Sankoh 3, Sei Naito 3 1 School of Computer

More information

Statistical Modeling of Huffman Tables Coding

Statistical Modeling of Huffman Tables Coding Statistical Modeling of Huffman Tables Coding S. Battiato 1, C. Bosco 1, A. Bruna 2, G. Di Blasi 1, G.Gallo 1 1 D.M.I. University of Catania - Viale A. Doria 6, 95125, Catania, Italy {battiato, bosco,

More information

Kinect & 3D. Tamara Berg Advanced Mul7media

Kinect & 3D. Tamara Berg Advanced Mul7media Kinect & 3D Tamara Berg Advanced Mul7media Recovering 3D from Images How can we automa7cally compute 3D geometry from images? What cues in the image provide 3D informa7on? Slide credit: S. Narasimhan Visual

More information

Adapting Standard Video Codecs for Depth Streaming

Adapting Standard Video Codecs for Depth Streaming Joint Virtual Reality Conference of EuroVR - EGVE (11) R. Blach, S. Coquillart, M. D Cruz, A. Steed, and G. Welch (Editors) Adapting Standard Video Codecs for Depth Streaming Fabrizio Pece Jan Kautz Tim

More information

Non-Metric Image-Based Rendering for Video Stabilization

Non-Metric Image-Based Rendering for Video Stabilization Non-Metric Image-Based Rendering for Video Stabilization Chris Buehler Michael Bosse Leonard McMillan MIT Laboratory for Computer Science Cambridge, MA 02139 Abstract We consider the problem of video stabilization:

More information

Real-Time Depth-Image-Based Rendering for 3DTV Using OpenCL

Real-Time Depth-Image-Based Rendering for 3DTV Using OpenCL Real-Time Depth-Image-Based Rendering for 3DTV Using OpenCL Roberto Gerson de Albuquerque Azevedo, Fernando Ismério, Alberto Barbosa Raposo, and Luiz Fernando Gomes Soares Department of Informatics, Pontifical

More information

lecture 20 Image Compositing - chroma keying - alpha - F over B - OpenGL blending - chroma keying revisited: "pulling a matte"

lecture 20 Image Compositing - chroma keying - alpha - F over B - OpenGL blending - chroma keying revisited: pulling a matte lecture 20 Image Compositing - chroma keying - alpha - F over B - OpenGL blending - chroma keying revisited: "pulling a matte" Organization of Course 1: Viewing transformations 2: Visibility, geometry

More information

COIT 475 Multimedia Network Technology. Hossam M.J. Mustafa FCITR, KAU, Rabigh

COIT 475 Multimedia Network Technology. Hossam M.J. Mustafa FCITR, KAU, Rabigh COIT 475 Multimedia Network Technology Hossam M.J. Mustafa FCITR, KAU, Rabigh Part I Introduction to Multimedia Networking COIT 475 Multimedia Network Technology 2 Overview What is Multimedia? Characteristics

More information

Improved Interactive Reshaping of Humans in Images

Improved Interactive Reshaping of Humans in Images Improved Interactive Reshaping of Humans in Images Nidhi Arora IIT Delhi nidhi@cse.iitd.ac.in Harsh Kumar IIT Delhi harsh23031989@gmail.com Jagjeet S. Dhaliwal IIT Delhi cs5080212@cse.iitd.ac.in Prem Kalra

More information

Module 9 AUDIO CODING. Version 2 ECE IIT, Kharagpur

Module 9 AUDIO CODING. Version 2 ECE IIT, Kharagpur Module 9 AUDIO CODING Lesson 28 Basic of Audio Coding Instructional Objectives At the end of this lesson, the students should be able to : 1. Name at least three different audio signal classes. 2. Calculate

More information

Compression of Computer Graphics Images with Image-Based Rendering

Compression of Computer Graphics Images with Image-Based Rendering Compression of Computer Graphics Images with Image-Based Rendering Ilmi Yoon and Ulrich Neumann Computer Science Dept. University of Southern California Los Angeles, CA90089 ABSTRACT We present a new compression

More information

The path towards augmented reality with Renesas R-Car family

The path towards augmented reality with Renesas R-Car family The path towards augmented reality with Renesas R-Car family 2014.12 Smart Camera Autonomous Drive H.264 Low Latency Augmented Reality Radar 3D Surround View Functional Safety Image Recognition Ethernet

More information

Immersive 3-D Video Conferencing: Challenges, Concepts, and Implementations

Immersive 3-D Video Conferencing: Challenges, Concepts, and Implementations Proc. SPIE Visual Communications and Image Processing (VCIP), Lugano, Switzerland, July 2003. Immersive 3-D Video Conferencing: Challenges, Concepts, and Implementations Peter Eisert Image Processing Department

More information

Introduzione alle Biblioteche Digitali Audio/Video

Introduzione alle Biblioteche Digitali Audio/Video Introduzione alle Biblioteche Digitali Audio/Video Biblioteche Digitali 1 Gestione del video Perchè è importante poter gestire biblioteche digitali di audiovisivi Caratteristiche specifiche dell audio/video

More information

Overview: Video Coding Standards

Overview: Video Coding Standards Overview: Video Coding Standards Video coding standards: applications and common structure Relevant standards organizations ITU-T Rec. H.261 ITU-T Rec. H.263 ISO/IEC MPEG-1 ISO/IEC MPEG-2 ISO/IEC MPEG-4

More information

Vision based Vehicle Tracking using a high angle camera

Vision based Vehicle Tracking using a high angle camera Vision based Vehicle Tracking using a high angle camera Raúl Ignacio Ramos García Dule Shu gramos@clemson.edu dshu@clemson.edu Abstract A vehicle tracking and grouping algorithm is presented in this work

More information

SIGGRAPH Asia 2016 Conference at a Glance. Monday, 05 December 2016

SIGGRAPH Asia 2016 Conference at a Glance. Monday, 05 December 2016 Registration Categories At-A-Glance Entitlements Included For The Various Categories FC Full Conference Pass BC Basic Conference Pass EO Exhibits Only ET Electronic Theater Ticket 1D Full Conference One-Day

More information

Introduction to Video Encoding

Introduction to Video Encoding Introduction to Video Encoding INF5063 15. October 2010 History of MPEG Motion Picture Experts Group MPEG1 work started in 1988, published by ISO in 1993 Part 1 Systems, Part 2 Video, Part 3 Audio, Part

More information

A Novel Hole filling method based on Projection onto Convex Set in DIBR

A Novel Hole filling method based on Projection onto Convex Set in DIBR 3rd International Conference on Multimedia Technology ICMT 2013) A Novel Hole filling method based on Projection onto Convex Set in DIBR Weiquan Wang1 and Yingyun Yang2 and Qian Liang3 Abstract. Depth

More information