COMP : 096: Computational Photography

Transcription

1 COMP : 096: Computational Photography

2 Basic Info Instructor: Svetlana Lazebnik Office hours: By appointment, FB 244 Class webpage:

3 Today What is computational photography? Course overview Course requirements

4 What is Computational Photography? Definition 1: the use of photographic imagery to create graphics content

5 Traditional Computer Graphics 3D geometry projection simulation physics Slide credit: A. Efros

6 State of the Art Amazingly real But so sterile, lifeless, futuristic Slide credit: A. Efros

7 The richness of our everyday world Slide credit: A. Efros Pavia, Italy

8 Beauty in complexity Slide credit: A. Efros Blue Mountains, Australia

9 Which parts are hard to model? Slide credit: A. Efros

10 People On the Tube, London From Final Fantasy Slide credit: A. Efros

11 Faces / Hair From Final Fantasy Slide credit: A. Efros Photo by Joaquin Rosales Gomez

12 Urban Scenes Photo of l LA Virtual LA (SGI) Slide credit: A. Efros

13 Nature River Cherwell, Oxford Slide credit: A. Efros

14 The Realism Spectrum Computer Graphics Computational Photography Photography Realism Manipulation Ease of capture + easy to create new worlds + easy to manipulate objects/ viewpoint - very hard to look realistic + instantly realistic + easy to aquire - very hard to manipulate objects/viewpoint Slide credit: A. Efros

15 What is Computational Photography? Definition 1: the use of photographic imagery to create graphics content Definition 2: The use of computational techniques to overcome limitations of conventional photography

16 Limitations of traditional photography Blur, camera shake, noise, damage

17 Limitations of traditional photography Limited resolution

18 Limitations of traditional photography Bad color / no color

19 Limitations of traditional photography Unwanted objects

20 Limitations of traditional photography Unfortunate expressions

21 Limitations of traditional photography Limited dynamic range

22 Limitations of traditional photography Single viewpoint, static 2D picture

23 Limitations of traditional photography Single depth of focus

24 Course overview Data-driven image synthesis Image enhancement and restoration Removing camera shake and motion blur Color image manipulation Image completion Combining multiple images Compositing Panoramas, mosaics, collages Warping and morphing From 2D to 3D Single-view reconstruction Multiple-view reconstruction Beyond conventional cameras High dynamic range imaging Light field cameras, coded apertures Digital image forensics

25 Data-driven image synthesis Texture synthesis Image analogies Efros & Leung (1999), Efros & Freeman (2001) Hertzmann et al. (2001) Image super-resolution

26 Removing Camera Shake and Motion Blur Fergus et al. (2006) Levin (2006) Yuan et al. (2007)

27 Color image manipulation Colorization Color Harmonization Levin et al. (2004) White Balance Adjustment Cohen-Or et al. (2006) Hsu et al. (2008)

28 Inpainting Image completion Image completion using millions of photographs Bertalmio et al. (2000) Hays and Efros (2007) Image completion with structure propagation Sun et al. (2005)

29 Image compositing Interactive Digital Photomontage Agarwala et al. (2004) Face Swapping Bitouk et al. (2008)

30 Panoramas, collages, etc. Panorama stitching AutoCollage Multi-viewpoint panoramas Rother et al. (2006) Agarwala et al. (2006)

31 Image deformation using moving least squares Warping and morphing Face morphing Schaefer et al. (2006)

32 From 2D to 3D Modeling and Rendering Architecture from Photographs Photo Pop-up Debevec et al. (1996) Photo Tourism Hoiem et al. (2005) Snavely et al. (2006)

33 Beyond conventional cameras High Dynamic Range Imaging Gigapixel Images Kopf et al. (2007) Light Field Photography Ng et al. (2005)

34 Image Forensics How to detect manipulated images?

35 What will not be covered Advanced image-based modeling and rendering View interpolation Relighting Combining photographic imagery with standard graphics imagery

36 Paper presentation: 30% Course requirements Think of yourself as professor for one class Make sure you understand all the important techniques in the paper Integrate related papers, other material covered in the course Try to encourage discussion Participation: 20% Read all papers Come to class Ask and answer questions Talk to me about your presentation or project Final project: 50% Implementation Grading is based primarily on effort and initiative so try to be creative! Brief project presentation at the end of the course

37 Project Ideas Straightforward ideas: implement texture synthesis, panorama stitching, etc.

38 Texture transfer

39 Tour into the Picture Horry et al. (1997)

40 Creating Joiners David Hockney Flickr Hockneyesque pool L. Zelnik-Manor and P. Perona, Automating Joiners, 5th International Symposium on Non-Photorealistic Animation and Rendering, 2007

41 Face Morphing

42 Playing around with faces on hotornot.com B. Davis and S. Lazebnik, Analysis of Human Attractiveness Using Manifold Kernel Regression, ICIP 2008

43 Face beautification T. Leyvand, D. Cohen-Or, G. Dror and D. Lischinski, Data-Driven Enhancement of Facial Attractiveness, SIGGRAPH 2008

44 Hybrid images A. Oliva, A. Torralba, P.G. Schyns, Hybrid Images, SIGGRAPH 2006

45 Background Replacement Preliminary results by Sashi Kumar Penta

46 Infinite Panoramas J. Sivic, B. Kaneva, A. Torralba, S. Avidan, and W. Freeman, Creating and Exploring a Large Photorealistic Virtual Space, Internet Vision Workshop, 2008

47 Creating unlikely juxtapositions

48 Creating unlikely juxtapositions

49 Creating unlikely juxtapositions Jeff Wall, Flooded Grave Scott Mutter, Escalator

50 Visual rhyming R. Raguram and S. Lazebnik, Computing Iconic Summaries for General Visual Concepts, Internet Vision Workshop, 2008

51 Class projects: Policy Feel free to use code or software you find on the web, provided it does not make your project trivial All outside sources should be fully documented in the final project report Feel free to talk to other people about the project, but do your own implementation By default, I expect projects to be individual. In special cases, collaboration is possible, but each person should have a clearly identifiable part that they are responsible for. If you are thinking of collaborating on a project, come and talk to me

52 For next time Send four paper preferences by the end of the week Start thinking about final project ideas (feel free to talk to me) Read papers on texture synthesis