Learning hatching for pen-and-ink illustrations of surfaces

Transcription

1 Learning hatching for pen-and-ink illustrations of surfaces Evangelos Kalogerakis 1,2, Derek Nowrouzehahrai 1,3,4, Simon Breslav 1,5, Aaron Hertzmann 1 1 University of Toronto, 2 Stanford University, 3 Disney Research Zurich, 4 University of Montreal, 5 Autodesk Research

2 Goal: Synthesis of hatching illustrations Exemplar shape Artist s illustration

3 Goal: Synthesis of hatching illustrations Exemplar shape Artist s illustration Learned model of hatching

4 Goal: Synthesis of hatching illustrations Exemplar shape Artist s illustration Learned model of hatching Input shape Synthesized illustration

5 Challenge: understanding hatching styles

6 Related work: hatching smooth surfaces Iso-parametric curves [Saito and Takahashi 1990, Winkenbach and Salesin 1996]

7 Related work: hatching smooth surfaces Iso-parametric curves [Saito and Takahashi 1990, Winkenbach and Salesin 1996] Smooth curvature directions and shading-based tone [Elber 1998, Hertzmann and Zorin 2000]

8 Related work: hatching smooth surfaces Iso-parametric curves [Saito and Takahashi 1990, Winkenbach and Salesin 1996] Smooth curvature directions and shading-based tone [Elber 1998, Hertzmann and Zorin 2000] Shading gradients [Singh and Schaefer 2010]

9 Related work: hatching smooth surfaces Iso-parametric curves [Saito and Takahashi 1990, Winkenbach and Salesin 1996] Smooth curvature directions and shading-based tone [Elber 1998, Hertzmann and Zorin 2000] Shading gradients [Singh and Schaefer 2010] Real-time hatching [Praun et al. 2001, Kim et al. 2008]

10 Related work: hatching smooth surfaces Artist s illustration Smoothed curvature directions Smoothed image gradients [Hertzmann and Zorin 2000] [Singh and Schaefer 2010]

11 Related work: where do people draw lines? [Cole et al. 2008] Average images composed of artists drawings Predicted line drawing

12 Our approach Learns a model of hatching style from a single artist s drawing of an input shape

13 Our approach Learns a model of hatching style from a single artist s drawing of an input shape Can transfer the hatching style to different views of the exemplar shape as well as different shapes

14 Our approach Learns a model of hatching style from a single artist s drawing of an input shape Can transfer the hatching style to different views of the exemplar shape as well as different shapes The hatching style is determined by hatching properties related to hatching tone and orientations

15 Hatching level Hatching properties No hatching Hatching Cross hatching

16 Hatching level Stroke thickness Hatching properties

17 Hatching level Stroke thickness Stroke spacing Hatching properties

18 Hatching level Stroke thickness Stroke spacing Stroke length Hatching properties

19 Hatching level Stroke thickness Stroke spacing Stroke length Stroke intensity Hatching properties

20 Hatching level Stroke thickness Stroke spacing Stroke length Stroke intensity Hatching orientations Hatching properties

21 Hatching properties Hatching level Stroke thickness Stroke spacing Stroke length Stroke intensity Hatching orientations Artist s illustration Computer generated illustration

22 Learning stage

23 Learning stage Thickness Spacing Intensity Length Hatching level Orientations

24 Learning stage Shape features Thickness Spacing Intensity Image space features Length Hatching level Orientations

25 Learning stage Shape Hatching and image properties features yx Shape Hatching and image properties descriptors y x

26 Learning stage Shape and image features x Hatching properties y y = f(x)

27 Learning hatching orientations Linear model expressing hatching orientations as a weighted sum of selected orientation features.

28 Learning hatching orientations Linear model expressing hatching orientations as a weighted sum of selected orientation features.

29 Learning hatching orientations Artist s illustration Fitting a single model across the illustration

30 Learning orientation fields Artist s illustration

31 Mixture of experts model Simultaneous segmentation & model fitting for each segment

32 Learning stroke properties Map features to thickness, intensity, spacing, length

33 Learning stroke properties Map features to thickness, intensity, spacing, length

34 Learning stroke properties Map features to thickness Extracted thickness Learned thickness

35 Learning stroke properties Map features to intensity Extracted intensity Learned intensity

36 Learning stroke properties Map features to spacing Extracted spacing Learned spacing

37 Learning stroke properties Map features to length Extracted length Learned length

38 Learning hatching level and segment labels Map features to discrete values with Joint Boosting + CRF No hatching Hatching Cross hatching Extracted hatching level Learned hatching level

39 Synthesis stage

40 Synthesis stage Thickness Spacing Hatching level Intensity Length Orientations

41 Synthesis stage Thickness Spacing Hatching level Intensity Length Orientations

42 Artist s illustration

43 Artist s illustration

44 Artist s illustration

45 Artist s illustration

46 Artist s illustration

47 Artist s illustration

48 Artist s illustration

49 Artist s illustration

50 Artist s illustration

51 Artist s illustration

52 Artist s illustration

53 Artist s illustration

54 Artist s illustration

55 Artist s illustration

56 Artist s illustration

57 Artist s illustration

58 Artist s illustration

59 Artist s illustration

60 Artist s illustration

61 Artist s illustration

62 Orientation features: Analysis of features used

63 Orientation features: Principal curvatures and local symmetry axes dominate Analysis of features used

64 Orientation features: Principal curvatures and local symmetry axes dominate Also orientations aligned with feature lines are also important Analysis of features used

65 Analysis of features used Hatching level: image intensity, shading features Stroke thickness: shape descriptors, curvature, shading features, image gradients, location of feature lines, depth Spacing: shape descriptors, curvature, derivatives of curvature, shading features Intensity: shape descriptors, image intensity, shading features, depth, location of feature lines Length: shape descriptors, curvature, radial curvature, shading feature, image intensity, image gradient Segment label: shape descriptors

66 Summary An algorithm that learns hatching styles

67 Summary An algorithm that learns hatching styles Learns from a single drawing

68 Summary An algorithm that learns hatching styles Learns from a single drawing Synthesizes hatching illustrations in the input artist s style for novel views and shapes

69 Limitations We do not always exactly match the artist s illustration - aspects of hatching style are lost

70 Limitations We do not always exactly match the artist s illustration - aspects of hatching style are lost Pre-processing stage relies on thresholds to robustly extract hatching properties.

71 Limitations We do not always exactly match the artist s illustration - aspects of hatching style are lost Pre-processing stage relies on thresholds to robustly extract hatching properties. Computation time is large (5h-10h for training, 0.5-1h for synthesis)

72 Future Work Analyze larger set of drawings

73 Future Work Analyze larger set of drawings Extend our framework to analyze other forms of art

74 Future Work Analyze larger set of drawings Extend our framework to analyze other forms of art Applications to field design on surfaces

75 Thank you! Acknowledgements: Seok-Hyung Bae, Patrick Coleman, Vikramaditya Dasgupta, Mark Hazen, Thomas Hendry, Olga Vesselova, Olga Veksler, Robert Kalnins, Philip Davidson, David Bourguignon, Xiaobai Chen, Aleksey Golovinskiy, Thomas Funkhouser, Andrea Tagliasacchi, Richard Zhang, VAKHUN, Cyberware repositories