See the end of this document for submission instructions. Please see Piazza for questions and discussion from the class.

Transcription

1 CS 378H Hnrs Machine Learning and Visin Fall 2015 Assignment 1 Out: Thursday Sept 3 Due: Friday Sept 18 11:59 PM See the end f this dcument fr submissin instructins. Please see Piazza fr questins and discussin frm the class. I. Shrt answer prblems [30 pints] 1. Give an example f hw ne can explit the assciative prperty f cnvlutin t mre efficiently filter an image. 2. This is the input image: ([ ]. What is the result f dilatin with a structuring element [1 1 1]? The filter f [ 0,,0,,0] gives an estimate f the first derivative f the image in the x 2 2 directin. What is the crrespnding secnd derivative filter? (Hint: asymmetric filters must be flipped prir t cnvlutin.) f 4. Name tw specific ways in which ne culd reduce the amunt f fine, detailed edges that are detected with the Canny edge detectr. 5. Describe a pssible flaw in the use f additive Gaussian nise t represent image nise. 6. Design a methd that takes vide data frm a camera perched abve a cnveyr belt at an autmtive equipment manufacturer, and reprts any flaws in the assembly f a part. Yur respnse shuld be a list f cncise, specific steps, and shuld incrprate several techniques cvered in class thus far. Specify any imprtant assumptins yur methd makes.

2 II. Prgramming prblem: cntent-aware image resizing [70 pints] Fr this exercise, yu will implement a versin f the cntent-aware image resizing technique described in Shai Avidan and Ariel Shamir s SIGGRAPH 2007 paper, Seam Carving fr Cntent- Aware Image Resizing. The paper is available ff the curse website. The gal is t implement the methd, and then examine and explain its perfrmance n different kinds f input images. First read thrugh the paper, with emphasis n sectins 3, 4.1, and 4.3. Nte: chsing the next pixel t add ne at a time in a greedy manner will give sub-ptimal seams; the dynamic prgramming slutin ensures the best seam (cnstrained by 8-cnnectedness) is cmputed. Use the dynamic prgramming slutin as given in the paper and explained in class. Write Matlab cde with functins that can d the fllwing tasks: Cmpute the energy functin at each pixel using the magnitude f the x and y gradients (equatin 1 in the paper) Cmpute the ptimal vertical seam given an image Cmpute the ptimal hrizntal seam given an image Reduce the image size by a specified amunt in ne dimensin (width r height decrease) Display the selected seam n tp f an image Functins with the fllwing interface: [utput] = reducewidth(im, numpixels) [utput] = reduceheight(im, numpixels) These functins take an input image im, and a parameter specifying hw many seams t carve, frm the width r height, respectively. The image im will be a h x w x 3 uint8 matrix, which is what imread returns fr a clr image. Put these functins in file named reducewidth.m and reduceheight.m

3 Set up scripts s that yu can play with the seam remval and specify different cmbinatins f hrizntal and vertical remvals. Apply yur system t the prvided images. View the results in clr, but nte that the gradients shuld be cmputed with the grayscale cnverted image. Matlab hints: Useful functins: imfilter, im2duble, fspecial, imread, imresize, rgb2gray, imagesc, imshw, subplt; T plt pints n tp f a displayed image, use imshw(im); fllwed by hld n; fllwed by plt( ). Be careful with duble and uint8 cnversins as yu g between cmputatins with the images and displaying them filtering shuld be dne with dubles. Answer each f the fllwing, and include image displays where apprpriate: 1. [10 pints] Run yur reduceheight functin n the prvided prague.jpg with numpixels = 100 (in ther wrds, shrink the height by 100 pixels). Run yur reducewidth functin n the prvided mall.jpg with numpixels = 100 (in ther wrds, shrink the width by 100 pixels). Display the utputs. 2. [10 pints] Display (a) the energy functin utput (ttal gradient magnitudes e1(i)) fr the prvided image prague.jpg, and (b) the tw crrespnding cumulative minimum energy maps (M) fr the seams in each directin (use the imagesc functin). Explain why these utputs lk the way they d given the riginal image s cntent. 3. [10 pints] Fr the same image prague.jpg, display the riginal image tgether with (a) the first selected hrizntal seam and (b) the first selected vertical seam. Explain why these are the ptimal seams fr this image. 4. [10 pints] Make sme change t the way the energy functin is cmputed (i.e., filter used, its parameters, r incrprating sme ther a prir knwledge). Display the result and explain the impact n the results fr sme example. 5. [30 pints] Nw, fr the real results! Use yur system with different kinds f images and seam cmbinatins, and see what kind f interesting results it can prduce. The gal is t frm sme perceptually pleasing utputs where the resizing better preserves cntent than a blind resizing wuld, as well as sme examples where the utput lks unrealistic r has artifacts. Include results fr the tw prvided images, plus at least three images f yur wn chsing. Include an example r tw f a bad utcme. Be creative in the images yu chse, and in the amunt f cmbined vertical and hrizntal carvings yu apply. Try t predict types f images where yu might see smething interesting happen. It s k t fiddle with the parameters (seam sequence, number f seams, etc) t lk fr interesting and explainable utcmes. Fr each result, include the fllwing things, clearly labeled (see title functin): (a) the riginal input image, (b) yur system s resized image, (c) the result ne wuld get if instead a simple resampling were used (via Matlab s imresize), (d) the input and utput image dimensins, (e) the sequence f enlargements and remvals that were used, and (f) a qualitative explanatin f what we re seeing in the utput.

4 III. [OPTIONAL] Extra credit [up t 10 pints each, max pssible 20 pints extra credit] Belw are ways t expand n the system yu built abve. If yu chse t d any f these (r design yur wn extensin) include in yur writeup an explanatin f the extensin as well as images displaying the results and a shrt explanatin f the utcmes. Als include a line r tw f instructins telling what needs t be dne t execute that part f yur cde. 1. Allw a user t mark an bject t be remved, and then remve seams until all pixels n that bject are gne (as suggested in sectin 4.6 f the paper). Either hard-cde the regin specific t the image, r allw interactive chices (Matlab s ginput r imply functins are useful t get muse clicks r draw plygns). 2. Design an alternate energy functin, instead f the gradient magnitude. Explain yur chice, and shw hw it can influence the results as cmpared t using the gradient magnitude. Chse an image r tw that illustrates the differences well. 3. T avid warping regins cntaining peple s faces, have the system try t detect skinclred pixels, and let that affect the energy map. Try using the hue (H) channel f HSV clr space (see Matlab s rgb2hsv functin t map t HSV clr space). Think abut hw t translate thse values int energy functin scres. 4. Implement functins t increase the width r height f the input image, blending the neighbring pixels alng a seam. (See the Seam Carving paper fr details.) Demnstrate n an image that clearly shws the impact. 5. Implement the greedy slutin, and cmpare the results t the ptimal DP slutin.

5 Submissin instructins: what t hand in Electrnically: Yur dcumented Matlab cde (including functins reducewidth.m and reduceheight.m). Mark clearly in the cde where each required part is. A first pdf file named file1.pdf cntaining the fllwing: Yur name at the tp Yur answers t Part I, numbered. A brief explanatin f yur implementatin strategy: a shrt paragraph r tw describing in English what yu have cmputed. Yur respnses and image results fr questins 1 thrugh 4 in Part II, numbered. Insert image figures in the apprpriate places fr these questins. A secnd pdf file named file2.pdf cntaining the fllwing: Yur name at the tp Yur respnses and image results fr questin 5 in Part II. (ptinal): any results and descriptins fr extra credit prtins. Be sure t indicate which items yu did frm the extra credit list. This file will be psted nline be sure t credit any pht surces. Tip: Hw t save as pdf? If yu have a pdf printer installed n yur cmputer, yu can cnvert a dcument prepared in Wrd t pdf. The CS machines have penffice installed, which will als allw yu t Save as a pdf file. Or, if yu wrk in Latex, yu can use pdflatex, r cmpile t a ps and then cnvert. Submit all the abve in a single zip file n Canvas. Image acknwledgements: Thanks t the fllwing Flickr users fr sharing their phts under the Creative Cmmns license: mall.jpg is prvided by hey tiffany! prague.jpg david.niknvscann