Fast HighDimensional Filtering Using the Permutohedral Lattice


 Kellie McCormick
 1 years ago
 Views:
Transcription
1 EUROGRAPHICS x / NN an NN (Eitors) Volume (98), Number Fast HihDimensional Filterin Usin the Permutoheral Lattice Anrew Aams, Jonmin Baek, Myers Abraham Davis Stanfor University Abstract Many useful alorithms for rocessin imaes an eometry fall uner the eneral framework of hihimensional Gaussian filterin This family of alorithms inclues bilateral filterin an nonlocal means We roose a new way to erform such filters usin the ermutoheral lattice, which tessellates hihimensional sace with uniform simlices Our alorithm is the first imlementation of a hihimensional Gaussian filter that is both linear in inut size an olynomial in imensionality Furthermore it is arameterfree, aart from the filter size, an achieves a consistently hih accuracy relative to roun truth (> 45 B) We use this to emonstrate a number of interactiverate alications of filters in as hih as eiht imensions Cateories an Subject Descritors (accorin to ACM CCS): Enhancement Filterin I43 [Imae Processin an Comuter Vision]: Introuction Hihimensional Gaussian filterin (Equation ) is a owerful way to exress a smoothness rior on ata in an arbitrary Eucliean sace As such, it is an imortant comonent of many alorithms in imae rocessin an comuter vision Alorithms such as color or rayscale bilateral filterin [AW95] [SB97] [TM98], joint bilateral filterin [ED4] [PSA 4], joint bilateral usamlin [KCLU7], nonlocal means [BCM5], an satiotemoral bilateral filterin [BM5] can all be exresse as hihimensional Gaussian filters Recent work on acceleratin hihimensional Gaussian filters has focuse on exlicitly reresentin the hihimensional sace with oint samles, usin a reular ri of samles [PD9] or a clou of samles store in a ktree [AGDL9] When the sace is exlicitly reresente in this way, filterin is imlemente by resamlin the inut ata onto the hihimensional samles, erformin a hihimensional Gaussian blur on the samles, an then resamlin back into the inut sace (Fiure ) [AGDL9] terms these three staes slattin, blurrin, an slicin We roose acceleratin such filters by samlin the hihimensional sace at the vertices of the ermutoheral lattice (illustrate in Fiure ) The lattice is comose of ientical v i = n j=e i j v j Equation : Hihimensional Gaussian filterin associates an arbitrary osition i with each value v i to be filtere, an then mixes values with other values that have nearby ositions Usually the values are homoeneous ixel colors If the ositions are twoimensional ixel locations, then this exresses a Gaussian blur If the ositions are ixel locations combine with color, for a total of five imensions, this exresses a color bilateral filter If the osition vectors are erive from local neihborhoos aroun each ixel, then this exresses nonlocal means simlices (hihimensional tetrahera), an the enclosin simlex of any iven oint can be foun by a simle rounin alorithm Slattin an slicin can therefore be one by barycentric interolation, which is exonentially cheaer than the multilinear interolation of [PD9], an oes not suffer from the irreularity an arameterheavy nature of the ranomly bifurcatin ktree queries of [AGDL9] Similarly to the ri aroach, the blurrin stae can be one with a simle searable filter We escribe the lattice an its roerties in section 3 c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt Publishe by Blackwell Publishin, 96 Garsinton Roa, Oxfor OX4 DQ, UK an 35 Main Street, Malen, MA 48, USA
2 A Aams & J Baek & M A Davis / Fast HihDimensional Filterin Usin thepermutoheral Lattice Inut Outut Slat Blur Slice Fiure : Hihimensional Gaussian filterin can be imlemente by first embein the inut values v at ositions in a hihimensional sace (slattin), then erformin a Gaussian blur in that sace (blurrin), then samlin the sace at the oriinal ositions (slicin) The iaram above illustrates a bilateral filter of a oneimensional sinal usin this framework Usin the ermutoheral lattice for hihimensional filterin of n values in imensions has a time comlexity of O( n) an a sace comlexity of O(n) This comares favorably to existin techniques, an in fact this metho roves to be faster than the state of the art for a wie rane of filter sizes an imensionalities (Fiure 7) We escribe its erformance an accuracy in section 4 The runtime is sufficiently low that we can emonstrate the first realtime color bilateral filter (a fiveimensional filter) The qualitative chane brouht about by this see increase is that we enable interactive use of this class of filter, escribe in section 5 Prior Work Hihimensional Gaussian filterin inclues, as secial cases, the bilateral filter, an the nonlocal means filter In turn, hihimensional Gaussian filterin is a tye of Gauss transform, in which a weihte sum of Gaussians, centere at oints i, is samle at some other locations q i For our case (Equation ), the weihts are always homoeneous vectors, usually storin color, an i tyically equals q i There are thus several cateories of relate work: Bilateral filters an attemts to accelerate them; nonlocal means an corresonin accelerations; an more eneral attemts to accelerate comutation of the fast Gauss transform We will iscuss each in turn The Bilateral Filter Bilateral filterin [AW95] [SB97] [TM98] averaes ixels with other ixels that are nearby in both osition an intensity It is a fiveimensional Gaussian filter (Eq ) in which v i = [r i, i,b i,] (the homoeneous color at ixel i), an i = [ σ xi s, yi σ s, σ ri c, i σ c, σ bi c ], where σ s is the satial stanar eviation of the filter an σ c is the colorsace stanar eviation Grayscale bilateral filterin can be exresse by relacin r i, i, an b i with just luminance, an is hence a threeimensional filter Some notable uses of the bilateral filter inclue tone main [DD], halofree sharenin, an hotorahic tone transfer [BPD6] Bilateral filters of imensionalities other than 3 an 5 also occur Weber et al [WMM 4] use a fourimensional bilateral filter with a temoral term for smoothin hoton ensity mas for renerin raytrace sequences Aams et al [AGDL9] similarly a a temoral term to a color bilateral filter to enoise vieo, which results in a 6D filter Joint Bilateral Filters By erivin i from one imae an v i from another, one can smooth an imae in a way that oes not cross the ees of another This technique was invente ineenently by Eisemann an Duran [ED4] an Petschni et al [PSA 4] an was use by each for combinin imaes taken with an without flash This iea was extene by Kof et al [KCLU7] to use as an usamlin technique By slattin at ositions etermine by the lowresolution inut, an then slicin usin a hihresolution reference imae, the low resolution ata can be interolate in a way that resects ees in the hihresolution reference Fast Bilateral Filters Given the wie ranin utility of the bilateral filter, consierable research has one into acceleratin it Duran an Dorsey [DD] iscretize in intensity, an comute a reular Gaussian blur at each intensity level usin an FFT, which can then be interolate between to rouce a rayscale bilateral filter This aroach evolve into the bilateral ri [PD6] of Paris an Duran, which was imlemente on a GPU for various interactive alications by Chen et al [CPD7] The bilateral ri iscretizes in sace an intensity, an introuce the slatblurslice ieline for hihimensional filterin (Fiure ) Paris an Duran also escribe a fiveimensional ri for color bilateral filterin [PD9] The work of Duran an Dorsey [DD] was ineenently extene by Porikli [Por8] who observe that the FFTs were unnecessary, an use faster interalimae base methos to blur each intensity level, thus roucin interal historams Yan et al [YTA9] imrove on this by not exlicitly reresentin the entire sace, but instea sweein a lane throuh the intensity levels, comutin the outut in intensity orer Their lowmemory, cachefrienly alorithm is the fastest known rayscale bilateral filter However, the laneswee aroach oes not eneralize well to hiher imensions NonLocal Means Bennett an McMillan [BM5], while enoisin vieo with a combination of bilateral filters, note that the ositions i coul be usefully aumente by incluin the color or intensity of nearby ixels as well This is equivalent to nonlocal c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt
3 A Aams & J Baek & M A Davis / Fast HihDimensional Filterin Usin thepermutoheral Lattice means of Buaes et al [BCM5] [BCM8], which is an otimal enoiser for aitive white noise y Fast NonLocal Means There are numerous methos for acceleratin nonlocal means that o not exlicitly reresent the hihimensional sace For examle, Mahmoui an Sairo [MS5] reclassify reions of the imae accorin to averae intensity an raient irection in orer to restrict the search, whereas Darbon et al [DCC 8] comute interal imaes of certain error terms, as o Wan et al [WGY 6] However, this family of methos is restricte to osition vectors comose of rectanular imae atches, an oes not eneralize to less structure enoisin tasks, such as the eometry enoisin [AGDL9] an the aforementione hoton ensity enoisin [WMM 4] z (,, ) (,,) (, , ) x > y > z x  z < 3 x Fast Gauss Transforms Aroaches for evaluatin Equation as a eneric Gauss transform have focuse on treebase reresentations of the hihimensional sace of osition vectors Aams et al use a ktree coule with ranomize queries, whereas Brox et al [BKC8] use a cluster tree Yan et al [YDGD3] escribe the imrove fast Gauss transform, which uses a cluster tree in which each leaf stores not only a value, but also some Taylor series exansion terms escribin how the value varies about that leaf It is extremely accurate, but not as fast as other methos 3 The Permutoheral Lattice We now escribe the lattice we will use to accelerate hihimensional Gaussian filters The imensional ermutoheral lattice is the rojection of the scale reular ri ( + )Z + alon the vector = [,,] onto the hyerlane H : x =, which is the subsace of R + in which coorinates sum to zero It is hence sanne by the rojection of the stanar basis for ( + )Z + onto H : B = Each of the ( + ) basis vectors (columns of B ) has coorinates that sum to zero, an that each coorinate of each basis vector has a consistent remainer moulo + Both of these roerties are reserve when takin inteer combinations, so oints in the lattice are those oints with inteer coorinates that sum to zero an have a consistent remainer moulo + We escribe a lattice oint whose coorinates have a remainer of k as a remainerk oint In Fiure we show the lattice for =, an label each lattice oint by its remainer The ermutoheral lattice has several roerties that make Fiure : The imensional ermutoheral lattice is forme by rojectin the scale ri ( + )Z + onto the lane x = This forms the lattice ( + )A, which we term the ermutoheral lattice, as it escribes how to tile sace with ermutohera Lattice oints have inteer coorinates with a consistent remainer moulo + In the iaram above, which illustrates the case =, oints are labele an colore accorin to their remainer The lattice tessellates the lane with uniform simlices, each simlex havin one vertex of each remainer The simlices are all translations an ermutations of the canonical simlex (hihlihte), which is efine by the inequalities x > x > > x an x x < + it wellsuite for hihimensional filterin usin the slatblurslice ieline illustrate in Fiure Firstly, it tessellates hihimensional sace with uniform simlices, so we can use barycentric interolation to slat the sinal onto the lattice oints Seconly, the enclosin simlex of any oint, alon with its barycentric coorinates, can be comute quickly (in O( ) time), so slattin is fast Thirly, the neihbors of a lattice oint are trivial to comute, so the blur stae is also fast Finally, slicin can be one usin the barycentric weihts alreay comute urin the slattin stae, an so is also fast These roerties are escribe briefly below Full erivations, roofs, an further helful roerties can be foun in [BA9] The ermutoheral lattice tessellates H with uniform simlices Consier the imensional simlex whose vertices s,, s are iven by: s k = [k,,k, k ( + ),,k ( + ) ] } {{ } } {{ } + k k c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt
4 A Aams & J Baek & M A Davis / Fast HihDimensional Filterin Usin thepermutoheral Lattice y of l x (Fiure 3) As every oint belons to a unique simlex, which is a ermutation an translation of the canonical simlex, H is tessellate by uniform simlices z z > y > x y > z > x z > x > y y > x > z x > z > y x > y > z Fiure 3: When usin the ermutoheral lattice to tessellate the subsace H, any oint x H is enclose by a simlex uniquely ientifie by the nearest remainer lattice oint l (the zeroes hihlihte in re) an the orerin of the coorinates of x l The nearest remainer lattice oint can be comute with a simle rounin alorithm, an so ientifyin the enclosin simlex of any oint an enumeratin its vertices is comutationally chea (O( )) We term this simlex the canonical simlex For examle, when = 4 the vertices are the columns of: Note that s k is a lattice oint of remainer k (ie its coorinates are all conruent to k moulo + ), an the simlex inclues one oint of each remainer The vertices of this simlex are the bounary cases of the inequalities x x x an x x +, an a oint lies within the simlex if an only if it obeys these inequalities Now consier any ermutation ρ of the coorinates of the canonical simlex Each ρ inuces a corresonin orerin of the coorinates x ρ() x ρ() x ρ(), an the inequality x ρ() x ρ() < + Takin the union of these inequalities across all ( + )! simlices results in the set {x i max i x i min i x i +} (the central hexaon in Fiure 3), which is in fact the set of all oints which have the oriin as their closest remainer oint (See Proosition A) Hence, as the lattice is translation invariant, a oint x H, with closest remainer oint l, belons to a unique simlex etermine by l an the orerin of the coorinates x The vertices of the simlex containin any oint in H can be comute in O( ) time This roerty will be useful for the slat an slice staes of filterin The vertices of the simlex containin some oint x H can be enerate by first comutin the closest remainer oint l, an then sortin the ifference l x The resultin ermutation an translation can then be alie to the canonical simlex to comute the simlex vertices in O( ) oerations The closest remainer oint can be foun by first rounin each coorinate of x to the nearest multile of ( + ), an then, if the result is outsie the subsace H, reeily walkin back to H by rounin those coorinates that move the farthest in the other irection instea The sublattice forme by the remainer oints is calle A +, an this is the alorithm iven by Conway an Sloane ( [CS99] 446) for finin the closest oint in that lattice The nearest neihbors of a lattice oint can be comute in O( ) time This roerty will be useful urin the blur stae of filterin The basis vectors iven by B above are those of minimal lenth, so the nearest neihbors of a lattice oint l k are those searate by a vector of the form ±[,,,,,, ] The are ( + ) such neihbors, an each is escribe by a vector of lenth +, an so the neihbors can be fully enumerate in O( ) time 3 Filterin usin the Lattice There are four main staes for usin the ermutoheral lattice for hihimensional Gaussian filterin, illustrate in Fiure 4 We will escribe each in turn Generatin osition vectors Firstly, the osition vectors i, reresentin the locations in the hihimensional sace, must be enerate an embee in H Generatin the ositions is somewhat alication eenent For a color bilateral filter, we enerate 5D osition vectors of the form [ σ xi, yi σ, σ ri c, i σ c, σ bi c ], by aumentin the inut imae with two extra channels encoin satial location, an then scalin each channel by the inverse of the esire stanar eviation For nonlocal means we woul instea either extract local winows aroun each ixel, or comute some bank of filters aroun each ixel an recor the resonses Tyically we o the latter, usin PCA to etermine the otimal filter bank, as first roose by Tasizen [Tas8] We must then scale the osition vectors by the inverse of the stanar eviation of the blur inuce by the remainin stes, which totals 3 ( + ) in each imension (erive c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt
5 A Aams & J Baek & M A Davis / Fast HihDimensional Filterin Usin thepermutoheral Lattice Slat Blur Slice Fiure 4: To erform a hihimensional Gaussian filter usin the ermutoheral lattice, first the osition vectors i R are embee in the hyerlane H usin an orthoonal basis for H (not icture) Then, each inut value slats onto the vertices of its enclosin simlex usin barycentric weihts Next, lattice oints blur their values with nearby lattice oints usin a searable filter Finally, the sace is slice at each inut osition usin the same barycentric weihts to interolate outut values below) Next we embe the osition vectors in the subsace H The basis for H iven above is unsuitable for this task because it is not orthoonal, so we instea use the orthoonal basis: b = b j, j= b i = y i y i+ + E = 6 (+) We choose this basis because it allows us to comute E x in O() time usin the recurrence: (E x) = α x (E x) i = α i x i + x i /α i+ + (E x) i+ (E x) = x i /α + (E x) α i = i/(i + ) Slattin Once each osition has been embee in the hyerlane, we must ientify its enclosin simlex an comute barycentric weihts The enclosin simlex of any oint can be escribe by the ermutation an translation that mas the simlex back to the canonical simlex, which can be comute in O( lo) by usin the rounin alorithm escribe earlier to fin the nearest remainer oint, an then sortin the resiual Therefore, to comute barycentric coorinates for a oint E i in an arbitrary simlex, we can aly the translation an ermutation to ma E i to some y within the canonical simlex Barycentric coorinates b for y are then iven by the exression below, as shown by Proosition A Barycentric interolation is invariant to translation an ermutation, an so these barycentric coorinates for y within the canonical simlex are also the barycentric coorinates for E i within its simlex Once the barycentric weihts are comute, b k v i is ae to the value store at the remainerk lattice oint in the enclosin simlex of i (recall that v i is the homoeneous value associate with osition i ) The lattice oint values are store in a hash table Lattice oints that o not yet exist in the hash table are create when they are first referre to urin the slat stae, an start with an initial value of zero There are two ways to ientify each lattice oint for use as a hash table key One can aly the inverse ermutation an translation to the remainerk oint of the canonical simlex to comute the lattice oint s osition, an use that as a key Each key is a vector of lenth +, an so this results in a memory comlexity of O(l) for l lattice oints In rare cases where l > n, we can alternatively achieve a memory comlexity of O(n) for n inut values by searately storin the simlex enclosin each inut osition i, as a simlex can be ientifie uniquely in O() memory by its remainer oint an its ermutation We then ientify a lattice oint usin its remainer an a ointer to any simlex it belons to, for an aitional O(n) memory One lattice oint belons to many simlices, so key comarison is one by usin the simlex an remainer to comute the lattice oint s coorinates on the fly l is loosely boune by O(n), as each inut value creates at most + new lattice oints However, filters near this boun correson to very small filter sizes an are not very useful, as no share lattice oints means no crosstalk between ixels, an hence no filterin In ractice, we fin that l < n, so we refer the first, faster scheme In either case, c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt
6 A Aams & J Baek & M A Davis / Fast HihDimensional Filterin Usin thepermutoheral Lattice each hash table access costs O() time for key comarison Each ixel accesses the hash table O() times, an so the time comlexity of slattin is O( n) Barycentric interolation in the ermutoheral lattice is equivalent to convolution by the rojection of a uniformlyweihte ( +)imensional hyercube of sie lenth + onto H, an inuces a total variance of ( + ) / (See Proosition A3) Blurrin Now that our values are embee in the subsace H, the next stae of hihimensional Gaussian filterin is erformin a reular Gaussian blur within that subsace To o this we convolve by the kernel [ ] alon each lattice irection of the form ±[,,,,,,] (Fiure 4) This rouces an aroximate Gaussian kernel with total variance ( + ) / The blur stae sreas enery from each lattice oint to O(3 ) neihbors If we create hash table entries for new lattice oints reache urin the blur then the memory use woul row quite lare We therefore o not create new lattice oints urin the blur hase, which incurs some accuracy enalty relative to a naive comutation of Equation, as oints that may have transferre enery coul instea belon to isconnecte reions of the lattice Aams et al [AGDL9] observe a similar effect, an arue it may actually be avantaeous For examle, when bilateral filterin, the absence of these steinstone lattice oints will revent enery transfer from a white ixel to a black ixel across a har ee, but will allow enery transfer between a black ixel an a white ixel on either sie of a smooth raient The blur ste involves lookin u O() neihbors for each lattice oint Each looku takes O() time for hash table key comarison, an so the blur ste has time comlexity O( l) Slicin Slicin is ientical to slattin, excet that it uses the barycentric weihts to ather from the lattice oints instea of scatterin to them It rouces the same total variance of ( + ) /, which brins the total variance inuce by the alorithm to 3 ( +), which is equivalent to a stanar eviation in each imension of 3 ( + ) Slicin can be accelerate by storin the barycentric weihts an ointers to lattice oint values comute urin slattin This slicin table can then be scanne throuh in O(n) time to slice The entire alorithm thus has a time comlexity of O( (n + l)) 3 GPU Imlementation The above alorithm is fairly straihtforwar to arallelize on a GPU We constructe an imlementation usin NVIDIA s CUDA [Buc7], an achieve tyical seeus of Fiure 5: At the to is a 5x56 cro of the inut imae use for time an memory comarisons  a tyical 5 meaixel hotorah Below is the same cro of the outut of the ermutoheral lattice use to erform a color bilateral filter with a satial stanar eviation of 6 ixels an a color stanar eviation of 8 It is visually inistinuishable from the naive result The ermutoheral lattice rouces a result with a PSNR between 45 an 5 B relative to an exhaustive evaluation of Equation, eenin on filter size an imensionality 6x on a Geforce GTX8 comare to the sinlethreae CPU imlementation on an Intel Core i7 9 Clearly a hashtable benefits from a cache The main oint of ifference between the CPU an GPU versions is relate to the creation of hash table entries urin the slattin stae It is customary to attach locks to hash table entries an synchronize all accesses to a iven entry to revent erroneously insertin one key in multile laces We foun it faster to break the slattin stae into three First, we comute the slicin table, recorin which lattice oints each inut ixel slats to, an with what weihts While oin this, we insert the lattice oints foun into the hash table in a way which ermits iniviual keys bein inserte in multile locations Secifically, while we still lock each hash table entry before insertion, other simultaneous hash table insertions simly ski over locke entries while lookin for a free sot rather than waitin on the lock to see if the key matches This means we never have ata eenencies involvin one query reain the key that another query has written, so we can write the keys usin faster nonatomic writes, an only the smaller array of locks nees to be coherent Next, we rehash the entries of the slicin table an uate it so that every reference to a lattice oint refers to the unique earliest instance of that lattice oint in the hash c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt
7 A Aams & J Baek & M A Davis / Fast HihDimensional Filterin Usin thepermutoheral Lattice 3D 5D 8D 6D n n n n f f f s s f s s s Time Taken s r s s GB GB Memory Use s s s r MB MB MB Filter Satial Stanar Deviation (ixels) n Naive Dense Gri s Sarse Gri Permutoheral Lattice Gaussian KDTree f Fast Gauss Transform r RTBF Fiure 6: Here we show time taken an memory use as a function of filter size an imensionality for a variety of alorithms The inut was a tyical 5 meaixel hotorah (Fiure 5) Position vectors were comose of ixel locations, ixel luminance, ixel chrominance, an then increasin numbers of filter resonses over a Gaussianweihte 9x9 local winow, eenin on the esire imensionality These filters were erive usin PCA on the sace of such local winows, an are tyically local erivatives The stanar eviation for all but the satial terms is fixe at 8 The methos comare are the naive winowe bilateral filter (n), the ense bilateral ri () [CPD7], the sarse bilateral ri (s), the ermutoheral lattice (), the Gaussian KDTree () [AGDL9], the imrove fast Gauss transform (f) [YDGD3], for which memory use is unavailable, an realtime bilateral filterin (r) [YTA9], which oes not aly to imensionalities above three The arameters of each alorithm were tune so that it ran as quickly as ossible while achievin a PSNR of rouhly 45B relative to an exhaustive evaluation of Equation table Finally, we use the correcte slicin table to slat, aitively scatterin onto lattice oints as usual In an interactive settin, it is common to erform many filters whilst only chanin a few arameters in between runs If the osition vectors an filter sizes o not chane, then the ol slicin table an hash table can be reuse for a moerate seeu 4 Results an Comarisons Here we resent results of a comarison of a number of alorithms for erformin hihimensional filterin on a tyical hotorah All alorithms were run sinlethreae on an Intel Core i7 9 runnin at 67 GHz Only alorithms with source coe available were use, an all lowlevel otimizations were left u to the comiler The inut hotorah an the outut of the ermutoheral lattice are shown in Fiure 5 The runnin times an memory use of each alorithm for a few ifferent imensionalities are shown in Fiure 6 Fiure 7 illustrates the fastest metho for a variety of imensionalities an filter sizes The ermutoheral lattice is sinificantly faster than the state of the art for a lare rane of imensionalities an filter sizes 4 The Alorithms Naive: This alorithm consiers all ixels within three satial stanar eviations of each ixel an manually comutes Equation While this alorithm scales linearly with imensionality, it is quaratic in filter size, an oes not work for unstructure ata such as eometry Dense Gri: This is the bilateral ri of Paris et al [PD9], usin multilinear slattin an slicin, an a searable blur kernel of [ ] in each imension It scales exonentially with imensionality Sarse Gri: The two major ifferences between the ermutoheral lattice an the bilateral ri are the lattice use, an also the fact that the ermutoheral lattice stores values sarsely in a hash table In orer to isambiuate these two effects, we constructe a sarse bilateral ri alorithm that uses the same hash table imlementation Similarly to the Gaussian KDTree an the ermutoheral lattice, we o not allocate new lattice oints urin the blur stae However, the time an memory comlexity are still exonential in, an inee this technique is slower than the ense bilateral ri, while savin only a moerate amount of memory This c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt
8 A Aams & J Baek & M A Davis / Fast HihDimensional Filterin Usin thepermutoheral Lattice Fiure 7: This contour lot shows the fastest metho for each imensionality an satial filter size, an how many times faster it is than the secon fastest metho The bilateral ri [CPD7] is best for imensionalities three an four The Gaussian KDTree [AGDL9] is best for hih imensionalities an small filter sizes The ermutoheral lattice is the fastest metho for imensionalities from 5 (color bilateral filterin) u to aroun, eenin on the filter size Run times were samle at the small ots an interolate Only methos caable of arbitraryimensional filters were comare, an the arameters of each metho were tune to achieve an PSNR relative to roun truth of rouhly 45B Filter Dimensionality x 4x Gaussian KDTree x Bilateral Gri Permutoheral Lattice Filter Stanar Deviation x x x x 4x 4x 8x NonLocal Means Bilateral Filters shows that it is our choice of lattice, rather than our sarsity, which rovies the avantae Gaussian KDTree: This is the Gaussian KDTree of Aams et al [AGDL9] Its run time is fairly constant across filter size an imensionality, an its memory use is sinificantly lower than the other methos for imensionalities 5 an u Imrove Fast Gauss Transform: This is the fast multiole metho of evaluatin the Gauss Transform of C Yan et al [YDGD3] It is a fully eneral metho caable of extremely hih accuracy, but even when tune for see, it is not articularly fast comare to the more aroximate methos use in imae filterin RealTime O() Bilateral Filterin: This is the metho of Q Yan et al [YTA9] It is the fastest known metho for rayscale bilateral filters ( = 3), but oes not scale with imensionality 5 Interactive Alications Imaes rouce by alications of hihimensional Gaussian filterin such as nonlocal means an the bilateral filter can een heavily on the stanar eviations chosen The hih see an arallelizability of the ermutoheral lattice make these filters easier to use by lettin users interactively exlore the arameter sace Our CUDA imlementation of the lattice filters 8x6 imaes at fs We use this to imlement the followin interactive alications Reaers are encourae to view the vieo accomanyin this aer to see these alications emonstrate Interactive Color Bilateral Filterin The user is resente with sliers to control the stanar eviations for each of the imensions of the filter (x, y, r,, b) The result of chanin any of these values is resente immeiately, makin it easy to exlore the sace of bilateral filters Interactive Nonlocal Means As a rerocess, PCA is erforme on the sace of 7x7 atches centere about each ixel in the imae, in orer to reuce imensionality without losin istance relationshis We use six PCA terms in aition to the two satial terms as osition imensions Six imensions is the number sueste by Tasizen [Tas8], who emonstrate that erformin nonlocal means in this way rouces suerior results to nonlocal means without imensionality reuction Once aain the user is resente with sliers to control the stanar eviations, an the result is resente in real time, allowin for interactiverate enoisin Nonlocal Means Eitin Nonlocal means can also be use for makin eits to an imae that roaate across similar colors, textures, or brihtnesses, in the same manner as the roaatin eits of [AP8] By choosin aroriate osition imensions, local eits can be mae to resect bounaries with resect to any set of local escritors The user alies aroximate eits with a few strokes Each stroke aints values on a mask in those locations The mask is then filtere with resect to the osition imensions For our osition imensions, we use six PCA terms an two satial terms, which catures chanes in brihtness, color, an texture The filtere mask, which resects ees in the osition imensions, serves as an influence ma for how the eit shoul be alie For ajustin brihtness, the user aints ark or briht values into the mask, an we simly multily the inut imae by the filtere mask The filterin is one at interactive rates, so the user can see the fully roaate eit as they aly their rouh strokes c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt
Which Networks Are Least Susceptible to Cascading Failures?
Which Networks Are Least Susceptible to Cascaing Failures? Larry Blume Davi Easley Jon Kleinberg Robert Kleinberg Éva Taros July 011 Abstract. The resilience of networks to various types of failures is
More informationA New TwIST: TwoStep Iterative Shrinkage/Thresholding Algorithms for Image Restoration
SUBMITTED FOR PUBLICATION; 2007. 1 A Ne TIST: ToSte Iterative Shrinkage/Thresholding Algorithms for Image Restoration José M. BioucasDias, Member, IEEE, and Mário A. T. Figueiredo, Senior Member, IEEE
More informationNew Trade Models, New Welfare Implications
New Trae Moels, New Welfare Implications Marc J. Melitz Harvar University, NBER an CEPR Stephen J. Reing Princeton University, NBER an CEPR August 13, 2014 Abstract We show that enogenous firm selection
More informationComputing the Most Probable String with a Probabilistic Finite State Machine
Comuting the Most Probable String with a Probabilistic Finite State Machine Colin de la Higuera Université de Nantes, CNRS, LINA, UMR6241, F44000, France cdlh@univnantesfr Jose Oncina De de Lenguajes
More informationCrossOver Analysis Using TTests
Chapter 35 CrossOver Analysis Using ests Introuction his proceure analyzes ata from a twotreatment, twoperio (x) crossover esign. he response is assume to be a continuous ranom variable that follows
More informationDatabasefriendly random projections: JohnsonLindenstrauss with binary coins
Journal of Computer an System Sciences 66 (2003) 671 687 http://www.elsevier.com/locate/jcss Databasefrienly ranom projections: JohnsonLinenstrauss with binary coins Dimitris Achlioptas Microsoft Research,
More informationBOSCH. CAN Specification. Version 2.0. 1991, Robert Bosch GmbH, Postfach 30 02 40, D70442 Stuttgart
CAN Specification Version 2.0 1991, Robert Bosch GmbH, Postfach 30 02 40, D70442 Stuttgart CAN Specification 2.0 page 1 Recital The acceptance an introuction of serial communication to more an more applications
More informationImage denoising: Can plain Neural Networks compete with BM3D?
Image denoising: Can plain Neural Networks compete with BM3D? Harold C. Burger, Christian J. Schuler, and Stefan Harmeling Max Planck Institute for Intelligent Systems, Tübingen, Germany http://people.tuebingen.mpg.de/burger/neural_denoising/
More informationWhat Makes an Effective Coalition?
MARCH 2011 What Makes an Effective Coalition? EvidenceBased Indicators of Success Funded by and reared for: TCC Grou Team and Acknowledgements This aer was reared by Jared Raynor with extensive research
More informationThere are two different ways you can interpret the information given in a demand curve.
Econ 500 Microeconomic Review Deman What these notes hope to o is to o a quick review of supply, eman, an equilibrium, with an emphasis on a more quantifiable approach. Deman Curve (Big icture) The whole
More informationLarge Scale Online Learning of Image Similarity Through Ranking
Journal of Machine Learning Research 11 (21) 1191135 Submitted 2/9; Revised 9/9; Published 3/1 Large Scale Online Learning of Image Similarity Through Ranking Gal Chechik Google 16 Amphitheatre Parkway
More informationDistinctive Image Features from ScaleInvariant Keypoints
Distinctive Image Features from ScaleInvariant Keypoints David G. Lowe Computer Science Department University of British Columbia Vancouver, B.C., Canada lowe@cs.ubc.ca January 5, 2004 Abstract This paper
More informationFrom Few to Many: Illumination Cone Models for Face Recognition Under Variable Lighting and Pose. Abstract
To Appear in the IEEE Trans. on Pattern Analysis and Machine Intelligence From Few to Many: Illumination Cone Models for Face Recognition Under Variable Lighting and Pose Athinodoros S. Georghiades Peter
More informationScalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights
Seventh IEEE International Conference on Data Mining Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights Robert M. Bell and Yehuda Koren AT&T Labs Research 180 Park
More informationIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 4, APRIL 2006 1289. Compressed Sensing. David L. Donoho, Member, IEEE
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 4, APRIL 2006 1289 Compressed Sensing David L. Donoho, Member, IEEE Abstract Suppose is an unknown vector in (a digital image or signal); we plan to
More informationMultiresolution Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns
Ref: TPAMI 112278 Multiresolution Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns Timo Ojala, Matti Pietikäinen and Topi Mäenpää Machine Vision and Media Processing
More informationHighRate Codes That Are Linear in Space and Time
1804 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 7, JULY 2002 HighRate Codes That Are Linear in Space and Time Babak Hassibi and Bertrand M Hochwald Abstract Multipleantenna systems that operate
More informationON THE DISTRIBUTION OF SPACINGS BETWEEN ZEROS OF THE ZETA FUNCTION. A. M. Odlyzko AT&T Bell Laboratories Murray Hill, New Jersey ABSTRACT
ON THE DISTRIBUTION OF SPACINGS BETWEEN ZEROS OF THE ZETA FUNCTION A. M. Odlyzko AT&T Bell Laboratories Murray Hill, New Jersey ABSTRACT A numerical study of the distribution of spacings between zeros
More informationFeature Sensitive Surface Extraction from Volume Data
Feature Sensitive Surface Extraction from Volume Data Leif P. Kobbelt Mario Botsch Ulrich Schwanecke HansPeter Seidel Computer Graphics Group, RWTHAachen Computer Graphics Group, MPI Saarbrücken Figure
More informationSpaceTime Video Completion
SpaceTime Video Completion Y. Wexler E. Shechtman M. Irani Dept. of Computer Science and Applied Math The Weizmann Institute of Science Rehovot, 7600 Israel Abstract We present a method for spacetime
More informationVirtualize Everything but Time
Virtualize Everything but Time Timothy Broomhead Laurence Cremean Julien Ridoux Darryl Veitch Center for UltraBroadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering,
More informationTowards Lineartime Incremental Structure from Motion
Towards Lineartime Incremental Structure from Motion Changchang Wu University of Washington Abstract The time complexity of incremental structure from motion (SfM) is often known as O(n 4 ) with respect
More informationLearning Invariant Features through Topographic Filter Maps
Learning Invariant Features through Topographic Filter Maps Koray Kavukcuoglu Marc Aurelio Ranzato Rob Fergus Yann LeCun Courant Institute of Mathematical Sciences New York University {koray,ranzato,fergus,yann}@cs.nyu.edu
More informationOutOfCore Algorithms for Scientific Visualization and Computer Graphics
OutOfCore Algorithms for Scientific Visualization and Computer Graphics Course Notes for IEEE Visualization 2002 Boston, Massachusetts October 2002 Organizer: Cláudio T. Silva Oregon Health & Science
More informationSteering User Behavior with Badges
Steering User Behavior with Badges Ashton Anderson Daniel Huttenlocher Jon Kleinberg Jure Leskovec Stanford University Cornell University Cornell University Stanford University ashton@cs.stanford.edu {dph,
More informationBetter to Give than to Receive: Predictive Directional Measurement of Volatility Spillovers
Forthcomin, International Journal of Forecastin Better to Give than to Receive: Predictive Directional Measurement of Volatility Spillovers Francis X. Diebold University of Pennsylvania and NBER fdiebold@sas.upenn.edu
More informationCOSAMP: ITERATIVE SIGNAL RECOVERY FROM INCOMPLETE AND INACCURATE SAMPLES
COSAMP: ITERATIVE SIGNAL RECOVERY FROM INCOMPLETE AND INACCURATE SAMPLES D NEEDELL AND J A TROPP Abstract Compressive sampling offers a new paradigm for acquiring signals that are compressible with respect
More informationTHE PROBLEM OF finding localized energy solutions
600 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 45, NO. 3, MARCH 1997 Sparse Signal Reconstruction from Limited Data Using FOCUSS: A Reweighted Minimum Norm Algorithm Irina F. Gorodnitsky, Member, IEEE,
More informationHow to Use Expert Advice
NICOLÒ CESABIANCHI Università di Milano, Milan, Italy YOAV FREUND AT&T Labs, Florham Park, New Jersey DAVID HAUSSLER AND DAVID P. HELMBOLD University of California, Santa Cruz, Santa Cruz, California
More information