Fast High-Dimensional Filtering Using the Permutohedral Lattice

Transcription

1 EUROGRAPHICS x / NN an NN (Eitors) Volume (98), Number Fast Hih-Dimensional Filterin Usin the Permutoheral Lattice Anrew Aams, Jonmin Baek, Myers Abraham Davis Stanfor University Abstract Many useful alorithms for rocessin imaes an eometry fall uner the eneral framework of hih-imensional Gaussian filterin This family of alorithms inclues bilateral filterin an non-local means We roose a new way to erform such filters usin the ermutoheral lattice, which tessellates hih-imensional sace with uniform simlices Our alorithm is the first imlementation of a hih-imensional Gaussian filter that is both linear in inut size an olynomial in imensionality Furthermore it is arameter-free, aart from the filter size, an achieves a consistently hih accuracy relative to roun truth (> 45 B) We use this to emonstrate a number of interactive-rate alications of filters in as hih as eiht imensions Cateories an Subject Descritors (accorin to ACM CCS): Enhancement Filterin I43 [Imae Processin an Comuter Vision]: Introuction Hih-imensional Gaussian filterin (Equation ) is a owerful way to exress a smoothness rior on ata in an arbitrary Eucliean sace As such, it is an imortant comonent of many alorithms in imae rocessin an comuter vision Alorithms such as color or rayscale bilateral filterin [AW95] [SB97] [TM98], joint bilateral filterin [ED4] [PSA 4], joint bilateral usamlin [KCLU7], nonlocal means [BCM5], an satio-temoral bilateral filterin [BM5] can all be exresse as hih-imensional Gaussian filters Recent work on acceleratin hih-imensional Gaussian filters has focuse on exlicitly reresentin the hihimensional sace with oint samles, usin a reular ri of samles [PD9] or a clou of samles store in a ktree [AGDL9] When the sace is exlicitly reresente in this way, filterin is imlemente by resamlin the inut ata onto the hih-imensional samles, erformin a hihimensional Gaussian blur on the samles, an then resamlin back into the inut sace (Fiure ) [AGDL9] terms these three staes slattin, blurrin, an slicin We roose acceleratin such filters by samlin the hihimensional sace at the vertices of the ermutoheral lattice (illustrate in Fiure ) The lattice is comose of ientical v i = n j=e i j v j Equation : Hih-imensional Gaussian filterin associates an arbitrary osition i with each value v i to be filtere, an then mixes values with other values that have nearby ositions Usually the values are homoeneous ixel colors If the ositions are two-imensional ixel locations, then this exresses a Gaussian blur If the ositions are ixel locations combine with color, for a total of five imensions, this exresses a color bilateral filter If the osition vectors are erive from local neihborhoos aroun each ixel, then this exresses non-local means simlices (hih-imensional tetrahera), an the enclosin simlex of any iven oint can be foun by a simle rounin alorithm Slattin an slicin can therefore be one by barycentric interolation, which is exonentially cheaer than the multi-linear interolation of [PD9], an oes not suffer from the irreularity an arameter-heavy nature of the ranomly bifurcatin k-tree queries of [AGDL9] Similarly to the ri aroach, the blurrin stae can be one with a simle searable filter We escribe the lattice an its roerties in section 3 c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt Publishe by Blackwell Publishin, 96 Garsinton Roa, Oxfor OX4 DQ, UK an 35 Main Street, Malen, MA 48, USA

2 A Aams & J Baek & M A Davis / Fast Hih-Dimensional Filterin Usin thepermutoheral Lattice Inut Outut Slat Blur Slice Fiure : Hih-imensional Gaussian filterin can be imlemente by first embein the inut values v at ositions in a hih-imensional sace (slattin), then erformin a Gaussian blur in that sace (blurrin), then samlin the sace at the oriinal ositions (slicin) The iaram above illustrates a bilateral filter of a one-imensional sinal usin this framework Usin the ermutoheral lattice for hih-imensional filterin of n values in imensions has a time comlexity of O( n) an a sace comlexity of O(n) This comares favorably to existin techniques, an in fact this metho roves to be faster than the state of the art for a wie rane of filter sizes an imensionalities (Fiure 7) We escribe its erformance an accuracy in section 4 The run-time is sufficiently low that we can emonstrate the first real-time color bilateral filter (a five-imensional filter) The qualitative chane brouht about by this see increase is that we enable interactive use of this class of filter, escribe in section 5 Prior Work Hih-imensional Gaussian filterin inclues, as secial cases, the bilateral filter, an the non-local means filter In turn, hih-imensional Gaussian filterin is a tye of Gauss transform, in which a weihte sum of Gaussians, centere at oints i, is samle at some other locations q i For our case (Equation ), the weihts are always homoeneous vectors, usually storin color, an i tyically equals q i There are thus several cateories of relate work: Bilateral filters an attemts to accelerate them; non-local means an corresonin accelerations; an more eneral attemts to accelerate comutation of the fast Gauss transform We will iscuss each in turn The Bilateral Filter Bilateral filterin [AW95] [SB97] [TM98] averaes ixels with other ixels that are nearby in both osition an intensity It is a five-imensional Gaussian filter (Eq ) in which v i = [r i, i,b i,] (the homoeneous color at ixel i), an i = [ σ xi s, yi σ s, σ ri c, i σ c, σ bi c ], where σ s is the satial stanar eviation of the filter an σ c is the color-sace stanar eviation Grayscale bilateral filterin can be exresse by relacin r i, i, an b i with just luminance, an is hence a three-imensional filter Some notable uses of the bilateral filter inclue tone main [DD], halo-free sharenin, an hotorahic tone transfer [BPD6] Bilateral filters of imensionalities other than 3 an 5 also occur Weber et al [WMM 4] use a four-imensional bilateral filter with a temoral term for smoothin hoton ensity mas for renerin ray-trace sequences Aams et al [AGDL9] similarly a a temoral term to a color bilateral filter to enoise vieo, which results in a 6-D filter Joint Bilateral Filters By erivin i from one imae an v i from another, one can smooth an imae in a way that oes not cross the ees of another This technique was invente ineenently by Eisemann an Duran [ED4] an Petschni et al [PSA 4] an was use by each for combinin imaes taken with an without flash This iea was extene by Kof et al [KCLU7] to use as an usamlin technique By slattin at ositions etermine by the low-resolution inut, an then slicin usin a hih-resolution reference imae, the low resolution ata can be interolate in a way that resects ees in the hihresolution reference Fast Bilateral Filters Given the wie ranin utility of the bilateral filter, consierable research has one into acceleratin it Duran an Dorsey [DD] iscretize in intensity, an comute a reular Gaussian blur at each intensity level usin an FFT, which can then be interolate between to rouce a rayscale bilateral filter This aroach evolve into the bilateral ri [PD6] of Paris an Duran, which was imlemente on a GPU for various interactive alications by Chen et al [CPD7] The bilateral ri iscretizes in sace an intensity, an introuce the slat-blur-slice ieline for hih-imensional filterin (Fiure ) Paris an Duran also escribe a five-imensional ri for color bilateral filterin [PD9] The work of Duran an Dorsey [DD] was ineenently extene by Porikli [Por8] who observe that the FFTs were unnecessary, an use faster interal-imae base methos to blur each intensity level, thus roucin interal historams Yan et al [YTA9] imrove on this by not exlicitly reresentin the entire sace, but instea sweein a lane throuh the intensity levels, comutin the outut in intensity orer Their low-memory, cache-frienly alorithm is the fastest known rayscale bilateral filter However, the lane-swee aroach oes not eneralize well to hiher imensions Non-Local Means Bennett an McMillan [BM5], while enoisin vieo with a combination of bilateral filters, note that the ositions i coul be usefully aumente by incluin the color or intensity of nearby ixels as well This is equivalent to non-local c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt

3 A Aams & J Baek & M A Davis / Fast Hih-Dimensional Filterin Usin thepermutoheral Lattice means of Buaes et al [BCM5] [BCM8], which is an otimal enoiser for aitive white noise y Fast Non-Local Means There are numerous methos for acceleratin non-local means that o not exlicitly reresent the hih-imensional sace For examle, Mahmoui an Sairo [MS5] reclassify reions of the imae accorin to averae intensity an raient irection in orer to restrict the search, whereas Darbon et al [DCC 8] comute interal imaes of certain error terms, as o Wan et al [WGY 6] However, this family of methos is restricte to osition vectors comose of rectanular imae atches, an oes not eneralize to less structure enoisin tasks, such as the eometry enoisin [AGDL9] an the aforementione hoton ensity enoisin [WMM 4] z (,, ) (,,-) (, -, -) x > y > z x - z < 3 x Fast Gauss Transforms Aroaches for evaluatin Equation as a eneric Gauss transform have focuse on treebase reresentations of the hih-imensional sace of osition vectors Aams et al use a k-tree coule with ranomize queries, whereas Brox et al [BKC8] use a cluster tree Yan et al [YDGD3] escribe the imrove fast Gauss transform, which uses a cluster tree in which each leaf stores not only a value, but also some Taylor series exansion terms escribin how the value varies about that leaf It is extremely accurate, but not as fast as other methos 3 The Permutoheral Lattice We now escribe the lattice we will use to accelerate hihimensional Gaussian filters The -imensional ermutoheral lattice is the rojection of the scale reular ri ( + )Z + alon the vector = [,,] onto the hyerlane H : x =, which is the subsace of R + in which coorinates sum to zero It is hence sanne by the rojection of the stanar basis for ( + )Z + onto H : B = Each of the ( + ) basis vectors (columns of B ) has coorinates that sum to zero, an that each coorinate of each basis vector has a consistent remainer moulo + Both of these roerties are reserve when takin inteer combinations, so oints in the lattice are those oints with inteer coorinates that sum to zero an have a consistent remainer moulo + We escribe a lattice oint whose coorinates have a remainer of k as a remainer-k oint In Fiure we show the lattice for =, an label each lattice oint by its remainer The ermutoheral lattice has several roerties that make Fiure : The -imensional ermutoheral lattice is forme by rojectin the scale ri ( + )Z + onto the lane x = This forms the lattice ( + )A, which we term the ermutoheral lattice, as it escribes how to tile sace with ermutohera Lattice oints have inteer coorinates with a consistent remainer moulo + In the iaram above, which illustrates the case =, oints are labele an colore accorin to their remainer The lattice tessellates the lane with uniform simlices, each simlex havin one vertex of each remainer The simlices are all translations an ermutations of the canonical simlex (hihlihte), which is efine by the inequalities x > x > > x an x x < + it well-suite for hih-imensional filterin usin the slatblur-slice ieline illustrate in Fiure Firstly, it tessellates hih-imensional sace with uniform simlices, so we can use barycentric interolation to slat the sinal onto the lattice oints Seconly, the enclosin simlex of any oint, alon with its barycentric coorinates, can be comute quickly (in O( ) time), so slattin is fast Thirly, the neihbors of a lattice oint are trivial to comute, so the blur stae is also fast Finally, slicin can be one usin the barycentric weihts alreay comute urin the slattin stae, an so is also fast These roerties are escribe briefly below Full erivations, roofs, an further helful roerties can be foun in [BA9] The ermutoheral lattice tessellates H with uniform simlices Consier the -imensional simlex whose vertices s,, s are iven by: s k = [k,,k, k ( + ),,k ( + ) ] } {{ } } {{ } + k k c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt

4 A Aams & J Baek & M A Davis / Fast Hih-Dimensional Filterin Usin thepermutoheral Lattice y of l x (Fiure 3) As every oint belons to a unique simlex, which is a ermutation an translation of the canonical simlex, H is tessellate by uniform simlices z z > y > x y > z > x z > x > y y > x > z x > z > y x > y > z Fiure 3: When usin the ermutoheral lattice to tessellate the subsace H, any oint x H is enclose by a simlex uniquely ientifie by the nearest remainer- lattice oint l (the zeroes hihlihte in re) an the orerin of the coorinates of x l The nearest remainer- lattice oint can be comute with a simle rounin alorithm, an so ientifyin the enclosin simlex of any oint an enumeratin its vertices is comutationally chea (O( )) We term this simlex the canonical simlex For examle, when = 4 the vertices are the columns of: Note that s k is a lattice oint of remainer k (ie its coorinates are all conruent to k moulo + ), an the simlex inclues one oint of each remainer The vertices of this simlex are the bounary cases of the inequalities x x x an x x +, an a oint lies within the simlex if an only if it obeys these inequalities Now consier any ermutation ρ of the coorinates of the canonical simlex Each ρ inuces a corresonin orerin of the coorinates x ρ() x ρ() x ρ(), an the inequality x ρ() x ρ() < + Takin the union of these inequalities across all ( + )! simlices results in the set {x i max i x i min i x i +} (the central hexaon in Fiure 3), which is in fact the set of all oints which have the oriin as their closest remainer- oint (See Proosition A) Hence, as the lattice is translation invariant, a oint x H, with closest remainer- oint l, belons to a unique simlex etermine by l an the orerin of the coorinates x The vertices of the simlex containin any oint in H can be comute in O( ) time This roerty will be useful for the slat an slice staes of filterin The vertices of the simlex containin some oint x H can be enerate by first comutin the closest remainer- oint l, an then sortin the ifference l x The resultin ermutation an translation can then be alie to the canonical simlex to comute the simlex vertices in O( ) oerations The closest remainer- oint can be foun by first rounin each coorinate of x to the nearest multile of ( + ), an then, if the result is outsie the subsace H, reeily walkin back to H by rounin those coorinates that move the farthest in the other irection instea The sublattice forme by the remainer- oints is calle A +, an this is the alorithm iven by Conway an Sloane ( [CS99] 446) for finin the closest oint in that lattice The nearest neihbors of a lattice oint can be comute in O( ) time This roerty will be useful urin the blur stae of filterin The basis vectors iven by B above are those of minimal lenth, so the nearest neihbors of a lattice oint l k are those searate by a vector of the form ±[,,,,,, ] The are ( + ) such neihbors, an each is escribe by a vector of lenth +, an so the neihbors can be fully enumerate in O( ) time 3 Filterin usin the Lattice There are four main staes for usin the ermutoheral lattice for hih-imensional Gaussian filterin, illustrate in Fiure 4 We will escribe each in turn Generatin osition vectors Firstly, the osition vectors i, reresentin the locations in the hih-imensional sace, must be enerate an embee in H Generatin the ositions is somewhat alication eenent For a color bilateral filter, we enerate 5-D osition vectors of the form [ σ xi, yi σ, σ ri c, i σ c, σ bi c ], by aumentin the inut imae with two extra channels encoin satial location, an then scalin each channel by the inverse of the esire stanar eviation For non-local means we woul instea either extract local winows aroun each ixel, or comute some bank of filters aroun each ixel an recor the resonses Tyically we o the latter, usin PCA to etermine the otimal filter bank, as first roose by Tasizen [Tas8] We must then scale the osition vectors by the inverse of the stanar eviation of the blur inuce by the remainin stes, which totals 3 ( + ) in each imension (erive c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt

5 A Aams & J Baek & M A Davis / Fast Hih-Dimensional Filterin Usin thepermutoheral Lattice Slat Blur Slice Fiure 4: To erform a hih-imensional Gaussian filter usin the ermutoheral lattice, first the osition vectors i R are embee in the hyerlane H usin an orthoonal basis for H (not icture) Then, each inut value slats onto the vertices of its enclosin simlex usin barycentric weihts Next, lattice oints blur their values with nearby lattice oints usin a searable filter Finally, the sace is slice at each inut osition usin the same barycentric weihts to interolate outut values below) Next we embe the osition vectors in the subsace H The basis for H iven above is unsuitable for this task because it is not orthoonal, so we instea use the orthoonal basis: b = b j, j= b i = y i y i+ + E = 6 (+) We choose this basis because it allows us to comute E x in O() time usin the recurrence: (E x) = α x (E x) i = α i x i + x i /α i+ + (E x) i+ (E x) = x i /α + (E x) α i = i/(i + ) Slattin Once each osition has been embee in the hyerlane, we must ientify its enclosin simlex an comute barycentric weihts The enclosin simlex of any oint can be escribe by the ermutation an translation that mas the simlex back to the canonical simlex, which can be comute in O( lo) by usin the rounin alorithm escribe earlier to fin the nearest remainer- oint, an then sortin the resiual Therefore, to comute barycentric coorinates for a oint E i in an arbitrary simlex, we can aly the translation an ermutation to ma E i to some y within the canonical simlex Barycentric coorinates b for y are then iven by the exression below, as shown by Proosition A Barycentric interolation is invariant to translation an ermutation, an so these barycentric coorinates for y within the canonical simlex are also the barycentric coorinates for E i within its simlex Once the barycentric weihts are comute, b k v i is ae to the value store at the remainer-k lattice oint in the enclosin simlex of i (recall that v i is the homoeneous value associate with osition i ) The lattice oint values are store in a hash table Lattice oints that o not yet exist in the hash table are create when they are first referre to urin the slat stae, an start with an initial value of zero There are two ways to ientify each lattice oint for use as a hash table key One can aly the inverse ermutation an translation to the remainer-k oint of the canonical simlex to comute the lattice oint s osition, an use that as a key Each key is a vector of lenth +, an so this results in a memory comlexity of O(l) for l lattice oints In rare cases where l > n, we can alternatively achieve a memory comlexity of O(n) for n inut values by searately storin the simlex enclosin each inut osition i, as a simlex can be ientifie uniquely in O() memory by its remainer- oint an its ermutation We then ientify a lattice oint usin its remainer an a ointer to any simlex it belons to, for an aitional O(n) memory One lattice oint belons to many simlices, so key comarison is one by usin the simlex an remainer to comute the lattice oint s coorinates on the fly l is loosely boune by O(n), as each inut value creates at most + new lattice oints However, filters near this boun correson to very small filter sizes an are not very useful, as no share lattice oints means no cross-talk between ixels, an hence no filterin In ractice, we fin that l < n, so we refer the first, faster scheme In either case, c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt

6 A Aams & J Baek & M A Davis / Fast Hih-Dimensional Filterin Usin thepermutoheral Lattice each hash table access costs O() time for key comarison Each ixel accesses the hash table O() times, an so the time comlexity of slattin is O( n) Barycentric interolation in the ermutoheral lattice is equivalent to convolution by the rojection of a uniformlyweihte ( +)-imensional hyercube of sie lenth + onto H, an inuces a total variance of ( + ) / (See Proosition A3) Blurrin Now that our values are embee in the subsace H, the next stae of hih-imensional Gaussian filterin is erformin a reular Gaussian blur within that subsace To o this we convolve by the kernel [ ] alon each lattice irection of the form ±[,,,,,,] (Fiure 4) This rouces an aroximate Gaussian kernel with total variance ( + ) / The blur stae sreas enery from each lattice oint to O(3 ) neihbors If we create hash table entries for new lattice oints reache urin the blur then the memory use woul row quite lare We therefore o not create new lattice oints urin the blur hase, which incurs some accuracy enalty relative to a naive comutation of Equation, as oints that may have transferre enery coul instea belon to isconnecte reions of the lattice Aams et al [AGDL9] observe a similar effect, an arue it may actually be avantaeous For examle, when bilateral filterin, the absence of these stein-stone lattice oints will revent enery transfer from a white ixel to a black ixel across a har ee, but will allow enery transfer between a black ixel an a white ixel on either sie of a smooth raient The blur ste involves lookin u O() neihbors for each lattice oint Each looku takes O() time for hash table key comarison, an so the blur ste has time comlexity O( l) Slicin Slicin is ientical to slattin, excet that it uses the barycentric weihts to ather from the lattice oints instea of scatterin to them It rouces the same total variance of ( + ) /, which brins the total variance inuce by the alorithm to 3 ( +), which is equivalent to a stanar eviation in each imension of 3 ( + ) Slicin can be accelerate by storin the barycentric weihts an ointers to lattice oint values comute urin slattin This slicin table can then be scanne throuh in O(n) time to slice The entire alorithm thus has a time comlexity of O( (n + l)) 3 GPU Imlementation The above alorithm is fairly straihtforwar to arallelize on a GPU We constructe an imlementation usin NVIDIA s CUDA [Buc7], an achieve tyical seeus of Fiure 5: At the to is a 5x56 cro of the inut imae use for time an memory comarisons - a tyical 5 meaixel hotorah Below is the same cro of the outut of the ermutoheral lattice use to erform a color bilateral filter with a satial stanar eviation of 6 ixels an a color stanar eviation of 8 It is visually inistinuishable from the naive result The ermutoheral lattice rouces a result with a PSNR between 45 an 5 B relative to an exhaustive evaluation of Equation, eenin on filter size an imensionality 6x on a Geforce GTX8 comare to the sinle-threae CPU imlementation on an Intel Core i7 9 Clearly a hashtable benefits from a cache The main oint of ifference between the CPU an GPU versions is relate to the creation of hash table entries urin the slattin stae It is customary to attach locks to hash table entries an synchronize all accesses to a iven entry to revent erroneously insertin one key in multile laces We foun it faster to break the slattin stae into three First, we comute the slicin table, recorin which lattice oints each inut ixel slats to, an with what weihts While oin this, we insert the lattice oints foun into the hash table in a way which ermits iniviual keys bein inserte in multile locations Secifically, while we still lock each hash table entry before insertion, other simultaneous hash table insertions simly ski over locke entries while lookin for a free sot rather than waitin on the lock to see if the key matches This means we never have ata eenencies involvin one query reain the key that another query has written, so we can write the keys usin faster non-atomic writes, an only the smaller array of locks nees to be coherent Next, we rehash the entries of the slicin table an uate it so that every reference to a lattice oint refers to the unique earliest instance of that lattice oint in the hash c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt

7 A Aams & J Baek & M A Davis / Fast Hih-Dimensional Filterin Usin thepermutoheral Lattice 3D 5D 8D 6D n n n n f f f s s f s s s Time Taken s r s s GB GB Memory Use s s s r MB MB MB Filter Satial Stanar Deviation (ixels) n Naive Dense Gri s Sarse Gri Permutoheral Lattice Gaussian KD-Tree f Fast Gauss Transform r RTBF Fiure 6: Here we show time taken an memory use as a function of filter size an imensionality for a variety of alorithms The inut was a tyical 5 meaixel hotorah (Fiure 5) Position vectors were comose of ixel locations, ixel luminance, ixel chrominance, an then increasin numbers of filter resonses over a Gaussian-weihte 9x9 local winow, eenin on the esire imensionality These filters were erive usin PCA on the sace of such local winows, an are tyically local erivatives The stanar eviation for all but the satial terms is fixe at 8 The methos comare are the naive winowe bilateral filter (n), the ense bilateral ri () [CPD7], the sarse bilateral ri (s), the ermutoheral lattice (), the Gaussian KD-Tree () [AGDL9], the imrove fast Gauss transform (f) [YDGD3], for which memory use is unavailable, an real-time bilateral filterin (r) [YTA9], which oes not aly to imensionalities above three The arameters of each alorithm were tune so that it ran as quickly as ossible while achievin a PSNR of rouhly 45B relative to an exhaustive evaluation of Equation table Finally, we use the correcte slicin table to slat, aitively scatterin onto lattice oints as usual In an interactive settin, it is common to erform many filters whilst only chanin a few arameters in between runs If the osition vectors an filter sizes o not chane, then the ol slicin table an hash table can be reuse for a moerate seeu 4 Results an Comarisons Here we resent results of a comarison of a number of alorithms for erformin hih-imensional filterin on a tyical hotorah All alorithms were run sinle-threae on an Intel Core i7 9 runnin at 67 GHz Only alorithms with source coe available were use, an all low-level otimizations were left u to the comiler The inut hotorah an the outut of the ermutoheral lattice are shown in Fiure 5 The runnin times an memory use of each alorithm for a few ifferent imensionalities are shown in Fiure 6 Fiure 7 illustrates the fastest metho for a variety of imensionalities an filter sizes The ermutoheral lattice is sinificantly faster than the state of the art for a lare rane of imensionalities an filter sizes 4 The Alorithms Naive: This alorithm consiers all ixels within three satial stanar eviations of each ixel an manually comutes Equation While this alorithm scales linearly with imensionality, it is quaratic in filter size, an oes not work for unstructure ata such as eometry Dense Gri: This is the bilateral ri of Paris et al [PD9], usin multi-linear slattin an slicin, an a searable blur kernel of [ ] in each imension It scales exonentially with imensionality Sarse Gri: The two major ifferences between the ermutoheral lattice an the bilateral ri are the lattice use, an also the fact that the ermutoheral lattice stores values sarsely in a hash table In orer to isambiuate these two effects, we constructe a sarse bilateral ri alorithm that uses the same hash table imlementation Similarly to the Gaussian KD-Tree an the ermutoheral lattice, we o not allocate new lattice oints urin the blur stae However, the time an memory comlexity are still exonential in, an inee this technique is slower than the ense bilateral ri, while savin only a moerate amount of memory This c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt

8 A Aams & J Baek & M A Davis / Fast Hih-Dimensional Filterin Usin thepermutoheral Lattice Fiure 7: This contour lot shows the fastest metho for each imensionality an satial filter size, an how many times faster it is than the secon fastest metho The bilateral ri [CPD7] is best for imensionalities three an four The Gaussian KD-Tree [AGDL9] is best for hih imensionalities an small filter sizes The ermutoheral lattice is the fastest metho for imensionalities from 5 (color bilateral filterin) u to aroun, eenin on the filter size Run times were samle at the small ots an interolate Only methos caable of arbitraryimensional filters were comare, an the arameters of each metho were tune to achieve an PSNR relative to roun truth of rouhly 45B Filter Dimensionality x 4x Gaussian KD-Tree x Bilateral Gri Permutoheral Lattice Filter Stanar Deviation x x x x 4x 4x 8x Non-Local Means Bilateral Filters shows that it is our choice of lattice, rather than our sarsity, which rovies the avantae Gaussian KD-Tree: This is the Gaussian KD-Tree of Aams et al [AGDL9] Its run time is fairly constant across filter size an imensionality, an its memory use is sinificantly lower than the other methos for imensionalities 5 an u Imrove Fast Gauss Transform: This is the fast multiole metho of evaluatin the Gauss Transform of C Yan et al [YDGD3] It is a fully eneral metho caable of extremely hih accuracy, but even when tune for see, it is not articularly fast comare to the more aroximate methos use in imae filterin Real-Time O() Bilateral Filterin: This is the metho of Q Yan et al [YTA9] It is the fastest known metho for rayscale bilateral filters ( = 3), but oes not scale with imensionality 5 Interactive Alications Imaes rouce by alications of hih-imensional Gaussian filterin such as non-local means an the bilateral filter can een heavily on the stanar eviations chosen The hih see an arallelizability of the ermutoheral lattice make these filters easier to use by lettin users interactively exlore the arameter sace Our CUDA imlementation of the lattice filters 8x6 imaes at fs We use this to imlement the followin interactive alications Reaers are encourae to view the vieo accomanyin this aer to see these alications emonstrate Interactive Color Bilateral Filterin The user is resente with sliers to control the stanar eviations for each of the imensions of the filter (x, y, r,, b) The result of chanin any of these values is resente immeiately, makin it easy to exlore the sace of bilateral filters Interactive Non-local Means As a rerocess, PCA is erforme on the sace of 7x7 atches centere about each ixel in the imae, in orer to reuce imensionality without losin istance relationshis We use six PCA terms in aition to the two satial terms as osition imensions Six imensions is the number sueste by Tasizen [Tas8], who emonstrate that erformin non-local means in this way rouces suerior results to non-local means without imensionality reuction Once aain the user is resente with sliers to control the stanar eviations, an the result is resente in real time, allowin for interactive-rate enoisin Non-local Means Eitin Non-local means can also be use for makin eits to an imae that roaate across similar colors, textures, or brihtnesses, in the same manner as the roaatin eits of [AP8] By choosin aroriate osition imensions, local eits can be mae to resect bounaries with resect to any set of local escritors The user alies aroximate eits with a few strokes Each stroke aints values on a mask in those locations The mask is then filtere with resect to the osition imensions For our osition imensions, we use six PCA terms an two satial terms, which catures chanes in brihtness, color, an texture The filtere mask, which resects ees in the osition imensions, serves as an influence ma for how the eit shoul be alie For ajustin brihtness, the user aints ark or briht values into the mask, an we simly multily the inut imae by the filtere mask The filterin is one at interactive rates, so the user can see the fully roaate eit as they aly their rouh strokes c 9 The Author(s) Journal comilation c 9 The Eurorahics Association an Blackwell Publishin Lt