Robotics and Autonomous Systems. Cross-spectral visual simultaneous localization and mapping (SLAM) with sensor handover

Transcription

1 Robocs and Auonomous Sysems 61 (213) Conens lss avalable a ScVese ScenceDec Robocs and Auonomous Sysems jounal homepage: Coss-specal vsual smulaneous localzaon and mappng (SLAM) wh senso handove Mana Magnabosco, Toby P. Beckon School of Engneeng, Canfeld Unvesy, Uned Kngdom a c l e n f o a b s a c Acle hsoy: Receved 17 Apl 212 Receved n evsed fom 14 Sepembe 212 Acceped 21 Sepembe 212 Avalable onlne 23 Ocobe 212 Keywods: Themal magng SLAM Senso handove Coss-specal SLAM In hs wok, we examne he classc poblem of obo navgaon va vsual smulaneous localzaon and mappng (SLAM), bu noducng he concep of dual opcal and hemal (coss-specal) sensng wh he addon of senso handove fom one o he ohe. In ou appoach we use a novel combnaon of wo pmay sensos: co-egseed opcal and hemal cameas. Moble obo navgaon s dven by wo smulaneous camea mages fom he envonmen ove whch feaue pons ae exaced and mached beween successve fames. A beang-only vsual SLAM appoach s hen mplemened usng successve feaue pon obsevaons o denfy and ack envonmen landmaks usng an exended Kalman fle (EKF). Sx-degee-of-feedom moble obo and envonmen landmak posons ae managed by he EKF appoach llusaed usng opcal, hemal and combned opcal/hemal feaues n addon o handove fom one senso o anohe. Senso handove s pmaly ageed a a connuous SLAM opeaon dung vayng llumnaon condons (e.g., changng fom ngh o day). The fnal mehodology s esed n oudoo envonmens wh vaaon n he lgh condons and obo ajecoes poducng esuls ha llusae ha he addonal use of a hemal senso mpoves he accuacy of landmak deecon and ha he senso handove s vable fo solvng he SLAM poblem usng hs senso combnaon. 212 Elseve B.V. All ghs eseved. 1. Inoducon Auonomous obocs s an nceasngly gowng aea n eseach and developmen whn boh academa and ndusy [1]. A key pa of auonomous navgaon fo obocs used n a wde ange of applcaons s he ably o localze whn a gven envonmen and addonally map ha same envonmen. Ths s he classcal smulaneous localsaon and mappng (SLAM) poblem of moble obocs [2]. In hs wok, we unquely use co-egseed opcal and hemal camea sensos and develop a SLAM sysem on a moble obo plafom capable of explong and mappng a gven envonmen. Fuhemoe, we noduce he concep of senso handove, specfcally o addess he ssues of exeme changes n llumnaon ove a long-mescale SLAM msson (.e. mul-day mssons) whee he advanages of hemal sensng ae key unde cean wlgh/ngh llumnaon condons whls opcal sensng emans fo bghe llumnaon peods. The novel aspecs of hs wok ae he coss-specal SLAM (.e., combned hemal and opcal sensng) fo moble obo and addonally he concep of senso handove beween dffeen Coespondng auho. Tel.: E-mal addess: [email protected] (T.P. Beckon). URL: hp:// oby.beckon/ (T.P. Beckon). sensos on a gven SLAM msson. An example can be seen n Fg. 1, whee hemal feaues noduce addonal nfomaon o he scene no pesen fom he opcal senso alone. A smulaneous localzaon and mappng (SLAM) appoach based on puely opcal magey feaues s suscepble o changng envonmenal lghng condons fo long-duaon SLAM mssons and addonally lms feaue avalably fo nocunal SLAM opeaons [3,1,2,4,]. Fo long-duaon SLAM mssons whn a mlay/secuy opeaonal seng (e.g., anspo o seny), an auonomous sysem would eque connuous daylgh and nocunal opeaon. A coss-specal SLAM appoach offes a obus navgaon able o opeae dung day and ngh llumnaon anson whou neupon of opeaons. Fuhemoe, a equemen exss o pefom handove fom one sensng modaly o he ohe n ode o cay ou such connuous opeaons ove changng llumnaon condons. Ths s acheved usng a passve sensng appoach (.e., vsual SLAM). In addon, he coss-specal capably of dual opcal/hemal sensng faclaes ncludes scene deal and deecs objec pesence (Fg. 2) no eadly avalable n po opcal-only SLAM appoaches [3]. In hs wok, we combne he use of speed-up obus feaue pons (SURF [6]), he poposed vsual SLAM appoach of [4] and addonal obus RANSAC-based feaue machng [7] as a soluon o he vsual SLAM poblem ove coss-specal opcal and hemal magey. Addonally we consde he novel concep of senso handove whn hs mul-modal sensng conex /$ see fon mae 212 Elseve B.V. All ghs eseved. do:1.116/j.obo

2 196 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Fg. 1. Deecon of human pesence usng opcal and hemal cameas. Souce: SATURN pojec, Salsbuy Plan, Wlshe, UK, June 29. Fg. 2. Uban envonmen hemal sensng examples. 2. Po wok Fom he suvey of [1], moble obo navgaon can be summazed no hee pncpal subgenes: (a) map-based navgaon (he moble obo has an a po map of he envonmen), (b) mapbuldng-based navgaon (he obo consucs a map of he envonmen) and (c) mapless navgaon (he moble plafom does no use an explc model and navgaes based on objec denfcaon). The SLAM poblem s essenally a map-buldng-based navgaon n whch he sysem has a known knemac model of he moble obo pefomng he mappng ask. Sang fom an unknown poson nsde he envonmen, he am of SLAM s o localze he vehcle whn he envonmen and concuenly buld an ncemenal navgaon map usng he obseved landmaks fom senso nfomaon. A useful mehod o decease he posonng eo of he obo nsde he envonmenal map s he loop closng mehod. Ths mehod s key whn SLAM; consss of e-esmang he map when he obo euns o a pevously vsed locaon and, n hs case, he obo s able o ecognze he SLAM landmaks and ncease he accuacy of he oveall map. Ths mehod pems he consucon of map wh geae conssency hough educng he localzaon and he poson eos of he landmaks [8], nceasng he obusness of he oveall pose esmaon whn he envonmen. A wde ange of devces can be used fo obo navgaon and hey allow he deecon of he necessay SLAM landmak nfomaon. Commonly SLAM s acheved usng lase ange sensos [8,1], whch faclaes he ecovey of explc scene deph nfomaon o an alenave acve sensng appoach (e.g., LIDAR [11], mllmee wave ada [2]). By conas, vsual SLAM uses a passve sensng appoach based commonly on mulple vews fom a sngle camea (monocula SLAM) [11 1,4] o seeo-based appoaches [14,16] o ecove scene deph nfomaon. Vsual SLAM [9] s commonplace whn auonomous obocs and s fequenly augmened wh odomey, neal navgaon and saelle-based GPS navgaon sensos [11,2,16,17]. In geneal, one o moe sensos ae ulzed and combned usng a senso fuson appoach [18] o ncease he oveall SLAM obusness. In hs wok we consde he fuson of coss-specal vdeo souces (opcal/hemal) o age long-em llumnaonndependen vsual SLAM usng a passve sensng appoach. These ae augmened wh neal and GPS sensos on ou obo plafom Vsual SLAM Fom [9], whn he vsual SLAM poblem we can denfy wo man classes elaed o he mehod used o exac he nfomaon fom he sensng devces. The fs mehod s called feaue-based mehods [9], and consss of exacng a suffcen numbe of feaues (e.g., pons, lnes, edges) and sequenally machng hem beween successve mages. The machng sage s he key of all SLAM appoaches: eoneous machng means eoneous pose esmaon and hence eoneous map consucon. The second class of mehods based on [9] s he dec mehods. The equed nfomaon o paamees ae decly exaced fom he pxel nensy values (such as mage bghness and llumnaon-based coss-coelaon) [19]. Seveal echnques whn he leaue have been used fo SLAM, ncludng cone deecon [13,2], scene flow [9,21], genealzed feaue pons [6,22] and genealzed segmenaon [3,1]. In hs wok we unquely exend eale vsual SLAM appoaches o consde he use of such feaue-based mehods ove a cossspecal magey wh he age of beng able o pefom a senso handove fom opcal o hemal sensng enablng long-em

3 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Fg. 3. SURF feaues exaced fo (a) opcal camea and (b) hemal camea. vsual SLAM, fom passve mage-based sensng, unde exeme llumnaon vaance. 3. Opcal and hemal feaues In ou vsual SLAM we eque a feaue-based mehod o exac nfomaon fom ou coss-specal magey. To mee boh he scale and oaonal nvaance equemens of SLAM ogehe wh ou dese fo compuaonal effcency (enablng eal-me pefomance) we selec he SURF-based appoach of [6] SURF feaues The speeded-up obus feaue (SURF) mehod [6] s a obus mage feaue pon deeco and descpo paally based on he semnal scale-nvaan feaue ansfom (SIFT) mehod [22]. As wh he SIFT mehod, he SURF mehod s boh scale and oaon nvaan, bu s compuaonally fase, and hus makes a good canddae feaue deecon algohm fo ou applcaon. The fs sep of he SURF mehod s he deecon of chaacesc feaue pons n he mage whls he second sep s o assgn a unque descpo o each deeced pon. The descpo s a paamee veco of he feaue pon ha ams o be unque because s hen used o subsequenly mach feaue pons beween mages. I has also o be nose obus, and nvaan o scale and oaon n ode o mach he same pons n dffeen mages. The deeco used n he SURF mehod [6] s based on he Hessan max of he mage. Gven a pon x = {x, y} n an mage I, he Hessan max H(x, σ ) n x a scale σ s defned as follows: Lxx (x, σ ) L H (x, σ ) = xy (x, σ ), (1) L xy (x, σ ) L yy (x, σ ) whee L xx (x, σ ) s he convoluon of he Gaussan second-ode devave δ 2 /δx 2 g(σ ) wh he mage I a he pon x, and smlaly fo L xy (x, σ ) and L yy (x, σ ), as descbed n [6]. To oban a scalenvaan feaue, he mages ae successvely smoohed wh a Gaussan fle and spaally subsampled [23]. In geneal, he SURF mehod [6] has beng shown o pefom bee han ohe mehods elave o s feaue exacon pefomance agans compuaonal cos [24]. In ou mplemenaon we exac a 128-dmenson SURF descpo. SURF feaues ae nally deeced ndependenly n each of he opcal and hemal mages (Fg. 3). In ode o faclae he elave coespondence of feaues occung n ehe of hese mages, hemal o opcal senso calbaon s equed o calculae he elave mage plane ansfomaon beween he wo sensos Themal o opcal senso calbaon An nal calbaon of each opcal and hemal camea s caed ou o calculae he nnsc and exnsc camea paamees [2,26] denoed by opcal and hemal camea calbaon maces, K O and K T, especvely. These ae equed n ode o ecove he ansfomaon beween he wo mage planes pemng he use of common efeence fames fo he wo camea sensos (local and global; see Secon 4.1). Ths ansfomaon allows he mappng of feaue pons deeced n he hemal camea mage o he spaal efeence fame of he opcal camea mage, hus allowng he feaues deeced n each mage o be used n unson. Ths ansfomaon s denoed as he plana homogaphy ha pojecs one mage plane (hemal) o he ohe camea (opcal). Ths ype of mappng can be expessed n ems of max mulplcaon [2]. Gven a pon x T n he hemal mage and a coesponden pon x O n he opcal mage (aken fom he same scene) we can expess he opeaon of he homogaphy mappng as follows: x O = shx T, (2) whee x O = {x O, y O, 1} T and x T = {x T, y T, 1} T ae n homogeneous coodnaes, he paamee s s an abay scale faco and H s a 3 3 ansfomaon max. The homogaphy can be calculaed usng fou coesponden plana pons denfed n each of coespondng opcal and hemal scene mages [2]. In Fg., we see he hemal mage ansfomed usng he calculaed homogaphy H ovelan on he coespondng opcal mage based on he wo ognal opcal and hemal npu mages shown n Fg. 4. In ode o compue he homogaphy max H, fou pons n boh vews (.e. fou coesponden pons n he hemal mage and n he opcal mage) ae seleced and successvely calbaed usng he coesponden calbaon max (.e., K T and K O ) o coec fo camea lens and bael dsoons as follows: x T = K T x T and x O = K O x O, (3) whee x T = x T, y T, 1 T s a pon n he hemal mage afe he dsoon coecon and x T = x T, ỹ T, 1 T s he coespondng calbaed pon (x O = x O, y O, T 1 andxo = x O, ỹ O, T 1 ae he opcal pons afe dsoon coecon). Successvely, he pons x T and x O ae used o compue he homogaphy max H c beween he calbaed pons. Gven a calbaed hemal ponx T, he coespondng opcal pon s now esmaed as follows: x O = H c x T, (4) whee H c s now he homogaphy based on he coeced hemal and opcal mages wh espec o he dsoon chaacescs denfed n he magey. Ths calbaon pocedue o ecove he homogaphy H c s equed fo each ndvdual camea se up. Lookng a he deal of Fg. (b), we can see ha he opcal and hemal mages do no compleely mach due o paallax beween he mages, bu hs homogaphy s empcally a good esul fo he feaue machng equed n hs wok.

4 198 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Fg. 4. Images used o compue he homogaphy ha maps he (a) hemal mages o he (b) opcal mages. Fg.. (a) Unansfomed and (b) ansfomed mages (ovelay of he opcal and hemal mages usng calbaed homogaphy H c ). As noduced eale, he homogaphy max H s used o expess he hemal mage o hemal deeced feaues whn he efeence fame of he opcal camea (.e., he efeence fame used fo he SLAM sysem). To compue he ansfomaon fom a hemal feaue pon n he hemal mage plane o he opcal mage plane, we use he homogaphy max H and he nnsc camea paamees K T and K O esulng fom he sngle calbaon of he hemal and opcal camea, especvely. Followng hs appoach he ansfomed hemal feaue pons ae now expessed n he opcal camea efeence fame and hey ae eady fo use n he nalzaon of he SLAM appoach descbed subsequenly. 4. Fom a vdeo o a map Followng he mono-slam appoach of [4], we consuc a sysem based on a sngle camea senso capable of buldng a 3D envonmenal map and self-localzng self whn ha map ove a gven peod of me. Fom [4] we develop an nal soluon fo monocula SLAM usng opcal sensng, neal wheel encode npus and GPS upon whch hemally sensed feaues ae laely ovelan. In summay, he geneal appoach s o exac SURF feaue pons and subsequenly ehe nalze new feaues o mach deeced feaues agans hose whn an exsng feaue daabase (.e., known feaues fom pevous mages). New feaues ae nalzed o selec a se of possble 3D posons (.e., 3D possble coodnaes). Each new feaue s nalzed as a sum of Gaussans ha s hen updaed wh each subsequen obsevaon [4]. Fom seveal successve obsevaons, he deph of he feaue and s 3D coodnaes can be ecoveed, and he feaue s seleced as a canddae landmak o be added o he 3D envonmen map. The esulng map hus conans nfomaon abou he poson esmaon of each denfed landmak and s assocaed posonal eo [4]. In addon a dead eckonng appoach usng neal wheel encodes and a seconday GPS eceve ae used as addonal sensos o esolve he global obo moon and poson. As he obo connues n moon hough he envonmen, managemen and megng of hs nfomaon s equed o manan he envonmenal map. To faclae he combnaon of hs mul-senso daa fo he esmaon of he obo cuen poson, an mplemenaon of an exended Kalman fle [27] s used Feaue exacon and nalzaon Fom each capued opcal and hemal mage fame, SURF feaues [6] ae exaced. Subsequenly he exaced feaues ae mached beween successve fames on a pe-senso bass. Ths nal se of maches s used o buld an nal feaue daabase fo each senso (sepaaely). These daabases ae hen used o ack he mage o mage feaues as he obo anss hough he envonmen (opcal and hemal sepaaely). New feaue pons ae added o hese daabases evey n fames based on a paallel machng echnque. Ths echnque consss n machng feaues among m mage fames, allowng he sysem o add feaues whch ae moe fequenly pesen n he feld of vew of he camea sensos. Wh hs mehod we hen ae able o ncease he pobably of landmak deecon as he nal feaue pons used ae moe sable ove me. Afe a feaue s deeced we follow he appoach of [4], and each SURF feaue pon s nalzed wh a se of canddae posons and updaed usng successve obsevaons as descbed

5 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) n [4]. Fo each updae of he feaue posons, he 3D coodnaes of he nal se s puned unl a las updae selecs one canddae as he nal 3D coodnaes (see Secon 4.2). Fo a sngle feaue pon, once exaced fom he mage, he only nfomaon abou s poson n space ha can be mmedaely ecoveed s he 2D pxel coodnaes n he mage wh espec o he camea efeence fame. Gven a 3D pon P wh coodnaes X = {X, Y, Z} n he scene, we can expess s coespondng 2D mage pxel coodnaes usng he calbaon max K as follows: fsx o x λx = λ u v 1 = fs y o y 1 X Y Z = K X, () whee λ s he dsance (deph) of pon X along he z axs of he camea, x = {u, v} he 2D pxel coodnaes n he mage camea plane of P and f he camea focal lengh, whls s x and s y ae he lenghs of he pxel along he hozonal and vecal decons especvely and o x and o y ae he x and y coodnaes of he opcal cene of he mage plane. The max K and he elave elemens ae ecoveed fom po calbaon [26, see Secon 3.2]. Fom Eq. (), we can see ha he deph λ s he Z coodnae of a feaue, whch s n geneal unknown a hs sage of SLAM when usng a monocula camea model [4]. Mulplyng he 2D pxel coodnaes x = (u, v) by he nvese of he calbaon max, we oban he expesson o ecove he 3D coodnaes X of P up o a scala faco λ: X = X Y Z = λk 1 u v. (6) The scala faco λ epesens he unknown deph of he pon P. Dvson of Eq. (6) by λ gves: X/λ = X/λ Y/λ Z/λ = X/Z Y/Z 1 = x y 1 = K 1 u v 1. (7) The nex sage s he ecovey of he decon of he pon wh espec o he camea efeence fame. The decon of a feaue s epesened by he angles θ and φ: θ = φ acan (y/x) acan (1/ x 2 + y 2 ), (8) and hese angles efe o a pola efeence fame, as shown n Fg. 6. As we use mages fom a sngle camea, he cuen deph of a feaue pon P, anslaed as adus ρ n a pola coodnaes sysem (see Fg. 6), s unknown, as pevously oulned. Based on [4], we esmae he spaal poson (and hus he deph) of a feaue nalzng he coesponden 3D coodnaes usng a sum of Gaussans. Ths sum of Gaussans s successvely updaed usng obsevaons of he same feaue ove me based on ou feaue machng mehodology (o acheve conssen mach feaues we use a neaes-neghbou echnque [28] n combnaon wh he RANSAC algohm [7]). The man hypohess ha allows us o compue hs nal se of Gaussans fo a feaue pon s he esablshed specfc deph ange [ρ mn, ρ max ] fo any gven senso. Howeve, he nal values of ρ mn and ρ max ae somewha abay, and can be se empcally based on he envonmen and known senso capables. Followng he appoach of [4], he 3D coodnaes of a pon P can hus be expessed as: P (θ, φ, ρ) = Γ (θ, σ θ ) Γ (φ, σ φ ) ω Γ (ρ, σ ρ ), (9) whee P (θ, φ, ρ) epesens he spaal poson of P wh s elave eo n pola coodnaes and ω Γ (ρ, σ ρ ) epesens Fg. 6. Repesenaon of a feaue n pola coodnaes (θ, φ). he sum of Gaussans ha appoxmaes he a po knowledge of he deph. Accodng o [4] he deph of he feaue can be compued usng he followng geomec sees: ρ = ρ mn /(1 α), (1) ρ = β ρ, σ ρ = α ρ, ω ρ, (11) ρ n 2 < ρ max / (1 α), ρ n 1 ρ max /(1 α). (12) Once agan he values of α (=.2) and β (=2.) ae chosen empcally bu followng consans elaed o he dsbuon of a Gaussan ha we wan o oban [4]. Afe hs nalzaon, each Gaussan µ p = {ρ, θ, φ, }, p = σρ 2, σθ 2, σ φ 2 s conveed fom pola coodnaes o Caesan coodnaes: θ µ c = g = φ c = G ρ cos φ cos θ ρ cos φ sn θ ρ sn φ = x y z (13) p G T = σ 2 x, σ 2 y, σ 2 z, (14) whee G = g/ X (ρ,θ,φ). The efeence fame used o expess he nal 3D coodnaes of a feaue pon s he obo efeence fame a he nsan when he feaue s seen he fs me. Afe hs nalzaon, we oban n 3D coodnaes of he same feaue, and an updae sage s necessay o selec whch Gaussan bes appoxmaes he feaue pose. An example of he oupu of he nalzaon pocess s shown n Fg. 7, whee each Gaussan n he se s epesened as an ellpsod of eo and he oenaon s elaed o he fs elave decon n whch he feaue was nally seen Inal poson updae and landmak nalzaon Usng successve obsevaons of he same feaue pon, a selecon pocedue can be pefomed compung an esmaon of

6 2 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Z [m] Tajecoy of he Moble obo Fg. 7. Feaue nalzaon landmak selecon and esmaon elaed o he obo poson. fame. The obo efeence fame dffes fom he mage efeence fame by wo smple oaons along he z and x axes of π/2. Fom he specfcaon of he vaables, we can hus fomulae a compuaon fo ẑ, he pedcon of he h obsevaon n he cuen obo efeence fame, as follows: ẑ = h o ˆX, fom ˆX f, µc = H ˆX, ˆX f, µc, (16) Fg. 8. Feaue/landmak poson veco wh espec o dffeen obo efeence fames. Table 1 Descpon of he vaables used n ou fomulaon. Symbols X ˆX f ˆX f R, T f R, T Descpon global (o wold) efeence fame, aken as he fs poson of he moble obo obo efeence fame whee he landmak s nally seen, coodnaes ae expessed wh espec o he global efeence fame cuen obo efeence fame, coodnaes ae expessed wh espec o he global efeence fame oaon max, R, and anslaon veco, T, ha expess he oaon fom he obo efeence fame a me o he global efeence fame oaon max, R, and anslaon veco, T, ha expesses he oaon fom he global efeence fame o he cuen obo fame µ 3D coodnaes of he h landmak wh espec o he global efeence fame µ c 3D coodnaes of he h landmak wh espec o he obo efeence fame µ 3D coodnaes of he h landmak wh espec o he cuen obo efeence fame he nomalzed lkelhood fo each Gaussan, Γ. The lkelhood of Γ o be an esmaon of he obseved feaue s compued as follows: L 1 = 2π S exp 1 T z ẑ S 1 z ẑ, (1) 2 whee S s he covaance of he nnovaon z ẑ [4]. The pedcon of he obsevaon ẑ = (θ, φ ) s esmaed consdeng each Gaussan n he cuen obo fame (.e., a me ) and z s he obsevaon a he coespondng me. In Fg. 8 and Table 1 we pesen hee dffeen obo efeence fames geneally used and he noaon of he landmak veco wh espec o hese efeence fames (.e., µ, µc and µ ). All he nfomaon fom he mages s acqued n he mage efeence fame, whle he nfomaon soed n he EKF and n he subsequen 3D envonmenal map efes o he obo efeence whee µ = fom ˆX f, µc = f R µc + T, f (17) ˆX, µ µ = o = R µ T, (18) acan ( z /x) θ ẑ = = y φ acan x 2 +, (19) z 2 based on he fomulaon of [4]. Fom Eqs. (18) and (19) we can see how he vaable ẑ s he Caesan coodnaes µ ansfomed no he pola efeence fame, fom whee we can ndcae h() o be he ansfomaon fom Caesan coodnaes o pola coodnaes [4]. Fuhemoe, n Eq. (1) we use he max S, he covaance of he nnovaon z ẑ, whch s compued as follows: S = H 1 P X H T 1 + H 2P X f whee H 1 = H X + H 2 P T X,X f H T + 1 H 3, H 2 = H H T + 2 H 1P X,X f H T 2 c X f H T 3 + R, (2) and H 3 = H µ c (21) whee R s he covaance assocaed wh he obsevaon z and P (k) s he co-vaance max assocaed wh a gven efeence fame k fom Table 1 (whee efeence fame k n Table 1 s elave o he ogn wh assocaed ansfomaon done by T and R). Followng [4], o compae he lkelhood of each Gaussan, we use he nomalzed lkelhood; ha s, fo hypohess, he poduc of lkelhoods obaned fo Γ s Λ = j L L j. (22)

7 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Fg. 9. Fom an obseved feaue pon n he mage fame o a landmak n he 3D envonmenal map. The nomalzed lkelhood s compued evey me we have a new obsevaon z of he feaue pon fo each Gaussan, and he Gaussan assocaed wh he wos hypoheses s puned f Λ < τ (τ =./n, n = numbe of Gaussans emanng). Followng seveal obsevaons we end up wh a sngle Gaussan, and he assocaed 3D coodnaes of he feaue ae compaed wh he las obsevaon usng he χ 2 es [4,]. If he coodnaes pass he χ 2 es, he assocaed feaue s declaed as a landmak and s nseed n he landmak daabase and deleed fom he coesponden feaues daabase (.e., opcal o hemal). If hs s no he case, hs means ha he feaue s no n he pe-specfed deph ange [ρ mn, ρ max ] ha we have denfed n he nalzaon sage o alenavely he obsevaons wee no conssen enough so he feaue has been ejeced by he χ 2 es. The pocess fom nal feaue o a new landmak s shown n Fg. 9 wh he educon n spaal Gaussan (llusaed n wo dmensons). The lefmos mage (Fg. 9) epesens he nalzaon sage of a feaue as a sum of Gaussans. The successve wo mages n Fg. 9 show ha, hanks o successve obsevaons of he same feaue, some Gaussans ae subsequenly puned,leavng a sngle esmae of he 3D poson wh a Gaussan eo. When only one Gaussan emans and passes he χ 2 es, he feaue s declaed as a landmak and s pojeced no he global efeence fame. A he fnal sage, he pas obsevaons of he feaue pon ae used o updae he esmaon of he landmak poson and educe he oveall eo elaed o he Gaussan (see gh mage n Fg. 9) po o be added o he 3D envonmen map. As he numbe of landmaks deeced fom he envonmen nceases, we slowly buld up a 3D landmak map of he envonmen as he moble obo anss hough he envonmen Robo poson and 3D map updae Once we sa consucng he envonmen map, we use obsevaons of he denfed landmaks as addonal nfomaon npu o he exended Kalman fle (EKF) [27,29]. Ths s used n combnaon wh he neal wheel encodes and addonal GPS eceve daa o bee esmae he 3D map of he envonmen and he poson of he obo whn he envonmen map. The sae of he EKF s composed by he cuen obo poson, he pas n obo posons and he m landmak esmaes. The po n obo poson ae no subjeced o updae o pedcon whn he EKF famewok. These obo posons ae kep o allow he poson updae pocess o denfy he obo poson whee a feaue s nally deeced. The oveall EKF sae can be epesened as follows: X = X.. X X l,1 X l,m,. P X P X,X P X,X P l,1 X,X l,m P P = X P X,X P l,1 X,X l,m P X P l,1 X, l,1,x l,m P X l,m (23) whee X s he sae veco and X epesens he cuen poson of he obo, X s he fs saved pas poson of he obo and X l,=1,...,m s he poson of he h landmak a me. The max P s he covaance max of he enes of he EKF sae veco, and epesens he eo of he esmaons. Afe a gven amoun of eaon, he oveall sze of he EKF sae nceases due o he addon of new landmaks whn he envonmen map and he consan ecodng of po obo posons. In ode o educe boh he memoy foopn and he compuaonal cos of he oveall EKF pocess, he sze of he EKF sae veco s managed by emovng po obo posons ha ae no used fo subsequen feaue ackng fom he veco used n he nal poson updae pocess (see Secon 4.2).. Resuls Ths wok s ealzed on Ponee 3-AT moble obo plafom equpped wh boh opcal and hemal cameas (Vsonhech VC7WD-24 CCTV specal ange: 4 7 nm/themoeknx Fg. 1. Ponee 3-AT equpped wh opcal and hemal sensos.

8 22 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Fg. 11. Oudoo es envonmen: (a) opcal camea, (b) hemal camea he mage)]. By conas, fom he hemal camea a vaey of feaues ae sll deeced wh vayng sgnaue feaues, such as people and aspecs of buldng whn he envonmen (Fg. 11(a)). In geneal he feaue densy whn he hemal magey s lesse han he coespondng opcal magey (Fg. 3). Howeve, has o be noed ha he densy of he avalable opcal magey feaues s dependen on he llumnaon condon, wheeas he hemal feaues ae lagely consan and dependen on he hemal dynamcs of he envonmen ahe han vayng llumnaon condons Opcal SLAM As fs efeence mplemenaon we use a sngle camea sysem eplcang he poposed soluon of [4] usng only he opcal camea senso. Fo all he epoed examples, saelle magey [3] s ovelad ono he 2D envonmenal map (see Fg. 13(a)). Fo hs case 13 opcal landmaks ae deeced wh aveage eo ±2.1 m, ±.4 m and ±.21 m along he x, y and z axes, especvely. A second example s pesened n a dffeen envonmen seng fo compason wh he esulng 2D map/saelle magey shown n Fg. 12. Fo hs example case 129 opcal landmaks ae deeced wh aveage eo ±1.46 m, ±.6 m and ±.43 m along he x, y and z axes, especvely. Fg D envonmenal map fo he opcal senso SLAM analyss (case 2). MIRICLE 11K specal ange: 8 12 µm). Fuhemoe, he moble obo s equpped wh a GPS GlobalSa BU 33 eceve and s manually conolled a a vaable speed whn he es scenao used. The sysem confguaon s shown n Fg. 1. The sysem s esed ove a ange of oudoo envonmens aound he Canfeld Unvesy campus (Fg. 11(a) (b)). In geneal, whn hese es envonmens we see a dvese ange of feaues a a vayng deph deeced fom he opcal camea due o he lage level of deal whn he mage and ou choce of he feaue deeco [6, Refe o Fg. 3 and Fg. 11 (Fg. 3 fo he feaues and Fg. 11 fo he dffeence n he deals of.2. Themal SLAM The expemens fom Secon.1 ae epeaed usng he hemal senso (feaues deeced on he same physcal un of he obo). Boh cameas ae nalzed dencally, wh he only excepon beng he nal value of he deph ange lms [ρ mn, ρ max ]. In geneal, he mnmum ay ρ mn used o nalze new hemal feaue pons s lage han he one used fo opcal feaues (see Secon 4.1). The movaon fo hs s elaed o he dffeen feld of vew of he hemal senso, whch s sgnfcanly naowe han ha of he opcal senso. As a consequence, followng he nalzaon pocess (see Secon 4), he ellpses of eo fo he hemal landmak esuls ae lage han fo opcal

9 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) a b c Map legend: Robo Opcal Landmaks poson poson eo eo -1 encode daa Themal Landmaks GPS daa poson eo Fg D envonmenal map fo he (a) opcal senso SLAM analyss, (b) hemal senso SLAM analyss and (c) coss-specal SLAM analyss (case 1). landmaks (compae Fg. 13(a) and (b)). Fo he hemal case shown n Fg. 13(b), he analyss usng he hemal senso deecs seven hemal landmaks wh aveage eo ±8.31 m, ±1.3 m and ±.97 m along he x, y and z axes, especvely. Compang he opcal and hemal analyses fo hs case (compae Fg. 13(a) and (b)), we see a sgnfcan educon n he landmaks deeced dung he exploaon of he aea. Ths can be abued o a low level of hemal deal whn hs pacula poon of he envonmen and addonally he naowe feld of vew of he hemal camea senso. Fo he second hemal SLAM es case, he 2D envonmenal map s shown n Fg. 14; deecs 122 hemal landmaks wh aveage eo ±.94 m, ±.68 m and ±.31 m along he x, y and z axes, especvely. Fo hs case, we noce a smla level of landmak deecon as n he coespondng opcal case (see Fg. 12), and addonally noe he compaable aveage eos ove he se of deeced landmaks. Oveall we can see ha hemal senso-based SLAM can pefom a a compaable level o ha obaned usng he opcal senso..3. Coss-specal SLAM The expemenal analyss undeaken n Secons 3.1 and 3.2 s epeaed usng boh he opcal and hemal camea sensos fo ndependen feaue deecon as a condu o landmak deecon. Fo he fs analyss case, usng he coss-specal SLAM appoach (Fg. 13(c)), he level of opcal landmak deecon s compaable o he po opcal senso only case (Fg. 13(a)), bu we see a sgnfcan ncease n he numbe of hemal landmaks deeced Fg D envonmenal map fo he hemal senso SLAM analyss (case 2). wh espec o he po hemal senso only analyss (Fg. 13(b)). Fuhemoe, he envonmenal map s epesened n hee

10 24 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Z [m] Fg. 1. 3D envonmenal map fo he coss-specal SLAM analyss (case 1). Table 2 Fnal value of he EKF sae veco {X, Y, Z, γ, β, α} fo he coss-specal SLAM, he opcal senso only SLAM and hemal senso only SLAM (case 1). Coss-specal SLAM Opcal SLAM Themal SLAM X (m) 1.82 ± ± ±.4 Y (m) 9.9 ± ± ±.6 Z (m).22 ±.4.37 ±.4.6 ±.47 γ ( ). ± ± ± 9.6 β ( ) 2.73 ± ± ± 7.84 α ( ) ± ± ± 4.9 dmensons n Fg. 1 o gve an alenave llusaon of he oveall esmaed map followng he SLAM esul pesenaon syle of he ognal base wok [4]. Fo hs case, he combnaon of he opcal and hemal sensos allows he sysem o deec 96 opcal landmaks and 36 hemal landmaks wh aveage eo ±3.6 m, ±.34 m and ±.31 m along he x, y and z axes, especvely. In compason o he opcal senso only mplemenaon of [4] (Secon 4.1) we show ha he coss-specal SLAM appoach gves an ncease of 18.2% of he landmaks deeced. The s abuable o he fac ha he coss-specal SLAM usng wo sensos s able o exac moe nfomaon fom he exploed envonmen. Dung hs expemen a numbe of dynamc objecs (people) wh sgnfcan hemal and opcal feaues also ene he envonmen whn he feld of vew of he sensos. I s mpoan o oulne ha he pesence of hese dynamc objecs n he scene dd no affec he oveall opeaon of he appoach. Ths can be abued o he heshold added befoe he use of any gven obsevaon veco n he updae sage of he EKF ha allows he sysem o have a obus feaue/landmak machng pocess (Secons 4.2 and 4.3). Table 2 shows he fnal esmaed posons of he moble obo fo he hee senso vaaon appoaches (.e., coss-specal SLAM, opcal SLAM and hemal SLAM). The fnal eo assocaed wh he obo poson esmaon usng combned sensng s almos dencal o hose esmaed fom he opcal senso only case, bu noably he use of he hemal camea povdes addonal nfomaon dung he mappng phase of he SLAM appoach wh he advanage of povdng fuhe hemal feaues/landmaks. Ths s acheved whou noducng fuhe sgnfcan eos no he sysem. Fom Table 2, f we consde ha he moble obo s movng on a suface ha can be consdeed plana, we can obseve ha he esuls obaned fo he coss-specal SLAM sysem gve a bee esmae of he obo poson wh pacula egads o he obo oenaon along he x and y axes. Fg. 16 shows he 2D envonmenal map of he second analyss case fo combned coss-specal sensng. In compason wh he opcal senso only case (Fg. 12) and he hemal senso Fg D envonmenal map fo he coss-specal SLAM analyss (case 1). only case (Fg. 14) hs coss-specal SLAM analyss obaned 14 opcal landmaks and 67 hemal landmaks wh aveage eo of ±1.6 m, ±.43 m and ±.26 m along he x, y and z axes, especvely. Despe he decease of he numbe of hemal landmaks deeced n he combned sysem wh espec o he hemal senso only analyss (Secon.2), he posonal eos (ellpses n Fgs. 13(b) 14 and Fgs. 13(c) 16) fo he hemal landmaks n he coss-specal SLAM analyss ae acually smalle han hose ecoveed n he hemal senso only case. Compang he cossspecal SLAM sysem wh espec o he sngle opcal camea mplemenaon of [4] we have an ncease of 32% n he numbe of landmaks deeced and a decease of 27.4%, 28.3% and 39.% n he aveage landmak eo along he x, y and z axes, especvely. I s mpoan o noce ha he ncease n he oveall numbe of landmaks esuls n a hghe numbe of landmak obsevaons. These combned opcal and hemal landmak obsevaons ae dependen on he cuen obo poson (he landmak poson obsevaons ae ndec), and when used as npu o he EKF hs dependency on he cuen saus of he moble obo posonng allows us o educe he oveall EKF sae veco eo. The coespondng 3D map of he combned sysem fo hs second case s shown n Fg. 17.

11 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Z [m] Fg D envonmenal map fo he coss-specal SLAM analyss (case 2) Fg D envonmenal map fo he coss-specal SLAM analyss (case 3). Table 3 Fnal value of he EKF sae veco {X, Y, Z, γ, β, α} fo he coss-specal SLAM, he opcal senso only SLAM and hemal senso only SLAM (case 2). Coss-specal SLAM Opcal SLAM Themal SLAM X (m) 4.96 ± ± ±.96 Y (m) 2.28 ± ± ±.43 Z (m). ±.47.7 ±.47.1 ±.48 γ ( ) 3.6 ± ± ± 9.49 β ( ) ± ± ± 9.49 α ( ) ± ± ± 4.2 Fom Table 3 we can obseve ha fo hs case he coss-specal SLAM pefoms bee n ems of esmaed fnal obo poson han n he opcal senso only case, suggesng ha he cossspecal SLAM appoach benefs fom he addon of hemal landmak obsevaons mpovng he moble obo localzaon whn he envonmen. Ths expemen s caed ou fo a hd case and he 2D envonmenal map s shown n Fg. 18. In hs example we can see a pevalence of opcal landmaks a he begnnng of he navgaon whls a deecon of hemal landmaks seems o domnae he las pa of he moble obo oue. Ths s elaed o a hghe pesence of opcal feaues n he fs pa of he moble obo oue n compason o he end of he oue, whls fo hemal feaues he oppose end s appaen. In hs example 213 landmaks ae deeced: 141 ae opcal landmaks and 72 ae hemal landmaks, wh an aveage eo ±1.82 m, ±1.49 m and ±.8 m along he x, y and z axes, especvely. Fo hs hd case, dffeen opcal and hemal landmaks ae deeced dung he navgaon, and a wde ange of eo ellpses s pesen n he esulng envonmen map (Fgs. 18 and 19). Ths exen of landmak posonng eos s elaed o he nal coodnaes used fo a new landmak; landmaks ha ae deeced fuhe away fom he camea sensos ae gong o be nalzed wh a lage coesponden eo as he nal eo s popoonal o he dsance of he landmak fom he camea (see Secon 4.1). Fg. 19 epesens he 3D envonmenal map fo he combned sysem of hs hd case..4. Senso handove whn SLAM As shown n Secon.3, he combned use of opcal and hemal sensos allows he moble obo o pefom he SLAM ask ndependen of llumnaon condons and mpoves esmaon of he obo poson compaed o a sngle senso SLAM appoach [4]. Anohe advanage of hs appoach s ha f feaues fom one of he wo sensos become unavalable he sysem s sll able o exploe he envonmen, pefomng SLAM and deecng feaues/landmaks, as he daa fom he emanng senso s used o complee hs poon of he SLAM ask. To es he pefomance of he sysem, we smulae a senso handove suaon whee we essenally emove he opcal senso as an npu o he sysem whn he SLAM msson and ely on hemal feaues only fo a mdpoon of he msson. Ths s pefomed ove a sho peod of a longe msson o smulae a sgnfcan llumnaon change (dakness) affecng he avalably of opcal feaues fo vsual SLAM. In Fg. 2, we see a second analyss case (pevous case 2 fom Secon.3) whee opcal camea feaues ae no used fom fame 22 unl fame 16 of he oveall daa sequence (he opcal

12 26 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Z [m] Fg D envonmenal map fo he coss-specal SLAM analyss (case 3) Table 4 Fnal value of he EKF sae veco {X, Y, Z, γ, β, α} fo egula coss-specal SLAM and senso handove case (case 2). Coss-specal SLAM Handove X (m) 4.96 ± ±.97 Y (m) 2.28 ± ±.44 Z (m). ±.47. ±.47 γ ( ) 3.6 ± ± 9.47 β ( ) ± ± 9.39 α ( ) ± ± Fg. 2. 2D envonmenal map fo he handove analyss (case 2). feaues ae absen ove 3% of he msson). In hs fs handove case, 118 landmaks ae deeced, of whch 62 ae hemal and 6 ae opcal deeced landmaks, wh an aveage eo of ±.9 m, ±.22 m and ±.1 m along he x, y and z axes, especvely. Compang he senso handove condon wh he cossspecal SLAM case, s noable ha fewe landmaks ae deeced wh pacula efeence o opcal landmaks. Ths s somewha o be expeced because s he daa fom he opcal camea n whch we ae smulang an ouage, equng handove, fo appoxmaely a hd of he oveall moble obo navgaon oue. In Table 4, we epo he fnal poson of he obo fo boh he coss-specal SLAM and he senso handove condons. The esmaed poson fo he moble obo unde he senso handove condon s compaable n ems of he eo nevals wh he coss-specal SLAM case, and fom hs we can see ha he combned sensng appoach s able o cope wh he mssng senso daa ha could be caused by exeme changes n llumnaon condons (.e., day/ngh opeaon) o senso malfuncon. The same handove analyss s caed ou on he hd case (see Fg. 21). Smlaly, fo hs analyss case, he opcal camea s no beng used fo appoxmaely 3% of he navgaon oue o smulae a sgnfcan change n he lghng condon whch lms he usefulness of he opcal camea feaues. Ths esul s llusaed gaphcally n Fg. 21 and addonally wh compason o he coss-specal SLAM case n Table. In hs case he appoach s able o deec 167 landmaks; 62 ae hemal and 1 ae opcal landmaks, wh an aveage eo of ±1.29 m, ±.4 m and ±.9 m along he x, y and z axes, especvely. The numbe of landmaks deeced s agan fewe han n he coss-specal SLAM case bu hs s elaed o he absence of opcal daa nfomaon fo pa of he obo navgaon oue. The fnal obo poson esmaon s epoed n Table wh he elave eos and compaed o he coss-specal SLAM case. Fom Table we can see he dffeence n he obo poson esmaon of 4 m fo he X poson and 2 m fo he Y poson. In hs fnal case he hemal camea s noably posoned o an offse o he opcal senso feld of vew (see Fg. 22) n an aemp o maxmze he numbe of nea feld feaues deeced by he hemal senso. Ths confguaon n addon wh he naow feld of vew of he hemal senso (n compason wh he opcal camea) shows ha puely hemal camea navgaon s no deal and he oveall sysem suffes o a geae degee wh a lack of opcal daa dung he handove condon. Howeve hs does no affec he ably of he sysem o esmae s poson and deec landmaks, bu does appea o affec he localzaon of s oveall fnal pefomance. Fuue wok wll look a usng mulple hemal sensos o a wde feld of vew o gve geae concdence coveage of boh he opcal and hemal feld of vews whn hs ask (Fg. ). In geneal, we can see ha he SLAM appoach can cope well unde opcal o hemal senso handove condons, smulang daylgh o nocunal llumnaon changes, and we see ha he mpac on he oveall localzaon and mappng componens of he SLAM ask s mnmal.

13 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) Fg D envonmenal map fo he senso handove case analyss (case 3). Fg. 22. Example of he oupu dung he navgaon (case 3). Table Fnal value of he EKF sae veco {X, Y, Z, γ, β, α} fo egula coss-specal SLAM and senso handove case (case 3). Coss-specal SLAM Handove X (m) 148. ± ±.34 Y (m) 7.2 ± ±.47 Z (m).3 ± ±.34 γ ( ) 1.99 ± ± 2.33 β ( ) ± ±.27 α ( ) 1.9 ± ± Concluson We pesen a soluon fo he SLAM poblem usng combned opcal and hemal sensng. An mplemenaon of he echnque of [4] s successfully acheved, and pems he noducon of wo novel aspecs of hs wok: (1) he addonal use of a seconday hemal senso o complemen he exsng opcal senso n he SLAM navgaon ask (coss-specal SLAM navgaon) and (2) senso handove beween cameas opeang n dffeen pas of he specum ove a sngle SLAM msson. The evaluaon of he appoach pesened confms ha he nfomaon added by he hemal camea mpoves he pefomance of he monocula SLAM appoach (despe beng used as ndependen, non-seeo sensos) as nceases he numbe of deeced landmaks and deceases he aveage eo elaed o he landmak posons. Fuhemoe, mpoves he esmaon of he moble obo poson houghou he me negaon seps (.e., mpoved localzaon). Ths pefomance of he sysem s confmed fo dffeen ypes of envonmen and n vayng lghng condons ncludng he pesence of movng objecs whn he scene. In addon, we llusae he fs use of coss-specal senso handove whee fo a sngle SLAM msson we pefom pa of he oue wh combned opcal and hemal (coss-specal) sensng and pefom handove o a sngle senso (hemal camea) dung he msson. Ths wok exends he sae of he a wh espec o he monocula sngle senso SLAM appoach of [12,13,4] by llusang s exenson o coss-specal sensng and addonally wh he ably of handove beween mul-senso and sngle senso sensng wh mnmum effec on he oveall localzaon and mappng. Fuhe wok could develop he negaon of an opcal flow echnque [21] as a supplemenay ool fo he measuemen of he obo moon, coss-specal seeo [31] and addonally consdeaon of SLAM loop closng n a mul-sensng/cosssensng envonmen. Refeences [1] G.N. Desouza, A.C. Kak, Vson fo moble obo navgaon: a suvey, IEEE Tansacons on Paen Analyss and Machne Inellgence 24 (2) (22) [2] M.W.M.G. Dssanayake, P. Newman, S. Clak, H.F. Duan-Whye, M. Csoba, A soluon o he smulaneous localzaon and map buldng (SLAM) poblem, IEEE Tansacons on Robocs and Auomaon 17 (3) (21) [3] J. Maas, O. Chum, M. Uban, T. Pajdla, Robus wde-baselne seeo fom maxmally sable exemal egons, Image and Vson Compung 22 (1) (24) [4] T. Lemae, C. Bege, I. Jung, S. Lacox, Vson-based SLAM: seeo and monocula appoaches, Inenaonal Jounal of Compue Vson 74 (3) (27) [] Y. Lee, T. Kwn, J. Song, SLAM of a moble obo usng hnnng-based opologcal nfomaon, Inenaonal Jounal of Conol, Auomaon and Sysems () (27) [6] H. Bay, T. Tuyelaas, L.V. Gool, SURF: speeded up obus feaues, n: Euopean Confeence of Compue Vson, 26, p. 44. [7] M.A. Fschle, R.C. Bolles, Random sample consensus: a paadgm fo model fng wh applcaons o mage analyss and auomaed caogaphy, Communcaons of he Assocaon fo Compung Machney 24 (6) (1981) [8] A. Nüche (Ed.), 3D Roboc Mappng he Smulaneous Localzaon and Mappng Poblem wh Sx Degees of Feedom, n: Spnge Tacs n Advanced Robocs, 29. [9] C.C. Wang, C. Thope, Smulaneous localzaon and mappng wh deecon and ackng, n: IEEE Inenaonal Confeence on Robocs and Auomaon, vol. 3, 22, pp

14 28 M. Magnabosco, T.P. Beckon / Robocs and Auonomous Sysems 61 (213) [1] J. Cho, S. Ahn, W.K. Chung, Robus sona feaue deecon fo he SLAM of moble obo, n: Inenaonal Confeence on Inellgen Robos and Sysems, 2, pp [11] J.C. Las, R. Man, Technques used n auonomous vehcle sysems: a suvey, Unvesy of Nohen Iowa, 29. [12] B. Wllams, G. Klen, I. Red, Real-me SLAM elocalsaon, n: IEEE Inenaonal Confeence on Compue Vson, vol., 27, pp [13] Z. Zhang, Y. Huang, C. L, Y. Kang, Monocula vson smulaneous localzaon and mappng usng SURF, n: Wold Congess on Inellgen Conol and Auomaon, 28, pp [14] L.M. Paz, P. Pnes, J.D. Tados, J. Nea, Lage-scale 6-DOF SLAM wh seeon-hand, IEEE Tansacons on Robocs 24 () (28) [1] P. Sugess, K. Alaha, L. Ladckỳ, P.H.S. To, Combnng appeaance and sucue fom moon feaues fo oad scene undesandng, n: Bsh Machne Vson Confeence, London, Sepembe 29. [16] N. Muhammad, D. Fof, S. Anouz, Cuen sae of he a of vson based SLAM, n: Socey of Phoo-Opcal Insumenaon Engnees, SPIE, Confeence Sees, vol. 721, 29. [17] H. Chose, K. Nagaan, Topologcal smulaneous localzaon and mappng (SLAM): owad exac localzaon whou explc localzaon, IEEE Tansacons on Robocs and Auomaon 17 (2) (21) [18] J. Sola, A. Monn, M. Devy, T. Vdal-Calleja, Fusng monocula nfomaon n mulcamea SLAM, IEEE Tansacons on Robocs 24 () (28) [19] M. Ian, P. Anandan, Abou Dec Mehods, n: B. Tggs, A. Zsseman, R. Szelsk (Eds.), Vson Algohms: Theoy and Pacce, Spnge, Beln, Hedelbeg, 2, pp [2] C. Has, M. Sephens, A combned cone and edge deeco, n: Alvey Vson Confeence, Manchese, 1988, pp [21] F. Lu, V. Phlomn, Dspay esmaon n seeo sequences usng scene flow, n: Bsh Machne Vson Confeence, London, Sepembe 29. [22] D.G. Lowe, Objec ecognon fom local scale-nvaan feaues, n: Poceedngs of he Inenaonal Confeence on Compue Vson, 2, 1999, pp [23] P. Pnggea, T.P. Beckon, H. Bschof, On coss-specal seeomachng usng dense gaden feaues, n: Poc. Bsh MachneVson Confeence, 212. pp [24] K. Mkolajczyk, T. Tuyelaas, C. Schmd, A. Zsseman, J. Maas, F. Schaffalzky, T. Kad, L. VanGool, A compason of affne egon deecos, Inenaonal Jounal of Compue Vson 6 (1 2) (2) [2] Y. Ma, S. Soao, J. Kosecka, S.S. Sasy, An Invaon o 3-D Vson: Fom Images o Geomec Models, Spnge-Velag, 23. [26] Z. Zhang, A flexble new echnque fo camea calbaon, IEEE Tansacons on Paen Analyss and Machne Inellgence 22 (11) (2) [27] S. Schmd, Applcaons of sae space mehods o navgaon poblems, n: Advances n Conol Sysems, Vol. 3, 1966, pp [28] M. Moun, S. Aya, ANN: a lbay fo appoxmae neaes neghbo seachng, Veson Avalable a: hp:// (accessed: 4/29). [29] E. Nebo, Navgaon sysem desgn, Unpublshed Cene of Excellence fo Auonomous Thess, Unvesy of Sydney, Ausala, May 2. [3] Google Maps, Exploe he wold usng neacve maps. Avalable a: hp:// (accessed: ). [31] C.J. Solomon, T.P. Beckon, Fundamenals of Dgal ImagePocessng: A Paccal Appoach wh Examples n Malab, Wley-Blackwell, 21. Mana Magnabosco s Auomaon Conols Sysem Develope a Ocado Technology (UK). He wok s focused on he delvey of effcen, elable and obus sofwae soluons fo auomac dsbuon sysems coe o he company acves. Mana Magnabosco holds an M.Sc. by Reseach n Image Pocessng and Robocs (211) fom Canfeld Unvesy (UK), whee she appled an opcal hemal combned sensng fo auonomous navgaon. She also holds an M.Eng. (29) and a B.Eng. (27) n Aeospace Engneeng fom he Unvesy of Padova (Ialy). He key eseach neess ae elaed o auonomous sysems, mage pocessng and compue vson. Toby P. Beckon s cuenly a seno lecue whn he School of Engneeng, Canfeld Unvesy (UK). Hs key eseach neess le n he doman of compue vson and mage pocessng, and he leads a ange of eseach acvy n hs aea. D. Beckon holds a Ph.D. n nfomacs fom he Unvesy of Ednbugh (UK). He s a vsng membe of faculy a he Ecole Supeue des Technologes Induselles Avances (Fance) and has addonally held vsng posons a Nohwesen Polyechncal Unvesy (Chna), Shangha Jao Tong Unvesy (Chna) and Waseda Unvesy (Japan). D. Beckon s a Chaeed Engnee, Chaeed Scens and pofessonal membe of he IET and BCS. In addon, he s an Acceded Imagng Scens and an Assocae of he Royal Phoogaphc Socey. He led he developmen of magebased auomac hea deecon fo he 28 UK MoD Gand Challenge wnnes (R.J. Mchell Tophy, (28), IET Innovaon Awad (29)). Hs wok s ecognsed va he Royal Phoogaphc Socey Selwyn Awad fo ealy-caee conbuon o magng scence (211).