Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems
|
|
- Amos Chad Gibson
- 8 years ago
- Views:
Transcription
1 Fto: A Fater, Permutable Icremetal Gradet Method or Bg ata Problem Aaro J. eazo Tbéro S. Caetao Jut omke NICTA ad Autrala Natoal Uverty AARON.FAZIO@ANU.U.AU TIBRIO.CATANO@NICTA.COM.AU JUSTIN.OMK@NICTA.COM.AU Abtract Recet advace optmzato theory have how that mooth trogly covex te um ca be mmzed ater tha by treatg them a a black box batch problem. I th work we troduce a ew method th cla wth a theoretcal covergece rate our tme ater tha extg method, or um wth ucetly may term. Th method alo amedable to a amplg wthout replacemet cheme that practce gve urther peed-up. We gve emprcal reult howg tate o the art perormace.. Itroducto May recet advace the theory ad practce o umercal optmzato have come rom the recogto ad explotato o tructure. Perhap the mot commo tructure that o te um. I mache learg whe applyg emprcal rk mmzato we almot alway ed up wth a optmzato problem volvg the mmzato o a um wth oe term per data pot. The recetly developed SAG algorthm (Schmdt et al., 3) ha how that eve wth th mple orm o tructure, a log a we have ucetly may data pot we are able to do gcatly better tha black-box optmzato techque expectato or mooth trogly covex problem. I practcal term the derece ote a actor o or more. The requremet o ucetly large dataet udametal to thee method. We decrbe the prece orm o th a the bg data codto. etally, t the requremet that the amout o data o the ame order a the codto umber o the problem. The trog covexty requremet Proceedg o the 3 t Iteratoal Coerece o Mache Learg, Bejg, Cha, 4. JMLR: W&CP volume 3. Copyrght 4 by the author(). ot a oerou. Strog covexty hold the commo cae where a quadratc regularzer ued together wth a covex lo. The SAG method ad the Fto method we decrbe th work are mlar ther orm to tochatc gradet decet method, but wth oe crucal derece: They tore addtoal ormato about each data pot durg optmzato. etally, whe they revt a data pot, they do ot treat t a a ovel pece o ormato every tme. Method or the mmzato o te um have clacally bee kow a Icremetal gradet method (Berteka, ). The proo techque ued SAG der udametally rom thoe ued o other cremetal gradet method though. The derece hge o the requremet that data be acceed a radomzed order. SAG doe ot work whe data acceed equetally each epoch, o ay proo techque whch how eve o-dvergece or equetal acce caot be appled. A remarkable property o Fto the tghte o the theoretcal boud compared to the practcal perormace o the algorthm. The practcal covergece rate ee at mot twce a good a the theoretcally predcted rate. Th et t apart rom method uch a LBFGS where the emprcal perormace ote much better tha the relatvely weak theoretcal covergece rate would ugget. The lack o tug requred alo et Fto apart rom tochatc gradet decet (SG). I order to get good perormace out o SG, ubtatal laborou tug o multple cotat ha tradtoally bee requred. A multtude o heurtc have bee developed to help chooe thee cotat, or adapt them a the method progree. Such heurtc are more complex tha Fto, ad do ot have the ame theoretcal backg. SG ha applcato outde o covex problem o coure, ad we do ot propoe that Fto wll replace SG thoe ettg. ve o trogly covex problem SG doe ot exhbt lear covergece lke Fto doe.
2 Fto: A Fater, Permutable Icremetal Gradet Method or Bg ata Problem There are may mlarte betwee SAG, Fto ad tochatc dual coordate decet (SCA) method (Shalev-Shwartz & Zhag, 3). SCA oly applcable to lear predctor. Whe t ca be appled, t ha lear covergece wth theoretcal rate mlar to SAG ad Fto.. Algorthm We coder deretable covex ucto o the orm (w) (w). We aume that each ha Lpchtz cotuou gradet wth cotat L ad trogly covex wth cotat. Clearly we allow, vrtually all mooth, trogly covex problem are cluded. So tead, we wll retrct ourelve to problem atyg the bg data codto. Bg data codto: Fucto o the above orm aty the bg data codto wth cotat L Typcal value o are -8. I pla laguage, we are coderg problem where the amout o data o the ame order a the codto umber (L/) o the problem... Addtoal Notato We upercrpt wth (k) to deote the value o the crpted quatty at terato k. We omt the upercrpt o ummato, ad ubcrpt wth wth the mplcato that dexg tart at. Whe we ue eparate argumet or each, we deote them. Let (k) deote the average (k) P (k). Our tep legth cotat, whch deped o, deoted. We ue agle bracket otato or dot product h,... The Fto algorthm We tart wth a table o kow () value, ad a table o kow gradet ( () ), or each. We wll update thee two table durg the coure o the algorthm. The tep or terato k, a ollow:. Update w ug the tep: w (k) (k) ( (k) ).. Pck a dex j uormly at radom, or ug wthout-replacemet amplg a dcued Secto Set (k) j w (k) the table ad leave the other varable the ame ( (k) (k) or 6 j). 4. Calculate ad tore j ( (k) j ) the table. Our ma theoretcal reult a covergece rate proo or th method. Theorem. Whe the bg data codto hold wth, may be ued. I that ettg, we have talzed () all the ame, the covergece rate : h ( (k) ) (w ) apple 3 k ( () ). 4 See Secto 5 or the proo. I cotrat, SAG acheve a 8 rate whe. Note that o a per epoch ba, the Fto rate exp( /).66. To put that to cotext, epoch wll ee the error boud reduced by more tha 48x. Oe otable eature o our method the xed tep ze. I typcal mache learg problem the trog covexty cotat gve by the tregth cotat o the quadratc regularzer ued. Sce th a kow quatty, a log a the bg data codto hold may be ued wthout ay tug or adjutmet o Fto requred. Th lack o tug a major eature o Fto. I cae where the bg data codto doe ot hold, we cojecture that the tep ze mut be reduced proportoally to the volato o the bg data codto. I practce, the mot eectve tep ze ca be oud by tetg a umber o tep ze, a uually doe wth other tochatc optmato method. A mple way o atyg the bg data codto to duplcate your data eough tme o the hold. Th ot a eectve practce a jut chagg the tep ze, ad o coure t ue more memory. However t doe all wth the curret theory. Aother derece compared to the SAG method that we tore both gradet ad pot. We do ot actually eed twce a much memory however a they ca be tored ummed together. I partcular we tore the quatte p ( P ), ad ue the update rule w p. Th trck doe ot work whe tep legth are adjuted durg optmzato however. The torage o alo a dadvatage whe the gradet ( ) are pare but are ot pare, a t ca caue gcat addtoal memory uage. We do ot recommed the uage o Fto whe gradet are pare. The SAG algorthm der rom Fto oly the w update
3 Fto: A Fater, Permutable Icremetal Gradet Method or Bg ata Problem ad tep legth: w (k) w (k ) 6L 3. Radome key ( (k) ). By ar the mot teretg apect o the SAG ad Fto method the radom choce o dex at each terato. We are ot a ole ettg, o there o heret radome the problem. Yet t eem that a radomzed method requred. Nether method work practce whe the ame orderg ued each pa, or act wth ay o-radom acce cheme we have tred. It hard to emphaze eough the mportace o radome here. The techque o pre-permutg the data, the dog order pae ater that, alo doe ot work. Reducg the tep ze SAG or Fto by or order o magtude doe ot x the covergece ue ether. Other method, uch a tadard SG, have bee oted by varou author to exhbt peed-up whe radom amplg ued tead o order pae, but the derece are ot a extreme a covergece v.. ocovergece. Perhap the mot mlar problem that o coordate decet o mooth covex ucto. Coordate decet caot dverge whe o-radom orderg are ued, but covergece rate are ubtatally wore the o-radomzed ettg (Neterov, Rchtark & Takac ). Reducg the tep ze by a much larger amout, amely by a actor o, doe allow or o-radomzed orderg to be ued. Th gve a extremely low method however. Th the cae covered by the MISO (Maral, 3). A mlar reducto tep ze gve covergece uder oradomzed orderg or SAG alo. Covergece rate or cremetal ub-gradet method wth a varety o orderg appear the lterature alo (Nedc & Berteka, ). Samplg wthout replacemet much ater Other amplg cheme, uch a amplg wthout replacemet, hould be codered. I detal, we mea the cae where each pa over the data a et o amplg wthout replacemet tep, whch cotue utl o data rema, ater whch aother pa tart areh. We call th the permuted cae or mplcty, a t the ame a re-permutg the data ater each pa. I practce, th approach doe ot gve ay peedup wth SAG, however t work pectacularly well wth Fto. We ee peedup o up to a actor o two ug th approach. Th oe o the major derece practce betwee SAG ad Fto. We hould ote that we have o theory to upport th cae however. We are ot aware o ay aaly that prove ater covergece rate o ay optmzato method uder a amplg wthout replacemet cheme. A teretg dcuo o SG uder wthout-replacemet amplg appear Recht & Re (). The SCA method alo ometme ued wth a permuted orderg (Shalev-Shwartz & Zhag, 3), our expermet Secto 7 how that th ometme reult a large peedup over uorm radom amplg, although t doe ot appear to be a relable a wth Fto. 4. Proxmal varat We ow coder compote problem o the orm (w) (w) r(w), where r covex but ot ecearly mooth or trogly covex. Such problem are ote addreed ug proxmal algorthm, partcularly whe the proxmal operator or r: prox r (z) argm x kx zk r(x) ha a cloed orm oluto. A example would be the ue o L regularzato. We ow decrbe the Fto update or th ettg. Frt otce that whe we et w the Fto method, t ca be terpreted a mmzg the quatty: B(x) ( ) h ( ),x kx k, wth repect to x, or xed. Th related to the upper boud mmzed by MISO, where tead L. It traght orward to mody th or the compote cae: B r (x) r(x) ( ) h ( ),x kx k. The mmzer o the moded B r ca be expreed ug the proxmal operator a:! w prox r / ( ). Th trogly reemble the update the tadard gradet decet ettg, whch or a tep ze o /L w prox r /L w (k ) L (w (k ) ). We have ot yet developed ay theory upportg the proxmal varat o Fto, although emprcal evdece ugget t ha the ame covergece rate a the o-proxmal cae.
4 5. Covergece proo Fto: A Fater, Permutable Icremetal Gradet Method or Bg ata Problem We tart by tatg two mple lemma. All expectato the ollowg are over the choce o dex j at tep k. Quatte wthout upercrpt are at ther value at terato k. Lemma. The expected tep [w (k) ] w (w). I.e. the w tep a gradet decet tep expectato ( / L ). A mlar equalty alo hold or SG, but ot or SAG. Proo. [w (k) ] w apple (w j) j(w) j( j ) (w ) (w) ( ) Now mply (w ) a P ( ), o the oly term that rema (w). Lemma. (ecompoto o varace) We ca decompoe P kw k a kw k w. Proo. kw k w w w Ma proo w, w, Our proo proceed by cotructo o a Lyapuov ucto T ; that, a ucto that boud a quatty o teret, ad that decreae each terato expectato. Our Lyapuov ucto T T T T 3 T 4 compoed o the um o the ollowg term, T. T ( ), ( ) h ( ),w, T 3 kw k, T 4 We ow tate how each term chage betwee tep k ad k. Proo are oud the appedx the upplemetary materal: [T (k) ] T apple ( ),w L 3 kw k, [T (k) ] T apple T (w) ( ) 3 k(w) ( )k. w, (w) 3 h(w) ( ),w, [T (k) 3 ] T 3 ( )T 3 (w),w 3 k( ) (w)k, [T (k) 4 ] T 4 3 kw k. Theorem. Betwee tep k ad k,, ad the [T (k) ] T apple T. w apple Proo. We take the three lemma above ad group lke term to get [T (k) ] T apple ( ),w ( ) (w) h( ),w ( ) (w), w ( L ) kw k 3 h(w) ( ),w ( ) 3 k( ) (w)k
5 Fto: A Fater, Permutable Icremetal Gradet Method or Bg ata Problem w Next we cacel part o the rt le ug ( apple (w) ( ) w, baed o B3 the Appedx. We the pull term occurrg T together, gvg [T (k) ] T apple T ( ) ( (w),w apple ( ) (w) T ( L ) kw k 3 h(w) ( ),w ( ( ) 3 k( ) (w)k ) w ( ) Next we ue the tadard equalty (B5) ( ) ( ) (w),w apple ( ) w, whch chage the bottom row to ( ) w ( ) P. Thee two term ca the be grouped ug Lemma, to gve [T (k) ] T apple T L 3 kw k apple ( ) (w) T 3 h(w) ( ),w (.. ) 3 k( ) (w)k. We ue the ollowg equalty (Corollary 6 Appedx) to cacel agat the P kw k term: apple (w) T apple 3 h(w) ( ),w L 3 kw k 3 k(w) ( )k, ad the apply the ollowg mlar equalty (B7 Appedx) to partally cacel P k ( ) (w)k : apple Leavg u wth apple (w) T 3 k ( ) (w)k. [T (k) ] T apple T ( ) 3 k( ) (w)k. The remag gradet orm term o-potve uder the codto peced our aumpto. Theorem 3. The Lyapuov ucto boud ( ) (w ) a ollow: ( (k) ) (w ) apple T (k). Proo. Coder the ollowg ucto, whch we wll call R(x): R(x) ( ) h ( ),x kx k. Whe evaluated at t mmum wth repect to x, whch we deote w P ( ), t a lower boud o (w ) by trog covexty. However, we are evaluatg at w P ( ) tead the (egated) Lyapuv ucto. R covex wth repect to x, o by deto R(w) R apple w R( ) R(w ). Thereore by the lower boudg property ( ) R(w) ( ) R( ) ( ) ( ) ( ) (w ). Now ote that T ( ) R(w). So ( ) (w ) apple T. R(w ) (w ) () Theorem 4. I the Fto method talzed wth all the ame,ad the aumpto o Theorem hold, the the
6 Fto: A Fater, Permutable Icremetal Gradet Method or Bg ata Problem covergece rate : h ( (k) ) wth c. (w ) apple c Proo. By urollg Theorem, we get k [T (k) ] apple T (). Now ug Theorem 3 h ( (k) ) (w ) apple k ( () ), k T (). We eed to cotrol T () alo. Sce we are aumg that all tart the ame, we have that T () ( () ) ( () ) ( () ),w () () w() () ( () ),w () () ( () ) ( () ) ( () ). ( () ) 6. Lower complexty boud ad explotg problem tructure The theory or the cla o mooth, trogly covex problem wth Lpchtz cotuou gradet uder rt order optmzato method (kow a S,,L ) well developed. Thee reult requre the techcal codto that the dmeoalty o the put pace R m much larger tha the umber o terato we wll take. For mplcty we wll aume th the cae the ollowg dcuo. It kow that problem ext S,,L or whch the terate covergece rate bouded by: p! k w (k) w L/ p L/ w () w. I act, whe ad L are kow advace, th rate acheved up to a mall cotat actor by everal method, mot otably by Neterov accelerated gradet decet method (Neterov 988, Neterov 998). I order to acheve covergece rate ater tha th, addtoal aumpto mut be made o the cla o ucto codered. Recet advace have how that all that requred to acheve gcatly ater rate a te um tructure, uch a our problem etup. Whe the bg data codto hold our method acheve a rate.665 per epoch expectato. Th rate oly deped o the codto umber drectly, through the bg data codto. For example, wth L/,,, the atet poble rate or a black box method a.996, wherea Fto acheve a rate o.665 expectato or 4,,, or 4x ater. The requred amout o data ot uuual moder mache learg problem. I practce, whe quaewto method are ued tead o accelerated method, a peedup o -x more commo. 6.. Oracle cla We ow decrbe the (tochatc) oracle cla FS,,L, (Rm ) or whch SAG ad Fto mot aturally t. Fucto cla: (w) P (w), wth S,,L (Rm ). Oracle: ach query take a pot x R m, ad retur j, j (w) ad j (w), wth j choe uormly at radom. Accuracy: Fd w uch that [ w (k) w ] apple. The ma choce made ormulatg th deto puttg the radom choce the oracle. Th retrct the method allowed qute trogly. The alteratve cae, where the dex j put to the oracle addto to x, alo teretg. Aumg that the method ha acce to a ource o true radom dce, we call that cla S,,L, (Rm ). I Secto 3 we dcu emprcal evdece that ugget that ater rate are poble S,,L, (Rm ) tha or FS,,L, (Rm ). It hould rt be oted that there a trval lower boud rate or SS,,L, (R m ) o reducto per tep. It ot clear th ca be acheved or ay te. Fto oly a actor o o th rate, amely at, ad aymptote toward th rate or very large. SCA, whle ot applcable to all problem th cla, alo acheve the rate aymptotcally. Aother cae to coder the mooth covex but otrogly covex ettg. We tll aume Lpchtz cotuou gradet. I th ettg we wll how that or ucetly hgh dmeoal put pace, the (o-tochatc) lower complexty boud the ame or the te um cae ad caot be better tha that gve by treatg a a gle black box ucto. The ull proo the Appedx, but the dea a ollow: whe the are ot trogly covex, we ca chooe them uch that they do ot teract wth each other, a log a the
7 Fto: A Fater, Permutable Icremetal Gradet Method or Bg ata Problem dmeoalty much larger tha k. More precely, we may chooe them o that or ay x ad y ad ay 6 j, h (x), j (y) hold. Whe the ucto do ot teract, o optmzato cheme may reduce the terate error ater tha by jut hadlg each eparately. og o a -order aho gve the ame rate a jut treatg ug a black box method. For trogly covex, t ot poble or them to ot teract the above ee. By deto trog covexty requre a quadratc compoet each that act o all dmeo. 7. xpermet I th ecto we compare Fto, SAG, SCA ad LBFGS. We oly coder problem where the regularzer large eough o that the bg data codto hold, a th the cae our theory upport. However, practce our method ca be ued wth maller tep ze the more geeral cae, much the ame way a SAG. Sce we do ot kow the Lpchtz cotat or thee problem exactly, the SAG method wa ru or a varety o tep ze, wth the oe that gave the atet rate o covergece plotted. The bet tep-ze or SAG uually ot what the theory ugget. Schmdt et al. (3) ugget ug L tead o the theoretcal rate 6L. For Fto, we d that ug the atet rate whe the bg data codto hold or ay >. Th the tep uggeted by our theory whe. Iteretgly, reducg to doe ot mprove the covergece rate. Itead we ee o urther mprovemet our expermet. For both SAG ad Fto we ued a derg tep rule tha uggeted by the theory or the rt pa. For Fto, durg the rt pa, ce we do ot have dervatve or each yet, we mply um over the k term ee o ar w (k) k k (k) k k ( (k) ), where we proce data pot dex order or the rt pa oly. A mlar trck uggeted by Schmdt et al. (3) or SAG. Sce SCA oly apple to lear predctor, we are retrcted poble tet problem. We chooe log lo or 3 bary clacato dataet, ad quadratc lo or regreo tak. For clacato, we teted o the jc ad covtype dataet, a well a MNIST clayg - 4 agat 5-9. For regreo, we chooe the two dataet rom the UCI repotory: the mllo og year regreo cjl/ lbvmtool/dataet/bary.html dataet, ad the lce-localzato dataet. The trag porto o the dataet are o ze 5.3 5, 5. 4, 6. 4, ad repectvely. Fgure 6 how the reult o our expermet. Frtly we ca ee that LBFGS ot compettve wth ay o the cremetal gradet method codered. Secodly, the opermuted SAG, Fto ad SCA ote coverge at very mlar rate. The oberved derece are uually dow to the peed o the very rt pa, where SAG ad Fto are ug the above metoed trck to peed ther covergece. Ater the rt pa, the lope o the le are uually comparable. Whe coderg the method wth permutato each pa, we ee a clear advatage or Fto. Iteretgly, t gve very lat le, dcatg very table covergece. 8. Related work Tradtoal cremetal gradet method (Berteka, ) have the ame orm a SG, but appled to te um. etally they are the o-ole aalogue o SG. Applyg SG to trogly covex problem doe ot yeld lear covergece, ad practce t lower tha the lear-covergg method we dcu the remader o th ecto. Bede the method that all uder the clacal Icremetal gradet moker, SAG ad MISO (Maral, 3) method are alo related. MISO method all to the cla o upper boud mmzato method, uch a M ad clacal gradet decet. MISO eetally the Fto method, but wth tep ze tme maller. Whe ug thee larger tep ze, the method o loger a upper boud mmzato method. Our method ca be ee a MISO, but wth a tep ze cheme that gve ether a lower or upper boud mmato method. Whle th work wa uder peer revew, a tech report (Maral (4)) wa put o arv that etablhe the covergece rate o MISO wth tep ad wth a 3 per tep. Th mlar but ot qute a good a the rate we etablh. Stochatc ual Coordate decet (Shalev-Shwartz & Zhag, 3) alo gve at covergece rate o problem or whch t applcable. It requre computg the covex cojugate o each, whch make t more complex to mplemet. For the bet perormace t ha to take advatage o the tructure o the loe alo. For mple lear clacato ad regreo problem t ca be eectve. Whe ug a pare dataet, t a better choce tha Fto due to the memory requremet. For lear predctor, t theoretcal covergece rate o ( ) per tep a lttle ater tha what we etablh or Fto, however t doe ot appear to be ater our expermet.
8 Fto: A Fater, Permutable Icremetal Gradet Method or Bg ata Problem ull gradet orm ull gradet orm poch Fgure. MNIST poch Fgure 3. Covtype ull gradet orm ull gradet orm ull gradet orm poch Fgure. jc poch poch SAG Fto perm SCA perm Fto SCA LBFGS Fgure 5. lce Fgure 6. Covergece rate plot or tet problem Fgure 4. Mllo Sog 9. Cocluo We have preeted a ew method or mmzato o te um o mooth trogly covex ucto, whe there a ucetly large umber o term the ummato. We addtoally develop ome theory or the lower complexty boud o th cla, ad how the emprcal perormace o our method. Reerece Berteka, mtr P. Icremetal gradet, ubgradet, ad proxmal method or covex optmzato: A urvey. Techcal report,. Maral, Jule. Optmzato wth rt-order urrogate ucto. ICML, 3. Maral, Jule. Icremetal majorzato-mmzato optmzato wth applcato to large-cale mache learg. Techcal report, INRIA Greoble Rhe-Alpe / LJK Laboratore Jea Kutzma, 4. Nedc, Agela ad Berteka, mtr. Stochatc Optmzato: Algorthm ad Applcato, chapter Covergece Rate o Icremetal Subgradet Algorthm. Kluwer Academc,. Neterov, Yu. O a approach to the cotructo o optmal method o mmzato o mooth covex ucto. koomka Mateatcheke Metody, 4:59 57, 988. Neterov, Yu. Itroductory Lecture O Covex Programmg. Sprger, 998. Neterov, Yu. cecy o coordate decet method o huge-cale optmzato problem. Techcal report, COR,. Recht, Bejam ad Re, Chrtopher. Beeath the valley o the ocommutatve arthmetc-geometrc mea equalty: cojecture, cae-tude, ad coequece. Techcal report, Uverty o Wco-Mado,.
9 Fto: A Fater, Permutable Icremetal Gradet Method or Bg ata Problem Rchtark, Peter ad Takac, Mart. Iterato complexty o radomzed block-coordate decet method or mmzg a compote ucto. Techcal report, Uverty o dburgh,. Schmdt, Mark, Roux, Ncola Le, ad Bach, Frac. Mmzg te um wth the tochatc average gradet. Techcal report, INRIA, 3. Shalev-Shwartz, Sha ad Zhag, Tog. Stochatc dual coordate acet method or regularzed lo mmzato. JMLR, 3.
10 Appedx Bac covexty equalte The ollowg equalte are clacal. See Neterov 998 or proo. They hold or all x & y, whe S, (B) (y) apple (x)h (x),y x L kx yk (B) (y) (x)h (x),y x L k (x) (y)k (B3) (y) (x)h (x),y x kx yk (B4) h (x) (y),x y L k (x) (y)k (B5) h (x) (y),x y kx yk We alo ue varat o B ad B3 that are ummed over each, wth x ad y w: (w) ( ) h ( ),w L k (x) (y)k,l. (w) ( ) h ( ),w kw k Thee are ued the ollowg egated ad rearraged orm: (w) T (w) ( ) h ( ),w (B6) ) (w) T apple kw k (B7) (w) T apple L k (w) ( )k. Lyapuov term boud Smplyg each Lyapuov term arly traght orward. We ue extevely that or 6 j. Notealothat (k) j (B8) w (k) w (w j) j ( j ) j(w). Lemma 6. Betwee tep k ad k, the T ( ) term chage a ollow: [T (k) ] T apple ( ),w L 3 kw k. w, adthat (k) Proo. Frt we ue the tadard Lpchtz upper boud (B): (y) apple (x)h (x),y x L kx yk. We ca apply th ug y (k) (w j) ad x :
11 ( (k) ) apple ( ) We ow take expectato over j, gvg: [( (k) )] ( ),w j L kw jk. ( ) apple ( ),w L 3 kw k. Lemma 7. Betwee tep k ad k, the T [T (k) ] T apple Proo. We troduce the otato T T rt ug (k) j w: T (k) T Now we mply the chage T : T P P ( ) h ( ),w (w) ( ) 3 k(w) ( )k w, (w) 3 h(w) ( ),w. T (k) T ) T (k) T P ( ) ad T ( (k) ) ( ) j( j ) j( j ) ( (k) We ow mplyg the rt two term ug ( (k) ),w (k) The lat term o quato expad urther: ( (k) ),w (k) w j(w) ( ) ( (k) ),w (k) w w ),w (k) (k) j w: T P h ( ),w j(w) term chage a ollow: ( ) (k). We mply the chage T ( (k) ),w (k) w. () T T j ( j ),w j j (w),w w T j ( j ),w j. * ( ) j( j )j(w),w (k) w * ( ),w (k) The ecod er product term mple urther ug B8: j(w) j( j ),w (k) w j(w) j( j ), (w j) j (w) j( j ),w j w j(w) j( j ),w (k) w. () j ( j ) j(w) j (w) j( j ), j( j ) j(w).
12 We mply the ecod term: Groupg all remag term gve: j (w) j( j ), j( j ) j(w) j(w) j( j ). T (k) T apple j( j ) j ( j ),w j j(w) j(w) j( j ) j (w) j( j ),w j * ( ),w (k) w. We ow take expectato o each remag term. For the bottom er product we ue Lemma : * * ( ),w (k) w ( ), (w) w, (w). Takg expectato o the remag term traght orward. We get: [T (k) ] T apple ( ) (w) h( ),w 3 k(w) ( )k 3 h(w) ( ),w w, (w). Lemma 8. Betwee tep k ad k, the T 3 P kw k term chage a ollow: [T (k) 3 ] T 3 ( ) T 3 (w),w 3 k( ) (w)k. Proo. We expad a: T (k) 3 w (k) w (k) (k) w w w(k) w (k) w (k) w (k) w, w (3) (k). (4) We expad the three term o the rght eparately. For the rt term: w(k) w (w j) ( j( j ) j (w)) kw jk k j( j ) j (w)k h j( j ) j (w),w j. (5) 3
13 For the ecod term o quato 4, ug For the thrd term o quato 4: w (k) w, w w (k) (k) (k) j w: kw T 3 kw jk. * k kw jk w (k) w, w w (k) w, w j w (k) w, w The ecod er product term quato 6 become (ug B8): w (k) w, w j (w j) j ( j ) j(w),w j kw jk j ( j ) j(w),w j. w (k) w, w j. (6) Notce that the er product term here cacel wth the oe 5. Now we ca take expectato o each remag term. Recall that [w (k) ] w (w), othert er product term 6 become: "* # w (k) w, w (w),w. All other term do t mply uder expectato. So the reult : [T (k) 3 ] T 3 ( ) kw k (w),w 3 k ( ) (w)k. Lemma 9. Betwee tep k ad k, the T 4 P [T (k) 4 ] T 4 Proo. Note that (k) (w j), o: Now ug T (k) 4 P (k) (k) (k) (k) (w j) term chage a ollow: w 3 kw k. (k) (k), (k) w j,! (k). (k) (k) (w j) to mply the er product term: kw jk (k) hw j, j w 4
14 kw Takg expectato gve the reult. Lemma. Let S,L. The we have: (x) (y)h (y),x y jk (k) (k) kw jk (L ) k (x) (y)k j kw jk w kw jk. (7) L (L ) ky xk (L ) h (x) (y),y x. Proo. ee the ucto g a g(x) (x) kxk. The the gradet g (x) (x) x. g ha a lpchtz gradet wth wth cotat L. Bycovextywehave: Now replacg g wth : g(x) g(y)hg (y),x y (L ) kg (x) g (y)k. Note that (x) kxk (y) kyk h (y) y, x y (L ) k (x) x (y)yk. o: (L ) k (x) x (y)yk (L ) k (x) (y)k ky xk (L ) (L ) h (x) (y),y x, (x) (y)h (y),x y (L ) k (x) (y)k kxk kyk ky xk (L ) (L ) h (x) (y),y x hy, x y. Now ug: we get: kxk hy, x kyk kx yk, (x) (y)h (y),x y (L ) k (x) (y)k Note the orm y term cacel, ad: kyk kx yk kx yk (L ) (L ) h (x) (y),y x hy, y kx yk (L ) kx yk (L ) kx yk (L ) L (L ) kx yk. So: 5
15 (x) (y)h (y),x y (L ) h (x) (y),y x (L ) k (x) (y)k L ky xk (L ) Corollary. Take (x) vector: (x) P (x), wth the bg data codto holdg wth cotat ( ) h ( ),x k (x) ( )k L kx k h (x) ( ), x.. The or ay x ad Proo. We apply Lemma to each,butteadougtheactualcotatl, weue, whch uder the bg data aumpto larger tha L: (x) ( )h ( ),x k (x) ( )k L kx k h (x) ( ), x. Averagg over gve the reult. 3 Lower complexty boud I th ecto we ue the ollowg techcal aumpto, a ued Neterov (998): Aumpto : A optmzato method at tep k may oly voke the oracle wth a pot x (k) that o the orm: x (k) x () a g (), where g () the dervatve retured by the oracle at tep, ad a R. Th aumpto prevet a optmzato method rom jut gueg the correct oluto wthout dog ay work. Vrtually all optmzato method all to uder th aumpto. Smple ( )k boud Ay procedure that mmze a um o the orm (w) P (w) by uorm radom acce o retrcted by the requremet that t ha to actually ee each term at leat oce order to d the mmum. Th lead to a k rate expectato. We ow ormalze uch a argumet. We wll work R,matchgthedmeoalty o the problem to the umber o term the ummato. Theorem. For ay FS,,, (R ), we have that a k tep optmzato procedure gve: [(w)] (w ) k (w () ) (w ) Proo. We wll exhbt a mple wort-cae problem. Wthout lo o geeralty we aume that the rt oracle ace by the optmzato procedure at w. I ay other cae, we ht our pace the ollowg argumet approprately. Let (w) P h (w ) kwk. The clearly the oluto w or each, wth mmum o (w ) 4.Forw we have ().Scethedervatveoeach j o the th compoet we have ot yet ee,thevalueoeachw rema ule term ha bee ee. Let v (k) be the umber o uque term we have ot ee up to tep k. Betweetepk ad k, v decreae by wth probably v ad tay the ame otherwe. So [v (k) v (k) ]v (k) 6 v (k) v (k).
16 So we may dee the equece (k) [ (k) v (k) ] k v (k), whch the martgale wth repect to v, a (k). k [v (k) v (k) ] k v (k) Now ce k choe advace, toppg tme theory gve that [ (k) ][ () ].So [ ) [v (k) ] k v (k) ], k. By Aumpto, the ucto ca be at mot mmzed over the dmeo ee up to tep k. The ee dmeo cotrbute a value o 4 ad the uee term to the ucto. So we have that: [(w (k) )] (w ) [v (k) ] 4 [v(k) ] 4 [v(k) ] k 4 k h (w () ) (w ). 4 Mmzato o o-trogly covex te um It kow that the cla o covex, cotuou & deretable problem, wth L (Rm ), ha the ollowg lower complexty boud whe k<m: F, L Lpchtz cotuou dervatve (x (k) ) (k) (x ) L x () x 8(k ), whch proved va explct cotructo o a wort-cae ucto where t hold wth equalty. Let th wort cae ucto be deoted h (k) at tep k. We wll how that the ame boud apple or the te-um cae, o a per pa equvalet ba, by a mple cotructo. Theorem 3. The ollowg lower boud hold or k a multple o : (x (k) ) (k) (x ) L x () x 8( k ), whe a te um o term (x) P (x), wth each F, L (Rm ), ad wth m>k,uderthe oracle model where the optmzato method may chooe the dex to acce at each tep. Proo. Let h be a copy o h (k) redeed to be o the ubet o dmeo j, orj...k,orotherword, (x) h (k) ([x,x,...x j,...]). The we wll ue: h (k) (k) (x) 7 h (k) (x)
17 a a wort cae ucto or tep k. Sce the dervatve are orthogoal betwee h ad h j or 6 j, byaumpto,theboudoh (k) (x (k) ) h (k) (x ) deped oly o the umber o tme the oracle ha bee voked wth dex, oreach. Let th be deoted c. The we have that: (x (k) ) (k) (x ) L 8 x () x () (c ). Where k k () the orm o the dmeo j or j...k. We ca combe thee orm to a regular ucldea orm: (x (k) ) (k) (x L x () x ) 8 (c ). Now otce that P (c ) uder the cotrat P c k mmzed whe each c k.sowehave: (x (k) ) (k) (x ) L x () x 8 ( k, ) L x () x 8( k ), whch the ame lower boud a or k/ terato o a optmzato method o drectly. 8
Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems
Fto: A Faster, Permutable Icremetal Gradet Method for Bg Data Problems Aaro J Defazo Tbéro S Caetao Just Domke NICTA ad Australa Natoal Uversty AARONDEFAZIO@ANUEDUAU TIBERIOCAETANO@NICTACOMAU JUSTINDOMKE@NICTACOMAU
More informationConversion of Non-Linear Strength Envelopes into Generalized Hoek-Brown Envelopes
Covero of No-Lear Stregth Evelope to Geeralzed Hoek-Brow Evelope Itroducto The power curve crtero commoly ued lmt-equlbrum lope tablty aaly to defe a o-lear tregth evelope (relatohp betwee hear tre, τ,
More informationAPPENDIX III THE ENVELOPE PROPERTY
Apped III APPENDIX III THE ENVELOPE PROPERTY Optmzato mposes a very strog structure o the problem cosdered Ths s the reaso why eoclasscal ecoomcs whch assumes optmzg behavour has bee the most successful
More informationPreprocess a planar map S. Given a query point p, report the face of S containing p. Goal: O(n)-size data structure that enables O(log n) query time.
Computatoal Geometry Chapter 6 Pot Locato 1 Problem Defto Preprocess a plaar map S. Gve a query pot p, report the face of S cotag p. S Goal: O()-sze data structure that eables O(log ) query tme. C p E
More informationIn the UC problem, we went a step further in assuming we could even remove a unit at any time if that would lower cost.
uel Schedulg (Chapter 6 of W&W.0 Itroducto I ecoomc dpatch we aumed the oly lmtato were o the output of the geerator: m g. h aumed that we could et ge to ay value we dered wth the rage, at ay tme, to acheve
More informationData Analysis Toolkit #10: Simple linear regression Page 1
Data Aaly Toolkt #0: mple lear regreo Page mple lear regreo the mot commoly ued techque f determg how oe varable of teret the repoe varable affected by chage aother varable the explaaty varable. The term
More informationNumerical Methods with MS Excel
TMME, vol4, o.1, p.84 Numercal Methods wth MS Excel M. El-Gebely & B. Yushau 1 Departmet of Mathematcal Sceces Kg Fahd Uversty of Petroleum & Merals. Dhahra, Saud Araba. Abstract: I ths ote we show how
More informationGRADUATION PROJECT REPORT
SPAM Flter School of Publc Admtrato Computer Stude Program GRADUATION PROJECT REPORT 2007-I-A02 SPAM Flter Project group leader: Project group member: Supervor: Aeor: Academc year (emeter): MCCS390 Graduato
More informationCIS603 - Artificial Intelligence. Logistic regression. (some material adopted from notes by M. Hauskrecht) CIS603 - AI. Supervised learning
CIS63 - Artfcal Itellgece Logstc regresso Vasleos Megalookoomou some materal adopted from otes b M. Hauskrecht Supervsed learg Data: D { d d.. d} a set of eamples d < > s put vector ad s desred output
More informationSHAPIRO-WILK TEST FOR NORMALITY WITH KNOWN MEAN
SHAPIRO-WILK TEST FOR NORMALITY WITH KNOWN MEAN Wojcech Zelńsk Departmet of Ecoometrcs ad Statstcs Warsaw Uversty of Lfe Sceces Nowoursyowska 66, -787 Warszawa e-mal: wojtekzelsk@statystykafo Zofa Hausz,
More informationAbraham Zaks. Technion I.I.T. Haifa ISRAEL. and. University of Haifa, Haifa ISRAEL. Abstract
Preset Value of Autes Uder Radom Rates of Iterest By Abraham Zas Techo I.I.T. Hafa ISRAEL ad Uversty of Hafa, Hafa ISRAEL Abstract Some attempts were made to evaluate the future value (FV) of the expected
More informationSupply Chain Management Chapter 5: Application of ILP. Unified optimization methodology. Beun de Haas
Supply Cha Maagemet Chapter 5: Ufed Optmzato Methodology for Operatoal Plag Problem What to do whe ILP take too much computato tme? Applcato of ILP Tmetable Dutch Ralway (NS) Bu ad drver chedulg at Coeo,
More informationConstrained Cubic Spline Interpolation for Chemical Engineering Applications
Costraed Cubc Sple Iterpolato or Chemcal Egeerg Applcatos b CJC Kruger Summar Cubc sple terpolato s a useul techque to terpolate betwee kow data pots due to ts stable ad smooth characterstcs. Uortuatel
More informationSpeeding up k-means Clustering by Bootstrap Averaging
Speedg up -meas Clusterg by Bootstrap Averagg Ia Davdso ad Ashw Satyaarayaa Computer Scece Dept, SUNY Albay, NY, USA,. {davdso, ashw}@cs.albay.edu Abstract K-meas clusterg s oe of the most popular clusterg
More informationA general sectional volume equation for classical geometries of tree stem
Madera y Boque 6 (2), 2:89-94 89 NOTA TÉCNICA A geeral ectoal volume equato for clacal geometre of tree tem Ua ecuacó geeral para el volume de la eccó de la geometría cláca del troco de lo árbole Gldardo
More informationAn Effectiveness of Integrated Portfolio in Bancassurance
A Effectveess of Itegrated Portfolo Bacassurace Taea Karya Research Ceter for Facal Egeerg Isttute of Ecoomc Research Kyoto versty Sayouu Kyoto 606-850 Japa arya@eryoto-uacp Itroducto As s well ow the
More information10.5 Future Value and Present Value of a General Annuity Due
Chapter 10 Autes 371 5. Thomas leases a car worth $4,000 at.99% compouded mothly. He agrees to make 36 lease paymets of $330 each at the begg of every moth. What s the buyout prce (resdual value of the
More informationAnalysis of Multi-product Break-even with Uncertain Information*
Aalyss o Mult-product Break-eve wth Ucerta Iormato* Lazzar Lusa L. - Morñgo María Slva Facultad de Cecas Ecoómcas Uversdad de Bueos Ares 222 Córdoba Ave. 2 d loor C20AAQ Bueos Ares - Argeta lazzar@eco.uba.ar
More informationBasic statistics formulas
Wth complmet of tattcmetor.com, the te for ole tattc help Set De Morga Law Bac tattc formula Meaure of Locato Sample mea (AUB) c A c B c Commutatvty & (A B) c A c U B c A U B B U A ad A B B A Aocatvty
More informationThe simple linear Regression Model
The smple lear Regresso Model Correlato coeffcet s o-parametrc ad just dcates that two varables are assocated wth oe aother, but t does ot gve a deas of the kd of relatoshp. Regresso models help vestgatg
More informationCyber Journals: Multidisciplinary Journals in Science and Technology, Journal of Selected Areas in Telecommunications (JSAT), January Edition, 2011
Cyber Jourals: Multdscplary Jourals cece ad Techology, Joural of elected Areas Telecommucatos (JAT), Jauary dto, 2011 A ovel rtual etwork Mappg Algorthm for Cost Mmzg ZHAG hu-l, QIU Xue-sog tate Key Laboratory
More informationOn formula to compute primes and the n th prime
Joural's Ttle, Vol., 00, o., - O formula to compute prmes ad the th prme Issam Kaddoura Lebaese Iteratoal Uversty Faculty of Arts ad ceces, Lebao Emal: ssam.addoura@lu.edu.lb amh Abdul-Nab Lebaese Iteratoal
More information3.6. Metal-Semiconductor Field Effect Transistor (MESFETs)
.6. Metal-Semcouctor Fel Effect rator (MESFE he Metal-Semcouctor-Fel-Effect-rator (MESFE cot of a couctg chael potoe betwee a ource a ra cotact rego a how the Fgure.6.1. he carrer flow from ource to ra
More informationANOVA Notes Page 1. Analysis of Variance for a One-Way Classification of Data
ANOVA Notes Page Aalss of Varace for a Oe-Wa Classfcato of Data Cosder a sgle factor or treatmet doe at levels (e, there are,, 3, dfferet varatos o the prescrbed treatmet) Wth a gve treatmet level there
More informationRUSSIAN ROULETTE AND PARTICLE SPLITTING
RUSSAN ROULETTE AND PARTCLE SPLTTNG M. Ragheb 3/7/203 NTRODUCTON To stuatos are ecoutered partcle trasport smulatos:. a multplyg medum, a partcle such as a eutro a cosmc ray partcle or a photo may geerate
More informationRelaxation Methods for Iterative Solution to Linear Systems of Equations
Relaxato Methods for Iteratve Soluto to Lear Systems of Equatos Gerald Recktewald Portlad State Uversty Mechacal Egeerg Departmet gerry@me.pdx.edu Prmary Topcs Basc Cocepts Statoary Methods a.k.a. Relaxato
More informationApplications of Support Vector Machine Based on Boolean Kernel to Spam Filtering
Moder Appled Scece October, 2009 Applcatos of Support Vector Mache Based o Boolea Kerel to Spam Flterg Shugag Lu & Keb Cu School of Computer scece ad techology, North Cha Electrc Power Uversty Hebe 071003,
More informationOptimal multi-degree reduction of Bézier curves with constraints of endpoints continuity
Computer Aded Geometrc Desg 19 (2002 365 377 wwwelsevercom/locate/comad Optmal mult-degree reducto of Bézer curves wth costrats of edpots cotuty Guo-Dog Che, Guo-J Wag State Key Laboratory of CAD&CG, Isttute
More informationBanking (Early Repayment of Housing Loans) Order, 5762 2002 1
akg (Early Repaymet of Housg Loas) Order, 5762 2002 y vrtue of the power vested me uder Secto 3 of the akg Ordace 94 (hereafter, the Ordace ), followg cosultato wth the Commttee, ad wth the approval of
More informationSwarm Based Truck-Shovel Dispatching System in Open Pit Mine Operations
Swarm Baed Truck-Shovel Dpatchg Sytem Ope Pt Me Operato Yaah Br, W. Scott Dubar ad Alla Hall Departmet of Mg ad Meral Proce Egeerg Uverty of Brth Columba, Vacouver, B.C., Caada Emal: br@mg.ubc.ca Abtract
More informationIntegrating Production Scheduling and Maintenance: Practical Implications
Proceedgs of the 2012 Iteratoal Coferece o Idustral Egeerg ad Operatos Maagemet Istabul, Turkey, uly 3 6, 2012 Itegratg Producto Schedulg ad Mateace: Practcal Implcatos Lath A. Hadd ad Umar M. Al-Turk
More informationOnline Appendix: Measured Aggregate Gains from International Trade
Ole Appedx: Measured Aggregate Gas from Iteratoal Trade Arel Burste UCLA ad NBER Javer Cravo Uversty of Mchga March 3, 2014 I ths ole appedx we derve addtoal results dscussed the paper. I the frst secto,
More informationAverage Price Ratios
Average Prce Ratos Morgstar Methodology Paper August 3, 2005 2005 Morgstar, Ic. All rghts reserved. The formato ths documet s the property of Morgstar, Ic. Reproducto or trascrpto by ay meas, whole or
More informationCredit Risk Evaluation of Online Supply Chain Finance Based on Third-party B2B E-commerce Platform: an Exploratory Research Based on China s Practice
Iteratoal Joural of u- ad e- Servce, Scece ad Techology Vol.8, No.5 (2015, pp.93-104 http://dx.do.org/10.14257/juet.2015.8.5.09 Credt Rk Evaluato of Ole Supply Cha Face Baed o Thrd-party B2B E-commerce
More informationhal-00092005, version 2-12 Mar 2008
Author maucrpt, publhed "6th IFAC Sympoum o Fault Detecto, Supervo ad Safety of Techcal Procee, Safeproce'06, Beg : Cha (2006)" ODDS ALGORITHM -BASED OPPORTUNITY-TRIGGERED PREVENTIVE MAINTENANCE WITH PRODUCTION
More information6.7 Network analysis. 6.7.1 Introduction. References - Network analysis. Topological analysis
6.7 Network aalyss Le data that explctly store topologcal formato are called etwork data. Besdes spatal operatos, several methods of spatal aalyss are applcable to etwork data. Fgure: Network data Refereces
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationAnalysis of Two-Echelon Perishable Inventory System with Direct and Retrial demands
O Joural of Mathematc (O-JM) e-: 78-578 p-: 9-765X. Volume 0 ue 5 Ver. (ep-oct. 04) 5-57 www.oroural.org aly of Two-chelo erhable vetory ytem wth rect ad etral demad M. amehpad C.eryaamy K. Krha epartmet
More informationBayesian Network Representation
Readgs: K&F 3., 3.2, 3.3, 3.4. Bayesa Network Represetato Lecture 2 Mar 30, 20 CSE 55, Statstcal Methods, Sprg 20 Istructor: Su-I Lee Uversty of Washgto, Seattle Last tme & today Last tme Probablty theory
More informationFractal-Structured Karatsuba`s Algorithm for Binary Field Multiplication: FK
Fractal-Structured Karatsuba`s Algorthm for Bary Feld Multplcato: FK *The authors are worg at the Isttute of Mathematcs The Academy of Sceces of DPR Korea. **Address : U Jog dstrct Kwahadog Number Pyogyag
More informationTI-83, TI-83 Plus or TI-84 for Non-Business Statistics
TI-83, TI-83 Plu or TI-84 for No-Buie Statitic Chapter 3 Eterig Data Pre [STAT] the firt optio i already highlighted (:Edit) o you ca either pre [ENTER] or. Make ure the curor i i the lit, ot o the lit
More informationIDENTIFICATION OF THE DYNAMICS OF THE GOOGLE S RANKING ALGORITHM. A. Khaki Sedigh, Mehdi Roudaki
IDENIFICAION OF HE DYNAMICS OF HE GOOGLE S RANKING ALGORIHM A. Khak Sedgh, Mehd Roudak Cotrol Dvso, Departmet of Electrcal Egeerg, K.N.oos Uversty of echology P. O. Box: 16315-1355, ehra, Ira sedgh@eetd.ktu.ac.r,
More informationADAPTATION OF SHAPIRO-WILK TEST TO THE CASE OF KNOWN MEAN
Colloquum Bometrcum 4 ADAPTATION OF SHAPIRO-WILK TEST TO THE CASE OF KNOWN MEAN Zofa Hausz, Joaa Tarasńska Departmet of Appled Mathematcs ad Computer Scece Uversty of Lfe Sceces Lubl Akademcka 3, -95 Lubl
More informationWe present a new approach to pricing American-style derivatives that is applicable to any Markovian setting
MANAGEMENT SCIENCE Vol. 52, No., Jauary 26, pp. 95 ss 25-99 ess 526-55 6 52 95 forms do.287/msc.5.447 26 INFORMS Prcg Amerca-Style Dervatves wth Europea Call Optos Scott B. Laprse BAE Systems, Advaced
More informationChapter 3. AMORTIZATION OF LOAN. SINKING FUNDS R =
Chapter 3. AMORTIZATION OF LOAN. SINKING FUNDS Objectves of the Topc: Beg able to formalse ad solve practcal ad mathematcal problems, whch the subjects of loa amortsato ad maagemet of cumulatve fuds are
More informationRQM: A new rate-based active queue management algorithm
: A ew rate-based actve queue maagemet algorthm Jeff Edmods, Suprakash Datta, Patrck Dymod, Kashf Al Computer Scece ad Egeerg Departmet, York Uversty, Toroto, Caada Abstract I ths paper, we propose a ew
More informationOnline Tuning of Two Degrees of Freedom Fractional Order Control Loops
DOI:.7694/bajece.49 Ole Tug of Two Degree of Freedom Fractoal Order Cotrol Loop A. Ate, ad C. Yeroglu Abtract Th paper preet ole tug of Two Degree of Freedom cotrol loop wth fractoal order proportoaltegraldervatve
More informationA New Bayesian Network Method for Computing Bottom Event's Structural Importance Degree using Jointree
, pp.277-288 http://dx.do.org/10.14257/juesst.2015.8.1.25 A New Bayesa Network Method for Computg Bottom Evet's Structural Importace Degree usg Jotree Wag Yao ad Su Q School of Aeroautcs, Northwester Polytechcal
More informationGreen Master based on MapReduce Cluster
Gree Master based o MapReduce Cluster Mg-Zh Wu, Yu-Chag L, We-Tsog Lee, Yu-Su L, Fog-Hao Lu Dept of Electrcal Egeerg Tamkag Uversty, Tawa, ROC Dept of Electrcal Egeerg Tamkag Uversty, Tawa, ROC Dept of
More informationThe Gompertz-Makeham distribution. Fredrik Norström. Supervisor: Yuri Belyaev
The Gompertz-Makeham dstrbuto by Fredrk Norström Master s thess Mathematcal Statstcs, Umeå Uversty, 997 Supervsor: Yur Belyaev Abstract Ths work s about the Gompertz-Makeham dstrbuto. The dstrbuto has
More informationA Study of Unrelated Parallel-Machine Scheduling with Deteriorating Maintenance Activities to Minimize the Total Completion Time
Joural of Na Ka, Vol. 0, No., pp.5-9 (20) 5 A Study of Urelated Parallel-Mache Schedulg wth Deteroratg Mateace Actvtes to Mze the Total Copleto Te Suh-Jeq Yag, Ja-Yuar Guo, Hs-Tao Lee Departet of Idustral
More informationUsing Phase Swapping to Solve Load Phase Balancing by ADSCHNN in LV Distribution Network
Iteratoal Joural of Cotrol ad Automato Vol.7, No.7 (204), pp.-4 http://dx.do.org/0.4257/jca.204.7.7.0 Usg Phase Swappg to Solve Load Phase Balacg by ADSCHNN LV Dstrbuto Network Chu-guo Fe ad Ru Wag College
More informationThe Digital Signature Scheme MQQ-SIG
The Dgtal Sgature Scheme MQQ-SIG Itellectual Property Statemet ad Techcal Descrpto Frst publshed: 10 October 2010, Last update: 20 December 2010 Dalo Glgorosk 1 ad Rue Stesmo Ødegård 2 ad Rue Erled Jese
More informationECONOMIC CHOICE OF OPTIMUM FEEDER CABLE CONSIDERING RISK ANALYSIS. University of Brasilia (UnB) and The Brazilian Regulatory Agency (ANEEL), Brazil
ECONOMIC CHOICE OF OPTIMUM FEEDER CABE CONSIDERING RISK ANAYSIS I Camargo, F Fgueredo, M De Olvera Uversty of Brasla (UB) ad The Brazla Regulatory Agecy (ANEE), Brazl The choce of the approprate cable
More informationClassic Problems at a Glance using the TVM Solver
C H A P T E R 2 Classc Problems at a Glace usg the TVM Solver The table below llustrates the most commo types of classc face problems. The formulas are gve for each calculato. A bref troducto to usg the
More informationLoad Balancing Algorithm based Virtual Machine Dynamic Migration Scheme for Datacenter Application with Optical Networks
0 7th Iteratoal ICST Coferece o Commucatos ad Networkg Cha (CHINACOM) Load Balacg Algorthm based Vrtual Mache Dyamc Mgrato Scheme for Dataceter Applcato wth Optcal Networks Xyu Zhag, Yogl Zhao, X Su, Ruyg
More informationLecture 7. Norms and Condition Numbers
Lecture 7 Norms ad Codto Numbers To dscuss the errors umerca probems vovg vectors, t s usefu to empo orms. Vector Norm O a vector space V, a orm s a fucto from V to the set of o-egatve reas that obes three
More informationChapter Eight. f : R R
Chapter Eght f : R R 8. Itroducto We shall ow tur our atteto to the very mportat specal case of fuctos that are real, or scalar, valued. These are sometmes called scalar felds. I the very, but mportat,
More informationT = 1/freq, T = 2/freq, T = i/freq, T = n (number of cash flows = freq n) are :
Bullets bods Let s descrbe frst a fxed rate bod wthout amortzg a more geeral way : Let s ote : C the aual fxed rate t s a percetage N the otoal freq ( 2 4 ) the umber of coupo per year R the redempto of
More information1. The Time Value of Money
Corporate Face [00-0345]. The Tme Value of Moey. Compoudg ad Dscoutg Captalzato (compoudg, fdg future values) s a process of movg a value forward tme. It yelds the future value gve the relevat compoudg
More informationSecurity Analysis of RAPP: An RFID Authentication Protocol based on Permutation
Securty Aalyss of RAPP: A RFID Authetcato Protocol based o Permutato Wag Shao-hu,,, Ha Zhje,, Lu Sujua,, Che Da-we, {College of Computer, Najg Uversty of Posts ad Telecommucatos, Najg 004, Cha Jagsu Hgh
More informationCredibility Premium Calculation in Motor Third-Party Liability Insurance
Advaces Mathematcal ad Computatoal Methods Credblty remum Calculato Motor Thrd-arty Lablty Isurace BOHA LIA, JAA KUBAOVÁ epartmet of Mathematcs ad Quattatve Methods Uversty of ardubce Studetská 95, 53
More informationA particle swarm optimization to vehicle routing problem with fuzzy demands
A partcle swarm optmzato to vehcle routg problem wth fuzzy demads Yag Peg, Ye-me Qa A partcle swarm optmzato to vehcle routg problem wth fuzzy demads Yag Peg 1,Ye-me Qa 1 School of computer ad formato
More informationOn Error Detection with Block Codes
BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 9, No 3 Sofa 2009 O Error Detecto wth Block Codes Rostza Doduekova Chalmers Uversty of Techology ad the Uversty of Gotheburg,
More informationOptimal replacement and overhaul decisions with imperfect maintenance and warranty contracts
Optmal replacemet ad overhaul decsos wth mperfect mateace ad warraty cotracts R. Pascual Departmet of Mechacal Egeerg, Uversdad de Chle, Caslla 2777, Satago, Chle Phoe: +56-2-6784591 Fax:+56-2-689657 rpascual@g.uchle.cl
More informationNetwork dimensioning for elastic traffic based on flow-level QoS
Network dmesog for elastc traffc based o flow-level QoS 1(10) Network dmesog for elastc traffc based o flow-level QoS Pas Lassla ad Jorma Vrtamo Networkg Laboratory Helsk Uversty of Techology Itroducto
More informationSoftware Reliability Index Reasonable Allocation Based on UML
Sotware Relablty Idex Reasoable Allocato Based o UML esheg Hu, M.Zhao, Jaeg Yag, Guorog Ja Sotware Relablty Idex Reasoable Allocato Based o UML 1 esheg Hu, 2 M.Zhao, 3 Jaeg Yag, 4 Guorog Ja 1, Frst Author
More informationPass by Reference vs. Pass by Value
Pa by Reference v. Pa by Value Mot method are paed argument when they are called. An argument may be a contant or a varable. For example, n the expreon Math.qrt(33) the contant 33 paed to the qrt() method
More informationThe analysis of annuities relies on the formula for geometric sums: r k = rn+1 1 r 1. (2.1) k=0
Chapter 2 Autes ad loas A auty s a sequece of paymets wth fxed frequecy. The term auty orgally referred to aual paymets (hece the ame), but t s ow also used for paymets wth ay frequecy. Autes appear may
More informationHow To Value An Annuity
Future Value of a Auty After payg all your blls, you have $200 left each payday (at the ed of each moth) that you wll put to savgs order to save up a dow paymet for a house. If you vest ths moey at 5%
More informationFault Tree Analysis of Software Reliability Allocation
Fault Tree Aalyss of Software Relablty Allocato Jawe XIANG, Kokch FUTATSUGI School of Iformato Scece, Japa Advaced Isttute of Scece ad Techology - Asahda, Tatsuokuch, Ishkawa, 92-292 Japa ad Yaxag HE Computer
More informationOptimal Packetization Interval for VoIP Applications Over IEEE 802.16 Networks
Optmal Packetzato Iterval for VoIP Applcatos Over IEEE 802.16 Networks Sheha Perera Harsha Srsea Krzysztof Pawlkowsk Departmet of Electrcal & Computer Egeerg Uversty of Caterbury New Zealad sheha@elec.caterbury.ac.z
More informationMaximization of Data Gathering in Clustered Wireless Sensor Networks
Maxmzato of Data Gatherg Clustere Wreless Sesor Networks Taq Wag Stuet Member I We Hezelma Seor Member I a Alreza Seye Member I Abstract I ths paper we vestgate the maxmzato of the amout of gathere ata
More informationTI-89, TI-92 Plus or Voyage 200 for Non-Business Statistics
Chapter 3 TI-89, TI-9 Plu or Voyage 00 for No-Buie Statitic Eterig Data Pre [APPS], elect FlahApp the pre [ENTER]. Highlight Stat/Lit Editor the pre [ENTER]. Pre [ENTER] agai to elect the mai folder. (Note:
More informationMaintenance Scheduling of Distribution System with Optimal Economy and Reliability
Egeerg, 203, 5, 4-8 http://dx.do.org/0.4236/eg.203.59b003 Publshed Ole September 203 (http://www.scrp.org/joural/eg) Mateace Schedulg of Dstrbuto System wth Optmal Ecoomy ad Relablty Syua Hog, Hafeg L,
More informationConfidence Intervals for Linear Regression Slope
Chapter 856 Cofidece Iterval for Liear Regreio Slope Itroductio Thi routie calculate the ample ize eceary to achieve a pecified ditace from the lope to the cofidece limit at a tated cofidece level for
More informationCurve Fitting and Solution of Equation
UNIT V Curve Fttg ad Soluto of Equato 5. CURVE FITTING I ma braches of appled mathematcs ad egeerg sceces we come across epermets ad problems, whch volve two varables. For eample, t s kow that the speed
More informationThe Analysis of Development of Insurance Contract Premiums of General Liability Insurance in the Business Insurance Risk
The Aalyss of Developmet of Isurace Cotract Premums of Geeral Lablty Isurace the Busess Isurace Rsk the Frame of the Czech Isurace Market 1998 011 Scetfc Coferece Jue, 10. - 14. 013 Pavla Kubová Departmet
More informationSimple Linear Regression
Smple Lear Regresso Regresso equato a equato that descrbes the average relatoshp betwee a respose (depedet) ad a eplaator (depedet) varable. 6 8 Slope-tercept equato for a le m b (,6) slope. (,) 6 6 8
More informationOne way to organize workers that lies between traditional assembly lines, where workers are specialists,
MANUFACTURING & SERVICE OPERATIONS MANAGEMENT Vol. 7, No. 2, Sprg 2005, pp. 121 129 ss 1523-4614 ess 1526-5498 05 0702 0121 forms do 10.1287/msom.1040.0059 2005 INFORMS Usg Bucket Brgades to Mgrate from
More informationR. Zvan. P.A. Forsyth. paforsyth@yoho.uwaterloo.ca. K. Vetzal. kvetzal@watarts.uwaterloo.ca. University ofwaterloo. Waterloo, ON
Robust Numercal Methods for PDE Models of sa Optos by R. Zva Departmet of Computer Scece Tel: (59 888-4567 ext. 6 Fax: (59 885-8 rzvayoho.uwaterloo.ca P.. Forsyth Departmet of Computer Scece Tel: (59 888-4567
More informationCompressive Sensing over Strongly Connected Digraph and Its Application in Traffic Monitoring
Compressve Sesg over Strogly Coected Dgraph ad Its Applcato Traffc Motorg Xao Q, Yogca Wag, Yuexua Wag, Lwe Xu Isttute for Iterdscplary Iformato Sceces, Tsghua Uversty, Bejg, Cha {qxao3, kyo.c}@gmal.com,
More informationApproximation Algorithms for Scheduling with Rejection on Two Unrelated Parallel Machines
(ICS) Iteratoal oural of dvaced Comuter Scece ad lcatos Vol 6 No 05 romato lgorthms for Schedulg wth eecto o wo Urelated Parallel aches Feg Xahao Zhag Zega Ca College of Scece y Uversty y Shadog Cha 76005
More informationA particle Swarm Optimization-based Framework for Agile Software Effort Estimation
The Iteratoal Joural Of Egeerg Ad Scece (IJES) olume 3 Issue 6 Pages 30-36 204 ISSN (e): 239 83 ISSN (p): 239 805 A partcle Swarm Optmzato-based Framework for Agle Software Effort Estmato Maga I, & 2 Blamah
More informationReinsurance and the distribution of term insurance claims
Resurace ad the dstrbuto of term surace clams By Rchard Bruyel FIAA, FNZSA Preseted to the NZ Socety of Actuares Coferece Queestow - November 006 1 1 Itroducto Ths paper vestgates the effect of resurace
More informationOn k-connectivity and Minimum Vertex Degree in Random s-intersection Graphs
O k-coectivity ad Miimum Vertex Degree i Radom -Iterectio Graph Ju Zhao Oma Yağa Virgil Gligor CyLab ad Dept. of ECE Caregie Mello Uiverity Email: {juzhao, oyaga, virgil}@adrew.cmu.edu Abtract Radom -iterectio
More informationA technical guide to 2014 key stage 2 to key stage 4 value added measures
A technical guide to 2014 key tage 2 to key tage 4 value added meaure CONTENTS Introduction: PAGE NO. What i value added? 2 Change to value added methodology in 2014 4 Interpretation: Interpreting chool
More informationHow To Make A Supply Chain System Work
Iteratoal Joural of Iformato Techology ad Kowledge Maagemet July-December 200, Volume 2, No. 2, pp. 3-35 LATERAL TRANSHIPMENT-A TECHNIQUE FOR INVENTORY CONTROL IN MULTI RETAILER SUPPLY CHAIN SYSTEM Dharamvr
More informationCHAPTER 2. Time Value of Money 6-1
CHAPTER 2 Tme Value of Moey 6- Tme Value of Moey (TVM) Tme Les Future value & Preset value Rates of retur Autes & Perpetutes Ueve cash Flow Streams Amortzato 6-2 Tme les 0 2 3 % CF 0 CF CF 2 CF 3 Show
More informationA Parallel Transmission Remote Backup System
2012 2d Iteratoal Coferece o Idustral Techology ad Maagemet (ICITM 2012) IPCSIT vol 49 (2012) (2012) IACSIT Press, Sgapore DOI: 107763/IPCSIT2012V495 2 A Parallel Trasmsso Remote Backup System Che Yu College
More information0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5
Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.
More informationA COMPARATIVE STUDY BETWEEN POLYCLASS AND MULTICLASS LANGUAGE MODELS
A COMPARATIVE STUDY BETWEEN POLYCLASS AND MULTICLASS LANGUAGE MODELS I Ztou, K Smaïl, S Delge, F Bmbot To cte ths verso: I Ztou, K Smaïl, S Delge, F Bmbot. A COMPARATIVE STUDY BETWEEN POLY- CLASS AND MULTICLASS
More informationn. We know that the sum of squares of p independent standard normal variables has a chi square distribution with p degrees of freedom.
UMEÅ UNIVERSITET Matematsk-statstska sttutoe Multvarat dataaalys för tekologer MSTB0 PA TENTAMEN 004-0-9 LÖSNINGSFÖRSLAG TILL TENTAMEN I MATEMATISK STATISTIK Multvarat dataaalys för tekologer B, 5 poäg.
More informationGeometric Mean Maximization: Expected, Observed, and Simulated Performance
GM Mamzato Geometrc Mea Mamzato: Epected, Observed, ad Smulated Performace Rafael De Satago & Javer Estrada IESE Busess School 0/16 GM Mamzato Geometrc Mea Mamzato 1. Itroducto 2. Methodology 3. Evdece
More informationFINANCIAL MATHEMATICS 12 MARCH 2014
FINNCIL MTHEMTICS 12 MRCH 2014 I ths lesso we: Lesso Descrpto Make use of logarthms to calculate the value of, the tme perod, the equato P1 or P1. Solve problems volvg preset value ad future value autes.
More informationMDM 4U PRACTICE EXAMINATION
MDM 4U RCTICE EXMINTION Ths s a ractce eam. It does ot cover all the materal ths course ad should ot be the oly revew that you do rearato for your fal eam. Your eam may cota questos that do ot aear o ths
More informationOn Savings Accounts in Semimartingale Term Structure Models
O Savgs Accouts Semmartgale Term Structure Models Frak Döberle Mart Schwezer moeyshelf.com Techsche Uverstät Berl Bockehemer Ladstraße 55 Fachberech Mathematk, MA 7 4 D 6325 Frakfurt am Ma Straße des 17.
More informationof the relationship between time and the value of money.
TIME AND THE VALUE OF MONEY Most agrbusess maagers are famlar wth the terms compoudg, dscoutg, auty, ad captalzato. That s, most agrbusess maagers have a tutve uderstadg that each term mples some relatoshp
More informationModels for Selecting an ERP System with Intuitionistic Trapezoidal Fuzzy Information
JOURNAL OF SOFWARE, VOL 5, NO 3, MARCH 00 75 Models for Selectg a ERP System wth Itutostc rapezodal Fuzzy Iformato Guwu We, Ru L Departmet of Ecoomcs ad Maagemet, Chogqg Uversty of Arts ad Sceces, Yogchua,
More informationIncorporating demand shifters in the Almost Ideal demand system
Ecoomcs Letters 70 (2001) 73 78 www.elsever.com/ locate/ ecobase Icorporatg demad shfters the Almost Ideal demad system Jula M. Alsto, James A. Chalfat *, Ncholas E. Pggott a,1 1 a, b a Departmet of Agrcultural
More information