Zurich Rsarch Lab Challngs in Businss Analytics An industrial rsarch prspctiv 30 Octobr 2008 Elni Pratsini IBM Zurich Rsarch Laboratory Alxis Tsoukiàs LAMSADE-CNRS Univrsité Paris Dauphin Frd Robrts DIMACS Rutgrs Univrsity Copyright IBM Corporation 2005
60 Yars of IBM Rsarch 1952 San Jos California Almadn Establishd: 1986 Employs: 500 Th Sun Nvr Sts at IBM Rsarch 1945 1 st IBM Rsarch Lab in NY (Columbia Watson U) Establishd: 1961 Employs: 1750 Zürich Establishd: 1956 Employs: 300 Bijing Establishd: 1995 Employs: 90 Austin Establishd: 1995 Employs: 40 Haifa Establishd: 1972 Employs: 500 Dlhi Establishd: 1998 Employs: 60 Tokyo Establishd: 1982 Employs: 200 2 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Evolution of Rol Work on clint-spcific tchnology businss problms 1970's 1980's 1990's 2000's Corporat fundd rsarch agnda Tchnology transfr Cntrally fundd Collaborativ tam Shard agnda Effctivnss Joint programs Work on clint problms Crat businss advantag for clints Industry-focusd rsarch On Dmand Businss Rsarch Rsarch in th marktplac 3 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Ovrviw Businss Environmnt nd for analytics in a world of incrasing complxity Tchnical Challngs: Exampl from th Pharma Industry - Solution architctur - Data availability a plasur and a plagu quality uncrtainty missing - Prscriptiv modls dpnd on prdictiv modls Optimization undr uncrtainty Businss Challngs - Clint xtrnal projcts - Clint intrnal projcts - Acadmic projcts 4 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Businss Environmnt W liv in a highly constraind rsourc world Rapidly raching capacity for many rsourcs - natural infrastructur human Study: What do you s as th primary businss challngs currntly affcting your organization? - Improving oprational ffctivnss Analytics ar ssntial to addrss th constraind rsourcs challng. Th historical barrirs to th us of analytics ar rapidly disapparing nabling us to tackl complx high-valu problms. 5 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Analytics Landscap Stochastic Optimization Optimization How can w achiv th bst outcom including th ffcts of variability? How can w achiv th bst outcom? Prdictiv modling What will happn nxt? Analytics Comptitiv Advantag Simulation Forcasting Alrts Qury/drill down Ad hoc rporting What will happn if? What if ths trnds continu? What actions ar ndd? What xactly is th problm? How many how oftn whr? Rporting Standard Rporting What happnd? Dgr of Complxity Basd on: Compting on Analytics Davnport and Harris 2007 6 Challngs in Businss Analytics; An industrial rsarch prspctiv
Analytics Landscap Stochastic Optimization Optimization How can w achiv th bst outcom including th ffcts of variability? How can w achiv th bst outcom? Prscriptiv Prdictiv modling What will happn nxt? Comptitiv Advantag Simulation Forcasting Alrts Qury/drill down Ad hoc rporting What will happn if? What if ths trnds continu? What actions ar ndd? What xactly is th problm? How many how oftn whr? Prdictiv Dscriptiv Standard Rporting What happnd? Dgr of Complxity Basd on: Compting on Analytics Davnport and Harris 2007 7 Challngs in Businss Analytics; An industrial rsarch prspctiv
Challngs in a world of incrasing complxity Advancs in computr powr bring capabilitis (and xpctations) for solving largr problm sizs - Problm dcomposition not always obvious - Squntial optimization - Exact vs approximat solution or mixd - Hybrid mthods Information Explosion - Thr is a lot of data (too much!) but how about data quality? Not nough of th right data Missing valus Outdatd information - Data in multipl formats Can our optimization modls handl th information? 8 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
9 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Whr do w start? 10 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Ovrviw Businss Environmnt nd for analytics in a world of incrasing complxity Tchnical Challngs: Exampl from th Pharma Industry - Solution architctur - Data availability a plasur and a plagu quality uncrtainty missing - Prscriptiv modls dpnd on prdictiv modls Optimization undr uncrtainty Businss Challngs - Clint xtrnal projcts - Clint intrnal projcts - Acadmic projcts 11 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Exampl: Risk Basd Approach to Manufacturing FDA s GMPs for th 21st Cntury A Risk-Basd Approach 300000 Post-Markting Advrs Evnt Rports 247604 278143 266991 286755 191865 212843 Numbr 150000 156477 *Horowitz D.J. Risk-basd approachs for GMP programs PQRI Risk managmnt Workshop Fbruary 2005 0 1995 1996 1997 1998 1999 2000 2001 Calndar yar - Manag risk to patint safty Critical to Quality - Apply scinc in dvlopmnt and manufacturing - Adopt systms thinking build quality in 12 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
j j j j j j j Risk Managmnt and Optimization industry data clint Rv p X p a p Risk pj pj pj C pj X pj CT X pj 1 p Y jk 1 p j k Y jk = X pj j k X pj CAPj p j = Haz pjk Y jk X pj p j k pj T pj Rv p Risk pj k CR jk p j X pj T pj X pj X ipj 1 + T pj p j T pj 1 p T pj 1 X pj 1 p j Y jk Y jk 1 + R jk j k Y jk Y jk 1 Y jk R jk j k j k R jk 0 2000 4000 6000 8000 10000 12000 14000 16000 18000clint xprts industry / xprts Risk 0 X Y T R = 0 /1 Rvnu at Risk as a prcntag of Rvnu CAPE VIAL 3 11% CAPE VIAL 2 10.6% CAPE VIAL 1 5.3% CAPE BOT 3 1% CAPE BOT 2 1% CAPE BOT 1 1.3% CAPE PTCH 1 6.4% CAPE PCH 1 9.7% CAPE CRD 1 7% CAPE BLST 4 1.6% Phas 1: Idntify Risk Sourcs Phas 2: Quantify Risk Valus Phas 3: Optimal Transformation Using company s knowldg industry xprtis and industry guidlins & rgulations w idntify th sourcs of risk that nd to b considrd in th analysis. Basd on historical data industry xprtis and othr sourcs of causal rlationships w st th causal link strngths (conditional probabilitis) and dvlop a modl for risk quantification Givn th risk valus financial and physical considrations w dtrmin th bst squnc of corrctiv actions for minimizing xposur to risk and optimizing a givn prformanc mtric. 13 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008 13
Dfinition of Risk Fitnss for Us - Quality: product conforms to spcifications - Efficacy: nsurs product has a positiv ffct (drugs trats a patint) - Safty: nd product is not harmful (patint dos not suffr from fatal sid ffcts) Complianc: failur to comply with rgulations (quality fficacy and safty not fully undr control).g. lack of documntation 14 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Risk: Thr sourcs of hazards Products: Traditional focus; Basd on portfolio analysis - product is unstabl or contaminatd Tchnologis: som tchnologis may not b appropriat givn risk thy crat. - a nw blndr dos not work wll and th mixing is not don proprly Systms: basis for rgulatory inspction. Crat transfr risks btwn product profil classs. Systms capabilitis & tchnologis. - a biotch product is dvlopd and thr is no control of th spcification of its componnts 15 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Ovrviw of Approach Data Collction Envisioning supportd by R-factoring modl Implmntation Stratgy Establish Risk Profil Products Systms Tchnology Rviw Cost bas of Sits Collct ky data Assss radinss for chang 1 2 3 Input 4 Input Dfin Scnarios Product/Asst Configuration Input Rfactoring Modl Rout map Calculat Rvnu profil Output 5 Adapt scnarios Optimis rout-map for implmntation basd on Impact on: Rvnu @ Risk profil Invstmnt Stratgy Impact of Chang 6 Tst scnarios: Prioritis on impact on Rvnu Risk and Profitability Rviw rout maps slct final scnario options Slct scnario Dfin implmntation stratgy 16 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008 16
Ovrviw of Approach Data Collction Establish Risk Profil Products Systms Tchnology Rviw Cost bas of Sits Collct ky data Assss radinss for chang 1 2 3 Input 4 Input Dfin Scnarios Product/Asst Configuration Input Rfactoring Modl Rout map Calculat Rvnu profil optimization modl Output 5 Adapt modl 6 Tst solution: Prioritis on impact on Rvnu Risk and Profitability Implmntation Stratgy Rviw rout maps slct final solution options Dfin implmntation stratgy 17 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008 17
Effct of uncontrolld data collction A pharmacutical firm is assssing th risk of two of thir sits: Country A and Country X. Enginrs at both sits ar askd to collct non-compliant and dfctiv batchs in th last two yars. Analysis of th data indicats that sit A has 50% mor dfctiv / non-compliant batchs than sit X. Thy dduc that thy should mov thir manufacturing procsss to X or drastically chang thir oprations at A. Just bfor launching an audit of thir sit A thy rciv a lttr from th FDA mntioning many rcalls of batchs all coming from sit X. Furthr invstigation indicats that sit X dos not rport th many dfcts thy notic vry wk and thy ar indd mor risky. Lack of obsrvations dos not man no vnts; it mans no rporting of vnts Extract knowldg from non statistical sourcs.g. xprts 18 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Tchnology product and systm failur ar all linkd via common root causs. Th risk modl starts from building a dpndncy graph of all ths root causs. Caus Effct Analysis industry Exprtis Clint Constraints Past Exprincs Diffrnt sub-modls cratd: Dosag form variability Employ xcution Procss control Tchnology lvl 19 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Employ Ability to Excut Sub-modl Dirctly linkd to GMPs. Exampl vnts: rporting not don proprly quality control protocol not implmntd tc. 20 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Elicitation of Exprt Opinion - Challngs Ntwork paramtrs can b stimatd from data and manual ntry of probabilitis. Using xprt knowldg in Baysian ntworks can b an intractabl task du to th larg numbr of probabilitis that nd to b licitd W nd an licitation mthod that is fast and givs not only a probability but also a confidnc in th probability an aggrgation mthod for combining th licitatd probabilitis taking into account th xprts confidnc in thir assssmnts A probability stimation mthod to rduc th numbr of probabilitis to licit in a noisy MAX nod 21 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Elicitation / Aggrgation / Estimation 22 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Basd on location tchnologis or any prviously idntifid componnts involvd in th drug production procss th ntwork modl computs an stimat of th quality risk. Changing th way a drug is producd will (a priori) induc a chang in th quality risk stimat. Sit A Quality failur: 2.4% Sit B Optimization modl Quality failur: 1.9% 23 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008 23
Risk Exposur Prformanc masur for xposur to risk: (Insuranc prmium) Rvnu@risk p = Rvnup Risk pj j Risk valu: Risk pj = k Haz pjk Y jk X pj 24 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Formulation MAX ST p p a Risk Rv pj pj j k k X p j X pj j C pj X pj X 1 p pj Y 1 p j Y jk pj k CAP = pjk j jk X pj p jk pj j = Haz Y X p j Capacity Risk Stat Lvl j j CT pj T pj Rv p j pj T pj Risk pj j k CR jk X p j X X 1 + T p j pj T j pj ipj pj T 1 p jk pj 1 X pj 1 jk jk p j Y Y 1 + R j k Y jk Y jk 1 Transfr Y jk R jk Rmdiation j k j k R jk Risk 0 X Y T R = 0 /1 25 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Problm charactristics 17 product familis 3 systms plus on for subcontracting 14 tchnology platforms most availabl in all systms Planning horizon: 5 yars split into 10 priods of 6 months Rstriction on rat of chang Statistical analysis gav risk valus basd on historical data Rvnu projctions availabl Aggrgatd costs pr scnario latr disaggrgatd Simulation of givn scnarios (nding rsult); optimization 26 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Exampl Analysis Optimal squnc of actions for minimizing risk xposur Work cntr Currnt 2005 2006 2007 2008 2009 Scnario X Tch 1 Tch 2 Tch 3 Tch 4 Sit A Sit B Sit B Tch 5 Tch 6 Sit C Sit A Tch 7 Tch 8 Sit A / Sit Sit B Tch 9 C Tch 10 Tch 11 Tch 12 Sit A Sit C Tch 13 Sit C Tch 14 Sit D Scnario Comparison Rvnu@risk distributions Valus in 10^ -4 4.500 4.000 3.500 3.000 2.500 2.000 1.500 1.000 0.500 0.000 Calculat Businss Exposur Man=7208 3 5 7 9 11 Valus in Thousands 5% 90% 5% 5.7013 8.625 E(xposur) = rv i pi i 2 2 Var( xposur ) = rv i var (p i ) = rv i p i( 1 i i p i ) Dnsity Scnario 1 Scnario 2 As Is Fully Opt 0 5000 10000 15000 20000 Rvnu@risk 27 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
28 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Data Variability Th considration of th frquncy distribution of ach solution is ssntial whn thr is uncrtainty in th input paramtrs λ = 4300 Which Solution / Plan is bttr? ara = 5% frquncy distributi λ = 3900 6200 ara = 20% Plan A Plan B 0 2000 4000 6000 8000 10000 12000 14000 16000 Rv@Risk Robust Optimization tchniqus to dtrmin th solution that is optimal undr most ralizations of th uncrtain paramtrs 29 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Possibl Approachs Stochastic Programming - Avrag Valu crtainty quivalnt - Chanc constraints (known CDF continuous) Robust Optimization - Soystr (1973) - Brtsimas & Sim (2004) CVaR - Rockafllr & Uryasv (2000) 30 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Modling with th approach of Brtsimas and Sim Formulation adjustd to incorporat a constraint that could b violatd with a crtain probability: max s.t. x X E x ( ScurdPro fit(x) ) Rv@Risk(x t s x p t s p t s 1 {01} p p t s Uncrtain paramtrs hav unknown but symmtric distributions Uncrtain paramtrs tak valus in boundd intrvals: ) critical valu Haz ts [ Hˆ - H ; Hˆ + ] t s ts ts H ts Uncrtain paramtrs ar indpndnt 31 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Brtsimas & Sim cont d Providd guarant: Pr ( Rv@Risk( x ) > critical valu) γ γ = 1 2 n ( 1 μ) n + ν n l= ν +1 n l n = total numbr of uncrtain paramtrs ν = Γ+n μ = ν ν 2 Exampl Valus of Γ: 32 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
33 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008 Application to pharma xampl Robust Formulation: ( ) 0 0 0 {01} 1 ˆ max + Γ Z s t Y s t V s t p X p X s t Y X Y s t Y V Z V Z X s t s t s t p s t s t p s t p s t p s t s t s t ts pts H Rv valu critical Rv s.t. ts p ts p ts p s t + + H fit ScurdPro ) ( E x X x
Rsults! 25 20 Actual Probability of Constraint Violation 15 10 5 Idal Prformd 0 0 5 10 15 20 25 Thortical Bound 34 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Obsrvations Provids solutions that ar much mor consrvativ than xpctd Pr( Rv@Risk( x ) > critical valu) << γ Plans satisfying risk rquirmnt ar discardd by th optimization - suboptimal solutions Adjustmnts to calculation of paramtrs to improv rsults - Extrm distribution - Considr th numbr of xpctd basic variabls and not th total numbr of variabls affctd by uncrtain cofficints.g. production of 100 products 3 possibl sits uncrtainty in capacity absorption cofficint: - 300 possibl variabls (valu of n) - ach product can only b producd on on sit i.. only 100 variabls > 0 (us this valu for th calculation of Γ) 35 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Brtsimas & Sim approach Production Distribution Pharma Rgulatory Risk Actual Probability of Constraint Violation 25 20 15 10 continuous variabls Idal Original sttings: production not rstrictd to a singl sit Production on a singl sit: Γ computd with th numbr of activ uncrtain paramtrs Production on a singl sit: Γ computd with th total numbr of uncrtain paramtrs Actual Probability of Constraint Violation 25 20 15 10 binary variabls Idal Γ computd with th numbr of activ uncrtain paramtrs Γ computd with th total numbr of uncrtain paramtrs 5 5 0 0 5 10 15 20 25 Thortical Bound 0 0 5 10 15 20 25 Thortical Bound 36 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Rlatd work Robust optimization Modls for Ntwork-basd Rsourc Allocation Problms Prof. Cynthia Barnhart MIT - Robust Aircraft Maintnanc Routing (RAMR) minimizing xpctd propagatd dlay to gt schdul back on track (takn from Ian Clark and Barnhart 2005) Among multipl optimal solutions no incntiv to pick th solution with highst slack Rlationship of Γ (pr constraint) to ovrall solution robustnss? - Comparison with Chanc constraind Programming Explicitly diffrntiats solutions with diffrnt lvls of slack Extndd Chanc Constraind Programming - Introduc a budgt constraint stting an uppr bound on accptabl dlay - Improvd rsults 37 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Valu at Risk & Conditional Valu at Risk VaR Maximum loss with a spcifid confidnc lvl - Non sub-additiv Dnsity of f (x) β 1-β - Non-convx β-quantil CVaR β-quantil α β (x) (β=95%) E [ f (x) f (x) > α β (x)] Dnsity of f (x) CVaR(x) Advantags - Sub-additiv α β (x) - Convx 38 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
CVaR in Math Programming: Linar Program with Random Cofficints Considr a LP whr cofficint vctor ξ is random: Min T c x T ξ x b Dx = d x 0 CVaR can b usd to nsur 'robustnss' in th stochastic constraint: Min T c x T CVaR α ( ξ x) b Dx = d x 0 39 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Rformulation of Rockafllar & Uryasv (2000) Rformulation Each CVaR xprssion say a constraint: ξ CVaR ( ξ x) is rformulatd as: i th α 1 T + t + ( ξi x-t ) b N(1 α) i= 1 : i sampl of random vctor ξ (out of N total sampls) T b N Linar Rformulation 1 t + zi b N(1 α) z z i i 0 T i ξ x-t N i= 1 Good nws: Fully linar rformulation so prsntabl to any LP solvr Bad nws: Nw variabls and Nw constraints = O (Numbr of Sampls) - Rformulatd LP has grown in both dimnsions! - Larg numbr of CVaR constraints & larg numbr of sampls => Vry Larg LP 40 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Obsrvations Numbr of scnarios ncssary for a good stimation of CVaR(α x) and thus maningful rsults? >20000 scnarios ncssary for stabl rsults (Kunzi Bay A. and J. Mayr. Computational aspcts of minimizing conditional valu at risk NCCR FINRISK Working Papr No. 211 2005). Can handl up to 300 scnarios computational complxity 41 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Rcnt advancs in CVaR Kunzi-Bay & Mayr (2006) prsntd a 2-stag intrprtation if CVaR appars only in th Objctiv Function & thir algorithm CVaRMin ld to an ordr of magnitud spd-up P. Huang and D. Subramanian. Itrativ Estimation Maximization for Stochastic Programs with Conditional-Valu-at-Risk Constraints. IBM Tchnical Rport RC24535 2008 -Thy xploit a diffrnt linar rformulation motivatd by Ordr Statistics and this lads to a nw and fficint algorithm. -Th rsulting algorithm addrsss CVaR in both th Objctiv Function as wll as Constraints Intrsting work on Robust Optimization tchniqus & CVaR David B. Brown Duk Univrsity: - Risk and Robust Optimization (prsntation) - Constructing uncrtainty sts for robust linar optimization (with Brtsimas) - Thory and Applications of Robust Optimization (with Brtsimas and Caramanis) 42 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Ovrviw Businss Environmnt nd for analytics in a world of incrasing complxity Tchnical Challngs: Exampl from th Pharma Industry - Solution architctur - Data availability a plasur and a plagu quality uncrtainty missing - Prscriptiv modls dpnd on prdictiv modls Optimization undr uncrtainty Businss Challngs - Clint xtrnal projcts - Clint intrnal projcts - Acadmic projcts 43 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Clint xtrnal projcts Vry intrsting and challnging problms W might spak English / Grman / Frnch / but w do not spak thir languag! - Hav you usd simulation to prov your LP givs th optimal solution? Important to hav industry knowldg Oftn rly on consultants to sll our work - Ovrsll / undrsll Tim horizon - Thy want th optimal solution now Prfr maturd tchnology - Thy do not want to b th first to try a nw tchniqu 44 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
All thy nd is a simpl solution Th fastst travling salsman solution *http://xkcd.com/ 45 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Thy don t want to collct th data You gt what you pay for th amount and typ of information usd in th analysis affcts th output* *http://www.mnylon.com/blog/2008/04/ 46 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Clint intrnal projcts Intrsting ral lif application and clint agr to us advancd analytics oftn dal with vry tchnical businss partnrs so w undrstand ach othr Limitd in scop - oftn considr a small aspct of th whol problm Mdium tim horizon - Usually 1 yar but oftn continus for 2 or mor yars Intrnal politics! 47 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
Acadmic projcts Lif is good!! Acadmic collaborations: - Oftn killd by th lawyrs bcaus of IP (dis)agrmnt It dos not pay th bills!! 48 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008
49 Challngs in Businss Analytics; An industrial rsarch prspctiv 30 Octobr 2008