Lean%Analytics (a#sneak#peek)# by#alistair#croll#and#ben#yoskovitz# # #
1.#A#Quick#Preamble# WelcometothefirstsneakpeekofLean#Analytics.We veincludedafewofthe chaptersforyoutoread,providefeedbackonand hopefully enjoythinkofthis asourminimum#viable#product.ourblogisabigpartofourmvpaswell;it swhere weshareideas,testthingsandpublishcontentthatwe reeditingoutofthebook. Hopefullyyou reenjoyingit. Notsurprisingly,writingLean#Analyticsisverymuchlikestartingacompany.And we refollowingaleanstartupapproachinourefforts:wehavehypothesesaround whatwethinkpeoplewilllike,wewritesomecontent,wetestit(throughtheblog, givingpresentationsandsoon),measuretheresults(mostlyqualitatively,butthere aresomequantitativemetricstoo)andadjustaswelearn. Finally,we dliketopointoutthatwe reshamelesspromoters.we regivingyouthis lookatourworknotonlyforyourfeedback,butalsosoyoucanspreadtheword. Throughout#the#book,#we ve#flagged#some#of#the#core#points#we re#trying#to# make like#this.# Ifyoulikethem,feelfreetotweetthemtotheworld(andus) it llshowuswhich partsofthebookpeopleareresonatingwith,andhopefullyhelpustoreachaneven wideraudience.youcanclickonthetwittericonnexttoeachquote. WritingLeanAnalyticshasbeenagreatexperiencesofar,andwe vestillgotalong waystogo.sositback,enjoy,takenotesandsendusyourfeedback
2.#Introduction# Startupsuseanalyticstofindtheirwaytotherightproductandmarketbeforethe moneyrunsout.learninganditeratingisatthecoreoftheleanstartup methodology,butwithafloodofinformationavailable,it shardtoknowwhereto begin.ultimately,youdon twanttospendyourlifeandyourmoneybuilding somethingnobodywants. Theproblem,ofcourse,isthatit shardtofigureoutwhatpeoplewant.askingthem isagoodstart,butoftentheydon tknow andevenwhentheydo,theylie.what s worse,asafounderandentrepreneuryouhavestrong,almostoverwhelming preconceptionsabouthowotherpeoplethink,andthesecoloryourdecisionsin subtleandinsidiousways. TheLeanStartupmovementgalvanizedagenerationofentrepreneurs.Mostofits insightsboildownintoonesimplesentence: Don t#sell#what#you#can#make;#make#what#you#can#sell. Thedevil,ofcourse,isinthedetailsoffiguringoutwhatyoucansell andhowto turnaprofitdoingso.that swhatwewanttotackle. InLean#Analytics,wehopetoshowyouhowtofindtheonemetricthatmattersto yourbusinessforitscurrentstageofgrowth fromidentifyinganeed,tovalidating aparticularsolution,totestingnewfeatures,toimprovingrevenues.wealsowant tohelpyoutailoryourmetricstodifferentaudiences,whetheryou reenticing investors,promotingtobloggers,orreportingprogresstoadvisors.wewanttogive youanideaofwhat normal is,soyouknowifyou reaheadorbehindtheindustry onaparticularaspectofyourstartup.finally,wewanttolayoutpractical,proven stepstotakeyourstartupfrominitialideatoproduct/marketfit. We regoingtomakesomefairlyboldstatementsinthisbook.wethinkyoushould pickonlyonemetrictofocuson;thatyoushoulddrawalineinthesand;andthat youshouldbalancegutinstinctwithhard,impartialdata.thesearebitterpillsto swallow.we vehadfounderstellusdatawdrivenmindsetsdon tapplytocertain businesses,suchasgaming;orthatthey vealreadyfoundtheirproduct/marketfit anddon tneedtooptimize.that sok it swhywewrotethebook. Who#this#book#is#for# Inadigital,alwaysWonworld,everythingcanbemeasured.Anythingthat sreachable byamobiledevicecanbecollectedandquantified.hashtags,qrcodes,loyalty
cards,shorturls,nearwfieldcommunications,andsocialnetworkactivityareall gristfortheanalyticalmill,andallofthemcanpointthewaytotheidealproduct andgowtowmarketstrategyforyourbusiness.today,youcaninstrumenteveryfacet ofyourcompanytobetterunderstandyourcustomersandmarket. Recently,theWebAnalyticsAssociationrebrandeditselftheDigitalAnalytics Associationinrecognitionofthefarwiderimplicationsoftrackingandmeasuring theworldaroundus.eventraditionalbusinesseslikerestaurantsareembracinga lean,learnwfirstapproach:sanfrancisco swisesonsdeliusedatemporary popwup approachtooptimizetheirmenuandoperationsbeforelaunchingapermanent restaurant.thefoodtruckjustmightbetheleanstartupappliedtotherestaurant. ManymodernstartupshaveasignificantwebWbasedcomponent,whichmeans there salotanybusinesscanlearnfromwebtraffic.butwhetheryou reanentirely webwbasedbusiness,oralocalstorewithonlyafoursquarepresenceandaloyalty program,yourgoalremainsthesame:identifyandmeasurethesinglemost importantmetricforthestageofyourcompany,andtheniterateuntilyougetit right. Whilethisbookisprimarilyabouttechnologystartups,itslessonscanbeappliedfar beyondtheweb.manyofthetoolsandtechniqueswe llcoverwerefirstappliedto consumerwebapplications.today,however,theymattertoafarbroadersetof startups independentlocalbusinesses,businesswtowbusinessstartups,roguecivil servantstryingtochangethesystemfromwithin,and intrapreneurs 1 innovating withinbig,establishedorganizations. Inthatrespect,Lean#Analyticsisforanyonetryingtomaketheirorganizationmore effective fromtinystartupstoglobalcorporations,restaurantstosportsteams, communitiesandchurchgroupstointernationalcharities. Theprimaryaudience,however,istheleanentrepreneurorintrapreneurfocused ontheearlystagesofbuildingsomethinginnovative.we llwalkreadersthroughthe analyticalprocess,fromideagenerationtonearingorachievingproductwmarketfit, sothecontentisasapplicabletothosestartingtheirentrepreneurialjourneyasitis tothoseinthemiddleofit. Webanalystsandotherdatascientistsmayalsofindthisbookuseful,becauseitwill showthemhowtomovebeyondtraditional funnelvisualizations andhowto elevatethethingstheyworkonintomoremeaningfulbusinessdiscussions. Similarly,businessprofessionalsinvolvedinproductdevelopment,product 1 Anentrepreneurwithinalargeorganization,oftenfightingpoliticalratherthanfinancial battlesandtryingtoprovokechangefromwithin.
management,marketing,publicrelationsandinvestingwillfindmuchofthecontent relevant,asitwillhelpthemunderstandandassessstartups. The#building#blocks# LeanAnalyticsdoesn texistinavacuum.we reanextensionofleanstartup, heavilyinfluencedbycustomerdevelopmentandotherconceptsthathavecome before.it simportanttounderstandthosebuildingblocksbeforedivingin. Customer#Development# Customer#Development atermandprocessconceivedbyentrepreneurand professorsteveblank tookdirectaimatthewaterfallmethodforbuilding productsandcompanies.customerdevelopmentisfocusedoncollecting continuousfeedbackthathasamaterialimpactonthedirectionofaproductand business,everystepoftheway.stevefirstdefinedcustomerdevelopmentinhis book,the#four#steps#to#the#epiphany. Hecontinuestoevolvehisconceptsaroundbuildingandrunningstartups,and recentlypublishedanewbook,the#startup#owner s#manual. OneofthemostimportantconceptsinSteve sworkishisdefinitionofastartup, whichisextremelyrelevantthroughoutlean#analytics#:# Lean#Startup# A#startup#is#an#organization#formed#to#search#for#a#scalable#and#repeatable# business#model. # FromthefoundationofCustomerDevelopmentcomesLeanStartup,aprocess definedanddevelopedbyericries.itcombinescustomerdevelopment,agile SoftwareDevelopmentmethodologies,andLeanManufacturingpracticesintoa frameworkfordevelopingproductsandbusinessesquicklyandefficiently. ErichasdoneincredibleworkbringingLeanStartupnotonlytostartupsbutinto thecorporateworldaswell.afterall,leanisn taboutbeingcheaporsmall,it s abouteliminatingwasteandmoving quickly,whichorganizationsofanysizecan benefitfrom. OneofLeanStartup scoreconceptsisbuild# #Measure# #Learn theprocessbywhich aleanstartupdoeseverything,from establishingavisiontobuildingproduct features.thespeedatwhichan
organizationcaniteratethroughthecycleiskey. Everycompanystartswithanidea arealizationthatsomethingcouldbedone better,differently,morecompetitively.leanisaboutsubjectingthatideatorigorous analysisanddevelopmentaccordingtocarefullymeasuredsteps.akeytenetof LeanStartupistoidentifytheriskiestpartsofthebusinessataparticularstage,and iterateuntiltheriskisovercome.withinthatcycle,#lean#analyticsisfocused primarilyonthemeasure#stage,whereweusedataasanessentialtoolformaking progress. Mostofthetime,theriskisn twhetheryoucanbuildsomething,butrather,whether anyonewillcare.thereallyimportantquestionstoanswerarewhetheryou ve foundaproblempeoplecareabouthavingsolved,andwhetheryoucansolveit remarkablybetterthanothers.leanmakessureyoudon tputthe product cart beforethe need horse. We renotgoingtodoacompletereviewoftheleanstartup.whatwewilldo, however,isconsideranalyticsinthefourmainstagesanystartupwillgothrough,so youhaveastepwbywstepguidetobuildingyourstartupanalytically,knowingwhich metricstocareabout,howtocollectthem,andhowtointerpretandsharethem. Thefourstagesyou llgothroughare: Validate(a(problem:(findarealissuepeopleareawareofandneedsolved. Validate(a(solution:decideifyouhaveawayofsolvingtheproblemthatwill workforthem,andthattheycanconsumefromyou. Validate(a(market/revenue:figureouthowyou lldeliverthesolutionto theirproblemreliablyandrepeatedly,andwhatthey llgiveyouinreturn, andifthereareenoughpeoplewiththeproblemtosustainyou. Validate(the(channels(&(ecosystem:understandwhereyoustandinyour industry,whetheryou reastandaloneofferingorapartofsomethingbigger, who scompetingwithyou,andwhowillhelpyousellmorethingstomore peopleformoremoneymoreeffectively. 2 Keepthisinmindasyou rereadingthebooksoyoucanhoneinonthoseareasthat areofmostinteresttoyouandthestageyou reat. About#the#authors# AlistairCrollhasbeenanentrepreneur,author,andpublicspeakerfornearly20 years.he sworkedonavarietyoftopics,fromwebperformance,tobigdata,to cloudcomputing,tostartups,inthattime.in2001,hecowfoundedwebperformance 2 ThisishowSergioZyman,formerlytheCMOofCocaWcola,definesmarketing.
startupcoradiant,andsincethattimehasalsolaunchedrednod,cloudops, Bitcurrent,YearOneLabs,theBitnorthconference,theInternationalStartup FestivalandseveralotherearlyWstagecompanies. AlistairisthechairofO Reilly sstrataconference,cloudconnect,andinterop s EnterpriseCloudSummit.Thisishisfourthbookonanalytics,technology,and entrepreneurship.alistairlivesinmontreal,canadaandtriestomitigatechronic ADDbywritingaboutfartoomanythingsatSolveForInteresting (http://www.solveforinteresting.com.) BenYoskovitzisaserialentrepreneurwith15+yearsexperienceinwebbusinesses. Hestartedhisfirstcompanyin1996whilecompletinguniversity.In2007hecoW foundedstandoutjobs,ab2bsoftwarecompanyintherecruitmentspace.the companyraised$1.8mfromventureandangelinvestorsandexitedin2010.most recently,asvpofproductatgoinstant,heledthecompany sgrowthinthecow browsingspaceuntilitsacquisitionin2012bysalesforce.com,andcontinuesonin thatrole. Benhasbeenbloggingsince2006.The InstigatorBlog (http://instigatorblog.com) isrecognizedasoneofthetopblogsonstartupsandentrepreneurship.benisalso anactivementortonumerousstartupsandotheracceleratorprograms,including FounderFuel(aTechStarsWlikeprograminMontreal.)Heregularlyspeaksatstartup conferencesandevents,includingthemichiganleanstartupconference,internet MarketingConference,andtheLeanStartupconference. In2010,AlistairandBenweretwoofthecoWfoundersofYearOneLabs,anearly stageacceleratorthatprovidedfundingandupto1wyearofhandswonmentorshipto 5startups.YearOneLabsfollowedaLeanStartupprogram,makingitthefirst acceleratortoformalizesuchastructure.threeoffivecompaniesgraduatedfrom YearOneLabsandwentontoraisefollowWonfinancing.Agreatdealoftheir experienceandthoughtleadershiparoundleanstartupandanalyticsemerged duringthistime.
3.#We re#all#liars# Let sfaceit:you realiar. Wealllie;somebetterthanothers.Andthebestliarsareentrepreneurs. Entrepreneursareparticularlygoodatlyingtothemselves.Lyingmayevenbea prerequisiteforsucceedingasanentrepreneur afterall,youneedtoconvince othersthatsomethingistrueintheabsenceofgood,hardevidence.youneed believerstotakealeapoffaithwithyou.asanentrepreneur,youneedtoliveina semiwdelusionalstatejusttosurvivetheinevitablerollercoasterrideofrunning yourstartup. Smallliesareessential.Theycreateyourrealitydistortionfield.Smallliesarea necessarypartofbeinganentrepreneur.butifyoustartbelievingyourownhype, youwon tsurvive.you llgotoofarintothebubbleyou vecreated,andyouwon t comeoutuntilyouhitthewall hard andthatbubblebursts. You#need#to#lie#to#yourself,#but#not#to#the#point#at#which#you re#jeopardizing# your#business. That swheredatacomesin. Yourdelusions,nomatterhowconvincing,willwitherundertheharshlightofdata. Analyticsisthenecessarycounterweighttolying,theyintotheyangofhyperbole. Moreover,dataWdrivenlearningisthecornerstoneofsuccessinstartups.It show youlearnwhat sworkinganditeratetowardstherightproductandmarketbefore themoneyrunsout. Don tbelieveus?well,that spreciselythepointofthisbook. We renotsuggestingthatgutinstinctisabadthing.instinctsareinspiration,and you llneedtolistentoyourgutandrelyonitthroughoutthestartupjourney.but don tdisembowelyourself.gutsmatter;you vejustgottotestthem.instincts(are( experiments.(data(is(proof.( The#Lean#Startup#movement# Innovationishardwork harderthanpeoplerealize.it struewhetheryou realone startuptryingtodisruptanindustry,orarogueemployeechallengingthestatus quo,tiltingatcorporatewindmillsandsteeringaroundbureaucraticroadblocks.we getit.entrepreneurshipiscrazy,borderingonabsurd.
LeanStartupprovidesameasureofordertothechaos,aframeworkbywhichyou canmorerigorouslygoaboutthebusinessofcreatingsomethingnew.byrunning throughacycleofbuildingsomething,measuringwhathappens,andlearningfrom thatexperiencetobuildsomethingbetter,you reabletoidentifyrealrisksearlyon. LeanStartuphasbroughtagenerationofentrepreneursaheavydoseofintellectual honesty.followtheleanmodel,anditbecomesincreasinglyhardtolie,especially toyourself. There sareasontheleanstartupmovementhastakenoffnow.we reinthemidst ofafundamentalshiftinhowcompaniesarebuilt.it svanishinglycheaptocreate thefirstversionofsomething.cloudsarefree.socialmediaisfree.competitive researchisfree.evenbillingandtransactionsarefree 3.We(live(in(a(digital(world,( and(the(bits(don t(cost(anything.( Thatmeansyoucanbuildsomething,measureitseffect,andlearnfromittobuild somethingbetterthenexttime.youcaniteratequickly,decidingearlyonifyou shoulddoubledownonyourideaorfoldandmoveontothenextone.andthat s whereanalyticscomein.learningdoesn thappenaccidentally.it sanintegralpart oftheleanprocess. PeterDruckerfamouslyobserved, Ifyoucan tmeasureit,youcan tmanageit. NowhereisthistruerthanintheLeanmodel,wheresuccessfulentrepreneursbuild theproduct,thegowtowmarketstrategy,andthesystemsbywhichtolearnwhat customerswant simultaneously. Poking#a#hole#in#your#reality#distortion#field# Mostentrepreneurshavebeencrushed atleastonce,maybetwice.ifyouhaven t beensolidlytrouncedonaregularbasis,you reprobablydoingitwrong,andaren t takingtherisksyouneedtoinordertosucceed. Butthere samomentonthestartuprollercoasterwherethewholethingderails, andit strulyfinished.there slittlemoretodothanturnoffthewebsiteandclose downthebankaccount.you reoverwhelmed,thechallengesaretoogreat,andit s over.you vefailed. Youknewthiswasgoingtohappen.Longbeforetheactualderailment,youknewit wasn tworking.ifyou recompletelyhonestwithyourself,youknowit strue.butat 3 Whenwesay free, wemean freefromsignificantupfrontinvestment. Plentyofcloudand billingservicescostmoney sometimesmoremoneythandoingityourself onceyourbusinessis underway.butfree,here,meansfreeofoutlayinadvanceoffindingyourproduct/marketfit.youcan usepaypal,orgooglewallet,oreventbrite,ordozensofotherpaymentandticketingsystems,and passthecostofthetransactionontoyourconsumers.
thetime,yourrealitydistortionfieldwasstrongenoughtokeepyougoingonfaith andfumesalone.thisprobablymeantyouhitthewallatamillionmilesanhour, lyingtoyourselfthewholetime. We renotarguingagainsttheimportanceoftherealitydistortionfield butwedo wanttopokeafewholesinit.hopefully,asaresult,you llseethederailmentintime toavoidit.wewantyoutonothavetorelyontherealitydistortionfieldquiteas much,andinsteadrelyonleananalytics. Forstarters,herearetwodefinitionstorememberthroughoutthisbook: Lean(Startuphelpsstructureyourprogressandhoneinonwhatyoushould befocusingonatanygiventime. Lean(Analyticsisusedtomeasurethatprogress,helpingyoutoaskthemost importantquestionandgetclearanswersquickly. LeanAnalyticsisthegluethroughouttheLeanStartupprocess(andbeyond)that givesyouthenecessarydecisionwmakingcapabilitiestosucceed.it sthedashboard foreverystageofyourbusiness,fromvalidatingwhetheraproblemisreal,to learningwhoyourcustomersare,todecidingwhattobuild,topositioningyourself favorablywithapotentialacquirer.leananalyticsdrivesinsights.itcan tforceyou toactonthem butit llputthemfrontandcenter,makingthemhardtoignoreand givingyouabetterchanceofavoidingderailment.
Case#study:#AirBnB#Photography growth#within#growth# AirBnBisanincrediblesuccessstory.Injustafewyears,thecompanyhas becomeapowerhouseinthetravelindustry,providingtravelerswithan alternativetohotels,andprovidingindividualswhohaverooms,apartmentsor homestorentoutwithanewsourceofincome.in2012,theybookedover5 millionnightswiththeirservice.buttheystartedsmall,andasadherentstothe LeanStartupmindset,tookaverymethodicalapproachtotheirsuccess. AtSXSW2012,JoeZadeh,ProductLeadatAirBnB,sharedpartofthecompany s amazingstory.hefocusedononeaspectoftheirbusiness:#professional# photography.# Itstartedwithahypothesis: Hostswithprofessionalphotographywillgetmore business.andhostswillsignupforitasaservice. Thisiswheretheirgutcame in:theyhadasensethatsomethingwouldhelptheirbusiness.butratherthan implementingitoutright,theybuiltaconciergeminimumviableproduct(mvp) toquicklytesttheirhypothesis. IntheirinitialtestsoftheirMVP,AirBnBfoundthatprofessionalphotographed listingsgottwo#to#three#timesmorebookingsthanthemarketaverage.this validatedtheirfirsthypothesis.anditturnedoutthathostswerewildly enthusiastictoreceiveanofferfromairbnbtotakethosephotographsforthem. AirBnBthenwentontofurtherexperimentanditerate.Theywatermarked photostoaddauthenticitytothem.theygotcustomerservicetooffer professionalphotographyasaservicewhensomeonecalledin.theyincreased therequirementsonphotoquality.eachstepoftheway,theymeasuredthe resultsandadjustedasnecessary. Bridging#the#two#worlds# LeanAnalyticsishowwebridgethegapbetweenLeanStartupandAnalytics.We re bigbelieversintheeffectivenessofboth: Leanprovidesafantastictemplateforbuildingbusinesses.Itfocuseson methodicallybutquicklylearninganditerating,findingwhatwillwork(andwhat won t),mitigatingriskandminimizingeffortspentinthewrongplaces.we veseen LeanStartupworkinourowncompaniesandinothercompanieswe vementored
andadvised.leanstartupfundamentallychangesthewayyouthink.it sbotha prescriptiveprocessforbuildingyourstartup,andamindset. LikeLean,Analyticsisbothaprocessandamindset.It sasetofprocessesfor collecting,analyzing,andusingdatatoimprovesomething,whetherthat s validatingabusinessidea,findingamarket,acquiringcustomers,expanding channels,marketing,relatingtoinvestorsandthepublic,orreportingthemetrics thatdriveyourbusiness. Butit salsoamindset,becauseonceyou recommittedtousingdataineverything youdo,yourealizeit sawayofthinking,andnotjustaboutcrunchingnumbers. Analyticalthinkingisaboutaskingtherightquestions,andfocusingontheonekey metricthatwillproducethechangeyou reafter. Withthisbook,wehopetoprovideyouwiththeguidance,tools,andevidenceto embracedataasacorecomponentofyourstartup ssuccess. We#want#to#show#you#how#to#use#data#to#build#a#better#startup#faster.
4.#How#to#keep#score# Analyticsisabouttrackingthemetricsthatarecriticaltoyourbusiness.Usually, thosemetricsmatterbecausetheyrelatetoyourbusinessmodel wheremoney comesfrom,howmuchthingscost,howmanycustomersyouhave,andhow effectiveyouradvertisingis.analyticsmeasureswhetheryourcustomersaredoing whatyourbusinessmodelhopestheywill. Astartupisaspecial(somemightsaydysfunctional)caseofanalytics.Youdon t alwaysknowwhatyourkeybusinessmetricsare,becauseyou renotentirelysure whatyourbusinessmodelis,andthismeansyou refrequentlychangingthegoals youmeasureandtheactivityyouanalyze.you restilltryingtofindtheright product,ortherighttargetaudience.becauseofthis: In#a#startup,#the#purpose#of#analytics#is#to#find#your#way#to#the#right# product#and#market#before#the#money#runs#out. Sowe regoingtoneedtolearnhowtotalkanalytics. Metricsarethelanguageofanalytics.AnalystslookatKeyPerformanceIndicators (KPIs)totrackthehealthoftheirbusinesses.EveryindustryhasKPIs ifyou rea restaurantowner,it sthenumberofcovers(tables)inanight;ifyou reaninvestor, it sthereturnonaninvestment;ifyou reamediawebsite,it stheuniquevisitors; andsoon. Therearemanydifferentkindsofmetricsyou llneedtobefamiliarwith.three importantdimensionsinclude: Qualitative(and(quantitative(metrics.Qualitativemetricsareunstructured, anecdotal,revealing,andhardtoaggregate;quantitativemetricsinvolve numbersandstatistics,andprovidehardnumbersbutlessinsight. Vanity(and(actionable(metrics.Vanitymetricsmakeyou(andyour investors)feelgood,butdon tchangehowyou llact.actionablemetrics changeyourbehaviorbyhelpingyoupickacourseofaction. Exploratory(metrics(and(reporting(metrics.Exploratorymetricsare speculative,andtrytofindunknowninsightsthatcangiveyoutheupper hand,focusingontheinterestingandunexpectedandontryingnewthings, whilereportingmetricskeepyouabreastofthenormal,managerial,daywtow dayoperations.
Qualitative#and#quantitative# Quantitativedataiseasytounderstand.It sthenumberswetrackandmeasure.it s sportsscoresandmovieratings.assoonassomethingisranked,counted,orputon ascale,it squantified.it sniceandscientific,and(assumingyoudothemathright) youcanaggregateit,extrapolateit,andputitintoaspreadsheet.butit sseldom enoughtogetabusinessstarted.youcan twalkuptopeople,askthemwhat problemsthey refacing,andgetaquantitativeanswer.forthat,youneed qualitativeanswers. Qualitativedata,ontheotherhand,ismessy,subjective,andimprecise.It sthestuff ofinterviewsanddebates,andasitsnamesuggests,it shardtoquantify.youcan t measurequalitativedataeasily.ifquantitativedataanswers what and how much, qualitativedataanswersquestionslike why. Quantitativedataisbereftof emotion qualitativedatamarinatesinit. LeanStartupshowsusthatinitially,whenvalidatinganidea,you relookingfor qualitativedata.whenyou retryingtofigureoutifyouevenhaveaproblemworth solvinginthefirstplace,you renotmeasuringresultsnumerically.you respeaking topeople specifically,topeopleyouthinkarepotentialcustomersintheright targetmarket.you reexploring. You re#getting#out#of#the#building.# There saprocessforcollectingqualitativedatatherightway.youcan tinterview withoutpreparation.youneedspecificquestionsandgoalsinmind,andyouneedto askthemwithoutleadingthewitnessorskewingtheanswers.youhavetoavoid lettingyourenthusiasmandrealitydistortionruboffonyourinterviewsubjects. Doinginterviewswithoutpreparationwillleadtomeaningless orjustplain wrong results.thequalityofthequalitativefeedbackiscritical. Vanity#metrics#vs.#real#metrics# Lotsofcompaniesclaimthey redatawdriven.unfortunately,whiletheyembracethe datapartofthatmantra,fewfocusonthesecondword:driven.ifyouhaveapieceof dataonwhichyoucannotact,you resimplycollectingittostrokeyourownego.it s avanitymetric.youwantyourdatatoinform,toguide,tomakeyouwiser,to improveyourbusinessmodels,tohelpyoudecideonacourseofaction.yourguy mightbesmart,butyou llonlyrealizeyourfullpotentialwhenyoutemperyourgut withreality. Wheneveryoulookatametric,askyourself, whatwillidodifferentlybasedonthis information? Ifyoucan tanswerthatquestion,youprobablyshouldn tworryabout
themetrictoomuch.andifyoudon tknowwhatmetricswouldchangeyour organization sbehavior,youaren tbeingdatadriven.you rejustflounderingindata quicksand. Consider,forexample, totalnumberofusers. Thisisaclassicvanitymetric.The numbercanonlyincreaseovertime(aclassic upandtotheright graph.)more importantly,ittellsusnothingaboutwhatthoseusersaredoing,orwhetherthey re valuabletous.theymayhavesignedupfortheapplicationandvanishedforever. Totalactiveusers isabitbetter assumingthatyou vedoneadecentjobof defininganactiveuser butit sstillavanitymetric.itwillgraduallyincreaseover timetoo,unlessyoudosomethinghorriblywrong. Therealmetricofinterest theactionable 4 one is percentofusersthatare active. Thisisacriticalmetricbecauseittellsusaboutthelevelofengagementyour usersareexperiencingwithyourproduct.whenyouchangesomethingaboutthe product,thismetricshouldchange,andifyouchangeitinagoodway,itshouldgo up.thatmeansyoucanexperiment,learn,anditeratewithit. Anotherinterestingmetrictolookatis numberofusersacquiredovertimeperiod X. Often,thiswillhelpyoucomparedifferentmarketingapproaches forexample, afacebookcampaignonthefirstweek,aredditcampaignonthesecond,agoogle Adwordscampaignonthethird,andaLinkedIncampaignonthefourth.Segmenting experimentsbytimeinthiswayisn tprecise,butit srelativelyeasy. 5 Andit s actionable:iffacebookworksbetterthanlinkedin,youknowwheretospendyour money. Actionablemetricsaren tmagic.theywon ttellyouwhattodo intheprevious example,youcouldtrychangingyourpricing,oryourmedium,orthewayyouroffer isworded.thepointhereisthatyou redoingsomething,basedonthedatayou collect. Metricsarealsocomplex.Therearedozensoffactorsthataffectcustomer acquisition,fromthespeedwithwhichyourwebsiteloadstothetimeofdayyou tweetamessagetotheplacementofabannerad.youcan tcontrolallofthem;but 4 Once, actionable meant youcanbesued. ButsinceGeorgeW.Bushredefinedit,it sbeen acceptedintothedictionaryasmeaning somethingonwhichyoucanact, sowe lluseit grudgingly here. 5 Abetterwayistorunthefourcampaignsconcurrently,usinganalyticstosegmenttheusers youacquireintodistinctcohorts.you llgetyouranswerinoneweekratherthanfour,andcontrolfor othervariableslikeseasonalvariation.we llgetintomoredetailaboutsegmentationandcohort analysislater.
youcantrytoidentifytheonesthatreallymovetheneedle,andfocusonthemina deliberatemanner. Finally,metricsaren tthesameasvalue.youdon tknowwhatauserisworth,soit s hardtorelate engagedusers tomoneyuntilyouknowwhatauserdoesforyou. Thereareothermetricsyou llfocusonlater,suchasthelifetimevalueofacustomer ortheviralcoefficient,thatdealwiththethingsauserdoesforyou.intheearly stagesofyourcompany,youcareaboutgainingcustomerattention,andthe validation orrepudiation ofyourbusinessmodelandsolutionbythose customers.
Pattern:#Eight#vanity#metrics#to#watch#out#for# It seasytofallinlovewithnumbersthatgoupandtotheright.here salistof eightnotoriousvanitymetricsyoushouldavoidobsessingover. Number(of(hits:(Thisisametricfromtheearly,foolishdaysoftheWeb.If youhaveasitewithmanyobjectsonit,thiswillbeabignumber.count peopleinstead. Number(of(pageviews:(Onlyslightlybetterthanhits,sinceitcountsthe numberoftimessomeonerequestsapage.unlessyourbusinessmodel dependsonpageviews(i.e.displayadvertisinginventory)youshould countpeopleinstead. Number(of(visits:Isthisonepersonvisitingahundredtimesorarea hundredpeoplevisitingonce?fail. Number(of(unique(visitors:Theonlythingthisshowsyouishowmany peoplesawyourhomepage.ittellsyounothingaboutwhattheydid,why theystuckaround,oriftheyleft. Number(of(followers/friends/likes:Businesscanbeapopularity contestifyoucangetthosefriendstodosomething,butcounting followersratherthanactionsisabadidea,particularlysincemanypeople followbackautomatically.onceyouknowhowmanyfollowerswilldo yourbiddingwhenasked,you vegotsomething. Time(on(site/number(of(pages:(Theseareapoorsubstituteforactual engagementoractivityunlessyourbusinessistiedtothisbehavior.if customersspendalotoftimeonyoursupportorcomplaintspages,that couldbeabadthing. Emails(collected:(Abigmailinglistofpeopleexcitedaboutyournew startupisnice,butuntilyouknowhowmanywillopenyourmails and actonwhat sinsidethem thisisn tuseful.testsomeofthemandsee. Number(of(downloads:(Whileitsometimesaffectsyourplaceinapp storesandrankings,downloadsalonedon tleadtolifetimevalue.measure activations,accountcreations,orsomethingelse. Shhh vanity#metrics#are#sometimes#useful# Soarevanitymetricsalwaysbad? Notreally,butyoushouldn twastemuchtimeonthem,orusethemtolieto yourself,particularlyearlyonwhenyou restilltryingtodecidewhetheryou ve foundaproblemworthsolving.doyourbesttoeliminateallvanitymetrics.you ll lookatthem,ofcourse(afterall,you restillhuman)butyoushouldn tusethemfor decisionwmaking.thatpathleadstomadnessandselfwdelusion.
Vanitymetricshavetheirplace reportingdatatoinvestors,creatinginfographics forthepress,andsoon.we llsneakinabitmoreonvanitymetricslateron (becauseweknowyoulovethem,evenifyouknowthey rebadforyou) Exploration#versus#reporting# AsAvinashKaushikpointsout,DonaldRumsfeldknewathingortwoabout analytics. There#are#known#knowns;#there#are#things#we#know#that#we#know.#There#are#known# unknowns;#that#is#to#say#there#are#things#that,#we#now#know#we#don t#know.#but#there# are#also#unknown#unknowns there#are#things#we#do#not#know,#we#don t#know. # Understandingwhatkindofinformationwe redealingwithtellsusalotabouthow weshouldproceed.weneedtocheckourfacts;automatetherepetitivereporting; andteachourhiddenknowledgetotherestoftheorganization.butwhenwe re searchingforinsightandinnovation,whatwereallycareaboutisthefourthkindof information theunknownunknowns.thisiswherethemagiclives.it'swhere we'lllearnthingswedidn'tknow.table1showsusthesefourkindsofinformation. ( Things(we(know ( Things(we(don t(know ( (we(know( (we(don t(know( Facts(whichmaybe wrongandshouldbe checkedagainstdata.) Reporting(whichwe shouldautomate.) Intuition(whichwe shouldquantifyand teach.)# Exploration(whichis wheretheinteresting epiphanieslive.) Table1:Thefourkindsofknowledge Weworkwithdataintwo postures. Thingsweknow isareportingposture countingmoney,orusers,orlinesofcode.weknowwedon tknowthevalueofthe metric,sowegofindout.butweknowthemetricisneeded,andbigcompanies spendalotoftimedoingit.it stheworkofbusinessanalystswhoworkwith BusinessIntelligencesoftwareandDataWarehouses. Reportingmetricstrackprogressalonganassumedpath.Areportingmetricsays, Howmanywidgetsdidweselltoday? andtellsouraccountanthowmuchtaxto pay;anoptimizationmetricsays, Didthegreenortheredwidgetsellmore? and choosestheofficialcolorofourwidgets. Theotherposture, Thingswedon tknow, ismuchmorerelevanttostartups: exploring,tofindoutsomethingnewwithwhichtodisruptamarket.it siterative, exploratory,andimprecise.itleadsdownplentyofwrongpaths,andhopefully
towardssomekindofeurekamomentwhentheideafallsintoplace.thisfitswhat SteveBlanksaysastartupshouldspenditstimedoing:searchforascalable, repeatablebusinessmodel. Exploratorymetricsare whatif metrics,thescoresofourexperiments.weexpect themtochange;we retestingonethingagainstanother.insomecases,wemayfind thatacompletelyunexpectedcauseproducedthechangewewanted that sour secretadvantage. AnalyticshasaroletoplayinallfourofRumsfeld squadrants.itcancheckourfacts andassumptions,suchasopenratesorconversionrates,tobesurewe renot kiddingourselvesandcheckthatourbusinessplansareaccurate.itcantestour intuitions,turninghypothesesintoevidence.itcanprovidethedataforour spreadsheets,waterfallcharts,andboardmeetings.anditcanhelpusfindthe nuggetofopportunityonwhichtobuildabusiness.intheearlystagesofyour startup,thelattermattersmost. Case#study:#Exploring#your#way#to#success# MikeGreenfieldandhiscoWfoundersstartedCircleofFriendsinSeptember2007, shortlyafterfacebooklaunchedtheirdeveloperplatform.thetimingwas perfect:facebookbecameanopen,viralplacetoacquireusersasquicklyas possibleandbuildyourstartup.therehadneverbeenaplatformwithsomany users(facebookhad~50millionusersatthetime)thatwassoopentoreaching them. CircleofFriendswasasimpleidea:aFacebookapplicationthatallowedyouto organizeyourfriendsintocirclesfortargetedcontentsharing.mikegreenfield notesnowthatitwasbasically, Google+forFacebook (beforegoogle+existed.) BymidW2008,CircleofFriendshad10Musers.Greenfieldfocusedongrowth aboveeverythingelse. Itwasalandgrab, hesaid.andcircleoffriendswas clearlyviral.buttherewasaproblem.toofewpeoplewereactuallyusingthe product. AccordingtoGreenfield,lessthan20%ofcircleshadanyactivitywhatsoever aftertheirinitialcreation. Wehadafewmillionmonthlyuniquesfromthoseten millionusers,butasageneralsocialnetworkweknewthatwasn tgoodenough andmonetizationwouldlikelybepoor. SoMikewentdigging.
Hestartedlookingthroughtheirdatabaseofusersandwhattheyweredoing. Theydidn thaveaninwdepthanalyticaldashboardatthetime,butgreenfield couldstilldosomeexploratoryanalysis.andhefoundasegmentofusers moms tobeprecise thatbuckedthepoorengagementtrendofmostusers.here s whathefound: Theirmessagestooneanotherwereonaverage50%longer Theywere115%morelikelytoattachapicturetoaposttheywrote Theywere110%morelikelytoengageinathreaded(i.e.deep) conversation Circleowners friendswere50%morelikelytoengagewiththecircle Theywere75%morelikelytoclickonFacebooknotifications Theywere180%morelikelytoclickonFBnewsfeeditems Theywere60%morelikelytoacceptFBinvitations ThenumbersweresocompellingthatinJune2008,Greenfieldandhisteam switchedfocuscompletely.theypivoted.andinoctober2008theylaunched CircleofMomsonFacebook. CircleofMomsprovedtobeextremelypopular.Bylate2009,CircleofMomshad severalmillionusersandengagementcontinuedtobestrong.thecompanywent throughsomeupsanddownsafterthat,asfacebookstartedtoradicallychange theirplatformandlimitapplications abilitiestospreadvirally.ultimately,the companymovedofffacebook,grewindependently,andsoldtosugarinc.inearly 2012.Mikesharesmoreofthispartofthestory postthepivottocircleof Moms whenthecompanybecameparticularlyadeptattesting,measuringand learning,onhisblognumeratechoir. What#makes#a#good#metric?# Whatyoumeasureandwhenisaffectedbymanythings:yourbusinessmodel,stage ofyourcompany,andsoon.herearesomerulesofthumbforwhatmakesanumber thatwillproducethechangesyou relookingfor. A(good(metric(iscomparative.Beingabletocompareametrictoothertime periods,groupsofusers,orcompetitorshelpsyouunderstandwhichwaythingsare moving.increasedconversionfromlastweekismoremeaningfulthan 2% conversion.
A(good(metric(isunderstandable.Thismeansit snomorecomplicatedthanagolf handicap.otherwise,peoplewon trememberitanddiscussit,andit llbemuch hardertoturnachangeinthedataintoachangeintheculture. A(good(metric(is(a(ratio(or(a(rate.Accountantsandfinancialanalystshaveseveral ratiostheylook 6 attounderstand,ataglance,thefundamentalhealthofacompany. Youneedsometoo. Thereareseveralreasonsratiostendtobethebestmetrics: Ratiosare(easiertoacton.Thinkaboutdrivingacar.Distancetravelledis informational.butspeed distanceperhour issomethingyoucanacton, becauseittellsyouaboutyourcurrentstate,andwhetheryouneedtogo fasterorslowertogettoyourdestinationontime. Ratiosare(inherentlycomparative.Ifyoucompareadailymetrictothesame metricoveramonth,you llseewhetheryou relookingatasuddenspikeora longwtermtrend.inacar,speedisonemetric,butspeedrightnowover averagespeedthishourshowsyoualotaboutwhetheryou reacceleratingor slowingdown. Ratiosarealsogoodforcomparingfactorsthataresomehowopposed,orfor whichthere saninherenttension.inacar,thismightbedistancecovered dividedbytraffictickets.thefasteryoudrive,themoredistanceyoucover butthemoreticketsyouget.thisratiomightsuggestwhetheryoushouldbe breakingthespeedlimit. Leavingourcaranalogyforamoment,considerawebsitewithfreeandpaid versionsofitssoftware.thecompanymightchoosebetweenluringinnewusers withenticingfeatures,andtryingtoconvinceuserstopayfortheserviceinorderto unlockotherfeatures.havingafullwfeaturedfreeproductmightreducesales;but havingacrippledproductmightreducenewusers.theratioofpayingusersto churnratecouldcombinethesetwometrics. Anonlineretailerlivesanddiesbyitsconversionrate theratioofvisitorsto buyers.ifthesitesellstoohard,peoplewon tvisitit.ifitdoesn ttryhardenoughto sell,peoplewon tbuy.yearsago,executivesatgsicommercetoldustheytracked therealwtimeratioofrevenuetodaytothesamerevenueonthesameweekdaylast year.iftheratiodroppedbelowone,theyknewgrowthhadstalled,whichtriggered aninvestigation.foracompanythatmakesdozensofmerchandisingchangesaday, thiswasagoodoverallmetrictomanageallofthetweaksandadjustments. 6 ThisincludesfundamentalssuchasthepriceWtoWearningsratio,salesmargins,thecostof sales,revenueperemployee,andsoon.
A(good(metric(is(targeted(to(the(right(audience.Agoodnumberinthewrong handsisuseless.somethingsyoushareinternallywon thelpyouinaboard meeting;somemetricsthemediawilltalkaboutarejustvanitycontentthatwon t growthebusinessorfindproduct/marketfit.investorswon thavethecontextto understandcomplicateddata;overlysimpleinformationwon tbeusefulto developers. Forastartup,audiencesmayinclude: Internal(businessgroups,tryingtodecideonapivotorabusinessmodel Developers,prioritizingfeaturesandmakingexperimentalvalidationpartof the LeanQA process Marketersoptimizingcampaignstogeneratetrafficandleads Investors,whenwe retryingtoraisemoney Media,forthingslikeinfographicsandblogposts Asyou repickingthemetricsthatwillguideyourbusiness,keeptheaudiencein mind. A(good(metricchanges(the(way(you(behave.Thisisbyfarthemostimportant criteriaforametric:whatwillyoudodifferentlybasedontheanswer? For accounting metricslikedailysalesrevenue,makesurethatit s somethingwhich,whenenteredintoyourspreadsheet,makesyour predictionsmoreaccurate.anddoublewcheckthedata:thiswillbereviewed bytheboard,theinvestors,possibleacquirers,andeventhemedia. For experimental metricsyouusetooptimizetheproduct,pricing,or market,choosesomethingwhich,basedontheanswer,willsignificantly changeyourbehavior.betteryet,agreeonwhatthatchangewillbebefore youcollectthedata.drawingalineinthesandisagreatwaytoenforcea disciplinedapproach.
Case#study:#qidiq#changes#how#it#adds#users# OneoftheYearOneLabscompanies,qidiq,isatoolfordoingreallysimple surveysofasmallgroupviaemailormobileapplication.intheearlyiterationsof theproduct,asurveycreatorinvitedrespondentstojoinagroup.oncethose respondentshadsignedupandcreatedanaccount,theycouldanswersurveys deliveredbyemailorpushedtoaniphoneclient. Theproblemwasthatonlyasmallpercentageofpeoplewhowereinvited actuallycreatedanaccountandresponded.sowedevisedatest:whynotjust assumethattherecipienthasanaccount,andsendthemasurveyquestionthey canrespondtowithasingleclickortap?theactofrespondingistacitacceptance ofenrollment;later,iftherecipientwantstologintotheaccount,they llhaveto doapasswordrecovery,sincetheyneverprovidedone butthat seasyenough, andbythen,therecipientisengagedinthesurveysandthegroup. Theqidiqteamquicklychangedtheirapplicationandsentoutmoresurveys.The resultswerestriking:responserateswentfrom10w25percentwiththeenrollw firstmodel,eveninthebestofcases,to70w90percentwiththevotewfirstmodel. Thisinturnledtoarethinkingofthemobileapplicationdevelopment,since mobileapplicationscouldn tcompetewiththecrosswplatformubiquityand immediacyofewmail. Byfocusingonthekeymetricofresponserate,wewereabletoavoidthe temptationofwastingourenergyonthesexiermobileapp, sayscowfounder JonathanAbrams. Becauseitwastheresponseratethatmattered,itbecame clearearlyonthatemail,whilelesssexy,wasthebetterstrategyforourstartup. Themetrictheyweretracking,whichwasthebasisoftheirwholeproduct,was thenumberofpeoplewhowouldrespondtoaquestion.thatwastheright metric,andwhentheyfoundaproductchangethatmoveditdramaticallyinthe rightdirection,itmadethemrethinkthedesignoftheirentireservice. Moving#targets# Whenpickingatargetnumberearlyon,you redrawingalineinthesand not carvingitinstone.you rechasingmovingtargets,becauseyoureallydon tknow howtodefinesuccess.there salwaysasignificantelementofguessingandguts involved.adjustingyourgoalsforkeymetricsaswellashowyoudefinethose
metricsisokay,aslongasyou rebeinghonestwithyourself,recognizingthechange thismeansforyourbusiness,andnotjustloweringexpectationssothatyoucan keepgoinginspiteoftheevidence. WhenyourMVPisinthemarketandyou reacquiringearlyadopterwcustomersand testingtheiruseofyourproduct,youwon tevenknowhowthey rereallygoingto useit(althoughyou llhaveassumptions.)youmightassume,forexample,thatyour producthastobeusedonadailybasistobesuccessful onlytofindoutthat snot thecase.inasituationlikethat,it sreasonabletoupdateyourmetricsaccordingly providedthatyou resomehowabletovalidatethevaluecreated. Case#Study:#HighScore#House#defines#an# active#user # HighScoreHouseisaYearOneLabsportfoliocompany.Weworkedwiththe foundersfromdayone,rightfromthemomentofideation. WhenHighScoreHouselaunchedtheirMVP,theyhadseveralhundredfamilies readytotestitout.atthetime,highscorehousewasasimpleapplicationthat allowedparentstolistchoresandchallengesfortheirkidsandassignpoint valuesforcompletingthem.kidscouldcompletethosetasks,collectpoints,and redeemthemforrewardstheywanted. Thefoundersdrewalineinthesand:inorderfortheMVPtobeconsidered successful,parentsandkidswouldhavetoeachusetheapplicationfourtimes perweek.onlyfamiliesthatdidthiswereconsidered active families.itwasa high,butgood,bar.weallfeltthatiftheycouldachievethis,we dknowthatthey weredoingsomethingright. Afteramonthorsoofdevelopmentandtweaking,thepercentageofactive familieswasstillquitelow.thefoundersweredisappointedbutdeterminedto keepexperimenting.theybeganchanginganumberofthingstotryandimprove engagement: TheymodifiedthesignWupflow(makingitmoreeducationalandclearto increasequalitysignwupsandtoimproveonwboarding.) Theystartedsendingemailnotificationsasdailyreminderstoparents(to increaseengagement.) Therewasanincrementalimprovementeachtime,butnothingthatmovedthe needlesignificantlyenoughforustobelievethatthemvpwasasuccess.
ThencoWfounderandCEOKyleSeamandidsomethingcritical:he#picked#up#the# phone.hestartedcallingparentswhohadsignedup,butwhoweren tactive. FirsthereachedouttothosethathadabandonedHighscoreHousecompletely ( churnedout. )Formanyofthem,theapplicationjustwasn tsolvingabig enoughpainpoint.that sfine.weneverassumedthemarketwas allparents that sjusttoobroadadefinition,particularlyforafirstversionofaproduct. KylethencalledthosefamiliesthatwereusingHighScoreHouse,butnotusingit enoughtocrossthelinetheteamhaddrawninthesand.manyofthesefamilies respondedpositively: We reusinghighscorehouse.it sgreat.thekidsare makingtheirbedsconsistentlyforthefirsttimeever Theresponsefromparentswasasurprise.ManyofthemwereusingHighScore Houseonlyonceortwiceaweek,buttheyweregettingvalueoutoftheproduct. Fromthis,Kylelearnedaboutsegmentationandwhichtypesoffamilieswere moreorlessinterestedintheproduct.hebegantounderstandthattheinitial baselineofusagetheteamhadsetwasn tconsistentwithhowengaged customerswereusingtheproduct. Thatdoesn tmeantheteamshouldn thavetakenaguess.withoutthatinitialline inthesandtheywouldhavehadnobenchmarkforlearning,andkylemightnot havepickedupthephone.butnowhereallyunderstoodhiscustomers.the combinationofquantitativeandqualitativedatawaskey. Asaresultofthislearning,weallagreedtoredefinethe activeuser thresholdto moreaccuratelyreflectexistingusers behavior,becausewenowunderstoodthat gettingsignificantvaluefromhighscorehousedidn trequirethelevelof applicationusagewe dthoughtitdid. Thefirstlessonisthis:there snosubstituteforengagingwithcustomersand usersdirectly.allthenumbersintheworldcan texplainthe why whichis criticaltounderstand,particularlyattheearlystagesoftheminimumviable product.pickupthephonerightnowandcallacustomer,evenonethat s disengaged.you lllearn. Thesecondlessonisthatit sokaytoadjustyourkeymetricsaslongasyoutruly understandwhyyou redoingitandcanjustifythechange.loweringthebarfor thesakeofgettingpastitdoesn tcount:that sjustcheating,andlyingtoyourself. Butadjustingyourkeymetricsbasedonanunderstandingofyourcustomers behaviorandthevalueyou recreatingisareasonablebehaviorasyouiterate towardsproduct/marketfit.
5.#Deciding#what#to#do#with#your#life# Asafounder,you retryingtodecidewhattospendthenextfewyearsofyourlife workingon.thereasonyouwanttobeleanandanalyticalabouttheprocessisso thatyoudon twasteyourlifebuildingsomethingnobodywants. Hopefully,youhaveanideaofwhatyouwanttobuild.It syourblueprint,andit s whatyou lltestwithanalytics.youneedawayofquicklyandconsistently articulatingyourhypothesesaroundthatidea,soyoucangoandverify or repudiate themwithrealcustomers.todothis,werecommendashmaurya s Lean#Canvas. Butthecanvasisonlyhalfofwhatyouneed.It snotjustaboutfindingabusiness thatworks.youalsowanttofindabusinessthatyouwanttoworkon.budcaddell hasthreeclearcriteriafordecidingwhattospendyourtimeon:somethingthat you regoodat,thatyouwanttodo,andthatyoucanmakemoneydoing. Let slookatbothtoolsinabitmoredetail. The#Lean#Canvas# TheLeanCanvasisaoneWpagevisualbusinessplanthat songoingandactionable.it wascreatedbyashmaurya,whowasinspiredbyalexasterwalder sbusiness ModelCanvas.Itconsistsofnineboxesorganizedonasinglesheetofpaper, designedtowalkyouthroughthemostimportantaspectsofanybusiness.thelean Canvasisfantasticatidentifyingtheareasofbiggestriskandenforcingintellectual honesty. How#to#use#a#Lean#Canvas Unlikeatraditionalbusinessplan,theLeanCanvasismeanttobecontinuouslyused anditeratedupon.it sa living,breathing plan,notahypotheticaltomeofnonsense thatyouthrowouttheminuteyoustartactuallyworkingonyourstartup.once you vefilledouttheleancanvas(ormostofit),youstartrunningexperimentsto validateorinvalidatewhatyou vehypothesized. Initssimplestform,thinkofeachboxasa pass/fail :ifyourexperimentsfail,you don tgotothenextbox;rather,youkeepexperimentinguntilyouhitawall completelyorgettothenextstep.theonlyexceptionisthe KeyMetrics box, whichismeanttokeeparecordofthemostimportantmetricsyou retracking.you don trunexperimentsonthisbox,butit simportanttofillitoutanywaybecauseit s definitelyopentodebateanddiscussion.
Below,we veprovidedatablethatshowseightareasofleancanvasandthe correspondingmetricswethinkyoushouldconsider;tomeasurethehealthofthat area.thesemetricsmaychangedependingonthetypeofbusinessyou rein,butthe guidelinesarevaluablejustthesame.andwe llsharemoredetailslaterinthebook onkeymetricsbasedontypeofbusiness,aswellasbenchmarksyoucanaimfor. EachoftheboxesinAsh scanvashasrelevantmetricsyouneedtotrack.theyeither tieyouronewpagebusinessmodeltoreality,confirmingeachbox ortheysendyou backtothedrawingboard. Lean#Canvas#box# Problem Solution UniqueValueProposition CustomerSegments Channels UnfairAdvantage RevenueStreams CostStructure Relevant#metrics# Respondentswhohavethisneed;respondentswhoareawareof havingtheneed RespondentswhotrytheMVP;engagement;churn;most= used/least=usedfeatures Feedbackscores;independentratings;sentimentanalysis; customer=wordeddescriptions;surveys Reachability;uniquekeywordsegments;targetedfunneltraffic Leadsandcustomersperchannel;viralcoefficient;netpromoter score;openrate;affiliatemargins;clickthroughrate;pagerank; messagereach Respondents understandingoftheusp Lifetimecustomervalue;Averagerevenueperuser; Fixedcosts;costofcustomeracquisition;costofservicingthenth customer;supportcosts;keywordcosts WeencourageeverystartuptouseLeanCanvas.It sanenlighteningexperience,and wellworththeeffort. What#should#you#work#on?# TheLeanCanvasprovidesaformalframeworktohelpyouchooseandsteeryour business.butthere sanother,morehuman,sidetoallofthis. Do#you#want#to#do#it? Thisdoesn tgetaskedenough.investorssaytheylookforpassionatefounders,who reallycareaboutsolvingaproblem.butit sseldomcalledoutassomethingyou shoulddevotemuchthoughtto.butifyou regoingtosurviveasafounder,youhave
tofindtheintersectionofdemand(foryourproduct);ability(foryoutomakeit); anddesire(foryoutocareaboutit.) Thattrifectaisoftenoverlooked,witheringundertheharshlightofdataandaflood ofcustomerfeedback.butitshouldn t.don t#start#a#business#you re#going#to#hate:life istooshort,andyourwearinesswillshow. BudCaddellhasanamazinglysimplediagramofhowpeopleshouldchoosewhatto workon,showninfigure1. Figure(1:(What(to(work(on,(as(illustrated(by(Bud(Caddell( Bud sdiagramshowsthreeoverlappingrings:whatyouliketodo;whatyou regood# at;andwhatyoucan#be#paidtodo.foreachoftheintersectionsbetweenrings,he suggestsacourseofaction:
Ifyouwanttodosomethingandaregoodatit,butcan tbepaidtodoit,learn# to#monetize. Ifyou regoodatsomethingandcanbepaidtodoit,butdon tlikedoingit, learn#to#say#no. Ifyouliketodosomethingandcanbepaidtodoit,butaren tverygoodatit, learn#to#do#it#well. Thisisn tjustgreatadviceforcareercounselors:whenlaunchinganewventure,you needtoproperlyassessthesethreedimensionswell. First,askyourself:can#you#do#this#thing#you re#hoping#to#do,#well?thisisaboutyour abilitytosatisfyyourmarket sneedbetterthanyourcompetitors,andit sa combinationofdesignskill,coding,branding,andmyriadotherfactors.ifyou identifyarealneed,youwon tbetheonlyonesatisfyingit,andyou llneedallthe talentyoucanmuster.doyouhaveanetworkoffriendsandcontactsthatcangive youanunfairadvantage?today scustomersarefickle,andlife stooshorttodoa crappyjob. Second,figureoutwhether#you#like#doing#this#thing.Startupswillconsumeyourlife, andthey llbeaconstantsourceofaggravation.yourbusinesswillcompetewith yourfriends,yourpartner,yourchildren,andyourhobbies.youneedtobelievein whatyou redoingsothatyou llkeepatitandridethroughthegoodtimesandthe bad.wouldyouworkonitevenifyouweren tbeingpaid?isitaproblemworth solving,thatyou llbragabouttoothers?ifnot,maybeyoushouldkeeplooking. Finally,besureyou#can#make#money#doing#it 7.Thisisaboutthemarket sneed.this isbyfarthemostimportantofthethree;theothertwoareeasy,becausethey reup toyou.insales,aprospectmusthaveseveralattributesbeforethey rea real lead: need,budget,timeline,authority,andawareness. Needisthemostfundamentalthingofall.Aneedissomethingyoucan satisfyforanother,inreturnforsomething money,attention,referral,and soon. Budgetissimplywhethertheyhavemoneytheycanpartwithinreturnfor whatyou reselling. Timelineisasenseofurgencywithinwhichtheyneedtomakeapurchase. Timekillsalldeals.Ifthereisn tasensethattheclockisticking,andthat inactionwillberisky,yourrevenuestreamisatrisk. 7 Noteveryoneishopingtomakemoneywiththeirstartup.Somepeoplearedoingitfor attention,ortofixgovernment,ortomaketheworldabetterplace.ifthat syou,replace money with producetheresultsi mhopingtoachieve asyoureadthisbook.
Authoritymeanstheprospectcanmakethepurchasingdecision.Ifyou re sellingachildren stoy,thechildprobablydoesn thavepurchasingauthority. Sotheirneedsdon tcountunlesssomeonewhocanmakethepurchase understandsthem. Awarenessmeansthecustomerknowstheyneedtheirproblemsolved.If thepersonisn tawareoftheneed,thenyou llneedtocreateitforthem.your marketshouldimmediatelygrasphowyou llsolvetheirproblem.this doesn tmeanthecustomerisaskingforthesolutionalready thewalkman, ipadandminivanprovethatcustomersoftendon tknowtheyneed somethinguntiltheyseeit.buttheymustbeawareoftheirneed.sony, Apple,andChryslerdidn tjusthaveinnovativeideas theyhadhundredsof millionstospendonmarketing.ifyourmarketisn tawareofitsproblem, you llspendallyourtimeconvincingthemofthatneed,andthey lllietoyou aboutituntilthemoneyrunsout. Intheearlystagesofastartup,you llbedealingwithalotofdata.you rehearingall kindsofopinionsthatcanchangeyourbusinessmodeldaily.you reawashinthe tidesofopinion,andbuffetedbywhateverfeedbackyou veheardmostrecently. Neverforgetthatyou retryingtoanswerthreefundamentalquestions: Have#I#identified#a#problem#worth#solving?#Am#I#proposing#the#right# solution?#do#i#actually#want#to#solve#it? Or,moresuccinctly,( Should(I(go(build(this(thing?
6.#DataXdriven#vs.#DataXinformed# Dataisapowerfulthing.Itcanbeaddictive,makingyouoverWanalyzeeverything. You#don t#run#detailed#a/b#testing#before#deciding#what#pants#to#put#on#in# the#morning#or#you d#never#get#out#the#door. Muchofwhatwedoisunconscious,basedonpastexperienceandpragmatism.And withgoodreason:relyingonwisdomandexperience,ratherthanarigidanalysis, helpsusgetthroughourday. AsLeanStartup,withitsrigorousapproachtocustomerdevelopment,catchesonin businessesofallsizes,it ssparkedabacklashagainsttoomuchdata.it salmost hereticaltocriticizeleanloudly,butwehearwhisperedcomplaintsinthecow workingspacesandcoffeeshopsoftheworld stechhubs.ratherthanbeaslaveto thedata,thesecriticssay,weshoulduseitasatool. We#should#be#dataVinformed,#not#dataVdriven. Mostly,they rejustbeinglazy,andlookingforreasonsnottodothehardwork.but sometimes,theyhaveapoint:usingdatatooptimizeonepartofyourbusiness, withoutsteppingbackandlookingatthebigpicture,canbedangerous,evenfatal. ConsidertravelagencyOrbitzanditsdiscoverythatMacuserswerewillingto reserveamoreexpensivehotelroom.rogerliew,thecompany scto,toldthewall StreetJournal, wehadtheintuition[thatmacusersare40%morelikelytobooka fourworfivewstarhotelthanpcusersandtostayinmoreexpensiverooms],andwe wereabletoconfirmitbasedonthedata. 8 Ontheonehand,analgorithmthatdidn tincludeseeminglyunrelatedcustomer data(inthiscase,thetypeofcomputerthatvisitorswereusing)wouldn thave foundthisopportunitytoincreaserevenues.ontheotherhand,analgorithmthat blindlyoptimizesbasedoncustomerdata,regardlessofitsrelationshiptothesale, mayhaveunintendedconsequences likebadpr.datawdrivenmachine optimization,whennottemperedwithhumanjudgment,cancauseproblems. Yearsago,GailEnnis,thenCMOofOmniture,toldusthattheyhavetotemper machineoptimizationwithhumanjudgmentforotherreasons.lefttotheirown devices,weboptimizationsoftwarequicklylearnsthatscantilycladwomen generateafarhigherclickwthroughrateonwebpagesthanotherformsofcontent. 8 OnOrbitz,Macuserssteeredtohigherprices,August23,2012 (http://online.wsj.com/article/sb10001424052702304458604577488822667325882.html)
ButthatclickthroughrateisashortWtermgain,offsetbydamagetothebrandand imageofthecompany.sothecompany ssoftwareworkshandwinwhandwith curatorswhounderstandthebiggerpicture.thehumansgettheinspiration;the datadoesthevalidation. PerhapsthemosteloquentpushWbackcomesfromAndrewChen. Knowthe differencebetweendatawdrivenanddatawinformed, heimploresus. Ultimately, metricsaremerelyareflectionoftheproductstrategythatyoualreadyhavein placeandarelimitedbecausethey rebasedonwhatyou vealreadybuilt,whichis basedonyourcurrentaudienceandhowyourcurrentproductbehaves. 9 Inmathematics,localmaximaorminimaarethelargestorsmallestvaluesofa functionwithinagivenneighborhood. 10 Thatdoesn tmeanit sthelargestpossible value,justthelargestoneinaparticularrange.asananalogy,consideralakeona mountainside.thewaterisn tatitslowestpossiblelevel thatwouldbesealevel butit satthelowestpossiblelevelintheareasurroundingalake. Optimizationisallaboutfindingthelowestorhighestvaluesofaparticular function.oneofandrew skeypointsisthatdatausedforoptimizationpurposesis subjecttolocalmaxima.amachinecanfindtheoptimalsettingsforsomething,but onlywithintheconstraintsandproblemspaceit sawareof,inmuchthesameway thatthewaterinamountainsidelakecan tfindthelowestpossiblevalue,justthe lowestvaluewithintheconstraintsprovided. Tounderstandtheproblemwithconstrainedoptimization,imaginethatyou re giventhreewheelsandaskedyoutoevolvethebest,moststablevehicle.aftermany iterationsofpittingdifferentwheellayoutsagainstoneanother,youcomeupwitha tricyclewlikeconfiguration.it stheoptimalwheelconfiguration.datawdriven optimizationcanperformthiskindofiterativeimprovement. Whatitcan tdo,however,issay, youknowwhat?fourwheelswouldbeway better Math#is#good#at#optimization.#Humans#are#good#at#inspiration.#Change# favors#local#maxima. InhisbookA#River#Out#Of#Eden,RichardDawkinsusestheanalogyofaflowingriver todescribeevolution.evolution,heexplains,cancreatetheeye.infact,itcancreate dozensofversionsofit,forwasps,octopods,humans,eagles,andwhales.whatit 9 http://andrewchen.co/2012/05/29/knowwthewdifferencewbetweenwdatawinformedwandwversusw datawdriven/ 10 http://en.wikipedia.org/wiki/maxima_and_minima
can tdowellisgobackwards:onceyouhaveaneyethat suseful,slightmutations don tusuallyyieldimprovements.putanotherway,ahumanwon tevolveaneagle s eye,becausetheintermediatestepsareallbadeyesight. MachineWonlyoptimizationsuffersfromsimilarlimitationsasevolution.Ifyou re optimizingforlocalmaxima,youmightbemissingabigger,moreimportant opportunity.it syourjobtobetheintelligentdesignertodata sevolution.aschen says, noteverythingisanoptimizationproblem.anddelegatingyourdecisionw makingtoonlywhatyoucanmeasurerightnowoftendewprioritizesmore importantmacroaspectsoftheproblem. We vefoundthissentimentinmanyofthestartupswithwhomwe vespoken.they haveageneraluneasewiththewayinwhichtheircompaniesarebeingrunbythe numbers,withouttheopportunitytostepbackandlookatthebiggerpictureofthe marketthey rein,theproblemthey resolving,andtheirfundamentalbusiness models. Ultimately,#quantitative#data#is#great#for#testing#hypotheses,#but#it s#lousy# for#generating#new#ones. Lean#Startup#and#Big#Vision# TheretendstobeaspectrumofattitudestowardsdataWdrivenstartups.Some entrepreneursaremaniacally,almostcompulsively,datawobsessed,buttendtoget miredinanalysiswparalysis.othersarecasual,shootwfromwthewhipintuitionistswho ignoredataunlessitsuitsthem,andpivotlazilyfromideatoideawithoutdiscipline. AttherootofthisdivideisthefundamentalchallengethatLeanStartupadvocates face:howdoyouhaveaminimumviableproductandahugelycompellingvisionat thesametime? Havingabigvisionisimportant.Startingacompanywithoutoneleavesyoutoo susceptibletooutsideinfluences,betheyfromcustomers,investors,competition, press,oranythingelse.withoutabigvision,you lllackpurpose,andovertime you relikelytofindyourselfwanderingaimlessly. PlentyofpeopleuseLeanStartupasimplicitapprovalforstartingacompany withoutavision. It ssoeasytostartacompanythesedays, theyreason, the barriersaresolow,thateveryonecandoit,right? Oneclearresultisthatmore companiesarebeingcreated.unfortunately,toofewofthemhavearealvisionfor whatthey retryingtoaccomplish,andthataloneislikelytokillthem.
Soifabig,hairy,audaciousvisionisimportant onewithachangingwthewworldtype goal howdoesthatreconcilewiththestepwbywstep,methodicalapproachoflean Startup? Theanswerisactuallyprettysimple.YouneedtothinkofLeanStartupasthe processyouusetomovetowardsandachieveyourvision. EarlyVstage#founders#aren t#building#a#product.#they re#building#a#tool#to# learn#what#product#to#build. Thishelpsseparatethetaskathand findingasustainablebusinessmodel from thescreens,linesofcode,andmailingliststhey vecarefullybuiltalongtheway. LeanStartupisfocusedonlearningaboveeverythingelse,andencouragesbroad thinking,explorationandexperimentation.it snotaboutmindlesslygoingthrough themotionsof Build Measure Learn it saboutreallyunderstandingwhat s goingonandbeingopentonewpossibilities. Lean#Canvas#+#vision# TheLeanCanvasdoesn thaveasectionfor vision,becauseleanisallabout testing,andyoucan ttestabreadthofvision.larryandsergeimighthavehada visionofaglobalhivewmindforinformationwhentheyfirstsketchedoutthe Pagerankalgorithm,buttheycouldn ttestthatvision.theycouldcheckwhetherit inspiredpeople,andtheycouldholdtheideaintheirheadsastheygradually createdsearch,maps,android,andmore. Justbecausevisionisn texplicitlyspelledoutinleandoesn tmeanit snotessential. Visionmattersthemomentyoustartdescribingaproblemyouwanttosolve,and seeinghowitcapturestheattentionofothers.ifyou retacklingaproblemthat snot audaciousanddisruptive,you reunlikelytoelicitmuchofaresponse.if,onthe otherhand,youdiscoversomethingtrulymeaningfultopeople,something genuinelypainful,thenthere saverygoodchancethere sacompellingvisionofa betterworldthatgoesalongwiththatproblem. Expanding#your#vision#with#Lean#Startup# Alotofstartupssufferfrombeingtoosmallinscope.We vetalkedtofounderswho wanttobetheleadingproviderintheirstateorprovince.whynottheworld?even theallieshadtopickabeachhead;butlandinginnormandydidn tmeantheylacked abigvision.theyjustfoundagoodplacetostart. SomepeoplebelieveLeanStartupencouragesthatsmallness,butthat snotthecase atall.usedproperly,leanstartuphelpsexpandyourvisionbecauseyou re
encouragedtoquestioneverything.asyoudigdeeperandpeelawaymorelayersof whatyou redoing whetheryou relookingatproblems,solutions,customers, revenueoranythingelse you relikelytofindalotmorethanyouexpected.if you reopportunisticaboutit,youcanexpandyourvisionandunderstandhowto gettherefaster,allatthesametime.
What s#next?# We rehardatworkontherestofthebook,andbusyspeakingwithfoundersand aspiringentrepreneursaboutbuildingbettercompaniesfaster.wewanttohear fromyou joinusatwww.leananalyticsbook.com,andsignupforthemailinglist theresowecankeepintouch. Soon,you llbeabletobuyaprevieweditionofthebookfromo Reilly,andwill receiveregularupdatesofthe rough versionofthebookaswepublishit.wehope tobedoneinthenextfewmonths. Thanksfortakingthetimetoreadthis,andtoshareyourthoughtswiththerestof thestartupcommunity. Ben#&#Alistair.