Mac and PC Performance Analysis

Transcription

1 MacandPCPerformanceAnalysis DainK.Schroeder OfficeofScience,FacultyandStudentTeamProgram BigBendCommunityCollege,Washington LawrenceBerkeleyNationalLaboratory Berkeley,California August14,2008 PreparedinpartialfulfillmentoftherequirementoftheOfficeofScience,Departmentof Energy'sFacultyandStudentTeaminternshipunderthedirectionofCharlesE.Verboomin themac/pcworkstationsupportgroupatlawrenceberkeleynationallaboratory. Participant: Signature ResearchAdvisor: Signature i

2 TableofContents TableofContents... ii Abstract... iii Introduction... 1 MaterialsandMethods... 2 Results... 5 Thebesttoolforbenchmarking... 5 ThefastestOSonanIntelMac... 6 Thedifferencehardwaremakes.. 7 ComparisonofVistaandXP... 7 Discussionandconclusions... 8 FutureWork... 8 Acknowledgements... 8 References... 9 Tables Figures ii

3 ABSTRACT MacandPCPerformanceAnalysis.DAINKSCHROEDER(BigBendCommunityCollege, MosesLake,WA98837)CHARLIEEVERBOOM(LawrenceBerkeleyNationalLaboratory, Berkeley,CA94720). Overthecourseofthepastyear,thedesktopenvironmentatBerkeleyLabhasbeen impactedbyimprovementsinhardwareandchangesinoperatingsystemsoftware.the performanceofacomputerisbasedonacombinationoftheoperatingsystembeingused andthecomputer shardware.inordertoevaluatethecontributionsofthesedifferent factors,benchmarkingsoftwarewasusedonthesystems.thebenchmarkingproductsused weregeekbench,fromprimatelabs,cinebench,frommaxon,(cross platform)and PerformanceTest,fromPassMark(onlyavailableforWindows).Arecommendationfora singlelabbenchmarkproductwasnotmade,becauseeachoneemployeduniquecriteria, thusmakingitdifficulttoselecta best tool.experimentsweredoneonavarietyof commondesktopandlaptopsystems,fromapple,dell,andmpc.thesesystemsspanned therangeofprocessorsfromtheolderpentium4throughthelatestquadcorexeon processor.researchvalidatedmyexpectationthathardwaremakesasignificantdifference oncomputerperformance.theresearchshowedthatthefastestoperatingsystemonan IntelMacwasMacOSXLeopard,whichwasafull15%fasterthanWindowsVistaonthe samehardware.inaddition,windowsvistaisnotsignificantlyslowerthanxp,basedon theresultsoftheselectedbenchmarkingtoolsusedinthisresearch. iii

4 1.Introduction Thecomputerindustryisonethatisconstantlychanging;theleadingmanufacturers ofprocessors,intelandamd,releasemajorrevisionstotheirhardwareonaregularbasis. Inaddition,companiessuchasMicrosoftandAppleareconstantlyreleasingupdatesto theiroperatingsystems.theever changingnatureoftheindustrycanhaveaheavyimpact onbusinessdecisionsmadebycompanieswhostandardizetheirdesktopenvironments. OnJanuary30,2007,Microsoftunleashedtheirlatestoperatingsystem(OS), WindowsVista,tothepublic.Sincethen,LawrenceBerkeleyNationalLaboratory(LBNL) hasbeenworkingtoaddresstwomainissues.thefirstissueisthatwindowsxp, Microsoft smostpopularclientos,isnownolongerbeingsoldtoconsumers(thoughlbnl canstillpurchasedowngraderightstoxpwhenorderingasystemwithvista).inlightof this,lbnlmustdecidewhethertocontinueusingxpormaketheswitchtovista.another issuetoaddressisthegrowthofmacusageatthelab.forexample,thenewlyformedjoint BioEnergyInstitute,aswellastheMolecularFoundry.BothhavehigherthannormalMac adoption.thisresearchisbeingdonewithlawrenceberkeleylaboratory sinformation TechnologyDivision,intheMac/PCworkstationsupportgroup.WithMicrosoft smoveto Vista,LBNL(aswellasmanyothercompaniesandusers)aredebatingtheramificationsof upgradingtovistaorincreasingsupportforthemacintoshplatform. Tosomeusers,theOStheyuseisirrelevant.Thechoiceofoperatingsystemis drivenbytheneedforspecificresearchsoftwaretoolsandtheplatformsthosetoolsare deployedon.peoplehavedebatedtheissueofoperatingsystemsforyears everytimean OSisreleased,itisacertaintythattherewillbeamyriadofimportantquestionsaboutit.It isimportanttofindanswersforexactlythesekindsofquestions,becausetheycandirectly 1

5 influencethedecisionsofinformationtechnologydecisionmakers.specifically,iintendto focusonfindingtheanswerstofourdifferentquestions: Whatisthebest(ormost appropriate)benchmarkingtool?","whatisthefastestosonanintelmac?","howdoes hardwareimpactperformance?",andfinally"howdoeswindowsvistacompare (performance wise)towindowsxp? Asabi productofresearchrelatedtothesefour questions,thedifferencethatservicepacksmakeonwindowsoperatingsystemswasalso investigated. ThebenchmarkingproductsusedwereGeekBench,fromPrimateLabs,CineBench, frommaxon,(cross platform)andperformancetest,frompassmark(onlyavailablefor Windows).SeeTable1foradescriptionofwhateachtoolmeasures.Benchmarkingisa popularwaytomeasureacomputer sperformance[1].abenchmarkingprograminstructs acomputertoperformaspecificsetofpre definedtasksandmeasuresthetimeittakesto completethosetasks. 2.MaterialsandMethods Inthisresearch,Experimentsweredoneonavarietyofcommondesktopandlaptop systems,fromapple,dell,andmpc.thesesystemsspannedtherangeofprocessorsfrom theolderpentium4throughthelatestquadcorexeonprocessor.table2containsalistof thedesktopsused,whiletable3containsthelaptops.thesystemsweretestedusingfour differenttools.adescriptionofthetoolsusedisgivenbelow. GeekBench[2]:across platformbenchmarktoolthatmeasuresprocessorandmemory performance.ittestsfloating pointperformance,integerperformance,memory performance,andstreamingperformance. 2

6 CineBench[3]:across platformtoolthatrunsseveraltestsonasystemtomeasurethe capabilitiesofitsprocessor(s)andgraphicscard(s)inreal worldsituations.cinebench performstwomaintestsonasystem.inthefirsttest,arealisticimageisrenderedusing onlythemainprocessingcore,andthenusingallavailableprocessingcores.inthesecond test,a3danimationisplayedback,renderedinreal timeusingonlythegpu,inorderto testthecard'sabilitytorenderthesceneintheshortestpossibleamountoftime. PassMarkPerformanceTest[4]:aWindows onlytoolthatallowsyoutoperforma completebenchmarkonasystembyrunningawidevarietyoftestsandgivingtheoption tocomparetheresultstothoseofothercomputers.passmarkperformscputests,2d graphicstests,3dgraphicstests,disktests,andmemorytests,cd/dvdtests. OfficeBench[5]:thisisacompletelyversion independenttestingsuitethatperformsa largenumberofcommontaskswithinanymicrosoftofficeenvironment.itthenpresents theuserwithinformationregardinghowlongittooktocompletethesetoftasks. Inordertoconductthesebenchmarkingtests,someadditionaltoolswererequired. Thesetoolsincluded: OnyX:afreethird partyapplicationdesignedtodefragmentandoptimizemacintosh computers.macosxleoparddoesnotincludeabuilt indiskdefragmentationutility,butit isworthnotingthattheoperatingsystemdefragmentsitselfinthebackgroundaftermost fileoperations,aswellasautomaticallyat3:00ameachnight(assumingthesystemis runningatthattime).onyxwasusedintheseteststoforceadefragmentation,ensuring thateachsystemtestedwasoptimizedinasclosetothesamewayaspossible. BootCamp:afreeutilitymadebyApplethatcomesbuiltintoMacOSXLeopard.Itisused tocreateawindowspartitiononamacandassistintheprocessofcompletingawindows 3

7 installationonthesystem.italsogeneratesaspecialsetofdriversthatallowwindowsto functionproperlyonapplehardware. Beforebeginninganytesting,itwasimportanttocomeupwithastandardtest protocoltouse.thiswasdonetoensurethattherewerenooutsidefactorshavinganeffect onsystemperformance.theseprotocolsareoutlinedbelow. OnthePCplatform,twooperatingsystemswereused:WindowsXPandWindows Vista.FortheWindowsXPtests,bothServicePack2andthenewlyreleasedServicePack3 weretested.afreshinstallofxpwithservicepack2wasdone,andthesystemoptimized usingtwodifferentmethods.optimizationistheprocessofmakingsureasystemis runningideally.oncetheoperatingsystemswereinstalled,thesystemswererebooted threetimes.thisisdonebecause,accordingtomicrosoft[6],duringthefirstseveraltimes Windowsisbooted,theOSlooksatthesystemanddecideswhichfilesneedtobepreloaded,whichcanhavevariableeffectsonperformance.Thesecondoptimizationthatwas doneisasimplediskdefragmentation,performedtwice.thesystemswererebootedafter eachdefragmentation,foratotaloffiverebootsduringtheoptimizationprocess.the standardwindowsdiskdefragmentationtoolwasused. Afterperformingtheoptimizationsonthesystems,thebenchmarkingtoolstobe usedwereinstalledandrun.weexecutedthebenchmarksfivetimes,whichenabledusto gettheaverageofthescores.theresultswerethenobtainedandanalyzed.afterthis,the samesystemwasupgradedtoxpservicepack3,andtheprocesswasrepeated. FortheWindowsVistatests,theoriginalReleasetoManufacturing(RTM)version wastestedfirst.acleaninstallwasdone.afteroptimizingthesystem,thetoolswere installedandrun.theresultswerethenobtained.aftercataloguingtheresults,thesystem 4

8 wasupgradedtoservicepack1,andthesameprocessrepeated. FortheMacintoshtests,MacOSLeopard10.5.3wasinstalledonourtestsystems. ThesystemswerethenoptimizedusingOnyX,whichwasusedtodefragmentthesystem andverifythediskpermissions.thetoolsweretheninstalledandrun,andtheresults obtainedandcatalogued.forthetestsinvolvingrunningwindowsonamac,bootcamp wasused.theprocessofusingbootcampinvolvedcreatinga32gbwindowspartitionon themac'sharddrive,installingandoptimizingwindowsusingtheprocessdescribed above,andinstallingaspecialsetofdriversthatallowwindowstofunctioncorrectlyon Apple'shardware.Thetestswerethenrun,andtheresultscollected. 3.Results Besttoolforbenchmarking: Avarietyofdifferentbenchmarkingtoolswereused.Thecross platformprograms thatwereusedwerecinebenchandgeekbench.wealsousedpassmarkperformancetest, whichisverythoroughbutisnotcross platform.thetoolsthatwereusedeachhavetheir ownsetofdifferentparameterstoexamineforbenchmarking,asillustratedintable1, whichmakescomparingtheprogramsdifficult.forexample,whilecinebenchlooksonlyat graphicsandprocessing,geekbenchdoesnotexaminegraphicsperformanceatall. AnotherexampleisthatPassMarkandGeekBenchbothprovideanoverallratingofthe system,whilecinebenchdoesnot.however,forthepurposesofthesetests,theaverageof thethreeresultscinebenchdoesgivecanbefoundandusedasanaverage.fromthis research,weseethatpassmarkgivesanextremelydetailedresult,butitisimpossibleto objectivelystatethatitisabetterbenchmarkingtoolthangeekbenchorcinebench.see Table1foranaggregatelevelviewofwhateachprogramexamines. 5

9 Itisimportanttonotethatthesegenericbenchmarksdonotnecessarilyreflectwhat anindividualusermayexperience;forexample,ifsomeoneinterestedinplayinggames weretodependongeekbench,theymightbedisappointed(duetothefactthatitdoesnot measurevideoperformance).thatuserwould,instead,wanttouseatoollike3dmark, whichfocusessolelyonreal worldgamingbenchmarks.similarly,ifsomeonewantedto knowhowwellofficewouldrunontheirsystem,they'dwanttouseatoollikeofficebench, whichexaminesreal worldofficeworkloadsonasystem. WhichoperatingsystemisfasteronanIntelMac? Inthisresearch,itwasonlypossibletotesttheMacOSontrueApplehardware.This isbecausewindowsrunsonmacintosh basedcomputers,butwindows basedcomputers willnotrunmacosx.themacosisthefastestoperatingsysteminthesetests(onanew MacBook),asisevidencedbyFigure1.Figures2and3demonstratethatGeekBenchand CineBench(respectively)indicatethatthesameistrueonanewMacBookPro.Our GeekBenchresultsthatWindowsVistaisapproximately15%slowerthanLeopard,while XPisabout14%slowerthanLeopard.ThisresearchhasclearlydemonstratedthatMacOS XLeopardrunsnoticeablyfasteronaMacthanWindowsXPorWindowsVistado.These resultsalsoseemtoconfirmthatvistaisnotmuchslowerthanwindowsxp(whichis furtherdiscussedlaterinthispaper). Aninterestingobservationwemadeisthatourresearchresultsseemtoindicate thatwindowsxpandvistamayperformbetteronapplehardwarethanonanearly identical(specification wise)windows basedmachine.however,duetotimeconstraints, wewereunabletopursuethisfurther. 6

10 Howmuchdifferencedoesthehardwareplatformmake? Sinceanumberofdifferenthardwareplatformsweretestedinthisresearch,apart ofthestudywastodeterminehowmuchdifferencehardwaremakesonperformance.asis demonstratedinfigure4,performanceisproportionaltothehardware;ashigher end hardwareisintroduced,performanceincreases.thistrendcanalsobeseeninfigure5, wherewenotethattheprecisionworkstationhasmorethandoublethescoresofthe Optiplexline.WealsocomparedaMacBookandLatitudeD630,whichhaveidentical specifications(exceptforthelatituded630'sdedicatedvideocard,whichcanbeignored sincegeekbenchdoesnotmeasurevideoperformance).theresultsofthesetestscanbe seeninfigure4.similarly,anoptiplex745andanoptiplex755,whichisnewerbyone year,werecomparedinfigure6.itisinterestingtonotethatthedifferenceinprocessor speedsbetweentheoptiplex745and755makesasignificantdifferenceinperformance. IsthereadifferencebetweenVistaandXP? Inallofthetestsdoneforthisresearch,VistawasnotsignificantlydifferentfromXP inspeed.quiteinterestingly,theinstallationofvista'sfirstservicepackdoesnotseemto makeasignificantdifferenceinperformance,thoughmicrosoftclaimsthatperformance boostsareakeyfeatureoftheupdate.vistaiscertainlyslowerthanxp,butaccordingto ourpassmarkresults(seefigure7)thedifferenceissmallenoughthatacommonuser wouldnotevennoticethedifference.asfigure8indicates,geekbenchagreeswith PassMark'sresults.Afterexaminingtheseresultsthoroughly,Ifeelitissafetosaythat WindowsVistaisnotsignificantlyslowerthanXP,despitewhatthemediawouldhaveus believe.itisimportanttonote,however,thatthesegenericbenchmarkingtoolsdonot necessarilyreflectwhatausermayexperienceinreal worldsituations. 7

11 4.DiscussionandConclusions Duetotheresultsofmytests,IcansaywithsomecertaintythattheMacOSisfaster thanwindows,atleastonthesamehardware.itcanalsobesaidthat,judgingbytheresults givenbyeachprogramiused,windowsvistaisnotsignificantlyslowerthanwindowsxp. FortestsdonesolelyonWindows basedsystems,thebestbenchmarkingtoolis probablypassmark,duetothethoroughnessofitsresults.fortestingthatrequiresacrossplatformtool,i'dadvisetheusageofbothgeekbenchandcinebench;geekbenchprovides anexcellentlookatprocessorandmemoryperformance,andcinebenchgivesadetailed lookatgraphicscapabilities. 5.FutureWork Duetotheever changingnatureofcomputerhardware,benchmarkingtestshave becomeaprocessthatmustberepeatedeverytimeahardwarerevisionisreleasedtothe public.softwareplaysanequallyimportantrole,however;inthenearfuture,microsoft planstoreleasewindows7,andappleisslatedtoreleasemacos10.6"snowleopard". Whenthesenewoperatingsystemrevisionsarereleasedtothepublic,performing benchmarkingtestsonthemwillbenecessarybeforelbnlcanconsiderupgradingto eitherofthem. 6.Acknowledgments This research was completed at Lawrence Berkeley National Laboratory. I would like to thank Charles Verboom, my mentor, and Zachariah Tanko, my FaST team member, for their assistance and support. I would also like to thank the Office of Science, Lawrence Berkeley National Laboratory, CSEE, and the FaST program for allowing me to participate in this 8

12 internship. 7.References [1] MacVSPC:TheUltimateLabTestfornewDesktopsandLaptops PopularMechanics [2]"PrimateLabsGeekBench"PrimateLabs [3]"MAXONCineBench"MAXON [4]"PassMarkPerformanceTest"PassMark [5]"OfficeBench"exo.performance.network [6]"BenchmarkingonWindowsXP:HomeEditionandProfessional"Microsoft 9

13 TABLES Table1 ComparisonofBenchmarkingTools DellOptiplex745 DellOptiplex755 DellPrecision5400 MPCClientPro365 Cost $887 $823 $2629 $1594 CPU RAM GPU Core2Duo,2.13 GHZ 2GB Table2 Listofdesktops RadeonHD2400 Pro 256MB CPU:Core2Duo,3.0GHZ QuadCoreXeon,2.5GHZ Pentium4,3.4GHZ 2GB 8GB 512MB RadeonHD2400XT 256 MB QuadroFX MB GeForcePCX MB HDD 148GB 250GB 160GB 100GB 10

14 AppleMacBook AppleMacBookPro DellLatitude630 Cost $1401 $2524 $1335 CPU Core2Duo,2.2 GHZ Core2Duo,2.5GHZ Core2Duo,2.2GHZ RAM 2GB 4GB 2GB GPU IntelGMAX3100 (shared) GeForce8600MGT 512 MB QuadroNVS135M 128MB HDD 150GB 200GB 120GB Table3 ListofLaptops 11

15 FIGURES Figure1 Comparisonof32bitOperatingSystemsonApplehardware 12

16 Figure2 DetailedGeekBenchscoresonMacBookPro Figure3 DetailedCineBenchscoresonMacBookPro 13

17 Figure4 WindowsXPwithSP3ondifferenthardware(GeekBenchoverallscores) Figure5 ComparisonbetweenOptiplex745andPrecisiondesktops(PassMarkoveralls) 14

18 Comparison of Optiplex 745 and 755 systems in GeekBench Optiplex 745 Overall Integer Floating Point Memory Stream Optiplex 755 Overall Integer Floating Point Memory Stream XP SP XP SP Vista RTM Vista SP Figure6 ComparisonofOptiplex745and755inGeekBench OpIplex745:PassMarkRaIngs XPSP2 XPSP3 VistaRTM VistaSP1 LongerbarsrepresentbeEerperformance. Figure7 ComparisonofWindowsXPandVistaonOptiplex745inPassMark 15

19 OpIplex745 overallgeekbenchscores XPwithSP2 XPwithSP3 VistaRTM VistawithSP1 Higherbarsindicatefasterperformance Figure8 ComparisonofWindowsXPandVistaonOptiplex745inGeekBench 16