Joint Exploration of Hardware Prefetching and Bandwidth Partitioning in Chip Multiprocessors

Transcription

1 Joint Explortion o Hrdwre Preetching nd Bndwidth Prtitioning in Chip Multiprocessors Fng Liu liu3@ece.ncsu.edu Yn Solihin solihin@ece.ncsu.edu Deprtment o Electricl nd Computer Engineering North Crolin Stte University Rleigh, NC, USA ABSTRACT In this pper, we propose n nlyticl model-bsed study to investigte how hrdwre preetching nd memory bndwidth prtitioning impct Chip Multi-Processors (CMP) system perormnce nd how they interct. The model includes composite preetching metric tht cn help determine under which conditions preetching cn improve system perormnce, bndwidth prtitioning model tht tkes into ccount preetching eects, nd derives weighted speedup optimum bndwidth prtition sizes or dierent cores. Simultion results show tht selective preetching, guided by the composite preetching metric, coupled with dynmic bndwidth prtitioning cn improve system perormnce by 9.5% nd 2.8% on dul-core CMP nd qud-core CMP respectively.. INTRODUCTION Severl decdes o the persistent gp between microprocessor nd DRAM min memory speed improvement hs mde it impertive or tody s microprocessors to hide hundreds o processor clock cycles in memory ccess ltency. In most high perormnce microprocessors tody, this is chieved by employing severl levels o cches ugmented with hrdwre preetching [3, 8]. Hrdwre preetching works by predicting cche blocks tht re likely needed by the processor in the ner uture, nd etching them into the cche erly, llowing processor s memory ccesses to hit (i.e. ind dt) in the cche. Since preetching relies on prediction o uture memory ccesses, it is not 00% ccurte in its prediction, implying tht the ltency hiding beneit o preetching cuses n increse in o-chip memory bndwidth usge nd cche pollution. Unortuntely, the shit to multi-core design hs signiicntly incresed the pressure on the o-chip bndwidth. Moore s Lw continues to llow the doubling o trnsistors (cores) every two yers, incresing the o-chip bndwidth tric t Moore s Lw speed. However, the vilbility o o-chip bndwidth is only projected to grow t 5% nnully, due to the limittions in pin density nd power consumption [9]. This leds to problem clled the bndwidth wll, where system perormnce is incresingly limited by the vilbility o o-chip bndwidth [2]. Hence, the demnd or eicient use o o-chip bndwidth is tremendous nd incresing. Two methods hve been recently proposed to improve the eiciency o o-chip bndwidth usge. One method is to improve preetching policies. Some studies proposed throttling or eliminting the useless/bd preetches rom consuming bndwidth [, 8], nd tweking the memory scheduling policy to prioritize demnd nd proitble preetch requests [4]. Another method is to prtition the o-chip bndwidth mong cores [5, 7, 4, 3, 0, 7], by choosing prtition sizes to optimize system throughput or irness. However, these studies suer rom severl drwbcks. First, the studies ddress one technique but ignore the other: preetching studies do not include bndwidth prtitioning, wheres bndwidth prtitioning studies ssume systems tht hve no preetching. As result, the signiicnt interction between them ws missed, nd the opportunity or these techniques to work in synergy ws let unexplored. Second, the studies were perormed in n d-hoc mnner, yielding perormnce improvement but missing importnt insights. Weighted Speedup Bndwidth Shres Neither Pre PT Pre+PT str-gobmk mc-bzip2 hmmer-milc () core 2 (c2) core (c) PT Pre+PT PT Pre+PT PT Pre+PT str(c)-gobmk(c2) mc(c)-bzip2(c2) hmmer(c)-milc(c2) (b) Figure : Weighted speedup () nd optimum bndwidth lloction (b) or co-schedules running on dul-core CMP. To demonstrte the need or exploring preetching nd bndwidth prtitioning jointly, Figure () shows how preetching nd bndwidth prtitioning cn ect ech other. In the igure, system perormnce mesured s weighted speedup [] (reerring to Eqution ) o three pirs o SPEC2006 [9] benchmrks running s co-schedule on dul-core CMP system is shown with our conigurtions: bse system with no preetching or bndwidth prtitioning (Neither), hrdwre preetching only (Pre), bndwidth prtitioning only (PT), nd both preetching nd bndwidth prtitioning (Pre+PT). The three co-schedules highlight dierent interction cses. For hmmer-milc, preetching improves perormnce nd bndwidth prtitioning improves it urther due to osetting the eect o the preetcher s increse in o-chip bndwidth usge. For str-gobmk nd mc-bzip2, preetching hurts perormnce due to high o-chip bndwidth consumption. For mc-bzip2, pplying bndwidth prtitioning is not the right solution, since it cnnot recover the lost perormnce due to preetching (i.e. Pre+PT < Nei- A strem preetcher [3, 8] is used. Optimum bndwidth prtitioning scheme rom [7] is used. More detils cn be ound in Section 4.

2 ther). Such conclusion cnnot be derived in prior studies, when only bndwidth prtitioning ws studied [5, 7, 4, 3, 0, 7], or when preetchers were lwys turned on [4]. The igure points out the need to understnd when the preetcher o ech core should be turned on or o, nd how bndwidth prtitioning should be implemented in order to optimize system perormnce. Perorming the studies in n d-hoc mnner oten misses importnt insights. For exmple, it hs been ssumed tht core tht enjoys useul preetches should be rewrded with higher bndwidth lloction, wheres core or which preetching is less useul should be constrined with lower bndwidth lloction [4]. However, our experiments show the opposite. Mximum weighted speedup is chieved when core with highly useul preetching is given less bndwidth lloction, nd the resulting excess bndwidth is given to cores with less useul preetching. Figure (b) shows the bndwidth lloction or ech core tht mximizes the system throughput. The ppliction tht hs more useul preetching (higher preetching coverge nd ccurcy) runs on Core. Compring the two brs or ech co-schedule, it is cler tht in order to mximize system throughput, it is the pplictions tht show less useul preetching should receive higher bndwidth lloctions. The gol o this study is to understnd wht ctors contribute to the impct o preetching nd bndwidth prtitioning on CMP system perormnce, nd how they interct. We propose n nlyticl model-bsed study, bsed on the Cycle Per Instruction (CPI) model [6] nd queuing theory [2], tking vrious system prmeters (CPU requency, cche block size, nd vilble bndwidth), nd ppliction cche behvior metrics (miss requency, preetching requency, preetching coverge nd ccurcy, nd vrious CPIs) s input. Studying the model, coupled with empiricl evlution, we rrive t severl interesting indings, mong them re: Deploying preetching mkes the vilble bndwidth scrcer nd thereore increses the eectiveness o bndwidth prtitioning in improving system perormnce. The decision o turning preetchers on or o should be mde prior to employing bndwidth prtitioning, becuse the perormnce loss due to preetching in bndwidth-constrined systems cnnot be ully repired by bndwidth prtitioning. We discover new metric to decide whether to turn on or o the preetcher or ech core. The theoreticl optimum bndwidth lloctions cn be derived, nd simpliied version tht is implementble in hrdwre cn pproximte it quite well. The conventionl wisdom tht rewrds core tht hs more useul preetching with lrger bndwidth lloction is incorrect when preetchers o ll cores re turned on. Insted, its bndwidth lloction should be slightly more constrined. Our model explins why this is so. The indings in our study crry signiicnt implictions or CMP design nd perormnce optimiztion. As the number o cores on chip continues to double every 8-24 months, versus 0-5% ITRS projected growth in o-chip bndwidth [9], the o-chip bndwidth vilble per core will become incresingly scrcer, incresing the demnd or better o-chip bndwidth mngement. The reminder is orgnized s ollows: Section 2 reviews the relted work, Section 3 shows the construction o the nlyticl model, Section 4 vlidtes the model-driven indings through simultion bsed evlution, nd Section 5 concludes this pper. 2. RELATED WORK Bndwidth prtitioning. Reserchers hve proposed vrious bndwidth prtitioning techniques to mitigte CMP memory bndwidth contention-relted problems. The impliction is to prioritize memory requests rom dierent cores bsed on heuristics, in order to meet the objectives o Qulity o Service (QoS) [0], irness [4, 3], or throughput [5]. Liu et l. [7] investigted the impct o bndwidth prtitioning on CMP system perormnce nd its interction with shred cche prtitioning. Srikntih nd Kndemir [7] explored symbiotic prtitioning scheme on the shred cche nd o-chip bndwidth vi empiricl models. None o those bndwidth prtitioning techniques took preetching into ccount. Preetching. Hrdwre preetching is widely implemented in commercil microprocessors [3, 8] to hide the long memory ccess ltency. Due to the imperect ccurcy, preetching incurs cche pollution nd increses o-chip bndwidth consumption. Prior studies hve looked into how to mke preetching more eective in CMP. Techniques include throttling/iltering useless preetching requests [, 8] to reduce their extr bndwidth consumption. All o the bove studies ignore bndwidth contention tht rises rom demnd nd preetch requests coming rom dierent cores. Preetching nd Bndwidth Prtitioning. Only recently, preetching nd bndwidth prtitioning in CMP were studied together s inter-relted problems. Ebrhimi et l. [4] proposed to prtition bndwidth usge in the presence o preetching in CMP in order to reduce the inter-core intererence. While their results show improved system perormnce, mny questions re let unnswered. For exmple, wht undmentl ctors ect the eectiveness o preetching nd bndwidth prtitioning? How do they interct with ech other? Does combining preetching nd bndwidth prtitioning lwys chieve better perormnce thn using only one o them? Wht bndwidth shres cn produce optimum system perormnce? The gol o this pper is to ind out these nswers, tht re criticl or designing good policy or optimizing system perormnce. Due to the lck o understnding on these issues, the study [4] mde criticl error o lwys keeping the preetching engines o ll cores on regrdless o the situtions. Figure shows tht in some cses, it is better to turn the preetchers completely o. 3. ANALYTICAL MODELING 3. Assumptions nd Model Prmeters Assumptions. This study ssumes CMP with homogeneous cores, where ech core hs privte L (instruction nd dt) nd L2 (uniied) cches. Ech L2 cche hs hrdwre preetcher tht cn be turned on or o. The o-chip bndwidth is shred by ll cores through single queue interce, where ll o-chip memory requests re served in First-Redy First-Come-First-Serve (FR-FCFS) policy [6]. Our study ocuses on multi-progrmmed worklods, hence we ssume tht single-threded pplictions run on dierent cores nd do not shre dt. This mens the coherence tric is ignored. The write bcks re not modeled in the system, becuse they re much ewer thn red/write misses nd not on the criticl pth o perormnce. We deine bndwidth prtitioning s llocting rctions o ochip bndwidth to dierent cores nd enorcing these rctions s per-core quot. Thus, bndwidth prtitioning reduces inter-core intererence, without chnging the underlying memory ccess scheduling policy. The bndwidth prtitioning is implemented using token bucket lgorithm [7], with the gol to optimize the system throughput expressed s weighted speedup, which is the sum o individul Instruction Per Cycle (IPC) speedup rom ech core [] 2 : NX IP C i NX W S = = IP C lone,i i= i= CP I lone,i CP I i () 2 IP C i nd IP C lone,i re IPC o thred i when it runs in co-schedule nd when it runs lone in the system, respectively. Weighted speedup is widely used metric, nd includes some irness, s speeding up one core t the expense o others.

3 Model Prmeters. The input to the model includes system prmeters s well s per-thred prmeters listed in Tble. Tble : Input nd prmeters used in our model. System prmeters CPU clock requency (Hz) K Cche block size (Bytes) B Pek o-chip memory bndwidth (Bytes/sec) N Number o cores in the CMP Thred-speciic prmeters CP I L2,i Thred i s CPI ssuming ininite L2 cche size CP I lone,i Thred i s CPI when it runs lone in the CMP M i Thred i s L2 cche miss rte without preetching A i Thred i s L2 cche ccess requency (#ccesses/sec) M p,i Thred i s L2 cche miss rte ter preetching P i Thred i s L2 preetching rte (#preetches/#ccesses) c i Thred i s preetching coverge i Thred i s preetching ccurcy T m,i Thred i s verge memory ccess ltency (#cycles) β i Frction o bndwidth ssigned to thred i 3.2 Composite Metric or Preetching Trditionl metrics or preetching perormnce include coverge, deined s the rction o the originl cche misses tht re eliminted by preetching; nd ccurcy, deined s the rction o preetch requests tht successully eliminte cche misses. These metrics cnnot determine whether preetching should be used or not in limited o-chip bndwidth environment becuse they do not tke into ccount how much o-chip bndwidth is vilble, nd how memory ccess ltency is ected by preetching. To rrive t new metric, we strt rom the bsic Cycle per Instruction (CPI) model [6] nd dd the eect o dditionl queueing dely due to preetching tric. We use t to represent the queueing dely on the bus nd let T m be the verge ccess ltency to the memory 3, the CPI becomes: CP I = CP I L2 + h m (T m + t) (2) When preetching is pplied in the system, let CP I p present the new CPI, t p s the new queuing dely on the bus, h m,p s the new L2 misses per instruction, the CPI eqution then becomes: CP I p = CP I L2 + h m,p (T m + t p) (3) Compring Eqution 3 vs. Eqution 2, preetching my decrese the L2 miss per instruction, i.e. h m,p < h m, but increse the queueing dely, i.e. t p > t due to the extr tric generted by preetch requests. h m,p cn be expressed s the multipliction o miss requency (M pa p) nd the verge time tken to execute one instruction ( h m,p = Mp Ap CP Ip CP Ip ): Substituting Eqution 4 into Eqution 3, nd solving CP I p, we cn rrive t: CP I p = CP I L2 MpAp (T m + t p) Now we will derive t p using Little s lw. I we let λ p denote the rrivl rte o memory requests, then λ p is equl to the sum o requency o cche miss nd preetch requests: (4) (5) λ p = (M p + P )A p (6) 3 The verge memory ccess ltency T m is n mortized vlue tht implicitly includes the eect o memory bnk nd row buer conlicts, Instruction Level Prllelism (ILP) s well s Thred Level Prllelism (TLP). Little s lw [2] or queuing system sttes tht the verge queue length (L p) is equl to the rrivl rte (λ p) multiplied by the service time (T p), i.e. L p = λ p T p, while the service time on the o-chip bus is the cche block size (K) divided by the vilble bndwidth (B). Hence, the verge number o memory requests rriving during the service time is: L p = K (Mp + P )Ap (7) B I we ssume memory requests re not bursty, i.e. requests re processed bck to bck, the verge witing time o newly rriving request is equl to the L p requests tht re hed o it in the queue, multiplied by the service time. Thus, the witing time t p (in cycles) cn be expressed s: t p = K B K2 Lp = (Mp + P )Ap (8) B2 Substituting Eqution 8 into Eqution 5, expression or CP I p is: CP I p = MpApTm CP I L2 Mp(Mp+P )A2 p K2 (9) Similrly, the system without preetching hs the CPI o: CP I L2 CP I = (0) MATm M 2 A 2 K 2 In order or preetching to produce net beneit in perormnce, the CPI ter preetching must be smller thn CPI without preetching, i.e. CP I p < CP I. From the deinition o preetching coverge nd ccurcy, we hve (ssuming L2 cche ccess requency is not ected by preetching, i.e. A p = A): c = = MA MpAp MA MA MpAp P A p = M Mp M = M Mp P () (2) Rerrnging Eqution nd Eqution 2 to express M p nd P in terms o c nd, substituting them into the CP I p expression in Eqution 9, nd simpliying the inequlity CP I p < CP I, we obtin the inl expression or the inequlity: T m > MAK2 (c 2 + c ) (3) The right hnd side o the inequlity, MAK2 (c 2+ c ), is the composite metric tht tkes into ccount preetching coverge nd ccurcy, s well s cche block size, vilble bndwidth, nd miss requency. The let hnd side o the inequlity, T m, mesures the verge exposed memory ccess ltency. The inequlity essentilly provides brek-even point threshold or how lrge the verge memory ccess ltency should be or preetching to be beneicil. It is esier to meet the inequlity when the verge memory ccess ltency is lrge, the cche miss requency is smll, the vilble bndwidth is lrge, nd the coverge nd ccurcy re lrge. All these ctors mke sense qulittively, nd Eqution 3 cptures their quntittive contributions. This leds us to: OBSERVATION. Whether preetching improves or degrdes perormnce cnnot be mesured just by its coverge or ccurcy. Rther, preetching proitbility criterion in Inequlity 3 is needed, using input prmeters tht re esy to collect in hrdwre. Recll tht in the model construction, we ssumed memory requests re not bursty nd requests re processed bck to bck. This llows us to rech bndwidth utiliztion o 00% i the rrivl rte o memory requests is high. However, memory requests re bursty

4 in relity, even high rrivl rte cnnot chieve 00% utiliztion. This introduces smll inccurcy in our metric. Thus we cn deine prmeter α (0, ) such tht: T m > MAK2 (c 2 + c α ) (4) α loosely represents the degree o burstiness o memory requests nd cn be obtined rom empiricl evlution (s shown in Section 4.2.). Since α <, the minimum vlue o the right hnd side o the inequlity is MAK2 T m < MAK2 (c 2 + c ). Thus we cn conclude: OBSERVATION 2. I n ppliction running on core stisies (c 2 + c ), preetching is hrmul to peror- mnce. In ddition, i 0 > MAK2 B improves perormnce. 2 (c 2 + c ), preetching 3.3 Memory Bndwidth Prtitioning Model This model extends the bndwidth prtitioning model rom [7] by tking into ccount not just demnd etches, but lso preetch requests, nd preetching coverge nd ccurcy. In Eqution 5, we cn compute the queueing dely t p or thred i using Little s lw or the cse where we do not pply bndwidth prtitioning (i.e. requests rom ll cores contend with ech other or bndwidth ccess nturlly) nd compre it ginst the cse where we pply bndwidth prtitioning (i.e. bndwidth rction is llocted to ech core or its own requests). The CPI o thred i in system with nturl shring (no prtitioning) nd with bndwidth prtitioning cn be expressed s: CP I p,i,nopt = CP I p,i,bwpt = CP I L2,i M p,ia i T m,i (P N j= (M p,j +P j )A j) 2 M p,i K 2 (M p,i +P i ) (5) M p,ia i T m,i CP I L2,i M p,i(m p,i +P i )A 2 i K2 β 2 i B2 (6) where β i denote the rction o bndwidth llocted to thred i. Compring Eqution 6 nd 5, we cn conclude tht Eqution 5 is specil cse o Eqution 6 where: β i = which leds us to the ollowing observtion: (M p,i + P i)a i P N j= (Mp,j + Pj)Aj (7) OBSERVATION 3. In CMP system where o-chip bndwidth usge mong multiple cores is unregulted, the o-chip bndwidth is nturlly prtitioned between cores, where the nturl shre o bndwidth core uses is equl to the rtio o memory request requency (including both cche misses nd preetch requests) o the core to the sum o ll memory request requencies rom ll cores. Substituting CPI rom Eqution 6 into Eqution, with constrint o P N i=0 βi =, we cn mximize the weighted speedup using Lgrnge multipliers [7]. The solution provides the bndwidth prtition or thred i (β i) is: `CiM p,i(m p,i + P i)a 2 /3 i β i = (8) `CjM p,j(m p,j + P j)a 2 j /3 P N j= where C i = CP I lone,i CP I L2,i. This leds us to the next observtion: OBSERVATION 4. Weighted speedup-optimum bndwidth prtition or thred cn be expressed s unction o ll co-scheduled threds miss nd preetch requencies, ininite-l2 CPIs, nd CPIs when ech thred runs lone in the system, s in Eqution Prmeters in Eqution 8 my vry due to n ppliction s phse chnge 4. EMPIRICAL EVALUATION 4. Environment nd Methodology Simultion Prmeters. We use cycle-ccurte ull system simultor bsed on Simics [5] to model CMP system with dul nd qud cores. Ech core hs sclr in-order issue pipeline with 4GHz clock requency. To remove the eect o cche contention between cores, ech core hs privte L nd L2 cches. The L instruction cche nd dt cches re 6KB, 2-wy ssocitive, nd hve 2-cycle ccess ltency. The L2 cche is 52KB, 8-wy ssocitive, nd hs n 8-cycle ccess ltency. All cches use 64- byte block size, implement write-bck policy, nd LRU replcement. The bus to o-chip memory is split trnsction bus, with pek bndwidth o 800MB/s or single core system,.6gb/s or dul-core CMP nd 3.2GB/s or qud-core CMP. The verge min memory ccess ltency is 300 cycles. The CMP runs Linux OS tht comes with Fedor Core 4 distribution. Hrdwre Preetchers. The L2 cche o ech core is ugmented with typicl strem preetcher tht is commonly implemented in modern processors [3, 8] due to their low hrdwre cost nd high eectiveness or wide rnge o pplictions. Strem buers cn detect ccesses to block ddresses tht orm sequentil or stride pttern (clled strem), nd preetch the next ew blocks in the strem in nticiption o continuing ccesses rom the strem. We implement strem preetcher with our strems, nd up to our blocks preetched or ech strem coverge ccurcy str bzip2 gcc gobmk hmmer lbm libquntum mc milc nmd omnetpp perlbench povry sjeng soplex sphinx3 Figure 2: Benchmrks preetch coverge nd ccurcy. Worklod Construction. We consider seventeen C/C++ benchmrks rom the SPEC2006 benchmrk suite [9]. We compile the benchmrks using gcc compiler with O optimiztion level into x86 binries. We use the re input sets, simulte 250 million instructions ter skipping the initiliztion phse o ech benchmrk, by inserting brekpoints when mjor dt structures hve been initilized. We pir the benchmrks into co-schedules o two or our benchmrks. To reduce the number o co-schedules tht we need to evlute, we ctegorize the benchmrks bsed on the preetching chrcteristics. Figure 2 shows the preetch coverge nd ccurcy o ech benchmrk. From the igure, we pick up three representtive benchmrks tht hve low (bzip2), medium (str) nd high (hmmer) coverge nd ccurcy, nd pir ech one with the other benchmrks to cover ll representtive co-schedules. 4.2 Experimentl Results 4.2. Vlidting the Preetching Metric In this section, we investigte how well the composite preetching metric in Eqution 4 predicts preetching perormnce proitbility. We run ech ppliction on single core system with behvior. The eqution does not ssume them to be constnt, but ssumes the prmeters or ny speciic execution intervl re vilble.

5 vilble bndwidth o 800MB/s, nd mesure CPIs when preetching is turned on (CP I p) nd when preetching is turned o (CP I). Then we compute the dierence ( CP I = CP I p CP I) or ech benchmrk. A negtive number represents perormnce improvement while positive vlue represents degrdtion. Tble 2 shows the pplictions sorted by their CP Is. As comprison, we compute our composite preetching metric θ = MAK2 (c ) nd sort the pplictions in decresing order o θ. Finlly, conventionl wisdom evlutes preetching perormnce bsed on coverge or ccurcy, so we lso show the pplictions sorted in incresing order o ccurcy. Note tht Figure 2 shows tht coverge nd ccurcy re highly correlted, hence we only show ccurcy in the tble. Applictions whose rnks mtch with ones obtined rom CP I sorting re shown in bold. 2+ c Tble 2: Rnking o pplictions bsed on CP I, our composite metric θ, nd preetching ccurcy. CP I-sorted θ-sorted -sorted Benchmrk CP I Benchmrk θ Benchmrk bzip bzip bzip2 soplex soplex 48.6 gobmk mc mc gcc gobmk 0.28 gobmk soplex gcc gcc mc povry str povry nmd milc 0.40 str omnetpp povry 0.83 milc str -0.0 nmd -.47 sphinx3 milc omnetpp nmd perlbench perlbench perlbench hmmer hmmer -43. omnetpp sphinx libquntum lbm libquntum lbm hmmer lbm sphinx libquntum sjeng sjeng sjeng Let us compre the rnks o pplictions bsed on the ctul perormnce improvement due to preetching ( CP I), vs. bsed on our metric θ. The rnks o eight pplictions (shown in bold) out o sixteen exctly mtch. For the other eight pplictions, their rnks dier by t most three positions, indicting how well our metric θ correltes with the ctul perormnce. I ech ppliction is given rnk number, nd we compre rnks bsed on CP I vs. θ, the correltion coeicient computes to 95%, indicting high correltion between them. In contrst, the correltion o rnks bsed on CP I vs. ccurcy computes to 89%. Thereore, our composite metric correltes better with the ctul perormnce thn conventionl metrics. In ddition, conventionl metrics cnnot determine wht level o ccurcy or coverge is high enough to produce perormnce improvement, wheres our composite metric θ includes perormnce proitbility threshold (Eqution 4): T m > θ, where α (0, ) is α number tht relects how bursty memory ccesses re. Even when α is unknown, θ cn still determine preetching proitbility unmbiguously in some cses. For exmple, θ = or bzip2, which is much lrger thn the ully exposed memory ccess ltency T m = 300 cycles, hence θ predicts unmbiguously tht preetching will hurt perormnce (Observtion 2). In ddition, there re eight pplictions showing negtive θ vlues in Tble 2. θ predicts unmbiguously tht preetching will improve perormnce. Only or seven out o sixteen cses, the ctul vlue o α is needed to determine whether preetching is proitble. I we choose α = 0.2, ll benchmrks rnked bove str stisy T m θ, while ll benchmrks rnked t str nd below stisy α T m > θ. Thus, 0.2 provides resonble estimte or α. For α urther vlidtion, we run simultions on single core with dierent preetch lgorithms nd dierent bndwidth mount, ll results point to the sme conclusion tht α = 0.2 is good estimte or SPEC2006 benchmrks Co-Deciding Preetching nd Prtitioning In this section, we show results where our preetching proitbility metric in Inequlity 4 guides the decision to selectively turn on/o preetchers, coupled with dynmic bndwidth prtitioning. The bndwidth prtitioning is enorced using token bucket lgorithm [7], where token genertor distributes tokens to dierent per-core buckets t the rtes proportionl to the rction o bndwidth llocted to dierent cores. Ech cche miss or preetch request is llowed to go to the o-chip interce only when the core generting the request hs mtching tokens in its bucket. In the dynmic prtitioning scheme, the bndwidth shres re re-evluted every -million cycles. As in [7], we deploy simpliied prcticl implementtion by ssuming C i = in Eqution 8. Figure 3 shows the weighted speedup or our system conigurtions CMP system: no preetching or bndwidth prtitioning (Neither), only simpliied bndwidth prtitioning (Simp), ll preetchers turned on nd bndwidth prtitioning (AllPre+Simp), nd selective ctivtion o ech core s preetcher guided by preetching proitbility metric coupled with bndwidth prtitioning (Ctrl- Pre+Simp). Speciiclly, we ssume α = 0.2 s discussed in Section To mke preetching decision, we irst run the pplictions on CMP system with equl shre o the vilble bndwidth or ech core, nd proile the hrdwre counters needed or computing θ. Bsed on the computed θ, we stticlly turn on/o the preetcher o core, nd run the pplictions gin to collect perormnce. Note tht nothing prevents the preetching decision to be perormed dynmiclly t run time. However, mking preetching decision stticlly is suicient to demonstrte the eectiveness o the preetching proitbility metric. The experiments re evluted on dul-core CMP with vilble bndwidth o.6gb/s nd on qud-core CMP with vilble bndwidth o 3.2GB/s respectively. The igure shows tht in mny cses, turning on ll preetchers (AllPre+Simp) degrdes perormnce over no preetching (Simp) (i.e. 8 out o 6 co-schedules on dul core CMP, nd in 3 out o 0 co-schedules on qud-core CMP). Worse, perormnce o All- Pre+Simp is even lower thn the bse cse CMP without preetching or bndwidth prtitioning (Neither) in mjority o the bove cses. This indictes tht preetching cn be too hrmul or bndwidth prtitioning to oset. In both igures, when we use our preetching proitbility criterion to guide the decision to turn on/o ech core s preetcher, coupled with dynmic bndwidth prtitioning, the best weighted speedup cn be chieved or ll co-schedules. CtrlPre+Simp hs t lest the sme perormnce s the best o Simp nd AllPre+Simp, nd in mny cses outperorms both o them signiicntly. On verge, over Neither, Simp, nd AllPre+Simp, CtrlPre+Simp improves weighted speedup by 6.0%, 3.6% nd 9.5%, respectively on dul-core CMP, nd 7.2%, 2.5% nd 2.8%, respectively on qud-core CMP. 5. CONCLUSIONS The gol o this pper is to understnd how hrdwre preetching nd memory bndwidth prtitioning in CMP cn ect system perormnce nd interct with ech other. We hve presented n nlyticl model to chieve this gol. Firstly, we derived composite preetching metric tht cn help determine under which situtions hrdwre preetcher cn improve system perormnce. Then we constructed bndwidth prtitioning model tht cn derive weighted speedup-optimum bndwidth lloctions mong dierent cores. Finlly, we collected simultion results to vlidte the observtions obtined rom the nlyticl model, nd show system perormnce improvement tht cn be chieved by co-deciding preetching nd bndwidth prtitioning decisions using our preetching metric nd implementble dynmic bndwidth prtitioning.

6 ighted Speedup We Neither Simp AllPre+Simp CtrlPre+Simp Weighted Sp peedup () Weighted Speedup on Dul-core CMP Neither Simp AllPre+Simp CtrlPre+Simp (b) Weighted Speedup on Qud-core CMP Figure 3: Weighted speedup o optimum bndwidth prtitioning or no preetching, ll preetching, vs. selectively turning on preetchers using the composite preetching metric on dul-core CMP with.6gb/s bndwidth (), nd qud-core CMP with 3.2GB/s bndwidth (b). 6. REFERENCES [] A. Snvely et l. Symbiotic Job Scheduling or Simultneous Multithreding Processor. In Proc. o 9th Intl. Con. on Architecture Support or Progrmming Lnguge nd Operting Systems(ASPLOS), [2] B. Rogers et l. Scling the Bndwidth Wll: Chllenges in nd Avenues or CMP Scling. In Proc. o the 36th Intl. Con. on Computer Architecture (ISCA), [3] B. Sinhroy et. l. POWER5 System Microrchitecture. IBM Journl o Reserch nd Development, 49(4/5):505 52, [4] E. Ebrhimi et l. Coordinted Control o Multiple Preetchers in Multi-Core Systems. In Proc. o the 42th IEEE/ACM Intl. Symp. on Microrchitecture (MICRO), [5] E. Ipek et l. Sel-Optimizing Memory Controller: A Reinorcement Lerning Approch. In Proc.o the 35th Intl. Symp. on Computer Architecture (ISCA), [6] P. Emm. Understnding Some Simple Processor-Perormnce Limits. IBM Journl o Reserch nd Development, 4(3), 997. [7] F. Liu et l. Understnding How O-Chip Memory Bndwidth Prtitioning in Chip Multiprocessors Aects System Perormnce. In 6th Intl. Symp. on High Perormnce Computer Architecture, 200. [8] G. Hinton et l. The Microrchitecture o the Pentium 4 Processor. Intel Technology Journl, (Q), 200. [9] ITRS. Interntionl Technology Rodmp or Semiconductors: 2005 Edition, Assembly nd pckging. In [0] K.J. Nesbit et l. Fir Queuing Memory System. In Proc. o the 39th IEEE/ACM Intl. Symp. on Microrchitecture (MICRO), [] L. Sprcklen et l. Eective Instruction Preetching in Chip Multiprocessors or Modern Commercil Applictions. In th Intl. Symp. on High Perormnce Computer Architecture(HPCA), [2] J. Little. A Proo o Queueing Formul L = λw. Opertions Reserch, 9( ), 96. [3] N. Rique et l. Eective Mngement o DRAM Bndwidth in Multicore Processors. In Proc. o the 6th Intl. Con. on Prllel Architectures nd Compiltion Techniques(PACT), [4] O. Mutlu et l. Prllelism-Awre Btch Scheduling: Enhncing both Perormnce nd Firness o Shred DRAM Systems. In Proc.o the 35th Intl. Symp. on Computer Architecture (ISCA), [5] P.S. Mgnusson et l. Simics: A Full System Simultion Pltorm. IEEE Computer Society, 35(2):50 58, [6] S. Rixner et l. Memory Access Scheduling. In Proc.o the 27th Intl. Symp. on Computer Architecture (ISCA), [7] S. Srikntih et l. SRP: Symbiotic Resource Prtitioning o the Memory Hierrchy in CMPs. In In Proc. o Intl. Con. on High Perormnce Embedded Architectures nd Compilers (HiPEAC), 200. [8] S. Srinth et l. Feedbck Directed Preetching: Improving the Perormnce nd Bndwidth-eiciency o Hrdwre Preetchers. In 3th Intl. Symp. on High Perormnce Computer Architecture(HPCA), [9] Stndrd Perormnce Evlution Corportion. Spec cpu2006 benchmrks