White Paper. Performance Monitoring Unit Sharing Guide. Peggy Irelan and Shihjong Kuo

Transcription

1 White Paper Perfrmance Mnitring Unit Sharing Guide Peggy Irelan and Shihjng Ku

2 Table f Cntents 1. Abut this dcument Prgrammer s Reference Manual Vlume 3B and these Guidelines Sharing guidelines Appendix MSR Definitin(s)... 9 Acknwledgements This material has been reviewed and updated in crdinatin with ecsystem OEMs, OSVs, VMVs, ISVs and Tl prviders. Thanks t all the technical cntributrs and facilitatrs fr supprting the breath f the reviews. A special thanks t VMware* fr raising attentin t this issue and the impact t the ecsystem, and fr prviding the mtivatin t start the develpment f these guidelines. * Other names and brands may be claimed as the prperty f thers

3 Terms Term Definitin PerfMn PMU MSR PEBS CPL and Ring Level In-use Agent First-use Agent ISR PMI VMX SMM Shrt fr Perfrmance Mnitring. The term used fr the current cllective mnitring hardware resurces e.g. cunter/cntrls etc supprted n Intel platfrms. The Perfrmance Mnitring Unit f prcessrs supprting Intel 64 and IA-32 architectures. Generally, it cnsists f cllectins f MSRs. The cllectin f MSRs include cunter registers, event prgramming MSRs, glbal event cntrl MSRs. PMUs f lder prcessrs are mdel-specific; PMU interfaces in mre recent prcessrs are evlving twards higher degrees f architectural stability. Mdel Specific Register. PMU cunter and cunter cntrl registers are implemented as MSR registers. They are accessed via the RDMSR and WRMSR instructin. Certain cunter registers can be accessed via the RDPMC instructin. As defined in Vlume 2B f the Prgrammer s Reference Manual. RDPMC is available t sftware at any privilege level; RDMSR and WRMSR are available nly t sftware running at ring 0. Precise Event Based Sampling. A special cunting mde in which cunters can be cnfigured t verflw, interrupt the prcessr, and capture machine state at that pint. Intel prcessrs perate in privilege levels zer thrugh three. Typically perating system cde executes in privilege level 0. User (r privilege levels 1, 2, r 3) refers t less privileged states f executin. User cde typically executes at privilege level 3. Defines a sftware cnventin between sftware agent s interactins with PMU hardware with respect t the subscriptin f PMU hardware. A set f PMU hardware (a cunter cntrl register, assciated cunter MSR, r feature) is in-use if cnditins described in the guidelines sectin f this dcument are met. Shrt fr PMU agent. A PMU agent is a privileged sftware prcess that reads and writes t PMU hardware. Specifically, it culd be any ne f the fllwing: Firmware (BIOS), OS, hsted VMM, Perfrmance Mnitring tls, ISR, r any ther ring 0 sftware. A PMU agent that discvers a set f PMU hardware was nt in-use and prgrams that piece f PMU hardware t in-use state. Interrupt Service Rutine. Perfrmance Mnitring Interrupt. This interrupt is generated when a cunter verflws and has been prgrammed t generate an interrupt, r when the PEBS interrupt threshld has been reached. The interrupt vectr fr this interrupt is cntrlled thrugh the Lcal Vectr Table in the Lcal APIC. Virtual Machine extensins. Hardware extensins prvided t enable x86 hardware virtualizatin. System Management Mde.

4 Abut this dcument Until fairly recently, the perfrmance mnitring unit (PMU) f Intel prcessrs emplyed mdel-specific prgramming interfaces and required unique treatment by perfrmancemnitring users r sftware agents. In rder t facilitate simpler develpment and maintenance f perfrmance mnitring tls, Intel has begun the transitin t create an architecturally-defined apprach fr sftware agents t interacting with the PMU. This apprach is knwn as "architectural Perfrmance Mnitring (PerfMn)," as ppsed t the traditinal "mdel-specific Perfrmance Mnitring (PerfMn)." The purpse f architectural PerfMn is t prvide a functinally and lgically cnsistent set f capabilities with a cnsistent hardware interface that develpers can rely n nw and in the future. Perfrmance Mnitring (PerfMn) is a limited system resurce. While prgramming cntrl f PMU hardware is available n a first cme first serve basis, there is currently n hardware mechanism available t reserve a piece f PMU resurce. A sftware agent can intentinally r unintentinally verwrite previusly prgrammed PMU settings. This situatin can be imprved with sftware updates and PMU sharing guidelines between users r agents. This paper prvides a set f guidelines between multiple sftware agents sharing the PMU hardware n Intel prcessrs. These cnventins are nt attempting t address PerfMn virtualizatin but hw independent sftware cmpnents (e.g., BIOS and OS; e.g., hsted Virtual Machine Mnitr (VMM) and hsting OS) can crdinate use f PerfMn. In the case f PerfMn virtualizatin (HV and guest), the sftware is nt fully independent, and a HyperVisr/VMM can use different appraches, as lng as it uses these cnventins glbally relative t ther independent sftware (e.g., BIOS). All PerfMn resurce definitins are dcumented in Vlume 3B f the Intel 64 and IA-32 Architectures Sftware Develper's Manual. Usage and access, beynd Ring 0 prgramming, is nt restricted. These guidelines are fr sftware agents including Firmware (BIOS), VMM, OS and Tls that prgram and utilize r plan t utilize PerfMn resurces. These guidelines are nt setting requirement fr PerfMn virtualizatin under VMX. All cnventins are recmmendatins. There is n new hardware r hardware enfrcement being defined by this paper.

5 Prgrammer s Reference Manual Vlume 3B and these Guidelines PerfMn supprt and usage expectatins utlined Prgrammer s Reference Manual Vlume 3B must still be fllwed: Facility enumeratin: The versin identifier is retrieved by querying CPUID.0AH:EAX[bits 7:0]. If the versin identifier is greater than zer, architectural perfrmance mnitring capability is supprted. Sftware queries the CPUID.0AH fr the versin identifier first; it then analyzes the value returned in CPUID.0AH.EAX, CPUID.0AH.EBX t determine the facilities available. Bits 8 thrugh 15 f CPUID.0AH.EAX indicate the number f perfrmance mnitring cunters available n the lgical prcessr (each IA32_PERFEVTSELx MSR is paired t the crrespnding IA32_PMCx MSR). Bits 0 thrugh 5 f CPUID.0AH.EDX indicate the number f fixed-functin perfrmance cunters available per thread. Events nly supprted by enumeratin: A prcessr that supprts architectural perfrmance mnitring may nt supprt all the predefined architectural perfrmance events (Table 18-6). CPUID.0AH:EAX[31:24] indicates events nt available. N sftware precisin prgramming decisins r functinality: Prgramming decisins r sftware decisins fr functinality shuld nt be based n the event values r dependent n the existence f perfrmance mnitring events. Knwn starting state: Sftware requires a knwn starting state. After CPU reset, all cunters and cntrl registers are disabled and clear/reset t 0. The nly exceptin t this is the IA32_PERF_GLOBAL_CTRL cntrl MSR, all prgrammable cunter glbal enable bits are reset t 1. General PerfMn prgramming guidelines: Disable PerfMn cunter(s), using IA32_PERF_GLOBAL_CTRL r IA32_PERFEVTSELx.ENABLE, befre prgramming (writing) the cunter (IA32_PMCx) MSRs. Utilize kernel preemptin disabling facilities prvided befre reading the in-use indicatin and prgramming the cunter(s) t reduce race cnditins. T facilitate sharing agents must mve t read-mdify-write t access MSRs.

6 An in-use resurce is indicated by the fllwing: Prgrammable cunter is in-use if the Event select field in IA32_PERFEVNTSELx is nn-zer. A fixed cunter is in-use if its Enable field in IA32_FIXED_CTR_CTRL is nnzer. PMI r APIC Perfrmance Mnitring Interrupt Vectr is in-use if: The INT/PMI bit, bit 20, in any IA32_PERFEVNTSELx is nn-zer. r The INT/PMI bit, the 4 th bit in the crrespnding fixed cunter cntrl field in IA32_FIXED_CTR_CTRL, is nn-zer. E.g. If the platfrm supprts 3 fixed cunters. The PMI in-use check is IA32_FIXED_CTR_CTRL bits 3, 7 r 11 are nn-zer the PMI vectr is in-use. Mdel specific feature r event MSRs are in-use if the prgramming/enable MSR is nn-zer. Fr example n Intel Cre i7 prcessr : PEBS is in-use if (0x3F1) MS_PEBS_ENABLE[3:0]!= 0. Lad Latency feature is in-use if (0x3F1) MS_PEBS_ENABLE[35:32]!= 0. Extended filter MSRs fr the fllwing: Offcre Traffic events are in-use if (0x1A6/0x1A7) MS_OFFCORE_REQ0/1!= 0. LBR filter is enabled/ in-use if (0x1C8) MS_LBR_FILTER_SELECT!= 0. Nte: Mdel specific PerfMn events/features and the crrespnding MSRs are subject t change.

7 Sharing Guidelines Agents must prvide a sftware calling and launch cnfiguratin facility t relinquish r release the use f PerfMn resurces. Agents must be able t be cnfigured and functin withut PerfMn resurces. Discntinue r relinquish use f the cunter by zering the cunter cntrl register (IA32_PERFEVNTSELx) r cntrl field and the crrespnding cunter. Agents must discntinue r relinquish PerfMn resurce n exit r unlad. Nte: Tls agents that fllw these cnventins and utilize cunter resurces are at risk f being killed by the user such that the tl is nt able t cleanly release the cunter resurces. Tls shuld maintain a resurce usage histry and clean exit infrmatin file s that if they are launched again they can reclaim the previusly used resurces. If the Tl is nt able t cleanly exit, and des nt implement a kill-reclaim capability, these resurces will remain in-use until rebt. Tls shuld dcument the impact t the platfrm when the user kills the applicatin and des nt rebt the system. Agents shuld prgram the cunter at the first pprtunity. An agent that discvers a cunter is in-use, at first pprtunity r later, will assume the cunter has been claimed by anther agent and will nt change its prgramming r disable it. This includes the VMM. The VMM shuld ensure any cunter in-use by firmware is nt disabled. If an agent has prgrammed cunters at its first pprtunity, that agent reserves the right t "reclaim" these cunters if the cunter, at any pint, is reprgrammed by anther agent. Agents shuld cnsume a minimal set f resurces. If at all pssible, use a fixed functin cunter. If a prgrammable cunter is required, use the least capable cunter. Fr example, n Intel Cre prcessrs and Intel Cre Du prcessrs Precise Event Based Sample (PEBS) is nly available n cunter 0. If at all pssible, agents shuld avid using cunter 0 t allw PEBS t be utilized by anther agent. If at all pssible, cnfigure the fixed cunter(s) t be an always running (mntnically increasing) cunter(s) and nt reprgram. Prgram the cunters t cunt all ring levels and nt generate a PMI (perfrmance mnitring interrupt). This enables agents t check the crrespnding 4-bit cunter cntrl fr free running cnfiguratins fr pssible read-nly sharing.

8 Fr example, Fixed Cunter 0 is free running if IA32_FIXED_CTR_CTRL[3:0] = 3, Fixed Cunter 1 is free running if IA32_FIXED_CTR_CTRL[7:4] = 3, Fixed Cunter 2 is free running if IA32_FIXED_CTR_CTRL[11:8] = 3. Using Perfrmance Mnitring Interrupts (PMI): If the agents require sampling r enabling PMI, the agent must: Pll fr an available resurce including PMI vectr and check fr ther user(s). If ther agents are using cunters, d nt use the Freeze PerfMn n PMI feature in IA32_DEBUGCTL MSR. Setting this bit will disable all cunters upn interrupt, pssibly negatively impacting ther users. Sftware must utilize the sftware freeze, read-mdify-write t IA32_PERF_GLOBAL_CTRL MSR, in the interrupt service rutine and nly disable cunters under their prgramming cntrl. Agents are free t utilize the SMM Freeze feature in IA32_DEBUGCTL MSR. Firmware must nt disable this feature while in SMM.

9 Appendix MSR Definitin(s) MSR definitins belw fr Intel Cre i7 prcessr. Glbal Cntrl and Status MSR

10 IA32_PERFEVNTSELx MSR IA32_FIXED_CTR_CTRL MSR

11 IA32_PERF_CAPABILITIES MSR MS_PEBS_ENABLE MSR (Intel Cre i7 prcessr

12 IA32_DEBUGCTL MSR

13 Abut the Authrs Peggy Irelan is a Principal Engineer within the Sftware & Services Grup at Intel. She wrks n prcessr bservatin/mnitring hardware architecture and usage mdels. She has a particular interest in advancing smart sftware enabled thrugh hardware feedback, mnitring/sensing. In her spare time, Peggy likes t try new adventures, the latest f which is scuba diving this year reaching PADI Rescue Diver certificatin while explring Beqa, Fiji and Mrea, Tahiti. She can be reached at peggy.j.irelan@intel.cm Shihjng Ku is a technical lead in Server Platfrm Grup. He wrks n CPU feature dcumentatin and instructin set enabling. He can be reached at shihjng.ku@intel.cm. INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. UNLESS OTHERWISE AGREED IN WRITING BY INTEL, THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR. Intel may make changes t specificatins and prduct descriptins at any time, withut ntice. Designers must nt rely n the absence r characteristics f any features r instructins marked "reserved" r "undefined." Intel reserves these fr future definitin and shall have n respnsibility whatsever fr cnflicts r incmpatibilities arising frm future changes t them. The infrmatin here is subject t change withut ntice. D nt finalize a design with this infrmatin. The prducts described in this dcument may cntain design defects r errrs knwn as errata which may cause the prduct t deviate frm published specificatins. Current characterized errata are available n request. Cntact yur lcal Intel sales ffice r yur distributr t btain the latest specificatins and befre placing yur prduct rder. Cpies f dcuments which have an rder number and are referenced in this dcument, r ther Intel literature, may be btained by calling , r by visiting Intel's Web Site.