XC Series-AC Repair Procedures

Transcription

1 XC Series-AC Repair Procedures

2 Contents About Cray XC Series Repair Procedures...4 Safety Information...5 System Cabinet Repair Procedures...8 Power Down a System Cabinet...9 Power Up a System Cabinet...10 Open and Close the Cabinet Door...11 Remove a Power Supply...11 Install a Power Supply...12 Remove an Operator Interface...13 Install an Operator Interface...14 Remove an Enviromental Distribution Board...15 Install an Enviromental Distribution Board...16 Remove an Air Temperature Sensor...18 Install an Air Temperature Sensor...20 Remove a Chassis Backplane...21 Install a Chassis Backplane...26 Remove a Power Distribution Unit (PDU)...29 Install a Power Distribution Unit (PDU)...32 Remove a Power Supply Rack...34 Install a Power Supply Rack...37 Remove a Cabinet Controller...39 Install a Cabinet Controller...40 Remove a Cabinet Controller Backplane...41 Install a Cabinet Controller Backplane...43 Remove a Cabinet Controller DIMM...46 Install a Cabinet Controller DIMM...47 Remove a Chassis Host...49 Install a Chassis Host...50 Remove a Passive Electrical Cable...52 Install a Passive Electrical Cable...53 Remove a Blower Assembly...54 Install a Blower Assembly...56 Remove a VFD...58 Install a VFD...61 Remove an EMI Filter

3 Install a EMI Filter...65 XC Series Compute and IO Blades...67 Remove a Compute or IO Blade Using xtwarmswap...69 Install a Compute or IO Blade Using xtwarmswap...71 Remove a Compute or IO Blade...73 Install a Compute or IO Blade...74 Remove a Memory DIMM...76 Install a Memory DIMM...78 Remove a Voltage Module (VRM, VIVOC, and HIVOC)...79 Install a Voltage Module (VRM, VIVOC, and HIVOC)...81 Remove a Processor...83 Install a Processor...86 Remove a Quad Processor Daughter Card (QPDC)...90 Install a Quad Processor Daughter Card (QPDC)...91 Remove a Processor Daughter Card (XPDC)...93 Install a Processor Daughter Card (XPDC)...94 Remove a Kepler Processor Daughter Card (SGDC)...95 Install a Kepler Processor Daughter Card (SGDC)...97 Remove a Graphic Processor Daughter Card (GPDC)...98 Install a Graphic Processor Daughter Card (GPDC)...99 Remove a Haswell Processor Daughter Card (HPDC) Install a Haswell Processor Daughter Card (HPDC) Remove Components from an Aries Network Card (ANC) Replace Components on an Aries Network Card (ANC) Remove Components from an IO Base Board (IOBB) Install Components on an IO Base Board (IOBB) Remove a PCI Card Install a PCI Card System Maintenance Workstation (SMW) Boot RAID Cabinet

4 About Cray XC Series Repair Procedures This publication describes how to remove and replace the field replaceable units (FRUs) in Cray XC Series-AC and XC Series-LC systems. Scope and Audience This publication is intended for Cray service personnel who service XC Series systems in the field. Personnel must complete XC Series hardware training before performing these maintenance and replacement procedures. The procedures are organized as follows: Power-up and power-down procedures System cabinet field replacement procedures (FRPs) Blower cabinet FRPs Air Preconditioner FRPs Compute and IO blade FRPs SMW and Disk Cabinet FRPs The time required to complete each procedure is included. This estimate does not include time to power down the system, power up the system, or boot/troubleshoot the operating system. These times can vary significantly depending on the size of the system. Related Publications Table 1. XC Series Related Publications Publication Title Installing a XC Series-AC System, H Illustrated Parts Catalog for XC Series Systems, H Hardware Description for XC Series Systems, H Brief Description Describes site planning and environmental requirements for the XC Series-AC computer room, as well as the hardware installation and cabling procedures for the XC Series-AC system. Includes illustrated parts diagrams and identifies part numbers for the field-replaceable units (FRUs) in the XC Series system. Describes the hardware overview and hardware configuration information for XC Series systems. Includes architecture and mechnical packaging information, as well as required support hardware for all systems. 4

5 Publication Title ESD and Safety Guidelines, HGM-016 CLE System Administration Guide (S-2393) Brief Description Includes safety information that alerts maintenance and serviging personnel to, and protects them from, potential hazards. This publication must be read before performing maintenance or service procedures on XC Series cabinets. Includes Cray Linux Environment (CLE) software information and administration procedures. TORX Fasteners The XC Series cabinet and blades contain both TORX and TORX+ fasteners. The drivers and bits in the tool kit are TORX; you can use these TORX drivers and bits on both TORX and TORX+ fasteners. HSS Component Naming Conventions Several FRPs in the this document instruct you to enter hardware supervisory system (HSS) commands on the system maintenance workstation (SMW). This document refers to the HSS ID of the target component as the hostname. The man pages for the various commands refer to the target component as the hostname, physidlist, id-list, or some other name; these names all refer to the standard HSS ID for the target component. The FRPs include an example of the HSS ID for each command. Blade Handling Each blade weighs approximately 29 lbs (13 kg). Blades can be removed or installed using a lift. CAUTION: Personal injury and equipment damage Always observe the ESD and safety precautions described in the previous section when handling blades. Failure to do so could result in equipment damage. Use and proper lifting techniques when handling blades. Failure to do so could result in serious personal injury. Feedback Visit the Cray Publications Portal at and use the Contact Us button to submit comments about this publication. Safety Information Ensure that you read and comply with the following safety information when you service the XC Series system. DANGER: Electrocution Hazard Cray requires the power to be disconnected from the PDU when working on components inside the PDU chassis. If it is necessary to work on components in the PDU with power applied, a minimum 36- Safety Information 5

6 in working clearance must be maintained in front of the PDU. This workspace is required and must be maintained around all electrical equipment where parts of an energized system may be serviced. To maintain this clearance, remove the door from the cabinet facing the energized parts, and install an insulating barier to prevent any grounded equipment from coming in contact with the energized components in the PDU. Failure to do so could result in serious injury or death. WEEE Directive The Waste Electrical and Electronic Equipment (WEEE) directive of the European Union (EU) is designed to reduce waste management problems linked to heavy metals and flame retardants. It attempts to limit the amount of electrical and electronic equipment that enters landfills by encouraging reuse, recycling, and separate collection. Components that bear the crossed-out wheelie bin symbol require special disposal In accordance with European Directive 2002/96/EC on Waste Electrical and Electronic Equipment (WEEE), the presence of the crossed-out wheelie bin symbol on a product or on its packaging indicates that you must not dispose of that item in the normal unsorted municipal waste stream. Instead, it is your responsibility to dispose of that product by returning it to a collection point that is designated for the recycling of electrical and electronic equipment waste. Separate collection of this waste helps to optimize the recovery and recycling of any reclaimable materials and reduces the impact on human health and the environment. ESD Precautions Observe electrostatic discharge (ESD) precautions during the entire removal and installation process. Required apparel includes an ESD smock, ESD shoes, and an ESD wrist strap. ESD and Safety Guidelines, Cray publication HGM-016, contains safety information that alerts maintenance and servicing personnel to, and protects them from, potential hazards. Take the time to read these guidelines. You can access ESD and Safety Guidelines (HGM-016) via the Customer Training and Documentation (CDT) website at depts/customerservice/cdt/default.aspx. CAUTION: Equipment damage You must read ESD and Safety Guidelines, Cray publication HGM-016, before you perform maintenance or service procedures on the Cray XC30 cabinets. Observe all ESD precautions. Failure to do so can result in damage to the equipment. ESD Smock Wear a Cray approved static-dissipative smock when you service or handle an ESD-sensitive device. Completely button the smock and wear it as the outermost layer of clothing. You must have a portion of the smock s sleeves in direct contact with the skin of your arms. Skin contact is essential for a dissipative path-to-earth ground through your wrist strap. Tuck hair that exceeds shoulder length inside the back of the smock. ESD Shoes Wear approved static-dissipative shoes or approved dissipative heel straps on both shoes when you service or handle an ESD-sensitive device. When sensitive equipment is exposed to static discharge, ESD shoes provide a backup to the wrist straps and grounding cords and help prevent an excessive charge from building up on you when you contact the conductive flooring. Use dissipative footwear in addition to, not as an alternative to, a wrist strap. Safety Information 6

7 Safety Measures Observe the following safety measures when you remove, install, repair, or maintain the system. Do not wear watches or jewelry when you work on a system cabinet. Keep fingers and conductive tools away from high-voltage areas and from high-current areas. WARNING: Serious injury or death Keep fingers and conductive tools away from high-voltage areas. Serious injury or death will occur if you do not follow these precautions. Set circuit breakers to the OFF or OPEN position, when the procedure indicates that, before you start the removal and installation process. WARNING: Severe personal injury Ensure that the electrical circuit breakers for the equipment that you service are off, locked, and tagged. Failure to do so could result in severe shock and burns. Remove all tools from the system cabinets after you perform service. Replace all covers and panels that you removed from the system during servicing. Some procedures require that you power off the system; power off the system only after you shut down the system software in an orderly manner. CAUTION: Loss of data If you power off the system before you halt the operating system, you can lose customer data. Cray systems may reach noise levels where the use of hearing protection is required. The need for this protection depends on such factors as the number of installed cabinets, the speed of the blowers, and the physical design and the materials used in the computer room. The use of hearing protection is highly recommended for all personnel when working in computer rooms, even if noise exposure dosimetry results performed by the host site, if known, are below the USA OSHA or other country's regulatory levels. Safety Information 7

8 XC Series-AC System Cabinet Repair Procedures XC Series-AC (air cooled) cabinet repair procedures. There are two types of XC Series-AC cabinets: 400/480Vac and 200/208Vac. Depending on the system type, the PDU, Variable Frequency Drive (VFD), and blower assembly will require different parts. Table 2. XC Series-AC 400/480Vac and 200/208Vac Part Numbers PDU VFD Blower Assembly 400/480Vac PDU Assy, XC30-AC High Voltage Cabinet AC Drive, 7.5 HP 240V 3P High Voltage Cabinet Blower Assy, XC30- AC High-Voltage Cabinet 200/208Vac PDU Assy, XC30-AC Low Voltage Cabinet AC Drive, 7.5 HP 240V 3P Low Voltage Cabinet Blower Assy, XC30- AC Low Voltage Cabinet The figure shows the XC Series-AC system cabinet and cabling connections for the major components. Not every component shown in the figure is a FRU. 8

9 Figure 1. XC Series-AC Cabling Operator Interface Cabinet X Address: X01 From Cabinet Controller Backplane to Operator Interface ETH From Environmental Distribution Board to Cabinet Controller Backplane Cabinet Controller Backplane SMW SSB0 SSB1 SSB2 ETH0 ETH1 ETH2 Cabinet Cont roller (CC) Chassis Host SSB0 ETH0 Chassis Backplane Modules Blower Enclosure Environmental Distribution Board J1 J2 P8 P1 P2 P3 Shelf 2 Shelf 1 Shelf 0 P1 P2 P3 Power Supply Rack J1 J2 J3 Variable Frequency Drive P4 AC To blower P5 P6 P9 From Environmental Distribution Board to temp/velo strip Blower breaker to PDU J5 J1 J2 J3 EMI Filter CB1 PDU J4 Power Down a XC Series-AC Cabinet Time: 2-3 minutes Tools: None Use this procedure to power down a system cabinet when you perform maintenance on the cabinet. 1. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Power Down a XC Series-AC Cabinet 9

10 Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. 2. Open the rear door of the cabinet. 3. Place the main circuit breaker CB1 in the OFF (down) position. Figure 2. Circuit Breaker for XC Series Cabinet Power Up a XC Series-AC Cabinet Time: 2-3 minutes Tools: None Use this procedure to power up a system cabinet when you perform maintenance on the cabinet. 1. Place main circuit breaker CB1 in the ON (up) position. Figure 3. Circuit Breaker for XC Series Cabinet 2. Close the rear door of the cabinet. Power Up a XC Series-AC Cabinet 10

11 3. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Open and Close the Cabinet Door 1. Open the cabinet door. a. Pull out the bottom of the door handle. b. Use the handle to pull the door open. Figure 4. Front or Rear Door Handle 2. Close cabinet door. a. Pull out the bottom of the door handle. b. Firmly push the door shut. c. Push the bottom of the door handle in. Remove a XC Series-AC Power Supply Part Number: Power Supply, 52Vdc, 3KW, Vac Time: 2-3 minutes Tools: None For a XC Series-AC system, a XC Series-AC Power Supply Replacement video is avaliable that shows how to remove and install this component. The power supplies connect in a minimum N+1 configuration, enabling you to remove one power supply while the system is operating. 1. Open the rear door of the cabinet. Open and Close the Cabinet Door 11

12 2. Loosen the thumb screw completely and pull the extractor lever forward. 3. CAUTION: Equipment damage Use both hands to support the power supply and do not carry the power supply by the extraction lever. Damage to equipment may result. Pull on the extractor handle to disengage the power supply from the connector and slide the power supply out of the shelf. Figure 5. Power Supply Install a XC Series-AC Power Supply Part Number: Power Supply, 52Vdc, 3KW, Vac Time: 2-3 minutes Tools: None A XC Series-AC Power Supply Replacement video is avaliable that shows how to remove and install this component. The power supplies connect in a minimum N+1 configuration, enabling you to remove one power supply while the system is operating. 1. Inspect the connectors on the back of the replacement power supply; ensure that the pins are straight. 2. Slide the power supply into the rack and seat it firmly. Install a XC Series-AC Power Supply 12

13 3. Push the extractor handle in to completely seat the power supply and latch the handle. 4. Finger-tighten the thumb screw to secure the power supply latch. Figure 6. Power Supply 5. Close the rear door of the cabinet. Remove a XC Series-AC Operator Interface Part Number: Interface Module, Operator Time: 15 minutes Tools: Small flathead screwdriver T15 TORX driver Socket driver The X and Y cabinet address information, cabinet type, and coordinated cooling region (CCR) position is stored on the cabinet controller backplane SEEP. You do no need to reset this information when replacing the operator interface. 1. Open the rear door of the cabinet. 2. NOTE: Component loss Place a towel or protective covering under the operator interface to prevent losing the short serial cable screws. Remove a XC Series-AC Operator Interface 13

14 Loosen the two small captive flathead screws that secure the serial cable to the operator interface and disconnect the cable. 3. Remove the bottom two T15 TORX screws and lockwashers that attach the operator interface to the cabinet. While holding the operator interface, remove the top two screws. 4. Remove the operator interface from the cabinet. Figure 7. Operator Interface AC Operator interface Serial cable Screws (4) Install a XC Series-AC Operator Interface Part Number: Interface Module, Operator Time: 15 minutes Tools: Small flathead screwdriver T15 TORX driver Socket driver The X and Y cabinet address information, cabinet type, and coordinated cooling region (CCR) position is stored on the cabinet controller backplane SEEP. You do no need to reset this information when replacing the operator interface. 1. Position the operator interface in the cabinet and starting from the top, reinstall the four T15 TORX screws and lockwashers to secure the operator interface to the cabinet. 2. Reconnect the serial cable to the operator interface and tighten the two small captive flathead screws. 3. Close the rear door of the cabinet. Install a XC Series-AC Operator Interface 14

15 Figure 8. Operator Interface AC Operator interface Serial cable Screws (4) Remove a XC Series-AC Environmental Distribution Board Part Number: Environmental Control Assembly Time: 15 minutes Tools: Small flathead screwdriver Slotted screwdriver T20 TORX driver Socket driver A XC Series-AC Evironmental Distribution Board Replacement is avaliable which shows how to remove and install this component. 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Remove a XC Series-AC Environmental Distribution Board 15

16 Figure 9. Circuit Breaker for XC Series Cabinet 2. CAUTION: Equipment damage Do not drop disconnect serial cables. Damage to equipment may occur. Label any serial cables with missing labels, then loosen the two small captive flathead or slotted screws that secure each serial cable to the environmental distribution board. Disconnect the cables. Figure 10. Environmental Distribution Board AC Envrionmental distribution board Serial cable Screws (6) 3. Remove the six T20 TORX screws that attach the environmental distribution board to the cabinet and remove the board. Install a XC Series-AC Environmental Distribution Board 16

17 Install a XC Series-AC Environmental Distribution Board Part Number: Environmental Control Assembly Time: 15 minutes Tools: Small flathead screwdriver Slotted screwdriver T20 TORX driver Socket driver A XC Series-AC Evironmental Distribution Board Replacement is avaliable which shows how to remove and install this component. 1. Position the environmental distribution board in the cabinet and reinstall the T20 TORX screws. 2. Reconnect the serial cables to the environmental distribution board and tighten the captive screws. Figure 11. Environmental Distribution Board Connections J1 J2 P8 P1 P2 P3 P4 P5 P6 P9 To cabinet controller Blower control to blower enclosure Preconditioner (Optional) Power supply rack shelf 0 (bottom) Power supply rack shelf 1 (middle) Power supply rack shelf 2 (top) Air temp sensor Not used Not used Coil Control Board 3. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Install a XC Series-AC Environmental Distribution Board 17

18 Figure 12. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC Air Temperature Sensor Part Number: Inlet Air Sensor, Cabinet Time: 15 minutes Tools: Small flathead screwdriver T15 TORX driver 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Remove a XC Series-AC Air Temperature Sensor 18

19 Figure 13. Circuit Breaker for XC Series Cabinet 2. Open the front door of the cabinet. 3. Remove the lower blades from slots 2, 3, 4, and Working from the rear of the cabinet, remove the two screws that secure the metal plate to the chassis. 5. Disconnect the control cable from the environmental distribution board. Use a flathead screwdriver, if necessary. 6. Push a few inches of the control cable through the hole under the metal plate. 7. Working from the front of the cabinet, pull the sensor forward and disconnect the control cable from the sensor. Remove the sensor, along with the metal siding, from the cabinet. 8. On an ESD safe surface, remove the three T15 TORX screws that attach the air temperature sensor to the metal siding. Figure 14. XC Series-AC Air Temperature Sensor Remove a XC Series-AC Air Temperature Sensor 19

20 Install a XC Series-AC Air Temperature Sensor Part Number: Inlet Air Sensor, Cabinet Time: 15 minutes Tools: Small flathead screwdriver T15 TORX driver 1. To install the air temperature sensor, install the three T15 TORX screws that secure the replacement air temperature sensor to the metal siding. 2. Reconnect the control cable to the air temperature sensor. Figure 15. Air Temperature Sensor AC 3. Position the metal siding in its proper location in the cabinet and press it into place. 4. From the front of the cabinet, push the control cable through the rear hole. 5. From the rear of the cabinet, align the control cable in the cable indentation. Install a XC Series-AC Air Temperature Sensor 20

21 Figure 16. Cable Indentation 6. From the rear opening, use a screwdriver to push the cable tab to the left toward the guide rail. 7. Reinstall the lower blades into slots 2, 3, 4, and Close the front door of the cabinet. 9. Open the rear door of the cabinet. 10. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Figure 17. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC Chassis Backplane Part Number: Backplane Replacement kit Remove a XC Series-AC Chassis Backplane 21

22 Time: 1.5 hours Tools: NOTE: There are many screws securing the backplane and DC bus bars. An electric torque-driver is highly recommended. T10 TORX driver T15 TORX driver T20 TORX driver T25 TORX driver T30 TORX driver T20 TORX bit T30 TORX bit 1/4-in. socket, 1/4-in. square drive 7/16-in. socket, 1/4-in. square drive Socket driver, 1/4-in. square drive Adjustable torque driver, 5 40 in-lbs (P/N ) Torque Wrench, 1/4" Drive Inch-lbs. (P/N ) Passive electrical connector plug, quantity 40 Isopropyl alcohol Absorbent wipes This procedure may require a Cray contracted or customer contracted electrician to complete lockout/tagout procedures or disconnect facility power. Allow time in the preventive maintenance schedule for this requirement. 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Remove a XC Series-AC Chassis Backplane 22

23 Figure 18. Circuit Breaker for XC Series Cabinet 2. WARNING: Electrical hazard Make sure that the electrical circuit breaker for the cabinet is off, locked, and tagged. Failure to do so could result in severe shock and burns. Work with a site electrician to ensure that circuit breakers for the cabinet at the wall panel are off. Use appropriate lockout/tagout or other site-specific procedures to indicate that the breakers are off. 3. Open the front door of the cabinet. 4. Simultaneously push and release the buttons on each end of the extractor handles and unseat all blades about two inches. 5. Remove the chassis host. 6. Disconnect all passive electrical cables if working on a multicabinet Cray XC30-AC cabinet system. Place connector covers on each cable. 7. Remove two T30 TORX screws that attach the backplane cover to the cabinet. 8. Remove 31 TORX screws (T20) that attach the backplane cover to the cabinet. a. Remove three screws that are attached to the bus bar cover on the left side and bottom of the backplane cover. b. Remove 28 screws that are located on the attached top and bottom backplane cover. Six screws remain installed on the backplane cover. Remove a XC Series-AC Chassis Backplane 23

24 () Figure 19. Backplane Cover AC 9. Backplane cover Connector cover T30 TORX screws (2) T20 TORX screws (28) Screws (3) (2 screws not shown) Screws (6) remain installed CAUTION: Personal or equipment damage The backplane cover may be jammed between the upper or lower connector covers. It may be required to remove the upper or lower connector covers to free the backplane cover. Forcefully removing the cover can lead to equipment damage or personal injury. Remove the backplane cover by firmly grasping both sides and pulling outward. Figure 20. Backplane Cover Removal AC 10. Remove the six 7/16-in. nuts and conical washers that attach the vertical bus joiner plate to the horizontal bus plate and remove the plates. Note the orientation of the conical washers. The conical washers have a top and bottom. The cone side of the washer should face the nut Figure 21. Horizontal Bus Bar AC IP20 TORX screws (16) IP10 TORX+ screws (48) Standoffs (28) 7/16-in. nuts (6) (3 top, 3 bottom) Remove a XC Series-AC Chassis Backplane 24

25 11. Loosen the twenty-eight 1/4-in. standoffs that attach the backplane to the cabinet. Remove by hand to prevent losing any standoffs. 12. Remove ten T20 TORX screws that attach the upper bus bar cover to the cabinet. 13. Remove ten T20 TORX screws that attach the lower bus bar cover to the upper bus bar cover and the cabinet. 14. Remove five T25 TORX screws from the power supply bus bar (lower red bar) and five T25 TORX screws from the black vertical bus bar. Each T25 screw has a lock (small) and flat (large) washer. Figure 22. Vertical Bus Bars AC T25 TORX screws (5) (power supply bus bar) T25 TORX screws (5) (power supply bus bar) 15. Remove the upper and lower vertical bus bars and retain for installation. 16. Remove 15 IP20 TORX+ screws that attach the backplane to the cabinet. Leave the final screw in the upper right corner installed to secure the backplane to the cabinet. 17. CAUTION: Equipment damage The backplane can tip off the chassis when all screws are removed. Support the backplane while you remove the screws or damage to equipment may occur. Support the backplane as you remove the upper right screw. 18. Carefully tip the backplane assembly off of the chassis and place it on an ESD-safe surface. 19. Remove the 48 IP10 TORX+ screws that attach the bus bars to the backplane assembly and remove the bus bars. The backplane EMI gaskets are secured with reusable adhesive tape. Retain the EMI gaskets for installation on the replacement backplane assembly. 20. Remove the EMI gaskets from the edges of the backplane assembly. 21. Remove the insulator panel from the front side of the backplane assembly. Retain the insulator panel for installation on the replacement backplane assembly. Remove a XC Series-AC Chassis Backplane 25

26 Figure 23. EMI Gasket and Insultator Panel AC Insulator panel EMI gasket Install a XC Series-AC Chassis Backplane Part Number: Backplane Replacement kit Time: 1.5 hours Tools: NOTE: There are many screws securing the backplane and DC bus bars. An electric torque-driver is highly recommended. T10 TORX driver T15 TORX driver T20 TORX driver T25 TORX driver T30 TORX driver T20 TORX bit T30 TORX bit 1/4-in. socket, 1/4-in. square drive 7/16-in. socket, 1/4-in. square drive Socket driver, 1/4-in. square drive Adjustable torque driver, 5 40 in-lbs (P/N ) Torque Wrench, 1/4" Drive Inch-lbs. (P/N ) Passive electrical connector plug, quantity 40 Install a XC Series-AC Chassis Backplane 26

27 Isopropyl alcohol Absorbent wipes This procedure may require a Cray contracted or customer contracted electrician to complete lockout/tagout procedures or disconnect facility power. Allow time in the preventive maintenance schedule for this requirement. 1. Position the insulator panel on the front side of the new backplane assembly. Make sure the EMI gasket protrudes 1/16-in on all sides so a continuous EMI shield exists. 2. Install the adhesive EMI gaskets on all sides of the backplane. 3. Clean the bare metal conductive surfaces on the bus bars and the contact pads on the backplane PCA with isopropyl alcohol. 4. Position the bus bars on the backplane PCA and reinstall, but do not tighten, the 48 IP10 TORX+ screws. 5. Torque the 48 IP10 TORX+ screws to 6 in-lbs. 6. If necessary, move the operator interface cable to prevent interference and position the backplane assembly on the chassis frame. 7. Hand tighten the twenty-eight 1/4-in. standoffs that attach the backplane to the chassis. Start with the top right standoff for easier backplane handling. 8. Reinstall, but do not tighten, the 16 IP20 TORX+ screws that attach the backplane to the chassis. Figure 24. Cray XC Series-AC Standoff and Screw Positions 28 standoffs 16 screws 9. Torque the twenty-eight 1/4-in. standoffs and 16 IP20 screws to 14 in-lbs. 10. Position the lower (red) vertical bus bar inside the cabinet, making sure the piece fits into its correct position. 11. Install, but do not tighten, the five T20 TORX screws into the power supply bus bar. Install a XC Series-AC Chassis Backplane 27

28 12. Position the upper (black) vertical bus bar inside the cabinet, making sure the piece fits into its correct position. Figure 25. XC Series-AC Vertical Bus Bar Positions 13. Install, but do not tighten, the five T20 TORX screws into the upper vertical bus bar. 14. Install and tighten the six 1/4-in. nuts and conical washers to secure the horizontal bus bar to the vertical bus bar. 15. Tighten the five T20 TORX screws into the power supply bus bar and the five T20 TORX screws into the upper vertical bus bar. 16. Torque each nut to 75 in-lbs. 17. Position the backplane cover over the backplane assembly. If necessary, remove upper or lower connector cover to enable the correct placement. 18. Install the two screws that attach to the bus bar cover on the left side of the backplane cover. 19. Install the 31 TORX screws (T20) to secure the backplane cover to the cabinet. 20. CAUTION: Component loss The screws for the bus bar covers are short and easy to drop. Place a towel or foam barrier between the PDU and the side of the cabinet to avoid losing screws. Place the upper bus bar cover in the cabinet and install the 10 T20 TORX screws that secure the cover to the cabinet. The backplane EMI gaskets are secured with reusable adhesive tape. Retain the EMI gaskets for installation on the replacement backplane assembly. Install a XC Series-AC Chassis Backplane 28

29 21. Place the lower bus bar cover in the cabinet and install the 10 T20 TORX screws that secure the cover to the upper cover and the cabinet. 22. Reinstall the cabinet controller. 23. Seat all blades by sliding each blade back into the cabinet. 24. Simultaneously press each extractor handle to fully seat the blade. 25. Close the front door of the cabinet. 26. Work with the site electrician to verify that circuit breakers for the cabinet at the wall panel are turned on. 27. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Figure 26. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC PDU Part Number: PDU Assy, XC30-AC 400/480Vac Cabinet PDU Assy, XC30-AC 200/208Vac Cabinet NOTE: Use the correct part according to the customer site AC line voltage. Time: 15 minutes Tools: Remove a XC Series-AC PDU 29

30 T15 TORX driver T25 TORX driver Large slotted screwdriver 7/16-in. open end wrench This procedure may require a Cray contracted or customer contracted electrician to complete lockout/tagout procedures or disconnect facility power. Allow time in the preventive maintenance schedule for this requirement. DANGER: Electrocution Hazard Cray requires the power to be disconnected from the PDU when working on components inside the PDU chassis. If it is necessary to work on components in the PDU with power applied, a minimum 36- in working clearance must be maintained in front of the PDU. This workspace is required and must be maintained around all electrical equipment where parts of an energized system may be serviced. To maintain this clearance, remove the door from the cabinet facing the energized parts, and install an insulating barier to prevent any grounded equipment from coming in contact with the energized components in the PDU. Failure to do so could result in serious injury or death. 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Figure 27. Circuit Breaker for XC Series Cabinet 2. CAUTION: Electrical hazard Electrical hazard. Make sure the circuit breaker for the cabinet is off, locked, and tagged. Failure to do so could result in severe shock and burns. Remove a XC Series-AC PDU 30

31 Work with a site electrician to ensure that circuit breakers for the cabinet at the wall panel are off. Use appropriate lockout/tagout or other site-specific procedures to indicate that the breakers are off. 3. Have a Cray contracted or customer contracted electrician disconnect the input power cables from the PDU circuit breakers by using the following procedure: a. Remove the ten IP15 TORX+ screws that attach the front cover to the PDU and remove the cover. b. Use the digital multimeter to verify that no voltage is present on the input terminals of circuit breaker CB1. c. Label and disconnect the PDU power cable wires from the ground blocks and the input terminals of circuit breaker CB1. (use appropriate site procedures). d. Loosen the power-cable clamps and remove the power cables by pulling them through the bottom of the PDU. Figure 28. PDU Circuit Breaker Input Terminals AC Circuit breaker input terminals Ground block 4. Remove the six IP25 TORX+ screws that attach the PDU assembly to the cabinet frame. Figure 29. PDU Front View AC IP15 TORX+ screws (10) IP25 TORX+ screws (6) CB1 Cover 5. Disconnect cables J1 through J5 from the side of the PDU; press the tabs on the bottom of the connectors and pull the connectors straight out. Figure 30. Power and Control Cables AC Connector with tab at bottom Remove a XC Series-AC PDU 31

32 6. Slide the PDU assembly about 1 inch out of the cabinet to access the nut. Remove the 7/16-in. hex nut from the ground lug at back of the PDU. 7. Slide the PDU straight out of the cabinet. Figure 31. 7/16-in. Hex Nut AC 7/16-in. hex nut Install a XC Series-AC Power Distribution Unit Part Number: PDU Assy, XC30-AC 400/480Vac Cabinet PDU Assy, XC30-AC 200/208Vac Cabinet NOTE: Use the correct part according to the customer site AC line voltage. Time: 15 minutes Tools: T15 TORX driver T25 TORX driver Large slotted screwdriver 7/16-in. open end wrench This procedure may require a Cray contracted or customer contracted electrician to complete lockout/tagout procedures or disconnect facility power. Allow time in the preventive maintenance schedule for this requirement. 1. Position the back of the PDU assembly onto the support rails, and slide the PDU partially into the cabinet. 2. Reinstall the 1/4-in. hex nut on the ground lug at the back of the PDU. Install a XC Series-AC Power Distribution Unit 32

33 () Figure 32. 7/16-in. Hex Nut AC 7/16-in. hex nut 3. Reconnect cables J1 through J5 to the side of the PDU. Cables J1 through J3 connect top to top, middle to middle, and bottom to bottom to the power supply rack. Figure 33. Power and Control Cables AC Connector with tab at bottom 4. Slide the PDU fully into the cabinet. 5. Reinstall the six IP25 TORX+ screws that attach the PDU to the cabinet frame. Figure 34. PDU Power Cable Clamps AC Ground block Circuit breaker input terminals IP25 TORX+ screws (6) PDU power PDU power cable clamp cable clamp 6. Have a Cray contracted or customer contracted electrician reconnect the input power cables to the PDU circuit breakers by using the following procedure: a. Reinstall the input power cables through the support plates in the bottom of the PDU and reattach the cable clamp. Install a XC Series-AC Power Distribution Unit 33

34 b. Reconnect the PDU power cable wires to the ground blocks and the input terminals of circuit breaker CB1 (use appropriate site procedures). c. Replace the front cover on the PDU and reinstall the twenty screws. Figure 35. PDU Circuit Breaker Input Terminals AC Circuit breaker input terminals Ground block 7. Work with the site electrician to verify that circuit breakers for the cabinet at the wall panel are turned on. 8. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Figure 36. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC Power Supply Rack Part Number: Power Rack Assy, 33KW Time: minutes Tools: T15 TORX driver Remove a XC Series-AC Power Supply Rack 34

35 T25 TORX driver Large slotted screwdriver This procedure may require a Cray contracted or customer contracted electrician to complete lockout/tagout procedures or disconnect facility power. Allow time in the preventive maintenance schedule for this requirement. 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Figure 37. Circuit Breaker for XC Series Cabinet 2. WARNING: Electrical hazard Make sure the circuit breaker for the cabinet is off, locked, and tagged. Failure to do so could result in severe shock and burns. Work with the site electrician to verify that circuit breakers for the cabinet at the wall panel are off. Use appropriate lockout/tagout or other site-specific procedures to indicate that the breakers are off. 3. Open the front door of the cabinet. 4. Remove each power supply from the power supply rack by loosening each thumb screw at the top right of the power supply and pulling each extractor handle down. Remove the power supply by pulling it outward. Remove a XC Series-AC Power Supply Rack 35

36 Figure 38. Power Supply Rack AC IP25 TORX+ screws (8) with clip nuts Loosen each thumb screw and pull each extractor handle forward to allow for power supply removal. 5. Carefully place each power supply on an ESD-safe surface. 6. Label and disconnect the three control cables from the side of the power supply rack. 7. Label and disconnect the three power cables from the side of the power supply rack. Figure 39. Power Supply Rack Cabling AC Control cables Power cables 8. Remove the 10 T25 TORX screws from the lower vertical bus bar cover and remove the cover from the cabinet. 9. Remove the three T20 TORX screws from the power supply bus bar cover and remove the cover from the cabinet. Remove a XC Series-AC Power Supply Rack 36

37 Figure 40. Power Supply Bus Bar Cover AC Power supply bus bar cover 10. Remove 10 T25 TORX screws from the lower (red) vertical bus bar. 11. CAUTION: Component damage The eight IP25 TORX+ screws are clip nuts. Be careful when removing the screws to avoid damaging the clip nuts. Remove the eight IP25 TORX+ screws that attach the power supply rack to the cabinet frame. 12. Slide the power supply rack straight out of the cabinet and place the power supply rack on a sturdy work surface. 13. Remove the twelve 3/8-in. nuts and conical washers that secure the power supply rack bus bar to the power supply rack. Figure 41. Power Supply Rack Bus Bar AC Power supply bus bar 3/8-in. nuts (12) Install a XC Series-AC Power Supply Rack 37

38 Install a XC Series-AC Power Supply Rack Part Number: Power Rack Assy, 33KW Time: minutes Tools: T15 TORX driver T25 TORX driver Large slotted screwdriver This procedure may require a Cray contracted or customer contracted electrician to complete lockout/tagout procedures or disconnect facility power. Allow time in the preventive maintenance schedule for this requirement. 1. Reinstall the twelve 3/8-in. nuts, with conical washers, that attach the power supply bus bar to the new power supply rack. 2. Torque the 3/8-in. nuts to 19 in-lbs. 3. Position the back of the power supply rack assembly onto the support rails, and slide the power supply rack partially into the cabinet. When inserting the power supply rack, carefully align the bus bars so the power supply bus rack (red) is above the lower vertical bus bar (silver). 4. Reinstall the ten T25 TORX screws into the lower (red) vertical bus bar. Figure 42. Bus Bar Alignment AC Power supply bus bar in front of other bus bar 5. CAUTION: Component damage The eight IP25 TORX+ screws are clip nuts. Be careful when installing the screws to avoid damaging the clip nuts. Reinstall, but do not tighten, the eight IP25 TORX+ screws that attach the power supply rack to the cabinet frame. 6. Tighten the eight IP25 TORX+ screws. Install a XC Series-AC Power Supply Rack 38

39 7. Insert the lower bus bar cover into the cabinet and reinstall the ten T25 TORX screws. 8. Insert the power supply bus cover and reinstall the three T20 TORX screws. 9. Reconnect the three power cables to the side of the power supply rack. 10. Reconnect the three control cables to the side of the power supply rack. 11. Reinstall each power supply in the power supply rack by pushing it into the slot. Tighten each thumb screw at the top right of the power supply and pull each extractor handle up. 12. Work with the site electrician to verify that circuit breakers for the cabinet at the wall panel are turned on. 13. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Figure 43. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC Cabinet Controller Part Number: Module Assy, Cabinet Controller Time: 5 minutes Tools: Small slotted screwdriver A XC Series-AC Cabinet Controller Replacement is avaliable which shows how to remove and install this component. Remove a XC Series-AC Cabinet Controller 39

40 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Figure 44. Circuit Breaker for XC Series Cabinet 2. Open the front door of the cabinet. 3. Disconnect the Ethernet and serial cables from the front of the cabinet controller and label all cables. 4. Simultaneously pull out on both extractor handles until they stop. 5. Slide the cabinet controller straight out of the cabinet and set it on an ESD-safe surface. Figure 45. Cabinet Controller Ethernet cables Serial cables Extractor handles Install a XC Series-AC Cabinet Controller Part Number: Module Assy, Cabinet Controller Time: 5 minutes Tools: Small slotted screwdriver Install a XC Series-AC Cabinet Controller 40

41 A XC Series-AC Cabinet Controller Replacement video is avaliable which shows how to remove and install this component. 1. Align the guide rail on the cabinet controller with the guide slot in the front of the chassis. Figure 46. Cabinet Controller Ethernet cables Serial cables Extractor handles 2. Slide the cabinet controller into the chassis until it stops; push the center of the face plate to avoid pinching your fingers under the extractor handles. 3. Engage the tabs on the extractor handles with the extractor flanges on the sides of the chassis. 4. Simultaneously push in on both extractor handles until they lock in place. 5. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Figure 47. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC Cabinet Controller Backplane 41

42 Remove a XC Series-AC Cabinet Controller Backplane Part Number: Backplane Module Assy, Cabinet Controller Time: 30 minutes Tools: T15 TORX driver Small slotted screwdriver The X and Y cabinet address information, cabinet type, and coordinated cooling region (CCR) position for the cabinet controller is stored on the cabinet controller backplane SEEP. This information must be re-entered on the operator interface when replacing the cabinet controller backplane. 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Figure 48. Circuit Breaker for XC Series Cabinet 2. Open the front door of the cabinet. 3. Remove the cabinet controller. 4. Label and disconnect the environmental distribution board cable and operator interface cables from the cabinet controller backplane at the rear of the cabinet. 5. Remove the IP15 TORX+ screws that attach the cabinet controller backplane to the cabinet backplane. 6. Remove the module from slot 15. Remove a XC Series-AC Cabinet Controller Backplane 42

43 7. Push the backplane assembly towards the front of the cabinet, then working from the front, slide the backplane assembly out and set it on an ESD-safe surface. Figure 49. Cabinet Controller Cables Install a XC Series-AC Cabinet Controller Backplane Part Number: Backplane Module Assy, Cabinet Controller Time: 30 minutes Install a XC Series-AC Cabinet Controller Backplane 43

44 Tools: T15 TORX driver Small slotted screwdriver The X and Y cabinet address information, cabinet type, and coordinated cooling region (CCR) position for the cabinet controller is stored on the cabinet controller backplane SEEP. This information must be re-entered on the operator interface when replacing the cabinet controller backplane. 1. Align the guide rail on the cabinet controller backplane with the guide slot in the front of the chassis. 2. Slide the backplane assembly into the chassis until it stops. 3. Make sure slot 15 is empty. 4. Working from the rear of the cabinet, replace the 7 IP15 TORX+ screws that attach the cabinet controller backplane to the cabinet backplane. 5. Reconnect the environmental distribution board and operator interface cables to the backplane assembly. 6. Reinstall the cabinet controller. 7. Close the front door of the cabinet. Install a XC Series-AC Cabinet Controller Backplane 44

45 Figure 50. Cabinet Controller Cables 8. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Install a XC Series-AC Cabinet Controller Backplane 45

46 Figure 51. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC Cabinet Controller DIMM Part Number: DIMM, 2GB PC UNBFR DUAL RANK ECC Time: 5 minutes Tools: T15 TORX bit Adjustable torque driver, 5-40 in-lbs (P/N ) Small slotted screwdriver Dusting gas (P/N ) 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Remove a XC Series-AC Cabinet Controller DIMM 46

47 Figure 52. Circuit Breaker for XC Series Cabinet 2. Open the front door of the cabinet. 3. Disconnect the SMW ethernet cables and side band cables. 4. Remove the cabinet controller. 5. Remove the eleven T15 TORX screws that attach the shroud to the cabinet controller and remove the shroud. 6. Simultaneously push out on both DIMM cams with your thumbs to eject the DIMM from the socket. 7. Remove the DIMM and place it on an ESD-safe surface. Figure 53. Cabinet Controller DIMM Install a XC Series-AC Cabinet Controller DIMM Part Number: DIMM, 2GB PC UNBFR DUAL RANK ECC Time: 5 minutes Install a XC Series-AC Cabinet Controller DIMM 47

48 Tools: T15 TORX bit Adjustable torque driver, 5-40 in-lbs (P/N ) Small slotted screwdriver Dusting gas (P/N ) 1. Spray the DIMM contacts and the DIMM socket with dusting gas to remove any debris. Figure 54. Cabinet Controller DIMM 2. CAUTION: Equipment damage The DIMM connectors are keyed and the DIMM can be installed only in the proper orientation. If the connectors do not mate easily, rotate the DIMM 180 degrees. If you force the connectors together, you can damage the DIMM or the blade. CAUTION: Equipment damage Ensure that you push down simultaneously and evenly on both ends of the DIMM. If you rock the DIMM or seat one end before the other end, you can damage the DIMM or the blade. Position the DIMM into the socket and simultaneously push down on both ends of the card with your thumbs until the card seats and the cams snap closed. 3. Replace the shroud and reinstall the T15 TORX screws. Torque the screws to 9 in-lbs. 4. Reinstall the cabinet controller. 5. Connect the SMW Ethernet cables and side band cables. 6. Close the front door of the cabinet. 7. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Install a XC Series-AC Cabinet Controller DIMM 48

49 Figure 55. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC Chassis Host Part Number: Module Assy, Chassis Host Time: 5 minutes Tools: Small slotted screwdriver A XC Series-AC Chassis Host Replacement video is avaliable which shows how to remove and install this component. 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Remove a XC Series-AC Chassis Host 49

50 Figure 56. Circuit Breaker for XC Series Cabinet 2. Open the front door of the cabinet. 3. Label and disconnect the Ethernet and serial cables from the front of the chassis host. 4. Simultaneously push the release button on the end of both extractor handles and pull out on the handles until they stop. 5. Slide the chassis host straight out of the cabinet and set it on an ESD-safe surface. Figure 57. Chassis Host Install a XC Series-AC Chassis Host Part Number: Module Assy, Chassis Host Time: 5 minutes Tools: Small slotted screwdriver Install a XC Series-AC Chassis Host 50

51 A XC Series-AC Chassis Host Replacement video is avaliable which shows how to remove and install this component. 1. Align the guide rail on the chassis host with the guide slot in the front of the chassis. Figure 58. Chassis Host 2. Slide the chassis host into the chassis until it stops; push the center of the face plate to avoid pinching your fingers under the extractor handles. 3. Engage the tabs on the extractor handles with the extractor flanges on the sides of the chassis. 4. Simultaneously push in on both extractor handles until they lock in place. 5. Reconnect the Ethernet and serial cables to the front of the chassis host. 6. Close the door to the front cabinet. 7. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Figure 59. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. Install a XC Series-AC Chassis Host 51

52 c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC Passive Electrical Cable Part Number: Cable Assy, 10.0 GBS Electrical CXP 5 ft (1.5M) Cable Assy, 10.0 GBS Electrical CXP 10 ft (3.0M) Cable Assy, 10.0 GBS Electrical CXP 13 ft (4.0M) Cable Assy, 10.0 GBS Electrical CXP 16 ft (5.0M) Cable Assy, 10.0 GBS Electrical CXP 20 ft (6.0M) Time: 15 minutes Tools: T15 TORX driver T25 TORX driver 1/2-in. socket Socket driver You do not need to power off the cabinets or blade slots to remove optical network cables. 1. Label the replacement cable. 2. Route the replacement cable along the same path as the cable that you removed. All cables have a tendency to curve or twist in a particular direction; allow the cable to follow its natural path whenever possible. 3. Align the end of the cable with the backplane connector and push squarely and firmly on the cable to seat it in the connector. 4. Repeat step 1 through step 3 for the other end of the cable. 5. Replace the cable wraps that secure the cable to the adjacent cables. 6. Close the rear door of the cabinet. Remove a XC Series-AC Passive Electrical Cable 52

53 Figure 60. Passive Electrical Cables AC Cable release ring Install a XC Series-AC Passive Electrical Cable Part Number: Cable Assy, 10.0 GBS Electrical CXP 5 ft (1.5M) Cable Assy, 10.0 GBS Electrical CXP 10 ft (3.0M) Cable Assy, 10.0 GBS Electrical CXP 13 ft (4.0M) Cable Assy, 10.0 GBS Electrical CXP 16 ft (5.0M) Cable Assy, 10.0 GBS Electrical CXP 20 ft (6.0M) Time: 15 minutes Tools: T15 TORX driver T25 TORX driver 1/2-in. socket Socket driver You do not need to power off the cabinets or blade slots to remove optical network cables. 1. Label the replacement cable. 2. Route the replacement cable along the same path as the cable that you removed. All cables have a tendency to curve or twist in a particular direction; allow the cable to follow its natural path whenever possible. 3. Align the end of the cable with the backplane connector and push squarely and firmly on the cable to seat it in the connector. 4. Repeat step 1 through step 3 for the other end of the cable. 5. Replace the cable wraps that secure the cable to the adjacent cables. Install a XC Series-AC Passive Electrical Cable 53

54 6. Close the rear door of the cabinet. Remove a XC Series-AC Blower Assembly Part Number: Blower Assy, XC30-AC 400/480Vac Cabinet Blower Assy, XC30-AC 200/208Vac Cabinet Time: 5 minutes Tools: NOTE: Use the correct part according to the customer site AC line voltage. Small Phillips screwdriver Cable tie A XC Series-AC Blower Assembly Replacement video is avaliable which shows how to remove and install this component. 1. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. 2. Open the rear door of the cabinet. 3. Place the main circuit breaker CB1 in the OFF (down) position. Figure 61. Circuit Breaker for XC Series Cabinet 4. CAUTION: Electrical hazard Remove a XC Series-AC Blower Assembly 54

55 Electrical hazard. Make sure the circuit breaker for the cabinet is off, locked, and tagged. Failure to do so could result in severe shock and burns. Work with a site electrician to ensure that circuit breakers for the cabinet at the wall panel are off. Use appropriate lockout/tagout or other site-specific procedures to indicate that the breakers are off. 5. Remove the 14 Phillips screws that secure the blower assembly panel to the cabinet and remove the panel. Figure 62. Blower Assembly Panel AC Phillips screws (14) 6. Cut the cable tie that attaches the blower cable connector to the blower. Figure 63. Replace Cable Tie AC Cable tie 7. Disconnect the blower cable connector from the blower assembly. Remove a XC Series-AC Blower Assembly 55

56 Figure 64. Blower Cable Connector AC Blower cable connector 8. CAUTION: Personal injury or equipment damage The blower assembly weighs 116 lbs (53 kg). The blower assembly must be removed with the assistance of qualified personnel. Failure to do so may result in personal injury or equipment damage. With the assistance of qualified personnel, use proper lifting techniques to lift the blower assembly up and slide the blower assembly out of the cabinet. Figure 65. Blower Assembly Handling AC Blower insertion point Install a XC Series-AC Blower Assembly Part Number: Blower Assy, XC30-AC 400/480Vac Cabinet Blower Assy, XC30-AC 200/208Vac Cabinet NOTE: Use the correct part according to the customer site AC line voltage. Time: 5 minutes Install a XC Series-AC Blower Assembly 56

57 Tools: Small Phillips screwdriver Cable tie A XC Series-AC Blower Assembly Replacement video is avaliable which shows how to remove and install this component. 1. With the assistance of qualified personnel, use proper lifting techniques to slide the blower assembly into the racks. 2. Connect the blower cable connector to the blower assembly. 3. Attach a cable tie around the blower connector cable and the blower tie. Figure 66. Replace Cable Tie AC Cable tie 4. Position the blower assembly panel on the rack and install the 14 Phillips screws. 5. Work with the site electrician to verify that circuit breakers for the cabinet at the wall panel are turned on. 6. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Figure 67. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. Install a XC Series-AC Blower Assembly 57

58 c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC VFD Part Number: AC Drive, 7.5 HP 240V 3P 400/480Vac Cabinet AC Drive, 7.5 HP 240V 3P 200/208Vac Cabinet AC Drive, MODBUS Module NOTE: Use the correct part according to the customer site AC line voltage. Time: 15 minutes Tools: T25 TORX driver 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Figure 68. Circuit Breaker for XC Series Cabinet 2. CAUTION: Electrical hazard Electrical hazard. Make sure the circuit breaker for the cabinet is off, locked, and tagged. Failure to do so could result in severe shock and burns. Remove a XC Series-AC VFD 58

59 Work with a site electrician to ensure that circuit breakers for the cabinet at the wall panel are off. Use appropriate lockout/tagout or other site-specific procedures to indicate that the breakers are off. 3. Remove the blower assembly. 4. Remove the bottom right T25 TORX screw that attaches the ground wire from the inner VFD control to the VFD cover. Disconnect the ground wire from the screw. Figure 69. XC Series (Air Cooled) Variable Frequency Drive 5. Working from the rear of the cabinet, locate the outer VFD control cable coming from the environmental distribution board and disconnect it. Figure 70. XC Series (Air Cooled) VFD Outer Control Cable 6. Working from the rear of the cabinet, pinch the inner VFD control cable connector push it in to disconnect the control cable from the cabinet. Remove a XC Series-AC VFD 59

60 Figure 71. Inner VFD Control Cable AC Inner VFD control cable 7. Disconnect the VFD power cable connector from the EMI filter power cable connector. 8. NOTE: Component loss Insert foam or other material above the blower enclosure air hole to prevent screws from falling into the air hole. Remove the remaining three T25 screws that secure the VFD cover to the VFD. 9. Remove the VFD from the blower enclosure and set it on an ESD-safe surface. 10. Open the VFD terminal block cover. 11. Remove the black 5-pin terminal block connector from the cover by pulling it straight out of the modular unit. Figure 72. XC Series (Air Cooled) VFD Terminal Block 12. Note the position of the jumper wires on the 14-pin terminal block. 13. Label and disconnect the power and control wires from the 14-pin terminal block. 14. Remove the four 25IP TORX+ screws that attach the VFD to the bracket. Retain the bracket for installation on the new VFD. Remove a XC Series-AC VFD 60

61 Figure 73. XC Series (Air Cooled) VFD Install a XC Series-AC VFD Part Number: AC Drive, 7.5 HP 240V 3P 400/480Vac Cabinet AC Drive, 7.5 HP 240V 3P 200/208Vac Cabinet AC Drive, MODBUS Module NOTE: Use the correct part according to the customer site AC line voltage. Time: 15 minutes Tools: T25 TORX driver Phillips screwdriver 5/16 in. socket 1. If the replacement VFD has a metal plate where the terminal block cover attaches, use the following procedure to replace it with the blue plastic plate that is included in the box with the replacement VFD: a. Loosen the two Phillips screws that secure the terminal block cover and remove the cover. b. Remove the two 5/16-in. nuts that attach the metal plate to the VFD. c. Remove the metal plate and reinstall the 5/16-in. nuts. d. Snap out the three cable cutouts in the plastic plate. Install a XC Series-AC VFD 61

62 e. Engage the three tabs on the bottom of the plastic plate into the slots in the VFD, rotate the plate into position, and secure the two tabs on the top of the plate into the VFD. If no plastic plate is in the box with the new VFD, remove the plastic plate from the old VFD and install it in the new VFD. 2. Verify that the jumper wires are installed on the 14-pin VFD terminal block. If disconnected, refer to the old VFD to reattach the jumper wires. 3. Reinstall the four 25IP TORX+ screws to secure the VFD to the bracket. 4. Reconnect the power and control wires from the 14-pin terminal block. 5. Remove the empty 5-pin terminal block from the modular unit on the replacement VFD terminal block cover. 6. Reinstall the black 5-pin terminal block connector from the original terminal block cover into the replacement cover by pushing it straight into the modular unit. 7. Verify that the black cable that attaches the modular unit to the VFD is securely connected at both ends. 8. Reinstall the terminal block cover onto the VFD. 9. Position the VFD in the back of the blower enclosure and reinstall three of the four T25 screws to secure the VFD cover to the VFD. Do not attach the lower right screw. 10. Reconnect the ground wire to the inner VFD control cable and attach it to the lower right T25 screw. Reinstall the screw to the VFD cover. 11. Reconnect the VFD blower power cable connector to the EMI filter power cable connector. 12. Reconnect the inner VFD control cable to the blower enclosure. 13. Working from the rear of the cabinet, connect the outer VFD control cable. 14. Reinstall the blower assembly. 15. Work with the site electrician to verify that circuit breakers for the cabinet at the wall panel are turned on. 16. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Figure 74. Circuit Breaker for XC Series Cabinet Install a XC Series-AC VFD 62

63 b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Remove a XC Series-AC EMI Filter Part Number: Filter, Dust Fan Plenum Time: 10 minutes Tools: T25 TORX driver 1. Power down all of the cabinets in the row. a. Enter the following command in a terminal window on the SMW: smw$ xtcli power down hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or P0 (for a partition). The Cabinet ID can be found on the operator interface. a. Open the rear door of the cabinet. a. Place the main circuit breaker CB1 in the OFF (down) position. Figure 75. Circuit Breaker for XC Series Cabinet 2. CAUTION: Electrical hazard Electrical hazard. Make sure the circuit breaker for the cabinet is off, locked, and tagged. Failure to do so could result in severe shock and burns. Remove a XC Series-AC EMI Filter 63

64 Work with a site electrician to ensure that circuit breakers for the cabinet at the wall panel are off. Use appropriate lockout/tagout or other site-specific procedures to indicate that the breakers are off. 3. Remove the blower assembly. 4. Working from the rear of the cabinet, disconnect the outer blower power cable. Figure 76. Outer Blower Power Cable Connector AC Outer blower power cable connector 5. Working from the front of the cabinet, remove the inner blower power cable connector from the blower enclosure. It may be necessary to remove the VFD to access the blower power cable connector. Figure 77. Inner Blower Power Cable Connector AC Inner blower power cable connector 6. Remove the top two T25 screws from the EMI filter. 7. Loosen, but do not remove, the bottom two screws on the EMI filter. 8. Disconnect the EMI filter power cable connector from the VFD power cable connector. 9. Remove the EMI filter from the cabinet. Remove a XC Series-AC EMI Filter 64

65 Figure 78. EMI Filter AC T25 screws (remove top two, loosen bottom two) Bottom two screws not shown Install a XC Series-AC EMI Filter Part Number: Filter, Dust Fan Plenum Time: 10 minutes Tools: T25 TORX driver 1. Place the EMI filter on the bottom two screws. 2. Install the top two screws to secure the EMI filter to the blower enclosure (tighten all four T25 TORX screws). 3. Connect the inner blower power cable connector to the blower enclosure. 4. Working from the rear of the cabinet, connect the outer blower power cable. 5. Connect the EMI filter power cable connector to the VFD power cable connector. 6. If necessary, reinstall the VFD. 7. Install the blower assembly. 8. Work with the site electrician to verify that circuit breakers for the cabinet at the wall panel are turned on. 9. Power up all of the cabinets in the row. a. Place main circuit breaker CB1 in the ON (up) position. Install a XC Series-AC EMI Filter 65

66 Figure 79. Circuit Breaker for XC Series Cabinet b. Close the rear door of the cabinet. c. Enter the following command in a terminal window on the SMW: smw$ xtcli power up hostname Hostname is the cabinet, system, or partition ID; examples include c0-0 (for a cabinet), S0 (for a system), or p0 (for a partition). Install a XC Series-AC EMI Filter 66

67 XC Series Compute and IO Blades XC Series systems contain left and right compute blades and left IO blades. The procedure for replacing most blade FRUs is the same whether the FRU is on a left or right compute blade or an IO blade. Many of the illustrations in this section show the FRU on a right compute blade, but the procedure is the same when the component is on a left compute blade or IO blade, unless noted. The figures below show the location and name of each FRU on the left compute blade, and the IO blade FRUs. Each blade requires an Aries ASIC. On the right and left compute blades, the Aries ASIC is mounted on the right and left Aries network card (ANC). On the IO blade, the Aries ASIC is mounted on the IO base blade (IOBB). The FRU procedures for the compute and IO blades are identical for the XC Series-LC and XC Series-AC systems. 67

68 Figure 80. Left Compute Blade Component Identification Shroud Shroud screw (x12) Air duct QPDC power cables QPDC 0 Side diffuser rail VIVOC 1 Side diffuser rail QPDC 1 Aries ASIC ANC Aries VRM Backplane connector VIVOC 0 Electrical (copper) connectors 68

69 Figure 81. IO Blade Component Identification Shroud Shroud screw (x19) PCI mezzanine 0 (PCIe adapter top=1, bottom=0) PCI adapters DIMMs PCI mezzanine screws PCI mezzanine 1 (PCIe adapter top=1, bottom=0) PCI riser CPU VRM DIMMs CPU VRM HIVOC XPDC 0 IO base blade PCA VIVOC1 VIVOC0 VIVOC HIVOC XPDC 1 Aries ASIC Electrical (copper) connectors Backplane connector Remove a Compute or IO Blade Using xtwarmswap Part Number: Module Assy, Compute Right Module Assy, Compute Left Module Assy, I/O Left Module Assy, Diffuser Right ANC/QPDC Module Assy, Diffuser Left ANC/QPDC Time: 5 minutes Tools: T15 TORX driver Remove a Compute or IO Blade Using xtwarmswap 69

70 IMPORTANT: Only certain IO Blades can be warm swapped. Currently, an IO blade with only InfiniBand cards that is labeled LNET on the config sheet may be warm swapped. WARNING: System corruption Do not warm swap a boot node, SDB node, DVS node or login/network node. Serious system corruption may occur if an I/O blade is warm swapped during system operation. 1. CAUTION: Loss of data The following command terminates any applications that are running, so you must ensure that all applications on the blade are completed. Failure to do so can result in the loss of customer data. Avoid using xtwarmswap when mission critical applications are running. CAUTION: Equipment damage A slot cannot be left empty for more than two minutes on a running system. Leaving a slot empty for more than two minutes can lead to processor throttling. Avoid using xtwarmswap when mission critical applications are running. Before using xtwarmswap, verify that the system is re-routable using the rtr-s command.if the system cannot be routed, xtwarmswap cannot be used. CAUTION: Equipment overheating A filler block needs to be installed for ensure proper cooling. From the SMW as crayadm, use the following command to remove the suspect blade from the job allocation pool. smw$ xtprocadmin -k s admindown -n BladeID 2. Use the following command on the login node as root to halt jobs gracefully on the admindown slots. smw$ xtcli halt BladeID 3. Use the following command to route the high-speed network around the blade to be removed, and wait for this command to complete. smw$ xtwarmswap -r BladeID 4. Use the following command to power down the blade. This helps to identify the blade to pull. All lights on the blade will be off. smw$ xtcli power down BladeID 5. Open the front door of the cabinet. Remove a Compute or IO Blade Using xtwarmswap 70

71 6. Locate the blade that you need to remove. 7. Simultaneously push the release button on the end of both extractor handles and pull out on the handles until they stop. 8. Slide the blade approximately 3/4 of the way out of the cage. Grasp the front of the blade with one hand and the bottom of the blade near the center with the other hand and pull the blade completely out of the cage. Set the blade on an ESD-safe surface. 9. CAUTION: Equipment overheating If you cannot reinstall a blade in the empty slot within 2 minutes, install a filler blade assembly in the empty slot; use the "Installing the Compute Blade" procedure. Failure to do so can cause other blades in the system to overheat. Before returning a blade to Logistics, remove all of the QPDCs in the compute blade, XPDCs in the IO blade, voltage modules, and DIMMs. Figure 82. Extractor Handles Install a Compute or IO Blade Using xtwarmswap Part Number: Install a Compute or IO Blade Using xtwarmswap 71

72 Module Assy, Compute Right Module Assy, Compute Left Module Assy, I/O Left Module Assy, Diffuser Right ANC/QPDC Module Assy, Diffuser Left ANC/QPDC Time: 5 minutes Tools: T15 TORX driver 1. If you are installing an unpopulated spare blade, acclimatize the blade to the computer room before you remove it from the shipping case. Then reinstall the QPDCs or XPDCs, voltage modules, and DIMMs. 2. If you installed a filler blade assembly in the empty slot, remove the filler blade assembly. 3. Grasp the front of the blade with one hand and the bottom of the blade near the center with the other hand. 4. Align the left and right guide rails on the blade with the guide slots in the sides of the chassis. 5. Slide the blade straight into the cage until it stops; push the center of the face plate to avoid pinching your fingers under the extractor handles. 6. Engage the tabs on the extractor handles with the extractor flanges on the sides of the cage. 7. Simultaneously push in on both extractor handles until they lock in place. 8. Close the front door of the cabinet. 9. Use the following command to power on the blade. xtcli power up BladeID 10. Run the following command to verify a successful response from the blade controller on replacement blade. Wait for the blade to power on and respond to xtalive before continuing. smw$ xtalive BladeID 11. Run the following command to check the status of the blade slot. smw$ xtcli status -t BladeID 12. Run the the following command to power down then power up components in the Cray system. smw$ xtbounce BladeID 13. Run the xtzap -b BladeID command to update the BIOS, FPGA, and microcontroller firmware on all flash devices associated with the blade. a. If the SMW is running SMW 7.0.UP03.PS11 or later software, ensure the QPDC VRMs are updated using the following command. smw$ xtzap_vrm -t vdd BladeID Install a Compute or IO Blade Using xtwarmswap 72

73 b. Offline diagnostics can now be run on a single blade. 14. Run the following command to take down all HSN links on the specified blade. This prepares the blade for the warmswap. smw$ xtbounce linkdown BladeID 15. Run the the following command to activate routing on the newly installed blade. This command will xtbounce the blade. smw$ xtwarmswap -a BladeID 16. Run the the following command to boot compute node Linux. smw$ xtcli boot CNL0 BladeID 17. Run the the following command to mark the blade nodes up in the SDB. smw$ xtprocadmin -k s up -n BladeID Figure 83. Extractor Handles Remove a Compute or IO Blade Part Number: Remove a Compute or IO Blade 73

74 Module Assy, Compute Right Module Assy, Compute Left Module Assy, I/O Left Time: 5 minutes Tools: T15 TORX driver Two videos are avaliable that show how to install and remove these two components: XC Series-AC Compute Blade Replacement XC Series-AC IO Blade Replacement Use the xtwarmswap procedure when the operating system is running to remove a compute blade. Use this procedure when the operating system is not running. 1. Log on to the SMW as crayadm. 2. Input the following commands to power down a blade slot: a. Verify the operating system is not booted. smw# xtcli status cabinet_id b. Power down the blade slot. smw# xtcli power down_slot blade_id c. Check the status of the slot to ensure that it powered down properly. smw# xtcli status blade_id 3. Open the front door of the cabinet. 4. Locate the blade that you need to remove. 5. Simultaneously push the release button on the end of both extractor handles and pull out on the handles until they stop. 6. Slide the blade approximately 3/4 of the way out of the cabinet. Grasp the front of the blade with one hand and the bottom of the blade near the center with the other hand and pull the blade completely out of the cabinet. Set the blade on an ESD-safe surface. 7. Before you return a blade to Logistics, remove all of the QPDCs, voltage modules, DIMMs, etc. Install a Compute or IO Blade 74

75 Install a Compute or IO Blade Part Number: Module Assy, Compute Right Module Assy, Compute Left Module Assy, I/O Left Time: 5 minutes Tools: T15 TORX driver Two videos are avaliable that show how to install and remove these two components: XC Series-AC Compute Blade Replacement XC Series-AC IO Blade Replacement Use the xtwarmswap procedure when the operating system is running to install a compute blade. Use this procedure when the operating system is not running. 1. If you are installing an unpopulated spare blade, acclimatize the blade to the computer room before you remove it from the shipping case. Then reinstall the QPDCs, voltage modules, DIMMs, etc. 2. If a filler blade assembly is in the empty slot, remove the filler blade assembly. 3. Grasp the front of the blade with one hand and the bottom of the blade near the center with the other hand. 4. Align the left and right guide rails on the blade with the guide slots in the sides of the chassis. 5. Slide the blade straight into the cage until it stops; push the center of the face plate to avoid pinching your fingers under the extractor handles. 6. Engage the tabs on the extractor handles with the extractor flanges on the sides of the cage. 7. Simultaneously push in on both extractor handles until they lock in place. 8. Close the front door of the cabinet. 9. At the SMW, input the following commands to power up a blade slot: a. Power up the blade slot. smw# xtcli power up_slot blade_id b. Check the status of the slot to ensure that it powered up properly. smw# xtcli status blade_id c. Check for a response from the blade slot. Install a Compute or IO Blade 75

76 smw# xtalive d. Power up the components of the blade slot. smw# xtbounce blade_id e. Check the status of the slot to ensure it powered up properly. smw# xtcli status cabinet_id Remove a Memory DIMM You cannot mix different size and speed grade memory DIMMs on a blade and you must install identical (P/N) memory DIMMs in all of the slots that are common to a node. Part Number: DIMM, 4GB PC Reg Dual Rank ECC (Micron) DIMM, 4GB PC Reg Dual Rank ECC (Samsung) DIMM, 8GB PC Reg Dual Rank ECC (Hynix) DIMM, 8GB PC Reg Dual Rank ECC (Micron) DIMM, 8GB PC Reg Dual Rank ECC (Samsung) DIMM, 16GB PC Reg Dual Rank ECC (Hynix) DIMM, 16GB PC Reg Dual Rank ECC (Samsung) DIMM, 8GB DDR3 PC Reg Dual-Rank ECC (Hynix) DIMM, 8GB DDR3 PC Reg Dual-Rank ECC (Micron) DIMM, 8GB DDR3 PC Reg Dual-Rank ECC (Samsung) DIMM, 16GB PC Reg Dual-Rank ECC (Hynix) DIMM, 16GB PC Reg Dual-Rank ECC (Micron) DIMM, 16GB PC Reg Dual-Rank ECC (Samsung) Time: 7-8 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T10 TORX bit T15 TORX bit Dusting gas (P/N ) This procedure shows how to remove a memory DIMM from a compute or IO blade. 1. Locate the blade that contains the defective DIMM and remove the blade from the cabinet. Remove a Memory DIMM 76

77 2. Remove the T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Use the T10 to remove the air diffuser side rail closest to the defective DIMM, if necessary. Note how the air diffuser rail is positioned on the standoffs for proper installation. 4. If removing the DIMM from a compute blade, disconnect the power cable connectors and remove the power cable assembly. (Press the release tab on the connector and pull the cable straight out.) 5. Locate the defective DIMM. 6. Simultaneously push out on both DIMM cams with your thumbs to eject the DIMM from the socket. 7. Remove the memory DIMM and place it on an ESD-safe surface. Figure 84. QPDC Component Locations Figure 85. XPDC Component Locations XPDC 0 - Logical node 2 PCIe card PCIe slot 1 Slot mezzanine (upper) PCIe card PCIe slot 3 slot mezzanine (upper) PCIe slot 0 mezzanine (lower) PCIe slot 2 mezzanine (lower) PCIe IVOC J6 DIMM 0 J6 DIMM 0 J7 DIMM 1 J7 DIMM 1 CPU 0 XU1 CPU 0 XU1 J2, J3, Node VRM J2, J3, Node VRM J8 DIMM 3 J9 DIMM 2 J8 DIMM 3 J9 DIMM 2 U1 PATSBURG Node IVOC U1 PATSBURG Node IVOC J10 DIMM 0 J11 DIMM 1 J10 DIMM 0 J11 DIMM 1 CPU 1 Not XU1 Populated CPU Not 1 Populated XU1 J12 DIMM 3 J13 DIMM 2 J12 DIMM 3 J13 DIMM 2 IO Base Board J44 XPDC 1 - Logical node 1 Remove a Memory DIMM 77

78 Install a Memory DIMM You cannot mix different size and speed grade memory DIMMs on a blade and you must install identical (P/N) memory DIMMs in all of the slots that are common to a node. Part Number: DIMM, 4GB PC Reg Dual Rank ECC (Micron) DIMM, 4GB PC Reg Dual Rank ECC (Samsung) DIMM, 8GB PC Reg Dual Rank ECC (Hynix) DIMM, 8GB PC Reg Dual Rank ECC (Micron) DIMM, 8GB PC Reg Dual Rank ECC (Samsung) DIMM, 16GB PC Reg Dual Rank ECC (Hynix) DIMM, 16GB PC Reg Dual Rank ECC (Samsung) DIMM, 8GB DDR3 PC Reg Dual-Rank ECC (Hynix) DIMM, 8GB DDR3 PC Reg Dual-Rank ECC (Micron) DIMM, 8GB DDR3 PC Reg Dual-Rank ECC (Samsung) DIMM, 16GB PC Reg Dual-Rank ECC (Hynix) DIMM, 16GB PC Reg Dual-Rank ECC (Micron) DIMM, 16GB PC Reg Dual-Rank ECC (Samsung) Time: 7-8 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T10 TORX bit T15 TORX bit Dusting gas (P/N ) Install a Memory DIMM 1. Spray the DIMM contacts and the DIMM socket with dusting gas to remove any debris. 2. CAUTION: Equipment damage DIMMs are keyed so that you can insert it only in the proper orientation. If the connectors do not mate easily, rotate the DIMM 180 degrees. If you force the connectors together, you can damage the DIMM or the blade. Ensure that you push down simultaneously and evenly on both ends of the DIMM. If you rock the DIMM or seat one end before the other end, you can damage the DIMM or the blade. Install a Memory DIMM 78

79 Position the DIMM into the socket and simultaneously push down on both ends of the DIMM with your thumbs until the card seats and the cams snap closed. 3. If you are installing the DIMM in a compute blade, position the power cable assembly on the blade. Insert the two tabs on the center of the cable into the standoffs on the blade and then reconnect the cable connectors. 4. Reinstall the air diffuser side rail, if necessary. Diffuser has one T10 screw on compute blades and 4 on IO blades. Torque to 5.5 lbs. 5. Place the shroud and reinstall the T15 TORX shroud screws. Torque the screws to 9 in-lbs. 6. Reinstall the blade. Remove a Voltage Module (VRM, VIVOC, and HIVOC) Four types of voltage modules are attached to the compute and IO blades: Aries VRMs, vertical 48 12Vdc and 52-13Vdc VRMs (VIVOCs), horizontal 48 12Vdc and 52-13Vdc VRMs (HIVOCs), and Intel socket VRMs (IO blades only). The replacement procedure is identical for all types of VRMs on all blades, except as noted. Part Number: PCA, IR Aries VRM, Compute and IO Voltage Converter Module, Vertical 300W 52-13VDC, IO (IOBB VIVOC) Voltage Converter Module, Vertical 300W 48-12VDC, Compute (ANC VIVOC) Voltage Converter Module, Horizontal 456W 48-12VDC, QPDC and SGDC (HIVOC) Voltage Converter Module, Horizontal 430W 52-13VDC, XPDC (HIVOC) PCA, Intel Socket R Voltera VRM, XPDC Time: 7-8 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) This procedure shows how to remove a voltage module from a compute or IO blade. 1. Locate the blade that contains the defective voltage module and remove the blade from the cabinet. 2. Remove the T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Locate the defective voltage module. 4. Remove the following components as necessary for voltage module removal. Remove a Voltage Module (VRM, VIVOC, and HIVOC) 79

80 () a. Remove the air diffuser side rail closest to the defective voltage module removal, if necessary. Note how the air diffuser rail is positioned on the standoffs for proper installation. b. If removing a HIVOC from a compute blade, disconnect the power cable connectors and remove the power cable assembly. (Press the release tab on the connector and pull the cable straight out.) c. If removing a HIVOC, remove the two IP15 TORX+ screws that attach the voltage module to the QPDC or XDPC. Figure 86. Power Cable Tabs Figure 87. Power Cable Installation Connector Power cable connector 5. Use the appropriate removal procedure according to the type of voltage module. a. If removing an Aries VRM, or CPU VRM, grab the corners of the voltage module and carefully wiggle it side to side to disconnect it from the VRM connector. b. For all other voltage modules, grasp the edges of the voltage module and pull out to disconnect it from the VRM connector. Remove a Voltage Module (VRM, VIVOC, and HIVOC) 80

81 Figure 88. VRM Removal 6. Remove the voltage module and place it on an ESD-safe surface. Install a Voltage Module (VRM, VIVOC, and HIVOC) Four types of voltage modules are attached to the compute and IO blades: Aries VRMs, vertical 48 12Vdc and 52-13Vdc VRMs (VIVOCs), horizontal 48 12Vdc and 52-13Vdc VRMs (HIVOCs), and Intel socket VRMs (IO blades only). The replacement procedure is identical for all types of VRMs on all blades, except as noted. Part Number: PCA, IR Aries VRM, Compute and IO Voltage Converter Module, Vertical 300W 52-13VDC, IO (IOBB VIVOC) Voltage Converter Module, Vertical 300W 48-12VDC, Compute (ANC VIVOC) Voltage Converter Module, Horizontal 456W 48-12VDC, QPDC and SGDC (HIVOC) Voltage Converter Module, Horizontal 430W 52-13VDC, XPDC (HIVOC) PCA, Intel Socket R Voltera VRM, XPDC Time: 7-8 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) Install a Voltage Module (VRM, VIVOC, and HIVOC) 81

82 Install a Voltage Module 1. CAUTION: Equipment damage HIVOCs and VIVOCs have the same key. Verify the replacement components part number before inserting it into the connector. Inserting the wrong replacement component into a connector can result in equipment damage. The voltage module connectors are keyed for orientation and the voltage module can be installed only in the proper orientation. If the connectors do not mate easily, rotate the voltage module 180 degrees. If you force the connectors together, you can damage the voltage module or the blade. Spray the voltage module contacts and the voltage module socket with dusting gas to remove any debris. 2. Position the voltage module in the socket and simultaneously push down on both ends of the voltage module with your thumbs until the voltage module seats. Figure 89. VRM Removal 3. Reinstall the following components as necessary for voltage module replacement. a. If installing a HIVOC, replace the two IP15 TORX+ attachment screws. Torque the screws to 5.5 in-lbs. b. If installing a HIVOC in a compute blade, position the power cable assembly on the blade. Insert the two tabs on the center of the cable into the standoffs on the blade and then reconnect the cable connectors. c. Reinstall the air diffuser side rail, if necessary. Torque the screws to 5.5 in-lbs. Install a Voltage Module (VRM, VIVOC, and HIVOC) 82

83 () Figure 90. Power Cable Tabs Figure 91. Power Cable Installation Connector Power cable connector 4. Place the shroud and reinstall the T15 TORX attachment screws. Torque the screws to 9 in-lbs. 5. Reinstall the blade. Remove a Processor The replacement procedure is identical for all types of processors on all blades, except as noted. Part Number: IC, SANDYBRIDGE INTEL 8-CORE E GHZ FOR QPDC IC, SANDYBRIDGE INTEL 8-CORE E GHZ 20M CACHE IC, PROCESSOR E GHZ 6C 20MB SNB 135W 8.00 GT/S C2 Remove a Processor 83

84 IC, IVYBRIDGE INTEL 10-CORE E GHZ 25M CACHE 115W C IC, IVYBRIDGE INTEL 10-CORE E GHZ 25M CACHE 115W C IC, IVYBRIDGE INTEL 10-CORE E GHZ 25M CACHE 115W C IC, IVYBRIDGE INTEL 12-CORE E GHZ 30G CACHE 115W C0 QPDC IC, IVYBRIDGE INTEL 12-CORE E GHZ 30G CACHE 115W C1 QPDC IC, IVYBRIDGE INTEL 12-CORE E GHZ 30G CACHE 130W C0 QPDC IC, IVYBRIDGE INTEL 12-CORE E GHZ 30G CACHE 130W C1 QPDC IC, HASWELL INTEL 10-CORE E5-2650V3 2.5GHZ 105W M1 HPDC IC, HASWELL INTEL 12-CORE E5-2670V3 2.3GHZ 120W M1 HPDC IC, HASWELL INTEL 12-CORE E5-2680V3 2.5GHZ 120W M1 HPDC IC, HASWELL INTEL 12-CORE E5-2690V3 2.6GHZ 135W M1 HPDC IC, HASWELL INTEL 14-CORE E5-2695V3 2.3GHZ 120W C1 HPDC IC, HASWELL INTEL 16-CORE E5-2698V3 2.3GHZ 135W C1 HPDC IC, KNIGHTS CORNER (KNC) INTEL XEON PHI W/HEATSINKS FRU GPDC Time: 15 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T10 TORX bit T15 TORX bit Dusting gas (P/N ) Thermal grease (P/N ) Isopropyl alcohol Absorbent wipes Pen-Vac processor insertion/removal tool This procedure shows how to remove a processor from a compute or IO blade. 1. Locate the blade that contains the defective processor and remove the blade from the cabinet. 2. Remove the T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Remove the air diffuser side rail closest to the defective processor, if necessary. Note how the air diffuser rail is positioned on the standoffs for proper installation. 4. If removing the processor from a compute blade, disconnect the power cable connectors and remove the power cable assembly. (Press the release tab on the connector and pull the cable straight out.) 5. Locate the defective processor and remove the processor air duct covering it. 6. Loosen the four captive T15 TORX bolts that attach the processor heat sink to the blade in an alternate crisscross pattern until they are completely loose. Remove a Processor 84

85 Figure 92. Processor Heat Sink 7. Remove the heat sink and retain it for installation on the replacement processor. If removing more than one processor, note the position of each processor heat sink so each heat sink is returned to its original position. 8. Use a dry, clean cloth to remove the thermal grease from the processor; do not use a liquid, such as isopropyl alcohol. Ensure that the grease does not get under the socket cover. 9. Open the lever near the unlocked symbol and rotate the lever up 100 degrees. 10. Open the second lever and rotate the lever up 100 degrees. 11. Rotate the socket cover up and remove any remaining thermal grease from the processor and the socket cover. 12. Cray recommends using a pickup pen to pick up the processor. Use a pickup pen to carefully lift the processor out of the socket. DO NOT DROP the processor on the socket pins. Figure 93. Intel Processor Remove Tool - Attach Pen Press Figure 94. Intel Processor Remove Tool - Release Pen Release Remove a Processor 85

86 Figure 95. Processor Install a Processor Part Number: IC, SANDYBRIDGE INTEL 8-CORE E GHZ FOR QPDC IC, SANDYBRIDGE INTEL 8-CORE E GHZ 20M CACHE IC, PROCESSOR E GHZ 6C 20MB SNB 135W 8.00 GT/S C IC, IVYBRIDGE INTEL 10-CORE E GHZ 25M CACHE 115W C IC, IVYBRIDGE INTEL 10-CORE E GHZ 25M CACHE 115W C IC, IVYBRIDGE INTEL 10-CORE E GHZ 25M CACHE 115W C IC, IVYBRIDGE INTEL 12-CORE E GHZ 30G CACHE 115W C0 QPDC IC, IVYBRIDGE INTEL 12-CORE E GHZ 30G CACHE 115W C1 QPDC IC, IVYBRIDGE INTEL 12-CORE E GHZ 30G CACHE 130W C0 QPDC IC, IVYBRIDGE INTEL 12-CORE E GHZ 30G CACHE 130W C1 QPDC IC, HASWELL INTEL 10-CORE E5-2650V3 2.5GHZ 105W M1 HPDC IC, HASWELL INTEL 12-CORE E5-2670V3 2.3GHZ 120W M1 HPDC IC, HASWELL INTEL 12-CORE E5-2680V3 2.5GHZ 120W M1 HPDC IC, HASWELL INTEL 12-CORE E5-2690V3 2.6GHZ 135W M1 HPDC IC, HASWELL INTEL 14-CORE E5-2695V3 2.3GHZ 120W C1 HPDC IC, HASWELL INTEL 16-CORE E5-2698V3 2.3GHZ 135W C1 HPDC Install a Processor 86

87 IC, KNIGHTS CORNER (KNC) INTEL XEON PHI W/HEATSINKS FRU GPDC Time: 15 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T10 TORX bit T15 TORX bit Dusting gas (P/N ) Thermal grease (P/N ) Isopropyl alcohol Absorbent wipes Pen-Vac processor insertion/removal tool Install a Processor 1. CAUTION: Equipment damage The processor is keyed so that you can insert it only in the proper orientation. If the processor does not mate easily, rotate the processor 90 degrees. If you force the processor into the socket, you can damage the processor or the blade. The processor is difficult to grasp. If you drop the processor into the socket, you can damage the pins in the socket. Cray recommends using a pickup pen to pick up the processor. Use the pick up pen to pick up the processor and immediately place your other hand underneath. Figure 96. Processor Install Tool - Attach Protect socket pins Released 2. Orient the processor in the socket and set it into position. Install a Processor 87

88 Figure 97. Processor Install Tool - Insert Released 3. Press the vacuum release button on the pickup pen to release the processor. Figure 98. Processor Install Tool - Release Press 4. Close the socket cover. 5. Close the lever near the locked symbol and the second lever to engage the lock. Install a Processor 88

89 Figure 99. Processor 6. Use isopropyl alcohol to completely remove the thermal grease from the heat sink that you removed previously. 7. Apply thermal grease in four areas on top of the processor. Figure 100. Thermal Grease Application 8. Position the heat sink on the processor socket. If more than one processor heat sink was removed, each heat sink needs to be reinstalled in its original position. Processor heat sinks are not interchangeable. 9. CAUTION: Equipment damage Ensure that the heat sink is level as you tighten the attachment screws. If the heat sink is tilted, you can damage the threads in the processor socket. Install a Processor 89

90 Hold the heat sink level and with a hand tool, tighten the four T15 TORX bolts that attach the heat sink to the socket; tighten the bolts alternately in a criss-cross pattern, 1 turn each, until they are snug. 10. Torque the T15 TORX bolts to 9 in-lbs. Figure 101. Processor Heat Sink T15 TORX bolts (4 bolts) 11. Place the processor air duct over the processor. 12. If installing a processor in a compute blade, position the power cable assembly on the blade. Insert the two tabs on the center of the cable into the standoffs on the blade and then reconnect the cable connectors. 13. Reinstall the air diffuser side rail. Torque T10 screw to 5.5 in lbs. 14. Place the shroud and reinstall the T15 TORX screws. Torque the screws to 9 in-lbs. 15. Reinstall the blade. Remove a Quad Processor Daughter Card (QPDC) Part Number: Module Assy, QPDC (shipped without DIMMs and processors) Time: Tools: NOTE: If you are installing a spare QPDC, you must transfer the DIMMs and processors from the defective QPDC to the new QPDC. 15 minutes to remove and replace a populated QPDC 30 minutes to remove a QPDC, transfer the DIMMs and processors to an unpopulated spare QPDC, and install the QPDC Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) Remove a Quad Processor Daughter Card (QPDC) 90

91 This procedure shows how to remove a quad processor daughter card (QPDC) from a compute blade. 1. Locate the blade that contains the defective QPDC and remove the blade from the cabinet. 2. Remove the 12 T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Remove the air diffuser side rail from the compute blade. Note how the air diffuser side rail is positioned on the standoffs for proper installation. 4. Disconnect the power cable connectors and remove the power cable assembly. (Press the release tab on the connector and pull the cable straight out.) 5. Locate the defective QPDC and remove the processor air ducts covering it. 6. Use the adjustable torque driver to remove the eight standoffs that attach the QPDC to the blade (the bit holder on the torque driver is the same size as the standoff). 7. CAUTION: Equipment damage Carefully place the defective QPDC on a soft, ESD-safe surface with the components facing upward. Place the QPDC down directly and do not drag the QPDC on the surface. Failure to do so can lead to component damage. Grasp the edges of the QPDC and carefully wiggle the circuit board while pulling straight up to remove it from the blade. Figure 102. Compute Blade Component Identification LOGICAL NODE 2 LOGICAL NODE 1 LOGICAL NODE 3 LOGICAL NODE 0 J1000 DIMM 0 J1001 DIMM 1 J2002 DIMM 2 J2003 DIMM 3 CPU 0 XU1000 Physical Node 0 CPU 1 XU2000 J1003 DIMM 3 J1002 DIMM 2 J2001 DIMM 1 J2000 DIMM 0 U6000 PATSBURG J8001 IVOC CONN J8000 IVOC CONN U8000 QLOC FPGA U7000 PATSBURG J4000 DIMM 0 J4001 DIMM 1 J3002 DIMM 2 J3003 DIMM 3 CPU 1 XU4000 Physical Node 1 CPU 0 XU3000 J4003 DIMM 3 J4002 DIMM 2 J3001 DIMM 1 J3000 DIMM 0 J1000 DIMM 0 J1001 DIMM 1 J2002 DIMM 2 J2003 DIMM 3 CPU 0 XU1000 Physical Node 0 CPU 1 XU2000 J1003 DIMM 3 J1002 DIMM 2 J2001 DIMM 1 J2000 DIMM 0 U6000 PATSBURG J8001 IVOC CONN J8000 IVOC CONN U8000 QLOC FPGA U7000 PATSBURG J4000 DIMM 0 J4001 DIMM 1 J3002 DIMM 2 J3003 DIMM 3 CPU 1 XU4000 Physical Node 1 CPU 0 XU3000 J4003 DIMM 3 J4002 DIMM 2 J3001 DIMM 1 J3000 DIMM 0 Left or Right Aries network card QPDC 0 Board QPDC 1 Board Cray XC30/XC40 Compute Blade with 2 QDPCs Install a Quad Processor Daughter Card (QPDC) Part Number: Module Assy, QPDC (shipped without DIMMs and processors) Install a Quad Processor Daughter Card (QPDC) 91

92 Time: Tools: NOTE: If you are installing a spare QPDC, you must transfer the DIMMs and processors from the defective QPDC to the new QPDC. 15 minutes to remove and replace a populated QPDC 30 minutes to remove a QPDC, transfer the DIMMs and processors to an unpopulated spare QPDC, and install the QPDC Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) This procedure shows how to install a quad processor daughter card (QPDC) from a compute blade. 1. Position the replacement QPDC on the blade and press down firmly in the area of the connector to seat it in the connector on the blade. 2. Hand tighten the eight standoffs that attach the QPDC to the blade. Torque the standoffs to 14 in-lbs. (The bit holder on the torque driver is the same size as the standoff.) 3. Record the location of the DIMMs and processors on the defective QPDC so you can reinstall them in the same locations on the replacement QPDC. 4. Remove the processors from the defective QPDC. 5. Install the processors on the replacement QPDC. 6. Remove the memory DIMMs from the defective QPDC 7. Install the memory DIMMs on the replacement QPDC. Figure 103. Compute Blade Component Identification LOGICAL NODE 2 LOGICAL NODE 1 LOGICAL NODE 3 LOGICAL NODE 0 J1000 DIMM 0 J1001 DIMM 1 J2002 DIMM 2 J2003 DIMM 3 CPU 0 XU1000 Physical Node 0 CPU 1 XU2000 J1003 DIMM 3 J1002 DIMM 2 J2001 DIMM 1 J2000 DIMM 0 U6000 PATSBURG J8001 IVOC CONN J8000 IVOC CONN U8000 QLOC FPGA U7000 PATSBURG J4000 DIMM 0 J4001 DIMM 1 J3002 DIMM 2 J3003 DIMM 3 CPU 1 XU4000 Physical Node 1 CPU 0 XU3000 J4003 DIMM 3 J4002 DIMM 2 J3001 DIMM 1 J3000 DIMM 0 J1000 DIMM 0 J1001 DIMM 1 J2002 DIMM 2 J2003 DIMM 3 CPU 0 XU1000 Physical Node 0 CPU 1 XU2000 J1003 DIMM 3 J1002 DIMM 2 J2001 DIMM 1 J2000 DIMM 0 U6000 PATSBURG J8001 IVOC CONN J8000 IVOC CONN U8000 QLOC FPGA U7000 PATSBURG J4000 DIMM 0 J4001 DIMM 1 J3002 DIMM 2 J3003 DIMM 3 CPU 1 XU4000 Physical Node 1 CPU 0 XU3000 J4003 DIMM 3 J4002 DIMM 2 J3001 DIMM 1 J3000 DIMM 0 Left or Right Aries network card QPDC 0 Board QPDC 1 Board Cray XC30/XC40 Compute Blade with 2 QDPCs Install a Quad Processor Daughter Card (QPDC) 92

93 8. Place the processor air ducts and position the power cable assembly on the blade. 9. Insert the two tabs on the center of the power cable into the standoffs on the blade and reconnect the cable connectors. 10. Reinstall the air diffuser side rail. Torque the screw to 5.5 in-lbs. 11. Place the shroud and reinstall the 12 T15 TORX attachment screws. Torque the screws to 9 in-lbs. 12. Reinstall the blade. Remove a Processor Daughter Card (XPDC) Part Number: Module Assy, Processor Daughter Card Cascade (shipped without DIMMs, Intel socket VRMs, and processors) Time: Tools: NOTE: If you are installing a spare QPDC, you must transfer the DIMMs and processors from the defective QPDC to the new QPDC. 15 minutes to remove and replace a populated XPDC 30 minutes to remove a XPDC, transfer the DIMMs, VRMs, and processors to an unpopulated spare XPDC, and install the XPDC Adjustable torque driver, 5 40 in-lbs (P/N ) T15X TORX bit T10 TORX bit Dusting gas (P/N ) This procedure shows how to remove a processor daughter card from an IO blade. 1. Locate the blade that contains the defective XPDC and remove the blade from the cabinet. 2. Remove the 19 T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Remove the air diffuser rail closest to the defective XPDC. Note how the air diffuser rail is positioned on the standoffs for proper installation. 4. Locate the defective XPDC and remove the processor air duct covering it. 5. Use the adjustable hex driver to remove the six 1/4-in standoffs that attach the XPDC to the blade (the bit holder on the hex driver is the same size as the standoff). 6. NOTE: Remove a Processor Daughter Card (XPDC) 93

94 Equipment damage Carefully place the defective XPDC on a soft, ESD-safe surface with the components facing upward. Place the XPDC down directly and do not drag the XPDC on the surface. Failure to do so can lead to component damage. Grasp the edges of the XPDC and wiggle toward the front of the blade as you pull straight up to remove it from the blade. Figure 104. XPDC Component Locations XPDC 0 - Logical node 2 PCIe card PCIe slot 1 Slot mezzanine (upper) PCIe card PCIe slot 3 slot mezzanine (upper) PCIe slot 0 mezzanine (lower) PCIe slot 2 mezzanine (lower) PCIe IVOC J6 DIMM 0 J6 DIMM 0 J7 DIMM 1 J7 DIMM 1 CPU 0 XU1 CPU 0 XU1 J2, J3, Node VRM J2, J3, Node VRM J8 DIMM 3 J9 DIMM 2 J8 DIMM 3 J9 DIMM 2 U1 PATSBURG Node IVOC U1 PATSBURG Node IVOC J10 DIMM 0 J11 DIMM 1 J10 DIMM 0 J11 DIMM 1 CPU 1 Not XU1 Populated CPU Not 1 Populated XU1 J12 DIMM 3 J13 DIMM 2 J12 DIMM 3 J13 DIMM 2 IO Base Board J44 XPDC 1 - Logical node 1 Install a Processor Daughter Card (XPDC) Part Number: Module Assy, Processor Daughter Card Cascade (shipped without DIMMs, Intel socket VRMs, and processors) Time: Tools: NOTE: If you are installing a spare QPDC, you must transfer the DIMMs and processors from the defective QPDC to the new QPDC. 15 minutes to remove and replace a populated XPDC 30 minutes to remove a XPDC, transfer the DIMMs, VRMs, and processors to an unpopulated spare XPDC, and install the XPDC Adjustable torque driver, 5 40 in-lbs (P/N ) TO15 TORX bit T10 TORX bit Dusting gas (P/N ) Install a Processor Daughter Card (XPDC) 94

95 Install an XPDC 1. Position the replacement XPDC on the blade and press down firmly in the area of the connector to seat it in the connector on the blade. 2. Hand tighten the six standoffs that attach the XPDC to the blade. Torque the standoffs to 14 in-lbs. (The bit holder on the torque driver is the same size as the standoff.) 3. Record the location of the DIMMs, VRMs, and processors on the defective XPDC so you can reinstall them in the same locations on the replacement XPDC. 4. Remove the processors from the defective XPDC. 5. Install the processors on the replacement XPDC. 6. Remove the memory DIMMs from the defective XPDC. 7. Install the memory DIMMs on the replacement XPDC. 8. Remove the VRMs from the defective XDPC. 9. Install the VRMs on the replacement XPDC. Figure 105. XPDC Component Locations XPDC 0 - Logical node 2 PCIe card PCIe slot 1 Slot mezzanine (upper) PCIe card PCIe slot 3 slot mezzanine (upper) PCIe slot 0 mezzanine (lower) PCIe slot 2 mezzanine (lower) PCIe IVOC J6 DIMM 0 J6 DIMM 0 J7 DIMM 1 J7 DIMM 1 CPU 0 XU1 CPU 0 XU1 J2, J3, Node VRM J2, J3, Node VRM J8 DIMM 3 J9 DIMM 2 J8 DIMM 3 J9 DIMM 2 U1 PATSBURG Node IVOC U1 PATSBURG Node IVOC J10 DIMM 0 J11 DIMM 1 J10 DIMM 0 J11 DIMM 1 CPU 1 Not XU1 Populated CPU Not 1 Populated XU1 J12 DIMM 3 J13 DIMM 2 J12 DIMM 3 J13 DIMM 2 IO Base Board J44 XPDC 1 - Logical node Place the processor air duct over the XPDC. 11. Reinstall the air diffuser side rail. Torque to 5.5 in-lbs. 12. Place the shroud and reinstall the 19 T15 TORX attachment screws. Torque the screws to 9 in-lbs. 13. Reinstall the blade. Remove a Kepler Processor Daughter Card (SGDC) 95

96 Remove a Kepler Processor Daughter Card (SGDC) Part Number: Module Assy, SGDC (shipped without DIMMs and processors) Time: Tools: NOTE: If you are installing a spare SGDC, you must transfer the DIMMs and processors from the defective SGDC to the new SGDC. 15 minutes to remove and replace a populated SGDC 30 minutes to remove a SGDC, transfer the DIMMs and processors to an unpopulated spare SGDC, and install the SGDC Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) This procedure shows how to remove a kepler processor daughter card (SGDC) from a compute blade. 1. Locate the blade that contains the defective SGDC and remove the blade from the cabinet. 2. Remove the 12 T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Remove the air diffuser side rail from the compute blade. Note how the air diffuser side rail is positioned on the standoffs for proper installation. 4. Disconnect the power cable connectors and remove the power cable assembly. (Press the release tab on the connector and pull the cable straight out.) 5. Locate the defective SGDC and remove the processor air ducts covering it. 6. Remove the 8 standoffs that hold the PDC to the ANC. 7. CAUTION: Equipment damage Carefully place the defective SGDC on a soft, ESD-safe surface with the components facing upward. Place the SGDC down directly and do not drag the SGDC on the surface. Failure to do so can lead to component damage. Grasp the edges of the SGDC and carefully wiggle the circuit board while pulling straight up to remove it from the blade. 8. Use the adjustable torque driver to remove the two standoffs from the rail that attach the SGDCs to the blade (the bit holder on the torque driver is the same size as the standoff). Remove a Kepler Processor Daughter Card (SGDC) 96

97 Install a Kepler Processor Daughter Card (SGDC) Part Number: Module Assy, SGDC (shipped without DIMMs and processors) Time: Tools: NOTE: If you are installing a spare SGDC, you must transfer the DIMMs and processors from the defective SGDC to the new SGDC. 15 minutes to remove and replace a populated SGDC 30 minutes to remove a SGDC, transfer the DIMMs and processors to an unpopulated spare SGDC, and install the SGDC Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) This procedure shows how to install a kepler processor daughter card (SGDC) from a compute blade. 1. Hand tighten the two standoffs that attach the SPDC to the blade. Torque the standoffs to 14 in-lbs. (The bit holder on the torque driver is the same size as the standoff.) 2. Position the replacement SGDC on the blade and press down firmly in the area of the connector to seat it in the connector on the blade. 3. Hand tighten the eight standoffs that attach the SPDC to the blade. Torque the standoffs to 14 in-lbs. (The bit holder on the torque driver is the same size as the standoff.) 4. Record the location of the DIMMs and processors on the defective SGDC so you can reinstall them in the same locations on the replacement SGDC. 5. Remove the processors from the defective SGDC. 6. Install the processors on the replacement SGDC. 7. Remove the memory DIMMs from the defective SGDC. 8. Install the memory DIMMs on the replacement SGDC. 9. Place the processor air ducts over the SGDC processors. The air duct labeled "G" position the power cable assembly on the blade. 10. Insert the two tabs on the center of the power cable into the standoffs on the blade and reconnect the cable connectors. Install a Kepler Processor Daughter Card (SGDC) 97

98 11. Reinstall the air diffuser side rail. Torque to 5.5 in-lbs. 12. Place the shroud and reinstall the 12 T10 TORX attachment screws. Torque the screws to 9 in-lbs. 13. Reinstall the blade. Remove a Graphic Processor Daughter Card (GPDC) Part Number: Module Assy, GPDC (shipped without DIMMs and processors) Time: Tools: NOTE: If you are installing a spare GPDC, you must transfer the DIMMs and processors from the defective GPDC to the new GPDC. 15 minutes to remove and replace a populated GPDC 30 minutes to remove a GPDC, transfer the DIMMs and processors to an unpopulated spare GPDC, and install the GPDC Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) This procedure shows how to remove a graphic processor daughter card (GPDC) from a compute blade. 1. Locate the blade that contains the defective GPDC and remove the blade from the cabinet. 2. Remove the 12 T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Remove the air diffuser side rail from the compute blade. Note how the air diffuser side rail is positioned on the standoffs for proper installation. 4. Disconnect the power cable connectors and remove the power cable assembly. (Press the release tab on the connector and pull the cable straight out.) 5. Locate the defective GPDC and remove the processor air ducts covering it. 6. Use the adjustable torque driver to remove the eight standoffs that attach the GPDC to the blade (the bit holder on the torque driver is the same size as the standoff). 7. CAUTION: Equipment damage Remove a Graphic Processor Daughter Card (GPDC) 98

99 Carefully place the defective GPDC on a soft, ESD-safe surface with the components facing upward. Place the GPDC down directly and do not drag the GPDC on the surface. Failure to do so can lead to component damage. Grasp the edges of the GPDC and carefully wiggle the circuit board while pulling straight up to remove it from the blade. Install a Graphic Processor Daughter Card (GPDC) Part Number: Module Assy, GPDC (shipped without DIMMs and processors) Time: Tools: NOTE: If you are installing a spare GPDC, you must transfer the DIMMs and processors from the defective GPDC to the new GPDC. 15 minutes to remove and replace a populated GPDC 30 minutes to remove a GPDC, transfer the DIMMs and processors to an unpopulated spare GPDC, and install the GPDC Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) This procedure shows how to install a graphic processor daughter card (GPDC) from a compute blade. 1. Position the replacement GPDC on the blade and press down firmly in the area of the connector to seat it in the connector on the blade. 2. Hand tighten the eight standoffs that attach the GPDC to the blade. Torque the standoffs to 14 in-lbs. (The bit holder on the torque driver is the same size as the standoff.) 3. Record the location of the DIMMs and processors on the defective GPDC so you can reinstall them in the same locations on the replacement GPDC. 4. Remove the processors from the defective GPDC. 5. Install the processors on the replacement GPDC. 6. Remove the memory DIMMs from the defective GPDC 7. Install the memory DIMMs on the replacement GPDC. Install a Graphic Processor Daughter Card (GPDC) 99

100 Figure 106. Compute Blade Component Identification LOGICAL NODE 2 LOGICAL NODE 1 LOGICAL NODE 3 LOGICAL NODE 0 J1000 DIMM 0 J1001 DIMM 1 J2002 DIMM 2 J2003 DIMM 3 CPU 0 XU1000 Physical Node 0 CPU 1 XU2000 J1003 DIMM 3 J1002 DIMM 2 J2001 DIMM 1 J2000 DIMM 0 U6000 PATSBURG J8001 IVOC CONN J8000 IVOC CONN U8000 QLOC FPGA U7000 PATSBURG J4000 DIMM 0 J4001 DIMM 1 J3002 DIMM 2 J3003 DIMM 3 CPU 1 XU4000 Physical Node 1 CPU 0 XU3000 J4003 DIMM 3 J4002 DIMM 2 J3001 DIMM 1 J3000 DIMM 0 J1000 DIMM 0 J1001 DIMM 1 J2002 DIMM 2 J2003 DIMM 3 CPU 0 XU1000 Physical Node 0 CPU 1 XU2000 J1003 DIMM 3 J1002 DIMM 2 J2001 DIMM 1 J2000 DIMM 0 U6000 PATSBURG J8001 IVOC CONN J8000 IVOC CONN U8000 QLOC FPGA U7000 PATSBURG J4000 DIMM 0 J4001 DIMM 1 J3002 DIMM 2 J3003 DIMM 3 CPU 1 XU4000 Physical Node 1 CPU 0 XU3000 J4003 DIMM 3 J4002 DIMM 2 J3001 DIMM 1 J3000 DIMM 0 Left or Right Aries network card QPDC 0 Board QPDC 1 Board Cray XC30/XC40 Compute Blade with 2 QDPCs 8. Place the processor air ducts and position the power cable assembly on the blade. 9. Insert the two tabs on the center of the power cable into the standoffs on the blade and reconnect the cable connectors. 10. Reinstall the air diffuser side rail. Torque the screw to 5.5 in-lbs. 11. Place the shroud and reinstall the 12 T15 TORX attachment screws. Torque the screws to 9 in-lbs. 12. Reinstall the blade. Remove a Haswell Processor Daughter Card (HPDC) Part Number: Module Assy, HPDC (shipped without DIMMs and processors) Time: Tools: NOTE: If you are installing a spare HPDC, you must transfer the DIMMs and processors from the defective HPDC to the new HPDC. 15 minutes to remove and replace a populated HPDC 30 minutes to remove a HPDC, transfer the DIMMs and processors to an unpopulated spare HPDC, and install the HPDC Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) Remove a Haswell Processor Daughter Card (HPDC) 100

101 This procedure shows how to remove a haswell processor daughter card (HPDC) from a compute blade. 1. Locate the blade that contains the defective HPDC and remove the blade from the cabinet. 2. Remove the 12 T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Remove the air diffuser side rail from the compute blade. Note how the air diffuser side rail is positioned on the standoffs for proper installation. 4. Disconnect the power cable connectors and remove the power cable assembly. (Press the release tab on the connector and pull the cable straight out.) 5. Locate the defective HPDC and remove the processor air ducts covering it. Figure 107. HPDC Blade DIMM HPDC Processor HIVOC 6. Use the adjustable torque driver to remove the eight standoffs that attach the HPDC to the blade (the bit holder on the torque driver is the same size as the standoff). 7. CAUTION: Equipment damage Carefully place the defective HPDC on a soft, ESD-safe surface with the components facing upward. Place the HPDC down directly and do not drag the HPDC on the surface. Failure to do so can lead to component damage. Grasp the edges of the HPDC and carefully wiggle the circuit board while pulling straight up to remove it from the blade. Remove a Haswell Processor Daughter Card (HPDC) 101

102 Figure 108. HPDC Processor Locked symbol Unlocked symbol Install a Haswell Processor Daughter Card (HPDC) Part Number: Module Assy, HPDC (shipped without DIMMs and processors) Time: Tools: NOTE: If you are installing a spare HPDC, you must transfer the DIMMs and processors from the defective HPDC to the new HPDC. 15 minutes to remove and replace a populated HPDC 30 minutes to remove a HPDC, transfer the DIMMs and processors to an unpopulated spare HPDC, and install the HPDC Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) This procedure shows how to install a haswell processor daughter card (HPDC) from a compute blade. 1. Position the replacement HPDC on the blade and press down firmly in the area of the connector to seat it in the connector on the blade. 2. Hand tighten the eight standoffs that attach the HPDC to the blade. Torque the standoffs to 14 in-lbs. (The bit holder on the torque driver is the same size as the standoff.) 3. Record the location of the DIMMs and processors on the defective HPDC so you can reinstall them in the same locations on the replacement HPDC. 4. Remove the processors from the defective HPDC. Install a Haswell Processor Daughter Card (HPDC) 102

103 5. Install the processors on the replacement HPDC. 6. Remove the memory DIMMs from the defective HPDC 7. Install the memory DIMMs on the replacement HPDC. Figure 109. HPDC Processor Locked symbol Unlocked symbol 8. Place the processor air ducts and position the power cable assembly on the blade. 9. Insert the two tabs on the center of the power cable into the standoffs on the blade and reconnect the cable connectors. 10. Reinstall the air diffuser side rail. Torque the screws to 5.5 in-lbs. 11. Place the shroud and reinstall the 12 T15 TORX attachment screws. Torque the screws to 9 in-lbs. 12. Reinstall the blade. Remove Components from an Aries Network Card (ANC) Part Number: Blade Assy FRU, ANC Compute Left Cascade Blade Assy FRU, ANC Compute Right Cascade NOTE: The defective ANC is not removed from the compute blade. The defective ANC and compute blade are returned to logistics. You must transfer the QPDCs from the defective ANC compute blade to the replacement ANC compute blade. Time: 15 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit Remove Components from an Aries Network Card (ANC) 103

104 T10 TORX bit Dusting gas (P/N ) Remove Components from the Defective ANC Compute Blade 1. Locate the blade that contains the defective ANC and remove the blade from the cabinet. 2. Remove the 12 T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Remove both air diffuser side rail from the blade. 4. Disconnect the power cable connectors and remove the power cable assembly and air ducts. (Press the release tab on the connector and pull the cable straight out.) 5. Remove the processor air ducts covering the QPDCs. 6. Remove the QPDCs from the compute blade. When removing the QPDCs from the compute blade, do not remove any components from the QPDCs. Replace Components on an Aries Network Card (ANC) Part Number: Blade Assy FRU, ANC Compute Left Cascade Blade Assy FRU, ANC Compute Right Cascade NOTE: The defective ANC is not removed from the compute blade. The defective ANC and compute blade are returned to logistics. You must transfer the QPDCs from the defective ANC compute blade to the replacement ANC compute blade. Time: 15 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX Dusting gas (P/N ) Install Components on the Defective ANC Compute Blade 1. Install the QPDCs. 2. Place the processor air ducts over the QPDCs. 3. Insert the two tabs on the center of the power cable into the standoffs on the blade and then connect the cable connectors. Replace Components on an Aries Network Card (ANC) 104

105 4. Install the air diffuser side rail. Torque the screws to 5.5 in-lbs. 5. Place the shroud and reinstall the 12 T15 TORX attachment screws. Torque the screws to 9 in-lbs. 6. Reinstall the blade. Remove Components from an IO Base Board (IOBB) The defective IOBB is not removed from the IO blade. The defective IOBB and IO blade are returned to logistics. Part Number: Blade Assy FRU, IO Left Cascade NOTE: You must transfer the XPDCs, PCIe cards, PCIe risers, and PCIe mezzanines from the defective IOBB IO blade to the replacement IOBB IO blade. Time: 15 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) Remove Components from an IOBB IO Blade 1. Locate the blade that contains the defective IOBB and remove the blade from the cabinet. 2. Remove the 19 T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Remove both air diffuser side rails from the blade. Note how the air diffuser rails are positioned on the standoffs for proper installation. 4. Remove the PCIe cards, risers, and mezzanines. 5. Remove the processor air duct that covers the XPDCs. 6. Remove the XPDCs from the IO blade. When removing the XPDCs from the IO blade, do not remove any components from the XPDCs. Install Components on an IO Base Board (IOBB) The defective IOBB is not removed from the IO blade. The defective IOBB and IO blade are returned to logistics. Remove Components from an IO Base Board (IOBB) 105

106 Part Number: Blade Assy FRU, IO Left Cascade NOTE: You must transfer the XPDCs, PCIe cards, PCIe risers, and PCIe mezzanines from the defective IOBB IO blade to the replacement IOBB IO blade. Time: 15 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T15 TORX bit T10 TORX bit Dusting gas (P/N ) Remove Components on an IOBB IO Blade 1. Install the XPDCs that connect to the IOBB. 2. Place the processor air duct over the XPDCs. 3. Install the PCIe cards, risers, and mezzanines. 4. Install both air diffuser side rails. Torque the screw to 5.5 in-lbs. 5. Place the shroud and reinstall the 19 T10 TORX attachment screws. Torque the screws to 9 in-lbs. 6. Reinstall the blade Remove a PCI Card The IO blade accommodates several types of PCI cards. The replacement procedure is identical for each type of PCI card. Part Number: PCIE Card, Gen 2 FC FC8 HCA Dual-Port Low Profile PCIE Card, Gen 2 GIGE Quad-Port Copper Interface PCIE Card, Gen 2 10GIGE Dual-Port CX4 Interface PCIE Card, Gen 2 Dual-Port 10GBE LR (Order FRUs & ) PCIE Card, Gen 2 10GIGE Dual-Port SR Opt (Order FRUs PCIE & SFP, ) PCIe Card, Gen 2 SAS-2 6Gb/s Dual-Port PCIE Card, Gen 3 IB HCA Dual-Port FDR W/QSFP PCIE Card, Gen 3 40GB Ethernet Dual-Port optical Remove a PCI Card 106

107 PCIE Card, Gen 2 10GIGE Dual-Port CX4 Interface Time: 7-8 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T10 TORX bit T15 TORX bit #2 Phillips screwdriver Remove a PCI Card 1. Locate the blade that contains the defective PCI card and remove the blade from the cabinet. 2. Remove the 19 T15 TORX screws that attach the shroud to the blade and remove the shroud. 3. Remove one or both upper PCI mezzanines and risers as needed to remove the defective PCI card. a. Remove the two T10 TORX screws that secure the upper PCI mezzanine to the 1/4-in. spacers. Remove the upper mezzanine from the standoffs and PCI riser. b. Remove the PCI riser near the lower mezzanine. c. Repeat steps a and step b to remove the opposite PCI mezzanine and PCI riser, if necessary. 4. Remove the top T10 TORX screw that attaches the aluminum block to the PCI faceplate. Figure 110. Aluminum Block T10 Torx screw Aluminum block 5. NOTE: If the defective PCI card is underneath the upper PCI card, you must remove the upper PCI card first. Grasp the PCI card and the faceplate and gently pull away from the PCI riser to disconnect the PCI card. 6. Repeat step 5 to remove the lower PCI card, if necessary. Remove a PCI Card 107

108 Figure 111. PCI Card Removal Install a PCI Card The IO blade accommodates several types of PCI cards. The replacement procedure is identical for each type of PCI card. Part Number: PCIE Card, Gen 2 FC FC8 HCA Dual-Port Low Profile PCIE Card, Gen 2 GIGE Quad-Port Copper Interface PCIE Card, Gen 2 10GIGE Dual-Port CX4 Interface PCIE Card, Gen 2 Dual-Port 10GBE LR (Order FRUs & ) PCIE Card, Gen 2 10GIGE Dual-Port SR Opt (Order FRUs PCIE & SFP, ) PCIe Card, Gen 2 SAS-2 6Gb/s Dual-Port PCIE Card, Gen 3 IB HCA Dual-Port FDR W/QSFP PCIE Card, Gen 3 40GB Ethernet Dual-Port optical PCIE Card, Gen 2 10GIGE Dual-Port CX4 Interface Time: 7-8 minutes Tools: Adjustable torque driver, 5 40 in-lbs (P/N ) T10 TORX bit T15 TORX bit #2 Phillips screwdriver Install a PCI Card 108

109 Install a PCI Card 1. NOTE: If the defective PCI card is underneath the upper PCI card, you must reinstall the lower PCI card first. As you hold the faceplate, gently push the PCI card into the PCI riser to connect the PCI card. 2. Repeat step 1 for the second PCI card, if necessary. 3. Reinstall the top T10 TORX screw that attaches the aluminum block to the PCI faceplate. 4. Spray the PCI riser socket and mezzanine contacts with dusting gas to remove any debris, if necessary. Figure 112. Aluminum Block T10 Torx screw Aluminum block 5. Reinstall one or both upper PCI mezzanines and risers as needed to install the replacement PCI card. a. Reinstall the PCI riser near the lower mezzanine. The riser is keyed and can only be inserted in one orientation. b. Replace the upper PCI mezzanine on the spacers and PCI riser. Reinstall the two T10 TORX screws to secure the upper mezzanine to the 1/4-in. standoffs. c. Torque the screws to 5.5 in-lbs. d. Repeat step a through step c to reinstall the opposite PCI mezzanine and PCI riser, if necessary. 6. Place the shroud on the blade and reinstall the 19 T15 TORX screws that attach the shroud to the blade. Torque to 9 in lbs. 7. Reinstall the IO blade. Install a PCI Card 109