CRAY GOLD SUPPORT SUPPORT OPERATIONS HANDBOOK. GOLD-SOHB Page 1 of 22

Transcription

1 CRAY GOLD SUPPORT SUPPORT OPERATIONS HANDBOOK GOLD-SOHB Page 1 of 22

2 Table of Contents Document Version History... 4 About Cray Customer Service... 5 About this document Cray Gold Support X7 Online Self Services Software Upgrades, Updates, and Patches Case Management Documentation X5 Next Business Day (NBD) Services Call Back Response Accessing Inquiry assistance Remote diagnosis General Services Proactive Field Notices On-site Spare Parts Enhanced Spare Parts On-site Maintenance Tools Training Training Included Additional Value Added Training Manuals and Documentation Support Tools CrayPort Registering for CrayPort Defect Management Bugzilla Bug Severity Field Notices Remote Diagnostic Tools Notifications Spare Parts Management GOLD-SOHB Page 2 of 22

3 3. Case Lifecycle Reporting a Problem Defining a Case Monitoring Case Progress Definition of Case Status Case Stages Level 1 Support (1LS) Level 2 Support (2LS) Level 3 Support (3LS) Roles and Responsibilities Description of Cray Assistance to Customers Telephone Support Licensed Software Support Software Updates Degraded Operation Support Tools Customer s Obligations Glossary GOLD-SOHB Page 3 of 22

4 Document Version History Rev Description Originator Date 1.0 Document creation Paula C Salovich 3/18/ Change of name document and Paula C Salovich 3/20/13 general customer release 2.1 Added Roles and Paula C Salovich 06/13/2013 Responsibilities section 2.2 Deleted Feedback and Resolution guidelines Paula C Salovich 09/02/13 GOLD-SOHB Page 4 of 22

5 About Cray Customer Service Cray Customer Service strives to maintain high standards of performance. Cray service teams, backed by Cray support systems, are highly skilled and experienced in the tradition of high performance computing and equipped to provide world-class support for mission-critical systems. Cray support engineers, field engineers and field analysts undergo a full range of product, technology and support system training that qualifies them to deliver services including installation, troubleshooting, system maintenance, system administration, programming, and optimization for some of the world s most impressive high performance systems. Cray support and field engineers are backed by subject matter experts from the Cray Engineering and Development organization as well as third party technology partners. Cray support and field engineers are closely linked to Cray hardware and software product support groups, and have access to Cray s inhouse systems to reproduce hardware and software problems, which aids system troubleshooting and problem diagnostics. Cray Customer Service offers a variety of service levels to ensure coverage of all customer needs. This document focuses on Cray Gold Support, but Cray Customer Service recommends customers gain a full understanding of all Cray service offerings. Cray Silver Support is tailored to customers with technical expertise and capabilities. These customers are fully trained by Cray to take on the responsibility of maintaining the operations and availability of their own systems with limited contact and intervention from Cray. This is a low cost alternative for customers and support is provided remotely. For customers that require a greater level of engagement from Cray Customer Service, the next level offered is Cray Gold Support. This level of support includes all the benefits and service offerings of Cray Bronze Support. Additionally, it provides customers the assurance of having an experienced Cray technical support resource on site, if required, the next business day after an issue is reported. The highest levels of support Cray offers are Platinum and Diamond, which include 1 hour, 2 hour or 4 hour response times. These levels offer completely hands-free full service. Each customer will have dedicated and experienced Cray technical resources at their disposal during the contracted coverage and response times. Platinum and Diamond Support also offer a single point of contact into Cray to resolve system issues and procedural questions. GOLD-SOHB Page 5 of 22

6 About this document The Support Operations Handbook (SOHB) has been developed to describe the processes, policies and practices of Cray Customer Service. It is also intended to provide the processes and procedures that will ensure a successful working relationship between Cray and its customers. This document is not intended to replace any current legal agreements between Cray and the customer. This document is subject to review, change and update upon the creation of new services, processes, changes in existing services or policies, or support level structuring, without notice. Cray customers may copy or transmit this document for internal use only. GOLD-SOHB Page 6 of 22

7 1. Cray Gold Support Cray Gold Support is a comprehensive hardware and software maintenance offering designed for customers that desire to have Cray resources available, on-site when needed. Cray Gold Support provides assistance available 9 hours a day x 5 days a week, local business hours, Monday through Friday, 8 AM to 5PM, with next business day response. This means Cray support engineers are available to address system problems by the following business day from the time an incident is reported. Weekend and holiday support is not included. Cray Gold Support also provides online access to CrayPort, Cray case reporting tool, and Cray documentation center 24 hours per day x 365 per year. If, during the troubleshooting process, the support engineer working on the case has determined that a Cray technical recourse is needed on site to resolve the issue, one of Cray s technical resources will be assigned to travel to site to work on the issue on the next business day. If the issue cannot be resolved on site, the support engineer will leverage Cray s development and potentially third party support resources to find a resolution for the problem. If the issue is determined to be a bug, a ticket will be created in Cray s bug tracking tool, Bugzilla, and addressed by Cray s development team or by the thirdparty provider. Cray Gold Support also includes a spare parts kit that will be located at the customer site to cover the majority of hardware problems. The following services are available with a Cray Gold Support agreement: X7 Online Self Services All Cray customers with active service contracts receive access to Cray s customer service portal, CrayPort. CrayPort is the tool with which customers can report and manage system maintenance issues, find relevant product documentation, and receive Field Notices as well as downloadable software, firmware, upgrades, and patches Software Upgrades, Updates, and Patches The latest releases of software are provided at Updates include major releases, patches, documentation updates, and related release notes Case Management Customers under an active services contract can request access to Cray s case management tool, SalesForce.com. Customers can request access for as many users as needed. The case reporting tool is a self-service system that allows issues to be logged through cases 24 hours per day x 365 days per year. Customers must enter all the relevant information related to the issue being reported, capturing all critical information required to help diagnose a problem Documentation Cray provides a full suite of documentation for all software and hardware products delivered with Cray systems. All system documentation is kept current and is made GOLD-SOHB Page 7 of 22

8 available to customers with successive releases of system software, including access to third-party products provided by Cray. Documentation includes: Reference information in the form of man pages is provided with Cray software release packages using the man(1) command. For software updates between major releases, changes/fixes are documented in HTML pages included with the update download package. The CrayDoc documentation system is a collection of open-source software components that allows user with fast and easy access to Cray documentation from a web browser over a local path defined by the system administrator. HTML files of most documents and PDF files of all documents are provided. CrayDoc runs on any operating system based on UNIX or Linux and includes the following features: ability to search books, Cray-authored man pages, and the Cray glossary accommodates wild card (*) searches integrates non-cray documents which support third-party hardware and software highlights search matches Documentation is also available on Cray s public website. The documentation that is available on the web is updated with each software release to accurately reflect the true nature of the service X5 Next Business Day (NBD) Services Call Back Response Once an issue is reported in a case, initial response of all customer cases is provided by Cray Field Engineers, next business day, after the case is reported for a hardware failure, software defect or system problem, 9 hours a day x 5 days a week, local business hours, Monday through Friday 8 AM to 5PM. If deemed necessary, a Cray technical resource will be dispatched to the customer site to work on resolving the issue. On-site response will also be provided for system upgrades and should be scheduled in advanced by the customer. Weekend and holiday support is not included Accessing Inquiry assistance Cray customers under an active services contract are allowed to report a case to ask any procedural questions. Inquiry assistance covers processes, product functionality, and product feature-related questions. Inquiry questions are only applicable to products that have been properly installed and configured. Inquiry assistance is not a substitute for training and is not intended to replace Cray Service s involvement in implementing additional products. Cray reserves the right to review the case and refer the customer to GOLD-SOHB Page 8 of 22

9 Cray Professional Services for billable work or to Cray s training department for additional training. Inquiry assistance is available 9 hours a day x 5 days a week, local business hours, Monday through Friday 8 AM to 5PM. Initial response to all cases is provided the next business day after being reported. Weekend and holiday support is not included Remote diagnosis Customers under the Gold Support agreement may be required to provide full access to their system for remote diagnostics. Cray will work with the customer to remotely diagnose and identify products not performing in accordance with Cray s written specifications, and will provide the customer remote technical assistance or on-site technical assistance when deemed necessary. This arrangement is available on a 9x5 basis - 9 hours a day x 5 days a week, local business hours, next business day response, Monday through Friday 8 AM to 5PM. Weekend and holiday support is not included. 1.3 General Services Proactive Field Notices The configurable Field Notices deliver information about Cray s software and hardware by providing documentation, information updates, product updates, and breaking news. All existing field notices are available at On-site Spare Parts Under Cray Gold Support, Cray will provide on-site inventory of key spare parts Enhanced Spare Parts For customers that are highly sensitive to user downtime, Cray recommends the purchase of warm spares, which are fully assembled blades or power supplies. This reduces the amount of downtime caused by system hardware failures. To purchase these parts, contact your Cray District Service Manager or Account Manager for assistance On-site Maintenance Tools Depending on the geographical location of the Cray system and the proficiency of the customer s repair skills, Cray will provide several tools to help maintain the hardware. 1.4 Training Training Included Under Cray Gold Support, Cray will provide a one day hardware replacement training session for two designated customer employees. The training will be delivered during system installation at the customer s site or at Cray s training facility. The customer- GOLD-SOHB Page 9 of 22

10 designated personnel must complete hardware replacement training before performing maintenance on the Cray system. Software or system administration training is not included under Cray Gold Support; however, Cray requires that customers under Gold Support take a full system administration class to properly maintain the operations and availability of their systems Additional Value Added Training To get the best value from Cray products, additional training is available to customers for any and all software maintenance and applications. Cray recommends that Gold Support customers take additional system administration training in order to properly maintain their systems. Additional available training sessions can be found at: Manuals and Documentation All user and administrative documentation for software and hardware products sold by Cray can be found at CrayDoc, GOLD-SOHB Page 10 of 22

11 2. Support Tools The Cray customer portal, CrayPort, provides a comprehensive set of tools to allow customers to search independentry for information, solutions, and online case reporting. All customers with current maintenance contracts have full access to CrayPort, 24 hours a day, 7 days a week (24 x 7), excluding downtimes for routine maintenance or unforeseen outages. 2.1 CrayPort CrayPort is Cray's online customer portal. It provides access to a variety of information, including bug tracking, Field Notices, technical support cases, documentation, software downloads, and training registration. The Help tab at CrayPort includes a CrayPort Guide that describes the various features of the site. CrayPort provides customers access to technical bulletins, software fixes, and license keys. CrayPort also allows customers to monitor and track Bugzilla software defect correction activity. Registering for CrayPort To register for CrayPort, go to and select the Register for an Account link. The registration form requires the following information: Company or organization name. Cray system serial number. Contact name, address, phone number, and mailing address. Selection of bug tracking tool access. Upon receipt of this information, account setup and activation occurs within two business days. Customer login information is sent to the address provided. While every effort is made to prepare for all possible contingencies, there may be times where access to CrayPort may not be available due to unforeseen circumstances. If access problems are encountered, customers may contact the CrayPort support organization at csms_support@cray.com. 2.2 Defect Management Defects are reported as cases in CrayPort. All pertinent information needs to be captured at the time a case is created. While the issue is still a case in CrayPort, Cray support engineers triage and validate the issue and gather additional documentation if necessary. If a CrayPort case is confirmed to be a bug, the Cray support representative converts the case to a Bugzilla bug via an automated mechanism that is part of CrayPort's web interface. The automated mechanism will open a bug via Bugzilla's interface, passing along all the required information. GOLD-SOHB Page 11 of 22

12 2.2.1 Bugzilla Once a case received in CrayPort is confirmed to be a bug, a new bug ID is created in Bugzilla. The data (such as product, component and customer ID) that is used to generate the Bugzilla bug from CrayPort comes directly from the associated fields in the CrayPort case. Once the new bug ID is created, the bug ID is added to the CrayPort case, and then closes the case. All bugs, and all public bugs reported into Bugzilla, are available for review at CrayPort. Additionally, customers can subscribe to a daily update of their own reported bugs Bug Severity The severity of a bug refers to the importance the customer places on the reported bug. In general, these are the basic guidelines for defining the severity of a bug at Cray: Critical: Prevents the customer from meeting their normal production schedules or prevents acceptance testing from proceeding. If an acceptable workaround is available at all affected sites, then the bug can be downgraded to Urgent. Urgent: Limits the customer's ability to meet their normal production schedules or will cause significant impact on the acceptance test. If an acceptable workaround is available at all affected sites, then the bug can be downgraded to Major. Major: Does not interfere with the customer's normal production schedules Minor: Little or no impact on users and can be bypassed easily Details on the response guidelines and specific examples of when the above severities apply are available on Section Field Notices Field Notices (FN) are important communications specific to the products a customer owns. These notifications include information updates, product updates, security updates, critical notifications breaking news, and announcements on the availability of patches or new releases. All customers under a valid Gold Support agreement will receive, via , all the FNs that are related to their Cray products. Customers also have an option to customize their FN subscriptions at CrayPort to receive only those FNs that they consider appropriate for their role. GOLD-SOHB Page 12 of 22

13 2.4 Remote Diagnostic Tools The support engineers assigned to a particular customer case will assess each situation and determine those instances where remote problem diagnosis would best benefit the client in expediting the analysis of the case for resolution and troubleshooting. Support engineers may require system administrator capabilities to perform command line functions when connected to customers systems. Remote access should be provided by the customer and may include any or all of the following access methods: 1. Secure connection tools: a. Virtual private networks (VPN). b. Secure shell (SSH). c. Secure ID authentications. 2. Cray is also able to provide remote diagnosis using WebEx. WebEx provides collaboration tools for application sharing and problem diagnosis, and it gives customers control over the kind of information and access that they allow Cray Support Personnel to have Notifications All customers under a current, valid Support Agreement are able to subscribe to receive notifications. These notifications include all Field Notices, changes, and bug updates related to their products. Customers can set up their accounts for notifications by changing their CrayPort Profiles, located under the Help drop down menu on the main CrayPort menu bar. In CrayPort Profile, customers can enable notifications for bugs and/or Field Notices under the Preferences heading. 2.6 Spare Parts Management Under Cray Gold Support, Cray will provide on-site inventory of key spare parts. All spare parts will be new or warranted to perform as new. When the customer suspects a part has failed, the customer must report a case through CrayPort. Cray Support will either confirm the part failure or provide specific direction to the customer to resolve the issue. If Cray support confirms that the part has failed, a Cray technical resource will travel to site to replace or services the damaged part. After the failed part has been replaced, the Cray technical resource will return the damaged part to Cray. GOLD-SOHB Page 13 of 22

14 Parts that are installed into the customer system become the property of the customer; parts removed from the customer system become the property of Cray. Parts that are retained by customer for security reasons shall be purchased by the customer at Cray s then-current replacement part price. Once the failed parts have been returned to Cray, new (or warranted to perform as new) parts will be added to the customer s spare parts pool. GOLD-SOHB Page 14 of 22

15 3. Case Lifecycle Customers may contact Cray Support either by telephone or electronically during their contracted support hours. Customers can electronically submit cases on a 24 X 7 basis; however, cases will still be handled in accordance to the contracted coverage level and response times. 3.1 Reporting a Problem Customers can report a problem in two ways: 1. Open a case on the CrayPort website: a) Log in to and select the Case tab. b) Click the Create New Case button and enter the required information. c) Click the Submit button. The system assigns the case to the queue that supports your Cray system. An assigned monitor of the queue responds to your inquiry during your contracted period of service. Note: Refer to the CrayPort Guide on the Help tab for detailed case entry instructions. 2. Severity 1 only for quicker response during a critical situation, or an inoperable system, customers should call the Cray help center at CRAY ( ). The customer reporting the critical issue should specify that the issue being reported is a system down situation. All other cases should be reported through CrayPort. Note: Refer to the CrayPort Guide on the Help tab for detailed case entry instructions. 3.2 Defining a Case Customers should review the following information prior to entering and reporting a case. This information will be used in resolving the customer issue: Detailed description of the problem. The environment profile associated with the system that is exhibiting the problem (production, test, etc.). What steps were followed that resulted in the problem being reported? What parameters are set for the programs being accessed? Has this problem occurred before? What result was expected? Can the issue be duplicated (reproduced) on demand? Can the issue be described step-by-step to allow it to be reproduced? What troubleshooting steps have you taken to investigate the problem? Have any recent changes been made to the system? Have the processes for using the software been changed? Which programs were involved when the error occurred? If an error message appeared, what is the exact error message and number? What is the severity of the case? GOLD-SOHB Page 15 of 22

16 3.3 Monitoring Case Progress All cases that have been submitted to Cray can be viewed electronically via CrayPort. For customers to access the information online, they need to go to the Case/Bug Entry tab on CrayPort and select Search Cases. After login into SalesForce.com, customer will find information on all the cases they have reported to Cray. Cray also provides notifications every time the case is updated or a change in status occurs. 3.4 Definition of Case Status A case will move through various stages during its lifecycle. Below is the list of definitions of case status and the action currently in process: Case Status Open/Closed Status Description Unassigned Open Waiting to be assigned to an individual owner. Work in Progress Open Waiting for final HW diagnosis, collection of bug information, responses to other queries, and information gathering. Waiting Maintenance Closed Waiting for physical work or SW upgrade/patch installation. Unsuccessful part replacement will cause case remain in this state until case can be completed. Waiting Logistics Closed Waiting for any Logistics activity, such as part failure analysis or spare parts. Data Gathering / Monitoring Closed Waiting for additional data, or waiting to see if an intermittent issue re-occurs. Completed Closed Case completed. 3.5 Case Stages The following are the support levels used in support of escalation of cases. GOLD-SOHB Page 16 of 22

17 3.5.1 Level 1 Support (1LS) Level 1 Support is the first line of contact with Cray support personnel. Level 1 Support includes: First contact and direct end user interaction. Clarification of functions and features of the product and reference to documentation pertaining to the product. Guidance in the operation of the product, including, without limitation, Cray software. Error verification, analysis and correction to the extent possible by telephone. Information collection and analysis. Identification of whether the problem is known and has a known solution. Troubleshooting and problem reproduction. Problem report administration and tracking Level 2 Support (2LS) Level 2 Support is technical support and is typically where Cray product experts reside serving as the remediation contact for problems not solved by Level 1 support. Level 2 Support includes: Resolving all known problems. Addressing installation and configuration issues. Assisting with products, including the system software, firmware, and driver updates at the customer site. Seeking answers to customer problems by searching Cray s posted technical notes, knowledge database, and other technical information that will assist in problem resolution Level 3 Support (3LS) Cray s Level 3 technical support engages Cray s development team or third party technology vendor support organization in the resolution of problems. Should a 2LS product engineer or analyst be unable to resolve a reported case, either because of lack of expertise, exhausted troubleshooting knowledge, or identification of an undocumented defect, a 2LS representative may escalate the problem to Level 3 technical support for resolution. If a problem is escalated to Level 3, Cray must first create an incident report. To create an incident report, Cray 2LS product support engineers contact Cray s development team or the third party technology vendor s support organization by either: 1. Contacting Cray s development team or third party technology vendor support organization over the phone and communicating a description of the problem. GOLD-SOHB Page 17 of 22

18 2. Fully documenting and communicating electronically a detailed description of the problem. 3. If available, entering the incident report directly into the third party technology vendor support organization. Cray Level 3 technical support is available twenty-four (24) hours a day and every day of the year. Regardless of the method Cray Level 2 uses to create an incident report, the following information is required to open a Cray Level 3 Support incident report: Caller s name. Caller s phone number. Caller s address. Caller s location. End User identification. Cray s case reference number. Cray s assigned severity level for the incident. End user s current product revision or product type. A detailed description of the problem that is the subject of the incident report and any/all events, actions, and information leading up to the reported problem. System logfile output, or other supporting documentation, as applicable. 4. Roles and Responsibilities 4.1 Description of Cray Assistance to Customers Following the certification of at least two (2) designates to participate in support, Cray will provide the following assistance to enable customer to commence supporting customer s systems: Telephone Support. Unless other problem submission arrangements are defined, the customer designates will be given access to Cray s support number or a local alternative phone number, and Cray incident reporting tool. Cray shall respond within the timeframe specified in Response and Feedback Guidelines of this document. Cray will work with the customer to remotely diagnose and identify the products not performing in accordance with Cray s written specifications, and will provide the customer limited technical assistance regarding customer s use of products and related documentation Licensed Software Support. Cray will process the customer s licensed software/documentation bug reports and enhancement requests, at Cray s discretion, and will provide the customer with status on their bug reports and enhancement requests. GOLD-SOHB Page 18 of 22

19 4.1.3 Software Updates. Cray will provide the customer with access to updates for licensed software that the customer has licensed from Cray, on a general release of such updates to Cray s customers. The customer agrees that if a licensed software update is not installed by the customer within six (6) months after it is received from Cray, Cray may limit the support that Cray provides for such licensed software. Cray will provide support for the current and the immediately preceding major release of licensed software and will only support licensed software when used with equipment configurations approved in writing by Cray. Cray, at its sole discretion, will provide limited support for previous major releases Degraded Operation. Based upon the circumstance of a given system event or interrupt, Cray may, at its discretion and with the cooperation of the customer, remotely disable or down a failing node or reconfigure the system to allow for continued operation on the remainder of the system until such time that the customer designate can complete a service operation Support Tools. Cray shall provide support tools for the maintenance of the equipment to include an Electro- Static Discharge (ESD) kit. Appropriate ESD precautions will be observed at all times and the customer agrees to regularly inspect the ESD kit for wear and proper functionality. Cray will be notified immediately if any part of the kit is worn or not functioning correctly, and will provide a replacement if necessary. GOLD-SOHB Page 19 of 22

20 4.2 Customer s Obligations Customers under a valid support agreement are under the following obligations: Ensure that all equipment is installed in a location meeting the environmental conditions specified in Cray s documentation. Perform system administration and maintenance of their system. Ensure that Cray has remote access to the product. Ensure that customer has designated technically capable individuals as designates to be trained for, and to perform, service functions and that designates are available to assist Cray on any remote maintenance activity. Ensure that all problems encountered that require Cray action are logged in accordance with Cray s instructions. Limit access to Cray s technical information and support software in customer s possession solely to customer s employees specifically needing to use such information for operation of product. Perform any user maintenance operations on equipment authorized by Cray, including, but not limited to, cleaning equipment, performing operating checks and diagnostics, and other tasks described in Cray s documentation or other Cray manuals provided with the product; maintain at least one (1) current backup copy of all licensed software. Not change the configuration of any Product (e.g., by upgrade or addition of add-ins/add-ons or licensed software and the like). Not move products for which support is provided without providing prior written notice to Cray and providing any move is in accordance with Cray s instructions. Limit its use of licensed software support to circumstances when the licensed software fails to function as specified in the applicable Cray user s guide or other Cray documentation or when customer believes that Cray documentation is unclear as to the proper use or configuration of the licensed software, and maintain the equipment at an acceptable revision level (Cray will provide information to customer regarding equipment changes when applicable). Promptly notify Cray of any failures or bugs in the product, and perform reasonable product testing at the request of Cray. Ensure designates follow telephone support and direction provided by Cray support engineers and other technical support personnel to perform limited on site diagnosis and physical replacement of pre-defined system components. Ensure designates strictly follow safe working practices, the proper use of tools and ESD precautions. Designates will ensure that they are physically capable of undertaking any CRU replacement procedures without any risk of injury to themselves, others or damage to equipment. After successful completion of each maintenance event, log the details of the event via the CrayPort self-service portal. GOLD-SOHB Page 20 of 22

21 The customer is required to package and ship all failed parts at its expense and ship them as directed by Cray (at Cray s expense) and, within fourteen (14) days of removal from the system. Any applicable export regulatory requirements are the customer s responsibility. The customer agrees to return all replaced parts to Cray when upgrading or repairing equipment. GOLD-SOHB Page 21 of 22

22 Glossary Bug Error, defect, discrepancy. Critical Bug Defect that prevents the customer from meeting their normal production schedules and leaves the system inoperable. Urgent Bug Defect that limits the customer from meeting their normal production schedules but does not leave the system inoperable. Major Bug Defect that does not limit the customer from meeting their normal production schedule and does not leave the system inoperable. Minor Bug - Minor defect that has little or no impact on users and can be bypassed easily. Bugzila Ticket tracking tool used by Cray to develop or create fixes for a bug reported in a case. Case (Incident, Call) A case is a single problem, or symptom in the customer s operation of the solution that negatively affects the functionality of the Cray system. A case is made up of all the activities required to resolve the issue once Cray services has been contacted. Every case is assigned a unique number (case ID) that is used to track its progress and referenced when communicating with customers. CrayPort CrayPort is Cray s portal to customers. Documentation, case reporting and product information is found in CrayPort. Fix (Action, Correction, or Patch) A fix is the solution to a bug. Level 1 Support - Level 1 support means the service provided by Cray Field Engineers, in response to the initial web contact or telephone call by a Customer which identifies and documents a problem using a product supplied by Cray. This includes problem source identification assistance, problem analysis, problem resolution, installation planning information, knowledge base search, and preventative and corrective service information. Level 2 Support Level 2 support means the service provided by Cray s technical experts after a case is escalated by Level 1 support. Level 3 Support Level 3 support means the service provided for fixes or workarounds. This service involves Cray development team and/or third-party providers. Release A release is the edition of a Cray product after it has been made generally available by Cray. SFDC Salesforce.com Cray s case reporting tool. GOLD-SOHB Page 22 of 22