North America ATS Storage Accelerate with ATS DS8000 Hardware Management Console (HMC) Best practices and Remote Support Configuration September 23rd, 2014 Thomas Fiege 1 2014 2014 IBM IBM Corporation Corporation
Accelerate with ATS Storage Webinars The Free IBM Storage Technical Webinar Series Continues in 2014... ATS Technical Experts cover a variety of Storage topics Audience: Clients who are either currently have IBM Storage products or considering acquiring IBM Storage products. Business Partners and IBMers are also welcome. How to sign up? To automatically receive announcements of the Accelerate with ATS Storage webinar series, Clients, Business Partners or IBMers can send an email to accelerate-join@hursley.ibm.com Information, schedules, and archives: Located in the Accelerate with ATS Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/accelerate/?lang=en 2
DS8000 Hardware Management Console (HMC) Best practices and Remote Support Configuration 3
DS8000 HMC Service Interface & Remote Connectivity Gateway Reliability, Availability, and Serviceability HMC is integrated into the DS8000 rack Connects the customer and the IBM SSR to the DS8000 Primary HMC inside the rack; external optional secondary HMC Customized Lenovo laptop as hardware platform Server grade disk drives; No wireless adapter No battery since the HMC is part of the DS8000 power system HMC is the center of DS8000 networking Two redundant DS8000 internal ethernet networks controlled by the HMC One ethernet port for customer network connectivity Firewalls; Secured communication; No unnecessary network services Audit and Activity logging HMC Legacy HMC started its life as an x-server Linux based customized appliance operating system Common parts with z-series and p-series HMC Business Critical Storage for the World s Most Demanding Clients 4
DS8000 HMC Overview HMC controls the DS8000 internal components Manages the two p-series servers and other hardware components inside the DS8000 Service, customer, and remote user and account management Problem analysis: Analyze events & notify IBM Support and customer if required Provides infrastructure for initial DS8000 installation, concurrent code loads, and loading of critical code patches Repair & Verify framework to fix DS8000 problems Hosts various service utilities Controls back-ups for seamless recovery from management drive failures Pass-through for encryption key manager 5
DS8000 HMC Overview HMC connects the DS8000 to the outside world The HMC has one GbE interface to connect the DS8000 to the customer network Provides connectivity for a secure HTTPS based service user interface for the IBM Service Representative (SSR) Focal point for remote support (call home, remote access) and network security Firewalls on the HMC only open the ports that are actually needed Requires Internet connectivity provided and secured by the customer Hosts the servers and their configuration for DS Storage Manager DS Command Line Interface CIM Agent TPC-R 6
DS8000 HMC Overview IBM SSR uses the HMC via the Service User Interface to: Install the DS8000 at the customer account Check on the DS8000 health is everything ok? Diagnose and Repair problems Conduct code loads and HMC rebuilds Perform MES (hardware upgrades) Customer uses the HMC to: Connect the DS Storage Manager UI and DS Command Line Interface (DS CLI) for User management Storage configuration and management Copy Services setup and notifications Check for open DS8000 problems Setup email and SNMP notification for DS8000 events Miscellaneous utilities to query the DS8000 state 7
DS8000 HMC Connectivity Summary DS8000 Private Network Secondary HMC (optional) TPC Corporate Network 2 to 4 Encryption Key Servers Encryption Key Servers - Provides key management for encryption of data at rest Primary HMC Firewall Storage Admin Phone Line IBM Support Center Remote support - Call home - Remote access - Connectivity via Internet SSL or Internet VPN* or modem* Internet Storage Admin - Maintain storage configuration - Establish replication - Receive alert messages via email - Review system performance data - Uses web browser, DSCLI, TPC 8
DS8000 HMC Concurrent Code loads Concurrent Code Loads (CCL) IBM SSR uses the HMC to acquire code images from DVD/CD Also supports acquire over the network or FTP server Preload distributes code packages to the various DS8000 hard- and software components and then performs a daily health check IBM SSR usually updates the HMC as part of Preload When customer is ready the IBM SSR activates the preloaded code CCL duration between 1.5 4.5 hours, depending on code load contents A look into the future IBM is constantly improving the code load process: Decrease CCL times Better recovery from failures (self-healing) Load code images from USB attached media 9
DS8000 HMC Concurrent Repair Concurrent Repair The HMC is the center of all repair activity Every service action is guided by the DS8000 micro code and executed by the IBM SSR (guided maintenance) When the DS8000 microcode detects a problem it creates a Serviceable Event (SE) on the HMC, containing problem details and information on what part(s) needs to be replaced The SE is being called home to IBM for first analysis by the IBM remote support center and, if required, to dispatch an IBM SSR for parts replacement If configured the SE is also sent as email and SNMP trap The IBM SSR uses the SE to start the guided repair 10
DS8000 HMC Concurrent Repair Example of a Serviceable Event 11
DS8000 HMC Remote Support Why does DS8000 need remote support? In the rare event the DS8000 has a problem it will be fixed faster and with less attempts if remote support is enabled Automatically Call Home (RETAIN PMR) problem reports & notification to IBM before there is any customer impact Periodically send machine statistics and heart beat information (I'm alive!) to IBM Transmit log data to IBM for faster error analysis if required Allow for Remote Access in case IBM support personnel or development needs to recover from catastrophic events Start of the art security technology from end-to-end Over 90% of DS8000 customers have remote support enabled At no time we are sending customer data over the network By design customer data cannot be accessed from the management network and the components connected to it Flexible remote support configuration to meet our customer's security requirements Audit and user access logging capabilities Constant review of Common Vulnerabilities and Exposures (CVE) on a corporate level, incl. accelerated fix plans 12
DS8000 HMC Remote Support What is IBM Call Home? DS8000 micro code constantly monitors the health of the entire machine If a Serviceable Event (SE) is opened on the HMC it will be sent encrypted to IBM over Internet or modem connectivity Final destination is a call management system (RETAIN) which starts the support process and eventually may dispatch an IBM SSR if required Together with the SE a small set of system log files is also sent to IBM 13
DS8000 HMC Remote Support What is IBM Call Home? Example of a Call Home Record (PMR in RETAIN) Contact info is the only 'personal' type info in the data being sent home 14
DS8000 HMC Remote Support What is IBM Call Home? Knowledge-based Systems (KBS) provide real-time updates to predefined repair actions 15
DS8000 HMC Remote Support Machine Reportable Product Data (MRPD) MRPD is also known a Vital Product Data (VPD) Contains DS8000 hardware information like location codes, part numbers, serial numbers, and firmware levels Also contains logical configuration meta data and DS8000 machine configuration (Service Agent) Useful for capacity planning, LIC feature support, and IBM service planning Does not contain customer / sensitive data Heart Beat (HB) A Heart Beat (HB) is a very small data package periodically sent to IBM to indicate the HMC is still alive and healthy It also indicates that the HMC's connection to IBM is working properly and can be used for call home and remote access (if configured) If a HB is not arriving at its scheduled time IBM support will be notified and look into the issue 16
DS8000 HMC Remote Support Log and trace data offload When needed the IBM SSR or IBM remote support engineer can offload additional log and trace data needed for error analysis and recovery Contains system hardware and software error logs, traces, and dumps Does not contain customer / sensitive data Can be offloaded over a secure network connection or onto removable media Used by IBM Product Engineering and development Deleted after a certain period of time (180 days) 17
DS8000 HMC Remote Support Remote Connectivity Options Internet SSL (recommended) Modem* Internet VPN* FTP SSL/TLS encrypted port 443 connection through customer network Support for SSL proxies, including user id and password authentication 1024 bit asymmetric key for initial handshake; data encryption via symmetric 128 bit RC4 or 256 bit AES Global AT&T dial-up Internet access (Fenced Internet) Actual communication is SSL/TLS encrypted, see below IPSec implementation; 192 bit 3DES encryption key for ESP + 160 bit MD5 hash authentication key Uses protocol 50 (ESP) and port 500 UDP; UDP ports 500 and 4500 when Network Address Translation (NAT) is utilized Standard FTP connection for log and trace data offload only Final destination can be IBM or any FTP server on a customer network Support for various FTP proxies/firewalls) Footnote *: Will be deprecated in a future DS8000 version 18
DS8000 HMC Remote Support Remote Access Options Assist On-Site AOS (recommended) Modem dial-in* Internet VPN* SSL based remote access tool for IBM support personnel Easy to configure; only needs outbound 443 traffic enabled in firewall infrastructure Runs on a customer provided and maintained gateway server or on the DS8000 HMC Allows the customer to be in control of the remote access session Simple ASCII terminal point-to-point dial-in connection IBM support personnel can use a VPN connection initiated by the local IBM SSR or customer via DSCLI The VPN connection is always established by the DS8000 HMC This is not a Business-to-Business VPN; the remote access VPN is only active when needed Same technology as the Internet VPN connectivity for call home Footnote *: Will be deprecated in a future DS8000 version 19
DS8000 HMC Remote Support More details on AOS 1: The AOS gateway establishes and maintains a permanent and secure connection between the customer and the IBM AOS server (green arrows) 2: IBM support personnel requests a remote access session with the IBM AOS server for a particular customer 3: The AOS Server verifies the requests and establishes the remote access session between the IBM remote support personnel and the AOS gateway 4: The AOS gateway controls the application data flow between the DS8000 and the IBM support personnel Starting with DS8870 R7.1 the AOS gateway is also included in the HMC AOS supports multiple gateways (redundancy) and can be used by other IBM products (Tape, XIV, SoNAS) The IBM AOS redbook at http://www.redbooks.ibm.com/redpieces/abstracts/redp4889.html?open provides additional information on AOS as a secure remote service solution. 20
DS8000 HMC AOS Customer Flyer Available as PDF Promotional video in the making 21
DS8000 HMC Remote Support & Network connectivity General Security Design HMC acts as a proxy/firewall between customer network and internal private networks No IP forwarding between customer network and internal private networks Unnecessary network services are removed (telnetd etc) Authentication Authorization Accounting IBM can identify who of its employees has accessed a DS8000 at what time Periodic review and fixes for CVEs Two-factor authentication for remote ID's (Challenge/Response) IBM SSR log-on only from local HMC Only IBM employees with a need to know can access customer machine (IBM internal ACL's) Different access levels of authority for IBM personnel The DS8000 design is such that customer data is kept in kernel space; There is no way to get to the data from user space 22
DS8000 HMC Configuration and best practices HMC configuration Mandatory one HMC per DS8000; installed in the same rack Optional second HMC (priced feature); installed outside the DS8000 rack Both HMCs are fully operational and functionally equivalent Possibility to concurrently change the HMC configurations IBM recommends the optional second HMC for redundancy Use two HMCs when encryption is enabled on the DS8000 Provide two paths for DS8000 encryption key retrieval Use two HMCs whenever copy services is managed by TPC-R or DS CLI/DS Storage Manager Two HMCs provide redundancy in case on HMC is not available, for example code loads Alternative HMC for install, repair, and concurrent code load activities Backup for call home and remote access functions Customized network attachment options Attach one HMC to a secure 'internal' customer network, for DSCLI/DSGUI access for example Attach the other HMC to a portion of the customer network that has Internet connectivity for call home remote access 23
DS8000 Encryption network environment 24
Trademarks The following are trademarks of the International Business Machines Corporation in the United States, other countries, or both. Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market. Those trademarks followed by are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States. For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml: *, AS/400, e business(logo), DBE, ESCO, eserver, FICON, IBM, IBM (logo), iseries, MVS, OS/390, pseries, RS/6000, S/30, VM/ESA, VSE/ESA, WebSphere, xseries, z/os, zseries, z/vm, System i, System i5, System p, System p5, System x, System z, System z9, BladeCenter The following are trademarks or registered trademarks of other companies. Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce. * All other products may be trademarks or registered trademarks of their respective companies. Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area. All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Information about non-ibm products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography. 25