Technical White Paper ETERNUS DX8700 S2 Hardware Architecture Table of contents Management Summary and General Remarks 2 System Architecture 3 Controller Enclosure 5 Controller Module () 5 Cache and Backup Disk 6 Channel Adapter 6 Disk Enclosure () 7 Disk Drives 8 Firmware 9 Remote Service 10 Environmental Improvements 10 Summary 11 Page 1 of 12
Management Summary and General Remarks Based on a unique and consistent system design, ETERNUS DX is a seamless family of disk storage systems ranging from economy, entrylevel, and midrange systems up to the entry-level ETERNUS enterprise class. The major design principles of ETERNUS DX are compatibility and modularity. All models utilize the same disk types, disk shelves, racks, cables and other components, plus uniform management software. Only the controller module is model-specific. The idea is to minimize operational, training and migration work, as well as costs. Furthermore, this family concept allows upgrading a smaller model to a larger one by exchanging the control module. All other components can be reused. The uniform management system enables easy interaction between models of different sizes, for example, ETERNUS DX90 entrylevel and DX400 midrange models in remote locations can replicate data to an ETERNUS DX8700 enterprise system. The comprehensive system configuration flexibility of ETERNUS DX makes it easy to balance user requirements in terms of speed, capacity and costs. The ETERNUS DX8700 S2 enterprise disk storage system is purpose-built for the requirements of big data centers. The latest S2 version is a complete redesign of the system with the objective of achieving maximum modularity and configuration flexibly, while also delivering highly scalable performance and operational robustness. Thus enterprises are no longer forced to accurately anticipate future capacity demand levels or select a specific high-end model with a fixed data capacity range. The traditional, fixed-capacity, model-based approach typically required IT organizations to invest in modules that were bigger than actually needed, just to be sure that future capacity would be covered and to avoid unplanned and high migration costs. Therefore, Fujitsu based the ETERNUS DX8700 S2 generation on a completely modular architecture that can be flexibly expanded according to business needs. With minimum initial costs, customers can start with a dualcontroller module system and four drive enclosures. Maximum performance with up to 128 front-end ports for host connectivity can be achieved easily by scaling up to 8 controller modules. In order to have simultaneous connectivity to different network types, FC, FCoE and iscsi host interfaces can be included in mixed configurations. Fujitsu s Flexible Data Management concept enables a higher level of automation for storage tiering, including support for SAS, Nearline SAS and SSD drive options, plus capacities that ensure an optimum service level in dynamic and fast-changing environments. The concept also reduces the overall total cost of storage. The challenges stemming from data exposure protection are addressed using innovative Self-Encrypting Drives (SED). System robustness that is supported by redundant components ensures high availability for today s non-stop business. This white paper describes the architecture and hardware features of ETERNUS DX8700 S2. Software functionalities and features are the subject of other publications. Page 2 of 12
System Architecture Figure 1 illustrates the schematic architecture of the ETERNUS DX8700 S2. The basic functional blocks are the Channel Adapters () which realize the connection to the host, the Controller Module () which runs the firmware-based operating system and the Back-End Router (), which connects the to the disks located in Disk Enclosures (). Legend Channel Adapter IOC I/O Controller Controller Module Back-End Router Memory The minimum configuration of an ETERNUS DX8700 S2 consists of two controllers and can be enhanced by adding pairs of Controller Modules up to the maximum configuration of eight controllers. The performance of the system scales with the number of controllers. IOC The Controller Modules are interconnected via the Front-End Routers (FRT) consisting of redundant PCI Express switches, which enable high-speed data interchange between the controllers (Figure 2). Figure 1 The disk enclosures are connected via SAS 2.0 redundant paths to the Back-End Routers while each Back-End Router is connected to each controller module. 0 PCIe PCIe SW 1 The combination of Front-End Router PCIe and Back-End Router SAS2.0 interconnect is key to enable outstanding data throughput in any ETERNUS DX8700 S2 system. The schematic configuration of the highest possible configuration is shown in Figure 3. IOC SAS2.0 SAS SW Memory FRT IOC SAS2.0 SAS SW Memory A fully extended ETERNUS DX8700 S2 consists of 8 Controller Modules with 16 Channel Adapters and 128 Disk Enclosures holding up to 3,072 disk drives. Legend Channel Adapter IOC I/O Controller SAS SW SAS Switch Figure 2 Back-End Router FRT Front-End Router PCIe SW PCIe Switch CE FRT PEX FRT PEX BBU FRT PEX FRT PEX SVC Legend CE Controller Enclosure Drive Enclosure Channel Adapter FRT Front-End Router Back-End Router EXP SAS Expander IOC I/O Controller I/O Module BBU Battery Backup Unit SVC Service Controller CPSU Controller Power Supply Unit Figure 3 C CPSU C Cooling Fan Power Input Temperature Sensor Page 3 of 12
The system populates one 19, 42U rack at minimum and can be expanded up to 9 fully populated racks (Figure 4). This architecture is the basis for the extremely flexible configuration options of Fujitsu s high-end system. Unlike other systems, the Back- End Routers are not part of the controller, but are separate modules which are installed according to the number of disk enclosures. In addition, 3.5 and 2.5 disk enclosures can be mixed in the system. Thus the number of controllers is not related to the number of disks and vice versa. If, for example, a high-capacity configuration is needed, a dual-controller configuration can be equipped with the maximum number of disks. If, on the other hand, the highest level of performance is required, an eight-controller system with a low number of disks is the choice. Another remarkable feature is the fact that the number of SSD s in the system is not limited, so that system performance can be tailored in many ways. The mesh architecture delivers considerable advantages in terms of high performance and high availability, because the internal data paths are point-to-point connections, thus avoiding latencies which are typical of switched or matrix-like back-end concepts. Furthermore, it enables a wide choice for scaling capacity in terms of the number of disks that can be connected without having to add controllers, while simultaneously determining which disk tier classes are utilized according to applications requirements. The same wide choice applies to the host connectors, which are just modules within the controller and can be easily exchanged as fabric requirements change. Minimum configuration: 1 rack, 4, 8 drives Maximum configuration: 9 racks, 128, 3,072 drives Figure 4 Page 4 of 12
Controller Enclosure (CE) Controller Module () The Controller Enclosure is the central part of the ETERNUS DX8700 S2 and is installed once per system. It is a 16U, standard 19 frame and holds the Controller Modules, Front-End Routers, Back-End Routers, the Service Controller, Power Supply and the Battery Backup Units. Its architecture design ensures that all components can be upgraded or exchanged without interrupting a running system. Thus all components in the Controller Enclosure are hot-pluggable. The Controller Module is the central component of an ETERNUS DX8700 S2 system. It holds the an Intel Xeon processor and the main memory comprised of DDR3 DIMM modules while up to 4 Channel Adapters () per realize the host connections. Legend BBU Battery Backup Unit Back-End Router Channel Adapter Controller Module FRT PSU SVC Front-End Router Power Supply Unit Service Controller Page 5 of 12
Controller Module () Cache and Backup Disks Controller Module () Channel Adapter () The minimum main memory for one is 6 GB and can be expanded to 96 GB, so that a fully extended ETERNUS DX8700 S2 has 768 GB of cache. The combination of highly dynamic cache allocation mechanisms with a very lean operating system kernel enables very efficient and performance-optimized use of the main memory for caching purposes and not for system overhead. Should a power outage occure, the Battery Backup Unit keeps the alive until the cache data is saved to a mirrored pair of Backup Disks (BUDs). The BUDs are very fast non-volatile SSD drives. This feature ensures that the cache data is backed up at the highest speed possible, and that data retention time is unlimited, no matter how long the outage lasts. The BUDs also contain the operating system software and the system configuration data. The Channel Adapters (s) realize the connectivity to the hosts via the storage network as well as the array-to-array connectivity for remote mirroring. Up to 4 Channel Adapters can be mounted on each Controller Module. To date four different types of are available: 4 port FibreChannel 8 GB/s 2 port iscsi 10 GB/s 2 port iscsi 1 GB/s 2 port FCoE 10 GB/s Remote replication between arrays can be realized with both FC and iscsi. All types of s can be mixed on a or in the whole system without limitations. This offers the highest flexibility in terms of connectivity and disaster recovery options during the entire lifecycle of the system. Even if new connectivity technologies or protocols come to market, the flexible / design enables quick adoption in existing systems, and thus offers additional investment protection (Figure 6). FC ETERNUS DX8700 S2 FC Expand 8 FC (128 port) FC (4 port) Min. configuration Mix FC iscsi 1G iscsi10g FCoE FC + iscsi1g + iscsi10g + FCoE Figure 6 Page 6 of 12
Disk Enclosure () Starting with ETERNUS DX80 S2 and ranging up to the ETERNUS DX8700 S2, the utilized types are identical. s are 2U, standard 19 enclosures, and the ETERNUS DX8700 S2 can be equipped with a maximum of 128 s in a total of 9 cabinets. Two different types are available: a small form factor (SFF) holds up to 24 x 2.5 disk drives a large form factor (LFF) holds up to 12 x 3.5 disk drives Legend I/O Module PSU Power Supply Unit Different disk types having the same form factor can be mixed without restriction within one. As the operating system and the system configuration data resides on the backup disks in the controller enclosure, all disks in the disk enclosures can be used for the storage of customer data. Page 7 of 12
Disk Enclosure () Disk Drives All Disk Drives in ETERNUS DX S2 systems are equipped with a SAS 6B/s interface to realize the connection to the Back-End Routers. Three different main types are available and realize different storage tiers in terms of performance and reliability: Very fast Solid State Drives (SSD) for the highest IOPS and response time requirements High-performance SAS Drives with both 15k and 10k rpm Low spinning Nearline SAS Drives with 7.2 rpm for backup, archiving and low access data All these types are available with different capacities in both LFF and SFF, and optionally as Self-Encrypting Disks (SED). The ETERNUS DX8700 S2 thus offers flexibility with an almost unlimited combination of disk options to ensure a tailored fit with any kinds of applications and customer requirements (Figure 7). Disk Enclosure () Small form factor Disk Enclosure () Large form factor SSD SSD SAS Drive 15k rpm or SAS Drive 15k rpm SAS Drive 10k rpm SAS Drive 10k rpm SAS Drive 7.2k rpm SAS Drive 7.2k rpm Figure 7 Page 8 of 12
Firmware The ETERNUS DX8700 S2 offers multifold firmware-based features and functionalities which, in addition to the hardware design, improve system reliability and performance. Advanced RAID organization ensures data access even during a drive enclosure failure. Each disk drive in a RAID group is located in a different Drive Enclosure. This keeps the RAID group accessible even if one complete Drive Enclosure is not functioning. Available RAID levels are RAID1, 1+0, 5, 5+0, 6. Data Block Guard guarantees very high data integrity and improved system reliability. This function adds check codes for identification of all data written. It can also confirm/guarantee data integrity at multiple checkpoints along the data transmission route. Disk drive patrol is a background process that constantly checks the disk drives to detect errors and write failures. Erroneous data is reconstructed and written to a new valid area on the disk drive. Thus disk write failures are avoided in a preemptive way. Encrypted communication between ETERNUS DX and user terminals via SSL/SSH guarantees secure transfer of data across a network and prevents unauthorized access and information theft. Thin provisioning helps lower initial investments in disk drives while offering virtual pools of disk capacity. Quality of service options help administrators improve or maintain system performance for the most demanding and business-critical applications. A more detailed description of these features and the even more sophisticated functionalities of the ETERNUS SF Management Suite (which complements and enhances ETERNUS DX systems) can be found in the ETERNUS DX software white paper. Controller based encryption prevents unauthorized data access LUNwise, whereas self-encrypting disks do not offer this kind of granularity. Encryption technology can be chosen between the world-standard 128bit AES technology and Fujitsu s unique encryption with high-process performance. Page 9 of 12
Remote Service Environmental Improvements For deployments in the Europe, Middle East, Africa and India region (EMEA-I) the AIS Connect Remote Support Gateway is an optional hardware and software solution that offers call home functionality in cases of alerts, component outages and other issues. It enables server-less remote access to ETERNUS storage systems for remote services in customer environments. The ETERNUS DX8700 S2 was engineered to reduce the environmental impact of both the manufacturing process and system operation. The main objective during development was to ensure for low power consumption, which was achieved by the deployment of power-saving SFF and SSD hard drives, by the use of highly efficient power supplies, and by the intelligent control of fan rotation. Furthermore, power and cooling can be reduced with the ECO mode feature, which supports drive spindle rotation control and disk spin down for idling disk drives. Due to the power-saving implementation of the Backup Disk on the Controller Module, the required battery capacity has been significantly reduced. This was also a prerequisite for the deployment of NiMH batteries, which save 50 percent in weight and 80 percent in volume when compared with the lead batteries formerly used. The device consists of a 1U, standard 19 enclosure which is mounted in the CE cabinet of the ETERNUS DX8700 S2. The avoidance of hazardous chemical substances is a top priority throughout the entire manufacturing process, not to mention the 3R design of systems and manufacturing Recycle, Reuse, Reduce. Implementation of this this real-time service device keeps the potential downtimes of ETERNUS storage systems to a minimum. The call home feature can alert the maintenance provider preventively, long before a potential failure can become system imminent. After allowing individual access, system specialists can remotely access the system for preemptive maintenance and to ensure fast analysis and troubleshooting in cases of failures. For deployments in geographies outside EMEA-I Fujitsu recommends to enable REMote Customer Support (REMCS). REMCS is a software based remote maintenance tool that monitors the storage array s status continuously and sends configuration, environmental, and failure event information via encrypted email directly to the local Fujitsu customer support. Thus the support organization is able to analyze failures and resolve problems within shortest response times, also enabling it to conduct preemptive action before serious trouble occurs. Figure 8: Real-time monitoring of power consumption and temperature with ETERNUS SF helps raise storage administrators awareness for saving energy. Page 10 of 12
Summary The ETERNUS DX8700 S2 provides non-disruptive capacity upgrades up to 3,072 disk drives, just by adding disks or complete drive enclosures. Maximum performance is delivered by configurations having from 2 to 8 controller units, with 128 front-end ports (maximum) for host connectivity. FC, FCoE and iscsi host interfaces can be included in mixed configurations for simultaneous connectivity to different network types. Tiered storage optimization using a range of SAS, Nearline SAS and SSD drive options lowers the overall total cost of storage. The challenges of data exposure protection are addressed by Self- Encrypting Drives (SED). Flexible management across different quality of service profiles, using different drive types for each application scenario, ensures efficient assignment of system resources matched to the performance priorities of different applications. System robustness with redundant interconnections to the storage array, internal redundant components and local data copy options ensure high availability for today s non-stop business. Remote replication to a second storage device protects all valuable data assets across the entire enterprise against site outages. Page 11 of 12
www./eternus Published by Fujitsu Limited Copyright 2012 Fujitsu Limited www./eternus All rights reserved, including intellectual property rights. Technical data subject to modifications and delivery subject to availability. Any liability that the data and illustrations are complete, actual or correct is excluded. Designations may be trademarks and/or copyrights of the respective manufacturer, the use of which by third parties for their own purposes may infringe the rights of such owner. For further information see www./eternus Page 12 of 12