Exadata HW Overview. Marek Mintal

Transcription

1 Exadata HW Overview Marek Mintal Oracle Day

2 Exadata Hardware Architecture Scalable Grid of industry standard servers for Compute and Storage Eliminates long-standing tradeoff between Scalability, Availability, Cost Intelligent Storage Grid Database Grid 8 Dual-processor x64 database servers 14 High-performance low-cost storage servers OR 2 Eight-processor x64 database servers InfiniBand Network Redundant 40Gb/s switches Unified server & storage network Copyright 2010, Oracle Corporation and/or its affiliates 100 TB High Performance disk, or 336 TB High Capacity disk 5.3 TB PCI Flash Data mirrored across storage servers

3 Exadata software Exadata Smart Scans Reduce data transfer between DB server and Storage Exadata Storage indexes Reduce disk IO Hybrid Columnar Compression (HCC) Raise the disk capacity and reduce data transfers Exadata Smart Flash Cache Raise IOPs Accelerate transactions I/O Resource Manager Control the IO resources Inter-leaved Grid Disks Most recently used data on faster disk spins

4 DB Performance overview CPU speed, cores Memory Parallelism interconnect Disk IO...

5 Bottlenecks LAN interconnect SAN Disk storage Tape storage

6 DB performance and CPU Cores, speed 8 x 12cores X2-2 or 2 x 64 cores X2-8 Interrupt handling on IO Disk IO Network IO User space app. processing Compression / encryption

7 RAC Network IO performance Public Network Client access Backup Interconnect Latency Reliability Throughput

8 DB and disk IO over SAN SAN 4 Gb ~ 250 MB/s 8 Gb ~ 500 MB/s NAS NFS ISCSI Disks arrays IOPS Cache, disk performance SATA, SAS1, SAS2, FC SSD technologies

9 Sun InfiniBand in Exadata Version 2 Exadata QUAD Rack IB Network TWO Sun Datacenter InfiniBand Switch 36 switches One Sun InfiniBand Dual port 4x QDR PCI-E LP HCA in each Database and Storage server 2x 40Gbps interconnections between Database Servers and Storage Servers IB connections to remote Backup Servers Exadata Database Servers IB Leaf Switch Exadata Storage Servers IB Leaf Switch IB Core Switch To Backup Servers

10 Exadata Database Machine X2-2 Quarter Rack Pre-Configured for Extreme Performance 2 x64 Dual-procesor Database Servers (Sun Fire X4170 M2) 24 cores (12 per server) 192 GB memory (96GB per server) 10 GigE connectivity to Data Center 4 x 10GbE ports (2 per server) 3 Exadata Storage Servers X2-2 All with High Performance 600GB SAS disks 2 Sun Datacenter InfiniBand Switch port Managed QDR (40Gb/s) switch 1 Admin Cisco Ethernet switch Keyboard, Video, Mouse (KVM) hardware Redundant Power Distributions Units (PDUs) Copyright 2010 Oracle Corporation and/or its affiliates

11 Sun Oracle Database Machine Quad Rack 2 x.-sun Datacenter Infiniband Switch 36 20,24 3rd switch is part of full rack upgrade 1 x Cisco Catalyst 48 port gigabit ethernet switch. (U23) 1 x. Avocent 32 port KVM switch. (U21) 2x -Database Server 3 x Exadata Storage Server Each containing 4 Flash PCIE 1x.- Avocent 1U USB Keyboard,video mouse drawer (U22)

12 Sun Rack II 1242E 42 RU spaces available 1200mm depth 600mm wide, 19-inch rails Enterprise Packaging suitable for shipping fully loaded Zero-RU PDUs with metering features Ideal for high density deployments requiring high amperage in a single cabinet

13 Database Server X4170 M2 Specification 2x Intel Xeon X5670 Intel Xeon 2.93 GHz I/O Oracle Enterprise Linux Oracle Real Application Clusters Maximum Reliability, Availability, and Serviceability Ease of Deployment and Manageability 4 x 300GB 10000rpm 2.5 SAS HDDs SAS RAID PCIe 8 port HBA. Installed in slot PCIE-0 Sun QDR InfiniBand PCIe Low Profile HCA.(X4237A) Installed in slot PCIE-2 Management ILOM Service Processor

14 Database Server 1 xsun QDR InfiniBand PCIe Installed in slot PCIE-2 2 x six core 2.93 GHz Intel Xeon, 96GB RAM 4x300GB 10K SAS HDD's

15 Exadata Storage Server Specification 2x Intel Xeon E5540, 6C, 2.26 GHz, 6x DDR GB RDIMMs I/O 12 x 600GB 15000rpm SAS or SATA HDDs X4270 M2 Oracle Enterprise Linux Oracle Exadata Storage Server Software Additional Drivers, Utilities SAS RAID PCIe 8 port HBA. Installed in slot PCIE-0 Qty x 4 Flash PCIE 96GB Sun QDR InfiniBand PCIe Low Profile HCA.(X4237A) Installed in slot PCIE-3 Management ILOM Service Processor 2TB

16 Exadata Storage Server 12 x 600GB SAS2 15k 2 x 2.26 GHz, Intel Xeon 8MB L3, 24GB RAM Sun QDR Infiniband pcie Low profile 4 x F20 flash accelerator

17 Sun F20 PCIe Flash Accelerator Capacity 100k IOPS per PCIe slot 96 GB of Flash in one 8-lane PCIe slot Attach up to 8 Internal SAS/SATA HDD Performance 100k read IOPS (4K) 84k write IOPS (4K) <.4ms random read/write service times 1 GB/s sequential read (.5 GB/s write) Reliability Redefining Storage Performance and Efficiency Unprecedented power efficiency Unmatched space efficiency Unparalleled flexibility Advanced wear-leveling, write endurance Integrated Super CAP: no batteries! ENTERPRISE READY Connectivity 2 x 4-wide3Gb/s internal SAS ports SAS expander, HBA and Flash Standard HBA drivers Power Rating 16.5 Watts

18 Exadata Product Capacity (Uncompressed) X2-8 X2-2 Full Rack Full Rack Raw Disk 1 X2-2 Half Rack X2-2 Quarter Rack High Perf Disk 100 TB 100 TB 50 TB 21 TB High Cap Disk 336 TB 336 TB 168 TB 72 TB 5.3 TB 5.3 TB 2.6 TB 1.1 TB Raw Flash1 Usable Capacity with ASM normal redundancy2 High Perf Disk 45 TB 45 TB 22.5 TB 9.25 TB High Cap Disk 150 TB 150TB 75 TB 31.5 TB Usable Capacity with ASM high redundancy3 High Perf Disk 30 TB 30 TB 15 TB 6.25 TB High Cap Disk 100 TB 100TB 50 TB 21.5 TB 1 - Raw capacity calculated using standard disk drive raw space terminology of 1 GB = 1000 x 1000 x 1000 bytes and 1 TB = 1000 x 1000 x 1000 x 1000 bytes. 2 - Actual space available for a database after mirroring (ASM normal redundancy) and leaving one empty disk to handle disk failures. Capacity calculated using normal space terminology of 1 TB = 1024 * 1024 * 1024 * 1024 bytes. 3 - Actual space available for the database computed after triple mirroring (ASM high redundancy). Capacity calculated using normal space terminology of 1 TB = 1024 * 1024 * 1024 * 1024 bytes. Copyright 2010, Oracle Corporation and/or its affiliates 18

19 Infiniband Network Qty x 2 or 3 Sun Datacenter Infiniband Switch 36 A 1RU 36 port managed QDR InfiniBand Switch. The switch utilizes standard QSFP connectors All Database Servers and Sun Exadata Storage Servers Data Facing Servers Dual Port HCA QSFP connection from Server to two separate switches Provides Redundant single HCA Path for all servers All SQL Queries and Replies for all Servers utilize IB Network Embedded OpenSM for IB Fabric

20 Infiniband Sun Infiniband 36 QDR switch 40 Gb/s point to point ~ 3.38 GB/s Low latency, reliable HA and scalable architecture Dual port HCA in each DB and storage server Connection to remote backup server IPoIB capability (2 MB/s)

21 Infiniband and RDS RDS (reliable datagram socket) Low overhead, high bandwidth, reliable, IPC transport system Matches Oracle's existing IPC model for RAC Based on socket API Limit the SW stack (latency) Utilizing IB features HA port failure within HCA...

22 Infiniband IO perfomance InfiniBand maximizes Disk utilization Delivers 46% higher IO traffic than 10GE InfiniBand Interconnect 10GigE Interconnect

23 Network Block Diagram Non-bonded configuration KVM NET3 NET2 NET1 NET0 Management/ILOM Client Access Ethernet switch ILOM InfiniBand Additional network 1 (optional) Additional network 2 (optional) ILOM Sun Fire X4170 Oracle Database servers NET0 Exadata Storage Servers BOND0 Network Key BOND0 InfiniBand switch InfiniBand switch

24 Exadata Demo Center diagram FC Backup server X4170M2 Exadata Public net 1Gb DB servers X4170M2 RDS IDB Sun Datacenter InfiniBand Switch 36 Storage servers X4270 M2

25 Exadata V2 Quad power, cooling, weight Three-Phase High-Voltage EMEA Sun Rack II 600 x 1200 mm 42U Weight: 342 kg Specification (per 1 PDU) 15 kva, 3ph Number of Inputs 1 x 25A 3ph 5 wire Voltage 220/ /415VAC 3ph Frequency 50/60Hz Current 25A max. per phase Power Rating 14.4 kva Group Protection 20A Receptacle IEC309 32A 5-PIN 230/400V 3ph IP44 Cooling Max Cooling Typical 10,800 BTUs/hr 8,200 BTUs/hr

26 Thanks / Questions? April 09