AW206: The Need for Speed: Sizing and Scaling Nutanix for Performance

Transcription

1 AW206: The Need for Speed: Sizing and Scaling Nutanix for Performance Gary Little, Performance Engineering Andy Daniel, Technical #nextconf #AW206

2 Disclaimer Forward-Looking Statement Disclaimer This presentation and the accompanying oral commentary may include express and implied forward-looking statements, including but not limited to statements concerning our business plans and objectives, product features and technology that are under development or in process and capabilities of such product features and technology, our plans to introduce product features in future releases, the implementation of our products on additional hardware platforms, strategic partnerships that are in process, product performance, competitive position, industry environment, and potential market opportunities. These forward-looking statements are not historical facts, and instead are based on our current expectations, estimates, opinions and beliefs. The accuracy of such forward-looking statements depends upon future events, and involves risks, uncertainties and other factors beyond our control that may cause these statements to be inaccurate and cause our actual results, performance or achievements to differ materially and adversely from those anticipated or implied by such statements, including, among others: failure to develop, or unexpected difficulties or delays in developing, new product features or technology on a timely or cost-effective basis; delays in or lack of customer or market acceptance of our new product features or technology; the failure of our software to interoperate on different hardware platforms; failure to form, or delays in the formation of, new strategic partnerships and the possibility that we may not receive anticipated results from forming such strategic partnerships; the introduction, or acceleration of adoption of, competing solutions, including public cloud infrastructure; a shift in industry or competitive dynamics or customer demand; and other risks detailed in our Form 10-Q for the fiscal quarter ended April 30, 2017, filed with the Securities and Exchange Commission. These forward-looking statements speak only as of the date of this presentation and, except as required by law, we assume no obligation to update forward-looking statements to reflect actual results or subsequent events or circumstances. Any future product or roadmap information is intended to outline general product directions, and is not a commitment, promise or legal obligation for Nutanix to deliver any material, code, or functionality. This information should not be used when making a purchasing decision. Further, note that Nutanix has made no determination as to if separate fees will be charged for any future product enhancements or functionality which may ultimately be made available. Nutanix may, in its own discretion, choose to charge separate fees for the delivery of any product enhancements or functionality which are ultimately made available. Certain information contained in this presentation and the accompanying oral commentary may relate to or be based on studies, publications, surveys and other data obtained from third-party sources and our own internal estimates and research. While we believe these third-party studies, publications, surveys and other data are reliable as of the date of this presentation, they have not independently verified, and we make no representation as to the adequacy, fairness, accuracy, or completeness of any information obtained from third-party sources. Trademark Disclaimer 2017 Nutanix, Inc. All rights reserved. Nutanix and the Nutanix logo are registered trademarks or trademarks of Nutanix, Inc. in the United States and other countries. All other brand names and logos mentioned herein are for identification purposes only and are the property of their respective holder(s), Nutanix may not associated with, or sponsored or endorsed by such holder(s).

3 What Drives The Need For Speed? NOT A COMPETITIVE DRAG RACE Customer Demand - More intensive workloads Enterprise Applications Tier 1 Database (Oracle, MS SQL) Big Data, MapReduce (Hadoop, mongodb, Splunk) Application Vendor Requirements - SAP Low latency write - EPIC Consistency Day 2 Performance Resilience to failure Noisy neighbor resistance Big Data

4 #ThrowBackThursday - Where have we come from? JOURNEY TO TIER 1 PERFORMANCE 4 Node 4KB Random Read IOPS 250 VDI Knowledge Worker Density Login VSI. 500, , , , , IOPS 250, , , ,000 50, NX Dec NX Oct-15 NX Mar-16 NX 3060-G4 4.6 Mar-16 NX 3060-G4 4.7 Jul-16 NX 9060-G4 4.7 Jul-16 NEXT-G5 5.0 Jan-17 Desktops Per Host KB Write Response time (RF2 Lower is better) Response Time (ms) Cores 3060-G4 10 Cores 3060-G4 12 Cores 3175-G4 14 Cores 3060-G5 14 Cores 0.00 NX Dec-14 NX Oct-15 NX Mar-16 NX 3060-G4 4.6 Mar-16 NX 3060-G4 4.7 Jul-16 NX 9060-G4 4.7 Jul-16 NEXT-G5 5.0 Jan-17

5 Where are we now? 8K Random Read Peak (Single Node All Flash, SSD) Desktop Density NX-3060-G Ghz Client Resp time usec ,000 40,000 60,000 80, , , , ,000 Desktops Per Host ESX/VMW View AHV/Xen Desktop IOPS Sequential Read Sustained 1MB IO Size Single Node (AF, SSD) Sequential Write Sustained 1MB IO Size Single Node (AF, SSD) MB/s MB/s Queue Depth Queue Depth

6 Where are we now? (AHV w/ahv Turbo*) Key Takeaways Single-Node NX-3060 All Flash SSD (Non-Cached 8K Random Read) - AHV Turbo reduces guest limitation QEMU not designed for high I/O rates - Requires client ability to drive Linux SCSI multi-queue - Flexible resource consumption Consume all resources for data load Locality for steady-state and noisyneighbor protection ESX (Single Container) ESX (Dual Cointainer) AHV FRODO Turbo (Hosted Local) AHV FRODO Turbo (Scale-Out) *AHV Turbo w/planned datapath tuning

7 Where are we now? Beyond simple micro-benchmarks Scenario-centric testing - X-Ray Consistently reproduce tests Realistic Scenarios Database Colocation Node Failure HCI Workflow OLTP Rolling Upgrade Learn more about X-Ray in session PT307 tomorrow at 10:05!

8 Where are we now? Benchmarking with X-Ray - Database colocation scenario - AOS data locality provides predictable performance

9 Where are we now? Benchmarking with X-Ray - OLTP in failure scenario - AOS rebuilds data in a few minutes with minimal performance impact - Other testbed starts rebuild after hour, takes hours to complete, and significantly impacts performance Other

10 Where are we now? ALL-FLASH HARDWARE DOMINANCE NVMe SSD SATA SSD SAS HDD SATA HDD K 12ms 100K 500 6ms ms 0.03ms IOPS Higher is better Latency Lower is better Bandwidth (MB/sec) Higher is better Flash delivers maximum performance and minimum latency regardless of working set size

11 Nutanix All-Flash STORAGE CPU SCALE WITH DATA LOCALITY CPU scale for storage processing Data locality for lowest latency

12 Data Efficiency Means Performance Too ZERO BLOCK SUPPRESSION INLINE COMPRESSION Zero Blocks Metadata Update Works well for broad application set New LZ4/LZ4HC Algorithms (AOS 5.0)

13 Data Efficiency Means Performance Too DEDUPLICATION ERASURE CODING D D D P Cluster Size (nodes) EC Strip Size (data/parity blocks) EC Savings (vs. RF2) Strip Size (data/parity) EC Savings (vs. RF3) After 4 2/ N/A N/A 5 3/1 1.5 N/A N/A 6 4/1 1.6 N/A N/A 7+ 4/ /2 2.0 Cluster-wide dedupe Inline fingerprinting Works well for specific use cases Most interesting to broad user base Dynamic EC strip increase (AOS 5.1) Overwrite optimization (AOS 5.1)

14 Adding All-Flash The Easy Way HYBRID CLUSTERS Design Enhancement - Smart placement of data based on disk fitness (queue depth and free space) - Skew-based disk balancing tasks Benefits - Improved mixed workload support within the same cluster (general, databases, VDI) - Enables transition to All-Flash - Enables ABS deployments within existing clusters Hybrid Nodes + All-Flash Nodes

15 New Technology NX-9030 Platform Debut NVMe RDMA* Distance to App SATA Driver SATA Controller Flash Controller Flash Media Latency Distance NVMe Driver Flash Media NVMe devices directly connected to CPU via PCIe Low local access latency (~300μs) Oplog striped across NVMe Latency Remote Direct Memory Access RDMA over Converged Ethernet (RoCE) Low remote access latency (<400μs) *Capable hardware at release, full software support in AOS Obelix release

16 Performance Testing: Not a science project

17 Assessing Desktop Density Key Takeaways - Insufficient mage optimization is the #1 cause of VDI performance issues - Sizing is complex engage the experts - VDI on Nutanix is CPU and Memory bound. - Use optimizer tools BP Analyzer VMW Optimizer Fling Vendor BPG Tools - LoginVSI - View Planner

18 Database: Transactional Workloads Key Takeaways - DB requires CPU, memory & IOPS - Never overcommit DB memory - Ensure working-set is held in SSD - Account for growth - snapshots, etc. may need short term retention in SSD Tools - HammerDB - SLOB - Benchmark Factory

19 Acropolis File Services: User home directories scaling Key Takeaways - Home directory servers are metadata operation heavy - Raw IOP testing will not accurately predict success - Scaling AFS can scale up and/or out Mix AFS with VDI or separate out Tools - Microsoft fsct FSCT Users FSCT Users per AFS node - SFS Benchmark 0 4 vcpu 16 GB 6 vcpu 24 GB 6 vcpu 32 GB

20 General I/O Performance hints Keep active working-set on SSD Account for snapshots in calculations Very large working-sets (>1TB) may require more memory given to CVM Optimized write performance up to 6 GB per vdisk (burst support) Generally no longer need many vdisks All-Flash provides larger working-set and higher IOP rate with lower sustained latency > 100K IOPS per <1 ms latency Client observable response times 500 μs Read 750 μs Write

21 Where are we now? ALL-FLASH HARDWARE DOMINANCE NVMe SSD SATA SSD SAS HDD SATA HDD K 12ms 100K 500 6ms ms 0.03ms IOPS Higher is better Latency Lower is better Bandwidth (MB/sec) Higher is better Flash delivers maximum performance and minimum latency regardless of working set size

22 Where do we go from here? STORAGE CLASS MEMORY 3D XPoint SSD* NVMe SSD SATA SSD 500K > K 100K 500μs 300μs <100μs 500 IOPS Higher is better Latency Lower is better Bandwidth (MB/sec) Higher is better SCM delivers maximum performance and minimum latency regardless of queue depth *Generalized, preliminary, manufacturer specifications

23 Where do we go from here? MARRYING NUTANIX WEB-SCALE & INTEL HYPERSCALE Making AHV the performance hypervisor Pass-through drivers directly consumable Modern storage stack limited by CPU - SPDK and DPDK shorten the I/O path - CVM gains bare-metal access to hardware but benefits of VM Intel SPDK & DPDK - SPDK Storage Performance - DPDK Network Performance - Running at user level - Poll mode drivers ki/os I/O performance on single Intel Xeon core Number of NVMe SSDs Linux Kernel SPDK

24 #nextconf #AW206

25 Like What You Heard ATTEND NF101 Nutanix Technology Basics READ Nutanix Enterprise Cloud Datasheet USE Nutanix Community Edition Nutanix Sizer PT205 Microsegmentation Made Easy Nutanix Acropolis Datasheet Nutanix Prism Datasheet AW103 The Need for Speed: Sizing and Scaling Nutanix for Performance The Nutanix Bible

26 Gary #nextconf #AW206

27 Where are we now? AHV Turbo Design Enhancement Use DPDK libraries to enable faster UVM <-> CVM communication Parallelize IO processing end to end between UVM and CVM Key Takeaways Reduced latencies Performance optimized full stack with RDMA, NVMe and AHV Turbo User Space Kernel Space Host virtio Driver QEMU Old virtio Driver AHV Turbo HW CVM CVM New