HP Enterprise Data Warehouse Deep Dive. Steve Tramack, Sr. Engineering Manager, I2A Solutions, HP

Transcription

1 Enterprise Data Warehouse Deep Dive Steve Tramack, Sr. Engineering Manager, IA Solutions,

2 and Microsoft Strategic Partnership Igniting the contribution of IT to the business Microsoft World-leading software and hardware companies -year, $0M joint investment Portfolio of solutions delivers rapid time to value Integrated and Microsoft Services Bridge the path to the cloud

3 Converged Systems deliver bigger business impact Deploy in days or weeks, not months Boost application performance on balanced systems Optimize Performance Accelerate Time to Value Minimize Risk Deliver with confidence and ensure project success Simplify Operations Streamline management and lower TCO

4 Months Weeks Accelerate time to value Deploy in weeks rather than months Design Build Deploy Use Custom-built solution Assess workload Define architecture Evaluate options Acquire SW, HW Install components Load data Test with POC Balance SW, HW Integrate in environment Monitor and troubleshoot Extract and manipulate data Generate reports Make decisions + Integrated, optimized appliance Design Build Deploy Use Choose appliance for workload Acquire appliance Install appliance Extract data from existing MS apps Stand-up in production Manage and adapt Extract and manipulate data Generate reports Make decisions Weeks, Not Months + Months

5 DATA WAREHOUSING LANDSCAPE AND OFFERINGS Confidential NDA until June

6 Source TDWI, Next Generation Data Warehouse Platforms > 0TB % % % Appliances % 9% Growing Market Massive Parallel Processing %

7 and Microsoft Data Warehousing Offerings Business Data Warehouse Appliance An affordable SMP solution for data warehousing on optimized hardware Tier Offerings Fast Track Data Enterprise Warehouse Scalable and reliable platform for data warehousing on any hardware Reference architectures offering best price performance for data warehousing Appliance for high-end data warehousing requiring highest scalability, performance, or complexity Ideal for small data marts up to TB Ideal for data marts or small to mid-sized enterprise data warehouses (EDWs) Ideal for data marts or small to large data warehouses with scan-centric workloads Offers flexibility in hardware and architecture Appliance Software only Reference Architecture (software and hardware) Appliance Scale Up Data Warehousing Scale up data warehousing Scale up data warehousing Scale out data warehousing with massively parallel processing (MPP) Up to TB 0s of terabytes 0 terabytes 0s 00s of terabytes $0K (includes hardware) $.K per processor $.K per server $ per CAL $.K per Proc only (Datacenter) $0K $K ( Procs; includes hardware) $. $. million per rack (includes hardware) $K K per terabyte Tier Services and Support

8 The Appliance Engineering Approach Each solution matching and balancing four main elements Workload Architecture Software Hardware Driven by a fundamental understanding of the workload Architecture followed by and supported by components

9 Reference Architecture Maturity Model General Guidance HW & SW Guidelines Workload Target Vendor-Specific Configurations HW & SW Identified Validated Configuration Appliance Products HW & SW Optimized Factory Built & Tested Successfully Design a new system using best practices Successfully Buy a new configuration with known abilities Successfully Deploy a new solution with proven performance

10 & Microsoft Data Warehousing Continuum: SQL 00 R Server Fast Track, Enterprise Data Warehouse Appliance Scales from TB to Hundreds of TBs Balanced solutions ideal for data marts - EDW with scan-centric workloads Packaged and custom support From SMB to Enterprise Built on G Reference Architectures Appliance Business Data Warehouse DL0 (P) Up to Cores Internal HDD (TB) Basic RA DLx (P) Up to Cores P000 G (up to 0TB) Mainstream RAs DLx G (P) Up to Cores P000 G (Up to 0TB) Premium RA DL90 G (P) Up to Cores P000 G (Up to 0+ TB) Enterprise Data Warehouse Per /rack:x DL0 (P) 0 x P000G ( 0TB) Up to data racks (00TB)

11 Tier Enterprise Data Warehouse Value to Customer Enterprise-class scalability to 00s of terabytes High performance Interoperability with leading BI products Tier support and maintenance Mature SQL Server platform with high security and robust engineering process Strong data warehouse vision and roadmap that includes industry leading technologies Supporting Features MPP with shared nothing architecture Distributed query optimization Balanced hardware with pre-tested and pre-tuned appliances optimized for data warehousing third-party product integration (for example, Microstrategy, Business Objects, Informatica) Tier support and maintenance Roadmap includes column store, petabyte scalability, real-time data warehousing, MDM, and Data Quality

12 Affordability & Choice Value to Customer Low Cost: Low hardware cost Low maintenance cost Less risk and disruption on hardware standards Choice of deployment options Through distributed architecture that supports MPP hub and SMP spokes Enables customer to support users with different needs and SLAs Supporting Features Broad range of options with rack to rack, plus Dev/Test appliances Use of industry standard hardware Lower acquisition and maintenance cost than Teradata, Oracle, and IBM Distributed architecture supports PDW as the Hub and Fast Track data marts as spokes

13 The Most Complete Data Warehouse Solution Value to Customer Compelling End-user BI experience Agility to support new business groups with integrated data marts Fast time to value for BI through integration with Microsoft BI Complete data warehouse solution spanning desktop, data marts, and EDW Supporting Features PowerPivot, Reporting Services, and Microsoft Excel Distributed Data Warehouse Architecture integrates both MPP and SMP data warehouses Integration with Microsoft BI tools, including Analysis Services, Integration Services, and Microsoft SharePoint Most comprehensive toolset, including ETL, BI, MDM, and StreamInsight

14 ENTERPRISE DATA WAREHOUSE APPLIANCE Confidential NDA until June

15 Enterprise Data Warehouse Appliance Transforming SQL deployment BEFORE AFTER The world s most scalable, easy-to-manage enterprise data warehousing solution No single view of data across the enterprise Multiple siloed SQL data marts Proprietary, expensive EDW appliances 00TB+ = 0x Library of Congress printed collection

16 Enterprise Data Warehouse Appliance Optimized for SQL Server 00 R Parallel Data Warehouse Cut the complexity and expense of deploying very large data marts with industry-leading price/ performance and mission-critical support Complete All-in-one data warehouse appliance for mission-critical environments; New consulting services for Enterprise Data Warehouse Optimized Single view of information across the enterprise One-step solution update Enterprise-class scalability for complex queries at marketleading $/TB Agile The world s most scalable, easy-tomanage enterprise data warehousing solution Offers x scale to over 00 TB, and 0x to 00x faster queries

17 Corporate Network Private Network 9 SID DL0 G SID DL0 G SID DL0 G StorageWorks MSA000. inch drive Enclosure. inch drive Enclosure Passive Compute Node SID DL0 G SID DL0 G DL0 G SID DL0 G SID DL0 G SID DL0 G DL0 G SID DL0 G SID StorageWorks MSA000 Run data movement service DL0 G SID Perform local query processing; G DP 0k 00 GB G DP 0k 00 GB DL0 G G DP 0k 00 GB 00 GB G DP 0k G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB Active Server Control Rack Dual Fiber Channel. inch drive Enclosure StorageWorks MSA000. inch drive Enclosure StorageWorks MSA000 StorageWorks MSA000 DL0 G Backup Server and Storage Landing zone DL0 G SID Dual InfiniBand Monitoring Management nodes Active / Passive G DP 0k 00 GB. inch drive Enclosure G DP 0k 00 GB Dual Port k 0GB Dual Port k 0GB DL0 G Dual Port k 0GB Dual Port k 0GB Dual Port k 0GB PROC PROC 00 GB G DP 0k 00 GB Dual Port k G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k Dual Port k 0GB 0GB Dual Port k 0GB Dual Port k 0GB Dual Port k 0GB G DP 0k G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB G DP 0k 00 GB 00 GB PROC PS 99 PS 9 PROC ONLINE SPARE MIRROR CAP OVER TEMP Data Loading / ETL Central point for all Data HW Center monitoring CAP CAP G DP 0k OVER OVER TEMP PROC PROC FANS 99 ONLINE AMP SPARE STATUS FANS MIRROR DIMMS CAP CAP OVER OVER TEMP MIRROR SUPPLY SUPPLY ONLINE AMP SPARE STATUS 99 DIMMS PROC PROC SUPPLY SUPPLY PROC PROC CAP CAP OVER OVER TEMP PROC PROC FANS FANS SUPPLY SUPPLY DIMMS FANS ONLINE AMP SPARE STATUS MIRROR SUPPLY SUPPLY SUPPLY SUPPLY 99 G DP 0k 00 GB G DP 0k 00 GB PROC PROC 99 SUPPLY SUPPLY Controls DMS on all nodes 99 FANS Queries MPP engine runs User here Control nodes Active / Passive Control node Where clients apps connect Data Backup Enterprise Data Warehouse Architecture Data Rack Compute Nodes Compute nodes: Dedicated Storage Store user data; StorageWorks MSA000 StorageWorks MSA000. inch drive Enclosure StorageWorks MSA000. inch drive Enclosure DL0 G SID StorageWorks MSA000. inch drive Enclosure StorageWorks MSA000. inch drive Enclosure StorageWorks MSA000. inch drive Enclosure StorageWorks MSA000. inch drive Enclosure StorageWorks MSA000. inch drive Enclosure StorageWorks MSA000. inch drive Enclosure StorageWorks MSA000. inch drive Enclosure. inch drive Enclosure

18 Enterprise Data Warehouse Components Data Racks 0 + compute nodes per rack Infiniband Backup: DL0 G P000 G Control Nodes: DL0 G Fibre Channel Ethernet Management Nodes: DL0 G Landing Zone: DL- G Compute Nodes: DL0 G P000 G

19 Enterprise Data Warehouse HA Features Failover Clusters Redundant fabrics Redundant PDUs Redundant power Redundant cooling RAID 0 and hot plug disks Pre-failure alerting

20 Infiniband High-Level PDW Architecture Database Server Nodes Data Rack Storage Nodes (MSA) Fibre Channel

21 Architecture Storage Node Anatomy MSA P000 G x P000 G K.in ENT HDD, OR x P000 TB G.K.in MDL HDD, OR x G 0K.in DP ENT HDD, OR x 00GB G 0K.in DR ENT HDD

22 Infiniband High-Level PDW Architecture Database Server Nodes Data Rack Storage Nodes (MSA) Fibre Channel

23 Database Server node DL0 G (X0) 9GB RAM (x GB Rx PC-000R-9) DL0 SFF HD Backplane MB P-Series BBWC Upgrade x G 0K.in DP ENT HDD IB X DDR PCI-e DUAL PORT 0 mem HCA x 0W HE V Hotplug Power Supplies ilo Advance license internal tempdb disks OS disks

24 Measured Performance and Scalability Metric IO Scan Rate SQL Scan Rate Capacity per data rack Load rates Backup rates Restore rates Measurement.GB/sec per MPP node;.gb/sec per data rack ( x TB LFF MDL : 0.GB/sec/rack. Load and backup rates similar to SFF).GB/sec per MPP node;.gb/sec per data rack TB ( SFF) to TB (00GB SFF).TB/hour.TB/hour to TB/hour (depending on compression rates).tb/hour to 0TB/hour (depending on compression rates) System Under Test: Data Rack EDW with SFF disks. AU version of PDW Workload: Industry standard DW workload

25 Parallel Data Warehouse Hub and Spoke Plug Fast Track data marts into MPP cluster as spokes, receiving data from MPP hub Departments or business units keep their existing data marts Business Data Warehouse Enterprise data can be maintained on an EDW hub

26 Enterprise Data Warehouse Case Study Challenge Need better storage and access for report generation, balance checking and data analysis Solution Chose Enterprise Data Warehouse Optimized for SQL Server 00 R Parallel Data Warehouse Quick to install Results Reduced implementation risk Delivered X the performance at / the cost Can now generate reports in minutes instead of four hours Our expectations for setup, ease of management, and processing speed were exceeded at every point by and Microsoft we have tested the solution to generate reports in under minutes for what used to take over four hours.". Rich Hochron, CTO, Direct Edge

27 Our success to manage the business for more than 0 retail stores depends on our ability to gain insight into those stores to make the most optimal decisions, so that we can proactively manage fluctuations in demand, enhance operations and bolster business growth. Microsoft Parallel Data Warehouse based Enterprise Data Warehouse (EDW) Appliance s turn-key approach, coupled with and MSFT s best practices allowed us to deploy the EDW Appliance in a few days. It only takes hours to load data, which used to take more than days to load. We are able to make business decisions significantly faster as report generation and analysis of the information is performed in a more timely manner. - Assistant VP of Data Warehousing, Large US-based retail chain

28 EDW Support Pack Installation tools Default password update tool Time/Date synch tool BIOS setting synch tool Troubleshooting tools EDW Hardware Diagnostics tool Ongoing operational tools MRA Compliance Checker Push Button Update Utility (for firmware and drivers)

29 DEMO: Push Button Update Capabilities 9

30 Dramatic Simplification Game-changing solutions from and Microsoft,000 components updated with one button Enterprise Data Warehouse Appliance Consistent, validated and up-to-date environment 0

31 Converged Systems are Built on Converged Infrastructure. The right foundation for the workload Joint and Microsoft services for Seamless support: Collaborative partnering of Services with Microsoft Services Consulting expertise: Expertise with on-premises, outsource, or cloud delivery models Comprehensive offering: Over,000 combined and Microsoft professionals worldwide deliver a complete lifecycle Confidential NDA until June

32 Best-of-breed Partnership Delivering faster time to value + Microsoft co-innovation Integration of Converged Infrastructure through application services R&D + Services + Partner investments

33