IBM PureData System for Transactions. Technical Deep Dive. Jonathan Rossi, PureSystems Specialist

Transcription

1 IBM expert integrated system Technical Deep Dive Maria N. Schwenger, PureSystems Specialist Jonathan Rossi, PureSystems Specialist

2 IBM PureData System for Transactions - Agenda Feature Overview System Architecture and Setup System Console Overview Patterns and Deployment Database Console Overview Applications Monitoring Security 2

3 IBM expert integrated systems Feature Overview Slides - 10

4 Expertise builds champions

5 The Time Has Come For a New Breed of Systems Systems with integrated expertise that combine the flexibility of a general purpose system, the elasticity of cloud and the simplicity of an appliance Fundamentally changing the experience and economics of IT Built-in Expertise: Capturing and automating what experts do from the infrastructure to the application Integration by Design: Deeply integrating and tuning hardware and software in a ready-to-go workload optimized system Simplified Experience: Making every part of the IT lifecycle easier; Integrated management of the entire system; A broad, open ecosystem of optimized solutions

6 IBM Family of PureSystems Overview IBM PureData System Infrastructure Application Platform Data Platform Delivering Infrastructure Services Delivering Platform Services Delivering Data Services 6

7 2013 Announcements

8 Building blocks definition

9 Workload optimized performance Data load ready in hours Integrated management Automated maintenance Single point of support Data Platform Delivering Data Services 9

10 Different data workloads have different characteristics System for Transactions Database services that handle large volumes of transactions with high availability, scalability and integrity System for Analytics powered by Netezza technology Data Warehouse services for complex analytics and reporting on data up to petabyte scale - with minimal administration System for Operational Analytics Operational Warehouse services for continuous ingest of operational data, complex analytics, and a large volume of concurrent operational queries

11 IBM PureData Systems for Transactions Integrated Design, Built-in Expertise and Simplified Experience All Inclusive Comprehensive System Management Storage Data Management Networking Databases Minimum Set-up Time Pre-installed, Pre-integrated Single pane-of-glass management, administration and maintenance Virtualization in PureSystems Management (PSM) Node Servers Deployment 11

12 Simplified Experience Single Pane of Glass IBM PureData Console IBM PureData Console Network Chassis Compute ITE Compute ITE Compute ITE Compute ITE Workload Console Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE System Console Compute ITE Compute ITE Network Storage Unit 12

13 Loading Dock to Loading Data in Just Hours Plug-in system and network No assembly required Power-on Integrate system in your network Use the Workload Console to: Create the clustered instances Configure administrative access to the instance Create and Deploy a database Use the Database Operation Console to: Configure access control Add disk storage to the database Load Objects and Data Use IBM Data Studio In Just Hours From Loading Dock To Loading Data 13

14 IBM expert integrated systems System Architecture and Setup Slides - 36

15 System Architecture and Setup Hardware, Software and Setup 15

16 Hardware Architecture Small, Medium and Large Configuration P D U P D U Top of Rack Switch Top of Rack Switch V7000 Controller V7000 Expansion V7000 Controller V7000 Expansion V7000 Controller V7000 Expansion V7000 Controller V7000 Expansion System Console Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE Compute ITE PSM 1 PSM P D U P D U 42U 41U 40U 39U 38U 37U 36U 35U 34U 33U 32U 31U 30U 29U 28U 27U 26U 25U 24U 23U 22U 21U 20U 19U 18U 17U 16U 15U 14U 13U 12U 11U 10U 09U 08U 07U 06U 05U 04U 03U 02U 01U On-line Upgrade On-line Upgrade 16

17 Hardware Architecture Hardware Details External Network Connectivity Dual 10Gb Ethernet Switches for external and rack-to-rack communication Storage System Utilizes Storwize v7000 chassis and expansion units Holds up to 192 disks in large configuration 1:3 workload balanced SSD to HDD ratio Easy tier Balanced HDD & SSD 48-disk module (12 SSD + 36 HDD) to optimize for performance & cost RAID 10 for extreme storage reliability Up to 9.5 TB SSD and 64 TB HDD capacity Flex System Chassis Up to 2 per rack (Large) Holds up to 14 compute nodes each Back plane with fully redundant Network (Ethernet) and Storage (SAN) connectivity Compute Nodes Up to 24 per rack Serves as CF and Member nodes for purescale instances Pure System Manager 2 per rack for redundancy Integrate management for all system resources

18 Software Architecture Included with IBM PureData System Compute ITEs Red Hat Enterprise Linux 6.2 IBM DB2 Enterprise Server Edition 10.1 FP1 with DB2 purescale Feature Storage optimization Feature Workload Management Q Replication for business continuity TSM client for policy based backup and restore PureSystems Manager Nodes Optim Performance Manager IBM Data Studio System Console 18

19 Software Architecture Driving principles for the design of DB2 for PureData Extreme Capacity Designed for OLTP Start small Grow easily, with your business Move from small to medium to large config Application Transparency Avoid the risk and cost of changing applications because of the database topology Automatic Work Load Balancing, Automatic Client Reroute, and Client Affinity Continuous Availability Maintain service across unplanned events Maintain service across planned events using maintenance 19

20 Software Architecture PureData System DB2 Features Combining DB2 with proven IBM clustering technologies such as RSCT, GPFS etc. Clients connect using Ethernet Clustering Services for hosts (RSCT) and storage (GPFS) Interrupt free RDMA calls through the network SAN based database 20

21 Software Architecture CF and DB2 Members DB2 member in PureData: DB2 engine address space can be logical or physical All members access the same shared data (True data sharing) Each member has its own buffer pool, memory, log files, etc. Cluster caching facility (CF) in PureData Inherits golden standards from System z parallel sysplex Group Buffer Pool (GBP), Global Lock Management (GLM), and Shared Communication Area (SCA) Efficient communications between CFs and Members via RDMA Multiple DB2 member in a cluster Cluster caching facility (CF) for global lock and buffer pool management 21

22 Software Architecture Cluster Elasticity IBM PureData Choose from pre-configured topology Up to 3 PureData instances for the small configuration Up to 6 for medium and 12 PureData instances for large. 22

23 Software Architecture PureData Database Ease of Use No cluster manager scripting or configuration required Setup automatically upon installation Automatic install of all components across all desired hosts Automatically configured using best practices Application Transparency No cluster awareness in the application No affinity routing (Can be configured if desired) No application / data partitioning Automatic Work Load Balancing Member hardware failure is transparent to the applications Automatic connections reroute to the active members Member re-integration in cluster is transparent to the applications DBAs add capacity without re-tuning and re-testing 23

24 IBM expert integrated systems System Console Overview Slides - 12

25 Launch Console System Console IBM PureData Console 1. System Console 2. Workload Console 25

26 System Console Managing Hardware Resources Infrastructure Map Compute Nodes Management Nodes Storage Devices Network Devices Hosts 26

27 Managing Hardware Resources Hardware Infrastructure Map View performance data Check Status Check LED lights Check temperature for each part Monitor jobs associated with the hardware device See name and serial number of each hardware device 27

28 Managing Hardware Resources Hardware Compute Nodes 1. Power On 2. Power Off 3. Start 4. Suspend 5. Maintenance 6. Monitoring 28

29 Managing Hardware Resources Hardware Storage Devices Select a Storage Device to monitor Events Jobs Types Firmware Status Capacity Location Temperature Physical Cores Disk Drives Operating System volumes Storage volumes LUNs Storage controller ports Storage node statistics LEDs 29

30 Managing Hardware Resources Hardware Network Devices View Network Device Events Jobs Switch type Description Location Firmware level Software version Model Customer ports Network ports 30

31 Managing Hardware Resources Hardware Hosts Configure Hosts Access granted to Description Storage volumes IP addresses Managing Hosts Jobs Description Status Created On Updated On CPU Memory In cloud group On compute node Operating system volume Storage volume Access grants DB2 purescale instances 31

32 Managing System Settings Configure , Date/Time, DNS, Events, and Backup/Restore Configure System Console System System Settings Expand Mail Delivery Specify SMTP server etc. Managing Date / Time System Console System System Settings Expand Date and Time Specify IP address of NTP server etc. Configure Event Forwarding System Console System System Settings Expand Event Forwarding Specify SNMP destination and download MIB file Managing DNS System Console System System Settings Expand Domain Name Service (DNS) Specify DNS Server IP addresses Managing Backup / Restore System Console System System Settings Expand Backup and Restore Complete External Backup Server configuration 32

33 IBM expert integrated systems Patterns and Deployments Slides - 10

34 Patterns of Expertise Decades of Experience Decades of clients experience captured, lab tested and optimized into a deployable form What is a pattern? The pre-defined architecture of cluster, database For each cluster Pre-installation of an OS Pre-integration across components Pre-configured and tuned Pre-configured security Pre-configured monitoring Complete lifecycle management What is a deployment? Using predefined pattern to deploy Repeatable deployment with full lifecycle management 34

35 Do It Yourself OR Built by Experts Taking drudgery out of complexity Traditional System build-it yourself Takes several weeks Procure HW/SW/Network and configure Define HA topology Set up storage pools Install multiple operating systems Install database instances Set up backup policies Test, tune and reconfigure Traditional System Manual, brittle Install Deploy PureData System Point and click PureData System built-in expertise Time consuming Pattern-based Just in hours Choose topology pattern and click OK. OK Manage Manual optimizations Pre-optimized by experts 35

36 Instance Patterns Small Configuration Instance Patterns Automatically creates, configures, and deploys an Instance cluster topology with built-in redundancy and high performance Small Configuration Rack Maximum 3 Instances 3 Instances of 2 CF and 2 Members on 2 ITE each 2 Instances of 2 CF + 2 Members on 2 ITE and 2 CF + 2 Members on 4 ITE 1 Instance of 2 CF and 4 Members on 6 ITE Legend: CF and Member collocated Only CF Only Member 36

37 Storage Patterns Optimized for Instance / Databases Small Configuration 1 st Instance / 1 st DB 2 nd Instance / 1 st DB 3 rd Instance / 1 st DB 3 rd Instance / 2 nd DB V (Pool 1) /db2plog/db2inst1/dbname1 /db2fs/db2inst2/dbname1 /db2mlog/db2inst2/dbname1 /db2plog/db2inst3/dbname1 /db2fs/db2inst3/dbname4 /db2mlog/db2inst3/dbname4 EXP 1 (Pool 2) /db2fs/db2inst1/dbname1 /db2mlog/db2inst1/dbname1 /db2plog/db2inst2/dbname1 /db2fs/db2inst3/dbname1 /db2mlog/db2inst3/dbname1 /db2plog/db2inst3/dbname4 37

38 IBM expert integrated systems Workload Console Overview Slides - 8

39 Launch Console Workload Console IBM PureData Console 1. System Console 2. Workload Console 39

40 Creating / Managing Instances, Databases and Tools Workload Console DB2 purescale Instances Databases Database Patterns Database Workload Standards 40

41 Creating Instance Database DB2 purescale Instances Create DB2 purescale Instance Name Description Size Small/Medium/Large Compatibility mode Database version Database level Port number Maximum number of databases Click OK OK 41

42 Creating Database Pattern Database Database Patterns Create Database Pattern Name Description Pattern Database version Database level Database size Compatibility mode Schema file Advanced options Click OK 42

43 Deploying Database Using a Pattern Database Databases Deploy Database Name Description Default user Password Instance Name Click OK 43

44 Administering a Database Easy Management Manage Database Schedule database backups Create a database image List all database images Database Storage Management Add more storage 44

45 Manage Instance Grow Instance Growing DB2 purescale Instance Change size to large Click Submit 45

46 Simplified maintenance with pre-integrated fixes Reduce risk and eliminate manual errors when applying maintenance All hardware firmware and OS software patches integrated and tested together at the factory Can apply hardware and OS maintenance with zero downtime Single line of support Integrated stack support

47 High Availability and Disaster Recovery PureData System for Transactions is highly available by design Redundant hardware, networking and storage Recovery from hardware and software failures with DB2 purescale Additional capacity can be added and utilized when needed PureData systems can be configured with Q-Replication or InfoSphere Change Data Capture (CDC) to provide DR capabilities Not installed on PureData - installed on separate replication server Q Replication is now licensed with system CDC not bundled/installed/licensed with system CDC/Q-Replication

48 IBM expert integrated systems Applications Slides - 8

49 PureData Systems for Transactions Application Transparency Work Load Balancing (WLB) Distributes application requests in a balanced fashion among DB2 members WLB is handled based upon capacity information returned by the purescale server Lower capacity value than other members indicate that the member is busier than others Automatic Client Reroute (ACR) Provides a mechanism to reroute application requests from the failing members to the other available members With ACR enabled, the planned or unplanned outages do not affect applications so long as there is one surviving DB2 member Seamless ACR happens when failover was at the start of the UOW Non-seamless ACR returns error and connection is rerouted automatically Client Affinity Controls how DB2 members are used by the applications Client wants to connect to a particular member only If failover happens, it will reroute connection to other member defined in the list Should failed member become available again, client will reconnect to the available server again A must read white paper for DB2 purescale application: or 49

50 IBM expert integrated systems Monitoring Slides - 16

51 PureData Systems for Transactions Monitoring Single Pane of Glass for Monitoring IBM PureData Console System Monitoring Management Nodes Compute Nodes Network Storage Hosts Database Monitoring Single Sign-on and Trusted Context Monitored databases End-to-end Performance Monitoring of Java and non-java applications (License included) 51

52 Database performance management from integrated console

53 Database Performance Monitor Key Performance Monitoring Health Summary Alerts Configuration Dashboards Overview SQL Statements Buffer Pool and I/O Locking Logging Memory System Utilities Utilities Workload Connection Extended Insight Reports Custom Reports Predefined Reports 53

54 Database Performance Monitor Extended (Top-down) and Core (Bottom-up) Monitoring Application Program PureData DB2 Servers Top-down With Extended Insight OPM Server Bottom-up With Core Capabilities DB2 OPM Repository 54

55 IBM expert integrated systems Security Slides - 11

56 System Level Security Role Based Security Separation of Duties Workload Administration Auditing Administration Cloud group Administration Role Based Security Separation of Duties Hardware Administration Database configuration Management Security Administration 56

57 Security Model At System Level Allows users to view / manage system console and hardware elements Create users & user groups At Instance Level Controls access to instance and to databases within that instance Create instance users & user groups At Database Level Access to objects within the database Create database users & user groups 57

58 System Level Security Authentication Methods LDAP based User authentication and group membership is managed through a central authority Users connecting to database can still be defined at LDAP even if you choose not to use LDAP for system-level authentication Operating System OS level users and groups Can mix local and LDAP users and groups 58

59 When to use PureData System for Transactions? Looking for fast time-to-value with the IBM s workload optimized systems Require out of the box Continuous Availability for business application workloads Are struggling to deal with increasing volumes of data and skyrocketing data management costs Require maximum business application transaction performance to maintain a competitive edge May or may not have DB2 purescale and/or DB2 installed May have looked at DB2 purescale in the past but were not ready for it Currently have PureApplication System installed and looking to expand OLTP environments Reduce complexity The entire system lifecycle is simplified from acquisition to retirement Accelerate time to value Data management expertise is built in each system and ready for immediate use Improve IT economics Each system tuned for the different needs of different analytic and transaction processing

60 Backup Slides

61 Hardware Architecture Available Configurations 61 Continued.

62 Hardware Architecture Available Configurations (Continued ) 62

63 Hardware Architecture Storage Design Small Rack Small Rack has 12 SSD and 36 HDD 6 MDISK for HDD (Each with 6 drive RAID 10 *) 2 MDISK for SSD (Each with 6 drive RAID 10) 2 Storage Pool each with 3 HDD MDISK + 1 SSD MDISK Each storage pool space 9.3 TB (13% of SSD) Each storage pool is divided into generic volumes (VDISKS) * RAID 10 Disks are mirrored and striped 48 Internal Drives 8 MDisks 2 Pools 128 VDisks 63