IBM Mobile and Big Data with System z Enterprise assets Shyh-Mei F. Ho shyhmei@us.ibm.com IBM Distinguished Engineer IMS On Demand SOA Chief Architect SVL, San Jose, CA. USA 2013 年 11 月 11 日 星 期 一 1
Mobile is a mandatory transformation in the evolution of computing Mobile/Wireless/Cloud Web/Desktop Client/Server ISPF Mobile enables business: Premium Customer Service hotel bookings, flight check-in, etc Broaden Marketing Reach Put your branded icon on your customers smart device Increase Revenue Increase Operational Efficiency extend your existing enterprise systems to the mobile platform 2
Characteristics of Mobile Workloads On any platform Increased web traffic. Mobile applications drive an increase in overall transaction rates. When a user can check their bank balance anytime, they tend to do it more often. Increased off-peak web traffic. Traditional workload peaks change or are smoothed out when more mobile devices are driving the traffic. New mobile applications are often first deployed rapidly to cloud-based servers. According to Intel -- A new cloud server is required for every 600 smart phones (or 120 tablets) sold. This leads to over a million new servers required in 2013. Mobile applications could cause huge spikes in transactions. Consider a timesensitive offer sent to mobile users. Since they are more likely to see and respond to the offer quickly this could cause a huge and sharp spike in transaction invocations. This drives the move toward light-weight data transports like JSON. Think Sensors and Actuators. Mobile is more than smartphones. Think of any device relaying information to a server. 3
Uniqueness of Mobile Mobile devices are shared more often Mobile devices have multiple personas Mobile devices are diverse. Mobile devices are used in more locations Mobile devices prioritize the user Personal phones and tablets shared with family Enterprise tablet shared with coworkers Social norms of mobile apps vs. file systems Work tool Entertainment device Personal organization Security profile per persona? OS immaturity for enterprise mgmt BYOD dictates multiple OSs Vendor / carrier control dictates multiple OS versions A single location could offer public, private, and cell connections Anywhere, anytime Increasing reliance on enterprise WiFi Conflicts with user experience not tolerated OS architecture puts the user in control Difficult to enforce policy, app lists 4
z-unique Characteristics to support Mobile Applications Massive scalability in a single footprint, to handle the workload of millions of devices and sensors. Workload Management to provide a quick reaction to sharp spikes in demand Hardware encryption speeds SSL applications. Business Resiliency for critical mobile apps. Integration of co-located existing Applications, Services and Systems of Record 5
IBM MobileFirst Platform Overview of the IBM MobileFirst application and data platform 6
Delivering for multiple mobile platforms IBM Worklight Fast and cost-effective development, integration and management of rich, cross-platform mobile applications Client Challenge Using standards-based technologies and tools and delivering an enterprise-grade services layer that meets the needs of mobile employees and customers Key Capabilities Mobile optimized middleware Open approach to 3rd-party integration Mix native and HTML Strong authentication framework Encrypted offline availability Enterprise back-end connectivity Unified push notifications Data collection for analytics Direct updates and remote disablement Packaged runtime skins zlinux support 7
IBM Worklight: An Overview Studio Leading tools for cross platform hybrid development that maximize code reuse and enable per-device optimization Operational console Operational management for deployments, concurrent versions, and infrastructure access. Operational analytics provide real-time insight into application usage Application center A cross-platform private mobile application store focused on the needs of a development organization or a team Application runtime Cross platform services for the mobile channel (client & server componentry). Client APIs are introduced for both pure native as well as hybrid development 8
Integrated tools for all components of the mobile solution COBOL IBM Worklight Studio with Rational Developer for zenterprise Business Development Design Code Test Test Deploy WL Adapters Java Native App with HTML, JS, CSS Built on Eclipse Common code base across all mobile platforms (with ability to override at platform level) Build, preview, and deploy within the IDE Mobile simulator (for unit test) End-to-end debug Integrate with thirdparty SDKs (e.g. Android Development Tools) System z WebSphere Application Server with IBM Worklight Server Mobile IMS
IBM Integration portfolio IBM Cast Iron Synching data with SaaS apps to leverage new cloud economy IBM Caching Appliance Cache grids improve scale and performance IBM PureApplication System Enterprises looking to achieve more with less by better managing IT resources as collectives Private Cloud IBM DataPower Gateway Appliance B2B Integration Gateway for secure collaboration with communities of trading partners Public Cloud DMZ.NET.JCAPs SAP DB Pattern Pattern DB2 IMS CICS ODM BPM WAS DB DB DMZ Trading partner communities Mobile IBM Worklight Productive multidevice development and management Internet of Things IBM MQTT Reliable, efficient, scalable messaging for mobiles and sensors IBM Message Broker Integration Bus provides universal connectivity for heterogeneous environments across enterprise processes, applications, and data IBM MQ messaging Messaging backbone provides reliable transport and data delivery across data center IBM DataPower Gateway Appliance Integration Gateway for secure & controlled access to enterprise resources, while optimizing workload delivery Developer Communities IBM Web API Management Manage your APIs to open up access encouraging innovation from App Developers 10
Securely & Rapidly connect Mobile Apps with Enterprise Services Ensures application requests flow through as expected Legacy Apps IBM DataPower Gateway Appliance Web Apps, Services e.g. REST & JSON over HTTPS SSL Offload Threat Protection Validation, Filtering Authentication Authorization Transformation Rate Limiting Routing Intelligent Load Balancing Caching e.g. SOAP over HTTPS Message Oriented Apps IBM DataPower Gateway Appliances are the industry-leading Security & Integration gateway that helps provide security, control, integration and optimized access to a full range of Mobile, Web API, SOA, B2B and Cloud workloads 11
Mobile Enablement for System Z with WebSphere DataPower DataPower as a Mobile Gateway WebSphere DataPower is an appliance-based ubiquitous security and integration gateway. It provides Security, control, integration and optimization needs for mobile workload DataPower can play different roles in mobile enablement RESTful service facade DMZ proxy to secure your mobile network Seamless enterprise integration for IBM Worklight 12
DataPower and IBM Worklight 13 13
Mobile Enablement with DataPower for System Z Trusted zone DMZ RESTful Service Facade WebSphere DataPower e.g. REST (JSON/XML) over HTTPS WebSphere DataPower HTTP(S) with security token e.g. LTPA Integration with Worklight Worklight Server WebSphere DataPower System z Transactions & Database (IMS TM, IMS DB, CICS and DB2) 14
IBM WebSphere DataPower Family Service Gateway XG45 Entry-level device, slim footprint (1U) Security gateway (AAA, XML threat, etc) Service level management and monitoring Intelligent load distribution & dynamic routing Lightweight integration functions (optional module) Integration Appliance XI52 High density 2U form Any-to-Any conversion at wire-speed Bridges multiple transport protocols Mainframe integration & enablement Integration Blade XI50B/XI50z Functionally equivalent to XI52 Form factor flexibility XI50B: BladeCenter form factor XI50z: zenterprise BladeCenter Extension (zbx) form factor B2B Appliance XB62 High density 2U form B2B Messaging (AS1/AS2/AS3/ebMS) Trading Partner Profile Management B2B Transaction Viewer 15
Customer Demo: DataPower as RESTful Service Facade 16
IBM Big Data Platform Visualization & Discovery Application Development Systems Management Big Data Accelerators Hadoop System Stream Computing Data Warehouse Information Integration & Governance 17
On a Smarter Planet, unprecedented changes are occurring Cloud Business models under constant pressure Mobile Social Internet of Things Customers are more demanding and connected Great relationships trump great products 18
And leaders are responding by Providing a Great Experience Offering Value In Every Interaction Innovating Across the Ecosystem 19
Forward thinking organizations are creating value from Big Data The power of Data coming together to deliver improved business outcomes 1. Enrich your information base with Big Data Exploration with the power of Technology 2. Improve customer interaction with Enhanced 360º View of the Customer 3. Optimize operations with Operations Analysis 4. Gain IT efficiency and scale with Data Warehouse Augmentation 5. Prevent crime with Security and Intelligence Extension 20
What is Big data? Google can give you nearly 2 Billion options Vendors have even more definitions Here is how Gartner defines Big Data Big data is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative information processing for enhanced insight and decision making. 21
We ve moved into a new era of computing - V 4 Radical Flexibility 12 terabytes of Tweets create daily Volume Extreme Scalability 5 million Velocity trade events per second We have for the first time an economy based on a key resource [Information] that is not only renewable, but self-generating. 100 s Of video feeds from surveillance cameras Variety Information from everywhere Veracity Only 1 in 3 Decision makers trust their information Running out of it is not a problem, but drowning in it is. John Naisbitt 22
Demand for differently structured data to be seamlessly integrated, to augment analytics / decisions Analytics and decision engines reside where the DWH / transaction data is Noise surrounds the core business data Social Media, emails, docs, telemetry, voice, video, content Expanding our insights getting closer to the truth Lower risk and cost Increased profitability Data Warehouse Integration Business Analytics DB2 for z/os IMS Information Governance 23 Circle of trust widens
The Big Data starting point Where are organizations getting the most return on Big Data projects? 24 Source: 2012 IBM Global Big Data Online Survey
Big Data use cases Big Data Exploration Find, visualize, understand all big data to improve decision making Enhanced 360 o View of the Customer Extend existing customer views (MDM, CRM, etc) by incorporating additional internal and external information sources Security/Intelligence Extension Lower risk, detect fraud and monitor cyber security in real-time 25 Operations Analysis Analyze a variety of machine data for improved business results Data Warehouse Augmentation Integrate big data and data warehouse capabilities to increase operational efficiency
The IBM Big Data platform InfoSphere BigInsights Hadoop-based analytics for variety and volume as well as Data-At-Rest Hadoop IBM PureData System for Hadoop Reliable, enterpriseready appliance Easy-to-use, manageable system Built-in analytics InfoSphere Information Server High volume data integration and transformation Information Integration Netezza High Capacity Appliance Queryable Archive for Structured Data MPP Data Warehouse Stream Computing Smart Analytics System Netezza 1000 Operational Analytics on BI+Ad Hoc Analytics on Structured Data Structured Data InfoSphere Streams Low Latency Analytics for streaming data Velocity, Variety & Volume Data-In-Motion 26 InfoSphere Warehouse Large structured data analytics Informix Timeseries Time-structured analytics
The role of zenterprise in Big Data Analytics A large percent of the data that is accessed for analytics originates/resides on IBM zenterprise 2/3 of business transactions for U.S. retail banks 80% of world s corporate data Businesses that run on zenterprise 66 of the top 66 worldwide banks 24 of the top 25 U.S. retailers 10 of the top 10 global life/health insurance providers 1,300+ ISVs run zenterprise today, more than 275 of these selling over 800 applications on Linux The downtime of an application running on System z equates to approximately 5 minutes per year 27
DB2 Analytics Accelerator Accelerating decisions to the speed of business Blending System z and Netezza technologies to deliver unparalleled, mixed workload performance for complex analytic business needs. Get more insight from your data timely Fast, predictable response times for right-time analysis Accelerate analytic query response times Improve price/performance for analytic workloads Minimize the need to create data marts for performance Highly secure environment for sensitive data analysis Transparent to the application 28
Enhancing IMS analytics on System z with Big Data Much of the world s operational data resides on z/os Unstructured Observation points data lead sources to new are business growing opportunities fast There Observation is a need points to gleaned merge from this data both archived with trusted data OLTP and live data from System Score business z data events, sourcestrack claims evolution, and more IMS Make intends the data to available provide the to people connectors who can and do the something DB capability meaningful to allow BigInsights with it to easily and efficiently access the IMS data source 29
Machine Data Accelerator (MDA) - Log Data Analysis Custom Applications Shrink Wrap Solutions IT use cases: Server, performance, troubleshooting Health Care Networking Insurance Telco x2020 Unity Domain Specific MDA Accelerator IBM Big Data Platform Tools Client Specific Customizations, Visualization tools ( zinsights ) Telco Financial services Retail Healthcare Business use cases: Click stream and transaction analysis Optimize production, advance planning, IMS intends to provide Generic Parsers and Extractors Federated Discovery, Pattern (applications, services, Discovery, Search, Visualization Tools servers and devices ) for root cause analysis Hadoop System Information Integration & Governance Stream Computing Data Warehouse IBM Big Data Platform Visualization & Discovery Application Development Accelerators Systems Management Hadoop System Stream Computing Data Warehouse Information Integration & Governance 30
IMS Transaction and Big Data Business Scenario IMS transaction goes outbound synchronously to Big Data (BigInsights) to grab data and then return back to IMS transaction IMS OTMA Descriptor IMS Application Synch Callout (ICAL) BigInsights IMS Connect 31
32
Query execution process flow with DB2 & IDAA DB2 Application DB2 Optimizer SPU Query execution run-time for queries that cannot be or should not be off-loaded to IBM DB2 Analytic Accelerator IBM DB2 Analytic Accelerator DRDA Requestor SMP Host CPU FPGA Memory SPU CPU FPGA Memory SPU CPU FPGA Memory SPU CPU FPGA Memory DB2 IBM DB2 Analytics Accelerator (IDAA) 33 Heartbeat (IBM DB2 Analytics Accelerator availability and performance indicators) Queries executed without IBM IMS Analytics Accelerator Queries executed with IBM IMS Analytics Accelerator