BI and Data Warehousing on the Mainframe

BI and Data Warehousing on the Mainframe WebFOCUS for IBM System z Data Warehousing A White Paper

Table of Contents 1 Introduction 3 Addressing the Myths 3 Myth 1: Capacity Planning Is a Challenge 3 Myth 2: Data Warehouses Can t Fully Leverage Mainframes 3 Myth 3: Processing Is Cost-Prohibitive 4 Myth 4: Graphs Are Difficult to Generate 4 Myth 5: Making Information Available to Many Users Is Complex 4 Myth 6: Data Cannot Be Accessed in Real Time 5 Why IBM and Information Builders? 5 Data Loader for DB2 6 Data Reporter for DB2 7 Conclusion

Introduction One of the amazing things about technology is that older ideas often have a chance to evolve, not just simply fade away. One person s Cloud is another s server farm, and chances are that server farm comprises mainframe hardware IBM System z, for instance as well as overstuffed data warehouses. As data-warehouse appliances and larger in-memory data caches have secured a toehold in the ever-shifting market, it s important for IBM System z customers to realize that the data-warehouse infrastructure already exists. In the late 80s, IBM researchers Barry Devlin and Paul Murphy developed the business data warehouse concept. The term was coined in the 1988 IBM System Journal, Vol. 27, No. 1, when Devlin stated, A data warehouse is simply a single, complete, and consistent store of data obtained from a variety of sources and made available to users in a way they can understand and use in a business context. 1 However your business changes, one thing remains constant: inconsistent data translates into inconsistent actions. Enterprises rely on the data-warehouse and related business intelligence (BI) infrastructure to understand and react appropriately to change. Business executives who need analytical information at their fingertips turn to BI to transform data from the warehouse into actionable information. Several myths exist about typical data-warehousing environments, including: Data is cleansed and organized before it enters the warehouse Managers understand how data is collected, constructed, and reported Warehouses cannot scale to large data volumes or support a large, concurrent user count Warehouses cannot support mixed workloads Warehouses cannot support the Cloud or virtualization Warehouses are not dynamic Mainframes provide poor support of query and advanced analytics operations The biggest knock against data warehouses on System z is that they are typically expensive, unwieldy, inflexible, and require vast resources to maintain and the mainframe only adds exponentially to those costs. A data-warehouse appliance combines commodity processors and hardware with BI and data-warehouse software within a single box. Hardware and processors can be added as the warehouse grows, while data is segregated into yet another server in a profusion of distributed data stores. New islands of data are typically introduced using an appliance that represents nothing more than adding another server to the burgeoning farm population, thereby creating yet another copy of data to maintain, update, and control. A different approach is to integrate warehousing applications within the operational fabric of your business to ensure accurate, consistent, and timely access to information. Today, CIOs want the ability to streamline IT infrastructure and costs by consolidating data into a single view, returning it to a centralized, secure environment that supports the enterprise with current, consistent information. 1 Devlin, Barry, Data Warehouse: From Architecture to Implementation, Addison-Wesley, 1997. 1 Information Builders

Information Builders and IBM mitigate the risks and data latency, while optimizing information delivery options from data warehouses based on System z mainframes. Organizations can access all their data to fuel BI operations, no matter where it is located, or what application it s in. Data can even be quickly and easily retrieved from any platform, including UNIX and Intel. Access to continuous real-time data remains crucial to a business s ability to function optimally. The payoff is substantial for enterprises that have continually invested in System z architecture, as System z remains the centralized information hub that can power Cloud, Software-as-a-Service (SaaS), or any other initiative. 2 BI and Data Warehousing on the Mainframe

Addressing the Myths There are many myths when it comes to data warehousing in mainframe environments. But IBM s continuous investment in System z and specialty engines like the IFL, ziip, and zaap enhances performance by enabling data warehouses to be dynamic and scalable. Myth 1: Capacity Planning Is a Challenge The appeal of the Cloud is dynamic allocation of storage when the data volume of the warehouse varies unpredictably, which traditionally makes capacity planning difficult. But off-loading processes to a ziip processor mutes this criticism by improving overall performance and response time, allowing the system to scale to larger volumes and manage a mixed workload. Myth 2: Data Warehouses Can t Fully Leverage Mainframes The problem is not that most data warehouses can t take advantage of the mainframe it s that they don t. For example, they require the purchase of commodity hardware and additional servers requiring more maintenance as you grow. But the ultra-high reliability and high-capacity workload of a mainframe can be achieved for less than it would cost to build a new environment on Unix or Windows. In fact, on System z, you can reduce off-loading and staging complexity by leveraging the ziip. Centralizing your data warehouse on the hardware where your data and applications reside reduces operating costs and improves performance and efficiency. It also allows you to scale your data warehouse to enterprise usage. Myth 3: Processing Is Cost-Prohibitive The truth is, simply transferring processing requirements to the ziip chip debunks this myth. In fact, up to 80 percent of processing requirements can be transferred to the ziip chip. Processing costs on a mainframe are always measured by the amount of MSUs (MIPS) used. With the ziip, the workload transferred to the chip is not counted in the size of the computer or the cost of processing in other words, it s free. Information Builders IBM s partner in the WebFOCUS for IBM System z Data Warehousing solution is one of the only companies in the world that makes software to intelligently switch processing between the general processor and the ziip chip in a z/os environment. Think of this chip as a BI accelerator. Once the I/O is taken care of, business intelligence must provide rich Internet formatting, calculations, analytics, and other CPU-intensive operations, all of which is off-loaded to the ziip processor. So, if you want to deploy a large customer-facing analytics application from your System z data warehouse with real-time data from the mainframe, you can do more with what you have. IBM offers significantly lower prices for a ziip engine. The net result is that you get a big discount on your MIPS usage and all the computer power you need without having to buy backup or peak-load capacity. 3 Information Builders

Myth 4: Graphs Are Difficult to Generate Another common computational requirement is the generation of graphs. A great deal of BI information is presented in graphs, and the WebFOCUS graph engine runs in Java. If a z/os site wants to process Java more economically, it can use the IBM zaap chip, which is another special-purpose chip that doesn t increase the computer s rating. It speeds up Java applications by moving the processing of the Java Virtual Machine (JVM) off the general processor to the zaap chip, thus reducing cost by the same amount. Myth 5: Making Information Available to Many Users Is Complex The IBM z/os mainframe is universally acclaimed for its ability to process large-scale transactions. Large-scale, custom-built, transaction-processing applications like data warehouses are the source of the massive amount of data that resides on the mainframe. There is a corresponding ability to build another class of application on the mainframe that is not oriented toward processing transactions, but rather toward processing information that is delivered via Web browser, in a rich visual context to large populations of internal and external users who have no end-user training. The business driver is the need for up-to-date information to enable people to make better decisions. For example, sharing information with customers can reduce errors, encourage brand loyalty, create entirely new ways of doing business, and lead to new revenue opportunities. Such information and reporting applications cannot use BI tools per se; rather, they are built upon BI technology. The advent of ziip technology allows these applications to be implemented directly on the mainframe more economically than moving data off the mainframe and building new environments. For example, the State of Florida Treasury built a self-service reporting system for more than 200,000 vendors. The entire state s vendor invoices and payments were housed in a central system run on the Treasury s z/os system. To get information, vendors used to call the department for answers to questions as simple as, Was my invoice received? Now vendors can find their own information and check account details. Errors are caught much more quickly and do not propagate in the system. Thirty state staff members are also free to do other work. Myth 6: Data Cannot Be Accessed in Real Time Another stumbling block to data warehousing and analytics is the perceived lack of real-time data access for reporting. Integration technology allows trickle-feeding of the data warehouse, which means new records are added right away. As soon as new data is entered into the operational system, it is extracted, transformed, and loaded into a real-time repository. For instance, if your customer is a telephone company with a data warehouse that includes information about customer accounts, invoices, service orders, products, support histories, and more, their call-center representatives will always have up-to-the-minute information about customer accounts and inquiries, and customers don t get passed from one division to another. 4 BI and Data Warehousing on the Mainframe

Why IBM and Information Builders? The mandate from management is that information needs to be available now, but a close eye must be kept on cost. The IT group must figure out how to make the information architecture less complex, using less hardware and resources to keep costs contained while mitigating the risk of failure. Complexities in appliances, and the segmentation of data and workloads, are the enemy of cost-effective operations because they increase costs and operational management while raising questions about performance. Data-warehousing appliances theoretically met the need for a solution with a faster time to market. However, while appliances have been more widely used, businesses are beginning to realize that using commodity hardware does come with tradeoffs, such as higher administration costs and reduced availability. There is an alternative for businesses invested in the System z mainframe platform. Improvements in technology, environmental impact, cost of ownership, and data-warehousing and BI software capabilities have made System z a highly desirable platform for data warehousing and BI. With WebFOCUS for System z Data Warehousing, businesses can align the advantages of the platform with the requirements of the business application. WebFOCUS for System z Data Warehousing is the industry s only single-environment solution, providing a comprehensive and fully integrated suite of data warehousing and business intelligence software products. It combines the power, flexibility, and reliability of IBM s System z Solution Edition for Data Warehousing with native z/os-based, easy-to-use tools that deliver data warehousing and BI with dramatically reduced implementation complexity and TCO, as compared to similar distributed systems or dual operating environment approaches. The solution comprises two components: DB2 Data Loader and DB2 Data Reporter. Data Loader for DB2 Data Loader for DB2 is a specialized version of iway Software s z/os-based DataMigrator product, designed for optimal loading and management of IBM DB2 database data synchronization processes. Data Loader for DB2 enables organizations to: Load/synchronize a DB2-based enterprise data warehouse through comprehensive access to more than 200 database, file, and application sources Finely tune metadata creation and management capabilities using common metadata introspection, modeling, and generation techniques, regardless of information resource, type, or location Optimize DB2 data-load capabilities on a real-time, near real-time, or batch basis Federate data in a highly sophisticated manner with drill-through capabilities that provide access to detailed operational data via a DB2 warehouse model, implement, manage, and audit warehouse loading 5 Information Builders

Data Reporter for DB2 Data Reporter for DB2 is a specialized version of Information Builders z/os-based WebFOCUS BI platform, designed for building Internet-based BI applications or as a guided ad hoc query tool for businesses using a DB2 enterprise data warehouse. A highly graphic developer workbench enables developers to quickly and easily design, deploy, and manage Internet-based applications or query solutions for large or small user groups that don t require end-user training or software licensing for each user. Data Reporter for DB2 offers: Graphically rich, fully integrated, and easy-to-use design tools for developing medium- to verylarge-scale BI applications End-user, guided ad hoc query capabilities to support any amount of Web-based usage Shared server capabilities that enable BI applications/users and IT to use a common server architecture for data access, manipulation, metadata creation/utilization, and data federation within z/os The scalable, reliable, and flexible server architecture within IBM system z technology such as the ziip specialty engine dramatically lowers the operational costs of using a DB2-based data warehouse. Optional service-based components for data cleansing and data quality management are available. 6 BI and Data Warehousing on the Mainframe

Conclusion The WebFOCUS Framework for Data Warehousing is a completely integrated software solution that uses a common server architecture and implementation tools. When combined with IBM s System z Solution Edition for Data Warehousing, it forms a complete and unified hardware and software solution for implementing enterprise data warehousing. Designed as a software solution to be deployed within a single, native z/os operating environment, WebFOCUS Framework for Data Warehousing uses a dedicated System z LPAR or standalone System z server to deliver a simple approach to the design, implementation, and management of a data-warehousing solution all at a lower acquisition cost and overall TCO compared to similar multi-tiered or environmental solutions. Building your data warehouse on IBM System z is the best way to consolidate and organize all your data so it can be easily managed, accessed, and analyzed. With a single management interface, as well as self-tuning and self-diagnostic features, DB2 simplifies maintenance of your ever-expanding data warehouse. IT likely will find significant benefits from having fewer specialized resources to manage System z, which centralizes the data warehouse and analytics where the data resides. This results in fewer instances of RDBMSs requiring tuning and optimization for queries and reports, eliminating many of the complex network connections that distributed environments use to communicate. Companies can reduce the number of IT personnel who only keep the lights on and redeploy the most skilled of them to respond to business needs for data assets. It is clear that WebFOCUS for IBM System z Data Warehousing offers significant potential benefits for organizations that need to store and manage rapidly growing volumes of data and provide that data for business intelligence and analytics to increasing numbers of users. As you analyze your current and future business needs, take these factors into account: Achieving TCO. Apply unified server architecture, data integration on IBM System z, Data Loader for DB2, and Data Reporter for DB2 Improved Performance. A package designed for analytics that keeps up with user demand and increased data Turnkey Solutions. Proven solutions to reduce time to launch and minimize required maintenance Scalability. Adding capacity is a simple matter of leveraging a ziip to off-load non-essential processes to another processor Respond to user demands for power, speed, and flexibility while answering executives calls for solutions that deliver performance for the lowest cost. Evaluate existing System z environments and current trajectory of costs versus benefits, analyze the options for supplementing or replacing them, and create a plan that can help them be more agile in turning data into decisions and ultimately value. 7 Information Builders

Worldwide Offices North America United States Atlanta,* GA (770) 395-9913 Baltimore, MD Professional Services: (703) 247-5565 Boston,* MA (781) 224-7660 Channels, (800) 969-4636 Chicago,* IL (630) 971-6700 Cincinnati,* OH (513) 891-2338 Dallas,* TX (972) 490-1300 Denver,* CO (303) 770-4440 Detroit,* MI (248) 641-8820 Federal Systems,* DC (703) 276-9006 Hartford, CT (860) 249-7229 Houston,* TX (713) 952-4800 Los Angeles,* CA (310) 615-0735 Minneapolis,* MN (651) 602-9100 New Jersey* Sales: (973) 593-0022 New York,* NY Sales: (212) 736-7928 Professional Services: (212) 736-4433, ext. 4443 Orlando,* FL (407) 562-1852 Philadelphia,* PA Sales: (610) 940-0790 Phoenix, AZ (480) 346-1095 Pittsburgh, PA Sales: (412) 494-9699 St. Louis,* MO (636) 519-1411 San Jose,* CA (408) 453-7600 Seattle, WA (206) 624-9055 Washington,* DC Sales: (703) 276-9006 Professional Services: (703) 247-5565 Canada Information Builders (Canada) Inc. Calgary (403) 437-3479 Montreal* (514) 421-1555 Ottawa (613) 233-7647 Toronto* (416) 364-2760 Vancouver (604) 688-2499 Mexico Information Builders Mexico Mexico City 52-55-5062-0660 Australia Information Builders Pty. Ltd. Melbourne* 61-3-9631-7900 Sydney* 61-2-8223-0600 Europe Belgium* Information Builders Belgium Brussels 32-2-7430240 France* Information Builders France S.A. Paris 33-14-507-6600 Germany Information Builders (Deutschland) Eschborn* 49-6196-77576-0 Netherlands* Information Builders (Netherlands) B.V. Amsterdam 31-20-4563333 Portugal Information Builders Portugal Lisbon 351-217-217-400 Spain Information Builders Iberica S.A. Barcelona 34-93-344-32-70 Bilbao 34-94-452-50-15 Madrid* 34-91-710-22-75 Switzerland Information Builders Switzerland AG Dietlikon 41-44-839-49-49 United Kingdom* Information Builders (UK) Ltd. London 44-845-658-8484 Representatives Austria Raiffeisen Informatik Consulting GmbH Vienna 43-12-1136-3870 Bahrain InfoBuild Middle East Dubai 973-17-536-222, ext. 312 Brazil InfoBuild Brazil Ltda. São Paulo 55-11-3285-1050 China Information Builders China (IBC) Beijing 86-10-5128-9680 Beijing Xinrong Software Technology Co., Ltd. Beijing 86-10-5873-2031 Denmark InfoBuild AB Kista, SE 46-735-23-34-97 Egypt InfoBuild Middle East Abu Dhabi 971-2-627-5911 Dubai 971-4-3914391 Ethiopia MKTY IT Services Plc Addis Ababa 251-11-5501933 Finland InfoBuild Oy Espoo 358-207-580-840 Greece Applied Science Athens 30-210-699-8225 Guatemala IDS de Centroamerica Guatemala City 502-2412-4212 India* InfoBuild India Chennai 91-44-42177082 Israel SRL Group Ltd. Tel Aviv 972-3-7662030 Italy NessPRO Italy S.p.A. Genoa 39-010-64201-224 Milan 39-02-2515181 Turin 39-011-5513-211 Japan K.K. Ashisuto Osaka 81-6-6373-7113 Tokyo 81-3-5276-5863 Jordan InfoBuild Middle East Abu Dhabi 971-2-627-5911 Dubai 971-4-3914391 Korea UVANSYS Seoul 82-2-832-0705 Kuwait InfoBuild Middle East Dubai 965-22322926 Lebanon InfoBuild Middle East Dubai 961-4-533162 Norway InfoBuild Norway Oslo 47-48-20-40-30 Oman InfoBuild Middle East Abu Dhabi 971-2-627-5911 Dubai 971-4-3914391 Poland/Central and Eastern Europe InfoBuild SP.J. Warsaw 48-22-657-00-14 Qatar InfoBuild Middle East Dubai 974-467-7311 Russian Federation FOBOS Plus Co., Ltd. Moscow 7-495-926-3358 Saudi Arabia InfoBuild Middle East Riyadh 996-1-2180280 Singapore Automatic Identification Technology Ltd. Singapore 65-6286-2922 South Africa InfoBuild South Africa (Pty.) Ltd. Gauteng 27-83-4600800 Fujitsu Services (Pty.) Ltd. Johannesburg 27-11-2335911 Taiwan Galaxy Software Services Taipei 886-2-2586-7890 Thailand Datapro Computer Systems Co. Ltd. Bangkok 662-679-1927, ext. 200 Turkey InfoBuild Middle East Ankara 90-312-266-33-00 Istanbul 90-212-325-4114 United Arab Emirates InfoBuild Middle East Abu Dhabi 971-2-627-5911 Dubai 971-4-3914391 Venezuela InfoServices Consulting Caracas 58-212-763-1653 Toll-Free Number Sales, ISV, VAR, and SI Partner Information (800) 969-4636 * Training facilities are located at these branches. Corporate Headquarters Two Penn Plaza, New York, NY 10121-2898 (212) 736-4433 Fax (212) 967-6406 DN7506391.0910 informationbuilders.com askinfo@informationbuilders.com Canadian Headquarters 150 York St., Suite 1000, Toronto, ON M5H 3S5 (416) 364-2760 Fax (416) 364-6552 For International Inquiries +1(212) 736-4433 Copyright 2010 by Information Builders. All rights reserved. [92] All products and product names mentioned in this publication are trademarks or registered trademarks of their respective companies. Java and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. Printed in the U.S.A. on recycled paper