BI and Data Warehousing on the Mainframe

Transcription

1 BI and Data Warehousing on the Mainframe WebFOCUS for IBM System z Data Warehousing A White Paper

2 Table of Contents 1 Introduction 3 Addressing the Myths 3 Myth 1: Capacity Planning Is a Challenge 3 Myth 2: Data Warehouses Can t Fully Leverage Mainframes 3 Myth 3: Processing Is Cost-Prohibitive 4 Myth 4: Graphs Are Difficult to Generate 4 Myth 5: Making Information Available to Many Users Is Complex 4 Myth 6: Data Cannot Be Accessed in Real Time 5 Why IBM and Information Builders? 5 Data Loader for DB2 6 Data Reporter for DB2 7 Conclusion

3 Introduction One of the amazing things about technology is that older ideas often have a chance to evolve, not just simply fade away. One person s Cloud is another s server farm, and chances are that server farm comprises mainframe hardware IBM System z, for instance as well as overstuffed data warehouses. As data-warehouse appliances and larger in-memory data caches have secured a toehold in the ever-shifting market, it s important for IBM System z customers to realize that the data-warehouse infrastructure already exists. In the late 80s, IBM researchers Barry Devlin and Paul Murphy developed the business data warehouse concept. The term was coined in the 1988 IBM System Journal, Vol. 27, No. 1, when Devlin stated, A data warehouse is simply a single, complete, and consistent store of data obtained from a variety of sources and made available to users in a way they can understand and use in a business context. 1 However your business changes, one thing remains constant: inconsistent data translates into inconsistent actions. Enterprises rely on the data-warehouse and related business intelligence (BI) infrastructure to understand and react appropriately to change. Business executives who need analytical information at their fingertips turn to BI to transform data from the warehouse into actionable information. Several myths exist about typical data-warehousing environments, including: Data is cleansed and organized before it enters the warehouse Managers understand how data is collected, constructed, and reported Warehouses cannot scale to large data volumes or support a large, concurrent user count Warehouses cannot support mixed workloads Warehouses cannot support the Cloud or virtualization Warehouses are not dynamic Mainframes provide poor support of query and advanced analytics operations The biggest knock against data warehouses on System z is that they are typically expensive, unwieldy, inflexible, and require vast resources to maintain and the mainframe only adds exponentially to those costs. A data-warehouse appliance combines commodity processors and hardware with BI and data-warehouse software within a single box. Hardware and processors can be added as the warehouse grows, while data is segregated into yet another server in a profusion of distributed data stores. New islands of data are typically introduced using an appliance that represents nothing more than adding another server to the burgeoning farm population, thereby creating yet another copy of data to maintain, update, and control. A different approach is to integrate warehousing applications within the operational fabric of your business to ensure accurate, consistent, and timely access to information. Today, CIOs want the ability to streamline IT infrastructure and costs by consolidating data into a single view, returning it to a centralized, secure environment that supports the enterprise with current, consistent information. 1 Devlin, Barry, Data Warehouse: From Architecture to Implementation, Addison-Wesley, Information Builders

4 Information Builders and IBM mitigate the risks and data latency, while optimizing information delivery options from data warehouses based on System z mainframes. Organizations can access all their data to fuel BI operations, no matter where it is located, or what application it s in. Data can even be quickly and easily retrieved from any platform, including UNIX and Intel. Access to continuous real-time data remains crucial to a business s ability to function optimally. The payoff is substantial for enterprises that have continually invested in System z architecture, as System z remains the centralized information hub that can power Cloud, Software-as-a-Service (SaaS), or any other initiative. 2 BI and Data Warehousing on the Mainframe

5 Addressing the Myths There are many myths when it comes to data warehousing in mainframe environments. But IBM s continuous investment in System z and specialty engines like the IFL, ziip, and zaap enhances performance by enabling data warehouses to be dynamic and scalable. Myth 1: Capacity Planning Is a Challenge The appeal of the Cloud is dynamic allocation of storage when the data volume of the warehouse varies unpredictably, which traditionally makes capacity planning difficult. But off-loading processes to a ziip processor mutes this criticism by improving overall performance and response time, allowing the system to scale to larger volumes and manage a mixed workload. Myth 2: Data Warehouses Can t Fully Leverage Mainframes The problem is not that most data warehouses can t take advantage of the mainframe it s that they don t. For example, they require the purchase of commodity hardware and additional servers requiring more maintenance as you grow. But the ultra-high reliability and high-capacity workload of a mainframe can be achieved for less than it would cost to build a new environment on Unix or Windows. In fact, on System z, you can reduce off-loading and staging complexity by leveraging the ziip. Centralizing your data warehouse on the hardware where your data and applications reside reduces operating costs and improves performance and efficiency. It also allows you to scale your data warehouse to enterprise usage. Myth 3: Processing Is Cost-Prohibitive The truth is, simply transferring processing requirements to the ziip chip debunks this myth. In fact, up to 80 percent of processing requirements can be transferred to the ziip chip. Processing costs on a mainframe are always measured by the amount of MSUs (MIPS) used. With the ziip, the workload transferred to the chip is not counted in the size of the computer or the cost of processing in other words, it s free. Information Builders IBM s partner in the WebFOCUS for IBM System z Data Warehousing solution is one of the only companies in the world that makes software to intelligently switch processing between the general processor and the ziip chip in a z/os environment. Think of this chip as a BI accelerator. Once the I/O is taken care of, business intelligence must provide rich Internet formatting, calculations, analytics, and other CPU-intensive operations, all of which is off-loaded to the ziip processor. So, if you want to deploy a large customer-facing analytics application from your System z data warehouse with real-time data from the mainframe, you can do more with what you have. IBM offers significantly lower prices for a ziip engine. The net result is that you get a big discount on your MIPS usage and all the computer power you need without having to buy backup or peak-load capacity. 3 Information Builders

6 Myth 4: Graphs Are Difficult to Generate Another common computational requirement is the generation of graphs. A great deal of BI information is presented in graphs, and the WebFOCUS graph engine runs in Java. If a z/os site wants to process Java more economically, it can use the IBM zaap chip, which is another special-purpose chip that doesn t increase the computer s rating. It speeds up Java applications by moving the processing of the Java Virtual Machine (JVM) off the general processor to the zaap chip, thus reducing cost by the same amount. Myth 5: Making Information Available to Many Users Is Complex The IBM z/os mainframe is universally acclaimed for its ability to process large-scale transactions. Large-scale, custom-built, transaction-processing applications like data warehouses are the source of the massive amount of data that resides on the mainframe. There is a corresponding ability to build another class of application on the mainframe that is not oriented toward processing transactions, but rather toward processing information that is delivered via Web browser, in a rich visual context to large populations of internal and external users who have no end-user training. The business driver is the need for up-to-date information to enable people to make better decisions. For example, sharing information with customers can reduce errors, encourage brand loyalty, create entirely new ways of doing business, and lead to new revenue opportunities. Such information and reporting applications cannot use BI tools per se; rather, they are built upon BI technology. The advent of ziip technology allows these applications to be implemented directly on the mainframe more economically than moving data off the mainframe and building new environments. For example, the State of Florida Treasury built a self-service reporting system for more than 200,000 vendors. The entire state s vendor invoices and payments were housed in a central system run on the Treasury s z/os system. To get information, vendors used to call the department for answers to questions as simple as, Was my invoice received? Now vendors can find their own information and check account details. Errors are caught much more quickly and do not propagate in the system. Thirty state staff members are also free to do other work. Myth 6: Data Cannot Be Accessed in Real Time Another stumbling block to data warehousing and analytics is the perceived lack of real-time data access for reporting. Integration technology allows trickle-feeding of the data warehouse, which means new records are added right away. As soon as new data is entered into the operational system, it is extracted, transformed, and loaded into a real-time repository. For instance, if your customer is a telephone company with a data warehouse that includes information about customer accounts, invoices, service orders, products, support histories, and more, their call-center representatives will always have up-to-the-minute information about customer accounts and inquiries, and customers don t get passed from one division to another. 4 BI and Data Warehousing on the Mainframe

7 Why IBM and Information Builders? The mandate from management is that information needs to be available now, but a close eye must be kept on cost. The IT group must figure out how to make the information architecture less complex, using less hardware and resources to keep costs contained while mitigating the risk of failure. Complexities in appliances, and the segmentation of data and workloads, are the enemy of cost-effective operations because they increase costs and operational management while raising questions about performance. Data-warehousing appliances theoretically met the need for a solution with a faster time to market. However, while appliances have been more widely used, businesses are beginning to realize that using commodity hardware does come with tradeoffs, such as higher administration costs and reduced availability. There is an alternative for businesses invested in the System z mainframe platform. Improvements in technology, environmental impact, cost of ownership, and data-warehousing and BI software capabilities have made System z a highly desirable platform for data warehousing and BI. With WebFOCUS for System z Data Warehousing, businesses can align the advantages of the platform with the requirements of the business application. WebFOCUS for System z Data Warehousing is the industry s only single-environment solution, providing a comprehensive and fully integrated suite of data warehousing and business intelligence software products. It combines the power, flexibility, and reliability of IBM s System z Solution Edition for Data Warehousing with native z/os-based, easy-to-use tools that deliver data warehousing and BI with dramatically reduced implementation complexity and TCO, as compared to similar distributed systems or dual operating environment approaches. The solution comprises two components: DB2 Data Loader and DB2 Data Reporter. Data Loader for DB2 Data Loader for DB2 is a specialized version of iway Software s z/os-based DataMigrator product, designed for optimal loading and management of IBM DB2 database data synchronization processes. Data Loader for DB2 enables organizations to: Load/synchronize a DB2-based enterprise data warehouse through comprehensive access to more than 200 database, file, and application sources Finely tune metadata creation and management capabilities using common metadata introspection, modeling, and generation techniques, regardless of information resource, type, or location Optimize DB2 data-load capabilities on a real-time, near real-time, or batch basis Federate data in a highly sophisticated manner with drill-through capabilities that provide access to detailed operational data via a DB2 warehouse model, implement, manage, and audit warehouse loading 5 Information Builders

8 Data Reporter for DB2 Data Reporter for DB2 is a specialized version of Information Builders z/os-based WebFOCUS BI platform, designed for building Internet-based BI applications or as a guided ad hoc query tool for businesses using a DB2 enterprise data warehouse. A highly graphic developer workbench enables developers to quickly and easily design, deploy, and manage Internet-based applications or query solutions for large or small user groups that don t require end-user training or software licensing for each user. Data Reporter for DB2 offers: Graphically rich, fully integrated, and easy-to-use design tools for developing medium- to verylarge-scale BI applications End-user, guided ad hoc query capabilities to support any amount of Web-based usage Shared server capabilities that enable BI applications/users and IT to use a common server architecture for data access, manipulation, metadata creation/utilization, and data federation within z/os The scalable, reliable, and flexible server architecture within IBM system z technology such as the ziip specialty engine dramatically lowers the operational costs of using a DB2-based data warehouse. Optional service-based components for data cleansing and data quality management are available. 6 BI and Data Warehousing on the Mainframe

9 Conclusion The WebFOCUS Framework for Data Warehousing is a completely integrated software solution that uses a common server architecture and implementation tools. When combined with IBM s System z Solution Edition for Data Warehousing, it forms a complete and unified hardware and software solution for implementing enterprise data warehousing. Designed as a software solution to be deployed within a single, native z/os operating environment, WebFOCUS Framework for Data Warehousing uses a dedicated System z LPAR or standalone System z server to deliver a simple approach to the design, implementation, and management of a data-warehousing solution all at a lower acquisition cost and overall TCO compared to similar multi-tiered or environmental solutions. Building your data warehouse on IBM System z is the best way to consolidate and organize all your data so it can be easily managed, accessed, and analyzed. With a single management interface, as well as self-tuning and self-diagnostic features, DB2 simplifies maintenance of your ever-expanding data warehouse. IT likely will find significant benefits from having fewer specialized resources to manage System z, which centralizes the data warehouse and analytics where the data resides. This results in fewer instances of RDBMSs requiring tuning and optimization for queries and reports, eliminating many of the complex network connections that distributed environments use to communicate. Companies can reduce the number of IT personnel who only keep the lights on and redeploy the most skilled of them to respond to business needs for data assets. It is clear that WebFOCUS for IBM System z Data Warehousing offers significant potential benefits for organizations that need to store and manage rapidly growing volumes of data and provide that data for business intelligence and analytics to increasing numbers of users. As you analyze your current and future business needs, take these factors into account: Achieving TCO. Apply unified server architecture, data integration on IBM System z, Data Loader for DB2, and Data Reporter for DB2 Improved Performance. A package designed for analytics that keeps up with user demand and increased data Turnkey Solutions. Proven solutions to reduce time to launch and minimize required maintenance Scalability. Adding capacity is a simple matter of leveraging a ziip to off-load non-essential processes to another processor Respond to user demands for power, speed, and flexibility while answering executives calls for solutions that deliver performance for the lowest cost. Evaluate existing System z environments and current trajectory of costs versus benefits, analyze the options for supplementing or replacing them, and create a plan that can help them be more agile in turning data into decisions and ultimately value. 7 Information Builders

10 Worldwide Offices North America United States Atlanta,* GA (770) Baltimore, MD Professional Services: (703) Boston,* MA (781) Channels, (800) Chicago,* IL (630) Cincinnati,* OH (513) Dallas,* TX (972) Denver,* CO (303) Detroit,* MI (248) Federal Systems,* DC (703) Hartford, CT (860) Houston,* TX (713) Los Angeles,* CA (310) Minneapolis,* MN (651) New Jersey* Sales: (973) New York,* NY Sales: (212) Professional Services: (212) , ext Orlando,* FL (407) Philadelphia,* PA Sales: (610) Phoenix, AZ (480) Pittsburgh, PA Sales: (412) St. Louis,* MO (636) San Jose,* CA (408) Seattle, WA (206) Washington,* DC Sales: (703) Professional Services: (703) Canada Information Builders (Canada) Inc. Calgary (403) Montreal* (514) Ottawa (613) Toronto* (416) Vancouver (604) Mexico Information Builders Mexico Mexico City Australia Information Builders Pty. Ltd. Melbourne* Sydney* Europe Belgium* Information Builders Belgium Brussels France* Information Builders France S.A. Paris Germany Information Builders (Deutschland) Eschborn* Netherlands* Information Builders (Netherlands) B.V. Amsterdam Portugal Information Builders Portugal Lisbon Spain Information Builders Iberica S.A. Barcelona Bilbao Madrid* Switzerland Information Builders Switzerland AG Dietlikon United Kingdom* Information Builders (UK) Ltd. London Representatives Austria Raiffeisen Informatik Consulting GmbH Vienna Bahrain InfoBuild Middle East Dubai , ext. 312 Brazil InfoBuild Brazil Ltda. São Paulo China Information Builders China (IBC) Beijing Beijing Xinrong Software Technology Co., Ltd. Beijing Denmark InfoBuild AB Kista, SE Egypt InfoBuild Middle East Abu Dhabi Dubai Ethiopia MKTY IT Services Plc Addis Ababa Finland InfoBuild Oy Espoo Greece Applied Science Athens Guatemala IDS de Centroamerica Guatemala City India* InfoBuild India Chennai Israel SRL Group Ltd. Tel Aviv Italy NessPRO Italy S.p.A. Genoa Milan Turin Japan K.K. Ashisuto Osaka Tokyo Jordan InfoBuild Middle East Abu Dhabi Dubai Korea UVANSYS Seoul Kuwait InfoBuild Middle East Dubai Lebanon InfoBuild Middle East Dubai Norway InfoBuild Norway Oslo Oman InfoBuild Middle East Abu Dhabi Dubai Poland/Central and Eastern Europe InfoBuild SP.J. Warsaw Qatar InfoBuild Middle East Dubai Russian Federation FOBOS Plus Co., Ltd. Moscow Saudi Arabia InfoBuild Middle East Riyadh Singapore Automatic Identification Technology Ltd. Singapore South Africa InfoBuild South Africa (Pty.) Ltd. Gauteng Fujitsu Services (Pty.) Ltd. Johannesburg Taiwan Galaxy Software Services Taipei Thailand Datapro Computer Systems Co. Ltd. Bangkok , ext. 200 Turkey InfoBuild Middle East Ankara Istanbul United Arab Emirates InfoBuild Middle East Abu Dhabi Dubai Venezuela InfoServices Consulting Caracas Toll-Free Number Sales, ISV, VAR, and SI Partner Information (800) * Training facilities are located at these branches. Corporate Headquarters Two Penn Plaza, New York, NY (212) Fax (212) DN informationbuilders.com askinfo@informationbuilders.com Canadian Headquarters 150 York St., Suite 1000, Toronto, ON M5H 3S5 (416) Fax (416) For International Inquiries +1(212) Copyright 2010 by Information Builders. All rights reserved. [92] All products and product names mentioned in this publication are trademarks or registered trademarks of their respective companies. Java and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. Printed in the U.S.A. on recycled paper