Technology Innovations for Enhanced Database Management and Advanced BI

Transcription

1 Have 40 Technology Innovations for Enhanced Database Management and Advanced BI Claudia Imhoff, Intelligent Solutions, Inc. Colin White, BI Research May 2013 Sponsored by IBM

2 Table of Contents Executive Summary Introduction The What: Drivers That Help Both the Business and IT The Why: Improved Efficiency and Effectiveness Evolution of Database Management and Business Intelligence Customer Front-Office Operations Financial and Risk Management Industry-Specific Solutions The How: Technologies That Enable Innovation Historical Perspective Online Transaction Processing Systems Relational Database Management Systems Data Warehousing and BI Big Data The Two Main Categories of Technology Innovation RDBMS Directions New Sources of Data Business Ease of Use and Self Service Improved Performance Reduced IT Costs and Increased IT Flexibility Vendor Example: IBM Conclusions Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved.

3 Executive Summary Enterprises today are entering a new era of computing fueled by big data, advanced business intelligence, cloud computing, and mobile devices. These have produced significant technological innovation and disruption for existing IT architectures. IT leaders across all industries need to review and enhance their information architecture to support these requirements and consider how they can exploit the recent innovations in data management and business intelligence (BI) capabilities. Their assessment requires a deeper understanding in three critical areas the What, the Why and the How behind these innovations. The What: These describe the drivers for the technologies that help both business and IT acquire, analyze and understand all sources of information flowing into the enterprise today. The Why: These consist of the benefits derived from improved Business/IT efficiencies and effectiveness achievable through the utilization of technological advances. The How: These are the technological advances, derived from the What and Why factors, that enable the modern enterprise possible today. These three areas drive the rest of the paper. We include case studies to demonstrate the improved business efficiency and effectiveness achieved by re-architecting an IT environment, discuss four key areas of technological innovation having a major positive impact on IT (new sources of data, business ease of use and self-service, improved performance, and reduced IT costs and increased IT flexibility), and two categories of technological innovation (advanced BI and enhanced data management) that are driving the directions for relational database management system innovations today. These innovations are put into perspective by examining the key new online transaction processing (OLTP) and BI capabilities found in IBM s DB2 relational DBMS and other related IBM solutions. The innovations discussed give IT a number of different deployment options: The flexibility offered by software solutions that can be optimized by IT for either OLTP or BI processing. The convenience of appliances, pre-optimized for specific OLTP and BI workloads. The agility and fast deployment provided by a cloud-based computing approach. There is no single solution that meets all needs, and IT is likely to use all three options based on business and IT requirements. The vendors that provide all three options together with solid integration and interoperability capabilities between them are likely to be the winners given the increasing complexity of today s IT infrastructure. Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 1

4 Introduction Enterprises today are entering a new era of computing fueled by big data, advanced business intelligence, cloud computing, and mobile devices. These have produced significant technological innovation and disruption for existing IT architectures. IT leaders across all industries need to review and enhance their information architecture to support the requirements for this new era and to enable the many business opportunities it offers. A key component of their review is to consider how they can exploit the recent innovations in data management and business intelligence capabilities being offered by vendors today. Their assessment requires a deeper understanding in three critical areas the What, the Why and the How behind these innovations and their corresponding technological advances. The What: These describe the drivers for the technologies that help both business and IT acquire, analyze and understand all sources of information flowing into the enterprise today. The Why: These consist of the benefits derived from improved Business/IT efficiencies and effectiveness achievable through the utilization of technological advances. The How: These are the technological advances, derived from the What and Why factors, that enable the modern enterprise possible today. The What: Drivers That Help Both the Business and IT Obvious changes to the status quo of IT are new and unusual sources of data our architectures must deal with. Massive volumes of data are now available from machine sensors, social media feeds, RFID tags, and even text messages. These mostly multistructured data sources add tremendous value to the overall knowledge base for enterprises but present significant challenges in terms of acquiring, integrating, and analyzing them. The next driver for innovation comes from the business itself. The information workers today place ease of use for BI technology high on their list of requirements. They want to be able to create their own reports and analytics without IT intervention. This push for self-service BI does not mean self-sufficiency though a stable and robust BI architecture is still needed to ensure consistency, data security, and reliability. It does mean that BI technologies must be usable by a much broader audience many of whom may not be technologically savvy. This raises ease of use to a new level! Next is the demand for better performance. From a BI perspective, this comes from two directions. There is a strong push for operational intelligence analytics on more current, even real-time data requiring sub-second or a few seconds in response time. The second push comes from the increasingly complex analytics performed on higher volumes of data. Performance becomes a critical component of success or failure in modern IT architectures. Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 2

5 Of course, we cannot overlook the economic environment for today s enterprises. Reduced IT costs is a constant cry even in the face of these demands. New technologies must fit within the ever-shrinking IT budgets. You can t enhance and extend existing technologies with new ones if they break the budget. Fortunately we are seeing many innovations that are based on open-source systems, use commodity hardware, or combine multiple functions in the database to eliminate costs. Reduced technology costs are not the only concern for IT though. The last driver for innovation is IT s ease of use and flexibility. IT must consider the costs of installation, integration, deployment, administration and maintenance of these new technologies. Even if the technology is low or no cost, if it takes significant effort to install and manage it, it may still be challenging in terms of overall costs. The Why: Improved Efficiency and Effectiveness Evolution of Database Management and Business Intelligence Both online transaction processing (OLTP) and business intelligence (BI) systems are undergoing great changes. The OLTP systems that drive the operational business processes of our modern enterprises are experiencing massive increases in transaction volumes especially in the area of web traffic. BI systems are equally experiencing significant growth. BI has evolved from simple reporting functionality to very sophisticated, complex diagnostic, investigative and optimization capabilities. Like OLTP systems, the volume and types of data being handled by BI systems are also growing as organizations become increasingly interested in analyzing new sources of data such as web and social media data. We now have three distinct categories of BI: Descriptive BI: This tells us what has happened in our enterprise. It consists of standard and ad hoc reporting, comparative analytics (what happened yesterday, last week, last month, etc.) and query/drill down functionality. These are good at simple questions like how many products did we sell, from which stores, how often, and for what average price. Predictive BI: This category consists of the more sophisticated algorithms used in statistical, simulation, and forecasting analytics. They have been used for years to generate predictions of outcomes based on past trends and activities. It can answer questions about what will happen if we stay on this trajectory, what else could happen, and what actions are needed to change the trajectory. Prescriptive BI: This relatively new form of BI focuses on understanding how an enterprise can achieve the best outcome for an event while taking into consideration the effects of variability. It uses optimization and stochastic optimization techniques and algorithms. Enterprises today use it to help their employees perform the optimal activities given a particular situation (e.g., decision trees and guided decision making). Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 3

6 Changes in both OLTP and BI are leading to highly complex and mixed workloads, which are stretching the capabilities of IT systems. The challenge here is maintaining service level requirements in the face of this growth. Organizations therefore need to be thoughtful in their choice of hardware and software solutions if they are to leverage the business benefits offered by the many innovations in both data management and business intelligence. Given these changes, let s look at examples of how companies have improved business efficiency and effectiveness by re-architecting their IT environments and implementing innovative technologies today. Customer Front-Office Operations Customers are the lifeblood of every enterprise. Without them, no company would exist. Therefore it is understandable that many analytical efforts are focused on attracting the right customers, understanding their needs and wants, and preserving their loyalty and continued business. Here are a few examples of complex and highly developed customer analytics. A mobile communications company identifies in real-time the behavior of its subscribers so they can serve them better through targeted campaigns. They analyze customer usage data and trends, develop appropriate campaigns to reduce churn, then offer subscribers incentives of interest to them. The company keeps its higher usage customers satisfied by offering services and plans that are just right for them. To extend its reach, a large retail company launched a web initiative to offer a more localized, personalized and smarter retail customer experience nationwide. Customer s information is aggregated, analyzed and insights extracted from its web site and used to craft the customer web experience. For example, customer preferences (both implicit and explicit) are combined with recent purchase data to create customized recommendations on the fly (like a complementary clothing accessory or color). Because of the new retail paradigm to customer centricity, the company had to change the way it measures success from the traditional by-thenumbers metrics to longer-term ones of customer engagement, retention, and wallet share. Financial and Risk Management Given today s climate of regulatory compliance, economic stress, and strict security/privacy policies, many organizations have turned to complex algorithms and streaming real-time analytics to manage their financial and operational risk. The ability to detect fraudulent behavior faster, predict operational failures before they happen, or determine potential threats to the enterprise before they can occur is top of mind for many CEOs today. Financial and operational risk mitigation can take many forms; here are a few examples. One of South Africa s largest short-term insurance companies uses advanced analytics to automatically assess the risk of fraud and improve the time to settle legitimate claims. By building business rules based on many risk factors combined Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 4

7 with sophisticated predictive algorithms, the company is able to develop a reliable model of fraudulent behavior. Claims are segmented and scored according to their level of risk allowing the company to focus its energies on investigating the high-risk claims while rewarding good customers with fast settlements and better service. A utility company can now pinpoint with remarkable accuracy the damage from fallen trees and downed poles and lines predicted to occur from incoming storms. These predictions provide valuable lead-time for the company so it can be far more proactive in allocating people and equipment to the potentially stricken areas before the storm hits. The ultimate benefit is the ability to restore power as quickly and efficiently as possible in the aftermath of the storm. Industry-Specific Solutions Of course, BI is not limited to customer analytics or financial and risk management. Every aspect of an enterprise s business has analytic requirements. These include supply and demand chain management, operational processing efficiency, market research, geospatial analysis (store placement, logistics planning, location-based offers, etc.), call center optimization, and so on. Following are a few of these forms of analytics. A security company creates covert intelligence and surveillance sensor systems to provide organizations with advanced security solutions for infrastructures and extended borders. Obviously securing the scientific intelligence, technology and resources for governments and companies alike is vital to their survival. Their solutions capture and transmit real-time, streaming acoustical data from around the organization s premises, enabling security personnel to hear what is going on even when the disturbance is miles away. A potential security threat is identified quickly by collecting and integrating data from video and airborne surveillance systems and appropriate actions are taken. One of the largest North American freight transportation companies supplies its personnel with continuous access to data to confirm shipments are delivered on schedule, optimize delivery schedules and evaluate operational costs. Company dispatchers rely on daily shipment and scheduling data to determine when to increase velocity on the railroad to make sure shipments are delivered on time. Operations staff also uses this data to evaluate cost structures and performance so the company can stay competitive with other forms of transportation. The How: Technologies That Enable Innovation Historical Perspective Organizations have spent over fifty years automating their business processes and during that time there have been some key technology advances that have had a major positive impact on IT, which in turn has led to significant benefits to the business. Four of these innovations are directly relevant to this paper: Online transaction processing systems in the early 1960s Relational database management systems in the early 1980s Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 5

8 Data warehousing and business intelligence in the early 1990s Big data, the latest technology innovation to bring significant business benefits to organizations Online Transaction Processing Systems OLTP systems dramatically improved the ability of companies to automate business processes and react rapidly to changing operational business requirements. One of the first OLTP systems built was the Sabre airline reservation system, developed by American Airlines in conjunction with IBM. The initial Sabre system handled some 84,000 transactions per day, which at the time was considered to be an extreme transaction volume, since it stretched the limits of what was possible using available hardware and software at that time. High availability and fault tolerance were also critical to the success of the system. The concept of what is considered to be extreme or big has changed over the years as the introduction of point-of-sale terminals, bank tellers machines, mobile devices, the Internet, and sensor networks has led to more automation of business processes and corresponding higher transaction and data volumes. Sabre today, for example, now processes some 60,000 transactions per second. As a result both hardware and software have been improved and optimized to support these extreme transaction and data volumes and maintain required service levels. Relational Database Management Systems The introduction of relational database management systems (RDBMSs) gave IT developers and business users for the first time a standard data access language (SQL) and logical data structures (relational tables and views) that were independent of the way the data was physically stored, accessed and managed. This technology not only improved usability, but also enhanced portability and interoperability. Today, RDBMSs dominate the database marketplace for both OLTP and BI applications. Data Warehousing and BI As the number of data sources and the volume of transaction data grew so too did the problem of accessing this data for reporting and business decision making and planning. This issue led to the birth of the data warehouse where historical transaction data was consolidated into a single logical data store for use by BI reporting and analysis applications. This development radically improved the ability of users to analyze and optimize business operations and make more informed operational and strategic decisions. As in the past, the new data warehousing and BI technologies quickly stretched the limits of existing hardware and software capabilities as growing transaction and data volumes inevitably led to a corresponding increase in the size of data warehouses. Today, multiterabyte data warehouses are no longer the exception as they once were, and some companies are now managing several petabytes or more of data in their data warehousing systems. Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 6

9 To control data warehouse growth, and also to support the growing demand for close to real-time operational BI, some companies are now filtering and analyzing data in-motion as it flows through business processes. The results of this processing are then stored in a data warehousing system for downstream use. This analyze-and-store model of data management offers a useful addition to the store-and-analyze approach of traditional data warehousing. The term stream processing is often used to describe products that support the processing of in-motion data. Big Data The concept of big data evolved from solutions developed initially by web companies such as Google and Yahoo. These companies needed to manage and index huge volumes of web data and existing technologies were unable to support this in a timely or cost effective manner. To solve this problem, these web companies developed their own solutions. Most of these solutions were oriented towards IT programmers, rather than business users. The term NoSQL is often associated with these solutions because they do not use relational database technology. This paper, however, will use the term nonrelational instead since some of these systems do support a subset of SQL. Several of these non-relational systems have been donated to the open source community, which has led to the growing use of them in a wide range of industries. These systems enhance and extend the existing data warehousing environment by enabling many new types of data to be managed and analyzed, and offering improved price/performance for certain types of workloads. The advent of big data and increasing interest in blending new sources of data (web, social computing and sensor data, for example) into the business decision making process has also resulted in significant advances in relational database technology. Today s relational database products now support a broad range of different data types and have been enhanced to support growing data volumes. Many of them also provide a rich set of analytic capabilities that enhance traditional descriptive BI as well as providing more advanced predictive and prescriptive BI techniques. The four disruptive technologies outlined above have led to a rich set of capabilities for automating business processes and deploying data warehousing and BI solutions. The challenge of course is how to determine which technology to use for any given project. The next section looks at these technologies in more detail. The Two Main Categories of Technology Innovation In general, the innovations provided by current BI and OLTP solutions can be split into two main categories: advanced BI and enhanced data management. Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 7

10 Advanced BI Advanced BI is supported by the following technology capabilities: Enhanced analytic capabilities for improving existing descriptive, predictive and prescriptive BI. These capabilities are sometimes delivered in the form of an optimized analytic platform (see Figure 1) that can be used as an analytics accelerator, analytics sandbox for experimenting with data, or as a platform for new analyticsdriven line-of-business (LOB) applications. Such platforms may be optimized for analytic performance to lower IT costs or to enable new types of data to be analyzed. These platforms enhance and extend existing systems, making integration and interoperability with existing systems a key requirement. Operational intelligence solutions that support BI services embedded in OLTPdriven business processes (Figure 1). These services may provide data, analytics or analytic models to improve the effectiveness of operational business processes. New data discovery and visualization tools that make it easier to explore, analyze and visualize new types and sources of data. These tools may be provided as a component of an optimized analytic platform. Figure 1: Technologies for Enhancing and Extending Existing Systems Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 8

11 Enhanced Data Management Enhanced data management incorporates the following technologies (see Figure 1): RDBMSs that offer improved price/performance for both OLTP and BI analytic processing. In the case of BI, these systems may be employed to support a traditional BI and data warehousing environment, an optimized platform for analytic processing, or a cloud-based BI system. Non-relational systems such as Hadoop for handling new sources of data, especially multi-structured data such as web and social media data. These systems will typically coexist and interoperate with RDBMS products. One common use of Hadoop, for example, is as a data hub or staging area that brings together many different types and sources of data. Subsets of data in the hub can then distributed to underlying systems, such as a data warehousing system, as required. Stream processing systems for filtering and analyzing in-motion data in real time. The results of this processing may be delivered to OLTP-driven business processes, business users, or to downstream data warehousing and BI systems. In-memory computing that exploits large main-memory spaces to gain high performance. This style of computing may be used by RDBMSs, non-relational systems, streaming systems, or by analytic BI tools. Although there are many new data management technologies appearing in the marketplace, relational database technology still forms the core of both OLTP and BI processing. The rest of this section therefore focuses primarily on new RDBMS innovations. It will, however, review how RDBMSs are being enhanced to interoperate with both non-relational and stream processing systems. RDBMS Directions The most appropriate way of reviewing technology innovations in RDBMSs is in terms of the four business and IT benefits outlined earlier: new sources of data, business ease of use and self-service, improved performance, and reduced IT costs and increased IT flexibility. The first two benefits apply primarily to the BI environment, whereas the latter two benefits apply to both the OLTP and BI environments. New Sources of Data The main objective here is to extract useful business information from new types and sources of data for use by existing BI applications and for building new and advanced BI solutions. This can be done by either converting the data into a more usable format before loading into a data warehouse, or directly loading the data into a data warehouse and processing it there. To support the first approach, vendors are enhancing data integration products to access data from new data sources and to transform it into a structured format. RDBMS vendors, on the other hand, are extending their products to provide connectivity to new data Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 9

12 sources, and to directly support the storing, management and manipulation of the data in an RDBMS. Supporting more complex data types in an RDBMS is not new. Many RDBMSs, for example, support text and XML data, and also multi-media data. The pace at which new data types are being added to RDBMSs, however, is accelerating. Factors that distinguish products are their ability to efficiently store, manage manipulate these new types of data. Options to consider include data compression, SQL extensions and functions to manipulate the data, and enhancements to the relational optimizer and data manager to intelligently handle the data to provide good performance. Business Ease of Use and Self Service The four main objectives of ease of use and self-service are to: Make it easy to access data. The ability to handle new sources of data applies equally here because it enables business users to use the RDBMS to blend new data with existing data. Make a data warehouse fast to deploy and easy to manage. One of the key technology directions in this area is the ability to quickly deploy a solution that allows more experienced power users, business analysts and data scientists to experiment with data outside of the production or enterprise data warehouse to look for ways of improving existing BI applications and building new ones. This solution has various names including analytics sandbox (as shown in Figure 1), investigative computing platform, and data discovery platform. Such a solution is particularly useful for experimenting with a combination of existing and new data prior to possibly integrating the new data into a production data warehouse. In some cases deploying this solution in a cloud environment can provide faster time to value. Cloud deployment would require the underlying RDBMS to support a cloud-operating environment and to provide interoperability between the cloud environment and onpremises systems. Make BI tools easier to use (easy BI). There are a number of developments here. A key one is the availability of analytics function libraries that make it easier for business users to use more advanced analytic capabilities including the ability to process and analyze new types of data. These libraries eliminate the need for users to know how to develop and code these functions they do of course still have to know how to use them and interpret the results. From an RDBMS perspective, the ability to have these functions run inside the DBMS and exploit features such as parallel and in-memory computing can provide a significant performance boost. Make BI results easy to consume (easy UI). Again there are many options here including improved visualization, BI automation such as alerts and guided analysis, office product integration, and collaborative and mobile interfaces. Most of these are independent of the underlying RDBMS, with the exception perhaps of mobile computing where support for mobile data formats such as JavaScript Object Notation (JSON) improves ease of use for IT developers. Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 10

13 Improved Performance Performance has always been one of the main focus areas of RDBMS enhancements. With today s OLTP and BI performance requirements, the RDBMS must be to handle both the data and workloads required to support business needs. Performance has many dimensions. In terms of data management, the dimensions are the amount of data to be managed (data volume), its rate of generation or change (data velocity), the types of data to be managed (data variety), and the number of data sources, structures and relationships involved (data complexity). In the case of workloads, the dimensions are the types and complexity of the application processing (workload complexity), data currency and response time requirements (workload agility) and the makeup of the overall workload (workload mix). The product and technology selected to support any given project will be determined by the actual performance requirements in any of these data and workload dimensions. The higher the requirement in any dimension, the more important it will be to choose an RDBMS that can be optimized to provide good performance in that dimension. This is illustrated in Figure 2. Figure 2. The Many Variables of Performance To stretch the boundaries of RDBMS performance across the many dimensions shown in Figure 2, vendors are enhancing their products in three key areas: New data structures. Examples here include support for columnar as well as rowbased data stores within the same RDBMS, data stores structured to suit different varieties of data, and enhanced data compression techniques. Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 11

14 Hardware exploitation. Hardware is constantly improving in price/performance and it is important that RDBMS products exploit hardware performance improvements such as large hardware clusters, multicore processors, new processor capabilities such as SIMD (single instruction, multiple data), large memory spaces for in-memory processing, and hybrid data storage from high-performance solid-state drives to highcapacity but slower disk drives. DBMS extensions. Technologies here include in-database analytic functions (as discussed earlier), an intelligent query optimizer that understands and can exploit new data structures and hardware exploitation features provided by the DBMS, and an intelligent workload manager that can efficiently handle mixed and complex workloads. Reduced IT Costs and Increased IT Flexibility The overall objective in this area is to increase the ROI of IT investments in OLTP and BI. This objective can be split into two requirements: Reducing the cost of purchasing, installing, maintaining and administering IT systems. Many companies focus primarily on the initial costs of purchasing the hardware and software for any given project and neglect to take into account the ongoing costs of maintaining and administering the system. This accounts for the growing interest in non-relational systems such as Hadoop, which typically employ open source software and low-cost commodity or white-box hardware. These latter systems, however, are more resource intensive to maintain and administer and this must be taken into account when selecting any given solution. Relational database vendors, on the other hand, have invested heavily in improving the software to automate installation, maintenance and administration, while at the same time providing high availability and security with robust features for disaster recovery. Both relational and non-relational system providers are also moving in direction of providing packaged hardware and software platforms (or appliances) that are optimized for certain types of workload. This approach can significantly reduce the effort involved in deploying systems. Many of these providers are also offering cloud-based solutions that enable organizations to rapidly deploy projects without the need for significant up-front investment in on-premises hardware and software. Improving the speed and flexibility of applications development. The task of improving the usability of IT and business user development tools is an ongoing process throughout the IT industry. These improvements coupled with optimized packaged appliance and cloud-computing deployment options have significantly improved the speed of applications development. Application flexibility, however, is also an important requirement. Ideally an application should be able to connect to any data source and deliver results to any target business process or user. Data integration (including data virtualization techniques) and system interoperability are therefore key requirements as are support for new and evolving technologies such as cloud and mobile computing. Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 12

15 Vendor Example: IBM IBM has always been a leader in both OLTP and BI innovation and its support of big data and advanced BI continues this trend. IBM has made significant investments in developing solutions that support the information architecture shown in Figure 1. At the heart of IBM s data management product set is the IBM DB2 relational DBMS. The latest release of the product is version 10.5, which incorporates many innovative enhancements for both OLTP and BI processing. There is insufficient room here to describe all of the new DB2 features in detail. Instead we will limit discussion to a review of the capabilities that support the new technologies discussed in this paper. This discussion will be segmented into the four main business and IT drivers and benefits outlined earlier. New sources of data. IBM s strategy here is to both extend DB2 and provide connectivity to other products such as IBM InfoSphere BigInsights. In the case of DB2, in addition to existing support for XML data, IBM is working on JSON support, which is available as a database technology preview program. Business ease of use and self-service. As discussed above, requirements here are largely independent of the underlying DBMS. It is important to note, however, that IBM is focusing in this release on the use of DB2 in both the mobile and cloudcomputing environments, and on packaged appliances that are pre-optimized for specific OLTP and BI workloads. Improved performance. The performance features in DB are grouped under the label BLU Acceleration. Capabilities include in-memory columnar processing that speeds up analytical processing, compression techniques that allow data to be processed without the need for decompression, the ability to skip unnecessary processing of irrelevant data, and parallel processing improvements including exploitation of multi-core processors and processor SIMD instructions. These new capabilities not only improve OLTP and BI performance, but also reduce data storage requirements. Reduced IT costs and increased IT flexibility. The new release of DB2 contains a variety of different improvements here. Key enhancements include high-availability and disaster recovery for OLTP environments, enhanced on-line administration and maintenance, integration and automation of BLU Acceleration capabilities, simplified and automated workload management, and Oracle RDBMS compatibility enhancements. It is also interesting to note that DB2 supports both shared-disk and shared-nothing hardware architectures, which makes it easier to optimize for either an OLTP or a BI environment. IBM DB2 is packaged into a series of IBM PureData hardware and software appliances that are optimized for specific workloads. The PureData System for Transactions, for example, is an integrated, ready-to-run database appliance designed and tuned explicitly for OLTP workloads. IBM also offers the PureData System for Operational Analytics, which is optimized for operational BI. Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 13

16 Several other IBM products complement the capabilities of IBM DB2. These include: IBM InfoSphere BigInsights is a non-relational and Apache Hadoop-based solution for managing and analyzing massive volumes of structured and multi-structured data. The latest release of this product includes an SQL interface and an enhanced file system. BigInsights is also the cornerstone of IBM s PureData System for Hadoop packaged hardware and software appliance. IBM InfoSphere Streams is a stream processing system for filtering and analyzing large volumes of in-motion data. IBM Cognos and SPSS software is used to build descriptive, predictive and prescriptive BI applications. When combined, these products support the key components of the information architecture shown in Figure 1. Conclusions Organizations now have a wide range of OLTP and BI solutions and options available to them. This rich set of choices can bring significant business benefits, but also makes the task of platform and production selection more complex for the IT organization. As with any IT projects, the starting point is determining needs of the business, and then evaluating how any given technology satisfies those needs. Balancing business benefits against the IT costs of achieving those benefits is also a key consideration. As explained in this paper there are four areas that need to be considered when evaluating the benefits of new and innovative technologies: access to new types and sources of data, business ease of use and self service, improved performance, and reduced IT costs and increased IT flexibility. These technology innovations can be used to enhance existing application systems or to deploy new ones. Enhancing an existing system is typically more difficult that deploying a new one, since existing technologies may limit the improvements that can be made. Regardless of whether new technologies are used to improve existing systems or to build new ones, it is quite clear that a one size fits all approach to technology selection is not viable given the complexity of today s business needs and growing transaction and data volumes. Instead organizations will need to deploy technologies based on a range of different, and often mutually exclusive, business and IT requirements. A flexible and integrated information architecture similar to the one in Figure 1 is therefore required. As discussed, IBM provides a product suite that supports such an information architecture with IBM DB2 acting as the cornerstone of this solution. IT also has a number of different deployment options. Application systems can be deployed on-premises or in a cloud-based operating environment. On-premises systems can be optimized by IT, or purchased as pre-optimized hardware and software appliances. The choice therefore is between: Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 14

17 The flexibility offered by software solutions that can be optimized by IT for either OLTP or BI processing. The convenience of appliances, pre-optimized for specific OLTP and BI workloads. The agility and fast deployment provided by a cloud-based computing approach. There is no single solution that meets all needs, and IT is likely to use all three options based on business and IT requirements. The vendors that provide all three options together with solid integration and interoperability capabilities between them are likely to be the winners given the increasing complexity of today s IT infrastructure. Big data coupled with advanced BI and significant improvements to RDBMS technology dramatically improves IT s ability to deploy both OLTP and BI solutions that enhance and extend existing systems. Careful selection and deployment of these technologies can bring significant benefits to the business. Organizations should therefore review and improve their information architectures so that they are in a position to take advantage of this new generation of business process and BI innovations. About BI Research BI Research is a research and consulting company whose goal is to help organizations understand and exploit new developments in business intelligence, data integration, and data management. About Intelligent Solutions, Inc. Intelligent Solutions is a research and professional services firm, founded in 1992 by Dr. Claudia Imhoff, dedicated to assessing, planning, and guiding business intelligence (BI) through its services. Copyright 2013 BI Research and Intelligent Solutions, Inc., All Rights Reserved. 15