A TECHNICAL WHITE PAPER ATTUNITY VISIBILITY

Transcription

1 A TECHNICAL WHITE PAPER ATTUNITY VISIBILITY Analytics for Enterprise Data Warehouse Management and Optimization

2 Executive Summary Successful enterprise data management is an important initiative for growing organizations across the globe. However, due to the complexity of the IT environment and growing demands from business users, it is often difficult for IT teams to deliver high performance improvements in a timely manner at a reasonable cost. IT professionals need solutions that enable them to shine a light on enterprise data usage analytics. This kind of insight can provide actionable information on which data is used throughout the enterprise, by which departments, and how often, so that guesswork is removed from the decision about which data should be placed where, and what department should be charged back for its use. Attunity Visibility provides the best in class data usage analytics for enterprise data warehouse optimization and management. Using Visibility, organizations can get unprecedented insight into Business Activity, Data Usage and Workload Performance across different platforms (Including Teradata, Exadata, DB2, Pure Data and Hadoop). Enterprise customers use Attunity Visibility in large data warehousing environments to reduce costs, improve performance and justify investments. Attunity Visibility provides the level of robust intelligence required to run an Enterprise Data Warehousing Environment effectively and efficiently Attunity LTD WHITE PAPER: Attunity Visibility: Analytics for Data Warehouse Management and Optimization 2

3 Table of Contents Executive Summary: Enterprise Data Mgmt Market Overview & Challenges... 2 Attunity Visibility: Shining a Light on Enterprise Data Usage Analytics... 2 Use Cases/How Attunity Visibility Helps Companies... 5 Gain insight into business activity... 5 Analyze Data Usage... 5 Assess and Manage workload performance... 5 Rebalance data and workloads to Hadoop... 6 Justify costs and prioritize resource investment... 6 Retain and optimize relevant data & processes...6 Respond faster and ensure scalability and performance... 6 Attunity Visibility s Key Features... 7 Leveraging Attunity Visibility for Enterprise Data Analytics Use Cases... 9 Conclusion About Attunity Attunity LTD WHITE PAPER: Attunity Visibility: Analytics for Data Warehouse Management and Optimization 3

4 Executive Summary Enterprise Data Management Market Overview and Challenges Data is the lifeblood of businesses today. However, the magnitude and velocity of data growth are exploding and show no signs of stopping. Enterprises are struggling to manage it efficiently, especially when delivery of real-time data is essential to meet the needs of the business. Line of business owners want the right information at the right place and at the right time, in order to extract value from it as quickly as possible. Today s environment presents several challenges to IT teams:: - Data must be managed from many diverse sources, including ERP, CRM, POS, legacy systems, logs, sensor files, and more. - IT teams must manage multiple technology platforms, such as Teradata, Hadoop, Oracle, DB2, SAP, and cloud just to name a few. This makes it difficult to manage data efficiently and optimize the environment. - The cost and complexity of managing information are increasing rapidly. - While numerous business stakeholders can access data and systems, IT has limited visibility into what data is being used in the organization, by whom, how often, and its overall effect on system performance. - Without the ability charge back or show data and system utilization by business line, it s hard for IT to prioritize and justify additional investments. - Even though data is growing exponentially, IT budgets are staying flat. It s not possible to invest in infrastructure at a rate needed to keep pace with data growth. Most BI and data warehousing inefficiencies arise from a lack of understanding about how applications and data are used across organizations. Legacy application monitoring tools are designed for tracking transactional applications, rather than revealing how the business uses analytical data. Ideally, IT teams should take a proactive, flexible, and forward-thinking approach to system performance, but it s impossible to do that without visibility into how the business uses data. It all goes back to the old adage: you can t manage what you don t measure. Attunity Visibility: Shining a Light on Enterprise Data Usage Analytics Attunity Visibility enables true enterprise data usage analytics by giving organizations unprecedented insight into Big Data systems. It is the only solution that provides detailed, granular visibility into business activity, data usage, and workload performance across heterogeneous data platforms. With this information, IT teams can reduce costs and optimize performance Attunity LTD WHITE PAPER: Attunity Visibility: Analytics for Data Warehouse Management and Optimization 4

5 Attunity Visibility helps organizations: 1. Gain insight into business activity. To analyze workloads across data platforms to deliver increased operational efficiency and reduce costs. The screenshot above highlights dentification of intensive ETL workloads. 2. Analyze Data Usage: Identify unused data that can be archived rather than consuming valuable EDW capacity and loading cycles. Unused data can be retired, while active business data can be managed using data retention policies that ensure compliance. 3. Assess and Manage workload performance: Easily identify individual application users and BI reports that impact system performance on the data warehouse. Attunity Visibility s integrated dashboard enables IT teams to diagnose and assess performance bottlenecks. Almost 60% of CPU is used to load and ingest data The screenshot above highlights dentification of intensive ETL workloads Attunity LTD WHITE PAPER: Attunity Visibility: Analytics for Data Warehouse Management and Optimization 5

6 4. Rebalance data and workloads to Hadoop: Enabling enterprises to adopt and scale out Hadoop intelligently and strategically in a cross-platform data warehousing environment. This screenshot breaks down the level of usage by servers, databases and tables. The result is that companies improve system performance and put the right workloads on the right systems, across data warehouses, business intelligence applications, and Hadoop. The business usage monitoring provided by Attunity Visibility helps IT teams: - Justify costs and prioritize resource investment based on business utilization. Organizations can assess and identify underutilized IT assets, along with highly utilized but underperforming assets. This information reduces operational costs and supports efficient planning for future capacity and resources. - Retain and optimize the most relevant data and processes. With better insight into how data is being used, IT services can be better aligned with the business while reducing the costs of storage and data management. - Respond faster and ensure scalability and performance. Attunity Visibility gives CIOs and multi-functional IT teams end-to-end visibility into systems and user activity. With Attunity Visibility, IT teams can address exploding data growth with confidence, proactively manage BI and data warehouse performance, and realize economies through the use of Hadoop and the Cloud Attunity LTD WHITE PAPER: Attunity Visibility: Analytics for Data Warehouse Management and Optimization 6

7 Attunity Visibility s Key Features Attunity Visibility facilitates performance and cost optimizations by identifying unused data and intensive ETL processing that can be moved to Hadoop. This enables organizations to achieve better utilization across their EDW and Hadoop platforms. Key features include: - Analysis of all application activity & data usage in the context of business. By assessing ETL and analytical workloads that consume excessive resources, Attunity Visibility helps IT teams discover workloads that are promising candidates for offloading. - Data history and usage. Attunity Visibility constructs a history of data used from large tables, based on an analysis of calendar dates used in queries. Application activity and data usage by business units can also be analyzed, which supports chargeback and justification of investments. The screenshot above shows the historical use of data in a large Fact table Attunity LTD WHITE PAPER: Attunity Visibility: Analytics for Data Warehouse Management and Optimization 7

8 Attunity Visibility s Key Features (cont.) - Integration with leading business intelligence and analytic applications. This enables Attunity Visibility to correlate application users and report activity with usage. The screenshot above shows the timing of activity correllating with application usage. - Intuitive, integrated dashboard. This user interface diagnoses and assesses performance bottlenecks, providing IT teams with the means to optimize multiple data delivery platforms. Organizations that use Attunity Visibility appreciate the fact that they can rebalance data and workloads to the right platform by identifying unused data and resource intensive workloads. In addition, IT teams can measure and audit utilization of information assets. Auditing can be automated for data governance and user activity related to sensitive tables and columns can be tracked. Storage and performance can also be optimized, since data can be tiered based on its frequency of use Attunity LTD WHITE PAPER: Attunity Visibility: Analytics for Data Warehouse Management and Optimization 8

9 Leveraging Attunity Visibility for Enterprise Data Analytics Use Cases Organizations in diverse industries have implemented Attunity Visibility to address their most pressing enterprise data management use cases. Here are a few examples: - Improving Data Warehouse Performance. A pharmaceutical manufacturer had a massive data warehouse with over one million queries per day from more than 3,000 users. Performance was a problem, as were regulatory and audit risks. The company implemented Attunity Visibility to generate BI and data warehouse business activity and data usage analytics. The system delivered correlated, granular insight into user activity, application data, and database information without disruption of production warehouse systems. With this information, the IT team was able to reduce costs, improve delivery on service level agreements, and ensure trusted use of information. Insight into data usage enabled the organization to reduce data warehouse storage and make changes that reduced ETL load times by 75%. These actions resulted in cost savings and improved performance. In addition, user activity data generated from Attunity Visibility led to a reduction in the number of user IDs. - Identifying workload performance issues: A large personal computer manufacturing and consumer services firm uses Visibility to manage a large data warehouse environment that supports over 10,000 users accessing over 300TB of data. The application DBA s responsible for deploying applications and managing the data on the data warehouse were spending a lot of time writing custom queries against logs to get information users and applications running problematic workloads. They found that the existing tools provided insight into workload performance issues from the database system perspective which is useful for the operational DBAs, but was not useful for the Application DBAs. With Visibility, they are saving 30-50% of the time they used to spend diagnosing the sources (Users and Applications) related to performance issues. They are able to point-click-and-drill to get answers about the users and applications in minutes anytime they have performance issues. In addition the historical trending and data usage analysis provides the Application DBAs the ability to tune the environment based on how data is being used and impacting resource consumption. - Auditing Data Warehouse Infrastructure to Mitigate Regulatory Risk. A financial services firm needed a way to continuously and proactively audit its large BI and data warehouse environment to mitigate regulatory risk. Attunity Visibility has given the security and compliance team access to detailed user information. In addition, teams can set notifications and track data access violations through Attunity Visibility. - Identifying and offload unused data: A large financial institution s card services data warehouse, was nearly at capacity. They were expecting their data growth to grow by over 100TB in one year. They estimated that in just 18 months they would have had to spend over $15 million, adding new capacity. Instead they began using Visibility to identify unused data to show the statistics to the business on a monthly basis. By having the ability to collaborate with the business with factual information on data usage, they started to eliminate loading data that was not needed, and archive data that was not required and also offloaded data into Hadoop. In addition, every month they identifying frequently and infrequently used data that is moved to Tier 1 and Tier 2 storage for optimal data placement. Since customer data is exposed on the data warehouse, they use Visibility to generate weekly reports on which users are accessing sensitive customer data. These reports are sent to the data owners in the business as part of their process to meet risks and compliance requirements Attunity LTD WHITE PAPER: Attunity Visibility: Analytics for Data Warehouse Management and Optimization 9

10 Conclusion If enterprise data management & optimization is top of mind for your organization, it may be time to take a closer look at Attunity Visibility. With Attunity Visibility, it s possible to take a proactive and flexible approach to IT architecture and performance. With insight into how the business uses data, you will have the metrics and information needed to manage systems in intelligent, cost-effective ways. Simply put, having access to data usage analytics is key to minimizing the risk of data becoming a liability and maximizing the inherent value of it. About Attunity Attunity is a leading provider of data management software solutions that enable access, management, sharing and distribution of data, including Big Data, across heterogeneous enterprise platforms, organizations, and the cloud. Our software solutions include data replication, data flow management, test data management, change data capture (CDC), data connectivity, enterprise file replication (EFR), managed file transfer (MFT), data warehouse automation, data usage analytics, and cloud data delivery. Attunity has supplied innovative software solutions to its enterprise-class customers for over 20 years and has successful deployments at thousands of organizations worldwide. Attunity provides software directly and indirectly through a number of partners such as Microsoft, Oracle, IBM and HP. Headquartered in Boston, Attunity serves its customers via offices in North America, Europe, and Asia Pacific and through a network of local partners. For more information, visit or our In Tune blog and join our community on Twitter, Facebook, LinkedIn and YouTube Attunity LTD WHITE PAPER: Attunity Visibility: Analytics for Data Warehouse Management and Optimization 10

11 Over 2,000 companies including half of the Fortune 500 trust Attunity solutions. Attunity exclusively focuses on engineering high-performance data management solutions that are fast to deploy and easy to operate, empowering enterprises to simply and cost-effectively ensure business-critical information is well managed and accessible when, where and how it s needed to become a more agile, intelligent enterprise. Contact Us North America T: sales@attunity.com Europe, Middle East & Africa T: +44 (0) info-uk@attunity.com Asia Pacific T: + (852) info-hk@attunity.com Connect With Us blog.attunity.com You Tube 2015 Attunity LTD. All rights reserved.