Big Insights from Little Data: A Spotlight on Unlocking Insights from the Log Data That Matters

Size: px
Start display at page:

Download "Big Insights from Little Data: A Spotlight on Unlocking Insights from the Log Data That Matters"

Transcription

1 Big Insights from Little Data: A Spotlight on Unlocking Insights from the Log Data That Matters

2 Introduction Logentries is a leading SaaS provider for collecting, analyzing and managing machine- generated log data. Logentries processes billions of log events every day from users across 100 countries. In this report, researchers at Logentries show how they have analysed a sample of this data over a 14 day period and highlight how the challenge of dealing with big data can be especially difficult when a user is looking to identify specific events in large volumes of data. The research team analyzed over 22 billion events produced across a 14 day period by 6000 Heroku apps, which make up a sample of the Logentries user base. The team analyzed these events from the perspective of a DevOps professional, someone responsible for building and running cloud applications. They determined that a significant amount of the most useful and actionable data for a given use case goes unnoticed, because it is a vanishingly small percentage of the overall dataset. Why Log Data Matters DevOps professionals are commonly tasked with building, deploying, managing and maintaining applications, both in the cloud and on premise. To help ensure their applications are performing properly, DevOps often rely on log data for troubleshooting, diagnostics and application systems monitoring. However, they are challenged by the large amount of log data that an application generates during the normal course of business. Even a relatively small application can generate millions of log events per day. The challenge for DevOps is to not only manage increasing volumes of log data, but is also to quickly and efficiently sift through the millions or even billions of log events collected to find the individual log entries containing the important information they re looking for. In many instances, this information includes the application errors, exceptions, warnings and other critical log events related to application and system performance needed to monitor and improve performance or reliability. By reviewing and taking action based on these types of events, DevOps can quickly react to potential issues in real time, preventing outages by proactively addressing potential issues before they adversely impact an application. This report highlights the challenge facing DevOps teams today. The report finds, after looking across 6000 applications producing over 22 billion Heroku log events, that on average 99.82% of log events are noise (for the given use case), with 0.18% of log events containing critical information for DevOps concerned with application performance and reliability (i.e. application/platform errors, warnings or exceptions). Given that 0.18% of log data contains valuable information for this particular end user, it demonstrates that while big data receives most of the

3 attention in the media, it s actually the little data that counts and which often provides the big insights. The report also shows that this issue is significantly exasperated as systems grow in size, with only 0.02% (a 50 th of 1%) of log data containing valuable information for this DevOps use case (i.e % noise), and where the little data that matters, becomes a smaller percentage of overall log volumes. The report also gives a more fine grained analysis of the breakdown of this data and gives insight into the characteristics of the log data produced by Heroku apps of varying sizes. The Modern Needle in the Big Data Haystack Over a 14 day period Logentries analyzed log data from a sample of over 6,000 Heroku applications, providing a view into typical log volumes, error rates and performance characteristics. For the purposes of the analysis, the Heroku applications were split into two groups, moderately sized applications and large applications. Moderately sized applications were defined as those that generated less than 2GB of log data per day. Larger applications were defined as those that generated more than 2GB of log data per day. Relevant events from a DevOps perspective were defined as entries that contained any Heroku Error code or application exceptions that occurred in an application % 0.031% % 0.181% 0.010% 0.052% Signal Noise Fatal Events Cri\cal Events Warnings Excep\ons

4 Within the 0.181% of events containing relevant information for DevOps as defined above, 0.088% were warnings, 0.052% were critical events, 0.01% were fatal and 0.031% were application exceptions. Excep\ons 17% Fatal Events 5% Cri\cal Events 29% Warnings 49% Fatal Events Cri\cal Events Warnings Excep\ons Looking at the total amount of relevant information found as a whole, 49% of the total were warnings, 29% were critical events, 5% were fatal and 17% were application exceptions. Sample 1: Moderately Sized Heroku Applications Moderately sized Heroku applications were defined as those applications that generated less than 2 GB of log data per day. Based on this definition, our sampling included 11.4 billion log events. Within this sample, 99.67% of the events sampled were signal noise, burying the 0.33% of the log events that a DevOps person would likely be most interested in.

5 Log Event Breakdown, Moderate Heroku App % % % % Signal Noise Warnings Cri\cal Events Fatal Events Excep\ons Within the 0.33% events, 49% were warnings, 27% were warnings, 6% were fatal and 18% were application exceptions. Moderate Heroku App - Error Events Breakdown 18% 6% 49% 27% Warnings Cri\cal Events Fatal Events Excep\ons

6 Sample 2: Larger Heroku Applications Larger Heroku applications were defined as those applications that generated more than 2 GB of log data per day. Based on this definition, our sampling included approximately 11 million log events. Within this sample, 99.98% of the events sampled were noise, burying the 0.02% of the log events that a DevOps person would likely be most interested in. Log Event Breakdown, Large Heroku App 0.006% 0.001% % 0.017% 0.000% 0.010% Signal Noise Fatal Events Cri\cal Events Warnings Excep\ons Within the 0.02% events, 35% were warnings, 59% were critical events, 1% were fatal and 5% were exceptions. Large Heroku App - Error Events Breakdown Excep\ons 5% Fatal Events 1% Warnings 35% Cri\cal Events 59% Fatal Events Cri\cal Events Warnings Excep\ons

7 Examining the Needle Relevant events were defined with input from Heroku Engineers and were grouped based on Heroku error codes. Note, HTTP errors started with the letter H, runtime errors started with R and logging errors started with L. The individual events were categorized as follows: Warnings: Warning error codes relate to less severe error codes but also refused connections and timeouts that can often ultimately lead to critical or fatal errors. They include H20 App boot timeout, R13 Attach Error, H21 Backend connection refused, H19 Backend connection timeout, H22 Connection limit reached, R16 Detached, L10 Drain buffer overflow, R12 Exit timeout, H80 Maintenance Mode, H17 Poorly Formatted HTTP response, H16 Redirect to Herokuapp.com, H18 Request Interrupted, and L11 Tail buffer overflow. Critical events: Critical events were error codes related to degraded performance and possibly some dropped requests. They included H12 Request Timeout, R14 Memory Quota Exceeded, H15 Idle Connection, H13 Connection closed without response, and H11 Backlog too deep. Fatal: Fatal events were error codes related to application or dyno crashing - i.e. your user request will not be served. Note: H99 represents an error on the Heroku platform. Unlike all of the other errors which will require action from you to correct, this one does not require action from you. The included H10 App Crashed, R10 Boot timeout, H99 Platform Error, and R15 Memory Quota Vastly Exceeded. Exceptions: Included all application exceptions and relate to exceptions generated by individual applications. Digging into the data a little more shows how averages can sometimes be misleading. Many applications did not have events from any of the above categories. In fact, looking at a single day s worth of log data for all 6000 applications, the total number of applications producing at least one event from each event category were as follows: Warnings: 3357, Critical Events: 1415, Fatal: 462, Application Exceptions: Warning Cri\cal Fatal Excep\on

8 Warnings were the most prevalent category and appeared in over 50% of the applications. This is not surprising, since warnings can often be safely ignored and can be left unaddressed by developers as they build their apps. Critical events, fatal events and application exceptions appeared more sporadically across smaller numbers of apps, and are in general likely a result of a more severe application error. The volume of events in these categories also tend to increase as application log volumes grew as systems under load usually produce more exceptions when something goes awry. Large spikes in error events in your logs can often mean something serious is happening in your system. While undesirable, it s usually pretty noticeable if you have a major outage. Log management technologies can be used to assess, diagnose and resolve the issue. Intermittent issues and anomalies in your log data, however, can be much more difficult to track down and can often go unnoticed for long periods, resulting in a loss of business and undesirable user experiences. While log management solutions have largely focused on search capabilities, it is not possible to search for something you are unaware of. The anomalies in the little data are the small percentage of key insights that you really need to know about. Typical examples of such issues from a DevOps perspective include: A small percentage of request response times are outside an acceptable range. For example <1% of requests are taking more than 3 seconds. This could easily go unnoticed during test but will hurt your bottom line if a section of your user base is going elsewhere due to a poor user experience. A fraction of users receiving request timeouts (i.e. site looks like it is down). Similar to the issue above if a small section of your user base is having problems with your service, you may never find out about it and your system may be failing silently. Ongoing warnings that go unaddressed may lead to more severe problems. A typical example is a memory leak, which continues to leak until you get a dreaded out of memory error resulting in a much more serious outage. An example outside of DevOps scenario can include a change in the rate of signups, or in any KPI for that matter. From our analysis above, if on average less than 0.18% of log data contains useful information, only a fraction of this 0.18% relates to those hard to find issues. Thus finding these nuggets of information within billions of events can prove extremely challenging without the right tools. Unfortunately, most log management tools to date have focused on providing powerful search capabilities, which does not greatly help to solve the issue of finding the anomalies in your data.

9 Searching assumes you know what you looking for. However for intermittent issues or anomalies in your data you need different capabilities to easily identify such problems. To find the little data that has the big impact our strong belief is that building a log management solution solely around searching is fundamentally the wrong approach: rather, it is our belief that capabilities such as visualization, pre- analysis of data and the ability to plug in intelligence for different use cases are key to identifying the little data that matters. Heroku Heroku (pronounced her- OH- koo) is a cloud, platform as a service (PaaS) that enables application developers to build applications without having to deploy, operate or scale the underlying hardware of software. Supporting Ruby, Node.js, Clojure, Java, Python and Scala, Heroku developers can choose between 1x and 2x dynos to scale their applications based on growth and scale needs. Apps can be deployed in US and European geographic regions. Heroku was founded in 2007 and is owned by Salesforce.com. DevOps DevOps is the blending of tasks performed by a company's application development and systems operations teams. Traditional application development teams are in charge of gathering business requirements and building the application. The development team tests the program in a development environment for quality assurance (QA) and once the appropriate testing is completed, the code is released to operations for use. The traditional operations team is tasked with deploying and maintaining the software, and ensuring the application is available for end- users and runs smoothly. The challenge with the traditional model where application development teams and the operations team are separate, the development team may not be aware of operational roadblocks that prevent the program from working as anticipated. And the operations team may not be aware of how the application was designed, architected and built. DevOps is both a philosophy that promotes better communication, collaboration, and integration between the two teams, and also an emerging and critical role where individuals possess the skills and operate as both a developer

10 and a systems operations engineer. DevOps seeks to help organizations more rapidly produce applications and update systems to meet changing market and customer needs. As an example, in many software companies today software development teams deploy multiple software updates per today to their operational systems. Conclusions & Recommendations After taking a sample of our entire user base and analyzing 6000 Heroku applications which produced over 22 billion log events across a 14 day period, the research team at Logentries have given a real world view into how the challenge of big data is not simply scaling to consume and manage it, but rather is having the ability to easily and quickly find critical information needed to take action. Time to insight is a key driver in how quickly DevOps can react to system issues and prevent major outages. In this case, Logentries highlighted the role of the DevOps professional in building and running cloud based applications, and ensuring those cloud applications are meeting user needs. By sampling its Heroku community and the roughly 22 billion log events generated by it over a 2 week period, Logentries illustrated the unique challenges posed to DevOps in being able to consume and process large quantities of machine generated data. More specifically, the data collected showed for individuals looking to maintain application health and performance that on average across 6000 heroku applications 99.82% of log data is noise. In this case, it s the 0.18% of log events that contain the critical application health and performance information needed by DevOps - the cloud application equivalent of a needle in the log data haystack. For DevOps building and managing Heroku applications, we recommend: Log your Heroku events to a log management tool and use that tool to track and diagnose application health and performance. If you aren t using a solution today, we d obviously recommend Logentries as it uses a real- time analytics engine and a collective intelligence model to automatically pre- process, index and tag Heroku log events in real- time, and provide users with a view in to their Heroku logs out of the box. Logentries automatically identifies Heroku error codes and categorizes them into the groups outlined in this report (warnings, critical events, fatal and application exceptions) and provides a view into these events through the Logentries Heroku Dashboard which helps you identify anomalies in your data. Logentires doesn t require any programming, advanced search queries or costly setup to start using. Review your specific Heroku applications and the error codes being reported in your event logs. Use these error codes to investigate potential issues and the underlying sources to determine changes that should be made.

11 Evaluate whether your Heroku applications are performing in line with acceptable norms and compare them against these report averages. You can also evaluate your application against either the moderately sized application generated sampling statistics or the larger sized application generated sampling statistics. If an application is higher that the averages reported we recommend you bring your application within Heroku norms. Setup real- time monitoring and alerting of your Heroku log stream so you can be proactively notified if an error occurs. You can set the frequency and duration for receiving notifications of errors logged.

12 About the Authors Trevor Parsons, PhD Co- Founder, Chief Scientist Trevor Parsons is Chief Scientist and Co- founder of Logentries. Trevor has over 10 years experience in enterprise software and in particular has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland. Benoit Guadin, PhD Sr. System Architect Benoit obtained his Ph.D at the University of Rennes in France in 2004, on controlling discrete event systems. After holding a teaching position for a year in France and a postodoctoral position at the Fraunhoffer institute in Berlin, Benoit came to Ireland in He was a postdoctoral fellow at University College Dublin and then a research fellow at the University of Limerick. Finally Benoit joined Logentries in October 2012, bringing the expertise he previously gathered in topics such as Formal Methods, Control Theory, Autonomic Systems, Dynamic Analysis, Automatic System Modeling and Ontologies. About Logentries Logentries is a SaaS offering for collecting and analyzing huge quantities of machine- generated log data and making that data easily accessible to individual developers, small teams, and enterprise customers. While traditional log management solutions require advanced technical skills to use or are costly to setup, Logentries provides a simply accessible alternative. With Logentries your log files are filtered, pre- processed and correlated up front for quicker and easier retrieval of the individual log entries that matter most. In turn, this pre- processing is combined with a collective insights model that enables important information to be dynamically tagged, automatically routed, and easily shared across teams and computing platforms. Logentries eliminates the need for an in- house data expert or

13 specialist to interpret and use the data. We are easy to setup and free to use, with flexible pricing options available. Resources Try Logentries, it s free - start/ Build a Heroku application today Check out an overview of the Heroku error codes Concerned your Heroku app is too slow, checking our blog post, How do I know if my Heroku app is slow, (H12 and H11 errors) Don t let your Heroku apps fail silently? Checkout another classic Logentries blog post on Heroku error monitoring - Additional information on Heroku errors can be found at the following: H12, request time out errors: timeout H70, Access to Banboo HTTP endpoint denied errors: domains#custom- subdomains

Logentries Insights: The State of Log Management & Analytics for AWS

Logentries Insights: The State of Log Management & Analytics for AWS Logentries Insights: The State of Log Management & Analytics for AWS Trevor Parsons Ph.D Co-founder & Chief Scientist Logentries 1 1. Introduction The Log Management industry was traditionally driven by

More information

Augmented Search for Software Testing

Augmented Search for Software Testing Augmented Search for Software Testing For Testers, Developers, and QA Managers New frontier in big log data analysis and application intelligence Business white paper May 2015 During software testing cycles,

More information

Web applications today are part of every IT operation within an organization.

Web applications today are part of every IT operation within an organization. 1 Introduction Web applications today are part of every IT operation within an organization. Independent software vendors (ISV) as well as enterprises create web applications to support their customers,

More information

Perform-Tools. Powering your performance

Perform-Tools. Powering your performance Perform-Tools Powering your performance Perform-Tools With Perform-Tools, optimizing Microsoft Dynamics products on a SQL Server platform never was this easy. They are a fully tested and supported set

More information

Buying vs. Building Business Analytics. A decision resource for technology and product teams

Buying vs. Building Business Analytics. A decision resource for technology and product teams Buying vs. Building Business Analytics A decision resource for technology and product teams Introduction Providing analytics functionality to your end users can create a number of benefits. Actionable

More information

effective performance monitoring in SAP environments

effective performance monitoring in SAP environments WHITE PAPER September 2012 effective performance monitoring in SAP environments Key challenges and how CA Nimsoft Monitor helps address them agility made possible table of contents executive summary 3

More information

z/os Log Analysis Product Shoot-Out: CorreLog, Syncsort/Splunk and IBM Session 17442 IBM Log Analysis

z/os Log Analysis Product Shoot-Out: CorreLog, Syncsort/Splunk and IBM Session 17442 IBM Log Analysis z/os Log Analysis Product Shoot-Out: CorreLog, Syncsort/Splunk and IBM Session 17442 IBM Log Analysis Paul Smith (Smitty) (paulmsm@us.ibm.com) IBM z Systems Service Management / zanalytics Architect Anuja

More information

Augmented Search for IT Data Analytics. New frontier in big log data analysis and application intelligence

Augmented Search for IT Data Analytics. New frontier in big log data analysis and application intelligence Augmented Search for IT Data Analytics New frontier in big log data analysis and application intelligence Business white paper May 2015 IT data is a general name to log data, IT metrics, application data,

More information

VMware vcenter Log Insight Delivers Immediate Value to IT Operations. The Value of VMware vcenter Log Insight : The Customer Perspective

VMware vcenter Log Insight Delivers Immediate Value to IT Operations. The Value of VMware vcenter Log Insight : The Customer Perspective VMware vcenter Log Insight Delivers Immediate Value to IT Operations VMware vcenter Log Insight VMware vcenter Log Insight delivers a powerful real-time log management for VMware environments, with machine

More information

Frequently Asked Questions Plus What s New for CA Application Performance Management 9.7

Frequently Asked Questions Plus What s New for CA Application Performance Management 9.7 Frequently Asked Questions Plus What s New for CA Application Performance Management 9.7 CA Technologies is announcing the General Availability (GA) of CA Application Performance Management (CA APM) 9.7

More information

PUSH INTELLIGENCE. Bridging the Last Mile to Business Intelligence & Big Data. 2013 Copyright Metric Insights, Inc.

PUSH INTELLIGENCE. Bridging the Last Mile to Business Intelligence & Big Data. 2013 Copyright Metric Insights, Inc. PUSH INTELLIGENCE Bridging the Last Mile to Business Intelligence & Big Data 2013 Copyright Metric Insights, Inc. INTRODUCTION... 3 CHALLENGES WITH BI... 4 The Dashboard Dilemma... 4 Architectural Limitations

More information

APPLICATION PERFORMANCE MONITORING

APPLICATION PERFORMANCE MONITORING APPLICATION PERFORMANCE MONITORING PRACTICAL WAYS TO MONITOR THE END USER EXPERIENCE WHITE PAPER Performance of key applications is a critical item to monitor in many IT environments where users depend

More information

Performance Testing. on Production System

Performance Testing. on Production System Performance Testing on Production System Abstract Performance testing is conducted to check whether the target application will be able to meet the real users expectations in the production environment

More information

A BUSINESS CASE FOR BEHAVIORAL ANALYTICS. White Paper

A BUSINESS CASE FOR BEHAVIORAL ANALYTICS. White Paper A BUSINESS CASE FOR BEHAVIORAL ANALYTICS White Paper Introduction What is Behavioral 1 In a world in which web applications and websites are becoming ever more diverse and complicated, running them effectively

More information

Best Practices for Monitoring: Reduce Outages and Downtime. Develop an effective monitoring strategy with the right metrics, processes and alerts.

Best Practices for Monitoring: Reduce Outages and Downtime. Develop an effective monitoring strategy with the right metrics, processes and alerts. Best Practices for Monitoring: Reduce Outages and Downtime. Develop an effective monitoring strategy with the right metrics, processes and alerts. 1 TABLE OF CONTENTS INTRODUCTION WHAT TO MONITOR ENSURING

More information

Capacity planning with Microsoft System Center

Capacity planning with Microsoft System Center Capacity planning with Microsoft System Center Mike Resseler Veeam Product Strategy Specialist, MVP, Microsoft Certified IT Professional, MCSA, MCTS, MCP Modern Data Protection Built for Virtualization

More information

Licensing for BarTender s Automation Editions

Licensing for BarTender s Automation Editions The World's Leading Software for Label, Barcode, RFID & Card Printing White Paper Licensing for BarTender s Automation Editions Understanding Printer-Based Licensing and How to Configure Seagull License

More information

Modern IT Operations Management. Why a New Approach is Required, and How Boundary Delivers

Modern IT Operations Management. Why a New Approach is Required, and How Boundary Delivers Modern IT Operations Management Why a New Approach is Required, and How Boundary Delivers TABLE OF CONTENTS EXECUTIVE SUMMARY 3 INTRODUCTION: CHANGING NATURE OF IT 3 WHY TRADITIONAL APPROACHES ARE FAILING

More information

Conducting a Successful Cloudmarket CIO

Conducting a Successful Cloudmarket CIO As companies emerge from challenging economic times and turn the corner to face tremendous opportunities, the CIO's role is more and more about strategy and optimizing business results. Today s complex

More information

Augmented Search for Web Applications. New frontier in big log data analysis and application intelligence

Augmented Search for Web Applications. New frontier in big log data analysis and application intelligence Augmented Search for Web Applications New frontier in big log data analysis and application intelligence Business white paper May 2015 Web applications are the most common business applications today.

More information

Aurora365 White Paper. Establishing a Network Operations Centre (NOC) Culture with Aurora365

Aurora365 White Paper. Establishing a Network Operations Centre (NOC) Culture with Aurora365 with Aurora365 1. Introduction While organisations recognise the potential of running their own Network Operations Centre (NOC), it s often assumed that USS Enterprise style control centres are only deployed

More information

BMC ProactiveNet Performance Management Application Diagnostics

BMC ProactiveNet Performance Management Application Diagnostics BMC ProactiveNet Performance Management Application Diagnostics BMC, BMC Software, and the BMC Software logo are the exclusive properties of BMC Software, Inc., are registered with the U.S. Patent and

More information

Command Center Handbook

Command Center Handbook Command Center Handbook P r o a c t i v IT e Monitoring Protecting Business Value Through Operational Excellence Abdul A Jaludi Copyright 2014 Abdul A Jaludi abby@tag-mc.net www.tag-mc.net All rights reserved.

More information

Resource Monitoring During Performance Testing. Experience Report by Johann du Plessis. Introduction. Planning for Monitoring

Resource Monitoring During Performance Testing. Experience Report by Johann du Plessis. Introduction. Planning for Monitoring Resource Monitoring During Performance Testing Experience Report by Johann du Plessis Introduction During a recent review of performance testing projects I completed over the past 8 years, one of the goals

More information

Six Steps to Improved SharePoint Performance & Operations

Six Steps to Improved SharePoint Performance & Operations Accelerating SharePoint Performance The guys at Application Performance quickly understood our SharePoint problem. They demonstrated that the combination of WebTuna and Riverbed SteelApp Web Accelerator

More information

Six Signs. you are ready for BI WHITE PAPER

Six Signs. you are ready for BI WHITE PAPER Six Signs you are ready for BI WHITE PAPER LET S TAKE A LOOK AT THE WAY YOU MIGHT BE MONITORING AND MEASURING YOUR COMPANY About the auther You re managing information from a number of different data sources.

More information

ROCANA WHITEPAPER How to Investigate an Infrastructure Performance Problem

ROCANA WHITEPAPER How to Investigate an Infrastructure Performance Problem ROCANA WHITEPAPER How to Investigate an Infrastructure Performance Problem INTRODUCTION As IT infrastructure has grown more complex, IT administrators and operators have struggled to retain control. Gone

More information

Monitoring Best Practices for

Monitoring Best Practices for Monitoring Best Practices for OVERVIEW Providing the right level and depth of monitoring is key to ensuring the effective operation of IT systems. This is especially true for ecommerce systems like Magento,

More information

Disrupting The Market: Predictive Analytics As A Service

Disrupting The Market: Predictive Analytics As A Service Disrupting The Market: Predictive Analytics As A Service 0 Problem 8.7 Billion Connected Devices 1 Growing 25% Annually What Does This Data Tell Us About Sensor Use? 1 Study conducted by Cisco 1 Solution

More information

White Paper. The Ten Features Your Web Application Monitoring Software Must Have. Executive Summary

White Paper. The Ten Features Your Web Application Monitoring Software Must Have. Executive Summary White Paper The Ten Features Your Web Application Monitoring Software Must Have Executive Summary It s hard to find an important business application that doesn t have a web-based version available and

More information

IBM API Management Overview. 2014 IBM Corporation

IBM API Management Overview. 2014 IBM Corporation IBM API Management Overview Please Note IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM s sole discretion. Information regarding potential

More information

7 Must-Haves for Application Performance Management. SlashGuide - July 2013

7 Must-Haves for Application Performance Management. SlashGuide - July 2013 7 Must-Haves for Application Performance Management SlashGuide - July 2013 7 Must-Haves for Application Performance Management 2 Picture these all-too-familiar application fails: A checkout transaction

More information

PLA 7 WAYS TO USE LOG DATA FOR PROACTIVE PERFORMANCE MONITORING. [ WhitePaper ]

PLA 7 WAYS TO USE LOG DATA FOR PROACTIVE PERFORMANCE MONITORING. [ WhitePaper ] [ WhitePaper ] PLA 7 WAYS TO USE LOG DATA FOR PROACTIVE PERFORMANCE MONITORING. Over the past decade, the value of log data for monitoring and diagnosing complex networks has become increasingly obvious.

More information

Web Analytics Understand your web visitors without web logs or page tags and keep all your data inside your firewall.

Web Analytics Understand your web visitors without web logs or page tags and keep all your data inside your firewall. Web Analytics Understand your web visitors without web logs or page tags and keep all your data inside your firewall. 5401 Butler Street, Suite 200 Pittsburgh, PA 15201 +1 (412) 408 3167 www.metronomelabs.com

More information

BizTalk Server Monitoring Top 15 Best Practices

BizTalk Server Monitoring Top 15 Best Practices BizTalk Server Monitoring Top 15 Best Practices Why do you need to worry about monitoring? The simplest example you can relate to when it comes to monitoring is your car dashboard. A car dashboard reveals

More information

PLUMgrid Toolbox: Tools to Install, Operate and Monitor Your Virtual Network Infrastructure

PLUMgrid Toolbox: Tools to Install, Operate and Monitor Your Virtual Network Infrastructure Toolbox: Tools to Install, Operate and Monitor Your Virtual Network Infrastructure Introduction The concept of Virtual Networking Infrastructure (VNI) is disrupting the networking space and is enabling

More information

Work Smarter, Not Harder: Leveraging IT Analytics to Simplify Operations and Improve the Customer Experience

Work Smarter, Not Harder: Leveraging IT Analytics to Simplify Operations and Improve the Customer Experience Work Smarter, Not Harder: Leveraging IT Analytics to Simplify Operations and Improve the Customer Experience Data Drives IT Intelligence We live in a world driven by software and applications. And, the

More information

SOLUTION OVERVIEW SmartELM - Smart Error & Log Management

SOLUTION OVERVIEW SmartELM - Smart Error & Log Management SOLUTION OVERVIEW SmartELM - Smart Error & Log Management Troubleshooting apps should be this easy Troubleshooting an application requires having as much information about the error as possible. Errors

More information

Licensing for BarTender s Automation Editions. Understanding Printer-Based Licensing WHITE PAPER

Licensing for BarTender s Automation Editions. Understanding Printer-Based Licensing WHITE PAPER Licensing for BarTender s Automation Editions Understanding Printer-Based Licensing and How to Configure Seagull License Server WHITE PAPER Contents Introduction to Printer-Based Licensing 3 Available

More information

Site24x7: Powerful, Agile, Cost-Effective IT Management from the Cloud. Ensuring Optimal Performance and Quality Web Experiences

Site24x7: Powerful, Agile, Cost-Effective IT Management from the Cloud. Ensuring Optimal Performance and Quality Web Experiences Site24x7: Powerful, Agile, Cost-Effective IT Management from the Cloud Ensuring Optimal Performance and Quality Web Experiences Must-know facts about Site24x7: We bring expertise gained from ManageEngine

More information

A New Foundation For Customer Management

A New Foundation For Customer Management The Customer Data Platform: A New Foundation For Customer Management 730 Yale Avenue Swarthmore, PA 19081 info@raabassociatesinc.com The Marketing Technology Treadmill Marketing automation. Inbound marketing.

More information

Monitoring the Real End User Experience

Monitoring the Real End User Experience An AppDynamics Business White Paper HOW MUCH REVENUE DOES IT GENERATE? Monitoring the Real End User Experience Web application performance is fundamentally associated in the mind of the end user; with

More information

Creating Service Relevance for M2M Data

Creating Service Relevance for M2M Data Creating Service Relevance for M2M Data Jon Upton October 2014 M2M Summit jupton@salesforce.com Safe Harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation

More information

WHITE PAPER. Five Steps to Better Application Monitoring and Troubleshooting

WHITE PAPER. Five Steps to Better Application Monitoring and Troubleshooting WHITE PAPER Five Steps to Better Application Monitoring and Troubleshooting There is no doubt that application monitoring and troubleshooting will evolve with the shift to modern applications. The only

More information

Monitoring can be as simple as waiting

Monitoring can be as simple as waiting Proactive monitoring for dynamic virtualized environments By David Weber and Veronique Delarue Virtualization can significantly increase monitoring complexity. By using BMC ProactiveNet Performance Management,

More information

HOLISTIC PERFORMANCE MANAGEMENT: A FAST-GROWING NECESSITY IN APJ

HOLISTIC PERFORMANCE MANAGEMENT: A FAST-GROWING NECESSITY IN APJ HOLISTIC PERFORMANCE MANAGEMENT: A FAST-GROWING NECESSITY IN APJ IMPROVING THE END-USER EXPERIENCE DRIVES COMPETITIVE ADVANTAGE IN ASIA-PACIFIC/ JAPAN As business in nearly every industry becomes increasingly

More information

How Route Analytics Enables Virtualization and Cloud Computing

How Route Analytics Enables Virtualization and Cloud Computing How Route Analytics Enables Virtualization and Cloud Computing By Jim Metzler, Ashton Metzler & Associates Distinguished Research Fellow and Co-Founder, Webtorials Editorial/Analyst Division Introduction

More information

OPTIMIZING PERFORMANCE IN AMAZON EC2 INTRODUCTION: LEVERAGING THE PUBLIC CLOUD OPPORTUNITY WITH AMAZON EC2. www.boundary.com

OPTIMIZING PERFORMANCE IN AMAZON EC2 INTRODUCTION: LEVERAGING THE PUBLIC CLOUD OPPORTUNITY WITH AMAZON EC2. www.boundary.com OPTIMIZING PERFORMANCE IN AMAZON EC2 While the business decision to migrate to Amazon public cloud services can be an easy one, tracking and managing performance in these environments isn t so clear cut.

More information

Splunk for VMware Virtualization. Marco Bizzantino marco.bizzantino@kiratech.it Vmug - 05/10/2011

Splunk for VMware Virtualization. Marco Bizzantino marco.bizzantino@kiratech.it Vmug - 05/10/2011 Splunk for VMware Virtualization Marco Bizzantino marco.bizzantino@kiratech.it Vmug - 05/10/2011 Collect, index, organize, correlate to gain visibility to all IT data Using Splunk you can identify problems,

More information

BIG DATA THE NEW OPPORTUNITY

BIG DATA THE NEW OPPORTUNITY Feature Biswajit Mohapatra is an IBM Certified Consultant and a global integrated delivery leader for IBM s AMS business application modernization (BAM) practice. He is IBM India s competency head for

More information

LOG INTELLIGENCE FOR SECURITY AND COMPLIANCE

LOG INTELLIGENCE FOR SECURITY AND COMPLIANCE PRODUCT BRIEF uugiven today s environment of sophisticated security threats, big data security intelligence solutions and regulatory compliance demands, the need for a log intelligence solution has become

More information

State of SIEM Challenges, Myths & technology Landscape 4/21/2013 1

State of SIEM Challenges, Myths & technology Landscape 4/21/2013 1 State of SIEM Challenges, Myths & technology Landscape 4/21/2013 1 Introduction What s in a name? SIEM? SEM? SIM? Technology Drivers Challenges & Technology Overview Deciding what s right for you Worst

More information

Making big data simple with Databricks

Making big data simple with Databricks Making big data simple with Databricks We are Databricks, the company behind Spark Founded by the creators of Apache Spark in 2013 Data 75% Share of Spark code contributed by Databricks in 2014 Value Created

More information

Monitoring and Log Management in Hybrid Cloud Environments

Monitoring and Log Management in Hybrid Cloud Environments Ingo Averdunk, Dipl.-Inform. November 11, 2015 IT Service Management Monitoring and Log Management in Hybrid Cloud Environments Agenda Overview Hybrid Service Management Monitoring Log Management Closing

More information

GeoInt 2015 Watson Workshop

GeoInt 2015 Watson Workshop GeoInt 2015 Watson Workshop Bluemix Building a Watson Question & Answer Service Hands-on Lab The lab is divided into three parts Part A: Getting started what you need and what you will be building Estimated

More information

Monitoring applications in multitier environment. Uroš Majcen uros@quest-slo.com. A New View on Application Management. www.quest.

Monitoring applications in multitier environment. Uroš Majcen uros@quest-slo.com. A New View on Application Management. www.quest. A New View on Application Management www.quest.com/newview Monitoring applications in multitier environment Uroš Majcen uros@quest-slo.com 2008 Quest Software, Inc. ALL RIGHTS RESERVED. Management Challenges

More information

How To Use The Dcml Framework

How To Use The Dcml Framework DCML Framework Use Cases Introduction Use Case 1: Monitoring Newly Provisioned Servers Use Case 2: Ensuring Accurate Asset Inventory Across Multiple Management Systems Use Case 3: Providing Standard Application

More information

Application Performance Management for Enterprise Applications

Application Performance Management for Enterprise Applications Application Performance Management for Enterprise Applications White Paper from ManageEngine Web: Email: appmanager-support@manageengine.com Table of Contents 1. Introduction 2. Types of applications used

More information

PATROL From a Database Administrator s Perspective

PATROL From a Database Administrator s Perspective PATROL From a Database Administrator s Perspective September 28, 2001 Author: Cindy Bean Senior Software Consultant BMC Software, Inc. 3/4/02 2 Table of Contents Introduction 5 Database Administrator Tasks

More information

Personal Business Analytics. The Top 5 Reasons You Need to Understand Your Customer s Customer

Personal Business Analytics. The Top 5 Reasons You Need to Understand Your Customer s Customer The Top 5 Reasons You Need to Understand Your Customer s Customer Personal Business Analytics 1 Table of Contents 3 4 6 9 12 14 19 Introduction Renew Your Focus Get Out in Front Create More Targeted Marketing

More information

How To Develop An Application

How To Develop An Application What is Application Lifecycle Management? David Chappell Sponsored by Microsoft Corporation Copyright 2014 Chappell & Associates Defining application lifecycle management (ALM) isn t easy. Different people

More information

Virtual Desktop Infrastructure Optimization with SysTrack Monitoring Tools and Login VSI Testing Tools

Virtual Desktop Infrastructure Optimization with SysTrack Monitoring Tools and Login VSI Testing Tools A Software White Paper December 2013 Virtual Desktop Infrastructure Optimization with SysTrack Monitoring Tools and Login VSI Testing Tools A Joint White Paper from Login VSI and Software 2 Virtual Desktop

More information

Adopting Site Quality Management to Optimize Risk-Based Monitoring

Adopting Site Quality Management to Optimize Risk-Based Monitoring Adopting Site Quality Management to Optimize Risk-Based Monitoring Medidata and other marks used herein are trademarks of Medidata Solutions, Inc. All other trademarks are the property of their respective

More information

Francois Ajenstat, Tableau Stephanie McReynolds, Aster Data Steve e Wooledge, Aster Data

Francois Ajenstat, Tableau Stephanie McReynolds, Aster Data Steve e Wooledge, Aster Data Deep Data Exploration: Find Patterns in Your Data Faster & Easier Curt Monash, Founder and President, Monash Research Francois Ajenstat, Tableau Stephanie McReynolds, Aster Data Steve e Wooledge, Aster

More information

APPLICATION MANAGEMENT: RACING AHEAD OF THE COMPETITION WHITE PAPER

APPLICATION MANAGEMENT: RACING AHEAD OF THE COMPETITION WHITE PAPER APPLICATION MANAGEMENT: RACING AHEAD OF THE COMPETITION WHITE PAPER Imagine that you re driving a Formula One car down a track at the Grand Prix. The black-and-white checkered flag is up ahead, signaling

More information

Optimizing Oracle Forms System Performance Through Real-Time Client-Side Monitoring. October, 2011

Optimizing Oracle Forms System Performance Through Real-Time Client-Side Monitoring. October, 2011 Optimizing Oracle Forms System Performance Through Real-Time Client-Side Monitoring October, 2011 All rights reserved. OraPlayer 2011 1 Executive Summary Efficient utilization of computing resources is

More information

A Ranger4 Guide to. Application Performance Management. www.ranger4.com Ranger4 2014 1

A Ranger4 Guide to. Application Performance Management. www.ranger4.com Ranger4 2014 1 A Ranger4 Guide to Application Performance Management www.ranger4.com Ranger4 2014 1 Contents 1.0 What is Application Performance Management? 1.1 APM and DevOps 2.0 Why should you do it? 3.0 What you should

More information

Meeting the Challenge of Big Data Log Management: Sumo Logic s Real-Time Forensics and Push Analytics

Meeting the Challenge of Big Data Log Management: Sumo Logic s Real-Time Forensics and Push Analytics Meeting the Challenge of Big Data Log Management: Sumo Logic s Real-Time Forensics and Push Analytics A Sumo Logic White Paper Executive Summary The huge volume of log data generated by today s enterprises

More information

Assignment # 1 (Cloud Computing Security)

Assignment # 1 (Cloud Computing Security) Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual

More information

We are Big Data A Sonian Whitepaper

We are Big Data A Sonian Whitepaper EXECUTIVE SUMMARY Big Data is not an uncommon term in the technology industry anymore. It s of big interest to many leading IT providers and archiving companies. But what is Big Data? While many have formed

More information

Monitoring Best Practices for

Monitoring Best Practices for Monitoring Best Practices for OVERVIEW Providing the right level and depth of monitoring is key to ensuring the effective operation of IT systems. This is especially true for ecommerce systems like Magento,

More information

Geeks with...sql Monitor

Geeks with...sql Monitor Case Study Geeks with...sql Monitor How Geekswithblogs.net uses SQL Monitor to look after their servers and keep users around the world happy. Introducing Geekswithblogs.net Problems faced How SQL Monitor

More information

Streaming Analytics and the Internet of Things: Transportation and Logistics

Streaming Analytics and the Internet of Things: Transportation and Logistics Streaming Analytics and the Internet of Things: Transportation and Logistics FOOD WASTE AND THE IoT According to the Food and Agriculture Organization of the United Nations, every year about a third of

More information

T he complete guide to SaaS metrics

T he complete guide to SaaS metrics T he complete guide to SaaS metrics What are the must have metrics each SaaS company should measure? And how to calculate them? World s Simplest Analytics Tool INDEX Introduction 4-5 Acquisition Dashboard

More information

Intel IT s Cloud Journey. Speaker: [speaker name], Intel IT

Intel IT s Cloud Journey. Speaker: [speaker name], Intel IT Intel IT s Cloud Journey Speaker: [speaker name], Intel IT Accelerating The Corporate IT Journey Cloud enables ubiquitous access to resources and applications, and workload flexibility Cloud IaaS Infrastructure

More information

XpoLog Center Suite Data Sheet

XpoLog Center Suite Data Sheet XpoLog Center Suite Data Sheet General XpoLog is a data analysis and management platform for Applications IT data. Business applications rely on a dynamic heterogeneous applications infrastructure, such

More information

Machine Data Analytics with Sumo Logic

Machine Data Analytics with Sumo Logic Machine Data Analytics with Sumo Logic A Sumo Logic White Paper Introduction Today, organizations generate more data in ten minutes than they did during the entire year in 2003. This exponential growth

More information

Qlik Sense Enabling the New Enterprise

Qlik Sense Enabling the New Enterprise Technical Brief Qlik Sense Enabling the New Enterprise Generations of Business Intelligence The evolution of the BI market can be described as a series of disruptions. Each change occurred when a technology

More information

A Modern Approach to Monitoring Performance in Production

A Modern Approach to Monitoring Performance in Production An AppDynamics Business White Paper WHEN LOGGING ISN T ENOUGH A Modern Approach to Monitoring Performance in Production Ten years ago, the standard way to troubleshoot an application issue was to look

More information

Analyzing IBM i Performance Metrics

Analyzing IBM i Performance Metrics WHITE PAPER Analyzing IBM i Performance Metrics The IBM i operating system is very good at supplying system administrators with built-in tools for security, database management, auditing, and journaling.

More information

Oracle Database Performance Management Best Practices Workshop. AIOUG Product Management Team Database Manageability

Oracle Database Performance Management Best Practices Workshop. AIOUG Product Management Team Database Manageability Oracle Database Performance Management Best Practices Workshop AIOUG Product Management Team Database Manageability Table of Contents Oracle DB Performance Management... 3 A. Configure SPA Quick Check...6

More information

Oracle Application Performance Monitoring Cloud Service Application Visibility for DevOps

Oracle Application Performance Monitoring Cloud Service Application Visibility for DevOps Oracle Application Performance Monitoring Cloud Service Application Visibility for DevOps Rajiv Maheshwari, VP Product Development John Sullivan, Product Management Jurgen de Leijer, Product Management

More information

Monitoring Best Practices for COMMERCE

Monitoring Best Practices for COMMERCE Monitoring Best Practices for COMMERCE OVERVIEW Providing the right level and depth of monitoring is key to ensuring the effective operation of IT systems. This is especially true for ecommerce systems

More information

Benefits of Network Licensing

Benefits of Network Licensing Benefits of Network Licensing Submitted by: Tony O Dowd, CEO, Alchemy Software Development July 2006 Version 1.2 Abstract Network licensing is a powerful concept used by a number of workstationbased software

More information

Solution Brief TrueSight App Visibility Manager

Solution Brief TrueSight App Visibility Manager Solution Brief TrueSight App Visibility Manager Go beyond mere monitoring. Table of Contents 1 EXECUTIVE SUMMARY 1 IT LANDSCAPE TRENDS AFFECTING APPLICATION PERFORMANCE 1 THE MOBILE CONSUMER MINDSET DRIVES

More information

How Cisco Actively Manages Voice Availability and Quality

How Cisco Actively Manages Voice Availability and Quality How Cisco Actively Manages Voice Availability and Quality Cisco IT automates daily testing of all voice elements in the global network. Cisco IT Case Study / Unified Communications / Unified Operations

More information

SmartCloud Analytics Log Analysis

SmartCloud Analytics Log Analysis SmartCloud Analytics Log Analysis Clyde Richardson (richarcl@us.ibm.com) Technical Sales Specialist Anuja Deedwaniya (anujad@us.ibm.com) IBM z Systems Enterprise Architect Paul Smith (Smitty) (paulmsm@us.ibm.com)

More information

MERAKI WHITE PAPER Cloud + Wireless LAN = Easier + Affordable

MERAKI WHITE PAPER Cloud + Wireless LAN = Easier + Affordable MERAKI WHITE PAPER Cloud + Wireless LAN = Easier + Affordable Version 1.0, August 2009 This white paper discusses how a cloud-based architecture makes wireless LAN easier and more affordable for organizations

More information

Applications Performance Management for Mobile Applications September 18, 2013

Applications Performance Management for Mobile Applications September 18, 2013 Applications Performance Management for Mobile Applications September 18, 2013 Copyright 2013 Vivit Worldwide Brought to you by Vivit Business Service Management Special Interest Group (SIG) Leaders: Jim

More information

The Importance of Software License Server Monitoring White Paper

The Importance of Software License Server Monitoring White Paper LicenseMonitor The Importance of Software License Server Monitoring The Importance of Software License Server Monitoring White Paper Introduction In the electronic design automation (EDA) and other CAD

More information

Using Predictive Maintenance to Approach Zero Downtime

Using Predictive Maintenance to Approach Zero Downtime SAP Thought Leadership Paper Predictive Maintenance Using Predictive Maintenance to Approach Zero Downtime How Predictive Analytics Makes This Possible Table of Contents 4 Optimizing Machine Maintenance

More information

Zend and IBM: Bringing the power of PHP applications to the enterprise

Zend and IBM: Bringing the power of PHP applications to the enterprise Zend and IBM: Bringing the power of PHP applications to the enterprise A high-performance PHP platform that helps enterprises improve and accelerate web and mobile application development Highlights: Leverages

More information

VMware Performance and Capacity Management Accelerator Service

VMware Performance and Capacity Management Accelerator Service AT A GLANCE The VMware Performance and Capacity Management Accelerator Service rapidly deploys a performance management, capacity optimization, and log management solution focused on a limited predefined

More information

SOLUTION BRIEF. TIBCO LogLogic A Splunk Management Solution

SOLUTION BRIEF. TIBCO LogLogic A Splunk Management Solution SOLUTION BRIEF TIBCO LogLogic A Splunk Management Solution Table of Contents 3 State of Affairs 3 The Challenges 5 The Solution 6 How it Works 7 Solution Benefits TIBCO LogLogic A Splunk Management Solution

More information

the missing log collector Treasure Data, Inc. Muga Nishizawa

the missing log collector Treasure Data, Inc. Muga Nishizawa the missing log collector Treasure Data, Inc. Muga Nishizawa Muga Nishizawa (@muga_nishizawa) Chief Software Architect, Treasure Data Treasure Data Overview Founded to deliver big data analytics in days

More information

Accelerate Testing Cycles With Collaborative Performance Testing

Accelerate Testing Cycles With Collaborative Performance Testing Accelerate Testing Cycles With Collaborative Performance Testing Sachin Dhamdhere 2005 Empirix, Inc. Agenda Introduction Tools Don t Collaborate Typical vs. Collaborative Test Execution Some Collaborative

More information

Data Driven Success. Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA)

Data Driven Success. Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA) Data Driven Success Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA) In business, data is everything. Regardless of the products or services you sell or the systems you support,

More information

Agile speech analytics: a simple and effective way to use speech analytics in contact centres

Agile speech analytics: a simple and effective way to use speech analytics in contact centres Agile speech analytics: a simple and effective way to use speech analytics in contact centres Whitepaper Contact centres have successfully used speech analytics to help reduce avoidable calls, improve

More information

Optimizing your IT infrastructure. 2012 IBM Corporation

Optimizing your IT infrastructure. 2012 IBM Corporation Optimizing your IT infrastructure 2012 IBM Corporation Please Note: IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM s sole discretion.

More information

Big Data for Good or Evil Lessons from the NSA PRISM Scandal

Big Data for Good or Evil Lessons from the NSA PRISM Scandal Big Data for Good or Evil Lessons from the NSA PRISM Scandal Jason Bloomberg Chief Evangelist jason@enterpriseweb.com @theebizwizard Copyright 2014, EnterpriseWeb LLC EnterpriseWeb is a Registered Trademark

More information