Data Visualization and Discovery Market Trends and Requirements

Transcription

1 Discovery December 31, 2013 Brad Shimmin, Research Director, Business Technology Software Summary Brad Shimmin Current Analysis Research Director A picture is worth a thous words, a well-designed data visualization is worth a thous spreadsheets filled with both words numbers. How is the market for data visualization discovery tools evolving to replace augment traditional analytical tools such as spreadsheets white green printed reports? In this market analysis, we will look at big data in the palm of your h -- the tools we use to derive meaning from the billions of rows of data residing in corporate data stores, the web itself even within spreadsheets themselves. Along the way we discuss current requirements of enterprise customers exploring the advantages of big data, we will also discuss future opportunities being explored by technology providers operating within this rapidly evolving marketplace. Perspective How do we use the data taking up space in those data warehouses business intelligence platforms to underst something big like whether or not a clinical trial was a success, or something small like when to reorder a stock item? Most importantly, how do we even know which questions to ask of our data? That s where data visualization discovery tools enter the scene. Consider the following visualization created by Ferna Viégas Martin Wattenberg, who were co-directors of Google s Big Picture visualization research group. This was created in 2012, when they took hourly data from the National Digital Forecast base to show how the wind flows across America. The intensity of the white lines represents the average force of individual gusts. Europe +33 (0) Or visit our Web site: 1

2 Discovery This one simple image sums up the point of data visualization discovery tools, namely the discovery of actionable knowledge hidden within vast sums of information. In this case, this image forms a very unique useful way to show near real-time data that is both informative artful. As the image above illustrates, when vast amounts of disparate data are brought together, meaningful patterns emerge. For instance, by pooling a wide array of local data points, analysts can better underst local weather patterns even predict potentially dangerous weather events. Such visualizations generated by products such as QlikView by QlikTech, Spotfire by TIBCO, Tableau, Lumira by SAP numerous others seek therefore to illuminate, extract, compose communicate valuable knowledge by: Blending the aesthetic informational Creating a narrative from raw information Encouraging users to learn rather than simply digest knowledge Engaging both visual auditory aspects of the human mind Encoding details more forcefully through storytelling Filtering out the unimportant For business users, this translates into a faster time to understing. visualization discovery therefore is really about speeding human thought in making business decisions. It is about the discovery of opportunities, not just the validation of existing hypothesis. Moreover, it is about the democratization of knowledge, bringing it to a wider audience when where it will do the most good. Imagine if an insurance company could use this sort of wind-flow information for more accurately understing then predicting when where a tornado is most likely to strike now, or a year from now? Imagine how that company could fine tune its policies to lower its own risk while providing a fairer pricing structure for its customers. Instead of overcharging everyone for claims submitted last year, they could keep rates low for those living well clear of potential, severe weather events. Solution Benefits These objects are nothing new. Business intelligence (BI) solutions such as SAP Business Objects IBM Cognos have sought to deliver the same business opportunities for decades. What constitutes a data visualization discovery tool, how do such tools differ from these traditional BI solutions or even personal analytical tools such as a Microsoft Excel spreadsheet? First foremost, these tools are not a replacement for BI solutions. They represent a new entry point for such solutions, as well as for data warehouses, Hadoop clusters, data streams, of course, spreadsheets themselves. They are the natural evolution of the ideas that originated in business intelligence solutions. As stated above, they speed access to knowledge, allowing users to field retrieve query results in a matter of minutes rather than over the course of a few weeks. They feature rely heavily upon rebuilt reports, even suggested reports data visualizations that are based upon information currently being considered. They are also built for a wider audience, reaching far beyond data scientists data analysts to serve the needs of line-of-business users. They do this by removing or at least abstracting away traditional knowledge requirements surrounding parameterized reports, data transformation, model design the like. They are built for flexibly combining disparate data sources such as a traditional RDBMS, data warehouse, line-of-business applications more, whether housed locally or externally on a public cloud service such as Salesforce.com. What follows is a short list of some of the more common features found within these solutions. Multi-dimensional: Affords multiple dimension navigation with data point expansion (showing all underlying data), as well as drill up down Dynamic: Supports large data set snapshots; blends the two universes of massively scalable file systems (Hadoop) traditional row column databases Animated: Shows changes over time supports geospatial overlays (maps, buildings, etc.) Europe +33 (0) Or visit our Web site: 2

3 Discovery Packaged: Offers componentized visualizations with suggested view layouts/org based upon library of best practices set down by the company or the product itself Cinematic: Provides storyboarding tools to combine multiple visualizations to create a story Exploratory: Allows users to move between various hierarchies without imposing data model changes Proactive: Offers KPI custom alerts from cockpit gauges other dashboard tools Mobile: Delivers the entire user experience via mobile device with native alerting capabilities (even with interactivity built in) These capabilities truly benefit existing investments in data warehousing business intelligence platforms, greatly improving the value of such platforms, which have built a poor reputation over the past decade for being expensive, insular inflexible. For example, with data visualization discovery tools, users can turn the tables on traditional BI approaches which start with an outcome in mind (such as Do current subscribers in the U.S. fail to renew their full data plan if they use fewer than 10 minutes of talk time a month? ). Instead, users can start with a simple, innocent question (such as Why are our data plan renewals trending downward? ). Solution Drawbacks Unfortunately, this democratization of data beyond data scientists, which comes with the capability to meld disparate data sources, near zero provisioning deployment times, a reliance upon pre-built models, queries reports, brings with it a tremendous amount of risk responsibility. Since these are used to make important decisions, there are a number of risks users must consider when choosing to deploy a data visualization discovery solution. The Human Factor: The manner in which people interact with computers data both generates is greatly influenced by natural biases. These must be taken into account not as much by the software, but those who define how the software is used. Too Much Information: Even with these tools, it is very difficult for users to correctly visualize underst millions millions of data points. Wave vs. Particle: Conversely, it s equally difficult both to see breath to underst depth at the same time. Both must be considered for a full understing of a complex problem. What Is That Thing?: Navigating using advanced visualizations requires knowledge, often highly specialized. Note, however, that tools are actively seeking to rectify this problem with smarter visualizations that are capable of understing when things are going to go wrong with your model or your question. Who Do You Trust?: Combining disparate data sources without imposing data governance constraints practices raises questions of trust accuracy. Location, Location, Location: The size type of data output (monitor size, for example) limit not just what you can view, but also what you can underst. Rush to Judgment: Effective compelling visualizations can trigger uninformed biases lure users into making flawed conclusions. Vendors Snapshot With such an extensive ambitious feature set, data visualization discovery tools vary wildly from vendor to vendor depending upon each supplier s history, engineering moxy adjacent product capabilities. The following list is by no means comprehensive, but instead represents those vendors either specializing in data visualization discovery or those with broad portfolios including those solutions. In the coming months, we will develop a comparative product review featuring many of these vendors. Europe +33 (0) Or visit our Web site: 3

4 Company Product Key Differentiator Major Challenge QlikTech QlikView Market pioneer with 29,000 customers worldwide Partner dependent with high-end, BI-oriented pricing model Tableau TIBCO Spotfire MicroStrategy Analytics Platform SAS Visual Analytics IBM Cognos, SPSS Microsoft Power BI, HDInsight Oracle Exalytics SAP Business Objects, Lumira BIME Fan-friendly, favorite for those looking to communicate effectively, not just analyze data Development-savvy offering with a strong predictive analytics capability Early mobile player cross-firewall friendly Full suite including modeling mining, prediction, forecasting simulation Gr old gentleman (70s) of BI with powerful opportunities through engineered systems DIY master utilizing familiar tools for both big little data base-capable solution built to use engineered systems support line-of-business application In-memory focus for line-of-business applications, powered by SAP HANA database Flexible cloud model built to support numerous thirdparty platforms (Google BigQuery, Amazon Redshift, etc.) Not built to ingest large data sets; skewed toward executive communications over data scientists Comparatively late to tackle large-scale, in-memory/database analysis; same goes for modern visualization user experience support Only recently revamped to target self-service big data desktop visualization workloads Newly launched solution must market against legacy of high-end business intelligence portfolio Still oriented toward traditional, multi-year development cycles, targeting a more rarified user constituency (data analysts, business owners, etc.). Company channel not equipped to tackle businessoriented consultative engagements Slow to embrace current market trends toward the data democratization via the public cloud Product portfolio very much in transition from premises to cloud (HANA Cloud Platform) from discontinuity to unification (Project Fiori) Departmental in nature, only beginning to focus on the U.S. market Discovery Near Term Drivers DIY Big Projects Now Possible: With powerful flexible cloud-based data storage analysis services emerging such as Amazon Redshift Google BigQuery, enterprises of all sizes can now readily build large, advanced data sets upon which to run data visualization routines. Cloud-based BI Achieves Legitimacy: Pushing further into SMB departmental opportunities, vendors will rely heavily upon software-as-a-service (SaaS) delivery. This will also be driven by a need to more readily accommodate public cloud data sources such as Google Salesforce.com. Mobile User Experience a Top Priority: Customers expect to gain access to data both where when they need it, making rich, native mobile applications the preferred interface for data visualization discovery tools. The Demise of the Scientist: The emergence of self-service analytics coupled with cloud-based delivery a focus on mobility, data visualization analytics offerings will enable all business users to take on the role of data scientist. This will greatly offload IT requirements for all but the most complicated models queries. Analytics Now Live in Line-of-Business Applications: Through open APIs mature development frameworks, vendors will begin pulling data directly from line-of-business applications rather than traditional data warehouse sources. Increasingly, vendors will also begin delivering analytics in-situ, within those same line-of-business applications. Europe +33 (0) Or visit our Web site: 4

5 Discovery Competitor Response & Recommendations IBM must accelerate its planned delivery of core InfoSphere, Cognos SPSS functionality via the public cloud. Now that it has successfully acquired already begun to leverage cloud provider Soft- Layer, IBM must deliver user-facing data analytics, integration, governance analytics capabilities as pure SaaS offerings. This was the case in September 2013, when IBM released a cloud-based rendition of Cognos TM1 via the SoftLayer cloud. Microsoft should market against the perception that top-down, company-wide enterprise data analytics projects are the only way to make use of big data. The very concept of big data is in danger of turning into the next service-oriented architecture (SOA) -- a good idea that simply cannot be realized, at least not in its entirety. On the other h, Microsoft can position its diminutive self-service business intelligence solution, Power BI for Office 365, as highlighting the value of working with smaller tools from the bottom up. Oracle must do a better job of promoting its support for involvement in open source projects surrounding data analytics. Its heavy investment in solution-complete offerings belies its claims of openness customer choice, inviting further competitor claims of vendor lock-in. Continuing investments in support of big data strive to keep analytical processing rooted within Oracle s own structured unstructured database technology. A broader partner ecosystem further productized solutions utilizing open source offerings will go a long way in this effort. SAP must counter IBM Oracle s push toward a coordinated effort spanning both fully selfcontained hardware (e.g., IBM Pure Systems) prescriptive software patterns (such as IBM PureApplication System). With the same capabilities delivered as either a complete hardware solution or a software appliance, these firms their partners can rapidly bring new technologies to a wide array of customer types. Recommended Buyer Actions Potential customers looking for a do-it-yourself big data solution have many options from which to choose at present. However, for implementations that scale beyond the desktop to incorporate multiple data sources or data at scale, potential customers should plan on obtaining a professional services component, which can easily match the cost of software alone. Potential customers working within a large IT enterprise who are looking to roll out a data analytics project should strongly consider an engineered system such as Oracle Big Appliance. The use of an engineered system allows project leaders to keep ownership of the entire system without having to coordinate with or jointly support others in allocating storage, server networking resources. Potential customers business partners should look for technology providers that are developerfriendly. This was not always the case with smaller vendors, though most players within this market frequently utilize open source projects. However, many larger players also offer full, cloud-based IDEs, a full platform-as-a-service (PaaS) offering SDKs for their mobile clients, offering users ISVs a lifecycle-complete development experience. Europe +33 (0) Or visit our Web site: 5