Understanding Data Visualization

Transcription

1 Understanding Data Visualization A point of view on the process, people, and technology that are required to make sense of and communicate data through data visualization

2 1

3 Introduction Why Data Visualization? Data is everywhere, and people have been collecting, processing, and comprehending it since the beginning of our existence. After all, it helps us to understand and interact with the world. Now, many businesses are collecting data at a rapid rate. Whether for compliance, reporting, or data visualization purposes, the idea is that this data could be valuable. But true value is only achieved after data is processed, understood, and, ultimately, acted upon. Without these abilities, data is meaningless. Certain characteristics of data can only be seen when data is represented graphically. Thus, the power of data visualization, the art of representing data visually, lies in its ability to turn raw data into meaning and meaning into understanding. It can make sense of and communicate data. When successfully executed, data visualization transforms mere numbers or words into a story. In order to gather real learnings from their data, businesses need to adopt data visualization as a new common language for data exploration and communication. 1 Why Now? Several advancements in technology have made data visualization more salient now than ever before. Massive amounts of data, known as big data, can now be collected, stored, and made accessible, and these methods are improving every day. In fact, there are an increasing number of public big data assets, which can be used to supplement private data sources. Therefore, for those companies who have chosen to invest in big data, they run the risk of getting lost in an inundation of data. In this case, data visualization becomes even more valuable because it is a way for companies to see their data, make sense of it, and communicate new insights. At many companies, there has been a typical progression of data visualization technology. First, there were simple tables and charts made by hand, which was followed by the introduction of Excel. Excel became the tool of choice and often still is at many companies because of its legacy and relative ease of use. Next, databases became a popular way to store and access data, so companies invested in traditional business intelligence (BI) platforms, which claimed to come with data integration, data infrastructure, data analysis, and data presentation capabilities. These presentation capabilities began as reports and were followed by dashboards, with dashboards becoming increasingly interactive. In the age of big data, however, when data discovery, analysis and visualization capabilities are even more salient, traditional BI tools are falling short. In order to more easily enable insights into large and complex datasets, the visualization market has grown significantly in the past few years and continues to grow. Light-weight data discovery tools are one of the fastest growing areas of BI, and the more traditional BI software tools are reshaping their offerings to address this. 2 Moreover, with mediums such as computers, tablets, and mobile phones becoming ubiquitous, people are now able to interact with their data more immediately than ever before. With current tools undergoing rapid development and new tools coming out every day, it is difficult to recognize which are the most appropriate and effective for specific use-cases and needs. This very issue will be discussed further in the Technology section of this paper. Data visualization capabilities enable easier interaction with and understanding of data, which becomes increasingly important in the age of big data. Given this and the burgeoning data visualization market, it is too easy for companies to get started without a solid foundation. In order for data visualization to successfully be adopted as the common language for data exploration and communication, an investment first needs to be made in the process, people, and technology. 2

4 Overview Definition Data visualization is the visual representation of data meant to allow people to both understand and communicate information through graphical means. 3 One form or subset of data visualization frequently exposed to the public through means such as newspapers is infographics. Infographics tend to be created for a specific dataset in order to express an editorial point of view. While this paper will touch on infographics, it will largely focus on Visual Advantage Data visualization is especially effective because people are extremely well suited for visual analysis. People are very good at pattern matching and organizing what they see in order to make sense of it. Successful data visualizations use differences in properties such as size, color, and shape in order to take advantage of our preattentitive (before conscious awareness) visual processing. In fact, [Human brains] thrive primarily on pattern matching, something with which computers other types of data visualizations which are more repeatable and objective and therefore more suitable in enterprise. The definition of data visualization references its two objectives: understanding and communication; this can also be referred to as exploratory and explanatory data visualization respectively. This distinction is often overlooked, but it is extremely critical in the process of creating a successful data visualization. If the motivation is to struggle. 4 Moreover, data visualizations can consolidate lots of information in one place, allowing people to more easily and fully understand the data. Anscombe s quartet is a very famous example of just how effective data visualization can be (See Figure 1), especially in a case where using simple statistics falls short. It consists of four datasets, first shown and compared in tabular format and then after performing explore and make sense of data, then the data visualization should be exploratory in nature. However, if the analysis of the data is complete and the data visualization is meant to explain and communicate a finding, then an explanatory data visualization or infographic should be used. Without being conscious of one s data visualization motivations and goals, the process can be inefficient, misguided, or altogether unsuccessful. a simple analysis. It is very difficult to find meaning from the table of numbers itself, and the mean, variance, correlation, and linear regression of the numbers are also practically identical. It is not until after one graphs the datasets that patterns become immediately apparent. Influence Unfortunately, there is very little research that examines the exact influence that data visualization has on decision makers. Certainly, it depends on the effectiveness of the data visualization itself, but many other factors also matter. Often, these variables, such as pre-existing knowledge or one s mood, can have an illogical or even counterintuitive effect on the decision making process. 5 Hans Rosling, a leader in the field of data visualization who advocates a fact-based view, has said that people do not need more data but rather a new mindset, a new way of thinking about and accepting data and the insight it provides. 6 While this may seem to detract from the significance of data visualization, it should instead highlight the importance of the context around which data is presented. Duncan Swain, the creative partner at Information is Beautiful, says they always start with the story, the context, and what [they] are trying to tell [their] audience. 7 3

5 Figure 1. Anscombe s quartet in tabular and graphical forms. 8 Anscombe s quartet I II III IV x y x y x y x y Property Value Mean of x 9 Variance of x 11 Mean of y 7.50 Variance of y or Correlation Linear regression y = x y 1 8 y x 1 x y 3 8 y x 3 x 4 4

6 Process The process of creating a data visualization or an infographic is multi-disciplinary and includes a wide variety of sub-processes that must be well-integrated in order to be successful [Figure 2]. The most crucial step of the process is setting a goal from the beginning; it is important to have a clear understanding of who the data visualization is for, as well as its purpose. Without this information, companies risk wasting time and energy making an ineffective data visualization. The Represent step is also very important because it is where the data visualization design process takes place. Using leading practices to choose an effective, rather than flashy, way to display and communicate data is vital. While this paper does not discuss data visualization design principles in depth, there are numerous books which go into detail on this topic (authors include Edward Tufte, Colin Ware, Noah Illinsky and Julie Steele). In order to create a successful data visualization, each step must be complete before continuing to the next. This is not to imply that the process is perfectly linear. A step could appear to be complete but then have to be re-examined at some point in the future; parts of the process could be quite iterative. The problem comes in, however, when companies are not aware of the process or skip over steps, assuming they are unnecessary or overly time consuming. This will either result in a poor end product or inefficiencies throughout the process when steps need to be completely redone. Thus, the process is absolutely critical to follow. Figure 2. A summary of the data visualization process (Reference A shows detailed workflow). Process Skills Summary Set goal Understand the motivation and define the appropriate end goal. This step is vital to a successful visualization. Then, based on the current status, determine from which process step it is appropriate to start. Acquire Computer science Acquire the relevant data, and make sure it is a as complete and accurate as possible. If helpful, supplement with public data. Format Computer science Parse and format the acquired data into a usable form. If there are multiple datasets, make sure they are integrated. Filter Analyze Statistics and data mining Statistics and data mining Filter the data, such that the dataset only includes the data necessary. Choose the appropriate tool(s) for analysis. Then, mine, model, and analyze the data. Use exploratory data visualization as appropriate. If others will interact with this data, then continue on in the process. Otherwise, this is the last step; reiterate the above steps if applicable. Represent Graphic design Choose the appropriate tool(s) to represent the data visualization. Then, design and create the data visualization or infographic. Refine Graphic design Refine the data visualization or infographic so that it is suitable for the targeted audience. Interact Human computer interaction Publish, deploy, and interact with the data visualization. If the final product is unsatisfactory, determine which step of the process to which to return, and then repeat the following steps. Otherwise, if the goal has been achieved, then the process is finished. Exploratory Exploratory or explanatory 5

7 People People are closely related to the process, so additionally, a wide variety of skills are necessary to create a successful data visualization. The skills draw from the disciplines of computer science, statistics and data mining, graphic design, and human computer interaction. 9 The most advanced data analysis and data visualization companies use teams consisting of specialists interspersed with a few generalists. The x-factor that makes these teams so successful is their ability to work together, despite tendencies of experts to remain isolated and immersed in their areas of study. 10 Currently, data science and data visualization skills are highly technical and require some amount of coding. Unfortunately, these skillsets are also in high demand. Enrico Bertini, a leading researcher in the field, believes that the biggest problem right now is training people or finding people who are experts in visualization The bottleneck in visualization is not necessarily the tools but more having the right skills for doing high quality data visualization. 11 At the same time, this presents an opportunity for technology to better cater to less technical persons and effectively reduce this reliance on data visualization specialists. If advanced data analysis and visualization are part of a company s core focus (The New York Times, LinkedIn, Trulia), then they should invest in creating teams of specialists. It may be hard to hire people with these skills, as they are in high demand, so companies may want to invest in developing some of these skillsets internally, as well. 12 For those companies whose core processes do not include data visualization, it is recommended that they decide how advanced they want their data visualizations to be. If all that is required is simple day-to-day reports, then they should invest in teaching their employees basic data visualization techniques. However, for more advanced data visualizations, these companies should seek help from outside. Given the number of specialists that are required, it does not make sense to waste time and effort hiring new full-time employees or taking current employees away from the work they are currently doing, while also risking poor training due to lack of proper guidance. For creating a repeatable data visualization or data visualization application, outside help will likely be required. For more specific one-time infographics to use in marketing, for example, graphic design firms should be employed. 6

8 Technology While data visualization experts often point to Tableau as the leading tool in its field, many also believe that an ideal tool does not yet exist. In fact, it may be impossible to create a single tool that can meet every possible need of its users. Therefore, many people who produce highly unique and customizable data visualizations frequently use a variety of tools, while others who rely on more repeatable solutions may focus on just one. 7

9 There is a still much room for improvement in existing tools, particularly in certain areas, such as gathering and cleansing data, data integration, and real-time data visualization. One of these pain points that may often be overlooked is that it is still very difficult to integrate data from various sources and explore that data through data visualization. While Business Intelligence tools may have more of the back-end integration, they are lacking the frontend exploration and final presentation sophistication. On the other hand, lighter weight data visualization tools may have the front-end capabilities, but they cannot yet easily connect disparate data sources on the back-end. The software that is best able to accomplish this first will be well-positioned in the market going forward. 8

10 Given the plethora of visualization tools out on the market, it can be overwhelming to choose the appropriate one(s). However, the field can be narrowed into one of four categories based on the end goal of the data visualization. These four categories are outlined below, as well as several popular tools within each of the categories. Business Intelligence (BI) Tools Currently, BI tools are the most commonly deployed tools with data visualization capabilities. They are very good at reporting and usually have some sort of dashboard functionality. However, the dashboards tend to remain static or limited dynamically, and they do not have a wide range of out-of-the-box data visualization display types. This sort of data visualization works well for communicating simple, standard charts and graphs, and they are good for displaying high-level business data, such as KPIs. However, for exploratory data analysis or an interactive application, these tools fall short. BI companies are aware of the success of light-weight data visualization specific tools, and many are working to incorporate similar functionality in their offerings, as well. Examples: Microsoft, IBM, SAS, SAP, MicroStrategy Analytic Tools Here, analytic tools are referred to those that are particularly good at statistical data analysis but have limited data visualization capabilities. Usually, these products involve coding in order to see the data. The analysts who typically use these tools are using data visualization to help themselves explore the data or test models and not for sharing with or communicating the data to others. In this case, function is more important than form. In order to make these data visualizations presentable, it is recommended to use another tool in conjunction, such as Adobe Illustrator. Examples: R, SPSS Modeler, SAS, Excel, Matlab Visualization Tools In this paper, visualization tools are referred to as tools with very advanced data visualization functionality, but they tend to have less sophisticated data analysis capabilities. The best tools focus on making data visualization creation as seamless and non-technical as possible, reducing the reliance on a team of highly trained, technical specialists. These tools are the best at simple exploratory data analysis, but they tend to lack the ability or ease to handle involved statistical analysis. It may be necessary to use another tool in addition to carry out complex data analysis. Examples: Tableau, Spotfire, QlikView, Advizor, Excel Tools for Custom Work There are also many data visualization tools that require advanced skills and technical knowledge in order to implement. They would most commonly be used for one-time publication-worthy data visualizations. While they are more aesthetically pleasing, they take much more time, effort, and knowledge to create. Examples: D3, Processing, Adobe Illustrator 9

11 Use Cases Geospatial Data Visualization in the Oil and Gas Industry In the oil and gas industry, expertise in the downstream supply chain often relies on vast knowledge and intuition developed over years of experience. Even then, optimizing inventory, setting schedules, and mitigating outages may be incredibly difficult. This is due to a variety of reasons. For example, the schedule and forecast data may be coming from a variety of sources and tough to comprehend all at once for multiple locations. There may also be unexpected outages due to weather or equipment failure. When supply chain schedulers attempt to better regulate inventory and mitigate this risk, they are often challenged to quickly determine the optimal way to route product. Geospatial data visualization provides one way to improve decision making, reduce the risk of outages, and effectively save the company money. By displaying terminals, inventories, and pipelines on maps, visualizing and understanding the downstream supply chain becomes much easier and more intuitive. With all of the data in one central, location-based, and visual repository, a scheduler can be alerted to when an inventory is getting too low, and an optimal routing of product to that location can be determined more easily. A scheduler could even proactively monitor inventory by noticing that a forecast is consistently low at a location and increase product to that site. Alternatively, if weather data is incorporated into the application, the scheduler could see a storm predicted to hit, and the amount of product going to that location could be increased to avoid or delay a probable outage. Not only would this visual display of information be helpful to the downstream supply chain schedulers, but it would be helpful to many others in the industry, as well. New employees could more quickly and intuitively understand how the downstream supply chain works. In addition, senior management could more easily monitor the status of their supply chain and the work of their schedulers to ensure it is healthy. In these ways, the ability to geospatially visualize data can effectively allow a company to perform better and save money. Geospatial Data Visualization to Monitor the Health of an Electric Grid Typically, the health of an electric grid is monitored reactively. This means, for example, that when there is a power outage, the electric company must wait to get a call from a customer reporting the outage. This is detrimental to the company for a variety of reasons. First, it may make it difficult for the company to meet their service level agreement (SLA). Second, for every second that the electricity is out and a repair has not been made, the company is losing money. Therefore, the faster that an electric company can identify and fix an outage, the better off it will be. By geospatially visualizing the electric grid and associated sensor data, an electric company can immediately identify when and where an outage occurs. From there, a repairman can be dispatched immediately to the source of the outage. The ability to geospatially visualize this data, or any network data, can improve a company s ability to meet their SLAs and reduce outage time, effectively allowing the company to perform better and save money. 10

12 Conclusion By itself, data is meaningless. It only becomes valuable when it can be analyzed, understood, and strategically acted upon. Therefore, as the amount of data being collected grows, so does the need for data visualization, which can both make sense of and communicate data. Businesses should invest in learning from their data through data visualization, but in order to do so, they must understand the process, people, and technology required. Currently, these three components are quite technical and interdisciplinary, so the challenge compounds; they must work tightly and function together. Many leaders in the field hope and believe that in the future, data visualization will become more real-time, interactive, and accessible for all. Decisionmakers will be able to react better and faster as technology improves and data visualizations become real-time. Additionally, as data visualizations become more interactive, they will allow people to more easily explore their data on the fly. Simply by improving these real-time and interactive data visualizations, the ability to understand data will be attainable by many more people. Moreover, there will be improvements in technology to make data visualizations easier to develop. This will, in turn, make data visualization accessible to many more people who may not have the highly technical skills that are currently required. Enrico Bertini says it best: Interactive data visualization [can be used] as a way to give powerful tools to scientists and engineering and professionals of any kind to really make sense of the data they work with every day [and to] expose what is happening And this is the kind of evolution that I really want to see and we will probably see in the future because almost every kind of professional around the world is dealing with much more data than there used to be even five or ten years ago, and this is not just statisticians or data analysts [but others who] have no skills to deal with that. I think that s the biggest challenge but also the most exciting because all of these people need new tools and probably custom tools to really help them make sense of their own data. 11 Companies must rise to the occasion to adopt data visualization as their mode of data exploration and communication. In the long run, not only will it drive the improvement of data visualization technologies, but more importantly, it will supply businesses with the information and understanding they need to strategically learn from their data and act upon it. 11

13 12

14 Works Cited 1 Adam Bly, Gaining Understanding from Data Visualization, World Economic Forum, content/gaining-understanding-datavisualization-0 2 Dan Sommer, Rita L. Sallam, James Richardson, Emerging Technology Analysis: Visualization-Based Data Discovery Tools, Gartner, June 17, Vitaly Friedman, Data Visualization and Infographics, Smashing Magazine, July 9, com/2008/01/14/monday-inspirationdata-visualization-and-infographics/ 4 Erich Vieth, The Brain is not a Computer, May 18, dangerousintersection.org/2006/05/18/ the-brain-is-not-a-computer/ 9 Ben Fry, Computational Information Design, April phd/dissertation c.pdf 10 Gil Press, Mok Oh: To Do Data Science, You Need a Team of Specialists, Smart Data Collective, June 15, gilpress/52052/mok-oh-do-datascience-you-need-team-specialists 11 Enrico Bertini, Telephone Interview, July 11, Dan Woods, LinkedIn s Monica Rogati On What Is A Data Scientist? Forbes, November 11, com/sites/danwoods/2011/11/27/ linkedins-monica-rogati-on-what-is-adata scientist/ 5 Andy Kirk, Discussion: Can visualization influence people? Can we prove it? April 13, visualisingdata.com/index.php/2011/04/ discussion-can-visualisation-influencepeople-can-we-prove-it/ 6 Hans Rosling, Hans Rosling: Let my dataset change your mindset, TED, June hans_rosling_at_state.html 7 Duncan Swain, Telephone Interview, July 18, Anscombe s quartet Wikipedia, May 3, Anscombe s_quartet 13

15 Appendix Figure 3. Complete data visualization process diagram. Set goal Start Define goal Is data satisfactory? Yes Is analysis satisfactory? No No Yes Acquire Acquire data, supplement with public data if necessary Re-acquire Format Parse and format data Integrate multiple datasets Yes Filter Filter to only include relevant data Has goal been achieved? End Re-analyze No No Analyze Choose tool(s) Analyze/mine/ model data Will others view or interact with the data? Need to re-acquire or re-analyze? Yes Represent Choose tool(s) Create visualization Refine Refine visualization Refine Re-analyze Interact Publish/deploy/ interact with data Has goal been achieved? No Need to re-acquire, re-analyze or refine? End Yes 14

16 Contact us Krista Schnell Nathan Shetterley About Accenture Accenture is a global management consulting, technology services and outsourcing company, with approximately 266,000 people serving clients in more than 120 countries. Combining unparalleled experience, comprehensive capabilities across all industries and business functions, and extensive research on the world s most successful companies, Accenture collaborates with clients to help them become high-performance businesses and governments. The company generated net revenues of US$27.9 billion for the fiscal year ended Aug. 31, Its home page is About Accenture Technology Labs Accenture Technology Labs, the dedicated technology research and development (R&D) organization within Accenture, has been turning technology innovation into business results for more than 20 years. Our R&D team explores new and emerging technologies to create a vision of how technology will shape the future and invent the next wave of cutting-edge business solutions. Working closely with Accenture s global network of specialists, Accenture Technology Labs help clients innovate to achieve high performance. The Labs are located in Silicon Valley, California; Sophia Antipolis, France; Arlington, Virginia; Beijing, China and Bangalore, India. For more information, please visit: Copyright 2013 Accenture All rights reserved. Accenture, its logo, and High Performance Delivered are trademarks of Accenture