Neelesh Kamkolkar, Product Manager. A Guide to Scaling Tableau Server for Self-Service Analytics

Transcription

1 Neelesh Kamkolkar, Product Manager A Guide to Scaling Tableau Server for Self-Service Analytics

2 2 Many Tableau customers choose to deliver self-service analytics to their entire organization. They strategically transform their organizations to embrace an analytical culture by empowering their end users to use analytics, but also by enabling IT to deliver rapid-fire analytics at scale. Table of Contents Abstract... 4 Why self-service analytics at scale?... 4 Land and expand...5 Tableau Server scales...6 Enterprise-scale deployment...7 Plan your capacity... 8 WEDU Informing capacity and server configuration... 8 Where to start... 9 Evaluation matrix... 9 Tableau Online Example deployments High risk tolerance single-server deployment for users Low risk tolerance multiple-server deployment for users...13 Expanding to support data refresh needs...13 Scaling up with Repository failover...15 Summary...16

3 3 Abstract This paper provides an overview and recommendations for deploying Tableau Server 9.0 at scale. It includes example deployment scenarios and planning considerations to help you ensure that you size your deployment for success. The recommendations covered in this paper are informed by scale testing we have performed and published in Tableau Server 9.0 Scalability: Powering Self-Service Analytics at Scale. Why self-service analytics at scale? The more accessible your organization makes self-service analytics, the more users in your organization will be empowered to make informed business decisions. Data Anywhere Anarchy of Shadow IT Self Service At Scale Centralized Data Access Traditional BI Central IT Ad Hoc Analytics Empowered End Users Figure 1: Enabling self-service analytics at scale This concept is illustrated in Figure 1. The origin of the horizontal axis is an organization where data and analytics are created and managed in a traditional BI report factory. At the furthest end of the horizontal axis, users are empowered with the data to answer their own questions. They rely on self-service analytics to connect to, analyze, and share data when they make business decisions. The vertical axis shows the range of data access and availability within the organization. When you restrict access to data, keeping it locked under centralized control, your organization cannot realize the potential of self service at scale.

4 4 IT has a unique role to enable users with access to relevant and timely data, yet still maintain data integrity and security. Conversely, if you open up data without any controls, multiple chaotic answers become the norm and create data anarchy. Once again, IT is key in curating meta data and developing data connections and extracts that build confidence in the data and create a reliable single source of truth. A successful self-service analytical culture is a balance of managed data access and empowered business users. Tableau Server provides secure, scalable access to your business data. Tableau Desktop provides the powerful analytic client experience that your employees can use to evaluate, pivot, interrogate, visualize, and share data. Land and expand When you think about scaling a Tableau Server deployment, it s useful to understand how Tableau often grows through an organization. We call this the land and expand phenomenon. A land is an entry point where someone in an organization discovers Tableau and downloads Tableau Desktop, then champions a Tableau Server installation for a pioneer group. Tableau Server combined with growing Tableau Desktop penetration (the expand ), leads to more groups sharing content securely and easily without requiring extensive help from IT. As usage expands, IT support increases to help the business in deploying, maintaining, and supporting the internal Tableau user community. Figure 2: Tableau expansion means growth in People, Processes, and Technology; all in support of accessing more data.

5 5 In addition, Tableau Drive provides a methodology to allow teams to successfully adopt and scale Tableau with Agile best practices in mind. As usage grows, customers invest in Tableau Server to further support their user community on a much larger scale, often globally. Figure 2 shows that expanding access to data requires growing the people, processes and technology capabilities within the company. Tableau Server scales Scalability was a top-priority goal for the Tableau Server 9.0 release. In our scalability testing we found that Tableau Server 9.0 was much more scalable than version 8.3 under comparable conditions. Figure 3 shows a throughput comparison between the two versions. The most significant throughput improvements for any Tableau deployment are typically achieved by adding nodes to the cluster. Figure 3: Throughput comparison: Tableau Server 8.3 and Tableau Server 9.0

6 6 The Tableau Public data center is powered by Tableau Server. In the Tableau Public deployment, our server cluster runs across 32 server-class machines. Supporting more than 100,000 authors, who have published more than 500,000 visualizations, that have been viewed more than 450 million times. On average, we serve up to 800,000 views each day. At peak during the 2012 presidential elections, our deployment supported more than 30,000 views in a single hour. Enterprise-Scale Deployment We often see questions about how flexible Tableau Server is for supporting complex enterprise deployment topologies and scenarios. The Tableau Server architecture enables deployment behind reverse proxy servers, which support internal and external user scenarios, single sign-on experiences, and allows integration with Active Directory. Firewall Firewall Firewall Firewall Client SSO Policy Server Directory Shadow Sync Directory LB LB Server Reverse Proxy Server Database Desktop Maps Untrusted Zone (Internet) Public DMZ App Zone DB Zone Intranet Zone Figure 4: Distributed Tableau Server deployment.

7 7 Figure 4 shows a typical network deployment for Tableau Server in a tiered, firewalled enterprise network topology. You should work with your Tableau team to ensure you size your deployment correctly and set yourself up for success for the long term when you plan a large-scale deployment for your organization. Ensure that you have a good monitoring and management strategy in place for Tableau Server. Plan your capacity To effectively scale your server deployment, factor in both the frequency of analytics use and the required freshness of data to get a cumulative capacity plan. WEDU Informing capacity and server configuration WEDU stands for workload, environment, data, and usage: the combination of factors that should inform your capacity planning and configuration. Start with the following questions: Workload characteristics How many workbooks do you expect your users to publish in a week? How many data sources are used in a business-critical workbook? Do you connect to big data sources, or to Excel files? Environment characteristics Do you plan to deploy on a physical or virtual environment? For Virtual Machines, do you have any deployment constraints on the size of the VMs (vcpus, shared RAM, shared storage)? Do you plan to use vblocks or other virtualized infrastructure for Tableau Server? What is your typical VM configuration (number of cores, RAM, disk, network)? Data characteristics Do you expect to use in-memory analysis (using Tableau extracts), or to connect live to your backend data sources? Does your business require you to regulate or restrict data access to some users? Usage characteristics Do you expect a small number of workbooks to be used by a large number of users at the same time? Answering these questions will help you determine where to start to ensure you realize self service at scale.

8 8 Where to start As you might expect, scalability and performance vary depending upon many factors: hardware, network saturation, degree of shared resources, server process configuration, and tuning all contribute to effective scale. All factors matter. And each factor informs how you tune Tableau Server over time. But to begin, focus on these two questions: How frequently do you use or expect to use analytics to make business decisions? What is your business tolerance for the age of data that Tableau connects to? In other words, how frequently must the underlying data be refreshed? Evaluation matrix Analytics Use for Effective Business Decisions High (every second) Moderate (once an hour) Low (once a day) 7. WW Data Exploration Tableau Public (US Presidential Election) 30K Views/Hour 4. Daily Store Inventory Insurance Customer Analysis Marketing (targeting) 1. Engineering - Ship Room Mortgage Inventory Traditional BI 8. Sales Quota Dashboard, Tableau on TV 5. Patient Capacity Dealer Management 2. Who s Hot Sales Lead Tracking 9. Air Traffic Controller Monitoring Finance Trade Execution 6. Support Escalation Dashboard Finance Portfolio Dashboard Fraud Investigation 3. Highway Web Traffic Dashboards Low (once a day) Moderate (once an hour) High (every second) Data Refresh Frequency for Effective Business Decisions Figure 5: Use profile matrix

9 9 Plot your organization s use profile on the matrix above to help determine your starting place for initial Tableau deployment and further tuning. Tableau Server scales effectively both horizontally and vertically. Scaling your deployment is primarily a function of adding more process instances or additional cluster nodes (workers) to the corresponding use profile. Understanding Tableau Server processes and how they support various server load functions is essential to tuning and scaling your Tableau Server deployments. See Tableau Server Processes for more information. In the matrix in Figure 5, the vertical axis represents frequency of active use of analytics on Tableau Server. The processes associated with this functionality are considered user facing, and they include VizQL Server, Data Server, Data Engine, and Cache Server processes. The horizontal axis represents the frequency of data refreshes necessary to provide the latest data to your organization to make business decisions. The primary process that supports data refresh is the Backgrounder process. Figure 6 shows the relationship between use profile and process scaling. Frequency of Data Analytics Use to Make Decisions High (every second) Moderate (once an hour) Low (once a day) Scale VizQL, Data Server (Published DS), Data Engine, and Cache Servers Low (once a day) High to Moderate Query Cache Use Scale VizQL, Backgrounders Data Servers Scale Backgrounders Low to Moderate Query Cache Use Moderate (once an hour) Live Live (every second) Frequency of Data Refresh Needs to Make Effective Business Decisions Figure 6: Use profile mapped to Tableau Server processes

10 10 To scale to support more active users using analytics with extract-based workloads, you would add VizQL Server, Cache Server, and Data Engine process instances. However, if your users make heavier use of published data sources, consider adding additional Data Server instances. To scale out for more frequent data refreshes, separate your Backgrounder processes onto worker node computers in a cluster. This strategy pays off if you can expand horizontally with worker nodes that you can dedicate to the Backgrounder workloads. And finally, an important factor that informs both performance and scale is how users author Tableau visualizations. Anyone can author Tableau visualizations easily with drag and drop, but when they create a workbook that will get very high usage, your users should follow best practices for authoring Tableau visualizations that are performant and scalable. For more information, see Designing Efficient Workbooks. Tableau Online For some organizations, the best way to manage scalability is to have someone else think about it for them. If your organization does not have a professional IT team to deploy and manage large-scale hardware and software environments, Tableau can do it for you. Consider Tableau Online as your best option, because we ensure that you have the latest software updates, reliable performance, scalability, and availability. Example deployments The following sections provide some examples of scaling Tableau Server. The examples show how relatively straightforward server configuration can improve scalability and performance. Your results may vary, but you can leverage a load-testing tool like TabJolt or engage with Tableau customer solutions teams to help you determine your precise configuration. High risk tolerance single-server deployment for users For this scenario, where your risk profile allows you to tolerate some downtime, consider an 8-16 core server with 64 GB 128 GB RAM. Ensure that there is sufficient disk IO bandwidth (>500 MBPS) available for Tableau Server.

11 11 Cluster VizQL Server Controller Application Server Backgrounder Cache Server Data Server Data Engine File Storage Repository Search & Browse Figure 7: Single-server Tableau Server deployment We have engineered the default, out-of-the box, Tableau Server configuration to optimize for smaller deployments and departmental servers. But the Tableau Server architecture gives you flexibility in how you scale to support your use profile. When you add server processes, keep your corresponding infrastructure capacity in mind. For example, while we recommend minimum RAM and disk for installation purposes, best practice for production server deployments is to use a minimum of 8 GB of RAM per each core allocated to Tableau Server. If you add a new instance of any of the server processes, consider a corresponding increase of 2 GB of RAM per process, rounded to available RAM sizes. As for cores, you can start with 8, but to support future growth in analytics usage, be sure your computer can scale up to 16 cores over time. A single-server deployment is easy to set up, configure, maintain, and manage for smaller teams with little or no IT support. But a single-server deployment also represents a single source of availability failure. While Tableau Server provides out-of-the-box process-level high availability (processes restart automatically if they fail), in this scenario a hardware failure will result in downtime. In the case of catastrophic failure, Tableau Server is quick to install on any available Windows computer that meets Tableau Server specifications. Installing a new Tableau Server takes less than 30 minutes. And most single-server restorations from backup media take less than an hour, though backup and restore time depends on how large your backup files are.

12 12 Low risk tolerance multiple-server deployment for users If you have very low risk tolerance, consider deploying a multiple-server distributed cluster that gives flexibility of deployment. To increase availability and scale, you can add more servers to the cluster as needed over time. Do not split a distributed cluster deployment across multiple under-powered computers. In all cases, each computer in a cluster must meet the minimum hardware recommendations for production use. More cores on each computer will help move parallel workloads quickly through the system. Primary Cluster VizQL Server Controller Application Server Backgrounder Worker 1 Cluster VizQL Server Controller Application Server Cache Server Cache Server Data Server Data Server File Storage Repository Search & Browse Search & Browse Figure 8: Dual-node distributed deployment The deployment illustrated in Figure 8 could support anywhere from 1000 to 5000 users across a cluster with 16 to 44 cores. Use 8 GB per core depending on how many cores you choose, and ensure there is sufficient (>500 MBPS) disk IO bandwidth for the server processes. To further reduce risk and improve cluster availability, you could add a Repository process on Worker 1 and configure automatic failover. See Example 4 below for more information on repository failover. Expanding to support data refresh needs As users discover the power of making business decisions with self-service analytics, they often require access to real-time data, or data that is refreshed more frequently than periodic extracts. For this scenario, the scale unit that matters the most is the Backgrounder process. Among other improvements, adding Backgrounder processes accelerates extract refreshes and ensures subscriptions to your visualizations go out in a timely manner.

13 13 Primary Cluster VizQL Server Controller Application Server Cache Server Data Server Data Engine File Storage Background Worker Cluster Controller Backgrounder Repository Search & Browse Figure 9: Scaling to support higher data-refresh requirements This type of a deployment also reduces risks because it ensures that the services related to data freshness (refresh extracts and subscriptions) run separately from the services that provide responsive end-user visualizations, caching, and metadata services. When adding Backgrounder processes to dedicated worker nodes, you should ensure that there are at least the same number of Backgrounder processes as there are cores on a given machine. However, the maximum number of Backgrounder processes for this scenario must not exceed twice the number of cores. For example, consider a computer dedicated as a Backgrounder worker-node with 16 cores. At a minimum, you would configure this computer with 16 Backgrounder processes, but you should not exceed 32. This guidance is different when you share the Backgrounder workload with other server processes. You can scale vertically with Backgrounder processes on existing worker machines, but it s important to understand that Backgrounder processes compete for CPU, network, and disk resources. Resource competition is especially intense in the scenario where extract refreshes or subscriptions occur simultaneously with user interaction with analytics. In this case, Tableau Server processes that serve end users (such as VizQL Server) compete with Backgrounder processes. So if you add Backgrounder processes to existing worker computers, you should add a number of Backgrounder processes between one-fourth and one-half the number of cores for a given worker computer.

14 14 Scaling up with Repository failover To further scale out and minimize risk, Tableau Server supports flexible cluster topologies. To minimize risk, consider running an active-passive Repository configuration on a cluster deployment. The Tableau Server repository is a database that stores server data. This data includes information about Tableau Server users, groups and group assignments, permissions, projects, data sources, and extract metadata and refresh information. If the Repository fails, then users cannot access Tableau Server. To minimize this risk, you configure failover for the repository as shown in Figure 10. Primary Cluster Controller Worker 1 Cluster VizQL Server Controller Application Server Worker 2 Cluster VizQL Server Controller Application Server Cache Server Cache Server Data Server Data Engine File Storage Data Server Data Engine File Storage Active Repository Passive Repository Search & Browse Search & Browse Search & Browse Background Worker Backgrounder Background Worker Backgrounder Figure 10: Scaling up with repository failover In this configuration, if the active repository fails, the passive repository automatically takes over and becomes active. Downtime is brief, generally taking just a few minutes as the cluster completes the failover workflow. As you plan for this scale of deployment, you should work with Tableau to optimize performance, high availability, and licensing requirements according to your usage profile estimates.

15 15 Summary The following table provides a summary of the guidance from the example scenarios above. Use this guidance as a starting place for scalability planning: In this whitepaper, we reviewed some of the common ways Tableau customers get to self-service at scale. While content management, permissions planning, and other elements are also important, etc., this paper reviewed how to approach capacity planning to ensure that you are set up for long-term success with your Tableau Server deployments.

16 16 About Tableau Tableau Software helps people see and understand data. Offering a revolutionary new approach to business intelligence, Tableau allows you to quickly connect, visualize, and share data with a seamless experience from the PC to the ipad. Create and publish dashboards and share them with colleagues, partners, or customers no programming skills required. See how Tableau can help your organization by starting your free trial at tableau.com/trial. Additional Resources Download Free Trial Tableau Server Administrator Guide Related Whitepapers Tableau Server 9.0 Scalability: Powering Self-Service Analytics at Scale Tableau Server 9.0 High Availability: Delivering Mission-Critical Analytics in the Flow See All Whitepapers Explore Other Resources Product Demo Training & Tutorials Community & Support Customer Stories Solutions Tableau and Tableau Software are trademarks of Tableau Software, Inc. All other company and product names may be trademarks of the respective companies with which they are associated.