Platfora Installation Guide

Transcription

1 Platfora Installation Guide Version 4.5 For On-Premise Hadoop Deployments Copyright Platfora 2015 Last Updated: 10:14 p.m. June 28, 2015

2 Contents Document Conventions... 5 Contact Platfora Support...6 Copyright Notices... 6 Chapter 1: Installation Overview (On-Premise)...8 On-Premise Hadoop Deployments... 8 Master vs Worker Node Installations...9 Preinstall Check List High-Level Install Steps Chapter 2: System Requirements (On-Premise) Platfora Server Requirements...13 Port Configuration Requirements...14 Ports to Open on Platfora Nodes...15 Ports to Open on Hadoop Nodes...15 Supported Hadoop and Hive Versions Hadoop Resource Requirements...17 Browser Requirements...18 Chapter 3: Configure Hadoop for Platfora Access Create Platfora User on Hadoop Nodes...19 Create Platfora Directories and Permissions in Hadoop HDFS Tuning for Platfora Increase Open File Limits...21 Increase Platfora User Limits Increase DataNode File Limits Allow Platfora Local Access MapReduce Tuning for Platfora...23 YARN Tuning for Platfora Chapter 4: Install Platfora Software and Dependencies...27 About the Platfora Installer Packages Install Using RPM Packages Install Dependencies RPM Package Install Optional Security RPM Package...29 Install Platfora RPM Package (Master Only)...30 Install Using the TAR Package...31 Create the Platfora System User Set OS Kernel Parameters...33 Install Dependent Software...35

3 Platfora Installation Guide - Contents Install Platfora TAR Package (Master Only) Install PDF Dependencies (Master Only) Chapter 5: Configure Environment on Platfora Nodes...43 Install the MapR Client Software (MapR Only)...43 Configure Network Environment Configure /etc/hosts File...45 Verify Connectivity Between Platfora Nodes...46 Verify Connectivity to Hadoop Nodes...47 Open Firewall Ports Configure Passwordless SSH...49 Verify Local SSH Access...49 Exchange SSH Keys (Multi-Node Only)...49 Synchronize the System Clocks Create Local Storage Directories...51 Verify Environment Variables...52 Chapter 6: Configure Platfora for Secure Hadoop Access About Secure Hadoop...53 Configure Kerberos Authentication to Hadoop Obtain Kerberos Tickets for a Platfora Server Auto-Renew Kerberos Tickets for a Platfora Server Configure Secure Impersonation in Hadoop...55 Chapter 7: Initialize Platfora Master Node Connect Platfora to Your Hadoop Services...57 Understand How Platfora Connects to Hadoop Obtain Hadoop Configuration Files Create Local Hadoop Configuration Directory...59 Initialize the Platfora Master Configure SSL for Client Connections...71 Configure SSL for Catalog Connections About System Diagnostic Data...74 Troubleshoot Setup Issues View the Platfora Log Files Setup Fails Setting up Catalog Metadata Service...75 TEST FAILED: Checking integrity of binaries Chapter 8: Start Platfora...78 Start the Platfora Server Log in to the Platfora Web Application Add a License Key...81 Change the Default Admin Password...81 Page 3

4 Platfora Installation Guide - Contents Load the Tutorial Data Chapter 9: Initialize a Worker Node Appendix A: Command Line Utility Reference...85 setup.py hadoop-check hadoopcp hadoopfs install-node platfora-catalog platfora-catalog ssl...97 platfora-config platfora-export platfora-import platfora-license platfora-license install platfora-license uninstall platfora-license view platfora-node platfora-node add platfora-node config platfora-services platfora-services start platfora-services stop platfora-services restart platfora-services status platfora-services sync platfora-syscapture platfora-syscheck Appendix B: Glossary Page 4

5 Preface This guide provides information and instructions for installing and initializing a Platfora cluster. This guide is intended for system administrators with knowledge of Linux/Unix system administration and basic Hadoop administration. This on-premise installation guide is for data center environments (either physical or virtual data centers) that have a permanent, managed Hadoop cluster. Platfora is installed in the same network as your Hadoop cluster. Document Conventions This documentation uses certain text conventions for language syntax and code examples. Convention Usage Example $ Command-line prompt - proceeds a command to be entered in a command-line terminal session. $ sudo Command-line prompt for a command that requires root permissions (commands will be prefixed with sudo). $ ls $ sudo yum install open-jdk-1.7 UPPERCASE italics [ ] (square brackets)... (elipsis) Function names and keywords are shown in all uppercase for readability, but keywords are caseinsensitive (can be written in upper or lower case). Italics indicate a usersupplied argument or variable. Square brackets denote optional syntax items. An elipsis denotes a syntax item that can be repeated any number of times. SUM(page_views) SUM(field_name) CONCAT(string_expression[,...]) CONCAT(string_expression[,...]) Page 5

6 Platfora Installation Guide - Introduction Contact Platfora Support For technical support, you can send an to: support@platfora.com Or visit the Platfora support site for the most up-to-date product news, knowledge base articles, and product tips. To access the support portal, you must have a valid support agreement with Platfora. Please contact your Platfora sales representative for details about obtaining a valid support agreement or with questions about your account. Copyright Notices Copyright Platfora Corporation. All rights reserved. Platfora believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. PLATFORA CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any Platfora software described in this publication requires an applicable software license. Platfora, You Should Know, Interest Driven Pipeline, Fractal Cache, and Adaptive Job Synthesis are trademarks of the Platfora Corporation. Apache Hadoop and Apache Hive are trademarks of the Apache Software Foundation. All other trademarks used herein are the property of their respective owners. Embedded Software Copyrights and License Agreements Platfora contains the following open source and third-party proprietary software subject to their respective copyrights and license agreements: Apache Hive PDK dom4j freemarker GeoNames Google Maps API javassist Page 6

7 Platfora Installation Guide - Introduction javax.servlet Mortbay Jetty OWASP CSRFGuard 3 PostgreSQL JDBC Scala sjsxp : Unboundid Page 7

8 Chapter 1 Installation Overview (On-Premise) This section provides an overview of the Platfora installation process for environments that will use an onpremise deployment of Hadoop with Platfora. Topics: On-Premise Hadoop Deployments Master vs Worker Node Installations Preinstall Check List High-Level Install Steps On-Premise Hadoop Deployments An on-premise Hadoop deployment means that you already have an existing Hadoop installation in your data center (either a physical data center or a virtual private cloud). Page 8

9 Platfora Installation Guide - Installation Overview (On-Premise) Platfora connects to the Hadoop cluster managed by your organization, and the majority of your organization's data is stored in the distributed file system of this primary Hadoop cluster. For on-premise Hadoop deployments, the Platfora servers should be on their own dedicated hardware co-located in the same data center as your Hadoop cluster. A data center can be a physical location with actual hardware resources, or a virtual private cloud environment with virtual server instances (such as Rackspace or Amazon EC2). Platfora recommends putting the Platfora servers on a network with at least 1 Gbps connectivity to the Hadoop nodes. Platfora users access the Platfora master node using an HTML5-compliant web browser. The Platfora master node accesses the HDFS NameNode and the MapReduce JobTracker or YARN Resource Manager using native Hadoop protocols. The Platfora worker nodes access the HDFS DataNodes directly. If using a firewall, Platfora recommends placing the Platfora servers on the same side of the firewall as your Hadoop cluster. Platfora software can run on a wide variety of server configurations on as little as one server or scale across multiple servers. Since Platfora runs best with all of the active lenses readily available in RAM, Platfora recommends obtaining servers optimized for higher RAM capacity and a minimum of 8 CPUs. Master vs Worker Node Installations If you are installing Platfora for the very first time, you begin by installing, configuring and initializing the Platfora master node. Once you have the master node up and running, you can then add in additional worker nodes as needed. Page 9

10 Platfora Installation Guide - Installation Overview (On-Premise) All nodes in a Platfora cluster (master and workers) must meet the minimum system requirements and have the required prerequisite software installed. If you are using the RPM installer packages, you can use the base installer package to install the required software on each Platfora node. If you are using the TAR installer packages, you must manually install the required software on each Platfora node. You only need to install the Platfora server software, however, on the master node. Platfora copies the server software from the master to the worker nodes during the worker node initialization process. All nodes in a Platfora cluster also require you to configure the network environment so that all the nodes can talk to each other, as well as to the Hadoop cluster nodes. If you are adding additional worker nodes to an existing Platfora cluster, make sure to follow the instructions for installing dependencies and configuring the environment. You can skip any tasks denoted as 'Master Only' - these tasks are only required for first-time installations of the Platfora master node. Preinstall Check List Here is a list of items and information you will need in order to install a new Platfora cluster with an onpremise Hadoop deployment. Platfora must be able to connect to Hadoop services during setup, so you will also need information from your Hadoop installation. Platfora Checklist This is a list of things you will need in order to install Platfora nodes. What You Need Platfora License Platfora Software (MapR Only) MapR Client Software Description Platfora Customer Support must issue you a license file. Trial period licenses are available upon request for pilot installations. A Platfora customer support representative can give you the download link to the Platfora installation package for your chosen operating system and Hadoop distribrution and version. Platfora provides both rpm and tar installer packages. If you are using a MapR Hadoop cluster with Platfora, you will need the MapR client software for the version of MapR you are using. The MapR client software must be installed on all Platfora nodes. Page 10

11 Platfora Installation Guide - Installation Overview (On-Premise) Hadoop Checklist This is a list of things you will need from your Hadoop environment in order to install Platfora. What You Need Hadoop Distribution and Version Number Hadoop Hostnames and Ports Hadoop Configuration Files Description When you install Platfora, you need to specify what Hadoop distribution you have (Cloudera, Hortonworks, MapR, etc.) and what version you are running. You will need to know the hostnames and ports of your Hadoop services (NameNode, Resource Manager or JobTracker, Hive Server, DataNodes, etc.) Platfora requires local versions of Hadoop's configuration files. It uses these files to connect to Hadoop services: core-site.xml and hdfs-site.xml for HDFS mapred-site.xml and yarn-site.xml for data processing hive-site.xml for the Hive metastore The locations of these files varies based on your Hadoop distribution. Platfora Data Directory Location in HDFS Platfora requires a directory location in HDFS to store its library files and output (lenses). High-Level Install Steps This section lists the high-level steps involved in installing Platfora to work with an on-premise Hadoop cluster. Note that there are different procedures if you are installing a new Platfora cluster verses adding a worker node to an existing Platfora cluster. New Platfora Installation When installing Platfora for the first time, you begin with installing and configuring the Platfora master node first. After the master node is installed, initialized and connected to the Hadoop services it needs, then you can use the master node to add additional worker nodes into the cluster. These are the high-level steps for installing Platfora for the first time: 1. Make sure your systems meet the minimum System Requirements. Page 11

12 Platfora Installation Guide - Installation Overview (On-Premise) 2..Configure Hadoop for Platfora Access. 3. Install Platfora Software and Dependencies. 4. Configure Environment on Platfora Nodes. 5. (Secure Hadoop Only) Configure Platfora for Secure Hadoop Access. 6. Obtain a Copy of Your Hadoop Configuration Files. 7. Configure Access to Your Hadoop Services. 8. Initialize the Platfora Master. 9. Start Platfora. 10.Login to the Platfora Application. 11.Install the License File. 12.(Optional) Load the Tutorial Data (as a quick way to test that everything works). 13.Add Worker Nodes. Additional Worker Node Installation Once you have a Platfora master node up and running, you can use it to initialize additional worker nodes. Before you can initialize a worker node, however, you must make sure that it has the required dependencies installed. These are the high-level steps for adding a worker node to an existing Platfora cluster: 1. Install the prerequisite software only directly on the worker node. If using the RPM installer packages, Install Dependencies RPM Package. If using the TAR installer packages, you must manually Create the Platfora System User, Set OS Kernel Parameters, and Install Dependent Software. 2. Configure Environment on Platfora Nodes. 3. (Secure Hadoop Only) Configure Kerberos Authentication to Hadoop. 4. Add Worker Node to Platfora Cluster. Page 12

13 Chapter 2 System Requirements (On-Premise) The Platfora software runs on a scale-out cluster of servers. You can install Platfora on a single node to start, and then scale up storage and processing capacity by adding additional nodes. Platfora requires access to an existing, compatible Hadoop implementation in order to start. Users then access the Platfora application using a compatible web browser client. This section describes the system requirements for on-premise deployments of the Platfora servers, Hadoop source systems, network connectivity, and web browser clients. Topics: Platfora Server Requirements Port Configuration Requirements Supported Hadoop and Hive Versions Hadoop Resource Requirements Browser Requirements Platfora Server Requirements Platfora recommends the following minimum system requirements for Platfora servers. For multi-node installations, the master server and all worker servers must be the same operating system (OS) and system configuration (same amount of memory, CPU, etc.). 64-bit Operating System or Amazon Machine Image (AMIs) CentOS (7.0 is not supported) RHEL (7.0 is not supported) Scientific Linux 6.2 Amazon Linux AMI Oracle Enterprise Linux 6.x Ubuntu LTS or higher Security-Enhanced Linux If you wish to install Security-Enhanced Linux, refer to Platfora's Support site for installation instructions. Page 13

14 Platfora Installation Guide - System Requirements (On-Premise) Software Java 1.7 Python 2.6.8, 2.7.1, through (3.0 not supported) PostgreSQL , 9.2.5, or 9.3 (master only) OpenSSL or higher 2 Unix Utilities Memory rsync, ssh, scp, cp, tar, tail, sysctl, ntp, wget 64 GB minimum, 256 recommended The server needs enough memory to accommodate actively used lens data. Additionally, it needs 1-2 GB reserved for normal operations and the lens query engine workspace. CPU Disk Network 8 cores minimum, 16 recommended All Platfora nodes (master or worker) require 300MB for the Platfora installation. Every node requires high-speed local storage and a local disk cache configured as a single logical volume. Hardware RAID is recommended for the best performance. All nodes combined require appropriate free space for aggregated data structures (Platfora lenses). At a minimum, you will need twice the amount of disk space as the amount of system memory. The Platfora master node requires an additional, approximately 700 MB for metadata catalog (dataset definitions, vizboard and visualization definitions, lens definitions, etc.) 1 Gbps reliable network connectivity between Platfora master server and query processing servers 1 Gbps reliable network connectivity between Platfora master server and Hadoop NameNode and JobTracker/ResourceManager node Network bandwidth should be comparable to the amount of memory on the Platfora master server Port Configuration Requirements You must open ports in the firewall of your Platfora nodes to allow client access and intra-cluster communications. You also must open ports within your Hadoop cluster to allow access from Platfora. This section lists the default ports required. 2 Only required if you want to enable SSL for secure communications between Platfora servers Page 14

15 Platfora Installation Guide - System Requirements (On-Premise) Ports to Open on Platfora Nodes Your Platfora master node must allow HTTP connections from your user network. All nodes must allow connections from the other Platfora nodes in a multi-node cluster. On Amazon EC2 instances, you must configure the port firewall rules on the Platfora server instances in addition to the EC2 Security Group Settings. Platfora Service Default Port Allow connections from Master Web Services Port (HTTP) Secure Master Web Services Port (HTTPS) Master Server Management Port Worker Server Management Port 8001 External user network Platfora worker servers localhost 8443 External user network Platfora worker servers localhost 8002 Platfora worker servers localhost 8002 Platfora master server other Platfora worker servers localhost Master Data Port 8003 Platfora worker servers localhost Worker Data Port 8003 Platfora master server other Platfora worker servers localhost Master PostgreSQL Database Port 5432 Platfora worker servers localhost Ports to Open on Hadoop Nodes Platfora must be able to access certain services of your Hadoop cluster. This section lists the Hadoop services Platfora needs to access and the default ports for those services. Page 15

16 Platfora Installation Guide - System Requirements (On-Premise) Note that this only applies to on-premise Hadoop deployments or to self-managed Hadoop deployments in a virtual private cloud, not to Amazon Elastic MapReduce (EMR). Hadoop Service Default Ports by Hadoop Distro Allow connections from CDH, HDP, Pivotal Apache Hadoop MapR HDFS NameNode N/A Platfora master and worker servers HDFS DataNodes N/A Platfora master and worker servers MapRFS CLDB N/A N/A 7222 Platfora master and worker servers MapRFS DataNodes N/A N/A 5660 Platfora master and worker servers MRv1 JobTracker Platfora master server MRv1 JobTracker Web UI YARN ResourceManager YARN ResourceManager Web UI YARN Job History Server YARN Job History Server Web UI HiveServer Thrift Port External user network (optional) Platfora master server External user network (optional) Platfora master server External user network (optional) Platfora master server Hive Metastore DB 9083 Port (HDP2) N/A 9083 Platfora master server 3 If connecting to Hive directly using JDBC Page 16

17 Platfora Installation Guide - System Requirements (On-Premise) Supported Hadoop and Hive Versions This section lists the Hadoop distributions and versions that are compatible with the Platfora installation packages. If using Hive as a data source for Platfora, the version of Hive must be compatible with the version of Hadoop you are using. Hadoop Version Hive M/R Platfora Package Distro Version Version Cloudera 5 CDH YARN cdh5 CDH YARN cdh5 CDH YARN cdh52 CDH YARN cdh52 CDH YARN cdh54 Hortonworks HDP 2.1.x YARN hadoop_2_4_0_hive_0_13_0 HDP 2.2.x YARN hadoop_2_6_0_hive_0_14_0 MapR MapR YARN mapr4 MapR YARN mapr402 MapR YARN mapr402 Pivotal Labs PivotalHD YARN hadoop_2_6_0_hive_0_14_0 Amazon EMR (AMI 3.7.x) Hadoop YARN hadoop_2_4_0_hive_0_13_0 Hadoop Resource Requirements Platfora must be able to connect to an existing Hadoop installation. Platfora also requires permissions and resources in the Hadoop source system. This section describes the Hadoop resource requirements for Platfora. Platfora uses the remote Distributed File System (DFS) of the Hadoop cluster for persistent storage and as the primary data source. Optionally, you can also configure Platfora to use a Hive metastore server as a data source. Page 17

18 Platfora Installation Guide - System Requirements (On-Premise) Platfora uses the Hadoop MapReduce services to process data and build lenses. For larger lens builds to succeed, Platfora requires minimum resources on the Hadoop cluster for MapReduce tasks. DFS Disk Space DFS Permissions MapReduce Permissions Platfora requires a designated persistent storage directory in the remote distributed file system (DFS) with appropriate free space for Platfora system files and data structures (lenses). The location is configurable. The platfora system user needs read permissions to source data directories and files. The platfora system user needs write permissions to Platfora's persistent storage directory on DFS. The platfora system user needs to be added to the submit-jobs and administer-jobs access control list (or added to a group that has these permissions). DFS Resources Minimum Open File Limit = 5000 MapReduce Resources Minimum Memory for Task Processes = 1 GB Browser Requirements Users can connect to the Platfora web application using the latest HTML5-compliant web browsers. Platfora supports the latest releases of the following web browsers: Chrome (preferred browser) Firefox Safari Internet Explorer with the Compatibility View feature disabled (versions prior to IE 10 are not supported) Platfora supports these web browsers on desktop machines only. Page 18

19 Chapter 3 Configure Hadoop for Platfora Access Before initializing and starting Platfora for the first time, you must make sure that Platfora can connect to Hadoop and access the directories and services it needs. The tasks in this section are performed in your Hadoop environment, and apply to on-premise Hadoop installations only (not to Amazon EMR). Topics: Create Platfora User on Hadoop Nodes Create Platfora Directories and Permissions in Hadoop HDFS Tuning for Platfora MapReduce Tuning for Platfora YARN Tuning for Platfora Create Platfora User on Hadoop Nodes Platfora requires a platfora system user account on each node in your Hadoop cluster. The Platfora server uses this system user account to submit jobs to the Hadoop cluster and to access the necessary files and directories in the Hadoop distributed file system (HDFS). Creating a system user requires root or sudo permissions. 1. Create the platfora user: $ sudo useradd -s /bin/bash -m -d /home/platfora platfora 2. Set a password for the platfora user: $ sudo passwd platfora Create Platfora Directories and Permissions in Hadoop Platfora requires read and write permissions to a designated directory in the Hadoop file system where it can store its metadata and MapReduce output. Platfora connects to HDFS as the platfora user and also runs its MapReduce jobs as this same user. Page 19

20 Platfora Installation Guide - Configure Hadoop for Platfora Access Create a data directory for Platfora and set the platfora system user as its owner. In the example below, the Hadoop file system has a user called hdfs, the directory is called /platfora and the Platfora server is running as the platfora system user: $ sudo -u hdfs hadoop fs -mkdir /platfora $ sudo -u hdfs hadoop fs -chown platfora /platfora $ sudo -u hdfs hadoop fs -chmod 711 /platfora Note that for MapR, run the command as the mapr user: $ sudo -u mapr hadoop fs -mkdir /platfora $ sudo -u mapr hadoop fs -chown platfora /platfora $ sudo -u mapr hadoop fs -chmod 711 /platfora The platfora system user needs access to the location where MapReduce writes its staging files. Depending on your Hadoop distribution, the location of the staging area is different. In Cloudera, MapR, Pivotal, and Hortonworks, the default location is /user/username. In Apache, the location is / tmp/xxx/xxx/username. Make sure this location exists and is writable by the platfora system user. For example, on Cloudera: $ sudo -u hdfs hadoop fs -mkdir /user/platfora $ sudo -u hdfs hadoop fs -chown platfora /user/platfora For example, on MapR: $ sudo -u mapr hadoop fs -mkdir /user/platfora $ sudo -u mapr hadoop fs -chown platfora /user/platfora During lens build processing, the platfora system user needs to be able to write to the intermediate and log directories on the Hadoop nodes. Check the following Hadoop configuration properties and make sure the specified locations exist in HDFS and are writable by the platfora system user. Property Hadoop Configuration File Description mapreduce.cluster.local.dir mapred-site.xml Tells the MapReduce servers where to store intermediate files for a job. mapreduce.jobtracker.system.dir mapred-site.xml The directory where MapReduce stores control files. mapreduce.cluster.temp.dir mapred-site.xml A shared directory for temporary files. Page 20

21 Platfora Installation Guide - Configure Hadoop for Platfora Access Property Hadoop Configuration File Description mapr.centrallog.dir (MapR Only) mapred-site.xml The central job log directory for MapR Hadoop. The platfora system user also needs to be added to the submit-jobs and administer-jobs access control lists (or added to a group that has these permissions). The platfora system user also needs read permissions to the source data directories and files that you want to analyze in Platfora. HDFS Tuning for Platfora Platfora opens files on the Hadoop NameNode and DataNodes as it does its work to build the lens. This section describes how to ensure your Hadoop cluster has file limits that support lens build operations. Increase Open File Limits Platfora opens files on the Hadoop NameNode and DataNodes as it builds the lens. For multiple lens builds or for lenses that have a lot of fields selected, a lens build can cause your Hadoop nodes to exceed the maximum open file limit. When this limit is exceeded, Platfora lens builds will fail with a "Too many open files..." exception. Linux operating systems limit the number of open files and connections a process can have. This prevents one application from slowing down the entire system by requesting too many file handlers. When an application exceeds the limit, the operating system prevents the application from requesting more file handlers, causing the process to fail with a "Too many open files..." error. Verify your file limits are adequate on each Hadoop node. Increase the limits on your Hadoop nodes where the limts are too low. There are two places file limits are set in the Linux operating system: A global limit for the entire system (set in /etc/sysctl.conf) A per-user process limit (set in /etc/security/limits.conf) You can check the global limit by running the command: $ cat /proc/sys/fs/file-nr This should return a set of three numbers like this: The first number is the number of currently opened file descriptors. The second number is the number of allocated file descriptors. The third number is the maximum number of file descriptors for the whole system. The maximum should be at least Page 21

22 Platfora Installation Guide - Configure Hadoop for Platfora Access To increase the global limit, edit /etc/sysctl.conf (as root) and set the property: fs.file-max = Increase Platfora User Limits You can check the per-user process limit by running the command: $ ulimit -n This should return the file limit for the currently logged in user, for example: 1024 This limit should be at least 5000 for the platfora system user (or whatever user runs Platfora lens build jobs). To increase the limit, edit /etc/security/limits.conf (as root) and add the following lines (the * increases the limit for all system users): * hard nofile * soft nofile root hard nofile root soft nofile You must reboot the server whenever you change OS kernel settings. Increase DataNode File Limits A Hadoop HDFS DataNode has an upper bound on the number of files that it can serve at any one time. In your Hadoop configuration, make sure the DataNodes are tuned to have an upper bound of at least 5000 by setting the following properties in the hdfs-site.xml file (located in the conf directory on your Hadoop NameNode): Framework hdfs-site.xml Property Minimum Value MapReduce v1 dfs.datanode.max.xcievers 5000 YARN dfs.datanode.max.transfer.threads 5000 Allow Platfora Local Access If the platfora system user is not able to make HDFS calls during lens build processing, lens build jobs in Platfora will stall at 0% progress. To prevent this, make sure your hdfs-site.xml files contain the dfs.block.local-path-acess.user parameter and that its value includes the platfora system user. For example: <property> <name>dfs.block.local-path-access.user</name> <value>gpadmin,hdfs,mapred,yarn,hbase,hive,platfora</value> </property> Page 22