Cloudera Manager Administration Guide

Transcription

1 Cloudera Manager Administration Guide

2 Important Notice (c) Cloudera, Inc. All rights reserved. Cloudera, the Cloudera logo, Cloudera Impala, and any other product or service names or slogans contained in this document are trademarks of Cloudera and its suppliers or licensors, and may not be copied, imitated or used, in whole or in part, without the prior written permission of Cloudera or the applicable trademark holder. Hadoop and the Hadoop elephant logo are trademarks of the Apache Software Foundation. All other trademarks, registered trademarks, product names and company names or logos mentioned in this document are the property of their respective owners. Reference to any products, services, processes or other information, by trade name, trademark, manufacturer, supplier or otherwise does not constitute or imply endorsement, sponsorship or recommendation thereof by us. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Cloudera. Cloudera may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Cloudera, the furnishing of this document does not give you any license to these patents, trademarks copyrights, or other intellectual property. For information about patents covering Cloudera products, see The information in this document is subject to change without notice. Cloudera shall not be liable for any damages resulting from technical errors or omissions which may be present in this document, or from use of this document. Cloudera, Inc Page Mill Road Bldg 2 Palo Alto, CA [email protected] US: Intl: Release Information Version: 5.1.x Date: September 8, 2015

3 Table of Contents About this Guide...7 Managing the Cloudera Manager Server and Agents...9 Starting, Stopping, and Restarting the Cloudera Manager Server...9 Configuring Cloudera Manager Server Ports...9 Moving the Cloudera Manager Server to a New Host...9 Starting, Stopping, and Restarting Cloudera Manager Agents...10 Configuring Cloudera Manager Agents...11 Viewing Cloudera Manager Server and Agent Logs...14 Changing Hostnames...14 Backing up Databases...16 Backing Up PostgreSQL Databases...17 Backing Up MySQL Databases...17 Backing Up Oracle Databases...17 Cloudera Management Service...19 Managing Users and Authentication...23 Cloudera Manager User Accounts...23 Changing the Logged-In Internal User Password...24 Adding an Internal User Account...24 Assigning User Roles...24 Changing an Internal User Account Password...24 Deleting Internal User Accounts...24 Configuring External Authentication...25 Configuring Authentication Using Active Directory...25 Configuring Authentication Using an OpenLDAP-compatible Server...25 Configuring Authentication Using an External Program...27 Configuring Authentication Using SAML...27 Configuring TLS Security for Cloudera Manager...31 Configuring TLS Encryption only for Cloudera Manager...31 Step 1: Create a Cloudera Manager Server certificate Step 2: Enable TLS encryption and specify Server keystore properties...32

4 Step 3: Enable and configure TLS on the Agent hosts...32 Step 4: Restart the Cloudera Manager Server...32 Step 5: Restart the Cloudera Manager Agents...33 Step 6: Verify that the Server and Agents are communicating...33 Configuring TLS Authentication of Server to Agents...33 Step 1: Configure TLS encryption...33 Step 2: Provide the Server's server certificate and CA certificate...33 Step 3: Copy the Server's server.pem file to the Agents...36 Step 4: Enable TLS Encryption in Cloudera Manager...36 Step 5: Restart the Cloudera Manager Server...36 Step 6: Restart the Cloudera Manager Agents...36 Step 7: Verify that the Server and Agents are communicating...36 Configuring TLS Authentication of Agents to Server...37 Step 1: Configure TLS encryption...37 Step 2: Configure TLS Authentication of Server to Agents...37 Step 3. Generate the private key for the Agent using openssl...37 Step 4: Generate a certificate for the agent...37 Step 5: Create a file that contains the password for the key...37 Step 6: Configure the Agent with its private key and certificate...37 Step 7: Import the Agent's certificate into the Server's truststore...38 Step 8: Repeat steps 3 through 7 for every agent in your cluster...38 Step 9: Enable Agent authentication and configure the Server to use the new truststore...38 Step 10: Restart the Server...38 Step 11: Restart the Cloudera Manager Agents...38 Step 12: Verify that the Server and Agents are communicating...38 Configuring TLS Encryption for Cloudera Manager Admin Console...38 Step 1: Create a Cloudera Manager Server certificate Step 2: Enable TLS encryption and specify Server keystore properties...39 Step 3: Restart the Cloudera Manager Server...39 Step 4: Restart the Cloudera Management Services...40 Step 5: Verify that the Server and browser are using TLS to communicate...40 HTTPS Communication in Cloudera Manager...40 Cloudera Manager Agent...40 Cloudera Management Services...41 Upgrading Cloudera Manager...43 Understanding Upgrades...43 Upgrading Cloudera Manager...43 Upgrading CDH...44 Database Considerations for Cloudera Manager Upgrades...44 Back up Databases...44 Modify Databases to Support UTF Modify Databases to Support Appropriate Maximum Connections...45

5 Next Steps...46 Upgrading Cloudera Manager 5 to the Latest Cloudera Manager...46 Review Warning...46 Perform Prerequisite Steps...47 Stop Selected Services and Roles...47 Stop Cloudera Manager Server, Database, and Agent...47 (Optional) Upgrade JDK on Cloudera Manager Server Host and Agent Hosts...48 Upgrade Cloudera Manager Server Packages...48 Start the Cloudera Manager Server...50 Upgrade Cloudera Manager Agent Packages...50 Verify the Upgrade Succeeded...52 (Optional) Configure SSL for Cloudera Management Service...52 Deploy JDK Upgrade...53 Start Selected Services and Roles...53 Deploy Updated Client Configurations...53 Test the Installation...54 (Optional) Upgrade CDH...54 Upgrading Cloudera Manager 4 to Cloudera Manager Review Warnings and Notes...55 Perform Prerequisite Steps...56 Stop Selected Services...57 Stop Cloudera Manager Server, Database, and Agent...57 (Optional) Upgrade JDK on Cloudera Manager Server Host and Agent Hosts...58 Upgrade Cloudera Manager Server Packages...58 Start the Cloudera Manager Server...60 Upgrade Cloudera Manager Agent Packages...60 Verify the Upgrade Succeeded...63 Add Hive Gateway Roles...63 Configure Cluster Version for Package Installs...63 Upgrade Impala...64 (Optional) Configure SSL for Cloudera Management Service...64 (Optional) Deploy Cloudera Manager Agent Upgrade...64 Deploy JDK Upgrade...64 (Optional) Deploy Monitoring Upgrade...65 Start Selected Services and Roles...65 Deploy Updated Client Configurations...65 Test the Installation...66 (Optional) Upgrade CDH...66 Upgrading Cloudera Manager 3.7.x...66 Re-Running the Cloudera Manager Upgrade Wizard...66 Reverting a Failed Cloudera Manager Upgrade...67 Reinstall the Cloudera Manager Server Packages...67 Start the Server...69

6 Other Cloudera Manager Settings...71 Administration Settings...71 User Interface Language Settings...72 Managing Licenses...72 Managing Alerts...75 Configuring Alert Delivery...75 Configuring Alert SNMP Delivery...76 Kerberos...77 Sending Usage and Diagnostic Data to Cloudera...78 Configuring a Proxy Server...78 Managing Anonymous Usage Data Collection...78 Managing Hue Analytics Data Collection...79 Diagnostic Data Collection...79 Configuring Network Settings...81 Importing Cloudera Manager Settings...82 Backing up your Current Deployment...82 Building a Cloudera Manager Deployment...82 Uploading a Cloudera Manager 4.x Configuration Script...82

7 About this Guide About this Guide This guide is for system administrators who need to manage a Cloudera Manager server installation. This guide covers managing the Cloudera Manager Server and Agents, managing the Cloudera Management Service, adding and managing Cloudera Manager users, configuring TLS security, upgrading Cloudera Manager, adding or upgrading licenses, configuring the Alert Publisher, and other similar features. Cloudera Manager Administration Guide 7

8

9 Managing the Cloudera Manager Server and Agents Managing the Cloudera Manager Server and Agents This section covers information on managing the Cloudera Manager Server and Agents that run on each host of the cluster. Starting, Stopping, and Restarting the Cloudera Manager Server To start the Cloudera Manager Server: $ sudo service cloudera-scm-server start You can stop (for example, to perform maintenance on its host) or restart the Cloudera Manager Server without affecting the other services running on your cluster. Statistics data used by activity monitoring and service monitoring will continue to be collected during the time the server is down. To stop the Cloudera Manager Server: $ sudo service cloudera-scm-server stop To restart the Cloudera Manager Server: $ sudo service cloudera-scm-server restart Configuring Cloudera Manager Server Ports Required Role: 1. Select Administration > Settings. 2. Under the Ports and Addresses category, set the following options as described below: Setting HTTP Port for Admin Console HTTPS Port for Admin Console Agent Port to connect to Server Description Specify the HTTP port to use to access the Server via the Admin Console. Specify the HTTPS port to use to access the Server via the Admin Console. Specify the port for Agents to use to connect to the Server. 3. Click Save Changes. 4. Restart the Cloudera Manager Server. Moving the Cloudera Manager Server to a New Host You can move the Cloudera Manager Server if the database information is still available for either of the following reasons: The database server is still available. A current back up of the Cloudera Manager database is available. To move Cloudera Manager Server: Cloudera Manager Administration Guide 9

10 Managing the Cloudera Manager Server and Agents 1. Record the old host's name hostname and IP address. It is not absolutely necessary to have the old Cloudera Manager server hostname and IP address, but it simplifies the process. You could use a new hostname and IP address, but this would require updating the configuration of every Agent to use this new information. Because it is easier to use the old server hostname and address in most cases, using a new hostname and IP address is not described. 2. Identify a new host on which to install Cloudera Manager. Assign the failed Cloudera Manager Server's hostname and IP address to the new host. Note: If the Agents were configured with the server's hostname, you do not need to assign the old host's IP address to the new host. Simply assigning the hostname will suffice. 3. Install Cloudera Manager on a new host, using the method described under Install the Cloudera Manager Server Packages. Do not install the other components, such as CDH and databases. 4. If the database server is not available, a. Install the database packages on the host that will host the restored database. This could be the same host on which you have just installed Cloudera Manager or it could be a different host. The details of which package to install varies based on which database was initially installed on your system. If you used the embedded PostgreSQL database, install the PostgreSQL package as described in Embedded PostgreSQL Database. If you used an external MySQL, PostgreSQL, or Oracle database, reinstall that following the instructions in Cloudera Manager and Managed Service Databases. b. Restore the backed up databases to the new database installations. 5. Update /etc/cloudera-scm-server/db.properties with the necessary information so that the Cloudera Manager Server connects to the restored database. This information is typically the database name, database instance name, user name, and password. 6. Start the Cloudera Manager Server. At this point, Cloudera Manager should resume functioning as it did before the failure. Because you restored the database from the backup, the server should accept the running state of the Agents, meaning it will not terminate any running processes. The process is similar with secure clusters, though files in /etc/cloudera-scm-server must be restored in addition to the database. Starting, Stopping, and Restarting Cloudera Manager Agents Starting Agents To start Agents, the supervisord process, and all managed service processes, use one of the following commands: Start $ sudo service cloudera-scm-agent start Clean Start $ sudo service cloudera-scm-agent clean_start The directory /var/run/cloudera-scm-agent is completely cleaned out; all files and subdirectories are removed, and then the start command is executed. /var/run/cloudera-scm-agent contains on-disk running Agent state. Some Agent state is left behind in /var/lib/cloudera-scm-agent, but you shouldn't delete that. For further information, see Server and Client Configuration and Process Management. Stopping and Restarting Agents To stop or restart Agents while leaving the managed processes running, use one of the following commands: 10 Cloudera Manager Administration Guide

11 Managing the Cloudera Manager Server and Agents Stop $ sudo service cloudera-scm-agent stop Restart $ sudo service cloudera-scm-agent restart Hard Stopping and Restarting Agents Warning: The hard_stop, clean_restart, or hard_restart commands kill all running managed service processes on the host(s) where the command is run. To stop or restart Agents, the supervisord process, and all managed service processes, use one of the following commands: Hard Stop $ sudo service cloudera-scm-agent hard_stop Hard Restart $ sudo service cloudera-scm-agent hard_restart Hard restart is useful for the following situations: 1. You're upgrading Cloudera Manager and the supervisord code has changed between your current version and the new one. To properly do this upgrade you'll need to restart supervisor too. 2. supervisord is hung and needs to be restarted. 3. You want to clear out all running state pertaining to Cloudera Manager and managed services. Clean Restart $ sudo service cloudera-scm-agent clean_restart Runs hard_stop followed by clean_start. Checking Agent Status To check the status of the Agent process, use the command: $ sudo service cloudera-scm-agent status Configuring Cloudera Manager Agents Required Role: Cloudera Manager Agents can be configured globally using properties you set in the Cloudera Manager Admin Console and by setting properties in individual Agent configuration files. Configuring Agent Heartbeat and Health Status Options You can configure the Cloudera Manager Agent heartbeat interval and timeouts to trigger changes in Agent health as follows: 1. Select Administration > Settings. Cloudera Manager Administration Guide 11

12 Managing the Cloudera Manager Server and Agents 2. Under the Performance category, set the following option: Property Send Agent Heartbeat Every Description The interval in seconds between each heartbeat that is sent from Cloudera Manager Agents to the Cloudera Manager Server. Default: 15 sec. 3. Under the Monitoring category, set the following options: Property Set health status to Concerning if the Agent heartbeats fail Set health status to Bad if the Agent heartbeats fail Description The number of missed consecutive heartbeats after which a Concerning health status is assigned to that Agent. Default: 5. The number of missed consecutive heartbeats after which a Bad health status is assigned to that Agent. Default: Click Save Changes. Configuring the Host Parcel Directory To configure the location of distributed parcels: 1. Click Hosts in the top navigation bar. 2. Click the Configuration tab. 3. Configure the value of the Parcel Directory property. The setting of the parcel_dir property in the Cloudera Manager Agent configuration file overrides this setting. 4. Click Save Changes to commit the changes. Agent Configuration File The Cloudera Manager Agent supports different types of configuration options in the /etc/cloudera-scm-agent/config.ini file. You must update the configuration on each host. Property server_host, server_port, listening_port, listening_hostname, listening_ip Description Hostname and ports of the Cloudera Manager Server and Agent and IP address of the Agent. Also see Configuring Cloudera Manager Server Ports on page 9 and Ports Used by Cloudera Manager. The Cloudera Manager Agent configures its hostname automatically. However, if your cluster hosts are multi-homed (that is, they have more than one hostname), and you want to specify which hostname the Cloudera Manager Agent uses, you can update the listening_hostname property. If you want to specify which IP address the Cloudera Manager Agent uses, you can update the listening_ip property in the same file. To have a CNAME used throughout instead of the regular hostname, an Agent can be configured to use listening_hostname=cname. In this case, the CNAME should resolve to the same IP address as the IP address of the hostname on that machine. Users doing this will find that the host inspector will report problems, but the CNAME will be used in all configurations where that's appropriate. This practice is particularly useful for users who would like clients to use 12 Cloudera Manager Administration Guide

13 Managing the Cloudera Manager Server and Agents Property log_file lib_dir parcel_dir max_collection_wait_seconds metrics_url_timeout_seconds task_metrics_timeout_seconds use_tls,verify_cert_file, client_key_file, client_keypw_file, client_cert_file mgmt_home Description namenode.mycluster.company.com instead of machine1234.mycluster.company.com. In this case, namenode.mycluster would be a CNAME for machine1234.mycluster, and the generated client configurations (and internal configurations as well) would use the CNAME. The path to the Agent log file. If the Agent is being started via the init.d script, /var/log/cloudera-scm-agent/cloudera-scm-agent.out will also have a small amount of output (from before logging is initialized). Default: /var/log/cloudera-scm-agent/cloudera-scm-agent.log. Directory to store Cloudera Manager Agent state that persists across instances of the agent process and system reboots. The agent's UUID is stored here. Default: /var/lib/cloudera-scm-agent. Directory to store unpacked parcels. Default: /opt/cloudera/parcels. Maximum time to wait for all metric collectors to finish collecting data. Default: 10 sec. Maximum time to wait when connecting to a local role's web server to fetch metrics. Default: 30 sec. Maximum time to wait when connecting to a local TaskTracker to fetch task attempt data. Default: 5 sec. Security-related configuration. See Configuring TLS Authentication of Agents to Server on page 37 Configuring TLS Authentication of Server to Agents on page 33 Specifying the Cloudera Manager Server Certificate Adding a Host to the Cluster Directory to store Cloudera Management Service files. Default: /usr/share/cmf. cloudera_mysql_connector_jar, cloudera_oracle_connector_jar, cloudera_postgresql_jdbc_jar Location of JDBC drivers. See Cloudera Manager and Managed Service Databases. Default: MySQL - /usr/share/java/mysql-connector-java.jar Oracle - /usr/share/java/oracle-connector-java.jar PostgreSQL - /usr/share/cmf/lib/postgresql-version-build.jdbc4.jar Cloudera Manager Administration Guide 13

14 Managing the Cloudera Manager Server and Agents Viewing Cloudera Manager Server and Agent Logs To help you troubleshoot problems, you can view the Cloudera Manager Server and Agent logs. You can view these logs in the Logs page or in specific pages for the logs. Viewing Cloudera Manager Server and Agent Logs in the Logs Page 1. Select Diagnostics > Logs on the top navigation bar. 2. Click Select Sources to display the log source list. 3. Uncheck the All Sources checkbox. 4. Check the Cloudera Manager checkbox to view both Agent and Server logs, or click to the left of Cloudera Manager, and check either the Agent or Server checkbox. 5. Click Search. For more information about the Logs page, see Logs. Viewing the Cloudera Manager Server Log 1. Select Diagnostics > Server Log on the top navigation bar. Note: You can also view the Cloudera Manager Server log at /var/log/cloudera-scm-server/cloudera-scm-server.log on the server host. Viewing the Cloudera Manager Agent Log 1. Click the Hosts tab. 2. Click the link for the host where you want to see the Agent log. 3. In the Details panel, click the Details link in the Host Agent field. 4. Click the Agent Log link. Note: You can also view the Cloudera Manager Agent log at /var/log/cloudera-scm-agent/cloudera-scm-agent.log on the Agent hosts. Changing Hostnames Required Role: Important: The process described here requires Cloudera Manager and cluster downtime. After you have installed Cloudera Manager and created a cluster, you may need to update the names of the hosts running the Cloudera Manager Server or cluster services. To update a deployment with new hostnames, follow these steps: 1. Verify if SSL/TLS certificates have been issued for any of the services and make sure to create new SSL/TLS certificates in advance for services protected by TLS/SSL. Review Cloudera Manager and CDH documentation at Cloudera Documentation. Tip: Search for SSL and TLS in the documentation. 2. Export the Cloudera Manager configuration using one of the following methods: Open a browser and go to this URL Save the displayed configuration. 14 Cloudera Manager Administration Guide

15 Managing the Cloudera Manager Server and Agents From terminal type: $ curl -u admin:admin > cme-cm-export.json If Cloudera Manager SSL is in use, specify the -k switch: $ curl -k -u admin:admin > cme-cm-export.json where cm_hostname is the name of the Cloudera Manager host and api_version is the correct version of the API for the version of Cloudera Manager you are using. For example, 3. Stop all services on the cluster. 4. Stop the Cloudera Management Service. 5. Stop the Cloudera Manager Server. 6. Stop the Cloudera Manager Agents on the hosts that will be having the hostname changed. 7. Back up the Cloudera Manager Server database using mysqldump, pg_dump, or another preferred backup utility. Store the backup in a safe location. 8. Update names and principals: a. Update the target hosts using standard per-os/name service methods (/etc/hosts, dns, /etc/sysconfig/network, hostname, and so on). Ensure that you remove the old hostname. b. If you are changing the hostname of the host running Cloudera Manager Server do the following: a. Change the hostname per step 8.a. b. Update the Cloudera Manager hostname in /etc/cloudera-scm-agent/config.ini on all Agents. c. If the cluster is configured for Kerberos security, do the following: a. In the Cloudera Manager database, set the merged_keytab value: PostgreSQL MySQL update roles set merged_keytab=null; update ROLES set MERGED_KEYTAB=NULL; b. Remove old hostname cluster service principals from the KDC database using one of the following: Use the delprinc command within kadmin.local interactive shell. From the command line: kadmin.local -q "listprincs" grep -E "(HTTP hbase hdfs hive httpfs hue impala mapred solr oozie yarn zookeeper)[^/]*/ [^/]*@" > cluster-princ.txt Open cluster-princ.txt and remove any non-cluster service principal entries within it. Make sure that the default krbtgt and other principals you created, or were created by Kerberos by default, are not removed by running the following: for i in `cat cluster-princ.txt`; do yes yes kadmin.local -q "delprinc $i"; done. c. Start the Cloudera Manager database and Cloudera Manager Server. d. Start the Cloudera Manager Agents on the newly renamed hosts. The Agents should show a current heartbeat in Cloudera Manager. e. Within the Cloudera Manager Admin Console recreate all the principals based on the new hostnames: a. Select Administration > Kerberos. Cloudera Manager Administration Guide 15

16 Managing the Cloudera Manager Server and Agents b. Do one of the following: If there are no principals listed, click the Generate Principals button. If there are principals listed, click the top checkbox to select all principals and click the Regenerate button. 9. If one of the hosts that was renamed has a NameNode configured with High Availability and automatic failover enabled, reconfigure the ZooKeeper failover controller znodes to reflect the new hostname. Warning: Do not perform this step if you are also running JobTracker in a High Availability configuration, as clearing the hadoop-ha znode will negatively impact JobTracker HA. All other services, and most importantly HDFS, should not be running. a. Start ZooKeeper services. Note: Make sure the ZooKeeper Failover Controller role is stopped within the HDFS service; start only the ZooKeeper Server role instances. b. On one of the hosts that has a ZooKeeper Server role, log into the Zookeeper CLI to delete the Nameservice znode: On a package-based installation zkcli.sh is found at: /usr/lib/zookeeper/bin/zkcli.sh On a parcel-based installation zkcli.sh is found at: $/opt/cloudera/parcels/cdh/lib/zookeeper/bin/zkcli.sh a. Verify that the HA znode exists: zkcli$ ls /hadoop-ha b. Delete the old znode: zkcli$ rmr /hadoop-ha/nameservice1 c. In the Cloudera Manager Admin Console, go to the HDFS service. d. Click the Instances tab. e. Select Actions > Initialize High Availability State in ZooKeeper For each of the Cloudera Management Service roles (Host Monitor, Service Monitor, Reports Manager, Activity Monitor, Navigator) go to their configuration and update the Database Hostname property. 11. Start all cluster services. 12. Start the Cloudera Management Service. 13. Deploy client configurations. Backing up Databases Cloudera recommends that you periodically back up the databases that Cloudera Manager uses to store configuration, monitoring, and reporting data and for managed services that require a database: Cloudera Manager - Contains all the information about what services you have configured, their role assignments, all configuration history, commands, users, and running processes. This is a relatively small database (<100 MB), and is the most important to back up. A monitoring database contains monitoring information about service and host status. In large clusters, this database can grow large. Activity Monitor - Contains information about past activities. In large clusters, this database can grow large. Reports Manager - Keeps track of disk utilization and processing activities over time. Medium-sized. Hive Metastore - Contains Hive metadata. Relatively small. Sentry Server - Contains authorization metadata. Relatively small. Cloudera Navigator Audit Server - Contains auditing information. In large clusters, this database can grow large. 16 Cloudera Manager Administration Guide

17 Backing Up PostgreSQL Databases The procedure for backing up a PostgreSQL database is the same whether you are using an embedded or external database: 1. Log in to the host where the Cloudera Manager Server is installed. 2. Run the following command as root: cat /etc/cloudera-scm-server/db.properties. The db.properties file contains: # Auto-generated by scm_prepare_database.sh # Mon Jul 27 22:36:36 PDT 2011 com.cloudera.cmf.db.type=postgresql com.cloudera.cmf.db.host=host:7432 com.cloudera.cmf.db.name=scm com.cloudera.cmf.db.user=scm com.cloudera.cmf.db.password=nnyfwijlbk 3. Run the following command as root using the parameters from the preceding step: # pg_dump -h host -p U scm > /tmp/scm_server_db_backup.$(date +%Y%m%d) 4. Enter the password specified for the com.cloudera.cmf.db.password property on the last line of the db.properties file. If you are using the embedded database, Cloudera Manager generated the password for you during installation. If you are using an external database, enter the appropriate information for your database. Backing Up MySQL Databases To back up the MySQL database, run the mysqldump command on the MySQL host, as follows: $ mysqldump -hhostname -uusername -ppassword database > /tmp/database-backup.sql For example, to back up the Activity Monitor database amon on the local host as the root user, with the password amon_password: $ mysqldump -pamon_password amon > /tmp/amon-backup.sql Managing the Cloudera Manager Server and Agents To back up the sample Activity Monitor database amon on remote host myhost.example.com as the root user, with the password amon_password: $ mysqldump -hmyhost.example.com -uroot -pcloudera amon > /tmp/amon-backup.sql Backing Up Oracle Databases For Oracle, work with your database administrator to ensure databases are properly backed up. Cloudera Manager Administration Guide 17

18

19 Cloudera Management Service Cloudera Management Service The Cloudera Management Service implements various management features as a set of roles: Activity Monitor - collects information about activities run by the MapReduce service. This role is not added by default. Host Monitor - collects health and metric information about hosts Service Monitor - collects health and metric information about services and activity information from the YARN and Impala services Event Server - aggregates relevant Hadoop events and makes them available for alerting and searching Alert Publisher - generates and delivers alerts for certain types of events Reports Manager - generates reports that provide an historical view into disk utilization by user, user group, and directory, processing activities by user and YARN pool, and HBase tables and namespaces. This role is not added in Cloudera Express. Cloudera Manager manages each role separately, instead of as part of the Cloudera Manager Server, for scalability (for example, on large deployments it's useful to put the monitor roles on their own hosts) and isolation. In addition, for certain editions of the Cloudera Enterprise license, the Cloudera Management Service provides the Navigator Audit Server and Navigator Metadata Server roles for Cloudera Navigator. Displaying the Cloudera Management Service Status 1. Do one of the following: Select Clusters > Cloudera Management Service > Cloudera Management Service. On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link. Starting the Cloudera Management Service Required Role: 1. Do one of the following: 1. Select Clusters > Cloudera Management Service > Cloudera Management Service. 2. Select Actions > Start. 1. On the Home page, click to the right of Cloudera Management Service and select Start. 2. Click Start to confirm. The Command Details window shows the progress of starting the roles. 3. When Command completed with n/n successful subcommands appears, the task is complete. Click Close. Stopping the Cloudera Management Service Required Role: 1. Do one of the following: 1. Select Clusters > Cloudera Management Service > Cloudera Management Service. 2. Select Actions > Stop. 1. On the Home page, click to the right of Cloudera Management Service and select Stop. 2. Click Stop to confirm. The Command Details window shows the progress of stopping the roles. Cloudera Manager Administration Guide 19

20 Cloudera Management Service 3. When Command completed with n/n successful subcommands appears, the task is complete. Click Close. Restarting the Cloudera Management Service Required Role: 1. Do one of the following: 1. Select Clusters > Cloudera Management Service > Cloudera Management Service. 2. Select Actions > Restart. 1. On the Home page, click to the right of Cloudera Management Service and select Restart. 2. Click Restart to confirm. The Command Details window shows the progress of stopping and then starting the roles. 3. When Command completed with n/n successful subcommands appears, the task is complete. Click Close. Configuring Management Service Database Limits Required Role: Each Cloudera Management Service role maintains a database for retaining the data it monitors. These databases (as well as the log files maintained by these services) can grow quite large. For example, the Activity Monitor maintains data at the service level, the activity level (MapReduce jobs and aggregate activities), and at the task attempt level. Limits on these data sets are configured when you create the management services, but you can modify these parameters through the Configuration settings in the Cloudera Manager Admin Console. For example, the Event Server lets you set a total number of events to store, and Activity Monitor gives you "purge" settings (also in hours) for the data it stores. There are also settings for the logs that these various services create. You can throttle how big the logs are allowed to get and how many previous logs to retain. 1. Do one of the following: a. Select Clusters > Cloudera Management Service > Cloudera Management Service. b. On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link. 2. Click the Configuration tab. 3. In the left-hand column, select the Default role group for the role whose configurations you want to modify. 4. Edit the appropriate properties: Activity Monitor - the Purge or Expiration period properties are found in the top-level settings for the role. Host and Service Monitor - see Data Storage for Monitoring Data. Log Files - log file size settings will be under the Logs category under the role group. 5. Click Save Changes. Adding and Starting Cloudera Navigator Roles Required Role: 1. Do one of the following: a. Select Clusters > Cloudera Management Service > Cloudera Management Service. b. Select Actions > Restart. 2. Click the Instances tab. 20 Cloudera Manager Administration Guide

21 3. Click the Add Role Instances button. The Customize Role Assignments page displays. 4. Assign the Navigator role to a host. a. Customize the assignment of role instances to hosts. The wizard evaluates the hardware configurations of the hosts to determine the best hosts for each role. The wizard assigns all worker roles to the same set of hosts to which the HDFS DataNode role is assigned. These assignments are typically acceptable, but you can reassign role instances to hosts of your choosing, if desired. Click a field below a role to display a dialog containing a pageable list of hosts. If you click a field containing multiple hosts, you can also select All Hosts to assign the role to all hosts or Custom to display the pageable hosts dialog. The following shortcuts for specifying hostname patterns are supported: Range of hostnames (without the domain portion) Cloudera Management Service Range Definition [1-4] host[1-3].company.com host[07-10].company.com Matching Hosts , , , host1.company.com, host2.company.com, host3.company.com host07.company.com, host08.company.com, host09.company.com, host10.company.com IP addresses Rack name Click the View By Host button for an overview of the role assignment by hostname ranges. 5. When you are satisfied with the assignments, click Continue. The Database Setup page displays. 6. Configure database settings: a. Choose the database type: Leave the default setting of Use Embedded Database to have Cloudera Manager create and configure required databases. Make a note of the auto-generated passwords. Select Use Custom Databases to specify external databases. 1. Enter the database host, database type, database name, username, and password for the database that you created when you set up the database. b. Click Test Connection to confirm that Cloudera Manager can communicate with the database using the information you have supplied. If the test succeeds in all cases, click Continue; otherwise check and correct the information you have provided for the database and then try the test again. (For some servers, if you are using the embedded database, you will see a message saying the database will be created at a later step in the installation process.) The Review Changes page displays. 7. Review and accept any configuration changes (typically there are none). Click Accept. This returns you to the Instances page. 8. Check the checkboxes next to the Navigator Audit Server and Navigator Metadata Server roles. 9. Select Actions for Selected > Start and confirm Start in the pop-up. 10. Click Close. Cloudera Manager Administration Guide 21

22

23 Managing Users and Authentication Managing Users and Authentication This chapter covers managing user accounts, and configuring external authentication. Cloudera Manager User Accounts on page 23 Configuring External Authentication on page 25 Cloudera Manager User Accounts Required Role: Access to Cloudera Manager features is controlled by user accounts. A user account identifies how a user is authenticated and determines what privileges are granted to the user. When you are logged in to the Cloudera Manager Admin Console, the username you are logged in as is at the far right of the top navigation bar for example, if you are logged in as admin you will see. An user in the Administrator role manages user accounts through the Administration > Users page. User Authentication Cloudera Manager provides several mechanisms for authenticating users. You can configure Cloudera Manager to authenticate users against the Cloudera Manager database or against an external authentication service. The external authentication service can be an LDAP server (Active Directory or an OpenLDAP compatible directory), or you can specify another external service. Cloudera Manager also supports using the Security Assertion Markup Language (SAML) to enable single sign-on. If you are using LDAP or an external service you can configure Cloudera Manager so that it can use both methods of authentication (internal database and external service), and you can determine the order in which it performs these searches. If you select an external authentication mechanism, Administrator users can always authenticate against the Cloudera Manager database. This is to prevent locking everyone out if the authentication settings are misconfigured such as with a bad LDAP URL. With external authentication, you can restrict login access to members of specific groups, and can specify groups whose members are automatically given Administrator access to Cloudera Manager. Users accounts in the Cloudera Manager database page show Cloudera Manager in the User Type column. User accounts in an LDAP directory or other external authentication mechanism show External in the User Type column. User Roles A user's role determines the Cloudera Manager features visible to the user and the actions the user can perform. All the tasks in the Cloudera Manager documentation indicate which role is required to perform the task. Note: Currently there is no indication in the Cloudera Manager Admin Console of the role a logged in user is assigned to. To determine your role, contact a user that has the Administrator role. A user account can be assigned one of the following roles: Read-Only - Allows the user to view service and monitoring information but cannot add services or take any actions that affect the state of the cluster. Limited Operator - Allows the user to view service and monitoring information and decommission hosts (except hosts running Cloudera Management Service roles), but cannot add services or take any other actions that affect the state of the cluster. Cloudera Manager Administration Guide 23

24 Managing Users and Authentication Operator - Allows the user to view service and monitoring information, stop, start, and restart clusters, services, and roles (except the Cloudera Management Service and roles), decommission and recommission hosts (except hosts running Cloudera Management Service roles), and decommission and recommission roles (except Cloudera Management Service roles), but cannot add services, roles, or hosts, or take any other actions that affect the state of the cluster. Configurator - Allows the user to perform all Operator operations, configure services (except the Cloudera Management Service), enter and exit maintenance mode, and manage dashboards (including Cloudera Management Service dashboards). Administrator - Allows the user to add, change, delete, and configure services, roles, and hosts and administer user accounts. Even if you are using an external authentication mechanism for user authentication, users with Administrator privileges can log in to Cloudera Manager using their local Cloudera Manager username and password. (This prevents the system from locking everyone out if the external authentication settings get misconfigured.) The full set of roles are available with Cloudera Enterprise; Cloudera Express supports only the Read-Only and Administrator roles. Changing the Logged-In Internal User Password 1. Right-click the logged-in username at the far right of the top navigation bar and select Change Password. 2. Enter the current password, and a new password twice and then click Update. Adding an Internal User Account 1. Select Administration > Users. 2. Click the Add User button. 3. Enter a username and password. 4. In the Role drop-down, select a role for the new user. 5. Click Add. Assigning User Roles 1. Select Administration > Users. 2. Check the checkbox next to one or more usernames. 3. Select Actions for Selected > Assign User Roles. 4. In the drop-down, select the role. 5. Click the Assign Role button. Changing an Internal User Account Password 1. Select Administration > Users. 2. Click the Change Password button next to a username with User Type Cloudera Manager. 3. Type the new password and repeat it to confirm. 4. Click the Update button to make the change. Deleting Internal User Accounts 1. Select Administration > Users. 2. Check the checkbox next to one or more usernames with User Type Cloudera Manager. 3. Select Actions for Selected > Delete. 4. Click the OK button. (There is no confirmation of the action.) 24 Cloudera Manager Administration Guide

25 Managing Users and Authentication Configuring External Authentication Required Role: Important: This feature is available only with a Cloudera Enterprise license. For other licenses, the following applies: Cloudera Express - The feature is not available. Cloudera Enterprise Data Hub Edition Trial - The feature is available until you end the trial or the trial license expires. To obtain a license for Cloudera Enterprise, please fill in this form or call After you install a Cloudera Enterprise license, the feature will be available. Cloudera Manager supports user authentication against an internal database and against an external service. The following sections describe how to configure the supported external services. Configuring Authentication Using Active Directory 1. Select Administration > Settings. 2. In the left-hand column, select the External Authentication category. 3. In the Authentication Backend Order field, select the order in which Cloudera Manager should attempt its authentication. You can choose to authenticate users using just one of the methods (using Cloudera Manager's own database is the default), or you can set it so that if the user cannot be authenticated by the first method, it will attempt using the second method. 4. For External Authentication Type, select Active Directory. 5. In the LDAP URL property, provide the URL of the Active Directory server. 6. In the Active Directory NT Domain property, provide the NT domain to authenticate against. 7. In the LDAP User Groups property, optionally provide a comma-separated list of case-sensitive LDAP group names. If this list is provided, only users who are members of one or more of the groups in the list will be allowed to log into Cloudera Manager. If this property is left empty, all authenticated LDAP users will be able to log into Cloudera Manager. For example, if there is a group called CN=ClouderaManagerUsers,OU=Groups,DC=corp,DC=com, add the group name ClouderaManagerUsers to the LDAP User Groups list to allow members of that group to log in to Cloudera Manager. 8. To automatically assign a role to users when they log in, provide a comma-separated list of LDAP group names in the following properties: LDAP Administrator Groups LDAP Configurator Groups LDAP Operator Groups LDAP Limited Operator Groups If you specify groups in these properties, users must also be a member of at least one of the groups specified in the LDAP User Groups property or they will not be allowed to log in. If these properties are left empty, users will be assigned to the Read-Only role and any other role assignment must be performed manually by an Administrator. Configuring Authentication Using an OpenLDAP-compatible Server For an OpenLDAP-compatible directory, you have several options for searching for users and groups: You can specify a single base Distinguished Name (DN) and then provide a "Distinguished Name Pattern" to use to match a specific user in the LDAP directory. Search filter options let you search for a particular user based on somewhat broader search criteria for example Cloudera Manager users could be members of different groups or organizational units (OUs), so a Cloudera Manager Administration Guide 25

26 Managing Users and Authentication single pattern won't find all those users. Search filter options also let you find all the groups to which a user belongs, to help determine if that user should have login or admin access. 1. Select Administration > Settings. 2. In the left-hand column, select the External Authentication category. 3. In the Authentication Backend Order field, select the order in which Cloudera Manager should attempt its authentication. You can choose to authenticate users using just one of the methods (using Cloudera Manager's own database is the default), or you can set it so that if the user cannot be authenticated by the first method, it will attempt using the second method. 4. For External Authentication Type, select LDAP. 5. In the LDAP URL property, provide the URL of the LDAP server and (optionally) the base Distinguished Name (DN) (the search base) as part of the URL for example ldap://ldap-server.corp.com/dc=corp,dc=com. 6. If your server does not allow anonymous binding, provide the user DN and password to be used to bind to the directory. These are the LDAP Bind User Distinguished Name and LDAP Bind Password properties. By default, Cloudera Manager assumes anonymous binding. 7. To use a single "Distinguished Name Pattern", provide a pattern in the LDAP Distinguished Name Pattern property. Use {0} in the pattern to indicate where the username should go. For example, to search for a distinguished name where the uid attribute is the username, you might provide a pattern similar to uid={0},ou=people,dc=corp,dc=com. Cloudera Manager substitutes the name provided at login into this pattern and performs a search for that specific user. So if a user provides the username "foo" at the Cloudera Manager login page, Cloudera Manager will search for the DN uid=foo,ou=people,dc=corp,dc=com. If you provided a base DN along with the URL, the pattern only needs to specify the rest of the DN pattern. For example, if the URL you provide is ldap://ldap-server.corp.com/dc=corp,dc=com, and the pattern is uid={0},ou=people, then the search DN will be uid=foo,ou=people,dc=corp,dc=com. 8. You can also search using User and/or Group search filters, using the LDAP User Search Base, LDAP User Search Filter, LDAP Group Search Base and LDAP Group Search Filter settings. These allow you to combine a base DN with a search filter to allow a greater range of search targets. For example, if you want to authenticate users who may be in one of multiple OUs, the search filter mechanism will allow this. You can specify the User Search Base DN as dc=corp,dc=com and the user search filter as uid={0}. Then Cloudera Manager will search for the user anywhere in the tree starting from the Base DN. Suppose you have two OUs ou=engineering and ou=operations Cloudera Manager will find User "foo" if it exists in either of these OUs, that is, uid=foo,ou=engineering,dc=corp,dc=com or uid=foo,ou=operations,dc=corp,dc=com. You can use a user search filter along with a DN pattern, so that the search filter provides a fallback if the DN pattern search fails. The Groups filters let you search to determine if a DN or username is a member of a target group. In this case, the filter you provide can be something like member={0} where {0} will be replaced with the DN of the user you are authenticating. For a filter requiring the username, {1} may be used, as memberuid={1}. This will return a list of groups the user belongs to, which will be compared to the list in the group properties discussed in step 8 of Configuring Authentication Using Active Directory on page 25. Configuring Cloudera Manager to Use LDAPS If the LDAP server certificate has been signed by a trusted Certificate Authority (that is, VeriSign, GeoTrust, and so on), steps 1 and 2 below may not be necessary. 1. Copy the CA certificate file to the Cloudera Manager Server host. 2. Import the CA certificate(s) from the CA certificate file to the local truststore. The default truststore is located in the $JAVA_HOME/jre/lib/security/cacerts file. This contains the default CA information shipped with the JDK. Create an alternate default file called jssecacerts in the same location as the cacerts file. You can now safely append CA certificates for any private or public CAs not present in the default cacerts file, while keeping the original file intact. 26 Cloudera Manager Administration Guide

27 For our example, we will follow this recommendation by copying the default cacerts file into the new jssecacerts file, and then importing the CA certificate to this alternate truststore. $ cp $JAVA_HOME/jre/lib/security/cacerts \ $JAVA_HOME/jre/lib/jssecacerts Managing Users and Authentication $ /usr/java/latest/bin/keytool -import -alias nt_domain_name \ -keystore /usr/java/latest/jre/lib/security/jssecacerts -file path_to_cert Note: The default password for the cacerts store is changeit. The alias can be any name (not just the domain name). 3. Configure the LDAP URL property to use ldaps://ldap_server instead of ldap://ldap_server. Configuring Authentication Using an External Program You can configure Cloudera Manager to use an external authentication program of your own choosing. Typically, this may be a custom script that interacts with a custom authentication service. Cloudera Manager will call the external program with the username as the first command line argument. The password is passed over stdin. Cloudera Manager assumes the program will return the following exit codes identifying the user role for a successful authentication: 0 - Read-Only 1 - Administrator 2 - Limited Operator 3 - Operator 4 - Configurator and a negative value is returned for a failure to authenticate. To configure authentication using an external program: 1. Select Administration > Settings. 2. In the left-hand column, select the External Authentication category. 3. In the Authentication Backend Order field, select the order in which Cloudera Manager should attempt its authentication. You can choose to authenticate users using just one of the methods (using Cloudera Manager's own database is the default), or you can set it so that if the user cannot be authenticated by the first method, it will attempt using the second method. 4. For External Authentication Type, select External Program. 5. Provide a path to the external program in the External Authentication Program Path property. Configuring Authentication Using SAML Cloudera Manager supports the Security Assertion Markup Language (SAML), an XML-based open standard data format for exchanging authentication and authorization data between parties, in particular, between an identity provider (IDP) and a service provider (SP). The SAML specification defines three roles: the principal (typically a user), the IDP, and the SP. In the use case addressed by SAML, the principal (user agent) requests a service from the service provider. The service provider requests and obtains an identity assertion from the IDP. On the basis of this assertion, the SP can make an access control decision in other words it can decide whether to perform some service for the connected principal. The primary SAML use case is called web browser single sign-on (SSO). A user wielding a user agent (usually a web browser) requests a web resource protected by a SAML SP. The SP, wishing to know the identity of the requesting user, issues an authentication request to a SAML IDP through the user agent. In the context of this terminology, Cloudera Manager operates as a SP. This topic discusses the Cloudera Manager part of the Cloudera Manager Administration Guide 27

28 Managing Users and Authentication configuration process; it assumes that you are familiar with SAML and SAML configuration in a general sense, and that you have a functioning IDP already deployed. Note: Cloudera Manager supports both SP- and IDP-initiated SSO. The logout action in Cloudera Manager will send a single-logout request to the IDP. SAML authentication has been tested with specific configurations of SiteMinder and Shibboleth. While SAML is a standard, there is a great deal of variability in configuration between different IDP products, so it is possible that other IDP implementations, or other configurations of SiteMinder and Shibboleth, may not interoperate with Cloudera Manager. To bypass SSO if SAML configuration is incorrect or not working, you can login via a Cloudera Manager local account using the URL: Setting up Cloudera Manager to use SAML requires the following steps. Preparing Files You will need to prepare the following files and information, and provide these to Cloudera Manager: A Java keystore containing a private key for Cloudera Manager to use to sign/encrypt SAML messages. The SAML metadata XML file from your IDP. This file must contain the public certificates needed to verify the sign/encrypt key used by your IDP per the SAML Metadata Interoperability Profile The entity ID that should be used to identify the Cloudera Manager instance How the user ID is passed in the SAML authentication response: As an attribute. If so, what identifier is used. As the NameID. The method by which the Cloudera Manager role will be established: From an attribute in the authentication response: What identifier will be used for the attribute What values will be passed to indicate each role From an external script that will be called for each use: The script takes user ID as $1 The script sets an exit code to reflect the assigned role: 0 - Administrator 1 - Read-Only 2 - Limited Operator 3 - Operator 4 - Configurator and a negative value is returned for a failure to authenticate. Configuring Cloudera Manager Required Role: 1. Select Administration > Settings. 2. In the left-hand column, select the External Authentication category. 3. Set the External Authentication Type property to SAML (the Authentication Backend Order property is ignored for SAML). 4. Set the Path to SAML IDP Metadata File property to point to the IDP metadata file. 28 Cloudera Manager Administration Guide

29 5. Set the Path to SAML Keystore File property to point to the Java keystore prepared earlier. 6. In the SAML Keystore Password property, set the keystore password. 7. In the Alias of SAML Sign/Encrypt Private Key property, set the alias used to identify the private key for Cloudera Manager to use. 8. In the SAML Sign/Encrypt Private Key Password property, set the private key password. 9. Set the SAML Entity ID property if: There is more than one Cloudera Manager instance being used with the same IDP (each instance needs a different entity ID). Entity IDs are assigned by organizational policy. 10. In the Source of User ID in SAML Response property, set whether the user ID will be obtained from an attribute or the NameID. 11. If an attribute will be used, set the attribute name in the SAML attribute identifier for user ID property. The default value is the normal OID used for user IDs and so may not need to be changed. 12. In the SAML Role assignment mechanism property, set whether the role assignment will be done from an attribute or an external script. If an attribute will be used: In the SAML attribute identifier for user role property, set the attribute name if necessary. The default value is the normal OID used for OrganizationalUnits and so may not need to be changed. In the SAML Attribute Values for Roles property, set which attribute values will be used to indicate the user role. If an external script will be used, set the path to that script in the Path to SAML Role Assignment Script property. Make sure that the script is executable (an executable binary is fine - it doesn t need to be a shell script). 13. Save the changes. Cloudera Manager will run a set of validations that ensure it can find the metadata XML and the keystore, and that the passwords are correct. If you see a validation error, correct the problem before proceeding. 14. Restart the Cloudera Manager Server. Configuring the IDP After the Cloudera Manager Server is restarted, it will attempt to redirect to the IDP login page instead of showing the normal CM page. This may or may not succeed, depending on how the IDP is configured. In either case, the IDP will need to be configured to recognize CM before authentication will actually succeed. The details of this process are specific to each IDP implementation - refer to your IDP documentation for details. 1. Download the Cloudera Manager s SAML metadata XML file from 2. Inspect the metadata file and ensure that any URLs contained in the file can be resolved by users web browsers. The IDP will redirect web browsers to these URLs at various points in the process. If the browser cannot resolve them, authentication will fail. If the URLs are incorrect, you can manually fix the XML file or set the Entity Base URL in the CM configuration to the right value, and then re-download the file. 3. Provide this metadata file to your IDP using whatever mechanism your IDP provides. 4. Ensure that the IDP has access to whatever public certificates are necessary to validate the private key that was provided to Cloudera Manager earlier. 5. Ensure that the IDP is configured to provide the User ID and Role using the attribute names that Cloudera Manager was configured to expect, if relevant. 6. Ensure the changes to the IDP configuration have taken effect (a restart may be necessary). Verifying Authentication and Authorization Managing Users and Authentication 1. Return to the Cloudera Manager Admin Console and refresh the login page. 2. Attempt to log in with credentials for a user that is entitled. The authentication should complete and you should see the Home page. Cloudera Manager Administration Guide 29

30 Managing Users and Authentication 3. If authentication fails, you will see an IDP provided error message. Cloudera Manager is not involved in this part of the process, and you must ensure the IDP is working correctly to complete the authentication. 4. If authentication succeeds but the user is not authorized to use Cloudera Manager, they will be taken to an error page by Cloudera Manager that explains the situation. If an user who should be authorized sees this error, then you will need to verify their role configuration, and ensure that it is being properly communicated to Cloudera Manager, whether by attribute or external script. The Cloudera Manager log will provide details on failures to establish a user s role. If any errors occur during role mapping, Cloudera Manager will assume the user is unauthorized. 30 Cloudera Manager Administration Guide

31 Configuring TLS Security for Cloudera Manager Configuring TLS Security for Cloudera Manager Important: Cloudera strongly recommends that you set up a fully-functional CDH cluster and Cloudera Manager before you begin configuring it to use TLS. Once Level 3 TLS is configured, if you want to add new hosts running Agents, you must manually deploy the Cloudera Manager Agent and daemon packages for your platform, issue a new certificate for the host, configure /etc/cloudera-scm-agent/config.ini to use SSL/TLS and then bring the host online. Conversely, you can disable TLS to add the host, configure the new host for TLS, then re-enable with the proper configuration in place. Either approach is valid, based on your needs. Transport Layer Security (TLS) provides encryption and authentication in the communications between the Cloudera Manager Server and Agents. Encryption prevents snooping of communications, and authentication helps prevent malicious Servers or Agents from causing problems in your cluster. Cloudera Manager supports three levels of TLS security: Level 1 (Good) - Encrypted communications between the Server and Agents only; no authentication of Server and Agents. See Configuring TLS Encryption only for Cloudera Manager on page 31. Level 2 (Better) - Encrypted communications and authentication of Server to Agents and users; no authentication of Agents to Server. See Configuring TLS Authentication of Server to Agents on page 33. Level 3 (Best) - Encrypted communications, authentication of Server to Agents, and authentication of Agents to Server. See Configuring TLS Authentication of Agents to Server on page 37. To enable TLS encryption for all connections between your Web browser running the Cloudera Manager Admin Console and the Cloudera Manager Server, see Configuring TLS Encryption for Cloudera Manager Admin Console on page 38. Configuring TLS Encryption only for Cloudera Manager Required Role: Use the keytool to manage the public keys and certificates for the Cloudera Manager Server. Before configuring TLS security for Cloudera Manager, create a keystore, as described in the documentation at the preceding link. For example, you might use a command similar to the following: keytool -genkey -alias jetty -keystore truststore Step 1: Create a Cloudera Manager Server certificate. Warning: You must use an Oracle JDK keytool. 1. Use keytool to generate a certificate for the Cloudera Manager Server. For example: $ keytool -validity 180 -keystore <path-to-keystore> -alias jetty -genkeypair -keyalg RSA The -validity option specifies the certificate lifetime in number of days. If no validity value is specified, the default value is used. The default varies, but is often 90 days. Cloudera Manager Administration Guide 31

32 Configuring TLS Security for Cloudera Manager The <path-to-keystore> must be a path to where you want to save the keystore file, and where the Cloudera Manager Server host can access. 2. When prompted by keytool, create a password for the keystore. Save the password in a safe place. 3. When prompted by keytool, fill in the answers accurately to the questions to describe you and your company. The most important answer is the CN value for the question "What is your first and last name?" The CN must match the fully-qualified domain name (FQDN) or IP address of the host where the Server is running. For example, cmf.company.com or Important: For the CN value, be sure to use a FQDN if possible, or a static IP address that will not change. Do not specify an IP address that will change periodically. When Agents connect to the server using TLS, they check whether the key uses the same name as the one they are using to connect to the server. If the names do not match, Agents do not heartbeat. Step 2: Enable TLS encryption and specify Server keystore properties. 1. Log into the Cloudera Manager Admin Console. 2. Select Administration > Settings. 3. Click the Security category. 4. Configure the following TLS settings: Setting Use TLS Encryption for Agents Path to TLS Keystore File Keystore Password Description Enable TLS encryption between the Server and Agents. The full filesystem path to the keystore file. Enable TLS encryption between the Server and Agents. The password for keystore. 5. Click Save Changes to save the settings. Step 3: Enable and configure TLS on the Agent hosts. To enable and configure TLS, you must specify values for the TLS properties in the /etc/cloudera-scm-agent/config.ini configuration file on all Agent hosts. 1. On the Agent host, open the /etc/cloudera-scm-agent/config.ini configuration file: 2. Edit the following property in the /etc/cloudera-scm-agent/config.ini configuration file. Property use_tls Description Specify 1 to enable TLS on the Agent, or 0 (zero) to disable TLS. 3. Repeat these steps on every Agent host. Step 4: Restart the Cloudera Manager Server. Note: Perform this step only if you are using a self-signed server certificate. Restart the Cloudera Manager Server with the following command to activate the TLS configuration settings. $ sudo service cloudera-scm-server restart 32 Cloudera Manager Administration Guide

33 Configuring TLS Security for Cloudera Manager Step 5: Restart the Cloudera Manager Agents. On every Agent host, restart the Agent: $ sudo service cloudera-scm-agent restart Step 6: Verify that the Server and Agents are communicating. In the Cloudera Manager Admin Console, open the Hosts page. If the Agents heartbeat successfully, TLS encryption is working properly. Configuring TLS Authentication of Server to Agents Required Role: This is the second highest level of TLS security and requires that you provide a server certificate for the Server that is signed through a chain to a trusted root CA. You must also provide the certificate of the Certificate Authority (CA) that signed the Server's server certificate. If you are not working in a production environment, you can also use a self-signed server certificate. Note: If the Server's server certificate or the associated CA certificate is missing or expired, the Agents do not allow communications with the Server. Step 1: Configure TLS encryption. If you have not already done so, you must configure TLS encryption to use this second level of security. For instructions, see Configuring TLS Encryption only for Cloudera Manager on page 31. Step 2: Provide the Server's server certificate and CA certificate. If you want to use a Certificate Authority-signed server certificate, you can use keytool to request a server certificate from an existing CA, you can skip down to Using a CA-signed server certificate on page 33. Alternatively, if you want to generate your own self-signed server certificate, you can use keytool to generate a public certificate for the Server, see Using a self-signed server certificate on page 35. Using a CA-signed server certificate 1. Generate a new RSA key: Use keytool provided by the Java SDK to create a new keystore containing a keypair for the Cloudera Manager server. Replace the $ keytool -validity 180 -keystore cm_keystore.jks -alias jetty -genkeypair -keyalg RSA Enter keystore password: Re-enter new password: What is your first and last name? [Unknown]: host_1.example.com What is the name of your organizational unit? [Unknown]: Support What is the name of your organization? [Unknown]: Cloudera What is the name of your City or Locality? [Unknown]: London What is the name of your State or Province? [Unknown]: What is the two-letter country code for this unit? [Unknown]: GB Is CN=host_1.example.com, OU=Support, O=Cloudera, L=London, ST=Unknown, C=GB correct? Cloudera Manager Administration Guide 33

34 Configuring TLS Security for Cloudera Manager [no]: yes Enter key password for <jetty> (RETURN if same as keystore password): 2. Create a Certificate Signing Request (CSR): Use the key created in the previous step to create a CSR for the Cloudera Manager server. $ keytool -certreq -alias jetty -keystore cm_keystore.jks > jetty.csr Enter keystore password: 3. Request a new server certificate: To request a certificate from a recognised Certificate Authority (CA), provide the CSR generated in step 2. The example below uses a private CA created using OpenSSL. $ openssl ca -config openssl.cnf -out jetty.crt -infiles jetty.csr Using configuration from openssl.cnf Enter pass phrase for cakey.pem: Check that the request matches the signature Signature ok <--SNIP--> Certificate is to be certified until Apr 19 10:49: GMT (3650 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated 4. Import CA trust certificates into the Cloudera Manager keystore: Import the root and intermediate CA certificates to the keystore created in step 1. Generate a new RSA key. $ keytool -import -keystore cm_keystore.jks -alias int_ca -file intermediate.crt Enter keystore password: Owner: CN=COE Intermediate Test CA, OU=Customer Operations, O=Cloudera, ST=London, C=GB Issuer: CN=COE Root Test CA, OU=Customer Operations, O=Cloudera, L=Shoreditch, ST=London, C=GB Serial number: 1 Valid from: Tue Apr 22 02:02:26 PDT 2014 until: Wed Apr 22 02:02:26 PDT 2015 <--SNIP--> Trust this certificate? [no]: yes Certificate was added to keystore $ keytool -import -keystore cm_keystore.jks -alias root_ca -file root.crt Enter keystore password: Owner: CN=COE Root Test CA, OU=Customer Operations, O=Cloudera, L=Shoreditch, ST=London, C=GB Issuer: CN=COE Root Test CA, OU=Customer Operations, O=Cloudera, L=Shoreditch, ST=London, C=GB Serial number: 928f80538cdfe523 Valid from: Tue Apr 22 01:58:53 PDT 2014 until: Sun Apr 21 01:58:53 PDT 2019 <--SNIP--> Trust this certificate? [no]: yes Certificate was added to keystore 5. Import certificate into Cloudera Manager keystore: 34 Cloudera Manager Administration Guide

35 Import the signed server certificate supplied by the CA, using the same alias that was used to create the key pair in step 1, so the key and the certificate are linked together in the keystore. Make sure you see the message, Certificate reply was installed in keystore. $ keytool -import -keystore cm_keystore.jks -alias jetty -file jetty.crt Enter keystore password: Certificate reply was installed in keystore 6. Create trusted keystore: Create a trusted keystore using the keytool command as in step 1. Generate a new RSA key. Import the CA intermediate and root certificates to this new keystore, in this case, trusted.jks. Alternatively, you can use the existing Cloudera Manager keystore, containing the CA intermediate and root certificates, as the trust store. $ keytool -import -keystore trusted.jks -alias int_ca -file intermediate.crt Enter keystore password: Owner: CN=COE Intermediate Test CA, OU=Customer Operations, O=Cloudera, ST=London, C=GB Issuer: CN=COE Root Test CA, OU=Customer Operations, O=Cloudera, L=Shoreditch, ST=London, C=GB Serial number: 1 Valid from: Tue Apr 22 02:02:26 PDT 2014 until: Wed Apr 22 02:02:26 PDT 2015 <--SNIP--> Trust this certificate? [no]: yes Certificate was added to keystore $ keytool -import -keystore trusted.jks -alias root_ca -file root.crt Enter keystore password: Owner: CN=COE Root Test CA, OU=Customer Operations, O=Cloudera, L=Shoreditch, ST=London, C=GB Issuer: CN=COE Root Test CA, OU=Customer Operations, O=Cloudera, L=Shoreditch, ST=London, C=GB Serial number: 928f80538cdfe523 Valid from: Tue Apr 22 01:58:53 PDT 2014 until: Sun Apr 21 01:58:53 PDT 2019 <--SNIP--> Trust this certificate? [no]: yes Certificate was added to keystore Configuring TLS Security for Cloudera Manager Using a self-signed server certificate 1. Use keytool to generate a public certificate for the Server by typing the following command on the Server host: $ keytool -validity 180 -keystore <path-to-keystore> -alias jetty -genkeypair -keyalg RSA 2. When prompted by keytool, create a password for the keystore. Save the password in a safe place. 3. When prompted by keytool, fill in the answers accurately to the questions to describe you and your company. The most important answer is the CN value for the question "What is your first and last name?" The CN must match the fully-qualified domain name (FQDN) or IP address of the host where the Server is running. For example, cmf.company.com or Important: For the CN value, be sure to use a FQDN if possible, or a static IP address that will not change. Do not specify an IP address that will change periodically. When agents connect to the server using TLS, they check whether the key uses the same name as the one they are using to connect to the server. If the names do not match, agents do not heartbeat. Cloudera Manager Administration Guide 35

36 Configuring TLS Security for Cloudera Manager 4. On the Server host, run the following command to export the server certificate from your keystore in the binary DER format: $ keytool -exportcert -keystore <path-to-keystore> -alias jetty -file server.der 5. Convert the binary DER format to a.pem file that can be used on the Agents by using openssl (available for download here.) $ openssl x509 -out server.pem -in server.der -inform der Step 3: Copy the Server's server.pem file to the Agents. 1. Copy the Server's server.pem file (for example, server.pem) to the Agent host in any directory. If you have used a CA-signed certificate, copy the CA's root certificate in PEM format to the Agent host. For example, copy the.pem file to /etc/cmf. 2. On the Agent host, open the /etc/cloudera-scm-agent/config.ini configuration file and edit the following properties in the /etc/cloudera-scm-agent/config.ini configuration file. Property verify_cert_file use_tls Description Enter the path to the Server's server.pem file. For example, /etc/cmf/server.pem. Set this property to Repeat these steps on every Agent host. Step 4: Enable TLS Encryption in Cloudera Manager. 1. Log into the Cloudera Manager Admin Console. 2. Select Administration > Settings. 3. Click the Security category. 4. Configure the following TLS setting: Setting Use TLS Encryption for Agents Description Enable TLS encryption between the Server and Agents. 5. Click Save Changes to save the settings. Step 5: Restart the Cloudera Manager Server. $ sudo service cloudera-scm-server restart Step 6: Restart the Cloudera Manager Agents. On every Agent host, restart the Agent: $ sudo service cloudera-scm-agent restart Step 7: Verify that the Server and Agents are communicating. In the Cloudera Manager Admin Console, open the Hosts page. If the Agents heartbeat successfully, the Server and Agents are communicating. If not, check the Agent log /var/log/cloudera-scm-agent/cloudera-scm-agent.log which shows errors if the connection fails. 36 Cloudera Manager Administration Guide

37 Configuring TLS Security for Cloudera Manager Configuring TLS Authentication of Agents to Server Required Role: This is the highest level of TLS security and requires you to use openssl to create private keys and public certificates for every Agent on your cluster, and import those Agents' certificates into the Server's truststore. Step 1: Configure TLS encryption. If you have not already done so, you must configure TLS encryption to use this third level of security. For instructions, see Configuring TLS Encryption for Cloudera Manager. Step 2: Configure TLS Authentication of Server to Agents. If you have not already done so, you must configure TLS Authentication of Server to Agents. For instructions, see Configuring TLS Authentication of Server to Agents. Step 3. Generate the private key for the Agent using openssl. 1. Run the following openssl command on the agent: $ openssl genrsa -des3 -out agent.key 2. Provide a password for the key file. Note it in a safe place. Step 4: Generate a certificate for the agent. 1. Run the following openssl command. $ openssl req -new -x509 -days 365 -key agent.key -out agent.pem The key is output in a.pem file. In the preceding example, the optional days argument results in a certificate that is valid for 365 days. 2. Fill in the answers to the questions about the certificate. Note that the CN must match the hostname or IP address of the Agent host. Step 5: Create a file that contains the password for the key. The Agent reads the password from a text file instead of from a command line. The file allows you to use file permissions to protect the password. For example, name the file agent.pw. Step 6: Configure the Agent with its private key and certificate. 1. On the Agent host, open the /etc/cloudera-scm-agent/config.ini configuration file: 2. Edit the following properties in the /etc/cloudera-scm-agent/config.ini configuration file. Property client_key_file client_keypw_file client_cert_file Description Name of client key file Name of client key pw file Name of client certificate file 3. Repeat these steps on every Agent host. Cloudera Manager Administration Guide 37

38 Configuring TLS Security for Cloudera Manager Step 7: Import the Agent's certificate into the Server's truststore. The Server's truststore contains the certificates that are required to authenticate clients. Use the following command to import a certificate called, for example, agent.pem into a new truststore called, for example, truststore. $ keytool -keystore <path-to-truststore> -import -alias <agent-name> -file agent.pem Step 8: Repeat steps 3 through 7 for every agent in your cluster. Important: Each Agent's private key and certificate that you import into the Server's truststore must be unique. Step 9: Enable Agent authentication and configure the Server to use the new truststore. 1. Log into the Cloudera Manager Admin Console. 2. Select Administration > Settings. 3. Click the Security category. 4. Configure the following TLS settings: Setting Use TLS Authentication of Agents to Server Path to Truststore Truststore Password Description Select this option to enable TLS Authentication of Agents to the Server. Specify the full filesystem path to the truststore located on the Cloudera Manager Server host. Specify the password for the truststore. 5. Click Save Changes to save the settings. Step 10: Restart the Server. $ sudo service cloudera-scm-server restart Step 11: Restart the Cloudera Manager Agents. On every Agent host, restart the Agent: $ sudo service cloudera-scm-agent restart Step 12: Verify that the Server and Agents are communicating. In Cloudera Manager Admin Console, open the Hosts page. If the Agents heartbeat successfully, the Server and Agents are communicating. If they are not, you may get an error in the Server, such as a null CA chain error. This implies either the truststore doesn't contain the Agent certificate or the Agent isn't presenting the certificate. Double check all of your settings. Check the Server's log to verify whether TLS and Agent validation have been enabled correctly. Configuring TLS Encryption for Cloudera Manager Admin Console Required Role: 38 Cloudera Manager Administration Guide

39 This level of security is for users connecting to the Cloudera Manager Admin console. Step 1: Create a Cloudera Manager Server certificate. Configuring TLS Security for Cloudera Manager Note: If you have already completed this step when configuring TLS encryption for Cloudera Manager, you do not need to repeat it. Warning: You must use an Oracle JDK keytool. 1. Use keytool to generate a certificate for the Cloudera Manager Server. For example: $ keytool -validity 180 -keystore <path-to-keystore> -alias jetty -genkeypair -keyalg RSA The -validity option specifies the certificate lifetime in number of days. If no validity value is specified, the default value is used. The default varies, but is often 90 days. The <path-to-keystore> must be a path to where you want to save the keystore file, and where the Cloudera Manager Server host can access. 2. When prompted by keytool, create a password for the keystore. Save the password in a safe place. 3. When prompted by keytool, fill in the answers accurately to the questions to describe you and your company. The most important answer is the CN value for the question "What is your first and last name?" The CN must match the fully-qualified domain name (FQDN) or IP address of the host where the Server is running. For example, cmf.company.com or Important: For the CN value, be sure to use a FQDN if possible, or a static IP address that will not change. Do not specify an IP address that will change periodically. When agents connect to the server using TLS, they check whether the key uses the same name as the one they are using to connect to the server. If the names do not match, agents do not heartbeat. Step 2: Enable TLS encryption and specify Server keystore properties. 1. Log into the Cloudera Manager Admin Console. 2. From the Administration tab select Settings, then go to the Security category. 3. Configure the following three TLS settings: Setting Use TLS Encryption for Admin Console Path to TLS Keystore File Keystore Password Description Select this option to enable TLS encryption between the Server and user's web browser. Specify the full filesystem path to the keystore file. Specify the password for keystore. 4. Click Save Changes to save the settings. Step 3: Restart the Cloudera Manager Server. Restart the Cloudera Manager Server with the following command to activate the TLS configuration settings. $ sudo service cloudera-scm-server restart Log out and then log in into Cloudera Manager to test the certificate. You may see an warning message to accept the certificate if the root certificate is not installed in your browser. Cloudera Manager Administration Guide 39

40 Configuring TLS Security for Cloudera Manager Step 4: Restart the Cloudera Management Services. Restart the Cloudera Management Services by clicking the Services link and choosing Restart on the Actions menu for the Cloudera Management Services. Click Restart that appears in the next screen to confirm. When you see a Finished status, the service has restarted. Step 5: Verify that the Server and browser are using TLS to communicate. Open the Cloudera Manager Admin Console page in your browser. Every browser has its own way of indicating a successful TLS connection. Some browsers indicate this by displaying a lock icon in the URL bar while others display an error message if the connection is unencrypted. HTTPS Communication in Cloudera Manager Both the Cloudera Manager Agent and the daemons that make up the Cloudera Management Service participate in HTTPS communication with Cloudera Manager and CDH components. This topic aims to explain how the how the various aspects of HTTPS communication are handled by the Cloudera Manager agents and the Cloudera Management Services daemons. Cloudera Manager Agents communicate with a number of the local running CDH services over HTTPS to collect monitoring data. As of Cloudera Manager 5.1.x, monitoring data for HBase, HDFS, MapReduce and YARN can also be collected over HTTPS. Previous releases only supported monitoring Impala daemons using HTTPS. Cloudera Manager Agent The process of configuring TLS communication between the Cloudera Manager Server and agents is outlined here. The process is identical to that for previous releases except for one new parameter. You can now configure the certificates available for server certificate verification using the verify_cert_dir parameter in the Agent's config.ini file. See the comments in the config.ini file for a detailed explanation of this property. This change is strictly additive. The existing verify_cert_file parameter can still be used. The following points outline the Cloudera Manager Agent's behavior when it communicates with CDH services using HTTPS. If verify_cert_file and/or verify_cert_dir are configured in the Agent's config.ini then the Agent will use these settings to perform certificate verification on the server certificates. If these settings are not configured, no certificate verification will be performed. That is, it is not possible to perform certificate verification for the Cloudera Manager Server but not for CDH daemons. An Agent never participates in mutual TLS authentication with any CDH service. Instead each service has it's own authentication scheme. For most services this is Kerberos authentication, while Impala uses HTTP digest. User Impact This depends on how you are using certificates. If you are not interested in certificate verification, do not configure verify_cert_file or verify_cert_dir. Remember that this leaves you vulnerable to man-in-the-middle attacks. If you are using a CA-signed certificate, configure the Agent accordingly. Adding new services or enabling SSL/TLS on a service will not require any changes to the Agent configuration since the CA should be able to verify the certificates used by any new servers you bring online. If you are using self-signed certificates, then the addition of each new service that uses HTTPS will require making the certificate available to the Agent. This will involve modifying the file pointed to by verify_cert_file (agent restart required), or the directory pointed to by verify_cert_dir, to contain the new certificate. 40 Cloudera Manager Administration Guide

41 Cloudera Management Services A number of Cloudera Management Service roles act as HTTPS clients in communications with the Cloudera Management Service and other CDH entities. As of Cloudera Manager 5.1.x, there are two ways to configure verification of server certificates. You can configure a truststore through Cloudera Manager to perform certificate verification on the certificates of the various servers it communicates with. If this truststore is configured, it is used to verify server certificates. OR If no truststore is configured through Cloudera Manager, the default Java truststore (cacerts) will be used to verify certificates. The following table shows Cloudera Manager roles that act as HTTPS servers as other roles communicate with them as HTTPS clients. This table does not depict the entirety of the roles' communication, just communications over HTTPS. Table 1: HTTPS Communication Between Cloudera Manager Roles Roles as HTTPS Servers Communicating HTTPS Client(s) Roles Activity Monitor Cloudera Manager Server JobTracker Web Server Oozie server (may involve the Load Balancer in an HA configuration) Host Monitor Cloudera Manager Server Service Monitor Cloudera Manager Server NameNode(s) Web Server Impala StateStore Web Server YARN ResourceManager(s) Web Server(s) YARN JobHistory Web Server Oozie server (directly, never through the Load Balancer) Event Server Cloudera Manager Server Reports Manager Cloudera Manager Server NameNode(s) Web Servers Configuring TLS Security for Cloudera Manager Note: The Cloudera Navigator roles also act as HTTPS clients, but are outside the scope of this document. The Cloudera Manager roles behave as follows when communicating using HTTPS: If the Cloudera Management Service's SSL Client Truststore File Location parameter is configured, the roles will use this truststore to perform certificate verification on the server certificates. If this parameter is not set, certificate verification will be performed using the default Java truststore. This means that it is not possible, without the use of safety valves, to perform certificate verification for some Cloudera Management Service roles and not for others. Nor is it possible to perform certificate verification for only a subset of the HTTPS communication by a role. The Cloudera Management Service roles never participate in mutual TLS authentication with any CDH service or with the Cloudera Manager Server. Instead each service has it's own authentication scheme. For most services this is Kerberos authentication, while Impala uses HTTP digest. For the Cloudera Manager Server, this is session-based authentication. Cloudera Manager Administration Guide 41

42 Configuring TLS Security for Cloudera Manager User Impact This depends on how you are using certificates: If you are using a CA-signed certificate, configure the Cloudera Manager Service's SSL Client Truststore File Location parameter to point to a truststore that contains the CA certificate. Adding a new service or enabling TLS on an existing service will not require any changes to the Cloudera Management Service configuration since the CA certificate should be able to verify the certificates used by any new servers you bring online. Alternatively, this CA-signed certificate may be added to the default Java truststore. If you are using self-signed certificates, the addition of each new server using HTTPS will require making the certificate available. This will involve modifying the truststore pointed to by the Cloudera Manager Service's SSL Client Truststore File Location parameter. Truststore changes will be needed on each host on which a Cloudera Management Service daemon is running. Changes to the truststore will not require a role restart, and should be picked up within 10 seconds by default. If the Cloudera Manager Service's SSL Client Truststore File Location is not in use, the certificate must be made available in the default Java truststore. The Cloudera Management Service daemon will need to be restarted for this change to take effect. 42 Cloudera Manager Administration Guide

43 Upgrading Cloudera Manager Upgrading Cloudera Manager Upgrading Cloudera Manager preserves existing data and settings, while enabling the use of the new features provided with the latest product versions. To enable new features, some new settings are added, and some additional steps may be required, but no existing configuration is removed. Note: When an upgraded Cloudera Manager adds support for a new feature (for example, Sqoop 2, WebHCat, and so on), it does not install the software on which the new feature depends. If you install CDH and managed services from packages, you must add the packages to your managed hosts first, before adding a service or role that supports the new feature. Understanding Upgrades The process for upgrading Cloudera Manager varies depending on the starting point. The categories of tasks to be completed include the following: Install any databases required for the release. In Cloudera Manager 5, the Host Monitor and Service Monitor roles use an internal database that provides greater capacity and flexibility for current and future uses. You no longer need to configure an external database for this purpose. If you are upgrading from Cloudera Manager 4, this transition is handled automatically. If you are upgrading a Free Edition installation and you are running a MapReduce service, you are asked to configure an additional database for the Activity Monitor that is part of Cloudera Express. Upgrade the Cloudera Manager Server. Upgrade the Cloudera Manager Agent. This can be done via an upgrade wizard that is invoked when you connect to the Admin Console or by manually installing the Cloudera Manager Agent packages. Upgrading Cloudera Manager You can upgrade from any version of Cloudera Manager 4 running CDH 4, to Cloudera Manager 5 or from Cloudera Manager 5 to a later version of Cloudera Manager 5. See the instructions at: Upgrading Cloudera Manager 5 to the Latest Cloudera Manager on page 46. After upgrading Cloudera Manager 5, the following is true: Database schema is upgraded to reflect the current version. The Cloudera Manager Server and all supporting services are updated. Client configurations are redeployed to ensure client services have the most current configuration. Upgrading Cloudera Manager 4 to Cloudera Manager 5 on page 54 and Upgrading Cloudera Manager 3.7.x on page 66. After upgrading to Cloudera Manager 5, the following is true: Database schema is upgraded to reflect the current version. Data from the existing Host and Service Monitor databases is migrated. The Cloudera Manager Server and all supporting services are updated. Client configurations are redeployed to ensure client services have the most current configuration. Cloudera Manager 5 continues to support a CDH 4 cluster with an existing High Availability deployment using NFS shared edits directories. However, if you disable High Availability in Cloudera Manager 5, you will only be able to re-enable High Availability using Quorum-based Storage. CDH 5 does not support enabling NFS shared edits directories with High Availability. Cloudera Manager Administration Guide 43

44 Upgrading Cloudera Manager Upgrading CDH Cloudera Manager 5 can manage both CDH 4 and CDH 5, so upgrading existing CDH 4 installations is not required. However, to get the benefits of the most current CDH features, you may want to upgrade CDH. For more information on upgrading CDH, see Upgrading CDH and Managed Services. Database Considerations for Cloudera Manager Upgrades Cloudera Manager uses databases to store information about system configurations and tasks. Before upgrading, complete the pre-upgrade database tasks that apply in your environment. Note: Cloudera Manager 4.5 added support for Hive, which includes the Hive Metastore Server role type. This role manages the Metastore process when Hive is configured with a remote Metastore. When upgrading from Cloudera Manager prior to 4.5, Cloudera Manager automatically creates new Hive service(s) to capture the previous implicit Hive dependency from Hue and Impala. Your previous services will continue to function without impact. If Hue was using a Hive Metastore backed by a Derby database, then the newly created Hive Metastore Server will also use Derby. Since Derby does not allow concurrent connections, Hue will continue to work, but the new Hive Metastore Server will fail to run. The failure is harmless (because nothing uses this new Hive Metastore Server at this point) and intentional, to preserve the set of cluster functionality as it was before upgrade. Cloudera discourages the use of a Derby backed Hive Metastore due to its limitations. You should consider switching to a different supported database. After you have completed these steps, the upgrade processes automatically complete any additional updates to database schema and service data stored. You do not need to complete any data migration. Back up Databases Before beginning the upgrade process, shut down the services that are using databases. This includes the Cloudera Manager Management Service roles, the Hive Metastore server, and Cloudera Navigator, if it is in use. Cloudera strongly recommends that you then back up all databases, however backing up the Activity Monitor database is optional. This is especially important if you are upgrading from Cloudera Manager 4 to Cloudera Manager 5. For information on backing up databases see Backing up Databases on page 16. If any additional database will be required as a result of the upgrade, complete any required preparatory work to install and configure those databases. For example, if you are upgrading from Cloudera Manager Free Edition, Cloudera Manager 5 with Cloudera Express requires a database for the Activity Monitor. The upgrade instructions assume all required databases have been prepared. For more information on using databases, see Cloudera Manager and Managed Service Databases. Modify Databases to Support UTF-8 Cloudera Manager 4.0 adds support for UTF-8 character sets. Update any existing databases in your environment that are not configured to support UTF Cloudera Manager Administration Guide

45 Modifying MySQL to Support UTF-8 To modify a MySQL database to support UTF-8, the default character set must be changed and then you must restart the mysql service. Use the following commands to complete these tasks: mysql> alter database default character set utf8; mysql> quit $ sudo service mysql restart Upgrading Cloudera Manager Modifying PostgreSQL to Support UTF-8 There is no single command available to modify an existing PostgreSQL database to support UTF-8. As a result, you must complete the following process: 1. Use pg_dump to export the database to a file. This creates a backup of the database that you will import into a new, empty database that supports UTF Drop the existing database. This deletes the existing database. 3. Create a new database that supports Unicode encoding and that has the same name as the old database. Use a command of the following form, replacing the database name and user name with values that match your environment: CREATE DATABASE scm_database WITH OWNER scm_user ENCODING 'UTF8' 4. Review the contents of the exported database for non-standard characters. If you find unexpected characters, modify these so the database backup file contains the expected data. 5. Import the database backup to the newly created database. Modifying Oracle to Support UTF-8 Work with your Oracle database administrator to ensure any Oracle databases support UTF-8. Modify Databases to Support Appropriate Maximum Connections Check existing databases configurations to ensure the proper maximum number of connections is supported. Update the maximum configuration values, as required. Modify the Maximum Number of MySQL Connections Allow 100 maximum connections for each database and then add 50 extra connections. For example, for two databases set the maximum connections to 250. If you store five databases on one host (the databases for Cloudera Manager Server, Activity Monitor, Reports Manager, Cloudera Navigator, and Hive Metastore), set the maximum connections to 550. Modify the Maximum Number of PostgreSQL Connections Update the max_connection parameter in the /etc/postgresql.conf file. You may have to increase the system resources available to PostgreSQL, as described at Modify the Maximum Number of Oracle Connections Work with your Oracle database administrator to ensure appropriate values are applied for your Oracle database settings. You must determine the number of connections, transactions, and sessions to be allowed. Allow 100 maximum connections for each database and then add 50 extra connections. For example, for two databases set the maximum connections to 250. If you store five databases on one host (the databases for Cloudera Manager Server, Activity Monitor, Reports Manager, Cloudera Navigator, and Hive Metastore), set the maximum connections to 550. Cloudera Manager Administration Guide 45

46 Upgrading Cloudera Manager From the maximum number of connections, you can determine the number of anticipated sessions using the following formula: sessions = (1.1 * maximum_connections) + 5 For example, if a host has two databases, you anticipate 250 maximum connections. If you anticipate a maximum of 250 connections, plan for 280 sessions. Once you know the number of sessions, you can determine the number of anticipated transactions using the following formula: transactions = 1.1 * sessions Continuing with the previous example, if you anticipate 280 sessions, you can plan for 308 transactions. Work with your Oracle database administrator to apply these derived values to your system. Using the sample values above, Oracle attributes would be set as follows: alter system set processes=250; alter system set transactions=308; alter system set sessions=280; Next Steps After you have completed any required database preparatory tasks, continue to Upgrading Cloudera Manager 4 to Cloudera Manager 5 on page 54. Upgrading Cloudera Manager 5 to the Latest Cloudera Manager Required Role: This process applies to upgrading all versions of Cloudera Manager 5. In most cases it is possible to complete the following upgrade without shutting down most CDH services, although you may need to stop some dependent services. CDH daemons can continue running, unaffected, while Cloudera Manager is upgraded. The upgrade process does not affect your CDH installation. After upgrading Cloudera Manager you may also want to upgrade CDH 4 clusters to CDH 5. Upgrading from a version of Cloudera Manager 5 to the latest version of Cloudera Manager involves the following broad steps. Review Warning Cloudera Management Service SSL configuration If you have enabled TLS security for the Cloudera Manager Admin Console, as of Cloudera Manager 5.1 Cloudera Management Service roles will try to communicate with Cloudera Manager using TLS, and will fail to start until SSL properties have been configured. Navigator If you have enabled auditing with Cloudera Navigator, during the process of upgrading to Cloudera Manager 5 auditing is suspended and is only restarted when you restart the roles of audited services. JDK upgrade If you choose to upgrade the JDK during the installation of the Cloudera Manager Agent, you must restart all services. 46 Cloudera Manager Administration Guide

47 Perform Prerequisite Steps Ensure that you have performed the following steps: Obtain host credentials - If you want Cloudera Manager to upgrade the Agent packages you must have SSH access and be able to log in using a root account or an account that has password-less sudo permission. See Cloudera Manager Requirements for more information. Stop running commands - Use the Admin Console to check for any running commands. You can either wait for commands to complete or abort any running commands. For more information on viewing and aborting running commands, see Viewing Running and Recent Commands. Prepare databases - See Database Considerations for Cloudera Manager Upgrades on page 44. Stop Selected Services and Roles Upgrading Cloudera Manager If your cluster meets any of the conditions listed in the following table, you must stop the indicated services or roles. Condition Running a version of Cloudera Manager that has the Cloudera Management Service Running Cloudera Navigator Procedure Stop the Cloudera Management Service. Stop any of the following roles whose service's Queue Policy configuration (navigator.batch.queue_policy) is set to SHUTDOWN: HDFS - NameNode HBase - Master and RegionServers Hive - HiveServer2 Hue - Beeswax Server Stopping these roles renders any service depending on these roles unavailable. For the HDFS - NameNode case this implies most of the services in the cluster will be unavailable until the upgrade is finished. Stop Cloudera Manager Server, Database, and Agent 1. On the host running the Cloudera Manager Server, stop the Cloudera Manager Server: $ sudo service cloudera-scm-server stop 2. If you are using the embedded PostgreSQL database for Cloudera Manager, stop the database: sudo service cloudera-scm-server-db stop Important: If you are not running the embedded database service and you attempt to stop it, you will get a message to the effect that the service cannot be found. If instead you get a message that the shutdown failed, this means the embedded database is still running, probably due to services connected to the Hive Metastore. Do not proceed with the installation until you have stopped all your Metastore-dependent services and the database successfully shuts down (restart the Cloudera Manager server to shut down services as necessary). If you continue without solving this, your upgrade will fail and you will be left with a non-functional Cloudera Manager installation. 3. If the Cloudera Manager host is also running the Cloudera Manager Agent, stop the Cloudera Manager Agent: $ sudo service cloudera-scm-agent stop Cloudera Manager Administration Guide 47

48 Upgrading Cloudera Manager (Optional) Upgrade JDK on Cloudera Manager Server Host and Agent Hosts If you are manually upgrading the Cloudera Manager Agent packages in Upgrade Cloudera Manager Agent Packages on page 50, and you plan to upgrade to CDH 5, ensure that the Oracle JDK is installed on the Agent hosts following the instructions in Java Development Kit Installation. If you are not running Cloudera Manager Server on the same host as a Cloudera Manager Agent, and you want all hosts to run the same JDK version, optionally install the Oracle JDK on that host. Upgrade Cloudera Manager Server Packages 1. To upgrade the Cloudera Manager Server Packages, you can upgrade from the Cloudera repository at or you can create your own repository, as described in Understanding Custom Installation Solutions. Creating your own repository is necessary if you are upgrading a cluster that does not have access to the Internet. a. Find the Cloudera repo file for your distribution by starting at and navigating to the directory that matches your operating system. For example, for Red Hat or CentOS 6, you would navigate to Within that directory, find the repo file that contains information including the repository's base URL and GPG key. The contents of the cloudera-manager.repo file might appear as follows: [cloudera-manager] # Packages for Cloudera Manager, Version 5, on RedHat or CentOS 6 x86_64 name=cloudera Manager baseurl= gpgkey = gpgcheck = 1 For Ubuntu or Debian systems, the repo file can be found by navigating to the appropriate release directory, for example, The repo file, in this case, cloudera.list, may appear as follows: # Packages for Cloudera Manager, Version 5, on Debian 7.0 x86_64 deb wheezy-cm5 contrib deb-src wheezy-cm5 contrib b. Replace the repo file in the configuration location for the package management software for your system. Operating System RHEL SLES Ubuntu or Debian Commands Copy cloudera-manager.repo to /etc/yum.repos.d/. Copy cloudera-manager.repo to /etc/zypp/repos.d/. Copy cloudera.list to /etc/apt/sources.list.d/. c. Run the following commands: Operating System RHEL Commands $ sudo yum clean all $ sudo yum upgrade 'cloudera-*' 48 Cloudera Manager Administration Guide

49 Upgrading Cloudera Manager Operating System Commands Note: yum clean all cleans up yum's cache directories, ensuring that you download and install the latest versions of the packages If your system is not up to date, and any underlying system components need to be upgraded before this yum update can succeed. yum will tell you what those are. SLES Ubuntu or Debian $ sudo zypper clean --all $ sudo zypper up -r To download from your own repository: $ sudo zypper clean --all $ sudo zypper rr cm $ sudo zypper ar -t rpm-md $ sudo zypper up -r Use the following commands to clean cached repository information and update Cloudera Manager components: $ sudo apt-get clean $ sudo apt-get update $ sudo apt-get dist-upgrade $ sudo apt-get install cloudera-manager-server cloudera-manager-agent cloudera-manager-daemons As this process proceeds, you may be prompted concerning your configuration file version: Configuration file `/etc/cloudera-scm-agent/config.ini' ==> Modified (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. You will receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both these prompts. The config.ini file should be carefully inspected and the files merged together to ensure the new entries are incorporated. At the end of this process you should have the following packages, corresponding to the version of Cloudera Manager you installed, on the host that will become the Cloudera Manager Server host. OS RPM-based distributions Ubuntu or Debian Packages $ rpm -qa 'cloudera-manager-*' cloudera-manager-agent cm5.p0.932.el6.x86_64 cloudera-manager-server cm5.p0.932.el6.x86_64 cloudera-manager-daemons cm5.p0.932.el6.x86_64 ~# dpkg-query -l 'cloudera-manager-*' Desired=Unknown/Install/Remove/Purge/Hold Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) / Name Version Description +++-======================-======================-============================================================ ii cloudera-manager-agent cm5.p0.932~sq The Cloudera Manager Agent Cloudera Manager Administration Guide 49

50 Upgrading Cloudera Manager OS Packages ii cloudera-manager-daemo cm5.p0.932~sq Provides daemons for monitoring Hadoop and related tools. ii cloudera-manager-serve cm5.p0.932~sq The Cloudera Manager Server You may also see an entry for the cloudera-manager-server-db-2 if you are using the embedded database, and additional packages for plug-ins, depending on what was previously installed on the server host. If the cloudera-manager-server-db-2 package is installed, and you don't plan to use the embedded database, you can remove this package. Start the Cloudera Manager Server On the Cloudera Manager Server host (the system on which you installed the cloudera-manager-server package) do the following: 1. If you are using the embedded PostgreSQL database for Cloudera Manager, start the database: $ sudo service cloudera-scm-server-db start 2. Start the Cloudera Manager Server: $ sudo service cloudera-scm-server start You should see the following: Starting cloudera-scm-server: [ OK ] If the Cloudera Manager Server does not start, see Troubleshooting Installation and Upgrade Problems. Upgrade Cloudera Manager Agent Packages Important: All hosts in the cluster must have access to the Internet if you plan to use archive.cloudera.com as the source for installation files. If you do not have Internet access, create a custom repository. 1. Log in to the Cloudera Manager Admin Console. 2. Upgrade hosts using one of the following methods: Cloudera Manager installs Agent software 1. Select Yes, I would like to upgrade the Cloudera Manager Agent packages now and click Continue. 2. Select the release of the Cloudera Manager Agent to install. Normally, this will be the Matched Release for this Cloudera Manager Server. However, if you used a custom repository (that is, a repository other than archive.cloudera.com) for the Cloudera Manager server, select Custom Repository and provide the required information. The custom repository allows you to use an alternative location, but that location must contain the matched Agent version. 3. Click Continue. (Cloudera Manager 5.1.3) Leave Install Oracle Java SE Development Kit (JDK) checked to allow Cloudera Manager to install the JDK on each cluster host or uncheck if you plan to install it yourself. If your local laws permit you to deploy unlimited strength encryption and you are running a secure cluster, check the Install Java Unlimited Strength Encryption Policy Files checkbox. Click Continue to proceed to the Upgrade Cloudera Manager Agent Packages screen.. 4. Specify credentials and initiate Agent installation: a. Select root or enter the user name for an account that has password-less sudo permission. 50 Cloudera Manager Administration Guide

51 b. Select an authentication method: If you choose to use password authentication, enter and confirm the password. If you choose to use public-key authentication provide a passphrase and path to the required key files. c. You can choose to specify an alternate SSH port. The default value is 22. d. You can specify the maximum number of host installations to run at once. The default value is Click Continue. The Cloudera Manager Agent packages are installed. 6. Click Continue. The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If there are problems, you can make changes and then re-run the inspector. When you are satisfied with the inspection results, click Finish. Manually install Agent software 1. On all cluster hosts except the Cloudera Manager server host, stop the Agent: $ sudo service cloudera-scm-agent stop Upgrading Cloudera Manager 2. In the Cloudera Admin Console, select No, I would like to skip the agent upgrade now and click Continue. 3. Copy the appropriate repo file as described in Upgrade Cloudera Manager Server Packages on page Run the following commands: Operating System RHEL Commands $ sudo yum clean all $ sudo yum upgrade 'cloudera-*' Note: yum clean all cleans up yum's cache directories, ensuring that you download and install the latest versions of the packages If your system is not up to date, and any underlying system components need to be upgraded before this yum update can succeed. yum will tell you what those are. SLES Ubuntu or Debian $ sudo zypper clean --all $ sudo zypper up -r To download from your own repository: $ sudo zypper clean --all $ sudo zypper rr cm $ sudo zypper ar -t rpm-md $ sudo zypper up -r Use the following commands to clean cached repository information and update Cloudera Manager components: $ sudo apt-get clean $ sudo apt-get update $ sudo apt-get dist-upgrade $ sudo apt-get install cloudera-manager-agent cloudera-manager-daemons As this process proceeds, you may be prompted concerning your configuration file version: Configuration file `/etc/cloudera-scm-agent/config.ini' ==> Modified (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it? Your options are: Cloudera Manager Administration Guide 51

52 Upgrading Cloudera Manager Operating System Commands Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. You will receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both these prompts. The config.ini file should be carefully inspected and the files merged together to ensure the new entries are incorporated. 5. On all cluster hosts, start the Agent: $ sudo service cloudera-scm-agent start 3. Click Continue. The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If there are problems, you can make changes and then re-run the inspector. When you are satisfied with the inspection results, click Finish. 4. Review the configuration changes to be applied. Confirm the settings entered for file system paths. The file paths required vary based on the services to be installed. Warning: DataNode data directories should not be placed on NAS devices. Click Continue. The wizard starts the services. 5. Click Finish. All services (except for the services you stopped in Stop Selected Services and Roles on page 47) should be running. Verify the Upgrade Succeeded If the commands to update and start the Cloudera Manager Server complete without errors, you can assume the upgrade has completed as desired. For additional assurance, you can check that the server versions have been updated. 1. In the Cloudera Manager Admin Console, click the Hosts tab. 2. Click Host Inspector. On large clusters, the host inspector may take some time to finish running. You must wait for the process to complete before proceeding to the next step. 3. Click Show Inspector Results. All results from the host inspector process are displayed including the currently installed versions. If this includes listings of current component versions, the installation completed as expected. (Optional) Configure SSL for Cloudera Management Service If you have enabled TLS security for the Cloudera Manager Admin Console, as of Cloudera Manager 5.1 Cloudera Management Service roles will try to communicate with Cloudera Manager using TLS, and will fail to start until SSL properties have been configured. Configure Cloudera Management Service roles to communicate with Cloudera Manager over SSL as follows: 1. Do one of the following: Select Clusters > Cloudera Management Service > Cloudera Management Service. On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link. 2. Click the Configuration tab. 3. Expand the Service-Wide > Security category. 52 Cloudera Manager Administration Guide

53 Upgrading Cloudera Manager 4. Edit the following SSL properties according to your cluster configuration. Property SSL Client Truststore File Location SSL Client Truststore File Password Description Path to the client truststore file used in HTTPS communication. The contents of this truststore can be modified without restarting the Cloudera Management Service roles. By default, changes to its contents are picked up within ten seconds. Password for the client truststore file. 5. Click Save Changes to commit the changes. 6. Restart the Cloudera Management Service. For more information, see HTTPS Communication in Cloudera Manager on page 40. Deploy JDK Upgrade If you upgraded the JDK when installing the Cloudera Manager Agents, do the following: 1. If the Cloudera Manager Server host is also running a Cloudera Manager Agent, restart the Cloudera Manager Server: $ sudo service cloudera-scm-server restart If the Cloudera Manager Server does not start, see Troubleshooting Installation and Upgrade Problems. 2. Restart all services: a. From the Home page click next to the cluster name and select Restart. b. In the confirmation dialog that displays, click Restart. Start Selected Services and Roles Start services you shut down in Stop Selected Services and Roles on page 47: 1. If you are running Cloudera Navigator, start the following roles of audited services: HDFS - NameNode HBase - Master and RegionServers Hive - HiveServer2 Hue - Beeswax Server 2. From the Home page click next to the name of each service you shut down and select Start. 3. In the confirmation dialog that displays, click Start. 4. From the Home page click next to the Cloudera Management Service and select Start. 5. In the confirmation dialog that displays, click Start. Deploy Updated Client Configurations The services whose client configurations require redeployment are indicated with icon on the Home page Status tab. To ensure clients have current information about resources, update client configuration: 1. From the Home page click next to the cluster name and select Deploy Client Configuration. 2. In the confirmation dialog that displays, click Deploy Client Configuration. Cloudera Manager Administration Guide 53

54 Upgrading Cloudera Manager Test the Installation When you have finished the upgrade to Cloudera Manager, you can test the installation to verify that the monitoring features are working as expected; follow instructions under Testing the Installation. (Optional) Upgrade CDH Cloudera Manager 5 can manage both CDH 4 and CDH 5, so upgrading existing CDH 4 installations is not required, but you may want to upgrade to the latest version. For more information on upgrading CDH, see Upgrading CDH and Managed Services. Upgrading Cloudera Manager 4 to Cloudera Manager 5 Required Role: This process applies to upgrading all versions of Cloudera Manager 4 to Cloudera Manager 5. In most cases it is possible to complete the following upgrade without shutting down most CDH services, although you may need to stop some dependent services. CDH daemons can continue running, unaffected, while Cloudera Manager is upgraded. The upgrade process does not affect your CDH installation. However, to take advantage of Cloudera Manager 5 features, after the upgrade all services must be restarted. After upgrading Cloudera Manager you may also want to upgrade CDH 4 clusters to CDH 5. Upgrading from a version of Cloudera Manager 4 to the latest version of Cloudera Manager involves the following broad steps. 54 Cloudera Manager Administration Guide

55 Upgrading Cloudera Manager Review Warnings and Notes Warning: Cloudera Management Service databases Cloudera Manager 5 stores Host Monitor and Service Monitor data in a local datastore instead of in an embedded PostgreSQL or external database. The Cloudera Manager upgrade process automatically migrates data from existing databases to the local datastore. For further information, see Data Storage for Monitoring Data. The Host Monitor and Service Monitor databases are stored on the partition hosting /var. Ensure that you have at least 20 GB available on this partition. If you have been storing the data in an external database, you can drop those databases after upgrade completes. Cloudera Management Service SSL configuration If you have enabled TLS security for the Cloudera Manager Admin Console, as of Cloudera Manager 5.1 Cloudera Management Service roles will try to communicate with Cloudera Manager using TLS, and will fail to start until SSL properties have been configured. Impala Cloudera Manager 5 supports Impala or later. If the version of your Impala service is 1.1 or earlier, the following upgrade instructions will work, but once the upgrade has completed, you will see a validation warning for your Impala service and you will not be able to restart your Impala (or Hue) services until you upgrade your Impala service to or later. If you want to continue to use Impala 1.1 or earlier, do not upgrade to Cloudera Manager 5. Navigator If you have enabled auditing with Cloudera Navigator, during the process of upgrading to Cloudera Manager 5 auditing is suspended and is only restarted when you restart the roles of audited services. JDK upgrade If you choose to upgrade the JDK during the installation of the Cloudera Manager Agent, you must restart all services. Hard Restart of Cloudera Manager Agents Certain circumstances will require that you hard restart the Cloudera Manager Agent on each host: To deploy a fix to an issue where Cloudera Manager didn't always correctly restart services To take advantage of the maximum file descriptor feature To enable HDFS DataNodes to start if you plan to perform the step (Optional) Upgrade CDH on page 66 after upgrading Cloudera Manager Cloudera Manager Administration Guide 55

56 Upgrading Cloudera Manager Important: Hive Cloudera Manager 4.5 added support for Hive, which includes the Hive Metastore Server role type. This role manages the Metastore process when Hive is configured with a remote Metastore. When upgrading from Cloudera Manager prior to 4.5, Cloudera Manager automatically creates new Hive service(s) to capture the previous implicit Hive dependency from Hue and Impala. Your previous services will continue to function without impact. If Hue was using a Hive Metastore backed by a Derby database, then the newly created Hive Metastore Server will also use Derby. Since Derby does not allow concurrent connections, Hue will continue to work, but the new Hive Metastore Server will fail to run. The failure is harmless (because nothing uses this new Hive Metastore Server at this point) and intentional, to preserve the set of cluster functionality as it was before upgrade. Cloudera discourages the use of a Derby backed Hive Metastore due to its limitations. You should consider switching to a different supported database. Cloudera Manager provides a Hive configuration option to bypass the Hive Metastore Server. When this configuration is enabled, Hive clients, Hue, and Impala connect directly to the Hive Metastore database. Prior to Cloudera Manager 4.5, Hue and Impala connected directly to the Hive Metastore database, so the bypass mode is enabled by default when upgrading to Cloudera Manager 4.5 or later. This is to ensure the upgrade doesn't disrupt your existing setup. You should plan to disable the bypass mode, especially when using CDH 4.2 or later. Using the Hive Metastore Server is the recommended configuration and the WebHCat Server role requires the Hive Metastore Server to not be bypassed. To disable bypass mode, see Disabling Bypass Mode. Cloudera Manager 4.5 or later also supports HiveServer2 with CDH 4.2. In CDH 4 HiveServer2 is not added by default, but can be added as a new role under the Hive service (see Role Instances). In CDH 5, HiveServer2 is a mandatory role. Note: When you install on EC2 using the Cloud wizard, the wizard creates a security group that by default opens ports used by Cloudera Manager and CDH components. Before upgrading, you must manually open these ports: Upgrades from Cloudera Manager or earlier for the Cloudera Manager Event Server. Upgrades from Cloudera Manager beta 2 or earlier and for the Spark master and worker web UI ports. If you are upgrading from Cloudera Manager Free Edition 4.5 or earlier you are upgraded to Cloudera Express, which includes a number of features that were previously available only with Cloudera Enterprise. Of those features, activity monitoring requires a database. Thus, upon upgrading to Cloudera Manager 5, you must specify Activity Monitor database information. You have the option to use the embedded PostgreSQL database, which Cloudera Manager can set up automatically. Perform Prerequisite Steps Warning: Cloudera Manager 5 does not support CDH 3 and you cannot upgrade Cloudera Manager 4 to Cloudera Manager 5 if you have a cluster running CDH 3. Therefore, to upgrade CDH 3 clusters to CDH 4 using Cloudera Manager you must use Cloudera Manager 4. Perform the following before upgrading to Cloudera Manager 5: Upgrade Cloudera Manager 3.7.x to Cloudera Manager 4 - See Upgrading Cloudera Manager 3.7.x on page Cloudera Manager Administration Guide

57 Upgrade all CDH 3 clusters to CDH 4 - See Upgrading CDH 3. If you attempt to upgrade to Cloudera Manager 5 and Cloudera Manager 4 is managing a CDH 3 cluster, the Cloudera Manager 5 server will not start, and you will be notified that you must downgrade to Cloudera Manager 4. Instructions for downgrading may be found here: Reverting a Failed Cloudera Manager Upgrade on page 67. After downgrading, you must upgrade your CDH 3 cluster to CDH 4 before you can upgrade Cloudera Manager. See Upgrading CDH 3. Obtain host credentials - If you want Cloudera Manager to upgrade the Agent packages you must have SSH access and be able to log in using a root account or an account that has password-less sudo permission. See Cloudera Manager Requirements for more information. Stop running commands - Use the Admin Console to check for any running commands. You can either wait for commands to complete or abort any running commands. For more information on viewing and aborting running commands, see Viewing Running and Recent Commands. Prepare databases - See Database Considerations for Cloudera Manager Upgrades on page 44. Cloudera Manager 5 supports HDFS High Availability only with Automatic Failover. If your cluster has enabled High Availability without Automatic Failover, you must enable Automatic Failover before upgrading to Cloudera Manager 5. See Configuring HDFS High Availability. Stop Selected Services If your cluster meets any of the conditions listed in the following table, you must stop the indicated services or roles. Condition Running a version of Cloudera Manager that has the Cloudera Management Service Upgrading from Cloudera Manager 4.5 or later, and using the embedded PostgreSQL database for the Hive Metastore Procedure Stop the Cloudera Management Service. Stop the services that have a dependency on the Hive Metastore (Hue, Impala, and Hive). You will not be able to stop the Cloudera Manager Server database while these services are running. If you attempt to upgrade while the embedded database is running, the upgrade will fail. Stop services that depend on the Hive Metastore in the following order: 1. Stop the Hue and Impala services. 2. Stop the Hive service. Upgrading Cloudera Manager Running Cloudera Navigator Stop any of the following roles whose service's Queue Policy configuration (navigator.batch.queue_policy) is set to SHUTDOWN: HDFS - NameNode HBase - Master and RegionServers Hive - HiveServer2 Hue - Beeswax Server Stopping these roles renders any service depending on these roles unavailable. For the HDFS - NameNode case this implies most of the services in the cluster will be unavailable until the upgrade is finished. Stop Cloudera Manager Server, Database, and Agent 1. On the host running the Cloudera Manager Server, stop the Cloudera Manager Server: $ sudo service cloudera-scm-server stop Cloudera Manager Administration Guide 57

58 Upgrading Cloudera Manager 2. If you are using the embedded PostgreSQL database for Cloudera Manager, stop the database: sudo service cloudera-scm-server-db stop Important: If you are not running the embedded database service and you attempt to stop it, you will get a message to the effect that the service cannot be found. If instead you get a message that the shutdown failed, this means the embedded database is still running, probably due to services connected to the Hive Metastore. Do not proceed with the installation until you have stopped all your Metastore-dependent services and the database successfully shuts down (restart the Cloudera Manager server to shut down services as necessary). If you continue without solving this, your upgrade will fail and you will be left with a non-functional Cloudera Manager installation. 3. If the Cloudera Manager host is also running the Cloudera Manager Agent, stop the Cloudera Manager Agent: $ sudo service cloudera-scm-agent stop (Optional) Upgrade JDK on Cloudera Manager Server Host and Agent Hosts If you are manually upgrading the Cloudera Manager Agent packages in Upgrade Cloudera Manager Agent Packages on page 60, and you plan to upgrade to CDH 5, install the Oracle JDK on the Agent hosts following the instructions in Java Development Kit Installation. If you are not running Cloudera Manager Server on the same host as a Cloudera Manager Agent, and you want all hosts to run the same JDK version, optionally install the Oracle JDK on that host. Upgrade Cloudera Manager Server Packages 1. To upgrade the Cloudera Manager Server Packages, you can upgrade from the Cloudera repository at or you can create your own repository, as described in Understanding Custom Installation Solutions. Creating your own repository is necessary if you are upgrading a cluster that does not have access to the Internet. a. Find the Cloudera repo file for your distribution by starting at and navigating to the directory that matches your operating system. For example, for Red Hat or CentOS 6, you would navigate to Within that directory, find the repo file that contains information including the repository's base URL and GPG key. The contents of the cloudera-manager.repo file might appear as follows: [cloudera-manager] # Packages for Cloudera Manager, Version 5, on RedHat or CentOS 6 x86_64 name=cloudera Manager baseurl= gpgkey = gpgcheck = 1 For Ubuntu or Debian systems, the repo file can be found by navigating to the appropriate release directory, for example, The repo file, in this case, cloudera.list, may appear as follows: # Packages for Cloudera Manager, Version 5, on Debian 7.0 x86_64 deb wheezy-cm5 contrib deb-src wheezy-cm5 contrib b. Replace the repo file in the configuration location for the package management software for your system. 58 Cloudera Manager Administration Guide

59 Upgrading Cloudera Manager Operating System RHEL SLES Ubuntu or Debian Commands Copy cloudera-manager.repo to /etc/yum.repos.d/. Copy cloudera-manager.repo to /etc/zypp/repos.d/. Copy cloudera.list to /etc/apt/sources.list.d/. c. Run the following commands: Operating System RHEL Commands $ sudo yum clean all $ sudo yum upgrade 'cloudera-*' Note: yum clean all cleans up yum's cache directories, ensuring that you download and install the latest versions of the packages If your system is not up to date, and any underlying system components need to be upgraded before this yum update can succeed. yum will tell you what those are. SLES Ubuntu or Debian $ sudo zypper clean --all $ sudo zypper up -r To download from your own repository: $ sudo zypper clean --all $ sudo zypper rr cm $ sudo zypper ar -t rpm-md $ sudo zypper up -r Use the following commands to clean cached repository information and update Cloudera Manager components: $ sudo apt-get clean $ sudo apt-get update $ sudo apt-get dist-upgrade $ sudo apt-get install cloudera-manager-server cloudera-manager-agent cloudera-manager-daemons As this process proceeds, you may be prompted concerning your configuration file version: Configuration file `/etc/cloudera-scm-agent/config.ini' ==> Modified (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. You will receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both these prompts. The config.ini file should be carefully inspected and the files merged together to ensure the new entries are incorporated. At the end of this process you should have the following packages, corresponding to the version of Cloudera Manager you installed, on the host that will become the Cloudera Manager Server host. Cloudera Manager Administration Guide 59

60 Upgrading Cloudera Manager OS RPM-based distributions Ubuntu or Debian Packages $ rpm -qa 'cloudera-manager-*' cloudera-manager-agent cm5.p0.932.el6.x86_64 cloudera-manager-server cm5.p0.932.el6.x86_64 cloudera-manager-daemons cm5.p0.932.el6.x86_64 ~# dpkg-query -l 'cloudera-manager-*' Desired=Unknown/Install/Remove/Purge/Hold Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) / Name Version Description +++-======================-======================-============================================================ ii cloudera-manager-agent cm5.p0.932~sq The Cloudera Manager Agent ii cloudera-manager-daemo cm5.p0.932~sq Provides daemons for monitoring Hadoop and related tools. ii cloudera-manager-serve cm5.p0.932~sq The Cloudera Manager Server You may also see an entry for the cloudera-manager-server-db-2 if you are using the embedded database, and additional packages for plug-ins, depending on what was previously installed on the server host. If the cloudera-manager-server-db-2 package is installed, and you don't plan to use the embedded database, you can remove this package. Start the Cloudera Manager Server On the Cloudera Manager Server host (the system on which you installed the cloudera-manager-server package) do the following: 1. If you are using the embedded PostgreSQL database for Cloudera Manager, start the database: $ sudo service cloudera-scm-server-db start 2. Start the Cloudera Manager Server: $ sudo service cloudera-scm-server start You should see the following: Starting cloudera-scm-server: [ OK ] If the Cloudera Manager Server does not start, see Troubleshooting Installation and Upgrade Problems. Upgrade Cloudera Manager Agent Packages Important: All hosts in the cluster must have access to the Internet if you plan to use archive.cloudera.com as the source for installation files. If you do not have Internet access, create a custom repository. 1. Log in to the Cloudera Manager Admin Console. 2. Upgrade hosts using one of the following methods: Cloudera Manager installs Agent software 1. Select Yes, I would like to upgrade the Cloudera Manager Agent packages now and click Continue. 2. Select the release of the Cloudera Manager Agent to install. Normally, this will be the Matched Release for this Cloudera Manager Server. However, if you used a custom repository (that is, a repository other than archive.cloudera.com) for the Cloudera Manager server, select Custom Repository and provide 60 Cloudera Manager Administration Guide

61 the required information. The custom repository allows you to use an alternative location, but that location must contain the matched Agent version. 3. Click Continue. (Cloudera Manager 5.1.3) Leave Install Oracle Java SE Development Kit (JDK) checked to allow Cloudera Manager to install the JDK on each cluster host or uncheck if you plan to install it yourself. If your local laws permit you to deploy unlimited strength encryption and you are running a secure cluster, check the Install Java Unlimited Strength Encryption Policy Files checkbox. Click Continue to proceed to the Upgrade Cloudera Manager Agent Packages screen.. 4. Specify credentials and initiate Agent installation: a. Select root or enter the user name for an account that has password-less sudo permission. b. Select an authentication method: If you choose to use password authentication, enter and confirm the password. If you choose to use public-key authentication provide a passphrase and path to the required key files. c. You can choose to specify an alternate SSH port. The default value is 22. d. You can specify the maximum number of host installations to run at once. The default value is Click Continue. The Cloudera Manager Agent packages are installed. 6. Click Continue. The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If there are problems, you can make changes and then re-run the inspector. When you are satisfied with the inspection results, click Finish. Manually install Agent software 1. On all cluster hosts except the Cloudera Manager server host, stop the Agent: $ sudo service cloudera-scm-agent stop Upgrading Cloudera Manager 2. In the Cloudera Admin Console, select No, I would like to skip the agent upgrade now and click Continue. 3. Copy the appropriate repo file as described in Upgrade Cloudera Manager Server Packages on page Run the following commands: Operating System RHEL Commands $ sudo yum clean all $ sudo yum upgrade 'cloudera-*' Note: yum clean all cleans up yum's cache directories, ensuring that you download and install the latest versions of the packages If your system is not up to date, and any underlying system components need to be upgraded before this yum update can succeed. yum will tell you what those are. SLES $ sudo zypper clean --all $ sudo zypper up -r To download from your own repository: $ sudo zypper clean --all $ sudo zypper rr cm $ sudo zypper ar -t rpm-md $ sudo zypper up -r Cloudera Manager Administration Guide 61

62 Upgrading Cloudera Manager Operating System Ubuntu or Debian Commands Use the following commands to clean cached repository information and update Cloudera Manager components: $ sudo apt-get clean $ sudo apt-get update $ sudo apt-get dist-upgrade $ sudo apt-get install cloudera-manager-agent cloudera-manager-daemons As this process proceeds, you may be prompted concerning your configuration file version: Configuration file `/etc/cloudera-scm-agent/config.ini' ==> Modified (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. You will receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both these prompts. The config.ini file should be carefully inspected and the files merged together to ensure the new entries are incorporated. 5. On all cluster hosts, start the Agent: $ sudo service cloudera-scm-agent start 3. If you are upgrading from a free version of Cloudera Manager prior to 4.6: a. Click Continue to assign the Cloudera Management Services roles to hosts. b. If you are upgrading to Cloudera Enterprise, specify required databases: a. Configure database settings: a. Choose the database type: Leave the default setting of Use Embedded Database to have Cloudera Manager create and configure required databases. Make a note of the auto-generated passwords. Select Use Custom Databases to specify external databases. 1. Enter the database host, database type, database name, username, and password for the database that you created when you set up the database. b. Click Test Connection to confirm that Cloudera Manager can communicate with the database using the information you have supplied. If the test succeeds in all cases, click Continue; otherwise check and correct the information you have provided for the database and then try the test again. (For some servers, if you are using the embedded database, you will see a message saying the database will be created at a later step in the installation process.) The Review Changes page displays. 4. Review the configuration changes to be applied. Confirm the settings entered for file system paths. The file paths required vary based on the services to be installed. Click Finish. 5. If you are upgrading from Cloudera Manager prior to 4.5: a. Select the host for the Hive Metastore Server role. b. Review the configuration values and click Accept to continue. 62 Cloudera Manager Administration Guide

63 Upgrading Cloudera Manager Note: If Hue was using a Hive Metastore backed by a Derby database, then the newly created Hive Metastore Server will also use Derby. Since Derby does not allow concurrent connections, Hue will continue to work, but the new Hive Metastore Server will fail to run. The failure is harmless (because nothing uses this new Hive Metastore Server at this point) and intentional, to preserve the set of cluster functionality as it was before upgrade. Cloudera discourages the use of a Derby backed Hive Metastore due to its limitations. You should consider switching to a different supported database. Prior to Cloudera Manager 4.5, Hue and Impala connected directly to the Hive Metastore database, so the bypass mode is enabled by default when upgrading to Cloudera Manager 4.5 or later. This is to ensure the upgrade doesn't disrupt your existing setup. You should plan to disable the bypass mode, especially when using CDH 4.2 or later. Using the Hive Metastore Server is the recommended configuration and the WebHCat Server role requires the Hive Metastore Server to not be bypassed. To disable bypass mode, see Disabling Bypass Mode. After changing this configuration, you must re-deploy your client configurations, restart Hive, and restart any Hue or Impala services configured to use that Hive. If you are using CDH 4.0 or CDH 4.1, see known issues related to Hive in Known Issues and Workarounds in Cloudera Manager If you are upgrading from Cloudera Manager prior to 4.8, and have an Impala service, assign the Impala Catalog Server role to a host. All services (except for the services you stopped in Stop Selected Services on page 57) should be running. Verify the Upgrade Succeeded If the commands to update and start the Cloudera Manager Server complete without errors, you can assume the upgrade has completed as desired. For additional assurance, you can check that the server versions have been updated. 1. In the Cloudera Manager Admin Console, click the Hosts tab. 2. Click Host Inspector. On large clusters, the host inspector may take some time to finish running. You must wait for the process to complete before proceeding to the next step. 3. Click Show Inspector Results. All results from the host inspector process are displayed including the currently installed versions. If this includes listings of current component versions, the installation completed as expected. Add Hive Gateway Roles If you are upgrading from a release prior to Cloudera Manager 4.5, add Hive gateway roles to any hosts where Hive clients should run. 1. Go to the Hive service. 2. Click the Instances tab. 3. Click the Add Role Instances button. 4. Select the hosts on which you want a Hive gateway role to run. This will ensure that the Hive client configurations are deployed on these hosts. Configure Cluster Version for Package Installs If you have installed CDH as a package, after an install or upgrade make sure that the cluster CDH version matches the package CDH version, using the procedure in Configuring the CDH Version for a Cluster in Managing Clusters with Cloudera Manager. If the cluster CDH version does not match the package CDH version, Cloudera Manager will incorrectly enable and disable service features based on the cluster's configured CDH version. Cloudera Manager Administration Guide 63

64 Upgrading Cloudera Manager Upgrade Impala If your version of Impala was 1.1 or earlier, upgrade to Impala or later. (Optional) Configure SSL for Cloudera Management Service If you have enabled TLS security for the Cloudera Manager Admin Console, as of Cloudera Manager 5.1 Cloudera Management Service roles will try to communicate with Cloudera Manager using TLS, and will fail to start until SSL properties have been configured. Configure Cloudera Management Service roles to communicate with Cloudera Manager over SSL as follows: 1. Do one of the following: Select Clusters > Cloudera Management Service > Cloudera Management Service. On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link. 2. Click the Configuration tab. 3. Expand the Service-Wide > Security category. 4. Edit the following SSL properties according to your cluster configuration. Property SSL Client Truststore File Location SSL Client Truststore File Password Description Path to the client truststore file used in HTTPS communication. The contents of this truststore can be modified without restarting the Cloudera Management Service roles. By default, changes to its contents are picked up within ten seconds. Password for the client truststore file. 5. Click Save Changes to commit the changes. 6. Restart the Cloudera Management Service. For more information, see HTTPS Communication in Cloudera Manager on page 40. (Optional) Deploy Cloudera Manager Agent Upgrade Several conditions require that you hard restart the Cloudera Manager Agents: To deploy a fix to an issue where Cloudera Manager didn't always correctly restart services To take advantage of the maximum file descriptor feature To enable HDFS DataNodes to start if you plan to perform the step (Optional) Upgrade CDH on page 66 after upgrading Cloudera Manager To address any of these conditions, at a convenient time perform the following steps: 1. Stop all CDH and managed services. 2. On all hosts with Cloudera Manager Agents, run the command: $ sudo service cloudera-scm-agent hard_restart Before performing this step, ensure you understand the semantics of the hard_restart command by reading Hard Stopping and Restarting Agents on page Start all services. Deploy JDK Upgrade If you upgraded the JDK when installing the Cloudera Manager Agents, do the following: 64 Cloudera Manager Administration Guide

65 1. If the Cloudera Manager Server host is also running a Cloudera Manager Agent, restart the Cloudera Manager Server: $ sudo service cloudera-scm-server restart If the Cloudera Manager Server does not start, see Troubleshooting Installation and Upgrade Problems. 2. If you have not restarted services in previous steps, restart all services: a. From the Home page click next to the cluster name and select Restart. b. In the confirmation dialog that displays, click Restart. Upgrading Cloudera Manager (Optional) Deploy Monitoring Upgrade Cloudera Manager 5 has added monitoring support for all roles that were not previously monitored. However, the Cloudera Manager Agent will not start sending monitoring data for these roles until: 1. The Cloudera Manager Agent has been upgraded and restarted. 2. The monitored roles have been restarted. Until you restart the roles, some data will not be present in the monitoring charts and health tests. This is generally innocuous and simply results in charts that do not display any data and unknown health tests. To enable monitoring for all roles, if you have not restarted services in previous steps, at a convenient time perform the following steps: 1. Restart all services. From the Home page click next to the cluster name and select Restart. 2. In the confirmation dialog that displays, click Restart. 3. From the Home page click next to the Cloudera Management Service and select Start. 4. In the confirmation dialog that displays, click Start. Start Selected Services and Roles Start services and roles you shut down in Stop Selected Services on page 57 that have not been started in other steps: 1. Do one of the following, depending on which services you shut down: From the Home page click next to the cluster name and select Start. From the Home page click next to the name of each service you shut down and select Start. 2. In the confirmation dialog that displays, click Start. 3. If are running Cloudera Navigator and you haven't performed (Optional) Deploy Monitoring Upgrade on page 65, restart the following roles of audited services: HDFS - NameNode HBase - Master and RegionServers Hive - HiveServer2 Hue - Beeswax Server Deploy Updated Client Configurations The services whose client configurations require redeployment are indicated with icon on the Home page Status tab. To ensure clients have current information about resources, update client configuration: 1. From the Home page click next to the cluster name and select Deploy Client Configuration. 2. In the confirmation dialog that displays, click Deploy Client Configuration. Cloudera Manager Administration Guide 65

66 Upgrading Cloudera Manager Test the Installation When you have finished the upgrade to Cloudera Manager, you can test the installation to verify that the monitoring features are working as expected; follow instructions under Testing the Installation. (Optional) Upgrade CDH Cloudera Manager 5 can manage both CDH 4 and CDH 5, so upgrading existing CDH 4 installations is not required, but you may want to upgrade to the latest version. For more information on upgrading CDH, see Upgrading CDH and Managed Services. Upgrading Cloudera Manager 3.7.x Warning: Cloudera Manager 3 and CDH 3 have reached End of Maintenance (EOM) as of June 20, Cloudera does not support or provide patches for Cloudera Manager 3 and CDH 3 releases. You cannot upgrade directly from Cloudera Manager 3.7.x to Cloudera Manager 5; you must upgrade to Cloudera Manager 4 first before upgrading to Cloudera Manager 5. Follow the instructions for upgrading Cloudera Manager 3.7.x to Cloudera Manager 4 in Upgrade Cloudera Manager 3.7.x to the Latest Cloudera Manager. Note that the last step in the Cloudera Manager upgrade process is an optional step to upgrade CDH. If you are running CDH 3, this step is not optional. Cloudera Manager 5 does not support CDH 3 and will not allow you to complete the upgrade if it detects a CDH 3 cluster. You must upgrade to CDH 4 before you can upgrade to Cloudera Manager 5. Follow the steps at Upgrading CDH 3 to CDH 4 in a Cloudera Manager Deployment before you attempt to upgrade your Cloudera Manager Server to version 5. Re-Running the Cloudera Manager Upgrade Wizard Required Role: The first time you log in to the Cloudera Manager server after upgrading your Cloudera Manager software, the upgrade wizard runs. If you did not complete the wizard at that time, or if you had hosts that were unavailable at that time and still need to be upgraded, you can re-run the upgrade wizard: 1. Click the Hosts tab. 2. Click Re-run Upgrade Wizard. This takes you back through the installation wizard to upgrade Cloudera Manager Agents on your hosts as necessary. 3. Select the release of the Cloudera Manager Agent to install. Normally, this will be the Matched Release for this Cloudera Manager Server. However, if you used a custom repository (that is, a repository other than archive.cloudera.com) for the Cloudera Manager server, select Custom Repository and provide the required information. The custom repository allows you to use an alternative location, but that location must contain the matched Agent version. 4. Specify credentials and initiate Agent installation: a. Select root or enter the user name for an account that has password-less sudo permission. b. Select an authentication method: If you choose to use password authentication, enter and confirm the password. If you choose to use public-key authentication provide a passphrase and path to the required key files. c. You can choose to specify an alternate SSH port. The default value is 22. d. You can specify the maximum number of host installations to run at once. The default value is Cloudera Manager Administration Guide

67 Upgrading Cloudera Manager When you click Continue the Cloudera Manager Agent is upgraded on all the currently managed hosts. You cannot search for new hosts through this process. To add hosts to your cluster, click the Add New Hosts to Cluster button. Reverting a Failed Cloudera Manager Upgrade If you have a CDH 3 cluster running under Cloudera Manager 4, you cannot upgrade to Cloudera Manager 5 because it does not support CDH 3. Likewise, an upgrade from Cloudera Manager 3 to Cloudera Manager 5 is not supported. In either case, the Cloudera Manager 5 server will not start, and you must now downgrade your Cloudera Manager server, back to the version you were using prior to attempting the upgrade. Important: The following instructions assume that a Cloudera Manager upgrade failed, and that the upgraded server never started, so that the remaining steps of the upgrade process were not performed. The steps below are not sufficient to revert from a running Cloudera Manager 5 deployment. Reinstall the Cloudera Manager Server Packages In this step, you install the Cloudera Manager Server packages to the version you were running previously. You must reinstall the same version of Cloudera Manager you were using previously, so that the version of your Cloudera Manager Agents match the server. The steps below assume that the Cloudera Manager Server is already stopped (as it failed to start after the attempted upgrade). 1. If you are using the embedded PostgreSQL database for Cloudera Manager, stop the database on the Cloudera Manager Server host: $ sudo service cloudera-scm-server-db fast_stop 2. Reinstall the same Cloudera Manager Server version that you were previously running. You can reinstall from the Cloudera repository at or alternately, you can create your own repository, as described in Understanding Custom Installation Solutions. a. Find the Cloudera repo file for your distribution by starting at and navigating to the directory that matches your operating system. For example, for Red Hat or CentOS 6, you would navigate to Within that directory, find the repo file that contains information including the repository's base URL and GPG key. On CentOS 6, the contents of the cloudera-manager.repo file might appear as follows: [cloudera-manager] # Packages for Cloudera Manager, Version 4, on RedHat or CentOS 6 x86_64 name=cloudera Manager baseurl= gpgkey = gpgcheck = 1 For Ubuntu or Debian systems, the repo file can be found by navigating to the appropriate directory, for example, The repo file, in this case, cloudera.list, may appear as follows: # Packages for Cloudera's Distribution for Hadoop, Version 4, on Debian 6.0 x86_64 deb squeeze-cm4 contrib deb-src squeeze-cm4 contrib Cloudera Manager Administration Guide 67

68 Upgrading Cloudera Manager You must edit the file if it exist and modify the URL to reflect the exact version of Cloudera Manager you are using (unless you want the downgrade to also upgrade to the latest version of Cloudera Manager 4). The possible versions are shown in the directory on archive. Setting the URL (an example): OS RHEL Ubuntu or Debian Command Replace baseurl= with baseurl= Replace deb squeeze-cm4 contrib with deb squeeze-cm4.7.3 contrib b. Copy the repo file to the configuration location for the package management software for your system: Operating System RHEL SLES Ubuntu or Debian Commands Copy cloudera-manager.repo to /etc/yum.repos.d/. Copy cloudera-manager.repo to /etc/zypp/repos.d/. Copy cloudera.list to /etc/apt/sources.list.d/. c. Run the following commands: Operating System RHEL SLES Ubuntu or Debian Commands $ sudo yum downgrade 'cloudera-*' $ sudo zypper clean --all $ sudo zypper dup -r To download from your own repository: $ sudo zypper clean --all $ sudo zypper dup -r There's no action that will downgrade to the version currently in the repository. Read DowngradeHowto, download the script described therein, run it, and then run apt-get install for the name=version pairs that it provides for Cloudera Manager. At the end of this process you should have the following packages, corresponding to the version of Cloudera Manager you installed, on the Cloudera Manager Server host. For example, for CentOS, $ rpm -qa 'cloudera-manager-*' cloudera-manager-daemons cm473.p0.163.el6.x86_64 cloudera-manager-server cm473.p0.163.el6.x86_64 cloudera-manager-agent cm473.p0.163.el6.x86_64 For Ubuntu or Debian, you should have packages similar to those shown below. ~# dpkg-query -l 'cloudera-manager-*' Desired=Unknown/Install/Remove/Purge/Hold Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend / Err?=(none)/Reinst-required (Status,Err: uppercase=bad) / Name Version Description +++-======================-======================-============================================================ ii cloudera-manager-agent cm473.p0.163~sq The Cloudera Manager Agent 68 Cloudera Manager Administration Guide

69 Upgrading Cloudera Manager ii cloudera-manager-daemo cm473.p0.163~sq Provides daemons for monitoring Hadoop and related tools. ii cloudera-manager-serve cm473.p0.163~sq The Cloudera Manager Server You may also see an entry for the cloudera-manager-server-db if you are using the embedded database, and additional packages for plug-ins, depending on what was previously installed on the server host. If the commands to update the server complete without errors, you can assume the upgrade has completed as desired. For additional assurance, you will have the option to check that the server versions have been updated after you start the server. Start the Server On the Cloudera Manager Server host (the system on which you installed the cloudera-manager-server package) do the following: 1. If you are using the embedded PostgreSQL database for Cloudera Manager, start the database: $ sudo service cloudera-scm-server-db start 2. Start the server: $ sudo service cloudera-scm-server start You should see the following: Starting cloudera-scm-server: [ OK ] Note: If you have problems starting the server, such as database permissions problems, you can use the server's log /var/log/cloudera-scm-server/cloudera-scm-server.log to troubleshoot the problem. Cloudera Manager Administration Guide 69

70

71 Other Cloudera Manager Settings Other Cloudera Manager Settings From the Administration tab you can select options for configuring settings that affect how Cloudera Manager interacts with your cluster. Administration Settings The Settings page provides a number of categories as follows: Performance - Set the Cloudera Manager Agent heartbeat interval. Advanced - Enable API debugging and other advanced options. Monitoring - Set Agent health status parameters. For configuration instructions, see Configuring Cloudera Manager Agents on page 11. Security - Set TLS encryption settings to enable TLS encryption between the Cloudera Manager Server, Agents, and clients. For configuration instructions, see Configuring TLS Security for Cloudera Manager on page 31. You can also: Set the realm for Kerberos security and point to a custom keytab retrieval script. For configuration instructions, see Configuring Hadoop Security with Cloudera Manager. Specify session timeout and a "Remember Me" option. Ports and Addresses - Set ports for the Cloudera Manager Admin Console and Server. For configuration instructions, see Configuring Cloudera Manager Server Ports on page 9. Other Enable Cloudera usage data collection For configuration instructions, see Managing Anonymous Usage Data Collection on page 78. Set a custom header color and banner text for the Admin console. Set an "Information Assurance Policy" statement this statement will be presented to every user before they are allowed to access the login dialog. The user must click "I Agree" in order to proceed to the login dialog. Disable/enable the auto-search for the Events panel at the bottom of a page. Support Configure diagnostic data collection properties. See Diagnostic Data Collection on page 79. Configure how to access Cloudera Manager help files. By default, when you click the Help link under the Support menu in the Cloudera Manager Admin console, Help files from the Cloudera web site are opened. This is because local Help files are not updated after installation. You can configure Cloudera Manager to open either the latest Help files from the Cloudera web site (this option requires Internet access from the browser) or locally-installed Help files by configuring the property Open latest Help files from the Cloudera website. External Authentication - Specify the configuration to use LDAP, Active Directory, or an external program for authentication. See Configuring External Authentication on page 25 for instructions. Parcels - Configure settings for parcels, including the location of remote repositories that should be made available for download, and other settings such as the frequency with which Cloudera Manager will check for new parcels, limits on the number of downloads or concurrent distribution uploads. See Parcels for more information. Network - Configure proxy server settings. Custom Service Descriptors - Configure custom service descriptor properties for Add-on Services. Cloudera Manager Administration Guide 71

72 Other Cloudera Manager Settings User Interface Language Settings You can change the language of the Cloudera Manager Admin Console User Interface through the language preference in your browser. Information on how to do this for the browsers supported by Cloudera Manager is shown under the Administration page. You can also change the language for the information provided with activity and health events, and for alert messages by selecting Language, selecting the language you want from the drop-down list on this page, then clicking Save Changes. Managing Licenses Required Role: When you install Cloudera Manager, you can choose to select Cloudera Express (no license required), a 60-day Cloudera Enterprise Data Hub Edition trial license, or Cloudera Enterprise (which requires a license). You can later end a trial license or upgrade your license. About Trial Licenses You can use the trial license only once; once the 60-day trial period has expired or you have ended the trial, you cannot restart it. When a trial ends, features that require a Cloudera Enterprise license immediately become unavailable. However, data or configurations associated with the disabled functions are not deleted, and become available again when you install a Cloudera Enterprise license. Trial expiration or termination has the following effects: Only local users can log in (no LDAP or SAML authentication). Configuration history is unavailable. Alerts cannot be delivered as SNMP traps. Operational reports are inaccessible (but remain in the database). Commands such as Rolling Restart, History and Rollback (under the Configuration tab), Send Diagnostic Data, and starting Cloudera Navigator roles are disabled or not available. Accessing the License Page To access the license page, select Administration > License. If you have a license installed, the license page indicates its status (for example, whether your license is currently valid) and displays the license details: the license owner, the license key, and the expiration date of the license, if there is one. At the right side of the page a table shows the usage of licensed components based on the number of hosts with those products installed. You can move the cursor over the to see an explanation of each item. Basic Edition - a cluster running core CDH services: HDFS, Hive, Hue, MapReduce, Oozie, Sqoop, YARN, and ZooKeeper. Flex Edition - a cluster running core CDH services plus one of the following: Accumulo, HBase, Impala, Navigator, Solr, Spark. Data Hub Edition - a cluster running core CDH services plus any of the following: Accumulo, HBase, Impala, Navigator, Solr, Spark. Ending a Cloudera Enterprise Data Hub Edition Trial If you are using the trial edition the License page indicates when your license will expire. However, you can end the trial at any time (prior to expiration) as follows: 1. On the License page, click End Trial. 72 Cloudera Manager Administration Guide

73 2. Confirm that you want to end the trial. 3. Restart the Cloudera Management Service, HBase, HDFS, and Hive services to pick up configuration changes. Upgrading from Cloudera Express to a Cloudera Enterprise Data Hub Edition Trial To start a trial, on the License page, click Try Cloudera Enterprise Data Hub Edition for 60 Days. 1. Cloudera Manager displays a pop-up describing the features enabled with Cloudera Enterprise Data Hub Edition. Click OK to proceed. At this point, your installation is upgraded and the Customize Role Assignments page displays. 2. Under Reports Manager click Select a host. The pageable host selection dialog displays. The following shortcuts for specifying hostname patterns are supported: Range of hostnames (without the domain portion) Other Cloudera Manager Settings Range Definition [1-4] host[1-3].company.com host[07-10].company.com Matching Hosts , , , host1.company.com, host2.company.com, host3.company.com host07.company.com, host08.company.com, host09.company.com, host10.company.com IP addresses Rack name 3. Select a host and click OK. 4. When you are satisfied with the assignments, click Continue. The Database Setup page displays. 5. Configure database settings: a. Choose the database type: Leave the default setting of Use Embedded Database to have Cloudera Manager create and configure required databases. Make a note of the auto-generated passwords. Select Use Custom Databases to specify external databases. 1. Enter the database host, database type, database name, username, and password for the database that you created when you set up the database. b. Click Test Connection to confirm that Cloudera Manager can communicate with the database using the information you have supplied. If the test succeeds in all cases, click Continue; otherwise check and correct the information you have provided for the database and then try the test again. (For some servers, if you are using the embedded database, you will see a message saying the database will be created at a later step in the installation process.) The Review Changes page displays. 6. Review the configuration changes to be applied. Confirm the settings entered for file system paths. The file paths required vary based on the services to be installed. Warning: DataNode data directories should not be placed on NAS devices. Click Continue. The wizard starts the services. 7. At this point, your installation is upgraded. Click Continue. 8. Restart Cloudera Management Services and audited services to pick up configuration changes. The audited services will write audit events to a log file, but the events are not transferred to the Cloudera Navigator Audit Server until you add and start the Cloudera Navigator Audit Server role as described in Adding and Starting Cloudera Navigator Roles on page 20. For information on Cloudera Navigator, see Cloudera Navigator documentation. Cloudera Manager Administration Guide 73

74 Other Cloudera Manager Settings Upgrading from a Cloudera Enterprise Data Hub Edition Trial to Cloudera Enterprise 1. Purchase a Cloudera Enterprise license from Cloudera. 2. On the License page, click Upload License. 3. Click the document icon to the left of the Select a License File text field. 4. Navigate to the location of your license file, click the file, and click Open. 5. Click Upload. Upgrading from Cloudera Express to Cloudera Enterprise 1. Purchase a Cloudera Enterprise license from Cloudera. 2. On the License page, click Upload License. 3. Click the document icon to the left of the Select a License File text field. 4. Navigate to the location of your license file, click the file, and click Open. 5. Click Upload. 6. Cloudera Manager displays a pop-up describing the features enabled with Cloudera Enterprise Data Hub Edition. Click OK to proceed. At this point, your installation is upgraded and the Customize Role Assignments page displays. 7. Under Reports Manager click Select a host. The pageable host selection dialog displays. The following shortcuts for specifying hostname patterns are supported: Range of hostnames (without the domain portion) Range Definition [1-4] host[1-3].company.com host[07-10].company.com Matching Hosts , , , host1.company.com, host2.company.com, host3.company.com host07.company.com, host08.company.com, host09.company.com, host10.company.com IP addresses Rack name 8. When you are satisfied with the assignments, click Continue. The Database Setup page displays. 9. Configure database settings: a. Choose the database type: Leave the default setting of Use Embedded Database to have Cloudera Manager create and configure required databases. Make a note of the auto-generated passwords. Select Use Custom Databases to specify external databases. 1. Enter the database host, database type, database name, username, and password for the database that you created when you set up the database. b. Click Test Connection to confirm that Cloudera Manager can communicate with the database using the information you have supplied. If the test succeeds in all cases, click Continue; otherwise check and correct the information you have provided for the database and then try the test again. (For some servers, if you are using the embedded database, you will see a message saying the database will be created at a later step in the installation process.) The Review Changes page displays. 10. Review the configuration changes to be applied. Confirm the settings entered for file system paths. The file paths required vary based on the services to be installed. Warning: DataNode data directories should not be placed on NAS devices. Click Continue. The wizard starts the services. 74 Cloudera Manager Administration Guide

75 11. At this point, your installation is upgraded. Click Continue. 12. Restart Cloudera Management Services and audited services to pick up configuration changes. The audited services will write audit events to a log file, but the events are not transferred to the Cloudera Navigator Audit Server until you add and start the Cloudera Navigator Audit Server role as described in Adding and Starting Cloudera Navigator Roles on page 20. For information on Cloudera Navigator, see Cloudera Navigator documentation. If you want to use the Cloudera Navigator Metadata Server, add its role following the instructions in Adding and Starting Cloudera Navigator Roles on page 20. Renewing a License 1. Download the license file and save it locally. 2. In Cloudera Manager, go to the Home page. 3. Select Administration > License. 4. Click Upload License. 5. Browse to the license file you downloaded. 6. Click Upload. You do not need to restart Cloudera Manager for the new license to take effect. Other Cloudera Manager Settings Managing Alerts Required Role: The Administration > Alerts page provides a summary of the settings for alerts in your clusters. Alert Type The left column lets you select by alert type (Health, Log, or Activity) and within that by service instance. In the case of Health alerts, you can look at alerts for Hosts as well. You can select an individual service to see just the alert settings for that service. Health/Log/Activity Alert Settings Depending on your selection in the left column, the right hand column show you the list of alerts that are enabled or disabled for the selected service type. To change the alert settings for a service, click the next to the service name. This will take you to the Monitoring section of the Configuration tab for the service. From here you can enable or disable alerts and configure thresholds as needed. Recipients You can also view the list of recipients configured for the enabled alerts. Configuring Alert Delivery When you install Cloudera Manager you can configure the mail server you will use with the Alert Publisher. However, if you need to change these settings, you can do so under the Alert Publisher section of the Management Services configuration tab. Under the Alert Publisher role of the Cloudera Manager Management Service, you can configure or SNMP delivery of alert notifications. Configuring Alert Delivery Required Role: Sending A Test Alert Select the Administration > Alerts tab and click the Send Test Alert link. Configuring the List Of Alert Recipient Addresses 1. Do one of the following: Cloudera Manager Administration Guide 75

76 Other Cloudera Manager Settings Select the Administration > Alerts tab and click the 1. Do one of the following: to the right of Recipient(s). Select Clusters > Cloudera Management Service > Cloudera Management Service. On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link. 2. Click the Configuration tab. 3. Select the Alert Publisher Default Group role group. 2. Configure the Alerts: Mail Message Recipients property. 3. Click the Save Changes button at the top of the page to save your settings. 4. Restart the Alert Publisher role. Configuring Alert Properties 1. Display the Cloudera Management Service status page. 2. Click the Configuration tab. 3. Select the Alert Publisher Default Group role group to see the list of properties. In order to receive alerts you must set (or verify) the following settings: Enable alerts protocol to use. Your mail server hostname and port. The username and password of the user that will be logged into the mail server as the "sender" of the alert s. A comma-separated list of addresses that will be the recipients of alert s. The format of the alert message. Select json if you need the message to be parsed by a script or program. 4. Click the Save Changes button at the top of the page to save your settings. 5. Restart the Alert Publisher role. Configuring Alert SNMP Delivery Required Role: Important: This feature is available only with a Cloudera Enterprise license. For other licenses, the following applies: Cloudera Express - The feature is not available. Cloudera Enterprise Data Hub Edition Trial - The feature is available until you end the trial or the trial license expires. To obtain a license for Cloudera Enterprise, please fill in this form or call After you install a Cloudera Enterprise license, the feature will be available. Enabling, Configuring, and Disabling SNMP Traps 1. Before you enable SNMP traps, configure the trap receiver (Network Management System or SNMP server) with the Cloudera MIB. 2. Do one of the following: Select Clusters > Cloudera Management Service > Cloudera Management Service. On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link. 76 Cloudera Manager Administration Guide

77 3. Click the Configuration tab. 4. Select Alert Publisher Default Group > SNMP. Enter the DNS name or IP address of the Network Management System (SNMP server) acting as the trap receiver in the SNMP NMS Hostname property. In the SNMP Security Level property, select the version of SNMP you are using: SNMPv2, SNMPv3 without authentication and without privacy (noauthnopriv), or SNMPv3 with authentication and without privacy (authnopriv) and specify the required properties: SNMPv2 - SNMPv2 Community String. SNMPv3 without authentication (noauthnopriv) - SNMP Server Engine Id and SNMP Security UserName. SNMPv3 with authentication (authnopriv) - SNMP Server Engine Id, SNMP Security UserName, SNMP Authentication Protocol, and SNMP Authentication Protocol Pass Phrase. You can also change other settings such as the port, retry, or timeout values. 5. Click Save Changes when you are done. 6. Restart the Alert Publisher role. To disable SNMP traps, remove the hostname from the SNMP NMS Hostname property (alert.snmp.server.hostname). Viewing the Cloudera MIB 1. Do one of the following: Select Clusters > Cloudera Management Service > Cloudera Management Service. On the Status tab of the Home page, in Cloudera Management Service table, click the Cloudera Management Service link. 2. Click the Configuration tab. 3. Select Alert Publisher Default Group > SNMP. 4. In the Description column for the first property (SNMP NMS Hostname) click the SMNP Mib link. Other Cloudera Manager Settings Kerberos Required Role: After enabling and configuring Hadoop security using Kerberos on your cluster, you can view and regenerate the Kerberos principals for your cluster. If you make a global configuration change in your cluster, such as changing the encryption type, you would use the Kerberos page to regenerate the principals for your cluster. In a secure cluster, the Kerberos page lists all the Kerberos principals that are active on your cluster. Regenerating Kerberos Principals If you make a global configuration change in your cluster, such as changing the encryption type, you must use the following instructions to regenerate the principals for your cluster. Cloudera Manager Administration Guide 77

78 Other Cloudera Manager Settings Important: Regenerate principals using the following steps in the Cloudera Manager Admin Console and not directly using kadmin shell. Do not regenerate the principals for your cluster unless you have made a global configuration change. Before regenerating, be sure to read Appendix B - Set up a Cluster-dedicated MIT KDC and Default Domain for the Hadoop Cluster to avoid making your existing host keytabs invalid. If you are using Active Directory, delete the AD accounts with the userprincipalname (or login names) that you want to manually regenerate before continuing with the steps below. To view and regenerate the Kerberos principals for your cluster: 1. Select Administration > Kerberos. 2. The currently configured Kerberos principals are displayed. If you are running HDFS, the hdfs/hostname and host/hostname principals are listed. If you are running MapReduce, the mapred/hostname and host/hostname principals are listed. The principals for other running services are also listed. 3. Only if necessary, select the principals you want to regenerate. 4. Click Regenerate. The Security Inspector The Security Inspector uses the Host Inspector to run a security-related set of commands on the hosts in your cluster. It reports on things such as how Java is configured for encryption and on the default realms configured on each host: 1. Select Administration > Kerberos. 2. Click Security Inspector. Cloudera Manager begins several tasks to inspect the managed hosts. 3. After the inspection completes, click Download Result Data or Show Inspector Results to review the results. Sending Usage and Diagnostic Data to Cloudera Required Role: Cloudera Manager collects anonymous usage information and takes regularly-scheduled snapshots of the state of your cluster and automatically sends them anonymously to Cloudera. This helps Cloudera improve and optimize Cloudera Manager. If you have a Cloudera Enterprise license, you can also trigger the collection of diagnostic data and send it to Cloudera Support to aid in resolving a problem you may be having. Configuring a Proxy Server To configure a proxy server through which usage and diagnostic data is uploaded, follow the instructions in Configuring Network Settings on page 81. Managing Anonymous Usage Data Collection Cloudera Manager sends anonymous usage information using Google Analytics to Cloudera. The information helps Cloudera improve Cloudera Manager. By default anonymous usage data collection is enabled. 1. Select Administration > Settings. 2. Under the Other category, set the Allow Usage Data Collection property. 3. Click Save Changes to commit the changes. 78 Cloudera Manager Administration Guide

79 Other Cloudera Manager Settings Managing Hue Analytics Data Collection Required Role: Hue tracks anonymised pages and application versions in order to gather information to help compare each application's usage levels. The data collected does not include any hostnames or IDs. For example, the data is of the form: /2.3.0/pig, /2.5.0/beeswax/execute. You can restrict data collection as follows: 1. Go to the Hue service. 2. Click the Configuration tab. 3. Expand the Service-Wide category. 4. Uncheck the Enable Usage Data Collection checkbox. 5. Click Save Changes to commit the changes. 6. Restart the Hue service. Diagnostic Data Collection To help with solving problems when using Cloudera Manager on your cluster, Cloudera Manager collects diagnostic data on a regular schedule, and automatically sends it to Cloudera. By default Cloudera Manager is configured to collect data weekly and to send it automatically. You can schedule the frequency of data collection on a daily, weekly, or monthly schedule, or disable the scheduled collection of data entirely. You can also send a collected data set manually. Note: Automatically sending diagnostic data requires the Cloudera Manager Server host to have Internet access, and be configured for sending data automatically. If your Cloudera Manager server does not have Internet access, and you have a Cloudera Enterprise license, you can manually send the diagnostic data as described in Manually Triggering Collection and Transfer of Diagnostic Data to Cloudera on page 81. Automatically sending diagnostic data may fail sometimes and return an error message of "Could not send data to Cloudera." To work around this issue, you can manually send the data to Cloudera Support. What Data Does Cloudera Manager Collect? Cloudera Manager collects and returns a significant amount of information about the health and performance of the cluster. It includes: Up to 1000 Cloudera Manager audit events: Configuration changes, add/remove of users, roles, services, etc. One day's worth of Cloudera Manager events: This includes critical errors Cloudera Manager watches for and more Data about the cluster structure which includes a list of all hosts, roles, and services along with the configurations that are set through Cloudera Manager. Where passwords are set in Cloudera Manager, the passwords are not returned. Cloudera Manager license and version number. Current health information for hosts, service, and roles. Includes results of health tests run by Cloudera Manager. Heartbeat information from each host, service, and role. These include status and some information about memory, disk, and processor usage. The results of running Host Inspector. One day's worth of Cloudera Manager metrics. Note: If you are using Cloudera Express, host metrics are not included. Cloudera Manager Administration Guide 79

80 Other Cloudera Manager Settings A download of the debug pages for Cloudera Manager roles. For each host in the cluster, the result of running a number of system-level commands on that host. Logs from each role on the cluster, as well as the Cloudera Manager server and agent logs. Which parcels are activated for which clusters. Whether there's an active trial, and if so, metadata about the trial. Metadata about the Cloudera Manager server, such as its JMX metrics, stack traces, and the database/host it's running with. HDFS/Hive replication schedules (including command history) for the deployment. Impala query logs. Configuring the Frequency of Diagnostic Data Collection By default, Cloudera Manager collects diagnostic data on a weekly basis. You can change the frequency to daily, weekly, monthly, or never. If you are a Cloudera Enterprise customer and you set the schedule to never you can still collect and send data to Cloudera on demand. If you are a Cloudera Express customer and you set the schedule to never, data is not collected or sent to Cloudera. 1. Select Administration > Settings. 2. Under the Support category, click Scheduled Diagnostic Data Collection Frequency and select the frequency. 3. To set the day and time of day that the collection will be performed, click Scheduled Diagnostic Data Collection Time and specify the date and time in the pop-up control. 4. Click Save Changes to commit the changes. You can see the current setting of the data collection frequency by viewing Support > Scheduled Diagnostics: in the main navigation bar. Specifying the Diagnostic Data Directory You can configure the directory where collected data is stored. 1. Select Administration > Settings. 2. Under the Support category, set the Diagnostic Data Bundle Directory to a directory on the host running Cloudera Manager Server. The directory must exist and be enabled for writing by the user cloudera-scm. If this field is left blank, the data is stored in /tmp. 3. Click Save Changes to commit the changes. Collecting and Sending Diagnostic Data to Cloudera Important: This feature is available only with a Cloudera Enterprise license. For other licenses, the following applies: Cloudera Express - The feature is not available. Cloudera Enterprise Data Hub Edition Trial - The feature is available until you end the trial or the trial license expires. To obtain a license for Cloudera Enterprise, please fill in this form or call After you install a Cloudera Enterprise license, the feature will be available. Disabling the Automatic Sending of Diagnostic Data from a Manually Triggered Collection If you do not want data automatically sent to Cloudera after manually triggering data collection, you can disable this feature. The data you collect will be saved and can be downloaded for sending to Cloudera Support at a later time. 1. Select Administration > Settings. 2. Under the Support category, uncheck the box for Send Diagnostic Data to Cloudera Automatically. 80 Cloudera Manager Administration Guide

81 Other Cloudera Manager Settings 3. Click Save Changes to commit the changes. Note: The Send Diagnostic Data form that displays when you collect data in one of the following procedures indicates whether the data will be sent automatically. Manually Triggering Collection and Transfer of Diagnostic Data to Cloudera 1. Optionally change the System Identifier property: a. Select Administration > Settings. b. Under the Other category, set the System Identifier property and click Save Changes. 2. Under the Support category, choose Send Diagnostic Data. The Send Diagnostic Data form displays. 3. Fill in or change the information here as appropriate: Cloudera Manager populates the End Time based on the setting of the Time Range selector. You should change this to be a few minutes after you observed the problem or condition that you are trying to capture. The time range is based on the timezone of the host where Cloudera Manager Server is running. If you have a support ticket open with Cloudera Support, include the support ticket number in the field provided. 4. Depending on whether you have disabled automatic sending of data, do one of the following: Click Collect and Send Diagnostic Data. A Running Commands window shows you the progress of the data collection steps. When these steps are complete, the collected data is sent to Cloudera. Click Collect Diagnostic Data. A Command Details window shows you the progress of the data collection steps. 1. In the Command Details window, click Download Result Data to download and save a zip file of the information. 2. Send the data to Cloudera Support by doing one of the following: 1. Download the phone_home script. 2. Copy the script and the downloaded data file to a host that has Internet access. 3. Run the following command on that host: python phone_home.py --file <downloaded data file> Contact Cloudera Support and arrange to send the data file. Configuring Network Settings To configure a proxy server thorough which data is downloaded to and uploaded from the Cloudera Manager Server, do the following: 1. Select Administration > Settings. 2. Click the Network category. 3. Configure proxy properties. 4. Click Save Changes to commit the changes. Cloudera Manager Administration Guide 81

82 Other Cloudera Manager Settings Importing Cloudera Manager Settings Backing up your Current Deployment To back up your current deployment, follow the instructions in Back up Databases on page 44. The import feature should not be relied on for backup and recovery at this time. Building a Cloudera Manager Deployment You can use the Cloudera Manager API to programmatically build a Cloudera Manager Deployment a definition of all the entities in your Cloudera Manager-managed deployment clusters, service, roles, hosts, users and so on. See the Cloudera Manager API documentation on how to manage deployments using the /cm/deployment resource. Uploading a Cloudera Manager 4.x Configuration Script Importing a deployment should be done using the Cloudera Manager API. See the Cloudera Manager API documentation for details. 82 Cloudera Manager Administration Guide