Cloudera Manager Introduction

Transcription

1 Cloudera Manager Introduction

2 Important Notice (c) Cloudera, Inc. All rights reserved. Cloudera, the Cloudera logo, Cloudera Impala, and any other product or service names or slogans contained in this document are trademarks of Cloudera and its suppliers or licensors, and may not be copied, imitated or used, in whole or in part, without the prior written permission of Cloudera or the applicable trademark holder. Hadoop and the Hadoop elephant logo are trademarks of the Apache Software Foundation. All other trademarks, registered trademarks, product names and company names or logos mentioned in this document are the property of their respective owners. Reference to any products, services, processes or other information, by trade name, trademark, manufacturer, supplier or otherwise does not constitute or imply endorsement, sponsorship or recommendation thereof by us. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Cloudera. Cloudera may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Cloudera, the furnishing of this document does not give you any license to these patents, trademarks copyrights, or other intellectual property. The information in this document is subject to change without notice. Cloudera shall not be liable for any damages resulting from technical errors or omissions which may be present in this document, or from use of this document. Cloudera, Inc. 220 Portage Avenue Palo Alto, CA info@cloudera.com US: Intl: Release Information Version: Date: July 11, 2013

3 Table of Contents Cloudera Manager Frequently Asked Questions...5 General Questions...5 Using Cloudera Manager...7 Cloudera Manager Guide Overview...9 Getting Help and Support...11 Community Support...11 Cloudera Enterprise Support...11 Get Announcements about New CDH and Cloudera Manager Releases...11 Report Issues...11

4

5 Cloudera Manager Frequently Asked Questions Cloudera Manager Frequently Asked Questions General Questions What are the new features of Cloudera Manager 4.6? For a list of new features in Cloudera Manager 4.6, see New Features in Cloudera Manager 4. What operating systems are supported? See Requirements for Cloudera Manager for more detailed information on which platforms are supported. What database platforms are supported? See Requirements for Cloudera Manager for more detailed information on which database systems are supported. What are the differences between the Standard and the Enterprise Versions of Cloudera Manager? Cloudera Standard includes a free version of the commercially-available Cloudera Manager product. Both the Standard and Enterprise versions automate the installation, configuration, and monitoring of CDH3 or CDH4 on an entire cluster. Cloudera Enterprise provides additional functionality. The full-featured version of Cloudera Manager is available as part of the Cloudera Enterprise subscription offering, and requires a license. See the matrix at What are the Differences Between the Cloudera Standard and Enterprise Versions? for a comparison of the two versions. You can also view a discussion of the differences at If you are not an existing Cloudera customer, contact Cloudera Sales for information about obtaining the full-featured version of Cloudera Manager. If you are already a Cloudera customer and you need to upgrade the from Cloudera Standard to Cloudera Enterprise, contact Cloudera Support to obtain a license. What version of CDH is required for Cloudera Manager 4.6? See Requirements for Cloudera Manager for detailed information. Can I upgrade CDH using Cloudera Manager? Yes, as of Cloudera Manager 4.5 you can upgrade to CDH4.1.2 or later from within the Cloudera Manager Admin Console using parcels. Furthermore, once you have installed or upgraded CDH using parcels, you can perform rolling upgrades on your CDH services. If you have HDFS High Availability configured and enabled, you can perform a rolling upgrade on your cluster without taking the entire cluster down. Important: Cloudera Manager version 3.x and CDH3 have reached End of Maintenance (EOM) as of June 20, Cloudera will not support or provide patches for any of the Cloudera Manager version 3.x and CDH3 releases. Even though Cloudera Manager 4.x will continue to support CDH3, it is strongly recommended that you upgrade to CDH4. See Upgrading existing installations of CDH3 to CDH4 for more details. Cloudera Manager Introduction 5

6 Cloudera Manager Frequently Asked Questions For instructions on upgrading CDH4 to a newer version, see Upgrading to the Latest Version of CDH4 in a Cloudera Manager Deployment. For instructions on upgrading from CDH3 to CDH4, see Upgrading CDH3 to CDH4 in a Cloudera Managed Deployment. What version of CDH does Cloudera Manager 4.6 install? Cloudera Manager 4.6 allows you to install any version of CDH3 or CDH4. What upgrade paths are available for Cloudera Manager, and what's involved? For instructions about upgrading, see Upgrading Cloudera Manager. How do I install Cloudera Manager 4.6 in a walled-off environment (no Internet access)? You can set up a local repository and use it in the installer. For instructions, see Appendix A - Understanding Custom Installation Solutions. Do slave nodes need access to the Cloudera public repositories for an install with Cloudera Manager? As of Cloudera manager 4.5, you can perform an installation or upgrade using the new parcel format. When using parcels, only the Cloudera manager server requires access to the Cloudera public repositories. Distribution of the parcels to slave nodes is done between the Cloudera manager server and the slave nodes. See Using Parcels for more information. If you want to install using the traditional packages, nodes only require access to the installation files. For both parcels and packages, it is also possible to create local repositories that serve these files to the nodes that are being upgraded. If you have established local repositories, no access to the Cloudera public repository is required. For more information, see Creating and Using your own Repository. Can I use the Service Monitoring features of Cloudera Manager without using Service and Configuration Management? No. To understand the desired state of the system, we need the global configuration that the service management feature of Cloudera Manager gathers and provides. The Cloudera Manager agent doubles as both the agent for supervision and for monitoring. Can I run the Cloudera Management services and the Hadoop services on the host where the Cloudera Manager Server is running? Yes. This is especially common in deployments that have a small number of hosts. Which sudo commands does Cloudera Manager run? The sudo commands are: yum (Red Hat/CentOS/Oracle) zypper (SLES) apt-get (Debian/Ubuntu) apt-key (Debian/Ubuntu) sed service /sbin/chkconfig (Red Hat/CentOS/Oracle) /usr/sbin/update-rc.d (Debian/Ubuntu) id rm mv chown 6 Cloudera Manager Introduction

7 Cloudera Manager Frequently Asked Questions install Does Cloudera Manager Support an API? Yes. A comprehensive set of APIs for the various features is supported in this version of Cloudera Manager. For more information about the Cloudera Manager API, see the Cloudera Manager API documentation. You can download this Cloudera Manager API example that shows how to integrate with Nagios or other systems. Using Cloudera Manager How do I set the rack topology script? Cloudera Manager includes internal rack awareness scripts, but you must specify the racks where the hosts in your cluster are located. If your cluster contains more than 10 hosts, Cloudera recommends that you specify the rack for each host. HDFS and MapReduce will automatically use the racks you specify. For instructions, see Specifying the Racks for Hosts. How do I configure my daemons to use a custom GC parameter? Every Java-based role in Cloudera Manager has a configuration setting called "Additional Java configuration options for <role>" where you can enter command line options. Commonly, garbage collection flags or extra debugging flags would be passed here. To find the appropriate configuration setting, select the service you want to modify in the Cloudera Manager UI, then use the keyword search to search for the Additional Java configuration options. You can add configuration options for all instances of a given role by making this configuration change at the Service level. For example, to modify the setting for all DataNodes, select the HDFS service, then modify the "Additional Java configuration options for DataNode" setting). To modify a configuration option for a given instance of a role, select the service, then select the particular role instance (e.g. a specific DataNode) you want to modify. The configuration settings you modify will apply to the selected role instance only. For detailed instructions see the Modifying Service Configurations section of Managing Clusters with Cloudera Manager. Cloudera Manager Introduction 7

8

9 Cloudera Manager Guide Overview Cloudera Manager Guide Overview The following documents are included in the Cloudera Manager documentation set: Guide Description Cloudera Manager Release Notes These release notes provide information on the new features and known issues and limitations for Cloudera Manager. Cloudera Manager Introduction This guide provides a comprehensive introduction to Cloudera Manager. Cloudera Manager Installation Guide This guide explains how to install Cloudera Manager and CDH. Cloudera Manager 4 supports managing CDH3 and CDH4. Managing Clusters with Cloudera Manager This guide is for Apache Hadoop developers and system administrators who want to use the Cloudera Manager Admin Console to change CDH configuration settings, add new services, add new clusters, add hosts to existing clusters, upgrade your CDH or component versions, configure High Availability, and other similar functions. Cloudera Manager Monitoring and Diagnostics Guide This guide is for Apache Hadoop developers and system administrators who want to use Cloudera Manager to monitor and diagnose their CDH installation. You can use the Cloudera Manager Admin Console to monitor cluster usage, activities, and health, and view status and logs to troubleshoot problems. Cloudera Manager Administration Guide This guide is for system administrators who need to manage a Cloudera Manager server installation. This guide covers adding and managing Cloudera Manager users, adding or upgrading licenses, configuring TLS security, configuring the Alert Publisher, and other similar features. Cloudera Backup and Disaster Recovery This guide describes Cloudera Backup and Disaster Recovery (BDR), a separately-licensed Cloudera product that provides an integrated, easy-to-use management solution for enabling data protection in the Hadoop platform. Configuring Hadoop Security with Cloudera Manager This guide describes how to use Cloudera Manager to automate many of the manual tasks of implementing Kerberos security on your CDH cluster. Cloudera Manager Introduction 9

10

11 Getting Help and Support Getting Help and Support Cloudera Manager documentation is available under the Support menu in the Cloudera Manager Admin console. In addition there are several other channels to obtain help with and information about Cloudera Manager: Community Support Any user of Cloudera Manager (Standard or Enterprise), can get help through community user groups: Cloudera Manager Users group CDH Users group As a Cloudera Manager Standard user, you can also access the Cloudera Manager support mailing list from within the Cloudera Manager Admin console by selecting Support > Mailing List. Cloudera Enterprise Support Cloudera can help you install, configure, optimize, tune, and run Hadoop for large scale data processing and analysis. Cloudera supports Hadoop whether you run our distribution on servers in your own data center, or on hosted infrastructure services such as Amazon EC2, Rackspace, SoftLayer, or VMware vcloud. If you are a Cloudera Enterprise customer, you can access the Cloudera Support Portal from within the Cloudera Manager Admin console, by selecting Support > Cloudera Support Portal. From there you can: Create a Cloudera Support Ticket Visit the Cloudera Knowledge Base Learn how to register for an account to create a support ticket Get Announcements about New CDH and Cloudera Manager Releases Cloudera provides the following public mailing lists that send announcements about new CDH and Cloudera Manager product releases and updates: To receive CDH release announcements, subscribe to the CDH-announce list. To receive Cloudera Manager release announcements, subscribe to the CM-announce list. Report Issues Cloudera tracks software and documentation bugs and enhancement requests for CDH on issues.cloudera.org. Your input is appreciated, but before filing a request, please search the Cloudera issue tracker for existing issues and send a message to the CDH user's list, cdh-user@cloudera.org, or the CDH developer's list, cdh-dev@cloudera.org. Cloudera Manager Introduction 11

12