EMC Elastic Cloud Storage (ECS)

Size: px
Start display at page:

Download "EMC Elastic Cloud Storage (ECS)"

Transcription

1 EMC Elastic Cloud Storage (ECS) Version 2.0 ECS Documentation

2 Copyright EMC Corporation. All rights reserved. Published in USA. Published June, 2015 EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC², EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. For the most up-to-date regulatory document for your product line, go to EMC Online Support ( EMC Corporation Hopkinton, Massachusetts In North America EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

3 CONTENTS Part 1 What's New 11 Chapter 1 New features in ECS New features Part 2 Concepts 17 Chapter 2 What is ECS? 19 Overview ECS platform Portal services Storage services...20 Provisioning service Fabric service Infrastructure service Chapter 3 Understanding data protection 23 Overview Storage service Object creates...24 Object reads...27 Erasure coding Recovery on disk and node failures Site fail over and recovery Part 3 Install and Upgrade 31 Chapter 4 Plan an installation 33 Overview Site preparation ECS installation readiness checklist Connecting ECS appliances in a single site Multi-site requirements Part 4 Configure and Manage 37 Chapter 5 Getting started with the ECS Portal 39 Introduction Log in to the ECS Portal Change Password...41 Access to portal areas Ordering and searching tables in the portal EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation 3

4 CONTENTS Chapter 6 Configure storage pools, VDCs, and replication groups 47 Configure storage pools, VDCs, and replication groups...48 Storage pools...48 Create storage pools Virtual data centers (VDCs)...49 Create a VDC for a single site...50 Add a VDC to a federation Fail over a site/delete a VDC Replication groups Create replication groups Configure ConnectEMC Chapter 7 Add users and assign roles 55 Introduction Understanding users and roles in ECS Users in ECS...56 User roles...57 Domain and local users...58 User scope: global or namespace...59 Working with the users at the ECS Portal Add a new object user...62 Add a domain user as an object user...63 Create a local management user or assign a domain user to a management role Create a namespace administrator...64 Working with the authentication providers at the ECS Portal...65 Add an authentication provider Authentication provider settings Considerations when adding authentication providers...70 Understanding the mapping of users into a namespace Map domain users into a namespace Chapter 8 Configure a namepace for a tenant 75 Introduction Understanding tenants...76 Understanding namespace settings Working with namespaces at the ECS portal Create and configure a namespace...78 Chapter 9 Obtain and upload a license file to the ECS Portal 81 Licensing...82 Obtain the EMC ECS license file...82 Chapter 10 Manage a tenant 83 Introduction Quotas Retention periods and policies Lock buckets and users...86 Metering Audit buckets EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

5 CONTENTS Chapter 11 Fail over an ECS site 91 Fail over a site/delete a VDC...92 Part 5 Monitor 93 Chapter 12 Monitor resources: hardware, network traffic, disk bandwidth, node and process health 95 About resource monitoring Using monitoring pages Monitor hardware...98 Monitor network traffic Monitor disk bandwidth Monitor node and process health Chapter 13 Monitor storage: metering and capacity 105 Introduction Using monitoring pages Monitor capacity Storage capacity data Monitor metering data Metering data Chapter 14 Monitor service logs 113 Service logs ECS service log locations Chapter 15 Monitor services: chunks, erasure coding, geo-replication, and recovery status 115 About service monitoring Using monitoring pages Monitor chunks Monitor erasure coding Monitor recovery status Monitor geo-replication: rate and chunks Monitor geo-replication: Recovery Point Objective (RPO) Monitor geo-replication: failover Monitor geo replication: bootstrap processing Chapter 16 Monitor events: audit portal, API, and CLI events and system alerts 127 About event monitoring Filter events by date and sort the results by column Part 6 Enable Data Access 129 Chapter 17 Address ECS object storage and use the Base URL 131 Introduction Bucket addressing EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation 5

6 CONTENTS DNS Configuration Base URL Add a Base URL Example of a Base URL Chapter 18 Create and configure buckets 137 Introduction Bucket concepts and attributes Bucket ACLs Create a bucket using the ECS Portal Edit a bucket Set the bucket ACL permissions for a user Set the bucket ACL permissions for a pre-defined group Create bucket using the object APIs Create a bucket using the S3 API (with s3curl) Bucket and key naming conventions S3 bucket and object naming in ECS OpenStack Swift container and object naming in ECS Atmos bucket and object naming in ECS CAS pool and object naming in ECS Chapter 19 Obtain secret key to access object storage 151 Introduction Create a key for an object user Generate a secret key from the ECS Portal Create an S3 secret key using the ECS Management REST API Create an S3 secret key: self-service Working with self-service keys Chapter 20 Configure support for CAS SDK applications with the ECS Portal 157 Setting up CAS support in ECS CAS retention in ECS Set up namespace retention policies Set up a bucket for CAS Set up a CAS object user Set up bucket ACLs for a user ECS REST APIs that support CAS users Part 7 Configure ECS HDFS 167 Chapter 21 HDFS Overview 169 What is ECS HDFS? Configuring Hadoop to use ECS HDFS ECS HDFS URI for file system access Hadoop authentication modes File system interaction Unsupported Hadoop applications Chapter 22 Configure HDFS in a non-secure Hadoop cluster 177 Configure ECS HDFS EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

7 CONTENTS Plan the ECS HDFS and Hadoop integration Obtain the ECS HDFS installation and support package Create bucket for HDFS Deploy the ECS HDFS JAR Use a Cloudera Parcel to install Hadoop on a cluster Edit Hadoop core-site.xml file Edit HBASE hbase-site.xml Chapter 23 Configure HDFS in a secure Hadoop cluster 187 Configure secure Hadoop cluster to use ECS HDFS Plan the ECS HDFS and secure Hadoop cluster integration Obtain the ECS HDFS installation and support package Create bucket for secure HDFS Deploy the ECS HDFS JAR in a secure cluster Use a Cloudera Parcel to install Hadoop on a secure cluster Configure ECS nodes with the ECS Service Principal Edit core-site.xml Guidance on Kerberos configuration ECS administration tool reference Chapter 24 Troubleshooting an ECS HDFS configuration 203 Troubleshooting Verify AD/LDAP is correctly configured with secure Hadoop cluster.204 Permission denied for AD user Restart services after hbase configuration Pig test fails: unable to obtain Kerberos principal Enable Kerberos client-side logging Debug Kerberos on the KDC Eliminate clock skew Chapter 25 ECS HDFS core site property reference 207 Hadoop core-site.xml properties for ECS HDFS Sample core-site.xml for simple authentication mode Part 8 Access ECS Data 213 Chapter 26 ECS Amazon S3 Object Service API support 215 Amazon S3 API S3 API Supported and Unsupported Features API Extensions Authenticating with the S3 service Using SDKs to access the S3 service Chapter 27 ECS OpenStack Swift Object Service API support 229 OpenStack Swift API OpenStack Swift supported operations API Extensions OpenStack Version 1 authentication OpenStack Version 2 authentication Authorization on Container EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation 7

8 CONTENTS Chapter 28 ECS EMC Atmos Object Service API support 241 EMC Atmos API Supported Features Supported EMC Atmos REST API Calls Unsupported EMC Atmos REST API Calls Subtenant Support in EMC Atmos REST API Calls API Extensions Appending data to an object Part 9 Use the REST API 247 Chapter 29 Use the ECS Management REST API 249 Introduction Authenticate with the ECS Management REST API Authenticate with AUTH-TOKEN Authenticate with cookies Logout Whoami ECS Management REST API summary Part 10 Hardware Maintenance 257 Chapter 30 ECS hardware 259 ECS Appliance hardware components ECS Appliance configurations and upgrade paths U-Series single-phase AC power cabling U-Series three-phase AC power cabling Switches Private switch: Arista Public switch: Arista 7150S Public switch: Arista 7124SX Network cabling Nodes Server front views Server rear view Rack and node host names Disk drives Integrated disk drives Disks and enclosures Disk drives in DAEs LCC Fan control module ICM DAE power supply U-Series SAS cabling Chapter 31 Replace an ECS storage disk in a U-Series Appliance 293 Drive replacement planning Output for the cs_hal list disks command Replace drives EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

9 CONTENTS Chapter 32 Replace an ECS storage disk in third-party hardware 305 Drive replacement planning Output for the cs_hal list disks command Replace drives Chapter 33 Add a 60-disk upgrade to a U-Series ECS Appliance disk upgrade planning cs_hal commands Perform a 60-disk upgrade Chapter 34 ECS third-party rack requirements 325 ECS Third-Party Rack Requirements EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation 9

10 CONTENTS 10 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

11 PART 1 What's New Chapter 1, " New features in ECS 2.0 " What's New 11

12 What's New 12 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

13 CHAPTER 1 New features in ECS 2.0 New features New features in ECS

14 New features in ECS 2.0 New features Describes the new features and additions for ECS 2.0. Controller In previous releases, ECS used the ViPR controller to provide a number of services, such as authorization, authentication, licensing, and logging. These capabilities are now part of ECS and ViPR is no longer required. In addition, the need for a virtual machine to install ECS has also been removed as the first node in an ECS appliance now acts as the installer for all nodes. Portal ECS was previously configured from the ViPR UI. With ECS 2.0, a new portal is provided that provides configuration, management, and monitoring capabilities to ECS and its tenants. Monitoring and Diagnostics ECS now provides comprehensive monitoring and diagnostics capabilities through the ECS Portal and the ECS API. The following features are supported: Metering Metering of namespace and bucket utilization Namespace and bucket time-based metering by capacity, bandwidth, and object count. Capacity Utilization Monitoring of storage pool utilization. Physical environment monitoring Monitoring of node and disk health Monitoring of node health - CPU, memory, and NIC usage Monitoring of process health. Storage efficiency Monitoring of erasure coding job performance Monitoring of chunk level usage Monitoring of recovery operations Monitoring of back end traffic for storage operations (erasure coding, georeduction using XOR, recovery). Geo-replication monitoring Replication monitoring Failover monitoring Bootstrap monitoring. 14 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation Geo Enhancements The following enhancements have been made that improve the use of the ECS georeplication mechanism: Geo Enhancement: Temporary Site Failover on page 15 Geo Enhancement: Improved Recover Point Objective (RPO) on page 15 Geo Enhancement: Multi-Site Access Performance Improvements through Geocaching on page 15

15 New features in ECS 2.0 Geo Enhancement: Temporary Site Failover ECS 2.0 has automatic and sophisticated ways of handling temporary site failures and failbacks. With this new functionality, applications have access to data even when connectivity between federated VDCs is unavailable. There are some restrictions on some of the operations, such as creating new buckets while the sites are down. ECS will automatically resync the sites and reconcile the data when all the sites are operational and connected to each other again. Geo Enhancement: Improved Recover Point Objective (RPO) Prior to ECS 2.0, ECS object chunks were of a fixed size (128MB) and were sealed and replicated to a remote site only when the chunk was filled. Although this strategy was more efficient, it had the drawback that if an entire site or a rack went down, there could be many chunks with less than 128MB of data that had not been replicated. To overcome this, ECS 2.0 now starts the replication process as soon as a chunk starts receiving data. Geo Enhancement: Multi-Site Access Performance Improvements through Geo-caching Prior to ECS 2.0, where multiple VDCs were federated, any attempt to read an object had to go back to the VDC that owned the object. This meant that every time a user in another site accessed the data, it incurred a cost in terms of WAN bandwidth as well as slower performance caused by WAN latency. ECS 2.0 solves this problem by caching objects at the secondary sites so that users can access the data locally without a WAN transfer. Object Metering Object metering enables key statistics to be retrieved for a tenant and for the buckets associated with a tenant. Metering data includes capacity, object count, objects created, objects deleted, and bandwidth (inbound as well as outbound) and can be retrieved for a specific point in time, or for a time range. Retention Policies Retention periods can be applied to object containers (object containers are referred to as buckets in Amazon S3, containers in OpenStack Swift, and subtenants in EMC Atmos) and objects to prevent critical data being modified within a specified period. In addition, ECS provides the ability to define policies that can be applied to objects so that the retention period is applied in accordance with the defined policy rather than a specific period. Quota Management Quota limits can be set on a bucket or namespace. This enables a tenant to define the maximum amount of storage that can be used for a namespace and enables tenants to create buckets that are quota limited. Hard and soft quotas can be applied independently or together, to record and event prior to imposing a hard limit. Bucket and User Locking Locking can be applied at the bucket and user level, using the ECS Management REST API, as a means of preventing access. Bucket Auditing Bucket auditing is provided to enable create, update, and delete buckets operations and bucket access permission changes to be logged. Enable Rack Level Awareness Racks can be treated as separate fault domains enabling object store chunks to be distributed across these domains so that failure of a rack does not cause failure of the whole site. New features 15

16 New features in ECS EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

17 PART 2 Concepts Chapter 2, "What is ECS?" Chapter 3, "Understanding data protection" Concepts 17

18 Concepts 18 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

19 CHAPTER 2 What is ECS? Overview ECS platform What is ECS? 19

20 What is ECS? Overview ECS is a complete software-defined cloud storage platform that supports the storage, manipulation, and analysis of unstructured data on a massive scale on commodity hardware. ECS is specifically designed to support mobile, cloud, big data, and social networking applications. It can be deployed as a turnkey storage appliance or as a software product that can be installed on a set of qualified commodity servers and disks. The ECS scale-out, geo-distributed architecture is a cloud platform that provides: Lower cost than public clouds Unmatched combination of storage efficiency and data access Anywhere read/write access with strong consistency that simplifies application development No single point of failure to increase availability and performance Universal accessibility that eliminates storage silos and inefficient ETL/data movement processes ECS platform The ECS platform includes the following software layers and services: Figure 1 ECS platform services Portal services Portal services include interfaces for provisioning, managing, and monitoring storage resources. The interfaces are: GUI: A built-in browser-based graphical user interface called the portal Portal. REST: A RESTful API that you can use to develop your own portal Portal. CLI: A command-line interface that enables you to perform the same tasks as the browser-based interface. Storage services 20 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation Storage services are provided by the unstructured storage engine (USE) which ensures data availability and protection against data corruption, hardware failures, and data

21 What is ECS? Provisioning service center disasters. It enables global namespace management across geographically dispersed data centers and geo-replication. The USE enables the following storage services: Object service: Provides the ability to store, access, and manipulate unstructured data. The object service is compatible with existing Amazon S3, OpenStack Swift APIs, EMC CAS and EMC Atmos APIs. HDFS: Enables you to use your portal storage infrastructure as a Big Data repository that you can run Hadoop analytic applications against (in-place). The provisioning service manages the provisioning of storage resources and user access. Specifically, it handles: User management: Keeps track of which users have rights to administer the system, provision storage resources, and access objects via REST requests. portal supports both local and domain users. Authorization and authentication for all provisioning requests: Queries the authentication domain to determine if users are authorized to perform management, provisioning, and access operations. Resource management: Enables authorized users to create storage pools, Virtual Data Centers, and replication groups. Multi-tenancy: Manages the namespace that represents a tenant, and their associated buckets and objects. Fabric service The fabric service is a distributed cluster manager that is responsible for: Cluster health: Aggregates node-specific hardware faults and reports on the overall health of the cluster. Node health: Monitors the physical state of the nodes, and detects and reports faults. Infrastructure service Disk health: Monitors the health of the disks and file systems. It provides raw, fast, lock-free read/write operations to the storage engine, exposes information about the individual disk drives and their status so the storage engine can place data across the disk drives according to the storage engine's built-in data protection algorithms. Software management: Provides command line tools for installing and running services, and for installing and upgrading the fabric software on nodes in the cluster. This layer provides the Linux OS running on the commodity nodes and it implements network interfaces and other hardware-related tools. Provisioning service 21

22 What is ECS? 22 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

23 CHAPTER 3 Understanding data protection Overview Storage service...24 Object creates Object reads...27 Erasure coding Recovery on disk and node failures Site fail over and recovery Understanding data protection 23

24 Understanding data protection Overview Storage service Object creates Learn about how ECS protects unstructured data against node, disk, and site failures through replication and erasure coding. ECS ensures durability, reliability, and availability of objects by creating and distributing three copies of objects and their metadata across the set of nodes in the local site. After the three copies are successfully written, ECS erasure-codes the object copies to reduce storage overhead. It handles failure and recovery operations automatically with no additional backup software or devices required. The storage service layer handles data availability and protection against data corruption, hardware failures, and data center disasters. The unstructured storage engine (USE) is part of the storage services layer. It is a distributed shared service that runs on each node, and it manages transactions and persists data to nodes. The USE enables global namespace management across geographically dispersed data centers through geo-replication. The USE writes all object-related data (such as, user data, metadata, object location data) to logical containers of contiguous disk space known as chunks. Chunks are open and accepting writes, or closed and not accepting writes. After chunks are closed, the storage engine erasure-codes them. The storage engine writes to chunks in an append-only pattern so that existing data is never overwritten or modified. This strategy improves performance because locking and cache validation is not required for I/O operations. All nodes can process write requests for the same object simultaneously while writing to different chunks. The storage engine tracks object location through an index that records object name, chunk id, and offset. Chunk location is separately tracked through an index that records chunk id and a set of disk locations. The chunk location index contains three disk location pointers before erasure coding, and multiple location pointers after erasure coding. The storage engine performs all of the storage operations (such as, erasure coding and object recovery) on chunks. Object creates: one VDC The following figure shows how the storage engine writes object data when there is a single VDC. In this example, there is a single appliance deployed at the site, but the same principles apply when more appliances are deployed. The eight nodes are in a single storage pool within a single replication group. 24 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

25 Understanding data protection Figure 2 Single site: object creates 1. An application creates an object in a bucket. 2. The storage engine writes the object to one chunk. The disk locations corresponding to this chunk are on three different disks/nodes, so the writes go to three different disks/nodes in parallel. The storage engine can write the object to any of the nodes that belong to the bucket's replication group. The VDC where the object is created is the object's owner. 3. The storage engine records the disk locations of the chunk in the chunk location index, and the chunk id and offset in the object location index. 4. The storage engine writes the object location index to one chunk and the disk locations corresponding to the chunk to three different disks/nodes, so the writes go to three different disks/nodes in parallel. The index locations are chosen independently from the object chunk locations. 5. After all of the disk locations are written successfully, the storage engine acknowledges the write to the application. When object chunks are full, the storage engine erasure-codes them. It does not erasure code the object location index chunks. Object creates: federated VDCs (2 sites) In a federated deployment of two VDCs, the storage engine writes object chunks to the local VDC and also to the remote VDC. Object creates 25

26 Understanding data protection Figure 3 Two site: object creates 1. An application creates an object in a bucket. 2. The storage engine writes the object to one chunk at the site where it is ingested. The disk locations corresponding to this chunk are on three different disks/nodes, so the writes go to three different disks/nodes in parallel. The storage engine can write the object to any of the nodes that belong to the bucket's replication group. The storage engine records the disk locations of the chunk in the chunk location index, and the chunk id and offset in the object location index. The site where the object is originally ingested is the object's owner. 3. After all of the disk locations are written successfully, the storage engine acknowledges the write to the application. 4. The storage engine replicates the chunk to three nodes in the federated site. It records the chunk locations in the object location index (not shown in this diagram) also on three different nodes at the federated site. When the chunks are full, the storage engine erasure-codes the object chunks. It does not erasure code the object location index chunks. When two VDCs are in a replication group, both VDCs have a readable copy of the object. 26 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

27 Understanding data protection Three sites: object creates Figure 4 Object creates: federated VDCs (3 or more sites) 1. An application creates an object in a bucket. 2. The storage engine writes the object to one chunk at the site where it is ingested. The disk locations corresponding to this chunk are on three different disks/nodes, so the writes go to three different disks/nodes in parallel. It can write the object to any of the nodes that belong to the bucket's replication group. The storage engine records the disk locations of the chunk in the chunk location index, and the chunk id and offset in the object location index (not shown in this diagram). The VDC where the write received is the object's owner, and it contains a readable copy of the object. 3. After all of the disk locations are written successfully, the storage engine acknowledges the write to the application. 4. The storage engines replicates the chunks to nodes in another VDC within the replication group. To improve storage efficiency, the storage engine XOR's the chunks with other chunks from other objects also stored on the node. When the chunks are full, the storage engine erasure-codes the XOR'd chunks. When possible, it writes XOR chunks directly in erasure-coded format without going through the replication phase. It does not erasure code the object location index chunks. Object updates When an application fully updates an object, the storage engine writes a new object (following the principles described earlier). The storage engine then updates the object location index to point to the new location. Because the old location is no longer referenced by an index, the original object is available for garbage collection. Object reads Object reads: single VDC In a single site deployment, when a client submits a read request, the storage engine uses the object location index to find which chunks are storing the object, it retrieves the chunks or erasure-coded fragments, reconstructs and returns the object to the client. Object reads 27

28 Understanding data protection Object reads: federated VDCs (2 sites) In a two-site federation, the storage engine reads the object chunk or erasure coded fragments from the nodes on the VDC where the application is connected. In a two-site federation, object chunks exist on both sites. Object reads: federated VDCs (3 sites or more sites) If the requesting application is connected to the VDC that owns the object, the storage engine reads the object chunk or erasure coded fragments from the nodes on the VDC. If the requesting application is not connected to the owning VDC, the storage engine retrieves the object chunk or erasure coded fragments from the VDC that owns the object, copies them to the VDC the application is connected to, and returns the object to the application. The storage engine keeps a copy of the object in its cache in case another request is made for the object. If another request is made, the storage engine compares the timestamp of the object in the cache with the timestamp of the object in the owning VDC. If they are the same, it returns the object to the application; if the timestamps are different, it retrieves and caches the object again. Erasure coding ECS uses erasure coding to provides better storage efficiency without compromising data protection. The storage engine implements the Reed Solomon 12/4 erasure coding scheme in which an object is broken into 12 data fragments and 4 coding fragments. The resulting 16 fragments are dispersed across the nodes in the local site. The storage engine can reconstruct an object from any of the 12 fragments. Table 1 Storage overhead when deploying multiple sites Number of sites Storage Overhead ECS requires a minimum of four nodes running the object service in a single site. It tolerates failures based on the number of nodes. Table 2 Node failure tolerance at a single site Total nodes Node failure tolerance for writes EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

29 Understanding data protection When an object is erasure coded, the original chunk data is present as a single copy that consists of 16 fragments dispersed throughout the cluster. When an object has been erasure-coded, ECS can read objects directly without any decoding or reconstruction. ECS only uses the code fragments for object reconstruction when there is hardware failure. Recovery on disk and node failures ECS continuously monitors the health of the nodes, their disks, and objects stored in the cluster. Since ECS disperses data protection responsibilities across the cluster, it is able to automatically re-protect at-risk objects when nodes or disks fail. Disk health ECS reports disk health as Good, Suspect, or Bad. Good The disk s partitions can be read from and written to. Suspect The disk has not yet met the threshold to be considered bad. Bad A certain threshold of declining hardware performance has been met. Once met, no data can be read or written. ECS writes only to disks in good health; it does not write to disks in suspect or bad health. ECS reads from good disks and from suspect disks. When two of an object s chunks are located on suspect disks, ECS writes the chunks to other nodes. Node health ECS reports node health as Good, Suspect, Degraded, or Bad. Good: The node is available and responding to I/O requests in a timely manner. Internal health monitoring indicates that it is in good health. Suspect: The node is available, but is reporting internal health information such as a fan failure (if there are multiple fans), a single power supply failure (if there are redundant power supplies). Or, the node is unreachable by the other nodes, but it is visible to BMC probes and is in an unknown state. Degraded: The node is available but is reporting bad or suspect disks. Bad: The node is reachable, but internal health monitoring indicates poor health. For example, the node's fans are offline, the CPU temperature is too high, there are too many memory errors, and so on. Bad health can also be reported when the node is offline, and BMC probes indicate the health is not acceptable. ECS writes only to nodes in good health; it does not write to nodes in suspect, degraded, or bad health. ECS reads from good and suspect nodes. When two of an object s chunks are located on suspect nodes, ECS writes two new chunks of it to other nodes. When a node is reported as suspect or bad, all of the disks it manages are also considered suspect or bad. Data recovery When there is a failure of a node or drive in the site, the storage engine: 1. Identifies the chunks or EC fragments affected by the failure. 2. Writes copies of the affected chunks or EC fragments to good nodes and disks that do not currently have copies. Recovery on disk and node failures 29

30 Understanding data protection Site fail over and recovery ECS provides protection against a site failure due to a disaster or other problem that causes a site to go offline or to be disconnected from the other sites in a geo-federated deployment. Temporary site failure Temporary site failures occur when network connectivity is interrupted between federated VDCs or when a VDC goes down temporarily. When a VDC goes down, the Replication Group page displays the status Temporarily unavailable for the VDC that is unreachable. When buckets are configured with the Access During Outage property set to On, applications can read objects while connected to any site. When applications are connected to a site that is not the bucket's owner, the application must explicitly access the bucket to write to it or to view the contents. If an application modifies an object or bucket while connected to a VDC that is not the owner, the storage engine transfers ownership to the site where the change is initiated. The following operations cannot be completed at any site in the geo-federation until the temporary failure is resolved regardless of the Access During Outage setting: Bucket: create or rename bucket, modify bucket properties, list buckets for a namespace when the namespace owner site is not reachable Namespace: create User: create After the sites are reconnected, the storage engine starts a resync operation in the background. Use the portal's Monitor > Recovery Status to monitor the progress of the resync operation. Permanent site fail over If a disaster occurs at a site and the VDC cannot be brought back online, you must delete it. 30 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

31 PART 3 Install and Upgrade Chapter 4, "Plan an installation" Install and Upgrade 31

32 Install and Upgrade 32 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

33 CHAPTER 4 Plan an installation Overview Site preparation ECS installation readiness checklist Connecting ECS appliances in a single site Multi-site requirements Plan an installation 33

34 Plan an installation Overview Learn about the physical environment, data center, and multi-site requirements as well as the private management network topologies. Site preparation Review the Site Preparation Guide to learn about the environmental requirements associated with the 40U-D cabinet used by the ECS Appliance. ECS installation readiness checklist Review this list for the infrastructure components required for a successful installation. An ECS appliance deployment consists of the following components: One or more racks. The rack must be up linked to the customer network for both data traffic and remote management. The rack and all nodes must be powered on. The nodes must have valid IP addresses assigned by DHCP or configured statically. Infrastructure requirements: The data center environment must include the following servers that are reachable from all nodes. DHCP server (if you are assigning IP addresses via DHCP) DNS server (or forwarder) NTP server SMTP server SSH must be enabled on all nodes. The following ports are opened and used by the installer: Docker registry: 5000 Lifecycle agent: 9240 Object: ,9040,9091, ,8088,9898,1095,1096,1098,9100,9101,9111,3 218 ZooKeeper: 9277,9278,9279 See the Security Guide for the list of ports that must be open. Connecting ECS appliances in a single site The ECS appliance management networks are connected together through the Nile Area Network. The NAN is created by connecting either port 51 or 52 to another turtle switch of another ECS appliance. Through these connections nodes from any segment can communicate to any other node in the NAN. 34 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

35 Plan an installation The simplest topology to connect the ECS appliances together does not require extra switch hardware. All the turtle switches can be connected together a linear or daisy chain fashion. Figure 5 Linear or daisy-chain topology In this topology, if there is a loss of connectivity a split-brain can occur. Figure 6 Linear or daisy-chain split-brain For a more reliable network, the ends of the daisy chain topology can be connected together to create a ring network. The ring topology is more stable because it would require two cable link breaks in the topology for a split-brain to occur. The primary drawback to the ring topology is that the RMM ports cannot be connected to the customer network unless an external customer or aggregation switch is added to ring. Figure 7 Ring topology The daisy-chain or ring topologies are not recommended for large installations. When there are four or more ECS appliances, an aggregation switch is recommended. The addition of an aggregation switch in a star topology can provide better fail over by reducing split-brain issues. Connecting ECS appliances in a single site 35

36 Plan an installation Figure 8 Star topology Multi-site requirements When planning for a multi-site ECS installation, ensure these requirements are met: A minimum of two VDCs is required. Each VDC in the multi-site configuration requires IP connectivity to the other VDCs. Network latency: Ensure a maximum latency of 1000 ms between sites. Free storage: If your disaster plan includes running for a period of time with one site permanently failed (instead of promptly recovering the site), each site needs enough free storage across all sites to accommodate data rebalancing. Across all sites, the amount of free space left should be, in total: free space across n sites =1.33*x/(n-1)/(n-2) where x is the total amount of user data across all n sites. This amount of free space is not required if you add a new site soon after the fail over, and do not continue to operate with (N-1) sites indefinitely. 36 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

37 PART 4 Configure and Manage Chapter 5, " Getting started with the ECS Portal " Chapter 6, "Configure storage pools, VDCs, and replication groups" Chapter 7, " Add users and assign roles " Chapter 8, " Configure a namepace for a tenant " Chapter 9, "Obtain and upload a license file to the ECS Portal" Chapter 10, " Manage a tenant " Chapter 11, "Fail over an ECS site" Configure and Manage 37

38 Configure and Manage 38 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

39 CHAPTER 5 Getting started with the ECS Portal Introduction Log in to the ECS Portal Change Password...41 Access to portal areas Ordering and searching tables in the portal Getting started with the ECS Portal 39

40 Getting started with the ECS Portal Introduction The ECS Portal enables ECS to be configured, managed, and monitored. In addition, the portal provides facilities for tenants to manage and monitor their namespace and to create and configure buckets within their namespace. The portal is primarily intended for access by ECS management users, system administrators, and namespace/tenant administrators. Object storage users access ECS using the supported object protocols using clients that support those protocols. You can read more about ECS users and roles in Add users and assign roles on page 56. The portal makes use of the public ECS Management REST API and it is possible to develop custom clients that use this API to perform operations. Log in to the ECS Portal You can log in to the ECS Portal from your browser by specifying the IP address of any node in the cluster. Before you begin You can log in to the ECS Portal if your are assigned to the System Admin or Namespace Admin ECS roles. A "root" user account, assigned to the System Admin role, is provided for initial access. You are automatically logged out after 15 minutes of inactivity. Procedure 1. Type the public IP address of the first node in the system, or the address of the load balancer that has been configured as the front-end to ECS, in the following form: For example: The login screen is displayed. 40 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

41 Getting started with the ECS Portal 2. Enter the username and password. If you log in using the initial root credentials (root/changeme), it is recommended that you change the password. Your session ends when you close the browser, or log out. Logging out always closes the session. If you are unable to log in, contact your administrator. Change Password When you are logged in at the ECS Portal, you can change your password. Before you begin System Admin and Namespace Admin users have access the Password page. If you are logged in as the root user, this account is not the same account as the root account used to provide command line access to a node. So, changing the password here will not change the password for the node root account. Procedure 1. At the ECS Portal, select Settings > Password 2. Enter a new password in the Password field and enter it again as confirmation. 3. Click Save. Access to portal areas The portal provides a left navigation menu and a page area. The System Admin can access all pages, a Namespace Admin can access a limited number of pages and perform only tenant-specific operations. The followings sections detail the access provided for different management users. Change Password 41

42 Getting started with the ECS Portal System Admin on page 42 Namespace Admin on page 43 System Admin The following table lists the menu items that can be accessed and provides a link to documentation articles that provide more information on their use. Area Menu Operations Supported Monitor Metering View object metering for namespace or bucket. Capacity Utilization Monitor storage pool, node, and disk capacity. For more information, see: Monitor storage: metering and capacity on page 106. Events Traffic Metrics Hardware Health Node and Process Health Disk Bandwidth View audit events. For more information, see: Monitor events: audit portal, API, and CLI events and system alerts on page 128. Provides monitoring details, as follows: Monitor read and write bandwidth and latency. Monitor storage node and disk status for each storage pool. Monitor health of nodes and processes by memory and CPU utilization. Monitor disk bandwidth usage. For more information, see: Monitor resources: hardware, network traffic, disk bandwidth, node and process health on page 96 Chunk Summary Erasure Coding Recovery Status Geo Replication Provides monitoring details, as follows: Monitor chunks and chunks status. Monitor erasure coding status. Monitor recovery status. Monitor geo-replication activity. For more information, see: Monitor services: chunks, erasure coding, geo-replication, and recovery status on page 116. Manage Storage Pools Virtual Data Center Enables the following operations: Add a storage pool and specify the nodes that it comprises. Replication Group Add a VDC and define its connection details. Configure a replication group by adding storage pools belonging to a VDC. For more information, see: Configure storage pools, VDCs, and replication groups on page 48. Authentication Namespace Add an authentication provider that can authenticate domain users. See Add users and assign roles on page 56. Enables the following operations: Create a new namespace. Set quota for namespace. 42 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

43 Getting started with the ECS Portal Area Menu Operations Supported Map object users into a namespace. For more information, see: Configure a namepace for a tenant on page 76 Users Enables the following operations: Create object users for the namespace. Edit object users. Create secret keys. For more information, see: Add users and assign roles on page 56 Bucket Enables the following operations: Create bucket. Assign ACLs to bucket owner and object users. For more information, see: Create and configure buckets on page 138 Settings Object Base URL Set the Base URL to determine which part of object address is the bucket and namespace. For more information, see: Address ECS object storage and use the Base URL on page 132 Change Password Connect EMC Licensing Change own password. For more information, see: Change Password on page 41 Configure sending of events to EMC. View license status and upload a license. For more information, see: Obtain and upload a license file to the ECS Portal on page 82 Namespace Admin The following table lists the menu items that can be accessed and provides a link to documentation articles that provide more information on their use. Area Menu Operations Supported Monitor Metering View object metering for namespace or bucket. Manage Namespace Edit the namespace. For more information, see Monitor storage: metering and capacity on page 106 For more information, see Configure a namepace for a tenant on page 76 Users Enables the following operations: Create object users for the namespace. Edit object users Access to portal areas 43

44 Getting started with the ECS Portal Area Menu Operations Supported Create secret keys for object users For more information, see Add users and assign roles on page 56 Bucket Enables the following operations: Create bucket. Assign ACLs to bucket owner and object users. For more information, see Create and configure buckets on page 138 Settings Change Password Change own password. For more information, see Change Password on page 41 Ordering and searching tables in the portal When a data set presented at the portal is large, and especially when it runs onto multiple pages, it is useful to be able to reorder a table and to search for information in the table. An example of a page containing a table is shown below. 44 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation Reordering Table Columns You can reorder the rows in a table based on the ordering of a selected column. A table column can be ordered by clicking on the table header. Columns that contain textual data are sorted alphabetically. For example, if you select the Namespace field in the users table, that column will be ordered alphabetically and will

45 Getting started with the ECS Portal drive the ordering of rows. When you reenter the page, the default ordering will be applied. Similarly, refreshing the page will return the page to the default ordering. Using Search The Search facility enables table rows to be filtered based on matching text strings. As you type text in the Search box, rows that contain strings that match the search string are displayed. The order in which the rows that match the search criteria are displayed depends on the ordering applied by the table column ordering (see Reordering Table Columns on page 44). Refreshing a Page A refresh control is provided on pages that contain table data. Using refresh will return the table to its default ordering. Ordering and searching tables in the portal 45

46 Getting started with the ECS Portal 46 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

47 CHAPTER 6 Configure storage pools, VDCs, and replication groups Configure storage pools, VDCs, and replication groups...48 Storage pools...48 Virtual data centers (VDCs)...49 Replication groups Configure ConnectEMC Configure storage pools, VDCs, and replication groups 47

48 Configure storage pools, VDCs, and replication groups Configure storage pools, VDCs, and replication groups Learn how to use the portal to create, modify, and delete storage pools, VDCs, and replication groups for single or federated deployments, and how configure ConnectEMC for the object service. Users must be assigned to the System Admin role to perform these procedures. Storage pools Storage pools let you organize storage resources based on business requirements. For example, if you require physical separation of data, you can partition the storage into multiple different storage pools. Use the Storage Pool Management page available from Manage > Storage Pools to view the details of existing storage pools, to create new storage pools, to modify existing storage pools, and to delete storage pools. Figure 9 Storage Pool Management page Table 3 Storage pool properties Field Name Description The name of the storage pool. # Nodes The number of nodes assigned to the storage pool. Status The current state of the storage pool and of the nodes. Storage pool states are: Ready: At least four nodes are installed and all nodes are in the ready to use state. Not Ready: A node in the storage pool is not in the ready to use. Partially Ready: There are less than four nodes and all nodes are in the ready to use state. Host Name Node IP address Rack ID Actions The fully qualified host name assigned to the node. The public IP address assigned to the node. The name assigned to the rack that contains the nodes. Actions are: 48 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

49 Configure storage pools, VDCs, and replication groups Table 3 Storage pool properties (continued) Field Description Edit: Use to change storage pool's name and the set of nodes included in the storage pool. Delete: Use to delete storage pools. All nodes in storage pool must be removed before you can delete a storage pool. You cannot delete the system storage pool which is the first storage pool created. If the system storage pool has empty nodes, the empty nodes can be deleted if the number of nodes is greater than four. Create storage pools Use this procedure to assign nodes to storage pools. Storage pools must contain a minimum of four nodes. The first storage pool that is created is known as the system storage pool because it stores system metadata. The system storage pool cannot be deleted. Procedure 1. From the portal, select Manage > Storage Pools. 2. Click New Storage Pool. 3. Type the storage pool name. For example: StoragePool1. 4. Select the nodes to add to the storage pool from the Available Nodes list. a. To select nodes one-by-one, click the + icon next for each node. b. To select all available nodes, click the + icon at the top of the Available Nodes list. c. To narrow the list of available nodes, type the node's public IP address or host name in the search field. 5. When you have completed the node selection, click Save. 6. Wait 10 minutes after the storage pool is in the Ready state before you perform other configuration tasks. This allows the storage pool time to initialize. If you do not wait long enough, you receive the following error message: Error 7000 (http: 500): An error occurred in the API Service. An error occurred in the API service.cause: error insertvdcinfo. Virtual Data Center creation failure may occur when Data Services has not completed initialization. If you receive this error, wait a few more minutes before attempting any further configuration. Virtual data centers (VDCs) VDCs are logical constructs. They are the top-level resource that represents the collection of ECS infrastructure to manage as a unit. Use the Virtual Data Center Management page available from Manage > Virtual Data Centers to view VDC details, to create a new VDC, to modify existing VDCs, to delete VDCs and to federate multiple VDCs for a multi-site deployment. The following example shows the Manage Virtual Data Center page for a multi-site, federated deployment. It is configured with three sites. The VDCs are named vdc1, vdc2, and vdc3. Create storage pools 49

50 Configure storage pools, VDCs, and replication groups Figure 10 VDCManagement page Table 4 VDC properties Field Name Endpoints Status Description The VDC's name. The public IP addresses of the nodes in the storage pools that comprise the VDC. States are: Online Permanently Failed: The VDC was deleted. Actions Actions are: Edit: Use to modify the VDC's name, the access key, and the public IP addresses of the nodes in the VDC's storage pools. Delete: Use to delete a VDC. The delete operation triggers permanent fail over of the VDC so you cannot add it back using the same name. You cannot delete a VDC that is part of a replication group until you first remove it from the replication group. You cannot delete a VDC when you are logged in to the VDC you are trying to delete. Create a VDC for a single site Use this procedure when you are creating a VDC for a single site deployment, or when you are creating the first VDC in a multi-site federation. Before you begin One or more storage pools are available and in the Ready state. Procedure 1. From the ECS Portal, select Manage > Virtual Data Center. 2. Click New Virtual Data Center. 3. Type a name. For example: VDC1. The name cannot have includes spaces or underscores. 4. Click Get VDC Access Key. The VDC Access Key is used as a symmetric key for encrypting replication traffic between VDCs in a multi-site federation. 5. In the Endpoints text box, enter the public IP addresses of each node in the VDC's storage pools. Supply them as a comma-separated list. 6. Click Save. 50 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

51 Configure storage pools, VDCs, and replication groups Add a VDC to a federation Use this procedure when you are adding a VDC (for example, VDC2) to an existing VDC (for example, VDC1) to create a federation. Before you begin Obtain the ECS Portal credentials for the root user, or for a user with system administrator credentials, to log in to both sites. Ensure you have the list of public IP addresses for the nodes from the site you are adding (VDC2). Ensure the site you are adding (VDC2) has a valid license uploaded and has at least one storage pool in the Ready state. Procedure 1. Log in to the ECS Portal at the site you are adding (VDC2). The default credentials are root/changeme. 2. Select Manage > Virtual Data Center. 3. Click Get VDC Access Key. 4. Select the access key, and copy it using Ctrl-C to save it in the buffer. 5. Log out of the ECS Portal at the site you are adding (VDC2). 6. Log in to the ECS Portal of the first VDC (VDC1). 7. Select Manage > Virtual Data Center. 8. Click New Virtual Data Center. 9. Enter the VDC's name. For example: VDC2. 10.Click into the Key field and paste (CTRL-V) the Key you copied from the site you are adding ( VDC2) from Steps 3 and 4 above. 11.Enter the public IP addresses of the site you are adding. Enter them as a commaseparated list. 12.Click Save. Fail over a site/delete a VDC Use this procedure to delete a VDC. Deleting a VDC initiates site fail over when the VDC you are deleting is part of a multi-site federation. If a disaster occurs, an entire VDC can become unrecoverable. ECS initially treats the unrecoverable VDC as a temporary site failure. If the failure is permanent, you must remove the VDC from the federation to initiate fail over processing which reconstructs and reprotects the objects stored on the failed VDC. The recovery tasks run as a background process. Review the recovery process by using the Monitor > Geo Replication > Failover Procesing. Procedure 1. Log in to one of the operational VDCs in the federation. 2. Go to Manage > Replication Group. 3. Click Edit for the replication group that contains the VDC to delete. 4. Click Delete in the row that contains the VDC and storage pool to remove. Add a VDC to a federation 51

52 Configure storage pools, VDCs, and replication groups Replication groups 5. Click Save. 6. Go to Manage > VDC. The status for the permanently removed VDC changes to Permanently failed. 7. Select Delete from the drop down in the row of the VDC to remove. 8. Click Save. Replication groups are logical constructs that define where storage pool content is protected. Replication groups can be local or global. Local replication groups protect objects within the same VDC against disk or node failures. Global replication groups protect objects against disk, node, and site failures. Use the Manage Replication Groups page to view replication group details, to create new replication groups, and to modify existing replication groups. You cannot delete replication groups in this release. Figure 11 Manage Replication Groups page Table 5 Replication Group properties Field Name VDC Storage Pool Status Description The replication group name. The number of VDCs in the replication group and the names of the VDCs where the storage pools are located. The names of the storage pools and their associated VDCs. States are: Online Temp Unavailable: Replication traffic to this VDC has failed. If all replication traffic to the same VDC is in the Temp Unavailable state, further investigation about the cause of the failure is recommended. Actions Edit: Use to modify the replication group name and the set of VDCs and storage pools in the replication group. Create replication groups Use this procedure to create replication groups. To create global replication groups, choose storage pools from multiple VDCs. Procedure 1. From the ECS Portal, select Manage > Replication Group. 2. Click New Replication Group. 52 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

53 Configure storage pools, VDCs, and replication groups 3. Type a name. For example: ReplicationGroup1. 4. Click Add VDC. 5. Select a Virtual Data Center and Storage Pool from the dropdown. Repeat this step to add the VDCs and Storage pools required for object protection. 6. Click Save. Configure ConnectEMC Use this procedure to configure ConnectEMC for the object service. Procedure 1. From the portal, click Settings > ConnectEMC. 2. Select the transport type and whether the communications should encrypted. 3. If the transport type is FTPS, use the table below for assistance in completing the configuration. Option Encryption Hostname Server Addresses Description Choose whether to encrypt the service data. Specify corpusfep3.emc.com. Specify the SMTP server is used in the environment. The address to send ConnectEMC service notifications to. 4. If the transport type is SMTP, use the table below for assistance in completing the configuration. Option Encryption Authentication SMTP Server SMTP Port From Address Description Choose whether to encrypt the service data. Choose an authenticate type for authenticating to the SMTP server or NONE. If authentication is required, you must also supply the UserName and Password for authenticating to the SMTP server. Enter the SMTP server hostname or IP address. Enter the port for the SMTP server. The address to display as the From address for ConnectEMC service notifications. Addresses Additional address to send ConnectEMC service notifications to. 5. Click Save. For alerts to be sent out automatically, the system must be licensed, have ConnectEMC, storage pools, a VDC, and replication groups configured. Configure ConnectEMC 53

54 Configure storage pools, VDCs, and replication groups 54 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

55 CHAPTER 7 Add users and assign roles Introduction Understanding users and roles in ECS Working with the users at the ECS Portal Working with the authentication providers at the ECS Portal...65 Understanding the mapping of users into a namespace Add users and assign roles 55

56 Add users and assign roles Introduction This article describes the types of users supported by ECS and the roles to which they can be assigned. It introduces the main concepts around ECS users and roles: Understanding users and roles in ECS on page 56 Working with the users at the ECS Portal on page 60 and then describes how to add management users or object users: Add a new object user on page 62 Add a domain user as an object user on page 63 Create a local management user or assign a domain user to a management role on page 63 Create a namespace administrator on page 64 In addition, it shows you how you can set up an authentication provider and perform the mapping of domain users into a namespace: Add an authentication provider on page 65 Map domain users into a namespace on page 72 Understanding users and roles in ECS ECS defines different user types and roles to determine access to ECS management facilities and to the object store. The main concepts relating to users and roles are described in the following topics: Users in ECS on page 56 User roles on page 57 Domain and local users on page 58 User scope: global or namespace on page 59 Users in ECS ECS requires two types of user: management users, who can perform administration of ECS, and object users, who access the object store to read and write objects and buckets using the supported data access protocols (S3, EMC Atmos, OpenStack Swift, and CAS). Management users can access the ECS Portal. Object users cannot access the ECS Portal but can access the object store using clients that support the ECS data access protocols. Management users and object users are stored in different tables and their credentials are different. Management users require a local username and password, or a link to a domain user account. Object users require a username and a secret key. Hence you can create a management user and an object user with the same name, but they are effectively different users as their credentials are different. In addition, management and object user names can be unique across the ECS system or can be unique within a namespace. This is referred to as user scope and is described in: User scope: global or namespace on page 59. Details of the supported user types are provided in the following sections: 56 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

57 Add users and assign roles Management Users on page 57 Object users on page 57 Root user on page 57 Management Users Management users can perform the configuration and administration of the ECS system and of tenants configured in ECS. Management users can be local users whose credentials are stored by ECS and are authenticated by ECS against the locally held credentials, or they can be domain users defined in Active Directory (AD) or Lightweight Directory Access Protocol (LDAP) and authenticated against users held in those systems. You can find out more about domain and local users in Domain and local users on page 58. Management users are not replicated across geo-federated VDCs. Object users Root user User roles Object users are end-users of the ECS object store and access it through object clients using the ECS supported object protocols (S3, EMC Atmos, Openstack Swift, and CAS). Object users are defined by a username and a secret key that can be used to access the object store. Usernames can be local names or can be domain-style user names that include a "@" in their name. A management user can create an object user account and can assign a secret key to the object user account when the account is created or at any time thereafter. When created by a management user, the object users secret key is distributed by or other means. For domain users, a secret key can be obtained by the object user using the ECS selfservice capability, using a client that talks to the ECS REST API (object users do not have access to the ECS portal). You can read more about domain users in: Domain and local users on page 58, and you can refer to Obtain secret key to access object storage on page 152 for information on creating a secret key. However, object users use ECS-held credentials, and no link to an AD account is stored, so ECS does not check whether the name exists in AD or is mapped into the namespace. Object users are global resources, so an object user created at a VDC can be given privileges to read and write buckets, and objects, within the namespace to which they are assigned, from any VDC. The root user is available at system initialization and is pre-assigned to the System Admin role. The root user should only be used for initial access to the system. On initial access, the root user password should be changed at the Settings > Password page and one or more new System Admin accounts should be created. From an audit perspective, it is important to know which user carried out changes to the system, so root should not be used, and each System Admin user should have their own account. ECS defines roles to determine the operations that a user account can perform at the ECS Portal or when accessing ECS using the ECS Management REST API. Management users User roles 57

58 Add users and assign roles can be assigned to administration roles in ECS and can be either local users or domain users. If a management user who is a domain users is to be assigned to the Namespace Admin role, the user must be mapped into the namespace, The following management roles are defined: System Admin on page 58 Namespace Admin on page 58 System Admin Namespace Admin The System Admin role can configure ECS and specify the storage used for the object store, how the store is replicated, how tenant access to the object store is configured, and which users have permissions on an assigned namespace. The System Admin can also configure namespaces and perform namespace administration, or can assign a user who belongs to the namespace as the Namespace Admin. The System Admin has access to the ECS Portal and system administration operations can also be performed from programmatic clients using the ECS Management REST API. Because management users are not replicated across site, a System Admin must be created at each VDC that requires one. If a management user who is a domain users is to be assigned to the Namespace Admin role, the user must be mapped into the namespace, The Namespace Admin is a management user who can access the ECS Portal to configure namespace settings, such as quotas and retention periods, and can map domain users into the namespace and assign local users as object users for the namespace. Namespace Admin operations can also be performed using the ECS Management REST API. A Namespace Admin can only be the administrator of a single namespace. Because authentication providers and namespaces are replicated across sites (they are ECS global resources), a domain user who is a Namespace Admin can log in at any site and perform namespace administration from that site. Local management accounts are not replicated across sites, so a local user who is a Namespace Admin can only log in at the VDC at which the management user account was created. If you want the same username to exist at another VDC, the user must be created at the other VDC. As they are different accounts, changes to a same-named account at one VDC, such as a password change, will not be propagated to the account with the same name at the other VDC. Domain and local users ECS provides support for local and domain users. 58 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation Local users are user accounts whose credentials are stored by ECS. Both management users and object users can be defined locally to ECS. In the case of object users, the credentials are global resources and are available at all ECS VDCs. Local users make it very simple to start using ECS, however, the use of AD/LDAP enables an existing user database to be leveraged and allows a large number of users to be given access to the object store without having to create accounts for them. Domain users are users defined in an Active Directory AD/LDAP database and ECS must talk to the AD or LDAP server to authenticate user login request. ECS uses a construct

59 Add users and assign roles User scope: global or namespace called an authentication provider to supply the credentials it needs to talk to the AD/ LDAP server and to specify the domains and groups that should be made available to ECS. Domain users are defined in the form and ECS will attempt to authenticate user names in that form using the authentication providers that have been configured. User names will be authenticated against the local user database. Domain users assigned to management roles can be authenticated against their AD/LDAP credentials to allow them to access ECS and perform ECS administration operations. Administration operations can be performed from the ECS Portal or using the ECS Management API. Domain users can also be assigned as object users. To save the administrative overhead of manually creating large numbers of object user accounts in ECS, a self-service capability is provided that allows ECS to authenticate domain users and automatically add them as object users and assign a secret key to them. To make use of this, a domain user must be mapped into a namespace and ECS provides a mechanism for mapping domain users into a namespace based on their domain and group membership and on attributes associated with their account. For an object user, ECS supports names that include "@", so it is possible to set up object users with names that are the same as their domain name. This enables users to access the ECS object store using a name that is familiar. However, authentication uses a locally held secret key, not AD credentials. Where an object user is also a domain user and is mapped into a namespace, it is possible to use the ECS REST API to obtain a new secret key to access the ECS object store using the S3 API. This is referred to a self-service capability. The scope of object users depends on the user scope that has been set. The setting affects all users, in all namespaces across all federated VDCs The user scope can be either GLOBAL or NAMESPACE. In global scope, object user names are unique across all VDCs in the ECS system. In namespace scope, object user names are unique within a namespace, so the same object user account names can exist in different namespaces. The default setting is GLOBAL. If you intend to use ECS in a multi-tenant configuration and you want to ensure that tenants are not prevented from using names that are in use in another namespace, you should change this default configuration to NAMESPACE. Note The user scope setting must be made before the first object user is created. Setting the User Scope The user scope can be set using the PUT /config/object/properties API and passing the user scope in the payload. An example of a payload that sets the user_scope to NAMESPACE is shown below. PUT /config/object/properties/ <property_update> <properties> <properties> <entry> <key>user_scope</key> User scope: global or namespace 59

60 Add users and assign roles <value>namespace</value> </entry> </properties> </property_update> Working with the users at the ECS Portal The ECS Portal provides a Manage > Users page to enable local users to be created and assigned as object users for a namespace. It also enables system administrators to create local management users and assign them to administration roles and to assign domain users to administration roles. The Manage > Users page provides two sub-pages: Object Users View on page 60 Management Users View on page 61 The Management Page is only accessible if you are a System Admin (or root user) for ECS. Object Users View The Object Users view provides an Object Users table that lists the local users that have been created, the namespace to which the users have been assigned, and the actions that can be performed on the user. If you are a System Admin you will see the object users for all namespaces. If you are a Namespace Admin, you will only see the users belonging to your namespace. The Object Users view is shown below. The Object Users table provides access to the following information and operations. 60 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

61 Add users and assign roles Attribute Name Namespace Actions Description The name of the user. The namespace to which the user is assigned. Provides a selection menu for the actions that are available. The actions that are available are: Edit and Delete. The Object Users pane additionally provides access to the the following controls: Control Description New Object User The New Object User button enables an object user to be added. Management Users View The Management Users view provides a Management Users table that lists the management users that have been created and the actions that can be performed on the user. This page is only visible to users with the System Admin role. The Management Users view is shown below. The Management Users table provides access to the following information and operations. Column Description Name Actions The name of the user. Provides a selection menu for the actions that are available. The actions that are available are: Edit and Delete. In addition, the Management Users view provides the following controls: Control Description New Management User The New Management User button enables the addition of a management user that may be assigned as the System Admin role for ECS. Working with the users at the ECS Portal 61

62 Add users and assign roles Add a new object user You can create new local users and configure them to use the supported object access protocols. Once created, you can edit a user configuration by adding or removing access to an object protocol, or by creating a new secret key for the user. Before you begin If you are an ECS System Admin, you can assign users for any namespace. If you are a Namespace Admin, you can assign users for the namespaces for which you are the administrator. If you want your domain users to be enabled as object users you should refer to Add a domain user as an object user on page 63. When assigning a password for a Swift user, the user will be added to the Swift Admin group. Note Do not use the ECS Portal to perform this operation if you want users to be assigned to different Swift groups. You can refer to Working with the users at the ECS Portal on page 60 for information about the Manage > Users page. Procedure 1. At the ECS Portal, select Manage > Users. The Object Users Page is shown by default and displays the Object Users table which lists the local users that have been created and the namespace to which they are assigned. 2. Select New Object User. The New Object User page is displayed. 3. Enter a name for the user. This is a name for a local user that will be created. You can use domain-style names that include "@". For example, "some.name@emc.com". However, this is a convenience to enable you to keep names unique and consistent with AD names, authentication is performed using a secret key assigned to the username, not through AD or LDAP. 4. Select the namespace to which the local user will be assigned. Once you have selected the namespace, you can Save the user and return later to edit the user and assign a secret key to access an object protocol. Alternatively, you can select Add Passwords and specify passwords or secret keys to access the ECS object protocols. 5. To set up secret keys for the user, select Add Passwords. 6. For each of the object protocols that you want to use to access the ECS object store, enter or generate a key for use in accessing the S3, Swift, or CAS, and save the key. Select Add Password to save the key. 7. Specify a password for each of the object interfaces that you want the user to be able to access. For S3 and CAS you can generate the password. 62 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

63 Add users and assign roles 8. The secret keys and passwords are saved automatically and you can click the Close button to return to the Users page. Add a domain user as an object user You can configure domain users so that they can access ECS and generate secret keys for themselves and, by doing so, add themselves as object users. Before you begin Procedure 1. Ensure an authentication provider that connects to the appropriate AD/LDAP system has been configured. Adding an authentication provider must be performed by a System Admin and is described in Add an authentication provider on page Map domain users into the namespace as described in Map domain users into a namespace on page 72. This can be performed by the Namespace Admin. 3. Allow users to create secret keys using the instructions in Obtain secret key to access object storage on page 152. Create a local management user or assign a domain user to a management role You can add a local management user and assign a local management user or a domain user to a management role from the ECS Portal. Management users are required to perform system-level administration (VDC administration) and namespace administration. Where a user is no longer needed to perform administration operations, you can remove the role assignment. Before you begin You must be a System Admin to create a local management user or assign a management role. The ECS root user has the System Admin role by default and can perform the initial assignment of a user to the System Admin role. If you want to assign a domain user to a management role, you must first ensure that an authentication provider has been added. See Add an authentication provider on page 65. If you want to assign a Namespace Admin, you must create a management user using the operation defined here and perform the role assignment at the portal Namespace page (see Configure a namepace for a tenant on page 76). The user will not be able to log in until they have been assigned to the Namespace Admin role (or the System Admin role). You can refer to Working with the users at the ECS Portal on page 60 for information about the Manage > Users page. Procedure 1. At the ECS Portal, select Manage > Users. The Object Users Page is displayed by default and you need to change to the Management Users page. 2. Select Management Users. Add a domain user as an object user 63

64 Add users and assign roles The Management Users page is displayed which shows any users that have currently been assigned and provide a New Management User button. 3. Select New Management User. The New Management User pages is displayed which enables you to create a local user and assign the new user to the management role, or assign a domain user to the management role. 4. Select Local User or AD/LDAP User. For a local user you will need to define a password; for a domain user, the user and password credentials that ECS will use to authenticate a user are held in AD/LDAP, so you don't need to define a password. 5. Enter the name the user. If you have selected AD/LDAP, the user must exist and have been made available by adding an authentication provider to ECS. If you select local user, a new local management user will be created 6. If you want to assign the user to the System Admin role, select Yes at the System Administrator selector. If you are creating a management user who will be assigned to the Namespace Admin role for a namespace, you should leave this as No. If you select Yes, but at a later date you want to remove System Administrator privileges from the user, you can edit the user settings and change this to No. 7. If you have not selected System Admin because you intend to assign the user to the Namespace Admin role, you must check the "In order to log in, non-system Admin users..." box. The "In order to log in, non-system Admin users..." box acknowledges that the user will not be able to log in until assigned to the Namespace Admin role. 8. Select Save. Create a namespace administrator You can assign a local or domain user as a Namespace Admin. Before you begin You must be a System Admin to create a management user and assign a user to the Namespace Admin role. You can refer to Working with the users at the ECS Portal on page 60 for information about the Manage > Users page. Procedure 1. If you want to assign a local management user to the Namespace Admin role, you need to create a management user as described in Create a local management user or assign a domain user to a management role on page 63. If you want to assign a domain user to the Namespace Admin role, you do not need to explicitly assign the user to a management role. 2. At the Manage > Namespace page. a. Select the Edit action for the namespace. 64 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation b. Add the user to the Namespace Admin field. If there is more than one Namespace Admin, their usernames should be a comma separated list.

65 Add users and assign roles A user can only be assigned as the Namespace Admin for a single namespace. c. Save the namespace. You can read more about configuring a namespace in: Configure a namepace for a tenant on page 76. Working with the authentication providers at the ECS Portal The ECS Portal provides a Manage > Authentication page to enable authentication providers to be added. The Authentication Provider Page is only accessible if you are a System Admin (or root user) for ECS. The Authentication Provider Page provides an Authentication Provider table that lists the authentication provider that have been created. An example is shown below. The table provides access to the following information and operations. Attribute Description Name Type Domains Enabled Actions The name that has been given to the authentication provider. Indicates whether the authentication provider is an Active Directory (AD) or Lightweight Directory Access Protocol (LDAP) server. Domains that the authentication provider provides access to. Indicated whether the authentication provider is currently Enabled or Disabled. Provides a selection menu for the actions that are available. The actions that are available are: Edit and Delete. The Authentication Provider Page additionally provides access to the following controls: Control Description New Authentication Provider The New Authentication Provider button enables an authentication provider to be added. Add an authentication provider User authentication for domain users is performed using one or more authentication providers added to ECS. An authentication provider is a construct that enables ECS to Working with the authentication providers at the ECS Portal 65

66 Add users and assign roles connect an AD/LDAP server and identifies the domains and groups that the AD/LDAP should make available to ECS. Before you begin To add an authentication provider you must be assigned to the System Admin role in ECS. The root user has the System Admin role. You need access to the authentication provider information listed in Authentication provider settings on page 66. Note especially the requirements for the Manager DN user. Procedure 1. At the ECS Portal, select Manage > Authentication > New Authentication Providers. 2. Enter values for the attributes. Refer to Authentication provider settings on page Save. Authentication provider settings 4. To verify the configuration, add a user from the authentication provider at Manage > Users > Management Users, then try to log in as the new user. You need to provide certain information when adding or editing an authentication provider. Table 6 Authentication provider settings Field name Name Description Type Domains Description and requirements The name of the authentication provider. You can have multiple providers for different domains. Free text description of the authentication provider. Active Directory or LDAP. Active Directory and LDAP allow administrators to organize objects of a network (such as users, computers, and devices) into a hierarchical collection of containers. Domains are a collection of administratively defined objects that share a common directory database, security policies, and trust relationships with other domains. In this way, each domain is an administrative boundary for objects. A single domain can span multiple physical locations or sites and can contain millions of objects. A typical entry in this field of the authentication provider would look like this: mycompany.com If an alternate UPN suffix is configured in the Active Directory, the Domains list should also contain the alternate UPN configured for the domain. For example, if myco is added as an alternate UPN suffix for mycompany.com, then the Domains list should contain both myco and mycompany.com. Server URLs ldap or ldaps (secure LDAP) with the domain controller IP address. Default port for ldap is 389 and ldaps is EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

67 Add users and assign roles Table 6 Authentication provider settings (continued) Field name Description and requirements Usage: one or more of ldap://<domain controller IP >:<port> (if not default port) or ldaps://<domain controller IP >:<port> (if not default port) If the authentication provider supports a multidomain forest, use the global catalog server IP and always specify the port number. Default is 3268 for ldap, 3269 for ldaps. Usage: ldap(s)://<global catalog server IP>:<port> Manager DN Indicates the Active Directory Bind user account that ECS uses to connect to Active Directory or LDAP server. This account is used to search Active Directory when a ECS administrator specifies a user for role assignment, for example. Requirement: This user must have Read all inetorgperson information in Active Directory. The InetOrgPerson object class is used in several non-microsoft, Lightweight Directory Access Protocol (LDAP) and X.500 directory services to represent people in an organization. To set this privilege in Active Directory, open Active Directory Users and Computers, right click on the domain, and select Delegate Control.... Click Next, then select the user that you are using for managerdn and click Next. The required permission is on the next screen "Read all inetorgperson information." Example: CN=Manager,CN=Users,DC=mydomaincontroller,DC=com In this example, the Active Directory Bind user is Manager, in the Users tree of the mydomaincontroller.com domain. Usually managerdn is a user who has fewer privileges than Administrator, but has sufficient privileges to query Active Directory for users attributes and group information. WARNING You must update this value in ECS if the managerdn credentials change in Active Directory. Manager Password The password of the managerdn user. WARNING You must update this value in ECS if the managerdn credentials change in Active Directory. Providers Select Disabled if you want to add the server to ECS but not immediately use it for authentication. (Regardless of whether Authentication provider settings 67

68 Add users and assign roles Table 6 Authentication provider settings (continued) Field name Description and requirements this property is true, ECS validates that the provider's name and domain are unique.) Group Attribute Indicates the Active Directory attribute that is used to identify a group. Used for searching the directory by groups. Example: CN Active Directory only. Does not apply to other authentication providers. Note Once this value is set for a provider, it cannot be changed, because of the tenants that are using this provider may already have role assignments and permissions configured using group names in a format using the current attribute. Group Whitelist Optional. One or more group names as defined by the authentication provider. This setting will filter the group membership information that ECS retrieves about a user. When a group or groups are included in the whitelist, it means that ECS will be aware of a user's membership in the specified group[s] only. Multiple values (one per line in ECS portal, comma-separated in CLI and API) and wildcards (for example MyGroup*,TopAdminUsers*) are allowed. Blank value (default) means that ECS will be aware of any and all groups that a user belongs to. Asterisk (*) is the same as blank. Example: UserA belongs to Group1 and Group2. If the whitelist is blank, ECS knows that UserA is a member of Group1 and Group2. If the whitelist is "Group1", ECS knows that UserA is a member of Group1, but does not know that UserA is a member of Group2 (or of any other group). Use care when adding a whitelist value. For example, if mapping a user to a tenant is based on group membership, then ECS must be aware of the user's membership in the group. To restrict access to a namespace to users of certain group(s) only, one must: add these group(s) to the namesapce user mapping (using the CLI command viprcli tenant add-group), so the tenant is configured to accept only users of these group(s). add these group(s) to the whitelist, so that ECS is authorized to receive information about them 68 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

69 Add users and assign roles Table 6 Authentication provider settings (continued) Field name Description and requirements Note that by default, if no groups are added to the tenant user mapping, users from any groups are accepted, regardless of the whitelist configuration. Active Directory only. Does not apply to other authentication providers. Search Scope Search Base One Level (search for users one level under the search base) or Subtree (search the entire subtree under the search base). Indicates the Base Distinguished Name that ECS uses to search for users at login time and when assigning roles or setting ACLs. Example: CN=Users,DC=mydomaincontroller,DC=com This example searches for all users in the Users container. Example: CN=Users,OU=myGroup,DC=mydomaincontroller,DC=com This example searches for all users in the Users container in the mygroup organization unit. Note that the structure of the searchbase value begins with the "leaf" level and goes up to the domain controller level--the reverse of the structure seen in the Active Directory Users and Computers UI. Search Filter Indicates the string used to select subsets of users. Example: userprincipalname=%u Note ECS does not validate this value when you add the authentication provider. If an alternate UPN suffix is configured in the Active Directory, the Search Filter value must be of the format samaccountname= %U where %U is the username, and does not contain the domain name. (This setting not available in the UI.) (not applicable) Value that controls the maximum number of objects returned in a single search result. This is independent of size of the each returned object. If specified must be greater than 0. Cannot be higher than the max page size configured on the authentication provider. When ldaps protocol is used, SSL validates the certificate from the authentication provider. Default is false. If set to true, the LDAP needs to have a valid CA certificate. Authentication provider settings 69

70 Add users and assign roles Considerations when adding authentication providers When you configure ECS to work with Active Directory, you must decide whether to manage several domains in a single authentication provider, or to add separate authentication providers for each domain. The decision to add a single authentication provider, or multiple, depends on the number of domains in the environment, and the location on the tree from which the manager user is able to search. Authentication providers have a single search_base from which searches are conducted. They have a single manager account who must have read access at the search_base level and below. Use a single authentication provider for multiple domains if you are managing an Active Directory forest and: the manager account has privileges to search high enough in the tree to access all user entries the search will be conducted throughout the whole forest from a single search base, not just the domains listed in the provider. Otherwise, configure an authentication provider for each domain. Note that even if you are dealing with a forest and you have the correct privileges, you might not want to manage all the domains with a single authentication provider. You would still use one authentication provider per domain when you need granularity and tight control on each domain, especially to set the search base starting point for the search. Since there is only one search base per configuration, it needs to include everything that is scoped in the configuration in order for the search to work. The search base needs to be high enough in the directory structure of the forest for the search to correctly find all the users in the targeted domains. If the forest in the configuration contains ten domains but you target only three, do not use a single provider configuration, because the search will unnecessarily span the whole forest, and this may adversely affect performance. In this case, use three individual configurations. If the forest in the configuration contains ten domains and you want to target ten domains, a global configuration is a good choice, because there is less overhead to set up. Understanding the mapping of users into a namespace Domain users can be added to ECS using authentication providers. To make users available as namespace users they need to be mapped into the namespace. The authentication provider makes users belonging to specified domains and whitelisted groups available to ECS and they can be assigned to system roles. To associate users with a namespace and make them eligible to be object users for the namespace, you must associate the domain to which the users belong with the namespace and, if necessary, apply finer grained filtering based on the groups that belong to the domain and the attributes that have been assigned to the domain users. A domain can be mapped to a single namespace or can provide users for multiple namespaces. The ECS Portal and the ECS Management REST API provide the ability to specify mappings when a new namespace is registered and provide support for updating the mappings for all namespaces. Creating a namespace is an operation that requires System Admin 70 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

71 Add users and assign roles privileges; modifying a tenant and performing user mappings operations can be performed by a Namespace Admin. The user mappings assigned to different namespaces must not overlap, so if the Accounts namespace maps users from the same domain as the HR namespace, it must provide additional mappings to differentiate its users. In the example below, the Accounts namespace uses the corp.sean.com domain but maps users with specific attributes, in this case, those with their Department attribute set to Accounts in Active Directory. Figure 12 User mappings for a tenant using AD attributes The example below shows the use of multiple mapping criteria. All members of the corp.sean.com domain who belong to the Storage Admins group and have their Department attribute set to Accounts AND Company set to Acme, OR belong to the Storage Admins group and have their Department set to Finance, will be mapped into the namespace. Understanding the mapping of users into a namespace 71

72 Add users and assign roles Figure 13 Using multiple mapping criteria Map domain users into a namespace The ECS portal provides the ability to map users into a namespace based on the AD/LDAP domain, groups, and attributes associated with users. Before you begin An authentication provider must have been registered with ECS and must provide access to the domain from which you want to map users. The administrator of the AD must have configured the groups or users in AD before mapping the users from the ECS Portal. If you are using attribute mapping, each user must have the appropriate attribute value set in AD. You should understand the concepts associated with user mapping, described in Understanding the mapping of users into a namespace on page EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

73 Add users and assign roles Procedure 1. At the ECS portal, select Manage > Namespace. 2. In the Namespaces table, click on the Edit action for the namespace to open it for editing. 3. If a domain hasn't already been specified, click Add to add a mapping and enter the domain name in the Domain field. 4. Specify any groups that you want to use to map users into the namespace. The group or groups that you specify must exist in AD. 5. If you want to use attributes to map users into the namespace enter the name of the attribute and the value or values for the attribute. If you do not want to use attributes to map users into the namespace, click the delete button to remove the attribute fields from the current mapping. For users to be mapped into the domain, the attribute value set for the user must match the attribute value specified in ECS. 6. Save the namespace settings. Map domain users into a namespace 73

74 Add users and assign roles 74 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

75 CHAPTER 8 Configure a namepace for a tenant Introduction Understanding tenants...76 Understanding namespace settings Working with namespaces at the ECS portal Create and configure a namespace...78 Configure a namepace for a tenant 75

76 Configure a namepace for a tenant Introduction Namespaces provide the mechanism by which multiple tenants can access the ECS object store and ensures that the objects and buckets written by users of a tenant are segregated from users of other tenants. This article introduces some concepts around tenants and namespace settings: Understanding tenants on page 76 Understanding tenants Understanding namespace settings on page 77 Working with namespaces at the ECS portal on page 78 and describes the operations required to configure a namespace using the ECS portal: Create and configure a namespace on page 78 While the configuration operations described in this article use the ECS portal, the concepts described in Understanding tenants on page 76 and Understanding namespace settings on page 77 apply whether you are using the portal or the REST API. ECS supports access by multiple-tenants, where each tenant is defined by a namespace and the namespace has a set of configured users who can store and access objects within the namespace. Namespaces are global resources in ECS and a System Admin or Namespace Admin accessing ECS at any federated VDC can configure the namespace settings. In addition, object users assigned to a namespace are global and can access the object store from any federated VDC. The key characteristic of a namespace is that users from one namespace cannot access objects belonging to another namespace. In addition, ECS enables an enterprise to configure namespaces and to monitor and meter their usage, and enables management rights to be granted to the tenant so that it can perform configuration and monitoring and metering operations. It is also possible to use buckets as a means of creating sub-tenants. The bucket owner is the sub-tenant administrator and can assign users to the sub-tenant using access control lists. However, sub-tenants do not provide the same level of segregation as tenants; any user belonging to the tenant could be assigned privileges on a sub-tenant, so care must be taken when assigning users. The following scenarios are supported: Enterprise single tenant All users access buckets and objects in the same namespace. Sub-tenants (buckets) can be created to allow a subset of namespace users to access the same set of objects. A sub-tenant could be a department within the enterprise. Enterprise multi tenant Different departments within an organization are assigned to different namespaces and department users are assigned to each namespace. Cloud Service Provider single tenant A single namespace is configured and the Service Provider provides access to the object store for users within the enterprise or outside the enterprise. 76 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

77 Configure a namepace for a tenant Cloud Service Provider multi tenant The Service Provider assigns namespaces to different companies and assigns an administrator for the namespace. The namespace administrator for the tenant can then add users and can monitor and meter the use of buckets and objects. The features provided to enable management of tenants are described in Manage a tenant on page 84. Each tenant has access to the replication groups made available by the System Admin. Depending on the access patterns of a tenant, they may require replication groups that include sites in specific geographies. For example, if a client tenant is located in China, they might prefer to access replication groups that include VDCs located in China. Understanding namespace settings A namespace provides a mechanism by which objects and buckets can be segregated so that an object in one namespace can have the same name as an object in another namespace. ECS will always know which object is required by the namespace qualifier. The namespace is also configured with attributes that define which users can access the namespace and what characteristics the namespace has. You can think of an ECS namespace as a tenant. Users with the appropriate privileges can create buckets, and can create objects within buckets, in the namespace. The way in which namespace and bucket names are used when addressing objects in ECS is described in Address ECS object storage and use the Base URL on page 132. An ECS namespace has the following attributes: Default Replication Group The replication group in which a bucket will be created if no replication group is specified in a request. You can find out more information about the configuration of replication groups in Configure storage pools, VDCs, and replication groups on page 48 Namespace Administrators Users assigned to the Namespace Admin role for the namespace. The Namespace Admin is an ECS management user and can be a local or domain user. User Mappings The domains, groups, and attributes that identify the users who can be assigned as object users for a namespace. The way in which users are added to ECS and mapped to a specific namespace is described in Add users and assign roles on page 56. Allowed (and Disallowed) Replication Groups The ECS Management REST API enables a client to specify which replication groups can be used by the namespace. This is not available from the ECS portal. It is also possible to specify retention policies and specify a quota for the namespace. Further information on using these features is provided in Manage a tenant on page 84. Quota When enabled, a quota size set against the namespace can cause an event to be logged (a soft quota) or access to be blocked (hard quota) when a specified storage limit is reached. Understanding namespace settings 77

78 Configure a namepace for a tenant Retention Policy A namespace can have a number of associated retention polices, where each policy defines a retention period. By applying a retention policy to a number of objects, rather than applying a retention period directly, a change the retention policy will cause the retention period to be changed for the complete set of objects to which the policy has been applied. A request to modify an object that falls before the expiration of the retention period will be disallowed. Working with namespaces at the ECS portal The namespace portal page, Manage > Namespace, enables namespaces to be created and provides a namespace table which lists the namespaces that exist and allows them to be edited. Figure 14 Namespace management page The namespace table comprises the following fields: Field Name Quota Replication Group Notification Quota Max Quota Actions Description Name of the namespace. Quota, if any, that has been assigned to the namespace. Default replication group for the namespace. Quota limit at which notification is generated. Quota limit at which writes to the namespace will be blocked. Actions that can be performed on the namespace. Edit and Delete actions are available. Create and configure a namespace You can create a new namespace or change the configuration of an existing namespace at the Manage > Namespace page. Before you begin 78 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation To perform this operation, you must be assigned to the System Admin role in ECS. A replication group must exist. The replication group provides access to storage pools in which object data is stored.

79 Configure a namepace for a tenant If you want to allow domain users to access the namespace, an authentication provider must have been added to ECS. In addition, if you intend to configure domain object users, you should plan how you want to map users into the namespace. You can refer to Add users and assign roles on page 56 for more information on mapping users. You should ensure you are familiar with the general information about namespaces provided in Understanding namespace settings on page 77. Procedure 1. At the ECS portal, select Manage > Namespace 2. To create a new namespace, select New Namespace. To edit the configuration of an existing namespace, choose the Edit action associated with the existing namespace. 3. Specify appropriate value for each of the fields. Guidance on the settings for each field is provided in the table below. Field Name Admin Replication Group Quota Retention Policies Domain Description The name of the namespace. This name must be in all lowercase characters. Once created, you cannot change this name. User Id of one or more users who you want to assign to the Namespace Admin role; a list of users should be comma separated. Namespace Admins can be local or domain users. If you want the Namespace Admin to be a domain user, you will need to ensure that an authentication provider has been added to ECS. Refer to Add users and assign roles on page 56 for details. The default replication group for the namespace. This control allows a quota to be enabled and for the behavior when the quota is reached to be specified. See Step 4 on page 79 Enables one or more retention policies to be added and configured. Existing retention policies can be edited, disabled, or deleted. See Step 5 on page 79 Enables AD/LDAP domains to be specified and the rules for including users from the domain to be configured. This enables users belonging to the domain to use the ECS self-service capability to register as object users. See Step 6 on page Enable and configure a quota. a. Set the Quota control to Enabled if you want to set a quota for the namespace. b. Choose Notification Only or Block Access If you choose to block access when a specified storage limit is reached, you can also specify a percentage of that limit at which a notification will be sent. 5. Add and Configure Retention Policies. a. In the Retention Policies area, select Add to add a new policy. Create and configure a namespace 79

80 Configure a namepace for a tenant b. Enter a name for the policy. c. Specify the period for the Retention Policy. This can be a value in minutes or you can select the Infinite checkbox to ensure that buckets to which this retention policy is assigned are never deleted. 6. Specify an AD/LDAP domain whose users can log in to ECS and perform administration tasks for the namespace. Enter the name of the domain and specify groups and attributes to provide finer grained control over the domain users that will be allowed to access ECS in the current namespace. To perform more complex mappings using groups and attributes, you should refer to Add users and assign roles on page Select Save. 80 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

81 CHAPTER 9 Obtain and upload a license file to the ECS Portal Licensing...82 Obtain the EMC ECS license file...82 Obtain and upload a license file to the ECS Portal 81

82 Obtain and upload a license file to the ECS Portal Licensing EMC ECS licensing is capacity-based. At a minimum you need to obtain at least an ECS license and upload it to the appliance. The Settings > License page provides additional details. Obtain the EMC ECS license file You need to obtain the license file (.lic) from the EMC license management web site for uploading to ECS. Before you begin In order to obtain the license file, you must have the License Authorization Code (LAC), which was ed from EMC. Procedure 1. Go to support.emc.com 2. Select Support > Service Center. 3. Select Get and Manage Licenses. 4. Select ECS from the list of products. 5. On the LAC Request page, enter the LAC code and Activate. 6. Select the entitlements to activate and Start Activation Process. 7. Select Add a Machine to specify any meaningful string for grouping licenses. The "machine name" does not have to be a machine name at all; enter any string that will help you keep track of your licenses. 8. Enter the quantities for each entitlement to be activated, or select Activate All. Click Next. If you are obtaining licenses for a multisite (geo) configuration, you should distribute the controllers as appropriate in order to obtain individual license files for each virtual data center. 9. Optionally specify an addressee to receive an summary of the activation transaction. 10.Click Finish. 11.Click Save to File to save the license file (.lic) to a folder on your computer. This is the license file that is needed during initial setup of ECS, or when adding a new license later in the ECS Portal (Settings > Licensing). 12.From the ECS Portal, select Settings > Licensing. 13.Select Choose File, select your license file, and select Upload. The license display updates. 82 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

83 CHAPTER 10 Manage a tenant Introduction Quotas Retention periods and policies Lock buckets and users...86 Metering Audit buckets...88 Manage a tenant 83

84 Manage a tenant Introduction ECS provides a number of features to support the management of a tenant. The following features are supported: Users The ability to assign a Namespace Admin for the namespace and to create object users for the namespace is described in Add users and assign roles on page 56. Quotas The ability to set quotas on namespaces and buckets is described in Quotas on page 84. Retention Periods The ability to create retention policies is described in Retention periods and policies on page 85. Lock buckets and users The ability to lock buckets and users is described in Lock buckets and users on page 86. Metering The ability to meter the writing of data to buckets and namespaces is described in Metering on page 87. Audit buckets The ability to audit the operations associated with buckets is described in Audit buckets on page 88. Quotas You can set soft and hard quotas on a namespace and on buckets created within a namespace. Soft quotas cause events to be logged to inform you that the quota has been reached; hard quotas provide a hard limit on the amount of object storage that can be used for a bucket or namespace - when the limit is reached, access to the bucket or namespace is blocked. Quotas can be set from the ECS Portal or using the API and the CLI. Setting quotas from the portal You can set quotas for a namespace from the Manage > Namespace page, as described in Configure a namepace for a tenant on page 76. Quotas for a bucket are set from the Manage > Bucket page, as described in Create and configure buckets on page 138. Setting quotas using the API The following API paths provide the ability to set quotas: Method Description PUT/GET/DELETE /object/namespaces/ namespace/{namespace}/quota Sets the quota for a namespace. The payload specifies hard and soft quotas. 84 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

85 Manage a tenant Method PUT/GET/DELETE /object/bucket/ {bucketname}/quota Description Sets the quota for a bucket. The payload specifies hard and soft quotas. Retention periods and policies You can find more information about the ECS Management REST API in: Use the ECS Management REST API on page 250 and the online reference is here. ECS provides the ability to prevent data being modified within a specified retention period. Retention periods can be defined in metadata associated with objects and buckets and is checked each time a request to modify an object is made. Retention periods are supported on all object interfaces S3, Swift, Atmos, and CAS. However, CAS data is immutable so the retention period when applied to CAS refers to the ability to delete CAS objects. While the retention period for a bucket can be set at the ECS Portal, the assignment of a retention period, or policy, to an object must be performed using the object interface. There are two ways of defining retention: retention periods and retention policies. Retention Periods Retention periods are assigned at the object or bucket level. Where a retention period is assigned on a bucket, each time an attempt is made to modify an object within a bucket, the retention period for the bucket is checked and an expiration time calculated. For example, where a financial document must be retained for 3 years from the date on which it is created. It is also possible to specify that the object is retained indefinitely. Retention Policies Retention policies enable retention use cases to be captured and applied to objects. For example, different types of documents could have different retention periods. You could require the following retention periods: Financial - 3 years Legal - 5 years - 6 months Where a retention policy is applied to a number of objects, by changing the policy, the retention period for all objects to which the policy has been applied can be changed. How to create retention policies You can configure the retention policies that are available for the namespace from the ECS Portal, refer to: Configure a namepace for a tenant on page 76 or you can create them using the ECS Management REST API, a summary of which is provided below. Method PUT /object/bucket/{bucketname}/retention Description The retention value for a bucket defines a default retention period which is applied to every object within a bucket. So, if you set a Retention periods and policies 85

86 Manage a tenant Method Description retention period of 1 year, an object from the bucket can not be deleted for one year. GET /object/bucket/{bucketname}/retention POST /object/namespaces/namespace/ {namespace}/retention PUT /object/namespaces/namespace/ {namespace}/retention/{class} GET /object/namespaces/namespace/ {namespace}/retention Returns the retention period that is currently set for a specified bucket. For namespaces, the retention setting acts like a policy, where each policy is a <Name>: <Retention period> pair. You can define a number of retention policies for a namespace and you can assign a policy, by name, to an object within the namespace. This allows you to change the retention period of a set of objects that have the same policy assigned by changing the corresponding policy. Updates the period for a retention class that is associated with a namespace. Returns the retention classes defined for a namespace. Lock buckets and users You can find out how to access the ECS Management REST API in the following article: Use the ECS Management REST API on page 250 and the online reference is here. How to apply retention policies and periods You can apply retention periods to buckets at the ECS Portal. When you create objects or buckets using the object service protocols, for example, when you create an S3 bucket using a client that supports the S3 protocol, you can apply the retention period or retention policy using x-ems headers. When you create objects, you can apply the following retention period and retention policy headers: x-emc-retention-period x-emc-retention-policy When you create a bucket, you can set the retention period using the x-emc-retentionperiod header. ECS provides the ability to prevent access to a bucket and to prevent user access. Support for the bucket and user lock operations is provided by the ECS Management REST API. There is no support for locking buckets and users in the ECS Portal. The following calls are supported: Method PUT /object/bucket/{bucketname}/lock DELETE /object/bucket/{bucketname}/lock Description Locks a bucket so that all writes to the bucket are disallowed. Unlocks a bucket so that writes to the bucket are re-enabled. 86 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

87 Manage a tenant Method PUT /object/users/{userid}/lock DELETE /object/user/{userid}/lock Description Locks an object user (not AD user) such that all subsequent API operations performed by the user return an error. Unlocks an object user (not AD user) such that the user is re-enabled to perform further API operations. You can find out how to access the ECS Management REST API in the following article: Use the ECS Management REST API on page 250 and the online reference is here. Metering ECS provides support for metering the use of the object storage at the namespace and bucket level. Metering using the portal You can use the ECS Portal to monitor the use of namespace and buckets. The Monitor > Metering page enables a namespace or a specific bucket from a namespace to be selected and its metering data displayed. Table 7 Bucket and namespace metering Attribute Total Size (GB) Object Count Objects Created Objects Deleted Bandwidth Ingress (MB) Bandwidth Egress (MB) Description Total size of the objects stored in the selected namespace or bucket. Number of objects associated with the selected namespace or bucket at the end time specified in the filter. Number of objects created in the selected namespace or bucket in the time period. Number of objects deleted from the selected namespace or bucket in the time period. Total of incoming object data (writes) for the selected namespace or bucket during the specified period. Total of outgoing object data (reads) for the selected namespace or bucket during the specified period. Note Metering data is not available immediately as it can take a significant amount of time to gather the statistics for data added to the system and deleted from the system. Refer to Monitor storage: metering and capacity on page 106 for more information on accessing these details. Metering using the API The following API paths provide the ability to retrieve metering information: Metering 87

88 Manage a tenant Method GET /object/billing/buckets/{namespace}/ {bucket}/info?sizeunit=<kb MB GB> Description Gets the current usage for a bucket in a specified namespace. GET /object/billing//buckets/{namespace}/ {bucket}/sample? start_time=<iso8061_format>&end_time=<iso8 061_format>&marker=<string_marker> &sizeunit=<kb MB GB> GET /object/billing/namespace/ {namespace_name}/info? marker=<string_marker>&include_bucket_detail =<true false>&sizeunit=<kb MB GB> Samples a bucket activity for the given time slice. By default, a bucket's minimum sample resolution is 5 minutes and samples will be retained for 30 days. If the start time and end time do not fall on sample boundaries, an error will be returned. If the time range spans multiple low-level samples, the data will be aggregated for the time period to produce one data point. Gets usage information for all of the buckets in a namespace. Note When bucket details are included, the total size on disk might be different to the total size without bucket details. This is due to bucket size being rounded and summed to give the total size. GET /object/billing/namespace/ {namespace_name}/sample? start_time=<iso8061_format>&end_time=<iso8 061_format>&marker=<string_marker> &include_bucket_detail=<true false>&sizeunit=<kb MB GB> Gets a snapshot for a particular time sample for a namespace. By default, buckets and namespaces will be sampled every 5 minutes and samples will be retained for 30 days. If the start time and end time do not fall on sample boundaries an error will be returned. If the time range spans multiple low-level samples, the data will be aggregated for the time period to produce one data point. You can find more information about the ECS Management REST API in: Use the ECS Management REST API on page 250 and the online reference is here. Audit buckets The controller API provides the ability to audit the use of the S3, EMC Atmos, and OpenStack Swift object interfaces. The following operations on object containers (S3 buckets, EMC Atmos subtenants, and OpenStack Swift containers) are logged. Create Bucket Delete Bucket Update Bucket Set Bucket ACL Change Bucket Owner Set Bucket Versioning 88 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

89 Manage a tenant Set Bucket Versioning Source Set Bucket Metadata Set Bucket Head Metadata Set Bucket Expiration Policy Delete Bucket Expiration Policy Set Bucket Cors Configuration Delete Bucket Cors Configuration Audit logging at the portal You can use the Portal Monitor > Events page to detect the generation of an audit log event. The root user should only be used for initial access to the system. On initial access, the root user password should be changed at the Settings > Password page and one or more new System Admin accounts should be created. From an audit perspective, it is important to know which user carried out changes to the system, so root should not be used, and each System Admin user should have their own account. You can refer to Monitor events: audit portal, API, and CLI events and system alerts on page 128 for more information on using the events log. Audit API Support for bucket auditing is provided by the following ECS Management REST API calls: Method Description GET /monitoring/events Retrieves the audit events for a specified namespace and time interval. You can find more information about the ECS Management REST API in: Use the ECS Management REST API on page 250 and the online reference is here. Audit buckets 89

90 Manage a tenant 90 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

91 CHAPTER 11 Fail over an ECS site Fail over a site/delete a VDC...92 Fail over an ECS site 91

92 Fail over an ECS site Fail over a site/delete a VDC Use this procedure to delete a VDC. Deleting a VDC initiates site fail over when the VDC you are deleting is part of a multi-site federation. If a disaster occurs, an entire VDC can become unrecoverable. ECS initially treats the unrecoverable VDC as a temporary site failure. If the failure is permanent, you must remove the VDC from the federation to initiate fail over processing which reconstructs and reprotects the objects stored on the failed VDC. The recovery tasks run as a background process. Review the recovery process by using the Monitor > Geo Replication > Failover Procesing. Procedure 1. Log in to one of the operational VDCs in the federation. 2. Go to Manage > Replication Group. 3. Click Edit for the replication group that contains the VDC to delete. 4. Click Delete in the row that contains the VDC and storage pool to remove. 5. Click Save. 6. Go to Manage > VDC. The status for the permanently removed VDC changes to Permanently failed. 7. Select Delete from the drop down in the row of the VDC to remove. 8. Click Save. 92 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

93 PART 5 Monitor Chapter 12, "Monitor resources: hardware, network traffic, disk bandwidth, node and process health" Chapter 13, " Monitor storage: metering and capacity " Chapter 14, "Monitor service logs" Chapter 15, "Monitor services: chunks, erasure coding, geo-replication, and recovery status" Chapter 16, "Monitor events: audit portal, API, and CLI events and system alerts" Monitor 93

94 Monitor 94 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

95 CHAPTER 12 Monitor resources: hardware, network traffic, disk bandwidth, node and process health About resource monitoring Using monitoring pages Monitor hardware...98 Monitor network traffic Monitor disk bandwidth Monitor node and process health Monitor resources: hardware, network traffic, disk bandwidth, node and process health 95

96 Monitor resources: hardware, network traffic, disk bandwidth, node and process health About resource monitoring Describes the ECS Portal features for resource monitoring. Resource monitoring includes: Hardware health on page 98 Network traffic metrics on page 99 Disk bandwidth on page 100 Node and process health on page 102 Each of these monitoring areas has a corresponding page under the ECS Portal Monitor menu. See the following articles for information about other ECS Portal monitoring features: Monitor storage: metering and capacity on page 106 Monitor services: chunks, erasure coding, geo-replication, and recovery status on page 116 Monitor events: audit portal, API, and CLI events and system alerts on page 128 Using monitoring pages Introduces the basic techniques for using monitoring pages in the ECS Portal. The ECS Portal monitoring pages share a set of common interactions. These are: Search: the search icon appears when you can narrow monitoring results by matching search text to result rows containing the search text. Figure 15 Search Refresh: the refresh icon allows you to update the monitoring display with the latest data. Figure 16 Refresh Filter: fill in filter fields and the date range and select Filter to display result rows that match all filter fields. The default date range is always yesterday and today. Figure 17 Filter Sort by column: Select a column head once to sort the results by that column in ascending order. Select again to change the sort order to descending order. 96 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

97 Monitor resources: hardware, network traffic, disk bandwidth, node and process health Figure 18 Sort Drill down displays with breadcrumbs: Breadcrumbs let you quickly drill up when you have drilled down into detail screens. See the example below. History charts with left to right mouse-overs: Get detailed charts showing hourly snapshots for the last five days worth of data which you can browse through using your mouse as a left-to-right chart cursor. See the example below. Figure 19 Navigating with breadcrumbs Highlighted text in a table row indicates a link to a detail display. Selecting the link drills down to the next level of detail. On drill down displays, a path string shows your current location in the sequence of drill down displays. This path string is called a breadcrumb trail or breadcrumbs for short. Selecting any highlighted breadcrumb jumps up to the associated display. Figure 20 Data chart with active cursor When you select a History button, all available charts for that row display below the table. Mouse over a chart from left to right to see a vertical line that helps you find a Using monitoring pages 97

98 Monitor resources: hardware, network traffic, disk bandwidth, node and process health Monitor hardware specific date-time point on the chart. A pop-up display shows the value and timestamp for that point. Describes how to use the Monitor > Hardware Health page. Hardware health is designated by three states: Good: The hardware component is in normal operating condition. Suspect: Either the hardware component is transitioning from good to bad because of decreasing hardware metrics, or there is a problem with a lower-level hardware component, or the hardware is not detectable by the system because of connectivity problems. Bad: The hardware needs replacement. In the case of disks, these states have the following meanings: Good: The system is actively reading from and writing to the disk. Suspect: The system no longer writes to the disk but will read from it. Note that "swarms" of suspect disks are likely caused by connectivity problems at a node. These disks will transition back to Good when the connectivity issues clear up. Bad: The system neither reads from nor writes to the disk. Replace the disk. Once a disk has been identified as bad by the ECS system, it cannot be reused anywhere in the ECS system. Because of ECS data protection, when a disk fails, copies of the data that was once on the disk are recreated on other disks in the system. A bad disk only represents a loss of capacity to the system--not a loss of data. When the disk is replaced, the new disk does not have data restored to it. It simply becomes raw capacity for the system. Procedure 1. Select Monitor > Hardware Health. 2. Locate the table row for the target storage pool. 3. Optionally, select a storage pool name to drill down to the node display. 4. Optionally, select a node endpoint to drill down to the disk display. 98 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

99 Monitor resources: hardware, network traffic, disk bandwidth, node and process health Figure 21 Hardware Health Note See Replace an ECS storage disk in a U-Series Appliance on page 294for information on using disk IDs to help identify disks needing replacement. Monitor network traffic Describes the ECS Portal monitoring page for network traffic. The Monitor > Traffic Metrics page provides network traffic metrics at the virtual data center or the individual node level. The charts show data for the last seven days. Table 8 Network traffic metrics Metric label R Avg. Latency (ms) W Avg. Latency (ms) R Bandwidth Description Average latency for reads in milliseconds. Average latency for writes in milliseconds. Bandwidth for reads in megabytes per second. Monitor network traffic 99

100 Monitor resources: hardware, network traffic, disk bandwidth, node and process health Table 8 Network traffic metrics (continued) Metric label W Bandwidth R Transactions (per/s) W Transactions (per/s) Recent Transaction Failures per type Description Bandwidth for writes in megabytes per second. Read transactions per second. Write transactions per second. For each error code that occurred in the monitoring period, display that code's percent of the total errors. Procedure 1. Select Monitor > Traffic Metrics. 2. Locate the target VDC name. 3. Optionally, select the VDC name to drill down to the nodes display. 4. Select History button ofr the target VDC or node. Figure 22 Network traffic charts for a VDC Monitor disk bandwidth Describes the ECS Portal monitoring page for disk bandwidth. The Monitor > Disk bandwidth page provides disk use metrics at the virtual data center or the individual node level. There is one row for read and another for write for each VDC or node. The charts show data for the last seven days. 100 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

101 Monitor resources: hardware, network traffic, disk bandwidth, node and process health Table 9 Disk bandwidth metrics Metric label Total Hardware Recovery Erasure Encoding XOR Consistency Checker Geo User Traffic Description Total disk bandwidth used in kilobytes per second for either read or write operations. Amount of disk bandwidth used to recover data after hardware failures. Amount of disk bandwidth used in system erasure coding operations. Amount of disk bandwidth of the total used in the system's XOR data protection operations. Note that XOR operations occur for systems with three or more sites (VDCs). Amount of disk bandwidth used to check for inconsistencies between protected data and its replicas. Amount of disk bandwidth used to support geo replication operations. Amount of disk bandwidth used by object users. Procedure 1. Select Monitor > Disk Bandwidth. 2. Locate the target VDC name and either the Read or Write table row for that VDC. 3. Optionally, select the Node Count to drill down to a table with rows for the nodes in the VDC. 4. Select the History button for the VDC or node. Figure 23 Disk Bandwidth Monitor disk bandwidth 101

102 Monitor resources: hardware, network traffic, disk bandwidth, node and process health Monitor node and process health Describes the ECS Portal monitoring page for node and process health. The Monitor > Node & Process Health page provides metrics that can help assess the health of the VDC, node, or node process. Table 10 VDC, node, and process health metrics Metric label Level Description Avg. NIC Bandwidth VDC and Node Average bandwidth of the network interface controller hardware used by the selected VDC or node. Avg. CPU Usage (%) VDC and Node Average percent of the CPU hardware used by the selected VDC or node. Avg. Memory Usage VDC and Node Average usage in megabytes of the aggregate memory available to the VDC or node. Relative NIC (%) VDC and Node Percent of the available bandwidth of the network interface controller hardware used by the selected VDC or node. Relative Memory (%) VDC and Node Percent of the memory used relative to the memory available to the selected VDC or node. CPU (%) Process Percent of the node's CPU used by the process. Memory Usage Process The memory in megabytes used by the process. Relative Memory (%) Process Percent of the memory used relative to the memory available to the process. Avg. # Thread Process Average number of threads used by the process. Last Restart Process The last time the process restarted on the node. Procedure 1. Locate the table row for the target VDC. 102 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

103 Monitor resources: hardware, network traffic, disk bandwidth, node and process health 2. Optionally, select the VDC name to drill down to a table with rows for each node in the VDC. 3. Optionally, select the a node endpoint to drill down to a table with rows for each process running on the node. 4. Select the History button for the target VDC, node, or process. Figure 24 Node & Process Health Monitor node and process health 103

104 Monitor resources: hardware, network traffic, disk bandwidth, node and process health 104 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

105 CHAPTER 13 Monitor storage: metering and capacity Introduction Using monitoring pages Monitor capacity Monitor metering data Monitor storage: metering and capacity 105

106 Monitor storage: metering and capacity Introduction ECS provides the ability to meter the creation and deletion of objects at the bucket and namespace level. In addition, ECS enables the capacity utilization to be monitored. Each of these monitoring capabilities has a corresponding panel under the ECS Portal Monitor menu and the mechanism for accessing it, and the data provided by the panel, is described in the following topics: Monitor metering data on page 110 Monitor capacity on page 108 See the following articles for information about other ECS Portal monitoring features: Monitor resources: hardware, network traffic, disk bandwidth, node and process health on page 96 Monitor services: chunks, erasure coding, geo-replication, and recovery status on page 116 Monitor events: audit portal, API, and CLI events and system alerts on page 128 Using monitoring pages Introduces the basic techniques for using monitoring pages in the ECS Portal. The ECS Portal monitoring pages share a set of common interactions. These are: Search: the search icon appears when you can narrow monitoring results by matching search text to result rows containing the search text. Figure 25 Search Refresh: the refresh icon allows you to update the monitoring display with the latest data. Figure 26 Refresh Filter: fill in filter fields and the date range and select Filter to display result rows that match all filter fields. The default date range is always yesterday and today. Figure 27 Filter Sort by column: Select a column head once to sort the results by that column in ascending order. Select again to change the sort order to descending order. 106 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

107 Monitor storage: metering and capacity Figure 28 Sort Drill down displays with breadcrumbs: Breadcrumbs let you quickly drill up when you have drilled down into detail screens. See the example below. History charts with left to right mouse-overs: Get detailed charts showing hourly snapshots for the last five days worth of data which you can browse through using your mouse as a left-to-right chart cursor. See the example below. Figure 29 Navigating with breadcrumbs Highlighted text in a table row indicates a link to a detail display. Selecting the link drills down to the next level of detail. On drill down displays, a path string shows your current location in the sequence of drill down displays. This path string is called a breadcrumb trail or breadcrumbs for short. Selecting any highlighted breadcrumb jumps up to the associated display. Figure 30 Data chart with active cursor When you select a History button, all available charts for that row display below the table. Mouse over a chart from left to right to see a vertical line that helps you find a Using monitoring pages 107

108 Monitor storage: metering and capacity Monitor capacity Storage capacity data specific date-time point on the chart. A pop-up display shows the value and timestamp for that point. You can monitor the capacity utilization of storage pools, nodes, and disks. The capacity tables and displays are shown in Storage capacity data on page 108. Each table has an associated History display that enables you to see how the table data has changed over time. Using the ECS Management REST API you can retrieve data programatically using custom clients. Support for this feature and other features that enable a tenant to be managed is provided in Manage a tenant on page 84. The ECS Management REST API Reference is provided here. Procedure 1. At the ECS Portal, select Monitor > Capacity. 2. You can drill down into the nodes and to individual disks by selecting the appropriate link in the table. Guidance on navigating the tables is provided in Using monitoring pages on page To display the way in which the capacity has changed over time, select History for the storage pool, node, or disk that you are interested in. Storage capacity for storage pools, nodes, and disks can be displayed at the Monitor > Capacity Utilization page. The capacity utilization areas are described in: Storage Pool Capacity on page 108 Node Capacity Utilization on page 109 Disk Capacity Utilization on page 110 Refer to Using monitoring pages on page 96 for guidance on navigating the capacity displays. Storage Pool Capacity Table 11 Capacity Utilization: Storage Pool Attribute Storage Pool Nodes Disks Usable Capacity Used Capacity Available Capacity Description Name of the storage pool. Number of nodes in the storage pool. Click node number to open: Node Capacity Utilization on page 109 Number of disks in the storage pool. Total usable capacity. This is total of the capacity already used and the capacity still free for allocation. Used capacity in the storage pool. Capacity available for use. 108 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

109 Monitor storage: metering and capacity Table 11 Capacity Utilization: Storage Pool (continued) Attribute Actions Description The following action is provided: History. History displays the capacity usage history graphically. The history display for the storage pool capacity utilization table is shown below. Node Capacity Utilization Table 12 Capacity Utilization: Node Attribute Nodes Disks Usable Capacity Used Capacity Available Capacity Node Status Actions Description IP address of the node. Number of disks associated with the node. Click node number to open: Disk Capacity Utilization on page 110 Total usable capacity provided by the disks within the node. This is total of the capacity already used and the capacity still free for allocation. Capacity used within the node. Remaining capacity available in the node. Check mark indicates the node status is Good. The following action is provided: History History displays the capacity usage history graphically. The history display for the node capacity utilization table is shown below. Storage capacity data 109

110 Monitor storage: metering and capacity Disk Capacity Utilization Table 13 Capacity Utilization: Disk Attribute Disks Usable Capacity Used Capacity Available Capacity Disk Status Actions Description Disk identifier. Usable capacity provided by the disk. Capacity used on the disk. Remaining capacity available on the disk. Check mark indicates the node status is Good. The following action is provided: History History displays the capacity usage history graphically. The history display for the disk utilization table is shown below. Monitor metering data 110 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation You can choose the namespace, or bucket within a namespace, for which you want to display metering data, and you can choose the time period for which the metering data is displayed. The available metering data is detailed in Metering data on page 111. Using the ECS Management REST API you can retrieve data programatically using custom clients. Support for this feature and other features that enable a tenant to be managed is

111 Monitor storage: metering and capacity provided in Manage a tenant on page 84. The ECS Management REST API Reference is provided here. Procedure 1. At the ECS Portal, select Monitor > Metering. 2. Select the namespace for which you want to display metering data. If you are a Namespace Admin, you will only be able to select your namespace. 3. Optionally, enter the name of the bucket for which you want to display object data. If you do not specify a bucket, the object metering data will be the totals for all buckets in the namespace. 4. Use the From and To calendars to choose the time period for which data will be displayed. Metering data is kept for around 30 days. 5. Click Filter to display the metering data for the selected namespace and bucket, and time period. Metering data Object metering data for a specified namespace, or a specified bucket within a namespace, can be obtained for a defined time period at the ECS portal Monitor > Metering page. Table 14 Bucket and namespace metering Attribute Total Size (GB) Object Count Objects Created Objects Deleted Bandwidth Ingress (MB) Bandwidth Egress (MB) Description Total size of the objects stored in the selected namespace or bucket. Number of objects associated with the selected namespace or bucket at the end time specified in the filter. Number of objects created in the selected namespace or bucket in the time period. Number of objects deleted from the selected namespace or bucket in the time period. Total of incoming object data (writes) for the selected namespace or bucket during the specified period. Total of outgoing object data (reads) for the selected namespace or bucket during the specified period. Note Metering data is not available immediately as it can take a significant amount of time to gather the statistics for data added to the system and deleted from the system. Metering data 111

112 Monitor storage: metering and capacity 112 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

113 CHAPTER 14 Monitor service logs Service logs ECS service log locations Monitor service logs 113

114 Monitor service logs Service logs Describes the location and function of the ECS service logs. Storage administrators can access ECS service logs if you have permission to SSH into a node and access the logs. Using the Monitoring pages of the ECS Portal is usually a better way to understand the state of your system. ECS service log locations Describes the location and content of ECS service logs. You can access ECS service logs directly by an SSH session on a node. Change to the following directory: /opt/emc/caspian/fabric/agent/services/object/ main/log to find object service logs: authsvc.log: Records information from the authentication service. blobsvc*.log: These logs record aspects of the blob service. cassvc*.log: These logs record aspects of the CAS service. coordinatorsvc.log: Records information from the coordinator service. ecsportalsvc.log: Records information from the ECS Portal service. eventsvc*.log: These logs record aspects of the event service. This information is available in the ECS Portal Monitoring menu. hdfssvc*.log: These logs record aspects of the HDFS service. objcontrolsvc.log: Records information from the object service. objheadsvc*.log: These logs record aspects of the various object heads supported by the object service. provisionsvc*.log: These logs record aspects of the ECS provisioning service. resourcesvc*.log: These logs record aspects of the ECS chunking functions. 114 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

115 CHAPTER 15 Monitor services: chunks, erasure coding, georeplication, and recovery status About service monitoring Using monitoring pages Monitor chunks Monitor erasure coding Monitor recovery status Monitor geo-replication: rate and chunks Monitor geo-replication: Recovery Point Objective (RPO) Monitor geo-replication: failover Monitor geo replication: bootstrap processing Monitor services: chunks, erasure coding, geo-replication, and recovery status 115

116 Monitor services: chunks, erasure coding, geo-replication, and recovery status About service monitoring Describes the ECS Portal features for service monitoring. Service monitoring includes: Chunk Summary on page 118 Erasure Coding on page 119 Recovery Status on page 120 Geo-replication: Rates and Chunks on page 121 Geo-replication: RPO (recovery point objective) on page 122 Geo-replication: Failover Processing on page 123 Geo-replication: Bootstrap Processing on page 123 Each of these monitoring areas has a corresponding page under the ECS Portal Monitor menu. The Geo Replication page has four separate displays within it. See the following articles for information about other ECS Portal monitoring features: Monitor resources: hardware, network traffic, disk bandwidth, node and process health on page 96 Monitor storage: metering and capacity on page 106 Monitor events: audit portal, API, and CLI events and system alerts on page 128 Using monitoring pages Introduces the basic techniques for using monitoring pages in the ECS Portal. The ECS Portal monitoring pages share a set of common interactions. These are: Search: the search icon appears when you can narrow monitoring results by matching search text to result rows containing the search text. Figure 31 Search Refresh: the refresh icon allows you to update the monitoring display with the latest data. Figure 32 Refresh Filter: fill in filter fields and the date range and select Filter to display result rows that match all filter fields. The default date range is always yesterday and today. Figure 33 Filter 116 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation Sort by column: Select a column head once to sort the results by that column in ascending order. Select again to change the sort order to descending order.

117 Monitor services: chunks, erasure coding, geo-replication, and recovery status Figure 34 Sort Drill down displays with breadcrumbs: Breadcrumbs let you quickly drill up when you have drilled down into detail screens. See the example below. History charts with left to right mouse-overs: Get detailed charts showing hourly snapshots for the last five days worth of data which you can browse through using your mouse as a left-to-right chart cursor. See the example below. Figure 35 Navigating with breadcrumbs Highlighted text in a table row indicates a link to a detail display. Selecting the link drills down to the next level of detail. On drill down displays, a path string shows your current location in the sequence of drill down displays. This path string is called a breadcrumb trail or breadcrumbs for short. Selecting any highlighted breadcrumb jumps up to the associated display. Figure 36 Data chart with active cursor When you select a History button, all available charts for that row display below the table. Mouse over a chart from left to right to see a vertical line that helps you find a Using monitoring pages 117

118 Monitor services: chunks, erasure coding, geo-replication, and recovery status Monitor chunks specific date-time point on the chart. A pop-up display shows the value and timestamp for that point. Describes the ECS Portal monitoring page for chunks. This page reports statistics for sealed chunks in the local zone. A sealed chunk is one that can no longer accept writes. It is immutable. Table 15 Chunk tables Table Chunk Count of Each Type Total Length of Each Chunk Type Avg Sealed Length of Each Type Description Shows number and percentage of sealed chunks for different chunk types per each storage pool configured in the local zone. Shows total logical size of sealed chunks for different chunk types per each storage pool configured in the local zone. Shows average logical size of sealed chunks for different chunk types per each storage pool configured in the local zone. Table 16 Chunk metrics Metric label Storage pools Repo User data Level-0 btree Level-0 journal Level-1 btree Level-1 journal Geo copy XOR Description This column provides the list of storage pools configured in the local VDC. Each row provides chunk metrics for the specified storage pool. Repo (repository) chunks hold user data. This column provides relevant data for the repo chunks in the storage pool. B-tree chunks contain system metadata about the arrangement and location of user data. Level-0 refers to the lowest level of data in the system, like object tables and basic lookup tables. This column provides relevant data for the B-tree level-0 chunks in the storage pool. Journal chunks contain failure recovery data. Level-0 refers to the lowest level of data in the system, like object tables and lookup tables. This column provides relevant data for the journal level-0 chunks in the storage pool. B-tree chunks contain system metadata about the arrangement and location of user data. Level-1 refers to more complex data in the system. This column provides relevant data for the B-tree level-1 chunks in the storage pool. Journal chunks contain failure recovery data. Level-1 refers to more complex data in the system. This field provides relevant data for the journal level-1 chunks in the storage pool. Geo chunks are chunks containing replicas of data from other zones (VDCs). This field provides relevant data for the geo-copy chunks in the storage pool. XOR chunks are chunks that save disk space by using the XOR algorithm to compress data from other chunks and replace those chunks with an XOR chunk. 118 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

119 Monitor services: chunks, erasure coding, geo-replication, and recovery status Table 16 Chunk metrics (continued) Metric label Description This field provides relevant data for the XOR chunks in the storage pool. Total The total number of chunks in the storage pool. Chunk metrics Figure 37 Chunk Summary Monitor erasure coding Describes how to use the Monitor > Erasure Coding page. The erasure coding display monitors the amount of total user data and erasure coded data in a local storage pool. It also shows the amount of data pending erasure coding the current rate, and estimated completion time. Charts hold seven days worth of data. Table 17 Erasure coding metrics Column Description Storage Pool Total User Data Total Erasure Coded Data Rate of Erasure Coding Est Time to Complete The total logical size of the user data chunks in the storage pool. The total logical size of all erasure coded chunks in the storage pool. The rate at which any current data waiting for erasure coding is being processed. The estimated completion extrapolated from the current ensure coding rate. Procedure 1. Select Monitor > Erasure Coding. Monitor erasure coding 119

120 Monitor services: chunks, erasure coding, geo-replication, and recovery status 2. Locate the table row for the target storage pool. 3. Select the History button. Monitor recovery status Describes how to use the Monitor > Recovery Status page. Recovery is the process of rebuilding data after any local condition that results in bad data (chunks). This table includes one row for each storage pool in the local zone. Table 18 Recovery metrics Column Storage Pool Total Amount Data to be Recovered Recovery Rate Time to Completion Description Lists each storage pool in the local zone. Logical size of the data yet to be recovered. Rate data is being recovered in the specified storage pool in bytes per second. Estimated time to complete the recovery extrapolated from the current recovery rate. Procedure 1. Select Monitor > Recovery Status. 2. Locate the table row for the target storage pool. 3. Select the History button. 120 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

121 Monitor services: chunks, erasure coding, geo-replication, and recovery status Monitor geo-replication: rate and chunks Describes the table fields found in the ECS Portal Monitor > Geo-Replication > Rate and Chunks page. This page provides fundamental metrics about the network traffic for a replication and the chunks waiting for replication by VDC or replication group. Table 19 Rate and Chunk columns Column Remote Zone Replication Group Write Traffic Read Traffic Repo Chunks Pending Replication Total Space Journal Chunks Pending Replication Total Space Chunks Pending XOR Total Space Description Lists the remote VDCs that have replication groups that include the local VDC. The data listed is the uuid of the VDC. This column lists the replication group this zone (VDC) participates in. The current rate of writes to the VDC or replication groupin gigabytes per second. The current rate of reads to the VDC or replication group in gigabytes per second. The total logical size of all user data (repository) chunks waiting for replication. The total logical size of all system metadata chunks waiting for replication. The total logical size of all chunks waiting to be processed by the XOR compression algorithm. Monitor geo-replication: rate and chunks 121

122 Monitor services: chunks, erasure coding, geo-replication, and recovery status Figure 38 Geo replication: Rate and Chunks Monitor geo-replication: Recovery Point Objective (RPO) Describes the table fields found in the ECS Portal Monitor > Geo-Replication > RPO page. A recovery point objective (RPO) is your business rule for the limit of how long a service can be down. This display lets you see how long a recovery is taking for a replication group or zone. Table 20 RPO columns Column Remote Replication Group\Remote Zone Overall RPO (minutes) Description At the VDC level, lists all remote replication groups the local zone participates in. At the replication group level, this column lists the remote zones in the replication group. The data listed is the system identifier for the VDC or replication group as an URN. The time elapsed in the last recovery for the replication group or zone. Figure 39 RPO 122 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

123 Monitor services: chunks, erasure coding, geo-replication, and recovery status Monitor geo-replication: failover Describes the metrics found in the ECS Portal Monitor > Geo-Replication > Failover Processing page. The Failover Processing page provides metrics on the process to re-synchronize a replication group after connectivity between VDCs is restored following an outage. Table 21 Failover columns Field Replication Group Failed Zone Repo chunks pending rereplication (# / Total Space) Journal chunks pending rereplication (# / Total Space) Btree chunks pending rereplication (# / Total Space) Repo chunks pending XOR decoding (# / Total Space) Description Lists the replication groups that the local zone is a member of. The data listed is the system identifier for the replication group as an URN. Identifies one failed zone that is part of the replication group. If more than one zone has failed, there will be multiple rows for the replication group in the table. The data shown is the system identifier for the zone as an URN. Shows chunks that contain user data (repository data) that need to be replicated again because the system cannot verify that the original chunks were replicated correctly due to the failed zone. The field lists the number of these chunks waiting for re-replication as well as the total logical size of these chunks. Shows chunks that contain failure recovery data that need to be replicated again because the system cannot verify that the original chunks were replicated correctly due to the failed zone. The field lists the number of these chunks waiting for re-replication as well as the total logical size fo these chunks. Shows chunks that contain system metadata that need to be replicated again because the system cannot verify that the original chunks were replicated correctly due to the failed zone. The field lists the number of these chunks waiting for re-replication as well as the total logical size of these chunks. Shows the count and total logical size of chunks waiting to be retrieved by the XOR compression scheme. Failover State BLIND_REPLAY_DONE REPLICATION_CHECK_DONE: The process that makes sure that all replication chunks are in an acceptable state has completed successfully. CONSISTENCY_CHECK_DONE: The process that makes sure that all system metadata is fully consistent with other replicated data has completed successfully. ZONE_SYNC_DONE: The synchronization of the failed zone has completed successfully. ZONE_BOOTSTRAP_DONE: The bootstrap process on the failed zone has completed successfully. ZONE_FAILOVER_DONE: The failover process has completed successfully. Failover Progress A percentage indicator for the overall status of the failover process. Monitor geo-replication: failover 123

124 Monitor services: chunks, erasure coding, geo-replication, and recovery status Figure 40 Failover Monitor geo replication: bootstrap processing Describes the fields found in the ECS Portal Monitor > Geo Replication > Bootstrap Processing page. Bootstrapping refers to the process of copying necessary metadata to a replication group that has added a zone. Table 22 Bootstrap Processing columns Column Replication Group Added Zone Repo Chunks Pending Replication Total Space Journal Chunks Pending Replication Total Space Btree Chunks Pending Replication Total Space Description This column provides the list of replication groups the local zone participates in with new zones being added. Each row provides metrics for the specified replication group. The zone being added to the specified replication group. The logical size of all user data (repository) chunks waiting replication to the failed zone. The logical size of all failure recovery chunks waiting replication to the failed zone. The logical size of all system metadata (chunks waiting replication to the failed zone. Bootstrap State Started: The system has begun preparing to add the zone to the replication group. BlindReplayDone ReplicationCheckDone: The process that checks to make sure that all replication chunks are in an acceptable state has completed successfully. ConsistencyCheckDone: The process that makes sure that all system metadata is fully consistent with other replicated data has completed successfully. ZoneSyncDone: The synchronization of the failed zone has completed successfully. 124 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

125 Monitor services: chunks, erasure coding, geo-replication, and recovery status Table 22 Bootstrap Processing columns (continued) Column Description ZoneBootstrapDone: The bootstrap process on the failed zone has completed successfully. Done: The entire bootstrap process has completed successfully. Bootstrap Progress (%) The completion percent of the entire bootstrap process. Figure 41 Bootstrap processing Monitor geo replication: bootstrap processing 125

126 Monitor services: chunks, erasure coding, geo-replication, and recovery status 126 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

127 CHAPTER 16 Monitor events: audit portal, API, and CLI events and system alerts About event monitoring Filter events by date and sort the results by column Monitor events: audit portal, API, and CLI events and system alerts 127

128 Monitor events: audit portal, API, and CLI events and system alerts About event monitoring Describes the event monitoring functions of the ECS Portal. The Events page under the Monitor menu displays: All activity by users working with portal, the ECS REST API, or the ECS CLI. Alerts fired by the ECS system. Alerts fired by ECS services. Event data through the ECS Portal is limited to 5 days. If you need to keep event data for longer periods, consider using ViPR SRM. See the following articles for information about other ECS Portal monitoring pages: Monitor resources: hardware, network traffic, disk bandwidth, node and process health on page 96 Monitor storage: metering and capacity on page 106 Monitor services: chunks, erasure coding, geo-replication, and recovery status on page 116 Filter events by date and sort the results by column Use the filter feature to narrow events to a specific date range and sort the results. Procedure 1. Enter a beginning date in the From field and an ending date in the To field or select the calendar icons to pick the dates. Notice that the default entry in these fields is always the yesterday and today range. 2. Select Filter. 3. Select a column header to sort the data by User ID, Description, Namespace, or Timestamp in ascending order. Select the column again to change the sort to descending order. Figure 42 Events display filtered by a date rang and sorted by User ID with descending order 128 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

129 PART 6 Enable Data Access Chapter 17, " Address ECS object storage and use the Base URL " Chapter 18, " Create and configure buckets " Chapter 19, " Obtain secret key to access object storage " Chapter 20, "Configure support for CAS SDK applications with the ECS Portal" Enable Data Access 129

130 Enable Data Access 130 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

131 CHAPTER 17 Address ECS object storage and use the Base URL Introduction Bucket addressing Add a Base URL Address ECS object storage and use the Base URL 131

132 Address ECS object storage and use the Base URL Introduction Applications that are written to use Amazon S3 can be enabled to use ECS object storage by setting the Base URL parameter. The Base URL is set by default to amazonaws.com. This article describes how to set the Base URL and ensure that requests are routed to ECS. The following sections describe the addressing scheme supported by ECS, the use of the Base URL parameter, and the mechanism for setting the Base URL parameter. Bucket addressing on page 132 Add a Base URL on page 135 Bucket addressing The ECS S3 service provides a number of ways in which to identify the bucket against which the operation defined in a request should be performed. When using the Amazon S3 service, all buckets names must be unique. However, the ECS S3 service supports the use of a namespace, which can be used in addition to the bucket name and allows buckets in different namespaces to have the same name. By assigning a namespace to each tenant, a tenant can assign bucket names without regard for the names currently used by other tenants. If no namespace is specified in a request, ECS uses the default namespace associated with the tenant to which the user making the request belongs. The namespace that refers to the location of an object can be specified in the x-emcnamespace header of an HTTP request. ECS also supports extraction of the location from the host header and allows the following Amazon S3 compatible addressing schemes: Virtual Host Style Addressing on page 132 Path Based Addressing on page 132 Virtual Host Style Addressing In the virtual host addressing scheme, the bucket name appears in the hostname. For example, the bucket called "mybucket" on host ecs1.yourco.com, would be accessed using: In addition, ECS also allows the inclusion of a namespace in the address. For example: <bucketname>.<namespace>.ecs1.yourco.com To use this style of addressing, you need to configure ECS so that it knows which part of the URL is the bucket name. This is done by configuring the Base URL. In addition, you need to ensure that your DNS system can resolve the address. The following sections provide more information: DNS Configuration on page 133 Base URL on page 133 Path Based Addressing In the path based addressing scheme, the bucket name is added to the end of the path. For example: ecs1.yourco.com/mybucket 132 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation When including a namespace, use the following format:

133 Address ECS object storage and use the Base URL ecs1.yourco.com/mynamespace/mybucket DNS Configuration When accessing ECS storage using the S3 service, you will need to ensure that the URL resolves to the address of the ECS data node, or the data node load balancer. Where your application uses path-style addressing, this is simply a case of ensuring that the base name is resolvable by DNS. For example, if your application normally issues requests in the form ecs1.yourco.com/bucket, you will need to have a DNS entry that resolves ecs1.yourco.com to the IP address of your load balancer used for access to ECS nodes. If you are using the Amazon service this URI will be of the form: s3-euwest-1.amazonaws.com. Where your application is using virtual host style addressing, the URL will include the bucket name and can include a namespace. Under these circumstances, you will need to ensure that you include a DNS entry that will resolve the virtual host style address. You can do this by using a wildcard in the DNS entry. For example, if your application normally issues requests in the form bucket.s3.yourco.com, you will need to have two DNS entries. ecs1.yourco.com *.ecs1.yourco.com Or, if If you are using an application that previously connected to the Amazon S3 service, using bucket.s3.amazonaws.com, the entries would be: s3.amazonaws.com *.s3.amazonaws.com These entries allow the base name to be resolved when issuing service-level commands (for example, list buckets) and the virtual host style bucket address to be resolved. If you are creating an SSL certificate for this service, it should have the wildcard entry on the name of the certificate and the non-wildcard version as a Subject Alternate Name. Base URL If you have an S3 application that uses virtual host style addressing and you want to use it to connect to ECS, the Base URL must be set to enable ECS to know which part of the address refers to the bucket and, optionally, namespace. The Base URL can be set using the ECS Portal, or using the ECS Management REST API, and requires the ECS System Administrator role. The Base URL Management page shows the Base URLs that have been created and how ECS should use the them. DNS Configuration 133

134 Address ECS object storage and use the Base URL In order that ECS knows how to treat the bucket location prefix, the Base URL must be configured by choosing one of the following options. Use Base URL with namespace Use Base URL without namespace When processing a request, ECS will: 1. Try to extract namespace from the x-emc-namespace header. If found, skip the steps below and process the request. 2. Get the hostname of the URL from the host header and check if the last part of the address matches any of the configured Base URLs. 3. Where there is a BaseURL match, use the prefix part of the hostname (the part left when the Base URL is removed), to obtain the bucket location. Some applications, such as Amazon's S3 Browser, embed a static endpoint, such as s3.amazonaws.com in their generated requests. Many of these applications have no provision for changing endpoint with which they are communicating. The S3 Browser always tries to send its requests to s3.amazonaws.com. This token is in the URI and encoded in the authentication string even if you have the S3 Browser configured to communicate with ECS. Using BaseURL, you can use an application like the S3 Browser with ViPR Controller. Set the BaseURL to s3.amazonaws.com to make the S3 Browser work. The following examples demonstrate how ECS handles incoming HTTP requests with different structures. Example1 Host: baseball.image.emc.finance.com BaseURL: finance.com Use BaseURL with namespace enabled Namespace: Bucket Name: emc baseball.image Example 2 Host: baseball.image.emc.finance.com BaseURL: finance.com Use BaseURL without namespace enabled Namespace: Bucket Name: null (Use other methods to determine namespace) baseball.image.emc Example 3 Host: BaseURL: Namespace: Bucket Name: baseball.image.emc.finance.com not configured null (Use other methods to determine namespace.) null (Use other methods to determine the bucket name.) ViPR Controller treats this request as a path-style request. 134 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

135 Address ECS object storage and use the Base URL Add a Base URL This operation is only necessary if you use object clients that encode the location of an object, its namespace and bucket, in a URL. In that case you can specify a base URL that will be used, together with the namespace, as the path to objects in a tenant. Before you begin This operation requires the System Admin role in ECS. You must ensure that the domain specified in a request that uses a URL to specify an object location resolves to the location of the ECS data node or a load balancer that sits in front of the data nodes. Procedure 1. At the ECS Portal, select Settings > Object Base URLs. 2. Select New Base URL. The New Base URL page is displayed. Example of a Base URL 3. Enter the name of the Base URL. This will provide additional information about the base URL when looking at the base URL table. 4. Enter the Base URL. If your objects location URLs are in the form: mybucket.mynamespace.acme.com (that is, bucket.namespace.baseurl ) or (that is, bucket.baseurl), the base URL would be acme.com. You can specify which format in the Namespace selector. 5. Choose the format in which your object address is encoded in the URL: with a namespace or without a namespace. 6. Select Save. As an example of the use of the Base URL, an object named myphoto.jpg in the images bucket with a base URL of s3.amazonaws.com and a tenant namespace of "ecsdata", Add a Base URL 135

136 Address ECS object storage and use the Base URL would be addressable using the following URL: images.ecsdata.s3.amazonaws.com/myphoto.jpg Figure 43 Base URL example 136 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

137 CHAPTER 18 Create and configure buckets Introduction Bucket concepts and attributes Bucket ACLs Create a bucket using the ECS Portal Edit a bucket Set the bucket ACL permissions for a user Set the bucket ACL permissions for a pre-defined group Create bucket using the object APIs Bucket and key naming conventions Create and configure buckets 137

138 Create and configure buckets Introduction Containers are required to store object data. In S3 these containers are called buckets and this term has been adopted as a general term in ECS. In Atmos, the equivalent of a bucket is a subtenant, in Swift, the equivalent of a bucket is a container, and for CAS, a bucket is a CAS pool. In ECS, buckets are assigned a type which can be S3, Swift, Atmos, or CAS. In addition, S3, Atmos, or Swift buckets can be configured to support HDFS, and a bucket configured for HDFS access can be read and written using its object protocol and using the HDFS protocol. Buckets can be created for each object protocol using its API, usually using a client that supports the appropriate protocol. Additional support for creating S3, HDFS, and CAS buckets is provided by the ECS portal and the ECS Management API. The ability to create buckets from the portal makes it easy to create buckets for HDFS and CAS and makes it easy to take advantage of some of the more advanced bucket configuration options provided by ECS, such as quotas and retention periods. Where you want to create buckets using the object protocols, you can use special x-emc headers to control bucket configuration. This article describes how to create and edit buckets, and set ACLs for a bucket, using the ECS portal and also describes the additional x-emc headers that you can be used to control bucket configuration when using the supported object protocols. This was in 2.1. In addition, S3, Atmos, or Swift buckets can be configured to support HDFS, and a bucket configured for HDFS access can be read and written using its object protocol and using the HDFS protocol. The following use cases are described: Create a bucket using the ECS Portal on page 142 Create bucket using the object APIs on page 145 In addition, the ability to edit the configuration of a bucket and to set the ACL for a bucket are described: Edit a bucket on page 143 Set the bucket ACL permissions for a user on page 144 Bucket concepts and attributes 138 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation Buckets are object containers and can be used to control access to objects and to set properties that define attributes for all contained objects, such as retention periods and quotas. As namespaces are also global resources, an object can be addressed using its bucket and namespace from any linked VDC. Bucket access Buckets are associated with a replication group. Where the replication group spans multiple VDCs, the bucket contents are similarly replicated across the VDCs. Objects in a bucket that belongs to a replication group which spans two VDCs, VDC1 and VDC2, for example, can be accessed from either VDC1 or VDC2. Objects in a bucket that belongs to a replication group that is only associated with VDC1, can only be accessed from VDC1, they cannot be accessed from other VDCs in a federated ECS system. The identity of a bucket and its metadata, such as its ACL, are global management information in ECS, which means that they are replicated across the system storage pools

139 Create and configure buckets and can be seen from all VDCs in the federation. However, the bucket can only be listed from a VDC that is part of the replication group to which the bucket belongs. In addition, buckets in ECS always belong to a namespace. Bucket ownership A bucket belongs to a namespace and object users are also assigned to a namespace. Each object user can create buckets only in the namespace to which they belong, however, any ECS object user can be assigned as the owner of a bucket or object, or a grantee in a bucket ACL, even if the user does not belong to the same namespace as the bucket or object. This enables buckets and objects to be shared between users in different namespaces. For example, in an enterprise where a namespace is a department, a bucket or object can be shared between users in different departments. When an object user wants to access a bucket in a namespace that they don't belong to, the namespace must be specified using the x-emc-namespace header. Access to a bucket during temporary site outage ECS provides a temporary site outage mechanism that enables objects to be retrieved even if the primary copy of the object is not available due to the site that hosts the primary being unavailable. Because there is a risk that object data retrieved during a temporary site outage is not the most recent, the user must indicate that they are prepared to accept this by marking the bucket as available during an outage. Bucket attributes The ECS Portal enables buckets to be created and managed at the Manage > Buckets page. The Bucket Management page provides a bucket table which displays the buckets for a selected namespace. The table displays bucket attributes and provides Edit Bucket, Edit ACL, and Delete actions for each bucket. The attributes associated with a bucket are described in the following table. To view and change attributes that are not displayed on the Bucket Management page, you can select Edit Bucket. Table 23 Bucket attributes Attribute Name Created Description Name of the bucket. You can refer to the following topic for guidance on bucket naming: Bucket and key naming conventions on page 148. Creation time of the bucket. Bucket concepts and attributes 139

140 Create and configure buckets Table 23 Bucket attributes (continued) Attribute Replication Group Namespace Owner Notification Quota Max Quota File System Enabled CAS Access During Outage Bucket Retention Description Replication group in which the bucket will be created. Namespace with which the bucket is associated. Bucket owner. Quota setting at which you will be notified. This is a soft quota and can be set on its own or can be set in addition to a hard quota. The quota cannot be set less than 1GB. The quota associated with a bucket can be changed during its lifetime. More information on quotas is provided in: Manage a tenant on page 84. Hard quota which, when reached, will cause access to the bucket to be prevented. A soft can be set to trigger before the hard quota is reached. Indicates that ECS will allow the bucket to be used as a Hadoop Distributed File System (HDFS). This must be set at the time of bucket creation. It cannot be changed after bucket creation. A File System Enabled bucket can't be set for access during an outage. Indicates that the bucket is enabled for CAS data. A flag set on the bucket which specifies the behavior when accessing data in the bucket when there is a temporarily unavailable zone in a geofederated setup. If you set this flag ON, and a temporary site outage occurs, objects that you access in this bucket might have been updated at the failed site but changes might not have been propagated to the site from which you are accessing the object. Hence, you are prepared to accept that the objects you read might not be up to date. If the flag is turned OFF, data in the zone which has the temporary outage is not available for access from other zones and object reads for data which has its primary in the failed site will fail. This flag can only be turned on if File System Enabled flag is not turned on. Sets the retention period for a bucket. Information on retention period is provided in: Manage a tenant on page 84. The expiration of a retention period on an object within a bucket is calculated when a request to modify an object is made and is based on the value set on the bucket and the objects themselves. The retention period can be changed during the lifetime of the bucket. 140 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

141 Create and configure buckets Bucket ACLs The privileges a user has when accessing a bucket are set using an ACLs. When you create a bucket and assign an owner to it, an ACL is created that assigns a default set of permissions to the bucket owner - the owner is, by default, assigned full control. You can modify the permissions assigned to the owner or you can add new permissions for a user by selecting the Edit ACL operation for the bucket. At the ECS Portal, the Bucket ACLs Management page provides User ACLs and Group ACLs panels to manage the ACLs associated with individual users and with pre-defined groups. User ACLs The User ACL panel show the ACLs that have been applied to users and enables ACLs to be assigned to a user using the Add operation. The ACL meanings are provided in the following table. Table 24 Bucket ACLs ACL Read Read ACL Write Write ACL Execute Full Control Privileged Write Delete None Permission Allows user to list the objects in the bucket. Allows user to read the bucket ACL. Allows user to create or update any object in the bucket. Allows user to write the ACL for the bucket. Sets the execute permission when accessed as a file system. This permission has no effect when the object is accessed using the ECS object protocols. Allows user to Read, Write, Read ACL, and Write ACL. Allows user to perform writes to a bucket or object when the user doesn't have normal write permission. Required for CAS buckets. Allows user to delete buckets and objects. Required for CAS buckets. User has no privileges on the bucket. Bucket ACLs 141

142 Create and configure buckets Note Because the ECS Portal supports S3, HDFS, and CAS buckets, the range of permissions that can be set are not applicable to all bucket types. Group ACLs You can set permissions for a set of pre-defined groups. The following groups are supported: public All users authenticated or not. all users All authenticated users. other Authenticated users but not the bucket owner. log delivery Not supported. The permissions that can be assigned are listed in Table 24 on page 141. Create a bucket using the ECS Portal The ECS portal enables the creation of buckets and provides the ability to specify the configuration of the bucket. Buckets created at the portal can be either S3, S3+HDFS, or CAS buckets. Before you begin You must be a Namespace Admin or a System Admin to create a bucket at the ECS portal. If you are a Namespace Admin you can create buckets in your namespace. If you are System Admin you can create a bucket belonging to any namespace. Procedure 1. At the ECS portal, select Manage > Buckets. 2. Select New Bucket. 3. Select the replication group that the bucket associated with. 4. Select the namespace that the bucket and its objects will belong to. If you are a System Admin and the ECS system has more than one namespace, select the namespace to which the bucket will belong. If you are a Namespace Admin, you will only be able to select your own namespace. 5. Specify a bucket owner. The bucket owner should be an ECS object user for the namespace. If you don't specify a user, you will be assigned as the owner, however, you will not be able to access the bucket unless your username is also assigned as an object user. The user that you specify will be given Full Control. 6. If you want the bucket to be a CAS bucket, set the CAS control to ON. By default, the bucket will be marked as an S3 bucket. 142 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

143 Create and configure buckets 7. If you want the bucket to support HDFS operation, set the File System Enabled control to ON. The bucket will be an S3 bucket that supports HDFS. 8. If the bucket is a CAS bucket or an S3 bucket on which you have not set File System Enabled, you can set Access During outage to ON. 9. If required, specify a quota for the bucket The settings that you can apply are described in: Bucket concepts and attributes on page If required, set a bucket retention period for the bucket. You can read more about retention periods in: Bucket concepts and attributes on page Select Save to create the bucket. Results You can assign users to the bucket and set permissions for users (or pre-defined groups) from the buckets table Actions menu. Edit a bucket You can edit some bucket settings after the bucket has been created and after it has had objects written to it. Before you begin You must be a Namespace Admin or a System Admin to edit a bucket. If you are a Namespace Admin you can edit the setting for buckets belonging to your namespace. If you are System Admin you can edit the settings for a bucket belonging to any namespace. Procedure 1. At the ECS portal, select Manage > Buckets. 2. In the Buckets table, select the Edit action for the bucket for which you want to change the settings. 3. You can edit the following bucket attributes: Quota Bucket Owner Access During Outage Note Not available if File System Enabled was set on the bucket. You cannot change the following attributes of the bucket: Replication Group File Access Enabled CAS enabled Edit a bucket 143

144 Create and configure buckets You can find out more information about these settings in: Bucket concepts and attributes on page Select Save. Set the bucket ACL permissions for a user The ECS portal enables the ACL for a bucket to be set for a user or for a pre-defined group. Before you begin You must be a Namespace Admin or a System Admin to edit the ACL for a bucket. If you are a Namespace Admin you can edit the ACL settings for buckets belonging to your namespace. If you are System Admin you can edit the ACL settings for a bucket belonging to any namespace. Procedure 1. At the ECS portal, select Manage > Buckets. 2. In the Buckets table, select the Edit ACL action for the bucket for which you want to change the settings. 3. To set the ACL permissions for a user, select the User ACLs button. To select the ACL for a group, select Group ACLs. You can refer to Set the bucket ACL permissions for a pre-defined group on page 144 for more information on setting group ACLs. 4. You can edit the permissions for a user that already has permissions assigned, or you can add a user that you want to assign permissions for. To set (or remove) the ACL permissions for a user that already has permissions, select Edit (or Remove) from the Action column in the ACL table. To add a user to which you want to assign permissions, select Add. The user that you have set as the bucket owner will have already have default permissions assigned. 5. If you have added an ACL, enter the username of the user that the permissions will apply to. 6. Specify the permissions that you want to apply to the user. More information on ACL privileges is provided in Bucket concepts and attributes on page Select Save. Set the bucket ACL permissions for a pre-defined group The ECS portal enables the ACL for a bucket to be set for a user or for a pre-defined group. Before you begin You must be a Namespace Admin or a System Admin to edit the group ACL for a bucket. If you are a Namespace Admin you can edit the group ACL settings for buckets belonging to your namespace. 144 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

145 Create and configure buckets If you are System Admin you can edit the group ACL settings for a bucket belonging to any namespace. Procedure 1. At the ECS portal, select Manage > Buckets. 2. In the Buckets table, select the Edit ACL action for the bucket for which you want to change the settings. 3. To set the ACL permissions for a group, select the Group ACLs button. To select the ACL for a user, select User ACLs. You can refer to Set the bucket ACL permissions for a user on page 144 for more information on setting user ACLs. 4. Select the group that you want to assign ACL privileges for. You can read more about the pre-defined privileges in: Bucket concepts and attributes on page Select the privileges that you want to assign to the group. 6. Select Save. Create bucket using the object APIs When creating buckets using the object APIs or using tools that call the object APIs, there are a number of headers that determine the behavior. The following x-emc headers are provided: x-emc-dataservice-vpool Determines the replication group that will be used to store the objects associated with this bucket. If you do not specify a replication group using the x-emcdataservice-vpool header, ECS will choose the default replication group associated with the namespace. x-emc-file-system-access-enabled Configures the bucket for HDFS access. The header must not conflict with the interface that is being used. That is, a create bucket request from HDFS cannot specify x-emc-file-system-access-enabled=false. x-emc-namespace Specifies the namespace to be used for this bucket. If the namespace is not specified using the S3 convention of host/path style request, then it can be specified using the x-emc-namespace header. If the namespace is not specified as this header, the namespace associated with the user is used. x-emc-retention-period Specifies the retention period that will be applied to objects in a bucket. Each time a request is made to modify an object in a bucket, the expiration of the retention period for the object is calculated based on the retention period associated with the bucket. x-emc-is-stale-enabled Flag that indicates whether the bucket can be accesses during a temporary VDC outage in a federated configuration. x-emc-project-id Specifies the project ID to associate with the new bucket. This is relevant when reporting metering data for object usage. If this header is not present, the default project id defined for the associated tenant is used. Create bucket using the object APIs 145

146 Create and configure buckets An example of using the S3curl tool to create a bucket is provided: Create a bucket using the S3 API (with s3curl) on page 146 Create a bucket using the S3 API (with s3curl) You can use the S3 API to create a bucket in an replication group. Because ECS uses custom headers (x-emc), the string to sign must be constructed to include these headers. In this procedure the s3curl tool is used; there are also a number of programmatic clients you can use, for example, the S3 Java client. Before you begin ECS must have at least one replication group configured. Perl must be installed on the Linux machine on which you will run s3curl. You will need to have curl installed and you will need the s3curl module, which acts as a wrapper around curl. To use s3curl with x-emc headers, minor modifications must be made to the s3curl script. These modifications are described in the procedure. Procedure 1. Obtain a secret key for the user who will create the bucket. Refer to the article: Obtain secret key to access object storage on page 152 for details. 2. Obtain the identity of the replication group in which you want the bucket to be created. You can obtain the replication group identity by using the ECS REST API: GET IP Address>:4443/vdc/data-service/vpools The response provides the name and identity of all data services virtual pools. For example: <data_service_vpools> <data_service_vpool> <creation_time> </creation_time> <id>urn:storageos:replicationgroupinfo:8fc8e19b-edf0-4e81- bee8-79accc867f64:global</id> <inactive>false</inactive> <tags/> <description>isilonvpool1</description> <name>isilonvpool1</name> <varraymappings> <name>urn:storageos:virtualdatacenter:1de0bbc2-907c-4edeb133-f5331e03e6fa:vdc1</name> <value>urn:storageos:virtualarray:793757ab-ad b80a-682e124eb25e:vdc1</value> </varraymappings> </data_service_vpool> </data_service_vpools> Here the ID is urn:storageos:replicationgroupinfo:8fc8e19bedf0-4e81-bee8-79accc867f64:global. 3. Set up s3curl by creating a.s3curl file in which to enter the user credentials. The.s3curl file must have permissions 0600 (rw-/---/---) when s3curl.pl is run. 146 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

147 Create and configure buckets In the example below, the profile "my_profile" is used to reference the user credentials for the account, and "root_profile" references the credentials for the root account. %awssecretaccesskeys = ( my_profile => { id => 'user@yourco.com', key => 'szrctzyk93iwukhegq3evpjevpuq4asl8nre0awn' }, root_profile => { id => 'root', key => 'szrctzyk93iwukhegq3evpjevpuq4asl8nre0awn' }, ); 4. Add the endpoint that you want to use s3curl against to the.s3curl file. This will be the address of your data node or the load balancer that sits in front of your data node. For example: ( ' ', 'lglw3183.lss.emc.com', ); 5. Modify the s3curl.pl script so that it includes the x-emc headers in its "string to sign". Replace the following lines: elsif ($header =~ /^(?'header'[xx]-(([aa][mm][zz]) ([Ee][Mm][Cc]))- [^:]+): *(?'val'.+)$/) { my $name = lc $+{header}; my $value = $+{val}; with: elsif ($header =~ /^([Xx]-(?:(?:[Aa][Mm][Zz]) (?:[Ee][Mm][Cc]))- [^:]+): *(.+)$/) { my $name = lc $1; my $value = $2; 6. Create the bucket using s3curl.pl. Specify the following: Profile of the user. Identity of the replication group in which to create the bucket (<vpool_id>). This must be set using the x-emc-dataservice-vpool header. x-emc-file-system-access-enabled header to enable file system access. Name of the bucket (<BucketName>). Create a bucket using the S3 API (with s3curl) 147

148 Create and configure buckets The fully specified command looks like this:./s3curl.pl --debug --id=my_profile --acl public-read-write --createbucket -- -H 'x-emc-file-system-access-enabled:true' -H 'x-emc-dataservice-vpool:<vpool_id>' <BucketName> Note that the -acl public-read-write argument is optional, but is needed if you plan to access the bucket in an anonymous environment. For example, if you intend to access to bucket as HDFS from an environment that is not secured using Kerberos. If successful (with --debug on) you should see output similar to the following: s3curl: Found the url: host= ; port=9020; uri=/s3b4; query=; s3curl: ordinary endpoint signing case s3curl: StringToSign='PUT\n\n\nThu, 12 Dec :58: \nxamz-acl:public-read-write \nx-emc-file-system-access-enabled:true\nx-emc-dataservice-vpool: urn:storageos:replicationgroupinfo:8fc8e19b-edf0-4e81- bee8-79accc867f64:global:\n/s3b4' s3curl: exec curl -H Date: Thu, 12 Dec :58: H Authorization: AWS root:aitcfmdhsi6isq2ribhezon0wno= -H x-amz-acl: public-read-write - L -H content-type: --data-binary -X PUT -H x-emc-file-system-access-enabled:true -H x-emc-dataservice- vpool:urn:storageos:objectstore:e0506a04-340b-4e78- a694-4c389ce14dc8: You can list the buckets using the S3 interface, using:./s3curl.pl --debug --id=my_profile Bucket and key naming conventions Bucket and object/key names must conform to the specification presented here. S3 bucket and object naming in ECS on page 149 OpenStack Swift container and object naming in ECS on page 149 Atmos bucket and object naming in ECS on page 150 CAS pool and object naming in ECS on page 150 Note If you want to use a bucket for HDFS, you should not use underscores in the bucket name as they are not supported by the URI Java class. For example, viprfs:// my_bucket.ns.site/ will not work as this is an invalid URI and is thus not understood by Hadoop. Namespace name The following rules apply to the naming of ECS namespaces: 148 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

149 Create and configure buckets Cannot be null or an empty string Length range is (Unicode char) Valid characters are defined by regex /[a-za-z0-9-_]+/. Hence: Alphanumeric characters S3 bucket and object naming in ECS Special characters: hyphen (-) and underscore (_). This topic details the rules that apply to the naming of buckets and objects when using the ECS S3 Object API. Bucket name The following rules apply to the naming of S3 buckets in ECS: Names must be between one and 255 characters in length. (S3 requires bucket names to be from 1 to 255 characters long). Names can include dot (.), hyphen (-), and underscore (_) characters and alphanumeric characters ([a-za-z0-9]). Names can start with a hyphen (-) or alphanumeric character. The name does not support: Starting with a dot (.) Containing a double dot (..) Ending with a dot (.) Name must not be formatted as IPv4 address. You can compare this with naming restriction specified by the S3 specification: docs.aws.amazon.com/amazons3/latest/dev/bucketrestrictions.html. Object Name The following rules apply to the naming of ECS S3 objects: Cannot be null or an empty string Length range is (Unicode char) No validation on characters. OpenStack Swift container and object naming in ECS This topic details the rules that apply to the naming of buckets and objects when using the ECS OpenStack Swift Object API. Container Name The following rules apply to the naming of Swift containers: Cannot be null or an empty string Length range is (Unicode char) Valid characters are defined by regex /[a-za-z0-9\\.\\-_]+/ Alphanumeric characters Special characters: dot (.), hyphen (-), and underscore (_). Object Name The following rules apply to the naming of Swift objects: S3 bucket and object naming in ECS 149

150 Create and configure buckets Cannot be null or an empty string Length range is (Unicode char) No validation on characters. Atmos bucket and object naming in ECS This topic details the rules that apply to the naming of buckets and objects when using the ECS Atmos Object API. Subtenant (bucket) This is created by the server, so the client does not need to know the naming scheme. Object name The following rules apply to the naming of Atmos objects: Cannot be null or an empty string Length range is (Unicode char) No validation on characters. CAS pool and object naming in ECS Name should be percent-encoded UTF-8. This topic details the rules that apply to the naming of CAS pools and objects ('clips' in CAS terminology) when using the CAS API. CAS pool naming The following rules apply to the naming of CAS pools in ECS: a maximum of 255 characters cannot contain: ' " / &? * < > <tab> <newline> or <space> Clip naming There are no user defined keys in the CAS API. When an application using CAS API creates a clip, it opens a pool, creates a new clip, and adds tags, attributes, streams etc. After a clip is complete it is written to a device. A corresponding clip ID is returned by CAS engine and can be referred to using <pool name>/<clip id>. 150 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

151 CHAPTER 19 Obtain secret key to access object storage Introduction Create a key for an object user Create an S3 secret key: self-service Obtain secret key to access object storage 151

152 Obtain secret key to access object storage Introduction Users of the ECS object services require a secret key in order to authenticate with a service. Secret keys can be created and made available to the object user in the following ways: Administrator creates a a keys and distributes to the object user (Create a key for an object user on page 152). Object user who is a domain user creates a new key using the self-service API provided by the ECS Management REST API (Create an S3 secret key: self-service on page 153). Create a key for an object user Generate a secret key from the ECS Portal ECS Management users can create a secret key for an object user. Generate a secret key from the ECS Portal on page 152 Create an S3 secret key using the ECS Management REST API on page 152 You can generate a secret key at the ECS Portal. Before you begin You must be an ECS System Admin or Namespace Admin If you are a System Admin, you can create a secret key for an object user belonging to any namespace. If you are a Namespace Admin, you can create a secret key for an object users who belongs to your namespace. Procedure 1. At the ECS Portal, select the Manage > Users page. 2. In the Object Users table, select Edit for the user to which you want to assign a secret key. 3. For S3, select Generate & Add Password. 4. Copy the generated key and to the object user. Create an S3 secret key using the ECS Management REST API The ECS Management REST API enables a management user to create a secret key for an S3 object user. The APIs is as follows: API Path /object/usersecret-keys/ {uid} Description API to allow secret keys to be assigned to object users and enable secret keys to be managed. Namespace Admin can create keys for users in their namespace. System Admin can assign keys to users in any namespace. 152 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

153 Obtain secret key to access object storage You can find out more information about the API call in the ECS Management REST API reference. Create an S3 secret key: self-service The ECS Management REST API provides the ability to allow authenticated domain users to request a secret key to enable them to access the object store. The ECS Management REST API reference can be used where you want to create a custom client to perform certain ECS management operations. For simple operations domain users can use curl or a browser-based HTTP client to execute the API to create a secret key. When a user runs the object/secret-keys API, ECS automatically creates an object user and assigns a secret key. API Path /object/ secret-keys Description API to allow S3 client users to create a new secret key that enables them to access objects and buckets within their namespace. This is also referred to as a self-service API. The payload for the /object/secret-keys can include an optional existing key expiry time. <secret_key_create_param> <existing_key_expiry_time_mins></existing_key_expiry_time_mins> </secret_key_create_param> If you are creating a secret key for the first time, you can omit the existing_key_expiry_time_mins parameter and a call would be: POST object/secret-keys Request body <?xml version="1.0" encoding="utf-8"?> <secret_key_create_param/> Working with self-service keys Response <user_secret_key> <secret_key>...</secret_key> <key_timestamp>...</key_timestamp> <link rel="..." href="..." /> </user_secret_key> There are a number of operations that you might want to perform with self-service secret keys using the ECS Management REST API Reference. The examples provided use the curl tool to demonstrate the following activities. Log in as a domain user on page 154 Generate first key on page 154 Generate second key on page 154 Check keys on page 155 Delete all secret keys on page 155 Create an S3 secret key: self-service 153

154 Obtain secret key to access object storage Log in as a domain user You can log in as a domain user and obtain an authentication token that can be used to authenticate subsequent requests. curl -ik -u user@mydomain.com:<password> login HTTP/ OK Date: Mon, 27 Apr :29:38 GMT Content-Type: application/xml Content-Length: 107 Connection: keep-alive X-SDS-AUTH-TOKEN: BAAcaVAzNU16eVcwM09rOWd2Y1ZoUFZ4QmRTK2JVPQMAQQIADTE0MzAwNzQ4ODA1NTQDAC 51cm46VG9rZW46YWJmODA1NTEtYmFkNC00ZDA2LWFmMmMtMTQ1YzRjOTdlNGQ0AgAC0A8= <?xml version="1.0" encoding="utf-8" standalone="yes"?> <loggedin> <user>tcas@corp.sean.com</user> </loggedin> Check user details You can check your user details.check the details for the user. In this case the user belongs to namespace "ns1" and is a NAMESPACE_ADMIN. curl -ks -H "X-SDS-AUTH-TOKEN: BAAcaVAzNU16eVcwM09rOWd2Y1ZoUFZ4QmRTK2JVPQMAQQI ADTE0MzAwNzQ4ODA1NTQDAC51cm46VG9rZW46YWJmODA1NTEtYmFkNC00ZDA2LWFmMmMtM TQ1YzRjOTdlNGQ0AgAC0A8=" xmllint --format - <?xml version="1.0" encoding="utf-8" standalone="yes"?> <user> <common_name>user@mydomain.com</common_name> <distinguished_name/> <namespace>ns1</namespace> <roles> <role>namespace_admin</role> </roles> </user> Generate first key You can generate a secret key. curl -ks -H "X-SDS-AUTH-TOKEN: BAAcaVAzNU16eVcwM09rOWd2Y1ZoUFZ4QmRTK2JVPQMAQQIADTE0MzAw NzQ4ODA1NTQDAC51cm46VG9rZW46YWJmODA1NTEtYmFkNC00ZDA2LWFmMmMtMTQ1YzRjOT dlngq0agac0a8=" -H "Content-Type: application/json" -X POST -d "{}" xmllint --format - <?xml version="1.0" encoding="utf-8" standalone="yes"?> <user_secret_key> <link rel="self" href="/object/user-secret-keys/ tcas@corp.sean.com"/> <secret_key>7hxz9/ehtvvmfuyly/z3ghpihxteux/vzxdxddbd</secret_key> <key_expiry_timestamp/> <key_timestamp> :39:13.813</key_timestamp> </user_secret_key> Generate second key You can generate a second secret key and set the expiration for the first key. curl -ks -H "X-SDS-AUTH-TOKEN: BAAcaVAzNU16eVcwM09rOWd2Y1ZoUFZ4QmRTK2JVPQMAQQIADTE0MzAwN zq4oda1ntqdac51cm46vg9rzw46ywjmoda1ntetymfknc00zda2lwfmmmmtmtq1yzrjotd 154 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

155 Obtain secret key to access object storage lngq0agac0a8=" -H "Content-Type: application/json" -X POST -d "{\"existing_key_expiry_time_mins\": \"10\"}" xmllint --format - <?xml version="1.0" encoding="utf-8" standalone="yes"?> <user_secret_key> <link rel="self" href="/object/user-secret-keys/ tcas@corp.sean.com"/> <secret_key>l3fpcufcg/bxooxcpzoyupwhxrstwu0f1kfdarur</secret_key> <key_expiry_timestamp/> <key_timestamp> :40:12.506</key_timestamp> </user_secret_key> Check keys You can check the keys that you have been assigned. In this case there are two keys with the first having an expiration date/time. curl -ks -H "X-SDS-AUTH-TOKEN: BAAcaVAzNU16eVcwM09rOWd2Y1ZoUFZ4QmRTK2JVPQMAQQIADTE0MzAw NzQ4ODA1NTQDAC51cm46VG9rZW46YWJmODA1NTEtYmFkNC00ZDA2LWFmMmMtMTQ1YzRjOT dlngq0agac0a8=" xmllint --format - <?xml version="1.0" encoding="utf-8" standalone="yes"?> <user_secret_keys> <secret_key_1>7hxz9/ehtvvmfuyly/z3ghpihxteux/vzxdxddbd</ secret_key_1> <secret_key_2>l3fpcufcg/bxooxcpzoyupwhxrstwu0f1kfdarur</ secret_key_2> <key_expiry_timestamp_1> :50:12.369</ key_expiry_timestamp_1> <key_expiry_timestamp_2/> <key_timestamp_1> :39:13.813</key_timestamp_1> <key_timestamp_2> :40:12.506</key_timestamp_2> <link rel="self" href="/object/secret-keys"/> </user_secret_keys> Delete all secret keys If you need to delete your secret keys before regenerating them. You can use the following. curl -ks -H "X-SDS-AUTH-TOKEN: BAAcaVAzNU16eVcwM09rOWd2Y1ZoUFZ4QmRTK2JVPQMAQQIADTE0MzAw NzQ4ODA1NTQDAC51cm46VG9rZW46YWJmODA1NTEtYmFkNC00ZDA2LWFmMmMtMTQ1YzRjOT dlngq0agac0a8=" -H "Content-Type: application/json" -X POST -d "{}" :4443/object/secret-keys/deactivate Working with self-service keys 155

156 Obtain secret key to access object storage 156 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

157 CHAPTER 20 Configure support for CAS SDK applications with the ECS Portal Setting up CAS support in ECS CAS retention in ECS Set up namespace retention policies Set up a bucket for CAS Set up a CAS object user Set up bucket ACLs for a user ECS REST APIs that support CAS users Configure support for CAS SDK applications with the ECS Portal 157

158 Configure support for CAS SDK applications with the ECS Portal Setting up CAS support in ECS Introduces CAS (content addressable storage) support in ECS. ECS CAS allows CAS SDK-based client applications to store, retrieve, and delete fixed content objects from ECS storage. The underlying ECS storage must be provisioned before you can configure your ECS set up. Provisioning is usually completed when a new ECS rack is installed. This includes setting up a storage pool, VDC, and replication group. In ECS CAS, there are no CASspecific features in these objects, so you can use the standard documentation if you need to create or edit these objects to support CAS. See Configure storage pools, VDCs, and replication groups on page 48. Next, set up your namespaces, users, and buckets, using the standard documentation: Configure a namepace for a tenant on page 76 Add users and assign roles on page 56 Create and configure buckets on page 138 This document describes how to modify your basic configuration to support CAS: CAS retention in ECS on page 158 Set up namespace retention policies on page 160 Set up a bucket for CAS on page 161 Set up a CAS object user on page 161 Set up bucket ACLs for a user on page 162 ECS REST APIs that support CAS users on page 165 CAS retention in ECS A CAS C-Clip can have a retention period that governs the length of time the associated object is retained in ECS storage before an application can delete it. Retention periods Retention periods are assigned in the C-Clip for the object by the CAS application. For example, if a financial document must be retained for three years from its creation date, then a three-year retention period is specified in the C-Clip associated with the financial document. It is also possible to specify that the document is retained indefinitely. Retention policies (retention classes) Retention policies enable retention use cases to be captured and applied to C-Clips. For example, different types of documents could have different retention periods. You could require the following retention periods: Financial: 3 years Legal: 5 years 6 months When a retention policy is applied to a number of C-Clips, by changing the policy, the retention period changes for all objects to which the policy applies. 158 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

159 Configure support for CAS SDK applications with the ECS Portal Retention policies are associated with namespaces in ECS and are recognized by the CAS application as retention classes. Bucket-level retention not recommended for CAS ECS provides bucket-level retention for all object services. Since bucket-level retention overrides all CAS object-level retention, it will likely interfere with the external CAS application. CAS precedence When multiple retention periods are applied to a CAS object in ECS, the retention period with the higher value has precedence no matter how the retention was applied. How to apply CAS retention You can define retention polices (retention classes) to namespaces in the ECS Portal or with the ECS REST API. See Set up namespace retention policies on page 160. Your external CAS application can assign a fixed retention period or a retention class to the C-Clip during its creation. When applying retention periods through APIs, specify the period in seconds. Note that ECS CAS takes the creation time of the C-Clip for all retention related calculations and not the migration time. How to create retention policies with the ECS REST API. You can create retention periods and policies using the ECS Management REST API, a summary of which is provided below. Table 25 ECS REST API resources for retention Method PUT /object/bucket/{bucketname}/retention GET /object/bucket/{bucketname}/retention POST /object/namespaces/namespace/ {namespace}/retention PUT /object/namespaces/namespace/ {namespace}/retention/{class} GET /object/namespaces/namespace/ {namespace}/retention Description The retention value for a bucket defines a default retention period which is applied to every object within a bucket. So, if you set a retention period of 1 year, an object from the bucket cannot be deleted for one year. Returns the retention period that is currently set for a specified bucket. For namespaces, the retention setting acts like a policy, where each policy is a <Name>: <Retention period> pair. You can define a number of retention policies for a namespace and you can assign a policy, by name, to an object within the namespace. This allows you to change the retention period of a set of objects that have the same policy assigned by changing the corresponding policy. Updates the period for a retention class that is associated with a namespace. Returns the retention classes defined for a namespace. You can find more information about the ECS Management REST API in: Use the ECS Management REST API on page 250 and the online reference is here. CAS retention in ECS 159

160 Configure support for CAS SDK applications with the ECS Portal Set up namespace retention policies Provides CAS-specific set up instructions for namespace retention policies. The Retention Policy feature for namespace provides a way to define and manage CAS retention classes for all C-Clips created in the namespace. A namespace can have many retention polices, where each policy defines a retention period. By applying a retention policy to a number of C-Clips (with the API), a change to the retention policy changes the retention period for all objects associated with the policy. For CAS, retention classes are applied to an object's C-Clip by the application. If an object is under a retention period, requests to modify the object are not allowed. Procedure 1. At the ECS portal, select Manage > Namespace. 2. To edit the configuration of an existing namespace, choose the Edit action associated with the existing namespace. 3. Add and Configure Retention Policies. a. In the Retention Policies area, select Add to add a new policy. b. Enter a name for the policy. c. Specify the period for the Retention Policy. Select the Infinite checkbox to ensure that objects with this policy are never deleted. Figure 44 New Retention Policy 4. Select Save. 160 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

161 Configure support for CAS SDK applications with the ECS Portal Figure 45 Retention policies for a namespace Set up a bucket for CAS Configure a bucket to support CAS. Note Bucket-level retention policies are not recommended for CAS. Procedure 1. At the ECS portal, select Manage > Bucket. 2. To edit the configuration of an existing bucket, choose the Edit Bucket action associated with the existing bucket. 3. Select the CAS On control to enable CAS for the bucket. 4. Select Save. Set up a CAS object user Set up an object user to use CAS. When you set up an object user, you can assign CAS features to the profile that make up the elements of a CAS profile. You will be able to view the resulting PEA file for use in your CAS applications. Procedure 1. At the ECS portal, select Manage > Users. 2. To edit the configuration of an existing object user, choose the Edit action associated with the user. Set up a bucket for CAS 161

162 Configure support for CAS SDK applications with the ECS Portal Figure 46 CAS settings for object users 3. In the CAS area, type a password (secret) or choose Generate to have the portal create one for you. 4. Choose Set Password. 5. Choose Generate PEA File to generate the PEA file your application will need to authenticate to the CAS storage on ECS. 6. By setting a default bucket, every action the user takes that does not specify a bucket will use the specified default bucket. Type the name of the default bucket and chooseset Bucket. 7. Choose Add Attribute to add a metadata tag to the user. 8. Add the metadata tag name and value. See the CAS SDK documentation for more info on metadata tags. 9. Choose Save Metadata. Set up bucket ACLs for a user Edit a bucket's access control list to limit a user's access. Edit bucket ACLs for CAS users. Some ECS bucket ACLs map to CAS permissions and some have no meaning for CAS data. Procedure 1. At the ECS portal, select Manage > Bucket. 2. To edit the ACLs of an existing bucket, choose the Edit ACL action associated with the existing bucket. 162 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

163 Configure support for CAS SDK applications with the ECS Portal Figure 47 Edit bucket ACL 3. Choose the Edit associated with the user. Figure 48 Bucket ACLs Management 4. Modify the permissions. Set up bucket ACLs for a user 163

164 Configure support for CAS SDK applications with the ECS Portal Table 26 Bucket ACLs CAS SDK capability ECS ACL ACL definition Read read Allows user to open C- Clips in the bucket. Write write Allows user to create C-Clips in the bucket. Exist read Allows user to open C- Clips in the bucket. Delete delete Allows user to delete C-Clips from the bucket. Privileged Delete privileged write Allows user to perform a privileged delete of a C-Clip. Privileged delete allows the user to delete a C-Clip that is under retention. Clip-Enumeration Retention Period Not currently supported in ECS Not currently supported in ECS Note Only Centera * supports LitHold (litigation hold). Note Other ECS ACLs have no meaning to CAS. 5. Select Save. 6. You can also edit the ACLs at the group level. Groups are predefined and membership in the group is automatic based on user criteria. Choose Group ACLs. 7. Choose Add. 8. Select the group you want to edit from the Group Name list. 164 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

165 Configure support for CAS SDK applications with the ECS Portal Table 27 Bucket ACL groups Bucket ACL group Description public all users other log delivery All users authenticated or not. All authenticated users. Authenticated users but not the bucket owner. Not supported. 9. Edit the ACLs and select Save. ECS REST APIs that support CAS users Describes ECS REST API resources that you can use to manage CAS user and profile settings. The ECS Management REST API contains resources for managing CAS-specific data for a user. REST API resource descriptions: GET /object/user-cas/secret/{uid} : Gets the CAS secret for the specified user. GET /object/user-cas/secret/{namespace}/{uid}: Gets the CAS secret for the specified namespace and user. POST /object/user-cas/secret/{uid}: Creates or updates the CAS secret for a specified user. GET /object/user-cas/secret/{namespace}/{uid}/pea: Generates a PEA file for the specified user. POST /object/user-cas/secret/{uid}/deactivate: Deletes the CAS secret for a specified user. GET /object/user-cas/bucket/{namespace}/{uid}: Gets the default bucket for the specified namespace and user. GET /object/user-cas/bucket/{uid}: Gets the default bucket for a specified user. POST /object/user-cas/bucket/{namespace}/{uid}: Updates the default bucket for the specified namespace and user. GET /object/user-cas/applications/{namespace}: Gets the CAS registered applications for a specified namespace. POST /object/user-cas/metadata/{namespace}/{uid}: Updates the CAS registered applications for a specified namespace and user. GET /object/user-cas/metadata/{namespace}/{uid}: Gets the CAS user metadata for the specified namespace and user. See the ECS Management REST API Reference for more information. ECS REST APIs that support CAS users 165

166 Configure support for CAS SDK applications with the ECS Portal 166 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

167 PART 7 Configure ECS HDFS Chapter 21, "HDFS Overview" Chapter 22, "Configure HDFS in a non-secure Hadoop cluster" Chapter 23, "Configure HDFS in a secure Hadoop cluster" Chapter 24, "Troubleshooting an ECS HDFS configuration" Chapter 25, "ECS HDFS core site property reference" Configure ECS HDFS 167

168 Configure ECS HDFS 168 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

169 CHAPTER 21 HDFS Overview What is ECS HDFS? HDFS Overview 169

170 HDFS Overview What is ECS HDFS? ECS HDFS is a Hadoop Compatible File System (HCFS) that enables you to run Hadoop 2.0 applications on top of your ECS storage infrastructure. You can configure your Hadoop distribution to run against the built-in Hadoop file system, against ECS HDFS, or any combination of HDFS, ECS HDFS, or other Hadoop Compatible File Systems available in your environment. The following figure illustrates how ECS HDFS integrates with an existing Hadoop cluster. Figure 49 ECS HDFS integration in a Hadoop cluster Hadoop Cluster Hadoop Client MapReduce Request Job Tracker Task Tracker Task Tracker Task Tracker MapReduce Task MapReduce Task MapReduce Task ViPRFS JAR ViPRFS JAR ViPRFS JAR ECS data nodes ECS data nodes ECS data nodes Appliance Commodity 170 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation In a Hadoop environment configured to use ECS HDFS, each of the ECS HDFS data nodes functions as a traditional Hadoop NameNode which means that all of the ECS HDFS data nodes are capable of accepting HDFS requests and servicing them. When you set up the Hadoop client to use ECS HDFS instead of traditional HDFS, the configuration points to ECS HDFS to do all the HDFS activity. On each ECS HDFS client node, any traditional Hadoop component would use the ECS HDFS client (JAR) to perform the HDFS activity. To integrate ECS HDFS with an existing Hadoop environment, you must have the following:

171 HDFS Overview A Hadoop cluster already installed and configured. The following table lists the supported distributions: Supported Hadoop distributions Pivotal 2.1 Cloudera 5.0, Cloudera Hortonworks 2.0 Hadoop installed and configured to support ECS HDFS, which requires: An ECS unstructured license with Object and HDFS access methods. An ECS replication group. Configuring Hadoop to use ECS HDFS One or more buckets that support HDFS access. Hadoop stores system configuration information in a file called core-site.xml. Editing core-site.xml is a required part of the ECS HDFS configuration. There are several types of properties to add or modify in core-site.xml including: ECS HDFS Java classes: This set of properties defines the ECS HDFS implementation classes that are contained in the ECS HDFS client JAR. They are required. File system location properties: These properties define the file system URI (scheme and authority) to use when running Hadoop jobs, and the IP addresses to the ECS data nodes for a specific ECS file system. Identity translation properties: These properties allow you to map anonymously owned objects to users, as well as specify user realms. Kerberos realm and service principal properties: These properties are required only when you are running in a Hadoop environment where Kerberos is present. These properties map Hadoop and ECS HDFS users. core-site.xml resides on each node in the Hadoop cluster. You must add the same properties to each instance of core-site.xml. Note With Cloudera distributions it is better to use Cloudera Safety Valve, and with Hortonworks it is better to use Hortonworks Ambari, to make these changes so that they are persistent across the cluster. ECS HDFS URI for file system access After you configure Hadoop to use the ECS file system, you can access it by specifying the ECS HDFS URI with viprfs:// as the scheme and a combination of ECS bucket, tenant namespace, and user-defined installation name for the authority. The ECS HDFS URI looks like this: viprfs://bucket_name.namespace.installation/path The bucket_name corresponds to a HDFS-enabled bucket. It contains the data you want to analyze with Hadoop. The namespace corresponds to a tenant namespace, and the installation_name is a name you assign to a specific set of ECS nodes or a load balancer. Configuring Hadoop to use ECS HDFS 171

172 HDFS Overview Hadoop authentication modes ECS HDFS resolves the installation_name to a set of ECS nodes or to a load balancer by using the fs.vipr.installation.[installation_name].hosts property, which includes the IP addresses of the ECS nodes or load balancer. If the installation_name maps to a set of ECS ECS nodes, you can specify how often to query ECS for the list of active nodes by setting the fs.vipr.installation. [installation_name].resolution to dynamic, and the fs.vipr.installation. [installation_name].resolution.dynamic.time_to_live_ms to specify how often to query ECS for the list of active nodes. With a Hadoop environment that uses simple security, you can specify the ECS HDFS URI as the default file system in core-site.xml by setting it as the value of the fs.defaultfs property, but this is not a requirement. With a Hadoop environment secured with Kerberos, setting ECS HDFS as a default filesystem is not supported. Whether or not to set this value to ECS HDFS requires careful consideration as part of your overall Hadoop on ECS HDFS integration planning. Where ECS HDFS is not the default file system, you must use the full URI including the path each time you access ECS data. If you have existing applications that already use a different default file system, you need to update those applications. ECS HDFS integrates with Hadoop clusters configured to use either simple or Kerberos authentication modes. To handle each mode, you must set different properties in coresite.xml to ensure that users and services are able to access the data they need. Hadoop applications access data stored in ECS buckets, so the Hadoop users must have permissions to read the objects they are trying to read, and permissions to write to the buckets they are trying to write to. Hadoop services (such as mapred, hive, and hbase) must have permissions to write system files. Table 28 Hadoop authentication mode summary Authentication mode Hadoop user ECS user Bucket permissions Simple Unix users can be users or services. Some examples include: root anonymous All Users Full Control (Required) sally cloudera mapred Kerberos A user authenticates via kinit, for example: # kinit sally@emc.com Service users are defined by the viprfs.security.principal property in core-site.xml. User writes files as sally@emc.co m Hadoop services writes system files as mapred@emc.c om Assign permissions to users as needed. The following table lists the default ACLs applied to files and directories in both simple and Kerberos authentication mode. 172 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

173 HDFS Overview Table 29 Default ACLs Name Simple authentication Kerberos authentication File Directory Hadoop simple authentication mode In a Hadoop cluster running in simple mode, when users create files, directories and objects, those files and directories are created with the owner as the empty string and are referred to as anonymous or anonymously-owned objects. This can cause problems for some applications that do their own ACL checking. To resolve this problem, set the fs.viprfs.auth.anonymous_translation property in coresite.xml to CURRENT_USER. This setting allows anonymously owned files to be displayed as if they were owned by the current Unix user. This example shows what happens when the setting for fs.viprfs.auth.anonymous_translation is set to NONE: # hadoop fs -ls / 14/01/30 11:36:23 INFO vipr.viprfilesystem: Initialized ViPRFS (atom ) for viprfs://hdfs.s3.docsite d------rwx :42 /bar d------rwx :34 /foodir d------rwx :15 /tmp If you change the setting to CURRENT_USER, and the logged in user is root, you see that root becomes the owner: # hadoop fs -ls / 14/01/30 11:30:37 INFO vipr.viprfilesystem: Initialized ViPRFS (atom ) for viprfs://hdfs.s3.docsite Found 3 items drwx root :42 /bar drwx root :34 /foodir drwx root :15 /tmp In anonymous mode, set up your ECS buckets to allow access from Everyone to ensure that Hadoop processes can access ECS buckets. ECS HDFS provides the fs.viprfs.auth.identity_translation as a way to map users to a realm when Kerberos is not present. If you must chown a file, you can specify the realm to use. For example: hdfs dfs -chown sally@myrealm.com /sallys/new/file When you specify NONE, users must type the realm each time they chown a file. Otherwise, you can specify FIXED_REALM as a convenience, then specify the actual realm to use in the fs.viprfs.auth.realm property. <property> <name>fs.viprfs.auth.identity_translation</name> <value>fixed_realm</value> </property> <property> <name>fs.viprfs.auth.realm</name> <value>myrealm.com</value> </property> Hadoop authentication modes 173

174 HDFS Overview Hadoop Kerberos authentication mode Using chown in anonymous mode is not recommended. When you chown files or directories, you change the owner from the empty string to an actual owner. Once the files or directories have an owner, anonymous users no longer have access to it. When Kerberos and the ECS Active Directory server are integrated, the Kerberos realm provides a single namespace of users so that the Hadoop users authenticated with kinit are recognized as credentialed ECS users. In a Hadoop cluster running in Kerberos mode, there must be a one-way cross-realm trust from the Kerberos realm to the Active Directory realm used to authenticate your ECS users. The following identity translation properties in core-site.xml are used to ensure the proper Hadoop-to-ECS user translation: fs.permissions.umask-mode: Set the value to 027. fs.viprfs.auth.anonymous_translation: Set the value to CURRENT_USER. fs.viprfs.auth.identity_translation: Set the value to CURRENT_USER_REALM so the realm of users is auto-detected; alternatively set to FIXED_REALM if you want to hardcode the user's realm by using the fs.viprfs.auth.realm property. In addition, you must set the following properties in core-site.xml to define a service principal: viprfs.security.principal PIG and ACLs Because PIG performs its own ACL checking to determine if a user has proper permissions to an object, you must do one of the following when running PIG on top of ECS HDFS. Option 1: Set the following properties in core-site.xml: fs.<scheme>.auth.identity_translation = CURRENT_USER_REALM (in Kerberos authentication mode) or FIXED_REALM (in simple authentication mode) fs.<scheme>.auth.anonymous_translation = CURRENT_USER Option 2: Set the following environment variable: $VIPR_LOCAL_USER_MODE ="true" SymLink support In standard HDFS, a symbolic link that does not specify the full URI to a file points to a path in the same HDFS instance. The same rule is true in ECS HDFS. When you do not specify the full URI in a symlink, ECS HDFS uses the current namespace and bucket as the root. To provide a symlink to a file outside of the current namespace and bucket, you must provide the full URI that includes both the scheme and the authority. File system interaction Note Hadoop 2.2 does not support SymLinks. When you are interacting directly with ECS HDFS, you might notice the following differences from interaction with the standard HDFS file system: 174 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

175 HDFS Overview Unsupported Hadoop applications Applications that expect the file system to be an instance of DistributedFileSystem do not work. Applications hardcoded to work against the built-in HDFS implementation require changes to use ECS HDFS. ECS HDFS does not support checksums of the data. When you use the listcorruptfileblocks function, all blocks are reported as OK because ECS HDFS has no notion of corrupted blocks. The replication factor is always reported as a constant N, where N=1. The data is protected by the ECS SLA, not by Hadoop replication. ECS HDFS does not support a small subset of Hadoop applications. HttpFS Hue Cloudera Impala Apache Oozie The following Hadoop applications are not supported with a secure (Kerberos) Hadoop cluster. HBase Hive Unsupported Hadoop applications 175

176 HDFS Overview 176 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

177 CHAPTER 22 Configure HDFS in a non-secure Hadoop cluster Configure ECS HDFS Configure HDFS in a non-secure Hadoop cluster 177

178 Configure HDFS in a non-secure Hadoop cluster Configure ECS HDFS This article describes how to configure your existing Hadoop distribution to use the data in your ECS storage infrastructure with ECS HDFS. Use this step-by-step procedure if your Hadoop distribution is configured to use simple authentication and not Kerberos authentication. If your Hadoop distribution is configured for Kerberos authentication, follow the steps described here on page 188. To perform this integration procedure, you must have: A working knowledge of your Hadoop distribution and its associated tools. The Hadoop credentials that allow you to log in to Hadoop nodes, to modify Hadoop system files, and to start and stop Hadoop services. Plan the ECS HDFS and Hadoop integration Use this list to verify that you have the information necessary to ensure a successful integration. Table 30 ECS HDFS configuration prerequisites Element Hadoop cluster ECS cluster:ecs nodes ECS cluster: replication group ECS cluster: tenant namespace What to do Verify the cluster is installed and operational. Record the admin credentials for use later in this procedure. Record the ECS node IP addresses for use later in this procedure. Verify that ECS has a replication group. HDFS requires a bucket enabled for HDFS to be created within an ECS replication group and the bucket is accessed as a filesystem using the namespace and bucket name. Verify a tenant namespace is configured. Record the name. To integrate ECS HDFS with your Hadoop cluster, perform the following tasks: 1. Obtain the ViPR HDFS installation and support package on page Create bucket for HDFS on page Deploy the ViPR HDFS JAR on page 180 or Use a Cloudera Parcel to install Hadoop on a cluster on page Edit core-site.xml on page Restart the following services: HDFS MapReduce Pivotal HAWQ (only if using this service) YARN (when using Hortonworks) 178 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

179 Configure HDFS in a non-secure Hadoop cluster Note If you use Cloudera Manager Safety Valve to change core-site.xml properties then, by default, then Cloudera Manager will restart all services. 6. Confirm the services restart correctly. Note Some services, such as namenode datanode will not start as default.fs is set to ViPRFS and ECS takes over these tasks in a non-kerberos setup 7. Verify that you have file system access. When using HBase, perform these additional tasks: 1. Edit HBASE hbase-site.xml on page Restart the HBase services. Obtain the ECS HDFS installation and support package Create bucket for HDFS The ECS HDFS JAR and HDFS support tools are provided in a ZIP file, hdfsclient <version>.zip, that you can download from the ECS support pages on support.emc.com. The ZIP file contains \tools\bin, \playbooks, \client, and \parcels directories. Before you unzip the file, create a directory to hold the zip contents (your unzip tool might do this for you), then extract the contents to that directory. After you extract the files, the directories will contain the following: \tools\bin: Contains the ViPRAdminTools.sh which enables the creation of buckets that support HDFS access without needing to use ECS object protocols or to use the ECS portal. \playbooks: Contains Ansible playbooks for configuring a secure Hadoop environment to talk to ECS HDFS. \client: Contains the following files: ECS (ViPPRFS) JAR files (viprfs-client-<ecs version>-hadoop- <hadoop version>.jar): Used to configure different Hadoop distributions. libvipr-<version>.so: Used to configure Pivotal HAWQ for use with ECS HDFS. \parcels Cloudera distributions in "Parcel" format. Parcels include the appropriate ViPRFS JAR file. Hadoop HDFS support in ECS uses the ECS object store. Buckets for use by HDFS can be created in a number of ways, but must be marked for HDFS access. Where you are creating buckets to support a Hadoop cluster that uses simple security (non-kerberos), you can use the ECS Portal or you can use the object APIs, as described in the following article: Create and configure buckets on page 138 Refer to Set bucket permissions for HDFS on page 180 for information on setting permissions. Obtain the ECS HDFS installation and support package 179

180 Configure HDFS in a non-secure Hadoop cluster Note You should not use underscores in bucket names as they are not supported by the URI Java class. For example, viprfs://my_bucket.ns.site/ will not work as this is an invalid URI and is thus not understood by Hadoop. Set bucket permissions for HDFS Deploy the ECS HDFS JAR Before you can use a bucket for HDFS access, you need to ensure that permissions are appropriately set. Buckets created using the ECS Data Access protocols have Full Control for the user who created the bucket. However, in order for a bucket to be used for HDFS, it must be set so that All Users have full control. You can set bucket permissions using a graphical client, such as the S3 Browser, or you can use a command line tool, such as s3curl. Use this procedure to put the ECS HDFS JAR on the classpath of each client node in the ECS cluster. Before you begin Obtain the ECS HDFS JAR for your ECS distribution from the EMC Support site for ECS as described in Obtain the ECS HDFS installation and support package on page 179. Table 31 ECS HDFS JARs Hadoop distribution Pivotal HD Cloudera ECS HDFS JAR PHD 2.1: viprfs-client hadoop-2.2.jar For Cloudera versions lower than CDH5, use: viprfsclient hadoop alpha.jar For Cloudera versions CDH5.0.x, use: viprfs-client hadoop-2.2.jar For Cloudera versions higher than CDH5.1.x, use: viprfsclient hadoop-2.3.jar Hortonworks HDP 2.0: viprfs-client hadoop-2.2.jar Note If you are using a Cloudera Hadoop distribution, you can use the supplied Cloudera Parcels to install a Hadoop distribution that is pre-configured with the ViPRFS JAR. See Use a Cloudera Parcel to install Hadoop on a cluster on page 181. Procedure 1. Log in to a ECS client node. 180 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation 2. Run the classpath command to get the list of directories in the classpath: # hadoop classpath 3. Copy ECS HDFS JAR to one of folders listed by the classpath command that occurs after the /conf folder.

181 Configure HDFS in a non-secure Hadoop cluster ECS distribution Pivotal HD Cloudera Hortonworks Classpath location (suggested) /usr/lib/gphd/hadoop/lib /usr/lib/hadoop/lib /usr/lib/hadoop/lib 4. Repeat this procedure on each ECS client node. Use a Cloudera Parcel to install Hadoop on a cluster Cloudera uses Parcels as a mechanism to distribute software to CDH Clusters from the Cloudera Manager Admin console. ECS provides Cloudera Parcels, referred to as ViPRFS Parcels, that are pre-configured to use ECS HDFS. Before you begin Cloudera Parcels that include the ECS (ViPRFS) Client are provided in the ECS HDFS installation and support package (see Obtain the ECS HDFS installation and support package on page 179). Procedure 1. Create a parcel repository To create a Cloudera Parcel repository, put the ViPRFS Parcel files in a directory published by a web server. This is typically hosted inside your network. 2. Add the ECS parcel repository from the Cloudera Manager UI. a. Navigate to Administration > Settings > Parcels. b. Set the Parcel Update Frequency to 1 minute to speed discovery. c. Click Save Changes 3. Download the parcel. Navigate to Hosts > Parcels > Downloadable and download the latest ViPRFS Parcel. 4. Distribute the parcel to your cluster. It is possible to distribute multiple versions of ViPRFS Parcels but only one version can be activated at any time. To delete a parcel, you need to de-activate the parcel first, and then remove the parcel from your hosts. All the above operations can be performed in the Parcel UI. 5. Activate the Parcel Edit Hadoop core-site.xml file A distributed parcel can be activated by pushing the Activate button. Cloudera Manager will prompt you to restart the cluster to enable running processes to use the new parcel. 6. Click Restart to restart the cluster. Use this procedure to update core-site.xml with the properties needed to integrate ECS HDFS with a Hadoop cluster that uses simple authentication mode. Before you begin You must have a set of user credentials that enable you to log in to Hadoop nodes and modify core-site.xml. Use a Cloudera Parcel to install Hadoop on a cluster 181

182 Configure HDFS in a non-secure Hadoop cluster The location of core-site.xml depends on the distribution you are using. Table 32 core-site.xml locations Hadoop Distribution core-site.xml location Nodes to update Pivotal HD /etc/ghpd/hadoop/conf ComputeMaster and clients Cloudera /etc/hadoop/conf All client nodes Hortonworks /etc/hadoop/conf All nodes core-site.xml resides on each node in the Hadoop cluster. You must modify the same properties in each instance. You can make the change in one node, and then use secure copy command (scp) to copy the file to the other nodes in the cluster. See core_site.xml property reference for more information about each property you need to set. Procedure 1. Log in to one of the HDFS nodes where core-site.xml is located. 2. Make a backup copy of core-site.xml. cp core-site.xml core-site.backup 3. Using the text editor of your choice, open core-site.xml for editing. Note With Cloudera distributions it is better to use Cloudera Safety Valve, and with Hortonworks it is better to use Hortonworks Ambari, to make these changes so that they are persistent across the cluster. 4. Add the following properties and values to define the Java classes that implement the ECS HDFS file system: <property> <name>fs.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.viprfilesystem</value> </property> <property> <name>fs.abstractfilesystem.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.viprabstractfilesystem</value> </property> 5. Add the fs.vipr.installations property. In the following example, the value is set to Site1. <property> <name>fs.vipr.installations</name> <value>site1</value> </property> 6. Add the fs.vipr.installation.[installation_name].hosts property as a comma-separated list of ECS data nodes or load balancer IP addresses. In the following example, the installation_name is set to Site EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

183 Configure HDFS in a non-secure Hadoop cluster Note The use of a load balancer adds no value to a HDFS scenario as the client has the logic to connect to the nodes directly. For this reason, it is recommended that you provide a list of data nodes in this property, not the address of a load balancer. In addition, you should set the fs.vipr.installation.[installation_name].resolution to "dynamic". <property> <name>fs.vipr.installation.site1.hosts</name> <value> , , </value> </property> 7. Add the fs.vipr.installation.[installation_name].resolution property, and set it to one of the following values: Option dynamic fixed Description Use when accessing ECS data nodes directly without a load balancer. Use when accessing ECS data nodes through a load balancer. In the following example, installation_name is set to Site1. <property> <name>fs.vipr.installation.site1.resolution</name> <value>dynamic</value> </property> a. If you set fs.vipr.installation.[installation_name].resolution to dynamic, add the fs.vipr.installation.[installation_name].resolution.dynamic.time_to_live_ms property to specify how often to query ECS for the list of active nodes. In the following example, installation_name is set to Site1. <property> <name>fs.vipr.installation.site1.resolution.dynamic.time_to_live _ms</name> <value>900000</value> </property> 8. Locate the fs.defaultfs property and modify the value to specify the ECS file system URI.. This setting is optional and you can specify the full file system URL to connect to the ECS ViPRFS. Use the following format: viprfs:// <bucket_name.namespace.installation_name, where bucket_name: The name of the bucket that contains the data you want to use when you run Hadoop jobs. If running in simple authentication mode, the owner of the bucket must grant permission to Everybody. In the following example, the bucket_name is set to mybucket. namespace: The tenant namespace where bucket_name resides. In the following example, the namespace is set to mynamespace. installation_name: The value specified by the fs.vipr.installations property. In the following example, installation_name is set to Site1. <property> <name>fs.defaultfs</name> Edit Hadoop core-site.xml file 183

184 Configure HDFS in a non-secure Hadoop cluster <value>viprfs://mybucket.mynamespace.site1/</value> </property> 9. Locate fs.permissions.umask-mode, and set the value to 022. In some configurations, this property might not already exist. If it does not, then add it. <property> <name>fs.permissions.umask-mode</name> <value>022</value> </property> 10.Add the fs.viprfs.auth.anonymous_translation property; use it to specify whether to map anonymously owned objects to the current user so the current user has permission to modify it. Option NONE (default) CURRENT_USER Description Do not map anonymously owned objects to the current user. Map anonymously owned objects to the current Unix user. <property> <name>fs.viprfs.auth.anonymous_translation</name> <value>current_user</value> </property> 11.Add the fs.viprfs.auth.identity_translation property. It provides a way to assign users to a realm when Kerberos is not present. Option FIXED_REALM Description When specified, ECS HDFS gets the realm name from the value of the fs.vipr.auth.realm property. NONE (default) ECS HDFS does no realm translation. <property> <name>fs.viprfs.auth.identity_translation</name> <value>none</value> </property> 12.If you set the fs.viprfs.auth.identity_translation property to FIXED_REALM, add the fs.viprfs.auth.realm property. 13.If you want to use the Pivotal HAWQ service, add the hawq.vipr.endpoint property. Specify the value using the following format: bucket_name.namespace.installation_name. Where: bucket_name:the name of the bucket that contains the data you want to use when you run Hadoop jobs. If running in simple authentication mode, the owner of the bucket must grant permission to Everybody. In the following example, bucket_name is set to mybucket. namespace: The tenant namespace where bucket_name resides. In the following example, the namespace is set to mynamespace. installation_name: The value specified by the fs.vipr.installations property. In the following example, the installation_name is set to Site EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

185 Configure HDFS in a non-secure Hadoop cluster You must be running a version of ECS that supports Pivotal HAWQ. For more information, see the Support Matrix. <property> <name>hawq.vipr.endpoint</name> <value>mybucket.mynamespace.site1</value> </property> 14.Save core-site.xml. 15.Update the core-site.xml on the required nodes in your Hadoop cluster. 16.If you are using a Cloudera distribution, use Cloudera Manager to update the coresite.xml safety valve with the same set of properties and values. 17.Restart the Hadoop services. Hadoop Distribution Pivotal HD Commands ComputeMaster: # service hadoop-yarn-resourcemanager restart Data Nodes: # service hadoop-hdfs-datanode restart # service hadoop-yarn-nodemanager restart NameNode: # service hadoop-yarn-nodemanager restart When you configure the Pivotal Hadoop cluster to use ECS HDFS as the default file system (specified by fs.defaultfs in coresite.xml), you cannot use the icm_client's cluster start/stop functionality, instead, you must start all cluster services (except HDFS) individually. For example: icm_client start -s yarn icm_client start -s zookeeper and so on. Cloudera Apache Use Cloudera Manager to restart the HDFS and MapReduce services # stop-all.sh # start-all.sh 18.Test the configuration by running the following command to get a directory listing: # hdfs dfs -ls viprfs://mybucket.mynamespace.site1/ 13/12/13 22:20:37 INFO vipr.viprfilesystem: Initialized ViPRFS for viprfs://mybucket.mynamespace.site1/ If you have set fs.defaultfs, you can use: # hdfs dfs -ls / Edit Hadoop core-site.xml file 185

186 Configure HDFS in a non-secure Hadoop cluster Edit HBASE hbase-site.xml When you use HBASE with ECS HDFS, you must set the hbase.rootdir in hbasesite.xml to the same value as the core-site.xml fs.defaultfs property. hbase-site.xml is located in one of the following locations: Table 33 hbase-site.xml locations Hadoop Distribution Pivotal HD Cloudera Hortonworks hbase-site.xml location /etc/ghpd/hbase/conf/ /etc/hbase/conf/ /etc/hbase/conf/ Procedure 1. Open hbase-site.xml. 2. Set the hbase.rootdir property to the same value as fs.defaultfs adding /hbase as the suffix. 3. Save your changes. a. On Cloudera, add the hbase.rootdir property to the HBase Service Configuration Safety Valve for hbase-site.xml. 4. Restart the services for your distribution. HadoopDistribution Pivotal HD Description Run this command on the hbase master node: # service hbase-master restart Run this command on the hbase region server: # service hadoop-regionserver restart Cloudera Hortonworks Use Cloudera manager to restart the HBase service. # bin/start-hbase.sh Example 1 hbase.rootdir entry <property> <name>hbase.rootdir</name> <value>viprfs://testbucket.s3.testsite/hbase</value> </property> 186 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

187 CHAPTER 23 Configure HDFS in a secure Hadoop cluster Configure secure Hadoop cluster to use ECS HDFS Configure HDFS in a secure Hadoop cluster 187

188 Configure HDFS in a secure Hadoop cluster Configure secure Hadoop cluster to use ECS HDFS This article describes how to configure your existing Hadoop distribution to use the data in your ECS storage infrastructure with ECS HDFS. Use this step-by-step procedure if your Hadoop cluster is configured to use Kerberos authentication. If your Hadoop cluster is configured for simple authentication, follow the steps described here on page 178. This procedure has not been qualified on the ECS Appliance. To perform this integration procedure, you must have: A working knowledge of your Hadoop cluster and its associated tools. The Hadoop credentials that allow you to log in to Hadoop nodes, to modify Hadoop system files, and to start and stop Hadoop services. Plan the ECS HDFS and secure Hadoop cluster integration Use this list to verify that you have the information necessary to ensure a successful integration. It is a best practice to get your Hadoop cluster working with Kerberos before you configure ECS HDFS. Table 34 ECS HDFS configuration prerequisites Element Hadoop cluster ECS cluster:ecs nodes ECS cluster: replication group ECS cluster: tenant namespace What to do Verify the cluster is installed and operational. Record the admin credentials for use later in this procedure. Record the ECS node IP addresses for use later in this procedure. Verify that ECS has a replication group. HDFS requires a bucket enabled for HDFS to be created within an ECS replication group and the bucket is accessed as a filesystem using the namespace and bucket name. Verify a tenant namespace is configured. Record the name. In addition, verify that a Kerberos KDC is installed and configured to handle authentication of the Hadoop service principals. If you are using Active Directory to authenticate ECS users, you must set up a cross-realm trust between the Kerberos realm and the ECS user realm. Help with setting up the Kerberos KDC and configuring trust is provided in Guidance on Kerberos configuration on page 198. To integrate ECS HDFS with your secure Hadoop cluster, complete the following tasks: 1. Obtain the ECS HDFS installation and support package on page Create bucket for secure HDFS on page Deploy the ECS HDFS JAR in a secure cluster on page 190 or Use a Cloudera Parcel to install Hadoop on a secure cluster on page Configure ECS nodes with the ECS Service Principal on page Edit core-site.xml on page Restart the HDFS and MapReduce services 188 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

189 Configure HDFS in a secure Hadoop cluster 7. Confirm the services restart correctly 8. Verify that you have file system access Obtain the ECS HDFS installation and support package The ECS HDFS JAR and HDFS support tools are provided in a ZIP file, hdfsclient <version>.zip, that you can download from the ECS support pages on support.emc.com. The ZIP file contains \tools\bin, \playbooks, \client, and \parcels directories. Before you unzip the file, create a directory to hold the zip contents (your unzip tool might do this for you), then extract the contents to that directory. After you extract the files, the directories will contain the following: \tools\bin: Contains the ViPRAdminTools.sh which enables the creation of buckets that support HDFS access without needing to use ECS object protocols or to use the ECS portal. \playbooks: Contains Ansible playbooks for configuring a secure Hadoop environment to talk to ECS HDFS. \client: Contains the following files: ECS (ViPPRFS) JAR files (viprfs-client-<ecs version>-hadoop- <hadoop version>.jar): Used to configure different Hadoop distributions. libvipr-<version>.so: Used to configure Pivotal HAWQ for use with ECS HDFS. \parcels Create bucket for secure HDFS Cloudera distributions in "Parcel" format. Parcels include the appropriate ViPRFS JAR file. Where the Hadoop cluster is secured using Kerberos, and you want users authenticated against a Kerberos domain to be able to create buckets, you can create an S3 bucket and enable it for HDFS access or you can use the ECS Admin tool. The Admin tool provides a convenient way of creating a bucket without having access to the ECS portal or without having to use the ECS Management API or the S3 Object Service API. These two mechanisms are described here: Create and configure buckets on page 138 Create a bucket for HDFS using the ViPRAdminTool on page 189 Note You should not use underscores in bucket names as they are not supported by the URI Java class. For example, viprfs://my_bucket.ns.site/ will not work as this is an invalid URI and is thus not understood by Hadoop. Create a bucket for HDFS using the ViPRAdminTool From a Hadoop cluster secured using Kerberos, you can use the ViPRAdminTool.sh to create a bucket for use by Object and HDFS protocols, without requiring knowledge of the ECS REST API or Object Data Access APIs. The tool is a wrapper around a Java class within Obtain the ECS HDFS installation and support package 189

190 Configure HDFS in a secure Hadoop cluster the ECS HDFS JAR, so can be run once the Hadoop cluster has been configured to use ECS HDFS. Before you begin Obtain the ViPRAdminTool.sh as described in Obtain the ECS HDFS installation and support package on page 189 Hadoop must be installed and the machine on which you are running the tool must have the ECS HDFS JAR installed and the Hadoop cluster configured to access the ECS HDFS. Kerberos security must be configured. If you do not have Kerberos security configured, you will need to create a bucket using the S3 API or the ECS REST API. Reference information for the ViPRAdminTool tool is provided in ECS administration tool reference on page 200. Procedure 1. Use the ViPRAdminTool.sh script and specify: the createbucket command, the path to the ECS node and namespace, the name of the new bucket, and the permissions to set for the bucket. The following command creates a bucket called "newbucket" and sets its permissions as 0755 (rwx/r-x/r-x). The object virtual pool in which the bucket is created is the default pool and the bucket is assigned to the default project. ViPRAdminTool.sh createbucket viprfs://<ecs Node Address>/ mynamespace newbucket 0755 Deploy the ECS HDFS JAR in a secure cluster If you want to specify a project and a virtual pool, you can use the ECS REST API or CLI to obtain these values. Use this procedure to put the ECS HDFS JAR on the classpath of each client node in the ECS cluster. Before you begin Obtain the ECS HDFS JAR for your ECS distribution from the EMC Support site for ECS as described in Obtain the ECS HDFS installation and support package on page 189. Table 35 ECS HDFS JARs Hadoop distribution Pivotal HD Cloudera ECS HDFS JAR PHD 2.1: viprfs-client hadoop-2.2.jar For Cloudera versions lower than CDH5, use: viprfsclient hadoop alpha.jar For Cloudera versions CDH5.0.x, use: viprfs-client hadoop-2.2.jar For Cloudera versions higher than CDH5.1.x, use: viprfsclient hadoop-2.3.jar Hortonworks HDP 2.0: viprfs-client hadoop-2.2.jar 190 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

191 Configure HDFS in a secure Hadoop cluster Note If you are using a Cloudera Hadoop distribution, you can use the supplied Cloudera Parcels to install a Hadoop distribution that is pre-configured with the ViPRFS JAR. See Use a Cloudera Parcel to install Hadoop on a secure cluster on page 191. Procedure 1. Log in to a ECS client node. 2. Run the classpath command to get the list of directories in the classpath: # hadoop classpath 3. Copy ECS HDFS JAR to one of folders listed by the classpath command that occurs after the /conf folder. ECS distribution Pivotal HD Cloudera Hortonworks Classpath location (suggested) /usr/lib/gphd/hadoop/lib /usr/lib/hadoop/lib /usr/lib/hadoop/lib 4. Repeat this procedure on each ECS client node. Use a Cloudera Parcel to install Hadoop on a secure cluster Cloudera uses Parcels as a mechanism to distribute software to CDH Clusters from the Cloudera Manager Admin console. ECS provides Cloudera Parcels, referred to as ViPRFS Parcels, that are pre-configured to use ECS HDFS. Before you begin Cloudera Parcels that include the ECS (ViPRFS) Client are provided in the ECS HDFS installation and support package (see Obtain the ECS HDFS installation and support package on page 189). Procedure 1. Create a parcel repository To create a Cloudera Parcel repository, put the ViPRFS Parcel files in a directory published by a web server. This is typically hosted inside your network. 2. Add the ECS parcel repository from the Cloudera Manager UI. a. Navigate to Administration > Settings > Parcels. b. Set the Parcel Update Frequency to 1 minute to speed discovery. c. Click Save Changes 3. Download the parcel. Navigate to Hosts > Parcels > Downloadable and download the latest ViPRFS Parcel. 4. Distribute the parcel to your cluster. It is possible to distribute multiple versions of ViPRFS Parcels but only one version can be activated at any time. To delete a parcel, you need to de-activate the parcel first, and then remove the parcel from your hosts. All the above operations can be performed in the Parcel UI. Use a Cloudera Parcel to install Hadoop on a secure cluster 191

192 Configure HDFS in a secure Hadoop cluster 5. Activate the Parcel A distributed parcel can be activated by pushing the Activate button. Cloudera Manager will prompt you to restart the cluster to enable running processes to use the new parcel. 6. Click Restart to restart the cluster. Configure ECS nodes with the ECS Service Principal The ECS service principal and its corresponding keytab file must reside on each ECS data node. Use the Ansible playbooks provided to automate these steps. Before you begin You must have the following items before you can complete this procedure: Access to the Ansible playbooks. Obtain the Ansible playbooks from the ECS HDFS software package as described in Obtaining the ViPR HDFS installation and support package on page 189, and copy it to the node were you intend to install Ansible. The list of ECS node IP addresses. IP address of the KDC. The DNS resolution where you run this script should be the same as the DNS resolution for the Hadoop host, otherwise the vipr/_host@realm will not work. ECS provides reusable Ansible content called 'roles', which consist of python scripts, YAML-based task lists, and template files. vipr_kerberos_config: Configures a ECS node for Kerberos. vipr_jce_config: Configures a ECS data node for unlimited-strength encryption by installing JCE policy files. vipr_kerberos_principal: Acquires a service principal for an ECS node. Procedure 1. Install Ansible. yum install epel-release && yum install ansible 2. Decompress the hdfsclient <version>.zip file The steps in this procedure use the playbooks contained in the viprfsclient <version>/playbooks/samples directory and the steps are also contained in viprfs-client <version>/playbooks/ samples/readme.md. 3. Install the supplied Ansible roles. ansible-galaxy install -r requirements.txt -f 4. Copy the contents of the viprfs-client <version>/playbooks/ samples directory to a working directory. 5. Edit inventory.txt to refer to the ECS data nodes and KDC server. The default entries are shown below. [data_nodes] [100:200] 192 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

193 Configure HDFS in a secure Hadoop cluster [kdc] Download the "unlimited" JCE policy archive from oracle.com, and extract it to the UnlimitedJCEPolicy directory. Kerberos may be configured to use a strong encryption type, such as AES-256. In that situation, the JRE within the ECS nodes must be reconfigured to use the 'unlimited' policy. Note This step should be performed only if you are using strong encryption type. 7. Copy the krb5.conf file from the KDC to the working directory. 8. Edit the generate-vipr-keytabs.yml as necessary and set the domain name. For example. [root@nile3-vm22 samples]# cat generate-vipr-keytabs.yml --- ### # Generates keytabs for ViPR/ECS data nodes. ### - hosts: data_nodes serial: 1 roles: - role: vipr_kerberos_principal kdc: "{{ groups.kdc first }}" principals: - name: vipr/_host@ma.emc.com keytab: keytabs/_host@ma.emc.com.keytab In this example, the default value (vipr/_host@example.com) has been replaced with (vipr/_host@ma.emc.com) and the domain is MA.EMC.COM. 9. Run export ANSIBLE_HOST_KEY_CHECKING=False 10.Run the Ansible playbook to generate keytabs. ansible-playbook -v -k -i inventory.txt generate-vipr-keytabs.yml 11.Edit the setup-vipr-kerberos.yml file as necessary. The default file contents are shown below. # cat setup-vipr-kerberos.yml --- ### # Configures ViPR/ECS for Kerberos authentication. # - Configures krb5 client # - Installs keytabs Configure ECS nodes with the ECS Service Principal 193

194 Configure HDFS in a secure Hadoop cluster # - Installs JCE policy ### - hosts: data_nodes roles: - role: vipr_kerberos_config krb5: config_file: krb5.conf service_principal: name: vipr/_host@example.com keytab: keytabs/_host@example.com.keytab - role: vipr_jce_config jce_policy: name: unlimited src: UnlimitedJCEPolicy/ In this example, the default value (vipr/_host@example.com) has been replaced with (vipr/_host@ma.emc.com) and the domain is MA.EMC.COM. Note Remove the "vipr_jce_config" role if you are not using strong encryption type. 12.Run the Ansible playbook to configure the data nodes with the ECS service principal. Make sure the./viprfs-client <version>/playbooks/ samples/keytab directory exist and the krb5.conf file is in the working directory viprfs-client <version>/playbooks/samples directory. ansible-playbook -v -k -i inventory.txt setup-vipr-kerberos.yml Verify that the correct ECS service principal, one per data node, has been created (from the KDC): # kadmin.local -q "list_principals" grep vipr vipr/nile3-vm42.centera.lab.emc.com@ma.emc.com vipr/nile3-vm43.centera.lab.emc.com@ma.emc.com Verify that correct keytab is generated and stored in location: /data/hdfs/ krb5.keytab on all ECS data nodes. You can use the "strings" command on the keytab to extract the human readable text, and verify that it contains the correct principal. For example: dataservice :~ # strings /data/hdfs/krb5.keytab MA.EMC.COM vipr nile3-vm42.centera.lab.emc.com In this case the principal is vipr/nile3-vm42.centera.lab.emc.com. Edit core-site.xml Use this procedure to update core-site.xml with the properties that are required when using ECS HDFS with a ECS cluster that uses Kerberos authentication mode. Before you begin Obtain the credentials that enable you to log in to ECS nodes and modify coresite.xml. 194 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

195 Configure HDFS in a secure Hadoop cluster See core_site.xml property reference for more information about each property you need to set. core-site.xml resides on each node in the Hadoop cluster, and you must modify the same properties in each instance. You can make the change in one node, and then use secure copy command (scp) to copy the file to the other nodes in the cluster. As a best practice, back up core-site.xml before you start the configuration procedure. The location of core-site.xml depends on the distribution you are using. Table 36 Location of core-site.xml files ViPR Controller Distribution core-site.xml location Nodes to update Pivotal HD /etc/ghpd/hadoop/conf ComputeMaster and clients Cloudera /etc/hadoop/conf Client nodes Hortonworks /etc/hadoop/conf All nodes Procedure 1. Log in to one of the HDFS nodes where core-site.xml is located. 2. Make a backup copy of core-site.xml. cp core-site.xml core-site.backup 3. Using the text editor of your choice, open core-site.xml for editing. Note With Cloudera distributions it is better to use Cloudera Safety Valve, and with Hortonworks it is better to use Hortonworks Ambari, to make these changes so that they are persistent across the cluster. 4. Add the following properties and values to define the Java classes that implement the ECS HDFS file system: <property> <name>fs.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.viprfilesystem</value> </property> <property> <name>fs.abstractfilesystem.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.viprabstractfilesystem</value> </property> 5. Add the fs.vipr.installations property. In the following example, the value is set to Site1. <property> <name>fs.vipr.installations</name> <value>site1</value> </property> 6. Add the fs.vipr.installation.[installation_name].hosts property as a comma-separated list of ECS data nodes or load balancer IP addresses. In the following example, the installation_name is set to Site1. Edit core-site.xml 195

196 Configure HDFS in a secure Hadoop cluster Note The use of a load balancer adds no value to a HDFS scenario as the client has the logic to connect to the nodes directly. For this reason, it is recommended that you provide a list of data nodes in this property, not the address of a load balancer. In addition, you should set the fs.vipr.installation.[installation_name].resolution to "dynamic". <property> <name>fs.vipr.installation.site1.hosts</name> <value> , , </value> </property> 7. Add the fs.vipr.installation.[installation_name].resolution property, and set it to one of the following values: Option dynamic fixed Description Use when accessing ECS data nodes directly without a load balancer. Use when accessing ECS data nodes through a load balancer. In the following example, installation_name is set to Site1. <property> <name>fs.vipr.installation.site1.resolution</name> <value>dynamic</value> </property> a. If you set fs.vipr.installation.[installation_name].resolution to dynamic, add the fs.vipr.installation.[installation_name].resolution.dynamic.time_to_live_ms property to specify how often to query ECS for the list of active nodes. In the following example, installation_name is set to Site1. <property> <name>fs.vipr.installation.site1.resolution.dynamic.time_to_live _ms</name> <value>900000</value> </property> 8. Locate the fs.defaultfs property and modify the value to specify the ECS file system URI.. This setting is optional and you can specify the full file system URL to connect to the ECS ViPRFS. Use the following format: viprfs:// <bucket_name.namespace.installation_name, where bucket_name: The name of the bucket that contains the data you want to use when you run Hadoop jobs. If running in simple authentication mode, the owner of the bucket must grant permission to Everybody. In the following example, the bucket_name is set to mybucket. namespace: The tenant namespace where bucket_name resides. In the following example, the namespace is set to mynamespace. installation_name: The value specified by the fs.vipr.installations property. In the following example, installation_name is set to Site1. <property> <name>fs.defaultfs</name> 196 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

197 Configure HDFS in a secure Hadoop cluster <value>viprfs://mybucket.mynamespace.site1/</value> </property> 9. Locate fs.permissions.umask-mode, and set the value to 027. In some configurations, this property might not already exist. If it does not, then add it. <property> <name>fs.permissions.umask-mode</name> <value>027</value> </property> 10.Add the fs.viprfs.auth.anonymous_translation property; use it to specify whether to map anonymously owned objects to the current user so the current user has permission to modify it. Option NONE (default) CURRENT_USER Description Do not map anonymously owned objects to the current user. Map anonymously owned objects to the current Unix user. <property> <name>fs.viprfs.auth.anonymous_translation</name> <value>current_user</value> </property> 11.Add the fs.viprfs.auth.identity_translation property, and set it to CURRENT_USER_REALM, which maps to the realm of the user signed in via kinit. <property> <name>fs.viprfs.auth.identity_translation</name> <value>current_user_realm</value> </property> 12.Add the viprfs.security.principal property. This property tells the KDC who the ECS user is. The principal name can include "_HOST" which is automatically replaced by the actual data node FQDN at run time. <property> <name>viprfs.security.principal</name> <value>vipr/_host@example.com</value> </property> 13.Restart the HDFS and MapReduce services. 14.Test the configuration by running the following command to get a directory listing: # kinit <service principal> # hdfs dfs -ls viprfs://mybucket.mynamespace.site1/ 13/12/13 22:20:37 INFO vipr.viprfilesystem: Initialized ViPRFS for viprfs://mybucket.mynamespace.site1/ If you have set fs.defaultfs, you can use: # hdfs dfs -ls / Edit core-site.xml 197

198 Configure HDFS in a secure Hadoop cluster Guidance on Kerberos configuration Provides guidance on configuring Kerberos in the Hadoop cluster. Set up the Kerberos KDC Set up the Kerberos KDC by following these steps. Procedure 1. Install krb5-workstation. Use the command: yum install -y krb5-libs krb5-server krb5-workstation 2. Modify /etc/krb5.conf and change the realm name and extensions. 3. Modify /var/kerberos/krb5kdc/kdc.conf and change the realm name to match your own. 4. If your KDC is a VM, recreate /dev/random (otherwise your next step of creating the KDC database will take a very long time). a. Remove using: # rm -rf /dev/random b. Recreate using: # mknod /dev/random c Create the KDC database. # kdb5_util create -s Note If you made a mistake with the initial principals. For example, you ran "kdb5_util create -s" incorrectly, you might need to delete these principals explicitly in the /var/kerberos/krb5kdc/ directory. 6. Modify kadm5.acl to specify users that have admin permission. */admin@det.emc.com * 7. Modify /var/kerberos/krb5kdc/kdc.conf and take out any encryption type except des-cbc-crc:normal. Also modify the realm name. 8. Ensure iptables and selinux are off on all nodes (KDC server as well as Hadoop nodes). 9. Start KDC services and create a local admin principal. kadmin.local 198 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

199 Configure HDFS in a secure Hadoop cluster # service krb5kdc start # service kadmin start # /usr/kerberos/sbin/kadmin.local-q "addprinc root/admin" # kinit root/admin 10.Copy the krb5.conf file to all Hadoop nodes. Any time you make a modification to any of the configuration files restart the below services and copy the krb5.conf file over to relevant Hadoop host and ECS nodes. 11.Restart the services. service krb5kdc restart service kadmin restart Configure AD user authentication for Kerberos 12.You can visit following link to setup a Kerberos KDC based on steps at Where you have a Hadoop environment configured with Kerberos security, you can configure it to authenticate against the ECS AD domain. Make sure you have an AD user for your ADREALM. The user "detscr" for ADREALM CAMBRIDGE.EMC.COM is used in the example below. Create a one-way trust between the KDCREALM and the ADREALM as shown in the example. Do not try to validate this realm using "netdom trust". On Active Directory You must set up a one-way cross-realm trust from the KDC realm to the AD realm. To do so, run the following commands at a command prompt. ksetup /addkdc KDC-REALM <KDC hostname> netdom trust KDC-REALM /Domain:AD-REALM /add /realm / passwordt:<trustpassword> ksetup /SetEncTypeAttr KDC-REALM <enc_type> For example: ksetup /addkdc LSS.EMC.COM lcigb101.lss.emc.com netdom trust LSS.EMC.COM /Domain:CAMBRIDGE.EMC.COM /add /realm / passwordt:changeme ksetup /SetEncTypeAttr LSS.EMC.COM DES-CBC-CRC For this example, encryption des-cbc-crc was used. However, this is a weak encryption that was only chosen for demonstration purposes. Whatever encryption you choose, the AD, KDC, and clients must support it. On your KDC (as root) To set up a one-way trust, you will need to create a "krbtgt" service principal. To do so, the name is krbtgt/kdc-realm@ad-realm. Give this the password ChangeMe, or whatever you specified to the /passwordt argument above. 1. On KDC (as root) # kadmin kadmin: addprinc -e "des-cbc-crc:normal" krbtgt/ LSS.EMC.COM@CAMBRIDGE.EMC.COM Guidance on Kerberos configuration 199

200 Configure HDFS in a secure Hadoop cluster Note When deploying, it is best to limit the encryption types to the one you chose. Once this is working, additional encryption types can be added. 2. Add the following rules to your core-site.xml hadoop.security.auth_to_local property: RULE:[1:$1@$0](^.*@CAMBRIDGE\.EMC\.COM$)s/^(.*)@CAMBRIDGE\.EMC\.COM $/$1/g RULE:[2:$1@$0](^.*@CAMBRIDGE\.EMC\.COM$)s/^(.*)@CAMBRIDGE\.EMC\.COM $/$1/g 3. Verify that AD or LDAP is correctly setup with the Kerberos (KDC) server. User should be able to "kinit" against an AD user and list local HDFS directory. Note ECS administration tool reference If you are configuring your Hadoop cluster and ECS to authenticate through an AD, create local Linux user accounts on all Hadoop nodes for the AD user you will be kinit'ed as, and also make sure that all Hadoop host are kinit'ed using that AD user. For example, if you kinit as userx@adrealm, create userx as a local user on all Hadoop hosts, and kinit using: 'kinit userx@adrealm' on all hosts for that user. In the example below, we will authenticate as "kinit detscr@cambridge.emc.com", so will create a user called "detscr" and kinit as this user on the Hadoop host. As shown below: [root@lviprb159 ~]# su detscr [detscr@lviprb159 root]$ whoami detscr [detscr@lviprb159 root]$ kinit detscr@cambridge.emc.com Password for detscr@cambridge.emc.com: [detscr@lviprb159 root]$ klist Ticket cache: FILE:/tmp/krb5cc_1010 Default principal: detscr@cambridge.emc.com Valid starting Expires Service principal 12/22/14 14:28:27 03/02/15 01:28:30 krbtgt/ CAMBRIDGE.EMC.COM@CAMBRIDGE.EMC.COM renew until 09/17/17 15:28:27 [detscr@lviprb159 root]$ hdfs dfs -ls / Found 4 items drwx---rwx - yarn hadoop :11 /app-logs drwx---rwt - hdfs :48 /apps drwx---r-x - mapred :11 /mapred drwx---r-x - hdfs :11 /mr-history The ECS administration tool (ViPRAdminTool.sh) is provided to enable buckets that support HDFS to be created from a Hadoop cluster without needing to interact with the ECS Portal or API. The tool can also be used to list, delete, and obtain the status of buckets. Obtaining the tool The ViPRAdminTool.sh tool is a wrapper around a Java class within the ECS HDFS JAR, so can be run once the Hadoop cluster has been configured to use ECS HDFS. It can be obtained as described in Obtain the ECS HDFS installation and support package on page EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

201 Configure HDFS in a secure Hadoop cluster Usage To run the tool, use: ViPRAdminTool.sh <COMMAND> [ARGUMENTS] You can run the tool without the script and calling the class directly using: hadoop com/emc/hadoop/fs/vipr/vipradminclient <COMMAND> [ARGUMENTS] A command is always required, denoted by <>, arguments are optional, denoted by []. Note You must supply all arguments up to the last one that you want to specify. Commands createbucket <uri><name>[permission][vpoolid] [projectid] [objecttype] Creates a bucket. statbucket <uri> <name> Gets the status for specified bucket. deletebucket <uri><name> Deletes the specified bucket. listbucket <uri> Lists all buckets the current user has permission to read. Arguments name uri Bucket name to create/stat/delete. A pointer to the ECS deployment, in <scheme>://<datanode Address>/<namespace> format. permission Valid for createbucket only. Specify POSIX permissions in octal format, such as '0755'. Default value: vpoolid Valid for createbucket only. Identity of the replication group to be used. Default replication group is used if not specified. objecttype Valid for createbucket only. Specifies the objecttype allowed for this bucket; only S3 is allowed. ECS administration tool reference 201

202 Configure HDFS in a secure Hadoop cluster 202 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

203 CHAPTER 24 Troubleshooting an ECS HDFS configuration Troubleshooting Troubleshooting an ECS HDFS configuration 203

204 Troubleshooting an ECS HDFS configuration Troubleshooting This area provides workarounds for issue that may be encountered when configuring ECS HDFS. Verify AD/LDAP is correctly configured with secure Hadoop cluster You should verify that AD or LDAP is correctly set up with Kerberos (KDC) and the Hadoop cluster. When your configuration is correct, you should be able to "kinit" against an AD/LDAP user. In addition, if the Hadoop cluster is configured for local HDFS, you should check that you can list the local HDFS directory before ECS gets added to the cluster. Workaround If you cannot successfully authenticate as an AD/LDAP user with the KDC on the Hadoop cluster, you should address this before proceeding to ECS Hadoop configuration. An example of a successful login is shown below: [kcluser@lvipri054 root]$ kinit kcluser@qe.com Password for kcluser@qe.com: [kcluser@lvipri054 root]$ klist Ticket cache: FILE:/tmp/krb5cc_1025 Default principal: kcluser@qe.com Valid starting Expires Service principal 04/28/15 06:20:57 04/28/15 16:21:08 krbtgt/qe.com@qe.com renew until 05/05/15 06:20:57 If the above is not successful, you can investigate using the following checklist: Check /etc/krb5.conf on the KDC server for correctness and syntax. Realms can be case sensitive in the config files as well as when used with the kinit command. Check that /etc/krb5.conf from the KDC server is copied to all the Hadoop nodes. Check that one-way trust between AD/LDAP and the KDC server was successfully made. Refer to appropriate documentation on how to do this. Make sure that the encryption type on the AD/LDAP server matches that on the KDC server. Check that /var/kerberos/krb5kdc/kadm5.acl and /var/kerberos/ krb5kdc/kdc.conf are correct. Try logging in as a service principle on the KDC server to indicate that the KDC server itself is working correctly. Try logging in as the same AD/LDAP user on the KDC server directly. If that does not work, the issue is likely to be on the KDC server directly. 204 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

205 Troubleshooting an ECS HDFS configuration Permission denied for AD user When running an application as an AD user, a "Permission denied" error is raised. Workaround Set the permissions for the /user directory as: hdfs dfs -chmod 1777 /user Restart services after hbase configuration After editing the hbase.rootdir property in hbase-site.xml, the hbase service does not restart correctly. Workaround The following steps should be performed when this issue arises on Cloudera or Hortonworks to get hbase-master running. 1. Connect to the zookeeper cli. hbase zkcli 2. Remove the hbase directory. rmr /hbase 3. Restart the hbase service. On Cloudera restart all services. Pig test fails: unable to obtain Kerberos principal Enable Kerberos client-side logging Pig test fails with the following error: "Info:Error: java.io.ioexception: Unable to obtain the Kerberos principal" even after kinit as AD user, or with "Unable to open iterator for alias firstten". This issue is caused due to the fact that Pig (<0.13) doesn't generate a delegation token for ViPRFS as a secondary storage. Workaround Append the viprfs://bucket.ns.installation/ to the mapreduce.job.hdfs-servers configuration setting. For example: set mapreduce.job.hdfs-servers viprfs://kcdhbucktm2.s3.site1 To troubleshoot authentication issues, you can enable verbose logging on the Hadoop cluster node that you are using. Verbose logging is enabled using an environment variable that applies only to your current SSH session. export HADOOP_OPTS="-Dsun.security.krb5.debug=true" Permission denied for AD user 205

206 Troubleshooting an ECS HDFS configuration Debug Kerberos on the KDC Tail the KDC's /var/log/krb5kdc.log file when you do an HDFS operation to make it easier to debug. Eliminate clock skew tail -f /var/log/krb5kdc.log It is important to ensure that time is synchronized between client and server as Kerberos relies on time being accurate. If your AD has a clock skew with your data nodes/kdc, you will have configure its NTP server. You can do this as follows: 1. Use Remote Desktop to connect to your AD server. 2. Run the following commands: a. w32tm /config /syncfromflags:manual /manualpeerlist:<ntp-server1>,<ntpserver2> b. net stop w32time c. net start w32time 206 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

207 CHAPTER 25 ECS HDFS core site property reference Hadoop core-site.xml properties for ECS HDFS ECS HDFS core site property reference 207

208 ECS HDFS core site property reference Hadoop core-site.xml properties for ECS HDFS When configuring the Hadoop core-site.xml file, use this table as a reference for the properties and their related values. Table 37 Hadoop core-site.xml properties Property Description File system implementation properties fs.viprfs.impl <property> <name>fs.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.viprfilesystem</value> </property> fs.abstractfilesystem.vi prfs.impl <property> <name>fs.abstractfilesystem.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.viprabstractfilesystem</value> </property> Properties that define the authority section of the ECS HDFS file system URI fs.vipr.installations A comma-separated list of names. The names are further defined by the fs.vipr.installation. [installation_name].hosts property to uniquely identify sets of ECS data nodes. The names are used as a component of the authority section of the ECS HFDS file system URI. For example: <property> <name>fs.vipr.installations</name> <value><site1>,<abc>,<testsite></value> </property> fs.vipr.installation. [installation_name].host s The IP addresses of the ECS cluster's data nodes or the load balancers for each name listed in the fs.vipr.installations property. Specify the value in the form of a comma-separated list of IP addresses. For example: <property> <name>fs.vipr.installation.<site1>.hosts</name> <value> , , </value> </property> <property> <name>fs.vipr.installation.<abc>.hosts</name> <value> , , </value> </property> <property> <name>fs.vipr.installation.<testsite>.hosts</name> <value> , , </value> </property> fs.vipr.installation. [installation_name].reso lution Specifies how the ECS HDFS software knows how to access the ECS data nodes. Values are: dynamic: Use this value when accessing ECS data nodes directly without a load balancer. 208 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

209 ECS HDFS core site property reference Table 37 Hadoop core-site.xml properties (continued) Property Description fixed: Use this value when accessing ECS data nodes through a load balancer. <property> <name>fs.vipr.installation.testsite.resolution</name> <value>dynamic</value> </property> fs.vipr.installation. [installation_name].reso lution.dynamic.time_to_ live_ms When the fs.vipr.installation.[installation_name].resolution property is set to dynamic, this property specifies how often to query ECS for the list of active nodes. Values are in milliseconds. The default is 10 minutes. <property> <name>fs.vipr.installation.<testsite>.resolution.dynamic.time_to_live_ms</name> <value>600000</value> </property> ECS file system URI fs.defaultfs A standard Hadoop property that specifies the URI to the default file system. Setting this property to the ECS HDFS file system is optional. If you do not set it to the ECS HDFS file system, you must specify the full URI on each file system operation. The ECS HDFS file system URI has this format: viprfs://[bucket_name].[namespace].[installation_name] bucket_name: The name of the HDFS-enabled bucket that contains the data you want to use when you run Hadoop jobs. namespace: The tenant namespace associated with the HDFS-enabled bucket. installation_name: The name associated with the set of ECS data nodes thathadoop can use to access ECS data. The value of this property must match one of the values specified in the fs.vipr.installations property. For example: <property> <name>fs.defaultfs</name> <value>viprfs://testbucket.s3.testsite</value> </property> HBase requires that a default file system be defined. Pivotal HAWQ service hawq.vipr.endpoint This property specifies the ECS endpoint the HAWQ service uses to talk to the ECS file system. The property has this syntax: bucket_name.namespace.installation_name bucket_name: The name of the HDFS-enabled bucket that contains the data you want to use with the HAWQ service. namespace: The tenant namespace associated with the HDFS-enabled bucket. Hadoop core-site.xml properties for ECS HDFS 209

210 ECS HDFS core site property reference Table 37 Hadoop core-site.xml properties (continued) Property Description installation_name: The name associated with the set of ECS data nodes that Hadoop can use to access the ECS data. The value of this property must match one of the values specified in the fs.vipr.installations property. For example: <property> <name>hawq.vipr.endpoint</name> <value>testbucket.s3.testsite</value> </property> UMASK property fs.permissions.umaskmode This standard Hadoop property specifies how ECS HDFS should compute permissions on objects. Permissions are computed by applying a umask on the input permissions. The recommended values are: When Kerberos is present: 027 When Kerberos is not present: 022 For example: <property> <name>fs.permissions.umask-mode</name> <value>027</value> </property> Identity translation properties fs.viprfs.auth.identity_tr anslation This property specifies how the ECS HDFS client determines what Kerberos realm a particular user belongs to if one is not specified. ECS data nodes store file owners as username@realm, while Hadoop stores file owners as just the username. The possible values are: NONE: Default. Users are not mapped to a realm. FIXED_REALM: Users are mapped to the realm specified in fs.viprfs.auth.realm. CURRENT_USER_REALM: Valid when Kerberos is present. The user's realm is auto-detected, and it is the realm of the currently signed in user. In the example below, the realm is EMC.COM because sally is in the EMC.COM realm. The file ownership is changed john@emc.com. # kinit sally@emc.com # hdfs dfs -chown john /path/to/file Realms provided at the command line takes precedence over the property settings. <property> <name>fs.viprfs.auth.identity_translation </name> <value>fixed_realm</value> </property> 210 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

211 ECS HDFS core site property reference Table 37 Hadoop core-site.xml properties (continued) Property fs.viprfs.auth.realm Description The realm assigned to users when the fs.viprfs.auth.identity_translation property is set to FIXED_REALM. <property> <name>fs.viprfs.auth.realm</name> <value>my_realm</value> </property> fs.viprfs.auth.anonymou s_translation This property maps anonymously owned objects to the current user, which allows the current user to modify it. Hadoop expects objects to have an owner. However, with ECS HDFS, anyone has the ability to update anonymously owned objects. The values are: CURRENT_USER: This gives the current user the ability to read and write objects that are anonymously owned. When an object is stat'd, the user that "owns" the blob is the current Kerberos user. NONE: (Default). No mapping is performed. Anonymously owned objects have an empty owner. Some applications do not allow the current user to access anonymously owned objects. <property> <name>fs.viprfs.auth.anonymous_translation</name> <value>current_user</value> </property> Kerberos realm and service principal properties viprfs.security.principal This property specifies the ECS service principal. This property tells the KDC about the ECS service. This value is specific to your configuration. The principal name can include "_HOST" which is automatically replaced by the actual data node FQDN at run time. <property> <name>viprfs.security.principal</name> </property> For example: <property> <name>viprfs.security.principal</name> </property> Using YARN When Kerberos is enabled with ECS HDFS, YARN's Resource Manager (RM) and Node Manager (NM) must run as the same Kerberos principal. Here is an example of how you would set it up in the core-site.xml properties: fs.vipr.auth.service.yarn.principal = rm/_host@acme.com fs.vipr.auth.service.yarn.keytab = /etc/security/keytab/rm.keytab yarn.nodemanager.keytab = /etc/security/keytab/rm.keytab yarn.nodemanager.principal = rm/_host@acme.com Hadoop core-site.xml properties for ECS HDFS 211

212 ECS HDFS core site property reference yarn.resourcemanager.keytab = /etc/security/keytab/rm.keytab yarn.resourcemanager.principal = rm/_host@acme.com Sample core-site.xml for simple authentication mode This core-site.xml is an example of ECS HDFS properties for simple authentication mode. Example 2 core-site.xml <property> <name>fs.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.viprfilesystem</value> </property> <property> <name>fs.abstractfilesystem.viprfs.impl</name> <value>com.emc.hadoop.fs.vipr.viprabstractfilesystem</value> </property> <property> <name>fs.vipr.installations</name> <value>site1</value> </property> <property> <name>fs.vipr.installation.site1.hosts</name> <value> , , </value> </property> <property> <name>fs.vipr.installation.site1.resolution</name> <value>dynamic</value> </property> <property> <name>fs.vipr.installation.site1.resolution.dynamic.time_to_live_ms</ name> <value>900000</value> </property> <property> <name>fs.defaultfs</name> <value>viprfs://mybucket.mynamespace.site1/</value> </property> <property> <name>fs.viprfs.auth.anonymous_translation</name> <value>current_user</value> </property> <property> <name>fs.viprfs.auth.identity_translation</name> <value>fixed_realm</value> </property> <property> <name>fs.viprfs.auth.realm</name> <value>my.test.realm</value> </property> 212 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

213 PART 8 Access ECS Data Chapter 26, " ECS Amazon S3 Object Service API support " Chapter 27, " ECS OpenStack Swift Object Service API support " Chapter 28, " ECS EMC Atmos Object Service API support " Access ECS Data 213

214 Access ECS Data 214 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

215 CHAPTER 26 ECS Amazon S3 Object Service API support Amazon S3 API ECS Amazon S3 Object Service API support 215

216 ECS Amazon S3 Object Service API support Amazon S3 API This article describes ECS support for the Amazon S3 API. The Amazon S3 Object Service is made available on the following ports. Table 38 Amazon S3 Object Service Ports Protocol Ports HTTP 9020 HTTPS 9021 The following sections describe the support for the S3 API, the extension provided by ECS, and describe how to authenticate with the service and how to use SDKs to develop clients to access the service: S3 API Supported and Unsupported Features on page 216 API Extensions on page 219 Authenticating with the S3 service on page 224 Using SDKs to access the S3 service on page 224 Some aspects of bucket addressing and authentication are specific to ECS. If you want to configure an existing application to talk to ECS, or develop a new application that uses the S3 API to talk to ECS, you should refer to the following article: Address ECS object storage and use the Base URL on page 132 S3 API Supported and Unsupported Features ECS supports a subset of the Amazon S3 REST API. The following sections detail the supported and unsupported APIs: Supported S3 APIs on page 216 Unsupported S3 APIs on page 218 Supported S3 APIs The following table lists the supported S3 API methods. Table 39 Supported S3 APIs Feature GET service Notes ECS supports marker and max-keys parameters to enable paging of bucket list. GET /?marker=<bucket>&max-keys=<num> For example: GET /?marker=mybucket&max-keys=40 DELETE Bucket DELETE Bucket cors 216 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

217 ECS Amazon S3 Object Service API support Table 39 Supported S3 APIs (continued) Feature DELETE Bucket lifecycle Notes Only the expiration part is supported in lifecycle. Policies related to archiving (AWS Glacier) are not supported. GET Bucket (List Objects) GET Bucket cors GET Bucket acl GET Bucket lifecycle Only the expiration part is supported in lifecycle. Policies related to archiving (AWS Glacier) are not supported. GET Bucket Object versions GET Bucket versioning HEAD Bucket List Multipart Uploads PUT Bucket PUT Bucket cors PUT Bucket acl PUT Bucket lifecycle Only the expiration part is supported in lifecycle. Policies related to archiving (AWS Glacier) are not supported. PUT Bucket versioning DELETE Object Delete Multiple Objects GET Object GET Object ACL HEAD Object PUT Object Supports chunked PUT PUT Object acl PUT Object - Copy OPTIONS object Initiate Multipart Upload Upload Part Upload Part - Copy Complete Multipart Upload ECS returns an ETag of "00" for this request. This differs from the Amazon S3 response. Abort Multipart Upload List Parts S3 API Supported and Unsupported Features 217

218 ECS Amazon S3 Object Service API support Table 40 Additional features Feature Pre-signed URLs Chunked PUT Notes ECS supports the use of pre-signed URLs to enable users to be given access to objects without needing credentials. More information can be found here. PUT operation can be used to upload objects in chunks. Chunked transfer uses the Transfer-Encoding header (Transfer- Encoding: chunked) to specify that content will be transmitted in chunks. This enables content to be sent before the total size of the payload is known. Unsupported S3 APIs The following table lists the unsupported S3 API methods. 218 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

219 ECS Amazon S3 Object Service API support Table 41 Unsupported S3 APIs Feature Notes DELETE Bucket policy DELETE Bucket tagging DELETE Bucket website GET Bucket policy GET Bucket location ECS is only aware of a single virtual data center. GET Bucket logging GET Bucket notification Notification is only defined for reduced redundancy feature in S3. ECS does not currently support notifications. GET Bucket tagging GET Bucket requestpayment ECS has its own model for payments. GET Bucket website PUT Bucket policy PUT Bucket logging PUT Bucket notification Notification is only defined for reduced redundancy feature in S3. ECS does not currently support notifications. PUT Bucket tagging PUT Bucket requestpayment ECS has its own model for payments. PUT Bucket website Object APIs GET Object torrent POST Object POST Object restore This operation is related to AWS Glacier, which is not supported in ECS. API Extensions A number of extensions to the object APIs are supported. The extensions and the APIs that support them are listed in the following table. Table 42 Object API Extensions Feature PUT Object (rangeupdate) PUT Object (rangeoverwrite) Notes Uses Range header to specify object range to be updated. Updating a byte range within an object on page 220 Uses Range header to specify object range to be overwritten. API Extensions 219

220 ECS Amazon S3 Object Service API support Table 42 Object API Extensions (continued) Feature Notes Overwriting part of an object on page 221 PUT Object (rangeappend) GET Object (rangeread) Uses Range header to specify object range appended. Appending data to an object on page 222 Uses Range header to specify object byte range to read. Reading multiple byte ranges within an object on page 223 Updating a byte range within an object An example of using the ECS API extensions to update a byte range of an object is provided below. First do a GET request on the object named object1 located in bucket1 to review the object. object1 has the value The quick brown fox jumps over the lazy dog. GET /bucket1/object1 HTTP/1.1 Date: Mon, 17 Jun :04: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:9qxkiht2h7upudpf86dvgp8vdvi= Accept-Encoding: gzip, deflate, compress HTTP/ OK Date: Mon, 17 Jun :04:40 GMT Content-Type: application/octet-stream Last-Modified: Mon, 17 Jun :04:28 GMT ETag: 6 Content-Type: application/json Content-Length: 43 The quick brown fox jumps over the lazy dog. Now you want to update a specific byte range within this object. To do this, the Range header in the object data request must include the start and end offsets of the object that you want to update. The format is: Range: bytes=<startoffset>-<endoffset> In the example below, the PUT request includes the Range header with the value bytes=10-14 indicating that bytes 10,11,12,13,14 are to be replaced by the value sent in the request. Here, the new value green is being sent. PUT /bucket1/object1 HTTP/1.1 Content-Length: 5 Range: bytes=10-14 ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :15: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:xhjcayaeqansklaf+/4pdlbhyam= Accept-Encoding: gzip, deflate, compress 220 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

221 ECS Amazon S3 Object Service API support green Overwriting part of an object HTTP/ No Content ETag: 10 x-amz-id-2: object1 x-amz-request-id: 027f037c-29ea de82d0e9f52a Content-Length: 0 Date: Mon, 17 Jun :15:16 GMT When reading the object again, the new value is now The quick green fox jumps over the lazy dog. (The word brown has been replaced with green.) You have updated a specific byte range within this object. GET /bucket1/object1 HTTP/1.1 Cookie: JSESSIONID=wdit99359t8rnvipinz4tbtu ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :16: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:ogvn4z8nv5vnsailqtdpv/fcqzu= Accept-Encoding: gzip, deflate, compress HTTP/ OK Date: Mon, 17 Jun :16:00 GMT Content-Type: application/octet-stream Last-Modified: Mon, 17 Jun :15:16 GMT ETag: 10 Content-Type: application/json Content-Length: 43 The quick green fox jumps over the lazy dog. An example of using the ECS API extensions to overwrite part of an object is provided below. You can overwrite part of an object by providing only the starting offset in the data request. The data in the request will be written starting at the provided offset. The format is: Range: <startingoffset>- For example, to write the data brown cat starting at offset 10, you would issue this PUT request: PUT /bucket1/object1 HTTP/1.1 Content-Length: 9 Range: bytes=10- ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :51: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:uwpjdagmazcp5lu77zvbo+cit4q= Accept-Encoding: gzip, deflate, compress brown cat HTTP/ No Content ETag: 25 x-amz-id-2: object1 x-amz-request-id: 65be45c2-0ee8-448a-a5a0-fff82573aa3b Content-Length: 0 Date: Mon, 17 Jun :51:41 GMT API Extensions 221

222 ECS Amazon S3 Object Service API support Appending data to an object When retrieving the object, you can see the final value The quick brown cat jumps over the lazy dog and cat. (green fox has been replaced with brown cat). You have overwritten part of the data in this object at the provided starting offset. GET /bucket1/object1 HTTP/1.1 Date: Mon, 17 Jun :51: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:/uqpdxnqztydkzgbk169gzhzmt4= Accept-Encoding: gzip, deflate, compress HTTP/ OK Date: Mon, 17 Jun :51:55 GMT Content-Type: application/octet-stream Last-Modified: Mon, 17 Jun :51:41 GMT ETag: 25 Content-Type: application/json Content-Length: 51 The quick brown cat jumps over the lazy dog and cat. An example of using the ECS API extensions to append data to an object is provided below. There may be cases where you need to append to an object, but determining the exact byte offset is not efficient or useful. For this scenario, ECS provides the ability to atomically append data to the object without specifying an offset (the correct offset is returned to you in the response). A Range header with the special value bytes=-1- can be used to append data to an object. In this way, the object can be extended without knowing the existing object size. The format is: Range: bytes=-1- A sample request showing appending to an existing object using a Range value of bytes=-1-. Here the value and cat is sent in the request. PUT /bucket1/object1 HTTP/1.1 Content-Length: 8 Range: bytes=-1- ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :46: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:/sqofl65riebswlg6t8hl0dfw4c= Accept-Encoding: gzip, deflate, compress and cat HTTP/ No Content ETag: 24 x-amz-id-2: object1 x-amz-request-id: 087ac237-6ff5-43e3-b587-0c8fe5c08732 Content-Length: 0 Date: Mon, 17 Jun :46:01 GMT 222 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

223 ECS Amazon S3 Object Service API support Reading multiple byte ranges within an object When retrieving the object again, you can see the full value The quick green fox jumps over the lazy dog and cat. You have appended data to this object. GET /bucket1/object1 HTTP/1.1 ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :46: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:d8fse8joll0mtqcfmd4ng1gmdtg= Accept-Encoding: gzip, deflate, compress HTTP/ OK Date: Mon, 17 Jun :46:56 GMT Content-Type: application/octet-stream Last-Modified: Mon, 17 Jun :46:01 GMT ETag: 24 Content-Type: application/json Content-Length: 51 The quick green fox jumps over the lazy dog and cat. An example of using the ECS API extensions to read multiple byte ranges within an object is provided below. To read two specific byte ranges within the object named object1, you would issue the following GET request for Range: bytes==4-8, The read response would be for the words quick and lazy. Note The Amazon S3 API only supports one range when using the HTTP header Range for reading; ECS supports multiple byte ranges. GET /bucket1/object1 HTTP/1.1 Date: Mon, 17 Jun :51: x-emc-namespace: emc Range: bytes==4-8,41-44 Content-Type: application/octet-stream Authorization: AWS wuser1:/uqpdxnqztydkzgbk169gzhzmt4= Accept-Encoding: gzip, deflate, compress HTTP/ Partial Content Date: Mon, 17 Jun :51:55 GMT Content-Type: multipart/byteranges;boundary=bound04acf7f0ae3ccc Last-Modified: Mon, 17 Jun :51:41 GMT Content-Length: bound04acf7f0ae3ccc Content-Type: application/octet-stream Content-Range: bytes 4-8/50 quick --bound04acf7f0ae3ccc Content-Type: application/octet-stream Content-Range: bytes 41-44/50 lazy --bound04acf7f0ae3ccc-- API Extensions 223

224 ECS Amazon S3 Object Service API support Authenticating with the S3 service Authenticating with the Amazon S3 API is described in the Amazon S3 documentation referenced below. This topic identifies any ECS-specific aspects of the authentication process. Authenticating against a ECS system with the Amazon S3 API is described in the sections that follow. The complete algorithm for S3 authentication is described in the Amazon S3 documentation. Amazon S3 uses an authorization header that must be present in all requests to identify the user and provide a signature for the request. When calling Amazon the header has the following format: Authorization: AWS <AWSAccessKeyId>:<Signature> In ECS, the AWSAccessKeyId maps to the ECS user id (UID). An AWS access key ID has 20 characters (some S3 clients, such as the S3 Browser, check this), but ECS data service does not have this limitation. The signature is calculated from elements of the request and the user's Secret Key as detailed in the Amazon S3 documentation: The following notes apply: Using SDKs to access the S3 service In the ECS object data service, the UID can be configured (through the ECS API or the ECS UI) with 2 secret keys. The ECS data service will try to use the first secret key, and if the calculated signature does not match, it will try to use the second secret key. If the second key fails, it will reject the request. When users add or change the secret key, they should wait 2 minutes so that all data service nodes can be refreshed with the new secret key before using the new secret key. In the ECS data service, namespace is also taken into HMAC signature calculation. When developing applications that talk to the ECS S3 service, there are a number of SDKs that will support your development activity. The EMC Community provides information on the various clients that are available and provides guidance on their use: EMC ViPR Developer Resources. The following topics describe the use of the Amazon S3 SDK and the use of the EMC ECS Java SDK. Using the Java Amazon SDK on page 224 Java SDK client for ECS on page 226 Note If you want to make use of the ECS API Extensions (see ), support for these extensions is provided in the EMC ECS Java SDK. If you do not need support for the ECS extensions, or you have existing applications that use it, you can use the Amazon Java SDK. Using the Java Amazon SDK 224 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation You can access ECS object storage using the Java S3 SDK. By default the AmazonS3Client client object is coded to work directly against amazon.com. This section shows how to set up the AmazonS3Client to work against ECS.

225 ECS Amazon S3 Object Service API support In order to create an instance of the AmazonS3Client object, you need to pass it credentials. This is achieved through creating an AWSCredentials object and passing it the AWS Access Key (your ECS username) and your generated secret key for ECS. The following code snippet shows how to set this up. AmazonS3Client client = new AmazonS3Client(new BasicAWSCredentials(uid, secret)); By default the Amazon client will attempt to contact Amazon WebServices. In order to override this behavior and contact ECS you need to set a specific endpoint. You can set the endpoint using the setendpoint method. The protocol specified on the endpoint dictates whether the client should be directed at the HTTP port (9020) or the HTTPS port (9021). Note If you intend to use the HTTPS port, the JDK of your application must be set up to validate the ECS certificate successfully; otherwise the client will throw SSL verification errors and fail to connect. In the snippet below, the client is being used to access ECS over HTTP: AmazonS3Client client = new AmazonS3Client(new BasicAWSCredentials(uid, secret)); client.setendpoint(" When using path-style addressing ( ecs1.emc.com/mybucket ), you will need to set the setpathstyleaccess option, as shown below: S3ClientOptions options = new S3ClientOptions(); options.setpathstyleaccess(true); AmazonS3Client client = new AmazonS3Client(new BasicAWSCredentials(uid, secret)); client.setendpoint(" client.sets3clientoptions(options); The following code shows how to list objects in a bucket. ObjectListing objects = client.listobjects("mybucket"); for (S3ObjectSummary summary : objects.getobjectsummaries()) { System.out.println(summary.getKey()+ " "+summary.getowner()); } The CreateBucket operation differs from other operations in that it expects a region to be specified. Against S3 this would indicate the datacenter in which the bucket should be created. However, ECS does not support regions. For this reason, when calling the CreateBucket operation, we specify the standard region, which stops the AWS client from downloading the Amazon Region configuration file from Amazon CloudFront. client.createbucket("mybucket", "Standard"); The complete example for communicating with the ECS S3 data service, creating a bucket, and then manipulating an object is provided below: public class Test { public static String uid = "root"; public static String secret = "KHBkaH0Xd7YKF43ZPFbWMBT9OP0vIcFAMkD/9dwj"; Using SDKs to access the S3 service 225

226 ECS Amazon S3 Object Service API support public static String viprdatanode = " public static String bucketname = "mybucket"; public static File objectfile = new File("/photos/cat1.jpg"); public static void main(string[] args) throws Exception { AmazonS3Client client = new AmazonS3Client(new BasicAWSCredentials(uid, secret)); S3ClientOptions options = new S3ClientOptions(); options.setpathstyleaccess(true); AmazonS3Client client = new AmazonS3Client(credentials); client.setendpoint(viprdatanode); client.sets3clientoptions(options); client.createbucket(bucketname, "Standard"); listobjects(client); client.putobject(bucketname, objectfile.getname(), objectfile); listobjects(client); client.copyobject(bucketname,objectfile.getname(),bucketname, "copy-" + objectfile.getname()); listobjects(client); } public static void listobjects(amazons3client client) { ObjectListing objects = client.listobjects(bucketname); for (S3ObjectSummary summary : objects.getobjectsummaries()) { System.out.println(summary.getKey()+ " "+summary.getowner()); } } } Java SDK client for ECS The ECS Java SDK builds on the Amazon S3 Java SDK and supports the ECS API extensions. An example of using the ViPRS3client is shown below. package com.emc.ecs.sample; import com.amazonaws.util.stringinputstream; import com.emc.vipr.services.s3.viprs3client; public class BucketCreate { private ViPRS3Client s3; public BucketCreate() { URI endpoint = new URI( ); String accesskey = fred@yourco.com ; String secretkey = pcqq20rdi2dhzoiwnkaug3wk4xjp9sqnzqbqjev3 ; BasicAWSCredentials creds = new BasicAWSCredentials(accessKey, secretkey); ViPRS3Client client = new ViPRS3Client(endpoint, creds); } 226 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

227 ECS Amazon S3 Object Service API support public static void main(string[] args) throws Exception { BucketCreate instance = new BucketCreate(); instance.runsample(); } public void runsample() { String bucketname="mybucket"; String key1 = "test1.txt"; String content = "Hello World!"; try { s3.createbucket(bucketname); s3.putobject(bucketname, key1, new StringInputStream(content), null); } catch (Exception e) { } } } Using SDKs to access the S3 service 227

228 ECS Amazon S3 Object Service API support 228 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

229 CHAPTER 27 ECS OpenStack Swift Object Service API support OpenStack Swift API ECS OpenStack Swift Object Service API support 229

230 ECS OpenStack Swift Object Service API support OpenStack Swift API ECS includes support for the OpenStack Swift API. This article describes the supported operations and describes the mechanisms for authorization and authentication. The OpenStack Swift Service is made available on the following ports. Table 43 OpenStack Swift Object Service Ports Protocol Ports HTTP 9024 HTTPS 9025 The following sections describe supported methods, the ECS extensions, and the mechanism for authentication: OpenStack Swift supported operations on page 230 API Extensions on page 231 OpenStack Version 1 authentication on page 236 OpenStack Version 2 authentication on page 237 Authorization on Container on page 239 Examples showing the use of the OpenStack Swift API can be found here: OpenStack API Examples OpenStack Swift supported operations The following sections list the OpenStack REST API requests that are supported by ECS. Supported OpenStack Swift calls on page 230 Unsupported OpenStack Swift calls on page 231 This information is taken from the Object Storage API V1 section of the OpenStack API Reference documentation. Supported OpenStack Swift calls The following OpenStack Swift REST API calls are supported in ECS. Table 44 OpenStack Swift supported calls Method Path Description GET v1/{account} Retrieve a list of existing storage containers ordered by names. GET v1/{account}/{container} Retrieve a list of objects stored in the container. PUT v1/{account}/{container} Create a container. DELETE v1/{account}/{container} Delete an empty container. POST v1/{account}/{container} Create or update the arbitrary container metadata by associating custom metadata headers with the container level URI. These headers must take the format X-Container-Meta-*. 230 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

231 ECS OpenStack Swift Object Service API support Table 44 OpenStack Swift supported calls (continued) Method Path Description HEAD v1/{account}/{container} Retrieve the container metadata. Currently does not include object count and bytes used. User requires administrator privileges. GET PUT DELETE HEAD POST v1/{account}/{container}/ {object} v1/{account}/{container}/ {object} v1/{account}/{container}/ {object} v1/{account}/{container}/ {object} v1/{account}/{container}/ {object} Retrieve the object's data. Write, or overwrite, an object's content and metadata. Used to copy existing object to another object using X- Copy-From header to designate source. Remove an object from the storage system permanently. In combination with the COPY command you can use COPY then DELETE to effectively move an object. Retrieve object metadata and other standard HTTP headers. Set and overwrite arbitrary object metadata. These metadata must take the format X-Object-Meta-*. X- Delete-At or X-Delete-After for expiring objects can also be assigned by this operation. But other headers such as Content-Type cannot be changed by this operation. Unsupported OpenStack Swift calls The following OpenStack Swift REST API calls are not supported in ECS. Table 45 OpenStack Swift unsupported calls Method Path Description HEAD v1/{account} Retrieve the account metadata, for example, the number of containers, the total bytes stored in OpenStackObject Storage for the account/tenant. POST v1/{account} Create or update an account metadata by associating custom metadata headers with the account level URI. These headers must take the format X-Account-Meta-*. COPY v1/{account}/{container}/ {object} Copy is supported using PUT v1/{account}/ {container}/{object} with X-Copy-From header. API Extensions A number of extensions to the object APIs are supported. The extensions and the APIs that support them are listed in the following table. API Extensions 231

232 ECS OpenStack Swift Object Service API support Table 46 Object API Extensions Feature Notes API Support PUT Object (range-update) PUT Object (range-overwrite) Uses Range header to specify object range to be updated. Updating a byte range within an object on page 220 Uses Range header to specify object range to be overwritten. Overwriting part of an object on page 221 S3/Atmos/Swift S3/Atmos/Swift PUT Object (range-append) GET Object (range-read) Uses Range header to specify object range appended. Appending data to an object on page 222 Uses Range header to specify object byte range to read. Reading multiple byte ranges within an object on page 223 S3/Atmos/Swift S3/Atmos/Swift Updating a byte range within an object An example of using the ECS API extensions to update a byte range of an object is provided below. First do a GET request on the object named object1 located in bucket1 to review the object. object1 has the value The quick brown fox jumps over the lazy dog. GET /bucket1/object1 HTTP/1.1 Date: Mon, 17 Jun :04: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:9qxkiht2h7upudpf86dvgp8vdvi= Accept-Encoding: gzip, deflate, compress HTTP/ OK Date: Mon, 17 Jun :04:40 GMT Content-Type: application/octet-stream Last-Modified: Mon, 17 Jun :04:28 GMT ETag: 6 Content-Type: application/json Content-Length: 43 The quick brown fox jumps over the lazy dog. Now you want to update a specific byte range within this object. To do this, the Range header in the object data request must include the start and end offsets of the object that you want to update. The format is: Range: bytes=<startoffset>-<endoffset> 232 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

233 ECS OpenStack Swift Object Service API support In the example below, the PUT request includes the Range header with the value bytes=10-14 indicating that bytes 10,11,12,13,14 are to be replaced by the value sent in the request. Here, the new value green is being sent. PUT /bucket1/object1 HTTP/1.1 Content-Length: 5 Range: bytes=10-14 ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :15: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:xhjcayaeqansklaf+/4pdlbhyam= Accept-Encoding: gzip, deflate, compress green Overwriting part of an object HTTP/ No Content ETag: 10 x-amz-id-2: object1 x-amz-request-id: 027f037c-29ea de82d0e9f52a Content-Length: 0 Date: Mon, 17 Jun :15:16 GMT When reading the object again, the new value is now The quick green fox jumps over the lazy dog. (The word brown has been replaced with green.) You have updated a specific byte range within this object. GET /bucket1/object1 HTTP/1.1 Cookie: JSESSIONID=wdit99359t8rnvipinz4tbtu ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :16: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:ogvn4z8nv5vnsailqtdpv/fcqzu= Accept-Encoding: gzip, deflate, compress HTTP/ OK Date: Mon, 17 Jun :16:00 GMT Content-Type: application/octet-stream Last-Modified: Mon, 17 Jun :15:16 GMT ETag: 10 Content-Type: application/json Content-Length: 43 The quick green fox jumps over the lazy dog. An example of using the ECS API extensions to overwrite part of an object is provided below. You can overwrite part of an object by providing only the starting offset in the data request. The data in the request will be written starting at the provided offset. The format is: Range: <startingoffset>- For example, to write the data brown cat starting at offset 10, you would issue this PUT request: PUT /bucket1/object1 HTTP/1.1 Content-Length: 9 Range: bytes=10- ACCEPT: application/json,application/xml,text/html,application/octetstream API Extensions 233

234 ECS OpenStack Swift Object Service API support Appending data to an object Date: Mon, 17 Jun :51: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:uwpjdagmazcp5lu77zvbo+cit4q= Accept-Encoding: gzip, deflate, compress brown cat HTTP/ No Content ETag: 25 x-amz-id-2: object1 x-amz-request-id: 65be45c2-0ee8-448a-a5a0-fff82573aa3b Content-Length: 0 Date: Mon, 17 Jun :51:41 GMT When retrieving the object, you can see the final value The quick brown cat jumps over the lazy dog and cat. (green fox has been replaced with brown cat). You have overwritten part of the data in this object at the provided starting offset. GET /bucket1/object1 HTTP/1.1 Date: Mon, 17 Jun :51: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:/uqpdxnqztydkzgbk169gzhzmt4= Accept-Encoding: gzip, deflate, compress HTTP/ OK Date: Mon, 17 Jun :51:55 GMT Content-Type: application/octet-stream Last-Modified: Mon, 17 Jun :51:41 GMT ETag: 25 Content-Type: application/json Content-Length: 51 The quick brown cat jumps over the lazy dog and cat. An example of using the ECS API extensions to append data to an object is provided below. There may be cases where you need to append to an object, but determining the exact byte offset is not efficient or useful. For this scenario, ECS provides the ability to atomically append data to the object without specifying an offset (the correct offset is returned to you in the response). A Range header with the special value bytes=-1- can be used to append data to an object. In this way, the object can be extended without knowing the existing object size. The format is: Range: bytes=-1- A sample request showing appending to an existing object using a Range value of bytes=-1-. Here the value and cat is sent in the request. PUT /bucket1/object1 HTTP/1.1 Content-Length: 8 Range: bytes=-1- ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :46: x-emc-namespace: emc Content-Type: application/octet-stream 234 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

235 ECS OpenStack Swift Object Service API support Authorization: AWS wuser1:/sqofl65riebswlg6t8hl0dfw4c= Accept-Encoding: gzip, deflate, compress and cat Reading multiple byte ranges within an object HTTP/ No Content ETag: 24 x-amz-id-2: object1 x-amz-request-id: 087ac237-6ff5-43e3-b587-0c8fe5c08732 Content-Length: 0 Date: Mon, 17 Jun :46:01 GMT When retrieving the object again, you can see the full value The quick green fox jumps over the lazy dog and cat. You have appended data to this object. GET /bucket1/object1 HTTP/1.1 ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :46: x-emc-namespace: emc Content-Type: application/octet-stream Authorization: AWS wuser1:d8fse8joll0mtqcfmd4ng1gmdtg= Accept-Encoding: gzip, deflate, compress HTTP/ OK Date: Mon, 17 Jun :46:56 GMT Content-Type: application/octet-stream Last-Modified: Mon, 17 Jun :46:01 GMT ETag: 24 Content-Type: application/json Content-Length: 51 The quick green fox jumps over the lazy dog and cat. An example of using the ECS API extensions to read multiple byte ranges within an object is provided below. To read two specific byte ranges within the object named object1, you would issue the following GET request for Range: bytes==4-8, The read response would be for the words quick and lazy. Note The Amazon S3 API only supports one range when using the HTTP header Range for reading; ECS supports multiple byte ranges. GET /bucket1/object1 HTTP/1.1 Date: Mon, 17 Jun :51: x-emc-namespace: emc Range: bytes==4-8,41-44 Content-Type: application/octet-stream Authorization: AWS wuser1:/uqpdxnqztydkzgbk169gzhzmt4= Accept-Encoding: gzip, deflate, compress HTTP/ Partial Content Date: Mon, 17 Jun :51:55 GMT Content-Type: multipart/byteranges;boundary=bound04acf7f0ae3ccc Last-Modified: Mon, 17 Jun :51:41 GMT Content-Length: bound04acf7f0ae3ccc API Extensions 235

236 ECS OpenStack Swift Object Service API support OpenStack Version 1 authentication Content-Type: application/octet-stream Content-Range: bytes 4-8/50 quick --bound04acf7f0ae3ccc Content-Type: application/octet-stream Content-Range: bytes 41-44/50 lazy --bound04acf7f0ae3ccc-- To communicate with ECS Procedure 1. Acquire a UID and password from ECS. The UID will be an LDAP or Active Directory name. Call the following ECS REST API to generate a password. ECS through the OpenStack Swift API, follow these steps: Request: PUT /object/user-password/myuser@emc.com <user_password_create> <password>mypassword</password> <namespace>emc_namespace</namespace> </user_password_create> Response: HTTP Call the OpenStack authentication REST API shown below. Use port 9024 for HTTP, or port 9025 for HTTPS. Request: GET /auth/v1.0 X-Auth-User: myuser@emc.com X-Auth-Key: mypassword Response: HTTP/ No Content Date: Mon, 12 Nov :32:21 GMT Server: Apache X-Storage-Url: X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cbb Content-Length: 0 Results If the UID and password are validated by ECS, the storage URL and token are returned in the response header. Further requests are authenticated by including this token. The storage URL provides the host name and resource address. You can access containers and objects by providing the following X-Storage-Url header: X-Storage-Url: EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

237 ECS OpenStack Swift Object Service API support The generated token expires 24 hours after creation. If you repeat the authentication request within the 24 hour period using the same UID and password, OpenStack will return the same token. Once the 24 hour expiration period expires, OpenStack will return a new token. In the following simple authentication example, the first REST call returns an X-Auth- Token. The second REST call uses that X-Auth-Token to perform a GET request on an account. $ curl -i -H "X-Storage-User: tim_250@sanity.local" -H "X-Storage- Pass: 1fO9X3xyrVhfcokqy3U1UyTY029gha5T+k+vjLqS" HTTP/ No Content X-Storage-Url: X-Auth-Token: 8cf4a4e943f94711aad1c91a08e98435 Server: Jetty(7.6.4.v ) $ curl -v -X GET -s -H "X-Auth-Token: 8cf4a4e943f94711aad1c91a08e98435" ecs.yourco.com:9024/v1/s3 * About to connect() to ecs.yourco.com port 9024 (#0) * Trying * Adding handle: conn: 0x7f c00 * Adding handle: send: 0 * Adding handle: recv: 0 * Curl_addHandleToPipeline: length: 1 * - Conn 0 (0x7f c00) send_pipe: 1, recv_pipe: 0 * Connected to ecs.yourco.com ( ) port 9024 (#0) > GET /v1/s3 HTTP/1.1 > User-Agent: curl/ > Host: ecs.yourco.com:9024 > Accept: */* > X-Auth-Token: 8cf4a4e943f94711aad1c91a08e98435 > < HTTP/ No Content < Date: Mon, 16 Sep :31:45 GMT < Content-Type: text/plain * Server Jetty(7.6.4.v ) is not blacklisted < Server: Jetty(7.6.4.v ) < * Connection #0 to host ecs.yourco.com left intact OpenStack Version 2 authentication ECS includes limited support for OpenStack Version 2 (Keystone) authentication. Before you begin OpenStack V2 introduces unscoped tokens. These can be used to query tenant information. An unscoped token along with tenant information can be used to query a scoped token. A scoped token and a service endpoint can be used to authenticate with ECS as described in the previous section describing V1 authentication. The two articles listed below provide important background information. OpenStack Keystone Workflow and Token Scoping OpenStack Version 2 authentication 237

238 ECS OpenStack Swift Object Service API support Authenticate for Admin API Procedure 1. Retrieve an unscoped token. curl -v -X POST -H 'ACCEPT: application/json' -H "Content-Type: application/json" -d '{"auth": {"passwordcredentials" : {"username" : "swift_user", "password" : "123"}}}' The response looks like the following. The unscoped token is preceded by id. {"access": {"token": {"id":"d668b72a011c4edf960324ab2e87438b","expires":" "l },"user": {"name": "sysadmin", "roles":[ ], "role_links": [ ] },"servicecatalog":[ ] }}, } 2. Retrieve tenant info associated with the unscoped token. curl -v -H 'X-Auth-Token: d668b72a011c4edf960324ab2e87438b' The response looks like the following. {"tenants_links":[], "tenants": [{"description":"s3","enabled":true, "name": "s3"}]} 3. Retrieve scoped token along with storageurl. curl -v -X POST -H 'ACCEPT: application/json' -H "Content-Type: application/json" -d '{"auth": {"tenantname" : "s3", "token":{"id" : d668b72a011c4edf960324ab2e87438b"}}}' v2.0/tokens An example response follows. The scoped token is preceded by id. {"access":{"token":{"id":"baf0709e30ed4b138c5db6767ba76a4e ","expires":" ","tenant": {"description":"s3","enabled":true,"name":"s3"}}, "user":{"name":"swift_admin","roles":[{"name":"member"}, {"name":"admin"}],"role_links":[]}, "servicecatalog":[{"type":"object-store", "name":"swift","endpoints_links":[],"endpoint":[{"internalurl": " :9024/v1/s3"}]}]}} 4. Use the scoped token and service endpoint URL for swift authentication. This step is the same as in V1 of OpenStack. curl -v -H "X-Auth-Token: baf0709e30ed4b138c5db6767ba76a4e" EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

239 ECS OpenStack Swift Object Service API support :9024/v1/s3/{container}/{object} Authorization on Container OpenStack Swift authorization targets only containers. Swift currently supports two types of authorization: Referral style authorization Group style authorization. ECS 2.0 supports only group-based authorization. Admin users can perform all operations within the account. Non-admin users can only perform operations per container based on the container's X-Container-Read and X- Container-Write Access Control Lists. The following operations can be granted to nonadmin users: Admin assigns read access to the container curl -XPUT -v -H 'X-Container-Read: {GROUP LIST}' -H 'X-Auth-Token: {TOKEN}' This command allows users belonging to the GROUP LIST to have read access rights to container1. After read permission is granted, users belongs to target group(s) can perform below operations: HEAD container - Retrieve container metadata. Only allowed if user is assigned to group that has Tenant Administrator privileges. GET container - List objects within a container GET objects with container - Read contents of the object within the container Admin assigns write access to the container curl -XPUT -v -H 'X-Container-Write: {GROUP LIST}' -H 'X-Auth-Token: {TOKEN}' The users in the group GROUP LIST are granted write permission. Once write permission is granted, users belongs to target group(s) can perform following operations: POST container - Set metadata. Start with prefix "X-Container-Meta". PUT objects within container - Write/override objects with container. Additional information on authorization can be found in: Container Operations Authorization on Container 239

240 ECS OpenStack Swift Object Service API support 240 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

241 CHAPTER 28 ECS EMC Atmos Object Service API support EMC Atmos API Supported Features Supported EMC Atmos REST API Calls Unsupported EMC Atmos REST API Calls Subtenant Support in EMC Atmos REST API Calls API Extensions ECS EMC Atmos Object Service API support 241

242 ECS EMC Atmos Object Service API support EMC Atmos API Supported Features ECS supports a subset of the EMC Atmos API. This article lists the supported operations and the ECS extensions. The EMC Atmos Object Service is made available on the following ports. Table 47 EMC Atmos Object Service Ports Protocol Ports HTTP 9022 HTTPS 9023 More information on the supported operations can be found in the Atmos Programmer s Guide which is available from EMC Supportzone. Atmos Programmer's Guide Wire format compatibility is provided for all supported operations. Therefore, the operations described in the Atmos Programmer's Guide apply to the API operations exposed by ECS. The Atmos Programmer's Guide also provides information on authenticating with the Atmos API and provides comprehensive examples for many of the supported features. Supported EMC Atmos REST API Calls ECS supports a subset of the EMC Atmos API. The following Atmos REST API calls are supported. Calls for the object and namespace interfaces are shown. Table 48 Supported Atmos REST API calls Method Path Description Service Operations GET /rest/service Get information about the system Object Operations POST DELETE PUT GET /rest/objects /rest/namespace/<path> /rest/objects/<objectid> /rest/namespace/<path> /rest/objects/<objectid> /rest/namespace/<path> /rest/objects/<objectid> /rest/namespace/<path> Create an object (See notes below) Delete object Update object (See notes below) Read object (or directory list) POST /rest/namespace/<path>?rename Rename an object 242 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

243 ECS EMC Atmos Object Service API support Table 48 Supported Atmos REST API calls (continued) Method Path Description MetaData Operations GET POST DELETE GET GET POST GET GET Head /rest/objects/<objectid>?metadata/user /rest/namespace/<path>?metadata/user /rest/objects/<objectid>?metadata/user /rest/namespace/<path>?metadata/user /rest/objects/<objectid>?metadata/user /rest/namespace/<path>?metadata/user /rest/objects/<objectid>?metadata/system /rest/namespace/<path>?metadata/system /rest/objects/<objectid>?acl /rest/namespace/<path>?acl /rest/objects/<objectid>?acl /rest/namespace/<path>?acl /rest/objects/<objectid>?metadata/tags /rest/namespace/<path>?metadata/tags /rest/objects/<objectid>?info /rest/namespace/<path>?info /rest/objects/<objectid> /rest/namespace/<path> Get user metadata for an object Set user metadata Delete user metadata Get system metadata for an object Get ACL Set ACL Get metadata tags for an object Get object info Get all object metadata Object-space Operations GET /rest/objects List objects GET /rest/objects?listabletags Get listable tags Note The x-emc-wschecksum header is not supported. HTML form upload is not supported. GET /rest/objects does not support different response types with x-emcaccept. For example, text/plain is not supported. Expiration and retention of objects is not supported. Unsupported EMC Atmos REST API Calls The following Atmos REST API calls are not supported. Unsupported EMC Atmos REST API Calls 243

244 ECS EMC Atmos Object Service API support Table 49 Unsupported Atmos REST API calls Method Path Description Object Versioning POST /rest/objects/<objectid>?versions Create a version of an object DELETE /rest/objects/<objectid>?versions Delete an object version GET /rest/objects/<objectid>?versions List versions of an object PUT /rest/objects/<objectid>?versions Restore object version Anonymous Access GET /rest/objects/<objectid>? uid=<uid>&expires=<exp>&signature=<sig> /rest/namespace/<path>? uid=<uid>&expires=<exp>&signature=<sig> Shareable URL POST /rest/accesstokens Create an access token GET /rest/accesstokens/<token_id>?info Get access token detail DELETE /rest/accesstokens/<token_id> Delete access token GET /rest/accesstokens List access tokens GET /rest/accesstokens/<token_id> Download content anonymously Note Checksum protection using the x-emc-wschecksum custom header is not supported. Subtenant Support in EMC Atmos REST API Calls ECS includes two native REST API calls that are specifically to add ECS subtenant support to Atmos applications. These calls are as follows: API Call Subtenant create Subtenant delete Example PUT Http url: /rest/subtenant Required headers: x-emc-uid (for example, x-emcuid=wuser1@sanity.local ) x-emc-signature. The subtenantid is set in the header "subtenantid" of the response. DELETE Http url: /rest/subtenants/{subtenantid} Required headers: x-emc-uid (for example, x-emcuid=wuser1@sanity.local ) x-emc-signature 244 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

245 ECS EMC Atmos Object Service API support API Extensions A number of extensions to the object APIs are supported. The extensions and the APIs that support them are listed in the following table. Table 50 Object API Extensions Feature PUT Object (rangeappend) Notes Uses Range header to specify object range appended. Appending data to an object on page 245 Appending data to an object An example of using the ECS API extensions to append data to an object is provided below. There may be cases where you need to append to an object, but determining the exact byte offset is not efficient or useful. For this scenario, ECS provides the ability to atomically append data to the object without specifying an offset (the correct offset is returned to you in the response). A Range header with the special value bytes=-1- can be used to append data to an object. In this way, the object can be extended without knowing the existing object size. The format is: Range: bytes=-1- A sample request showing appending to an existing object using a Range value of bytes=-1-. Here the value and cat is sent in the request. PUT /rest/namespace/myobject HTTP/1.1 Content-Length: 8 Range: bytes=-1- ACCEPT: application/json,application/xml,text/html,application/octetstream Date: Mon, 17 Jun :46: x-emc-date: Mon, 17 Jun :46: x-emc-namespace: emc x-emc-uid: fa4e31a68d3e4929bec2f964d4cac3de/wuser1@sanity.local x-emc-signature: ZpW9KjRb5+YFdSzZjwufZUqkExc= Content-Type: application/octet-stream Accept-Encoding: gzip, deflate, compress and cat HTTP/ OK x-emc-mtime: Date: Mon, 17 Jun :46:01 GMT x-emc-policy: default x-emc-utf8: true x-emc-request-id: 0af9ed8d:14cc314a9bc:112f6:9 x-emc-delta: 8 x-emc-append-offset: 24 Content-Length: 0 Server: Jetty(7.6.4.v ) The offset position at which the data was appended is returned in the x-emc-appendoffset header. API Extensions 245

246 ECS EMC Atmos Object Service API support When retrieving the object again, you can see the full value The quick green fox jumps over the lazy dog and cat. You have appended data to this object. 246 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

247 PART 9 Use the REST API Chapter 29, " Use the ECS Management REST API " Use the REST API 247

248 Use the REST API 248 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

249 CHAPTER 29 Use the ECS Management REST API Introduction Authenticate with the ECS Management REST API ECS Management REST API summary Use the ECS Management REST API 249

250 Use the ECS Management REST API Introduction This article describes how to access the ECS Management REST API, it describes how to authenticate with it, and provides a summary of the API paths. The ECS Management REST API enables the object store to be configured and managed. Once the object store is configured, subsequent object create, read, update, and delete operations are performed using the supported S3, EMC Atmos, or OpenStack Swift object protocols. You can refer to the following topic to get an understanding of how to access the REST API and how to authenticate: Authenticate with the ECS Management REST API on page 250 and the API paths are summarized in: ECS Management REST API summary on page 253 In addition, an API Reference is provided in: ECS Management REST API Reference The ECS Management REST API Reference is auto-generated from the source code and provides a reference for the methods available in the API. Authenticate with the ECS Management REST API Authenticate with AUTH-TOKEN ECS uses a token-based authentication system for all its REST API calls. Examples are provided for authentication with the ECS REST API, with cookies and without cookies. Once a user is authenticated against ECS, an authentication token is returned and can be used to authenticate the user in subsequent calls. An HTTP 401 code is returned if the client is automatically following redirects, indicating that you need to login and authenticate to obtain a new token. An HTTP 302 code is returned if the client is not automatically following redirects. The 302 code directs the client to where to get re-authenticated. You can retrieve and use authentication tokens by: Saving the X-SDS-AUTH-TOKEN cookie from a successful authentication request and sending that cookie along in subsequent requests. Reading the X-SDS-AUTH-TOKEN HTTP header from a successful authentication request and copying that header into any subsequent request. The REST API is available on port :4443 and clients access ECS by issuing a login request in the form: EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation The ECS Management REST API Reference provides a description and complete list of parameters for the REST API methods used in this article. This example shows how to use authentication tokens by reading the X-SDS-AUTH-TOKEN http header from a successful authentication request and copying that header into a subsequent request. This example does not use cookies. The examples here are written in curl and formatted for readability. This command executes a GET on the /login resource. The -u option indicates the user of basic authentication header. The user designation must be included in the request. Upon

251 Use the ECS Management REST API successful authentication, a HTTP 200 code is returned as well as the X-SDS-AUTH-TOKEN header containing the encoded token. curl -L --location-trusted -k -u "root:changeme" -v > GET /login HTTP/1.1 > Authorization: Basic cm9vddpdagfuz2vnzq== > User-Agent: curl/ (i386-pc-win32) libcurl/ OpenSSL/ 0.9.8t zlib/1.2.5 > Host: :4443 > Accept: */* > < HTTP/ OK < Date: Tue, 26 Nov :18:25 GMT < Content-Type: application/xml < Content-Length: 93 < Connection: keep-alive < X-SDS-AUTH-TOKEN: BAAcQ0xOd3g0MjRCUG4zT3NJdnNuMlAvQTFYblNrPQMAUAQADTEzODU0OTQ4NzYzNTICAA EABQA5dXJu OnN0b3JhZ2VvczpUb2tlbjo2MjIxOTcyZS01NGUyLTRmNWQtYWZjOC1kMGE3ZDJmZDU3Mm U6AgAC0A8= < <?xml version="1.0" encoding="utf-8" standalone="yes"?> <loggedin> <user>root</user> </loggedin> * Connection #0 to host left intact * Closing connection #0 * SSLv3, TLS alert, Client hello (1): The token can then be passed back in the next API call. You can copy the X-SDS-AUTH- TOKEN contents and pass it to the next request through curl's -H switch. curl -k -H "X-SDS-AUTH-TOKEN: BAAcOHZLaGF4MTl3eFhpY0czZ0tWUGhJV2xreUE4PQMAUAQADTEzODU0OTQ4NzYzNTICAA EABQA5dXJu OnN0b3JhZ2VvczpUb2tlbjpkYzc3ODU3Mi04NWRmLTQ2YjMtYjgwZi05YTdlNDFkY2QwZD g6agac0a8=" <?xml version="1.0" encoding="utf-8" standalone="yes"?> <namespaces> <namespace> <id>ns1</id> <link rel="self" href="/object/namespaces/namespace/ns1"/> <names>ns1</name> </namespace> </namespaces> Authenticate with AUTH-TOKEN 251

252 Use the ECS Management REST API Authenticate with cookies This example shows how to use authentication tokens by saving the cookie from a successful authentication request, then passing the cookie in a subsequent request. The examples here are written in curl and formatted for readability. In this example, you specify the?using-cookies=true parameter to indicate that you want to receive cookies in addition to the normal HTTP header. This curl command saves the authentication token to a file named cookiefile in the current directory. curl -L --location-trusted -k -u "root:password" -c cookiefile -v The next command passes the cookie with the authentication token through the -b switch, and returns the user's tenant information. curl -k -b cookiefile - v <?xml version="1.0" encoding="utf-8" standalone="yes"?> <namespaces> <namespace> <id>ns1</id> <link rel="self" href="/object/namespaces/namespace/ns1"/> <names>ns1</name> </namespace> </namespaces> Logout The logout API ends a session. A given user is allowed a maximum of 100 concurrent authentication tokens. Past this limit, the system refuses any new connection for this user until tokens free up. They can free up by expiring naturally, or by explicitly calling this URI: If you have multiple sessions running simultaneously, this URI forces the termination of all tokens related to the current user. GET An example logout request follows. Request GET X-SDS-AUTH-TOKEN:{Auth_Token} Pass in the header or cookie with the authentication token to logout. Response HTTP EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

253 Use the ECS Management REST API Whoami An ECS user can view their own user name, tenant association, and roles using the whoami API call. Request GET Response HTTP 200 GET /user/whoami <user> <common_name>root</common_name> <distinguished_name/> <namespace/> <roles> <role>system_admin</role> </roles> </user> HTTP 200 GET /user/whoami <user> <distinguished_name/> <namespace>ns1</namespace> <roles> <role>namespace_admin</role> </roles> </user> This example shows the whoami output for the root user and for a user who has been assigned to the NAMESPACE_ADMIN role for the "ns1" namespace. ECS Management REST API summary The ECS Management REST API enables the ECS object store to be configured and managed. In addition, other ECS services can be used to support the use of ECS object storage. The following table summarizes the ECS Management REST API. Table 51 ECS Management REST API- methods summary API Area Authentication Provider Billing Description /vdc/admin/authproviders API to enable authentication providers to be added and managed. /object/billing API to enable the metering of object store usage at the tenant and bucket level. See Manage a tenant on page 84 for more information. Base URL /object/baseurl Whoami 253

254 Use the ECS Management REST API Table 51 ECS Management REST API- methods summary (continued) API Area Description API to enable the creation of a Base URL that allows existing applications to work with the ECS object store. More information on Base URL can be found in:. Bucket /object/bucket API for provisioning and managing buckets. /object/bucket/{bucketname}/lock API to enable bucket access to be locked. See Manage a tenant on page 84 for more information. Call Home /vdc/callhome/ API to enable ConnectEMC to be configured and create alert events for ConnectEMC. Capacity /object/capacity API for retrieving the current managed capacity. CAS /object/user-cas API to enable secret keys to be assigned to CAS users and enables Pool Entry Authorization (PEA) file to be generated. Certificate /object-cert API for managing certificates. /object-cert/keystore API to enable the certificate chain used by EMC to be specified and for the certificate to be rotated. Configuration Properties /config/object/properties API to enable the user scope to be set as GLOBAL or NAMESPACE. In GLOBAL scope, users are global and are can be shared across namespaces. In this case, the default namespace associated with a user determines the namespace for object operations and there is no need to supply a namespace for an operation. If the user scope is NAMESPACE, a user is associated with a namespace, so there might be more than user with the same name, each associated with a different namespace. In NAMESPACE mode a namespace must be provided for every operation. Must be set before the first user is created. The default is GLOBAL. Data Store /vdc/data-stores API to enable the creation of data stores on file systems (/vdc/datastores/filesystems) or on commodity nodes (/vdc/datastores/commodity). Licensing /license API to enable a license to be added and license details to be retrieved. 254 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

255 Use the ECS Management REST API Table 51 ECS Management REST API- methods summary (continued) API Area Monitoring (Dashboard) Monitoring (Events) Description /dashboard API for retrieving system information for display on a dashboard. /vdc/events API for retrieving audit alerts. Namespace /object/namespaces API to enable a namespace to be created and managed. Also enables the retention period for the namespace to be set. See Manage a tenant on page 84 for more information. Node Password Group (Swift) Replication Group /vdc/nodes API for retrieving the nodes that are currently configured for the cluster. /object/user-password API to enable a password to be generated for use with OpenStack Swift authentication. /data/data-service/vpools API to enable the creation and administration of replication groups. Secret Key /object/user-secret-keys API to allow secret keys to be assigned to object users and to enable secret keys to be managed. Secret Key Self-Service /object/secret-keys API to allow S3 users to create a new secret key that enables them to access objects and buckets within their namespace in the object store. Storage Pool /vdc/data-services/varrays API to allow the creation and management of storage pools. Temporary Failed Zone User (Object) /tempfailedzone/ API to enable all temporary failed zones or the temporary failed zones for a specified replication group to be retrieved. /object/users API for creating and managing object users. Object users are always associated with a namespace. The API returns a secret key that can be used for S3 access. An object user assigned an S3 secret key can change it using the REST API. /object/users/lock. Enables user access to be locked. User (Management) /vdc/users ECS Management REST API summary 255

256 Use the ECS Management REST API Table 51 ECS Management REST API- methods summary (continued) API Area Description API for creating and managing management users. Management users can be assigned to the System Admin role or to the Namespace Admin role. Local management user password can be changed. Virtual Data Center /object/vdcs Enables a VDC to be added and its inter VDC endpoints and secret key to be specified for replication of data between ECS sites. 256 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

257 PART 10 Hardware Maintenance Chapter 30, "ECS hardware" Chapter 31, "Replace an ECS storage disk in a U-Series Appliance" Chapter 32, "Replace an ECS storage disk in third-party hardware" Chapter 33, "Add a 60-disk upgrade to a U-Series ECS Appliance" Chapter 34, "ECS third-party rack requirements" Hardware Maintenance 257

258 Hardware Maintenance 258 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

259 CHAPTER 30 ECS hardware ECS Appliance hardware components ECS Appliance configurations and upgrade paths U-Series single-phase AC power cabling U-Series three-phase AC power cabling Switches Network cabling Nodes Rack and node host names Disk drives Integrated disk drives Disks and enclosures ECS hardware 259

260 ECS hardware ECS Appliance hardware components Describes the hardware components that make up ECS Appliance hardware models. ECS Appliance series The ECS Appliance has two series: U-Series: Unstructured storage servers with separate disk array enclosures (DAEs) engineered to maximize storage capacity. C-Series: High-density compute servers with integrated disks engineered to maximize compute capacity. U-Series The U-Series ECS Appliance includes the following hardware components: Table 52 ECS Appliance: U-Series hardware components Component 40U rack Description EMC Titan D racks that include: Single-phase PDUs with four power drops (two per side) Optional three-phase WYE or delta PDUs with two power drops (one per side) Front and rear doors Racking by EMC manufacturing Private switch Public switch Nodes One 1 GbE switch Two 10 GbE switches Intel-based unstructured server in four and eight-node configurations Disk array enclosure (DAE) Disk array enclosure (DAE) drawers that hold up to 60 6TB 3.5- inch disk drives 260 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

261 ECS hardware Figure 50 U-Series ECS Appliance minimum and maximum configurations C-Series The C-Series ECS Appliance includes the following hardware components: Table 53 ECS Appliance: C-Series hardware components Component 40U rack Description EMC Titan D Compute racks that include: Four horizontal power drops Two single-phase PDUs in a 2U configuration with two power drops Optional two three-phase WYE or delta PDUs in a 2U configuration with two power drops Front and rear doors Racking by EMC manufacturing ECS Appliance hardware components 261

262 ECS hardware Table 53 ECS Appliance: C-Series hardware components (continued) Component Private switch Public switch Nodes Disks Description One or two 1 GbE switches. The second switch is required for configurations with more than six servers Two or four 10 GbE switches. The third and four switches are required for configurations with more than six servers Intel-based unstructured servers in eight through 48 node configurations 12 6TB 3.5-inch disk drives integrated with each server Figure 51 C-Series ECS Appliance minimum and maximum configurations Customers connect to an ECS Appliance by way of 10 GbE ports and their own interconnect cables. When multiple appliances are installed in the same data center, the private switches can be connected by daisy-chain or home-run connections to a customer-provided switch. 262 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

263 ECS hardware ECS Appliance configurations and upgrade paths Describes the available ECS Appliance configurations and the upgrade paths between the configurations. U-Series configurations The U-Series ECS Appliance is a dense storage solution using commodity hardware. Table 54 U-Series configurations Model number Nodes DAEs Disks in DAE 1 to 4 Disks in DAE 5 to 8 Storage capacity Switches U300 (minimum configuratio n) NA 360TB One private and two public U NA 720TB One private and two public U NA 1080TB One private and two public U NA 1440TB One private and two public U TB One private and two public U TB One private and two public U TB One private and two public U3000 (maximum configuratio n) TB One private and two public U-Series upgrade paths U-Series upgrades consist of the disks and infrastructure hardware needed to move from the existing model number to the next higher model number. To upgrade by more than one model level, order the upgrades for each level and apply them in one service call. ECS Appliance configurations and upgrade paths 263

264 ECS hardware Table 55 U-Series upgrades Model number U300 (minimum configuration) Disk upgrade (from next lower model) NA Hardware upgrade (from next lower model) NA U700 One 60-disk kit NA U1100 One 60-disk kit NA U1500 One 60-disk kit NA U1800 One 60-disk kit One server chassis (four nodes) and four DAEs U2100 One 60-disk kit NA U2500 One 60-disk kit NA U3000 (maximum configuration) One 60-disk kit NA C-Series configurations The C-Series ECS Appliance is a dense compute solution using commodity hardware. Table 56 C-Series configurations Phoenix-12 Compute Servers 2 (minimum configuration) Nodes Storage capacity Switches 8 144TB One private and two public TB One private and two public TB One private and two public TB One private and two public TB One private and two public TB Two private and four public TB Two private and four public TB Two private and four public TB Two private and four public TB Two private and four public 264 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

265 ECS hardware Table 56 C-Series configurations (continued) Phoenix-12 Compute Servers 12 (maximum configuration) Nodes Storage capacity Switches TB Two private and four public C-Series upgrade paths C-Series upgrades consist of the disks and infrastructure hardware needed to move from the existing model number to the next higher model number. To upgrade by more than one model level, order the upgrades for each level and apply them in one service call. Table 57 C-Series upgrades Model number 2 Phoenix-12 Compute Servers (minimum configuration) Disk upgrade (from next lower model) NA Hardware upgrade (from next lower model) NA 3 Phoenix-12 Compute Servers 12 integrated disks One server chassis (four nodes) 4 Phoenix-12 Compute Servers 12 integrated disks One server chassis (four nodes) 5 Phoenix-12 Compute Servers 12 integrated disks One server chassis (four nodes) 6 Phoenix-12 Compute Servers 12 integrated disks One server chassis (four nodes) 7 Phoenix-12 Compute Servers 12 integrated disks One server chassis (four nodes) and one private and two public switches 8 Phoenix-12 Compute Servers 12 integrated disks One server chassis (four nodes) 9 Phoenix-12 Compute Servers 12 integrated disks One server chassis (four nodes) 10 Phoenix-12 Compute Servers 12 integrated disks One server chassis (four nodes) 11 Phoenix-12 Compute Servers 12 integrated disks One server chassis (four nodes) 12 Phoenix-12 Compute Servers (maximum configuration) 12 integrated disks One server chassis (four nodes) U-Series single-phase AC power cabling Provides single-phase power cabling diagrams for the U-Series ECS Appliance. For each configuration, the switches plug into the front of the rack and route through the rails to the rear. U-Series single-phase AC power cabling 265

266 ECS hardware Figure 52 U-Series single-phase AC power cabling for four-node configurations 266 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

267 ECS hardware Figure 53 U-Series single-phase AC power cabling for eight-node configurations U-Series three-phase AC power cabling Provides cabling diagrams for three-phase AC power. U-Series three-phase delta AC power cabling The legend below maps colored cables shown in the diagram to EMC part numbers and cable lengths. U-Series three-phase AC power cabling 267

268 ECS hardware Figure 54 Cable legend for three-phase delta AC power diagram 268 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

269 ECS hardware Figure 55 Three-phase delta AC power cabling for eight-node configuration U-Series three-phase AC power cabling 269

270 ECS hardware Three-phase WYE AC power cabling The legend below maps colored cables shown in the diagram to EMC part numbers and cable lengths. Figure 56 Cable legend for three-phase WYE AC power diagram 270 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

271 ECS hardware Figure 57 Three-phase WYE AC power cabling for eight-node configuration Switches ECS Appliances include three switches: Private switch One 1 GbE private switch to handle management traffic. In a C-Series rack with more than six compute servers, a second private switch is added. Public switch Two 10 GbE switches to handle data traffic. There are two Arista models qualified for use. In a C-Series rack with more than six compute servers, two more public switches are added. Switches 271

272 ECS hardware Private switch: Arista 7048 The private switch is used for management traffic. It has 48 ports and dual power supply inputs. The switch is configured in the factory. Figure 58 Arista 7048 configuration 1. Ports 1-24 Management 2. Ports RMM/IPMI 3. Port Management connections to the 10 GbE public switches 4. Port 51 NAN connectivity. In the first rack, this port uplinks to the customer network when access to the RMMs is needed. 5. Port 52 NAN connectivity. Port 52 of the first rack connects to port 51 of the next rack and so on to all racks in the ECS site. Note Public switch: Arista 7150S-24 The NAN (Nile Area Network) links all ECS Appliances at a site. The 7150S-24 switch is a 24 port switch. The switch is equipped with 24 SFP+ ports, dual hot-swap power supplies, and redundant, field-replaceable fan modules. Figure 59 Arista 7150S Ports 1-8: Customer uplink 2. Ports 9-12: Connect to nodes as data ports 272 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

273 ECS hardware Public switch: Arista 7124SX 3. Ports 13-20: Connect to nodes as data ports 4. Ports 21-22: Interconnect for MLAG interfaces between public switches 5. Ports 23-24: Interconnect for MLAG interfaces between public switches 6. Switch management 1 network interface 7. Serial console: The console port is used to manage the switch via a serial connection and the Ethernet management port is connected to the 1 GbE management switch The Arista 7124SX switch is equipped with 24 SFP+ ports, dual hot-swap power supplies, and redundant field replaceable fan modules. Figure 60 Arista 7124SX Network cabling 1. Port 1-8 Customer uplink data ports. These ports provide the connection to the user's 10 GbE infrastructure 2. Port 9-20 Connected to the nodes as data ports 3. Port HA interconnect for MLAG interfaces between public switches 4. Port HA interconnect for MLAG interfaces between public switches 5. Switch management 1 network interface 6. Serial console Presents network cable diagrams for the public and private switches in an ECS Appliance. The network cabling diagrams apply to both the U-Series and C-Series ECS Appliance. To distinguish between the three switches in documentation, each switch has a nickname: Hare: the 10 GbE public switch located at the top of the rack. Rabbit: the second 10 GbE public switch located below hare. Turtle: the 1 GbE private switch located below rabbit. Public switch: Arista 7124SX 273

274 ECS hardware Figure 61 Public switch cabling for four nodes Figure 62 Private switch cabling for four nodes Nodes ECS Appliance has two node types: U-Series Phoenix-16 for Object and HDFS (June 2014) C-Series Phoenix-12 for Object and HDFS (December 2014) U-Series nodes have the following standard features: 274 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

275 ECS hardware Four-node servers (2U) with two CPUs per node 2.4 GHz four-core Ivy Bridge CPUs Four channels of native DDR3 (1333) memory One or two system disks per node LED indicators for each node Dual hot-swap chassis power supplies Two SAS interfaces per node C-Series nodes have the following standard features: Four-node servers (2U) with two CPUs per node 2.4 GHz four-core Ivy Bridge CPUs Four channels of native DDR3 (1333) memory One system disk per node LED indicators for each node Dual hot-swap chassis power supplies. Supports N + 1 power. One SAS interface per node hot-swap SAS hard drives per server (three for each node) Server front views Figure 63 U-Series chassis front view Figure 64 C-Series chassis front view LED indicators are on the left and right side of the server front panels. Server front views 275

276 ECS hardware Figure 65 LEDs (left side) Table 58 Server LEDs LED Used for Description A System power button with LED B System ID LED button Identifies the server from among several servers. Off by default, and blue when activated by the button or by software. C System status LED Shows the overall health of the system (green, blinking green, blinking amber, amber, off). D Network link/activity LED Server rear view Both the Phoenix-16 (U-Series) and the Phoenix-12 (C-Series) server chassis provide dual hot-swappable power supplies and four nodes. The chassis shares a common redundant power supply (CRPS) that enables HA power in each chassis shared across all nodes. The nodes are mounted on hot-swappable trays that fit into the four corresponding node slots accessible from the rear of the server. 276 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

277 ECS hardware Figure 66 Phoenix-12 and Phoenix-16 server chassis rear view 1. Node 1 2. Node 2 3. Node 3 4. Node 4 Figure 67 Rear ports on Phoenix-12 and Phoenix-16 nodes 1. 1 GbE: Connected to one of the 1 GbE switches data port. 2. RMM: A dedicated port for hardware monitoring (per node) 3. SAS to DAE (used on Phoenix-16 only) GbE (primary): The right 10 GbE data port of each node is connected to one of the 10 GbE switch's primary data ports GbE (secondary): The left 10 GbE data port of each node is connected to one of the 10 GbE switch's secondary data ports Server rear view 277

278 ECS hardware Rack and node host names Lists the default rack and node host names for an ECS appliance. Default rack IDs and color names are assigned in installation order as shown below: Table 59 Rack ID 1 to 50 Rack ID Rack color Rack ID Rack color Rack ID Rack color 1 red 18 carmine 35 cornsilk 2 green 19 auburn 36 ochre 3 blue 20 bronze 37 lavender 4 yellow 21 apricot 38 ginger 5 magenta 22 jasmine 39 ivory 6 cyan 23 army 40 carnelian 7 azure 24 copper 41 taupe 8 violet 25 amaranth 42 navy 9 rose 26 mint 43 indigo 10 orange 27 cobalt 44 veronica 11 chartreuse 28 fern 45 citron 12 pink 29 sienna 46 sand 13 brown 30 mantis 47 russet 14 white 31 denim 48 brick 15 gray 32 aquamarine 49 avocado 16 beige 33 baby 50 bubblegum 17 silver 34 eggplant Nodes are assigned host names based on their order within the server chassis and within the rack itself. The following table lists the default host names. Table 60 Default host names Nod e Host name Node Host name Node Host name 1 provo 17 memphis 33 tampa 2 sandy 18 seattle 34 toledo 3 orem 19 denver 35 aurora 4 ogden 20 portland 36 stockton 5 layton 21 tucson 37 buffalo 6 logan 22 atlanta 38 newark 278 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

279 ECS hardware Table 60 Default host names (continued) Nod e Host name Node Host name Node Host name 7 Lehi 23 fresno 39 glendale 8 murray 24 mesa 40 lincoln 9 boston 25 omaha 41 norfolk 10 chicago 26 oakland 42 chandler 11 houston 27 miami 43 madison 12 phoenix 28 tulsa 44 orlando 13 dallas 29 honolulu 45 garland 14 detroit 30 wichita 46 akron 15 columbus 31 raleigh 47 reno 16 austin 32 anaheim 48 laredo Nodes positioned in the same slot in different racks at a site will have the same host name. For example node 4 will always be called ogden, assuming you use the default node names. System outputs will identify nodes by a unique combination of node host name and rack name. For example, node 4 in rack 4 and node 4 in rack 5 will be identified as: ogden-green ogden-blue Disk drives Describes the disk drives used in ECS Appliances. ECS Appliances use the following disk drives for storage: Table 61 Storage disk drives Service Size RPM Type Object 6TB 7200 SATA All the disks in a DAE or integrated into a server chasis conform to these rules: All disk drives must be the same size All disk drives must be the same speed Integrated disk drives Describes storage disks integrated into the server chassis. In servers with integrated disks, the disks are accessible from the front of the server chassis. The disks are assigned equally to the four nodes in the chassis. Disk drives 279

280 ECS hardware Note The first disk drive assigned to each node is called disk drive zero (HDD0). These storage drives will contain some system data. For this reason, disk drive zero cannot be replaced without first contacting EMC Customer Support. Figure 68 Integrated disks with node mappings Disks and enclosures The U-Series provides disk array enclosures (DAE). The DAE is a drawer that slides in and out of the 40U dense rack. The disk drives, LCC, and cooling modules for the DAE are located inside the DAE. The DAE has the following features: 15, 30, 45, or inch disk drives in a single 4U drawer. Serviced from the front. One Link Control Card (LCC). Serviced from the front. One Inter Connect Module (ICM). Serviced from the back. Three fans or cooling modules; n+1 redundant. Serviced from the front. Two power supplies; n+1 redundant. Serviced from the back. C-Series servers have integrated disks: inch disk drives accessible from the front of the server. Disk drives in DAEs Disk drives are encased in cartridge-style enclosures. Each cartridge has a latch that allows you to snap-out a disk drive for removal and snap-in for installation. The inside of each DAE has physically printed labels located on the left and the front sides of the DAE that describe the rows (or banks) and columns (or slots) where the disk drives are installed in the DAE. The banks are labeled from A to E and the slots are labeled from 0 to 11. When describing the layout of disk drives within the DAE, the interface format for the DAE is called E_D. That is, E indicates the enclosure, and D the disk. For example, you could have an interface format of 1_B11. This format is interpreted as enclosure 1, in row (bank) B/slot number 11. Enclosures are numbered from 1 through 8 starting at the bottom of the rack. Rear cable connections are color-coded. The arrangement of disk disks in a DAE must match the prescribed layouts shown in the figures below. Looking at the DAE from the front and above, the following figure shows you the disk drive layout of the DAE. 280 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

281 ECS hardware Figure 69 U-Series disk layout for 15-disk configurations Disk drives in DAEs 281

282 ECS hardware Figure 70 U-Series disk layout for 30-disk configurations 282 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

283 ECS hardware Figure 71 U-Series disk layout for 45-disk configurations Disk drives in DAEs 283

284 ECS hardware Figure 72 U-Series disk layout for 60-disk configurations LCC Each DAE includes a hot-swappable LCC whose main function is to be a SAS expander and provide enclosure services. The LCC independently monitors the environment status of the entire enclosure and communicates the status to the system. The LCC includes a fault LED and a power LED. Table 62 DAE LCC status LED LED Color State Description Power Green On Power on Off Power off Power fault Amber On Fault Off No fault or power off 284 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

285 ECS hardware Figure 73 LCC with LEDs Fan control module Each DAE includes three fan control modules (cooling modules) located on the front of the DAE. The fan control module augments the cooling capacity of each DAE. It plugs directly into the DAE baseboard from the top of the DAE. Inside the fan control module, sensors measure the external ambient temperatures to ensure even cooling throughout the DAE. Table 63 Fan control module fan fault LED LED Color State Description Fan fault Amber On Fault detected. One or more fans faulted. Off No fault detected. Fans operating normally. Figure 74 Fan control module with LED ICM The ICM is the primary interconnect management element. It is a plug-in module that includes a USB connector, RJ-12 management adapter, Bus ID indicator, enclosure ID Fan control module 285

286 ECS hardware indicator, two input SAS connectors and two output SAS connectors with corresponding LEDs indicating the link and activity of each SAS connector for input and output to devices. Table 64 ICM bus status LEDs LED Color State Description Power fault Green On Power on Off Power off Power on Amber On Fault Off No fault or power off The ICM supports the following I/O ports on the rear: Four 6-Gb/s PCI Gen 2 SAS ports One management (RJ-12) connector to the SPS (field service diagnostics only) One USB connector 6-Gb/s SAS x8 ports It supports four (two input and two output, one used) 6-Gb/s SAS x8 ports on the rear of the ICM. This port provides an interface for SAS and NL-SAS drives in the DAE. Table 65 ICM 6-Gb/s port LEDs LED Color State Description Link/Activity Blue On Indicates a 4x or 8x connection with all lanes running at 6 Gb/s. Green On Indicates that a wide port width other than 4x or 8x has been established or one or more lanes is not running at full speed or disconnected. Off Not connected. 286 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

287 ECS hardware Figure 75 ICM LEDs 1. Power fault LED (amber) 2. Power LED (green) 3. Link activity LEDs (blue/green) DAE power supply The power supply is hot-swappable. It has a built-in thumbscrew for ease of installation and removal. Each power supply includes a fan to provide cooling to the power supply. The power supply is an auto-ranging, power-factor-corrected, multi-output, offline converter with its own line cord. Each supply supports a fully configured DAE and shares load currents with the other supply. The DAE, the power supplies provide four independent power zones. Each of the hot-swappable power supplies has the capability to deliver 1300 W at 12 V in its load-sharing highly-available configuration. Control and status are implemented throughout the I2C interface. DAE power supply 287

288 ECS hardware Table 66 DAE AC power supply/cooling module LEDs LED Color State Description AC power on (12 V power): one LED for each power cord. Green On OK. AC or SPS power applied. All output voltages are within respective operating ranges, not including fan fault. Off 12 V power is out of operation range, or in shutdown or fault detected within the unit. Power fault Amber On Under ICM control. On if any fans or outputs are outside the specified operating range while the unit is not in low power mode. Off All outputs are within the specified range, or in shutdown or fault detected within unit. Figure 76 DAE power supply 288 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

289 ECS hardware U-Series SAS cabling Provides wiring diagrams for the SAS cables that connect nodes to DAEs. Note Hardware diagrams number nodes starting with zero. In all other discussions of ECS architecture and software, nodes are numbered starting with one. U-Series SAS cabling 289

290 ECS hardware Figure 77 SAS cabling for four-node configurations 290 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

291 ECS hardware Figure 78 SAS cabling for eight-node configurations U-Series SAS cabling 291

292 ECS hardware 292 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

293 CHAPTER 31 Replace an ECS storage disk in a U-Series Appliance Drive replacement planning Output for the cs_hal list disks command Replace drives Replace an ECS storage disk in a U-Series Appliance 293

294 Replace an ECS storage disk in a U-Series Appliance Drive replacement planning Describes an efficient strategy for periodic grooming of an ECS Appliance to replace FAILED and SUSPECT storage drives. Note To replace the node's main (non-storage) drives, contact EMC Customer Support for assistance. This document makes a distinction between the terms "drive" and "disk." A drive is a physical spindle. A disk is a logical entity (that is, a partition on a drive). When the system labels a drive as FAILED, the data protection logic rebuilds the data on that drive on other drives in the system. The FAILED drive no longer participates in the system in any way. Replacing a drive does not involve restoring any data to the replacement drive. Therefore, a FAILED drive only represents a loss of raw capacity to the system. This characteristic of the built-in data protection lessens the need for an immediate service action when a drive fails. SUSPECT drives that are suspect because of physical errors, as opposed to connectivity issues, should also be replaced. When a drive is labeled SUSPECT, the system no longer writes new data to the drive, but the system will continue to read from it. The SUSPECT drive with physical errors will eventually be labeled FAILED when the physical errors exceed a certain threshold. While the drive remains SUSPECT, the drive is not participating fully in the storage system. Therefore, a disk on the corresponding drive should be manually set to Bad using the supplied cs_hwmgr command line tool. This step gives the system the opportunity to finish pending network interactions with the drive. Data on the drive will be reconstructed elsewhere in the system using copies of the data from other drives. Therefore, you can begin the replacement process as soon as the system indicates that the disk on the corresponding FAILED drive was successfully set to Bad. Note The ViPR UI lists a node as DEGRADED when the node has storage disks with the Suspect or Bad status. An efficient way to handle FAILED drives is to replace them periodically. The replacement period should be calculated using the manufacturer's mean failure rate of the drives such that there is no danger that a critical number of drives can fail by the next planned service date. This process is called "grooming." Grooming involves: Ordering a sufficient number of drives to cover the mean failure rate expected by the next service date. Identifying the FAILED drives. Identifying SUSPECT drives that are suspect because of physical errors and manually forcing the disks corresponding to them to Bad. Replacing the FAILED drives. Output for the cs_hal list disks command 294 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation Describes the types of output rows from the cs_hal list disks command. In the abbreviated cs_hal list disks output below, notice the different types of rows:

295 Replace an ECS storage disk in a U-Series Appliance The first three rows represent the RAID structure of the node's two internal drives. The next row shows a GOOD storage drive. Device names, slot numbers, and serial numbers can be used in cs_hal commands as long as they are unique. The enclosure name can also be used in cs_hal commands. The next row represents a FAILED storage drive. The Partition Name indicates the disk on the corresponding drive is assigned to the Object service, the disk is formatted, and the disk health is Bad. The next row represents a SUSPECT storage drive. The disk on the corresponding SUSPECT drive has a disk health status of Suspect. The last row represents either an empty slot or an undetectable drive. Find the definitive health state of a drive in the SMART Status column. Note Ignore health information in the Partition Name column. This information is not necessary for this procedure. If you are interested in partition health for other reasons, then the cs_hwmgr disk --list-by-service Object command will have the most up-to-date and accurate information on partition health in the Health column. The DiskSet column is reserved for future use. [root@layton-cyan ~]# cs_hal list disks Disks(s): SCSI Device Block Device Enclosure Partition Name Slot Serial Number SMART Status DiskSet n/a /dev/md0 RAID vol n/a n/a not supported n/a /dev/sg4 /dev/sdb internal 0 KLH6DHXJ GOOD /dev/sg5 /dev/sdc internal 1 KLH6DM1J GOOD /dev/sg8 /dev/sdf /dev/sg0 Object:Formatted:Good A08 WCAW GOOD /dev/sg9 /dev/sdg /dev/sg0 Object:Formatted:Bad A09 WCAW FAILED: self-test fail; read element; /dev/sg10 /dev/sdh /dev/sg0 Object:Formatted:Suspect A10 WCAW SUSPECT: Reallocated_Sector_Count(5)=11... unavailable /dev/sg0 E05 internal: 2 external: 30 total disks: 32 Replace drives Replace FAILED and SUSPECT storage drives using commands on the node. Note Do not place the node in maintenance mode to replace drives. Procedure 1. To access the ECS rack (cabinet) using the private ( xxx) network from a laptop: Replace drives 295

296 Replace an ECS storage disk in a U-Series Appliance a. From the rear of the rack, locate the 1 GbE private switch network ports by opening the rear door. b. On the Arista 1 GbE (turtle) switch, attach a network cable from your laptop to port 24 on the switch. Figure 79 Locate port 24 on the private 1 GbE switch c. Set the network interface on the laptop to the static address , subnet mask , with no gateway required. d. Verify that the temporary network between the laptop and rack's private management network is functioning by using the ping command. C:\>ping Pinging with 32 bytes of data: Reply from : bytes=32 time<1ms TTL=64 Reply from : bytes=32 time<1ms TTL=64 Reply from : bytes=32 time<1ms TTL=64 Reply from : bytes=32 time<1ms TTL=64 Ping statistics for : Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms Note If does not answer, try If neither responds, verify the laptop IP/subnet mask, network connection, and switch port connection. # cs_hal list disks Disks(s): e. From the laptop, SSH into Node 1 (provo) via , using the root login credentials (with default or customer-modified password). 2. Use the cs_hal list disks command and observe the drive health status of each drive in the SMART Status column: 296 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

297 Replace an ECS storage disk in a U-Series Appliance SCSI Device Block Device Enclosure Partition Name Slot Serial Number SMART Status DiskSet n/a /dev/md0 RAID vol n/a n/a not supported n/a /dev/sg4 /dev/sdb internal 0 KLH6DHXJ GOOD /dev/sg5 /dev/sdc internal 1 KLH6DM1J GOOD /dev/sg8 /dev/sdf /dev/sg0 Object:Formatted:Good A08 WCAW GOOD /dev/sg9 /dev/sdg /dev/sg0 Object:Formatted:Good A09 WCAW GOOD /dev/sg10 /dev/sdh /dev/sg0 Object:Formatted:Good A10 WCAW GOOD /dev/sg11 /dev/sdi /dev/sg0 Object:Formatted:Good B08 WCAW GOOD /dev/sg12 /dev/sdj /dev/sg0 Object:Formatted:Good B07 WCAW GOOD /dev/sg13 /dev/sdk /dev/sg0 Object:Formatted:Good B06 WCAW GOOD /dev/sg14 /dev/sdl /dev/sg0 Object:Formatted:Good A11 WCAW GOOD /dev/sg15 /dev/sdm /dev/sg0 Object:Formatted:Good B10 WCAW GOOD /dev/sg16 /dev/sdn /dev/sg0 Object:Formatted:Suspect B11 WCAW SUSPECT: Reallocated_Sector_Count(5)=11 /dev/sg17 /dev/sdo /dev/sg0 Object:Formatted:Good C11 WCAW GOOD /dev/sg18 /dev/sdp /dev/sg0 Object:Formatted:Good D11 WCAW GOOD /dev/sg19 /dev/sdq /dev/sg0 Object:Formatted:Good C10 WCAW GOOD /dev/sg20 /dev/sdr /dev/sg0 Object:Formatted:Good D10 WCAW GOOD /dev/sg21 /dev/sds /dev/sg0 Object:Formatted:Good C09 WCAW GOOD /dev/sg22 /dev/sdt /dev/sg0 Object:Formatted:Good D09 WCAW GOOD /dev/sg23 /dev/sdu /dev/sg0 Object:Formatted:Good E11 WCAW GOOD /dev/sg24 /dev/sdv /dev/sg0 Object:Formatted:Good E10 WCAW GOOD /dev/sg25 /dev/sdw /dev/sg0 Object:Formatted:Good E09 WCAW GOOD /dev/sg26 /dev/sdx /dev/sg0 Object:Formatted:Good C08 WCAW GOOD /dev/sg27 /dev/sdy /dev/sg0 Object:Formatted:Good D08 WCAW GOOD /dev/sg28 /dev/sdz /dev/sg0 Object:Formatted:Good C07 WCAW GOOD /dev/sg29 /dev/sdaa /dev/sg0 Object:Formatted:Good E06 WCAW GOOD /dev/sg30 /dev/sdab /dev/sg0 Object:Formatted:Good E08 WCAW GOOD /dev/sg31 /dev/sdac /dev/sg0 Object:Formatted:Good D06 WCAW GOOD /dev/sg32 /dev/sdad /dev/sg0 Object:Formatted:Good C06 WCAW GOOD /dev/sg33 /dev/sdae /dev/sg0 Object:Formatted:Good D07 WCAW GOOD /dev/sg34 /dev/sdaf /dev/sg0 Object:Formatted:Good E07 WCAW GOOD /dev/sg3 /dev/sda /dev/sg0 Object:Formatted:Good A06 WCAW GOOD /dev/sg6 /dev/sdd /dev/sg0 Object:Formatted:Good A07 WCAW GOOD /dev/sg7 /dev/sde /dev/sg0 Object:Formatted:Good B09 Replace drives 297

298 Replace an ECS storage disk in a U-Series Appliance WCAW GOOD unavailable /dev/sg0 E02 unavailable /dev/sg0 E03 unavailable /dev/sg0 E04 unavailable /dev/sg0 E05 internal: 2 external: 30 total disks: 32 Here the report shows the 2 internal drives of the node, 30 storage drives (one of them in a SUSPECT health state), and 30 empty slots labeled as unavailable. Note If the output of the above command indicates that the Partition Name of a drive is "Object:Formatted:Bad" or "Object:Formatted:Suspect" and its SMART status is "GOOD", stop here. This condition is the result of a bug in hardware manager that causes it to catch an exception when determining the health of a drive. Please, engage ECS Customer Support for assistance and wait until they run a workaround procedure that fixes the issue. Otherwise, proceed with the next step. Note If you have a disk ID from the ECS Portal, use the following command with grep to get the serial number of the disk in the sixth column. In this example, the Serial Number is AR31021EG62L9C: # cs_hwmgr disk --list-by-service Object grep a1d7c15f b d0154 a1d7c15f b d AR31021EG62L9C... Now use the cs_hwmgr drives --list command with grep and the Serial Number to get the slot ID in the last (sixth) column. In this example, the slot ID is A06: # cs_hwmgr drives --list grep AR31021EG62L9C ATA HGST HUS726060AL AR31021EG62L9C 6001 Good A06 You now have the information needed to use this procedure. 3. Mark down the Serial Number of the FAILED or SUSPECT drive. 4. If the target drive is SUSPECT: Determine if the drive is in the SUSPECT state because of hardware issues: Continue the replace drive procedure if the SMART Status column in the cs-hal command output shows "reallocated_sector_count" data. This data indicates hardware problems. Stop the replace drive procedure if you do not see "reallocated_sector_count" data. Drives can be in the SUSPECT state for non-hardware issues like connectivity problems. 298 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

299 Replace an ECS storage disk in a U-Series Appliance a. Use the cs_hwmgr --list command to get the VendorId and ProductID for the disk, which are found in the first two columns. # cs_hwmgr drive --list grep -i WCAW ATA HUS726060ALA640 WCAW Suspect B11 Note If the cs_hwgmr drive --list grep <disk SN#> command does not return any output for a "Suspect" drive, the drive is not being managed by the hardware manager. Engage ECS Customer Support for assistance. Refer to Knowledge Base entry b. Set the drive health status to "Bad" with the cs_hwmgr drive --sethealth-bad command using the <VendorID>, <ProductID> and <Disk SN#> values. # cs_hwmgr drive --set-health-bad ATA HUS726060ALA640 WCAW Setting drive health of ATA:HUS726060ALA640:WCAW to bad...suceeded c. Use the cs_hwmgr drive --remove command to remove the target drive (or more specifically the disk associated with that drive) from the configuration. Note that this command does not remove the drive from the output of the cs_hwmgr drive --list command. # cs_hwmgr drive --remove ATA HUS726060ALA640 WCAW Removing drive ATA:HUS726060ALA640:WCAW Suceeded Note If the cs_hwmgr drive --remove command fails, use the cs_hwmgr drive -- force-remove command using the <VendorID>, <ProductID>, <Disk SN#> values. 5. If the target drive is BAD: If the drive's Serial Number did not appear in the cs_hwmgr drive --list command output, skip this step. If the If the drive health reported using the cs_hwmgr drive --list command is "Bad", continue with the substep a below. a. Use the cs_hwmgr drive --remove command to remove the target drive (or more specifically the disk associated with that drive) from the configuration. Note that this command does not remove the drive from the output of the cs_hwmgr drive --list command. # cs_hwmgr drive --remove ATA HUS726060ALA640 WCAW Removing drive ATA:HUS726060ALA640:WCAW Suceeded Note If the cs_hwmgr drive --remove command fails, use the cs_hwmgr drive -- force-remove command using the <VendorID>, <ProductID>, <Disk SN#> values. Replace drives 299

300 Replace an ECS storage disk in a U-Series Appliance 6. Use the cs_hal led command and the serial number to blink the LED for the drive (to find the proper drive in the drawer). # cs_hal led WCAW blink 7. Locate the DAE with the FAILED drive. 8. Un-thread (counter clock-wise release) the shoulder screws from the mounting holes on both left and right sides of the enclosure so it can be accessed. Grasp the enclosure latch handles by the knurled edges, and pull them out to release the enclosure from the inner rails; slowly pull the enclosure out of the cabinet until it locks in the secure service position. Figure 80 Release Enclosure with Shoulder Screws and Pull Enclosure from the Cabinet Note If the locking latches do not easily pull out, push in on the drawer and try again. 9. Locate the drive to replace by looking for an illuminated (solid) amber fault LED. Verify the slot ID provided matches the drive with the fault LED illuminated. Disks are arranged in five rows of twelve drive assemblies each. The first (front) row is A, then B, C, D, and E; drive assemblies are numbered 0-11 from left to right in each row. 300 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

301 Replace an ECS storage disk in a U-Series Appliance Figure 81 DAE Drive Layout CL Attach an ESD wristband to your wrist and the enclosure. 11.Remove the drive from the DAE by pushing the orange release tab. Then lift the latch and remove the drive and place it on a static-free surface. Replace drives 301

302 Replace an ECS storage disk in a U-Series Appliance Figure 82 Removing a DAE Drive Module 2 3 CL Install the new drive in the same slot in the DAE. 13.Align the drive with the guides in the slot, then insert the new drive into the DAE. 14.With the drive module latch fully open, gently lower the module into the slot. 15.The drive module latch will begin to rotate downward. 16.Push the handle down to engage the latch. After the latch is engaged, push firmly on the edge of the module to verify that the drive is properly seated. The drive module s Active light flashes to reflect the drive s spin up sequence. 302 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

303 Replace an ECS storage disk in a U-Series Appliance Figure 83 Installing a DAE Drive Module CL Engage the two orange self-locking latches to secure the enclosure so it cannot slide back out of the cabinet. Align the two shoulder screws on each side with the mounting holes on the cabinet. Thread the shoulder screws into the mounting holes and fingertighten the shoulder screws. Replace drives 303

304 Replace an ECS storage disk in a U-Series Appliance Figure 84 Inserting the DAE into the Cabinet 18.Once the enclosure is closed, verify fan speeds return to normal operating levels. 19.Use the cs_hal list disks command to confirm that the system recognizes the new drive. The new drive has the same Slot ID as the replaced drive. Take note of the new Serial Number. The system automatically integrates the new disk into use by the appropriate services. 20.Use the cs_hal led command and the new Serial Number to turn off the LED for the drive. # cs_hal led <serial_number_of_new_drive> off Results The node and system automatically detect the drive and initialize it for use. 304 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

305 CHAPTER 32 Replace an ECS storage disk in third-party hardware Drive replacement planning Output for the cs_hal list disks command Replace drives Replace an ECS storage disk in third-party hardware 305

306 Replace an ECS storage disk in third-party hardware Drive replacement planning Describes an efficient strategy for periodic grooming of an ECS Appliance to replace FAILED and SUSPECT storage drives. Note To replace the node's main (non-storage) drives, contact EMC Customer Support for assistance. This document makes a distinction between the terms "drive" and "disk." A drive is a physical spindle. A disk is a logical entity (that is, a partition on a drive). When the system labels a drive as FAILED, the data protection logic rebuilds the data on that drive on other drives in the system. The FAILED drive no longer participates in the system in any way. Replacing a drive does not involve restoring any data to the replacement drive. Therefore, a FAILED drive only represents a loss of raw capacity to the system. This characteristic of the built-in data protection lessens the need for an immediate service action when a drive fails. SUSPECT drives that are suspect because of physical errors, as opposed to connectivity issues, should also be replaced. When a drive is labeled SUSPECT, the system no longer writes new data to the drive, but the system will continue to read from it. The SUSPECT drive with physical errors will eventually be labeled FAILED when the physical errors exceed a certain threshold. While the drive remains SUSPECT, the drive is not participating fully in the storage system. Therefore, a disk on the corresponding drive should be manually set to Bad using the supplied cs_hwmgr command line tool. This step gives the system the opportunity to finish pending network interactions with the drive. Data on the drive will be reconstructed elsewhere in the system using copies of the data from other drives. Therefore, you can begin the replacement process as soon as the system indicates that the disk on the corresponding FAILED drive was successfully set to Bad. Note The ViPR UI lists a node as DEGRADED when the node has storage disks with the Suspect or Bad status. An efficient way to handle FAILED drives is to replace them periodically. The replacement period should be calculated using the manufacturer's mean failure rate of the drives such that there is no danger that a critical number of drives can fail by the next planned service date. This process is called "grooming." Grooming involves: Ordering a sufficient number of drives to cover the mean failure rate expected by the next service date. Identifying the FAILED drives. Identifying SUSPECT drives that are suspect because of physical errors and manually forcing the disks corresponding to them to Bad. Replacing the FAILED drives. Output for the cs_hal list disks command 306 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation Describes the types of output rows from the cs_hal list disks command. In the abbreviated cs_hal list disks output below, notice the different types of rows:

307 Replace an ECS storage disk in third-party hardware The first three rows represent the RAID structure of the node's two internal drives. The next row shows a GOOD storage drive. Device names, slot numbers, and serial numbers can be used in cs_hal commands as long as they are unique. The enclosure name can also be used in cs_hal commands. The next row represents a FAILED storage drive. The Partition Name indicates the disk on the corresponding drive is assigned to the Object service, the disk is formatted, and the disk health is Bad. The next row represents a SUSPECT storage drive. The disk on the corresponding SUSPECT drive has a disk health status of Suspect. The last row represents either an empty slot or an undetectable drive. Find the definitive health state of a drive in the SMART Status column. Note Ignore health information in the Partition Name column. This information is not necessary for this procedure. If you are interested in partition health for other reasons, then the cs_hwmgr disk --list-by-service Object command will have the most up-to-date and accurate information on partition health in the Health column. The DiskSet column is reserved for future use. [root@layton-cyan ~]# cs_hal list disks Disks(s): SCSI Device Block Device Enclosure Partition Name Slot Serial Number SMART Status DiskSet n/a /dev/md0 RAID vol n/a n/a not supported n/a /dev/sg4 /dev/sdb internal 0 KLH6DHXJ GOOD /dev/sg5 /dev/sdc internal 1 KLH6DM1J GOOD /dev/sg8 /dev/sdf /dev/sg0 Object:Formatted:Good A08 WCAW GOOD /dev/sg9 /dev/sdg /dev/sg0 Object:Formatted:Bad A09 WCAW FAILED: self-test fail; read element; /dev/sg10 /dev/sdh /dev/sg0 Object:Formatted:Suspect A10 WCAW SUSPECT: Reallocated_Sector_Count(5)=11... unavailable /dev/sg0 E05 internal: 2 external: 30 total disks: 32 Replace drives Replace FAILED and SUSPECT storage drives using commands on the node. Note Do not place the node in maintenance mode to replace drives. Procedure 1. To access the ECS rack (cabinet) using the private ( xxx) network from a laptop: Replace drives 307

308 Replace an ECS storage disk in third-party hardware a. Locate the 1 GbE private switch network ports. b. On the 1 GbE (turtle) switch, attach a network cable from your laptop to port 24 on the switch. c. Set the network interface on the laptop to the static address , subnet mask , with no gateway required. d. Verify that the temporary network between the laptop and rack's private management network is functioning by using the ping command. C:\>ping Pinging with 32 bytes of data: Reply from : bytes=32 time<1ms TTL=64 Reply from : bytes=32 time<1ms TTL=64 Reply from : bytes=32 time<1ms TTL=64 Reply from : bytes=32 time<1ms TTL=64 Ping statistics for : Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms Note If does not answer, try If neither responds, verify the laptop IP/subnet mask, network connection, and switch port connection. e. From the laptop, SSH into Node 1 (provo) via , using the root login credentials (with default or customer-modified password). 2. Use the cs_hal list disks command and observe the drive health status of each drive in the SMART Status column: # cs_hal list disks Disks(s): SCSI Device Block Device Enclosure Partition Name Slot Serial Number SMART Status DiskSet n/a /dev/md0 RAID vol n/a n/a not supported n/a /dev/sg4 /dev/sdb internal 0 KLH6DHXJ GOOD /dev/sg5 /dev/sdc internal 1 KLH6DM1J GOOD /dev/sg8 /dev/sdf /dev/sg0 Object:Formatted:Good A08 WCAW GOOD /dev/sg9 /dev/sdg /dev/sg0 Object:Formatted:Good A09 WCAW GOOD /dev/sg10 /dev/sdh /dev/sg0 Object:Formatted:Good A10 WCAW GOOD /dev/sg11 /dev/sdi /dev/sg0 Object:Formatted:Good B08 WCAW GOOD /dev/sg12 /dev/sdj /dev/sg0 Object:Formatted:Good B07 WCAW GOOD /dev/sg13 /dev/sdk /dev/sg0 Object:Formatted:Good B06 WCAW GOOD /dev/sg14 /dev/sdl /dev/sg0 Object:Formatted:Good A11 WCAW GOOD /dev/sg15 /dev/sdm /dev/sg0 Object:Formatted:Good B10 WCAW GOOD /dev/sg16 /dev/sdn /dev/sg0 Object:Formatted:Suspect B11 WCAW SUSPECT: Reallocated_Sector_Count(5)=11 /dev/sg17 /dev/sdo /dev/sg0 Object:Formatted:Good C EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

309 Replace an ECS storage disk in third-party hardware WCAW GOOD /dev/sg18 /dev/sdp /dev/sg0 Object:Formatted:Good D11 WCAW GOOD /dev/sg19 /dev/sdq /dev/sg0 Object:Formatted:Good C10 WCAW GOOD /dev/sg20 /dev/sdr /dev/sg0 Object:Formatted:Good D10 WCAW GOOD /dev/sg21 /dev/sds /dev/sg0 Object:Formatted:Good C09 WCAW GOOD /dev/sg22 /dev/sdt /dev/sg0 Object:Formatted:Good D09 WCAW GOOD /dev/sg23 /dev/sdu /dev/sg0 Object:Formatted:Good E11 WCAW GOOD /dev/sg24 /dev/sdv /dev/sg0 Object:Formatted:Good E10 WCAW GOOD /dev/sg25 /dev/sdw /dev/sg0 Object:Formatted:Good E09 WCAW GOOD /dev/sg26 /dev/sdx /dev/sg0 Object:Formatted:Good C08 WCAW GOOD /dev/sg27 /dev/sdy /dev/sg0 Object:Formatted:Good D08 WCAW GOOD /dev/sg28 /dev/sdz /dev/sg0 Object:Formatted:Good C07 WCAW GOOD /dev/sg29 /dev/sdaa /dev/sg0 Object:Formatted:Good E06 WCAW GOOD /dev/sg30 /dev/sdab /dev/sg0 Object:Formatted:Good E08 WCAW GOOD /dev/sg31 /dev/sdac /dev/sg0 Object:Formatted:Good D06 WCAW GOOD /dev/sg32 /dev/sdad /dev/sg0 Object:Formatted:Good C06 WCAW GOOD /dev/sg33 /dev/sdae /dev/sg0 Object:Formatted:Good D07 WCAW GOOD /dev/sg34 /dev/sdaf /dev/sg0 Object:Formatted:Good E07 WCAW GOOD /dev/sg3 /dev/sda /dev/sg0 Object:Formatted:Good A06 WCAW GOOD /dev/sg6 /dev/sdd /dev/sg0 Object:Formatted:Good A07 WCAW GOOD /dev/sg7 /dev/sde /dev/sg0 Object:Formatted:Good B09 WCAW GOOD unavailable /dev/sg0 E02 unavailable /dev/sg0 E03 unavailable /dev/sg0 E04 unavailable /dev/sg0 E05 internal: 2 external: 30 total disks: 32 Here the report shows the 2 internal drives of the node, 30 storage drives (one of them in a SUSPECT health state), and 30 empty slots labeled as unavailable. Note If the output of the above command indicates that the Partition Name of a drive is "Object:Formatted:Bad" or "Object:Formatted:Suspect" and its SMART status is "GOOD", stop here. This condition is the result of a bug in hardware manager that causes it to catch an exception when determining the health of a drive. Please, engage ECS Customer Support for assistance and wait until they run a workaround procedure that fixes the issue. Otherwise, proceed with the next step. Replace drives 309

310 Replace an ECS storage disk in third-party hardware Note If you have a disk ID from the ECS Portal, use the following command with grep to get the serial number of the disk in the sixth column. In this example, the Serial Number is AR31021EG62L9C: # cs_hwmgr disk --list-by-service Object grep a1d7c15f b d0154 a1d7c15f b d AR31021EG62L9C... Now use the cs_hwmgr drives --list command with grep and the Serial Number to get the slot ID in the last (sixth) column. In this example, the slot ID is A06: # cs_hwmgr drives --list grep AR31021EG62L9C ATA HGST HUS726060AL AR31021EG62L9C 6001 Good A06 You now have the information needed to use this procedure. 3. Mark down the Serial Number of the FAILED or SUSPECT drive. 4. If the target drive is SUSPECT: Determine if the drive is in the SUSPECT state because of hardware issues: Continue the replace drive procedure if the SMART Status column in the cs-hal command output shows "reallocated_sector_count" data. This data indicates hardware problems. Stop the replace drive procedure if you do not see "reallocated_sector_count" data. Drives can be in the SUSPECT state for non-hardware issues like connectivity problems. a. Use the cs_hwmgr --list command to get the VendorId and ProductID for the disk, which are found in the first two columns. # cs_hwmgr drive --list grep -i WCAW ATA HUS726060ALA640 WCAW Suspect B11 Note If the cs_hwgmr drive --list grep <disk SN#> command does not return any output for a "Suspect" drive, the drive is not being managed by the hardware manager. Engage ECS Customer Support for assistance. Refer to Knowledge Base entry b. Set the drive health status to "Bad" with the cs_hwmgr drive --sethealth-bad command using the <VendorID>, <ProductID> and <Disk SN#> values. # cs_hwmgr drive --set-health-bad ATA HUS726060ALA640 WCAW Setting drive health of ATA:HUS726060ALA640:WCAW to bad...suceeded c. Use the cs_hwmgr drive --remove command to remove the target drive (or more specifically the disk associated with that drive) from the configuration. Note 310 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

311 Replace an ECS storage disk in third-party hardware that this command does not remove the drive from the output of the cs_hwmgr drive --list command. # cs_hwmgr drive --remove ATA HUS726060ALA640 WCAW Removing drive ATA:HUS726060ALA640:WCAW Suceeded Note If the cs_hwmgr drive --remove command fails, use the cs_hwmgr drive -- force-remove command using the <VendorID>, <ProductID>, <Disk SN#> values. 5. If the target drive is BAD: If the drive's Serial Number did not appear in the cs_hwmgr drive --list command output, skip this step. If the If the drive health reported using the cs_hwmgr drive --list command is "Bad", continue with the substep a below. a. Use the cs_hwmgr drive --remove command to remove the target drive (or more specifically the disk associated with that drive) from the configuration. Note that this command does not remove the drive from the output of the cs_hwmgr drive --list command. # cs_hwmgr drive --remove ATA HUS726060ALA640 WCAW Removing drive ATA:HUS726060ALA640:WCAW Suceeded Note If the cs_hwmgr drive --remove command fails, use the cs_hwmgr drive -- force-remove command using the <VendorID>, <ProductID>, <Disk SN#> values. 6. Use the cs_hal led command and the serial number to blink the LED for the drive (to find the proper drive in the drawer). # cs_hal led WCAW blink 7. Following the instructions from your hardware manufacturer, locate the disk and replace it in the same slot. 8. Use the cs_hal list disks command to confirm that the system recognizes the new drive. The new drive has the same Slot ID as the replaced drive. Take note of the new Serial Number. The system automatically integrates the new disk into use by the appropriate services. 9. Use the cs_hal led command and the new Serial Number to turn off the LED for the drive. # cs_hal led <serial_number_of_new_drive> off Results The node and system automatically detect the drive and initialize it for use. Replace drives 311

312 Replace an ECS storage disk in third-party hardware 312 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

313 CHAPTER 33 Add a 60-disk upgrade to a U-Series ECS Appliance 60-disk upgrade planning cs_hal commands Perform a 60-disk upgrade Add a 60-disk upgrade to a U-Series ECS Appliance 313

314 Add a 60-disk upgrade to a U-Series ECS Appliance 60-disk upgrade planning Planning considerations for a 60-disk upgrade for an ECS U-Series appliance A 60-disk upgrade adds 15 disks each to four DAE/node pairs. Determine which racks will be upgraded by referencing the sales order form to locate the PSNT number. The target racks will have this number on a tag prominently affixed to the rear of the rack. Before beginning, determine if the target nodes are running ECS 1.2 (or later) software or an older version of ECS software. The procedure is slightly different depending on the software version Before beginning the upgrade procedure, make sure the following rules will be followed: A DAE must have either 15, 30, 45, or 60 disks. The disk drives must be installed according to the layouts shown in the figures below. The bottom four DAEs must contain the same number of disk drives. The bottom four DAEs must have 60 disks before the rack can be upgraded with four more DAE/node pairs. The top four DAEs must contain the same number of disk drives. All disk drives in a DAE must be the same size and speed. If a DAE has bad (failed) disk drives, the upgrade procedure can proceed without replacing the bad disk drives. The first figure shows the minimum ECS U-Series Appliance configuration. Disk drive slots A0 through A11 are always populated. Diks drive slots B0 through B2 are always populated. 314 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

315 Add a 60-disk upgrade to a U-Series ECS Appliance Figure disk layout Determine which drive slots should have their filler removed and disk drive installed using the figures shown below. To upgrade from 15 to 30 disks per DAE: Populate disk drive slots B3 through B11. Populate disk drive slots C0 through C5. 60-disk upgrade planning 315

316 Add a 60-disk upgrade to a U-Series ECS Appliance Figure disk layout To upgrade from 30 to 45 disk drives: Populate disk drive slots C6 through C11. Populate disk drive slots D0 through D EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

317 Add a 60-disk upgrade to a U-Series ECS Appliance Figure disk layout To upgrade from 45 to 60 disk drives: Populate disk drive slots D9 through D11. Populate disk drive slots E0 through E disk upgrade planning 317

318 Add a 60-disk upgrade to a U-Series ECS Appliance Figure disk layout cs_hal commands Introduces the cs_hal commands used in this procedure. cs_hal list disks command In the abbreviated cs_hal list disks output below, notice the different types of rows: The first three rows represent the RAID structure of the node's two internal drives. The next row shows a GOOD storage drive in the SMART Status column. Device names, slot numbers, and serial numbers can be used in cs_hal commands as long as they are unique. The enclosure (DAE) name can also be used in cs_hal commands. The next row represents a FAILED storage drive. The Partition Name indicates the disk on the corresponding drive is assigned to the Object service, the disk is formatted, and the disk health is Bad. Note Use the value in the SMART Status column when determining the current health of the disk drive. The next row represents a SUSPECT storage drive. The last row represents either an empty slot or an undetectable drive. 318 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

319 Add a 60-disk upgrade to a U-Series ECS Appliance The DiskSet column is reserved for future use. [root@layton-cyan ~]# cs_hal list disks Disks(s): SCSI Device Block Device Enclosure Partition Name Slot Serial Number SMART Status DiskSet n/a /dev/md0 RAID vol n/a n/a not supported n/a /dev/sg4 /dev/sdb internal 0 KLH6DHXJ GOOD /dev/sg5 /dev/sdc internal 1 KLH6DM1J GOOD /dev/sg8 /dev/sdf /dev/sg0 Object:Formatted:Good A08 WCAW GOOD /dev/sg9 /dev/sdg /dev/sg0 Object:Formatted:Bad A09 WCAW FAILED: self-test fail; read element; /dev/sg10 /dev/sdh /dev/sg0 Object:Formatted:Suspect A10 WCAW SUSPECT: Reallocated_Sector_Count(5)=11... unavailable /dev/sg0 E05 internal: 2 external: 30 total disks: 32 cs_hal list daes Use this command to find the enclosure ID of the DAE paired with the node. # cs_hal list daes Enclosure(s): SCSI Device Ext Disks /dev/sg2 15 total: 1 cs_hal list node Use this command to find the name of the node you are working on. This lets you identify the DAE associated with the node by referring to a diagram. # cs_hal list node Node(s): Name HBAs Enclosures Int Disks Ext Disks provo-sage Perform a 60-disk upgrade Add 15 disks to four disk array enclosures (DAEs) to complete a 60-disk upgrade. Before you begin Install disk drives on one node/dae pair at a time. Procedure 1. To access the ECS rack using the private ( xxx) network from a laptop: a. From the rear of the rack, locate the 1 GbE private switch network ports by opening the rear door. Perform a 60-disk upgrade 319

320 Add a 60-disk upgrade to a U-Series ECS Appliance b. On the Arista 1 GbE (turtle) switch, attach a network cable from your laptop to port 24 on the switch. Figure 89 Locate port 24 on the private 1 GbE switch c. Set the network interface on the laptop to the static address , subnet mask , with no gateway required. d. Verify that the temporary network between the laptop and rack's private management network is functioning by using the ping command. C:\>ping Pinging with 32 bytes of data: Reply from : bytes=32 time<1ms TTL=64 Reply from : bytes=32 time<1ms TTL=64 Reply from : bytes=32 time<1ms TTL=64 Reply from : bytes=32 time<1ms TTL=64 Ping statistics for : Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms Note If does not answer, try If neither responds, verify the laptop IP/subnet mask, network connection, and switch port connection. e. Start an ssh session with the first node. 2. Use the cs_hal list disks command: # cs_hal list disks Note Disk drive slots listed as occupied in the output should match your expectations from the planning phase. 320 EMC Elastic Cloud Storage (ECS) 2.0 ECS Documentation

321 Add a 60-disk upgrade to a U-Series ECS Appliance 3. Locate the correct DAE: a. Use the cs_hal list node command to verify the name of the node: # cs_hal list node Node(s): Name HBAs Enclosures Int Disks Ext Disks provo-sage b. Use the following wiring diagram to physically identify the correct DAE: Figure 90 U-Series ECS Appliance wiring 4. Open the DAE which you identified with the wiring diagram. 5. Remove fillers and populate disk drives in slots as shown in the appropriate disk layout diagram. Perform a 60-disk upgrade 321

TECHNICAL WHITE PAPER: ELASTIC CLOUD STORAGE SOFTWARE ARCHITECTURE

TECHNICAL WHITE PAPER: ELASTIC CLOUD STORAGE SOFTWARE ARCHITECTURE TECHNICAL WHITE PAPER: ELASTIC CLOUD STORAGE SOFTWARE ARCHITECTURE Deploy a modern hyperscale storage platform on commodity infrastructure ABSTRACT This document provides a detailed overview of the EMC

More information

Elastic Cloud Storage (ECS)

Elastic Cloud Storage (ECS) Elastic Cloud Storage (ECS) Version 2.2.1 Planning Guide 302-002-790 02 Copyright 2013-2016 EMC Corporation. All rights reserved. Published in the USA. Published May, 2016 EMC believes the information

More information

ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE

ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE Hadoop Storage-as-a-Service ABSTRACT This White Paper illustrates how EMC Elastic Cloud Storage (ECS ) can be used to streamline the Hadoop data analytics

More information

EMC Data Domain Management Center

EMC Data Domain Management Center EMC Data Domain Management Center Version 1.1 Initial Configuration Guide 302-000-071 REV 04 Copyright 2012-2015 EMC Corporation. All rights reserved. Published in USA. Published June, 2015 EMC believes

More information

EMC ViPR Controller. Version 2.4. User Interface Virtual Data Center Configuration Guide 302-002-416 REV 01 DRAFT

EMC ViPR Controller. Version 2.4. User Interface Virtual Data Center Configuration Guide 302-002-416 REV 01 DRAFT EMC ViPR Controller Version 2.4 User Interface Virtual Data Center Configuration Guide 302-002-416 REV 01 DRAFT Copyright 2014-2015 EMC Corporation. All rights reserved. Published in USA. Published November,

More information

www.basho.com Technical Overview Simple, Scalable, Object Storage Software

www.basho.com Technical Overview Simple, Scalable, Object Storage Software www.basho.com Technical Overview Simple, Scalable, Object Storage Software Table of Contents Table of Contents... 1 Introduction & Overview... 1 Architecture... 2 How it Works... 2 APIs and Interfaces...

More information

Barracuda Link Balancer Administrator s Guide

Barracuda Link Balancer Administrator s Guide Barracuda Link Balancer Administrator s Guide Version 1.0 Barracuda Networks Inc. 3175 S. Winchester Blvd. Campbell, CA 95008 http://www.barracuda.com Copyright Notice Copyright 2008, Barracuda Networks

More information

EMC ViPR Controller. User Interface Virtual Data Center Configuration Guide. Version 2.4 302-002-416 REV 01

EMC ViPR Controller. User Interface Virtual Data Center Configuration Guide. Version 2.4 302-002-416 REV 01 EMC ViPR Controller Version 2.4 User Interface Virtual Data Center Configuration Guide 302-002-416 REV 01 Copyright 2014-2015 EMC Corporation. All rights reserved. Published in USA. Published November,

More information

EMC SCALEIO OPERATION OVERVIEW

EMC SCALEIO OPERATION OVERVIEW EMC SCALEIO OPERATION OVERVIEW Ensuring Non-disruptive Operation and Upgrade ABSTRACT This white paper reviews the challenges organizations face as they deal with the growing need for always-on levels

More information

Networking Guide Redwood Manager 3.0 August 2013

Networking Guide Redwood Manager 3.0 August 2013 Networking Guide Redwood Manager 3.0 August 2013 Table of Contents 1 Introduction... 3 1.1 IP Addresses... 3 1.1.1 Static vs. DHCP... 3 1.2 Required Ports... 4 2 Adding the Redwood Engine to the Network...

More information

EMC Data Protection Search

EMC Data Protection Search EMC Data Protection Search Version 1.0 Security Configuration Guide 302-001-611 REV 01 Copyright 2014-2015 EMC Corporation. All rights reserved. Published in USA. Published April 20, 2015 EMC believes

More information

VMware vcenter Log Insight Getting Started Guide

VMware vcenter Log Insight Getting Started Guide VMware vcenter Log Insight Getting Started Guide vcenter Log Insight 2.0 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by

More information

EMC ViPR Controller. ViPR Controller REST API Virtual Data Center Configuration Guide. Version 2.3.0.0 302-002-070 01

EMC ViPR Controller. ViPR Controller REST API Virtual Data Center Configuration Guide. Version 2.3.0.0 302-002-070 01 EMC ViPR Controller Version 2.3.0.0 ViPR Controller REST API Virtual Data Center Configuration Guide 302-002-070 01 Copyright 2013-2015 EMC Corporation. All rights reserved. Published in USA. Published

More information

Symantec Database Security and Audit 3100 Series Appliance. Getting Started Guide

Symantec Database Security and Audit 3100 Series Appliance. Getting Started Guide Symantec Database Security and Audit 3100 Series Appliance Getting Started Guide Symantec Database Security and Audit 3100 Series Getting Started Guide The software described in this book is furnished

More information

Cisco TelePresence Management Suite Provisioning

Cisco TelePresence Management Suite Provisioning Cisco TelePresence Management Suite Provisioning Troubleshooting guide D14427.03 December 2010 Introduction Table of Contents Introduction... 3 Provisioning logs... 4 Cisco TMS provisioning directory logs...

More information

Symantec NetBackup OpenStorage Solutions Guide for Disk

Symantec NetBackup OpenStorage Solutions Guide for Disk Symantec NetBackup OpenStorage Solutions Guide for Disk UNIX, Windows, Linux Release 7.6 Symantec NetBackup OpenStorage Solutions Guide for Disk The software described in this book is furnished under a

More information

Isilon OneFS. Version 7.2. OneFS Migration Tools Guide

Isilon OneFS. Version 7.2. OneFS Migration Tools Guide Isilon OneFS Version 7.2 OneFS Migration Tools Guide Copyright 2014 EMC Corporation. All rights reserved. Published in USA. Published November, 2014 EMC believes the information in this publication is

More information

EMC CENTERA VIRTUAL ARCHIVE

EMC CENTERA VIRTUAL ARCHIVE White Paper EMC CENTERA VIRTUAL ARCHIVE Planning and Configuration Guide Abstract This white paper provides best practices for using EMC Centera Virtual Archive in a customer environment. The guide starts

More information

Backup Exec Private Cloud Services. Planning and Deployment Guide

Backup Exec Private Cloud Services. Planning and Deployment Guide Backup Exec Private Cloud Services Planning and Deployment Guide Chapter 1 Introducing Backup Exec Private Cloud Services This chapter includes the following topics: About Backup Exec Private Cloud Services

More information

Isilon OneFS. Version 7.2.1. OneFS Migration Tools Guide

Isilon OneFS. Version 7.2.1. OneFS Migration Tools Guide Isilon OneFS Version 7.2.1 OneFS Migration Tools Guide Copyright 2015 EMC Corporation. All rights reserved. Published in USA. Published July, 2015 EMC believes the information in this publication is accurate

More information

VMware vsphere Data Protection

VMware vsphere Data Protection VMware vsphere Data Protection Replication Target TECHNICAL WHITEPAPER 1 Table of Contents Executive Summary... 3 VDP Identities... 3 vsphere Data Protection Replication Target Identity (VDP-RT)... 3 Replication

More information

EMC SYNCPLICITY FILE SYNC AND SHARE SOLUTION

EMC SYNCPLICITY FILE SYNC AND SHARE SOLUTION EMC SYNCPLICITY FILE SYNC AND SHARE SOLUTION Automated file synchronization Flexible, cloud-based administration Secure, on-premises storage EMC Solutions January 2015 Copyright 2014 EMC Corporation. All

More information

Veeam Cloud Connect. Version 8.0. Administrator Guide

Veeam Cloud Connect. Version 8.0. Administrator Guide Veeam Cloud Connect Version 8.0 Administrator Guide April, 2015 2015 Veeam Software. All rights reserved. All trademarks are the property of their respective owners. No part of this publication may be

More information

Big Data Operations Guide for Cloudera Manager v5.x Hadoop

Big Data Operations Guide for Cloudera Manager v5.x Hadoop Big Data Operations Guide for Cloudera Manager v5.x Hadoop Logging into the Enterprise Cloudera Manager 1. On the server where you have installed 'Cloudera Manager', make sure that the server is running,

More information

Interworks. Interworks Cloud Platform Installation Guide

Interworks. Interworks Cloud Platform Installation Guide Interworks Interworks Cloud Platform Installation Guide Published: March, 2014 This document contains information proprietary to Interworks and its receipt or possession does not convey any rights to reproduce,

More information

EMC ViPR Controller. Service Catalog Reference Guide. Version 2.3 XXX-XXX-XXX 01

EMC ViPR Controller. Service Catalog Reference Guide. Version 2.3 XXX-XXX-XXX 01 EMC ViPR Controller Version 2.3 Service Catalog Reference Guide XXX-XXX-XXX 01 Copyright 2015- EMC Corporation. All rights reserved. Published in USA. Published July, 2015 EMC believes the information

More information

VMware vcenter Log Insight Getting Started Guide

VMware vcenter Log Insight Getting Started Guide VMware vcenter Log Insight Getting Started Guide vcenter Log Insight 1.5 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by

More information

EMC ViPR Controller Add-in for Microsoft System Center Virtual Machine Manager

EMC ViPR Controller Add-in for Microsoft System Center Virtual Machine Manager EMC ViPR Controller Add-in for Microsoft System Center Virtual Machine Manager Version 2.3 Installation and Configuration Guide 302-002-080 01 Copyright 2013-2015 EMC Corporation. All rights reserved.

More information

Acronis Storage Gateway

Acronis Storage Gateway Acronis Storage Gateway DEPLOYMENT GUIDE Revision: 12/30/2015 Table of contents 1 Introducing Acronis Storage Gateway...3 1.1 Supported storage backends... 3 1.2 Architecture and network diagram... 4 1.3

More information

Barracuda Link Balancer

Barracuda Link Balancer Barracuda Networks Technical Documentation Barracuda Link Balancer Administrator s Guide Version 2.2 RECLAIM YOUR NETWORK Copyright Notice Copyright 2004-2011, Barracuda Networks www.barracuda.com v2.2-110503-01-0503

More information

Cloudera Manager Training: Hands-On Exercises

Cloudera Manager Training: Hands-On Exercises 201408 Cloudera Manager Training: Hands-On Exercises General Notes... 2 In- Class Preparation: Accessing Your Cluster... 3 Self- Study Preparation: Creating Your Cluster... 4 Hands- On Exercise: Working

More information

CA Performance Center

CA Performance Center CA Performance Center Single Sign-On User Guide 2.4 This Documentation, which includes embedded help systems and electronically distributed materials, (hereinafter referred to as the Documentation ) is

More information

Testing and Restoring the Nasuni Filer in a Disaster Recovery Scenario

Testing and Restoring the Nasuni Filer in a Disaster Recovery Scenario Testing and Restoring the Nasuni Filer in a Disaster Recovery Scenario Version 7.2 November 2015 Last modified: November 3, 2015 2015 Nasuni Corporation All Rights Reserved Document Information Testing

More information

Application Discovery Manager User s Guide vcenter Application Discovery Manager 6.2.1

Application Discovery Manager User s Guide vcenter Application Discovery Manager 6.2.1 Application Discovery Manager User s Guide vcenter Application Discovery Manager 6.2.1 This document supports the version of each product listed and supports all subsequent versions until the document

More information

PATROL Console Server and RTserver Getting Started

PATROL Console Server and RTserver Getting Started PATROL Console Server and RTserver Getting Started Supporting PATROL Console Server 7.5.00 RTserver 6.6.00 February 14, 2005 Contacting BMC Software You can access the BMC Software website at http://www.bmc.com.

More information

How To Install An Aneka Cloud On A Windows 7 Computer (For Free)

How To Install An Aneka Cloud On A Windows 7 Computer (For Free) MANJRASOFT PTY LTD Aneka 3.0 Manjrasoft 5/13/2013 This document describes in detail the steps involved in installing and configuring an Aneka Cloud. It covers the prerequisites for the installation, the

More information

Leverage Your EMC Storage Investment with User Provisioning for Syncplicity:

Leverage Your EMC Storage Investment with User Provisioning for Syncplicity: Leverage Your EMC Storage Investment with User Provisioning for Syncplicity: Automate and simplify Syncplicity user/group management tasks EMC Global Solutions Abstract Make the most of your existing EMC

More information

EMC ViPR for On-Demand File Storage with EMC Syncplicity and EMC Isilon or EMC VNX

EMC ViPR for On-Demand File Storage with EMC Syncplicity and EMC Isilon or EMC VNX EMC ViPR for On-Demand File Storage with EMC Syncplicity and EMC Isilon or EMC VNX EMC Solutions Abstract This document describes how to deploy EMC ViPR software-defined storage in an existing EMC Isilon

More information

RSA Security Analytics. S4 Broker Setup Guide

RSA Security Analytics. S4 Broker Setup Guide RSA Security Analytics S4 Broker Setup Guide Copyright 2010-2013 RSA, the Security Division of EMC. All rights reserved. Trademarks RSA, the RSA Logo and EMC are either registered trademarks or trademarks

More information

VMware Identity Manager Connector Installation and Configuration

VMware Identity Manager Connector Installation and Configuration VMware Identity Manager Connector Installation and Configuration VMware Identity Manager This document supports the version of each product listed and supports all subsequent versions until the document

More information

DEPLOYMENT GUIDE Version 1.2. Deploying F5 with Oracle E-Business Suite 12

DEPLOYMENT GUIDE Version 1.2. Deploying F5 with Oracle E-Business Suite 12 DEPLOYMENT GUIDE Version 1.2 Deploying F5 with Oracle E-Business Suite 12 Table of Contents Table of Contents Introducing the BIG-IP LTM Oracle E-Business Suite 12 configuration Prerequisites and configuration

More information

How To Use Insightiq

How To Use Insightiq Isilon InsightIQ Version 3.0 User Guide Copyright 2010-2014 EMC Corporation. All rights reserved. Published in USA. Published January, 2014 EMC believes the information in this publication is accurate

More information

ACS 5.x and later: Integration with Microsoft Active Directory Configuration Example

ACS 5.x and later: Integration with Microsoft Active Directory Configuration Example ACS 5.x and later: Integration with Microsoft Active Directory Configuration Example Document ID: 113571 Contents Introduction Prerequisites Requirements Components Used Conventions Background Information

More information

Configuring Failover

Configuring Failover Configuring Failover 2015 Bomgar Corporation. All rights reserved worldwide. BOMGAR and the BOMGAR logo are trademarks of Bomgar Corporation; other trademarks shown are the property of their respective

More information

VMware Identity Manager Administration

VMware Identity Manager Administration VMware Identity Manager Administration VMware Identity Manager 2.6 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new

More information

VMware vcenter Operations Manager Administration Guide

VMware vcenter Operations Manager Administration Guide VMware vcenter Operations Manager Administration Guide Custom User Interface vcenter Operations Manager 5.6 This document supports the version of each product listed and supports all subsequent versions

More information

EMC ViPR SRM. Alerting Guide. Version 3.7.1.0 302-002-455 01

EMC ViPR SRM. Alerting Guide. Version 3.7.1.0 302-002-455 01 EMC ViPR SRM Version 3.7.1.0 Alerting Guide 302-002-455 01 Copyright 2015-2016 EMC Corporation. All rights reserved. Published in the USA. Published February, 2016 EMC believes the information in this

More information

Installing and Configuring vcloud Connector

Installing and Configuring vcloud Connector Installing and Configuring vcloud Connector vcloud Connector 2.7.0 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new

More information

WhatsUp Gold v16.3 Installation and Configuration Guide

WhatsUp Gold v16.3 Installation and Configuration Guide WhatsUp Gold v16.3 Installation and Configuration Guide Contents Installing and Configuring WhatsUp Gold using WhatsUp Setup Installation Overview... 1 Overview... 1 Security considerations... 2 Standard

More information

Deploying Remote Desktop Connection Broker with High Availability Step-by-Step Guide

Deploying Remote Desktop Connection Broker with High Availability Step-by-Step Guide Deploying Remote Desktop Connection Broker with High Availability Step-by-Step Guide Microsoft Corporation Published: May 2010 Abstract This guide describes the steps for configuring Remote Desktop Connection

More information

Features of AnyShare

Features of AnyShare of AnyShare of AnyShare CONTENT Brief Introduction of AnyShare... 3 Chapter 1 Centralized Management... 5 1.1 Operation Management... 5 1.2 User Management... 5 1.3 User Authentication... 6 1.4 Roles...

More information

StorSimple Appliance Quick Start Guide

StorSimple Appliance Quick Start Guide StorSimple Appliance Quick Start Guide 5000 and 7000 Series Appliance Software Version 2.1.1 (2.1.1-267) Exported from Online Help on September 15, 2012 Contents Getting Started... 3 Power and Cabling...

More information

Metalogix SharePoint Backup. Advanced Installation Guide. Publication Date: August 24, 2015

Metalogix SharePoint Backup. Advanced Installation Guide. Publication Date: August 24, 2015 Metalogix SharePoint Backup Publication Date: August 24, 2015 All Rights Reserved. This software is protected by copyright law and international treaties. Unauthorized reproduction or distribution of this

More information

Enterprise Manager. Version 6.2. Administrator s Guide

Enterprise Manager. Version 6.2. Administrator s Guide Enterprise Manager Version 6.2 Administrator s Guide Enterprise Manager 6.2 Administrator s Guide Document Number 680-017-017 Revision Date Description A August 2012 Initial release to support version

More information

Netezza PureData System Administration Course

Netezza PureData System Administration Course Course Length: 2 days CEUs 1.2 AUDIENCE After completion of this course, you should be able to: Administer the IBM PDA/Netezza Install Netezza Client Software Use the Netezza System Interfaces Understand

More information

Setting Up a Unisphere Management Station for the VNX Series P/N 300-011-796 Revision A01 January 5, 2010

Setting Up a Unisphere Management Station for the VNX Series P/N 300-011-796 Revision A01 January 5, 2010 Setting Up a Unisphere Management Station for the VNX Series P/N 300-011-796 Revision A01 January 5, 2010 This document describes the different types of Unisphere management stations and tells how to install

More information

MODERNIZING THE DISPERSED ENTERPRISE WITH CLOUD STORAGE GATEWAYS AND OBJECT STORAGE

MODERNIZING THE DISPERSED ENTERPRISE WITH CLOUD STORAGE GATEWAYS AND OBJECT STORAGE MODERNIZING THE DISPERSED ENTERPRISE WITH CLOUD STORAGE GATEWAYS AND OBJECT STORAGE ABSTRACT To reduce the cost and complexity of Remote Office Branch Office IT, many firms are centralizing applications

More information

Zerto Virtual Manager Administration Guide

Zerto Virtual Manager Administration Guide Zerto Virtual Manager Administration Guide AWS Environment ZVR-ADVA-4.0U2-01-23-07-15 Copyright 2015, Zerto Ltd. All rights reserved. Information in this document is subject to change without notice and

More information

VMware vcenter Operations Manager Enterprise Administration Guide

VMware vcenter Operations Manager Enterprise Administration Guide VMware vcenter Operations Manager Enterprise Administration Guide vcenter Operations Manager Enterprise 5.0 This document supports the version of each product listed and supports all subsequent versions

More information

Installation Guide Avi Networks Cloud Application Delivery Platform Integration with Cisco Application Policy Infrastructure

Installation Guide Avi Networks Cloud Application Delivery Platform Integration with Cisco Application Policy Infrastructure Installation Guide Avi Networks Cloud Application Delivery Platform Integration with Cisco Application Policy Infrastructure August 2015 Table of Contents 1 Introduction... 3 Purpose... 3 Products... 3

More information

RealPresence Platform Director

RealPresence Platform Director RealPresence CloudAXIS Suite Administrators Guide Software 1.3.1 GETTING STARTED GUIDE Software 2.0 June 2015 3725-66012-001B RealPresence Platform Director Polycom, Inc. 1 RealPresence Platform Director

More information

VMware vrealize Automation

VMware vrealize Automation VMware vrealize Automation Reference Architecture Version 6.0 and Higher T E C H N I C A L W H I T E P A P E R Table of Contents Overview... 4 What s New... 4 Initial Deployment Recommendations... 4 General

More information

Running a Workflow on a PowerCenter Grid

Running a Workflow on a PowerCenter Grid Running a Workflow on a PowerCenter Grid 2010-2014 Informatica Corporation. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise)

More information

vsphere Replication for Disaster Recovery to Cloud

vsphere Replication for Disaster Recovery to Cloud vsphere Replication for Disaster Recovery to Cloud vsphere Replication 6.0 This document supports the version of each product listed and supports all subsequent versions until the document is replaced

More information

User Manual. Onsight Management Suite Version 5.1. Another Innovation by Librestream

User Manual. Onsight Management Suite Version 5.1. Another Innovation by Librestream User Manual Onsight Management Suite Version 5.1 Another Innovation by Librestream Doc #: 400075-06 May 2012 Information in this document is subject to change without notice. Reproduction in any manner

More information

Emerson Smart Firewall

Emerson Smart Firewall DeltaV TM Distributed Control System Product Data Sheet Emerson Smart Firewall The Emerson Smart Firewall protects the DeltaV system with an easy to use perimeter defense solution. Purpose built for easy

More information

SOA Software API Gateway Appliance 7.1.x Administration Guide

SOA Software API Gateway Appliance 7.1.x Administration Guide SOA Software API Gateway Appliance 7.1.x Administration Guide Trademarks SOA Software and the SOA Software logo are either trademarks or registered trademarks of SOA Software, Inc. Other product names,

More information

Trend Micro Incorporated reserves the right to make changes to this document and to the products described herein without notice.

Trend Micro Incorporated reserves the right to make changes to this document and to the products described herein without notice. Trend Micro Incorporated reserves the right to make changes to this document and to the products described herein without notice. Before installing and using the software, please review the readme files,

More information

Virtual Appliance Setup Guide

Virtual Appliance Setup Guide Virtual Appliance Setup Guide 2015 Bomgar Corporation. All rights reserved worldwide. BOMGAR and the BOMGAR logo are trademarks of Bomgar Corporation; other trademarks shown are the property of their respective

More information

Acronis Backup & Recovery 11.5 Quick Start Guide

Acronis Backup & Recovery 11.5 Quick Start Guide Acronis Backup & Recovery 11.5 Quick Start Guide Applies to the following editions: Advanced Server for Windows Virtual Edition Advanced Server SBS Edition Advanced Workstation Server for Linux Server

More information

Privileged Access Management Upgrade Guide

Privileged Access Management Upgrade Guide Privileged Access Management Upgrade Guide 2015 Bomgar Corporation. All rights reserved worldwide. BOMGAR and the BOMGAR logo are trademarks of Bomgar Corporation; other trademarks shown are the property

More information

Gigabyte Management Console User s Guide (For ASPEED AST 2400 Chipset)

Gigabyte Management Console User s Guide (For ASPEED AST 2400 Chipset) Gigabyte Management Console User s Guide (For ASPEED AST 2400 Chipset) Version: 1.4 Table of Contents Using Your Gigabyte Management Console... 3 Gigabyte Management Console Key Features and Functions...

More information

VMware vsphere Data Protection Evaluation Guide REVISED APRIL 2015

VMware vsphere Data Protection Evaluation Guide REVISED APRIL 2015 VMware vsphere Data Protection REVISED APRIL 2015 Table of Contents Introduction.... 3 Features and Benefits of vsphere Data Protection... 3 Requirements.... 4 Evaluation Workflow... 5 Overview.... 5 Evaluation

More information

An Oracle White Paper January 2013. A Technical Overview of New Features for Automatic Storage Management in Oracle Database 12c

An Oracle White Paper January 2013. A Technical Overview of New Features for Automatic Storage Management in Oracle Database 12c An Oracle White Paper January 2013 A Technical Overview of New Features for Automatic Storage Management in Oracle Database 12c TABLE OF CONTENTS Introduction 2 ASM Overview 2 Total Storage Management

More information

VMware vcenter Log Insight Administration Guide

VMware vcenter Log Insight Administration Guide VMware vcenter Log Insight Administration Guide vcenter Log Insight 1.5 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by

More information

RSA Security Analytics Virtual Appliance Setup Guide

RSA Security Analytics Virtual Appliance Setup Guide RSA Security Analytics Virtual Appliance Setup Guide Copyright 2010-2015 RSA, the Security Division of EMC. All rights reserved. Trademarks RSA, the RSA Logo and EMC are either registered trademarks or

More information

Setup Guide Access Manager Appliance 3.2 SP3

Setup Guide Access Manager Appliance 3.2 SP3 Setup Guide Access Manager Appliance 3.2 SP3 August 2014 www.netiq.com/documentation Legal Notice THIS DOCUMENT AND THE SOFTWARE DESCRIBED IN THIS DOCUMENT ARE FURNISHED UNDER AND ARE SUBJECT TO THE TERMS

More information

Deployment and Configuration Guide

Deployment and Configuration Guide vcenter Operations Manager 5 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for more recent editions

More information

이 기기는 업무용 급 으로 전자파적합등록을 한 기기이오니 판매자 또는 사용자는 이점을 주의하시기 바라며 가정 외의 지역에서 사용하는 것을 목적으로 합니다

이 기기는 업무용 급 으로 전자파적합등록을 한 기기이오니 판매자 또는 사용자는 이점을 주의하시기 바라며 가정 외의 지역에서 사용하는 것을 목적으로 합니다 020-101186-01 020-101186-01 이 기기는 업무용 급 으로 전자파적합등록을 한 기기이오니 판매자 또는 사용자는 이점을 주의하시기 바라며 가정 외의 지역에서 사용하는 것을 목적으로 합니다 Table of Contents About this Document... 1 Document Conventions... 1 Audience... 1 Related

More information

Testing and Restoring the Nasuni Filer in a Disaster Recovery Scenario

Testing and Restoring the Nasuni Filer in a Disaster Recovery Scenario Testing and Restoring the Nasuni Filer in a Disaster Recovery Scenario Version 7.0 July 2015 2015 Nasuni Corporation All Rights Reserved Document Information Testing Disaster Recovery Version 7.0 July

More information

WatchDox Administrator's Guide. Application Version 3.7.5

WatchDox Administrator's Guide. Application Version 3.7.5 Application Version 3.7.5 Confidentiality This document contains confidential material that is proprietary WatchDox. The information and ideas herein may not be disclosed to any unauthorized individuals

More information

EMC IRODS RESOURCE DRIVERS

EMC IRODS RESOURCE DRIVERS EMC IRODS RESOURCE DRIVERS PATRICK COMBES: PRINCIPAL SOLUTION ARCHITECT, LIFE SCIENCES 1 QUICK AGENDA Intro to Isilon (~2 hours) Isilon resource driver Intro to ECS (~1.5 hours) ECS Resource driver Possibilities

More information

WOS Cloud. ddn.com. Personal Storage for the Enterprise. DDN Solution Brief

WOS Cloud. ddn.com. Personal Storage for the Enterprise. DDN Solution Brief DDN Solution Brief Personal Storage for the Enterprise WOS Cloud Secure, Shared Drop-in File Access for Enterprise Users, Anytime and Anywhere 2011 DataDirect Networks. All Rights Reserved DDN WOS Cloud

More information

Installing Management Applications on VNX for File

Installing Management Applications on VNX for File EMC VNX Series Release 8.1 Installing Management Applications on VNX for File P/N 300-015-111 Rev 01 EMC Corporation Corporate Headquarters: Hopkinton, MA 01748-9103 1-508-435-1000 www.emc.com Copyright

More information

System Compatibility. Enhancements. Email Security. SonicWALL Email Security 7.3.2 Appliance Release Notes

System Compatibility. Enhancements. Email Security. SonicWALL Email Security 7.3.2 Appliance Release Notes Email Security SonicWALL Email Security 7.3.2 Appliance Release Notes System Compatibility SonicWALL Email Security 7.3.2 is supported on the following SonicWALL Email Security appliances: SonicWALL Email

More information

Apache CloudStack 4.x (incubating) Network Setup: excerpt from Installation Guide. Revised February 28, 2013 2:32 pm Pacific

Apache CloudStack 4.x (incubating) Network Setup: excerpt from Installation Guide. Revised February 28, 2013 2:32 pm Pacific Apache CloudStack 4.x (incubating) Network Setup: excerpt from Installation Guide Revised February 28, 2013 2:32 pm Pacific Apache CloudStack 4.x (incubating) Network Setup: excerpt from Installation Guide

More information

CDH AND BUSINESS CONTINUITY:

CDH AND BUSINESS CONTINUITY: WHITE PAPER CDH AND BUSINESS CONTINUITY: An overview of the availability, data protection and disaster recovery features in Hadoop Abstract Using the sophisticated built-in capabilities of CDH for tunable

More information

EMC Documentum Connector for Microsoft SharePoint

EMC Documentum Connector for Microsoft SharePoint EMC Documentum Connector for Microsoft SharePoint Version 7.1 Installation Guide EMC Corporation Corporate Headquarters Hopkinton, MA 01748-9103 1-508-435-1000 www.emc.com Legal Notice Copyright 2013-2014

More information

Syncplicity On-Premise Storage Connector

Syncplicity On-Premise Storage Connector Syncplicity On-Premise Storage Connector Implementation Guide Abstract This document explains how to install and configure the Syncplicity On-Premise Storage Connector. In addition, it also describes how

More information

CA Unified Infrastructure Management

CA Unified Infrastructure Management CA Unified Infrastructure Management Probe Guide for IIS Server Monitoring iis v1.7 series Copyright Notice This online help system (the "System") is for your informational purposes only and is subject

More information

User's Guide - Beta 1 Draft

User's Guide - Beta 1 Draft IBM Tivoli Composite Application Manager for Microsoft Applications: Microsoft Cluster Server Agent vnext User's Guide - Beta 1 Draft SC27-2316-05 IBM Tivoli Composite Application Manager for Microsoft

More information

SECURE, ENTERPRISE FILE SYNC AND SHARE WITH EMC SYNCPLICITY UTILIZING EMC ISILON, EMC ATMOS, AND EMC VNX

SECURE, ENTERPRISE FILE SYNC AND SHARE WITH EMC SYNCPLICITY UTILIZING EMC ISILON, EMC ATMOS, AND EMC VNX White Paper SECURE, ENTERPRISE FILE SYNC AND SHARE WITH EMC SYNCPLICITY UTILIZING EMC ISILON, EMC ATMOS, AND EMC VNX Abstract This white paper explains the benefits to the extended enterprise of the on-

More information

Virtual Web Appliance Setup Guide

Virtual Web Appliance Setup Guide Virtual Web Appliance Setup Guide 2 Sophos Installing a Virtual Appliance Installing a Virtual Appliance This guide describes the procedures for installing a Virtual Web Appliance. If you are installing

More information

EMC ViPR Controller. Backup and Restore Guide. Version Darth 302-XXX-XXX 01

EMC ViPR Controller. Backup and Restore Guide. Version Darth 302-XXX-XXX 01 EMC ViPR Controller Version Darth Backup and Restore Guide 302-XXX-XXX 01 Copyright 2015- EMC Corporation. All rights reserved. Published in USA. Published November, 2015 EMC believes the information in

More information

Windows Server Failover Clustering April 2010

Windows Server Failover Clustering April 2010 Windows Server Failover Clustering April 00 Windows Server Failover Clustering (WSFC) is the successor to Microsoft Cluster Service (MSCS). WSFC and its predecessor, MSCS, offer high availability for critical

More information

Alfresco Enterprise on AWS: Reference Architecture

Alfresco Enterprise on AWS: Reference Architecture Alfresco Enterprise on AWS: Reference Architecture October 2013 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 13 Abstract Amazon Web Services (AWS)

More information

VMware vrealize Automation

VMware vrealize Automation VMware vrealize Automation Reference Architecture Version 6.0 or Later T E C H N I C A L W H I T E P A P E R J U N E 2 0 1 5 V E R S I O N 1. 5 Table of Contents Overview... 4 What s New... 4 Initial Deployment

More information

IaaS Configuration for Cloud Platforms

IaaS Configuration for Cloud Platforms vrealize Automation 6.2.3 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for more recent editions

More information

Carisbrooke. End User Guide

Carisbrooke. End User Guide Carisbrooke Contents Contents... 2 Introduction... 3 Negotiate Kerberos/NTLM... 4 Scope... 4 What s changed... 4 What hasn t changed... 5 Multi-Tenant Categories... 6 Scope... 6 What s changed... 6 What

More information