Extreme Networks: Big Data A SOLUTIONS GUIDE

Transcription

1 Extreme Networks: Big Data A SOLUTIONS GUIDE

2 Copyright 2014 Extreme Networks, Inc. All Rights Reserved. AccessAdapt, Alpine, Altitude, BlackDiamond, Direct Attach, EPICenter, ExtremeWorks Essentials, Ethernet Everywhere, Extreme Enabled, Extreme Ethernet Everywhere, Extreme Networks, Extreme Standby Router Protocol, Extreme Turbodrive, Extreme Velocity, ExtremeWare, ExtremeWorks, ExtremeXOS, Go Purple Extreme Solution, ExtremeXOS ScreenPlay, ReachNXT, Ridgeline, Sentriant, ServiceWatch, Summit, SummitStack, Triumph, Unified Access Architecture, Unified Access RF Manager, UniStack, XNV, the Extreme Networks logo, the Alpine logo, the BlackDiamond logo, the Extreme Turbodrive logo, the Summit logos, and the Powered by ExtremeXOS logo are trademarks or registered trademarks of Extreme Networks, Inc. or its subsidiaries in the United States and/or other countries. sflow is the property of InMon Corporation. Specifications are subject to change without tice. All other registered trademarks, trademarks, and service marks are property of their respective owners. For additional information on Extreme Networks trademarks, see Big Data Solutions Guide 2

3 Table of Contents Big Data Requirements... 5 Architecture... 6 LAYER 3 AND ECMP...8 LAYER 2 ALTERNATIVES...8 Scalability... 8 Performance... 9 THROUGHPUT...9 CONGESTION MANAGEMENT...11 JUMBO FRAMES...11 LATENCY High Availability...12 LOAD-SHARING REDUNDANCY RECOVERY BFD Manageability...16 MANAGING THE CLUSTER WITH NETSIGHT RACK AWARENESS AUTO-PROVISIONING FLOW ANALYSIS PROACTIVE TECH SUPPORT CLI SCRIPTING Big Data Solutions Guide 3

4 Security...21 LIMITING UNAUTHORIZED HOSTS DENIAL OF SERVICE (DOS) PROTECTION Appendix...22 LEAF SWITCH SAMPLE CONFIGURATION...22 SPINE SWITCH SAMPLE CONFIGURATION...28 RACK AWARENESS SAMPLE SCRIPT Big Data Solutions Guide 4

5 BIG DATA SOLUTIONS GUIDE Abstract: Big Data analytics are fueling the multi-billion dollar investment to integrate Big Data applications into business operations, and they need a solid infrastructure to run on. Extreme Networks delivers a Big Data solution with a high-performing Ethernet-based architecture. Best-in-class switches and operating system offer a simple and affordable solution that is highly resilient. This paper discusses how Extreme s Big Data solution meets the five pillars of Big Data: Scalability Performance, High Availability, Manageability, and Security. Big Data Requirements Big Data analytics are fueling the multi-billion dollar investment to integrate Big Data applications into business operations, and they need a solid infrastructure to run on. That s why Extreme Networks delivers a Big Data solution with a high-performance Ethernet-based architecture. Extreme Networks best-in-class switches and operating system offer a simple and affordable solution that is also highly resilient. With the release of Hadoop 2.0 by the Apache open source community, Hadoop YARN provides the resource management and pluggable architecture for enabling a wide variety of data access methods. Along with the techlogies provided by Cloudera, Hortonworks, Pivotal, DataStax, MongoDB and other NoSQL applications, YARN also supports emerging use cases for search and streaming with Apache Solr, Spark and Storm. Big Data applications are evolving rapidly and enterprise architects can confidently deploy Extreme Networks best-in-class solution kwing that it will handle current and future diverse workloads. Extreme Networks hardware and software are designed to support various data processing requirements. 1. Online NOSQL HBase and Cassandra provide databases with support for massive tables (billions of rows, millions of columns) with fast random access 2. Batch SQL Techlogies such as Hive are designed for batch queries on Hadoop and are used primarily for queries on very large data sets and large ETL jobs 3. Analytic SQL Impala, HAWQ, Drill and Stinger provide interactive query capabilities to enable traditional business intelligence and analytics on Hadoop-scale databases 4. In-Memory Machine Learning and Stream Processing In-memory computing for Hadoop is called Shark, which is a new project that also uses in-memory computing while retaining full Hive compatibility to provide x faster queries than Hive. The growing Big Data application ecosystem is accelerating Big Data deployments they are longer isolated to just the research community or top Enterprises. Big Data is becoming just data that Enterprises are integrating into mission critical operations in order to manage all the enterprise data for new insights and to gain a competitive advantage. Big Data Solutions Guide 5

6 The network architecture has to meet the same requirements as the whole solution. Next-Generation clusters demand high performance from a wire-speed architecture that provides high throughput and low latency. Big Data processes massive amounts of data (both structured and unstructured) in distributed systems and the network is part of the critical path for these clusters because of the nature of the distributed data systems with an east-west traffic flow that need a network fabric for interconnecitivity. This paper discusses how Extreme Networks best-inclass Big Data solution meets the five pillars of Big Data: 1. Scalability 2. Performance 3. High Availability 4. Manageability 5. Security Architecture Big Data requires a scalable way to store and process an ever-growing massive amount of data, which inspired the open-source development of Apache Hadoop. The Hadoop Distributed Files System (HDFS) is designed to span a large cluster of servers that pool data in any format into a single scalable and reliable data storage file system. Hadoop s MapReduce is a distributed parallel processing framework that is used to retrieve data from the Hadoop cluster. HDFS and MapReduce work in tandem to process data with locality in consideration, executing the compute processing at the data location. By distributing the storage and processing across many servers, the cluster can stretch and scale with the demand. With data distributed across multiple servers, the same data is often replicated across multiple des and intentionally across multiple racks for redundancy. During rmal MapReduce operations, a high volume of data is shuffled between datades and then reported back to the user. In addition, HDFS was designed with the assumption that hardware failures are the rm rather than the exception, which is more probable in environments where there may be hundreds or thousands of datades. A datade failure can result in Hadoop managing the task workload across the still-functioning datades and can also result in spikes of network bandwidth usage to replicate the data blocks that had been stored on the failed datade. Applications must detect and address failures quickly, but the infrastructure should detect and address failures quickly. The high bandwidth and high density requirements of Big Data can be achieved with a Layer 3 Ethernet-based topology that is optimized for the east-west traffic patterns of the typical workloads between the datades within the cluster. The Layer 3 Equal Cost Multi Path (ECMP) fabric interconnects the Layer 3 enabled leaf switches to a Layer 3 spine, as shown below: The topology is a flat two-stage Clos switch fabric comprising: Big Data Solutions Guide 6

7 1. First stage: top-of-rack leaf switches that connect to compute resources 2. Second stage: core spine switches that interconnect the leaf switches The leaf switches attached to any given rack are t directly connected to leaf switches belonging to other racks because this would require full mesh connectivity between the leafs. Instead, the datapath between the leafs is via uplinks to the spines, which requires the leaf to be connected to every spine switch instead of every leaf. This modern architecture has low oversubscription which is ideal for Big Data workloads. The Extreme Networks Big Data architecture easily scales with the size of the Big Data cluster. Smaller clusters can start with a lower number of leaf switches and add on as the cluster grows. The larger clusters can easily scale out the leaf switches and accommodate the ever-growing densities within each rack: dozens of datades, hundreds of cores, and exabytes of internal storage. In turn, this puts higher and higher demands on the network since larger clusters will drive port densities and network bandwidth utilization. Extreme Networks Big Data solution is also based on the most popular transport techlogy Ethernet which makes it easy to manage and integrate into existing Ethernet networks. Extreme Networks already delivers Gbps, 40Gbps, 10Gbps and 1Gbps solutions, and Ethernet techlogies remain ubiquitous and competitive in its flexibility to provide performant solutions. For example Data Center Bridging (DCB) and future industry advancements like 400Gbps and RoCE (RDMA over Converged Ethernet) and iwarp (Internet Wide Area RDMA Protocol), which helps reduce datades Ethernet processor overhead. Big Data Solutions Guide 7

8 LAYER 3 AND ECMP For IP reachability between the datades in the cluster, the network fabric is simple in that it uses Equal Cost Multi Path (ECMP) and Open Shorted Path First (OSPF) for routing. Any rack is equidistant to any other rack with a maximum traversal of three switches (leaf spine leaf), achieving low and predictable latency, and there are multiple paths to loadshare across improving link utilization and providing redundancy. Leaf switches and spine switches participate in OSPF and perform Layer 3 forwarding. Leaf switches make forwarding decisions based on Layer 3 lookups of the IP address of the destination datade, which do t need Layer 2 adjacencies between one ather, just IP reachability. The routing tables on the Extreme Networks switches can easily scale to support the number of routes in the network plus the rack subnets. Thus the network topology with OSPF as a routing protocol scales to support small to large clusters while maintaining performance, and allows users to easily add/remove compute resources. LAYER 2 ALTERNATIVES Extreme Networks also offers a pure Layer 2 solution where the network may be configured completely without Layer 3 by leveraging Multi-Switch Link Aggregation (MLAG) in the network fabric between leaf switches and spine switches. MLAG provides device and network level resiliency, but then the whole network is in a single broadcast domain, so it is a suitable configuration only for small deployments. Scalability Big Data HDFS-based applications leverage a distributed environment to process data on multiple servers in parallel and can run multiple jobs at the same time. Generally speaking, job completion times can be decreased as the datades scale up, so as new racks are added to the cluster the Extreme Networks architecture maintains optimized performance and bandwidth utilization between the racks. Customers can start with a small number of servers and then expand the infrastructure as the need grows. Extreme Networks best-in-class ExtremeXOS software is a single operating system that runs across the switch portfolio and provides flexibility to deploy various Extreme Networks switches depending on the requirements. This paper focuses on the Summit X670 series switches as leaf switches and Summit X770 Series switches as spine switches, where each leaf switch has 10 GbE links down to each server and 40 GbE links up to each spine. The Summit X770 provides 32 40bE ports and for the above architecture, 4 ports connect to each pair of Summit X670 leaf switches, supporting 8 pairs of X670 switches (i.e., 8 compute racks). Each pair of X670 switches can connect to 40 servers (assuming 8x10Gbps ports are for the ISC and 4x40Gbps ports are uplinks to the spine switches), so this architecture will scale out to 40x8 = 320 servers. Higher-density Big Data deployments can also leverage the Summit X770 Series switches as leaf switches and the BlackDiamond X8 at the spine. Lower-density deployments can leverage any of the other Extreme Networks Summit switches, such as the Summit X460. Extreme Networks offers the flexibility for those lowerdensity deployments to leverage either the Layer 3 design which is the focus of this paper, or the Layer 2 MLAG-based design mentioned earlier. Big Data Solutions Guide 8

9 Performance THROUGHPUT HDFS brings data to the compute des and moves a massive amount of data around the network, and server or rack failures will cause a lot of data to be replicated elsewhere on other servers. The time it takes to process tasks depends on how fast the network can move that data. Extreme Networks provides higher speeds and feeds than anyone else in the industry with 10Gbps/40Gbps/Gbps Ethernet with higher density per slot. Extreme Networks portfolio includes 10Gbps down to datades to support emerging 10Gbps-enabled servers, especially with LAN-on-Motherboard (LOM) capabilities that are driving down the cost of 10Gbps networking, and helps optimize new server deployments with higher bandwidth uplinks. As well Extreme Networks offers 1Gbps down to datades. Big Data clusters need to move data as quickly as possible between datades, and the Exreme Big Data solution supports line rate forwarding for Layer 2 and Layer 3 traffic. See the independent Lippis report shown below: Big Data Solutions Guide 9

10 Open Industry Network Performance & Power Test for Private/Public Data Center Cloud Computing Ethernet Fabrics Report: Evaluating 10 GbE Switches Extreme Networks Summit X670V Top-of-Rack Switch The Extreme Networks X670V demonstrated % throughput as a percentage of line rate across all 48-10GbE and 4-40GbE ports. In other words, t a single packet was dropped while the Extreme Networks X670V was presented with eugh traffic to populate its 48-10GbE and 4-40GbE ports at line rate simultaneously for both L2 and L3 traffic flows. Extreme Networks Summit X670V RFC 2544 L2 & L3 Throughput Test Throughput % Line Rate % 80% 60% 40% 20% Layer 2 Layer 3 Two congestion test were conducted using the same methodology. A 10GbE and 40GbE congestion test stressed the X670V ToR switch s congestion management attributes for both 10GbE and 40GbE. The Extreme Networks X670V demonstrated % of aggregated forwarding rate as percentage of line rate during congestion conditions for both the 10GbE and 40GbE test. A single 10GbE port was flooded at 150% of line rate. In addition, a single 40GbE port was flooded at 150% of line rate. The Extreme Networks X670V did t use HOL blocking which means that as the 10GbE and 40GbE ports on the X670V became congested, it did t impact the performance of other ports. Back pressure was detected. The X670V did send flow control frames to the Ixia test gear signaling it to slow down the rate of incoming traffic flow, which is common in ToR switches. Frame (Bytes) % Layer 2 Layer 3 Extreme Networks Summit X670V Congestion Test 150% of Line Rate into a single 10GbE Agg Forwarding Rate (% Line Rate) 150% of Line Rate into a single 40GbE Head of Line Blocking Back Pressure Agg Flow Control Frames LAYER 2 LAYER Lippis Enterprises, Inc Evaluation conducted at Ixia s isimcity Santa Clara Lab on Ixia test equipment Big Data Solutions Guide 10

11 CONGESTION MANAGEMENT Bursty traffic patterns can happen in Big Data clusters, such as during the shuffle phase of a MapReduce job when multiple mappers terminate and transmit data to reducers. Other periods of transient congestion can occur when multiple traffic flows are trying to go to the same output port on a switch. To manage these situations, Extreme Networks switches leverage a Smart Buffer techlogy that transparently self-tunes and dynamically allows buffers to be utilized as needed during periods of congestion. Extreme Networks Smart Buffer techlogy provides a dynamic and adaptive onchip buffer allocation scheme that is surperior to static per-port allocation schemes and avoids latency incurred by off-chip buffers. Ports have dedicated buffers and in addition can get extra buffer allocation from a shared pool as needed, thereby demonstrating an effective management of and tolerance for microbursts. In contrast, arbitrarily large off-chip buffers can exacerbate congestion or can increase latency and jitter, which leads to less deterministic Big Data job performance, especially if chaining jobs. While the Extreme Networks hardware maximizes burst absorption capability and addresses temporary congestion, it also maintains fairness. Since Extreme Networks Smart Buffer techlogy is adaptive in the shared buffering allocations, uncongested ports do t get starved of access to the shared buffer pool and they are t throttled by congestion on other ports, while still allowing congested ports to get more of the buffers to address the traffic burst. Extreme Networks Smart Buffering techlogy is dynamic and adaptive in its default configuration; Extreme Networks also supports fine-tuning the Smart Buffer allocations from the dedicated and shared pools. Users can configure the percentage of the shared buffers that each port can consume. configure ports <port> shared-packet-buffer <percent> In addition, kwing specifically which Big Data applications and traffic flows that may be used for high bandwidth data transfers or that are congestion risks, users can configure an access-list to match on specific criteria in traffic flows (e.g. match on a TCP port), assign them to a QoS Profile (QP), and then set a percentage of the dedicated buffers to that QP. configure qosprofile <QP> maxbuffer <percentage> port <port> Users should understand the consequences of modifying the default behavior. For more details on Extreme Network s Smart Buffer techlogies, please refer to the Congestion Management and Buffering in Data Center Networks white paper: JUMBO FRAMES Distributed filesystems like HDFS have high network I/O requirements because they move massive amounts of data around the cluster in the form of blocks. Oneto-many writes for block replication may occur at the same time as many-to-one reads, which results in periods of high network I/O demanding high throughput. These blocks can account for a majority of the bandwidth utilization in the network fabric. The Block size default is 64 MB but can be configured to be several times larger than that. To avoid the overhead of small packet sizes and to further improve Big Data Solutions Guide 11

12 the performance of the network for Big Data applications, jumbo frame support and increased IP MTU can be advantageous and Extreme Networks switches X670 and X770 can perform line rate with jumbo frames enabled. enable jumbo-frame ports all configure ip-mtu 9194 vlan <vlan name> LATENCY Extreme Networks Big Data solution provides the lowest port-to-port latency in the I/O fabric, jumbo-frame support, and short-reach optics and cables. The X770 is the first to support 104 wire-speed 10GbE ports or 32x40GbE ports or a mix of 10GbE and 40GbE in 1RU for switch interconnections and is capable of forwarding up to 1904 million packets per second. It has less than 600 naseconds latency with 64-byte packets using cut-through forwarding. 10G switches down to the servers provide lower latency than 1G switches and are recommended for optimal performance. Cut-through forwarding commands are as follows: configure forwarding switching-mode cut-through show forwarding configuration High Availability LOAD-SHARING Parallel layer 3 links between leaf switches and spine switches provide multiple equal cost paths, to enhance load sharing and providing a backup in case of link failure. Which link a traffic flow will use is determined by a hashing algorithm that takes into account the source and destination IP addresses of the servers. In the Big Data environment, the administrator assigns subnets to each rack of datades so the routing table will have a n-diverse set of IP address subnets representing each rack. If the Layer 3 source and destination IP addresses and the Layer 4 source and destination ports do t vary much across traffic flows, then a sub-optimal hashing algorithm (especially one that assumes diverse Internet addresses) may unevenly distribute the traffic flows across the available equal cost paths. This may cause some links to be underutilized or perhaps t used at all, and other links to be overutilized and exceed capacity. ECMP uses hash algorithms based on source and destination parameters of the traffic (L3, or L3 and L4), regardless of current utilization, so suboptimal algorithms may also result in congestion. Figure 3 compares two load sharing configurations and the variance in the link utilization. In both cases the same amount of traffic was delivered but the variance in the link utilization was much smaller in the better loadsharing algorithm and much larger in the suboptimal load-sharing algorithm. Big Data Solutions Guide 12

13 In order to achieve optimal load sharing of multiple traffic flows across the available paths in Big Data s n-diverse routing table, administrators can increase the number of ECMP gateways from its default of four and configure the optimal hashing algorithm. ExtremeXOS provides the cluster administrator different options for fine tuning the load sharing algorithms. Here is an example of what could be configured in this environment, ting that it can be adapted to the number of paths and IP address variance in the network: configure forwarding sharing L3_L4 configure iproute sharing hash-algorithm crc upper configure iproute sharing max-gateways 8 REDUNDANCY For the topology presented here, each server has two 10GbE NICs (and they can be negotiated down if needed) connected to two leaf switch pairs. The server NICs are bonded together to form a virtual 20GbE pipe providing increased bandwidth and redundancy to ensure high availability. Then each leaf switch pair creates the perception of a common link aggregated group so that the server doesn t see its NICs connected to two different leaf switches. The server thinks it has two NICs connected to the same leaf switch and it doesn t see anything different from a link aggregation perspective, even though the link aggregated ports are w distributed across Leaf 1 and Leaf 2, thereby leading to the term Multi-Switch Link Aggregation (MLAG). In this Big Data design, each pair of leaf switches is configured as MLAG peers down to the servers. A Layer 3 VLAN exists between the MLAG peers for control communication, called the ISC, and multiple physical links should be used and configured as a single logical LAG link to protect the MLAG ISC from loss of a single link. The ports down to the servers can be configured for Link Aggregation Control Protocol (LACP) which dynamically maintains LAG state information. Please see the Appendix for sample configuration. In this design VRRP (Virtual Router Redundancy Protocol) allows pairs of leaf switches to provide redundant routing services to the racks of servers. VRRP must be enabled on the server VLANs, allowing multiple switches to provide redundant Big Data Solutions Guide 13

14 routing services to the servers. This ensures high availability by eliminating the single point of failure associated with a default gateway failure. VRRP in Active/ Active mode allows both leaf switches to simultaneously act as the default gateway for the subnet and can be configured with a policy that blocks the VRRP multicast address on the input (i.e. ingress) of both sides of the MLAG ISC. entry v4active { if match all { destination-address /32 ; } then { deny ; } } RECOVERY Extreme Networks Big Data architecture with ECMP makes it resilient. It can dynamically react to failures, reroute traffic intelligently, and recover quickly. Consider the following topology where bi-directional traffic between the racks are load-shared across the network fabric. Big Data Solutions Guide 14

15 If a power hit results in Leaf 1 rebooting, then because each leaf de is part of an MLAG/VRRP pair (Leaf 1 and Leaf 2; Leaf 3 and Leaf4) the network stability is maintained. On the south side, the rack is still connected to the other pair leaf switch Leaf 2 and on the rth side Layer 3 reachability is still maintained through OSPF. The spine switch will send traffic through Leaf 2 until the original Leaf 1 boots and OSPF converges. See the figure below taken from the reporting mechanisms available in Extreme Networks OneView 1. The graphs below show links to leaf pairs Leaf 1 and Leaf 2, and leaf pairs Leaf 3 and Leaf 4, and shows three incidents where Leaf 1 is rebooted. When it is rebooted, the traffic traverses Leaf 2 until Leaf 1 recovers, at which point the traffic resumes load sharing between the two. Because of the network s resiliency, the applications and the MapReduce jobs are unaware of the network failure. BFD OSPF ECMP enables rapid recovery from outages and convergence times can be improved using Bidirectional Forwarding Detection (BFD), which more quickly detects failures and allows OSPF to react faster. BFD is a hello protocol that provides the rapid detection of failures in the path and informs the clients (routing protocols) to initiate the route convergence. It is independent of media, routing protocols, and data protocols. BFD helps in the separation of forwarding plane connectivity and control plane connectivity. 1 OneView data is sourced from SNMP polling of the switch interfaces every minute and can provide bandwidth utilization of interfaces, among other things. However, because of the polling interval there is some variance in the data and it will t necessarily be precise. Big Data Solutions Guide 15

16 Different routing protocol hello mechanisms operate in variable rates of detection, but BFD detects the forwarding path failures at a uniform rate, thus allowing for easier network profiling and planning, and consistent and predictable re-convergence time. Manageability MANAGING THE CLUSTER WITH NETSIGHT Extreme Networks NetSight provides a rich set of integrated management capabilities for centralized visibility and highly efficient anytime, anywhere control of enterprise wired and wireless network resources. NetSight s granularity reaches beyond ports, VLANs, and SSIDs down to individual users, applications, and protocols. NetSight increases efficiency, enabling IT staff to avoid time-consuming manual device-by-device configuration tasks. NetSight fills the functionality gap between traditional element managers that offer limited vendorspecific device control, and expensive, complex enterprise management applications. The NetSight management application provides centralized visibility and granular management of the cluster, including: Managing inventory Monitoring performance Use of syslog Configuring and upgrading devices Instead of managing the cluster through separate management applications, NetSight OneView provides a single management view of the entire cluster, in this case including the leaf switches and spine switches plus the datades. Big Data Solutions Guide 16

17 Interface status is also reported through the same view, as shown below. Furthermore, NetSight s reporting and visualizations can graphically show bandwidth utilization on different interfaces. NetSight OneFabric Control Center also provides comprehensive network inventory and change management information, tracks and reports changes to network configuration, and simplifies configuration backups, firmware upgrades and capacity planning. For more information on NetSight, please visit: extremenetworks.com/product/netsight/ RACK AWARENESS Hadoop s rack awareness capability enables intelligent decisions for where blocks of data should be replicated to which datades, which helps with replication of blocks to different racks for improved fault tolerance. The rack awareness is achieved through topology information that maps datade IP addresses to rack names. Normally this is a manual and time-consuming administrative task that needs to be done repeatedly as datades are added or removed from the cluster, and any user errors would result in less than optimal block replication locations. The Extreme Networks Big Data solution provides a seamless method to dynamically generate the topology data. The user can configure all the leaf switches for authentication on the ports connecting to the datades as shown in the CLI below. Note that the CLI uses the keyword vm-tracking but it applies to physical servers as well. Big Data Solutions Guide 17

18 configure vm-tracking nms primary server <NAC IP address> 1812 client-ip <switch management IP address> shared-secret encrypted Gt}xolg5 vr VR-Mgmt enable vm-tracking configure vm-tracking authentication database-order nms enable vm-tracking ports <port list> When the switch detects traffic from a new datade or end system, the switch authenticates the datade with the Network Management System (NMS). The NMS can then respond to the tification by calling a mapping program that maps the datade IP address to its given rack. The figure below shows the NMS configuration for the Hadoop Rack Awareness tification which calls the mapping program with arguments of the datade IP address and the switch IP address. Big Data Solutions Guide 18

19 A sample mapping program is provided in the Appendix. The output of the mapping program as shown below becomes the input to the Hadoop application, specifically the topology file, which is the enabler for the rack awareness /rack /rack /rack /rack /rack /rack /rack /rack2 Users can validate the rack information in Cloudera or manually through HDFS on the cluster Big Data Solutions Guide 19

20 $ hdfs dfsadmin -printtopology Rack: /default/rack :50010 (dn1-10g) :50010 (dn2-10g) :50010 (dn3-10g) :50010 (dn4-10g) Rack: /default/rack :50010 (dn5-10g) :50010 (dn6-10g) :50010 (dn7-10g) :50010 (dn8-10g) AUTO-PROVISIONING Big Data clusters need to be simple to manage and to deploy new hardware as the cluster scales out. To reduce the time needed to configure the switches to connect server racks into the Big Data cluster, ExtremeXOS has an auto-provisioning feature. This feature automates the (usually) manual configuration operations and allows the configuration to be more modularly defined. The autoprovisioning feature enables a port on the switch to be connected to a domain with a DHCP server, receive its IP address, and then the switch can receive a configuration file to be loaded and optionally send an SNMP trap to inform the user of the succesful auto-provision. The auto-provisioning feature can be enabled with the command: enable auto-provision FLOW ANALYSIS Varied traffic patterns from different applications may be traversing the Big Data cluster, and ExtremeXOS supports an industry-standard techlogy for monitoring that traffic through statistical sampling of packets. This techlogy, sflow, enables clients to analyze traffic patterns to better understand cluster utilization and to be used for capacity planning. PROACTIVE TECH SUPPORT Extreme Networks offers an automated support mechanism for customers who want to easily engage the Extreme Networks Technical Assistance Center (TAC) team. There is a loadable application called Proactive Tech Support that can be upgraded independently at any time without restarting the switch, and it enables system information to be automatically pushed into a cloud-hosted collector where the Extreme Networks TAC team can analyze the data and identify problems to provide solutions. Big Data Solutions Guide 20