version 7.0 Planning Guide

version 7.0

Contents Preface 1 Intended Audience 1 Documentation History 1 Introduction to Mirantis OpenStack and Fuel 2 System Requirements 3 Fuel Master Node Hardware Recommendations 3 Target Node Server Hardware Recommendations 4 Sample Hardware Configuration for Target Nodes 4 Controller nodes (3) 5 Storage nodes (3) 5 Compute nodes (5) 5 Summary 6 Planning Summary 7 Choose Network Topology 8 Choosing the Storage Model 9 Glance: Storage for Images 9 Object Storage for Applications 10 Cinder: Block Storage for Volumes 10 Nodes and Roles 12 Plan Monitoring Facilities 14 Ceilometer and MongoDB 14 Planning resources for Ceilometer 14 Installing Ceilometer 15 Planning a Sahara Deployment 16 Preparing for vsphere Integration 20 vsphere Installation 20 ESXi Host Networks Configuration 22 Limitations 24 Preparing to run Fuel on vsphere 25 Calculate hardware requirements 26 Example of Hardware Requirements Calculation 27 2015, Mirantis Inc. Page i

Calculating CPU 27 Calculating Memory 28 Calculating Storage 28 Throughput 29 Calculating Network 30 Scalability and oversubscription 31 Hardware for this example 31 Mellanox ConnectX-3 network adapters 31 Reference configuration of hardware switches 33 Tagged ports 33 Untagged ports 34 HA testing summary 35 Hardware compatibility list 36 Example 1: HA + Nova-network FlatDHCP manager 37 Detailed Port Configuration 38 Nova-network Switch configuration (Cisco Catalyst 2960G) 40 Nova-network Switch configuration (Juniper EX4200) 43 Example 2: HA + Neutron with GRE 50 Detailed port configuration 51 Neutron Switch configuration (Cisco Catalyst 2960G) 51 Neutron Switch configuration (Juniper EX4200) 55 Example 3: HA + Neutron with VLAN + SR-IOV & iser 61 Neutron Switch configuration (Mellanox SX1036) 64 Index 67 2015, Mirantis Inc. Page ii

Preface Preface This documentation provides information on how to use Fuel to deploy OpenStack environments. The information is for reference purposes and is subject to change. Intended Audience This documentation is intended for OpenStack administrators and developers; it assumes that you have experience with network and cloud concepts. Documentation History The following table lists the released revisions of this documentation: May, 2015 Revision Date 6.1 GA Description 2015, Mirantis Inc. Page 1

Introduction to Mirantis OpenStack and Fuel Introduction to Mirantis OpenStack and Fuel OpenStack is an extensible, versatile, and flexible cloud management platform. It is a portfolio of cloud infrastructure services compute, storage, networking and other core resources that are exposed through ReST APIs. It enables a wide range of control over these services, both from the perspective of an Integrated Infrastructure as a Service (IaaS) controlled by applications and as a set of tools that enable automated manipulation of the infrastructure itself. Mirantis OpenStack is a productized snapshot of the open source technologies. It includes Fuel, a graphical web tool that helps you to quickly deploy your cloud environment. Fuel includes scripts that dramatically facilitate and speed up the process of cloud deployment, without requiring you to completely familiarize yourself with the intricate processes required to install the OpenStack environment components. This guide provides details to get you started with Mirantis OpenStack and Fuel on a set of physical servers ("bare-metal installation") See the User Guide for detailed instructions about how to download and install Fuel on the Fuel Master Node and then how to use the Fuel interface to deploy your OpenStack environment. Further reading is available in the following documents: Terminology Reference is an alphabetical listing of technologies and concepts that serves as both a glossary and a master index of information in the Mirantis docs and the open source documentation. Operations Guide gives information about advanced tasks required to maintain the OpenStack environment after it is deployed. Most of these tasks are done in the shell using text editors and command line tools. Reference Architecture provides background information about how Mirantis OpenStack and its supporting HA architecture is implemented. You have ways to use Mirantis OpenStack and Fuel other than the bare-metal installation: You can install Fuel and use it to deploy a Mirantis OpenStack environment on Oracle VirtualBox. VirtualBox deployment is useful for demonstrations and is a good way to begin your exploration of the tools and technologies. See Introduction. Note The environments you deploy on VirtualBox are for testing purposes only. Typically, a production environment requires extensive resources and network planning. You can install Fuel master node on vsphere and deploy a Mirantis OpenStack Environment on either baremetal or on vsphere. Mirantis OpenStack is also available on-demand, preconfigured, and ready to use with our Hosted Private Cloud product, Mirantis OpenStack Express. For community members or partners looking to take Fuel even further, see the developer documentation for information about the internal architecture of Fuel, instructions for building the project, information about interacting with the REST API and other topics of interest to more advanced developers. You can also visit the Fuel project for more detailed information and become a contributor. 2015, Mirantis Inc. Page 2

System Requirements System Requirements Fuel and Mirantis OpenStack run well on high-quality commodity hardware. The following sections give some basic information about minimal hardware requirements; choosing the right hardware requires an understanding of how your cloud environment will be used. Some general remarks: For optimal performance in production environments, configure enough servers and network adapters to minimize contention for resources in the environment. Understand the nature of the software that will run on your environment and configure the hardware appropriately. For example, an environment that mostly provides remote storage has different hardware requirements than an environment that is mostly used for extensive calculations. One Fuel Master node can manage multiple environments. If you need to support applications with very different hardware needs, consider deploying separate environments that are targeted to these needs; for example, deploy one environment that is optimized for data storage and another environment that is optimized for compute power. Select peripheral hardware that is supported by the operating system distribution that you are using for the target nodes and for the VM instances that will be deployed. Integrating new hardware drivers into the Mirantis OpenStack distribution is possible but complicated at this time. This will be made easier through a simplified plug-in architecture that is planned for an upcoming release. Note also that some proprietary drivers may work only with specific Linux kernel versions and not be compatible with the kernel versions that are shipped with Mirantis OpenStack. Other drivers may not be included in the Mirantis OpenStack distribution because of licensing restrictions. It may be necessary to modify kernel parameters to get some CPUs and peripheral devices to work. See How To: Modify Kernel Parameters. Fuel Master Node Hardware Recommendations To install the Fuel Master Node, you should base your hardware on the anticipated load of your server. Logically, deploying more node servers in your environment requires more CPU, RAM, and disk performance. Suggested minimum configuration for installation in production environment: Quad-core CPU 4GB RAM 1 gigabit network port 128GB SAS Disk IPMI access through independent management network Suggested minimum configuration for installation in lab environment: Dual-core CPU 2GB RAM 2015, Mirantis Inc. Page 3

Target Node Server Hardware Recommendations 1 gigabit network port 50GB disk Physical console access Target Node Server Hardware Recommendations Determining the appropriate hardware for the target nodes depends on the node type. Controller nodes Use at least three controller nodes for high-availability. Further resource scaling depends on your use case and requires extensive assessment of your environment and business needs. For all deployments, the total number of the controller nodes must be odd. Compute nodes The number and hardware configuration of the compute nodes depend on the following: Storage nodes Number of virtual machines Applications that you plan to run on these virtual machines Types of workloads The number and capacity of the storage nodes highly depend on the type of the storage, redundancy, and the workloads that you run on the compute nodes. The OpenStack documentation provides additional guidelines on how to plan your resources: OpenStack Architecture Design Guide OpenStack Operations Guide Use the information provided in these documents in conjunction with this. For more information, see: Sample Hardware Configuration for Target Nodes Nodes and Roles Ceilometer and MongoDB Calculate hardware requirements OpenStack Hardware Compatibility List How do you calculate how much hardware you need for your OpenStack cloud? Sample Hardware Configuration for Target Nodes This is a general-purpose medium-sized hardware configuration that is provides room for moderate growth and is adequate for a variety of installations. Note that this is not a recommendation or some sort of "best practices" guide; but provides a reasonable starting point that avoids common configuration mistakes. The characteristics of the environment are: 2015, Mirantis Inc. Page 4

Target Node Server Hardware Recommendations 12 servers (one Fuel Master node, 3 Controller nodes, 3 Storage nodes, 5 Compute nodes) High-availability configuration Neutron networking, using either the VLAN or GRE topology Ceph as the backend for Cinder, Glance, and Nova (for ephemeral storage) No Ceilometer/MongoDB monitoring is configured; three additional servers to be used as MongoDB nodes would be required to add Ceilometer/MongoDB to the environment Controller nodes (3) Controllers require sufficient resources to run the Ceph monitors and MySQL in addition to the core OpenStack components (Nova, Neutron, Cinder, and Glance). Each controller should have: 2 CPUs with at least 6 physical cores each 64GB RAM recommended (1000+ VMs); 24GB minimum (Note that, for testing, a Controller node can run with as little as 2GB of memory, but production systems need the larger memory size.) Hardware RAID1 Controller with at least 1TB formatted capacity for the Host operating system disk. Larger disks may be warranted depending on expected database ang log storage requirements. Note that it is non-trivial to expand the disk storage on running Controller nodes. 2 NICs, either 1 Gbit/s or 10 Gbit/s Storage nodes (3) We recommend separate Ceph nodes for scalability and robustness. The hardware estimate is based on the requirement of.5 cores per Ceph-OSD CPU and 1GB of RAM per TB of Ceph-OSD space. All Ceph storage and journal hard disks can be configured in JBOD (Just a Bunch of Disks) mode on the RAID controller, or can be plugged into free MB ports. For production installations, the Ceph replication factor should be set to 3 or more; see Storage. Single-socket CPU with at least 4 physical cores 24GB RAM RAID1 Controller with at least 500GB capacity for the Host operating system disk 2 NICs, either 1 Gbit/s or 10 Gbit/s 18 TB of Ceph storage (6 x 3TB) 1-2 SSDs, 64GB or more each, for the Ceph Journal Compute nodes (5) Virtual Machines are hosted on the Compute nodes. By default, Fuel configures the Scheduler to use a fairly aggressive 8:1 CPU overcommit ratio on the Compute nodes; if you do not know what your VM workload is going to be, reset this ratio to 1:1 to avoid CPU congestion. Each Compute node should have: 2015, Mirantis Inc. Page 5

Summary Dual-socket CPU with at least 4 physical cores per socket 64GB RAM RAID1 Controller with at least 500GB capacity for the Host operating system disk 2 NICs, either 1 Gbit/s or 10 Gbit/s Summary In general, your best bet is to choose a 2 socket server with a balance in I/O, CPU, Memory, and Disk that meets your project requirements. Some good options from Supermicro, HP and Lenovo for compute nodes include: Supermicro Superserver 5037mr-h8rtf HP ProLiant DL380p Gen8 E5-2609v2 1P 8GB-R P420i/ZM 460W PS Server/S-Buy HP Smart Array P222 Controller Lenovo ThinkServer RD540 Rack Server Note Servers that use some UEFI chips (such as the Lenovo W520) can not be used for Fuel target nodes because they do not emulate legacy BIOS in a way that is compatible with the grub settings used for the Fuel Master node. This issue affects servers used as Controller, Compute, and Storage nodes. They are booted from PXE ROM and then the chain32 loader boots from the hard drive, so it is possible to boot them with an operating system that is already installed, but it is not possible to install an operating system on them because the operating system distributions that are provided do not include UEFI images. See LP1291128 and the UEFI support blueprint. 2015, Mirantis Inc. Page 6

Planning Summary Planning Summary Before installation, determine the deployment type that is appropriate for you configuration needs. You may want to print this list and make notes indicating your selection so you can be sure you have planned your deployment correctly. The following table provides a list of configuration steps that you must complete to plan the Mirantis OpenStack deployment. Step Description Select a network topology Determine how many nodes to deploy and which roles to assign to each and the high-availability to implement. Plan monitoring facilities If you want to run Hadoop plan to install Sahara. If you want to run vcenter or vsphere Calculate the server and network hardware needed Prepare an IP address management plan and network association. See Choose Network Topology See Nodes and Roles See Plan Monitoring Facilities Additional Information See Planning a Sahara Deployment See Preparing for vsphere Integration See Calculate hardware requirements Identify the network addresses and VLAN IDs for your Public, floating, Management, Storage, and virtual machine (fixed) networks. Prepare a logical network diagram. 2015, Mirantis Inc. Page 7

Choose Network Topology Choose Network Topology Mirantis OpenStack supports two network modes, which in turn support five topologies. For architectural descriptions of the five topologies, see: Neutron with VLAN segmentation and OVS (default) Neutron with GRE segmentation and OVS Nova-network FlatDHCP Manager (legacy networking) Nova-network VLAN Manager (legacy networking) Nova-network is a simple legacy network manager. It can operate with predefined Private IP spaces only. If you do not want to split your VMs into isolated groups (tenants), you can choose the Nova-network with FlatDHCP topology. In this case, you will have one big network for all tenants and their VMs. The Nova-network with VLANManager topology is a form of provider network. This allows you to have multiple tenants, each of which supports approximately the same number of VMs and an equal amount of Private IP space. The maximum number of tenants and the private IP space size must be defined before you deploy your environment. You must also set up appropriate VLANs and gateways on your underlying network equipment. Neutron is the reference network manager for OpenStack. It separates tenants, decreases the requirements for the underlying network (physical switches and topology), and gives a great deal of flexibility for manipulating Private IP spaces. You can create Private IP spaces with different sizes and manipulate them on the fly. The Neutron with VLAN topology, like Nova-network with VLANManager, requires a predefined maximum number of tenants value and underlying network equipment configuration. The Neutron with GRE topology allows more flexibility in the maximum number of tenants (it supports up to 65535 tenants) and simplifies the network equipment configuration, but GRE encapsulation decreases the speed of communication between the VMs and increases the CPU utilization on the Compute and Controller nodes. Mirantis OpenStack 6.0 and later supports configuring multiple network domains per single OpenStack environment when using Neutron GRE. See Configuring Multiple Cluster Networks for instructions; Implementing Multiple Cluster Networks explains the internals. Some other considerations when choosing a network topology: OVS (Open vswitch), Bonding, and Murano can only be implemented on Neutron. VMware vcenter and vsphere can only be implemented on Nova-network. Bonding is not supported for SR-IOV over the Mellanox ConnectX-3 network adapters family. Mellanox SR-IOV and iser are supported only when choosing Neutron with VLAN. 2015, Mirantis Inc. Page 8

Choosing the Storage Model Choosing the Storage Model This section discusses considerations for choosing the storage model for your OpenStack environment. You need to consider two types of data: Persistent storage exists outside an instance. Ephemeral storage is allocated for an instance and is deleted when the instance is deleted. Fuel deploys storage for two types of persistent data: Glance, the image storage service, which can use either Swift or Ceph RBD as the storage backend Cinder, the block storage service, which can use either LVM or Ceph RBD as the storage backend The Nova compute service manages ephemeral storage. By default, Nova stores ephemeral drives as files on local disks on the Compute nodes but can instead use Ceph RBD as the storage backend for ephemeral storage. See: Calculating Storage for information about choosing the hardware to use for your storage objects Storage Decisions is an OpenStack community document that gives guidelines for choosing the storage model to use. Glance: Storage for Images Fuel configures a storage backend for the Glance image service. This provides data resilience for VM images; multiple Glance instances running on controller nodes can store and retrieve images from the same data set, while the object store takes care of data protection and HA. When you create your OpenStack environment, you choose one of the following storage backends for Glance: Swift, the standard OpenStack object storage component Ceph RBD, the distributed block device provided by the Ceph storage platform Note Older versions of Fuel provided the Multi-Node deployment mode that was used to deploy OpenStack environments that had a single Compute node and used the file system backend for Glance. This mode is still available in Fuel 5.1 but is deprecated; instead, use the HA deployment mode with a single controller. This mode creates an environment that is not highly available but is managed using the same services as those that manage the full HA environment and can be scaled up to have multiple controllers and be highly available. In this mode, you can choose either Swift or Ceph RBD as the Glance storage backend. Factors to consider when choosing between Swift and Ceph RBD for the storage backend for the Glance image server include: Ceph provides a single shared pool of storage nodes that can handle all classes of persistent and ephemeral data that is required for OpenStack instances. 2015, Mirantis Inc. Page 9

Object Storage for Applications Fuel deploys Ceph-OSD (the data storage component of Ceph) as its own role that can run on dedicated nodes so you can add capacity by adding adding additional Ceph-OSD nodes. Fuel deploys Swift on Controller nodes. When using Ceph, a single set of hard drives can serve as a backend for Glance, Cinder, and Nova. Otherwise, you must have dedicated disks for image storage, block storage, and ephemeral disks. Ceph's copy-on-write facility allows you to clone a system image and to start different VMs based on that image without any unnecessary data copy operations; this speeds up the time required to launch VMs and create volumes from images. Without the Ceph copy-on-write facility, OpenStack images and volumes must be copied to separate Glance, Cinder, and Nova locations. Ceph supports live migration of VMs with ephemeral drives whereas Cinder with LVM backend only supports live migration of volume backed VMs. Note that live migration of Ceph backed VMs requires that Ceph RBD is used for ephemeral volumes as well as Glance and Cinder. Swift provides multi-site support. Ceph is unable to replicate RBD block device data on a long-distance link which means you cannot replicate between multiple sites. Starting with Ceph 0.80 "Firefly" (integrated into Mirantis OpenStack beginning with Release 5.1), the Ceph Object Gateway (radosgw) supports multi-site replication of object data but this support is more limited than that of Swift and does not apply to the RBD backend for Glance. Ceph cannot tolerate clock drift greater than 50ms. If your servers drift out of sync, your Ceph cluster breaks. When using Ceph, it is extremely important that you configure NTP or some other time synchronization facility for your environment. Object Storage for Applications Mirantis OpenStack supports Ceph as an object storage for apllications. Ceph includes the optional Ceph Object Gateway component (radosgw) that applications can use to access RGW objects. Note that the radosgw implementation of the Swift API does not implement all operations. Ceph RBD uses RADOS directly and does not use the Swift API, so it is possible to store Glance images in Ceph and still use Swift as the object store for applications. Because the Ceph Object Gateway replaces Swift as the provider of the Swift APIs, it is not possible to have both radosgw and Swift running in the same OpenStack environment. Cinder: Block Storage for Volumes Cinder serves block devices for the OpenStack environment. You have two choices for the storage back-end: Cinder LVM (default): each volume is stored as a logical volume in an LVM volume group on one of your Cinder nodes. Ceph: each volume is stored as an object in the Ceph RADOS object storage system. 2015, Mirantis Inc. Page 10

Object Storage for Applications Note If you are using vcenter as the hypervisor, you must use the VMDK driver to store your volumes in the vcenter datastore. Factors to consider when choosing between Cinder LVM and Ceph for the Cinder storage backend include: Ceph provides a single shared pool of storage nodes as discussed above for image storage. Ceph provides object replication capabilities by storing Cinder volumes as Ceph RBD objects; each Ceph RBD object is stored as multiple RADOS objects. Ceph ensures that each replica of an object is stored on a different node. This means that your volumes are protected against hard drive and node failures or even the failure of the data center itself. The Ceph data replication rules (CRUSH map) can be customized separately for each object pool to modify the number of object replicas, add different types of failure domains, etc. LVM provides much less protection of your data than Ceph does. Even if you use RAID on each Cinder node, your data is only protected against a hard drive failure. If the Cinder node itself is lost, all volumes that were stored on that node are lost. Ceph consumes more disk space than LVM. LVM stores a single replica of the data whereas Ceph stores at least two copies of your data so that your actual raw storage capacity must be two to three times bigger than your data set. Beginning with the "Firefly" release of Ceph (supported by Mirantis OpenStack beginning with Release 5.1), you can manually implement erasure coding striping to reduce the data multiplication requirements of Ceph. Ceph provides multi-node striping and redundancy for block storage. If you combine Ceph RBD backends for Cinder and Glance, you gain another important advantage over Cinder LVM: copy-on-write cloning of Glance images into bootable Ceph volumes. Ceph supports live migration of VMs with ephemeral drives whereas LVM only supports live migration of volume backed VMs. If you use Cinder LVM, you have the following configuration options: Let Fuel create a JBOD partition that spans all the storage drives in a node. You can join all drives into a RAID array before deployment and have the array appear to Fuel as a single block device. When deploying Ceph, Fuel partitions the Ceph storage nodes so that most of the space is reserved for Ceph-OSD storage; all other partitions for the node consume a portion of the first drive. To improve system performance, you might configure one or two SSDs and assign them the "Ceph General" role on the Assign a role or roles to each node server screen. See Ceph Hardware Recommendations for details. 2015, Mirantis Inc. Page 11

Nodes and Roles Nodes and Roles Your OpenStack environment contains a set of specialized nodes and roles; see OpenStack Environment Architecture for a description. When planning your OpenStack deployment, you must determine the proper mix of node types and what roles will be installed on each. When you create your OpenStack environment, you will assign a role or roles to each node server. All production environments should be deployed for high availability although you can deploy your environment without the replicated servers required for high availability and then add the replicated servers later. But part of your Nodes and Roles planning is to determine the level of HA you want to implement and to plan for adequate hardware. Some general guiding principles: When deploying a production-grade OpenStack environment, it is best to spread the roles (and, hence, the workload) over as many servers as possible in order to have a fully redundant, highly-available OpenStack environment and to avoid performance bottlenecks. For demonstration and study purposes, you can deploy OpenStack on VirtualBox; see QuickStart Guide for more information. This option has the lowest hardware requirements OpenStack can be deployed on smaller hardware configurations by combining multiple roles on the nodes and mapping multiple Logical Networks to a single physical NIC. Many plugins require specific roles; be sure to check the requirements of any plugins you plan to use when allocating nodes and roles. When looking to maximize performance, carefully choose your hardware components and make sure that the performance features of the selected hardware are supported and enabled. For an example of what tight integration of hardware and software can do for you, see Mellanox ConnectX-3 network adapters. This section provides information to help you decide how many nodes you need and which roles to assign to each. The absolute minimum requirement for a highly-available OpenStack deployment is to allocate 4 nodes: 3 Controller nodes, combined with Storage 1 Compute node In production environments, it is highly recommended to separate storage nodes from controllers. This helps avoid resource contention, isolates failure domains, and allows to optimize hardware configurations for specific workloads. To achieve that, you will need a minimum of 5 nodes when using Swift and Cinder storage backends, or 7 nodes for a fully redundant Ceph storage cluster: 3 Controller nodes 1 Cinder node or 3 Ceph OSD nodes 1 Compute node 2015, Mirantis Inc. Page 12

Nodes and Roles Note You do not need Cinder storage nodes if you are using Ceph RBD as storage backend for Cinder volumes. Of course, you are free to choose how to deploy OpenStack based on the amount of available hardware and on your goals (such as whether you want a compute-oriented or storage-oriented environment). 2015, Mirantis Inc. Page 13

Plan Monitoring Facilities Plan Monitoring Facilities Mirantis OpenStack can deploy Ceilometer to query and store metering data that can be used for billing purposes. Ceilometer and MongoDB Fuel can deploy Ceilometer (OpenStack Telemetry) in your OpenStack environment. Ceilometer collects and shares measurement data from all OpenStack components. You can use this data to monitor utilization, as well as for capacity planning, and the alarming service. In addition, Ceilometer's REST API provides data to external monitoring software for third-party billing systems. Mirantis OpenStack installs MongoDB as a back-end database for OpenStack Telemetry. This helps to resolve the Ceilometer performance issues with MySQL database encountered in earlier releases, because MongoDB better handles the volume of concurrent read and write operations. Ceilometer collects the following types of data: Billable events Billable events include such events as "instance X was created" and "volume Z was deleted". Though the system sends these notifications continuously, monitoring of billable events uses little resources of the cloud environment. Metrics Metrics analyze activities in the cloud environment at the moment of sampling. This information is gathered by polling and may consume significant amounts of I/O processing resources and large amount of database storage. You can configure Ceilometer to collect the large amount of metering data and, therefore, process high volume of database writes. Planning resources for Ceilometer When planning your resources, consider the following: The MongoDB partition requires at least 10240 MB of free space. The resources consumed by metrics sampling are determined by the polling interval, the number of metrics being collected, and the number of resources from which metrics are collected. The amount of storage required is also affected by the frequency with which you offload or purge the data from the database. Frequent polling of metrics yields a better picture of what is happening in the cloud environment, but also significantly increases the amount of data being processed and stored. For example, in one test sampling, the same metrics for the same fairly small number of resources in the same environment resulted in the following: 1 minute polling accumulated 0.8 TB of data over a year. 30 second polling accumulated 1.4 TB of data over a year. 5 second polling accumulated 14.5 TB of data over a year. 2015, Mirantis Inc. Page 14

Installing Ceilometer Ceilometer consumes fairly small amounts of CPU. However, the I/O processing is extremely intensive when the data is written to the disk. Therefore, we recommend using dedicated MongoDB nodes rather than running the MongoDB role on the Controller nodes. In our lab tests, nearly 100% of the disk I/O resources on the Controller nodes were sometimes consumed by Ceilometer writing data to MongoDB when the database was located on the Controller node and a small polling interval was used. This configuration halted or interferred with all other OpenStack services on the Controller node and prevented other processes from running. Installing Ceilometer Before installing Ceilometer, verify that your environment meets the recommendations described in Planning resources for Ceilometer. You can add Ceilometer to your OpenStack environment when you configure an OpenStack environment in the deployment wizard or in the Settings tab. To install Ceilometer: 1. Select from the following options: In the deployment wizard: 1. In the Additional Services screen, select the Install Ceilometer checkbox. In the Settings tab.: 1. Select the Install Ceilometer checkbox. 2. Create the new OpenStack environment. 3. Click on the new environment. 4. In the Nodes tab, assign the Telemetry-MongoDB role to the required servers. Note We recommend that you run MongoDB on dedicated servers. The minimum number of MongoDB nodes must be equal or greater than the number of the Controller nodes deployed in the OpenStack environment. 2015, Mirantis Inc. Page 15

Planning a Sahara Deployment Planning a Sahara Deployment Sahara enables users to easily provision and manage Apache Hadoop clusters in an OpenStack environment. Sahara supports only 2.x Release of Hadoop. The Sahara control processes run on the Controller node. The entire Hadoop cluster runs in VMs that run on Compute Nodes. A typical set-up is: One VM that runs management and monitoring processes (Apache Ambari, Cloudera Manager, Ganglia, Nagios) One VM that serves as the Hadoop master node to run ResourceManager and NameNode. Many VMs that serve as Hadoop worker nodes, each of which runs NodeManager and DataNodes. You must have exactly one instance of each management and master processes running in the environment. Other than that, you are free to use other configurations. For example, you can run the NodeManager and DataNodes in the same VM that runs ResourceManager and NameNode; such a configuration may not produce performance levels that are acceptable for a production environment but it works for evaluation and demonstration purposes. You could also run DataNodes and TaskTrackers in separate VMs. Sahara can use either Swift Object Storage or Ceph for object storage. Note If you have configured the Swift public URL with SSL, Sahara will only work with the prepared Sahara images. Special steps are required to implement data locality for Swift; see Data-locality for details. Data locality is not available for Ceph storage backend. Plan the size and number of nodes for your environment based on the information in Nodes and Roles. When deploying an OpenStack Environment that includes Sahara for running Hadoop you need to consider a few special conditions. Plugin Capabilities The following table provides a plugin capability matrix: Plugin Feature Vanilla HDP Cloudera Spark MapR Nova network x x x x x Neutron network x x x x x Cluster Scaling x x x x N/A Swift Integration x x x x N/A Cinder Support x x x x x 2015, Mirantis Inc. Page 16

Planning a Sahara Deployment Data Locality x x N/A x N/A EDP x x x x x High Availability N/A x N/A N/A N/A Floating IPs Fuel configures Sahara to use floating IPs to manage the VMs. This means that you must provide a Floating IP pool in each Node Group Template you define. See Public and Floating IP address requirements for general information about floating IPs. A special case is if you are using Nova-Network and you have set the auto_assign_floating_ip parameter to true by checking the appropriate box on the Fuel UI. In this case, a floating IP is automatically assigned to each VM and the "floating ip pool" dropdown menu is hidden in the OpenStack Dashboard. In either case, Sahara assigns a floating IP to each VM it spawns so be sure to allocate enough floating IPs. However, if you have a limited number of floating IPs or special security policies you may not be able to provide access to all instances. In this case, you can use the instances that have access as proxy gateways. To enable this functionality, set the is_proxy_gateway parameter to true for the node group you want to use as proxy. Sahara will communicate with all other cluster instances through the instances of this node group. Note If use_floating_ips is set to true and the cluster contains a node group that is used as proxy, the requirement to provision a pool of floating IPs is only applied to the proxy node group. Sahara accesses the other instances through proxy instances using the private network. Note The Cloudera Hadoop plugin does not support the access to the Cloudera manager through a proxy node. Therefore, you can only assign the nodes on which you have the Cloudera manager as proxy gateways. Security Groups Sahara can create and configure security groups separately for each cluster depending on a provisioning plugin and Hadoop version. Security Groups VM Flavor Requirements Hadoop requires at least 1Gb of RAM to run. That means you must use flavors that have at least 1Gb of memory for Hadoop cluster nodes. Hardware-assisted virtualization In order for Sahara to work properly, hardware-assisted virtualization must be enabled for the hypervisor used by OpenStack. Its absence leads to frequent random errors during Hadoop deployment, because in that case VMs are 2015, Mirantis Inc. Page 17

Planning a Sahara Deployment too 'weak' to run such a heavywight application. To ensure that Sahara will work properly, you should do two things: While deploying OpenStack environment via Fuel UI, select hypervisor other than QEMU. Make sure that CPUs on compute nodes support hardware-assisted virtualization. To check that, run the following command on deployed compute nodes: cat /proc/cpuinfo grep --color "vmx\ svm" While most modern x86 CPUs support hardware-assisted virtualization, its support still might be absent on compute nodes if they are themselves running as virtual machines. In that case hypervisor running compute nodes must support passing through hardware-assisted virtualization to nested VMs and have it enabled. VirtualBox does not have that feature, and as a result environments deployed as described in the QuickStart Guide will have Sahara working poorly. Communication between virtual machines Be sure that communication between virtual machines is not blocked. Default templates Sahara bundles default templates that define simple clusters for the supported plugins. These templates are already added to the sahara database, therefore, you do not need to create them. Supported default templates for plugins There is an overview of the supported default templates for each plugin: Vanilla Apache Hadoop 2.6.0: There are 2 node groups created for this plugin. First one is named vanilla-2-master and contains all management Hadoop components - NameNode, HistoryServer and ResourceManager. It also includes Oozie server required to run Hadoop jobs. Second one is named vanilla-2-worker and contains components required for data storage and processing - NodeManager and DataNode. The cluster template is also represented for this plugin. It's named vanilla-2 and contains 1 master and 3 worker nodes. Cloudera Hadoop Distribution (CDH) 5.4.0: There are 3 node groups created for this plugin. First one is named cdh-5-master and contains all management Hadoop components - NameNode, HistoryServer and ResourceManager. It also includes Oozie server required to run Hadoop jobs. Second one is named cdh-5-manager and contains Cloudera Management component that provides UI to manage Hadoop cluster. Third one is named cdh-5-worker and contains components required for data storage and processing - NodeManager and DataNode. The cluster template is also represented for this plugin. It's named cdh-5 and contains 1 manager, 1 master and 3 worker nodes. Hortonworks Data Platform (HDP) 2.2: There are also 2 node groups created for this plugin. First one named hdp-2-2-master and contains all management Hadoop components - Ambari, NameNode, MapReduce HistoryServer, ResourceManager, YARN Timeline Server, ZooKeeper. It also includes Oozie server required to run Hadoop jobs. Second one 2015, Mirantis Inc. Page 18

Planning a Sahara Deployment named hdp-2-2-worker and contains components required for data storage and processing - NodeManager and DataNode. The cluster template is also represented for this plugin. It's named hdp-2-2 and contains 1 master and 4 worker nodes. For additional information about using Sahara to run Apache Hadoop, see the Sahara documentation. 2015, Mirantis Inc. Page 19

Preparing for vsphere Integration Preparing for vsphere Integration Fuel 5.0 and later can deploy a Mirantis OpenStack environment that boots and manages virtual machines in VMware vsphere. VMware provides a vcenter driver for OpenStack that enables the nova-compute service to communicate with a VMware vcenter server that manages one or more ESX host clusters. If your vcenter manages multiple ESX host clusters, Fuel 5.1 allows you to specify several or all clusters for a single OpenStack environment, so that one nova-compute service manages multiple ESX host clusters via single vcenter server. Please, note the following: Beginning with Fuel 6.1, vcenter cannot be integrated with NSX: NSX support is now deprecated. Due to Pluggable Architecture, it might be turned into a plugin in the future Fuel releases. In 5.x environments that use vcenter as the hypervisor, the nova-compute service runs only on Controller nodes. In 6.0 Fuel release, the relation between a nova-compute service and an ESXi host cluster is changed from one-to-many to one-to-one (so-called 1-1 mapping). In other words, to manage multiple ESXi host clusters, you now need to run multiple nova-compute services. In Fuel 6.1, each OpenStack environment can support more than one vcenter cluster. In Fuel 6.1, Ceilometer compute service is available for each vsphere cluster. That means,every agent polls resources about instances from those that only relate to their vsphere cluster. Every agent uses its own configuration file with authentication parameters for its specific vsphere cluster. See Related projects for vcenter for more details. The vcenter driver makes management convenient from both the OpenStack Dashboard (Horizon) and from vcenter, where advanced vsphere features can be accessed. This section summarizes the planning you should do and other steps that are required before you attempt to deploy Mirantis OpenStack with vcenter integration. For more information: See VMware vsphere Integration for information about how vcenter support is implemented in Mirantis OpenStack; Deploying vcenter gives instructions for creating and deploying a Mirantis OpenStack environment that is integrated with VMware vsphere. For background information about VMware vsphere support in OpenStack, see the VMware vsphere - OpenStack Manuals. The official vsphere installation guide can be found here: vsphere Installation and Setup. See Limitations section for more details on compatility with other VMware components and Mirantis lab setup. vsphere Installation Before installing Fuel and using it to create a Mirantis OpenStack environment that is integrated with VMware vsphere, the vsphere installation must be up and running. Please check that you completed the following steps: Install ESXi 2015, Mirantis Inc. Page 20

Preparing for vsphere Integration Install vcenter Configure vcenter Create DataCenter Create vcenter cluster Add ESXi hosts to clusters in vcenter 2015, Mirantis Inc. Page 21

ESXi Host Networks Configuration ESXi Host Networks Configuration The ESXi host(s) network must be configured appropriately in order to enable integration of Mirantis OpenStack with vcenter. Follow the steps below: 1. Open the ESXi host page, select the "Manage" tab and click on "Networking". vswitch0 and all its networks are shown. Click the Add Network button: 2. In the "Add Networking" wizard, select the Virtual Machine Port group: 2015, Mirantis Inc. Page 22

ESXi Host Networks Configuration 1. On the next page, select the "Virtual Machine Port Group" option to ensure that the network will be created in vswitch0: 2. Always name the network br100; this is the only value that works with Fuel; type a VLAN Tag in the VLAN ID field; (the value must be equal to the VLAN Tag at VM Fixed on Fuel s Configure the Network tab): 2015, Mirantis Inc. Page 23

Limitations Limitations Only vcenter versions 5.1 and later are supported Security groups are not supported. The only supported backend for Cinder is VMDK. Volumes that are created by Cinder appear as SCSI disks. To be able to read/write that disk, be sure that the operating system inside the instance supports SCSI disks. The CirrOS image that is shipped with Fuel supports only IDE disks, so even if the volume is attached to it, CirrOS is not able to use it. The Ceph backend for Glance, Cinder and RadosGW object storage is not supported. Murano is not supported. It requires Neutron and vcenter utilizes nova-network. Note Mirantis has the following lab setup tested for Mirantis OpenStack release 6.0 for VMware environment: vcenter 5.5. vcenter 5.5.u2 For background information about how vcenter support is integrated into Mirantis OpenStack, see VMware vsphere Integration. Follow the instructions in Deploying vcenter to deploy your Mirantis OpenStack environment with vcenter support. 2015, Mirantis Inc. Page 24

Preparing to run Fuel on vsphere Preparing to run Fuel on vsphere For information about how Fuel runs on vsphere, see Fuel running under vsphere. For instructions for setting up your vsphere and installing Fuel on vsphere, follow the instructions in Installing Fuel Master Node on vsphere. 2015, Mirantis Inc. Page 25

Calculate hardware requirements Calculate hardware requirements You can use the Fuel Hardware Calculator to calculate the hardware required for your OpenStack environment. When choosing the hardware on which you will deploy your OpenStack environment, you should think about: CPU -- Consider the number of virtual machines that you plan to deploy in your cloud environment and the CPU per virtual machine. Also consider how the environment will be used: environments used for heavy computational work may require more powerful CPUs than environments used primarily for storage, for example. Memory -- Depends on the amount of RAM assigned per virtual machine and the controller node. Storage -- Depends on the local drive space per virtual machine, remote volumes that can be attached to a virtual machine, and object storage. Networking -- Depends on the Choose Network Topology, the network bandwidth per virtual machine, and network storage. See Example of Hardware Requirements Calculation for some specific calculations you can make when choosing your hardware. 2015, Mirantis Inc. Page 26

Example of Hardware Requirements Calculation Example of Hardware Requirements Calculation When you calculate resources for your OpenStack environment, consider the resources required for expanding your environment. The example described in this section presumes that your environment has the following prerequisites: 100 virtual machines 2 x Amazon EC2 compute units 2 GHz average 16 x Amazon EC2 compute units 16 GHz maximum Calculating CPU Use the following formula to calculate the number of CPU cores per virtual machine: max GHz /(number of GHz per core x 1.3 for hyper-threading) Example: 16 GHz / (2.4 x 1.3) = 5.12 Therefore, you must assign at least 5 CPU cores per virtual machine. Use the following formula to calculate the total number of CPU cores: (number of VMs x number of GHz per VM) / number of GHz per core Example: (100 VMs * 2 GHz per VM) / 2.4 GHz per core = 84 Therefore, the total number of CPU cores for 100 virtual machines is 84. Depending on the selected CPU you can calculate the required number of sockets. Use the following formula: total number of CPU cores / number of cores per socket For example, you use Intel E5 2650-70 8 core CPU: 84 / 8 = 11 Therefore, you need 11 sockets. To calculate the number of servers required for your deployment, use the following formula: total number of sockets / number of sockets per server 2015, Mirantis Inc. Page 27

Calculating Memory Round the number of sockets to an even number to get 12 sockets. Use the following formula: 12 / 2 = 6 Therefore, you need 6 dual socket servers. You can calculate the number of virtual machines per server using the following formula: number of virtual machines / number of servers Example: 100 / 6 = 16.6 Therefore, you can deploy 17 virtual machines per server. Using this calculation, you can add additional servers accounting for 17 virtual machines per server. The calculation presumes the following conditions: No CPU oversubscribing If you use hyper-threading, count each core as 1.3, not 2. CPU supports the technologies required for your deployment Calculating Memory Continuing to use the example from the previous section, we need to determine how much RAM will be required to support 17 VMs per server. Let's assume that you need an average of 4 GBs of RAM per VM with dynamic allocation for up to 12GBs for each VM. Calculating that all VMs will be using 12 GBs of RAM requires that each server have 204 GBs of available RAM. You must also consider that the node itself needs sufficient RAM to accommodate core OS operations as well as RAM for each VM container (not the RAM allocated to each VM, but the memory the core OS uses to run the VM). The node's OS must run it's own operations, schedule processes, allocate dynamic resources, and handle network operations, so giving the node itself at least 16 GBs or more RAM is not unreasonable. Considering that the RAM we would consider for servers comes in 4 GB, 8 GB, 16 GB and 32 GB sticks, we would need a total of 256 GBs of RAM installed per server. For an average 2-CPU socket server board you get 16-24 RAM slots. To have 256 GBs installed you would need sixteen 16 GB sticks of RAM to satisfy your RAM needs for up to 17 VMs requiring dynamic allocation up to 12 GBs and to support all core OS requirements. You can adjust this calculation based on your needs. Calculating Storage When planning disk space, you need to consider several types of data: Ephemeral (the local drive space for a VM) Persistent object storage Persistent block storage 2015, Mirantis Inc. Page 28

Calculating Memory See Choosing the Storage Model for more information about the options for storage and how to choose the appropriate model. As far as local drive space that must reside on the compute nodes, in our example of 100 VMs we make the following assumptions: 150 GB local storage per VM 5 TB total of local storage (100 VMs * 50 GB per VM) 500 GB of persistent volume storage per VM 50 TB total persistent storage Returning to our already established example, we need to figure out how much storage to install per server. This storage will service the 17 VMs per server. If we are assuming 50 GBs of storage for each VMs drive container, then we would need to install 2.5 TBs of storage on the server. Since most servers have anywhere from 4 to 32 2.5" drive slots or 2 to 12 3.5" drive slots, depending on server form factor (i.e., 2U vs. 4U), you will need to consider how the storage will be impacted by the intended use. If storage impact is not expected to be significant, then you may consider using unified storage. For this example a single 3 TB drive would provide more than enough storage for seventeen 150 GB VMs. If speed is really not an issue, you might even consider installing two or three 3 TB drives and configure a RAID-1 or RAID-5 for redundancy. If speed is critical, however, you will likely want to have a single hardware drive for each VM. In this case you would likely look at a 3U form factor with 24-slots. Don't forget that you will also need drive space for the node itself, and don't forget to order the correct backplane that supports the drive configuration that meets your needs. Using our example specifications and assuming that speed is critical, a single server would need 18 drives, most likely 2.5" 15,000 RPM 146 GB SAS drives. Throughput As far as throughput, that's going to depend on what kind of storage you choose. In general, you calculate IOPS based on the packing density (drive IOPS * drives in the server / VMs per server), but the actual drive IOPS will depend on the drive technology you choose. For example: 3.5" slow and cheap (100 IOPS per drive, with 2 mirrored drives) 100 IOPS * 2 drives / 17 VMs per server = 12 Read IOPS, 6 Write IOPS 2.5" 15K (200 IOPS, four 600 GB drive, RAID-10) 200 IOPS * 4 drives / 17 VMs per server = 48 Read IOPS, 24 Write IOPS SSD (40K IOPS, eight 300 GB drive, RAID-10) 40K * 8 drives / 17 VMs per server = 19K Read IOPS, 9.5K Write IOPS Clearly, SSD gives you the best performance, but the difference in cost between SSDs and the less costly platter-based solutions is going to be significant, to say the least. The acceptable cost burden is determined by the balance between your budget and your performance and redundancy needs. It is also important to note that the rules for redundancy in a cloud environment are different than a traditional server installation in that entire servers provide redundancy as opposed to making a single server instance redundant. In other words, the weight for redundant components shifts from individual OS installation to server redundancy. It is far more critical to have redundant power supplies and hot-swappable CPUs and RAM than to have 2015, Mirantis Inc. Page 29

Calculating Network redundant compute node storage. If, for example, you have 18 drives installed on a server and have 17 drives directly allocated to each VM installed and one fails, you simply replace the drive and push a new node copy. The remaining VMs carry whatever additional load is present due to the temporary loss of one node. Remote storage IOPS will also be a factor in determining how you plan to handle persistent storage. For example, consider these options for laying out your 50 TB of remote volume space: 12 drive storage frame using 3 TB 3.5" drives mirrored 36 TB raw, or 18 TB usable space per 2U frame 3 frames (50 TB / 18 TB per server) 12 slots x 100 IOPS per drive = 1200 Read IOPS, 600 Write IOPS per frame 3 frames x 1200 IOPS per frame / 100 VMs = 36 Read IOPS, 18 Write IOPS per VM 24 drive storage frame using 1TB 7200 RPM 2.5" drives 24 TB raw, or 12 TB usable space per 2U frame 5 frames (50 TB / 12 TB per server) 24 slots x 100 IOPS per drive = 2400 Read IOPS, 1200 Write IOPS per frame 5 frames x 2400 IOPS per frame / 100 VMs = 120 Read IOPS, 60 Write IOPS per frame You can accomplish the same thing with a single 36 drive frame using 3 TB drives, but this becomes a single point of failure in your environment. Object storage When it comes to object storage, you will find that you need more space than you think. For example, this example specifies 50 TB of object storage. Object storage uses a default of 3 times the required space for replication, which means you will need 150 TB. However, to accommodate two hands-off zones, you will need 5 times the required space, which actually means 250 TB. The calculations don't end there. You don't ever want to run out of space, so "full" should really be more like 75% of capacity, which means you will need a total of 333 TB, or a multiplication factor of 6.66. Of course, that might be a bit much to start with; you might want to start with a happy medium of a multiplier of 4, then acquire more hardware as your drives begin to fill up. That calculates to 200 TB in our example. So how do you put that together? If you were to use 3 TB 3.5" drives, you could use a 12 drive storage frame, with 6 servers hosting 36 TB each (for a total of 216 TB). You could also use a 36 drive storage frame, with just 2 servers hosting 108 TB each, but its not recommended due to the high cost of failure to replication and capacity issues. Calculating Network Perhaps the most complex part of designing an OpenStack environment is the networking. An OpenStack environment can involve multiple networks even beyond the Public, Private, and Internal networks. Your environment may involve tenant networks, storage networks, multiple tenant private networks, and so on. Many of these will be VLANs, and all of them will need to be planned out in advance to avoid configuration issues. 2015, Mirantis Inc. Page 30

Mellanox ConnectX-3 network adapters In terms of the example network, consider these assumptions: 100 Mbits/second per VM HA architecture Network Storage is not latency sensitive In order to achieve this, you can use two 1 Gb links per server (2 x 1000 Mbits/second / 17 VMs = 118 Mbits/second). Using two links also helps with HA. You can also increase throughput and decrease latency by using two 10 Gb links, bringing the bandwidth per VM to 1 Gb/second, but if you're going to do that, you've got one more factor to consider. Scalability and oversubscription It is one of the ironies of networking that 1 Gb Ethernet generally scales better than 10Gb Ethernet -- at least until 100 Gb switches are more commonly available. It's possible to aggregate the 1 Gb links in a 48 port switch, so that you have 48 x 1 Gb links down, but 4 x 10 Gb links up. Do the same thing with a 10 Gb switch, however, and you have 48 x 10 Gb links down and 4 x 100b links up, resulting in oversubscription. Like many other issues in OpenStack, you can avoid this problem to a great extent with careful planning. Problems only arise when you are moving between racks, so plan to create "pods", each of which includes both storage and compute nodes. Generally, a pod is the size of a non-oversubscribed L2 domain. Hardware for this example In this example, you are looking at: 2 data switches (for HA), each with a minimum of 12 ports for data (2 x 1 Gb links per server x 6 servers) 1 x 1 Gb switch for IPMI (1 port per server x 6 servers) Optional Cluster Management switch, plus a second for HA Because your network will in all likelihood grow, it's best to choose 48 port switches. Also, as your network grows, you will need to consider uplinks and aggregation switches. Mellanox ConnectX-3 network adapters Beginning with version 5.1, Fuel can configure Mellanox ConnectX-3 network adapters to accelerate the performance of compute and storage traffic. This implements the following performance enhancements: Compute nodes use SR-IOV based networking. Cinder nodes use iser block storage as the iscsi transport rather than the default iscsi over TCP. These features reduce CPU overhead, boost throughput, reduce latency, and enable network traffic to bypass the software switch layer (e.g. Open vswitch). The Mellanox ConnectX-3 adapters family supports up to 40/56GbE. To reach 56 GbE speed in your network with ConnectX-3 adapters, you must use Mellanox Ethernet switches (e.g. SX1036) with the additional 56GbE license. The switch ports should be configured specifically to use 56GbE speed. No additional configuration is required on 2015, Mirantis Inc. Page 31

Mellanox ConnectX-3 network adapters the adapter side. For additional information about how to run in 56GbE speed, see HowTo Configure 56GbE Link on Mellanox Adapters and Switches. 2015, Mirantis Inc. Page 32

Reference configuration of hardware switches Reference configuration of hardware switches This section describes reference configuration for Cisco, Arista, Mellanox and Juniper network switches. Tagged ports Cisco Catalyst interface [Ten]GigabitEthernet[interface number] description [port description] switchport trunk encapsulation dot1q switchport trunk allowed vlan [vlan IDs for specific networks] switchport mode trunk spanning-tree portfast trunk switchport trunk native vlan [vlan ID] - if necessary one untagged VLAN Cisco Nexus/ Arista interface ethernet[interface number] description [port description] switchport switchport mode trunk switchport trunk allowed vlan [vlan IDs for specific networks] switchport trunk native vlan [vlan ID] - if necessary one untagged VLAN Mellanox interface ethernet [interface number] description [port description] switchport mode hybrid switchport hybrid allowed-vlan [add remove except] [vlan ID vlan IDs range] switchport hybrid allowed-vlan [vlan ID vlan IDs range] switchport access vlan [vlan ID] Juniper interfaces { [interface_name]-[interface_number] { unit 0 { family ethernet-switching { port-mode trunk; vlan { members [ vlan IDs or names of specific networks]; native-vlan-id [vlan ID] if necessary one untagged VLAN 2015, Mirantis Inc. Page 33

Untagged ports Untagged ports Cisco Catalyst interface [Ten]GigabitEthernet[interface number] description [port description] switchport access [vlan ID for specific network] switchport mode access spanning-tree portfast Cisco Nexus/Arista interface ethernet[interface number] description [port description] switchport switchport access vlan [vlan ID for specific network] Mellanox interface ethernet [interface number] description [port description] switchport mode access switchport access vlan [vlan ID] Juniper interfaces { [interface_name]-[interface_number] { unit 0 { family ethernet-switching { port-mode access; vlan { members [vlan ID or name for specific network]; 2015, Mirantis Inc. Page 34

HA testing summary HA testing summary Currently, HA testing scenarios are provided to enable better testing of the whole environment: regular scenarios for checking Neutron, nova-network, and bonding. failover scenarios for checking specific cases (Neutron L3 agent, nova-network with Ceph). To learn more about HA testing scenarios, see HA testing scenarios. 2015, Mirantis Inc. Page 35

Hardware compatibility list Hardware compatibility list You can always get the latest list of the hardware certified as compatible at the official Mirantis Hardware Compatibility List page. 2015, Mirantis Inc. Page 36

Example 1: HA + Nova-network FlatDHCP manager Example 1: HA + Nova-network FlatDHCP manager As a model example, the following configuration is used: Deployment mode: Multi-node HA Networking model: Nova-network FlatDHCP manager Hardware and environment: 7 servers with two 1Gb/s Ethernet NIC and IPMI 1 Cisco Catalyst 2960G switch Independent out of band management network for IPMI Connection to the Internet or/and DC network via a router called Gateway and IP 172.16.1.1 Node Server roles: 1 server as Fuel Node 3 servers as Controller Node 1 server as Cinder Node 2 servers as Compute Node Network configuration plan: Public network 172.16.1.0/24 Floating network 172.16.0.0/24 in VLAN 100 Management network 192.168.0.0/24 in VLAN 101 Storage network 192.168.1.0/24 in VLAN 102 Private (Fixed) network 10.0.0.0/24 in VLAN 103 Administrative network (for Fuel) 10.20.0.0/24 in VLAN 104 Network Parameters: Fuel server IP: 10.20.0.2/24 Default gateway: 10.20.0.1 DNS 10.20.0.1 Note The Internet and rest of DC access is available through the Public network (for OpenStack Nodes) and Administrative network (Fuel server) From the server node side, ports with the following VLAN IDs for networks are used: eth0 - Management VLAN 101 (tagged), Storage VLAN 102(tagged) and Administrative VLAN 104 (untagged) 2015, Mirantis Inc. Page 37

Detailed Port Configuration eth1 - Public/Floating VLAN 100 (tagged), Private VLAN 103 (tagged) Detailed Port Configuration The following table describes the detailed port configuration and VLAN assignment. Switch Port Server name Server NIC tagged / untagged G0/1 Fuel eth0 untagged 104 G0/2 Fuel eth1 untagged 100 VLAN ID G0/3 Compute Node 1 eth0 tagged 101, 102, 104 (untagged) G0/4 Compute Node 1 eth1 tagged 100, 103 G0/5 Compute Node n eth0 tagged 101, 102, 104 (untagged) G0/6 Compute Node n eth1 tagged 100, 103 G0/7 Controller Node 1 eth0 tagged 101, 102, 104(untagged) G0/8 Controller Node 1 eth1 tagged 100, 103 G0/9 Controller Node 2 eth0 tagged 101, 102, 104 (untagged) G0/10 Controller Node 2 eth1 tagged 100, 103 G0/11 Controller Node 3 eth0 tagged 101, 102, 104 (untagged) G0/12 Controller Node 3 eth1 tagged 100, 103 G0/13 Cinder Node eth0 tagged 101, 102, 104 (untagged) G0/14 Cinder Node eth1 tagged 100, 103 G0/24 Router (default gateway) --- untagged 100 Connect the servers to the switch as in the diagram below: 2015, Mirantis Inc. Page 38

Detailed Port Configuration The following diagram describes the network topology for this environment. 2015, Mirantis Inc. Page 39

Nova-network Switch configuration (Cisco Catalyst 2960G) Nova-network Switch configuration (Cisco Catalyst 2960G) Use the following configuration to deploy Mirantis OpenStack using a Cisco Catalyst 2960G network switch.: service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption service sequence-numbers hostname OpenStack\_sw1 logging count logging buffered 64000 informational logging rate-limit console 100 except errors logging console informational enable secret r00tme username root privilege 15 secret r00tme no aaa new-model aaa session-id common ip subnet-zero ip domain-name domain.ltd ip name-server [ip of domain name server] 2015, Mirantis Inc. Page 40

Nova-network Switch configuration (Cisco Catalyst 2960G) spanning-tree mode rapid-pvst spanning-tree loopguard default spanning-tree etherchannel guard misconfig spanning-tree extend system-id ip ssh time-out 60 ip ssh authentication-retries 2 ip ssh version 2 vlan 100 name Public vlan 101 name Management vlan 102 name Storage vlan 103 name Private vlan 104 name Admin interface GigabitEthernet0/1 description Fuel Node eth0 switchport access vlan 104 switchport mode access spanning-tree portfast interface GigabitEthernet0/2 description Fuel Node eth1 (optional to have direct access to Public net) switchport access vlan 100 switchport mode access spanning-tree portfast interface GigabitEthernet0/3 description Compute Node 1 eth0 switchport trunk native vlan 104 switchport trunk encapsulation dot1q switchport trunk allowed vlan 101, 102, 104 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/4 description Compute Node 1 eth1 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 103 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/5 description Compute Node 2 eth0 switchport trunk native vlan 104 switchport trunk encapsulation dot1q 2015, Mirantis Inc. Page 41

Nova-network Switch configuration (Cisco Catalyst 2960G) switchport trunk allowed vlan 101, 102, 104 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/6 description Compute Node 2 eth1 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 103 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/7 description Controller Node 1 eth0 switchport trunk native vlan 104 switchport trunk encapsulation dot1q switchport trunk allowed vlan 101, 102, 104 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/8 description Controller Node 1 eth1 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 103 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/9 description Controller Node 2 eth0 switchport trunk native vlan 104 switchport trunk encapsulation dot1q switchport trunk allowed vlan 101, 102, 104 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/10 description Controller Node 2 eth1 switchport trunk encapsulation dot1 switchport trunk allowed vlan 100, 103 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/11 description Controller Node 3 eth0 switchport trunk native vlan 104 switchport trunk encapsulation dot1q switchport trunk allowed vlan 101, 102, 104 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/12 description Controller Node 3 eth1 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 103 switchport mode trunk spanning-tree portfast trunk 2015, Mirantis Inc. Page 42

Nova-network Switch configuration (Juniper EX4200) interface GigabitEthernet0/13 description Cinder Node eth0 switchport trunk native vlan 104 switchport trunk encapsulation dot1q switchport trunk allowed vlan 101, 102, 104 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/14 description Cinder Node eth1 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 103 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/24 description Connection to default gateway switchport access vlan 100 switchport mode access interface Vlan100 ip address 172.16.1.254 255.255.255.0 ip address 172.16.0.254 255.255.255.0 secondary no shutdown ip route 0.0.0.0 0.0.0.0 172.16.1.1 ip classless no ip http server no ip http secure-server line con 0 session-timeout 15 privilege level 15 login local password r00tme line vty 0 15 session-timeout 15 login local password r00tme ntp server [ntp_server1] prefer ntp server [ntp_server2] Nova-network Switch configuration (Juniper EX4200) Use the following configuration to deploy Mirantis OpenStack using Juniper EX4200 network switch.: 2015, Mirantis Inc. Page 43

Nova-network Switch configuration (Juniper EX4200) system { host-name OpenStack_sw1; domain-name domain.ltd; authentication-order [ password ]; root-authentication { encrypted-password "xxxxxxxxxxxxxxxxxxx"; services { ssh; ntp { server [ntp_server1] prefer; server [ntp_server2]; interfaces { ge-0/0/0 { description Fuel Node eth0; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_104; ge-0/0/1 { description Fuel Node eth1 (optional to have direct access to Public net); unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_100; ge-0/0/2 { description Compute Node 1 eth0; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; 2015, Mirantis Inc. Page 44

Nova-network Switch configuration (Juniper EX4200) native-vlan-id vlan_104; ge-0/0/3 { description Compute Node 1 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_100, vlan_103; ge-0/0/4 { description Compute Node 2 eth0; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; native-vlan-id vlan_104; ge-0/0/5 { description Compute Node 2 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_100, vlan_103; ge-0/0/6 { description Controller Node 1 eth0; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; native-vlan-id vlan_104; 2015, Mirantis Inc. Page 45

Nova-network Switch configuration (Juniper EX4200) ge-0/0/7 { description controller Node 1 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_100, vlan_103; ge-0/0/8 { description Controller Node 2 eth0; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; native-vlan-id vlan_104; ge-0/0/9 { description Controller Node 2 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_100, vlan_103; ge-0/0/10 { description Controller Node 3 eth0; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; native-vlan-id vlan_104; 2015, Mirantis Inc. Page 46

Nova-network Switch configuration (Juniper EX4200) ge-0/0/11 { description Controller Node 3 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_100, vlan_103; ge-0/0/12 { description Cinder Node 1 eth0; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; native-vlan-id vlan_104; ge-0/0/13 { description Cinder Node 1 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_100, vlan_103; ge-0/0/23 { description Connection to default gateway; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_100; vlan { unit 100 { 2015, Mirantis Inc. Page 47

Nova-network Switch configuration (Juniper EX4200) family inet { address 172.16.1.254/24; address 172.16.0.254/24; routing-options { static { route 0.0.0.0/0 next-hop 172.16.1.1; protocols { dcbx { interface all; rstp { bridge-priority 32k; interface ge-0/0/0.0 { edge; interface ge-0/0/1.0 { edge; interface ge-0/0/23.0 { edge; bpdu-block-on-edge; lldp { interface all; vlans { vlan_1; vlan_100 { description Public; vlan-id 100; l3-interface vlan.100; vlan_101 { description Management; vlan-id 101; vlan_102 { description Storage; vlan-id 102; 2015, Mirantis Inc. Page 48

Nova-network Switch configuration (Juniper EX4200) vlan_103 { description Private; vlan-id 103; vlan_104 { description Admin; vlan-id 104; 2015, Mirantis Inc. Page 49

Example 2: HA + Neutron with GRE Example 2: HA + Neutron with GRE As a model example, the following configuration is used: Deploying mode: Multi-node HA Networking model: Neutron with GRE Hardware and environment: 7 servers with two 1Gb/s ethernet NIC and IPMI 1 Cisco Catalyst 3750 switch Independent out of band management network for IPMI Connection to the Internet or/and DC network via a router called Gateway and IP 172.16.1.1 Node servers roles: 1 server as Fuel Node 3 servers as Controller Node 1 server as Cinder Node 2 servers as Compute Node Network Configuration Plan: Floating/Public network 172.16.0.0/24 in VLAN 100 (untagged on servers) Floating IP range 172.16.0.130-254 Internal network (private) 192.168.111.0/24 Gateway 192.168.111.1 DNS 8.8.4.4, 8.8.8.8 Tunnel ID range 2-65535 Management network 192.168.0.0/24 in VLAN 101 Storage network 192.168.1.0/24 in VLAN 102 Administrative network (for Fuel) 10.20.0.0/24 in VLAN 103 Network Parameters Fuel server: IP 10.20.0.2/24 Default gateway: 10.20.0.1 DNS: 10.20.0.1 Note The Internet and rest of DC access via Public network (for OpenStack Nodes) and Administrative network (Fuel server). 2015, Mirantis Inc. Page 50

Detailed port configuration From server side, ports with following VLAN IDs are used: eth0 - Administrative VLAN 103 (untagged) eth1 - Public/Floating VLAN 100 (untagged), Management VLAN 101 (tagged), Storage VLAN 102 (tagged) Detailed port configuration The following table describes port configuration for this deployment. Switch Port Server name Server NIC tagged / untagged G0/1 Fuel eth0 untagged 103 G0/2 Fuel eth1 untagged 100 G0/3 Compute Node 1 eth0 untagged 103 VLAN ID G0/4 Compute Node 1 eth1 tagged 100(untagged), 101, 102 G0/5 Compute Node n eth0 tagged 103 G0/6 Compute Node n eth1 tagged 100(untagged), 101, 102 G0/7 Controller Node 1 eth0 tagged 103 G0/8 Controller Node 1 eth1 tagged 100(untagged), 101, 102 G0/9 Controller Node 2 eth0 tagged 103 G0/10 Controller Node 2 eth1 tagged 100(untagged), 101, 102 G0/11 Controller Node 3 eth0 tagged 103 G0/12 Controller Node 3 eth1 tagged 100(untagged), 101, 102 G0/13 Cinder Node eth0 tagged 103 G0/14 Cinder Node eth1 tagged 100(untagged), 101, 102 G0/24 Router (default gateway) untagged 100 Neutron Switch configuration (Cisco Catalyst 2960G) Use the following configuration to deploy Mirantis OpenStack using a Cisco Catalyst 2960G network switch. service timestamps debug datetime msec localtime show-timezone service timestamps log datetime msec localtime show-timezone service password-encryption service sequence-numbers hostname OpenStack_sw1 logging count 2015, Mirantis Inc. Page 51

Detailed port configuration logging buffered 64000 informational logging rate-limit console 100 except errors logging console informational enable secret r00tme username root privilege 15 secret r00tme no aaa new-model aaa session-id common ip subnet-zero ip domain-name domain.ltd ip name-server [ip of domain name server] spanning-tree mode rapid-pvst spanning-tree loopguard default spanning-tree etherchannel guard misconfig spanning-tree extend system-id ip ssh time-out 60 ip ssh authentication-retries 2 ip ssh version 2 vlan 100 name Public vlan 101 name Management vlan 102 name Storage vlan 103 name Admin interface GigabitEthernet0/1 description Fuel Node eth0 switchport access vlan 103 switchport mode access spanning-tree portfast interface GigabitEthernet0/2 description Fuel Node eth1 (optional to have direct access to Public net) switchport access vlan 100 switchport mode access spanning-tree portfast interface GigabitEthernet0/3 description Compute Node 1 eth0 switchport access vlan 103 switchport mode access spanning-tree portfast 2015, Mirantis Inc. Page 52

Detailed port configuration interface GigabitEthernet0/4 description Compute Node 1 eth1 switchport trunk native vlan 100 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 101 102 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/5 description Compute Node 2 eth0 switchport access vlan 103 switchport mode access spanning-tree portfast interface GigabitEthernet0/6 description Compute Node 2 eth1 switchport trunk native vlan 100 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 101 102 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/7 description Controller Node 1 eth0 switchport access vlan 103 switchport mode access spanning-tree portfast interface GigabitEthernet0/8 description Controller Node 1 eth1 switchport trunk native vlan 100 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 101 102 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/9 description Controller Node 2 eth0 switchport access vlan 103 switchport mode access spanning-tree portfast interface GigabitEthernet0/10 description Controller Node 2 eth1 switchport trunk native vlan 100 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 101 102 2015, Mirantis Inc. Page 53

Detailed port configuration switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/11 description Controller Node 3 eth0 switchport access vlan 103 switchport mode access spanning-tree portfast interface GigabitEthernet0/12 description Controller Node 3 eth1 switchport trunk native vlan 100 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 101 102 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/13 description Cinder Node eth0 switchport access vlan 103 switchport mode access spanning-tree portfast interface GigabitEthernet0/14 description Cinder Node eth1 switchport trunk native vlan 100 switchport trunk encapsulation dot1q switchport trunk allowed vlan 100, 101 102 switchport mode trunk spanning-tree portfast trunk interface GigabitEthernet0/24 description Connection to default gateway switchport access vlan 100 switchport mode access interface Vlan100 ip address 172.16.1.254 255.255.255.0 ip address 172.16.0.254 255.255.255.0 secondary no shutdown ip route 0.0.0.0 0.0.0.0 172.16.1.1 ip classless no ip http server no ip http secure-server line con 0 2015, Mirantis Inc. Page 54

Neutron Switch configuration (Juniper EX4200) session-timeout 15 privilege level 15 login local password r00tme line vty 0 15 session-timeout 15 login local password r00tme ntp server [ntp_server1] prefer ntp server [ntp_server2] Neutron Switch configuration (Juniper EX4200) Use the following configuration to deploy Mirantis OpenStack using Juniper EX4200 network switch. system { host-name OpenStack_sw1; domain-name domain.ltd; authentication-order [ password ]; root-authentication { encrypted-password "xxxxxxxxxxxxxxxxxxx"; services { ssh; ntp { server [ntp_server1] prefer; server [ntp_server2]; interfaces { ge-0/0/0 { description Fuel Node eth0; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_103; ge-0/0/1 { 2015, Mirantis Inc. Page 55

Neutron Switch configuration (Juniper EX4200) description Fuel Node eth1 (optional to have direct access to Public net); unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_100; ge-0/0/2 { description Compute Node 1 eth0; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_103; ge-0/0/3 { description Compute Node 1 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; native-vlan-id vlan_100; ge-0/0/4 { description Compute Node 2 eth0; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_103; ge-0/0/5 { description Compute Node 2 eth1; unit 0 { 2015, Mirantis Inc. Page 56

Neutron Switch configuration (Juniper EX4200) family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; native-vlan-id vlan_100; ge-0/0/6 { description Controller Node 1 eth0; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_103; ge-0/0/7 { description controller Node 1 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; native-vlan-id vlan_100; ge-0/0/8 { description Controller Node 2 eth0; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_103; ge-0/0/9 { description Controller Node 2 eth1; unit 0 { family ethernet-switching { port-mode trunk; 2015, Mirantis Inc. Page 57

Neutron Switch configuration (Juniper EX4200) vlan { members vlan_101, vlan_102; native-vlan-id vlan_100; ge-0/0/10 { description Controller Node 3 eth0; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_103; ge-0/0/11 { description Controller Node 3 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; native-vlan-id vlan_100; ge-0/0/12 { description Cinder Node 1 eth0; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_103; ge-0/0/13 { description Cinder Node 1 eth1; unit 0 { family ethernet-switching { port-mode trunk; vlan { members vlan_101, vlan_102; 2015, Mirantis Inc. Page 58

Neutron Switch configuration (Juniper EX4200) native-vlan-id vlan_100; ge-0/0/23 { description Connection to default gateway; unit 0 { family ethernet-switching { port-mode access; vlan { members vlan_100; vlan { unit 100 { family inet { address 172.16.1.254/24; address 172.16.0.254/24; routing-options { static { route 0.0.0.0/0 next-hop 172.16.1.1; protocols { dcbx { interface all; rstp { bridge-priority 32k; interface ge-0/0/0.0 { edge; interface ge-0/0/1.0 { edge; interface ge-0/0/2.0 { edge; interface ge-0/0/4.0 { edge; 2015, Mirantis Inc. Page 59

Neutron Switch configuration (Juniper EX4200) interface ge-0/0/6.0 { edge; interface ge-0/0/8.0 { edge; interface ge-0/0/10.0 { edge; interface ge-0/0/12.0 { edge; interface ge-0/0/23.0 { edge; bpdu-block-on-edge; lldp { interface all; vlans { vlan_1; vlan_100 { description Public; vlan-id 100; l3-interface vlan.100; vlan_101 { description Management; vlan-id 101; vlan_102 { description Storage; vlan-id 102; vlan_103 { description Admin; vlan-id 103; 2015, Mirantis Inc. Page 60

Example 3: HA + Neutron with VLAN + SR-IOV & iser Example 3: HA + Neutron with VLAN + SR-IOV & iser As a model example, the following configuration is used: Deploying mode: Multi-node HA Networking model: Neutron with VLAN Hardware and environment: 16 servers (1U) with two 1Gb/s ethernet NIC on-board, IPMI 16 Mellanox ConnectX-3 Pro 40/56GbE adapter card 16 40/56GbE cables (Mellanox OPN MC2207130) 1 Mellanox Ethernet switch SX1036 (36 ports of 40/56GbE) 1 (or 2) 1G Ethernet switch 1 storage server (2U) with two 1Gb/s ethernet NIC on-board, IPMI 2 External JBODs + SATA cables - Optional disk + adaptec RAID controller Note The Fuel node doesn't have to be equipped with Mellanox ConnectX-3 Pro adapter card, as it is not connected to the high speed network. Node servers roles: 1 server as Fuel Node 3 servers as Controller Nodes 1 server as Storage Node (Cinder) 12 servers as Compute Nodes Network Configuration Plan: The switches and servers is designed in this example to support five networks as follows: Cloud Networks Admin (PXE) network - 10.142.10.0/24 (no gateway, untagged via the 1GbE switch) Managemenet network - 192.168.0.0/24 (no gateway, VLAN 3 via Mellanox SX1036 40/56GbE switch) Storage network - 192.168.1.0/24 (no gateway, VLAN 4 via Mellanox SX1036 40/56GbE switch) Public network - 10.7.208.0/24 (gateway 10.7.208.1, untagged via the 1GbE switch) Private network - <any> (no gateway, use range of VLANs e.g. 5-15 via Mellanox SX1036 40/56GbE switch) 2015, Mirantis Inc. Page 61

Example 3: HA + Neutron with VLAN + SR-IOV & iser Note Internet access is acheived via the Public network. Note All nodes should be connected to all networks (besides the Fuel node). Note The 1GbE switch configuraiton for the Public and Admin (PXE) networks are similar to examples 1 and 2 above. Network Configuation Floating IP range 10.7.208.101-200 DNS 8.8.4.4, 8.8.8.8 Fuel server: Admin (PXE) IP 10.142.10.10/24 From server side (all nodes), ports with following VLAN IDs are used: eth0 - Admin (PXE) - untagged eth1 - Public - untagged eth2 (40/56GbE port) - Management VLAN 3, Storage VLAN 4 (+ VLANS 5-15 for the privte networks) 2015, Mirantis Inc. Page 62

Example 3: HA + Neutron with VLAN + SR-IOV & iser Here is an example of the network diagram: 2015, Mirantis Inc. Page 63

Neutron Switch configuration (Mellanox SX1036) Rack Design Here is a recommended rack design configuration. Design the rack as the 1G switches are on top, followed by the servers, then the 40/56GbE switch and then the storage server and JBODs. Neutron Switch configuration (Mellanox SX1036) Use the following configuration to deploy Mirantis OpenStack using a Mellanox SX1036 40/56GE 36 port switch The switch configuration is required prior to the Fuel installation. Prior the installation, the network connectivity between all hosts should be ready. Here is an example of Mellanox switch VLAN configuration and flow control: switch > enable switch # configure terminal switch (config) # vlan 1-20 switch (config vlan 1-20) # exit # Note that VLAN 1 is an untagged VLAN by default switch (config) # interface ethernet 1/1 switchport mode hybrid switch (config) # interface ethernet 1/1 switchport hybrid allowed-vlan 2-20 switch (config) # interface ethernet 1/2 switchport mode hybrid switch (config) # interface ethernet 1/2 switchport hybrid allowed-vlan 2-20 switch (config) # interface ethernet 1/3 switchport mode hybrid switch (config) # interface ethernet 1/3 switchport hybrid allowed-vlan 2-20 2015, Mirantis Inc. Page 64