Opsview in the Cloud. Monitoring with Amazon Web Services. Opsview Technical Overview

Transcription

1 Opsview in the Cloud Monitoring with Amazon Web Services Opsview Technical Overview

2 Page 2 Opsview In The Cloud: Monitoring with Amazon Web Services Contents Opsview in The Cloud... 3 Considerations... 3 Why use Amazon s Private Cloud?... 3 Cloud monitors included in Opsview... 5 AMIs & Your VPC... 6 Configuring Opsview s Cloud Monitors... 7 Security... 7 Amazon... 7 Configuration... 7 Your Amazon Host... 8 Attributes... 8 Example Installation... 8 Opsview GUI... 9 Conclusion... 11

3 Page 3 Summary ( VPC ) infrastructure. As more infrastructure services are hosted in the cloud, Why Use Amazon s Virtual Private Cloud organizations need to adjust their monitoring to ensure visibility to off-premise infrastructure is maintained. This paper will detail how Opsview can integrate with your Amazon s Web Services ( AWS ) platform to provide a complete monitoring solution for your EC2, Amazon provides detailed use cases to help you understand the different scenarios that a VPC can cater for. Amazon s documentation provides guidance on the scenarios a VPC can be utilized. Operating System, Relational Database Service and Elastic Load Balancer infrastructure. Opsview in The Cloud Host a simple, public-facing website: You can host a basic web application, such as a blog or simple website in a VPC, and gain the additional layers of privacy and security afforded by Amazon VPC. You can help secure Opsview provides Amazon Machine Images built against Ubuntu s Long Term Support release. Our AMI s support both Enterprise and Pro offerings. Deploying your Opsview instance in the cloud is as simple as clicking through to the Amazon Data Centre you wish to deploy your monitoring instance to via the downloads section of opsview.com. Upon selection you will be taken to the website by creating security group rules which allow the webserver to respond to inbound HTTP and SSL requests from the Internet while simultaneously prohibiting the webserver from initiating outbound connections to the Internet. You can create a VPC that supports this use case by selecting VPC with a Public Subnet Only from the Amazon VPC console wizard. your AWS account where you can load our AMI instance. Considerations Host multi-tier web applications: You can use Amazon VPC to host multi-tier web applications and strictly enforce access and security restrictions between your Opsview s licensing system works by linking your entitlement ID to the MAC address and hostname of your installation. It is therefore of crucial importance that both variables do not change. To prevent this, Opsview must be run in Amazon s Virtual Private Cloud webservers, application servers, and databases. You can launch webservers in a publicly accessible subnet and application servers and databases in non-publically accessible subnets. The application servers and databases can t be directly accessed from the Internet,

4 Page 4 but they can still access the Internet via a NAT instance to download patches, for example. You can control access between the servers and subnets using inbound and outbound packet filtering provided by network access control lists and security groups. To create a VPC that supports this use case, you can select VPC with Public and Private Subnets in the Amazon VPC console wizard. your network by connecting your VPC to your corporate network. Because your VPC can be hosted behind your corporate firewall, you can seamlessly move your IT resources into the cloud without changing how your users access these applications. You can select VPC with a Private Subnet Only and Hardware VPN Access from the Amazon VPC console wizard to create a VPC that supports this use case. Host scalable web applications in the AWS cloud that are connected to your datacenter: You can create a VPC where instances in one subnet, such as webservers, communicate with the Internet while instances in another subnet, such as application servers, communicate with databases on your corporate network. An IPsec VPN connection between your VPC and your corporate network helps secure all communication between the application servers in the cloud and databases in your datacenter. Webservers and application servers in your VPC can leverage Amazon EC2 elasticity and Auto Scaling features to grow and shrink as needed. You can create a VPC to support this Disaster Recovery: You can periodically backup your mission critical data from your datacenter to a small number of Amazon EC2 instances with Amazon Elastic Block Store (EBS) volumes, or import your virtual machine images to Amazon EC2. In the event of a disaster in your own datacenter, you can quickly launch replacement compute capacity in AWS to ensure business continuity. When the disaster is over, you can send your mission critical data back to your datacenter and terminate the Amazon EC2 instances that you no longer need. By using Amazon VPC for disaster recovery, you can have all the benefits of a disaster recovery site at a fraction of the normal cost. use case by selecting VPC with Public and Private Subnets and Hardware VPN Access in the Amazon VPC console wizard. It s clear that running your infrastructure within a VPC environment has a significant amount of benefits. Placing Opsview within a VPC should therefore become a Extend your corporate network into the cloud: You natural extension of your existing Cloud environment. can move corporate applications to the cloud, launch additional webservers, or add more compute capacity to

5 Cloud Monitors included in Opsview Page 5 With the release of Opsview v4.4, monitor support for a wide range of Amazon Web Services is included. Specifically, we will be looking at EC2, RDS and ELB monitoring. Our service checks utilize Amazon s CloudWatch API to allow full monitoring of your Cloud infrastructure. These service checks can be applied to any aspect of your VPC estate; from an EC2 AMI instance, to a Relational Database Server to the Elastic Load Balancer distributing traffic across your AMIs. Combining these checks with our comprehensive service checks for the different Operating Systems running in your Cloud environment from Windows Server to Red Hat Enterprise Linux provides total visibility into your Cloud infrastructure. In the next section, we will look at how to deploy the Amazon Web Services checks.

6 Page 6 AMIs and Your VPC You will see above, we have a fully configured VPC environment from which to deploy our Opsview AMI. From opsview.com, you can select an appropriate Opsview AMI to be loaded into the region you wish to run monitoring instance. Remember to select a minimum of M1. Small for your Instance Type when prompted.

7 Page 7 Next, we need to install software to support the EC2 monitors (these will be included by default with later Amazon AMI releases). Configuring Opsview s Amazon Cloud Monitors must be owned by nagios user, otherwise the values cannot be read by the plugin). Amazon You will need access to the Amazon Web Services account you wish to monitor, along with a valid Access Opsview requires that the CloudyScripts rubygem is installed. Our Opsview provided AMIs include everything you need to get up and running. However, if you have installed Opsview directly into an AMI, please follow the Key and its secret key. See Amazon s Documentation for information about using keys for security. Configuration steps below. Use the encrypt_aws_credentials.rb script to generate gem install CloudyScripts your credentials file: Security ruby /usr/local/nagios/libexec/ plugin.d/cloud-aws/encrypt_aws_ An RSA key must be generated and saved to enable encryption of your AWS details. Use the commands - credentials.rb -A (your Amazon access key ID) -S (your Amazon secret key) -f /usr/local/nagios/etc/aws_credentials. mkdir /etc/cloutomate/ cfg ssh-keygen -f /etc/cloutomate/ cloutomate.pem You can find your access key(s) by logging into the AWS site and accessing Security Credentials from your account s drop-down. If you don t know the secret key, entering a blank passphrase, to create this file. you may need to create a new key. NOTE: sudo may be required in order to execute these The created file must not be distributed down to slaves. commands. These files (cloutomate.pem and aws_

8 Your Amazon Host Attributes Page 8 You will need access to the Amazon Web Services account you wish to monitor, along with a valid Access Key and its secret key. See Amazon s Documentation for AWS_CLOUDWATCH_CREDENTIALS must have arg2 set to the Amazon Instance ID - also available from their Management Console. information about using keys for security. Example Installation Environment: Opsview 4.4 clean installation, running on Debian Squeeze. AWS: We already have our secret key and our access key. See above how to get these. We have an EC2 instance running, we have the PUBLIC DNS of it, and its INSTANCE ID (i-3232 ). sudo mkdir /etc/cloutomate sudo ssh-keygen -f /etc/cloutomate/cloutomate.pem sudo gem install CloudyScripts sudo ruby /usr/local/nagios/libexec/plugins.d/cloud-aws/encrypt_aws_ credentials.rb -A (Amazon Access ID) -S (Amazon Secret Key) -f /usr/local/ nagios/etc/aws_credentials.cfg sudo chown -R nagios:nagios /etc/cloutomate sudo chown nagios:nagios /usr/local/nagios/etc/aws_credentials.cfg

9 Opsview GUI Page 9 1. Click on Settings > Hosts 2. Click on Add (The plus icon) 3. Enter the public address in the Primary Hostname/IP field. For RDS, use the Endpoint: value, but remember to remove the :3306 from the address. 4. At the bottom of the same page, in Host Templates, ensure that Cloud - AWS - EC2 is added to your host: 5. Click Next at the bottom. 6. Click on Attributes button. 7. Add a new attribute using the plus icon.

10 Page In the drop down box, choose AWS_CLOUDWATCH_CREDENTIALS. 9. Click on the eye icon, next to the trash icon. You may need to enter a holding value in the value field, i.e. Credentials below. 10. Tick the box next to Override Arg2. In the now available text box, enter your AWS instance ID, i.e. i-323xxxx. For RDS, enter the DB instance name, i.e. Mysqldatabasename. 11. Click submit changes. 12. Click Settings > Apply Changes. Then click Reload configuration. 13. After your Opsview has finished reloading, Navigate to Monitoring > Hosts, and your newly monitored EC2 device will be visible:

11 Page 11 Conclusion This technical guide has detailed both the type of monitors available within Opsview and how to deploy them. The inclusion of Amazon Web Services health checks in Opsview provides a rapid way to monitor your elastic Amazon environment. Through Amazon s Cloudwatch API, you can monitor detailed health statistics without the requirement to install an Opsview monitoring agent on each EC2 instance you spin up. To ensure a quick registration, your enterprise can utilize Amazon s Opsworks service to automatically register new instances into Opsview using the fully featured REST API. Finally, your AWS cloud estate can made highly available using a combination of Amazon s RDS MySQL offering, in co-operation with Opsview s Professional Services who can ensure your Opsview monitoring server is quickly and easily recovered in another EC2 standby instance should an event impact upon the operation of the master.