Chef Integration. Chef Integration. with IDERA s Uptime Cloud Monitor. Simple, Smart, Seamless May 10, 2013 IDERA

Transcription

1 Chef Integration Chef Integration with IDERA s Uptime Cloud Monitor Simple, Smart, Seamless May 10, 2013 IDERA 1

2 Contents Preface Part I Chef + Uptime Cloud Monitor Infographic Part II Integrating Chef Workflows with Uptime Cloud Monitor Performance Monitoring Part III Bootstrap an EC2 Windows Server Instance Part IV Uptime Cloud Monitor Recipes Part V Uptime Cloud Monitor Probe LWRP Summary To learn more visit idera.com today! 2

3 Preface Across Uptime Cloud Monitor (formerly CopperEgg) users, there are numerous deployment methods being utilized to quickly get environments up and running. Chef, from Opscode, is a tool that allows for smart automation, deployment, and more. The Uptime Cloud Monitor Chef Cookbook provides recipes to help gain real-time visibility into the performance and availability of all Chefdeployed applications and services. This Whitepaper will help introduce the integration of Chef workflows with the Uptime Cloud Monitor service and application performance monitoring. 3

4 Part I - Chef + Uptime Cloud Monitor Infographic 4

5 Part II - Integrating Chef Workflows with Uptime Cloud Monitor Performance Monitoring The self-service, automated provisioning of IT resources has come of age and is rapidly becoming ubiquitous. In this paradigm a single DevOps person, armed with tools like Opscode Chef and Puppet, can (almost) instantly bring any number of virtual servers online, each with its own set of application and infrastructure services, and start everything up... and it all just works. The above is absolutely true, but obviously naïve. Achieving that level of automation is possible, but it takes a team of people with time and effort to get there, and an ongoing effort to stay there. While one person can trigger a deployment, it takes a team of folks to learn the tools and build and test all of the components necessary to make that possible. At every step along the path to maintaining a cost-effective automated service deployment, you must always be monitoring. Monitoring application performance, service availability, cost of service delivery, degree of under or over-provisioning, the sensitivity of your service response time to varying customer load, etc. (just to name a few!) is necessary to the process. The purpose of this blog is to introduce you to integrating your Chef workflows with Uptime Cloud Monitor service and application performance monitoring. Assumptions a Chef workstation configured with a chef-repo installed, a Hosted Chef account, or have a Chef server, and you have a cloud provider, from which you can provision and configure nodes, or chef- managed servers. 5

6 1. Setup Add the Uptime Cloud Monitor cookbook to your workstation chef-rep/cookbooks directory: Navigate to the IDERA Uptime Cloud Monitor page at the Opscode Community site, here: Download the copperegg.gz file to your chef-repo/cookbooks directory Unzip as follows: tar -xvzf copperegg.gz 4. With an editor, open the file chef-repo/cookbooks/copperegg/ attributes/default.rb. On line 13 you will see: default[:copperegg][:apikey] = YOUR_USER_APIKEY replace YOUR_USER_APIKEY with your Uptime Cloud Monitor User API Key (you can find it in the Uptime Cloud Monitor UI; click the Settings tab, and scroll down to the bottom of the page) OPTIONAL: Scroll down to the bottom of the default.rb file, and find default[:copperegg][:create_sshprobe] = false ; replace false with true, and an SSH probe will be created for each monitored server. This feature is disabled by default. save default.rb, and close it Upload your new cookbook to your chef server using knife: knife cookbook upload Uptime Cloud Monitor Your CopperEgg cookbook is now installed! Example 1: Bootstrap a Centos EC2 instance with Uptime Cloud Monitor s server monitoring Install the ec2 plug-in for knife: o gem install knife-ec2 Set up your knife.rb for AWS: knife[:aws_access_key_id] = YOUR_AWS_ACCESS_KEY_ID knife[:aws_secret_access_key] = YOUR_AWS_SECRET_ACCESS_KEY knife[:aws_ssh_key_id] = YOUR_AWS_SSH_KEY_ID knife[:identity_file] = PATH_TO_YOUR_PEMFILE On the command line: knife ec2 server create -f t1.micro -G default -I ami N Centos6 -r 6

7 Example 1: Bootstrap a Centos EC2 instance with Uptime Cloud Monitor s server monitoring Instance ID: i f Flavor: t1.micro Image: ami Region: us-east-1 Availability Zone: us-east-1b Security Groups: default Security Group Ids: default Tags: { Name => Centos6 } SSH Key: your-ec2keys Root Device Type: ebs Root Volume ID: vol-0a1234 Root Device Name: /dev/sda Public DNS Name: ec compute-1.amazonaws.com Public IP Address: Private DNS Name: ip ec2.internal Private IP Address: Environment: _default Run List: copperegg Now log in to your CopperEgg UI, and click on the Systems tab. You will see a widget representing your newly-provisioned EC2 Centos6 server instance, which will look similar to the below: 7

8 Things to notice: The -r copperegg appended to the knife ec2 server create command means run the default copperegg recipe. The final line of the response (Run List: copperegg) indicates that this command was carried out. Also notice the second-to-last line, Environment: _default. That is referring to the name of my Chef environment. By default, when you add the Uptime Cloud Monitor recipe to the run list of any chef-managed node, the Chef environment, roles assigned to the node, and any Chef tags applied to the node will be propagated to Uptime Cloud Monitor as tags. Notice that the _default tag appears in the widget in the above picture. If you set default[:copperegg][:create_sshprobe] = true in the attributes/ default.rb file, click on the Probes tab in the Uptime Cloud Monitor UI. There you will see a widget representing the newly created SSH probe that was created to monitor SSH port availability to your new server. It will appear similar to that shown below: Again, notice that the Chef environment tag appears on the probe widget. 8

9 Part III - Bootstrap an EC2 Windows Server Instance Below are details on how you can use Chef to bootstrap an EC2 Windows server instance, also with Uptime Cloud Monitor server monitoring up and running. Important: Do This First when deploying on Windows! The Uptime Cloud Monitor Cookbook contains a file named metadata.rb. Open this file with you favorite editor, and find the following lines: # Uncomment to include support for Windows #depends windows Un-comment the second line: depends windows Now save the file, and close. What s up with that? If your deployments do not include Windows, leaving this line commented- out makes the most sense, to minimize your chef run times. If your deployments include windows, you will need to uncomment this line. My understanding is that in a future release of Chef, there will be simple means of making depends platform-specific. 1. Example 2: Bootstrap a Windows EC2 instance with Uptime Cloud Monitor server monitoring First, prepare your EC2 Windows instance (clearly, this preparation should be done on a single EC2 instance, and the AMI saved; then do your bootstrapping using your saved AMI, which will include your auth credentials and this winrm prep.) Log in to your Windows server instance Open a command prompt 9

10 On Windows Server 2012, open a command prompt as follows: Open a PowerShell consoles as administrator o If necessary, update your execution policy Set-ExecutionPolicy Unrestricted cmd.exe /c start cmd /k Pslist A windows command prompt will appear Note also that you may have to change your Windows Firewall settings differently on Windows Server 2012 vs. previous releases. Execute the following commands : winrm quickconfig q winrm set 300 } winrm set } winrm set true } winrm set true } (this would be a good point to create your own AMI instance.) Install the ec2 plug-in for knife gem install knife-ec2 Install the windows plug-in for windows gem install knife-windows Set up your knife.rb for AWS: knife[:aws_access_key_id] = YOUR_AWS_ACCESS_KEY_ID knife[:aws_secret_access_key] = YOUR_AWS_SECRET_ACCESS_KEY knife[:aws_ssh_key_id] = YOUR_AWS_SSH_KEY_ID knife[:identity_file] = PATH_TO_YOUR_PEMFILE on the command line: knife bootstrap windows winrm ec2-xxx-xx-xx-xxx.compute- 1.amazonaws.com -x Administrator -P YourPassword -r copperegg node-name winserver-12 You should see a new widget on your Uptime Cloud Monitor System Dashboard for the newly bootstrapped Windows server. One last note, for those who are just getting started with Chef on Windows: you can fire off a chef run from your terminal with the following command: knife winrm ec2- XXX-XX-XX-XXX.compute-1.amazonaws.com chef-client -c c:/chef/client.rb -m -x Administrator -P YourPassword 10

11 Part IV Uptime Cloud Monitor Recipes In the Uptime Cloud Monitor Cookbook/recipes directory, you will find: default.rb copperegg-handler.rb reload.rb restart.rb start.rb stop.rb uninstall.rb The first two listed (default and copperegg-hander) are likely the only two recipes that you will ever use. The other 5 are provided for covering use cases that we can imagine, but don t expect to occur. default.rb This is the recipe that will be installed when you add copperegg to your run list. Its principle function is to install the Uptime Cloud Monitor collector, and will work for most Linux distributions and Windows operating systems. *** Important: When you first install the Uptime Cloud Monitor cookbook, you need to edit the associated attributes file, which is found here: copperegg/ attributes/default.rb. Open that file with your editor, and modify the line reproduced below by replacing YOUR_USER_APIKEY with your User API Key. default[:copperegg][:apikey] = YOUR_USER_APIKEY All of the other settings have defaults and shouldn t be a surprise, but I ll cover them later. 11

12 copperegg-handler.rb This is a plug-in to the Chef Reporting and Exception system. With the copperegg-handler installed, failed Chef runs will be posted as Annotations on your CopperEgg graphs. Optionally, you can post failed AND successful runs, which may seem like overkill, but are valuable if your do chef runs ad-hoc. Here are the steps to enable Annotations for a chef-managed instance: In each recipe or run list that you have included copperegg, add the following: chef_handler copperegg::copperegg-handler That s all! You will begin seeing new Annotations immediately after your next Chef run. reload.rb This recipe could be named reinstall because that is just what it does. Should a circumstance arise when you want to re-install the collector, you have two choices: uninstall / reinstall in 2 chef runs: replace copperegg with copperegg::uninstall ; chef run then replace copperegg::uninstall with copperegg ; chef run reload insert copperegg::reload above copperegg in your run list chef run At the end of the chef run with copperegg::reload included, the collector will have been uninstalled and re-installed, and copperegg::reload will be automatically removed from your run list. restart.rb start.rb stop.rb These scripts are included for you to start, stop, and restart the collector service, independent of the operating system upon which the collector is running. uninstall.rb Replace copperegg with copperegg::uninstall in your run list to stop the collector, and uninstall all CopperEgg software. 12

13 Part V - Uptime Cloud Monitor Probe LWRP Uptime Cloud Monitor recently introduced its probe LWRP, which is included in the Uptime Cloud Monitor cookbook. You will find the probe resources and provider here: copperegg/resources/probe.rb copperegg/providers/probe.rb This provider allows you to instantly add, update and remove any number of external probes on any chef run. **Example: Add an ssh probe for each EC2 server instance as it is brought up The following code could be placed in a recipe, and added to the run list for each EC2 Linux server that will be brought up: host = CheckPort22_#{node[ hostname ]} target = #{node[ ec2 ][ public_ hostname ]}:22 copperegg_probe host do provider copperegg_probe action :update probe_desc host probe_dest target type TCP end host is the string that we ll use to name or describe the probe we re creating. target is the destination IP:port to probe. ASIDE: The above is nearly identical to the code you will find in copperegg/ recipes/default.rb. In fact, an attribute exists in copperegg/attributes/ default.rb that if set, will cause a new ssh probe to be created for each of the Linux servers you boot (if they are EC2 instances). The attributes is: default[:copperegg][:create_sshprobe] = false 13

14 To remove the above probe, you would make a call similar to the above, with the action set for delete: host = CheckPort22_#{node[ hostname ]} target = #{node[ ec2 ][ public_hostname ]}:22 copperegg_probe host do provider copperegg_probe action :delete probe_desc host probe_dest target type TCP end (This is effectively the same code as that found in the uninstall recipe.) Key points: Three parameters combine to define a unique probe: probe_desc: the probe description (name) probe_dest: probe target address type: the probe type (GET, POST, TCP or ICMP). These three parameters are the minimum required to create a probe. All other parameters have reasonable defaults, which can be overridden. The various parameters are described in the probe resources file, copperegg/resources/probe.rb. Extensive details about the Uptime Cloud Monitor probe API can be found here: 14

15 Summary Chef is a great automation and deployment tool that goes hand in hand with Uptime Cloud Monitor server and application performance monitoring. Automated infrastructure is not complete without maintenance and monitoring. Many find integrating monitoring solutions into the automation process to be a seamless way to ensure performance. We hope you found this whitepaper a useful tool to learn how to install the Uptime Cloud Monitor cookbook, bootstrap EC2 servers, and integrate Uptime Cloud Monitor into the automation process. Be sure to check out the Uptime Cloud Monitor Chef Cookbook. For more information: 15