Deploying and Monitoring Ruby on Rails A practical guide

Transcription

1 Deploying and Monitoring Ruby on Rails A practical guide Mathias Meyer and Jonathan Weiss, Peritor GmbH

2 Who are we? Jonathan Weiss Consultant for Peritor GmbH in Berlin Specialized in Rails, Scaling, Deployment, and Code Review Webistrano - Rails deployment tool FreeBSD Rubygems and Ruby on Rails maintainer 2

3 Who are we? Mathias Meyer Independent Contractor Specialized in Rails, Performance/Database Tuning, Deployment, and Refactoring Macistrano Webistrano-Client for Mac OS X (and soon the iphone) 3

4 Agenda Infrastructure Deployment Practical Session Monitoring Q & A 4

5 Infrastructure Infrastructure 5

6 Simple Rails Setup One Rails instance handles all requests Rails is single-threaded: There is only one concurrent request 6

7 Rails Setup 7

8 Rails Setup 8

9 Typical Rails Setup A load-balancer distributes the incoming requests Some load-balancers will deliver static requests themselves Several Rails instances handle all requests Number of concurrent requests equals number of Rails instances 9

10 Rails Setup Options 10

11 Deployment Questions Mongrel? mod_proxy_balancer? Proxy? Apache? Pound? FastCGI? Load-balancer? Nginx? HA-Proxy? Ebb? Swiftiply? mod_rails? Reverse Proxy? Phusion Passenger? Thin? Rails Application Server? Lighttpd? Pen? 11

12 What we are going to cover today Rails Application Server FastCGI Mongrel mod_rails / Phussion Passenger JRuby + Glassfish & Co. Proxy/Web Server Apache2 Nginx Lighttpd HA-Proxy 12

13 FastCGI 13

14 FastCGI Protocol to communicate with long-running CGI applications Usage of either mod_fcgi with Apache 1.3 or mod_fcgi with Lighttpd Proxy local and remote FastCGI instances Oldest way of deploying Rails Deprecated and unstable Hard to debug (FastCGI protocol) 14

15 FastCGI 15

16 FastCGI 16

17 17

18 Mongrel 18

19 Mongrel Developed by Zed Shaw as an alternative to FastCGI Complete HTTP-Server that can load arbitrary Ruby-servlets Built-in Rails support 19

20 Mongrel 20

21 Mongrel Apache 2.2 mod_proxy_balancer Lighttpd Nginx HA-Proxy Pound 21

22 Mongrel Cluster Utility to manage several Mongrel instances 22

23 Mongrel Cluster Control Mongrels 23

24 Mongrel and Apache 2.2 Define Mongrel Cluster in Apache 24

25 Simple Mongrel and Apache 2.2 Redirect all traffic to the Mongel cluster 25

26 A more complex example Redirect dynamic requests Serve static content Support cached pages Support maintenance page Enable client-side caching of images, stylesheets, and JavaScript Compress output if supported 26

27 Mongrel Very robust Strict HTTP parser Easy to debug (HTTP!) Defacto standard deployment with Apache 2.2 and mod_proxy_balancer Can be a bit difficult to setup (mongrel_cluster, ports, Apache) Not so easy on mass/virtual hosting 27

28 mod_rails 28

29 mod_rails a.k.a Phusion Passenger Fairly new module for Apache 2.2 Allows Apache to control Rails instances Apache starts and stops application instances depending on the application load Very easy to setup Able to run any RACK-compatible Ruby application (Merb & Co.) Only manages Rails on one host - no remote instances Combine with HTTP-Proxy / balancing solution 29

30 Install Phusion Passenger Install Apache module Load and activate in Apache 30

31 Customized Phusion Passenger Control Rails instance number 31

32 Control Phusion Passenger Restart after deployment: 32

33 Phusion Passenger One machine 33

34 Phusion Passenger One machine Multiple machines 34

35 Phusion Passenger Fairly new but ready for production Makes setup easier on the single machine level Multiple servers still require load balancer Suitable for mass-hosting The upcoming standard way of deploying Rails 35

36 JRuby 36

37 Ruby Runtime on the Java Virtual Machine Implemented in Java and Ruby Compiles Ruby into Java-bytecode Integrates with Java code and libraries Java s promises of native threads and JIT Allows for Ruby/Rails applications to be packaged as WAR files WAR files deployable on any J2EE-container: Glassfish, JBoss, Tomcat, Jetty, 37

38 JRuby on Rails 38

39 JRuby on Glassfish One machine 39

40 JRuby on Glassfish One machine Multiple machines 40

41 Setup JRuby on Glassfish 1. Download JRuby and Glassfish 2. From 41

42 Warble Configuration Define min/max Rails runtimes 42

43 Rails Setup 43

44 Proxy Options 44

45 Proxy Requirements Hide cluster backend from the user Load-balancer backend instances Recognize down hosts Fair scheduler (Deliver static content) 45

46 Apache 2.2 Apache 2.2 introduced mod_proxy_balancer mod_proxy_balancer can speak to multiple backends and balance requests Apache can acts as a pure proxy or can also serve static files 46

47 Apache 2.2 Apache 2.2 introduced mod_proxy_balancer mod_proxy_balancer can speak to multiple backends and balance requests Apache can acts as a pure proxy or can also serve static files 47

48 Apache 2.2 Apache 2.2 introduced mod_proxy_balancer mod_proxy_balancer can speak to multiple backends and balance requests Apache can acts as a pure proxy or can also serve static files 48

49 Apache 2.2 Pro Stable, robust, and mature Many people know how to work with Apache Integrates well with other modules (SVN, DAV, Auth, ) Con Apache can be complicated to configure The stock Apache is quite resource-hungry compared to pure proxy solutions 49

50 Nginx From Russia with love Nginx - popular Russian webserver with good proxy support Can load-balance multiple backends and deliver static content Quite popular with Mongrel as the Rails backend 50

51 Nginx Configuration Simple proxy example Get complete version here: 51

52 Nginx Pro Stable, robust, and fast Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files Simpler configuration file Can directly talk to memcached - SSI Con More documentation would be nice No equivalent for many Apache modules 52

53 Lighttpd Lightweight and fast webserver Balancing proxy support Good FastCGI support Used to be popular until Mongrel came around 53

54 Lighttpd Configuration Simple proxy example 54

55 Lighttpd Pro Fast and lightweight Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files Simpler configuration file Con Unstable for some people Slow development cycle More documentation would be nice Configuration file can be too simple (virtual host aliasing) No equivalent for many Apache modules 55

56 HA-Proxy HAProxy reliable, high performance TCP/HTTP load balancer Proxying and content inspection No content serving, just a proxy Mature proxy module (fair scheduler) ACL support See also similar Pound and Pen 56

57 HAProxy Simple proxy example 57

58 HAProxy Pro Mature, stable, robust, and fast TCP and HTTP balancing Con Few Rails examples Usually not needed in a Rails setup 58

59 Recommended Setups 59

60 Small Site Recommendation Apache 2.2 with mod_rails / Phusion Passenger 60

61 Medium Site Recommendation Apache 2.2 as the frontend proxy Use Mongrel or mod_rails as the backend Deliver static files with Apache 61

62 Medium Site Recommendation Apache 2.2 as the frontend proxy Use Mongrel or mod_rails as the backend Deliver static files with Apache 62

63 Large Rails Setup Recommendation Redundant load-balancer Redundant proxy / web Mongrel / mod_rails 63

64 Heavy Static Files Recommendation Deliver static assets through separate web server farm Mongrel or mod_rails 64

65 Java Shop Recommendation Deliver a Rails-WAR file and you are done Integrate with existing Java landscape and infrastructure 65

66 Remarks 66

67 Ruby Enterprise Edition Copy-On-Write patches to Ruby 1.8 Saves memory when spawning several Rails instances Used by Phusion Passenger if available 67

68 Thin, Ebb, Evented Mongrel & Co. Alternatives to Mongrel Claim to be faster, lighter, and what have you Rendering Hello World is usually not your bottleneck Stick with stable and robust Mongrel 68

69 In The Future, Watch Fuzed Erlang based load balancing Dynamic registration JRuby Faster Runtime Java Integration Phusion Passenger Enterprise Ruby Performance Usability 69

70 Infrastructure Deployment 70

71 71

72 Deployment Options 72

73 Deployment Options 73

74 Deployment Options 74

75 Deployment Options 75

76 What does Capistrano do? 76

77 Capistrano Deployment Cycle 77

78 Requirements 78

79 What doesn t Capistrano do? Plan your initial server setup Configure basic services 79

80 Basic Ingredients The cap command Variables Roles Tasks Namespaces 80

81 Basic Ingredients - cap Your one-stop deployment shop 81

82 Basic Ingredients - Variables Configure basic project information Override Capistrano s default assumptions Once set, variables are available globally Defined using the set method 82

83 Basic Ingredients - Roles Define types of servers Default roles :www :app :db All can point to the same server But all three must be defined At least one database server needs to be primary 83

84 Basic Ingredients - Roles Define custom roles as you please Can be reused when defining tasks 84

85 Basic Ingredients - Tasks Define an atomic set of actions for Capistrano Can be called from the command line Or other tasks 85

86 Basic Ingredients - Tasks To find all the tasks available in your project, use 86

87 Basic Ingredients - Namespaces Group tasks together logically Namespaces and tasks are separated with : 87

88 Get Your Capistrano On 88

89 Get Your Capistrano On Capfile, the place to include more recipes 89

90 Get Your Capistrano On config/deploy.rb, application specific configuration 90

91 Capistrano s Defaults Your SCM is Subversion Deployment directory is /u/apps/#{application_name} User for SCM and SSH is the currently logged-in user Commands are run with sudo 91

92 Get Your Capistrano On 92

93 Get Your Capistrano On Capistrano expects a directory structure Can be created with cap deploy:setup 93

94 The Deployment Lifecycle 94

95 The Deployment Lifecycle Check the prerequisites: 95

96 The Deployment Lifecycle Set up your application for the first time 96

97 The Deployment Lifecycle The initial deployment 1. Checks the revision from the local machine 2. Checks out the code on the remote machines 3. Sets a link called current pointing to the lates release 4. Runs the migrations 5. Fires up application servers 97

98 The Deployment Lifecycle 98

99 The Deployment Lifecycle Subsequent deployments 1. Checks the revision from the local machine 2. Checks out the code on the remote machines 3. Updates current link 4. Restarts application servers 99

100 The Deployment Lifecycle 100

101 Common Capistrano Tasks Deploy and run migrations Run only the migrations Restart application servers Rollback to the previous release 101

102 Have Your Shell, and Eat It Too 102

103 Invoking any Command 103

104 Deployment Strategies 104

105 Deployment Strategies Direct checkout (from scratch) on the servers 105

106 Deployment Strategies Keep a cached copy of the current SCM head 106

107 Deployment Strategies Check out locally and transfer 107

108 Give it a little Spin Capistrano expects a script called spin in script/process For Passenger 108

109 Customizing Capistrano 109

110 Write your own Tasks 110

111 Write your own Tasks 111

112 Namespace your Tasks 112

113 Callbacks Execute a task before another runs Execute a task after another has finished Callbacks are run in the order they re defined 113

114 Useful Variables 114

115 Transactions 115

116 Transactions 116

117 Transactions 117

118 Transactions 118

119 The Rest Gem dependencies Support for deploying through gateway servers Server setup with deprec gem Lack of documentation 119

120 One Click Deploy 120

121 Webistrano 121

122 Webistrano Web-UI to Capistrano Manages projects and their stages Alerting and Accounting Scriptable and extendable BSD License 122

123 Macistrano Mac-GUI to Webistrano Fire and monitor deployments from your desktop 123

124 Infrastructure Pratical Session 124

125 Practical Capistrano 125

126 Get the slides Connect to RailsConf Deployment WLAN The slides are at The VMs are at Sample configurations files are in the GIT/SVN repositories 126

127 Infrastructure Monitoring 127

128 The two questions of monitoring 128

129 1. Is everything still running? 129

130 2. What are the trends? 130

131 Monit Process-level monitoring Checks PID-files, ports, and permissions Reacts by executing a script and/or alerting 131

132 Monit MySQL 132

133 Monit Apache 133

134 Monit Mongrel 134

135 Munin Host-level monitoring Master periodically asks nodes for local data Checks system resources and records historical data Allows to recognize trends and make predictions Alerting support 135

136 Munin 136

137 Munin 137

138 Munin 138

139 Other Tools Nagios Big Brother New Relic RPM FiveRuns JMX 139

140 Q & A 140

141 Peritor GmbH Teutonenstraße Berlin Telefon: +49 (0) Telefax: +49 (0) Internet: [email protected] Peritor GmbH - Alle Rechte vorbehalten