Scaling Web Applications in a Cloud Environment Emil Ong Caucho Technology 8621
2 AGENDA > This talk targets Java web developers and administrators interested in using cloud and enterprise virtualized environments > You'll learn about what features are necessary to make cloud computing useful and about an implementation of these features in Resin 4
3 AGENDA > > > > > Defining Cloud Computing Why Cloud Computing actually helps Features needed to make Clouds useful Example uses Demo
4 Defining Cloud Computing
5 What is Cloud Computing? > Lots of differing definitions! > Start simple: Cloud Computing provides an environment with the ability to add and remove computing, network, and/or storage capacity at runtime > Many new services emerging meet this definition, differing in APIs and implementation
6 Back to Earth: A Concrete Cloud > Virtualization is at the heart of Cloud Computing Cloud providers simply rent virtual instances You ask for a new instance with certain capabilities, they give it to you, usually at a metered rate > Many enterprises create their own virtualized environments > Google App Engine and similar offer a very different environment...
7 The promise of clouds > Scalability Long term Startup growth Short term Events Slashdotting > Reliability Disposable instances Geographic distribution > Management Let somebody else worry about it >... But can these promised be kept?
8 Making Cloud Computing Useful
9 What challenges do clouds introduce for Developers? > New APIs for data sharing API per cloud API per framework > Different development and deployment environments > What do we want? Code to standard APIs Easy transition to cloud Application scaling that doesn't rely on cloud Same application behavior whether on cloud or in development environment Portability!
10 What challenges do clouds introduce for Administrators? > Starting and stopping > What do we want? servers may interrupt Starting and stopping application state servers in a way that's > Deploying applications safe for applications across the cloud Consistent, cloud-wide Syncing new servers deployment of with existing applications applications > Load balancing A load balancer that can find all the cloud resources
11 Where should these capabilities be in the stack? > Cloud provider? No help for developers Opaque view of applications Portability problems Even if the providers could agree on APIs, enterprises would not get the advantages
12 Where should these capabilities be in the stack? > Framework level? Does your framework support cloud? Cannot manage application container-level resources Webapp lifecycle Sessions > Application Level? Portability problems Work!
13 Cloud capabilities in the Application Server > Usable in any virtualized environment External cloud Internal server farm > Capable of session management > Manages webapp lifecycle > Handles distribution of applications
14 Cloud enabing features > > > > > Distributed Sessions Remote deployment across all servers Load balancing Distributed Object Caching What makes this cloud computing? Application server support for Dynamic Servers
15 Cloud enabing features a User's View > Distributed Sessions Servlet API > Remote deployment across all servers Deploy once, deploy everywhere > Load balancing Should just work with servers coming and going > Distributed Object Caching Java Cache (javax.cache) > Goal Minimal disruption of developer and administrator process
16 Implementing the Cloud Features
17 How do we actually deliver these features? > Start with a collection of servers in a cloud...
18 A Proposed Architecture > Need a way to organize the servers for communication
19 Resin 4 Cloud Architecture > Triad servers Replicated base servers Backup application and temporary data > Dynamic servers Added and removed at will, during runtime Cache data for performance Not expected to be reliable
20 Dealing with Dynamic Servers > Adding new servers > Removing unnecessary servers > Handling with faulty servers
21 Adding Dynamic Servers > The cloud provider supplies a new virtual instance... then what? 1. Notify the existing servers that a new server is joining 2. Start the application server on the instance 3. New server joins in the application server network 4. Migrate applications, sessions, and cached objects > Issues Security in the cloud Race conditions Manageability
22 Notifying the existing servers 1. Notify an existing server that a new server will be started Provide an id, IP, and server port
23 Starting the Dynamic Server 2. Start the Resin server on the instance Provide id, IP, and port Use same configuration file as with existing servers Configuration contains authentication information
24 After a new dynamic server has started > Existing servers send applications to new dynamic server > Load balancer begins directing traffic to new server > New server can request cached objects from existing servers
25 Stopping a dynamic server > To remove the server stop app server, kill instance Can just kill the instance May not even lose the current request Sessions preserved in either case
26 Feature 1 Distributed Object Cache
27 Distributed Object Cache Features > Put an object in the cache > Requirements on one server, get it out on Reliability any other server Replication > Adding and removing Conflict resolution servers doesn't lose data Performance > Standard API (javax.cache) Don't over replicate Allows setting properties like expiration, et al.
28 Implementing the Distributed Object Cache > Triad servers replicate the data and are the authoritative copy > All other servers write-through a local memory cache to triads
29 Implementing the Distributed Object Cache > Cache values are broken into data and metadata M-Nodes contain hashes of the key and value, version, and other metadata Data is hash-indexed Content/key-based addressing for even distribution of data > Sessions implemented on top of distributed object cache
30 Feature 2 Distributed Sessions
31 Distributed Session Features > Distributed sessions give any server in the cloud the ability to handle sessions started on any other server > Reliability feature allows servers to fail or be shutdown without losing session data > For developers must use standard Servlet Session API Works for any framework using sessions
32 Distributed Sessions Requirements > Reliability Replication of sessions across several servers for failover > Performance Sticky sessions Pair sticky sessions with leases Don't over-replicate (avoid NxN problem)
33 Implementing Distributed Sessions > Use Object Cache for replication, distribution > Load balancer uses session ids and unique ordering of servers to implement sticky sessions Only state required is current list of servers
34 Sticky sessions and failover > A server currently handling the session has a lease on the session Leases are like update listeners for sessions Leases are not locks!
35 Feature 3 Cloud-wide Remote Deploy
36 Remote Deploy Features > Send applications once to deploy cloud-wide > Avoid unnecessary bandwidth usage by sending deltas between versions > Synchronize new servers with existing applications, without redeploying application manually > Safe, simple version rollover
37 Remote Deploy Architecture > Distributed Git repository stores applications > Deploy service on triads handles updates > New applications sent to all servers on deployment by synchronizing Git repositories > New servers synchronize repositories/applications on startup
38 Using Remote Deploy > Administrators use Ant, Maven, Eclipse, or a web console to deploy new applications as.war files > Applications are only started when tagged in repository
39 Feature 4 Load Balancing
40 Load Balancing Features > > > > Dynamic server discovery Ignore dead dynamic servers Sticky sessions Balanced load based on backend server capability Many cloud providers offer a variety of instance capacities
41 Load Balancing Architecture > Servers organized in clusters > Cluster server manifest sent to load balancing server(s) with runtime updates > Sticky sessions from session id > Monitor number of live requests per server > Include CPU load as part of balancing calculation
42 Implementing Load Balancing > Load balancer runs as a Servlet on a separate instance > Cluster manifest is kept in cache by backend cluster Load balancer server(s) read this cache on new requests > Sticky sessions use the cache value and the server id list to decide which server should receive this request > Based on CPU and number of requests, load assigned via Round-robin Least-loaded Green load balancing (last loaded, until full)
43 Use Cases
44 Use case: Startups > Instant infrastructure > Place almost all of compute power and storage in a cloud provider > Add capacity as load increases
45 Use case: Extra capacity for events > Planned events Sales Seasonal bursts (holidays, due dates, etc.) > Unplanned events News story > Maintain internal cluster for normal operations > Add capacity in cloud for extra load
46 Use case: Enterprise resource allocation > Fixed set of servers > Multiple applications > During different periods, different applications need more or less compute power > Create multiple clusters, one for each application Move physical servers or virtual instances between clusters based on load > Don't actually need virtual servers in this case!
47 Use case: Enterprise resource allocation
48 Demos
Emil Ong http://www.caucho.com/ http://blog.caucho.com/ Caucho Technology, Inc. emil@caucho.com
50 Appendix: Distributed Cache Put
51 Appendix: Distributed Cache Lookup
52 Multiple load balancers