Aeolus: Mastering the Complexity of Cloud Application Deployment

Transcription

1 Aeolus: Mastering the Complexity of Cloud Application Deployment Gianluigi Zavattaro University of Bologna - Italy FoCUS research team INRIA - France Based on joint work with: Tudor A. Lascu and Jacopo Mauro Univ. of Bologna Roberto Di Cosmo and Stefano Zacchiroli Paris Diderot

2 Aeolus: Aeolus is a project funded by Mastering the cloud complexity o (IRILL / UPD / INRIA) Aeolus 9/12/ IRILL 1 / 12 u Models, languages and tools for the administration of cloud applications n Cloud computing offers the possibility to build sophisticated software systems on virtualized infrastructures at a fraction of the cost necessary just few years ago n but the administration (deploying/ upgrading/reconfiguring) of such software systems is a serious challenge

3 New models and languages: an industrial need u Several industrial initiatives pursue the definition of high-level languages for the management of applications deployed on virtualized infrastructures

4 New models and languages: an industrial need u Cloud Foundry (launched by VMware) provides a PaaS with high-level primitives for service creation and binding $ cf create-service What kind?> 1 Name?> cleardb-e2006 Creating service cleardb-e OK $ cf bind-service 1: myapp Which application?> 1 1: cleardb-e2006 Which service?> 1 Binding cleardb-e2006 to myapp... OK

5 New models and languages: an industrial need u Juju (an Ubuntu initiative) provides similar primitives n service replication and scaling supported n includes GUI for application management

6 New models and languages: an industrial need

7 New models and languages: an industrial need u Phase 1: Define Puppet s declarative language lets you... describe your configurations and the relationships between resources

8 New models and languages: an industrial need u Declarative language: three kinds of resources service package { 'openssh-server': ensure => installed,} file hasrestart => true,} { '/etc/ssh/sshd_config': source => 'puppet:///modules/sshd/sshd_config', owner group mode => 'root', => 'root', => '640', { 'sshd': ensure => running, enable => true, notify => Service['sshd'], require => Package['openssh-server'],} hasstatus => true,

9 New models and languages: an industrial need u In all these approaches a lot of human intervention is needed for n Service selection n Deciding the service bindings n It is like having to manually select all the packages to be (un)installed at every reconfiguration of a Linux installation! u Aim of Aeolus: n automatize as much as possible the management of such applications

10 Structure of the talk u The Aeolus starting point u Formalizing the deployment problem u Solving the deployment problem n ExpSpace-hard in the general case n PolyTime without conflicts u The Aeolus Deployment Engine u Future and related work

11 Automatic management Distributions of package-based software systems a somewhat successful idea... u Developed rather sophisticated tools for FOSS (free and open-source software) opensuse, Debian, Red Hat, Ubuntu, Mandriva, Fedora, Linux Mint, PCLinuxOS, Slackware, Gentoo Linux, CentOS, FreeBSD, Arch, Sabayon, Puppy, Lubuntu, MEPIS, Ultimate, NetBSD, Tiny Core, Zenwalk, CrunchBang, Dreamlinux, Vector, Kubuntu, Maemo, aptosid, Peppermint, PC-BSD, Chakra, Salix, ClearOS, Aeolus: KNOPPIX, Mastering Xubuntu, the Complexity Super of Cloud OS, BackTrack, Application Deployment gos, TinyMe, Zentyal, EasyPeasy, ESOCC'13 Frugalware, Clonezilla, Pardus, Meego, OpenBSD, Quirky, PC/OS, Zorin, SystemRescue, Element, Unity, SliTaz, Macpup, wattos, Scientific, Mythbuntu, Slax, DragonFLY, Elive, linux-gamers, 64 Studio, mageia,

12 Package: wordpress Version: dfsg -0+squeeze1 Depends: httpd, mysql-client, php5, php5-mysql, libphp -phpmailer (>= ), The dependency/conflict model Package: mysql-server -5.5 Source: mysql -5.5 Version: u Tools are based on the dependency/conflict model Provides: mysql-server, virtual -mysql-server Depends: libc6 (>= 2.12), zlib1g (>= 1:1.1.4), debconf, [...] Pre -Depends: mysql-common (>= ), adduser (>= 3.40), debconf Package: apache2 Version: Maintainer: Debian Apache Maintainers <debian -apache@...> Depends: lsb-base, procps, perl, mime-support, apache2 -bin (= ), apache2 -data (= ) Conflicts: apache2.2-common Provides: httpd Description: Apache HTTP Server Fig. 1. Debian package metadata for WordPress, Mysql and the Apache web ser Use case 1 Package installation. Before considering the services that

13 Package configuration as a SAT problem u One boolean variable for each package n TRUE installed n FALSE not installed u Conflicts/dependencies can be formalized as boolean formulae u Finding a correct configuration is mapped to a satisfaction problem

14 Package configuration as a SAT problem u One boolean variable for each package n TRUE installed Advanced configuration tools u Conflicts/dependencies exploit state-of-the-art can SAT be solvers formalized as boolean formulae n FALSE not installed u Finding a correct configuration is mapped to a satisfaction problem

15 Structure of the talk u The Aeolus starting point u Formalizing the deployment problem u Solving the deployment problem n ExpSpace-hard in the general case n PolyTime without conflicts u The Aeolus deployment tool u Future and related work

16 The Aeolus component model u A component has provide and require ports Provide ports Require ports

17 The Aeolus component model u A component has provide and require ports u A component has an internal state machine Provide ports Require ports

18 The Aeolus component model u A component has provide and require ports u A component has an internal state machine u Ports are active or inactive according to the current internal state Provide ports Require ports

19 Packages in the Aeolus model u The packages example Towards a Formal Component Model for the Cloud R. Di Cosmo, S. Zacchiroli, and G. Zavattaro Package: wordpress (a) available components, not installed Version: dfsg -0+squeeze1 Depends: httpd, mysql-client, php5, php5-mysql, libphp -phpmailer (>= ), [...] Package: mysql-server -5.5 Source: mysql -5.5 Version:

20 Packages in the Aeolus model u Binding between two packages (a) available components, not installed 158 R. Di Cosmo, S. Zacchiroli, and G. Zavattaro Package: Version: wordpress dfsg -0+squeeze1 Depends: httpd, mysql-client, php5, php5-mysql, libphp -phpmailer (>= ), [...] Package: mysql-server -5.5 Source: mysql -5.5 Version: (b) installed components, bound together on the httpd port Fig. 2. Asimplegraphicaldescriptionofthebasicmodelofapackage requirements and provides of the different resources, that indicate which provide is ful-

21 Services in the Aeolus model u At the service level, also a running state becomes relevant: n wordpress need to know the network address of a running MySQL instance

22 Conflicts in the Aeolus model u Conflicts are expressed as special ports n The apache web server is in conflict with the lighttpd web server

23 Fig. 1: Typical Wordpress/Apache/MySQL deployment, modeled in Aeolus core. Formalizing the Definition 1 (Component type). The set Γ of component types of the Aeolus core model, ranged over by T,T 1,T 2,... contains 4-ples Q,q 0,T,D where: Fig. 1: Typical Wordpress/Apache/MySQL deployment, modeled in Aeolus core. deployment problem Q is a finite set of states containing the initial state q 0 ; T Q Q is the set of transitions; Definition D is a function 1 (Component from Q totype). a 3-ple The P,R,C set Γ of ofcomponent interface names types(i.e. of the P,R,C Aeolus I core ) indicating ranged the over provide, by T require,,t 1,T 2,... andcontains conflict ports 4-ples that Q,q each 0,T,D state where: activates. We assume model, that the initial state q 0 has no requirements and conflicts (i.e. D(q 0 )=P, /0, /0). Q is a finite set of states containing the initial state q 0 ; We T now Q define Q is the configurations set of transitions; that describe systems composed by components and their Dbindings. is a function Eachfrom component Q to a 3-ple has P,R,C a unique of identifier interface taken names from (i.e. the P,R,C set Z. I A ) configurationdicatingranged the provide, over by require, C 1,C 2 and,..., conflict is given ports bythat a set each of component state activates. types, We assume a set of in- components that the initial some state state, q 0 and has no a set requirements of bindings. and conflicts (i.e. D(q 0 )=P, /0, /0). Definition We now2 define (Configuration). configurations A configuration that describe systems C is a 4-ple composed U,Z,S,B by components where: and their bindings. Each component has a unique identifier taken from the set Z. A configuration, ranged over by C 1,C 2,..., is given by a set of component types, a set of U Γ is the finite universe of the available component types; Z Z is the set of the currently deployed components; components in some state, and a set of bindings. S is the component state description, i.e. a function that associates to components Definition Z a pair 2 (Configuration). T,q where TA configuration U is a component C is atype 4-ple Q,q U,Z,S,B 0,T,D, and where: q Q is the current component state; U Γ is the finite universe of the available component types; B I Z Z is the set of bindings, namely 3-ple composed by an interface, Z Z is the set of the currently deployed components; the component that requires that interface, and the component that provides it; we S is the component state description, i.e. a function that associates to components assume that the two components are different. in Z a pair T,q where T U is a component type Q,q 0,T,D, and q Q is the Aeolus: current Mastering component the Complexity state; of Cloud Application Deployment Configuration are equivalent if they have the same instances up to instance renaming. B I Z Z is the set of bindings, namely 3-ple composed by an interface, Definition 3 (Configuration equivalence). Two configurations U,Z,S,B and

24 r, is bound to an active port providing r, i.e. there exists a component z Z \{z} such that r,z,z B, C [z ]=T,q and r is in T.P(q ). Similarly, for conflicts, we write C Formalizing = cn f (z,c) to indicate that the conflict port c of component z is satisfied because no other component has an active port providing c, i.e. for every z Z \{z} with C deployment [z ]=T,q we have that c T problem.p(q ). The configuration C is correct if for every component z Z with S(z)=T,q we have that C = req (z,r) for every r T.R(q) and C = cn f (z,c) for every c T.C(q). Configurations evolve at the granularity of actions. Definition 5 (Actions). The set A contains the following actions: statechange(z 1,q 1,q 1,...,z n,q n,q n) where z i Z and i = j. z i = z j ; bind(r,z 1,z 2 ) where z 1,z 2 Z and r I ; unbind(r,z 1,z 2 ) where z 1,z 2 Z and r I ; newrsrc(z : T ) where z Z and T U is the component type of z; delrsrc(z) where z Z. Notice that we consider a set of state changes in order to deal with simultaneous installations like the one needed for Apache2 and Apache2-bin in Fig.??. The execution of actions is formalized as configuration transitions. Definition 6 (Reconfigurations). Reconfigurations are denoted by transitions C α C meaning Aeolus: Mastering thatthe the Complexity execution of Cloud of Application α ADeployment on the configuration CESOCC'13 produces a new configuration C. The transitions from a configuration C = U,Z,S,B are defined as follows:

25 unbind(r,z 1,z 2 ) where z 1,z 2 Z and r I ; newrsrc(z : T ) where z Z and T U is the component type of z; delrsrc(z) where z Z. Formalizing the Notice that we consider a set of state changes in order to deal with simultaneous installations like the one needed for Apache2 and Apache2-bin in Fig.??. The execution of deployment problem actions is formalized as configuration transitions. Definition 6 (Reconfigurations). Reconfigurations are denoted by transitions C α C meaning that the execution of α A on the configuration C produces a new configuration C. The transitions from a configuration C = U,Z,S,B are defined as follows: C statechange(z 1,q 1,q 1,...,z n,q n,q n) U,Z,S,B if i. C [z i ].state = q i and i. (q i,q i ) C [z i].trans C and S (z [zi ].type,q )= i if i. z = z i C [z ] otherwise C bind(r,z 1,z 2 ) U,Z,S,B r,z 1,z 2 if r,z 1,z 2 B and r C [z 1 ].req C [z 2 ].prov C unbind(r,z 1,z 2 ) U,Z,S,B \ r,z 1,z 2 if r,z 1,z 2 B C newrsrc(z:t ) U,Z {z},s,b if z Z, T U T,T.init if z and S (z )= = z C [z ] otherwise C delrsrc(z) U,Z \{z},s,b if z if S (z )= = z C [z ] otherwise and B = {r,z 1,z 2 B z {z 1,z 2 }} 4

26 Deployment problem u Input: n A set of component types (called Universe) n One target component type-state pair u Output: n Yes, if there exists a deployment plan n No, otherwise Deployment plan: a sequence of actions bringing to a final configuration containing at least one component of the given target type, in the given target state

27 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Deployment problem: example build. In the kerberos case, krb5 is built in the first stage u missing Consider out the the generation problem of the krb5-ldap of installing package. Then openldap can be built directly into its normal build satisfying kerberos with lpad support in Debian its dependencies. Once openldap is built, krb5 can also be build into its normal stage. This process would be modeled in Aeolus as depicted in Fig. 2. n Universe: packages krb5 and openldap n Target: krb5 in normal state Fig. 2: Representation of the krb5 and openldap components.

28 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Deployment problem: example build. In the kerberos case, krb5 is built in the first stage missing out the generation of the krb5-ldap package. Then newrsrc(krb5), openldap can be builtnewrsrc(openldap), directly into its normal build satisfying stage1(krb5), its dependencies. Once bind(libkrb,openldap,krb5), openldap is built, krb5 can also be build into its normal stage. This process would be modeled in normal(openldap), bind(libldap,krb5,openldap), Aeolus as depicted in Fig. 2. normal(krb5) u Deployment plan: Fig. 2: Representation of the krb5 and openldap components.

29 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Deployment problem: example build. In the kerberos case, krb5 is built in the first stage missing out the generation of the krb5-ldap package. Then newrsrc(krb5), openldap can be builtnewrsrc(openldap), directly into its normal build satisfying stage1(krb5), its dependencies. Once bind(libkrb,openldap,krb5), openldap is built, krb5 can also be build into its normal stage. This process would be modeled in normal(openldap), bind(libldap,krb5,openldap), Aeolus as depicted in Fig. 2. normal(krb5) u Deployment plan: Fig. 2: Representation of the krb5 and openldap components.

30 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Deployment problem: example build. In the kerberos case, krb5 is built in the first stage missing out the generation of the krb5-ldap package. Then newrsrc(krb5), openldap can be builtnewrsrc(openldap), directly into its normal build satisfying stage1(krb5), its dependencies. Once bind(libkrb,openldap,krb5), openldap is built, krb5 can also be build into its normal stage. This process would be modeled in normal(openldap), bind(libldap,krb5,openldap), Aeolus as depicted in Fig. 2. normal(krb5) u Deployment plan: Fig. 2: Representation of the krb5 and openldap components.

31 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Deployment problem: example build. In the kerberos case, krb5 is built in the first stage missing out the generation of the krb5-ldap package. Then newrsrc(krb5), openldap can be builtnewrsrc(openldap), directly into its normal build satisfying stage1(krb5), its dependencies. Once bind(libkrb,openldap,krb5), openldap is built, krb5 can also be build into its normal stage. This process would be modeled in normal(openldap), bind(libldap,krb5,openldap), Aeolus as depicted in Fig. 2. normal(krb5) u Deployment plan: Fig. 2: Representation of the krb5 and openldap components.

32 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Deployment problem: example build. In the kerberos case, krb5 is built in the first stage missing out the generation of the krb5-ldap package. Then newrsrc(krb5), openldap can be builtnewrsrc(openldap), directly into its normal build satisfying stage1(krb5), its dependencies. Once bind(libkrb,openldap,krb5), openldap is built, krb5 can also be build into its normal stage. This process would be modeled in normal(openldap), bind(libldap,krb5,openldap), Aeolus as depicted in Fig. 2. normal(krb5) u Deployment plan: Fig. 2: Representation of the krb5 and openldap components.

33 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Deployment problem: example build. In the kerberos case, krb5 is built in the first stage missing out the generation of the krb5-ldap package. Then newrsrc(krb5), openldap can be builtnewrsrc(openldap), directly into its normal build satisfying stage1(krb5), its dependencies. Once bind(libkrb,openldap,krb5), openldap is built, krb5 can also be build into its normal stage. This process would be modeled in normal(openldap), bind(libldap,krb5,openldap), Aeolus as depicted in Fig. 2. normal(krb5) u Deployment plan: Fig. 2: Representation of the krb5 and openldap components.

34 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Deployment problem: example build. In the kerberos case, krb5 is built in the first stage missing out the generation of the krb5-ldap package. Then newrsrc(krb5), openldap can be builtnewrsrc(openldap), directly into its normal build satisfying stage1(krb5), its dependencies. Once bind(libkrb,openldap,krb5), openldap is built, krb5 can also be build into its normal stage. This process would be modeled in normal(openldap), bind(libldap,krb5,openldap), Aeolus as depicted in Fig. 2. normal(krb5) u Deployment plan: Fig. 2: Representation of the krb5 and openldap components.

35 Structure of the talk u The Aeolus starting point u Formalizing the deployment problem u Solving the deployment problem n ExpSpace-hard in the general case n PolyTime without conflicts u The Aeolus Deployment Engine u Future and related work

36 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Can we reduce to well-known models? (as SAT for packages) build. In the kerberos case, krb5 is built in the first stage u Deployment missing out the generation plans ofrecall the krb5-ldap firing package. sequences Then openldap can be built directly into its normal build satisfying in Petri nets: its dependencies. Once openldap is built, krb5 can also be n build Tokens into its normal are moved stage. Thisfrom processource would beplaces modeled into Aeolus as depicted in Fig. 2. target places by transitions Fig. 2: Representation of the krb5 and openldap components.

37 ing between the ified by id 1 and tified by id 2 ; tate of the commore than one ponents. For inonfiguration dete the resources apache2, a), and three new comy. All these new e initial state for ponents can be e(a, uninst, inst) int, to be able the mysql inst inst,m,w). After Can we reduce to well-known models? (as SAT for packages) build. In the kerberos case, krb5 is built in the first stage missing out the generation of the krb5-ldap package. Then u In fact, reduction to Petri nets is possible openldap can be built directly into its normal build satisfying n its but dependencies. reachability Once openldap problems is built, in krb5petri can also nets be are build into its normal stage. This process would be modeled undecidable in the presence of inhibitor arcs Aeolus as depicted in Fig. 2. (necessary to simulate conflicts) Fig. 2: Representation of the krb5 and openldap components.

38 Decidability of the deployment problem [ICALP 13] u Backward search algorithm inspired by the theory of WSTS (Well-Structured Transition Systems) n WSTS are popular in the context of infinite state systems verification Initial conf.. Target conf.

39 Decidability of the deployment problem [ICALP 13] u Key point: ordering C 1 C 2 on configurations s.t. n if C 1 has a given component, also C 2 has it Initial conf.. Target conf.

40 Decidability of the deployment problem [ICALP 13] u Key point: ordering C 1 C 2 on configurations s.t. n if C 1 has a given component, also C 2 has it n if C 1 C 2 and C 1 à C 1 then C 2 à C 2 with C 1 C 2 Initial conf.. Target conf.

41 Decidability of the deployment problem [ICALP 13] u Key point: ordering C 1 C 2 on configurations s.t. n if C 1 has a given component, also C 2 has it n if C 1 C 2 and C 1 à C 1 then C 2 à C 2 with C 1 C 2 n is a wqo: finite basis and finite antichains. Initial conf.. Target conf.

42 Complexity [ICALP 13] u The complexity of the problem is ExpSpace-hard! (reduction from coverability in standard Petri nets)

43 Alg. 1 creates a reachability graph that visually could be seen as a pyramid an initial empty configuration. This is achieved through a forward symbolic reachability analysis that relies on an abstract representation of components. For each component its individual identity as well as the number of its instances are ignored, keeping only its component type and its state T,q. Also, we abstract away from individual bindings without considering delete actions. The abstraction on the bindings is possible since we can safely[sefm 12] assume that, given a set of components, all complementary ports on two distinct components are bound. Delete actions are superfluous since the presence of one component does u Forward reachability algorithm not hinder the reachability of a state in another component. Quadratic algorithm (without conflicts) n all reachable states computed by saturation Algorithm 1 Reachability graph construction 1: Nodes 0 = {T, T.init T U}; provport = T,q Nodes 0 {T.P(q)}; i = 0; 2: repeat 3: i = i + 1; 4: Arcs i, Nodes i = ; 5: for all T,q Nodes i 1 do 6: for all (q, q ) T.trans do 7: if T.R(q ) provport then 8: Nodes i.add(t, q ); 9: for all T,q Nodes i do 10: provport.add(t.p(q)); 11: Nodes i = Nodes i 1 Nodes i 12: for all T,q Nodes i 1, T,q Nodes i do 13: if (q, q ) T.trans then 14: Arcs i.add(t,q T,q); 15: if q == q then 16: Arcs i.add(t,q T,q); 17: until Nodes i 1 == Nodes i

44 n the 1 and 2; comone r indeurces, and comnew te for an be,inst) able l inst After d by figug the omeiolate nd to, can ports o not build. In the kerberos case, krb5 is built in the first stage missing out the generation of the krb5-ldap package. Then openldap Example: can be built directly into its normal build satisfying its dependencies. Once openldap is built, krb5 can also be build into its normal stage. This process would be modeled in Aeolus as depicted in Fig. 2. the kerberos case-study a pyramid of levels of component-states having arrows or arcs between two consecutive levels as the one in Fig. 3. Fig. 2: Representation of the krb5 and openldap components. All reachable A. Reachability analysis The first step in the proposed technique checks if the the desired states target state can be reached. To do so all reachable states are computed, for each of the component types in the given universe. In the following we use the pair T,q to denote a Aeolus: component Mastering type the Complexity T and one of Cloud of its Application state q. Deployment An increasing sequence of sets of component-state pairs S 0,...,S n is built in such a way that S i+1 extends S i with Initial states New states Fig. 3: Reachability graph for the kerberos running example.

45 Structure of the talk u The Aeolus starting point u Formalizing the deployment problem u Solving the deployment problem n ExpSpace-hard in the general case n PolyTime without conflicts u The Aeolus Deployment Engine u Future and related work

46 From deployment to planning u The deployment problem simply replies yes / no u A real deployment tool needs to know how to reach the target configuration n In other words, an actual deployment plan should be computed

47 Using standard planning tools u First attempt: exploit state-of-the-art planning tools n Translation from the Aeolus model to PDDL (planning domain definition language) n Exploit popular available planning tools u Results: w Metric-FF (based on the GraphPlan algorithm) w Madagascar-p (translation to SAT) n Less than 10 components can be managed! (exponential explosion of the state space)

48 Ad-hoc planning [ICTAI 13] ws or in Fig. 3. u Use the reachability graph bottom-up from the target state n select the bindings (red arrows) n select the predecessors (black arrows) (a) example. (b) Fig. 4: Generation of abstract plan for the kerberos example.

49 or 3. elected, we al path that nd reaches nce in the tarting le. from g the state penldap in identify the b5 the. ed ent e it ap ce lly, om ce its create, e added to.e. first the he deletion ed between terface r. Ad-hoc planning (a) replacing z,x,y with z,x,y where z,x,y is a delete action or it is the the first statechange reaching a state y that does not provide r. After applying these relaxations we obtain the final version of abstract plan that, for the kerberos case, is the one depicted in Fig. 6. (b) Time Fig. 4: Generation of abstract plan for the kerberos example. target component-state pair at the bottom of the pyramid. From the bottom level we then proceed upward selecting the components that are used to deploy the selected componentstate pairs at the lower level. To do so, for every selected component at level i + 1, we select at level i one of its predecessors (i.e. a component-state pair connected via the Fig. Aeolus: 6: Abstract Mastering plan the Complexity for the kerberos of Cloud Application example after Deployment relaxation. arrow) or a copy (i.e. a component-state pair connected via the arc). Moreover, for every require port activated by [ICTAI 13] u Generate an abstract plan (one component for each maximal path) Arrows represent a precedence relation: u blue: start requirement u red: end requirement

50 elected, we al path that nd reaches nce in the tarting from g the state penldap in identify the. its create, e added to.e. first the he deletion ed between terface r. Ad-hoc planning u Plan as a topological visit until target: newrsrc(krb5), newrsrc(openldap), replacing z,x,y with z,x,y where z,x,y is a delete action or it is the the first statechange reaching a state y that does not provide r. After applying these relaxations we obtain the final version of abstract plan that, for the kerberos case, is the one depicted in Fig. 6. Time stage1(krb5), bind(libkrb,openldap,krb5), normal(openldap), bind(libldap,krb5,openldap), normal(krb5) Fig. Aeolus: 6: Abstract Mastering plan the Complexity for the kerberos of Cloud Application example after Deployment relaxation. [ICTAI 13] Arrows represent a precedence relation: u blue: start requirement u red: end requirement

51 Ad-hoc planning [ICTAI 13] u Problem: cycles could forbid the topological visit u Example: krb5 in normal requires an openldap in uninst state

52 Ad-hoc planning kerberos example in which the component type krb5 in normal state requires not only one openldap in normal state, but also one in uninst state. In thisu case The the target abstract plan state will be as in Fig. 7 (note the addition of the pair of arcs labeled with cannot be visited! uninst). [ICTAI 13] Fig. 7: Abstract plan for the modified kerberos example. the insta duplicat node. C r arc the othe importan the new outgoing new ins instance from the Fig. 8. I have inc the case node so we appl this dup

53 to provide an interface during a specific phase of the plan, but this is not possible because in the same phase of the plan the instance is required to change its internal state. This problem can be solved by means of instance duplication: an additional component instance is deployed in such a way that the new instance can continue providing the required interface during Ad-hoc planning that specific phase of the plan. The application of instance duplication to the abstract plan in Fig. 7 is reported in Fig. 8, where we add a new instance y of type openldap that does not proceed further than state uninst. This new resource is used to satisfy the requirement uninst of z. Notice that the topological visit until the target node z, stage1, normal becomes now possible. [ICTAI 13] u Solution: component duplication Fig. 8: Abstract plan in Fig. 7 after duplication of instance w. the dependencies se reachability graph t dependencies canno nodes at a lower to In the context of very important appl developing language change and, more sp Since 1998, a dec defined for establish in order to allow dif This language is kn Our tool solves a to validate our ad-h do so we have defi problem into PDDL into one PDDL ob to state changes. T only when the othe required interfaces. unbind actions 2 and be concurrently use

54 T + 6 Test error T 7.22 s 7 error Metric-FF Madagascar-p Metric-FF timeout 0.05TABLE s I: Performances 0.05 s of standard planners s timeout Ad-hoc 1.96 s 7.22 s planning running example. We have considered configurations composed error by an increasing number of components having 3 states (excluding one component having only 2 states used to trigger the plan). formances of standard planners. have considered configurations comnumber of components having 3 states nent having only 2 states used to trigger require ports in such a way that a valid plan to reach "!! the state no duplication the components, then perform from cn to c0 the)! state change ent types considered in our tests s 0.41 s timeout s 1.96 s Fig. 9: Component types considered in our tests. As depicted in Fig.??, the components have provide and s2 of the rightmost component would require first to create from s0 to s1, and then perform the state change from s1 to s2 from c1 to cn. In these scenarios instance duplication '! is not needed during the generation of the concrete plan. In order to test also this feature we modified the component %! types by removing for nearly one fifth of the components the activation of the provide port duplication px 1 from the states s2, where #! X is the number of the selected component. Removing this activation of the provide port requires the duplication of the instance of the component type cx in order to contemporaneously satisfy the requirements of its two neighbor components. The activation of the provide port to be deleted were selected randomly. In the following we will denote respectively with T the test of. 9, the components have provide and way that a valid plan to reach the state omponent would require first to create perform from cn to c0 the state change the solver took more than 120s and was thus interrupted. Even though the best solver among the two is Madagascarp, it is able to solve scenarios with just 6 components for the simpler T test. Metric-FF, in the T + case, times out even when considering a scenario with 3 components. [ICTAI 13] These poor encoding into SAT becomes too big to be handled. performances are due to the fact that the size of the encoding of the planning problem u Solution: increases exponentially component w.r.t. the number of components thatduplication need to be deployed concurrently. Metric- FF times out because it wastes all its time trying to ground all the possible actions while Madagascar-p fails because the encoding into SAT becomes too big to be handled. +5/26748 "#! p, it is able to solve scenarios with just 6 components for the simpler T test. Metric-FF, in the T + case, times out even when considering a scenario with 3 components. These poor performances are due to the fact that the size of the encoding of the planning problem increases exponentially w.r.t. the number of components that need to be deployed concurrently. Metric- FF times out because it wastes all its time trying to ground all the possible actions while Madagascar-p fails because the n Efficiently done during the topological visit (limited overhead) +, + Fig. 10: Time performances using our tool!! "!! #!! $!! %!! &!! '!! (!! )!! *!! Fig.?? shows instead the time taken by our tool to produce a valid plan for tests T and T +. Clearly our ad-hoc planner outperforms the general purpose planners. In both cases it is able to produce plans in less than two minutes for scenarios involving more than 700 components. As expected, duplication affects the performance of the planner. The degradation of performances, however, is still manageable: plans can be provided in less than a minute even for scenarios with hundreds -./ of components that need replication. The tool was developed in OCaml and is about 3.5 K lines of code. The source code of our tool and the problem encoding

55 Mandriva ADE (Aeolus Deployment Engine) u Mandriva (the Aeolus industrial partner) is integrating the Aeolus technologies in their Mandriva Business Service platform = +

56 A proof-of-concept case study u Aeolus model has been extended with: n Capacity planning

57 A proof-of-concept case study u Aeolus model has been extended with: n Capacity planning n Locations (virtual machines)

58 A proof-of-concept case study u Automatic deployment: n A constraint solver computes the target configuration

59 A proof-of-concept case study u Automatic deployment: n The planning alg. computes the low-level deployment actions

60 A proof-of-concept case study u Automatic deployment: n The plan is translated in Mandriva Server Setup scripts

61 A proof-of-concept case study u Automatic deployment: n Demo video at:

62 Structure of the talk u The Aeolus starting point u Formalizing the deployment problem u Solving the deployment problem n ExpSpace-hard in the general case n PolyTime without conflicts u The Aeolus Deployment Engine u Future and related work

63 Work to be done u Hierarchical locations n from virtual machines to administrative domains, geograpical areas, u Restricted shapes for the automata to reduce the complexity also with conflicts n acyclic, states totally ordered w.r.t. the requirements on the environment, u High-level languages for resource description and specification of the target configuration u

64 class Machine { var os as OperatingSystem; var cpus as 1..4; var memory as int; } tion into MiniZinc. Finally, we provide experimental results demonstrating that our method outperforms previ- for web servers: a reference to any instance of the appropriate type, so if we always wanted ws1 to run on m1 we would also need ous work, before discussing the implications and directions for future work. var machine as ref Machi abstract class Role { to write: } 2 Modelling with ConfSolve Related work ws1.machine = m1; var m as Machine; class WebServer extends Ro ConfSolve provides the user with Which an is object-oriented an example of an equality var port constraint. as ; In which m is a Machine object declarative in thelanguage, global scope, with with a Java-like Constraints: syntax, which Constraints adheres cpus, to several an integer key subrange; principles: } are expressions which must members os, an enumeration; u ConfSolve [LISA 12] hold in any solution to the model. For example, introducing and memory, an unbounded integer. 1. Order never matters. Declaration and a database-server usage can oc- role which can be either a slave Member variables may also declare objects, allowing or master, and must be peered References: with another slave Associations or mas- the nesting of child objects within a parent object. For elled using reference types. example, we could add a network interface to the machine definition: enum DatabaseRole { Master, Consider Slave an } instance of the objects elsewhere in the mo w n Object-oriented cur in any orderlanguage with no difference for inservices meaning. and ter, as appropriate: machines 2. Everything is an expression, except declarations. 3. All classes are equal: there are no built-in classes n Type system with special for meanings checking such asconfiguration Machine or File. correctness class Machine { class DatabaseServer extends Variables and Classes: A ConfSolve model consists var ws1 Role as { WebServer;... var role as DatabaseRole; of a global scope in which strongly-typed variables, n Constraint solver for var en as NetworkInterface; classes, and enumerations may be declared. For example, a simple placement machine may of be defined as: machine was declared as a M In the previous declaration of } // slave or master automatic var peer as ref DatabaseServer; contains a reference to a m class NetworkInterface services on machines enum { OperatingSystem { Windows, UNIX, OSX } refer to m, as it is the only m var subnet as..3; // the peer cannot be itself clared. The solver will autom } peer!= this; class Machine { a reference to any instance o var os as OperatingSystem; // a master s peer mustwe bealways a slave, wanted ws1 to run An instance of NetworkInterface var cpus will as be created 1..4; whenever a Machine is instantiated. varthe memory lifetime as int; of the Net- role!= peer.role; // and a slave s peer must to write: be a master workinterface instance is} tied to that of its parent object, and is Aeolus: not shared Mastering between the Complexity different of Cloud instances Application of Ma- Deployment } ws1.machine = m1; chine. var m as Machine; This allow us to define two database server roles: Which is an example of an e

65 Related work u Engage [PLDI 12] n Architectural specification in terms of inside / peer / environment relationships n Automata with resource lifecycle and transient dependencies n Strong assumption on acyclic relationships (to always guarantee topological visit) Figure 5 shows the dependencies generated when processing partial instantiation specification of Figure 2. We have marked resource instances that were present in the specification with a inside inside Server f Tomcat running inside server, inside and an instance openmrs for [ active] inside he OpenMRS application running inside tomcat. In particular, the install start restart JDK JRE MySQL uninstalled inactive active ser does not have to explicitly give the other dependencies on Java env env peers nd MySQL. The partial installation specification may also define uninstall stop alues for individual Tomcat configuration portinside properties. In Open our example, MRS [ inactive] he hostname and os user name properties have been assigned alues. Unassigned configuration properties will take the default Figure 3: Resource driver for Tomcat alues defined infigure the associated 5:HypergraphforpartialinstallationspecificationinFigure 2 Aeolus: Mastering resourcethe types. Complexity of Cloud Application Deployment The configuration engine takes this partial installation. specifiation and expands out all the dependencies to generate a set of r oolean constraints such that the Boolean constraints are satisfi- ure 3 shows an example resource driver for Tomcat. An action Lemma 1. Let R be a set of well-formed resource types an a partial install specification. Then procedure GraphGen(R creates a directed hypergraph G = (V, E), such that: (i) for resource instance r I, wehaver V, and for each reso instance r V, either r I or there is some resource inst I that is transitively dependent on the key of r; (ii) each resource instance r V, if r.inside is not null, then t is a resource instance r I such that there is an inside

66 Publications and project web site u u u u Roberto Di Cosmo, Stefano Zacchiroli, Gianluigi Zavattaro. Towards a Formal Component Model for the Cloud. Proc. of SEFM 12: LNCS 7504, Springer. Roberto Di Cosmo, Jacopo Mauro, Stefano Zacchiroli, Gianluigi Zavattaro. Component Reconfiguration in the Presence of Conflicts. Proc. of ICALP 13: LNCS 7966, Springer Tudor A. Lascu, Jacopo Mauro, Gianluigi Zavattaro. A Planning Tool Supporting the Deployment of Cloud Applications. To appear in proc. of ICTAI 13. IEEE Press