1 What is Cloud Computing? Cloud Infrastructures OpenStack Amazon EC CAMF Cloud Application Management

Transcription

1 1 What is Cloud Computing? Cloud Infrastructures OpenStack Amazon EC CAMF Cloud Application Management Frameworks CAMF Framework for Eclipse Eclipse TOSCA Specification for Cloud Applications CAMF Requirements CAMF Installation Instructions Getting Started Open CAMF Perspective Create a new Cloud Project Manage Cloud Providers Add Cloud Provider Edit Cloud Provider Cloud Application Description Create application description file Fetch Cloud provider info Check authentication details Describe Cloud application structure/contextualization Specify application s elasticity requirements Submit application description to Cloud provider Deploy application to Cloud provider Create deployment file Watch deployment status Monitoring Create JCatascopia Monitoring Probe... 22

2 1 What is Cloud Computing? NIST defines Cloud Computing as a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction 1. Cloud computing has five essential characteristics: On-demand self-service: A consumer can provision computing capabilities as needed automatically without requiring interactions with the service provider Broad network access: Computing capabilities are available over the network Resource pooling: The provider s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. Rapid elasticity: Capabilities can be elastically provisioned and released, in some cases automatically, to satisfy consumer demand. Measured service: Resource usage can be monitored providing transparency for both the provider and consumer of the utilized service. Cloud computing service models 2 : Software as a service (SaaS): Cloud-based applications or software as a service (SaaS) run on distant computers in the cloud that are owned and operated by others and that connect to users computers via the Internet and, usually, a web browser. Platform as a service (PaaS): Platform as a service provides a cloud-based environment with everything required to support the complete lifecycle of building and delivering web-based (cloud) applications without the cost and complexity of buying and managing the underlying hardware, software, provisioning and hosting. Infrastructure as a service (IaaS): Infrastructure as a service provides companies with computing resources including servers, networking, storage, and data center space on a pay-per-use basis. Although Cloud Computing is currently being widely used, interoperability and portability are major barriers to broader adoption 3. With the continuously growing number of Cloud service providers 2 Cloud Infrastructures 2.1 OpenStack Many companies use OpenStack as the basis for their public Clouds. Rackspace, being one of them, has proven that OpenStack can power a geographically distributed, massive scale public Cloud. HP is another

3 company that uses the OpenStack technology for its open, enterprise-grade public Cloud named Helion. Furthermore, OpenStack is used by many enterprises for building their own private Clouds. So, what exactly is OpenStack? OpenStack 4 is an open source Cloud operating system that controls large pools of compute, storage and networking resources throughout a datacenter. These resources can be managed through a dashboard or via the OpenStack API. See Figure 1. Compute Figure 1 The OpenStack cloud operating system enables users to allocate on-demand computing resources, by provisioning and managing large networks of virtual machines. The compute architecture is designed to scale horizontally on standard hardware, enabling scaling compute up and down to meet demand for web resources and applications. Storage In addition to traditional enterprise-class storage technology, many organizations now have a variety of storage needs with varying performance and price requirements. OpenStack has support for both Object Storage and Block Storage, with many deployment options for each depending on the use case. Object Storage is ideal for cost effective, scale-out storage. It is not a traditional file system, but rather a distributed storage system for static data such as virtual machine images, photo storage, storage, backups and archives. Block Storage allows block devices to be exposed and connected to compute instances for expanded storage, better performance and integration with enterprise storage platforms Networking 4

4 OpenStack Networking manages IP addresses, allowing for dedicated static IPs or DHCP. OpenStack provides flexible networking models to suit the needs of different applications or user groups. Standard models include flat networks or VLANs for separation of servers and traffic. Users can also create their own networks, control traffic and connect servers and devices to one or more networks. OpenStack Networking ensures the network will not be the bottleneck or limiting factor in a cloud deployment and gives users real self-service, even over their network configurations. 2.2 Amazon EC2 Amazon Elastic Compute Cloud (Amazon EC2 5 ) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers. To use Amazon EC2, you simply: Select a pre-configured, templated Amazon Machine Image (AMI) to get up and running immediately. Or create an AMI containing your applications, libraries, data, and associated configuration settings. Configure security and network access on your Amazon EC2 instance. Choose which instance type(s) you want, then start, terminate, and monitor as many instances of your AMI as needed. Determine whether you want to run in multiple locations, utilize static IP endpoints, or attach persistent block storage to your instances. Amazon Machine Image (AMI) An Amazon Machine Image (AMI) provides the information required to launch an instance, which is a virtual server in the cloud. You specify an AMI when you launch an instance, and you can launch as many instances from the AMI as you need. You can also launch instances from as many different AMIs as you need. An AMI includes the following: A template for the root volume for the instance (for example, an operating system, an application server, and applications) Launch permissions that control which AWS accounts can use the AMI to launch instances A block device mapping that specifies the volumes to attach to the instance when it's launched Instance Type Amazon EC2 provides a wide selection of instance types optimized to fit different use cases. Instance types comprise varying combinations of CPU, memory, storage, and networking capacity and give you the flexibility to choose the appropriate mix of resources for your applications. Each instance type 5

5 includes one or more instance sizes, allowing you to scale your resources to the requirements of your target workload. 3 CAMF 3.1 Cloud Application Management Frameworks Cloud Application Management Frameworks are integrated products that provide for the management of Cloud applications. Such products incorporate self-service interfaces through which developers can configure the compute, storage and network resources to satisfy applications requirements overs its hosting environment. Furthermore developers can view the providers service catalogs, and select from the available offerings according to their needs. Cloud Application Management Frameworks may also support specification of elasticity policies so that deployments can scale at runtime. They might also provide application performance monitoring and analytics. 3.2 CAMF Framework for Eclipse CAMF is an open-source Cloud Application Management Framework through which users are able to define the description, deployment and management phases of their Cloud applications in a clean and intuitive graphical manner. It is built on top of the well-established Eclipse platform (Section 3.2.1) and it adheres to two highly desirable features of Cloud applications: portability and elasticity. In particular, CAMF implements the open, non-proprietary OASIS TOSCA specification (Section 3.2.2) for describing the provision, deployment and re-contextualization of applications across different Cloud infrastructures, thereby ensuring application portability. Furthermore, CAMF enables Cloud users to specify elasticity policies that describe how the deployed virtualized resources must be elastically adapted at runtime to match the needs of a dynamic application-workload Eclipse Eclipse 6 is an Integrated Development Environment (IDE) which uses plug-ins to provide all the functionality within and on top of the runtime system, following the OSGi core framework specification. Tool builders can contribute to the Eclipse platform by wrapping their tools in Eclipse plug-ins. A plug-in in Eclipse is a component that provides a certain type of service within the context of the Eclipse workbench. By a component here we mean an object that may be configured into a system at system deployment time. The Eclipse runtime provides an infrastructure to support the activation and operation of a set of plug-ins working together to provide a seamless environment for development activities TOSCA Specification for Cloud Applications TOSCA 7 provides a language to describe the structure of applications, together with their management operations. The structure of an application defines the components an application consists of and the relationships between them. Application components are described in TOSCA by means of Nodes, and

6 are used to represent specific types of VMs in a deployment (i.e. an application component can be a Tomcat application server in a 3-tier application). Each Node can have certain semantics that are defined in the corresponding Node Type. Such semantics include the Requirements a Node has against its hosting environment, the Capabilities it offers and the Policies that govern its execution, such as security or elasticity policies. Similarly, TOSCA Relationships, used to represent the relations in an application's structure, have their own semantics based on their Relationship Type (i.e. the semantics of a relationship might denote that there is some kind of dependency between the connected Nodes). The management aspects of an application are described in TOSCA by means of lifecycle operations or by more complex Management Plans. For each Node Type, its lifecycle operations can be defined, (i.e deploy or start an instance of the corresponding Node Type). On the other hand, there is no TOSCA specific way to describe Management Plans. Instead, they can be specified in any existing process modeling language, such as BPMN, and referenced through TOSCA. Both lifecycle operations and Managements Plans require some content to be realized, such as virtual machine images, configuration files etc. These contents are referred to as Artifacts. Figure 2 The application description together with the actual files required to realize the application's management operations that are specified in the description, are packaged into a single fully selfcontained archive, called CSAR (Cloud Service Archive). CSARs are portable across different Cloud vendors that implement a TOSCA supporting environment (also called TOSCA Container). 3.3 CAMF Requirements CAMF works with the latest versions of Eclipse IDE (Kepler or Luna). You can download the Eclipse IDE from CAMF Installation Instructions 1) Import -> Git -> Projects from Git -> Clone URI (URI: ) 2) Check-out branch ceclipse_aws_openstack_icwe 3) Open terminal and go under "<your git dir>/c-eclipse/app-submission/org.apache.jclouds" 4) Run "mvn clean:clean"

7 5) Run "mvn p2:site" - This will build a local p2-repository from which Eclipse will fetch all the JClouds libs needed to interface with openstack. At the end you should get an error with Maven exiting and complaining about ${jclouds.version}. This is expected (caused by wrong jclouds references in the Neutron POM) and is fixed by custom-build files I provide in the step below. 6) Download the corrected openstack-neutron JAR and POM files from here: (it s safe from my Dropbox account) 7) Copy the two files under folder ~/.m2/repository/org/apache/jclouds/labs/openstack-neutron/ ) Run mvn p2:site again under the directory specified above. You should not get any errors now, and the mvn p2 process should report: "[INFO] BUILD SUCCESS". 9) Run "mvn jetty:run" - This starts a local web server on port 8080 that is used to serve the created p2- repository. This is a thread so don't terminate it until we complete the next steps. You can check that everything is running normally by browsing: 10) Go to eclipse, expand plugin org.eclipse.camf.tycho.target and open file targetdefinition.target.xml 11) Click on one of the locations, click Update and wait for the process to finish. this might take a while, depending on the network speed and load of remote Eclipse repositories (watch the Progress indicator at lower-right corner). 12) When update completes, click Set as Target (upper right corner). Allow to complete again, save and close the target definition editor. 13) Right-click on "org.eclipse.camf.tycho" -> Maven -> Update project - > Select all -> Apply 14) Right-click on "org.eclipse.camf.tycho" -> Run As -> Maven install 15) If you get the Runtime configuration dialog, then add in the Goals field the following (without quotes): "clean install" 16) When everything completes, look in the Console View and make sure you have no error and you see a SUCCESS message. 17) Now you can terminate the jetty process on step 9. 18) Right-click on "org.eclipse.camf.tycho" -> Run As -> Run Configuration 19) New Eclipse Application 20) Give a name to your application and then go to the "Plug-ins" tab. 21) Change Launch With: "features selected below" and click Select All 22) Select All -> Apply -> Run

8 3.5 Getting Started Open CAMF Perspective The CAMF Perspective contains the Views and Editors utilized by CAMF. Open the CAMF Perspective: Window -> Open Perspective -> Other -> CAMF. (Figure 3) Figure 3 You can close the Cloud Information Tool view, since we are not going to need it for the current tutorial Create a new Cloud Project Right Click in the Cloud Project View -> New -> Other -> Cloud Application Management Framework -> Cloud Project. (Figure 4)

9 Figure 4 Next -> Give a name for the newly created project -> Next -> Select a Cloud Provider to associate the project with (Figure 5) -> Finish (if you have not already added providers to your workspace you can do so by clicking on the Edit Cloud Providers button and following the instructions in section 3.5.3). Figure 5

10 The new project is created and the project s folder structure is shown in the Cloud Project View (Figure 6). The created folders will be used as follows: Application Descriptions: holding topology blueprints defined using TOSCA. Each blueprint can be unique by specifying a different structure and management operations for the application at hand. Application Deployments: holding IaaS-flavored blueprints along with important operational records (past and current) regarding different deployment. Amongst other, these can include date/time, the target deployment IaaS, version of the application description, operational costs, etc. Artifacts: holding concrete software implementations required for the successful deployment and correct operation of the application. These include, and not limited to: executable and/or 3rd party libraries, custom virtual machine images, Chef cookbooks, O/S-specific configuration scripts, etc. The Cloud project structure, allows for artifacts to be referenced by multiple descriptions thus avoiding unwanted duplication. Monitoring: holding monitoring metric collectors prepared by the developer. These probes can be placed anywhere on the application stack ranging from the virtualization layer upwards to the application itself, in order to obtain meaningful metrics that report the runtime health and performance of the application. Figure Manage Cloud Providers Add Cloud Provider If the Cloud Providers wizard is not open go to Window -> Preferences -> Cloud Application Management Framework -> Cloud Providers Add -> OpenStack Cloud Provider (Figure 7) -> Next: Give a name for the OpenStack compliant provider to which you have already been registered and have acquired the necessary credentials (username and password). Fill in the Access Id (username) and Endpoint (URI) and click Finish -> OK. Here you can specify as many providers as you want. The selection of a provider to submit an application for deployment is done at a later stage.

11 Figure Edit Cloud Provider If the Cloud Providers wizard is not open go to Window -> Preferences -> Cloud Application Management Framework -> Cloud Providers Select the provider to be edited -> Edit -> OpenStack Cloud Provider -> Next: You can update the Access Id (username) and/or Endpoint (URI) (Figure 8). Click Finish -> OK.

12 Figure Cloud Application Description Create application description file Right Click on a Project in the Cloud Project View -> New -> Other -> Cloud Application Management Framework -> Application Description -> Next -> Give a name for the newly created description -> Finish. The application description file is created under the Application Descriptions folder with extension.tosca (Figure 9).

13 Figure Fetch Cloud provider info When creating the first application description in a project, the framework tries to fetch the following data from the specified OpenStack Cloud provider: Images: Virtual machine images that are made available through the OpenStack Image Service Networks: Virtual networks to enable compute servers to interact with each other and with the public network Key Pairs: User-created keys (i.e. rsa keys) that can be used to access the instances once they have been launched Flavors: A flavor represents a set of virtual resources. Flavors define how many virtual CPUs an instance has and the amount of RAM and size of its ephemeral disks. OpenStack provides a number of predefined flavors that you can edit or add to. Users must select from the set of available flavors defined on their cloud. A pop up window asks if you want to fetch this data (Figure 10). Figure 10 Click Yes and add the missing credentials (password) for the selected provider -> Finish (Figure 11).

14 Figure Check authentication details Once the user has provided her full credentials (username and password) for the specified endpoint and has been authenticated by the respective Cloud provider, her authentication details are shown in the Authentication Token UI view (Figure 12). Figure Describe Cloud application structure/contextualization Right click on the application description file -> Open With -> Tosca Diagram Editor. You can also open the xml TOSCA file at any time, by selecting Open With -> Text Editor. All the information required for the description can be found in the Palette (Figure 13) and/or in the Properties view (Figure 14). The Palette includes some generic elements (application components and relationships), as well as Cloud provider specific elements previously fetched (images, networks and key pairs). Elements from the Palette can be simply dragged and dropped onto the center Canvas of the tool.

15 Figure 13 The rest of the information (flavors) can be found in the Properties view (Figure 14). Figure 14 Users can also use image and configuration files stored in their local file system, simply by importing them into the respective folders of a Cloud Project (Virtual Machine Images, Deployment Scripts folders). Once the files are imported into the Cloud Project, they are also added in the Palette. Right Click -> Import -> General -> File System -> Next -> Browse -> Select a directory to import from. (Figure 15)

16 Figure 15 Firstly decide the number of application components to put in the description. Each application component corresponds to a virtual machine instance. Then, for each application component the image (green box), flavor and network(pink box) must be specified. Additional attributes can be specified, such as key pair (yellow box), deployment/configuration scripts (beige box), and initial number of instances (Figure 16). Figure 16

17 Specify application s elasticity requirements You can specify elasticity policies for your application so that it can scale at runtime based on the defined policies. There are two different types of elasticity policies: Elasticity Constraint: Used to express the constraints of an application, related to cost, performance and other application-quality metrics. Here the application user does not specify the exact actions to be enforced when a constraint is violated. Instead, the appropriate actions are determined by the underlying intelligent elastic Resource Provisioning System 8. Elasticity Strategy: Used to express specific strategies that should be enforced by the execution environment when specific constraints are violated. The policy presented in Figure 17 describes an elasticity constraint where the application provider wants the CPU usage of a database cluster to be less than 80%. In Figure 17 the application provider uses an elasticity strategy to specify that when the CPU usage of the cluster exceeds 80%, a new database VM should be added to the cluster. Figure 17 In order for the user to specify elasticity policies related to a monitoring metric, a corresponding monitoring probe (i.e. CPU Usage probe) from the Palette (Figure 18) must be assigned to the respective component by dragging the probe from the Palette and dropping it to the component. 8

18 Figure Submit application description to Cloud provider Once the application description is completed Right Click on the.tosca file under the Application Descriptions folder -> Application Submission (Figure 19). Give a name for the submission file (Figure 20). Figure 19

19 Select the provider to submit the file to (Figure 21). Figure 20 Figure 21 The submission file is created under the Application Submission folder (Figure 22).

20 Figure Deploy application to Cloud provider Create deployment file What remains is to send the deployment request to Cloud provider of your choice. Right Click on the.tosca file under the Application Submissions folder -> Application Deployment (Figure 23). Figure 23 Select the Cloud provider over which you wish to deploy your application -> Finish (Figure 24).

21 Figure Watch deployment status After deployment, the application developer can interact with the Deployment view (Figure 25), so as to instantly retrieve the applications operational status without leaving CAMF s workspace. The Deployment view provides a snapshot of all application deployments grouped per target IaaS. Each deployment is accompanied with provider-specific properties such as IP addresses of each component, instance IDs, uptime and cost information, if available. Figure Monitoring CAMF enables integration with different monitoring systems, enabling its users to collect performance metrics regarding any deployed application without being required to deal with external software environments.

22 Currently, CAMF is fully integrated with JCatascopia 9, an automated, multi-layer interoperable monitoring system for elastic Cloud environments. Besides the standard probe suite available in JCatascopia, CAMF allows application developers to define and implement custom metric collectors that adhere to JCatascopia s modular architecture. These probes can be placed anywhere on the application stack ranging from the virtualization layer upwards to the application itself, in order to obtain meaningful metrics that report the runtime health and performance of the application Create JCatascopia Monitoring Probe Users can write custom java monitoring probes in CAMF. Click on the Monitoring tab in the Properties view -> Create (Figure 26). Figure 26 A java class that extends the JCatascopia Probe interface is automatically generated (Figure 27), guiding the user on how to implement the required methods. The created probes are exported as.jar files and are packaged in the CSAR file together with the application deployment file. At deployment time, the exported.jar files are installed on the specified VMs, just like the rest of the JCatascopia probes. 9

23 Figure 27