Automatic, multi- grained elasticity- provisioning for the Cloud
|
|
- Gwen Scott
- 8 years ago
- Views:
Transcription
1 Automatic, multi- grained elasticity- provisioning for the Cloud Integration Prototype "CELAR System Prototype" V1 Companion Document Date: CELAR is funded by the European Commission DG- INFSO Seventh Framework Programmed, Contract no.:
2 Table of Contents 1 Introduction Integration map Components of the System Prototype V CELAR Client (c- Eclipse) CELAR Server CELAR Orchestrator and Provisioner Orchestrator Decision Module Multi- Level Metrics Evaluation Module Monitoring server and agents IaaS Cloud connectors for the Provisioner Orchestrator Interdependencies between Platform components for System Prototype V Source Code, Integration Platform and Process Packaging and Delivery CELAR Client CELAR Server Application Orchestrator VM Decision module Multi- level Metrics Evaluation module Application VMs System level testing Choice of the test application Tests Feedback Summary References List of Figures Figure 1: CELAR System Architecture V Figure 2: rsybl prototype structure... 8 Figure 3: MELA prototype structure... 9 Figure 4 CELAR Platform Deployment List of Tables Table 1 CELAR Orchestrator API / 23
3 Deliverable Title Integration Prototype "CELAR System Prototype" V1 Filename CELAR_D6.1_draft_v.X.doc Author(s) of companion document Konstantin Skaburskas, Dimitrios Tsoumakos, Nikos Papailiou, Ioannis Giannakopoulos, Christos Mantas, Craig Sheridan, Le Duc Hung, Stalo Sofokleous, Antonis Lioumis, Ioannis Mitsos Date Start of the project: Duration: 36 Project coordinator organisation: ATHENA RESEARCH AND INNOVATION CENTER IN INFORMATION COMMUNICATION & KNOWLEDGE TECHNOLOGIES (ATHENA) Deliverable title: Deliverable no.: Integration Prototype "CELAR System Prototype" V1 D6.1 Due date of deliverable: Actual submission date: 30 September September 2013 Dissemination Level X PU Public PP Restricted to other programme participants (including the Commission Services) RE Restricted to a group specified by the consortium (including the Commission Services) CO Confidential, only for members of the consortium (including the Commission Services) Deliverable status version control Version Date Author Konstantin Skaburskas Konstantin Skaburskas Dimitrios Tsoumakos Craig Sheridan Stalo Sofokleous Le Duc Hung Vangelis Floros Konstantin Skaburskas 3 / 23
4 Abstract Initial prototype of the CELAR System. This document represents complementary material to the released code to shortly describe the modules and their interaction. Keywords CELAR, integrated system V1.0, Elasticity provisioning, monitoring, application description, CELAR Server, CELAR Orchestrator, Decision- Making module 4 / 23
5 1 Introduction CELAR Integration Subprojects (ISPs) foster systematic evolution of project results towards the targeted CELAR platform and applications, ensuring horizontal integration of the RTD contributions of the project. ISPs are timed so as to follow key research contribution and reporting periods. The results of each subproject are then integrated into a prototype that showcases the contributions achieved during the subproject. WP6 s main activities produce the CELAR integrated system. In particular, WP6 ensures smooth integration of the input from the technical WPs (WP2 to WP5) into an operational system. This document describes D6.1, namely the first CELAR prototype that integrates technical contributions of the first year of the project. It describes the modules, their functionality, interactions, as well as the system setup and use. 2 Integration map ISP- 1 integrates and demonstrates Year 1 contributions. Main achievements during this period consist of the first versions of the Application Description and Cloud Monitoring tools, as well as Version 1.0 of the System Architecture and of the Elasticity Provisioning Platform. In D1.1, the consortium presented the first version of the CELAR System Architecture. CELAR enhances the functionality provided by current cloud infrastructures in order to provide automated, multi- grained, elastic resource provisioning for cloud- based applications. CELAR contributes methods and modules along the three following areas: 1. Application Management: Enable intelligent, and user- driven description and deployment of applications provided under the c- Eclipse framework. 2. Cloud Information and Performance Monitoring: Scalable, lightweight and customizable monitoring system that allows the collection and storage of statistics that come from the infrastructure, platform and the application layers. 3. Elasticity Platform: It provides automated resource allocation based on the application characteristics, the user- defined optimization and the incoming load. The elasticity platform is also responsible for the orchestration of cloud resources in the applications as well as methods for ensuring robustness and availability of the elastic operations w.r.t. the managed applications. This architecture along with the modules CELAR will develop in each area is depicted in Figure 1. 5 / 23
6 Application Management Platform Application Description Application Submission Information System Cloud Information and Performance Monitor Interceptor Monitoring System Multi- level Metrics Evaluation SaaS/ Pass IaaS Physical Layer Custom Applications Services VM VM Custom App 1 HBase Cassandra Cluster VM VM Custom App 2 Hive VM Hadoop Custom App 3 Other VS Storage VS Elasticity Platform Application Orchestration Resource Provisioner Cloud Orchestration Application Profiler Decision Module CELAR Manager CELAR DataBase Figure 1: CELAR System Architecture V1 Cloud Provider During the first year, substantial effort has been devoted for setting a visionary yet realistic plan on the overall CELAR system. The overall System Architecture (D1.1), Elasticity Platform Architecture (D3.1) and the Decision Process for On- Demand Elasticity (D5.1) have been delivered. Relative to individual modules, the first versions of the Application Description tool (D2.1) and Cloud Monitoring tool (D4.1) have been produced. In parallel to the aforementioned efforts, the consortium has progressed with the implementation of basic system components as well as their integration. A list of the implemented subsystems/modules as well as their functionality and features follows. The project has chosen an iterative approach to the development, integration and release process. One month long development cycles with regular checkpoints providing constant feedback to the development process were identified and followed. 2.1 Components of the System Prototype V1 The following components with the presented level of implementation constitute the first year System Prototype V CELAR Client (c- Eclipse) The c- Eclipse Application Management Platform functions as the interface of end- users with the CELAR system, and also serves as that information source through which application structure, elasticity requirements and deployment information are made available to the CELAR Orchestrator and Decision modules. c- Eclipse, contains the necessary graphical tools that enable Application Users to describe the application topology and model elasticity constraints and strategies, submit the application to the 6 / 23
7 underlying Cloud platform for deployment and execution, and finally monitor its complete life cycle. The c- Eclipse encompassed in the System Prototype V1 consists of the following tools: Application Description Tool: This tool interacts with the Application User who gives a description of the application that is to be deployed on the Cloud. The application description is translated into a formal XML specification that is passed through the Application Submission Tool to the CELAR Server. Application Submission Tool: This tool passes to the CELAR Server the application description, given by the Application User, enriched with some deployment information. The CELAR Server will then propagate the application description to the interested CELAR System modules. Monitoring Visualization Tool: This tool receives monitoring data from the CELAR DataBase about resource related metrics (CPU, memory usage, etc.) and application performance metrics and presents them graphically to the Application User CELAR Server The CELAR Server contains the CELAR Manager and Applications Provisioner. CELAR Manager is a web server that acts as a mediator between the different components of the CELAR Platform. This is where CELAR Client connects to request a deployment of a certain application, Monitoring module pushes aggregate metrics and Decision making module connects to discover the structure and historical data of the applications. For this purpose, the CELAR Manager runs a database to persist such information. Application Deployment Provisioner service is based on SlipStream [12], which is an open source multi- cloud coordinated provisioning and image factory engine. This service is responsible for the deployment of the applications under question on IaaS Cloud platforms. It requests instantiation of the virtual machines of the required configurations and orchestrates the deployment of the distributed multi- layered applications themselves. The service will be extended to support the acceptance and enforcement of the required application run- time elasticity actions on Cloud IaaS and application levels CELAR Orchestrator and Provisioner Orchestrator Those two components are responsible for actual materialization of the application deployment actions and are intended to be run in a special VM deployed by Provisioner at the initial phase of the deployment process. CELAR Orchestrator accepts scalability requests from Decision module and translates them to the ones understandable by the Provisioner. Provisioner Orchestrator is the SlipStream component whose task is to deploy the requested application, track it and apply the requested cloud IaaS level scalability actions. 7 / 23
8 2.1.4 Decision Module The current implementation of Decision Module is rsybl (runtime- SYBL) [6]. The main features of rsybl are evaluating application user s elasticity requirements in terms of cost, quality and resources, learning action effects and generating action plans. rsybl is composed of four core components: the SYBL Processing Engine, the Learning Engine, the Analysis Engine and the Planning Engine, and two components for interacting with the CELAR modules: the Data Processing Unit and the Reporting and Cloud Interaction Unit. For more details about these components, please refer to [D5.1]. A short description of the modules is presented in what follows The SYBL Processing Engine: Processes SYBL specifications and enforces described strategies when no planning is needed (e.g., if then strategy). The Learning Engine: Uses information from Data Processing Unit to model application behavior. The Analysis Engine: Determines elasticity requirements violations and traces the source of violated constraints to support action plan generation. The Planning Engine: Generates action plans for enforcing more complex strategies (e.g. maximization of a certain metric) and fixing violated constraints. The Data Processing Unit: Processes data coming from other CELAR modules such as application structure, load hints, or monitoring information. The Reporting and Cloud Interaction Unit: Interacts with other modules for sending action enforcement plans and application analysis information. Figure 2 depicts the structure of rsybl prototype which is a multi- modules Maven project. When building the parent project, Maven will automatically build sub- modules and create the web service in the folder: rsybl- analysis- engine/target/. Figure 2: rsybl prototype structure Multi- Level Metrics Evaluation Module The implementation of Multi- Level Metrics Evaluation Module is MELA [6], a service for monitoring and analysis elasticity of cloud application. The main features of MELA are composing multiple low level monitor metrics to high level metrics and evaluating application composed cost. The current MELA prototype is structured in two major modules: MELA- Core and MELA- Celar, which are short described as following. 8 / 23
9 - - MELA- Core: contains the multi- level cost composition and monitoring data analysis functionality. This module is structured in to three sub- modules. o MELA- DataService: Gets monitoring information from JCatascopia. o MELA- AnalysisService: Retrieves data from MELA- DataService, aggregates and analyzes it, and performs multi- level evaluation in terms of cost, quality and resource usage. This module exposes a RESTful web service that provides the composed and analyzed metrics to other interested CELAR modules, especially Decision Module. o MELA- Common: Holds common data model used in communication with the MELA- AnalysisService and the MELA- DataService. MELA- Celar: contains sub- modules for interacting with other CELAR modules. o MELA- JCatascopia- Client: Allows MELA- DataService to collect monitoring data from JCatascopia. o MELA- RSYBL- Client: Allows rsybl can use MELA RESTful API. Figure 3 depicts the MELA prototype structure. Maven will automatically build the sub- modules when building the parent project. The built packages are put in the folder MELA- AnalysisService/target and MELA- DataService/target. Figure 3: MELA prototype structure Monitoring server and agents The System Prototype V1 will encompass and fully utilize the first release of the JCatascopia Monitoring system (MS). JCatascopia is a fully automated, scalable, real- time Cloud Monitoring System, implemented in Java, which aims at supporting automated multi- grained Cloud platforms which offer elastic resource provisioning for the deployed Cloud applications. JCatascopia can be utilized to collect monitoring metrics from multiple layers of the underlying infrastructure as well as performance metrics from the deployed applications, and subsequently distribute them to subscribed users and platform operators. JCatascopia provides self- adaptive mechanisms to minimize the intrusiveness of its presence, and enhanced filtering to reduce communication costs for metric distribution. JCatascopia s elastic properties are further enhanced with the ability 9 / 23
10 to dynamically add/remove monitoring instances located on multiple levels of the infrastructure without any human intervention or the need to restart the monitoring system. Furthermore, JCatascopia supports, via a subscription mechanism, aggregation and grouping of low- level metrics to generate high- level metrics at runtime. Particularly, the following features are available in the current version (v1.0) of JCatascopia MS: Monitoring Probes: low level metric collectors utilized to gather raw metrics and generate time- stamped monitoring events. Probes are utilized to collect metrics concerning resource allocation and usage from multiple levels of the Cloud infrastructure, such as at VM, Virtual Cluster and Cloud level. Probes are also used to collect application performance metrics from the application level. Probes are written in Java and can be implemented by adhering to a comprehensive API provided by JCatascopia. Monitoring Agents: entities responsible for collecting, processing and distributing monitoring information, originating from the Probes, to JCatascopia Monitoring Server. An Agent can be considered as the Probe Manager for a particular VM, since it is responsible for activating/deactivating Probes, configuring accordingly their parameters and upon user request pull different metrics. By allowing monitoring Probes to be dynamically deployed at runtime, the JCatascopia MS is flexible and extensible. Monitoring Server: entities responsible for managing MS Agents. The basic functionality of a Server consists of receiving monitoring metrics from Agents, processing and storing them to the Monitoring Database and further distributing the acquired metrics to any interested entities. XProbe: a system (background) process that serves as an interface between third- party applications and the local Monitoring Agent. It enables developers to instrument their applications to report metrics either periodically or upon availability to the locally deployed JCatascopia Monitoring Agent. In the case of System Prototype V1, YCSB clients are instrumented to report throughput and latency metrics to the MS via XProbe. Subscription Mechanism: built into the Server to allow Application Users and various CELAR modules (e.g. Decision Module) via a RESTful API to apply aggregation filters and grouping functions upon low- level monitoring metrics gathered by individual monitoring Agents to create new high- level metrics. Monitoring Database: a relational database for storing the current state of the MS. Specifically, the current state refers to metadata concerning the Agents currently distributing metrics to the Server and the latest value of these metrics. The MS Server after processing freshly received metrics from Agents deployed on application VMs, stores these metrics locally for each application to an Application Monitoring Repository, in order to be consumed mainly by the Application User(s) (using the c- Eclipse Monitoring Visualization Tool) and the Decision Module via pull requests. JCatascopia Web Service: allows external entities such as Application Users or CELAR modules (i.e. Decision Module, Multi- Level Metrics Evaluation Module, etc.) to communicate / interact with the MS. For instance, Application Users can access monitoring information through the c- Eclipse Monitoring Visualization Tool. 10 / 23
11 2.1.7 IaaS Cloud connectors for the Provisioner Orchestrator The ultimate goal of the project is to implement the elasticity platform on top of the two chosen IaaS Cloud infrastructures: ~Okeanos and Flexiant FCO. The IaaS level provisioning is done by a special component the Provisioner Orchestrator, instantiated through SlipStream. SlipStream requires the connectors to the respective clouds to be developed. ~Okeanos. ~okeanos service builds on top of Synnefo open source, cloud software, developed by GRNET. Synnefo [11] implements most of the OpenStack [9] API. Since initially SlipStream already had an OpenStack connector the development of the one for Synnefo was based on the latter. Flexiant Cloud Orchestrator (FCO). For the Y1 prototype, the ~Okeanos platform is used to deploy the demo application and showcase project results. Work has progressed in parallel to this work for integration with Flexiant s FCO: The project platform at Flexiant has already been designed, built, configured, details passed to the consortium and individual accounts have been registered. Work has begun to expose the IaaS level metrics needed for the JCatascopia monitoring probe which includes a custom script called node- tool that exposes metrics from a physical compute node. Plans are in place to integrate the Flexiant platform with SlipStream and a list of IaaS actions identified for a wrapper to be written to communicate with the Orchestration VM of SlipStream. This work will be showcased in the Y2 prototype. 2.2 Interdependencies between Platform components for System Prototype V1 As described in D1.1, the CELAR Platform consists of the Decision Module, responsible for taking the elastic decisions, the Monitoring System which monitors the performance and the health of the application VMs and the Orchestrator module which applies the decisions taken by the Decision Module. These components communicate with each other, in order to deploy, orchestrate and elastically scale the application, described by the user using the c- Eclipse tool. The communication, which is crucial to the performance of the Platform in total, will be achieved through a proper and generic enough protocol of communication. This section describes how the communication between these modules occurs. 11 / 23
12 CELAR Client Inform ation Tool UI Application Description Tool Monitoring Tool UI Application Submission Tool c - Eclipse Platform Cloud Provider CELAR Server CELAR Application Orchestrator (Application A) CELAR Manager CELAR Orchestrator Decision Module Provisioner Server CELAR DataBase Provisioner Orchestrator Monitoring Server Monitoring Agent Application A Monitoring Agent Monitoring Agent Application VM Application VM Application VM Figure 4 CELAR Platform Deployment The Decision Module, the Monitoring System and the Application Orchestrator modules are located in a dedicated VM (referenced as the Application Orchestrator VM ), allocated when the application is initially deployed in order to monitor the application and apply all the necessary decisions needed to elastically scale the application. The Cloud Orchestrator module is located into a static VM (referenced as CELAR Server ), which is responsible for the allocation of the Application Orchestrator VM and the application VMs. These modules interact with each other through an API, which is exported as a set of REST web services. The communication between the modules is depicted in Figure 4, along with the communication between the platform and the application VMs. Each of the aforementioned modules provides a set of web services which will ease the communication and the cooperation between them. For example, when an elastic decision must be taken, the Decision Module fetches the latest monitoring metrics gathered by the Decision Module, through an HTTP call and when the decision is taken it will be forwarded to the Application Orchestrator using a second call. The Application Orchestrator will then contact the cloud Orchestrator (located in the CELAR Server) and 12 / 23
13 the necessary resources will be allocated. This procedure will be repeated each time a new decision is taken. The use of RESTful API calls eases the development process (as changes on the exported web services can happen in parallel for different modules) and enables the use of more than one VM for the application orchestrator if needed (if the modules interact through HTTP protocol then they can also communicate when located in different VMs). The prospect of breaking the Application Orchestrator into more than one VM will be examined if the performance of the platform deteriorates because of the shared resources. A brief overview of the CELAR Orchestrator API (which includes both the Application and Cloud Orchestrator calls) is provided in the following table ( Table 1). Table 1 CELAR Orchestrator API 13 / 23
14 URI Service Name Parameters /iaas/resources Get Provided Resources type (images, VMs, disks etc.) Description Returns a list of the provided resources by the IaaS. /iass/quotas Get quotas userid Returns a list of the remaining quotas of the specified user. /iaas/actions Get Resizing Actions None Returns a list of the allowed resizing actions by the IaaS. /iaas/probes Get Monitoring Probes None Returns a list of the available monitoring probes. /metrics/insert Insert metrics to DB metric (JSON) Inserts a new metrics into the database. /metrics/get Get metrics from DB Deploymentid, metricid, timeframe /probes/upload Upload custom probe Probe app. components /deployment/deploy Deploy an application ApplicationId, deployment configuration /deployment/smartdeploy Deploy an application ApplicationId, policy using a smart deployment configuration /deployment/shutdown Shut down an application deployment Returns a metric from the database for a specific deployment, metric type and timeframe. Upload a custom monitoring probe to an application VM. Deploys an application. Deploys a new application using the Decision Module to decide about a proper deployment configuration. DeploymentId Terminates a running deployment, /deployment/resize Apply resizing action Deploymentid, Performs a resizing action. resizing action (JSON/XML) /deployment/getconf Get deployment configuration DeploymentId, timestamp Returns the configuration of a deployment. /action/getactionhistory Get action history Resizing action id, Returns the actions applied (max results, state for a deployment in the before action) past (for a specific time frame). 3 Source Code, Integration Platform and Process CELAR project uses git [9] as SCM and source code can be found on GitHub in the CELAR organization There are ten multi- module projects, which 14 / 23
15 correspond to the different components and modules which in turn constitute the CELAR Platform. module Decision- Making Module metrics- evaluation Cloud Cost Evaluation ms Cloud Monitoring System orchestrator CELAR Orchestrator, Provisioner Orchestrator server CELAR Manager, Provisioner Server, CELAR DataBase Eclipse c- Eclipse related components is CELAR Cloud Information System Interceptor profiler Application Profiler provisioning- platform CELAR Elasticity Provisioning Platform For the demonstration purposes of the usage of the Maven [4] for managing and building single- and multi- module projects the following example projects were created hellocelar CELAR example Maven multi- module project helloworld CELAR example Maven single- module project For automating the build, test and release process Jenkins Continuous Integration and Nexus (Maven) servers were deployed. Both servers were deployed on ~Okeanos Cloud infrastructure and Jenkins CI server is available at vm.okeanos.grnet.gr Nexus server is available at vm.okeanos.grnet.gr/nexus Maven was used for the projects definitions of the corresponding CELAR components and facilitation of the build process. This materializes in the set of POM (Project Object Model) [10] files accompanying each software project, which define the project dependencies, build, test and distribution processes and goals. Development, integration, tests and artifacts archiving pipeline was set up as follows: for each GitHub project a corresponding build job was created in Jenkins CI; the jobs were configured in a polling mode to monitor git commits in the project with the interval of 5 min; 15 / 23
16 after developer writes, commits and pushes the changes to GitHub, Jenkins discovers the change- set, checks out the project and builds it using Maven; all Maven projects execute test goal to execute unit- tests; the final build goal is deploy which uploads the generated artifacts to the project s Nexus repository. Binary artifacts are available under the following URLs in the project s Nexus server vm.okeanos.grnet.gr/nexus/content/repositories/snapshots/ Snapshots vm.okeanos.grnet.gr/nexus/content/repositories/releases/ Releases To facilitate the development and integration process the project partners set up a weekly Integration teleconference meeting during which the most important issues and showstoppers were brought in, discussed and followed by the respective action points. This proved to be very useful and effective practice allowing every partner to be up- to- date with the global development and integration process as well as having a possibility to directly influence it. 4 Packaging and Delivery 4.1 CELAR Client c- Eclipse is implemented on top of the Eclipse platform, and follows its plug- in based architecture. It is platform independent, since it can be run on any platform supported by Eclipse (Windows, Linux, Sun Solaris, Mac OS X and others). It can be run on the client side by importing the implementation java packages into an Eclipse workspace and running it as an Eclipse Application. 4.2 CELAR Server The CELAR Server (machine or VM) will contain static information regarding the platform and the IaaS. The term static here is used in order to show that the existence and the functionalities of its components will not be affected by deployed applications and it will not interact immediately with the application, as the rest of the modules will. The CELAR Server will fetch and provide IaaS specific information (e.g. user quotas, available resizing actions, resource granularity, etc.) needed by other modules and it will also operate as an endpoint the CELAR Client application. In contrast to the CELAR Application Orchestrator VM, the CELAR Server will be constantly available in order to provide information and allocate resources from the IaaS for the already deployed applications. More specifically, the CELAR Server will be responsible for the following actions: 1. The Server will store aggregated and historic metrics of the previously deployed applications, in order to provide them to the CELAR Client when needed. For example, if a CELAR user wants to view performance metrics of terminated deployments, the data will be fetched by the CELAR Server, as the CELAR Application Orchestrator VM will be terminated with the termination of the 16 / 23
17 deployment by the user. Along with these metrics the CELAR Server will store information about the taken decisions, the allocated resources and any other available detail regarding the behavior of the application and its scaling reactions. 2. It will store profiling data (when a profiling task is executed) and serve them to the Decision Module in order to create a knowledge basis. 3. It will be the contact point between a CELAR client (c- Eclipse) and the platform. The CELAR Server will also accept the application description and initialize the deployment of the application. 4. Finally, the CELAR Server will be responsible for the Cloud- level Orchestration, as it will interact with the IaaS and it will (de)allocate new resources for the deployed applications. 4.3 Application Orchestrator VM The Application Orchestrator VM will be initially allocated when a new application deployment request is received by the CELAR Server. This VM will host the main components of the CELAR platform: the Decision Module, the Monitoring System and the CELAR Orchestrator module. These modules will interact with each other and guarantee that the application achieves the desired performance according to the user provided policy. The CELAR Orchestrator will receive requests from the Decision Module in order to update the application s configuration and perform the elastic operations, decided by the Decision Module. These actions could involve the allocation (deallocation) of new resources (previously allocated resources) respectively, or updating the applications configuration (if the user has given hints on the application operation) in order to improve the application performance according to the provided policy. Apart from this, the CELAR Orchestrator will be responsible for providing all the needed information regarding the application s current state to the Decision Module and it will manage the database used for storing all data, before they get flushed to CELAR Server. The aforementioned database is a classic RDBMS (MySQL v5.5) because of the ability of such databases to execute fast enough complex queries (possibly involving many joins between different tables). As discussed in Section 2.2, each module will export a set of web services in order to communicate with the rest of the modules. These services run in a servlet container (Apache Tomcat v 7.0) and they are written in Java. For the communication between the Application Orchestrator and the CELAR Server (when a resizing action must be applied, for instance), Python scripts should be executed from the Web Services (in order to allocate new resources through SlipStream) and Bash scripts should run after the allocation into the newly created VMs in order to Orchestrate them and ensure that the resources are utilized by the application. 17 / 23
18 4.3.1 Decision module Decision module is the rsybl tool and needs to be configured before deploying. The configuration file can be found under rsybl- analysis- engine/src/main/resources/ config.properties. rsybl needs following configuration - The Monitoring configuration: specifies the Monitoring Service URL as MELA - RESTful API, which allows rsybl to connect to MELA. The Enforcement configuration: specifies the necessary information for connecting to the Orchestration Service (e.g., Orchestration Service URL). In rsybl, the Analysis Engine exposes a RESTful service and coordinates the rest of the modules. The RESTful Web Service exposed by rsybl tool is rsybl- analysis- engine SNAPSHOT. To run rsybl, rsybl- analysis- engine SNAPSHOT.war needs to be deployed on a web application container Multi- level Metrics Evaluation module Multi- level Metrics Evaluation module is MELA tool and needs to be configured before deploying. The configuration file can be found under MELA- AnalysisService/src/main/ resources/config/config.properties. The configurations are the MELA- DataService IP and port, and others parameters such as interval time for collecting and aggregating data. MELA and consists of two artifacts to be deployed: - The MELA- AnalysisService: exposes a RESTful service which provides composed and analyzed metrics for other services. - The MELA- DataService: a background service which retrieves monitoring data from JCataScopia. Currently, to run MELA, MELA- DataService SNAPSHOT.jar needs to be executed like normal jar. Afterwards, MELA- AnalysisService SNAPSHOT.war must be deployed on a web application server. After the successful deployment, we can access to AnalysisService SNAPSHOT/ to view the visual monitoring metrics corresponding with application topology. Concerning the Monitoring System, the JCatascopia MS Server and Web Service reside on the Orchestrator VM. Particularly, the MS Server is re- distributed as a runnable Java archive (JAR), which itself is packaged into a TAR archive. An installation shell script (Bash) is used to untar the archive, copy it into a predefined directory on the Orchestrators VM s file system and set the necessary permissions. Once the above process is successfully completed, the JAR is executed and registered as a Linux service (i.e. daemon) using native OS libraries (/etc/init.d). On the other hand, the JCatascopia Web Service is packaged and re- distributed as a WAR (Web Application Archive) file for easy and transparent deployment into the Apache Tomcat web server running on the VM. 4.4 Application VMs Concerning the Monitoring System, only JCatascopia Agents, and consequently probes embedded to these Agents, reside on the Application VMs. Similarly to above, MS Agents are packaged as a runnable Java archive (JAR), and re- distributed in a TAR archive. An installation shell script (Bash) is used to untar the archive, copy it into a predefined directory on the Application VM s file system and set the necessary permissions. Once 18 / 23
19 the above process is successfully completed, the JAR is executed and the respective Agent is registered as a Linux service (i.e. daemon) using native OS libraries (/etc/init.d). 5 System level testing To verify that the fully integrated chain of an application description, deployment, decision- making and auto- scaling process works as expected a set of system level tests were conducted. 5.1 Choice of the test application In order to test the first prototype of the CELAR platform we have chosen the Apache Cassandra [1] distributed NoSQL database. Cassandra is a distributed key- value store that bases its distribution on Amazon s Dynamo [2] and its data model on Google s BigTable [3]. Cassandra is a good candidate for the CELAR platform because it offers linear scalability to the number of nodes participating in the Cassandra cluster and thus can be dynamically scaled to accommodate the desired user load. Furthermore, Cassandra has no single point of failure due to its distributed peer- to- peer ring architecture. Exploiting its modular peer- to- peer architecture Cassandra provides a lot of tools that can be used to achieve horizontal scaling. Nodes can be added and removed from a live Cassandra cluster without worrying about data loss and service unavailability. Finally, Cassandra is the central component of CELAR's Cloud Gaming use case and therefore lessons learned from the testing of the first prototype can be applied in the upcoming integration of the Cloud Gaming application. To generate variable workloads for our Cassandra deployment we use the YCSB (Yahoo! Cloud Serving Benchmark) [4] workload generator that is widely used for benchmarking cloud- based NoSQL databases. The architecture of the test application can be seen in the Figure 3. We use YCSB to create and load large datasets in our test Cassandra clusters and then generate a mix of workloads that help us profile the performance of Cassandra under different usage senarios. Cassandra s profiling with YCSB is used to derive good indicators about when a Cassandra cluster is over or under utilized. 19 / 23
20 Cassandra Node YCSB Client Cassandra Node Cassandra Node YCSB Client Cassandra Node Figure 3: Test application Cassandra Node Cassandra Node 5.2 Tests As stated in the previous section we used Cassandra in order to test the integration of the CELAR platform components. Cassandra deployment At first we generated SlipStream deployment scripts that can be used to deploy a Cassandra cluster with variable number of nodes to a cloud provider. This procedure fostered the integration between the CELAR Server and the SlipStream resource provisioner modules and can help synthesize the final CELAR application description structure. We tested the deployment of the application with variable number of both YCSB clients and Cassandra nodes, on the ~Okeanos [5] cloud infrastructure. For the Cassandra cluster we used 1-9 VMs each consisting of 1 virtual CPU, 1GB of main memory and 10GB of hard disk. For the YCSB clients we used 1-3 VMs each consisting of 1 virtual CPU, 512MB of main memory and 5GB of hard disk. Cassandra load tests During our tests we also experimented using various YCSB datasets. We loaded from 1 to 20 million records in order to see the impact of the dataset size on Cassandra s performance. Concerning the YCSB workloads, we used 2 basic workload types: Cached reads: All YCSB clients request random keys from a small subset of the dataset. The total amount of keys that are requested can fit in Cassandra s cache and doesn t produce costly I/O operations. Non Cached reads: All YCSB clients randomly request keys from the whole dataset resulting in high I/O overhead and lower request throughput. 20 / 23
21 We also vary the amount of requests that are issued by the YCSB clients through time. We created sinusoidal and step based workloads that can be used to stress out our Cassandra cluster and trigger resizing actions. Integration with monitoring module In order to test Cassandra's scalability we generated resizing scripts that add and remove nodes from an existing Cassandra cluster. The procedure shows that those operations are feasible and can be performed in a live cluster without losing data or availability. We also confirmed our initial expectations for the linear relation between the number of Cassandra nodes and the observed request throughput. c- Eclipse c- Eclipse has been tested on different Windows and MAC OS X versions (Windows 7, Windows 8, latest MAC OS X stable version) to see whether it can be executed on these platforms without arising any errors or problems. The Application Description/Submission Tools have also been tested in terms of user friendliness and ease of use by asking Cloud users of multiple expertise levels to describe an application to be submitted to the Cloud for deployment. The XML outcome of the Application Description Tool has also been tested whether it is a valid XML file according to the TOSCA specification. The Monitoring Visualization Tool has been tested in terms of correctly receiving metrics from the Monitoring DB and displaying them graphically to the c- Eclipse Users. JCatascopia The behavior of v1.0 of the JCatascopia Monitoring System was tested on both of the consortium IaaS platforms (Okeanos Public Cloud and Flexiant FCO) and on the UCY Nephelae private Cloud infrastructure, utilizing different Linux OS distributions (Ubuntu, Debian, CentOS) and various VM flavors (e.g. different no. of VCPUs, RAM and disk size). Testing involved: (i) checking if JCatascopia Monitoring Probes where collecting monitoring metrics, originating from the system level (CPU, Memory, Network etc.) and the application level (latency, throughput from YCSB clients, etc.), correctly, (ii) checking if metrics where distributed by JCatascopia Agents to their corresponding Monitoring Server, (iii) checking if JCatascopia Monitoring Servers where receiving, processing, storing and distributing monitoring metrics to Application Users and CELAR interested modules which access the Monitoring System via the c- Eclipse Monitoring Visualization Tool and/or via the JCatascopia RESTful API. 5.3 Feedback After running the above tests using the first prototype of the CELAR platform we have tested and confirmed the capability of the CELAR platform to perform automated deployment and resizing actions to a cloud application. We also confirmed our expectations about application elasticity and the need to automate the decision- making process. c- Eclipse tools have run on the different testing platforms without any changes required in the implementation packages. The Application Description and Submission Tools were successfully used by the Cloud users and the outcome of the Application 21 / 23
22 Description Tool is a valid XML file that can be parsed and read by XML editors. Finally, the c- Eclipse Monitoring Visualization Tool receives accurately and in time the monitoring metrics from the Monitoring DB and displays them graphically to the Application Users. Testing is an ongoing process for the CELAR Monitoring System. The feedback obtained after concluding various preliminary tests involved: (i) when testing the Monitoring Probes, we experienced problems collecting various resource- related metrics (i.e. disk usage, disk IO, etc.) from VMs which were not evident when monitoring physical machines. (ii) In large distributed infrastructures faults and unexpected errors are bound to present. When network connectivity errors appeared we experienced various problems while Monitoring Agents where attempting to push metrics to their corresponding Server. Such problems resulted in metric delivery failures and the worst scenario was that Monitoring Agents were disconnected from their Monitoring Server. After experiencing the aforementioned problems we enhanced the CELAR Monitoring System with heartbeat monitoring in order to take into consideration connectivity errors, allowing for Agents to appear as DOWN for a short time interval and we also enhanced Monitoring Agents with a better distribution mechanism allowing for messages to be re- send if not delivered. 6 Summary For the purpose of the code development, build, test and distribution the project partners have set up and successfully used the respective processes and the purposefully built infrastructure. This allowed in a short time produce the required components and integrate them into a working prototype of the CELAR Platform V1. The next identified points of improvement will be the generation of OS level packages (rpm and deb) as well as defining and implementing deployment and delivery processes for the CELAR Platform components. 7 References [1] [2] DeCandia, Giuseppe, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, and Werner Vogels. "Dynamo: amazon's highly available key- value store." In SOSP, vol. 7, pp [3] Chang, Fay, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber. "Bigtable: A distributed storage system for structured data." ACM Transactions on Computer Systems (TOCS) 26, no. 2 (2008): 4. [4] [5] [6] [7] [8] scm.com/ [9] 22 / 23
23 [10] [11] [12] 23 / 23
Monitoring Elastic Cloud Services
Monitoring Elastic Cloud Services trihinas@cs.ucy.ac.cy Advanced School on Service Oriented Computing (SummerSoc 2014) 30 June 5 July, Hersonissos, Crete, Greece Presentation Outline Elasticity in Cloud
More informationAutomatic, multi- grained elasticity- provisioning for the Cloud. Deliverable no.: 2.1 Date: 29-07- 2013
Automatic, multi- grained elasticity- provisioning for the Cloud Application Description Tool V1 Deliverable no.: 2.1 Date: 29-07- 2013 CELAR is funded by the European Commission DG- INFSO Seventh Framework
More information1 What is Cloud Computing?... 2 2 Cloud Infrastructures... 2 2.1 OpenStack... 2 2.2 Amazon EC2... 4 3 CAMF... 5 3.1 Cloud Application Management
1 What is Cloud Computing?... 2 2 Cloud Infrastructures... 2 2.1 OpenStack... 2 2.2 Amazon EC2... 4 3 CAMF... 5 3.1 Cloud Application Management Frameworks... 5 3.2 CAMF Framework for Eclipse... 5 3.2.1
More informationAutomatic, multi-grained elasticity-provisioning for the Cloud
Automatic, multi-grained elasticity-provisioning for the Cloud Cloud Information System Deliverable no.: 4.3 May 28 th, 2015 CELAR is funded by the European Commission DG-INFSO Seventh Framework Programmed,
More informationAutomatic, multi-grained elasticity-provisioning for the Cloud
Automatic, multi-grained elasticity-provisioning for the Cloud Cloud Monitoring Tool V1 Deliverable no.: 4.1 September 30 th, 2013 CELAR is funded by the European Commission DG-INFSO Seventh Framework
More informationTowards Smart and Intelligent SDN Controller
Towards Smart and Intelligent SDN Controller - Through the Generic, Extensible, and Elastic Time Series Data Repository (TSDR) YuLing Chen, Dell Inc. Rajesh Narayanan, Dell Inc. Sharon Aicler, Cisco Systems
More informationAssignment # 1 (Cloud Computing Security)
Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual
More informationOn- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform
On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform Page 1 of 16 Table of Contents Table of Contents... 2 Introduction... 3 NoSQL Databases... 3 CumuLogic NoSQL Database Service...
More informationCloudCenter Full Lifecycle Management. An application-defined approach to deploying and managing applications in any datacenter or cloud environment
CloudCenter Full Lifecycle Management An application-defined approach to deploying and managing applications in any datacenter or cloud environment CloudCenter Full Lifecycle Management Page 2 Table of
More informationCloud Computing in Distributed System
M.H.Nerkar & Sonali Vijay Shinkar GCOE, Jalgaon Abstract - Cloud Computing as an Internet-based computing; where resources, software and information are provided to computers on-demand, like a public utility;
More informationD3.3.1: Sematic tagging and open data publication tools
COMPETITIVINESS AND INNOVATION FRAMEWORK PROGRAMME CIP-ICT-PSP-2013-7 Pilot Type B WP3 Service platform integration and deployment in cloud infrastructure D3.3.1: Sematic tagging and open data publication
More informationCloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH
Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH CONTENTS Introduction... 4 System Components... 4 OpenNebula Cloud Management Toolkit... 4 VMware
More informationDevelopment of nosql data storage for the ATLAS PanDA Monitoring System
Development of nosql data storage for the ATLAS PanDA Monitoring System M.Potekhin Brookhaven National Laboratory, Upton, NY11973, USA E-mail: potekhin@bnl.gov Abstract. For several years the PanDA Workload
More informationFuture Prospects of Scalable Cloud Computing
Future Prospects of Scalable Cloud Computing Keijo Heljanko Department of Information and Computer Science School of Science Aalto University keijo.heljanko@aalto.fi 7.3-2012 1/17 Future Cloud Topics Beyond
More informationBig Data and Hadoop with components like Flume, Pig, Hive and Jaql
Abstract- Today data is increasing in volume, variety and velocity. To manage this data, we have to use databases with massively parallel software running on tens, hundreds, or more than thousands of servers.
More informationTopology Aware Analytics for Elastic Cloud Services
Topology Aware Analytics for Elastic Cloud Services athafoud@cs.ucy.ac.cy Master Thesis Presentation May 28 th 2015, Department of Computer Science, University of Cyprus In Brief.. a Tool providing Performance
More informationBenchmarking Couchbase Server for Interactive Applications. By Alexey Diomin and Kirill Grigorchuk
Benchmarking Couchbase Server for Interactive Applications By Alexey Diomin and Kirill Grigorchuk Contents 1. Introduction... 3 2. A brief overview of Cassandra, MongoDB, and Couchbase... 3 3. Key criteria
More informationIntroduction to Cloud Computing
Introduction to Cloud Computing Cloud Computing I (intro) 15 319, spring 2010 2 nd Lecture, Jan 14 th Majd F. Sakr Lecture Motivation General overview on cloud computing What is cloud computing Services
More informationDynamo: Amazon s Highly Available Key-value Store
Dynamo: Amazon s Highly Available Key-value Store Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall and
More informationA Middleware Strategy to Survive Compute Peak Loads in Cloud
A Middleware Strategy to Survive Compute Peak Loads in Cloud Sasko Ristov Ss. Cyril and Methodius University Faculty of Information Sciences and Computer Engineering Skopje, Macedonia Email: sashko.ristov@finki.ukim.mk
More informationBig Data and Hadoop with Components like Flume, Pig, Hive and Jaql
Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 7, July 2014, pg.759
More informationPlatform as a Service and Container Clouds
John Rofrano Senior Technical Staff Member, Cloud Automation Services, IBM Research jjr12@nyu.edu or rofrano@us.ibm.com Platform as a Service and Container Clouds using IBM Bluemix and Docker for Cloud
More informationC2C: An Automated Deployment Framework for Distributed Applications on Multi-Clouds
C2C: An Automated Deployment Framework for Distributed Applications on Multi-Clouds Flora Karniavoura, Antonis Papaioannou, and Kostas Magoutis Institute of Computer Science (ICS) Foundation for Research
More informationJenkins World Tour 2015 Santa Clara, CA, September 2-3
1 Jenkins World Tour 2015 Santa Clara, CA, September 2-3 Continuous Delivery with Container Ecosystem CAD @ Platform Equinix - Overview CAD Current Industry - Opportunities Monolithic to Micro Service
More informationEWeb: Highly Scalable Client Transparent Fault Tolerant System for Cloud based Web Applications
ECE6102 Dependable Distribute Systems, Fall2010 EWeb: Highly Scalable Client Transparent Fault Tolerant System for Cloud based Web Applications Deepal Jayasinghe, Hyojun Kim, Mohammad M. Hossain, Ali Payani
More informationFine-Grained Elasticity Support for Cloud Applications: The CELAR Approach
Fine-Grained Elasticity Support for Cloud Applications: The CELAR Approach (Marios Dikaiakos) University of Cyprus Hong Kong University of Science and Technology, May 8, 2015 Cloud Computing Ubiquitous,
More informationA Comparison of Clouds: Amazon Web Services, Windows Azure, Google Cloud Platform, VMWare and Others (Fall 2012)
1. Computation Amazon Web Services Amazon Elastic Compute Cloud (Amazon EC2) provides basic computation service in AWS. It presents a virtual computing environment and enables resizable compute capacity.
More informationDeploying Business Virtual Appliances on Open Source Cloud Computing
International Journal of Computer Science and Telecommunications [Volume 3, Issue 4, April 2012] 26 ISSN 2047-3338 Deploying Business Virtual Appliances on Open Source Cloud Computing Tran Van Lang 1 and
More informationDelivering IaaS for the Greek Academic and Research Community
Greek Research and Technology Network Internal Presentation 59 Delivering IaaS for the Greek Academic and Research Community Vangelis Koukis vkoukis@grnet.gr ki Technical Coordinator, okeanos Project Greek
More informationScalable Multiple NameNodes Hadoop Cloud Storage System
Vol.8, No.1 (2015), pp.105-110 http://dx.doi.org/10.14257/ijdta.2015.8.1.12 Scalable Multiple NameNodes Hadoop Cloud Storage System Kun Bi 1 and Dezhi Han 1,2 1 College of Information Engineering, Shanghai
More informationHands-on Cassandra. OSCON July 20, 2010. Eric Evans eevans@rackspace.com @jericevans http://blog.sym-link.com
Hands-on Cassandra OSCON July 20, 2010 Eric Evans eevans@rackspace.com @jericevans http://blog.sym-link.com 2 Background Influential Papers BigTable Strong consistency Sparse map data model GFS, Chubby,
More informationTOSCA Interoperability Demonstration
Topology and Orchestration Specification for Cloud Applications (TOSCA) Standard TOSCA Interoperability Demonstration Participating Companies: Join the TOSCA Technical Committee www.oasis-open.org, join@oasis-open.org
More informationLARGE-SCALE DATA STORAGE APPLICATIONS
BENCHMARKING AVAILABILITY AND FAILOVER PERFORMANCE OF LARGE-SCALE DATA STORAGE APPLICATIONS Wei Sun and Alexander Pokluda December 2, 2013 Outline Goal and Motivation Overview of Cassandra and Voldemort
More informationHow Bigtop Leveraged Docker for Build Automation and One-Click Hadoop Provisioning
How Bigtop Leveraged Docker for Build Automation and One-Click Hadoop Provisioning Evans Ye Apache Big Data 2015 Budapest Who am I Apache Bigtop PMC member Software Engineer at Trend Micro Develop Big
More informationWEBAPP PATTERN FOR APACHE TOMCAT - USER GUIDE
WEBAPP PATTERN FOR APACHE TOMCAT - USER GUIDE Contents 1. Pattern Overview... 3 Features 3 Getting started with the Web Application Pattern... 3 Accepting the Web Application Pattern license agreement...
More informationD5.3.2b Automatic Rigorous Testing Components
ICT Seventh Framework Programme (ICT FP7) Grant Agreement No: 318497 Data Intensive Techniques to Boost the Real Time Performance of Global Agricultural Data Infrastructures D5.3.2b Automatic Rigorous
More informationFile S1: Supplementary Information of CloudDOE
File S1: Supplementary Information of CloudDOE Table of Contents 1. Prerequisites of CloudDOE... 2 2. An In-depth Discussion of Deploying a Hadoop Cloud... 2 Prerequisites of deployment... 2 Table S1.
More informationCollaborative Open Market to Place Objects at your Service
Collaborative Open Market to Place Objects at your Service D4.1.2 Basic implementation of the COMPOSE runtime infrastructure Project Acronym Project Title COMPOSE Project Number 317862 Work Package WP4
More informationA Survey Study on Monitoring Service for Grid
A Survey Study on Monitoring Service for Grid Erkang You erkyou@indiana.edu ABSTRACT Grid is a distributed system that integrates heterogeneous systems into a single transparent computer, aiming to provide
More informationD5.4.4 Integrated SemaGrow Stack API components
ICT Seventh Framework Programme (ICT FP7) Grant Agreement No: 318497 Data Intensive Techniques to Boost the Real Time Performance of Global Agricultural Data Infrastructures Deliverable Form Project Reference
More informationWhite Paper. Cloud Native Advantage: Multi-Tenant, Shared Container PaaS. http://wso2.com Version 1.1 (June 19, 2012)
Cloud Native Advantage: Multi-Tenant, Shared Container PaaS Version 1.1 (June 19, 2012) Table of Contents PaaS Container Partitioning Strategies... 03 Container Tenancy... 04 Multi-tenant Shared Container...
More informationAlfresco Enterprise on AWS: Reference Architecture
Alfresco Enterprise on AWS: Reference Architecture October 2013 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 13 Abstract Amazon Web Services (AWS)
More informationShoal: IaaS Cloud Cache Publisher
University of Victoria Faculty of Engineering Winter 2013 Work Term Report Shoal: IaaS Cloud Cache Publisher Department of Physics University of Victoria Victoria, BC Mike Chester V00711672 Work Term 3
More informationBENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB
BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next
More informationMobile Cloud Computing T-110.5121 Open Source IaaS
Mobile Cloud Computing T-110.5121 Open Source IaaS Tommi Mäkelä, Otaniemi Evolution Mainframe Centralized computation and storage, thin clients Dedicated hardware, software, experienced staff High capital
More informationCollaborative Open Market to Place Objects at your Service
Collaborative Open Market to Place Objects at your Service D6.4.1 Marketplace integration First version Project Acronym COMPOSE Project Title Project Number 317862 Work Package WP6 Open marketplace Lead
More informationSurvey of the Benchmark Systems and Testing Frameworks For Tachyon-Perf
Survey of the Benchmark Systems and Testing Frameworks For Tachyon-Perf Rong Gu,Qianhao Dong 2014/09/05 0. Introduction As we want to have a performance framework for Tachyon, we need to consider two aspects
More informationSTeP-IN SUMMIT 2013. June 18 21, 2013 at Bangalore, INDIA. Performance Testing of an IAAS Cloud Software (A CloudStack Use Case)
10 th International Conference on Software Testing June 18 21, 2013 at Bangalore, INDIA by Sowmya Krishnan, Senior Software QA Engineer, Citrix Copyright: STeP-IN Forum and Quality Solutions for Information
More informationTHE HADOOP DISTRIBUTED FILE SYSTEM
THE HADOOP DISTRIBUTED FILE SYSTEM Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler Presented by Alexander Pokluda October 7, 2013 Outline Motivation and Overview of Hadoop Architecture,
More informationLiferay Portal Performance. Benchmark Study of Liferay Portal Enterprise Edition
Liferay Portal Performance Benchmark Study of Liferay Portal Enterprise Edition Table of Contents Executive Summary... 3 Test Scenarios... 4 Benchmark Configuration and Methodology... 5 Environment Configuration...
More informationOpen Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)
Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011 Open Cloud System University
More informationCloud Utilization for Online Price Intelligence
Lohnt sich Cloud Computing? Anwendungsbeispiele aus der Praxis Cloud Utilization for Online Price Intelligence 22.6.2010 OCG Competence Circle About Lixto Lixto extracts specific and precise data from
More informationCloud Application Development (SE808, School of Software, Sun Yat-Sen University) Yabo (Arber) Xu
Lecture 4 Introduction to Hadoop & GAE Cloud Application Development (SE808, School of Software, Sun Yat-Sen University) Yabo (Arber) Xu Outline Introduction to Hadoop The Hadoop ecosystem Related projects
More informationBIG DATA WEB ORGINATED TECHNOLOGY MEETS TELEVISION BHAVAN GHANDI, ADVANCED RESEARCH ENGINEER SANJEEV MISHRA, DISTINGUISHED ADVANCED RESEARCH ENGINEER
BIG DATA WEB ORGINATED TECHNOLOGY MEETS TELEVISION BHAVAN GHANDI, ADVANCED RESEARCH ENGINEER SANJEEV MISHRA, DISTINGUISHED ADVANCED RESEARCH ENGINEER TABLE OF CONTENTS INTRODUCTION WHAT IS BIG DATA?...
More informationSisense. Product Highlights. www.sisense.com
Sisense Product Highlights Introduction Sisense is a business intelligence solution that simplifies analytics for complex data by offering an end-to-end platform that lets users easily prepare and analyze
More information<Insert Picture Here> Private Cloud with Fusion Middleware
Private Cloud with Fusion Middleware Duško Vukmanović Principal Sales Consultant, Oracle dusko.vukmanovic@oracle.com The following is intended to outline our general product direction.
More informationEvaluation Methodology of Converged Cloud Environments
Krzysztof Zieliński Marcin Jarząb Sławomir Zieliński Karol Grzegorczyk Maciej Malawski Mariusz Zyśk Evaluation Methodology of Converged Cloud Environments Cloud Computing Cloud Computing enables convenient,
More informationGRNET Cloud Compu7ng Services An Overview
GRNET Cloud Compu7ng Services An Overview Panos Louridas louridas@grnet.gr GN3 Innova7on Workshop Copenhagen, October 10 11 2011 Greek Research and Technology Network 2 Outline u okeanos IaaS u pithos
More informationScalable Architecture on Amazon AWS Cloud
Scalable Architecture on Amazon AWS Cloud Kalpak Shah Founder & CEO, Clogeny Technologies kalpak@clogeny.com 1 * http://www.rightscale.com/products/cloud-computing-uses/scalable-website.php 2 Architect
More informationLiferay Portal s Document Library: Architectural Overview, Performance and Scalability
Liferay Portal s Document Library: Architectural Overview, Performance and Scalability Table of Contents EXECUTIVE SUMMARY... 1 HIGH LEVEL ARCHITECTURE... 2 User Interface Layer... 2 Service Layer....
More informationIntroduction to Big Data Training
Introduction to Big Data Training The quickest way to be introduce with NOSQL/BIG DATA offerings Learn and experience Big Data Solutions including Hadoop HDFS, Map Reduce, NoSQL DBs: Document Based DB
More informationCollaborative Open Market to Place Objects at your Service
Collaborative Open Market to Place Objects at your Service D6.2.1 Developer SDK First Version D6.2.2 Developer IDE First Version D6.3.1 Cross-platform GUI for end-user Fist Version Project Acronym Project
More informationEvaluating and Comparing the Impact of Software Faults on Web Servers
Evaluating and Comparing the Impact of Software Faults on Web Servers April 2010, João Durães, Henrique Madeira CISUC, Department of Informatics Engineering University of Coimbra {naaliel, jduraes, henrique}@dei.uc.pt
More informationManaging your Red Hat Enterprise Linux guests with RHN Satellite
Managing your Red Hat Enterprise Linux guests with RHN Satellite Matthew Davis, Level 1 Production Support Manager, Red Hat Brad Hinson, Sr. Support Engineer Lead System z, Red Hat Mark Spencer, Sr. Solutions
More informationEnterprise Service Bus
We tested: Talend ESB 5.2.1 Enterprise Service Bus Dr. Götz Güttich Talend Enterprise Service Bus 5.2.1 is an open source, modular solution that allows enterprises to integrate existing or new applications
More informationJoining Cassandra. Luiz Fernando M. Schlindwein Computer Science Department University of Crete Heraklion, Greece mattos@csd.uoc.
Luiz Fernando M. Schlindwein Computer Science Department University of Crete Heraklion, Greece mattos@csd.uoc.gr Joining Cassandra Binjiang Tao Computer Science Department University of Crete Heraklion,
More informationSeaClouds Project. Cloud Application Programming Interface. Seamless adaptive multi- cloud management of service- based applications
SeaClouds Project D4.2- Cloud Application Programming Interface Project Acronym Project Title Call identifier Grant agreement no. Start Date Ending Date Work Package Deliverable code Deliverable Title
More informationStratusphere Solutions
Stratusphere Solutions Deployment Best Practices Guide Introduction This guide has been authored by experts at Liquidware Labs in order to provide a baseline as well as recommendations for a best practices
More informationJAVA IN THE CLOUD PAAS PLATFORM IN COMPARISON
JAVA IN THE CLOUD PAAS PLATFORM IN COMPARISON Eberhard Wolff Architecture and Technology Manager adesso AG, Germany 12.10. Agenda A Few Words About Cloud Java and IaaS PaaS Platform as a Service Google
More informationFleSSR Project: Installing Eucalyptus Open Source Cloud Solution at Oxford e- Research Centre
FleSSR Project: Installing Eucalyptus Open Source Cloud Solution at Oxford e- Research Centre Matteo Turilli, David Wallom Eucalyptus is available in two versions: open source and enterprise. Within this
More informationWinery A Modeling Tool for TOSCA-based Cloud Applications
Institute of Architecture of Application Systems Winery A Modeling Tool for TOSCA-based Cloud Applications Oliver Kopp 1,2, Tobias Binz 2, Uwe Breitenbücher 2, and Frank Leymann 2 1 IPVS, 2 IAAS, University
More informationApache Stratos (incubating) 4.0.0-M5 Installation Guide
Apache Stratos (incubating) 4.0.0-M5 Installation Guide 1. Prerequisites 2. Product Configuration 2.1 Message Broker Configuration 2.2 Load Balancer Configuration 2.3 Cloud Controller Configuration 2.4
More informationApache Stratos Building a PaaS using OSGi and Equinox. Paul Fremantle CTO and Co- Founder, WSO2 CommiCer, Apache Stratos
Apache Stratos Building a PaaS using OSGi and Equinox Paul Fremantle CTO and Co- Founder, WSO2 CommiCer, Apache Stratos @pzfreo #wso2 #apache paul@wso2.com pzf@apache.org 1 About me CTO and Co- Founder
More informationACCELERATE DEVOPS USING OPENSHIFT PAAS
ACCELERATE DEVOPS USING OPENSHIFT PAAS September 3, 2014 AGENDA World we live in today IT organization: Charter, goals, and challenges DevOps: Problem statement, what, and why How to enable DevOps Application
More informationMr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo
Sensor Network Messaging Service Hive/Hadoop Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Contents 1 Introduction 2 What & Why Sensor Network
More informationAutomatic, multi-grained elasticity-provisioning for the Cloud. User Requirements and System Architecture V1. Deliverable no.: D1.
Automatic, multi-grained elasticity-provisioning for the Cloud User Requirements and System Architecture V1 Deliverable no.: D1.1 Date: 05-04-2013 CELAR is funded by the European Commission DG-INFSO Seventh
More informationwww.basho.com Technical Overview Simple, Scalable, Object Storage Software
www.basho.com Technical Overview Simple, Scalable, Object Storage Software Table of Contents Table of Contents... 1 Introduction & Overview... 1 Architecture... 2 How it Works... 2 APIs and Interfaces...
More informationGetting Started with Google Cloud Platform
Chapter 2 Getting Started with Google Cloud Platform Welcome to Google Cloud Platform! Cloud Platform is a set of modular cloud-based services that provide building blocks you can use to develop everything
More informationCloud Computing an introduction
Prof. Dr. Claudia Müller-Birn Institute for Computer Science, Networked Information Systems Cloud Computing an introduction January 30, 2012 Netzprogrammierung (Algorithmen und Programmierung V) Our topics
More informationSolution for private cloud computing
The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details System requirements and installation How to get it? 2 What is CC1? The CC1 system is a complete solution
More informationApplication Discovery Manager User s Guide vcenter Application Discovery Manager 6.2.1
Application Discovery Manager User s Guide vcenter Application Discovery Manager 6.2.1 This document supports the version of each product listed and supports all subsequent versions until the document
More informationLoad and Performance Load Testing. RadView Software October 2015 www.radview.com
Load and Performance Load Testing RadView Software October 2015 www.radview.com Contents Introduction... 3 Key Components and Architecture... 4 Creating Load Tests... 5 Mobile Load Testing... 9 Test Execution...
More informationMarketplace Functions: Experimental Prototype of Software Components and Documentation: D3.3.2
Ref. Ares(2013)119913-31/01/2013 Building the PaaS Cloud of the Future Marketplace Functions: Experimental Prototype of Software Components and Documentation: D3.3.2 Version 2.0 WP3 - Marketplace Functions
More informationNoSQL Performance Test In-Memory Performance Comparison of SequoiaDB, Cassandra, and MongoDB
bankmark UG (haftungsbeschränkt) Bahnhofstraße 1 9432 Passau Germany www.bankmark.de info@bankmark.de T +49 851 25 49 49 F +49 851 25 49 499 NoSQL Performance Test In-Memory Performance Comparison of SequoiaDB,
More informationMark Bennett. Search and the Virtual Machine
Mark Bennett Search and the Virtual Machine Agenda Intro / Business Drivers What to do with Search + Virtual What Makes Search Fast (or Slow!) Virtual Platforms Test Results Trends / Wrap Up / Q & A Business
More informationCSE 590: Special Topics Course ( Supercomputing ) Lecture 10 ( MapReduce& Hadoop)
CSE 590: Special Topics Course ( Supercomputing ) Lecture 10 ( MapReduce& Hadoop) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Spring 2016 MapReduce MapReduce is a programming model
More informationCloud computing - Architecting in the cloud
Cloud computing - Architecting in the cloud anna.ruokonen@tut.fi 1 Outline Cloud computing What is? Levels of cloud computing: IaaS, PaaS, SaaS Moving to the cloud? Architecting in the cloud Best practices
More informationThe Cloud to the rescue!
The Cloud to the rescue! What the Google Cloud Platform can make for you Aja Hammerly, Developer Advocate twitter.com/thagomizer_rb So what is the cloud? The Google Cloud Platform The Google Cloud Platform
More informationProject OpenSolaris TM Dynamic Service Containers featuring Nimsoft SLM
Project OpenSolaris TM Dynamic Service Containers featuring Nimsoft SLM Session: S311526 Robert Holt Customer Architect, Sun Microsystems Inc. Jason Carolan Distinguished Engineer, Sun Microsystems Inc.
More informationBitnami Packaging and Deployment Technology for Server Software
Bitnami Packaging and Deployment Technology for Server Software A TECHNICAL OVERVIEW Started in 2015, Bitnami s Software Partner Program provides a number of engineering benefits to Independent Software
More informationBuilding a Continuous Integration Pipeline with Docker
Building a Continuous Integration Pipeline with Docker August 2015 Table of Contents Overview 3 Architectural Overview and Required Components 3 Architectural Components 3 Workflow 4 Environment Prerequisites
More informationHow To Monitor A Server With Zabbix
& JavaEE Platform Monitoring A Good Match? Company Facts Jesta Digital is a leading global provider of next generation entertainment content and services for the digital consumer. subsidiary of Jesta Group,
More informationGraph Database Proof of Concept Report
Objectivity, Inc. Graph Database Proof of Concept Report Managing The Internet of Things Table of Contents Executive Summary 3 Background 3 Proof of Concept 4 Dataset 4 Process 4 Query Catalog 4 Environment
More informationSLA BASED SERVICE BROKERING IN INTERCLOUD ENVIRONMENTS
SLA BASED SERVICE BROKERING IN INTERCLOUD ENVIRONMENTS Foued Jrad, Jie Tao and Achim Streit Steinbuch Centre for Computing, Karlsruhe Institute of Technology, Karlsruhe, Germany {foued.jrad, jie.tao, achim.streit}@kit.edu
More informationLinstantiation of applications. Docker accelerate
Industrial Science Impact Factor : 1.5015(UIF) ISSN 2347-5420 Volume - 1 Issue - 12 Aug - 2015 DOCKER CONTAINER 1 2 3 Sawale Bharati Shankar, Dhoble Manoj Ramchandra and Sawale Nitin Shankar images. ABSTRACT
More informationIkasan ESB Reference Architecture Review
Ikasan ESB Reference Architecture Review EXECUTIVE SUMMARY This paper reviews the Ikasan Enterprise Integration Platform within the construct of a typical ESB Reference Architecture model showing Ikasan
More informationThe Hadoop Distributed File System
The Hadoop Distributed File System Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler Yahoo! Sunnyvale, California USA {Shv, Hairong, SRadia, Chansler}@Yahoo-Inc.com Presenter: Alex Hu HDFS
More informationComparing NoSQL Solutions In a Real-World Scenario: Aerospike, Cassandra Open Source, Cassandra DataStax, Couchbase and Redis Labs
Comparing NoSQL Solutions In a Real-World Scenario: Aerospike, Cassandra Open Source, Cassandra DataStax, Couchbase and Redis Labs Composed by Avalon Consulting, LLC June 2015 1 Introduction Specializing
More informationMilestone Solution Partner IT Infrastructure MTP Certification Report Scality RING Software-Defined Storage 11-16-2015
Milestone Solution Partner IT Infrastructure MTP Certification Report Scality RING Software-Defined Storage 11-16-2015 Table of Contents Introduction... 4 Certified Products... 4 Key Findings... 5 Solution
More informationConfiguration Manual Yahoo Cloud System Benchmark (YCSB) 24-Mar-14 SEECS-NUST Faria Mehak
Configuration Manual Yahoo Cloud System Benchmark (YCSB) 24-Mar-14 SEECS-NUST Faria Mehak Table of Contents 1 Introduction... 3 1.1 Purpose... 3 1.2 Product Information... 3 2 Installation Manual... 3
More information