Integrated Monitoring Approach for Seamless Service Provisioning in Federated Clouds Attila Kertész, Gabor Kecskemeti, Marc Oriol, Attila Csaba Marosi, Xavier Franch and Jordi Marco 20th Euromicro International Conference on Parallel, Distributed and Network-Based Processing Feb 17, 2012. www.s-cube-network.eu
Introduction Highly dynamic service environments require a novel infrastructure to handle on demand deployment and decommission of service instances. Cloud Computing allows: Outsourcing these dynamic environments Constructing extensible service-based applications Utilizes the latest achievements of Grid Computing, Service- oriented computing, business-processes and virtualization Virtual Appliances encapsulate metadata with a complete system (OS, libraries and applications) Infrastructure cloud systems (IaaS) allow to instantiate VA s on their virtualized resources: Virtual Machines Several public and private IaaS systems co-exist Only a Federated Cloud could offer the different capabilities as a whole S-Cube 2
Federated Cloud Management (FCM) An autonomic resource management solution Provides an entry point to a cloud federation Provides transparent service execution for users Cloud Following challenges are considered: Varying load of user requests Enabling virtualized management of applications Establishing interoperability and provider selection Minimizing Cloud usage costs Builds on meta-brokering, cloud brokering and automated on-demand service deployment Layered architecture Meta-broker Cloud Brokers Cloud infrastructure providers Cloud FCM Cloud Cloud S-Cube 3
FCM Architecture: overview 1 Generic Meta-Broker Service FCM Repository Top-level brokering Autonomously manage the interconnected cloud infrastructures Cloud a Cloud b Forms a federation with the help of Cloud Brokers S-Cube 4
FCM Architecture: overview Generic Meta-Broker Service 2 FCM Repository Manages VA distribution among the various cloud infrastructures Automated federationwide repository content management Cloud a Cloud b Offers current VA availability and estimates its future deployment S-Cube 5
FCM Architecture: overview Generic Meta-Broker Service FCM Repository 3 3 Interacts with a single IaaS system Manages resources Schedules service calls Cloud a Cloud b S-Cube 6
Generic Meta-Broker Service Service call (+requierments) Generic Meta-Broker Service Meta-Broker Core MatchMaker Call result Information Collector BPDL list Invoker FCM Repository VA x..va y IS Agent BPDL Broker Property Description Language Cloud Brokers are described Basic and aggregated dynamic properties Estimated availability time for a specific VA in the native repository Average VA deployment time CB Scheduling filters and ranks Cloud Brokers Cloud a Cloud b Cloud c S-Cube 7
Cloud Broker Generic Meta-Broker Service Q 1 VA x Cloud a VMQ x a VA y Cloud a VMQ y Cloud a VM Handler VM x 1 VM x 2 VM x n VM y 1 VM y 2 VM y m Native Repository FCM Repository VA x..va y Dynamic requirements may be specified with a service call Treated as a new VA type Some IaaS systems offer predefined classes of resources Resource class selected with at least the required resources Cloud a S-Cube 8
Cloud Broker: Scheduling of service calls The Cloud Broker performs scheduling of service calls to resources (VMs) Based on the monitoring information gathered May decide to start new resources based on: The number of running VM s to handle the service call The number of waiting service calls in the Service call queue The average execution time of service calls The average deployment time of VA s SLA constraints VM decommission Takes into account the billing period S-Cube 9
Service monitoring with SALMon Service Level Agreement Monitor (SALMon): designed for monitoring QoS of software services Capable of passive monitoring and testing purposes Supports any type of service technology (SOAP-based WS, RESTful services, etc.) It is itself an SBA with two main components: Monitor: retrieves values of quality metrics with the help of Measure Instruments Analyzer: evaluates conditions over these metrics It is suitable for monitoring running services in Cloud infrastructures S-Cube 10
The SALMon service monitoring framework S-Cube 11
Integrated Monitoring Approach for Seamless Service Provisioning (IMA4SSP) S-Cube 12
IMA4SSP details We have integrated SALMon to enable performance-driven service executions in Cloud federations Metrics related to service methods may be defined to monitor service operations Metric values are regularly refreshed in the Generic Service Registry (GSR) of the architecture Finally information stored in the GSR are used by the GMBS for Cloud infrastructure selection based on service reliability S-Cube 13
Enhanced monitoring with M3S 3. Fetch reliability info GMBS Users SALMon reports service metrics to a DB regularly checked by the meta-broker SALMon DDBB 2. Send results SALMon VM 1. Test metrics OpenNebula M3S Amazon The Minimal Metric Montoring Service (M3S) is used to monitor: Availability; Computing capability; and data transfer reliability. S-Cube 14
Miminizing monitoring costs Keeping the monitoring VMs in the Cloud can be costly To reduce these costs, the IS Agent component of GMBS has been extended to initiate the deployment and decommission of these VMs The monitored metric values have timestamps When the retrieved metric value of a service is outdated, the IS Agent contacts the responsible Cloud Broker to initiates a M3S and SALMon VM deployment When metric values with new timestamps are read from the registry, the IS Agent contacts the CB again, to decommission the monitoring VMs S-Cube 15
Conclusions We have designed a Federated Cloud Management solution that acts as an entry point to cloud federations Meta-brokering, cloud brokering and on-demand service deployment We have shown how Cloud Brokers manage the number and location of VMs for the various service requests We have extended FCM with enhanced monitoring capabilities with the SALMon framework, and created the IMA4SSP architecture We have shown a simplified version that uses the M3S reference service to monitor infrastructure reliability Our future work targets performance measurements in Cloud federations S-Cube 16
Thank You for your attention! Questions? https://www.lpds.sztaki.hu/cloudresearch S-Cube 17