Smart Metering of Cloud Services



Similar documents
ISBN: SDIWC 1

PRIVACY PRESERVATION ALGORITHM USING EFFECTIVE DATA LOOKUP ORGANIZATION FOR STORAGE CLOUDS

AN IMPLEMENTATION OF E- LEARNING SYSTEM IN PRIVATE CLOUD

A Hybrid Load Balancing Policy underlying Cloud Computing Environment

Beyond the Internet? THIN APPS STORE FOR SMART PHONES BASED ON PRIVATE CLOUD INFRASTRUCTURE. Innovations for future networks and services

The Hidden Extras. The Pricing Scheme of Cloud Computing. Stephane Rufer

Nessus or Metasploit: Security Assessment of OpenStack Cloud

THE CLOUD AND ITS EFFECTS ON WEB DEVELOPMENT

SLA-aware Resource Scheduling for Cloud Storage

Dynamic Resource Pricing on Federated Clouds

Dynamic resource management for energy saving in the cloud computing environment

CloudSim: A Toolkit for Modeling and Simulation of Cloud Computing Environments and Evaluation of Resource Provisioning Algorithms

Introduction to Cloud Computing

Figure 1. The cloud scales: Amazon EC2 growth [2].

An Efficient Checkpointing Scheme Using Price History of Spot Instances in Cloud Computing Environment

Infrastructure as a Service (IaaS)

Real Time Network Server Monitoring using Smartphone with Dynamic Load Balancing

MODIFIED BITTORRENT PROTOCOL AND ITS APPLICATION IN CLOUD COMPUTING ENVIRONMENT

Efficient Cloud Management for Parallel Data Processing In Private Cloud

An Experimental Study of Load Balancing of OpenNebula Open-Source Cloud Computing Platform

A Dynamic Resource Management with Energy Saving Mechanism for Supporting Cloud Computing

Affinity Aware VM Colocation Mechanism for Cloud

Geoprocessing in Hybrid Clouds

Oracle Platform as a Service and Infrastructure as a Service Public Cloud Service Descriptions-Metered & Non-Metered.

VIRTUAL RESOURCE MANAGEMENT FOR DATA INTENSIVE APPLICATIONS IN CLOUD INFRASTRUCTURES

Data Integrity Check using Hash Functions in Cloud environment

LabStats 5 System Requirements

IMPROVEMENT OF RESPONSE TIME OF LOAD BALANCING ALGORITHM IN CLOUD ENVIROMENT

Review of Cloud Computing Architecture for Social Computing

INVESTIGATION OF RENDERING AND STREAMING VIDEO CONTENT OVER CLOUD USING VIDEO EMULATOR FOR ENHANCED USER EXPERIENCE

Performance Analysis of VM Scheduling Algorithm of CloudSim in Cloud Computing

Investigations on Hierarchical Web service based on Java Technique

A Middleware Strategy to Survive Compute Peak Loads in Cloud

Analysis of Service Broker Policies in Cloud Analyst Framework

An Open MPI-based Cloud Computing Service Architecture

A Comparative Study on Load Balancing Algorithms with Different Service Broker Policies in Cloud Computing

Optimal Service Pricing for a Cloud Cache

International Journal of Computer Science Trends and Technology (IJCST) Volume 2 Issue 3, May-Jun 2014

Chapter 19 Cloud Computing for Multimedia Services

2) Xen Hypervisor 3) UEC

Benchmarking the Performance of XenDesktop Virtual DeskTop Infrastructure (VDI) Platform

Dell Virtualization Solution for Microsoft SQL Server 2012 using PowerEdge R820

Efficient and Enhanced Load Balancing Algorithms in Cloud Computing

Sistemi Operativi e Reti. Cloud Computing

Dynamic Resource allocation in Cloud

EFFICIENT AND SECURE DATA PRESERVING IN CLOUD USING ENHANCED SECURITY

International Journal of Computer & Organization Trends Volume21 Number1 June 2015 A Study on Load Balancing in Cloud Computing

Payment minimization and Error-tolerant Resource Allocation for Cloud System Using equally spread current execution load

GUEST OPERATING SYSTEM BASED PERFORMANCE COMPARISON OF VMWARE AND XEN HYPERVISOR

DEFINING CLOUD COMPUTING: AN ATTEMPT AT GIVING THE CLOUD AN IDENTITY.

A Review of Load Balancing Algorithms for Cloud Computing

Dragon Medical Enterprise Network Edition Technical Note: Requirements for DMENE Networks with virtual servers


PERFORMANCE ANALYSIS OF KERNEL-BASED VIRTUAL MACHINE

International Journal of Computer & Organization Trends Volume20 Number1 May 2015

A Survey on Aware of Local-Global Cloud Backup Storage for Personal Purpose

Green Cloud Computing 班 級 : 資 管 碩 一 組 員 : 黃 宗 緯 朱 雅 甜

A Comparative Study of Applying Real- Time Encryption in Cloud Computing Environments

This is an author-deposited version published in : Eprints ID : 12902

Resource usage monitoring for KVM based virtual machines

Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH

Dynamic Load Balancing of Virtual Machines using QEMU-KVM

Secure Attack Measure Selection and Intrusion Detection in Virtual Cloud Networks. Karnataka.

Managing the Real Cost of On-Demand Enterprise Cloud Services with Chargeback Models

BPM in Cloud Architectures: Business Process Management with SLAs and Events

Keywords Distributed Computing, On Demand Resources, Cloud Computing, Virtualization, Server Consolidation, Load Balancing

A Open Source Tools & Comparative Study on Cloud Computing

HP Intelligent Management Center Standard Software Platform

AUTOMATIC PROXY GENERATION AND LOAD-BALANCING-BASED DYNAMIC CHOICE OF SERVICES

EWeb: Highly Scalable Client Transparent Fault Tolerant System for Cloud based Web Applications

Cloud Infrastructure Pattern

High Performance Computing Cloud Computing. Dr. Rami YARED

7/15/2011. Monitoring and Managing VDI. Monitoring a VDI Deployment. Veeam Monitor. Veeam Monitor

DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING. Carlos de Alfonso Andrés García Vicente Hernández

StACC: St Andrews Cloud Computing Co laboratory. A Performance Comparison of Clouds. Amazon EC2 and Ubuntu Enterprise Cloud

Efficient Load Balancing using VM Migration by QEMU-KVM

Very Large Enterprise Network, Deployment, Users

Deploying Business Virtual Appliances on Open Source Cloud Computing

CLOUD COMPUTING. When It's smarter to rent than to buy

Managing Capacity Using VMware vcenter CapacityIQ TECHNICAL WHITE PAPER

How To Understand Cloud Computing

High performance computing network for cloud environment using simulators

PERFORMANCE ANALYSIS OF PaaS CLOUD COMPUTING SYSTEM

Energy Constrained Resource Scheduling for Cloud Environment

Heterogeneity-Aware Resource Allocation and Scheduling in the Cloud

Transcription:

Smart Metering of Cloud Services Akshay Narayan akshay.narayan@iiitb.org Gaurav Ranjan gaurav.ranjan@iiitb.org Kumar D kumar.d@iiitb.org Pavithrra V pavithrra.v@iiitb.org Vasudha S V vasudha.sv@iiitb.org International Institute of Information Technology Bangalore http://www.iiitb.ac.in/ Technical Report IIITB-OS-2011-5A April 2011

Abstract Smart Metering, prevalent in the power grids is a billing model where, the consumers are charged variable rates for the power consumed based on the load conditions of the grid. A similar model is developed for the Cloud Services in this project. The idea of Smart Metering of Cloud Services is to prevent wastage of compute power in low load conditions and improve the overall utilization of the cloud infrastructure. This objective is achieved by developing a varying tariff model based on the load prevailing on the cloud infrastructure. In this project, a mapping between the load condition for a particular time period and the pricing is achieved. Time period of utilization of the cloud service is divided into time slots. The pricing for every time slot is published and consumers can query for the same. It is the consumer s discretion to continue using the service or susp the usage for the time period. In the current implementation, a prediction of the load condition of the cloud infrastructure is obtained based on the previous load history. Statistical model, arima is used for this prediction. A prediction accuracy of up to 96% is achieved. Monitoring of the cloud infrastructure is performed using Hyperic, an open source monitoring framework. Hyperic SIGAR framework is used to monitor the per-consumer usage of compute resources. In the current setup, the implementation can scale up to three cloud instances. The pricing information obtained as a function of load condition is used for billing the consumers. The bill amount is a function of pricing for a time slot and the corresponding utilization of compute resource. The calculated bill is produced to the consumer. Project URL: https://sourceforge.net/projects/smartmetering/ c 2011 Akshay Narayan, Gaurav Ranjan, Kumar D, Pavithrra V & Vasudha SV This material is available under the Creative Commons Attribution-NonCommercial- ShareAlike License. See http://creativecommons.org/licenses/by-nc-sa/3.0/ for details.

CONTENTS Contents 1 Introduction 5 1.1 Problem Statement........................... 6 1.2 Similar Systems & Gap Analysis................... 6 2 System Design 8 2.1 Definitions................................ 8 2.2 Billing Model and Pricing....................... 8 2.3 Architecture............................... 10 2.4 Use Cases................................ 12 2.5 Module Interaction........................... 14 3 Implementation Details 16 3.1 Cloud Setup............................... 16 3.2 Monitoring............................... 16 3.3 Load Prediction............................. 18 3.4 Database Schema............................ 19 3.5 Pseudocode Listing........................... 20 4 Observations and Results 25 5 Conclusion 28 3

LIST OF FIGURES List of Figures 1 Smart Metering - Architecture.................... 10 2 Smart Metering - Administrator Use case.............. 12 3 Smart Metering - Consumer Use case................. 13 4 Smart Metering - Work flow...................... 14 5 Smart Metering - Cloud Setup.................... 16 6 Smart Metering - Database Schema.................. 19 7 ARIMA - Order Vs Prediction.................... 27 8 ARIMA - Prediction Vs Actual.................... 27 9 Hyperic Performance - Query Vs Load................ 28 4

1. INTRODUCTION 1 Introduction Cloud computing [1] provides for three categories of services namely, Infrastructure as a service, Platform as a service and Software as a service. The set of cloud services such as computation, storage and network are packaged as a computing resource and provided as metered utility service. Cloud services are sold on demand, both duration and quantum and are fully managed by the service providers. Multi tenancy [2] and on demand acquisition model make it very important to have a robust metering and billing mechanism. Smart meters [3] in the power domain are hardware devices which perform realtime registration of electricity usage. They allow obtaining the usage information both locally and remotely. These smart meters offer a two way communication between the consumer installation and the power grid which facilitates regulating the power consumption via the meter installation. Current implementation of metering mechanism [4] in cloud environment is based on a service level agreement which govern the provisioned capacity. It is a fixedcost, post-paid service model. The limitation of this model is that, the consumers are billed based on a predefined tariff for the utilization. This project caters to the need of a metering application in the cloud service context. The Smart Metering application is a software that provides for dynamic pricing of the cloud service based on the load condition of the cloud infrastructure. Resource usage per cloud instance is recorded and the consumer is billed according to two factors, his utilization and the load on the cloud during the period of utilization. The Smart Metering application facilitates improving the overall utilization of the cloud infrastructure, by implementing a varying tariff model based on the load prevailing on the infrastructure. Rest of this section introduces the problem statement, similar systems and gap analysis. Section 2 details the system design including the pricing and billing model implemented in the application. Architecture and use cases are also presented in section 2. Section 3 details the implementation of various elements described in the design. This section includes the pseudocode for the modules developed. Performance metrics and results are presented in Section 4. Section 5 presents some conclusions and scope for further development. 5

1. INTRODUCTION 1.1 Problem Statement The project is aimed at developing a technical model and an implementation of smart metering of cloud services. The project caters to the need of a dynamic pay-per-use model in the cloud environment based on real time pricing of the cloud service. The aim of this dynamic pay-per-use pricing model is regulation and improvement in overall utilization of the cloud infrastructure. With respect to the consumer, the computing resource needs may be time variant. The prevalent billing model [5] does not provide for a robust dynamic pay-per-use mechanism. There is a need to develop a smart metering mechanism which enables the consumer to be billed per usage based on dynamic tariff plans. This also enables the service provider to effectively manage the computing resources by having smart tariff plans based on the load condition on the infrastructure. We propose to solve the aforementioned problem to achieve a dynamic metering and billing mechanism for cloud services in this project. The result of the project is aimed to be analogous to the smart metering in power grids [3] [6]. 1.2 Similar Systems & Gap Analysis Smart metering and smart tariff is implemented in power grids in various countries across the globe [7]. These systems are related to physical entities like electricity, natural gas and potable water. These implementations are mainly concerned with the implementational hardware and communication system between the power meters and the central control hub. Commercial cloud service providers like Amazon provide for spot instances [8] which function based on bidding for the cloud instance. It works based on the bid amount which is the maximum price the consumer is willing to pay for the instance per hour. Gap Analysis The Virtual platform architecture for resource metering in data-centers [9] that is mostly implemented in private clouds provide for a pay-as-you-go billing mech- 6

1. INTRODUCTION anism. However, this implementation does not provide for dynamic metering. The billing mechanism is based on a predefined static pricing information. Also the literature indicates usage of Virtual Platform Architecture, which may not be completely applicable to a cloud infrastructure that is deployed by cloud service providers. The other existing systems for dynamic charge-back like Amazon Spot Instance [8] provides for a restrictive usage of cloud resources. Resource provisioning is based on the bid amount for the unused capacity. Once the cloud pricing amount increases beyond the bid amount, the cloud instance is terminated. The smart metering implementation proposed in this project aims at providing a dynamic billing mechanism for cloud service providers. The service provider specifies a pricing based on the current load on the cloud using the smart metering module. It is then the consumer s discretion to continue using the cloud instance by paying the specified amount or to terminate the cloud instance. 7

2. SYSTEM DESIGN 2 System Design 2.1 Definitions 1. Pricing rules: Service provider specified pricing information corresponding to the load on the system. 2. Load Range: The difference between maximum and minimum load values between which a price value holds. 3. Threshold load: The load values on the cloud, which specify the load condition. 4. Instance: A virtual machine deployed on the Eucalyptus cloud infrastructure. 5. Current price: The cost of utilizing the cloud service in a specified load condition. This is obtained from the pricing rules specified by the service provider and the current load estimated. 2.2 Billing Model and Pricing Pricing and billing are two important aspects addressed in the Smart Metering implementation. This section describes the mechanism of pricing and billing. Pricing Pricing for a cloud service can be applied based on multiple considerations. Current service providers like Amazon [10] and Rackspace [11] price the cloud instance mostly based on the configuration and duration of use. Another prevalent practice is to charge the consumers of fixed price for a lease period like the Amazon reserved instance [10]. 8

2. SYSTEM DESIGN Smart Metering proposes pricing based on the operational cost of running the service. In this pricing model base cost of running the service is specified by the service provider, C base. Pricing rules which define the pricing overhead for running the service under various load conditions are specified by the service provider, C loadi, where t is the time instance of operation. Current price, P t for the instance at a given load condition is given by equation 1. P t = C base C loadt (1) Where, P t operational price at time t C base base operational cost C loadt cost of running the service under load load t at time t. Billing Dynamic billing on cloud is a function of the instantaneous load on the cloud and the pricing information obtained as per the configuration specified by the service provider. Billing calculations involve determining the overall load on the cloud over a recent history and obtaining a weighted sum of the load on the entities and the corresponding pricing information as described above. Overall load L t on the cloud infrastructure at time t is the summation of the load on every node as given by: L t = Where, L t total load on the cloud at time t n l i (2) i=1 l i load index of the individual cloud component. The load obtained in equation 2 is mapped to a corresponding pricing value at a given instance of time, t. 9

2. SYSTEM DESIGN The billing amount is computed as a summation of the product of instantaneous pricing obtained in equation 1 and the utilization of the consumer, u t. Total bill amount is obtained as in equation 3 Bill = Where, P t operational price at time t u t resource utilization of the consumer at time t. n P t u t (3) t=1 2.3 Architecture Figure 1: Smart Metering - Architecture 10

2. SYSTEM DESIGN The main components of the Smart Metering application (indicated in Figure 1) are: Smart Meter module. Hyperic monitoring framework. Hyperic SIGAR framework. Back database. Thin client UI. The functioning of each of the component is described in this section. Smart Meter uses the service of Hyperic [12] open source system monitoring tool to monitor the compute load on the cloud infrastructure. Eucalyptus [13] cloud, deployed on Ubuntu server (version 10.10) is used to host the cloud instances. Smart Meter uses a MySQL database back to store the application data. Hyperic SIGAR API [14] is used to monitor the individual cloud instances. Hyperic is configured to monitor the load on the cloud every five minutes or every fifteen minutes as the configuration specifies. All other monitoring configurations are disabled currently. In its current form, the application does not compute the load on the network or storage to handle the metering and billing of cloud services. The application records the information about every deployed cloud instance. In the current form, the information about the deployed instances have to be manually fed by the administrator. This is later utilized for billing purpose. The Smart Meter application utilizes the statistical model ARIMA [15] to obtain the time series prediction of load. A mapping of the load to pricing is maintained in the application database, which is configurable by the service provider. Based on this mapping, the pricing for a time slot is published to the consumers. The consumer then decides to continue utilizing the cloud service or terminate the instance. The mapped price is later used to generate billing information on a per consumer basis. 11

2. SYSTEM DESIGN Figure 2: Smart Metering - Administrator Use case 2.4 Use Cases Use case - Administrator This use case is depicted in the Figure 2. The administrator or the cloud service provider controls the pricing and the load thresholds that govern the working of the Smart Meter. Interfaces are provided to the cloud service provider to change the pricing rules and threshold values. Pricing rules define the mapping between the load condition of the cloud and the pricing for the load period. The administrator is also able to monitor the number of users on the cloud at any given instance of time and the usage statistics of cloud as well as every deployed 12

2. SYSTEM DESIGN Figure 3: Smart Metering - Consumer Use case instance. The threshold values define the transitions between each defined load intervals. Smart Meter application uses the services of Hyperic monitoring tool and Hyperic SIGAR API to monitor the cloud infrastructure. Use case - Consumer This use case is depicted in the Figure 3. The consumer can view current pricing and obtain his usage statistics via the Smart Meter interface. Current pricing information is obtained on demand. It is based on this price information, the consumer decides to continue using the service or terminate the 13

2. SYSTEM DESIGN cloud instance. 2.5 Module Interaction The interaction between various modules in the Smart Meter application is as indicated in Figure 4. The working of each of the modules is explained in the rest of this section. Figure 4: Smart Metering - Work flow Data Collection Smart Metering application avails the service of Hyperic monitoring framework to monitor the load on the cloud infrastructure and Hyperic SIGAR framework to retrieve the individual instance resource utilization. 14

2. SYSTEM DESIGN Data collection (VM Instances) Process ID (PID) of the kvm process containing the cloud instance is identified and SIGAR API is invoked to retrieve resource utilization data for the identified process. SIGAR API returns the percentage CPU Utilization for the process whose PID is passed as argument. This process is repeated for all cloud instances listed in the application database. Data Collection (Cloud Infrastructure) Hyperic getmetrics() API is used to obtain the load average of a node in the cloud infrastructure. This API returns load values as a list to the caller. Load Prediction Load prediction in the Smart Metering application is done based on time series analysis. The monitored load data forms a time series with predefined intervals. Arima, a statistical model is used to predict the load for next interval. Billing The Smart Meter module maintains a mapping of the load to price. The load predicted above is mapped to a corresponding price value. This is used along with the utilization recorded for the instance and the bill is generated as the product of the two values mentioned. Back Database MySQL is used as the back database to maintain the application data. The database schema is explained in section 3 15

3. IMPLEMENTATION DETAILS 3 Implementation Details 3.1 Cloud Setup Figure 5: Smart Metering - Cloud Setup Eucalyptus was deployed on Ubuntu Server Version 10.10 for the project. The cloud controller, cluster controller and walrus storage engine setup was deployed on a Net-book running Intel Atom 1.6GHz processor with 1GB RAM. The node controller was installed on a more powerful Intel Core i7 based Dell laptop with 4GB RAM. The client machine was also installed on the same machine as the node controller. The cloud deployment is analogous to Figure 5. The instructions in the Eucalyptus Beginners guide [16] were followed to deploy the cloud setup. 3.2 Monitoring The monitoring tool for cloud infrastructure was chosen such that the basic criteria mentioned below are satisfied. Scalability: The monitoring tool must be scalable to at least one thousand nodes. 16

3. IMPLEMENTATION DETAILS It should have zero or minimum impact on the monitored machines resource utilization. The tool should automatically start monitoring the devices added dynamically in the infrastructure. It should consume minimal network bandwidth during data collation as network bandwidth in the cloud comes with a cost overhead. The same tool must be able to monitor various system resources like compute power, bandwidth and storage. Programmatic access to monitoring data should be provided by the tool, in the form of API. Hyperic [12] was was chosen among the evaluated tools, which include: Nagios Hyperic Zabbix Hyperic HQ Some of the main features of Hyperic HQ include auto discovery of system resources, services running on machines, monitoring resources and applications, persisting monitored values in database, administration of software resources, alerts and notification etc. Hyperic API s are used to fetch the metric data programmatically from the database created by Hyperic. Hyperic scalability benchmarks can be found in the Hyperic HQ and MySQL Back Performance Study [17] The Hyperic SIGAR API is exclusively used to monitor the deployed Virtual machines. This is necessary to obtain the usage information on per instance basis. This information is used to generate the bill. 17

3. IMPLEMENTATION DETAILS 3.3 Load Prediction Based on historical data collected through monitoring framework, the future load is predicted in the Smart Meter application. Predicting load plays an important part in the Smart Meter to publish an indicative pricing of the cloud service for the next time slot. In this project ARIMA is used to predict the load based on the cloud based on the metric data collected from the Hyperic monitoring tool. ARIMA The Auto Regressive model assumes that the current value of a process x t, can be described by a finite linear aggregate of the previous values of the process along with the current value of a white noise signal a t. Generally, AR models (denoted as AR(p)) are described using the following equation Where, x t = ϕ 1 x t 1 + ϕ 2 x t 2 + L + ϕ p x t p + a t (4) a t the unpredictable white noise sequence with zero mean and variance of σ 2 a ϕ 1, ϕ 2, ϕ p and L coefficients of AR model with p as the number of coefficients. If the p coefficients of AR models were estimated from the past data, the onestep-ahead prediction of this AR model can be estimated using these p coefficients along with the p historical data, x t 1, x t 2, L, x t p Mathematically this is described as follows: ˆx t = ϕ 1 x t 1 + ϕ 2 x t 1 + L + ϕpx + t p (5) Where, ˆx t one-step-ahead prediction value of time t The parameter p of AR models is the number of historical data points it uses. Run time cost increases if a higher order is used in the AR model to predict the load. If order chosen is too less, it results in an erroneous prediction. We assume to have a dedicated machine for smart metering and hence use a higher order value for p. In [18], Dinda defined the space of order p to be explored is from 18

3. IMPLEMENTATION DETAILS 1 to 32. And his recommation is to use AR models of relatively high order (16 or better) for load prediction. Hence AR (16) model which is equivalent to ARIMA(16, 0, 0) is used to predict the CPU load for Smart Metering application. 3.4 Database Schema The database schema used for the Smart Meter application is as shown below in Figure 6. Figure 6: Smart Metering - Database Schema The purpose of the tables is as listed below: customer info: Stores informations about every customer. pricing info: Contains load to price mapping as per the pricing scheme of the service provider. It is populated using the configurable XML data. 19

3. IMPLEMENTATION DETAILS time slots: Contains time interval in which the readings will be taken and pricing will be calculated. customer usage: Stores the usage of all the customers at every time slots of the day. This table is populated using data from SIGAR APIs. cloud load: Stores the actual and predicted state of load on cloud at every time slot. usage price: The calculated usage price per time slot is stored in this table. monthly bill: Contains the total usage and total monthly bill for every customer. 3.5 Pseudocode Listing This section lists the pseudocode for various modules implemented in the Smart Metering application. Data Collection Data Collection VM Instances The method getvmstats() invokes the SIGAR API getproccpu(). PID of the KVM process containing the cloud instance in question is passed as the argument to collect resource utilization data. The percentage CPU utilization returned by getproccpu() is captured in vmcpuusage. The above process is repeated for all cloud instances listed in the application database (listofvm). The procedure is listed in Algorithm 1. Data Collection Cloud Load The method getload() invokes Hyperic getmetrics() API. The unique identifier of the node controller is passed as a parameter to getmetrics(). The number of data points to be returned is configurable. getmetrics() 20

3. IMPLEMENTATION DETAILS API returns a list containing the load average values collected at every one, five or fifteen minutes. The process is repeated for all registered node controllers. The procedure is listed in Algorithm 2. Algorithm 1: Data Collection VM Instances Data: vmcpuusage Data: list listofvm Data: pid begin getinstanceutilization while true do foreach listofvm[ 1... n ] do vmcpuusage getvmstats(pid) store(vmcpuusage, timestamp) begin getvmstats Data: pid Result: vmcpuusage vmcpuusage sigar.getproccpu(pid) return vmcpuusage Load Prediction Data Fitting & Prediction The load data obtained earlier using getcloudload() which is a time series has to be fit into the arima model. The predictload() method uses ARIMA.fit() method to achieve this objective. Prediction of the next load instance is done by the ARIMA.predict() method. These methods are available as a part of R programming language. Smart Metering application uses the rjava interface to interact with R. Map The predicted load is mapped to an operational cost. This mapped pricing 21

3. IMPLEMENTATION DETAILS Algorithm 2: Data Collection Cloud Load Data: list loaddatapoints Data: list nodecontrollers begin getcloudload while true do foreach nodecontrollers[ i = 1... n ] do loaddatapoints getload(nodecontrollers[i]) return loaddatapoints begin getload Data: nodecontroller Result: loaddatapoints loaddatapoints hyperic.getmetrics(nodecontroller) information is published to the consumers. The map() method performs a table look up for the predicted load to obtain a corresponding pricing information. The predictedprice is returned to the calling function. These procedures are listed in Algorithm 3. Billing The actual resource utilization of instance obtained using the getinstanceutilization() method defined earlier and the predictedprice obtained using predictprice() method are stored is the SmartMetering database along with the timestamp. These two values are used to generate the bill amount for a particular cloud instance. Retrieve CPU Utilization The getcpuusageattimestamp() method executes an SQL query to retrieve the stored CPU utilization of every cloud instance. timestamp is passed as a parameter to this method. 22

3. IMPLEMENTATION DETAILS Algorithm 3: Load Prediction Data: list loadvalues Data: double predictedload Data: double processutilization Result: predictedprice begin predictprice while true do loadvalues getcloudload() predictedload predictload(loadvalues) predictedprice map(predictedload) This predicted price is published to the consumers. store(predictedprice, timestamp) begin predictload Data: loadvalues Data: predictedload Data: arimaref arimaref ARIMA.fit(loadValues) predictedload ARIMA.predict(arimaRef) return predictedload begin map Data: predictedload Data: constant LOAD TO PRICE TABLE Result: predictedprice for predictedload in keys of (LOAD TO PRICE TABLE) do predictedprice LOAD TO PRICE TABLE[predictedLoad] return predictedprice 23

3. IMPLEMENTATION DETAILS Retrieve Price Information The getpriceattimestamp() method executes an SQL query to retrieve the predicted price at given timestamp. These procedures are listed in Algorithm 4. Algorithm 4: Generate Billing Data: vmcpuusage Data: predictedprice Data: timetamp Data: listofvm begin generatebill foreach listofvm[ i = 1... n ] do vmcpuusage getcpuusageattimestamp(timestamp) predictedprice getpriceattimestamp(timestamp) billamount vmcpuusage * predictedprice store(listofvm[i], billamount) begin getcpuusageattimestamp Data: timestamp Result: vmcpuusage vmcpuusage get vmcpuusage at timestamp=timestamp return vmcpuusage begin getpriceattimestamp Data: timestamp Result: predictedprice predictedprice get predictedprice at timestamp=timestamp return predictedprice 24

4. OBSERVATIONS AND RESULTS 4 Observations and Results Results Load to price mapping A configurable load to price mapping was achieved in this project. The basic operational cost specified by the service provider and the pricing rules form the basis of this mapping. A sample mapping is listed in Table 4. Load Start Load End Base Price Price Multiplier 0 1 100 0.2 1 2 100 0.5 2 3 100 1 3 4 100 2.5 4 20 100 7.5 Table 1: Load to Price Mapping Dynamic tariff A tariff plan based on the above mapping was developed. Varying tariff based on the load condition of the cloud infrastructure was achieved. Table 4 depicts a sample dynamic tariff generated by the application. Load Condition Load Average Price (in Rs.) Low 1.76977 50 Low 1.19032 50 Medium 2.04024 100 Medium 2.53133 100 High 3.56575 250 High 3.48933 250 Extreme 9.44684 750 Extreme 8.19211 750 Extreme 4.69272 750 Table 2: Load to Price Mapping Itemized billing The current implementation of Smart Metering application is capable of 25

4. OBSERVATIONS AND RESULTS generating an itemized bill per consumer and per instance basis. A sample of the itemized billing for one consumer in the database is listed in Table 4 Instance ID Usage Price ( in Rs) i-3a1806e8 5.75 1148 i-40cf0798 15.65 3357.5 i-666a0b5f 1.1 882.5 Table 3: Itemized Usage Performance Metrics This section of the document lists the performance metrics evaluated as a part of the project. ARIMA Order for Prediction Smart Metering application uses previous 200 data points to predict the load of the next time interval. For this input size, it was experimentally determined that an order of P = 8 or P = 9 in ARIMA(P, D, Q) results in a prediction close to the actual load condition. The plot of Order Vs Prediction is shown in Figure 7 Prediction Using ARIMA model a prediction accuracy of up to 96% was achieved. However, the error factor is high when there is a steep jump or a steep drop in the load average. This behavior can be observed in the plot shown in 8 Time to Query It was observed that the time to query the data from Hyperic gradually increased as the load on the machine increased. However, the time to query was always within acceptable limits. Peak time to query being around 120ms when the corresponding load average was recorded as 7.5. A plot of this data is show in Figure 9 26

4. OBSERVATIONS AND RESULTS Figure 7: ARIMA - Order Vs Prediction Figure 8: ARIMA - Prediction Vs Actual 27

5. CONCLUSION Figure 9: Hyperic Performance - Query Vs Load 5 Conclusion An effective mapping between price and load was obtained in the current implementation of Smart Metering. Using this mapping a dynamic pricing of the cloud services based on the load condition of the cloud infrastructure was achieved. Based on this dynamic pricing information, a varying tariff model for the cloud services was developed in the project. An effective billing mechanism was developed for the cloud services which is analogous to the Smart Metering in power grids. Future Work A mechanism to handle time based billing can be incorporated to the current implementation of Smart Metering. The order for ARIMA used in this project was defined by [18], however, in [19] Liu et. al. suggest optimal order for ARIMA by considering four different criteria. We can incorporate this improvement in the Smart Meter application. 28

5. CONCLUSION In the current implementation, we use ARIMA as the only statistical prediction model. However, the robustness of the prediction can be improved by using a weighted average of various predictors like in the Potluck Problem [20]. The Smart Meter application in the current form deps on Hyperic system monitoring tool for load monitoring and R package for load prediction. The application can be made stand alone, by incorporating a monitoring module and a prediction engine written specifically for the Smart Meter. The current implementation of the Smart Meter is compatible with Eucalyptus cloud infrastructure. The system can be exted to work with other cloud service providers like Nimbus, Amazon etc. 29

REFERENCES References [1] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, A view of cloud computing, Commun. ACM, vol. 53, pp. 50 58, April 2010. [2] C.-P. Bezemer, A. Zaidman, B. Platzbeecker, T. Hurkmans, and A. t Hart, Enabling multi-tenancy: An industrial experience report, in Software Maintenance (ICSM), 2010 IEEE International Conference on, Sept 2010, pp. 1 8. [3] R. van Gerwen, S. Jaarsma, and R. Wilhite, Smart metering, July 2006. [Online]. Available: http://www.leonardo-energy.org/webfm s/435 [4] J. W. Ross, M. R. Vitale, and C. M. Beath, The untapped potential of it chargeback, MIS Quarterly, vol. 23, no. 2, pp. 215 237, June 1999. [5] Understanding cloud pricing. [Online]. Available: http://searchcloudcomputing.techtarget.com/tutorial/ Understanding-cloud-computing-pricing [6] P.Vijayapriya, G. Bapna, and Dr.D.P.Kothari, Smart tariff for smart meters in smart grid, International Journal of Engineering and Technology, vol. 2, no. 5, 2010. [7] H.-P. Siderius and A. Dijkstra, Smart metering for households: Cost and benefits for the Netherlands, EEDAL, 2006. [8] Amazon spot instance. [Online]. Available: http://aws.amazon.com/ec2/ spot-instances/ [9] R. Iyer, R. Illikkal, L. Zhao, D. Newell, and J. Moses, Virtual platform architectures for resource metering in datacenters, SIGMETRICS Perform. Eval. Rev., vol. 37, pp. 89 90, October 2009. [10] Amazon pricing. [Online]. Available: http://aws.amazon.com/ec2/pricing/ [11] Rackspace pricing. [Online]. Available: http://www.rackspace.com/cloud/ cloud hosting products/servers/pricing/ 30

REFERENCES [12] Hyperic open source monitoring. [Online]. Available: http://www.hyperic. com/products/open-source-systems-monitoring [13] Open eucalyptus. [Online]. Available: http://open.eucalyptus.com/ [14] Hyperic sigar api. [Online]. Available: http://www.hyperic.com/products/ sigar [15] G. E. P. Box, G. M. Jenkins, and G. C. Reinsel, Time series analysis: Forecasting and control, 2008. [16] Eucalyptus beginners guide, 2010. [Online]. Available: http://open. eucalyptus.com/forum/eucalyptus-beginners-guide-uec-edition [17] Hyperic HQ and MySQL Back Performance Study, 2008. [Online]. Available: http://download.hyperic.com/pdf/hyperic-wp-mysql-benchmark. pdf [18] P. A. Dinda and D. R. O Hallaron, An evaluation of linear models for host load prediction, High-Performance Distributed Computing, International Symposium on, vol. 0, p. 10, 1999. [19] J. Huo, L. Liu, L. Liu, Y. Yang, and L. Li, Selection of the order of autoregressive models for host load prediction in grid, Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, ACIS International Conference on, vol. 2, pp. 516 521, 2007. [20] P. K. Enumula and S. Rao, The potluck problem, Economics Letters, vol. 107, no. 1, pp. 10 12, 2010. 31