Context-Aware Optimization in Cloud Management Jakub Krzywda Umeå University Lund 2014-05-15 www.cloudresearch.org
BSc & MSc Studies Poznan University of Technology Master program Distributed data processing Master s Thesis Data mining in XML files 2
Internship at Inria Grenoble April September 2013 Team: STEEP sustainable development Topic: measuring urban sprawl Raport: http://hal.inria.fr/hal-00907081 3
PhD Studies since November 2013 Umeå University Department of Computing Science Distributed Systems Research Group 4
Table of Contents Context-Awareness Interesting problems Cactos project 5
Placement Initial Placement (including admission) Continuous Consolidation (VM migration) 6
Erik Elmroth, elmroth@cs.umu.se
Erik Elmroth, elmroth@cs.umu.se Jakub Krzywda, Virtual Machine = black box 8
Are we close to the limit? 9
Context-Awareness 10
Interesting problems 1. Request > Resource utilization model 2. Workload prediction 3. Influence of co-location 4. Transforming resource utilization between different PMs 11
Request > Resource utilization model Benchmarking applications Relation between #requests and resource utilization Requests are heterogeneous 12
Workload prediction New applications? Classification Workload history from previous IP 13
Influence of co-location 14
Transforming resource utilization between PM 15
Cactos
A very short view on CACTOS Partners Umeå Universitet, SE Ulm Universität, DE REALTECH AG, DE The Queen s University of Belfast, UK Flexiant Limited, UKFZI Forschungszentrum Informa5k, DE Dublin City University, IR Dura+on: Oct 2013 September 2016 Total cost: 4,761,232 Context- Aware Cloud Topology Op5misa5on and Simula5on h;p://cactosfp7.eu
Cactos in a nutshell Data Centre Operators/ Cloud Operators collect infrastructure and hardware data analyze datalogs collect application behavior data Cloud Middleware Developers, Cloud Infrastructure Providers, Data Centre Operators simulate optimization models CactoScale Cloud Middleware Developers, Cloud Infrastructure Providers, Data Centre Operators determine best fitting resource predict behavior of applications on different resources CactoSim CactoOpt automatic mapping of workloads validate and improve models find most appropriate provider 18
CactoOpt Architecture Cluster level resource utilization load mixing Data center level proactive plan multiple criteria / objective functions 19
Infrastructure model Data Center Aggregated - #CPU cores - Total memory - Network Atteched Storage Inter cluster VLAN Cluster Aggregated - #CPU cores - Total memory Node VLAN Storage VLAN Cluster Rack Power distribution: - Intra rack Power distribution Node (NodeID) CPU: - #CPUs - #CPU cores - Core frequency - CPU IDs Memory: - Total - Bandwidth - Frequency Storage: - Capacity - Bandwidth GUI/External Access VLAN Network: - #Interconnects For each interconnect - Nominal bandwidth - Type... Node Node Node 20 Rack
Load model Monitoring the utilization of resources CPU Memory Storage Network Energy At different levels VM PM 21
Actuators VM level start new suspend/resume migrate PM level suspend/resume Dynamic Voltage/Frequency Scaling 22
Summary It is hard to improve using blackbox approach Use Context-Awareness to make proactive decisions Modelling data center infrastructure and load in Cactos project 23