Context-Aware Optimization in Cloud Management

Transcription

1 Context-Aware Optimization in Cloud Management Jakub Krzywda Umeå University Lund

2 BSc & MSc Studies Poznan University of Technology Master program Distributed data processing Master s Thesis Data mining in XML files 2

3 Internship at Inria Grenoble April September 2013 Team: STEEP sustainable development Topic: measuring urban sprawl Raport: 3

4 PhD Studies since November 2013 Umeå University Department of Computing Science Distributed Systems Research Group 4

5 Table of Contents Context-Awareness Interesting problems Cactos project 5

6 Placement Initial Placement (including admission) Continuous Consolidation (VM migration) 6

7 Erik Elmroth,

8 Erik Elmroth, Jakub Krzywda, Virtual Machine = black box 8

9 Are we close to the limit? 9

10 Context-Awareness 10

11 Interesting problems 1. Request > Resource utilization model 2. Workload prediction 3. Influence of co-location 4. Transforming resource utilization between different PMs 11

12 Request > Resource utilization model Benchmarking applications Relation between #requests and resource utilization Requests are heterogeneous 12

13 Workload prediction New applications? Classification Workload history from previous IP 13

14 Influence of co-location 14

15 Transforming resource utilization between PM 15

16 Cactos

17 A very short view on CACTOS Partners Umeå Universitet, SE Ulm Universität, DE REALTECH AG, DE The Queen s University of Belfast, UK Flexiant Limited, UKFZI Forschungszentrum Informa5k, DE Dublin City University, IR Dura+on: Oct 2013 September 2016 Total cost: 4,761,232 Context- Aware Cloud Topology Op5misa5on and Simula5on h;p://cactosfp7.eu

18 Cactos in a nutshell Data Centre Operators/ Cloud Operators collect infrastructure and hardware data analyze datalogs collect application behavior data Cloud Middleware Developers, Cloud Infrastructure Providers, Data Centre Operators simulate optimization models CactoScale Cloud Middleware Developers, Cloud Infrastructure Providers, Data Centre Operators determine best fitting resource predict behavior of applications on different resources CactoSim CactoOpt automatic mapping of workloads validate and improve models find most appropriate provider 18

19 CactoOpt Architecture Cluster level resource utilization load mixing Data center level proactive plan multiple criteria / objective functions 19

20 Infrastructure model Data Center Aggregated - #CPU cores - Total memory - Network Atteched Storage Inter cluster VLAN Cluster Aggregated - #CPU cores - Total memory Node VLAN Storage VLAN Cluster Rack Power distribution: - Intra rack Power distribution Node (NodeID) CPU: - #CPUs - #CPU cores - Core frequency - CPU IDs Memory: - Total - Bandwidth - Frequency Storage: - Capacity - Bandwidth GUI/External Access VLAN Network: - #Interconnects For each interconnect - Nominal bandwidth - Type... Node Node Node 20 Rack

21 Load model Monitoring the utilization of resources CPU Memory Storage Network Energy At different levels VM PM 21

22 Actuators VM level start new suspend/resume migrate PM level suspend/resume Dynamic Voltage/Frequency Scaling 22

23 Summary It is hard to improve using blackbox approach Use Context-Awareness to make proactive decisions Modelling data center infrastructure and load in Cactos project 23

24