Memory-Aware Sizing for In-Memory Databases

Transcription

1 Memoy-Awae Sizing fo In-Memoy Databases Kasten Molka, Giuliano Casale, Thomas Molka, Laua Mooe Depatment of Computing, Impeial College London, United Kingdom {k.molka3, SAP HANA Cloud Computing, Systems Engineeing, Belfast, United Kingdom {kasten.molka, thomas.molka, Abstact In-memoy database systems ae among the technological dives of big data pocessing. In this pape we apply analytical modeling to enable efficient sizing of in-memoy databases. We pesent novel esponse time appoximations unde online analytical pocessing wokloads to model thead-level fokjoin and pe-class memoy occupation. We combine these appoximations with a non-linea optimization pogam to minimize memoy swapping in in-memoy database clustes. We compae ou appoach with state-of-the-at esponse time appoximations and tace-diven simulation using eal data fom an SAP HANA in-memoy system and show that ou optimization model is significantly moe accuate than existing appoaches at simila computational costs. Index Tems Optimization; In-memoy Databases; Pefomance; Closed Queueing Netwoks; Appoximation, SAP HANA. I. INTRODUCTION In-memoy database systems leveage new technologies, such as SDRAM, flash stoage, FPGAs and GPUs, to shaply optimize database thoughputs and latencies. Case studies show that in-memoy databases can achieve temendous speedups, outpefoming taditional disk-based database systems by seveal odes of magnitude []. As a esult, inmemoy systems ae in high commecial demand, in paticula as pat of cloud sofwae-as-a-sevice offeings []. This poses new challenges egading the management of these applications in cloud infastuctues, since thee is vitually no achitectual design, sizing and picing methodology focused explicitly on in-memoy technologies. This pape tackles this poblem by intoducing a novel povisioning famewok specifically tailoed to in-memoy databases. We popose a novel optimization-based methodology to povision these systems at a efeence timescale, minimizing costs. Ou methodology can be applied to inmemoy database clustes that ae continuously monitoed and feed pefomance measuements into ou famewok. Ou famewok enables what-if analyses fo vaious in-memoy database configuations egading pefomance and cost without the need to setup expeiments physically. In paticula, we seek fo load-dispatching outing pobabilities that can load balance in-memoy instances fo a set of clients especting the sevice level ageement (SLA) in place with the custome. We use a queueing modeling appoach to descibe the levels of contention at esouces, in ode to establish the likelihood that a sizing configuation will comply to SLAs. In paticula, since in-memoy systems ae memoy-bound applications, it is cucial that thei sizing models can captue memoy constaints, as memoy exhaustion and swapping ae moe likely to happen in this class of applications. Convesely, existing sizing methods fo entepise applications have pimaily focused on modeling mean CPU demand and equest esponse times. Memoy occupation is difficult to model as it equies the ability to pedict the pobability of a cetain mix of queies being active at a given time. Howeve, pobabilistic models tend to be expensive to solve, leading to slow iteation speed when used in combination with numeical optimization. To cope with this issue, we intoduce a famewok based on appoximate mean-value analysis (AMVA), a classic methodology to obtain pefomance estimates in queueing netwok models [3]. We obseve in paticula that cuent AMVA methods ae unable to coectly captue the effects of vaiable theading levels in in-memoy database systems and popose a coection that makedly impoves accuacy. Ou appoach etains the same computational popeties of AMVA and it is simple and inexpensive to integate in optimization pogams. We also demonstate that multi-stat inteio point methods and evolution stategies can be effectively used to solve the esulting optimization pogams, offeing diffeent tadeoffs between accuacy and scalability. In paticula, we popose a simple yet fast mutation function fo ou evolution stategy that tuns out to be competitive with inteio point methods. Finally, we validate ou appoach using eal taces fom a commecial in-memoy database appliance, SAP HANA []. The emainde of this pape is oganized as follows. Section II motivates ou eseach objective and gives the poblem statement. Section III intoduces the system chaacteistics of ou in-memoy database. A novel esponse time appoximation is developed in Section IV, combined with a non-linea optimization pogam in Section V and evaluated in Section VI by numeical tests. Finally, Section VII outlines elated wok, while Section VIII concludes this pape and gives futue wok. II. MOTIVATION AND PROBLEM STATEMENT In-memoy databases ae a completely new type of big data analysis systems capable of pocessing heavily memoy intensive wokloads in a paallel fashion. Thei esouce management is a complex and difficult task that includes memoyawae sizing of these systems acoss heteogeneous clustes. Analytical pefomance models of in-memoy databases can suppot these sizing decisions by enabling what-if analyses

2 Mean Relative Response Time Eo in % 8 6 AMVA FJ-AMVA Wokload Scenaio Fig.. Relative Response Time Eo compaed with Simulation fo vaious hadwae configuations. It is theefoe essential to develop models that ae able to captue the behavio of in-memoy databases acoss seveal dimensions. In paticula, pefomance measues such as esponse times and thoughputs ae key metics of in-memoy systems that need to be modeled accuately. Howeve, the extensive and vaiable theading-level we ae faced with cannot be coectly captued by existing analytical appoaches, such as AMVA [3], widely used to model the pefomance of multi-tie applications [5], and stateof-the-at techniques, i.e. fok-join AMVA (FJ-AMVA) [6]. To demonstate this, we paameteized these two methods fom eal taces of ou in-memoy database SAP HANA and compaed thei esponse time pedictions with a validated inmemoy database simulato [7]. We give an except of ou esults in Figue, which depicts the elative esponse time eo of AMVA and FJ-AMVA compaed with ou simulato unde diffeent wokload scenaios. We obseve that using both AMVA and FJ-AMVA can esult in pediction eos of moe than 5%. We will theefoe develop a new pefomance model that captues the chaacteistics of in-memoy databases moe accuately. Ou second challenge is coined by a capacity planning poblem, assigning esouces to in-memoy databases subject to memoy and utilization constaints. Optimizing memoy occupation fo such systems can be computationally expensive and can intoduce local optima due to non-convexity. Hence we addess this by poposing intelligent optimization stategies and combine these with ou new pefomance model. Summaizing, ou main contibutions ae: A novel analytic eponse time appoximation fo inmemoy databases that consides thead-level fok join An optimization-based fomulation fo seeking loaddispatching outing pobabilities to minimize memoy swapping fo such systems subject to esouce constaints An expeimental validation that eveals the applicability of local and global seach stategies Paameteization and evaluation of ou models with eal taces of an in-memoy database system To the best of ou knowledge thee ae no methods that combine thead-level fok join models with non-linea optimization of in-memoy databases, and thus epesents a novelty in this eseach aea. III. IN-MEMORY DATABASE CHARACTERISTICS A. OLAP Wokload Chaacteistics In-memoy databases ae optimized to execute analytical business tansactions, i.e. OLAP. These types of tansactions Nomalized Thead Level Paallelism Wam-up Memoy Occupation in GB Nomalized Peak Memoy Occupation Queies Quey Class (a) Thead Level Paallelism Queies Quey Class (c) Memoy Occupation Nomalized Thead Execution Time s t Nomalized Execution Time Queies Quey Class.5 (b) Execution Times s t s t.5 s t Thead ID (d) Thead Times fo t 8 Fig.. Wokload Chaacteistics nomalized by Quey Class epesent ead-only wokloads and can thus be entiely pocessed in main memoy. Due to thei analytical natue OLAP wokloads ae not only computationally intensive but also show high vaiability in thei theading levels. To emphasize these divese chaacteistics, we analyzed tace logs obtained fom benchmaking expeiments unning SAP HANA on an IBM X5 -socket database seve configued with TB main memoy [8]. The benchmak was un with a scale facto of x and compised a set of OLAP queies intoduced by SAP-H, an extension to the TPC-H benchmak with emphasis on analytical pocessing. We povide the esults of ou tace log analysis fo all quey classes in Figue (a)-(c). All values ae obtained fom isolated quey uns, nomalized by class fo confidentiality and shown with thei espective standad deviations. In Figue (a) we pesent the aveage numbe of CPU coes used by each quey class and denote this with thead level paallelism. As expected, we see a stong vaiability of the paallelism acoss all quey classes, which can incease contention fo esouces unde OLAP wokload mixes. This attains futhe distinction due to the vaying computational expense of OLAP queies, depicted in Figue (b). In addition, we eveal the memoy intensive chaacte of OLAP wokloads in Figue (c) by showing the physical memoy tempoaily occupied duing the pocessing of queies, which vaies on gigabyte scale. To emphasize the impotance of compession duing the execution of OLAP wokloads, we demonstate in Figue (c) that ou coesponding benchmak dataset with a size of.3 TB was educed to appoximately 65GB afte a wam-up un fo each quey class. B. Request Handling and Demand Chaacteization Quey planning and execution ae impotant stages duing the pocessing of OLAP wokloads. The fist stage involves a quey planne analyzing the quey stuctue and ceating an s t

3 Coe ID Case a 3 Coe Activity Change of Thead Affinity Job Execution Time d Case b Pocessing Phases b b b 3 b b 5 b 6 Coe ID 3 c = 3 c 6=3 Numbe of active Coes Case a Thead t Thead Time t Thead Execution Time s Case b Thead t Thead Time - odeed s discaded.s 7 t Thead Execution Time s Fig. 3. Sevice Demand Estimation fo an OLAP Quey appopiate execution plan. In the second stage, queies ae executed depending on thei assigned execution plan, which defines the numbe of theads to be equested fom an intenal thead pool in ode to sevice a quey. All theads petaining to a quey pocess ae then assigned to pocessing coes fo execution and ae synchonized befoe a quey can leave the system. Kaft et al. [7] captued this behavio fom ou taces to paameteize thei in-memoy database simulato. We theefoe eview this pocess biefly and subsequently extend it fo use with ou analytical model. Since ou taces contain infomation fom isolated uns fo all available quey classes, also denoted as job classes, we can estimate two impotant model paametes on a pe-class basis. In paticula, we ae consideing the sevice demand d and the paallelism l. While d accounts fo the aveage time equied by ou in-memoy system to sevice one job of class, l descibes the numbe of CPU coes used on aveage by quey class. In Section IV we want to compae ou pefomance model with FJ-AMVA and the simulato developed in [7]. Howeve, all thee appoaches equie a diffeent epesentation of d. Hence, we will show in the following how to extact d appopiately. To bette illustate this pocess, we epesent ou taces in Figue 3 by an exemplay job that consists of 7 theads and is executed on a -coe system. Figue 3 Case a shows the coe activity, which was sampled duing the execution of ou job. We see that ove time, all coes wee diffeently utilized, i.e. attibutable to stalling theads o changes in thead affinity. Based on the sampled coe activity, we divide the execution pocess of a quey into P pocessing phases, as illustated in Case b. Each pocessing phase is defined by its duation b p and its numbe of active pocessing coes c p, i.e. active coes in pocessing phase and no active coes in pocessing phase 3. As mentioned above, the extaction of pocessing phases and active coes was done in [7], since thei simulato equies the paametes b p and c p. Howeve, we extend this pocess fo use with ou analytical model and detemine d and l as aggegates of these measuements, d = P p= b p and l = d p b pc p. In addition to the coe activity, ou taces ecod the numbe of theads J petaining to a class job execution pocess as well as the execution times of each individual thead, excluding the duation in which a thead was not active. This infomation was not consideed by [7], and thus pompted us to extact it fom the aw taces. We illustate this in Figue 3 Case a, which lists all 7 theads that belong to ou exemplay job. We denote the execution time of each thead t petaining to a job of class with s t and since FJ-AMVA specifically equies this epesentation, we will use s t fo its paameteization in ou expeiments. Additionally, FJ-AMVA assumes that J I. Howeve, fo some classes and also fo ou example, with J = 7 and I =, this is not the case. Hence, we sot s t and use only the fist t I longest unning theads, shown in Case b. We justify this, as fo the majoity of classes in ou taces, whee J > I, s t.s fo t > I. This means that these theads wee not sampled accuately, since [8] used a sampling inteval of. seconds. IV. FORK-JOIN MODEL In this section, we study queueing pefomance models fo in-memoy databases based on multi-coe pocessos. In addition, we popose an efficient analytical appoximation to these and pesent all elevant notation in Table I A. Modeling an in-memoy Database Pefomance models fo in-memoy databases need to be awae of the complexity intoduced by OLAP wokloads and equie a contention model that accuately captues hadwae and application chaacteistics. Emphasized by the high level of quey paallelism shown in Figue (a), fok-join queues pove to be an appopiate choice fo modeling an in-memoy database. We theefoe apply fok-join queues to model the pocessing coes of such a system. In paticula, we conside pocesso shaing (PS) queues in the sense of Baskett et al. [9], i.e., whee sevice times ae i.i.d. geneally distibuted. We employ multiclass closed queueing netwoks (QN) with a think time model that epesents an abstaction of client think time and inte-activation times of woke theads in the database, which ae dependent on the admission buffe and thead pool size. In Figue we pesent ou queueing model. It captues the behavio of jobs split into seveal tasks on aival at the system, which ae then assigned to pocessing coes in a pobabilistic manne. This includes the synchonization aspect of paallel siblings at the join point and the etun to the think time buffe once a job is completed. Appoaches to solve this type of QNs via simulation, e.g. in [7], emphasize the difficulty in finding analytical solutions. We will theefoe discuss available appoximations to QNs, befoe we intoduce ou novel analytical esponse time coection to fok-join queues. B. Appoximations to Fok-Join Queues The widely used exact analytical solution fo closed QNs, known as mean-value analysis (MVA), detemines the e-

4 Symbol R b p, c p d i, s i l s t J N Z I i p i X i W i A i Q i U i M i TABLE I MAIN NOTATION Desciption Wokload Paametes Numbe of quey classes Length of pocessing phase p and numbe of active coes duing p Sevice demand and sevice time of class at queue i Numbe of coes used on aveage by class (aveage degee of paallelism) Sevice time of thead t of class Numbe of theads pe class Population vecto with numbe of jobs pe custome class: N,..., N R Vecto of pe-class think times Z,..., Z R Additional Paametes Numbe of available pocessing coes at seve i Pobability of class jobs being outed to queue i Pefomance Measues Pe-class thoughput at queue i Pe-class esidence time at queue i Queue length at aival instant of class at queue i Pe-class queue length at queue i Pe-class utilization of queue i Pe-class memoy utilization at seve i sponse time W i fo a job of class at queueing cente i depending on the total numbe of pe-class jobs N in a system as follows []: W i = d i ( + A i ( N) ). () Hee, the definition of W i includes the queueing time and the sevice demand d i = v i s i, the poduct of pe-class sevice time s i and visits v i. The aival instant queue A i ( N) counts fo the total numbe of jobs queuing o being seviced at i at the aival instant of a job of class. Based on the aival theoem fo closed QNs, A i ( N) can be expessed as Q i ( N ), which designates the queue length with one class job less. MVA is applied in ecusive fashion, but despite being analytical it gets intactable fo poblems with moe than a few custome classes. This is addessed by Bad-Schweitze [3], poposing an appoximate MVA (AMVA) that employs a fixed-point iteation and estimates A i via linea intepolation: A i ( N) (N ) Q i ( N R N) + Q is ( N). () s=,s Synchonization in fok-join queues intoduces tempoal delays that cannot be descibed with the above poduct-fom models. As MVA and AMVA ae not applicable in that case, moe ecent appoaches tied to addess this aspect [6], []. Alomai et al. [6] popose a esponse time appoximation called FJ-AMVA that sots pe-class esidence times in descending ode and scales them by an appopiate coefficient fo bette estimation of the synchonization ovehead. This appoach assumes s i to be the mean of the exponentially distibuted sevice times S i. It can be shown that if s i ae the same at evey queue fo a paticula class, max i (s i ) H J equals s = E[max i ( S i )], whee s becomes the maximum sevice time of a job and H J hamonic numbe fo job class. In the heteogeneous case, s i can vay acoss the queues fo each job class, which esults in less synchonization time. FJ-AMVA appoximates this by multiplying W i with /i instead of H J. Howeve, the fok-join appoximations in [6], [] ae less suitable fo ou model, as both assume exponential distibutions of s i. By contast, ou sevice times s i and s t show a geneally low vaiability. We point this out in Figue (b) and = J j= j denotes the jth Client think time p p Fok p3 Coe Coe Join Coe 6 Database Seve Fig.. Multiclass Fok-Join Queueing Model of a Database Seve (d), by listing the pe-class execution times and thei standad deviations as well as the fist 8 longest unning theads fo a subset of ou quey classes. This justifies the need fo a esponse time coection that does not ely on exponential sevice times. C. Response Time Coection Thead-level fok-join cannot be expessed with (). We theefoe popose an analytical esponse time coection called TP-AMVA, which consides the placement of tasks in fokjoin queues. We assume equal pobabilities of jobs being outed to a paticula queue and conside jobs to not cycle within the fok-join constuct. Hence, d = v s = s. In addition, we appoximate the fok-join constuct with only one single queue, which deceases pocessing time and simplifies its integation into ou optimization pogam. Ou coection has the following fom: ) R l s W = d ( + Q s δ s, (3) I s= whee the esponse time W is calculated as the sevice demand d inflated by a facto that epesents the sevice ate degadation unde pocesso shaing due to jobs, which aleady compete fo esouces at the same queue. The aival queue A s is estimated by employing Bad-Schweitze: N, s = δ s = N (), s, s,. Since we ecod thead-level infomation fo each quey class, we ae able to bette appoximate the fok-join featue. Fo this we coect A s by the facto l s /I to estimate the pe-coe queue length in a system with I coes. The pefomance measues W, thoughput X and Q can then be esolved by employing the AMVA fixed-point iteation. In addition we appoximate the utilization in a fok-join system with: R l U = U I. (5) Befoe we evaluate this model, we pesent an altenative appoximation to (3), which is an empiical calibation. It follows the idea that an aiving class job affects W depending on its pobability p being outed to a paticula queue in the fok-join constuct. Hence, we coect the class queue length Q by multiplying with p : ) R l s W = d ( + Q s δ s I p s, (6) s=

5 TABLE II RELATIVE ERROR (%) COMPARED WITH SIMULATION FOR SCENARIO S i Method S S S 3 S S 5 S 6 S 7 S 8 AMVA with (l /I) s AMVA with s FJ-AMVA pobabilistic TP-AMVA static TP-AMVA whee p s is defined as: l p s = I, s =, s, s,. We will show expeimentally that ou two appoximations poduce easonable esults unde diffeent wokload mixes and ae highly competitive compaed with FJ-AMVA. Duing ou evaluation we denote the implementation of (3) with static TP-AMVA and (6) with pobabilistic TP-AMVA. D. Evaluation In this section we will evaluate ou coection against the in-memoy database simulato in [7] unde diffeent scenaios and include the FJ-AMVA into ou compaison. a) Expeimental Setup: We implemented FJ-AMVA, and TP-AMVA in Matlab and conducted seveal expeiments fo diffeent wokload scenaios based on the categoies: light, medium and heavy. Wheeas light mixes contain mostly quey classes with small degees of paallelism and shote execution times, heavy mixes compise quey classes with high paallelism and longe execution times. To futhe vay the wokload, we inceased the numbe of concuent uses fom to 3. Thoughout all scenaios we used a fixed think time extacted fom the single use scenaio in ou tace logs. We used the following paameteization fo the simulato, TP-AMVA and FJ-AMVA. To incease its capability of captuing esouce contention moe accuately, we paameteized the simulato with the fine gained quey chaacteistics defined by b p and c p, intoduced in Section III-B. Fo TP-AMVA we used the aggegated sevice demand d. In contast, FJ- AMVA needs to be paameteized with the sevice times of jobs at each queue s i. We theefoe mapped s t, which natually epesents the sevice times needed by FJ-AMVA, onto s i. As a poblem of ou taces, thee was no infomation about the placement of theads available. Hence we addessed this by applying a Monte Calo Simulation choosing andom pemutations of s t = {s,..., s t } with t J and assigning them to queue t, t J, befoe unning FJ-AMVA. We then took the aveage esponse time of iteations, which seemed easonable to poduce stable esults. Moeove, the task scheduling system in the simulato equied equal outing pobabilities to each coe, as does ou implementation of TP-AMVA. FJ-AMVA in contast defines its outing pobabilities p as pobability that a single queue in the fok-join constuct is visited by job class. Fo this case we assumed J /I to be a suitable appoximation of p and thus we used p = J /I to paameteize FJ-AMVA. (7) Response Time in Seconds Response Time in Seconds 3 Quey Mix: Light FJ-AMVA pobabilistic TP-AMVA static TP-AMVA Sim 3 Numbe of Concuent Uses (a) Scenaio Quey Mix: Medium FJ-AMVA pobabilistic TP-AMVA static TP-AMVA Sim 3 Numbe of Concuent Uses (c) Scenaio 3 Response Time in Seconds Response Time in Seconds 8 6 Quey Mix: Medium FJ-AMVA pobabilistic TP-AMVA static TP-AMVA Sim 3 Numbe of Concuent Uses 8 6 (b) Scenaio Quey Mix: Heavy FJ-AMVA pobabilistic TP-AMVA static TP-AMVA Sim 3 Numbe of Concuent Uses (d) Scenaio Fig. 5. Response Time Results fo diffeent OLAP-based Wokload Scenaios b) Results: We show the esults of ou expeiments in Figue 5 accompanied by Table II, which depicts the mean elative esponse time eo compaed to the simulato. To give an impession of how the standad AMVA implementation () pefoms, we list its mean elative eo fo a un with d = s and a un that takes the visit atio into account by d = l /I s. As expected, AMVA clealy shows a poo oveall pefomance. In contast, both static and pobabilistic TP-AMVA pefom easonably well thoughout all scenaios and follow the tend of the simulato. Both appoximations tend to be moe pessimistic once the numbe of uses inceases, wheeas the esponse time pediction unde light load appeas to be slightly optimistic, i.e. scenaio 3. In geneal, ou pobabilistic TP-AMVA captues contention effects bette than its static vesion, staying below a % eo ate. Supisingly, FJ-AMVA lacks in its accuacy acoss most scenaios. Its esponse time pediction is too optimistic unde medium mixes, i.e. scenaio 3, and too pessimistic unde lightmedium load (scenaios and ). In most of ou scenaios, we obseved a vey pessimistic stat fo FJ-AMVA, when only few concuent uses ae active. This can be explained, when looking at the paallelism of ou quey classes. Some of ou queies, such as class, ae highly paallel with s t d, i, and thus contain almost no synchonization time. This is why the summation ove W i in FJ-AMVA, despite its scaling facto /i, esults in a esponse time that is too high. Howeve, this effect seems to diminish when the load gows and bette eflects the inceasing congestion fo those cases. Fom the esults, we conclude that FJ-AMVA in its poposed fom is not suitable fo modeling OLAP-based quey wokloads, wheeas ou coection tuns out to be easonably accuate and due to its simplistic model a good choice fo the optimization pogam we pesent in the next section.

6 In-memoy DB Cluste p p Load Dispatche Fig. 6. Model of an in-memoy Cluste Subject to Load Optimization V. OPTIMIZATION PROBLEM Given a paallel system with memoy and esouce constaints, such as an SAP HANA in-memoy database cluste, we ae inteested in how to dispatch wokloads while minimizing memoy swapping. We futhe include utilization constaints to avoid ove-povisioning in such envionments by limiting the CPU utilization pe seve. In ode to solve this constained poblem, we develop a non-linea optimization pogam based on ou appoximation pobabilistic TP-AMVA, taking above mentioned constaints into account. A. Cluste Model We conside a simplified model of an in-memoy database cluste with K seves, as depicted in Figue 6. This model contains a sepaate fok-join closed QN fo each seve, which simplifies the evaluation. Howeve, the seves in ou cluste ae not completely independent, but shae the same wokload N. We theefoe define N i = N p i, i K as the pecentage of wokload that goes to seve i, with p i designating the pobability of outing a class equest to seve i. Ou optimization pogam aims at minimizing memoy swapping, and thus minimizes the oveall memoy occupation M of an in-memoy database cluste. To educe the complexity of the optimization poblem, we employ a less complex model fo estimating M fo each seve by multiplying the peclass queue length with the pe-class physical peak memoy consumption m, (8b). We heeby make the pessimistic assumption that memoy occupation gows as a function of the queue length Q and neglect that quey classes could shae data esiding in main memoy. Additionally, we assume that foking of new theads and joining is not elated to the change of memoy consumption. B. Non-linea Optimization Poblem We minimize the memoy occupation M by finding the outing pobabilities p i to achieve nea optimal wokload placement. This esults in the optimization poblem given by Equation (8), with its decision vaiables p i, X i and W i. We integated ou pobabilistic TP-AMVA in (8f) and defined δ is = (N i )/N i (l i /I i ) fo s = and δ is = in case of s. In addition, we added memoy and utilization constaints in fom of Mi max and Ui max. Fom a pefomance point of view, ou method uses less vaiables compaed to FJ- AMVA, which would intoduce M additional binay vaiables to sot the esponse times. This gets futhe attention, when looking at the natue of ou optimization poblem, which is p3 p5 p be non-convex. Hence, we expect the numbe of local optima to gow when inceasing the numbe of classes and seves as well as intoducing diffeent constaints fo each seve. This exacebates the poblem of finding a globally optimal solution and equies stategies such as multi-stat optimization. M min = min p i,x i,w i K i= s.t.: M i = Q i m i, i M i l i U i = X i d i, i I i s= (8a) (8b) (8c) N i = p i N, i, (8d) Q i = X i W i, i, (8e) ) R l is Q i = U i ( + Q is δ is, i (8f) I Q i = N i X i Z i, W i d i, i, p i =, i (8g) (8h) (8i) p i, X i, W i i, (8j) M i Mi max, i (8k) U i Ui max, i (8l) VI. NUMERICAL RESULTS Ou goal is to get insights in how wokload placement effects the memoy occupation acoss a cluste of in-memoy databases unde given constaints. We ae futhe inteested in how the pefomance and accuacy of multi-stat based appoaches fo ou optimization poblem compae to each othe. Based on empiical evidence, we show that local optimization algoithms such as inteio-point ae able to find good solutions unde multi-stat in compaison to global optimization algoithms, such as evolution stategies. In addition, we find that local optimization algoithms delive solutions fo instances up to 8 Seves and classes in less than minutes but lack unde lage scenaios, wheeas ou evolution stategy handles up to 36 seves and classes given the same amount of time. A. Evaluation Scenaios We vaied the numbe of seve instances and classes in K, R =,, 8, 6 and the wokload N in 8K and K (light load) and 3K and K (heavy load). Ou pe-class populations N ae obtained by equally dividing N acoss all classes, allowing factional N. To investigate how R affects the total memoy occupation M, we clusteed ou set of classes with k-means (a pioi nomalized with z-scoe) acoss the thee dimensions paallelism l, sevice demand d and memoy occupation m, depicted fo R =,, 8 in Figue 7. Finally, we set diffeent constaints to affect the wokload placement: M max i M max i = 8GB, U max i = 5GB, U max i =.95 fo i K/ and =.99 fo i > K/.

7 l.5 R=.5 m d.5.5 R=.5.5 m d.5 R=8.5.5 m d Fig. 7. Nomalized k-means Clustes fo diffeent Numbes of Quey Classes B. Solution Methods We compae the minimization of memoy swapping fo thee diffeent methods (local and global): yp: YALMIP [] employing fmincon and the inteio point algoithm fm: fmincon configued with the inteio point algoithm es: (µ + λ) - evolution stategy [] Both methods fm and es, call ou extenal solve to appoximate the esponse time W i and thoughput X i. As ou methods ely on AMVA, which suppots only cases with N i, we applied the appoximation poposed in [3], setting Q i := when N i <. Unde method yp we had to avoid the indiect division by p i in δ is and intoduced an additional optimizing vaiable. With method es we implemented an evolutionay algoithm called evolution stategy. In contast to genetic algoithms, which epesent solution candidates by a sting of bits, evolution stategies employ a eal valued encoding of solution candidates and theefoe equie self-adaptive mutation functions. In paticula, we implemented a (µ + λ) evolution stategy (es), configued with a unifom paent selection and a best selection as envionmental selection. Individuals (i.e. solution candidates) ae epesented by the outing pobability matix p i. On a population of µ = paent individuals we pefom a unifom selection times to poduce λ = mutants in each geneation. In paticula, we had to develop ou own adaptive mutation to alte individuals obeying the constaints (8i) and (8j). To ceate a mutant we andomly select a seve x and a class y and subsequently modify the outing pobability p xy by adding a new value p. We choose p unifomly fom the inteval [p xy p xy / ((g mod ) + ), p xy + ( p xy ) / ((g mod ) + )]. The amount of change intoduced by this mutation in fom of p is adapted thoughout the optimization depending on the cuent numbe of geneations g. Moeove, we detemine the fitness of each individual by its memoy occupation M. To ensue compliance with the given constaints, ou implementation penalizes violations of (8k),(8l) by impaiing the fitness. Finally, fom the individuals afte mutation, the fittest ae selected into the next geneation. C. Evaluation Methodology We implemented ou appoaches in MATLAB employing fmincon and its inteio-point algoithm fo yp and fm,, while es elies on an evolution stategy. Moeove, method es does not depend on MATLAB popietay toolbox functions, and theefoe could be implemented in anothe pogamming language to futhe decease pocessing time. TABLE IV MEMORY OCCUPATION AND EXECUTION TIMES. TIMEOUT: 8S. Instances Memoy Occupation in GB Time in s K R N/K fm es es-fs fm es es-fs N/A timeout N/A timeout fm,es wee stopped afte 8s o in case of fm when P = solutions found No single solution found by fm Ou scenaios wee evaluated on an Intel Coe i5 CPU with.6ghz and two physical coes. To cope with diffeent local optima, we andomized P = initial points fo evey tuple (K, R, N/K) and an fmincon using MATLAB s MultiStat solve. Subsequently, we epot the aveage of the cumulative execution time fo all P local solve uns at a timeout of 8 seconds to undestand the pefomance at shot time scales. D. Results We show the esults of ou optimization pogam fo all thee methods (yp,fm,es) in Table III and Table IV. We futhe epot the cumulative execution time fo yp and fm and the time until the fist non-violating solution, es-fs, was found by es. At fist, we wee inteested in how clusteing of classes affects memoy optimization. In ou case we obseve an invesely popotional behavio of the memoy occupation M once the numbe of classes R inceases, i.e. instances (,,8) and (,8,8). This can be explained by the way we calculate N fom the fixed atio N/K, but compaing the instances (,,3) and (,8,3), whee N = 8 fo both cases, we see that a lage diffeence in M emains. We also discove non-monotonicity in the oveall cluste utilization U with inceasing R. We show this in Table V, fo a light and heavy load scenaio. We impose this on classes with high paallelism and long execution times that ae less often meged into a cluste with shot unning and sequential queies when R inceases, as depicted in Figue 7. Fom this we conclude that the moe classes ae aggegated, the moe inaccuate gets the estimate fo M and U, given that the atio N/K emains constant. Anothe question we wanted to addess is how ou optimization pogam handles wokload placement unde the given constaints Mi max and Ui max defined in section VI-A. We theefoe investigated the two instances (8,,8) and (8,,3) in moe detail, which epesent light and medium load scenaios. Table VI shows the outing pobabilities p i and pe-seve

8 TABLE III MEMORY OCCUPATION IN GB AFTER OPTIMIZATION AND EXECUTION TIMES (SECONDS). TIMEOUT: 8S. Instances Memoy Occupation M U #Success Time K R N/K yp fm es fm-wl es-fs util yp fm yp fm es-fs N/A N/A N/A N/A N/A N/A.56 N/A timeout timeout N/A N/A N/A timeout timeout.797 memoy occupation M i fo the two solutions found by fm. Unde light load we see that fm ties to achieve as little intefeence as possible between the two classes, esulting in max i (M i ) = 5.GB. Moeove, as no constaints ae violated, the placement can be an abitay pemutation acoss i, but needs to emain fixed fo. Once the wokload gows to N i = 8, which is a nomal scenaio fo SAP HANA dealing with moe than 8 paallel connections, the memoy constaints fo seve 5-8 ae violated. At this point we obseve a wokload shift towads seves -, occupying 5.9GB on each. This suggests that ou appoach is able to optimize constained wokload placement easonably well. All of ou methods (yp,fm,es) poduce simila esults egading M fo instances whee all P solve uns completed successfully, i.e. (,,). We explain this due to the same algoithm that is used to solve the queueing models. Though theoetically identical yp applies the inteio-point algoithm, wheeas fm and es depend on a fixed-point iteation, on which we impose the slightly diffeent esults between yp and fm. In contast, we expect a highe computational cost fo fm due to the extenal solve call. Howeve, duing ou expeiments we obseved that despite slightly moe efficient code poduced by YALMIP, yp needed moe iteations to convege than fm at same toleances levels fo the stopping citeia. Looking at the execution times, we see that yp and fm ae able to find solutions in less than 8 seconds fo seves and classes, if not necessaily fo all P initial conditions. Ou Evolution Stategy seems moe efficient, as it epots the fist solution afte 7.9 seconds in scenaios such as (6,8,), whee yp and fm wee timed out befoe a single successful solution was found. We explain this due to the implementation of es, which involves less evaluations of the objective function pe iteation. Due to the inceasing numbe of decision vaiables in lage scenaios, yp equied up to MB of main memoy unde (6,8,). By contast, ou extenal solve based methods fm and es equie only a few megabytes. Concluding the esults, TABLE V AVERAGE SERVER UTILIZATION FOR 8-SERVER SCENARIO (K=8) N/K=8 N/K=3 R= R= R=8 R= R= R= we showed that both techniques inteio point based methods and evolution stategies can be effectively used to solve ou constained memoy optimization poblems. VII. RELATED WORK Reseach into in-memoy database pefomance stated in when [6] intoduced fundamental cost models including the entie memoy hieachy in a database system. Nowadays, on-demand povisioning of these systems dives eseach futhe into database optimization employing QNs [7]. In [8] classification-based machine leaning is used to schedule tenants in multi-tenant databases. The authos chaacteize tenant and node-level behavio based on pefomance metics collected fom database and OS laye and validate thei famewok in a PostgeSQL envionment. Howeve, in this wok scheduling constaints ae only appoximated. Wokload chaacteization and esponse time pediction via non-linea egession techniques fo in-memoy databases ae poposed in [9]. The authos deive tenant placement decisions by employing fist fit deceasing scheduling, but evaluate on small scale only. [] popose a new famewok fo managing pefomance SLOs unde multi-tenancy scenaios. Thei wok combines mathematical optimization and boolean functions to enable what-if analyses egading SLOs, but elies on bute foce solves and ignoes OLAP wokloads. In [] quey demands ae quantified by a fine-gained CPU shaing model including lagest deficit fist policies and a deficit-based vesion of ound obin scheduling. The methodology applies to database-as-a-sevice platfoms and is validated on a pototype of Micosoft SQL Azue. This wok neglects chaacteistics fo memoy occupation. [], [3] intoduce famewoks fo non-linea cost optimization egading SLA violations and

9 TABLE VI WORKLOAD PLACEMENT UNDER SCENARIOS (8,,8) AND (8,,3) N/K i= i= i=3 i= i=5 i=6 i=7 i=8 7 p i p i M i.6gb 5.GB 5.GB.6GB 5.GB 5.GB 5.GB 5.GB 8 p i p i M i 5.9GB 5.9GB 5.9GB 5.9GB 8.GB.5GB 8.GB.5GB esouce usage, applied to web sevice based applications and cloud databases. Thei wok eithe elies puely on constaint definitions o does not conside closed QNs. [] poposes a famewok fo multi-objective optimization of powe and pefomance. The methodology applies to softwae-as-a-sevice applications and it is validated using a commecial softwae, SAP ERP. The appoach is based on simulation and does not conside thead-level fok-join. [5] [7] use multi-vaiate egession and analytical models of closed QNs to pedict quey pefomance based on logical I/O intefeence in multi-tenant databases. Howeve, these methods equie detailed quey access patten and ae evaluated fo small numbes of jobs and batch wokloads only. Ignoed by the latte, thead-level fok join is addessed by [7] and [6], but despite using simila techniques, thei appoaches ae eithe computationally expensive o ely on exponential sevice time distibutions. VIII. CONCLUSIONS AND FUTURE WORK In this pape we have made seveal contibutions, which include a novel analytic esponse time appoximation that models thead-level fok join and pe-class memoy occupation in in-memoy systems. In addition, we have developed an optimization based fomulation that facilitates ou analytic appoximation and efficiently seeks fo load dispatching outing pobabilities to minimize memoy swapping in inmemoy database clustes. Futhemoe, we have shown that ou models exceed the accuacy of existing appoaches using eal taces fom a commecial in-memoy database appliance, SAP HANA, fo validation. Diections fo futue wok include a moe sophisticated model to estimate memoy consumption as well as a poof of concept fo ou models though thoough expeimentation and validation with diffeent hadwae configuations. In addition, we plan to implement ou povisioning famewok in a eal in-memoy database system and extend it by new featues to include shaed memoy access and multi-tenancy. REFERENCES [] SAP HANA Pefomance: Efficient Speed and Scale- Out fo Real-Time Business Intelligence, 3. [Online]. Available: [] J. Schaffne, B. Eckat, C. Schwaz, J. Bunnet, D. Jacobs, and A. Zeie, Towads Analytics-as-a-Sevice Using an In-Memoy Column Database, in New Fonties in Infomation and Softwae as Sevices SE -, se. Lectue Notes in Business Infomation Pocessing, D. Agawal, K. Candan, and W.-S. Li, Eds. Spinge Belin Heidelbeg,, vol. 7, pp [3] P. J. Schweitze, Appoximate analysis of multiclass closed netwoks of queues, in Poc. of the Int l Conf. on Stoch. Contol and Optim., Amstedam, 979, pp [] F. Fäbe, S. K. Cha, J. Pimsch, C. Bonhövd, S. Sigg, and W. Lehne, SAP HANA database: data management fo moden business applications, SIGMOD Rec., vol., no., pp. 5 5, Jan.. [5] J. Rolia, G. Casale, D. Kishnamuthy, S. Dawson, and S. Kaft, Pedictive Modelling of SAP ERP Applications: Challenges and Solutions, in Poceedings of the Fouth Intenational ICST Confeence on Pefomance Evaluation Methodologies and Tools, se. VALUETOOLS 9, 9, pp. 9: -9:9. [6] F. Alomai and D. Menascé, Efficient Response Time Appoximations fo Multiclass Fok and Join Queues in Open and Closed Queuing Netwoks, p., 3. [7] S. Kaft, G. Casale, A. Jula, P. Kilpatick, and D. Gee, WIQ: Wok- Intensive Quey Scheduling fo In-Memoy Database Systems, in Poc. of IEEE CLOUD,. [8] A. Jula, Multicoe scalability and NUMA effects on HDB with SAP-H data and TPC- H queies, SAP, Tech. Rep.,. [9] F. Baskett, K. M. Chandy, R. R. Muntz, and F. G. Palacios, Open, Closed, and Mixed Netwoks of Queues with Diffeent Classes of Customes, Jounal of the ACM, vol., no., pp. 8 6, 975. [] E. Lazowska, J. Zahojan, G. Gaham, and K. Sevcik, Quantitative System Pefomance: Compute System Analysis Using Queueing Netwok Models. Pentice Hall, 98. [] E. Vaki, Response time analysis of paallel compute and stoage systems, IEEE Tansactions on Paallel and Distibuted Systems, vol., no., pp. 6 6,. [] H.-G. Beye and H.-P. Schwefel, Evolution stategies A compehensive intoduction, Natual Computing, vol., no., pp. 3 5,. [3] R. Sui and R. Shinde, Using Factional Numbes of Customes to Achieve Thoughputs in Queueing Netwoks, in Poceedings of the Industial Engineeing Reseach Confeence (IERC), 5. [] J. Loefbeg, Yalmip : A toolbox fo modeling and optimization in MATLAB, in Poc. of CACSD,. [5] K. Molka and G. Casale, Memoy-Awae Sizing fo In-Memoy Databases, Dept. of Computing, Impeial College London, Tech. Rep.,. The technical epot can be made available to the eviewes upon equest to the chai. [6] S. Manegold, P. Boncz, and M. L. Kesten, Geneic database cost models fo hieachical memoy systems, in Poceedings of the 8th intenational confeence on Vey Lage Data Bases, se. VLDB,, pp. 9. [7] R. Osman and W. J. Knottenbelt, Database System Pefomance Evaluation Models: A Suvey, Pefomance Evaluation, vol. 69, no., pp. 7 93,. [8] A. J. Elmoe, S. Das, A. Puche, D. Agawal, A. El Abbadi, and X. Yan, Chaacteizing tenant behavio fo placement and cisis mitigation in multitenant DBMSs, in Poceedings of the 3 Intenational Confeence on Management of Data, se. SIGMOD 3, 3, pp [9] J. Schaffne, B. Eckat, D. Jacobs, C. Schwaz, H. Plattne, and A. Zeie, Pedicting In-Memoy Database Pefomance fo Automating Cluste Management Tasks, in IEEE 7th Intenational Confeence on Data Engineeing,, pp [] W. Lang, S. Shanka, J. M. Patel, and A. Kalhan, Towads Multi-tenant Pefomance SLOs, in IEEE 8th Intenational Confeence on Data Engineeing,, pp [] S. Das, V. R. Naasayya, F. Li, and M. Syamala, CPU Shaing Techniques fo Pefomance Isolation in Multitenant Relational Database-asa-Sevice, in VLDB, 3, pp [] J. Almeida, V. Almeida, D. Adagna, C. Fancalanci, and M. Tubian, Resouce Management in the Autonomic Sevice-Oiented Achitectue, in 6 IEEE Intenational Confeence on Autonomic Computing, 6, pp [3] J. Roges, O. Papaemmanouil, and U. Cetintemel, A geneic autopovisioning famewok fo cloud databases, in IEEE 6th Intenational Confeence on Data Engineeing Wokshops (ICDEW),. [] H. Li, G. Casale, and T. Ellahi, SLA-diven planning and optimization of entepise applications, in Poceedings of the fist joint WOSP/SIPEW,. [5] J. Duggan, U. Cetintemel, O. Papaemmanouil, and E. Upfal, Pefo-

10 mance pediction fo concuent database wokloads, in Poceedings of the intenational confeence on Management of data, se. SIGMOD,, pp [6] W. Wu, Y. Chi, H. Hacigümüs, and J. F. Naughton, Towads Pedicting Quey Execution Time fo Concuent and Dynamic Database Wokloads, PVLDB, vol. 6, no., pp , 3. [7] R. Sui, S. Sahu, and M. Venon, Appoximate Mean Value Analysis fo Closed Queuing Netwoks with Multiple-Seve Stations, in Poc. of the 7 Industial Engineeing Reseach Confeence, 7.