Model Predictive Control for Insulin Administration in People with Type 1 Diabetes

Transcription

1 Model Predictive Control for Insulin Administration in People with Type 1 Diabetes Miriam MN Nærum Kongens Lyngby 2010 IMM-BSC

2 Technical University of Denmark Informatics and Mathematical Modelling Building 321, DK-2800 Kongens Lyngby, Denmark Phone , Fax

3 Summary People with type 1 diabetes are unable to regulate their blood glucose concentration because they do not produce insulin. The non-linear Hovorka model is used to simulate the responses of blood glucose concentration, when meals are consumed and insulin is injected subcutaneously. First the theory of the Hovorka model is presented, by describing the three parts of the model: Carbohydrate absorption from meals, subcutaneous insulin injection and absorption, and the gluco-regulatory system. Then model predictive control (MPC) is described. MPC-based algorithms attempt to regulate the blood glucose concentration by manipulating the insulin infusion rate based on predictions of future glucose trends. MPC can use three different prediction models: ARX, ARX, and E ARX. The theory of these models is examined. Futhermore the theory of a possible improvement of MPC, asymmetric penalty function MPC, is derived. The Hovorka model is used to calculate the appropriate basal insulin infusion rate. This is the delivery rate of insulin that results in a steady-state glucose concentration at normaglycemia, the optimal glucose concentration level. From the calculation it is observed that the basal insulin infusion rate is linearly dependent on the weight of the patient. Next, the Hovorka model is used to inspect the importance of the duration of meals. It is evident that the glucose concentrations obtained, when a meal is consumed over long and short durations, are almost the same. In order to obtain a realistic view of the Hovorka model, simulations are made with meals from a typical day. First insulin boluses found with trial and error are used, and then simulations are made using MPC.

4 ii Subsequently simulations are performed to analyse the influence of changes in insulin sensitivities in the Hovorka model. If no further actions are taken, changes in insulin sensitivities result in very large changes in glucose concentration. However, MPC using ARX or E ARX with a suitable tuning parameter are able to adjust the insulin infusion rate to offset the changes. An insulin injection directly into a vain gives a short absorption time. A simple proportional derivative (PD) controller can be used in this case, since the insulin absorption time is smaller than the meal absorption time. This controller appears to be unsuitable for subcutaneous insulin injection, however. Lastly, simulations are made to investigate how MPC can be used in connection with meals. It appears that MPC using ARX handles meal disturbances well, while MPC using ARX is effective when dealing with changes in insulin sensitivities. MPC using E ARX is a good compromise that handles both meal disturbances and insulin sensitivities changes.

5 Resumé Patienter med type-1 diabetes er ikke i stand til at regulere deres blodsukker, fordi de ikke kan producere insulin. Den ulinære Hovorka-model bruges til at simulere reaktionerne i kroppen, når et måltid indtages, og insulin indsprøjtes subkutant. Først gennemgås teorien om Hovorka-modellen ved en beskrivelse af de tre dele af modellen: Kulhydratabsorbering, subkutan insulinabsorbering og glukoseregulering. Derefter beskrives modelprædiktiv regulering (MPC), der kontrollerer insulinadministrationen ved hjælp af linære forudsigelser. MPC kan bruge tre forskellige modeller til forudsigelse: ARX, ARX, og E ARX. Teorien bag disse modeller gennemgås. Endvidere udledes en mulig udvidelse til MPC, asymmetric penalty function MPC. Dernæst bruges teorien bag Hovorka-modellen til at beregne den grundliggende insulintilførsel. Det er det insulinniveau, der skal til, for at opnå det optimale glukosekoncentrationsniveau. Af beregningerne viser det sig, at den grundliggende insulintilførsel er lineært afhængig af patientens vægt. Herefter bruges Hovorka-modellen til at undersøge betydningen af et måltids varighed. Det viser sig, at den glukosekoncentration, der opnås under et langvarigt måltid, er stort set den samme som under et kortvarigt. For at få en realistisk opfattelse af Hovorka-modellen, udføres simuleringer ud fra en typisk dags måltider. Ved at prøve sig frem findes der insulinmængder, der gives i impulser. Herefter foretages simuleringer ved hjælp af MPC. Dernæst undersøges betydningen af ændringer i insulinfølsomhederne i Hovorkamodellen ved simuleringer. Disse ændringer påvirker glukosekoncentrationen

6 iv meget, hvis der ikke gøres noget for at forhindre det. Det viser sig, at MPC med ARX eller E ARX med passende valg af tuningparameter kan bruges til at justere insulintilførslen. Forholdene ved en intravenøs insulintilførsel simuleres med en simpel proportional derivative (PD) controller. Ved en sådan insulintilførsel indsprøjtes insulinen direkte i en vene, så absorberingen går hurtigere. En sådan PD controller viser sig at være uegnet til subkutan indsprøjtning. Til slut foretages simuleringer for at undersøge brugen af MPC i forbindelse med måltider. Det viser sig, at MPC sammen med ARX er velegnet til at imødegå påvirkningen fra måltider. Til gengæld er ARX egnet til behandling af ændringer i insulinfølsomhederne. Et godt kompromis, som kan håndtere både måltider og ændringer i insulinfølsomhederne, er MPC med brug af E ARX.

7 Preface The aim of report is to analyze the use of model predictive control (MPC) to automate the insulin infusion rate for type 1 diabetes patients. It is written as a part of the bachelor project: "Model Predictive Control for Insulin Administration in People with Type 1 Diabetes". The bachelor project is made at Department of Informatics and Mathematical Modelling, DTU. Submission date is Lyngby, June, 2010 Miriam MN Nærum

8 Variables and Abbreviations Variables α α d d(t) ɛ(t) EGP 0 f(z(t), d(t), u(t)) G(t) I(t) λ N φ q j Q 1 (t) Q 2 r S ID S IE S IT t τ D τ S u(t) u basal Tuning parameter Tuning parameter Meal disturbance (g CHO/min) White normally distributed noise Amount of glucose produced by the liver at zero insulin level (mmol/min) Change in Hovorka states glucose concentration in the blood (mmol/l) Blood insulin concentration (mu/l) Damping parameter Number of predictions Objective function in optimization problem Shifting parameter Glucose in the blood stream (mmol) Amount of glucose in peripheral tissue (mmol) Setpoint (mmol/l) Insulin sensitivity of disposal (L/mU) Insulin sensitivity of endogeneous glucose production (L/mU) Insulin sensitivity of glucose transport and distribution (L/mU) Time (min) Glucose absorption time constant (min) Insulin absorption time constant (min) Insulin infusion rate (mu/min) Basal insulin infusion rate (mu/min)

9 vii U G (t) x 1 (t) x 2 (t) x 3 (t) y(t) z(t) Glucose absorption rate (mmol/min) Insulin action that influences glucose transport and distribution State depicting the insulin action of glucose disposal State depicting the insulin action on endogeneous production of glucose in the liver Glucose concentration (mmol/l) Vector containing all the states in the Hovorka model Abbreviations ARX ARX E ARX EGP MISO MPC P PD SISO Exogenous-input auto regressive exogenous auto regressive Extended exogenous auto regressive Endogenous glucose production Multi-input, single-output Model predictive control Proportional Proportional-derivative Single-input, single-output

10 Acknowledgements I thank my supervisors John Bagterp Jørgensen and Daniel Aaron Finan. I also thank Dimitri Boiroux and Jakob Kjøbsted Huusom.

11 ix

12 Contents Summary Resumé Preface Variables and Abbreviations Acknowledgements i iii v vi viii 1 Introduction Hypothesis Methodology The Hovorka Model Model Predictive Control Simulation Results and Analysis Basal Insulin Infusion Rate Simulations with Meals Simulations of a Typical Day Sensitivity Analysis Intravenous Insulin Infusion Model Predictive Control Simulations Discussion 76 5 Conclusion 78

13 CONTENTS xi A MatLab code for Methodology 80 A.1 The Hovorka Model A.2 Model Predictive Control B MatLab code for Simulation Results and Analysis 86 B.1 Basal Insulin Infusion Rate B.2 Simulations with Meals B.3 Simulations of a Typical Day B.4 Sensitivity Analysis B.5 Intravenous Insulin Infusion B.6 Model Predictive Control Simulations

14 Chapter 1 Introduction People with type 1 diabetes are not able to produce sufficient amounts of insulin themselves, contrary to people without diabetes who produce insulin in the pancreas. Insulin is a hormone that is crucial to the regulation of glucose concentration in the blood, and without it the blood glucose level is often too high (hyperglycemia), which can cause long-term problems like eye, nerve, kidney diseases, and strokes [6] and [14]. Low blood glucose (hypoglycemia) can, on the other hand, cause the patient to fall into coma or have other immediate consequences. Therefore insulin must be injected to regulate the blood glucose concentration. When insulin is injected, the blood glucose concentration decreases. Therefore it is important to inject a suitable amount. If too much insulin is injected, the blood glucose concentration decreases too much, and the patient goes into hypoglycemia. The blood glucose concentration is also affected when the patient consumes a meal. The carbohydrates from the meal are absorbed by the body and are converted into glucose in the blood. This causes the blood glucose concentration to increase. Therefore it is important to inject insulin in a suitable amount, when a meal is consumed. When insulin is injected glucose is absorbed by adipose tissue, e.g. muscles and fat. The insulin also makes the liver absorb glucose as glycogen [3].

15 2 In this project a non-linear model, the Hovorka model, is used to simulate what happens in the body of the diabetes patient, when meals are consumed and insulin is injected subcutaneously. The model is used to gain information about the behaviour of the body to help find a suitable way of administrating insulin injections. Insulin can be injected with an insulin pen or an insulin pump. The insulin pen is typically used for injecting insulin boluses at meal times, while the insulin pump is used for injecting insulin almost continually with an injection, e.g., every 5 minutes. The glucose concentration can be measured with a continuous glucose monitor. These measurements can be used by a control algorithm to calculate how much insulin the insulin pump should infuse at the current time measure. This calculation can be done using model predictive control (MPC). MPC uses the glucose concentration and meal input to predict future glucose concentrations. Typically, predictions are made with a linear model. If a high glucose concentration is predicted, more insulin is injected early enough to decrease the glucose concentration to a proper value at the predicted time. MPC predicts the glucose concentration using the insulin infusion rate from only the current time sample. The optimal insulin infusion rate should then be found to get the best possible predictions. This is done by setting up a quadratic optimization problem. This problem has a set of linear inequality constraints, which confines the insulin infusion rate and the difference in insulin infusion rate between consecutive time samples. The model predictions can be made using an exogenous autoregressive (ARX) model, ARX, or extended ARX (E ARX). Computer simulations have been made by [12] and [11]to investigate the possibility of using MPC for calculating the optimal insulin infusion rate. Futhermore, non-linear MPC has been used for an artificial beta-cell in [3] and [5]. The report is structured as follows: At chapter 2 the theory of the Hovorka model and MPC is described. Chapter 3 consists of 6 sections that contain the results of the various simulations used for the analysis as described above. The discussion can be read in chapter 4, and finally the conclusions are listed in chapter 5.

16 1.1 Hypothesis Hypothesis The purpose of this project is to investigate whether model predictive control (MPC) can be used to find an insulin infusion rate for a type I diabetes patient, that reduces the risk of developing diseases from having too high or too low blood glucose concentration levels.

17 Chapter 2 Methodology The methodology of the Hovorka model and model predictive control (MPC) is described. The Hovorka model consists of three two compartment models: Carbohydrate absorption from meals, subcutaneous insulin injection and absorption, and the gluco-regulatory system. MPC uses linear prediction to administer the insulin infusion rate. 2.1 The Hovorka Model The Hovorka model [5] simulates a person with type 1 diabetes. The model is summarized in Fig It has two inputs, meal disturbances and insulin infusions, and simulates the glucose and insulin concentration in the body in response to these inputs.

18 2.1 The Hovorka Model 5 Gluco-Regulatory System CHO Absorption F R F 01 G(t) d(t) D 1 U G D 2 Q 1 Q 2 EGP SC Insulin Absorption x 3 x 1 x 2 u(t) S 1 S 2 U I I I(t) Figure 2.1: Diagram over the Hovorka model Meals The meal input, d(t) is the amount of carbohydrate (CHO) per minute eaten by the virtual subject. The absorption of the meal is simulated using a two compartment model, that describes the absorption of the carbohydrate through meals:. First the carbohydrate input rate in mmol/min is calculated as [5]: D(t) = 1000 d(t) M wg Where d(t) is the meal input in g CHO/min, D(t) is the meal input in mmol/min and M wg is the molecular weight of glucose. Now the glucose absorption rate can be calculated, using that the model describes the following dynamics between the meal input and the states D 1 (t) (mmol) and D 2 (t) (mmol) in the respective compartments [5]: dd 1 (t) dt dd 2 (t) dt = A G D(t) D 1(t) τ D = D 1(t) τ D D 2(t) τ D (2.1a) (2.1b) Where A G is the carbohydrate bioavailability parameter, and is set to 0.8. From the mass balance on the second compartment, the glucose absorption rate can be calculated using the time constant, τ D = 40 min. This constant is an indication of how long it takes from the time the meal is consumed until it has been digested

19 2.1 The Hovorka Model 6 Figure 2.2: The glucose absorption as a function of time. Each of the functions has a meal at t = 0, of 10, 20, and 30 g CHO/min, repectively. Due to the glucose absorption constant, τ D they all peak at t = 40 min. and enters the blood stream as glucose. The glucose absorption rate is then [5]: U G (t) = D 2(t) τ D (2.2) Where U G (t) (mmol/min) is the glucose absorption rate. The importance of the glucose absorption constant shows in Fig. 2.2, where the glucose absorption rate is plotted as a function of time. The responses shown are for a meal at time t = 0, of 10, 20, and 30 g CHO/min corresponding to 50, 100, and 150 g CHO in total, repectively. It can be observed that the responses all peak at τ D = 40 min Glucose Once the meal related glucose has been absorbed it can occupy two compartments that represent the glucose subsystem, denoted by Q 1 (t) and Q 2 (t). Q 1 (t) (mmol) is the glucose in the blood stream and Q 2 (t) (mmol) indicates the amount of glucose in the peripheral tissue, e.g. in the muscles. The dynamics between these two compartments are stated in two mass balances. The first compartment Q 1 (t) includes contributions from a number of variables. First of

20 2.1 The Hovorka Model 7 all it depends on U G (t), which as mentioned before represents the glucose absorption rate from the meals. The glucose in the blood stream is also dependent on: F 01,c (t) (mmol/min) states how much glucose is used by the central nervous system (CNS). F R (t) (mmol/min) is the amount of glucose excreted by the kidneys. x 1 (t) is the insulin action that has influence on glucose transport and distribution. k 12 (1/min) is the transfer rate from the blood to the tissues. EGP 0 (mmol/min) is the amount of glucose produced by the liver at zero insulin level. x 3 (t) is the state depicting the insulin action on endogenous production of glucose in the liver. Q 2 (t) is dependent on x 1 (t), k 12, and x 2 (t), which is the state depicting the insulin action of glucose disposal. The glucose subsystem, then, is represented by [8]: dq 1 (t) dt dq 2 (t) dt = U G (t) F 01,c F R (t) x 1 (t)q 1 (t) + k 12 Q 2 (t) + EGP 0 (1 x 3 (t)) (2.3a) = x 1 (t)q 1 (t) (k 12 + x 2 (t))q 2 (t) (2.3b) In this model the glucose concentration, G(t) (mmol/l), in the blood can be found using that the glucose distribution volume, V G, is dependent on the body weight: G(t) = Q 1(t) (2.4) V G The glucose consumption of the central nervous system, F 01,c (t) is at high glucose concentration equal to, and at low concentration proportional to, the glucose consumption independent of insulin, F 01, using the following model: { F01 G(t) 4.5 mmol/l F 01,c = F 01G(t) 4.5 otherwise The production of glucose in the kidneys is zero for low glucose concentration and dependent on the glucose concentration when it is high [8]: { 0.003(G(t) 9)VG G(t) 9 mmol/l F R (t) = 0 otherwise

21 2.1 The Hovorka Model Insulin The most direct way for the body to recieve exogenous insulin is intravenously, meaning directly into the blood stream, because this method of delivery has no time delay. However, this is not practical due to its invasive nature, and is unsuitable for the patients [11] and [12]. It is much safer and more practical to give the insulin subcutaneously, meaning injecting the insulin under the skin. The downside of this method is that there is a large time constant associated with subcutaneous-to-intravenous absorption, estimated at τ S = 55 min. This is higher than the glucose arbsorption constant, τ D = 40 min. This means that the insulin will reach the blood stream more softly than the glucose, so a controller will always react with some delay. As with the meals, the insulin absorption is simulated using a two compartment model [8]: ds 1 (t) dt ds 2 (t) dt = u(t) S 1(t) τ S = S 1(t) τ S S 2(t) τ S (2.5a) (2.5b) Here u(t) (mu/min) is the amount of injected insulin. S 1 (t) (mu) and S 2 (t) (mu) are the amounts of insulin in the two compartments. From this the insulin absorption rate, U I (t) (mu/min) can be calculated: U I (t) = S 2(t) τ S (2.6) Insulin Actions The insulin concentration, I(t) (mu/l), is found by solving the following differential equation: di(t) dt = U I(t) V I k e I(t) (2.7) Here V I (L) is the insulin distribution volume and k e (1/min) is the insulin elimination rate. As earlier mentioned the three insulin dependent states influence transport and distribution, glucose disposal and endogenous production of glucose in the liver. They are calculated using the following three differential equations, depending

22 2.1 The Hovorka Model 9 on only the plasma insulin concentration and parameters [8]: dx 1 (t) dt dx 2 (t) dt dx 3 (t) dt = k a1 x 1 (t) + k b1 I(t) (2.8a) = k a2 x 2 (t) + k b2 I(t) (2.8b) = k a3 x 3 (t) + k b3 I(t) (2.8c) The constants k ai, i = 1, 2, 3 have been experimentally estimated [8] and the additional constants are calculated from the three corresponding insulin sensitivities, S IT, S ID and S IE [8]: k b1 = S IT k a1 = L mu 1 min 1 (2.9a) k b2 = S ID k a2 = L mu 1 min 1 (2.9b) k b3 = S IE k a3 = L mu 1 min 1 (2.9c) These sensitivities will vary from person to person and depend on what the person is doing. For example, in general the insulin sensitivities are influenced by physical activity and stress [7] Parameters The constants mentioned in the sections above are summarized for a typical person in Table 2.1 [8]. Four of these are dependent on the weight of the patient: V I, V G, EGP 0 and F 01. These are therefore stated in the Table only for one kg and the actual values can be calculated by multiplying the Table value with the weight of the patient, BW.

23 2.1 The Hovorka Model 10 Table 2.1: Hovorka model parameters [8] Symbol Value Unit Explanation k min 1 Transfer rate k a min 1 Deactivation rate k a min 1 Deactivation rate k a min 1 Deactivation rate k e min 1 Insulin elimination rate τ D 40 min CHO absorption constant τ S 55 min Insulin absorption constant A G 0.8 CHO to glucose utilization V I /BW 0.12 L kg 1 Insulin distribution volume V G /BW 0.16 L kg 1 Glucose distribution volume EGP 0 /BW mmol kg 1 min 1 Liver glucose production at zero insulin F 01 /BW mmol kg 1 min 1 Insulin independent CNS consumption S IT L/mU Insulin sensitivity of transport/distribution S ID L/mU Insulin sensitivity of disposal S IE L/mU Insulin sensitivity of EGP

24 2.2 Model Predictive Control Model Predictive Control Model predictive control (MPC) predicts future glucose concentrations from a model of the system. This model is a linear approximation of the system and predicts the future glucose concentration using only measurements of the glucose concentration and the insulin infusion rate from previous time measures. Given the measured glucose concentration the innovation error is calculated. To find the optimal insulin infusion rate profile for a number of near-future control actions, an optimization problem is stated such that the future glucose concentrations are predicted using the linear model. At each time measure a new optimization problem is solved, such that the predictions are recalculated due to the new glucose concentration measurement. The first value of the calculated insulin infusion rates is implemented (i.e., injected), the next glucose concentration measure is obtained, and the process repeats itself. The optimization problem is constrained such that the insulin infusion rate cannot be negative or too high to be practibly obtainable. It is also, for practical reasons, constrained how much the insulin infusion rate will be able to change between consecutive time measures. A damping parameter is also introduced to control how much the insulin infusion rate is allowed to change between time measures [9]. Here three different linear models are used for the predictions: The Exogenousinput Autoregressive model (ARX), the Exogenous-input Autoregressive model ( ARX), and the Extended Exogenous-input Autoregressive model (E ARX). The Exogenous-input Autoregressive model (ARX) uses previous glucose concentration values, the current and previous insulin infusion rates and normally distributed noise to calculate the current and future glucose concentration. This model can be used to calculate predictions of the glucose concentration by expecting the future error to be zero. This model can be expanded to include more inputs than the insulin infusion rate, e.g. meal disturbance. The Exogenous-input Autoregressive model ( ARX) is a variation of the ARX model. This is an integrated version of ARX, that can reject unmeasured step disturbances. This is useful when a parameter of the Hovorka model changes [10]. The Extended Exogenous-input Autoregressive model (E ARX) is a mixture of the two previous models. Using a tuning parameter it balances the advantages of the two models. The first model is good at rejecting meal disturbances, but

25 2.2 Model Predictive Control 12 cannot reject unmeasured step disturbances. On the other hand the ARX model is able to reject unmeasured step disturbances, but cannot reject meal disturbances. The E ARX uses the tuning parameter to weigh the advantages of the two other models [9]. The Hovorka model is based on physical prinicpals and gives a good non-linear approximation. As MPC uses a linear model to calculate glucose concentration predictions, it will be inadequate compared to the non-linear Hovorka model (and, for that matter, the non-linear reality). The linear approximation in MPC is used when calculating the predictions, because it is simpler and faster than using the full non-linear model. It is important, that the calculation is fast, since new computations are made within short time intervals, e.g. 5 minutes. Futhermore the computations are made in a computer with small capacity, e.g. a microchip housed within an insulin pump. Another advantage of using linear predictions is that a linear model can be more robust whereas an optimization problem based on a non-linear model can have a higher risk of diverging MPC Based on Insulin-Glucose Models The predictions used in the MPC calculations can be based on a variety of dynamic models. This thesis will focus on a specific kind of model, known as an autoregressive exogenous-input (ARX) model, due to its versatility and straightforward incorporation into MPC algorithms. In this section, models are used which only describe the dynamic relationship between insulin and glucose. Thus, these models treat meals as unmeasured, unmodeled disturbances. The ARX model describes the glucose concentration in connection with the insulin infusion rate as: here: A(q 1 )y(t) = B(q 1 )u(t) + ɛ(t) (2.10a) ɛ(t) N iid (0, σ 2 ) (2.10b) A(q 1 ) = 1 + a 1 q 1 + a 2 q a n q n (2.10c) B(q 1 ) = b 1 q 1 + b 2 q b n q n (2.10d) u(t) is the insulin infusion rate, which is the input. y(t) is the glucose concentration, which is the output. q j is the shift operator, such that q j x(t) = x(t j), meaning that the referenced value of the variable x is the value j steps before t.

26 2.2 Model Predictive Control 13 ɛ(t) is normally distributed noise. a i and b i are the model coefficients where i (1,..., n) and n is the model order. This model can be realized as a state space model. Here the state at a certain time t k+1 is calculated from the state and the insulin infusion rate of the previous time t k. The glucose concentration, y k can be calculated from the state. This gives [16]: x k+1 = Ax k + Bu k + Kɛ k (2.11a) y k = Cx k + ɛ k (2.11b) Where A = B = a a a n a n b 1 b 2. b n 1 b n a 1 a 2 K =. a n 1 a n C = [ ] In order to make the prediction, first the innovation term, e k is required. It is the difference between the actual glucose concentration, y k, and the predicted glucose concentration. The predicted value is calculated from the predicted state, ˆx k k 1 predicted at the last time measure t k 1. From (2.11b) we have: e k = y k C ˆx k k 1 (2.12) Now the glucose concentration for the next time measure can be predicted by using this error to calculate the next state. For the next j time measurements,

27 2.2 Model Predictive Control 14 the glucose concentration can be predicted by using the state space model with no noise term. ˆx k+1 k = Aˆx k k 1 + Bû k k + Ke k (2.13a) ŷ k+1 k = C ˆx k+1 k (2.13b) ˆx k+1+j k = Aˆx k+j k + Bû k+j k j = 1, 2,..., N 1 (2.13c) ŷ k+1+j k = C ˆx k+1+j k j = 1, 2,..., N 1 (2.13d) Here N is the prediction horizon. The optimal glucose concentration should be as close as possible to the setpoint. That is the preferable level of glucose, called normoglycemia, a person should have in a fasting state. The optimal insulin infusion rate is now estimated in order to minimize the least-squares difference between the predicted glucose trajectory and the setpoint. This is the objective function that will be minimized for each time measure. In order to prevent too large changes in the insulin infusion rate, a damping parameter, λ is introduced. It is multiplied with the difference, u k+j k between the insulin infusion rate at time t k+j k and the previous time t k+j 1, so that u k+j k = u k+j k u k+j 1 k. The objective function is now given as: φ = 1 2 N 1 j=0 ŷ k+1+j k r k+1+j k λ u k+j k 2 2 (2.14) Where r k+1+j k is the setpoint at time t k+1+j, so r is the desired glucose level, which may or may not be time varying. The predicted glucose concentration in the objective function has to be constrained to satisfy (2.13a)-(2.13d), since these are the model predictions. The glucose concentration predictions only depend on ˆx k k 1, y k and the insulin infusion rates. This means that every state prediction at time k can be calculated from the prediction made at time k 1, the error term, and the predicted insulin infusion rate from time k to time k + N 1. This will now be shown for N = 4. First the one step prediction is found directly from (2.13a) and (2.13b), resulting in: ŷ k+1 k = C ˆx k+1 k = CAˆx k k 1 + CBû k k + CKe k (2.15)

28 2.2 Model Predictive Control 15 For the next prediction j = 1 and the state is calculated from (2.13c). ˆx k+2 k = Aˆx k+1 k + Bû k+1 k = A(Aˆx k k 1 + Bû k k + Ke k ) + Bû k+1 k = A 2ˆx k k 1 + AKe k + ABû k k + Bû k+1 k (2.16) From this the predicted glucose can be calculated using (2.13d): ŷ k+2 k = C ˆx k+2 k = C(A 2ˆx k k 1 + ABû k k + AKe k + Bû k+1 k ) = CA 2ˆx k k 1 + CAKe k + CABû k k + Bû k+1 k (2.17) Using the same method, the predicted state and glucose concentration are calculated for j = 2: ˆx k+3 k = Aˆx k+2 k + Bû k+2 k = A(A 2ˆx k k 1 + AKe k + ABû k k + Bû k+1 k ) + Bû k+2 k = A 3ˆx k k 1 + A 2 Ke k + A 2 Bû k k + ABû k+1 k + Bû k+2 k ŷ k+3 k = C ˆx k+3 k = C(A 3ˆx k k 1 + A 2 Ke k + A 2 Bû k k + ABû k+1 k + Bû k+2 k ) = CA 3ˆx k k 1 + CA 2 Ke k + CA 2 Bû k k + CABû k+1 k + CBû k+2 k From these calculations it is seen that for an i-step prediction at time k, where i N, the prediction of the glucose concentration can be calculated directly from the prediction of the state at time k 1, the error and all the previously predicted insulin infusion rates: ŷ k+i k = (CA iˆx k k 1 + CA i 1 Ke k ) + H i û k k + H i 1 û k+1 k H 1 û k+i 1 k Where = (CA iˆx i 1 k k 1 + CA i 1 Ke k ) + H i l û k+l k l=0 H i = CA i 1 B From this the N = 4 glucose concentrations can be written in matrix form: ŷ k+1 k CA CK H û k k ŷ k+2 k ŷ k+3 k = CA CA ˆx k k 1 + CAK CA K k + H 2 H û k+1 k H 3 H 2 H 1 0 û k+2 k ŷ k+4 k CA 4 CA 3 K H 4 H 3 H 2 H 1 û k+3 k (2.18) This equation can be written in shorter form: Y k = Φ xˆx k k 1 + Φ e e k + ΓU k (2.19)

29 2.2 Model Predictive Control 16 Since the innovation is known from (2.12) and ˆx k k 1 is known from the previous time measure, the first two parts of the equation are gathered in b k : b k = Φ xˆx k k 1 + Φ e e k (2.20) It should now be ensured, that the insulin infusion rates are within physical limitations. This is done by introducing the constraints: In vector form for N = 4: u min u k+j k u max, j = 1, 2,..., N 1 U min U k U max (2.21) where: U min = U k = U max = u min u min u min u min û k k û k k+1 û k k+2 û k k+3 u max u max u max u max By using these constraints it can, for instance, be ensured that the insulin infusion rate is never negative. This is a necessary limit, since insulin cannot be extracted from the blood. There is also a limit to how much the insulin infusion rate should be changed between two consecutive time measures. This means that there should be similar constraints on u k+j k. This can also be expressed on vector form, which for N = 4 is: u k k = u k k u k 1 (2.22) u k+j k = u k+j k u k+j 1 k, j = 1, 2,..., N 1 (2.23) U k = ΨU k I 0 u k 1 (2.24)

30 2.2 Model Predictive Control 17 where: U k = Ψ = I 0 = u k k u k+1 k u k+2 k u k+3 k The constraints on the difference between the insulin infusion rate at two consecutive time measures, can now be re-written to linear constraints containing U k instead of U k, so it can be inserted into the optimization problem. The two bounderies for U k are called U min and U max, respectively. The constrains are: U min U k U max (2.25) U min ΨU k I 0 u k 1 U max (2.26) U min + I 0 u k 1 ΨU k U max + I 0 u k 1 (2.27) b l ΨU k b u (2.28) Where b l = U min + I 0 u k 1 and b u = U max + I 0 u k 1. The objective function from (2.14) can be rewritten using (2.19) and (2.20): φ = 1 2 N 1 j=0 ŷ k+1+j k r k+1+j k λ uk+j k 2 2 (2.29) = 1 2 Y k R k λ 2 U k 2 2 = 1 2 b k + ΓU k R k λ 2 ΨU k I 0 u k = 1 2 ( (ΓUk ) T ΓU k + b k R k 2 ) 2 + (bk R k ) T ΓU k + λ ( ) (ΨU k ) T ΨU k + I 0 u k λ(ψu k ) T ( I 0 u k 1 ) 2 = 1 2 U T k HU k + g T U k + ρ (2.30)

31 2.2 Model Predictive Control 18 where: R k = r k+1 k r k+2 k. r k+n k H = Γ T Γ + λψ T Ψ g = Γ T (b k R k ) + λψ T ( I 0 u k 1 ) ρ = 1 2 b k R k λ 2 I 0u k Since this is an objective function, the constant ρ can be fully disregarded since the minimum of φ corresponds to the same U k as the minimum of φ ρ. If only the insulin infusion rate is unknown, this can be solved as a quadratic optimization problem with linear inequality constraints [15]. The optimization problem can now be presented: min U k φ = 1 2 U T k HU k + g T U k + ρ (2.31a) s.t. U min U k U max (2.31b) b l ΨU k b u (2.31c) The optimal insulin infusion rate for time measure k can now be found. First by using (2.13a) the next expected state, ˆx k+j k can be found. By using the measured glucose concentration in the blood, y k, the error, e k, of the ARX model can be calculated by using (2.12). Now everything except U k is known in (2.21), (2.28) and (2.30). The quadratic minimization problem can now be solved. The MPC algorithm can now be set up [9]. The ARX model in (2.10) only takes the insulin infusion rate as input. Therefore it will react very poorly to meal disturbances. In order to take this into account, the ARX model will have to be expanded to take both the insulin infusion rate and meal disturbances as input MPC Based on Insulin-Meal-Glucose Models The ARX model used in the previous section only depends on previous states and the predicted insulin infusion rate. It is therefore called a Single Input Single Output (SISO) model. It does not however take all inputs into account. In order to take both the predicted insulin infusion rate and meal disturbances

32 2.2 Model Predictive Control 19 Algorithm 1 The MPC controller algorithm Require: ˆx k k 1, û k 1 k 1, y k, R, N Calculate: e k = y k C ˆx k k 1 (2.12) Calculate: b k = Φ xˆx k k 1 + Φ e e k (2.20) Calculate: g = Γ T (b k R k ) + λψ T ( I 0 u k 1 ) (2.31) Calculate: b l = U min + I 0 u k 1 Calculate: b u = U max + I 0 u k 1 Solve for U k : min U k φ = 1 2 U T k HU k + g T U k (2.31a) s.t. U min U k U max (2.31b) b l ΨU k b u (2.31c) Calculate: û k k = U k (1) Calculate: ˆx k+1 k = Aˆx k k 1 + Bû k k + Ke k (2.13a) return u k k and ˆx k+1 k as input, the model is expanded to a Multiple Input Single Output (MISO) ARX model. The model from (2.10) is rewritten as the following: A(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 )d(t) + ɛ(t) (2.32a) ɛ(t) N iid (0, σ 2 ) (2.32b) A(q 1 ) = 1 + a 1 q 1 + a 2 q a n q n (2.32c) B(q 1 ) = b 1 q 1 + b 2 q b n q n (2.32d) B d (q 1 ) = b d1 q 1 + b d2 q b dn q n (2.32e) Where d(t) is the meal disturbance, and b di are the meal input coefficients, with i (1,..., n) The state from (2.11a) and (2.11b) can be rewritten correspondingly: x k+1 = Ax k + Bu k + B d d k + Kɛ k (2.33) y k = Cx k + ɛ k (2.34) where: B d = b d1 b d2. b dn 1 b dn

33 2.2 Model Predictive Control 20 Now the predictions in (2.19) can be rewritten to include the meal disturbances: where for N = 4: Y k = Φ xˆx k k 1 + Φ e e k + ΓU k + Γ d D k (2.35) Γ d = H d H d2 H d1 0 0 H d3 H d2 H d1 0 H d4 H d3 H d2 H d1 H di = CA i 1 B d D k = ˆd k k ˆd k+1 k ˆd k+2 k ˆd k+3 k The meal at time k is always known being the meal actually eaten at this time, and it is d k k. The meal predictions can be handled in several ways. One possibility is that the future meals are known and are represented in d k+j k, j = 1, 2,..., N 1. The other possibility is that they are known only with some certainty, or they are unknown and are therefore just expected to be zero. Then d k+j k = 0, j = 1, 2,..., N 1. It is more practical to use the formulation, where future meals are unknown, because this situation is more realistic. If a future meal is expected, the controller will inject insulin early in order to reduce the ensuing meal peak as much as possible. This strategy is risky, however, because if the meal is not eaten, the patient may go into hypoglycemia. The objective function from (2.30) can now also be rewritten: R k = φ = 1 2 U T k HU k + g T U k + ρ (2.36) r k+1 k r k+2 k. r k+n k H = Γ T Γ + λψ T Ψ g = Γ T (b k R k ) + λψ T ( I 0 u k 1 ) ρ = 1 2 b k R k λ 2 I 0u k Because both D k and Γ d are constant they also occur in b k : b k = Φ xˆx k k 1 + Γ d D k + Φ e e k (2.37)

34 2.2 Model Predictive Control 21 Thus, the algorithm for the MISO MPC controller is identical to Algorithm 1 except for the b k calculation as shown above Exogenous-input Autoregressive Model The ARX model used by the MPC can be expanded such that it can reject an unmeasured step disturbance [9]. This is important when a parameter in the Hovorka model changes. Since the parameters cannot be measured, the controller can only detect what happens to the glucose concentration in the blood, and not the parameter itself. This is an unmeasured disturbance. It can occur, e.g. when a patient s insulin sensitivity changes due to situations like exercise or stress as stated in section The ARX (or ARIX) model is very similar to the ARX. The noise term is now divided into two sources. There is the normally distributed random noise, ɛ(t) as before and the new term, s, which is a fixed constant different from zero. The system from (2.32) is now reformulated to [9]: A(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 )d(t) + s + ɛ(t) (2.38a) ɛ(t) N iid (0, σ 2 ) (2.38b) A(q 1 ) = 1 + a 1 q 1 + a 2 q a n q n (2.38c) B(q 1 ) = b 1 q 1 + b 2 q b n q n (2.38d) B d (q 1 ) = b d1 q 1 + b d2 q b dn q n (2.38e) The total noise term can be modelled as integrated white noise, so s can be removed, since it is constant between time samples. The integrated noise, e(t) is given by the following: e(t) = s + ɛ k (s + ɛ k 1 ) = ɛ k ɛ k 1 = (1 q 1 )ɛ(t) (2.39a) The system can now be rewritten to use the integrated white noise, by integrating every term [10]: (1 q 1 )A(q 1 )y(t) = (1 q 1 )B(q 1 )u(t)+(1 q 1 )B d (q 1 )d(t)+(1 q 1 )ɛ(t) (2.40) By renaming the integrated terms, the ARX model can be written as [9]: Ā(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 )d(t) + e(t) (2.41)

35 2.2 Model Predictive Control 22 where: Ā(q 1 ) = (1 q 1 )A(q 1 ) (2.42a) B(q 1 ) = (1 q 1 )B(q 1 ) (2.42b) B d (q 1 ) = (1 q 1 )B d (q 1 ) (2.42c) e(t) = (1 q 1 )ɛ(t) (2.42d) These defintions are rewritten to functions similar to A(q 1 ), B(q 1 ) and B d (q 1 ): Ā(q 1 ) = (1 q 1 )A(q 1 ) = 1 + a 1 q 1 + a 2 q a n q n (q 1 + a 1 q 2 + a 2 q a n q n 1 ) = 1 + (a 1 1)q 1 + (a 2 a 1 )q (a n a n 1 )q n a n q n 1 (2.43a) B(q 1 ) = (1 q 1 )B(q 1 ) = b 1 q 1 + b 2 q b n q n (b 1 q 2 + b 2 q b n q n 1 ) = b 1 q 1 + (b 2 b 1 )q (b n b n 1 )q n b n q n 1 (2.43b) B d (q 1 ) = (1 q 1 )B d (q 1 ) = b d1 q 1 + b d2 q b dn q n (b d1 q 2 + b d2 q b dn q n 1 ) = b d1 q 1 + (b d2 b d1 )q (b dn b dn 1 )q n b dn q n 1 (2.43c) The ARX model can now be arranged as a state space model analogous to (2.11): x k+1 = Āx k + Bu k + B d d k + Kɛ k (2.44a) y k = Cx k + ɛ k (2.44b) where: Ā = B = a a 2 + a a n + a n a n b 1 b 2 b 1. b n b n 1 b n (2.45a) (2.45b)

36 2.2 Model Predictive Control 23 B d = K = b d1 b d2 b d1.. b dn b dn 1 b dn a a 2 + a 1. a n + a n 1 a n (2.45c) (2.45d) C = [ ] (2.45e) The new algorithm is the same as algorithm 1 except for the calculation of x k+1 and y k, which now are computed using (2.44) such that: ˆx k+1 k = Āˆx k k 1 + Bû k k + B d ˆdk k + Ke k (2.46a) e k = y k C ˆx k+1 k (2.46b) Extended Exogenous-input Autoregressive Model The MPC using ARX can reject an unmeasured step disturbance, but due to the integration, noise will be magnified. Therefore the Extended ARX (E ARX) (or ARIMAX) is a mix of ARX and ARX, balanced by the tuning parameter, α [0; 1]. The noise is now a moving average process that is integrated, and defined [9]: 1 αq 1 e(t) = s + ɛ(t) (2.47a) 1 q 1 (1 αq 1 )e(t) = (1 q 1 )(s + ɛ(t)) (2.47b) = s s + (1 q 1 )ɛ(t) (2.47c) = (1 q 1 )ɛ(t) (2.47d) 1 αq 1 e(t) = ɛ(t) (2.47e) 1 q 1 Similar to (2.10) the E ARX model can be set up: A(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 )d(t) + 1 αq 1 e(t) (2.48a) 1 q 1 (1 q 1 )A(q 1 )y(t) = (1 q 1 )B(q 1 )u(t) + (1 q 1 )B d (q 1 )d(t) + (1 αq 1 )e(t) (2.48b)

37 2.2 Model Predictive Control 24 In the limiting cases when α = 1 the basic ARX is obtained, and with α = 0 ARX is obtained. This will now be shown: It is seen that if α = 1 the noise term is: 1 αq 1 e(t) 1 q 1 = ɛ(t) (2.49a) e(t) = ɛ(t) (2.49b) From this it follows that the E ARX model becomes: A(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 )d(t) + 1 αq 1 e(t) (2.50a) 1 q 1 A(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 )d(t) + e(t) (2.50b) A(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 )d(t) + ɛ(t) (2.50c) This is the same as (2.10), which means that ARX has been obtained. Similarly if α = 0 the noise term is: With this noise term the E ARX model is: 1 αq 1 e(t) 1 q 1 = ɛ(t) (2.51a) 1 e(t) 1 q 1 = ɛ(t) (2.51b) e(t) = (1 q 1 )ɛ(t) (2.51c) A(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 )d(t) + 1 αq 1 e(t) 1 q 1 (2.52a) A(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 1 )d(t) + e(t) 1 q 1 (1 q 1 )A(q 1 )y(t) = (1 q 1 )B(q 1 )u(t) + (1 q 1 )B d (q 1 )d(t) + e(t) (1 q 1 )A(q 1 )y(t) = (1 q 1 )B(q 1 )u(t) + (1 q 1 )B d (q 1 )d(t) + (1 q 1 )ɛ(t) Ā(q 1 )y(t) = B(q 1 )u(t) + B d (q 1 )d(t) + e(t) (2.52b) where the function are defined as in (2.42). The resulting equation is identical to that of ARX, (2.40). Since the form of E ARX in (2.48b) is so similar to that of ARX in (2.40), most of the matrices in the state space form in (2.44) and (2.45) can be reused. The only change is due to the moving average polynomium, (1 αq 1 ). Since

38 2.2 Model Predictive Control 25 it is multiplied to e(t), the only change in the state space model is in K: a α a 2 + a 1 K =. (2.53) a n + a n 1 a n Asymmetric Penalty Function An extension to MPC in the previous sections could be to use an asymmetric quadratic penalty function [4] and [2]. The theory of an asymmetric penalty quadratic function is described in this section, but it is regarded only as a possible improvement, and has therefore not been implemented in this thesis. The penalty function used in the previous sections is a symmetric, quadratic function. It has the lowest penalty at 5 mmol/l, normoglycemia, and it penalises lower and higher glucose concentration symmetricly. Since the distance between normoglycemia and hypoglycemia is 1 mmol/l, it is smaller than the distance between normoglycemia and hyperglycemia, which is 3 mmol/l. These intervals in connection form the preferred interval, and the border at hypoglycemia and hypoglycemia should therefore have the same level of penalty. Furthermore the consequences of hypoglycemia are far worse for the patient than the consequences of hyperglycemia. Therefore hypoglycemia should be penalised more than hyperglycemia. In order to obtain an asymmetric penalty function for the optimization problem, the glucose concentration level is divided into four intervals. The glucose concentration below the target glucose concentration at normoglycemia is penalised quadraticly with the coefficient, κ ξ. Similarly, the glucose concentration above normoglycemia is penalised quadraticly with the coefficient, κ ω. The glucose concentrations below hypoglycemia is additionaly penalised with the quadratic function with κ v. Likewise, glucose concentrations above hyperglycemia are penalysed additionally with a quadratic function using κ w as coefficient. An example of an asymmetric quadratic penalty function is shown in Fig Here the coefficients are chosen such that the penalty hypoglycemia is equal to the penalty at hyperglycemia. Therefore it holds that κ ξ > κ ω. Furthermore is the hypoglycemic glucose concentrations penalised additionally and the glucose concentration below hypoglycemia is penalised further, such that κ v > κ w. In the example the coefficients are κ ξ = 4.5, κ ω = 0.5, κ v = 15, and κ w = 2.

39 2.2 Model Predictive Control 26 Figure 2.3: A symmetric and an asymmetric penalty function. The solid curve is the symmetric quadratic penalty function used in the previous sections. The dashed curve is an asymmetric penalty function with coefficients: κ ξ = 4.5, κ ω = 0.5, κ v = 15, and κ w = 2. For the glucose concentrations below normoglycemia to be penalised more than above normoglycemia, then it must be true that κ ξ > κ ω. Similarly hypoglycemia should be penalised higher than hyperglycemia, which means that κ v > κ w. The objective function now looks accordingly: φ = κ ξ 2 N 1 j=0 + λ 2 + κ w 2 min(ŷ k+1+j k r k+1+j k, 0) κ ω 2 N 1 j=0 N 1 j=0 u k+j k κ v 2 N 1 j=0 N 1 j=0 min(y k+1+j k y min, 0) 2 2 max(ŷ k+1+j k r k+1+j k, 0) 2 2 max(y k+1+j y max, 0) 2 2 (2.54) In order to give the objective function the same form as a quadratic program, which is much simplier to solve, four sets of slack variables are introduced. ξ j for j = 0,..., N 1 are positive when the glucose concentration is below normoglycemia and otherwise zero, ω j for j = 0,..., N 1 are positive above normoglycemia and otherwise zero, v j for j = 0,..., N 1 are positive below hypoglycemia and otherwise zero, and w j for j = 0,..., N 1 are positive above hyperglycemia and otherwise zero. When the slack variables have been intro-

40 2.2 Model Predictive Control 27 duced, the objective function is the following. The slack variables are defined using linear inequality constraints: φ = κ ξ 2 N 1 j=0 ξ j 2 2+ κ ω 2 N 1 j=0 ω j 2 2+ κ v 2 N 1 j=0 v j 2 2+ κ w 2 N 1 j=0 w j 2 2+ λ 2 N 1 j=0 u k+j k 2 2 (2.55) s.t. ξ j, ω j, v j, w j 0 for j = 0,..., N 1 (2.56a) ξ j ŷ k+1+j k + r k+1+j k for j = 0,..., N 1 (2.56b) ω j ŷ k+1+j k r k+1+j k for j = 0,..., N 1 (2.56c) v j ŷ k+1+j k + y min for j = 0,..., N 1 (2.56d) w j ŷ k+1+j k y max for j = 0,..., N 1 (2.56e) The slack variables are gathered in four vectors, and the objective function, and the constraints are written in vector form: φ = κ ξ 2 Ξ k κ ω 2 Ω k κ v 2 V k κ w 2 W k λ 2 U k 2 2 (2.57) s.t. Ξ k, Ω k, V k, W k 0 Ξ k Y k + R k Ω k Y k R k V k Y k + Y min W k Y k Y max (2.58a) (2.58b) (2.58c) (2.58d) (2.58e) where: Ξ k = [ ξ 0 ξ 1 ] T ξ N 1 Ω k = [ ω 0 ω 1 ] T ω N 1 V k = [ v 0 v 1 ] T v N 1 W k = [ w 0 w 1 ] T w N 1 Y min = [ y min y min ] T y min Y max = [ y max y max ] T y max

41 2.2 Model Predictive Control 28 U k is inserted from (2.24): φ = κ ξ 2 Ξ k κ ω 2 Ω k κ v 2 V k κ w 2 W k λ 2 ΨU k I 0 u k (2.59) s.t. Ξ k, Ω k, V k, W k 0 Ξ k (b k + ΓU k ) + R k Ω k b k + ΓU k R k V k (b k + ΓU k ) + Y min W k b k + ΓU k Y max (2.60a) (2.60b) (2.60c) (2.60d) (2.60e) The norms can now be replaced with matrix multiplications and written on the form of a quadratic program with the objective variable P k : φ = 1 2 P T k MP k + h T P k + ρ (2.61) where: P k = M = h = U k Ξ k Ω k V k W k λψ T Ψ κ ξ I κ ω I κ v I κ w I Ψ T I 0 λu k ρ = λ 2 I 0u k 1

42 2.2 Model Predictive Control 29 All the constraints are now: Ξ k, Ω k, V k, W k 0 Ξ k (b k + ΓU k ) + R k Ω k b k + ΓU k R k V k (b k + ΓU k ) + Y min W k b k + ΓU k Y max U min U k U max b l ΨU k b u Each of the constraints are rewritten, so they state upper limits for the five objective variables: P k U min Ξ k (b k + ΓU k ) + R k ΓU k Ξ k R k + b k Ω k b k + ΓU k R k ΓU k Ω k b k + R k V k (b k + ΓU k ) + Y min ΓU k V k Y min + b k W k b k + ΓU k Y max ΓU k W k Y max b k U k U max b l ΨU k b u ΨU k b u ΨU k b l All the inequality constraints are now composed so the five objective variables

43 2.2 Model Predictive Control 30 are collected in one vector, P k : Γ I Γ 0 I 0 0 Γ 0 0 I 0 Γ I I Ψ Ψ P k R k + b k b k + R k Y min + b k Y max b k U max b u b l (2.62) The optimization problem that should be used in algorithm 1 can now be stated: min P k φ = 1 2 P k T MP k + h T P k + ρ (2.63a) s.t. Γ I Γ 0 I 0 0 Γ 0 0 I 0 Y min + b k Γ I P k Y max b k (2.63b) I U max Ψ b u Ψ b l R k + b k b k + R k The optimization problem with an asymmetric penalty function has now be reformulated to a quadratic program with linear inequality constraints.

44 Chapter 3 Simulation Results and Analysis In this chapter different aspects of the Hovorka model and MPC are analysed. First an appropriate value for the basal insulin will be found. This is done such that the subject has glucose concentration at normoglycemia when in a fasting state. It will also be analysed how the Hovorka model reacts to meal disturbances. To get a realistic view of this simulations are made with meals from my own typical day. In order to determine how the Hovorka model reacts in different situations, simulations are made to see how sensitive the model is to changes in different parameters. This can be compared to how the MPC responds to these changes. Since it is simpler make a controller for situations where insulin is injected intravenously, this situation is analysed and the simple controller is made. 3.1 Basal Insulin Infusion Rate In order to find a suitable insulin-glucose steady state for the model, first the optimal basal insulin infusion rate should be found. This is the constant insulin

45 3.1 Basal Insulin Infusion Rate 32 infusion rate in a fasting period. First the state vector is defined containing all the states in the Hovorka model: z(t) = [ D 1 (t) D 2 (t) S 1 (t) S 2 (t) Q 1 (t) Q 2 (t) I(t) x 1 (t) x 2 (t) x 3 (t) ] T The states are found solving the differential equations that define the Hovorka model, and they are dependent on the meal disturbance and the insulin infusion rate. The D 1 (t) and D 2 (t) are calculated solving (2.1), S 1 (t) and S 2 (t) are found solving (2.5), and Q 1 (t) and Q 2 (t) are found by solving (2.3). From (2.7) I(t) is found, and x 1 (t), x 2 (t), and x 3 (t) are calculated solving (2.8). These are dependent only on the meal disturbances, the insulin infusion rate, and the parameters in the model, some of which are based on the patient s body weight. The body weight is here assumed to be constant, since it changes slowly and is easily measured. The calculation of the derivatives, ż(t) from the actual states is summarized in the following function: ż(t) = = dd 1(t) dt dd 2(t) dt ds 1(t) dt ds 2(t) dt dq 1(t) dt dq 2(t) dt di(t) dt dx 1(t) dt dx 2(t) dt dx 3(t) dt A G D(t) D1(t) τ D D 1(t) τ D D2(t) τ D u(t) S1(t) τ S S 1(t) τ S S2(t) τ S U G (t) F 01,c F R (t) x 1 (t)q 1 (t) + k 12 Q 2 (t) + EGP 0 (1 x 3 (t)) x 1 (t)q 1 (t) (k 12 + x 2 (t))q 2 (t) U I (t) V I k e I(t) k a1 x 1 (t) + k b1 I(t) k a1 x 2 (t) + k b2 I(t) k a3 x 1 (t) + k b3 I(t) = f(z(t), u(t), d(t)) (3.1) where the connection between U I (t) and u(t) is as given in (2.6), and the connection between U G (t) and d(t) is calculated using (2.2). Since it is a fasting period, there are no meals and d(t) = 0. In order for it to be a fasting period, it should also be true that none of the elements in the state

46 3.1 Basal Insulin Infusion Rate 33 vector must change, giving a steady state. This means that the derivatives of all these states must be zero, so: ż(t) = 0 (3.2a) f(z(t), u(t), 0) = 0 (3.2b) Aside from this it should also apply that the glucose concentration is at some desired value. This could e.g. be at normoglycemia at r = 5 mmol/l. The plasma glucose concentration is calculated from the state in the first glucose compartment, (2.4). Since it only depends on a state in z(t), a function, g(t), is defined, which has z(t) as input, and calculates the plasma glucose concentration: G(t) = g(z(t)) (3.3) G(t) = Q 1(t) V G = r (3.4) g(z(t)) r = 0 (3.5) Now a system of eleven equations with eleven unknowns can be set up: [ ] [ f(z(t), u(t), 0) 0 = g(z(t)) r 0] (3.6) [ ] z(t) Which can be solved for. This is done using the MatLab function fsolve, u(t) which returns the basal insulin infusion rate and the basal steady state. The basal insulin infusion rate is calculated using this method for different body weights. Here the desired steady state glucose concentration is set to be normoglycemia, at r = 5 mmol/l. The results are shown in Fig Fig. 3.1 shows that there is a linear relationship between the body weight and the calculated basal insulin infusion rate, increasing from mu/min as the body weight increases from kg.

47 3.1 Basal Insulin Infusion Rate 34 Figure 3.1: The calculated basal insulin infusion rate as function of the body weight.

48 3.2 Simulations with Meals 35 Figure 3.2: Meal 1 is 10 g CHO/min consumed over 10 minutes. Meal 2 is 20 g CHO/min consumed over 5 minutes. 3.2 Simulations with Meals In order to analyse the Hovorka model, simulations are made to see how it reacts to meal disturbances. For this, a number of different simulations with meals are performed. First it is examined how much the duration of a meal influences the resulting glucose concentration, and then some different insulin boluses are tested CHO Measurement As stated in section the meal disturbances are measured in grams of carbohydrate per minute. This means that the model takes into consideration how long it takes to eat the meal. It will now be examined what significance the length of a meal is. In order to do this, first a 10 min meal of 10 g CHO/min (100 g CHO total) is simulated one hour from the starting point of the simulation. Using 5 min time measures, this means that the meal disturbance is d 13 = d 14 = 10 g CHO/min and zero otherwise. Next this meal is simulated as for half of this duration, meaning a 5 min meal at equivalent 20 g CHO/min, resulting in the meal disturbance d 13 = 20 g CHO/min and zero otherwise. The two meal disturbances are shown in Fig. 3.2.

49 3.2 Simulations with Meals 36 Figure 3.3: Insulin infusion rate for the meal CHO simulations. At one hour, a 700 mu/min insulin bolus is simulated. The simulated patient is set to weigh 70 kg and the basal insulin infusion rate is calculated by solving (3.6) with the desired basal plasma glucose concentration at normoglycemia at r = 5 mmol/l. It is calculated to be u basal = mu/min. Using trial and error a suitable insulin infusion bolus is simulated at one hour at 700 mu/min for one sample. The complete insulin infusion rate is shown in Fig This insulin infusion rate is used in simulations of both meal situations, and the resulting plasma glucose concentration is shown in Fig From Fig. 3.4 it is seen that the two different meal disturbances give almost the same plasma glucose concentration. Therefore it is from now assumed, that all meals are 5 min meals, meaning that every meal is consumed in exactly one time measure.

50 3.2 Simulations with Meals 37 Figure 3.4: Plasma glucose concentration 1 is the response to the 10 g CHO/min meal over 10 min. Plasma glucose concentration 2 is the 20 g CHO/min meal over 5 min.

51 3.2 Simulations with Meals 38 Figure 3.5: Glucose concentration for a 10 g CHO/min meal at a simulation time of one hour with basal insulin only (solid curve), a 600 mu/min insulin bolus at meal time (dashed curve), and a 450 mu/min bolus (dotted curve). The glucose setpoint of 5 mmol/l, and acceptable upper and lower bounds at 8 mmol/l and 4 mmol/l, respectively, are also plotted Meals with Trial and Error Insulin Boluses In order to examine how the Hovorka model reacts to insulin boluses, first a meal of 10 g CHO/min is simulated at one hour. The basal insulin infusion rate is now calculated using (3.6) assuming that the person weighs BW = 70 kg. For a basal glucose level at 5 mmol/l, it is computed to be u basal = 6.68 mu/min. The resulting glucose concentration when using only basal insulin is the first glucose concentration in the Fig It is evident that the result will be better with an insulin bolus. Using trial and error a suitable insulin bolus at one hour will now be found. If the bolus is too high, e.g. u 13 = 600 mu/min, the blood glucose concentration will become too low and the patient will go into hypoglycemia as illustrated in Fig A better value is found to be u 13 = 450 mu/min, which also is illustrated in Fig Now the glucose level does not reach hypolycemia. It seems that even if the insulin bolus is too big, there is a peak shortly after the meal is taken. This is a result of the fact that the glucose absorption time constant, τ D = 40 min, is smaller than the insulin absorption time constant, τ S = 55 min, as shown in Table 2.1. This means that the glucose reaches the blood stream sooner than the insulin does when they are taken simultaneously.

52 3.3 Simulations of a Typical Day Simulations of a Typical Day In order to get a realistic view of the Hovorka model some simulations are now made using my meals from a typical day on Thursday Everything has been weighed and the amount of carbohydrates has been calculated using the label on each item. During the day in question, I had the meals that are summarized in Table 3.1. Table 3.1: My meals during a typical day Meal Item Quantity CHO content g CHO Breakfast Oatmeal 185 g 55 g CHO/100 g Milk 448 g 5 g CHO/100 g 22.4 Sugar g 100 g CHO/100 g Total Lunch Rye bread g 36 g CHO/100 g Salami 20 g 0.5 g CHO/100 g 0.1 Carrot 75 g 5 g CHO/100 g 3.75 Total Afternoon snack Crispbread 3 units 8.5 g CHO/unit 25.5 Apple 100 g 9 g CHO/100 g 9 Total 34.5 Dinner Rice 80 g 78 g CHO/100 g 62.4 Carrot 75 g 5 g CHO/100 g 3.75 Tomato 100 g 3 g CHO/100 g 3 Chicken 60 g 13 g CHO/100 g 7.8 Total Evening snack Crispbread 3 units 8.5 g CHO/unit 25.5 Apple 100 g 9 g CHO/100 g 9 Total 34.5 In order to calculate the amount of carbohydrate per minute consumed during every meal, the duration of each meal is needed. Then the meal disturbance is calculated by dividing the total carbohydrate amount with the duration of the meal. The calculated number is then used as the meal disturbances at the appropriate times. In order to make the meals bolus-like, the meals are gathered as described in section The simulation for this typical day starts at 6:00 am, and the time measures are in 5 minute intervals after this time. The meal times and bolus sizes are summarized in Tables 3.2 and 3.3. The meals are represented in Fig. 3.6.

53 3.3 Simulations of a Typical Day 40 Table 3.2: The times of all the meals in my typical day. The starting simulation sample time is the number of time measures since 6:00 am with a sample time of 5 min. Meal Starting time Duration (min) Starting simulation sample number Breakfast 9: Lunch 12: Afternoon snack 16: Dinner 18: Evening snack 21: Table 3.3: The total sizes of all the meals in my typical day. The rate is the total size divided with the duration of the meal, shown in Table 3.2. The bolus size is the size of the meal when gathered as a single bolus. It is the relative size multiplied by the number of time measures in the duration period. Meal Total size (g) Rate (g/min) Bolus size (g) Breakfast Lunch Afternoon snack Dinner Evening snack Now my basal insulin infusion rate is calculated from my weight BW = 53 kg by solving (3.6). The resulting value is u basal = 5.06 mu/min. The glucose concentration calculated with the given meals and only basal insulin infusion rate (no meal-related insulin boluses) is shown in Fig It is evident that without meal-related insulin boluses, the subject would have very high blood sugar, and be in hyperglycemia most of the day.

54 3.3 Simulations of a Typical Day 41 Figure 3.6: The meals of my typical day. The time starts at 6:00 am, and the meals are summarized in Tables 3.2 and 3.3. Figure 3.7: The glucose concentration in response to my typical meals using only basal insulin.

55 3.3 Simulations of a Typical Day Insulin Boluses In order to bring the glucose concentration in Fig. 3.7 down after each meal, insulin boluses will now be added. These are found using trial and error aiming to get the smallest possible glucose concentration while staying above hypoglycemia at 4 mmol/l. This is done such that an insulin bolus is added at the same time as each meal, taking only prior events into account. The found boluses are given in Table 3.4. Table 3.4: The insulin boluses for the meals from my typical day found by trial and error. Meal Insulin bolus (mu/min) Breakfast 700 Lunch 250 Afternoon snack 200 Dinner 490 Evening snack 210 The insulin infusion rate is shown in Fig. 3.8, and the calculated glucose concentration of my typical day using these boluses is also shown in Fig It can be observed from the plot that there is a great improvement compared to the glucose concentration when only basal insulin is used. The glucose concentration peaks occur shortly after every meal. This is as expected due to the time delay of the insulin absorption.

56 3.3 Simulations of a Typical Day 43 Figure 3.8: Insulin infusion rate (bottom) and glucose concentration (top) using trial and error boluses for my typical day meals.

57 3.4 Sensitivity Analysis 44 Table 3.5: Basal insulin infusion rates (mu/min) for varying insulin sensitivities for basal state r = 5mmol/L. Sensitivity 100% 75% 50% 25% S IT S ID S IE Sensitivity Analysis An important part of the Hovorka model is the insulin sensitivity parameters, S IT, S ID and S IE as described in section These sensitivities are different from person to person, and are dependent on what the person is doing. For example thay change during exercise or due to stress [7]. By changing these sensitivities one by one, the changes in the glucose concentration will be examined to determine the impact of these sensitivities. In order to find the significance of the sensitivities some meals are needed. Here are three meals: 9 g CHO/min meal at 1 h, 12 g CHO/min meal at 5 h, and 15 g CHO/min meal at 11 h. At the same time measures an insulin bolus is found using trial and error with the usual parameters given in Table 2.1. The found boluses are: 400 mu/min at 1 h, 440 mu/min at 5 h, and 500 mu/min at 11 h. All the sensitivities will seperately become smaller in order to see their effect. Four simulations are run when the respective sensitivity has 100%, 75%, 50%, and 25% of its original value. First the basal insulin infusion rate should be calculated. This will be different for the simulations because of a change in the respective k bi, where i (1, 2, 3) due to (2.9). This will cause a change in f(z(t), d(t), u(t)) in (3.1) which changes the found basal insulin infusion rate in (3.6). Therefore a basal insulin infusion rate is calculated for every sensitivity value. These basal insulin infusion rates are summarized in Table 3.5. The glucose concentration is now calculated with the different S IT sensitivities and are plotted in Fig It can be seen that when S IT is smaller, the glucose concentration is larger. When S IT is reduced, k b1 is reduced. From (2.8a) it follows that also x 1 (t) is reduced. Since a smaller x 1 (t) causes a smaller Q 2 (t) according to (2.3), less glucose will be absorbed by the peripherical tissue and the glucose in the blood stream, Q 1 (t), is increased. Therefore the plasma glucose concentration becomes larger for a smaller S IT which corresponds to the shown glucose concentration in Fig. 3.9.

58 3.4 Sensitivity Analysis 45 Figure 3.9: Glucose concentration with 100%,75%, 50% and 25% of the original distribution and transport insulin sensitivity, S IT. Simulations are made for the four different values of S ID. The resulting glucose concentrations are shown in Fig Similar to the situation before, when S ID is reduced, x 2 (t) is also reduced. This would reduce the glucose in the peripheral tissue, Q 2 (t), resulting in higher amount of glucose in the blood stream. This is confirmed by the calculated glucose concentrations in Fig The simulations made with different S IE values are shown in Fig Unlike the first two insulin sensitivities, this parameter has little effect on the glucose responses. This is due to the fact that the change in the ellements of (2.4) almost cancel each other out.

59 3.4 Sensitivity Analysis 46 Figure 3.10: Glucose concentration with 100%,75%, 50%, and 25% of the original disposal insulin sensitivity, S ID. Figure 3.11: Glucose concentration with 100%,75%, 50% and 25% of the original insulin sensitivity of EGP 0, that is S IE.

60 3.5 Intravenous Insulin Infusion Intravenous Insulin Infusion In order to find a model that calculates a suitable insulin infusion rate, it is investigated what happens when the insulin infusion rate is injected intravenously. The insulin absorption time constant is now very close to zero, which means that the insulin reaches the blood stream much faster than the carbohydrate. Now there is no time delay, so a model can be developed that reacts instantly to the glucose concentration Proportional Control A simple controller for the insulin infusion rate is a proportional model (P): u(t) = u basal + α(y(t) r) (3.7) where u(t) is the insulin infusion rate, u basal is the basal insulin infusion rate calculated by solving (3.6), α is a tuning parameter, y(t) is the measured glucose concentration, and r is the setpoint. The insulin infusion rate is constrained to be greater than or equal to zero because insulin cannot be extracted from the blood. The tuning parameter is found using trial and error. A simulation is made using 9, 12, and 15 g CHO/min meals at 1, 5, and 11 hours, respectively. To investigate how much the model depends on the body weight of the subject, this simulation is made for a 70 kg and a 100 kg subject. The setpoint is set to normoglycemia at 5 mmol/l, and the calculated basal insulin infusion rates are calculated as u basal1 = 6.68 mu/min and u basal2 = 9.54 mu/min, respectively. Using trial and error for the 70 kg subject a tuning parameter of α = 6 mu min 1 mmol 1 L is found suitable. The insulin infusion rates and glucose concentrations are shown in Fig In order to evaluate the quality of the results, the same simulations are run using only basal insulin infusion rates, and the glucose concentrations of the four different simulations are shown in figure It is seen that the proportional model gives a glucose concentration much better than when only basal insulin infusion rates are used. The overall glucose concentration is much lower while staying above hypoglycemia. It is also observed that the glucose concentration generally is higher for the 70 kg subject than for the 100 kg subject. This is mainly due to the fact that for a smaller body weight the glucose distribution volume, V G, is smaller which causes the glucose concentration, y(t), from (2.4) to be larger.

61 3.5 Intravenous Insulin Infusion 48 Figure 3.12: Intravenous insulin therapy with a 9, 12, and 15 g CHO/min meal at 1, 5, and 11 hours, respectively. The meals (bottom plot), the insulin infusion rate calculated by the propotional controller (middle plot), and the resulting glucose concentration (top plot). The simulation is made for a 70 kg and a 100 kg subject. The same simulation as before is now made for subcutaneously injected insulin. These results are shown in Fig It can be seen that the proportional model gives very poor results when the insulin is injected subcutaneously. At certain times the glucose concentration is much below hypoglycemia for long periods. This is due to the time delay between when the proportional model raises the insulin infusion rate, and when the glucose concentration decreases.

62 3.5 Intravenous Insulin Infusion 49 Figure 3.13: The glucose concentration calculated with the proportional model as in Fig shown with the glucose of the simulations with only basal insulin. Figure 3.14: Meals (bottom plot), the insulin infusion rate calculated by the proportional controller (middle plot), and the glucose concentration for a simulation with 9, 12, and 15 g CHO/min meals at 1, 5, and 11 hours, respectively. The simulation is made for a 70 kg and a 100 kg subject.

63 3.5 Intravenous Insulin Infusion Proportional-Derivative Control An expansion of the proportional model is made by introducing a derivative term. The proportional-derivative controller (PD) [1] and [13] takes the glucose concentration of both the current and the previous time measure into account: u k = u basal + α(y k r) + α d (y k y k 1 ) (3.8) where u k is the insulin infusion rate at time k, u basal is the basal insulin infusion rate calculated by solving (3.6), α is a tuning parameter for the proportional term, y k is the measured glucose concentration at time k, α d is a tuning parameter for the derivative term, and r is the setpoint. Again, the insulin infusion rate is constrained to be greater than or equal to zero. The same simulations as in section are made but now with the PD controller. With intravenously injected insulin, the tuning parameters are found for the 70 kg subject using trial and error to α = 19 mu min 1 mmol 1 L and α d = 50 mu min 1 mmol 1 L. The calculated insulin infusion rates are shown with the glucose concentrations, using the PD controller for the two subjects, in Fig It is seen that the glucose concentration only exceeds hyperglycemia once for a very short time, which is much better than when the P controller was used. These simulations show that the time delay associated with subcutaneously injected insulin is a major limitation to the control that can be obtained. These models only react to the glucose concentration that has just been measured, and when there is an increase in insulin, it takes so long to have an effect on the glucose concentration that too much insulin has already been given. Therefore it is necessary to react before the glucose concentration has risen too much. Future glucose concentrations should therefore be predicted, which makes MPC an attractive option for this application. It makes predictions of the glucose concentration, and can therefore react before the actual glucose concentration has changed too much.

64 3.5 Intravenous Insulin Infusion 51 Figure 3.15: Simulations with intravenously injected insulin using a PD controller. Meals of 9, 12, and 15 g CHO/min are simulated at 1, 5, and 11 h, respectively. The simulations are made for a 70 kg and a 100 kg subject.

65 3.6 Model Predictive Control Simulations Model Predictive Control Simulations Simulations using MPC can now be made. First a third order SISO ARX model are used, and the effects of choice of prediction horizon and damping parameter is analysed. Then a ninth order MISO ARX is used, and simulations are run to investigate the controller s response to meals. From the nine order model a ARX and an E ARX model is defined. New meal simulations are made with the two models to see how they each respond. Next, simulations are made to analyse how MPC with each of the three models react to unmeasured meal disturbances. This is done by changing each of the insulin sensitivity parameters. Finally, MPC is applied to the meals of a typical day in section Simulations with a Third Order ARX Model As described in section 2.2 model predictive control (MPC) can be used to find the optimal insulin infusion rate. First MPC is used with a third order SISO ARX model. The model coefficients are given in the following two functions as in (2.10): A(q 1 ) = q q q 3 (3.9a) B(q 1 ) = q q q 3 (3.9b) The constraints on the insulin infusion rate and on the change in insulin infusion rate should be chosen for all MPC simulations. The minimum constraint on the insulin infusion rate is set to zero, so in difference variables this is minus the basal insulin. This is due to the fact that insulin cannot be extracted from the blood stream. The maximum value for the insulin infusion rate is set such that the insulin infusion rate cannot be more than 600 mu/min above the basal insulin. This is done such that the model can react fast to big meals. The change in insulin infusion rate between consecutive time measures is set to 300 in both directions. This makes it possible to have relatively big changes, but more than two time measures are needed in order to go between the smallest and largest possible insulin infusion rate. This could be made smaller to force the controller to react slower. However, it is assessed that the damping parameter can do better regulations of this. In practice physical limitations may set these bounderies, e.g. the insulin pump could have a maximum amount of insulin possible to administer during one time measure. The objective of using MPC to control the insulin infusion rate is to return the patient to normoglycemia as fast as possible after a change in glucose concen-

66 3.6 Model Predictive Control Simulations 53 Figure 3.16: Glucose concentration and insulin infusion rate for MPC with third order ARX and setpoint change. After 3 h the setpoint changes from 8 mmol/l to 5 mmol/l. A damping parameter of 100 and 2 h model predictions are used. tration. To investigate the algorithm s ability to do so, a setpoint change is simulated. In the simulation, the setpoint is at 8 mmol/l the first three hours, and then changes to normoglycemia at 5 mmol/l. The simulation is made using 5 min data samples and 2 h model predictions, so N = 24 and the damping parameter λ = 100. The calculated insulin infusion rate and glucose concentration are shown in Fig It is observed that the glucose concentration decreases to hypoglycemia before it returns to normoglycemia. It takes the glucose concentration a long time to reach a steady state at normoglycemia. In order to see what influence the prediction horizon has on the resulting glucose concentration, the exact same simulation is made, but now with 4 h predictions. The result is shown in Fig Since the controller in the second simulation "knows" at time t = 0 that the setpoint will change at time t = 3 h, it can start reacting from the beginning. This is because the further into the future the setpoint change is foreseen, the sooner the controller can react. This means that it can counteract the insulin delay due to the absorption time, by raising the inuslin infusion rate, before the glucose concentration has actually changed. To see the effects of the damping parameter, the same simulation as before is run but now with λ = 0.01; the result is shown in Fig It can be seen that

67 3.6 Model Predictive Control Simulations 54 Figure 3.17: Glucose concentration and insulin infusion rate for MPC with third order ARX and setpoint change. After 3 h the setpoint changes from 8 mmol/l to 5 mmol/l. A damping parameter of 100 and 4 h predictions are used. the smaller the damping parameter, the more severely are the insulin infusion rate changes. This is as expected since the damping parameter in definition damps the difference in insulin infusion rate from one sample to the next. The third order model is not very useful since it does not take meals into account, which makes it too simple for non-fasting situations. Therefore a MISO ARX model is now used that takes both the insulin infusion rate and the meal disturbances as inputs. It is also an expansion compared to the previous model, and is now of order 9.

68 3.6 Model Predictive Control Simulations 55 Figure 3.18: Glucose concentration and insulin infusion rate for MPC with third order ARX and setpoint change. After 3 h the setpoint changes from 8 mmol/l to 5 mmol/l. A damping parameter of 0.01 and 4 h model predictions are used.

69 3.6 Model Predictive Control Simulations Simulations with a Ninth Order ARX Model with Meal Input The coefficients for the model with order 9 and meal input are given in these three polynomials: A(q 1 ) = q q q q q q q q q 9 (3.10a) B(q 1 ) = q q q q q q q q q 9 (3.10b) B d (q 1 ) = q q q q q q q q q 9 (3.10c) In order to see the effect of the higher model order the same setpoint simulation as before is made. The insulin infusion rate and the glucose concentration are shown in Fig It can be seen that hypoglycemia now is avoided and that the insulin infusion rate oscillations now decrease faster than in Fig Now a simulation with a meal can be made to see how much influence the number of predictions has in this situation. This is done with a single 9 g CHO/min meal at 1 h, 2 h model predictions and a damping parameter of λ = 100. The insulin infusion rate is calculated using MPC with the 9 th order MISO ARX model. The meals are announced to the controller at meal time, but not prior to the meal. The resulting insulin infusion rate is shown along with the meal disturbance and glucose concentration in Fig For comparison, a simulation is made with the same meal, but now the MPC uses 4 h predictions and the damping parameter is λ = 1. The resulting insulin infusion rate is shown in Fig In both cases the damping parameter is chosen such that the glucose level is as brief as possible below hypoglycemia, but still as low as possible. The second simulation gives the better result, and to investigate more, in Fig and 3.23 the glucose production is shown with the predictions found with the state space model, and from them it can be seen that the far predictions

70 3.6 Model Predictive Control Simulations 57 Figure 3.19: Glucose concentration and insulin infusion rate for MPC with 9 th order ARX and setpoint change. After 3 h the setpoint changes from 8 mmol/l to 5 mmol/l. A damping parameter of 0.01 and 4 h model predictions are used as in the simulation for Fig are very wrong especially in the beginning. The two hour predictions are very far from the resulting glucose concentration, and the 4 h prediction does not seem to resemble it at all. However, the found glucose concentration seams better when 4 h predictions are used than when only two hour predictions are used. Although the furthest predictions are far from the actual result, they still contain information, that improve the calculated insulin infusion rate.

71 3.6 Model Predictive Control Simulations 58 Figure 3.20: The glucose concentration, insulin infusion rate and meal disturbance of a simulation with a 9 g CHO/min meal at 1 h. The insulin infusion rate is found using MPC with the MISO ARX model defined in h model predictions and λ = 100 are used, and meals are announced to the controller only at meal times. Figure 3.21: The glucose concentration, insulin infusion rate and meal disturbance with a 9 g CHO/min meal at one hour. The results are made using MISO ARX MPC with 4 h predictions, λ = 1 and unknown future meals.

72 3.6 Model Predictive Control Simulations 59 Figure 3.22: The glucose concentration of the same simulation as shown in Fig Some of the predictions made with the state space model are shown. E.g. the 2 h prediction is calculated 2 h prior to the current time sample. The glucose concentration with only basal insulin is showed as a dot-dashed curve.

73 3.6 Model Predictive Control Simulations 60 Figure 3.23: The glucose concentration and some of the calculated predictions from the same simulation as that of Fig Some of the predictions made with the state space model are shown. E.g. the 2 h prediction is calculated 2 h prior to the current time sample. The glucose concentration with only basal insulin is showed as a dot-dashed curve.

74 3.6 Model Predictive Control Simulations MPC with E ARX As stated in section the MPC, which uses an ARX model, cannot reject an unmeasured disturbance. Instead an E ARX model is used as described in section 2.2.4, which using the 9 th order ARX from before gives the following coefficient matrices: Ā = (3.11a) B = (3.11b) B d = (3.11c)

75 3.6 Model Predictive Control Simulations 62 K = α (3.11d) A reasonable value for the tuning parameter, α [0; 1] should be found. To see the limiting case for α = 1 first, a simulation is run for a 9 g CHO/min meal at 1 h like in section 3.6.2, using ARX. 4 h model predictions and a damping parameter of λ = 5000 are used. The resulting insulin infusion rate and glucose concentration are shown in Fig The damping parameter is chosen by trial and error, such that the glucose concentration is as small as possible while avoiding too much hypoglycemia. When comparing to the similar simulation using ARX in Fig it can be seen that with ARX the maximum glucose concentration is much smaller. The glucose concentration also stabilises much faster while avoiding too low glucose concentrations. This shows that the ARX model is much more suitible for handling meal disturbances than ARX. Using the tuning parameter α an E ARX compromise should now be found. Using trial and error α = is found to be a good value. It turns out that it should be very close to one, and resemble ARX very much, to give good results. Again 4 h model predictions are used, and now the damping parameter is λ = 5. The results are shown in Fig The maximum glucose concentration is now much lower compared to the simulation using ARX in Fig The glucose concentration is also above hyperglycemia for a much shorter period. The E ARX simulation resembles the simulation using ARX in Fig much more, which shows that E ARX can be used for handling meal disturbances similar to ARX.

76 3.6 Model Predictive Control Simulations 63 Figure 3.24: Simulation with a 9 g CHO/min meal at 1 h using MPC with ARX. 4 h model predictions and a damping parameter of λ = 5000 are used. Figure 3.25: Simulation of a 9 g CHO/min meal at 1 h using MPC with E ARX. The tuning parameter α = is used. 4 h model predictions are used and the damping parameter is λ = 5.

77 3.6 Model Predictive Control Simulations Sensitivities The advantage of using ARX is that it handles unsmeasured step disturbances well. It will now be tested how MPC handles changes in the sensitivities using ARX, ARX, and E ARX. First the insulin sensitivity S IT is changed to zero. In section 3.4 three meals were simulated with four different values of S IT and it showed that when S IT is decreased, the glucose concentration is increased. It is now simulated that S IT drops at t = 1 h from L/mU to zero L/mU. 4 h model predictions are used. The simulation using ARX has the damping parameter λ = and the resulting insulin infusion rate and glucose concentration are shown in Fig From the plot it is observed that the controller does not return the simulated patient to a steady state at normoglycemia. For comparison, the same simulation is run using ARX. Now the damping parameter is λ = 100. The result is shown in Fig It can be observed that although the simulation is run for two days, a steady state has not been reached. Now the same simulation is run using E ARX with α = which showed to be useful for meal disturbances in section The used damping parameter is now λ = 0.01 and the resulting insulin infusion rate and the glucose concentration are shown in Fig Now the steady state at normoglycemia is found within two days. Now the insulin sensitivity that controls glucose disposal is investigated. Three different simulations are made using ARX, ARX, and E ARX, respectively. In every simulation S ID is changed from to zero at time t = 1 h and 4 h model predictions are used. The simulation using ARX has a damping parameter of λ = and the results are shown in Fig It is seen that there are only small oscillations, and that the amplitude decreases rapidly. For the simulation using ARX a damping parameter of λ = 10 is used. The results are shown in Fig In this simulation the oscillations are small in the beginning, and a steady state at normoglycemia is reached. A damping parameter of λ = is used for the simulation with E ARX. As in section a tuning parameter of α = is used and the results are shown in Fig It can be observed that there are only very small oscilla-

78 3.6 Model Predictive Control Simulations 65 Figure 3.26: Response to a step change in the insulin sensitivity S IT using ARX MPC. After 1 h S IT drops from L/mU to zero L/mU. 4 h model predictions and damping parameter λ = are used. tions, and that the amplitude decreases rapidly. Insulin sensitivity S IE which relates to the sensitivity of EGP, can now be simulated using ARX, ARX, and E ARX. In the first simulation the damping parameter is λ = 10 4 and the results are shown in Fig It is seen that there are mild oscillations at the beginning of the response, but the steady state obtained is above normoglycemia. The simulation using ARX has a damping parameter of λ = 1 and its results are shown in Fig It is seen that the glucose concentration now ends with a steady state at normoglycemia. Now the simulation using E ARX is performed. It has a damping parameter of λ = and the tuning parameter of α = from the previous section. The results are shown in Fig It can be seen that there are only small oscillations and the amplitude rapidly decreases and returns to a steady state at normoglycemia. The sensitivity of the endogenous glucose production at zero insulin concentration is now investigated. This is done with simulations using ARX, ARX, and

79 3.6 Model Predictive Control Simulations 66 Figure 3.27: Response to a step change in the insulin sensitivity S IT using ARX MPC. After 1 h S IT drops from L/mU to zero L/mU. 4 h model predictions and damping parameter λ = 100 are used. E ARX. In each of the simulations EGP 0 is changed at time t = 1 h from mmol kg 1 min 1 to 75% of the original value. The simulation using ARX has a damping parameter of λ = From the results shown in Fig it can be seen that the MPC using ARX is not able to return the glucose concentration to a steady state at normoglycemia. Insulin sensitivity EGP 0 simulated using ARX is now made. The damping parameter is here λ = 0.001, and the results are shown in Fig It can be seen that there are a lot of oscillations, and that the glucose concentration is very hypoglycemic for a time. The damping parameter is λ = 1 for the simulation which uses E ARX and has a tuning parameter of α = from the previous section. The results are shown in Fig From the plot it can be observed that a steady state at normoglycemia is reach within two days. For the three cases with step changes in the insulin sensitivities, the MPC using

80 3.6 Model Predictive Control Simulations 67 Figure 3.28: Insulin infusion rate and glucose concentration in response to a step change in the insulin sensitivity S IT using E ARX. After 1 h S IT is changed to zero. A tuning parameter of α = 0.995, 4 h model predictions, and a damping parameter of λ = 0.01 are used. ARX did not return the glucose concentration to steady state at normoglycemia. On the contrary this happens when ARX is used. This confirms that ARX does not respond well to unmeasured step disturbances, and that good results can be found using ARX. The observations above also confirms that E ARX, which gives good response to meal disturbances, is a good method for rejecting unmeasured step disturbances. The sensitivity simulations of EGP 0 show that when ARX is used, a steady state is obtained very fast, but below normoglycemia. When ARX is used there are many big oscillations, but when these are combined in E ARX the steady state is obtained slowly but accurately. This confirms that E ARX is a good method that also responds well to meal disturbances.

81 3.6 Model Predictive Control Simulations 68 Figure 3.29: Insulin infusion rate and glucose concentration in response to a step change in the insulin sensitivity S ID using ARX MPC. 4 h model predictions and a damping parameter of λ = are used. Figure 3.30: Insulin infusion rate and glucose concentration in response to a step change in the insulin sensitivity S ID which at t = 1 h is changed to zero. A damping parameter of λ = 10 is used.

82 3.6 Model Predictive Control Simulations 69 Figure 3.31: Insulin infusion rate and glucose concentration in response to a step change in the insulin sensitivity S ID at t = 1 h using E ARX. The tuning parameter is α = and the damping parameter is λ = Figure 3.32: Results of simulation of a step change in the insulin sensitivity S IE to zero at t = 1 h using ARX MPC with a damping parameter λ = 10 4.

83 3.6 Model Predictive Control Simulations 70 Figure 3.33: Simulation of response to a step change in the insulin sensitivity S IE using ARX. The damping parameter is here λ = 1. Figure 3.34: Simulation of the response to a step change in the insulin sensitivity S IE using E ARX with a damping parameter of λ =

84 3.6 Model Predictive Control Simulations 71 Figure 3.35: Response to a change in the endogenous glucose production at zero insulin, EGP 0 using ARX MPC. At t = 1 h EGP 0 is change from mmol kg 1 min 1 to 75% of the original value. The simulation has a damping parameter of λ = Figure 3.36: Response to a change in EGP 0 using ARX MPC. A damping parameter of λ = is chosen.

85 3.6 Model Predictive Control Simulations 72 Figure 3.37: Simulation, which shows the response of a step change ín EGP 0 when using E ARX MPC. The simulation has a damping parameter of λ = 1 and a tuning parameter of α =

86 3.6 Model Predictive Control Simulations 73 Figure 3.38: Simulation of meals from my typical day in section 3.3 using E ARX. Tuning parameter α = 0.995, 4 h predictions, and a damping parameter of λ = 10 is used Simulations of a Typical Day In order to get a realistic view of the MPC, a simulation is made using the meals from my typical day in section 3.3. When E ARX with a tuning parameter of α = is used as in the previous section, the best damping parameter is determined to be λ = h model predictions are used. The resulting insulin infusion rate and glucose concentration are shown in Fig From the plot it can be seen there is a long time period with a glucose concentration below hypoglycemia, and the minimum value is 2.5 mmol/l. This is too low and lasts for too long to be acceptable. In order to prevent this the tuning parameter is increased to α = A suitable value for the damping parameter is λ = 5 and the resulting insulin infusion rate and glucose concentration are shown in Fig It can be observed that there is a very high glucose concentration shortly after the first meal. This is as expected since breakfast is a big meal, and the insulin infusion rate follows the meal variances with a time delay through the whole day as expected. To compare the glucose concentration using the controller with the simulation using only basal insulin infusion rate, the glucose concentrations are shown in Fig It can be observed that although the first peak seems high, it is much lower than when only basal insulin

87 3.6 Model Predictive Control Simulations 74 Figure 3.39: Insulin infusion rate and glucose concentration for my typical day meals using MPC with E ARX. A tuning parameter of α = 0.999, 4 h predictions, and a damping parameter of λ = 5 are used. is used. The glucose concentration is only below hypoglycemia for a short period and the lowest value is only at 3.7 mmol/l. It can also be seen that the glucose concentration using the controller is a great improvement to the simulation with only basal insulin.

88 3.6 Model Predictive Control Simulations 75 Figure 3.40: Glucose concentration of two simulations using my typical day meals. In the first simulation the insulation infusion rate is calculated with E ARX MPC using 4 h predictions, a tuning parameter of α = and a damping parameter of λ = 5. The second simulation is made with the boluses found using trial and error in section

89 Chapter 4 Discussion A simple controller can be used if the insulin is injected intraveneously. A proportional derivative controller gives very good results as shown in section 3.5. However, both the proportional and the proportional derivative controllers are unsuitable controllers for subcutaneously injected insulin. This is due to the fact that both controllers base the insulin infusion rate on the previous glucose concentration measurements. As a result of this, the simple controller is always "behind schedule". In order to avoid this, a controller based on MPC should be chosen instead. MPC has the advantage that it uses predictions of the glucose concentration, so it can react before changes occur. In order to reject meal disturbances, inslin boluses can be administered simultaneously with the meals. This could be done using trial and error, and the results in Fig. 3.8 depict a reasonable glucose concentration. However, the method is not practical since several possible boluses have to be tested using the Hovorka model in order to find a single suitable bolus. This is another reason why MPC should be used instead. Simulations have shown that ARX MPC can give a good insulin infusion rate with meal disturbances, such as changes in insulin sensitivities. However, this controller does not respond well to unmeasured step disturbances. Therefore a controller is needed that can handle both meal disturbances and unmeasured step disturbances. To reject unmeasured step disturbances MPC with ARX

90 77 can be used, because the difference in glucose concentration between two consecutive samples can reject a change in the general trend[10]. Simulations in section confirm that ARX handles changes in the insulin sensitivities well. Here the controller returns the simulated patient to a normoglycemic steady state after the changes. However, with a change in the endogenous glucose production at zero insulin, a parameter in the model, the controller gives many big oscillations in glucose concentration, and in this case ARX will not be suitable. Another aspect is that ARX MPC does not respond well to meal disturbances, since the integration magnifies the consequences of an impulse. Therefore ARX should not be used as a controller for the insulin infusion rate in this case. The MPC optimization problem has two sets of constraints, a minimum and a maximum value for the calculated insulin infusion rate and minimum and maximum value for the change of insulin infusion rate between two consecutive time measures. There is a natural minimum of the insulin infusion rate at zero, because insulin cannot be extracted from the blood. The maximum insulin infusion rate should be high in order to ensure the possibility of giving large insulin boluses. The maximum insulin infuison rates and the constraints on change in insulin infusion rate, should be based on the actual mechanical limitations of the insulin pump. In order to use the advantages of both ARX and ARX, a good compromise is E ARX. With a very high tuning parameter, E ARX resembles ARX, which is suitable for meal disturbances. If the tuning parameter is low, E ARX behaves like ARX, which is why it handles unmeasured step disturbances well. With a suitable tuning parameter, these advantages can be balanced, and E ARX MPC will be a useful controller in both cases. The simulations in section and show that the tuning parameter has to be very close to one in order to reject meal disturbances, and that it can also handle unmeasured step disturbances. This is supported by the simulations in Fig MPC uses linear predictions, ARX, ARX, or E ARX, while the Hovorka model is a non-linear system. Using linear predictions gives larger irregularities due to the linear approximation, while a non-linear model more similar to the Hovorka model would be more accurate. On the other hand, the linear predictions are computed much faster and do not require as much calculation capacity as a non-linear model. This is an advantage, when calculations are made e.g., in a small computer controlling an insulin pump. The fast calculations are necessary in order to get a relatively small sample size, e.g., 5 minutes.

91 Chapter 5 Conclusion It can be seen from the simulations that MPC gives a better insulin infusion rate profile than the proportional and proportional derivative controllers. MPC is also preferred compared to trial and error insulin boluses. MPC gives better results when it is used with E ARX instead of ARX and ARX. The basal insulin infusion rate is important, because it is what keeps the glucose concentration at normoglycemia in a fasting state. The basal insulin is used when no parameters or inputs are changed, and it depends linearly on the weight of the virtual Hovorka subject. When a meal is consumed the CHO absorption is dependent on the duration of the meal. Simulations show that the differences in the resulting glucose concentration trajectory are small. Therefore meals can be regarded as impulses, which makes it easier for the user in the sense that it is not necessary to know the duration of the meal but only the amount of carbohydrates in the meal. A proportional derivative controller is useful if the insulin is injected intravenously. However, this injection form is invasive for the patient, and normally subcutaneous insulin injection is used. Here the proportional derivative controller cannot be used, due to the time delay in insulin absorption. The insulin sensitivities are of great importance. They control how different

92 79 parts of the Hovorka model react to the insulin concentration, similar to insulin sensitivities in the patient. Therefore it is important to estimate good insulin sensitivities in order to make the Hovorka model as close to the physical behaviour as possible. Since e.g. stress and exercise affect the insulin sensitivities, the controller should be able to take these into account, even though they cannot be measured. MPC using ARX or E ARX with a suitable tuning parameter gives good responses to unmeasured step disturbances. A good controller for insulin administration is MPC using E ARX with a very high tuning parameter (i.e., close to one). It is able to give a good response to meal disturbances, and also handles unmeasured step disturbances well. An expansion to the controller, is to use an asymmetric penalty function that penalises hypoglycemia more severely than hyperglycemia. This would be an improvement to the controller, since it is much more dangerous for the subject to be hypoglycemic than hyperglycemic. The MPC using ARX, ARX, and E ARX will always have errors in the predictions due to the linear approximations. This means that the predictions furthest into the future will, in general, be the most erroneous. The key is to find a balanced set of parameters in the MPC that optimises the safety and efficacy of the controller. To improve the controller an asymmetric penalty function could be used. As described in section such a penalty function could be used to penalise hypoglycemia higher than hyperglycemia since the consequences of the first are more severe than of the latter. Another improvement would be to use an adaptive controller. Then the parameters, e.g. the insulin sensitivities, are able to adapt to the new conditions at each time sample [8].

93 Appendix A MatLab code for Methodology The Matlab code for the methodology sections are written here in the same order as the sections. An implementation of the Hovorka model has been given in some MatLab functions; HovorkaModel.m, WrapHovorkaModel.m, HovorkaLinearModelSimulation.m, HovorkaBasalState.m, HovorkaModelSimulation.m, and HovorkaParameters.m. These functions have been handed out by IMM, DTU. A.1 The Hovorka Model A HovorkaFigures.m function plothandles = HovorkaFigures(T,D,U,G,Ghypo,Ghyper,R,single,multi,legG,legU) % plothandles = HovorkaFigures(T,D,U,G,Ghypo,Ghyper,R,single,multi,legG,legU) % plothandles = [pd1;pu1;pr1;pg1] % first with the 3 single figures, then % the subplots if( single && multi) error('single or multi have to be true in order to make any plots!')

94 A.1 The Hovorka Model 81 end figurenmbs = 10:13; spec = ['b ';'r ';'g.';'k: ';'m.']; linewidth = 2.5; fontsize1 = 18; % used for axes labels fontsize2 = 20; % used for subplot titles fontsize3 = 16; % used for legends if(single) figure(figurenmbs(1)) pd1 = stairs(t/60,d,'linewidth',linewidth); ylabel('d (g CHO/min)','FontSize',fontSize1); xlabel('time (h)',... 'FontSize',fontSize1); figure(figurenmbs(2)) if(size(u,2)==1) pu1 = stairs(t/60,u,'linewidth',linewidth); else hold on for j=1:size(u,2) spec(j,:) stairs(t/60,u(:,j),spec(j),'linewidth',linewidth); end hold off if(nargin>10) leg = legend(legu); set(leg,'fontsize',fontsize3) end end yl = ylim; if yl(1)<0 % it doesn't need to show below 0 since it can't be negative ylim([0 yl(2)]); end ylabel('u (mu/min)','fontsize',fontsize1); xlabel('time (h)','fontsize',... fontsize1); figure(figurenmbs(3)) pr1 = stairs(t/60,r,'g','linewidth',linewidth); hold on plot([t(1),t(end)/60],[ghypo Ghypo],'k') plot([t(1),t(end)/60],[ghyper Ghyper],'k') if(size(g,2)==1) pg1 = plot(t/60,g,'linewidth',linewidth); leg = legend([pr1 pg1],'setpoint','glucose concentration'); set(leg,'fontsize',fontsize3) else for j=1:size(g,2) pg1q(j) = plot(t/60,g(:,j),spec(j,:),'linewidth',linewidth); end if(nargin>9) leg = legend([pr1 pg1q],legg); set(leg,'fontsize',fontsize3); else leg = legend([pr1 pg1q(1)],'setpoint','glucose concentration'); set(leg,'fontsize',fontsize3)

95 A.1 The Hovorka Model 82 end end hold off ylabel('g (mmol/l)','fontsize',fontsize1); xlabel('time (h)','fontsize',... fontsize1); end if(multi(1)>0) figure(figurenmbs(4)) if(multi(1)==3) subplot(3,1,3) pd2 = stairs(t/60,d,'linewidth',linewidth); title('meals','fontsize',fontsize2) ylabel('d (g CHO/min)','FontSize',fontSize1); xlabel('time (h)',... 'FontSize',fontSize1); end subplot(multi(1),1,2) hold on if(size(u,2)==1) pu2 = stairs(t/60,u,'linewidth',linewidth); elseif(length(multi)>1) for j=1:size(u,2) pu21(j) = stairs(t/60,u(:,j),spec(j,1),'linewidth',linewidth); end if(nargin>9) leg = legend(pu21,legu) set(leg,'fontsize',fontsize3) end end hold off yl = ylim; if yl(1)<0 % it doesn't need to show below 0 since it can't be negative ylim([0 yl(2)]) end title('insulin administration','fontsize',fontsize2) ylabel('u (mu/min)','fontsize',fontsize1); xlabel('time (h)','fontsize',... fontsize1); subplot(multi(1),1,1) pr2 = stairs(t/60,r,'g','linewidth',linewidth); hold on plot([t(1),t(end)/60],[ghypo Ghypo],'k') plot([t(1),t(end)/60],[ghyper Ghyper],'k') if(length(multi)==1) pg2h = plot(t/60,g(:,1),'linewidth',linewidth); leg = legend([pr2 pg2h],'setpoint','glucose concentration') set(leg,'fontsize',fontsize3) else for j=1:size(g,2) pg2h(j) = plot(t/60,g(:,j),spec(j,:),'linewidth',linewidth); end if(nargin>9) leg = legend([pr2 pg2h],legg); set(leg,'fontsize',fontsize3) else

96 A.1 The Hovorka Model 83 end leg = legend([pr2 pg2h(1)],'setpoint','glucose concentration'); set(leg,'fontsize',fontsize3) end end hold off yl = ylim if yl(1)<0 % it doesn't need to show below 0 since it can't be negative ylim([0 yl(2)]) end title('plasma glucose concentration','fontsize',fontsize2); ylabel('g (mmol/l)','fontsize',fontsize1); xlabel('time (h)','fontsize',... fontsize1); if(nargout>0) plothandles =[]; if(single) plothandles = [plothandles;pd1;pu1;pr1]; end if(multi) plothandles = [plothandles;pd2;pu2;pr2]; end end A glucoseabsorptiondemo.m clear all close all clc % First call HovorkaParameters to calculate the model's parameters, which % are functions of the weight of the virtual subject. w = 70; % Weight of the virtual subject (kg) par = HovorkaParameters(w); %% % Next call HovorkaBasalState to calculate the steady state values of the % plasma glucose concentration (Gss, mmol/l), plasma insulin concentration % (Iss, mu/l), and the state vector (xss, mixed units). uss = basalinsulin(w,5,7,par); % Steady state insulin infusion rate % (mu/min); usually chosen % so that the resulting steady state glucose Gss is in % normoglycemia, or about 5 mmol/l. Basal insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par) % basal state

97 A.1 The Hovorka Model 84 %% % Finally, call HovorkaModelSimulation to simulate the model. First, % though, set up the input vectors T, x0, U, and D. T = (0:5:3*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. x0=xss; % Initial value for the state vector. U=uss*ones(length(T),1); % U is the insulin infusion rate (mu/min). This command specifies it as the % basal rate, uss, for the duration of the simulation. % U([ ]) = [ ]'+uss; % This command simulates insulin boluses of 1.5, 2.25, and 3.0 U (300, 450, % and 600 mu/min for 5 minutes) at simulation times of 1, 5, and 11 hours. % If the simulation started at 7.00, for example, then these times would % simulate an 8.00 breakfast, lunch, and dinner. D1=zeros(length(T),1); D2 = D1; D3 = D1; D1(1) = 10; D2(1) = 20; D3(1) = 30; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. %G: plasma glucose concentration (mmol/l) [T,G1,I,X1]=HovorkaModelSimulation(T,x0,U,D1,par); [T,G2,I,X2]=HovorkaModelSimulation(T,x0,U,D2,par); [T,G3,I,X3]=HovorkaModelSimulation(T,x0,U,D3,par); % plots of UG UG1 = X1(:,2)/par.tauD; UG2 = X2(:,2)/par.tauD; UG3 = X3(:,2)/par.tauD; figure(1) linewidth = 2.5; pu1 = plot(t,ug1,'linewidth',linewidth) hold on pu2 = plot(t,ug2,'r ','LineWidth',lineWidth) pu3 = plot(t,ug3,'g:','linewidth',linewidth) plot([40 40],[0 7],'k.','LineWidth',lineWidth) hold off xlabel('time (min)','fontsize',18) ylabel('u_g (mmol/min)','fontsize',20) leg = legend([pu1,pu2,pu3],'u_g(t) with 10 g/min meal',... 'U_G(t) with 20 g/min meal','u_g(t) with 30 g/min meal') set(leg,'fontsize',16)

98 A.2 Model Predictive Control 85 A.2 Model Predictive Control A SoftPenalty.m clc clear all close all disp('beginning') normo = 0; % normoglycemia ymin = 1; ymax = 3; e = 2:.01:4; % The penalty function pen = zeros(size(e)); % the coefficient above normoglycemia kappa2 =.5; pen(e normo) = kappa2*e(e normo).^2; kappa1 = pen(e==ymax)/ymin^2; % coefficient below normoglycemia such that % the penalty at ymin and ymax are equal kappa = [kappa1;kappa2;15;2] pen(e<normo) = kappa(1)*e(e<normo).^2; pen(e<ymin) = pen(e<ymin)+kappa(3)*e(e 0 & e>e(1) ymin).^2; pen(e ymax) = pen(e ymax)+kappa(4)*e(e 0 & e e(end) ymax).^2; figure(1) plot([5 5],[0 max(pen)],'g','linewidth',2.5) hold on plot(e+5,e.^2,'linewidth',2.5) plot(e+5,pen,'r ','LineWidth',2.5) plot([4 4],[0 max(pen)],'k') plot([8 8],[0 max(pen)],'k') hold off axis([min(e)+5 max(e)+5 min(pen) max(pen)]) xlabel('glucose concentration, G (mmol/l)','fontsize',18) ylabel('penalty function','fontsize',18) leg = legend('normoglycemia','symmetric penalty function',... 'Asymmetric penalty function'); set(leg,'fontsize',16)

99 Appendix B MatLab code for Simulation Results and Analysis The Matlab code for the simulation results and analysis sections are written here in the same order as the sections. B.1 Basal Insulin Infusion Rate B basalinsulin.m function [uss,xss] = basalinsulin(bw,r,uss0,par,tolfun) % Calculates the basal insulin induction to the given body weight using the % Hovorka model. % % [uss,xss] = basalinsulin(bw,r,uss0,par) % uss = basalinsulin(bw,r,uss0,par) % % Input: % bw: The body weight of the simulated person (kg) % r: The plasma glucose concentration (mmol/l) at normoglycemia % uss0: Initialisation of the basal insulin induction used in the % optimization (mu/min)

100 B.1 Basal Insulin Infusion Rate 87 % par: Hovorka model parameters (can be calculated from % HovorkaParameters.m) % % Output: % uss: The optimal basal insulin injection % xss: The steady state % % MMNN %% Initial values dss0 = 0; % The initial basal state [Gss0,Iss0,xss0] = HovorkaBasalState(uss0,dss0,par); % basal state xu0 = [xss0;uss0]; %% Basal insulin if(nargin == 5) options = optimset('nonleqnalgorithm','gn','display','off','tolfun'...,tolfun,'tolx',1e 10); else options = optimset('nonleqnalgorithm','gn','display','off'); end [xu,fval,exitflag] = fsolve(@basalinsulinderivatives,xu0,options,r,par); if(exitflag 1) bw warning(['in optimizing [f(x,u,d);g(x)]=0 fsolve returned the exitflag '... num2str(exitflag) '. (see help fsolve)']); end %% The resulting basal state uss = xu(end); if(nargout>1) xss = xu(1:10); end B BasalInsulinCalc.m clc clear all close all %% Multiple uss bwer = 40:0.2:160; r = 5; a = 0.09 % slope of the uss line b = 0 uss0er = bwer*a+b; uss = zeros(size(bwer)); for i = 1:length(bwer)% (kg) par = HovorkaParameters(bwer(i));% the general parameters for the

101 B.1 Basal Insulin Infusion Rate 88 end % Hovorka model uss(i) = basalinsulin(bwer(i),r,uss0er(i),par); % Basal insulin as function of bodyweight figure(1) plot(bwer,uss,'.','linewidth',2.5) xlabel('body weight (kg)','fontsize',18); ylabel('basal insulin (mu/min)',... 'FontSize',18) a = (uss(end) uss(1))/(bwer(end) bwer(1)) b = uss(end) a*bwer(end) % my optimal uss: myuss = uss(bwer==53) %% Checking a found uss id = bwer==45; uss1 = uss(id) par = HovorkaParameters(bwer(id)); [Gss,Iss,xss] = HovorkaBasalState(uss1,0,par) % basal state T = (0:1)'; U = ones(size(t))*uss1; D = zeros(size(t)); %G: plasma glucose concentration (mmol/l) [T,G,I,X]=HovorkaModelSimulation(T,xss,U,D,par); figure(3) plot([t(1),t(end)],[5 5],'g','LineWidth',2.5) hold on plot([t(1),t(end)],[4 4],'r') plot([t(1),t(end)],[8 8],'r') plot(t,g,'linewidth',2.5) hold off ylabel('g (mmol/l)','fontsize',18); xlabel('time (h)','fontsize',18); B BasalInsulinDerivatives.m function H=BasalInsulinDerivatives(xu,r,par) % Calculates the change in state and difference between the glucose % concentration and the optimal concentration % H=HovorkaBasalInsulin(xu,t,par) % % Input: % xu: Vector concisting of two variables: the state, x (as given % in 'HovorkaModel.m') and the basal insulin induction uss % (mu/min) % r: The plasma glucose concentration (mmol/l) at normoglycemia % par: The general parameters for the Hovorka model (can be calculated % with 'HovorkaParameters.m') % Output:

102 B.2 Simulations with Meals 89 % H: Vector concisting of two variables: the change in state, % xdot = f(x,u,d) and the difference between the glucose concentration % and the optimal concentration, y r = g(x) r % % MMNN % xu is divided into x and u x = xu(1:10); % the state uss = xu(11); % The meals: d = 0; % Since it is only basal, there are no meals % The change in state, f(x,u,d) is calculated xdot = HovorkaModel(0,x,uss,d,par); % The distance from the glucose concentration to the optimal concetration % of r = 5 mmol/l %r = 5; % the optimal glucose concentration (mmol/l) y = x(5)/par.vg; yr = y r; % The change in state and the difference between the glucose concentration % and the optimal concentration are returned H = [xdot;yr]; B.2 Simulations with Meals B meal13bolus.m clear all close all clc % First call HovorkaParameters to calculate the model's parameters, which % are functions of the weight of the virtual subject. w = 70; % Weight of the virtual subject (kg) par = HovorkaParameters(w); %% Basal insulin % Next call HovorkaBasalState to calculate the steady state values of the % plasma glucose concentration (Gss, mmol/l), plasma insulin concentration % (Iss, mu/l), and the state vector (xss, mixed units). uss = basalinsulin(w,5,7,par); % Steady state insulin infusion rate % (mu/min); usually chosen % so that the resulting steady state glucose Gss is in % normoglycemia, or about 5 mmol/l. Basal insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min).

103 B.2 Simulations with Meals 90 % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par); % basal state %% Simulations % Finally, call HovorkaModelSimulation to simulate the model. First, % though, set up the input vectors T, x0, U, and D. T = (0:5:24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. x0=xss; % Initial value for the state vector. U=uss*ones(length(T),1); U2 = U; U3 = U; U2(13) = 600; U3(13) = 450; % U is the insulin infusion rate (mu/min). This command specifies it as the % basal rate, uss, for the duration of the simulation. D=zeros(length(T),1); D(13) = 10; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. %G: plasma glucose concentration (mmol/l) [T,G,I,X]=HovorkaModelSimulation(T,x0,U,D,par); [T,G2,I,X2]=HovorkaModelSimulation(T,x0,U2,D,par); [T,G3,I,X3]=HovorkaModelSimulation(T,x0,U3,D,par); Ghypo = 4; Ghyper = 8; R = ones(size(t))*5; leg = {'Setpoint','Basal insulin',['bolus = ' num2str(u2(13))],... ['Bolus = ' num2str(u3(13))]}; HovorkaFigures(T,D,U,[G G2 G3],Ghypo,Ghyper,R,1,0,leg) B HovorkaTimePeriodTest.m clear all close all clc % Testing the glucose concentration with a meal consumed in five and in ten % minutes. % First call HovorkaParameters to calculate the model's parameters, which % are functions of the weight of the virtual subject.

104 B.2 Simulations with Meals 91 w = 70; % Weight of the virtual subject (kg) par = HovorkaParameters(w); %% % Next call HovorkaBasalState to calculate the steady state values of the % plasma glucose concentration (Gss, mmol/l), plasma insulin concentration % (Iss, mu/l), and the state vector (xss, mixed units). r = 5; uss0 = 7; uss = basalinsulin(w,r,uss0,par); % Steady state insulin infusion rate % (mu/min); usually chosen % so that the resulting steady state glucose Gss is in % normoglycemia, or about 5 mmol/l. Basal insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par) % basal state %% % Finally, call HovorkaModelSimulation to simulate the model. First, % though, set up the input vectors T, x0, U, and D. T = (0:5:.5*24*60)';% Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. x0 = xss; % Initial value for the state vector. U = uss*ones(length(t),1); % U is the insulin infusion rate (mu/min). This command specifies it as the % basal rate, uss, for the duration of the simulation. U(10) = 700+uss; % This command simulates insulin boluses of 1.5, 2.25, and 3.0 U (300, 450, % and 600 mu/min for 5 minutes) at simulation times of 1, 5, and 11 hours. % If the simulation started at 7.00, for example, then these times would % simulate an 8.00 breakfast, lunch, and dinner. D1=zeros(length(T),1); D2=D1; D1(13) = 20; D2(13:14) = [10 10]'; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. linewidth = 2.5; figure(1) stairs(t/60,d2,'linewidth',linewidth) hold on stairs(t/60,d1,'k ','LineWidth',lineWidth)

105 B.3 Simulations of a Typical Day 92 hold off fontsize1 = 18; fontsize2 = 20; fontsize3 = 16; legend1 = legend('meal 1','Meal 2') set(legend1,'fontsize',fontsize3) ylabel('d (g CHO/min)','FontSize',fontSize1); xlabel('time (h)','fontsize',fontsize1); figure(2) stairs(t/60,u,'linewidth',linewidth) ylabel('insluin infusion rate, U (mu/min)','fontsize',fontsize1); xlabel('time (h)','fontsize',fontsize1); %G: plasma glucose concentration (mmol/l) [T,G1,I1,X1]=HovorkaModelSimulation(T,x0,U,D1,par); [T,G2,I2,X2]=HovorkaModelSimulation(T,x0,U,D2,par); figure(3) H(1) = plot([t(1),t(end)/60],[5 5],'g','LineWidth',lineWidth) hold on plot([t(1),t(end)/60],[4 4],'k') plot([t(1),t(end)/60],[8 8],'k') H(2) = plot(t/60,g2,'linewidth',linewidth) H(3) = plot(t/60,g1,'k ','LineWidth',lineWidth) hold off legend2 = legend(h,'normoglycemia','glucose concentration 1',... 'Glucose concentration 2') set(legend2,'fontsize',fontsize3) ylabel('g (mmol/l)','fontsize',fontsize1); xlabel('time (h)','fontsize',fontsize1); figure(4) plot(t/60,i1,'linewidth',linewidth) plot(t/60,i2,'linewidth',linewidth) xlabel('time (h)','fontsize',fontsize1); ylabel('i (mu/l)','fontsize',fontsize1) B.3 Simulations of a Typical Day B MMNNSimulation.m clc clear all close all % This is a simulation of my meals on thursday %% Body weight and general parameters w = 53;% (kg)

106 B.3 Simulations of a Typical Day 93 par = HovorkaParameters(w);% the general parameters for the Hovorka model %% Steady state % the steady state parameters are calculated r = 5; % (mmol/l) glucose concentration at normoglycemia uss0 = 7; % (mu/min) starting guess for basal insulin infusion rateuss uss = basalinsulin(w,r,uss0,par); % steady state insulin fusion rate % (mu/min) (found in BasalInsulinCalc.m) dss = 0; % steady state meal disturbance (g CHO/min) (You don't eat 24/7) % From basal insulin and disturbance and the general parameters comes the % basal plasma glucose concetration, the basel plasma insulin concentration % and the steady state [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par); % basal state %% Meals and insulin boluses % The time: 24 h with 5 min intervals (starting at 6:00) T=(0:5:24*60)'; % The meals (g CHO/min) D = ones(size(t))*dss; % ================== % Meals (g/min): % Breakfast: D(42)=sum([ ]); % Lunch: D(78)=sum(ones(4,1)*1.83); % Afternoon snack D(120)=sum([ ]); % Dinner D(153)=sum(ones(4,1)*3.85); % Evening snack D(180)=sum([ ]); % ================== % Insulin (mu/min): U = ones(size(t))*uss; % basal insulin rate % U(42) = 700; % U(78) = 250; % U(120) = 200; % U(153) = 490; % U(180) = 210; %% The simulation [T,G,I,X] = HovorkaModelSimulation(T,xss,U,D,par); %% Figures Ghypo = 4; Ghyper = 8; R = ones(size(t))*5; HovorkaFigures(T,D,U,G,Ghypo,Ghyper,R,1,2)

107 B.4 Sensitivity Analysis 94 B.4 Sensitivity Analysis B SensitivityDemo.m clc clear all close all % Insulin sensitivity analysis %% Body weight and general parameters w = 70;% (kg) par1 = HovorkaParameters(w);% the general parameters for the Hovorka model par2 = par1; par3 = par1; par4 = par1; % The sensitivity to be analysed par2.kb1 = par2.kb1*.75; par3.kb1 = par3.kb1*.50; par4.kb1 = par4.kb1*.25; %% Steady state % the steady state parameters are calculated r = 5; % (mmol/l) glucose concentration at normoglycemia uss0 = 7; % (mu/min) starting guess for basal insulin infusion rateuss [uss1,xss1] = basalinsulin(w,r,uss0,par1); % steady state insulin fusion % rate (mu/min) (found in BasalInsulinCalc.m) [uss2,xss2] = basalinsulin(w,r,uss0,par2); % steady state insulin fusion % rate (mu/min) (found in BasalInsulinCalc.m) [uss3,xss3] = basalinsulin(w,r,uss0,par3); % steady state insulin fusion % rate (mu/min) (found in BasalInsulinCalc.m) [uss4,xss4] = basalinsulin(w,r,uss0,par4); % steady state insulin fusion % rate (mu/min) (found in BasalInsulinCalc.m) dss = 0; % steady state meal disturbance (g CHO/min) (You don't eat 24/7) %% Meals and insulin boluses % The time: 24 h with 5 min intervals (starting at 6:00) T=(0:5:2*24*60)'; % The meals (g CHO/min) D = ones(size(t))*dss; D([ ]') = [9;12;15]; % D(13) = 9; % D(61) = 12; % D(133) = 15; % ================== % Insulin (mu/min):

108 B.5 Intravenous Insulin Infusion 95 % U(13) = 400; % U(61) = 440; % U(133) = 500; Ubolus = zeros(size(t)); Ubolus([ ]') = [ ]'; % only the boluses! U1 = ones(size(t))*uss1+ubolus; % basal insulin rate U2 = ones(size(t))*uss2+ubolus; % basal insulin rate U3 = ones(size(t))*uss3+ubolus; % basal insulin rate U4 = ones(size(t))*uss4+ubolus; % basal insulin rate %% The simulation [T,G1,I,X1] = HovorkaModelSimulation(T,xss1,U1,D,par1); [T,G2,I,X2] = HovorkaModelSimulation(T,xss2,U2,D,par2); [T,G3,I,X3] = HovorkaModelSimulation(T,xss3,U3,D,par3); [T,G4,I,X4] = HovorkaModelSimulation(T,xss4,U4,D,par4); %% Figures Ghypo = 4; Ghyper = 8; R = ones(size(t))*5; legg = {'Setpoint','Glucose concentration with 100%',... 'Glucose concentration with 75%','Glucose concentration with 50%',... 'Glucose concentration with 25%'}; HovorkaFigures(T,D,U1,[G1 G2 G3 G4],Ghypo,Ghyper,R,1,0,legG) % Insulin concentration figure(4) plot(t/60,i,'linewidth',2.5) xlabel('time (h)','fontsize',18); ylabel('insulin concentration (mu/l)','fontsize',18); B.5 Intravenous Insulin Infusion B HovorkaFastDemo2Person.m clear all close all clc % Hovorka fast insulin absorbtion (insulin absorption constant set to 1 min), % 2 persons, alpha from trial & error on 70 kg person % First call HovorkaParameters to calculate the model's parameters, which % are functions of the weight of the virtual subject. w1 = 70; % Weight of the virtual subject (kg) w2 = 100; % second person for comparison par1 = HovorkaFastParameters(w1);

109 B.5 Intravenous Insulin Infusion 96 par2 = HovorkaFastParameters(w2); par1.taus = 1; par2.taus = 1; disp(['par1.taus is ' num2str(par1.taus)]) %% % Next call HovorkaBasalState to calculate the steady state values of the % plasma glucose concentration (Gss, mmol/l), plasma insulin concentration % (Iss, mu/l), and the state vector (xss, mixed units). %calculating optimal uss r = 5; uss0 = 7; [uss1 xss1] = basalinsulin(w1,r,uss0,par1); [uss2 xss2] = basalinsulin(w2,r,uss0,par2); % Steady state insulin infusion rate (mu/min); usually chosen % so that the resulting steady state glucose Gss is in % normoglycemia, or about 5 mmol/l. Basal insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. %% % Finally, call HovorkaModelSimulation to simulate the model. First, % though, set up the input vectors T, x0, U, and D. T = (0:5:24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. D=zeros(length(T),1); D([ ]') = [9;12;15]; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. %G: plasma glucose concentration (mmol/l) alpha = 6 % found by trial and error on 70 kg person [T,G1,I1,X1,U1]=HovorkaModelSimulationFast(T,xss1,uss1,D,r,alpha,par1); [T,G2,I2,X2,U2]=HovorkaModelSimulationFast(T,xss2,uss2,D,r,alpha,par2); %Simulations with basal insulin only [T,Gb1,Ib1,Xb1]=HovorkaModelSimulation(T,xss1,ones(size(T))*uss1,D,par1); [T,Gb2,Ib2,Xb2]=HovorkaModelSimulation(T,xss2,ones(size(T))*uss2,D,par2); %% Figures Ghypo = 4; Ghyper = 8; R = ones(size(t))*5; leg = {'Setpoint','Proportional model for 70 kg subject',... 'Proportional model for 100 kg subject'}; legu = {'Insulin infusion rate for 70 kg subject',...

110 B.5 Intravenous Insulin Infusion 97 'Insulin infusion rate for 100 kg subject'}; HovorkaFigures(T,D,[U1 U2],[G1 G2],Ghypo,Ghyper,R,1,[3 1],leg,legU) figure(13) subplot(3,1,1) ylim([4 13]) leg = {'Setpoint','Proportional model for 70 kg subject',... 'Proportional model for 100 kg subject','basal insulin for 70 kg subject',... 'Basal insulin for 100 kg subject'}; HovorkaFigures(T,D,[U1 U2],[G1 G2 Gb1 Gb2],Ghypo,Ghyper,R,1,[3 1],leg,legU) figure(12) ylim([4 30]) B HovorkaFastPDDemo2Person.m clear all close all clc % Hovorka fast insulin absorbtion (insulin absorption constant set to 1 min), % 2 persons, alpha from trial & error on 70 kg person % First call HovorkaParameters to calculate the model's parameters, which % are functions of the weight of the virtual subject. w1 = 70; % Weight of the virtual subject (kg) w2 = 100; % second person for comparison par1 = HovorkaFastPIDParameters(w1); par2 = HovorkaFastPIDParameters(w2); par1.taus = 1; par2.taus = 1; disp(['par1.taus is ' num2str(par1.taus)]) %% % Next call HovorkaBasalState to calculate the steady state values of the % plasma glucose concentration (Gss, mmol/l), plasma insulin concentration % (Iss, mu/l), and the state vector (xss, mixed units). %calculating optimal uss r = 5; uss0 = 7; [uss1 xss1] = basalinsulin(w1,r,uss0,par1); [uss2 xss2] = basalinsulin(w2,r,uss0,par2); % Steady state insulin infusion rate (mu/min); usually chosen % so that the resulting steady state glucose Gss is in % normoglycemia, or about 5 mmol/l. Basal insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. %%

111 B.6 Model Predictive Control Simulations 98 % Finally, call HovorkaModelSimulation to simulate the model. First, % though, set up the input vectors T, x0, U, and D. T = (0:5:24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. D=zeros(length(T),1); D([ ]') = [9;12;15]; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. %G: plasma glucose concentration (mmol/l) alpha = 19 % found by trial and error on 70 kg person alphad = 50 % found by trial and error on 70 kg person [T,G1,I1,X1,U1]=HovorkaModelSimulationFastPID(T,xss1,uss1,D,r,alpha,alphaD,par1); [T,G2,I2,X2,U2]=HovorkaModelSimulationFastPID(T,xss2,uss2,D,r,alpha,alphaD,par2); max(g1) %% Figures Ghypo = 4; Ghyper = 8; R = ones(size(t))*5; leg = {'Setpoint','Glucose concentration for 70 kg subject',... 'Glucose concentration for 100 kg subject'}; legu = {'Insulin infusion rate for 70 kg subject',... 'Insulin infusion rate for 100 kg subject'}; HovorkaFigures(T,D,[U1 U2],[G1 G2],Ghypo,Ghyper,R,1,[3 1],leg,legU) B.6 Model Predictive Control Simulations B.6.1 Simulations with a Third Order ARX Model B MPC3Demo.m clc clear all close all disp('beginning') % The ARX model identified from the Hovorka model. % Gives A, B and arxmod (se get(arxmod)) load('arxmodel.mat') % number of predictions m = 2.25*12; lambda = 10000;

112 B.6 Model Predictive Control Simulations 99 [A,B,C,K] = statespacemodel(a(2:end),b(2:end)); % model order n = length(b); %% MPC initialization w = 70; % Weight of the virtual subject (kg) par = HovorkaParameters(w); uss = basalinsulin(w,5,7,par); % Steady state insulin infusion rate (mu/min); % usually chosen so that the resulting steady state glucose Gss % is in normoglycemia, or about 5 mmol/l. Basal insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par); % basal state T = (0:5:24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. D=zeros(length(T),1); D(13) = 9; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. hstate0 = xss; u0 = uss; umin = uss; umax = 600; DeltaUmin = 300; DeltaUmax = 300; Rdiff = zeros(size(t)); mpcstate0 = zeros(n,1); % converges to this at steady state %% MPC [Udiff,mpcStateReturn,hStateReturn,Ydiff,eReturn,yPred] = computempc(m,... A,B,C,K,lambda,Rdiff,umin,umax,DeltaUmin,DeltaUmax,T,hState0,mpcState0,... D,u0,Gss,par); U = Udiff+uss; Y = Ydiff+Gss; R = Rdiff+5; %G: plasma glucose concentration (mmol/l) [T,G,I,X]=HovorkaModelSimulation(T,xss,U,D,par); Ghypo = 4; Ghyper = 8; leg = {'Setpoint, R','Glucose concentration, G',[num2str(round(m/8)/12)... 'h prediction'],[num2str(round(m/2)/12) 'h prediction'],... [num2str(round(m)/12) 'h prediction'], 'Basal insulin'}; HovorkaFigures(T,D,U,[Y ypred(:,round(m/8))+gss ypred(:,round(m/2))+gss...

113 B.6 Model Predictive Control Simulations 100 ypred(:,end)+gss G],Ghypo,Ghyper,R,1,1,leg); B MPC3SetpointDemo.m clc clear all close all disp('beginning') % Experimenting with changing the setpoint in the beginning. % The ARX model identified from the Hovorka model. % Gives A, B and arxmod (se get(arxmod)) load('arxmodel.mat') % model order n = length(a); m = 48; lambda = 1e 2; [A,B,C,K] = statespacemodel(a,b); %% MPC initialization w = 70; % Weight of the virtual subject (kg) par = HovorkaParameters(w); Rss = 8; % the initial setpoint (meaning 5 is usual) uss0 = 6; % starting guess for the basal insulin infusion rate uss = basalinsulin(w,rss,uss0,par,1e 8); % Steady state insulin infusion rate (mu/min); % usually chosen so that the resulting steady state glucose Gss is in % normoglycemia, or about 5 mmol/l. Basal insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par); % basal state T = (0:5:24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. D=zeros(length(T),1); % D(13) = 9; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. hstate0 = xss; u0 = uss; umin = uss; umax = 600; DeltaUmin = 300;

114 B.6 Model Predictive Control Simulations 101 DeltaUmax = 300; Rdiff = zeros(size(t)); Rdiff(12*3+1:end) = 5 Rss; mpcstate0 = zeros(n,1); % It starts at steady state %% MPC [Udiff,mpcStateReturn,hStateReturn,Ydiff,eReturn,yPred] = computempc(m,... A,B,C,K,lambda,Rdiff,umin,umax,DeltaUmin,DeltaUmax,T,hState0,... mpcstate0,d,u0,gss,par); U = Udiff+uss; Y = Ydiff+Gss; R = Rdiff+Gss; Ghypo = 4; Ghyper = 8; HovorkaFigures(T,D,U,Y,Ghypo,Ghyper,R,1,2,leg) B designmpc.m function [H,Phix,Phie,Psi,Gamma] = designmpc(m,a,b,c,k,lambda) % [H,Phix,Phie,Psi] = designmpc(m,a,b,c,k) % m: number of predictions % A, B, C, K should be given as in Bagterp slide 3 n = length(c); %% Phi % size of Phix will be as if m C vectors were placed above each other Phix = zeros(m,n); Phie = zeros(m,1); % since CK, CAK,... are skalars Phix(1,:) = C*A; Phie(1) = C*K; for i=2:m Phix(i,:)=Phix(i 1,:)*A; Phie(i) = C*A^(i 1)*K; end %% Psi % Bagterp slides 8 Psi = eye(m); Psi(2:m+1:m^2) = 1; % the sloping line right below the diagonal is set to 1 %% H mat = zeros(m,1); mat(1) = C*B; Gamma = mat(1)*eye(m); for j=2:m mat(j) = Phix(j 1,:)*B; Gamma(j:m+1:m^2+m j*m) = mat(j); % the sloping line j below the % diagonal is set to H(j) end

115 B.6 Model Predictive Control Simulations 102 H = Gamma'*Gamma+lambda*Psi'*Psi; B statespacemodel.m function [A,Bvec,C,K] = statespacemodel(avec,bvec) % Makes the coefficient matrices for the state space model (Bagterp slides % 3) from the two vectors of ARX coefficients. % % [A,B,C,K] = statespacemodel(avec,bvec) n = length(avec); % matrices in Bagterp slides 3 K = Avec; A = [K [eye(n 1); zeros(1,n 1)]]; C = eye(1,n); B computempc.m function [udiff,mpcstatereturn,hstatereturn,yreturn,ereturn,ypred] = computempc(m,... A,B,C,K,lambda,R,Umin,Umax,DeltaUmin,DeltaUmax,T,hState0,mpcState0,D,uss,Gss,par) %% Initialization % MPC model order n = length(b); ereturn = zeros(size(t)); % the mpc model coefficients [H,Phix,Phie,Psi,Gamma] = designmpc(m,a,b,c,k,lambda); % the found insulin infusion rates to be added to the basal insulin infusion rate udiff = zeros(size(t)); upred = 0; udiff(1) = upred; % the MPC state mpcstatereturn = zeros(length(t),n); mpcstate = mpcstate0; mpcstatereturn(1,:) = mpcstate'; mpcstatereturn(2,:) = mpcstate'; % the initial Hovorka state hstate = hstate0'; hstatereturn = zeros(length(t),length(hstate)); hstatereturn(1,:) = hstate; % the glucose concentration Yreturn = zeros(size(t)); [T1,y,I1,hState] = HovorkaModelSimulation(T(1),hState',uDiff(1)+uss,D(1),par); Yreturn(1) = y Gss;

116 B.6 Model Predictive Control Simulations 103 % the predicted glucose concentration ypred = zeros(length(t),m); % making R bigger so it can be used for all the predictions R = [R;R(end)*ones(m,1)]; % for quadprog options = optimset('largescale','off','display','off','maxiter',300); %% for j = 2:length(T) % updating glucose concentration [T1,y,I1,hState] = HovorkaModelSimulation(T(j 1:j),hState',... udiff(j 1:j)+uss,D(j 1:j),par); y = y(2) Gss; % glucose concentration at this time measure hstate = hstate(2,:); % Hovorka state at this time measure hstatereturn(j,:) = hstate; Yreturn(j) = y; end % model prediction error e = y C*mpcState; ereturn(j) = e; b = Phix*mpcState+Phie*e; c = b R(j:j+m 1); g = Gamma'*c Psi'*(eye(m,1)*lambda*upred); bl = DeltaUmin + eye(m,1)*upred; bu = DeltaUmax + eye(m,1)*upred; % The QP ineqcoef = [ Psi;Psi]; ineqright = [ bl;bu]; [U,fval,exitflag] = quadprog(h,g,ineqcoef,ineqright,[],[],umin,umax,[],... options); if(exitflag 1) warning(['in optimizing the QP, quadprog returned the exitflag '... num2str(exitflag) '. (see help quadprog)']); end % updating the insulin prediction upred = U(1); udiff(j) = upred; % the predicted glucose concentration ypred(j,:) = Phix*mpcState+Phie*e+Gamma*U; % updating the mpc state mpcstate = A*mpcState + B*upred + K*e; mpcstatereturn(j+1,:) = mpcstate';

117 B.6 Model Predictive Control Simulations 104 B.6.2 Simulations with a Ninth Order ARX Model with Meal Input B MPCMISODemo.m clc clear all close all disp('beginning') % The ARX model identified from the Hovorka model. % Gives A, B and arxmod (see get(arxmod)) load('arxmodel9.mat') % get(arxmod) % number of predictions m = 2*12; lambda = 100; [A,B,Bd,C,K] = statespacemodelmiso(arxmod.a(2:end)',arxmod.b(:,2:end)'); % model order n = length(b); %% MPC initialization w = 70; % Weight of the virtual subject (kg) par = HovorkaParameters(w); uss = basalinsulin(w,5,7,par); % Steady state insulin infusion rate % (mu/min); usually chosen so that the resulting steady state % glucose Gss is in normoglycemia, or about 5 mmol/l. Basal % insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par); % basal state back = 5; % how many hours back in time the simulation starts predicting T = ( back*60:5:24*60)'; % Time vector for the simulation, in minutes. To % simulate one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. D=zeros(length(T),1); D(13+back*12) = 9; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. hstate0 = xss; umin = uss; umax = 600;

118 B.6 Model Predictive Control Simulations 105 DeltaUmin = 300; DeltaUmax = 300; Rdiff = zeros(size(t)); mpcstate0 = zeros(n,1); % converges to this at steady state % stating if future meals are known knownfuture = false; %% MPC [Udiff,mpcStateReturn,hStateReturn,Ydiff,eReturn,yPred] = computempcmiso(m,... A,B,Bd,C,K,lambda,Rdiff,umin,umax,DeltaUmin,DeltaUmax,T,hState0,... mpcstate0,d,knownfuture,uss,gss,par); U = Udiff+uss; Ubas = ones(size(udiff))*uss; Y = Ydiff+Gss; R = Rdiff+5; %G: plasma glucose concentration (mmol/l) [T,Gbas,I,X]=HovorkaModelSimulation(T,xss,Ubas,D,par); Ghypo = 4; Ghyper = 8; leg = {'Setpoint','Glucose concentration',[num2str(round(m/8)/12)... 'h prediction'],[num2str(round(m/2)/12) 'h prediction'],... [num2str(round(m)/12) 'h prediction'], 'Basal insulin'}; postimes = T 0; plothandles = HovorkaFigures(T(posTimes),D(posTimes),U(posTimes),... [Y(posTimes) ypred(t round(m/8)*5 & T T(end) round(m/8)*5,... round(m/8))+gss ypred(t round(m/2)*5 & T T(end) round(m/2)*5,... round(m/2))+gss ypred(t m*5 & T T(end) m*5,end)+gss Gbas(posTimes)],... Ghypo,Ghyper,R(posTimes),1,3,leg); B MPCMISOSetpointDemo.m clc clear all close all disp('beginning') % The ARX model identified from the Hovorka model. % Gives A, B and arxmod (see get(arxmod)) load('arxmodel9.mat') % get(arxmod) % number of predictions m = 4*12; lambda = 1e 2; [A,B,Bd,C,K] = statespacemodelmiso(arxmod.a(2:end)',arxmod.b(:,2:end)'); % model order n = length(b);

119 B.6 Model Predictive Control Simulations 106 %% MPC initialization w = 70; % Weight of the virtual subject (kg) par = HovorkaParameters(w); Rss = 8; % the initial setpoint (meaning (Rss 5) higher than usual) uss = basalinsulin(w,rss,7,par); % Steady state insulin infusion rate % (mu/min); usually chosen so that the resulting steady state % glucose Gss is in normoglycemia, or about 5 mmol/l. Basal % insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par); % basal state T = (0:5:24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. D=zeros(length(T),1); % D(13) = 9; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. hstate0 = xss; umin = uss; umax = 600; DeltaUmin = 300; DeltaUmax = 300; Rdiff = zeros(size(t)); Rdiff(12*3+1:end) = 5 Rss; mpcstate0 = zeros(n,1); % converges to this at steady state % stating if future meals are known knownfuture = false; %% MPC [Udiff,mpcStateReturn,hStateReturn,Ydiff,eReturn,yPred] = computempcmiso(m,... A,B,Bd,C,K,lambda,Rdiff,umin,umax,DeltaUmin,DeltaUmax,T,hState0,... mpcstate0,d,knownfuture,uss,gss,par); U = Udiff+uss; Y = Ydiff+Gss; R = Rdiff+Rss; Ghypo = 4; Ghyper = 8; HovorkaFigures(T,D,U,Y,Ghypo,Ghyper,R,1,2) B designmpcmiso.m

120 B.6 Model Predictive Control Simulations 107 function [H,Phix,Phie,Psi,Gamma,Gammad] = designmpcmiso(m,a,b,bd,c,k,lambda) % Calculates the constant matrices of a MISO MPC problem. The formulars are % given in Bagterp slides % % [H,Phix,Phie,Psi,Gamma,Gammad] = designmpcmiso(m,a,b,bd,c,k,lambda) % m: number of predictions % A, B, C, K should be given as in Bagterp slide 3 % Bd is the coefficient vector for the meal input n = length(c); %% Phi % size of Phix will be as if m C vectors were placed above each other Phix = zeros(m,n); Phie = zeros(m,1); % since CK, CAK,... are skalars Phix(1,:) = C*A; Phie(1) = C*K; for i=2:m Phix(i,:)=Phix(i 1,:)*A; Phie(i) = C*A^(i 1)*K; end %% Psi % Bagterp slides 8 Psi = eye(m); Psi(2:m+1:m^2) = 1; % the sloping line right below the diagonal is set to 1 %% H, Gamma & Gammad mat = zeros(m,1); matd = zeros(m,1); mat(1) = C*B; matd(1) = C*Bd; Gamma = mat(1)*eye(m); Gammad = matd(1)*eye(m); for j=2:m mat(j) = Phix(j 1,:)*B; matd(j) = Phix(j 1,:)*Bd; Gamma(j:m+1:m^2+m j*m) = mat(j); end % the sloping line j below the % diagonal is set to H(j) Gammad(j:m+1:m^2+m j*m) = matd(j); % the sloping line j below the % diagonal is set to H(j) H = Gamma'*Gamma+lambda*Psi'*Psi; B statespacemodelmiso.m function [A,B,Bd,C,K] = statespacemodelmiso(avec,bvec) % Makes the coefficient matrices for the state space model (Bagterp slides % 3) from the two vectors of ARX coefficients. B is the coefficient vector % for the insulin infusion rate input, and Bd is the coefficient vector

121 B.6 Model Predictive Control Simulations 108 % for the meal input. % % [A,B,Bd,C,K] = statespacemodel(avec,bvec) n = length(avec); % matrices in Bagterp slides 3 K = Avec; A = [K [eye(n 1); zeros(1,n 1)]]; B = Bvec(:,1); Bd = Bvec(:,2); C = eye(1,n); B computempcmiso.m function [udiff,mpcstatereturn,hstatereturn,yreturn,ereturn,... ypred] = computempcmiso(m,a,b,bd,c,k,lambda,r,umin,umax,deltaumin,... DeltaUmax,T,hState0,mpcState0,D,knownFuture,uss,Gss,par) %% Initialization % MPC model order n = length(b); % the errors ereturn = zeros(size(t)); % the mpc model coefficients [H,Phix,Phie,Psi,Gamma,Gammad] = designmpcmiso(m,a,b,bd,c,k,lambda); % the found insulin infusion rates to be added to the basal insulin infusion rate udiff = zeros(size(t)); upred = 0; udiff(1) = upred; % the MPC state mpcstatereturn = zeros(length(t),n); mpcstate = mpcstate0; mpcstatereturn(1,:) = mpcstate'; mpcstatereturn(2,:) = mpcstate'; % the initial Hovorka state hstate = hstate0'; hstatereturn = zeros(length(t),length(hstate)); hstatereturn(1,:) = hstate; % the initial glucose concentration Yreturn = zeros(size(t)); [T1,y,I1,hState] = HovorkaModelSimulation(T(1),hState',uDiff(1)+uss,D(1),par); Yreturn(1) = y Gss; % the predicted glucose concentration ypred = zeros(length(t),m);

122 B.6 Model Predictive Control Simulations 109 % making R bigger so it can be used for all the predictions R = [R;R(end)*ones(m,1)]; % the prediction of the meals Dpred = zeros(m,length(d)); if(knownfuture) % the future meals are known for j = 1:m Dpred(j,1:end j+1) = D(j:end)'; end else Dpred(1,:) = D; end % for quadprog options = optimset('largescale','off','display','off'); %% for j = 2:length(T) % updating glucose concentration [T1,y,I1,hState] = HovorkaModelSimulation(T(j 1:j),hState',... udiff(j 1:j)+uss,D(j 1:j),par); y = y(2) Gss; % glucose concentration at this time measure hstate = hstate(2,:); % Hovorka state at this time measure hstatereturn(j,:) = hstate; Yreturn(j) = y; % model prediction error e = y C*mpcState; ereturn(j) = e; b = Phix*mpcState+Phie*e+Gammad*Dpred(:,j); c = b R(j:j+m 1); g = Gamma'*c Psi'*(eye(m,1)*lambda*upred); bl = DeltaUmin + eye(m,1)*upred; bu = DeltaUmax + eye(m,1)*upred; % The QP ineqcoef = [ Psi;Psi]; ineqright = [ bl;bu]; [U,fval,exitflag] = quadprog(h,g,ineqcoef,ineqright,[],[],umin,umax,... [],options); if(exitflag 1) warning(['in optimizing the QP, quadprog returned the exitflag '... num2str(exitflag) '. (see help quadprog)']); end % updating the insulin prediction upred = U(1); udiff(j) = upred; % the predicted glucose concentration ypred(j,:) = (Phix*mpcState+Phie*e+Gamma*U+Gammad*Dpred(:,j))'; % updating the mpc state mpcstate = A*mpcState + B*upred + K*e;% + Bd*D(j);

123 B.6 Model Predictive Control Simulations 110 end mpcstatereturn(j+1,:) = mpcstate'; B ARX.m function [X,Y] = ARX(T,A,B,Bd,C,K,U,D,E,x0) % Calculates the glucose using the ARX model. % Y = ARX(T,A,B,Bd,C,K,U,D,E,x0) x = x0; X = zeros(length(t),length(x0)); Y = zeros(size(t)); for j=1:length(t) x = A*x+B*U(j)+Bd*D(j)+K*E(j);% Hovorka state at this time measure X(j,:) = x'; Y(j) = C*x; % glucose concentration at this time measure end B.6.3 MPC with E ARX B MPCMISOEDDemo.m clc clear all close all disp('beginning') % The ARX model identified from the Hovorka model. % Gives A, B and arxmod (see get(arxmod)) load('arxmodel9.mat') % get(arxmod) % number of predictions m = 48; lambda = 5; alpha =.995; [A,B,Bd,C,K] = statespacemodelmisoed(arxmod.a(2:end)',arxmod.b(:,2:end)',alpha); % model order n = length(b); [H,Phix,Phie,Psi,Gamma,Gammad] = designmpcmiso(m,a,b,bd,c,k,lambda); %% MPC initialization w = 70; % Weight of the virtual subject (kg) par = HovorkaParameters(w);

124 B.6 Model Predictive Control Simulations 111 uss = basalinsulin(w,5,7,par); % Steady state insulin infusion rate % (mu/min); usually chosen so that the resulting steady state % glucose Gss is in normoglycemia, or about 5 mmol/l. Basal % insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par); % basal state T = (0:5:2*24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. D=zeros(length(T),1); D(13) = 9; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. hstate0 = xss; umin = uss; umax = 600; DeltaUmin = 300; DeltaUmax = 300; Rdiff = zeros(size(t)); mpcstate0 = zeros(n,1); % converges to this at steady state % stating if future meals are known knownfuture = false; %% MPC [Udiff,mpcStateReturn,hStateReturn,Ydiff,eReturn,yPred] = computempcmiso(m,... A,B,Bd,C,K,lambda,Rdiff,umin,umax,DeltaUmin,DeltaUmax,T,hState0,... mpcstate0,d,knownfuture,uss,gss,par); U = Udiff+uss; Ubas = ones(size(udiff))*uss; Y = Ydiff+Gss; R = Rdiff+5; %G: plasma glucose concentration (mmol/l) [T,G,I,X]=HovorkaModelSimulation(T,xss,Ubas,D,par); Ghypo = 4; Ghyper = 8; plothandles = HovorkaFigures(T,D,U,[Y G],Ghypo,Ghyper,R,1,3); B MPCMISOEDSetpointDemo.m clc clear all

125 B.6 Model Predictive Control Simulations 112 close all disp('beginning') % The ARX model identified from the Hovorka model. % Gives A, B and arxmod (see get(arxmod)) load('arxmodel9.mat') % get(arxmod) % number of predictions m = 48; lambda = 1e0; alpha =.99; [A,B,Bd,C,K] = statespacemodelmisoed(arxmod.a(2:end)',arxmod.b(:,2:end)',alpha); % model order n = length(b); %% MPC initialization w = 70; % Weight of the virtual subject (kg) par = HovorkaParameters(w); Rss = 8; % the initial setpoint (meaning (Rss 5) higher than usual) uss = basalinsulin(w,rss,7,par); % Steady state insulin infusion rate % (mu/min); usually chosen so that the resulting steady state % glucose Gss is in normoglycemia, or about 5 mmol/l. Basal % insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par); % basal state T = (0:5:24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. D=zeros(length(T),1); % D(13) = 9; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. hstate0 = xss; umin = uss; umax = 600; DeltaUmin = 300; DeltaUmax = 300; Rdiff = zeros(size(t)); Rdiff(12*3+1:end) = 5 Rss; mpcstate0 = zeros(n,1); % converges to this at steady state % stating if future meals are known knownfuture = false;

126 B.6 Model Predictive Control Simulations 113 %% MPC [Udiff,mpcStateReturn,hStateReturn,Ydiff,eReturn,yPred] = computempcmiso(m,... A,B,Bd,C,K,lambda,Rdiff,umin,umax,DeltaUmin,DeltaUmax,T,hState0,... mpcstate0,d,knownfuture,uss,gss,par); U = Udiff+uss; Ubas = ones(size(udiff))*uss; Y = Ydiff+Gss; R = Rdiff+Rss; %G: plasma glucose concentration (mmol/l) [T,G,I,X]=HovorkaModelSimulation(T,xss,Ubas,D,par); Ghypo = 4; Ghyper = 8; leg = {'Setpoint, R','Glucose concentration, G',[num2str(round(m/8)/12)... 'h prediction'],[num2str(round(m/2)/12) 'h prediction'],... [num2str(round(m)/12) 'h prediction']}; HovorkaFigures(T,D,U,[Y ypred(:,round(m/8))+gss ypred(:,round(m/2))+gss... ypred(:,end)+gss],ghypo,ghyper,r,1,1,leg) B statespacemodelmisoed.m function [A,B,Bd,C,K] = statespacemodelmisoed(avec,bvec,alpha) % Makes the coefficient matrices for the state space model for the Extended % Delta ARX model (Bagterp slides 3) and Huusom from the two vectors of ARX % coefficients. B is the coefficient vector for the insulin infusion rate % input, and Bd is the coefficient vector for the meal input. % % [A,B,Bd,C,K] = statespacemodel(avec,bvec) n = length(avec); % The Delta ARX A coefficients AvecBar = [1;Avec]; AvecBar(1:end 1) = AvecBar(1:end 1) Avec; % The Delta ARX B and Bd coefficients B = [Bvec(:,1);0]; B(2:end) = B(2:end) Bvec(:,1); Bd = [Bvec(:,2);0]; Bd(2:end) = Bd(2:end) Bvec(:,2); % The K vector (Huusom (20)) K = AvecBar; K(1) = K(1) alpha; A = [AvecBar [eye(n); zeros(1,n)]]; C = eye(1,n+1);

127 B.6 Model Predictive Control Simulations 114 B.6.4 Sensitivities B MPCMISOEDparDemo.m clc clear all close all disp('beginning') % The ARX model identified from the Hovorka model. % Gives A, B and arxmod (see get(arxmod)) load('arxmodel9.mat') % get(arxmod) % number of predictions m = 48; lambda =.001; alpha = 0 [A,B,Bd,C,K] = statespacemodelmisoed(arxmod.a(2:end)',arxmod.b(:,2:end)',alpha); % model order n = length(b); [H,Phix,Phie,Psi,Gamma,Gammad] = designmpcmiso(m,a,b,bd,c,k,lambda); %% MPC initialization T = (0:5:2*24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. w = 70; % Weight of the virtual subject (kg) par1 = HovorkaParameters(w); par2 = par1; par2.egp0 = par2.egp0*.75; par = [par1 par2]; parchoice = ones(length(t),1); parchoice(13:end) = 2; uss = basalinsulin(w,5,7,par1); % Steady state insulin infusion rate % (mu/min); usually chosen so that the resulting steady state % glucose Gss is in normoglycemia, or about 5 mmol/l. Basal % insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par1); % basal state D=zeros(length(T),1); % D(13) = 9; % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1,

128 B.6 Model Predictive Control Simulations 115 % 5, and 11 hours. hstate0 = xss; umin = uss; umax = 600; DeltaUmin = 300; DeltaUmax = 300; Rdiff = zeros(size(t)); mpcstate0 = zeros(n,1); % converges to this at steady state % stating if future meals are known knownfuture = false; %% MPC [Udiff,mpcStateReturn,hStateReturn,Ydiff,eReturn,yPred] =... computempcmisopar(m,a,b,bd,c,k,lambda,rdiff,umin,umax,deltaumin,... DeltaUmax,T,hState0,mpcState0,D,knownFuture,uss,Gss,par,parChoice); U = Udiff+uss; Ubas = ones(size(t))*uss; Y = Ydiff+Gss; R = Rdiff+5; %G: plasma glucose concentration (mmol/l) [T,Gbas,I,X]=HovorkaModelSimulationPar(T,xss,Ubas,D,par,parChoice); Ghypo = 4; Ghyper = 8; plothandles = HovorkaFigures(T,D,U,Y,Ghypo,Ghyper,R,1,3); subplot(3,1,3) parchange = ones(length(parchoice),1)*par(1).egp0; parchange(parchoice==2) = par(2).egp0; stairs(t/60,parchange,'linewidth',2.5) title('parameter change EGP_0','FontSize',20) xlabel('time (h)','fontsize',18) ylabel('(mmol/(kg min))','fontsize',18) B computempcmisopar.m function [udiff,mpcstatereturn,hstatereturn,yreturn,ereturn,ypred] =... computempcmisopar(m,a,b,bd,c,k,lambda,r,umin,umax,deltaumin,... DeltaUmax,T,hState0,mpcState0,D,knownFuture,uss,Gss,par,parChoice) % par: is a matrix with two sets of parameters % parchoice: a vector with ones where the first set of parameters should % be used, a 2 where second set etc. %% Initialization % MPC model order n = length(b); % the errors

129 B.6 Model Predictive Control Simulations 116 ereturn = zeros(size(t)); % the mpc model coefficients [H,Phix,Phie,Psi,Gamma,Gammad] = designmpcmiso(m,a,b,bd,c,k,lambda); % the found insulin infusion rates to be added to the basal insulin infusion rate udiff = zeros(size(t)); upred = 0; udiff(1) = upred; % the MPC state mpcstatereturn = zeros(length(t),n); mpcstate = mpcstate0; mpcstatereturn(1,:) = mpcstate'; mpcstatereturn(2,:) = mpcstate'; % the initial Hovorka state hstate = hstate0'; hstatereturn = zeros(length(t),length(hstate)); hstatereturn(1,:) = hstate; % the initial glucose concentration Yreturn = zeros(size(t)); [T1,y,I1,hState] = HovorkaModelSimulation(T(1),hState',uDiff(1)+uss,D(1),par(1)); Yreturn(1) = y Gss; % the predicted glucose concentration ypred = zeros(length(t),m); % making R bigger so it can be used for all the predictions R = [R;R(end)*ones(m,1)]; % the prediction of the meals Dpred = zeros(m,length(d)); if(knownfuture) % the future meals are known for j = 1:m Dpred(j,1:end j+1) = D(j:end)'; end else Dpred(1,:) = D; end % for quadprog options = optimset('largescale','off','display','off'); %% for j = 2:length(T) % updating glucose concentration [T1,y,I1,hState] = HovorkaModelSimulation(T(j 1:j),hState',... udiff(j 1:j)+uss,D(j 1:j),par(parChoice(j))); y = y(2) Gss; % glucose concentration at this time measure hstate = hstate(2,:); % Hovorka state at this time measure hstatereturn(j,:) = hstate; Yreturn(j) = y;

130 B.6 Model Predictive Control Simulations 117 end % model prediction error e = y C*mpcState; ereturn(j) = e; b = Phix*mpcState+Phie*e+Gammad*Dpred(:,j); c = b R(j:j+m 1); g = Gamma'*c Psi'*(eye(m,1)*lambda*upred); bl = DeltaUmin + eye(m,1)*upred; bu = DeltaUmax + eye(m,1)*upred; % The QP ineqcoef = [ Psi;Psi]; ineqright = [ bl;bu]; [U,fval,exitflag] = quadprog(h,g,ineqcoef,ineqright,[],[],umin,umax,... [],options); if(exitflag 1) warning(['in optimizing the QP, quadprog returned the exitflag '... num2str(exitflag) '. (see help quadprog)']); end % updating the insulin prediction upred = U(1); udiff(j) = upred; % the predicted glucose concentration ypred(j,:) = (Phix*mpcState+Phie*e+Gamma*U+Gammad*Dpred(:,j))'; % updating the mpc state mpcstate = A*mpcState + B*upred + K*e;% + Bd*D(j); mpcstatereturn(j+1,:) = mpcstate'; B.6.5 Simulations of a Typical Day B MPCMISOEDMMNNDemo.m clc clear all close all disp('beginning') % The ARX model identified from the Hovorka model. % Gives A, B and arxmod (see get(arxmod)) load('arxmodel9.mat') % get(arxmod) % number of predictions m = 48; lambda = 5; alpha = 0.999;

131 B.6 Model Predictive Control Simulations 118 [A,B,Bd,C,K] = statespacemodelmisoed(arxmod.a(2:end)',arxmod.b(:,2:end)',alpha); % model order n = length(b); [H,Phix,Phie,Psi,Gamma,Gammad] = designmpcmiso(m,a,b,bd,c,k,lambda); %% MPC initialization w = 53; % Weight of the virtual subject (kg) par = HovorkaParameters(w); uss = basalinsulin(w,5,7,par); % Steady state insulin infusion rate (mu/min); % usually chosen so that the resulting steady state glucose Gss % is in normoglycemia, or about 5 mmol/l. Basal insulin input. dss = 0; % Steady state value of the meal disturbance (g CHO/min). % Should be zero. [Gss,Iss,xss] = HovorkaBasalState(uss,dss,par); % basal state T = (0:5:24*60)'; % Time vector for the simulation, in minutes. To simulate % one day (i.e., 24 hours) of 5 minute data, then % T = [0:5:1440]'. D=zeros(length(T),1); % ================== % Meals (g/min): % Breakfast: D(42)=sum([ ]); % Lunch: D(78)=sum(ones(4,1)*1.83); % Afternoon snack D(120)=sum([ ]); % Dinner D(153)=sum(ones(4,1)*3.85); % Evening snack D(180)=sum([ ]); % D is the meals (g CHO/min). This will be zero except at meal times. This % command specifies meals of 45, 60, and 75 g CHO at simulation times of 1, % 5, and 11 hours. hstate0 = xss; umin = uss; umax = 600; DeltaUmin = 300; DeltaUmax = 300; Rdiff = zeros(size(t)); mpcstate0 = zeros(n,1); % converges to this at steady state % stating if future meals are known knownfuture = false; %% MPC [Udiff,mpcStateReturn,hStateReturn,Ydiff,eReturn,yPred] = computempcmiso(m,... A,B,Bd,C,K,lambda,Rdiff,umin,umax,DeltaUmin,DeltaUmax,T,hState0,...

132 B.6 Model Predictive Control Simulations 119 mpcstate0,d,knownfuture,uss,gss,par); U = Udiff+uss; Y = Ydiff+Gss; R = Rdiff+5; % ================== % Insulin (mu/min): Ubolus = ones(size(t))*uss; % basal insulin rate Ubolus(42) = 700; Ubolus(78) = 250; Ubolus(120) = 200; Ubolus(153) = 490; Ubolus(180) = 210; %G: plasma glucose concentration (mmol/l) [T,Gbolus,I,X]=HovorkaModelSimulation(T,xss,Ubolus,D,par); Ghypo = 4; Ghyper = 8; leg = {'Setpoint','Glucose concentration MPC', 'Glucose concentration bolus'}; plothandles = HovorkaFigures(T,D,U,[Y Gbolus],Ghypo,Ghyper,R,1,3,leg);

133 Bibliography [1] B. W. Bequette, A critical assessment of algorithms and challenges in the development of a closed-loop artificial pancreas, Diabetes technology and therapeutics, vol. 7, [2] D. Boiroux, D. A. Finan, J. B. Jørgensen, N. K. Poulsen, and H. Madsen. [3] D. Boiroux, D. A. Finan, N. K. Poulsen, J. B. Jørgensen, and H. Madsen, Nonlinear model predictive control for an artificial beta-cell. [4] D. Boiroux, D. A. Finan, N. K. Poulsen, H. Madsen, and J. B. Jørgensen, Computation of ideal insulin administration in people with type 1 diabetes using constrained nonlinear optimal control. [5] D. Boiroux, J. B. Jørgensen, D. A. Finan, N. K. Poulsen, and H. Madsen, Optimal insulin administration for people with type 1 diabetes. [6] C. Cobelli, C. D. Man, G. Sparacino, L. Magni, G. D. Nicolao, and B. P. Kovatchev, Diabetes: models, signals, and control, IEEE reviews in biomedical engineering, vol. 2, [7] M. Eren-Oruklu, A. Cinar, L. Quinn, and D. Smidth, Adaptive control strategy for regulation of blood glucose levels in patients with type 1 diabetes, Journal of process control, vol. 19, [8] R. Hovorka, V. Canonico, L. J. Chassin, U. Haueter, M. Massi-Benedetti, M. O. Federici, T. R. Pieber, H. C. Schaller, L. Schaupp, T. Vering, and M. E. Wilinska, Nonlinear model predictive control of glucose concentration in subjects with type 1 diabetes, Physiological Measurement, vol. 25, 2004.

134 BIBLIOGRAPHY 121 [9] J. K. Huusom, N. K. Poulsen, S. B. Jørgensen, and J. B. Jørgensen, Offsetfree model predictive control based on arx models, (Manuscript in preparation), [10] H. Madsen, Time series analysis. Chapman and Hall/CRC, [11] L. Magni, D. M. Raimondo, C. D. Man, G. D. Nicolao, B. Kovatchev, and C. Cobelli, Model predictive control of glucose concentration in type 1 diabetic patients: A silico trial, Biomedical signal processing and control, vol. 4, [12] L. Magni, D. M. Raimondo, L. Bossi, C. D. Man, G. D. Nicolao, B. Kovatchev, and C. Cobelli, Model predictive control of type 1 diabetes: An in silico trial, Journal of diabetes science and technology, vol. 1, [13] G. Marchetti, M. Barolo, L. Jovanovič, and H. Zisser, An improved pid switching control strategy for type 1 diabetes, Proceedings of the 28th IEEE EMBS annual international conference, [14] G. Marchetti, M. Barolo, L. Jovanovič, H. Zisser, and D. E. Seborg, A feedforward-feedback glucose control strategy for type 1 diabetes mellitus, Journal of process control, [15] J. Nocedal and S. J. Wright, Numerical optimization. Springer, [16] J. B. Rawlings, Tutorial overview of model predictive control, IEEE control systems magazine, 2000.