Adaptive anticipatory network traffic control using iterative optimization with model bias correction

KU Leuven Centre or Industrial Manaement, Traic and Inrastructure Research report Adaptive anticipatory networ traic control usin iterative optimization with model bias correction Wei Huan a, Francesco Viti b, Chris M.J. Tampère a a Leuven Mobility Research Center, CIB, KU Leuven, Belium b Faculty o Science, Technoloy and Communication, University o Luxembour, Luxembour This research wor has been submitted to Transportation Research Part C. I you want to cite this report, please use the ollowin reerence instead: Huan, W., Viti, F., Tampère, C.M.J., 2015. Adaptive anticipatory networ traic control usin iterative optimization with model bias correction. (submitted to Transportation Research Part C) KU Leuven Leuven Mobility Research Center, CIB Celestijnenlaan 300A PO Box 2422 3001 Heverlee (Belium) Email: wei.huan@uleuven.be http://set.uleuven.be/l-mob/ www.uleuven.be/traic

Adaptive anticipatory networ traic control usin iterative optimization with model bias correction Wei Huan a*, Francesco Viti b, Chris M.J. Tampère a a Centre or Industrial Manaement/Traic and Inrastructure, KU Leuven, Belium b Faculty o Science, Technoloy and Communication, University o Luxembour, Luxembour * Correspondin author. E-mail: wei.huan@uleuven.be (Wei Huan) Centre or Industrial Manaement, Traic and Inrastructure, KU Leuven, Celestijnenlaan 300A, PO Box 2422, 3001 Heverlee, Belium Tel.: +32 16 37 27 69 Abstract: Anticipatory sinal control in traic networs adapts the sinal timins with the aim o controllin the resultin (equilibrium) low patterns in the networ. This study investiates a method to support control decisions or successul applications in traic systems that operate repeatedly rom day to day. A main bottlenec in desinin the daily control scheme is the presence o model uncertainty. Conventional methods adopt a two-step procedure, iteratively updatin parameter estimation and control optimization. Inconsistency arises due to the inevitable structural modelity mismatch. This paper proposes an iterative optimizin control method to tacle this limitation and drive the traic networ towards the true optimal perormance despite o model uncertainty. This Iterative Optimizin Control with Model Bias Correction (IOCMBC) corrects model bias usin measurements and the resultin ity-tracin model is updated or the subsequent control optimization. Theoretical analysis on matchin between the IOCMBC optimal solution and the true optimum is presented. A local converence analysis is also elaborated to investiate conditions required or a converent scheme. One critical issue is the involvement o the sensitivity (Jacobian) inormation o the route choice behavior with respect to sinal control variables. To avoid perormin additional perturbations, we introduce a measurement-based implementation method or estimatin the operational Jacobian that is associated with the ity. Numerical tests in a small networ veriy the eectiveness o the proposed IOCMBC method in taclin model uncertainty, as well as a practical settin or reulatin the ity-tracin converence. Keywords: anticipatory traic control, adaptive sinal control, model uncertainty, iterative optimization, model bias correction 1 Introduction Traic control and manaement strateies oer hih opportunity to improve traic operations. Incorporatin travelers response is crucial to desin appropriate strateies. In particular, a low anticipatory control is proposed (Taale, 2008), whereby the controller anticipates route choice response, thus has the possibility to loo or sinal settins that correspond to optimal response rom the travelers in terms o e.. optimal travel time. In ame-theoretical terms, anticipatory control

reconizes control optimization as a leader s role in the Stacelber-type interaction between route choice (traic assinment) and control. As it suests a lobal optimization scheme with respect to the user-optimal assinment, anticipatory control is also nown as bi-level optimization or lobal optimization problem, in terms o sinal control optimization in literature (Yan and Yaar, 1995; Yan and Bell, 1998; Cascetta et al., 2006; Cantarella et al., 2012). For transportation plannin purposes, low anticipatory control is an eective way to uide a traic networ to more desirable equilibrium conditions by settin appropriately the traic sinals. From a perspective o operational level, the calculation o anticipatory sinal settins can be interpreted as determinin a responsive sinal control policy that is employed in daily repeated traic operations (Smith and Van Vuren, 1993; Hu and Mahmassani, 1997). This anticipatory control scheme usually locates at a strateic level in many adaptive control systems with a hierarchical structure such as UTOPIA (Mauro and Di Taranto, 1990) and RHODES (Head et al., 1992). Sinal timins operatin at the strateic level are desined to react to traveler route choice response in a loner temporal and spatial horizon, and could be subsequently passed to the local control level which has a iner time scale as a reerence sinal. The primary oal o this study is to desin an operational anticipatory control used or strateic level decision support in daily repeated traic operations. We concern ourselves with steady-state route choice response, i.e. equilibrium low patterns. The objective we consider here is optimizin total travel time or the system; however, any lobal objective (includin environmental oals) can in principle be considered. A main bottlenec with respect to operations is the presence o uncertainty in the model used to anticipate the route equilibrium response. In eneral, the route choice response can never be precisely described and it is usually approximated by equilibrium low models, while empirical evidence shows that only part o the users choices actually ollow optimal route choice principles (Vreeswij et al., 2014). In the presence o model uncertainty, due to e.. structural model mismatch, inaccurate parameters, measurement noise, and some other unexpected disturbances, implementin control settins in traic networs could result in suboptimal traic operations, lie or instance unpredicted traic conestion and spillbac. A natural way to deal with model uncertainty is to mae use o eedbac inormation derived rom measurement data. Usin eedbac mechanisms to compensate or model uncertainty has been extensively investiated or traic applications (Heyi et al., 2005; Dinopoulou et al., 2006; Aboudolas et al., 2009; Lin, 2011). The existin studies mainly concern on within-day traic evolution, whereas in this wor we intend to explore a new eature o the eedbac mechanism applyin alon a day-to-day dimension within an adaptive control scheme. Since traic operates repeatedly rom day to day, dierent measurements on daily traic states such as low or density are available. An iterative two-step method is usually adopted or an adaptive implementation scheme (Bellemans et al., 2002), repeatedly estimatin model parameters by usin measurements and usin the updated model to enerate new control settins via optimization. As suested by Wan et al. (2006), a -time motorway networ traic surveillance tool called RENAISSANCE that enables traic state estimation based on -time measurements can provide more accurate inormation and may help traic operators to mae more conident control decisions to improve traic operatin conditions. However, simple iteratin between successive solutions o an estimation and an optimization problem usually ail to convere to the true optimal condition (Roberts and Williams, 1981), especially reardin the structural model mismatch that is inevitable as models employed or optimization are only imperect abstractions o system behavior. In this

context, two issues are under concern: whether the iterations could convere, and i so, whether the converent point coincides with the optimum. To overcome the limitation o a traditional two-step approach, this study aims at desinin an adaptive anticipatory control that is capable o eneratin a sequence o control settins able to convere to the true optimal operation despite model uncertainty. This requirement on control desin motivates the utilization o a novel Iterative Learnin Control (ILC) technique, which has been widely adopted in robot manipulators with a similar concern or robotic control desin (Arimoto et al., 1984; Bristow et al., 2006). Indeed, or the repeated traic operations, there is a hih potential to learn the deviations o ity rom model expectations, hence to mae better control strateies. ILC wors on repeatedly operatin systems and it is capable o usin measurements to compensate or uncertainty alon the iteration domain. This property o iteration-domain eedbac enables ILC to wor well in devisin an adaptive traic sinal scheme. An iterative learnin approach was already proposed or the adaptive traic anticipatory control in a previous study (Huan et al., 2014), modiyin daily sinal settins such that the enerated control scheme eeps trac o the low patterns and yields converence on traic operations. The underlyin mechanism o the iterative learnin approach on tracin the low response is an implicit correction on a model bias, deined as the dierence between measured and predicted lows. The modiied model bias is used to enerate a new sinal settin in a way that the deviations are compensated or iteratively. However, the optimality o the converent point is not uaranteed in this implicit method. This new paper adds to our previous wor the improvement in solution optimality. It ollows the idea o iterative optimizin control on a basis o model bias. However, instead o implicitly updatin model bias in the constraint o the optimization problem, this paper extends the method by perormin an explicit irst-order correction on the model bias. Flow sensitivity (Jacobian) is calculated to obtain the derivatives o both model and lows with respect to sinal settins. The derivative inormation is then used to ormulate the irst-order approximation on model bias correction, proposin an iterative optimizin control method with explicit model bias correction. The advantae over an implicit method is that the true optimum can be achieved because o the Jacobian modiication; moreover the explicit correction could also be used or other purpose than optimal sinal control (e.. inormation provision). 1.1 Contributions o the paper In conclusion, the main contributions o the presented paper are summarized in the ollowin points: 1. In this paper, we ormulate an adaptive anticipatory traic control problem in the context o inaccurate networ equilibrium modelin. The main idea is to apply eedbac mechanism alon a day-to-day dimension and hence desin a control scheme used as a decision support or daily repeated traic operations. 2. We propose an iterative optimizin alorithm with two desirable properties: providin converence on the traic system, and providin consistence between the converent point and the true optimal operatin condition. 3. A irst-order model bias correction is perormed or the proposed alorithm, correctin both the value and curvature o the model bias. A ity-tracin model is thereby deined, which is then

utilized by the control optimization procedure. This enables the traic manaers to mae more conident control decisions, meanwhile, to improve the underlyin decision support iteratively. 4. We identiy and prove that a crucial role in improvin the solution optimality is played by Jacobian o the (equilibrium) lows with respect to sinal control variables. Furthermore, reardin traic operations in ity, we present an operational sensitivity analysis and introduce a measurementbased inite dierence method to estimate the operational Jacobian o the low response with respect to sinal settins. 1.2 Outline o the paper The remainder o the paper is oranized as ollows. The model uncertainty problem aced in desinin an adaptive anticipatory control is introduced and an iterative optimization problem is ormulated in section 2. We ocus on an unnown but deterministic uncertainty due to structural modelin error. Section 2 also introduces an iterative learnin approach or an adaptive traic control stratey. In section 3, the Iterative Optimizin Control with Model Bias Correction (IOCMBC) alorithm is proposed and elaborated. A Necessary Optimality Condition (NOC) is discussed ensurin that the optimal solution to the IOCMBC is consistent with the true optimum. Furthermore, the local converence properties are analyzed, ivin some insihts into suicient conditions or converence o IOCMBC. Section 4 discusses a crucial issue reardin requirement on the Jacobian inormation o the system. A measurement-based method is implemented or Jacobian estimation in this study in order to avoid an impractical way o perormin additional perturbations. The proposed IOCMBC is tested in section 5 in a toy traic networ, compared with a traditional two-step method o iterative parameter estimation and control optimization. Section 6 briely discusses assumptions o ull observability and absence o noise on lin low measurements, and relaxation o the assumptions or lare-scale applications. Section 7 shows the conclusions and discussions on uture study. 2 Iterative optimization or adaptive anticipatory control This section explains some undamental issues reardin the adaptive anticipatory control and a desin method usin iterative learnin approach. 2.1 Adaptive anticipatory control dealin with anticipatory equilibrium low Anticipatory control optimizes traic sinal timins tain into account that a modiication in traic control could chane low patterns as a result o route choice response (Taale and Van Zuylen, 2003; Taale, 2008). Several ormulations can be used or modelin a traic assinment procedure. In this paper, we describe a mutually consistent system, in that the low patterns determine travel costs includin sinal delays, while in turn travel costs decide low patterns throuh their impact on route choice behavior. A static networ loadin, more speciic (and without loss o enerality) a linearized BPR cost unction is used to represent traic supply. As or traic demand model, a stochastic route choice principle is adopted, which is ormulated as a discrete choice model derived rom random utility theory (Cascetta, 2009).

c C(, ) c A s 0 F( c) dbρ( B T c) (2.1) Here, c is the vector o lin costs, which is a unction o lin lows and sinal settins (we reer 0 to sinal reen splits here, thus omittin the cycle lenth). In the cost unction C (.), c is the reelow costs and s the vector o lin saturation lows, while A is the coeicient matrix in the cost unction; in this study, A is a diaonal matrix whose entries can be rearded as dierent values (in a BPR cost unction) or each lin. In the route choice model F (.), d denotes the traic demand and B the lin-route incidence matrix. Route choice probabilities are denoted by ρ dependin on lin costs. In this study, we ocus on the steady-state control and the solution to a ixed-point model is used to describe the networ equilibrium, assumin that this solution is unique. As shown in the ollowin equation, the equilibrium solution Eq depends on a iven reen split 0 (Cantarella et al., 2012). Eq F C Eq 0 ( (, )) In the context o daily traic operations, it is important to distinuish the traic system and the model. Let the equilibrium low o ity n n mea n or n lins be represented by a mappin :. The superscript means a perect mathematical description o ity, while mea indicates that this result o low response is usually measured rom traic networs. mea ( ) (2.2) Full observability and absence o noise are assumed or the lin low measurements in this study. We assume that all the lin lows are observable and noise-ree, as the objective is to develop and veriy a new adaptive traic control stratey. Applications in lare-scale networs in which measurement noise and missin data are detrimental to the quality o the control stratey will be discussed in section 6 o this paper. The steady-state anticipatory control optimization problem or ity is ormulated as ollows. mea min z(, ) (2.3a) mea st.. ( ) (2.3b) mea 0 (2.3c) L n U n (2.3d) Where: z : is the objective unction or control optimization, e.. total travel costs. In this paper, we assume that this unction is nown and can be evaluated rom measurements. Equation (2.3b) is the low prediction denotin that an equilibrium low pattern is calculated based on sinal settins. Formula (2.3c) is a non-neativity constraint on the mathematical modelin o the lows. (2.3d) restricts the boundary or the sinal settins.

In eneral, a perect mathematical description o ity is rarely available or control desin and an approximated model is usually adopted: Eq (, μ) (2.4) n n n With : representin the equilibrium low model, and μ n denotin a set o model parameters which is adjustable or modelin accuracy. In this paper we are concerned about demand side model uncertainty thus we assume that the lin cost unction is accurately nown. Sinal settins in the model and ity are the same. Two sources o model uncertainty are reconized or (2.3), one bein inaccurate values or μ and the other a structural model-ity mismatch (.) (.). Usin such model approximation, solution to the oriinal problem (2.3) can be approximated by solvin the ollowin model-based optimization problem. Eq min z(, ) (2.5 a) Eq s. t. (, μ) (2.5 b) Eq 0 (2.5 c) L U (2.5 d) First, we introduce a easible solution mappin with respect to the equilibrium low or this optimization problem (2.5): Eq L U : G( ) : { [, ]: (, μ) 0} (2.6) An Anticipatory Equilibrium Flow (AEF) is deined as the low pattern associated with the solution to the optimization problem (2.5). We call it a model-based AEF denoted as Eq *. The correspondin optimal sinal settin can be written as: G G * Eq Eq Eq * ( *) { ( ) : z(, ) z *} (2.7) Similarly, a true AEF is deined or the lobal optimization problem (2.3) denoted as mea * and the correspondin true optimal sinal settin is denoted as true *. Note that mea * is dierent rom the measured equilibrium low mea ; it relates to the true optimal perormance in traic networs. In this study, the concern or an adaptive anticipatory control is to iteratively adjust model-based AEF Eq * in order to trac true AEF mea * with the help o measurements. 2.2 An iterative learnin approach Implementin sinal settins calculated rom problem (2.5) in ity usually results in suboptimal operations resultin in even some unexpected traic conestions due to model-ity mismatch. Reardin the act that traic operates repeatedly e.. rom day to day and measurements on traic lows are available or handlin uncertainty, iterative optimizin control methods can be utilized or desinin adaptive anticipatory control strateies. In this context, and or the sae o illustration, one

iteration is rearded as a day, but one may as well consider periods o wees or months o operation beore the model is corrected and the sinal is updated. This interpretation ollows a control settin procedure that low observed today (more eneral: o the previous period) is used or updatin new sinal settins or tomorrow (more eneral: or the next period). A two-step implementation scheme is conventionally applied. The idea is to iteratively estimate selected model parameters and enerate new sinal settins via control optimization. A possible two-step scheme can be written usin the ollowin two equations representin the two problems o parameter estimation and control optimization: mea μ = ar min (, μ) (2.8 a) μ 1 ar min z(, (, μ )) (2.8 b) The constraints are nelected or simplicity. Formulation (2.8a) shows one instance o minimizin the sum o square errors between the measured and modeled lows or parameter estimation. However we do not exclude other estimation techniques. The parameter estimation procedure uses measurements obtained rom the current th iteration control settin to correct the parameters; and then the control optimization step determines a new sinal ater havin ixed these estimated parameters. The two problems interact and several iterations may be required until no urther improvement is obtained. Similar to comments on usin an iterative optimization assinment (IOA) approach to solve an interaction between traic assinment and traic control (Allsop, 1974; Gartner, 1975), it is well nown that simple iteratin between two steps is not uaranteed to convere even to a local optimum (Smith, 1979; Dicson, 1981). Indeed, as the estimation is perormed with a previously applied sinal settin, it miht be unrelated to the next control optimization. Thereore, minimizin the square error in low may not help in optimizin the total travel time. Fiure 1 shows a symbolic representation o a traditional two-step procedure. An optimal sinal * or the th iteration is solved based on the th estimated model parameters; then it is used or a (+1) th estimation which is then used or solvin * 1. A simple two-step approach will not convere to the true optimum on the system unless the model is structurally correct and the parameters are identiiable. However, in the presence o model mismatch, which is an inherent attribute in modelin, two questions immediately arise: 1) i the control settins enerated based on an inaccurate model can yield a converence on traic operations? and 2) i the converent solution matches the true optimal perormance?

Fiure 1 Symbolic representation o model-ity mismatch and a two-step approach: parameter correction and control update or the th and (+1) th day. ( *, *) is an optimal solution computed rom model, and mea is a measurement on lows when implementin * in ity In a previous study (Huan et al., 2014), an iterative learnin approach has been proposed or handlin the model mismatch. This iterative learnin method is constructed with an important property o ity-tracin, i.e. the enerated control settins eep trac o the true low patterns and yield converence on traic operations. It is motivated by an Iterative Learnin Control (ILC) technique, which has been widely applied or industrial applications on robotic manipulator as well as chemical process control. Many rule-based control laws have been elaborated or reulatin the controlled system to a reerence state (Par et al., 1999; Moore, 2001; Chen and Moore, 2002; Hou et al., 2008; Huan et al., 2013). This is ollowed by developments on optimization-based methods, in order to have a more systematic desin concept and enable more implications on applications (Lee et al., 2000; Gunnarsson and Norrlö, 2001; Owens and Hätönen, 2005). A similar iterative trialand-error implementation scheme has also been developed or traic tollin on road networs in the absence o demand unctions (Yan et al., 2004; Han et al., 2004). An adaptive anticipatory control is ormulated ollowin an optimization-based ILC desin paradim. The problem can be interpreted as the determination o the (+1) th control settin as an input to the traic system that can reduce the total travel cost in an optimal way, whilst addin penalties to account or the model uncertainty.

min. z(, )) W W (2.9 a) 1 T T 1 1 1 1 1 1 s. t. ( (, μ) - ) (2.9 b) mea Eq 1 1 1 0 1 1 L 1 U (2.9 c) (2.9 d) (2.9 e) The use o measurements or handlin uncertainty is mainly relected by (2.9b). It indicates that a prediction on low response 1, which is a modiication o an observation upon the completion o the th iteration, is used in the optimization procedure to calculate a new control settin 1. It should be enerated in a way that, under the basic model inormation, its deviation to a model output should be at least the same as observed model-ity mismatch rom the last iteration. Formula (2.9c) is the non-neativity constraint on the low. Equation (2.9d) shows the sinal update while (2.9e) indicates that the sinal reen time is bounded by minimal and maximal reen time., W n n and W Δ n n are weihts on total travel cost perormance, sinal inputs and sinal updates respectively. They are desin parameters which can be tuned to balance the adaptation o sinal and the converence o the alorithm. The objective or this iterative optimization problem (2.9) is to desin a sequence o anticipatory control settins that is capable o achievin a true optimal perormance associated with the traic operations. Two main aruments are here derived. 1) In a traditional two-step approach, the selected model parameters are updated by usin measurements. While the proposed ILC-based method eeps the parameters at some deault values (or a nominal model). This is justiiable or the situations under which model calibration or a better description on system is not a main concern and the ultimate and only oal is to desin control settins that can operate well in ity. 2) By introducin a concept o model bias b n deined as dierence between measured and model predicted low shown in (2.10), it can be observed that the main idea behind the ILC-based method is actually to manipulate the model bias or tracin ity. The model bias is updated at each iteration implicitly as a constraint to the optimization problem: b mea Eq ( ) (, μ) (2.10) As discussed, in the two-step approach, parameter estimation is addressed with a current control settin while control optimization is perormed with a determined parameter value. The reason o its ailure actually lies in that the objective o parameter estimation miht be unrelated to the objective o perormance optimization. Model bias plays a siniicant role in connectin the objectives o estimation and optimization. Some approaches to interate these objectives have been explored in literature applied in the -time optimization problems (Cutler and Perry, 1983; Chachuat et al., 2009). Adaptations are usually perormed on the objective unction and/or the constraint unctions (Roberts and Williams, 1981; Roberts, 1995; Srinivasan and Bonvin, 2002; Marchetti et al., 2007). Our proposed method ollows the interation idea; while dierently rom the previous approaches, we address the adaptation on the system model bias.

In order to present the role o model bias, let us reormulate (2.9a) as (2.11): (, μ) b (2.11) 1 1 Equation (2.11) indicates that model bias update introduces a one-step prediction on the deviations o model rom ity. Hence, the impact o the subsequent sinal settins is counted in modiyin the deviations. In this sense, model bias is in essence a connection between tracin ity and optimizin control. It is worth notin that in this approach, adaptation on model bias instead o model parameters is perormed or tracin ity, as well as or supportin the subsequent control decision-main. Deault values are applied or the model parameters. The ILC-based method is a direct control method and compensates or the model uncertainty via implicit model bias update toether with penalties in the objective unction. It suests an eective way o determinin optimal control capable o tracin operatin conditions and yields a satisactory practical application. However, the implicit model bias update therein is actually a partial correction in a sense that only the value o the modeled low is adjusted based on the measurements, leavin the sensitivity o the model unchaned. Thereore, penalties on the control settins are added or reulatin a ity-tracin converence. As a result, there is a trade-o on the optimal total travel cost reardin a converent scheme in the presence o model uncertainty; the converent point miht be suboptimal and does not match the true optimum. Further eorts are needed i an improvement in optimality is required. The present paper wors on matchin with the true optimum. A Jacobian modiication term is added to the model bias prediction and hence a ull correction on model bias is obtained. The model bias correction is explicitly perormed and used or a subsequent total travel cost optimization which is separated rom the penalized optimization problem. The approach proposed in this study is to construct and describe in detail an alorithm with a desired property o, in addition to ity-tracin, matchin o the converent solution with the true optimum. The ollowin section 3 elaborates on desin o an adaptive anticipatory control stratey usin iterative control optimization considerin explicit model bias correction. 3 Iterative control optimization with explicit model bias correction 3.1 Model bias correction As discussed in section 2, in the presence o model mismatch, traic operations miht deteriorate due to the inconsistency between the model-based optimum and the true optimum. A ull and explicit model bias correction is proposed in this section to compensate or the uncertainty. The basic idea is to use measurements to adjust both the value o the model output and the radient o the model, which captures the sensitivity o the output to the input chanes. More speciically, a irstorder prediction is perormed on the model bias, updatin both the value and the radient. Followin this way, we intend to reduce the numerical error enerated rom approximation by explicitly addin sensitivity inormation to the model bias correction. As the model bias is determined rom a complete set o lin low observations, the model bias correction is constructed based on the assumption o ull observability on lin lows. A brie discussion on how to relax this assumption is arraned in section 6.

Standin at a current th iteration, a low pattern 1 denotes a prediction on low patterns when the decision 1 or the next (+1) th iteration is implemented; thus 1 depends on 1. An n expected model bias b 1 is deined as its dierence rom a model prediction. The value o model bias at th iteration is observed as the distance between measured and predicted low patterns. Curvature o the bias is captured by the Jacobian around the current operatin point. Thereore a model bias correction is perormed as ollows: b b 1 b ( ) ( 1 ) (3.1) Note that b is an observation rom ity as shown in (2.8), whereas b 1 is a prediction on model bias or the next iteration: b (, μ) (3.2) 1 1 1 Substitutin equation (2.10) with (3.1) and (3.2): (, μ) ( ) (, μ) δ ( ) (3.3) 1 1 1 n n in which, δ denotes the Jacobian error between ity and modelδ. Thereore, the expectation low or the next control optimization, which is adjusted on the basis o model bias correction, is written as: (, μ) b (, μ) b δ ( ) (3.4) 1 1 1 1 1 n This brins a ity-tracin model :, which is associated with operations. Metamodels are utilized by the optimization procedure to calculate a new optimal sinal settin (Osorio and Bierlaire, 2013). We will reer to model (3.5) as model or short in the rest o the paper: (, μ) (, μ) b δ ( ) (3.5) Fiure 2 shows a symbolic representation o the correction on model bias alon one sinal input direction. Observation b is the value o the model bias denotin distance between the measured and the modeled lows, whereas δ corresponds to the radient o the model bias denotin slope error between the sensitivity o the and the modeled low to the sinal chanes. Deinition (3.5) tells that a (+1) th prediction on low i.e. the model, is derived by adjustin an underlyin system model with a model bias correction on both distance and slope. As shown in iure 2, upon the completion o the th iteration, the system model is oset with an observed distance b, meanwhile rotated with a slope error δ estimated in the th iteration. Note that the nominal system model itsel does not chane alon iterations. A critical manipulation durin the model bias correction and model adaption is calculatin radients o the low response. This ey issue o radient (or Jacobian i control has multiple dimensions) calculation will be elaborated in the next section. n n

Fiure 2 Model bias correction to match model and ity or control decision support 3.2 The IOCMBC method Tain into account an explicit and ull model bias correction, an alorithm called Iterative Optimizin Control with Model Bias Correction (IOCMBC) is proposed or an adaptive anticipatory control scheme. By explicitly perormin a ull model bias correction, the adaptive scheme actually ollows a combined two-stae ramewor o model bias correction and control settin update. The model (3.5), which is modiied based on the model bias correction serves as a basis or control optimization. Hence, one advantae o this two-stae procedure is that it enables the traic manaers to improve the underlyin control decision support system, throuh improvin the ity-tracin model. Fiure 3 shows the updatin procedure o the proposed control scheme.

Reality-tracin model (, μ) Control optimization Reality μ (, ) ( ) Fiure 3 Iterative Optimizin Control with Model Bias Correction procedure An optimal sinal settin * 1 or the (+1) th iteration is determined by solvin the ollowin model-based optimization problem: * ar min z(, (, μ)) (3.6 a) 1 s. t. (, μ) (, μ) b δ ( ) (3.6 b) (, μ) 0 (3.6 c) L U (3.6 d) A prediction on the low response that is used in the objective unction results rom the model. Equation (3.6b) describes that the model is a modiication on model prediction with a ull irstorder model bias correction. Non-neativity and boundary constraints are included in (3.6c) and (3.6d) respectively. Considerin that even * 1 as a solution to (3.6) will not be optimal at once due to omittin hiherorder terms in the bias correction, it may be cautious to consider the newly obtained * 1 as an optimization direction or the next iterate, rather than the next control value to implement or the next iteration. We then need to determine the step size into this optimization direction to ind new implementation sinal settin 1. Naturally, an exponential smoothin structure is adopted: ( I K) K * (3.7) 1 1 Model bias correction b ( ) (, μ) ( ) (, μ) δ Here K n n is a ain matrix representin a suitable step rom to * 1 value o K ( 1,..., ) n and usually taes a dia, in which 1,..., [0,1], hence allowin in principle dierent step sizes or the dierent dimensions o. In many cases, K is rearded as a desin parameter and n serves as a practical settin or reulatin converence. Note that when one chooses K equal to I, no smoothin, hence ull step size is considered.

The new control 1 is then implemented in ity. Model bias is updated, includin the observed value at the current iteration and the Jacobian that is also estimated around the current operatin point: b ( ) (, μ) (3.8 ) 1 1 1 a δ 1 (3.8 ) 1 b 1 The IOCMBC alorithm is now summarized as ollows. Step 1: Set a deault value or model parameter μ ; choose desin parameters K ; set initial sinal settin 0 ; Step 2: Apply the initial sinal settin; implement 0 in ity, calculate the initial Eq 0 and measure mea 0. Then solve * 1 rom the control optimization problem, set a step rom 0 to * 1 and derive 1, set =1; Step 3: Apply sinal settins; implement in ity, calculate mode prediction equilibrium low Eq and measure the resultin low mea ; Step 4: Model bias correction; update model bias b ( ) (, μ) mea Eq, estimate Jacobian or both model and ity at the current operatin point and derive the Jacobian error δ ; Step 5: Update ity-tracin model; update the model on the basis o model bias correction and derive a prediction low or updatin the next sinal control; (, μ) (, μ) b δ ( ) 1 1 1 1 Step 6: Find optimization direction; calculate * 1 by solvin the control optimization or the updated model, and derive an optimization direction ( 1 * ) ; Step 7: Set appropriate step size and update sinal control: 1 ( I K) K 1* ; Step 8: Chec termination; stop i the termination condition is satisied, otherwise set =+1 and o to step 3. 3.3 Matchin o necessary optimality conditions The oal o this study is to desin an adaptive control scheme in avor o operations, whereby the optimal solution rom the iterative optimization problem should match the true optimal operatin point. IOCMBC can brin up this beneit. Upon converence, the resultin optimal solution is also the solution to the true optimization problem. This property is a prerequisite or iteratively solvin the optimization problem (2.5) to approximate the true solution to the problem (2.3).

To show this capability o IOCMBC, irst the Necessary Optimality Conditions (NOC) or a eneral optimization problem is introduced. Recall the true optimization problem (2.3) in section 2, suppose that true * is a local minimum o (2.3) and that the required constraint qualiication holds at true * (Boyd and Vandenberhe, 2004), and the unction z (.) and (.) are dierentiable at true *. Then we introduce the Larane multiplier vectors l, l, l such that the ollowin conditions are satisied at ( true *, l, l, l U ): true L( *, l, l, l ) 0 (3.9a) true ( *) 0 (3.9b) * L true U L U L n L n U (3.9c) l 0, l 0, l 0 (3.9d) L U T true T true L T U true l ( *) 0, l ( * ) 0, l ( *) 0 (3.9e) L U true Where L( *, l, l, l ) is the Laranian unction and tain the derivatives as: L U z z true L( *, l, l, l ) l * * * l l L U true true true L U The irst-order NOC conditions are oten nown as Karush-Kuhn-Tucer (KKT) conditions (Boyd and Vandenberhe, 2004). We reer to the NOC point or problem (2.3) as a true NOC point. Similarly, a model-based NOC point is also deined at * based on the KKT conditions or the model- true based optimization problem (2.5), replacin ( *) with ( *, μ ). Based on the true NOC and model-based NOC, toether with the deinitions o true AEF and modelbased AEF mentioned introduced in section 2, we present an important eature o IOCMBC. That is, by implementin the proposed IOCMBC alorithm, the resultin AEF state or the model-based optimization problem (3.6) matches the true AEF state, and the resultin NOC point is also a true NOC point. This eature is ormalized in the ollowin Proposition 1. Proposition 1 (AEF and NOC matchin). Suppose that the ain matrix K is nonsinular and the IOCMBC alorithm described by (3.6), (3.7) and (3.8) converes to a model-based AEF state *; the converent point * lim * is a NOC point or the problem (3.6). Then converent implementation point lim coincides with the true NOC point or the traic control optimization problem (2.3), and * coincides with the optimal traic state mea *. n Proo. Let, rom (3.7) we have: * (3.10) Ater some manipulations on (3.6b), it can be shown that

( *, μ) ( *, μ) ( *) ( *, μ) (3.11) That is ( *, μ) ( *) (3.12) Calculatin the derivatives o (, μ) with respect to around * * * ( * *) (3.13) We obtain (3.14) * * By assumption * is a NOC point or the problem (3.6), it is obvious that it satisies the ollowin conditions with the speciied Larane multipliers l, l and l U L ( *, l, l, l ) 0 (3.15a) ( *, μ) 0 (3.15b) L * U L U L (3.15c) l 0, l 0, l 0 (3.15d) L U T T L T U l ( *, μ) 0, l ( * ) 0, l ( *) 0 (3.15e) L U z z In which L ( *, l, l, l ) * * * l l l From (3.10), (3.12) and (3.14), associated with the oriinal problem (2.3). Besides, the true AEF state mea * ( true *) (3.16) L U L U satisies (3.15) and is consistent with a true NOC point true * * mea is represented as (3.10), (3.12) and (3.16) veriies that model-based AEF * mea *. coincides with the true AEF A crucial issue or this matchin property is the availability and accuracy o the Jacobian inormation (especially or the low response). The calculation o Jacobian matrices or the system will be addressed in the next section.

Proposition 1 tells that under some conditions, a converent solution o the IOCMBC alorithm is also a true NOC, i.e. the proposed method inds a solution to the optimization o traic operations. However, converence o IOCMBC is not ensured. In the ollowin subsection, some speciications are provided reardin the choice o the desin parameters (in particular the ain matrix K ) that could reulate the converence. A local converence analysis is addressed subsequently. 3.4 Converence analysis This subsection discusses converence conditions or the proposed IOCMBC alorithm, in the absence o measurement noise. Under some speciications, applyin IOCMBC can ensure leadin the traic operations towards the optimal state. To acilitate the converence analysis, irst a constant term or the model bias correction is ε introduced with a mappin : ε ε( ) b δ (3.17) Thus the optimization problem (3.6) is rewritten as ollows: * ar min z(, (, μ)) (3.18 a) 1 s. t. (, μ) (, μ) ε δ (3.18 b) (, μ) 0 (3.18 c) L U n n showin the dependence on a current sinal input. (3.18 d) n n Ω We deine a solution mappin : current operation point such that or solutions to this optimization problem reardin a L U : Ω( ε( )) : { [, ]: (, μ) 0} (3.19) Then the optimal solution mappin is ormulated with the correspondin optimal perormance z *. Ω ε Ω ε z μ z * 1 * ( ( )) { ( ( )) : (, (, )) *} (3.20) Reardin the converent point o (3.6) * lim *, we have Ω ε * * ( ( )) In which lim is the converent solution to IOCMBC, and * as indicated by (3.7). The ollowin properties are irst assumed or the studied traic system. Assumption 1: Lin cost unction C(.) is lobally Lipschitz continuous in both and ; lin low unction F(.) is lobally Lipschitz continuous in c.

Assumption 2: The suicient optimality conditions or a local optimum o problem (3.18) at each iteration are satisied at * * or* Ω ( ε( 1)) with. Assumption 3: The objective unction z(.) is exactly represented and is dierentiable at both and. Assumption 4: All lin lows can be measured in the traic networ under consideration and the measurements are noise-ree. Then we ormalize the Proposition 2 as ollows: Proposition 2 (local converence o IOCMBC). Under the assumptions 1-4, the IOCMBC alorithm described by (3.18), (3.7) and (3.8) converes i the ain matrix K is chosen such that * Ω ε ( I K) K 1 (3.21) ε In which. represents any suitable norm. Proo. It ollows rom the assumption 1 and 2 that there is an optimal point * 1 to the problem (3.18). Accordin to (3.7) we ormulate a ixed-point mappin or the consecutive implementation n sinal settins Ψ : n : ( I K) K * Ψ ( ) (3.22) 1 1 Consider a linear approximation on Ψ around the converent optimal point Ψ Ψ( ) Ψ( ) ( ) (3.23) Now we deine a sinal error with respect to 1 1 (3.24) It is natural to derive that: Ψ 1 Ψ( ) Ψ( ) (3.25) Followin a conventional way o analyzin converence or a linear dynamical system (e.. Lo and Bie, 2006; Bie, 2008), we obtain the converence conditions required or the operator norm: Ψ 1 (3.26) In which * Ψ Ω ε ( I K) K ε

Alon with the converence o the sinal error 0, the implementation sinal settin converes to the optimal point 1. Proposition 2 establishes conditions required or a local converence o IOCMBC around the solution point. These conditions suest that properly selectin the step sizek can improve converence and eectively reulate traic system towards its true optimal operations. 4 Jacobian estimation As mentioned beore, a ey issue or applyin the proposed IOCMBC is the requirement on Jacobian inormation. Correction on model bias and the resultin modiication on low model include Jacobian inormation associated with both model and ity. Challene arises especially reardin the calculation o the Jacobian or the low response. This section ocuses on this issue. A most straihtorward approach or calculatin derivatives as discussed in literature is a inite dierence approximation method (Mansour and Ellis, 2003; Gao and Enell, 2005). Basically, it consists in perturbin each input variable individually and calculatin the correspondin derivative element. Implementin a inite dierence method or the low anticipation model, a component o the model Jacobian, which is a derivative o lin low with respect to sinal settin, is derived: i i 1 j n (...... i ) ( ) j (4.1) i is the low or lin i and j the sinal settin or lin j. i j denotes the ij th Jacobian component. This ij th component shows the derivative o the i th low reardin a small perturbation perormed on the j th sinal settin. Thereore, the Jacobian o equilibrium low response with respect to sinal control is calculated in (4.2): 1 1 n 1 1 1 n 1 (... ) ( ) (... ) ( )............ (4.2) n 1 n n n 1 n n (... ) ( ) (... ) ( )... However, it is obvious that this approach is impractical or implementation in ity as it requires perormin additional perturbations on each sinal settin and has to measure the low response to the correspondin perturbation. It is necessary to wait or each equilibrium low response in ity or obtainin all the measurements. Several methods have been proposed to avoid the additional perturbations. A so-called Broydon s method is proposed to estimate the Jacobian or ity (Fletcher, 1980; Roberts, 2000), in which a recursive ormula is deined and the output derivatives

are updated usin current and past measurements. A dierent way o implementin a inite dierence approximation is developed (Brdys and Tajewsi 1994), usin measurements observed in previous iterations instead o additional perturbations. In this wor, we employ a measurement-based inite dierence method or estimatin Jacobian or the low response, called operational Jacobian. The daily operational Jacobian is updated based on a measured low set containin (n +1) lows { mea, mea 1,..., settins. Operational Jacobian at the current th iteration ormula (4.3): mea n }, as well as the past (n +1) sinal is calculated by the ollowin mea mea 1 (, ) ( 1, ) μ μ 1 1 ( Δ).... ( Δ).... (4.3) mea mea (, μ) (, μ) n n T in which Δ [ 1,..., ] n. Thereore, a measurement-based estimation o the operational Jacobian can be implemented as ollows: Step 1: Set a set o initial sinal settins { 0, 1,..., observed low set { mea 0, mea 1,..., mea n }; calculate IOCMBC alorithm to compute the initial Jacobian error δ 1 ; set =1; Step 2: Apply ; record sinal settins {, 1,..., calculate ; n n 0 0 }; record the correspondin rom (4.3) and input it to the 0 0 mea } and lows { mea, 1 ; then solve,..., mea }; n Step 3: Input to IOCMBC alorithm and update 1 ; Step 4: Stop i the termination condition o IOCMBC is satisied; otherwise set =+1 and o to step 2. This measurement-based approximation method can estimate the operational Jacobian with suicient accuracy in the circumstance o measurement noise-ree and small number o variables. The impact o its accuracy on the IOCMBC alorithm in a lare-scale problem with multiple decision variables is beyond the scope o this paper; while it is an interestin topic or a uture study.

5 Numerical example 5.1 Simulation setup The eectiveness o the proposed IOCMBC method is illustrated in a test networ nown as the Braess networ. As shown in iure 4, this networ consists o one OD pair (rom node 1 to node 4), 5 lins and 3 routes. Lin travel cost is calculated usin a linearized BPR unction. Loit route choice models with the dispersion parameter are utilized or demand modelin. Node 3 represents a sinalized intersection operatin in a two-phase plan. Green split is the decision variable and sinal 2 3 loss time is not considered in this case thus 1. As one sinalized intersection is concerned, a sinle variable control optimization problem is illustrated in this case study. Fiure 4 The Braess test networ Mismatch between model and ity is simulated by usin dierent route choice models. Assume that in ity travelers ollow a nested loit structure (Cascetta 2009) or main route choice decisions. Reardin lin 4 as a hihway, it is true that travelers would have more inormation at node 2 than at node 1 thus separatin two choice levels. The probability o choosin route j can be expressed as: exp( c j ) exp( Y ) [ j]. (5.1) exp( c) exp( Y ) ii i h h Lin travel cost is denoted by c. Route choice set is divided into subsets I 1,, I,, called nests. 0 is the ratio o dispersion parameters 0 and associated with the irst and second choice ji level respectively. Y ln exp( c ) is the correspondin losum variable. j In eneral, this route choice response cannot be precisely described in the demand modelin. We assume that only one choice level is reconized and a multinomial loit model is used with the dispersion parameter. Thereore the probability is calculated by the model as: 1 [ j] (5.2) 1 exp[ ( c c)] i j j i As we ocus on demand side uncertainty, supply side modelin is assumed to be precise thus the lin travel cost is accurately modeled by a linearized BPR unction. Introducin a ree-low travel time 0 c,

a saturation low s and a coeicient, the lin cost is described by a unction o lin low and sinal settin : c C c 0 (, ) s Speciically, or non-sinalized lin, reen split equals 1. The equilibrium low pattern resultin rom the demand-supply interaction depends on the sinal control decisions implemented in ity: mea ( ) Total travel time z or this networ is also precisely ormulated as a unction o sinal and equilibrium low: z z(, ) C(, ). mea mea mea l A mismatched low model is simulated ollowin a multinomial loit structure. The dispersion parameter is concerned as an adjustable parameter or the equilibrium low model: Eq (, ) Control decisions are to be made based on the approximated model, and the objective is to minimize the total travel time on this networ. All optimization problems in this numerical example are solved usin MATLAB optimization toolbox. Moreover, OD demand is ixed in this illustrative case and all the lins have a same saturation low. Characteristics o the networ are listed in table 1. Table 1 Networ characteristics OD demand (veh/h) 2000 Parameter in cost unction 0.15 Saturation low (veh/h) 2000 Free-low travel time (h) lin 1 0.1 lin 2 0.2 lin 3 0.05 lin 4 0.4 lin 5 0.15 Dispersion parameters in ity 0 0.1 1 In the equilibrium low model, dispersion parameter taes a nominal value o 10. 5.2 Results o the numerical experiments The proposed IOCMBC method is implemented or 20 days in this test networ, comparin with a traditional two-step control scheme o iteratively calibratin parameter and optimizin sinal settins.

First, by solvin the anticipatory control optimization or ity, we have the results o true optimal sinal settins, lin lows and total travel cost presented in table 2. Table2 Optimal solution or ity: sinal settins, lin lows, total travel time Sinal reen split ( 2, 3 )=(0.66, 0.34) Lin lows (veh/h) Lin 1 1033 Lin 2 967 Lin 3 520 Lin 4 513 Lin 5 1487 Total travel cost (veh.h) 1182.4 As the equilibrium low model is an imperect abstraction on the travelers response, implementin control decisions derived rom the nominal model-based optimization in traic system cannot result in the true optimum. The mismatched results o lin lows and total travel cost are shown in table 3. Table 3 Optimal sinal settins derived rom model, and implement the model-based sinal settins in ity: compare lin lows and total travel time Model Reality Sinal reen split ( 2, 3 )=(0.90, 0.10) Lin lows (veh/h) Lin 1 882 1026 Lin 2 1118 974 Lin 3 285 459 Lin 4 597 567 Lin 5 1403 1433 Total travel cost (veh.h) 1173.0 1256.3 I a two-step approach is adopted and implemented to deal with the issue o model inaccuracy, it iteratively solves a parameter estimation problem, minimizin the error between measured lows and model prediction lows, and a control optimization problem, minimizin the total travel cost. As shown in iure 5, interaction between two steps enerates lip-loppin. The optimal total travel cost cannot be reached. Fiure 5(a) shows the lows on lin 1 durin the 20 days or both model prediction and ity, whereas iure 5(b) depicts the total travel cost or the networ. Simply iteratin two steps o parameter estimation and control update does not lead to an optimal networ perormance.

Flow on lin 1 (veh/h) Flow on lin 1 (veh/h) Flow on lin 1 (veh/h) Total travel cost (veh.h) 1190 1190 1190 1200 Model prediction Reality 1260 Model prediction Reality 1150 1250 1100 1050 1240 1230 1220 1000 1210 950 900 850 800 0 5 10 15 20 Iteration Fiure 5(a) 1200 1190 1180 1170 1160 0 5 10 15 20 Iteration Fiure 5(b) Fiure 5 Simulation results when implementin two-step approach in 20 days: 5(a) lows on lin 1; 5(b) total travel cost This inconsistency results rom the presence o structural model mismatch, which maes an accurate value o the model parameter irrelevant or an accurate system representation. In order to tacle the problem o model mismatch and to interate the objectives o model correction and perormance optimization, our proposed IOCMBC method introduce a ity-tracin model or determinin optimal control decisions. Instead o iteratively correctin model parameter, model bias is corrected to trac the networ state. IOCMBC is implemented or the test networ with lare step size (K=1). Fiure 6 shows trajectory o the two-step iterates and the IOCMBC iterates startin rom the nominal optimal point. Fiure 6(a) depicts the total travel cost contours o the true anticipatory control optimization problem. The true optimum is constrained by tain into account route choice behavior. Fiure 6(b) compares the iteration trajectories o a conventional two-step approach and the proposed IOCMBC approach. Startin rom the nominal point o (reen, low1)=(0.90, 1026), IOCMBC converes to the true optimum in 4 iterations, whereas the lip-loppin between two states occurs under the two-step approach. 2000 1350 1350 total travel cost true optimum 14001400 1450 1036 Two-step approach IOCMBC approach true optimum 1800 1600 1400 1350 1300 1300 1250 1300 1250 1350 1300 1500 1450 1400 1034 1032 1185 1200 1400 1450 1500 1200 1250 1030 1185 1185 1185 1190 1190 1190 1200 1200 1200 1195 1195 1195 1200 1200 1200 1195 1195 1195 1000 800 600 400 200 1350 1400 1450 1500 1250 1300 1350 1400 1450 1500 1200 1250 1300 1350 1200 1200 1250 1300 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Sinal reen split 1028 1026 1024 1022 1185 1185 1020 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 Sinal reen split Fiure 6(a) Fiure 6(b)

Flow on lin 1 (veh/h) Total travel cost (veh.h) Fiure 6 Total travel cost contours: 6(a) the true optimum; 6(b) trajectories or two-step approach and IOCMBC approach As discussed, a ey manipulation in IOCMBC is calculation o the operational Jacobian or the low response. A perturbation-based inite dierence method and a measurement-based inite dierence method are adopted and compared in iure 7, with iure 7(a) showin the lows on lin 1 and iure 7(b) the total travel costs durin the testin days. Both methods convere in this small networ. The perturbation-based method, which requires addition input perturbations, and the correspondin equilibrium states can enerate more accurate estimations. This is used as a benchmar in the comparison to show the impact o Jacobian estimation on converence. It is quite natural that the accurate method shows a aster converence than the measurement-based method. Onoin wor elaborates on the inluence o accuracy o the operational Jacobian estimations. 1034 1033.5 1033 1032.5 perturbation-based measurement-based 1190 1189 1188 1187 1186 perturbation-based measurement-based 1032 1031.5 1031 1030.5 1030 0 2 4 6 8 10 12 14 16 18 20 Iteration Fiure 7(a) 1185 1184 1183 1182 1181 1180 0 2 4 6 8 10 12 14 16 18 20 Iteration Fiure 7(b) Fiure 7 Comparison o perturbation-based method and measurement-based method in IOCMBC or calculatin operational Jacobian: 7(a) lows on lin 1; 7(b) total travel cost The impact o step size Now we examine the impact o the step size K perormed alon the optimization direction to update a new implementation sinal settin. Due to the similarity between the nested loit and multinomial loit structure, a converent control scheme is derived with a ull step (i.e. K=1) in calculatin the sinal control at each iteration. In order to show the role o the step size in reulatin converence, we use another simulation setup and mae a urther linearization on the equilibrium low model around each current operation point. In this way, a worse case o model approximation to the ity is considered in which the model varies alon the iterations. The ity is still simulated by a nested loit model. A Euclidean distance between the model-based results and the true optimum is used as an indicator or converence. Denote the Euclidean distance o low and total travel cost as z dis respectively, the distances durin iterations are recorded as: dis and * * 2 dis mea mea ( mea mea ) (5.3) lin 1/2

Euclidean distance o low Euclidean distance o total travel cost z dis z z * (5.4) Fiure 8 compares the converence procedures o IOCMBC with ive values or the step size, in terms o the Euclidean distance o low shown in iure 8(a) and the Euclidean distance o total travel cost shown in iure 8(b). As shown, perormin larer step (with K 0.5 ) could lead to non-converent procedures. It is obvious that selectin a proper step size plays an important role in desinin a successully operatin control scheme and thus in reulatin the ity-tracin converence which is our ultimate oal. 35 30 25 K=1/ K=0.1 K=0.3 K=0.5 K=0.7 20 18 16 14 K=1/ K=0.1 K=0.3 K=0.5 K=0.7 20 15 10 5 0 0 2 4 6 8 10 12 14 16 18 20 Iteration Fiure 8(a) 12 10 8 6 4 2 0 0 2 4 6 8 10 12 14 16 18 20 Iteration Fiure 8(b) Fiure 8 Converence o the IOCMBC method with ive dierent step size values: 8(a) Euclidean distance o lows; 8(b) Euclidean distance o total travel cost 6 Discussion Throuhout the discussion on the IOCMBC method, we assume that all lin lows are measurable and the measurements are noise-ree. However, these assumptions are too riid reardin applications in lare-scale road networs. Due to a limited set o traic sensors or missin data in lare traic networs, there is a mismatch between the dimensions o the measured low mea and the modeled low Eq as shown in equation (2.10), when calculatin the model bias. As a result, the model bias that is used or the subsequent control optimization cannot be determined rom the observed traic lows. A recently developed methodoloy or selectin optimal sensor locations has been shown to be eective in taclin the partial observability issue, i.e. when not all low variables are observed or observable (Viti et al., 2014). The impact o sensor positionin and o partial observability o lin low variables is beyond the scope o this study and is let or uture research. In addition, the presence o measurement noise is detrimental to the accuracy o the operational Jacobian estimates, thus aectin the quality o the IOCMBC solution. The diiculty o estimatin Jacobian rom past operatin points increases with the number o decision variables, as a lare

number o measurements are required or implementin the measurement-based inite dierence, which typically ampliies the eect o measurement noise. Althouh the problem o estimatin operational Jacobian particularly reardin lare-scale applications is crucial to the IOCMBC method, it is beyond the scope o this study. Estimation methods to reduce the impact o measurement error on the Jacobian approximation are to be investiated in a uture study. 7 Conclusions This paper proposes an iterative optimizin control method or desinin an adaptive anticipatory control scheme operatin in daily traic, which taes into account route choice behavior and ocus on steady-state (equilibrium low) in traic sinal control desin. It is nown that inaccurate model usually ails to enerate the true optimal solutions rom the model based control optimization problem as the approximated model could not have perect representation on system. Furthermore, the presence o structural model mismatch limits the perormance o a conventional two-step approach. The proposed method o Iterative Optimizin Control with Model Bias Correction (IOCMBC) allows traic manaers to tacle this limitation and to drive the traic system towards the true optimal perormance despite inevitable model uncertainty. First, IOCMBC method is ormulated based on the idea o handlin model bias instead o model parameter. A ull irst-order correction is perormed on model bias at each iteration thus eneratin a ity-tracin model or iteratively calculatin optimal sinal settins. We present a theoretical justiication on usin the IOCMBC method to approximate the true optimal solution. This is achieved by provin the matchin o Necessary Optimality Conditions (NOC) and the resultin anticipatory equilibrium low state or IOCMBC and ity. Moreover, we analyze conditions required or the converence o IOCMBC. Simulation test in a small networ veriies the eectiveness o IOCMBC, as well as the inluence o the step size on converence, which is perormed on sinal update. A price to pay or ensurin the true optimality is the need to estimate Jacobian at each iteration. A critical issue o estimatin the operational Jacobian or the traic low response is reconized. In order to avoid additional perturbations or derivin the sensitivity inormation, which is quite impractical, a measurement-based inite dierence method is introduced. Inluence o accuracy o the operational Jacobian is also examined in our small test networ. An onoin study elaborates estimation methods or handlin lare-scale problems in the presence o measurement noise. As discussed, the proposed IOCMBC method actually corrects the ity-tracin model used or determinin the optimal control settins. In essence it is a direct control update method, as it improves the networ perormance iteratively by adjustin the control settins usin measurements. The model used to describe ity is not adjusted. Indeed it is not necessary as our oal is to desin ity-tracin sinal settins and an accurate system description is not the research ocus at this point. However, reardin a need or parallel operations in consideration o operational control system desin, whereby model calibration is required and the dynamic low estimation is simultaneously implemented durin the process o main optimal control decisions (Han et al., 2010), it is necessary to yield a best system representation as close as possible to the ity at its

current operatin point. Under this concern, parameter estimation is perormed or a model correction. This ramewor o iterative model correction and control optimization is a topic or uture wor. Finally, in this study we ocus on steady-state control at a strateic level based on the arument that the areation o individual choice decisions ollows an equilibrium pattern. However in some circumstances such as ater a bride collapse (Zhu et al., 2010) day-to-day individual choice behavior would have stron eect on the collective choice behavior. Thus the areate day-to-day low dynamics is activated and transient behavior matters the day-to-day sinal control desin. The ramewor o iterative model correction and control optimization allows or the incorporation o transient route choice behavior and the resultin day-to-day low dynamics. In this case, day-to-day lows are interpreted as parameters to be estimated which are also iteration-varyin in ity inluenced by sinal control. This attractive property and other potentials brouht by the iterative ramewor are appealin and worth urther explorations. Reerence Aboudolas, K., Papaeoriou, M., Kosmatopoulos, E., 2009. Store-and-orward based methods or the sinal control problem in lare-scale conested urban road networs. Transportation Research Part C, 17(2), 163-174. Allsop, R.E., 1974. Some possibilities or usin traic control to inluence trip distribution and route choice. In Proceedins o the 6th International Symposium on Transportation and Traic Theory, Sydney, Australia. Arimoto, S., Kawamura, S., Miyazai, F., 1984. Betterin operation o dynamic systems by learnin: a new control theory or servomechanism or mechatronic systems. In Proceedins o 23rd Conerence Decision Control, Las Veas, NV, 1064 1069. Bellemans, T., De Schutter, B., De Moor, B., 2002. Model predictive control with repeated model ittin or ramp meterin. In Proceedins o the IEEE 5 th International Conerence on Intellient Transportation Systems, Sinapore, pp. 236 241. Bie, J., 2008. The dynamical system approach to traic assinment: the attainability o equilibrium and its application to traic system manaement. PhD thesis, Hon Kon University o Science and Technoloy. Boyd, S., Vandenberhe, L., 2004. Convex Optimization. University Press, Cambride. Brdy s, M., Tajewsi, P., 1994. An alorithm or steady-state optimizin dual control o uncertain plants. In Proceedins o the irst IFAC worshop on new trends in desin o control systems, pp. 249 254. Bristow, D. A., Tharayil, M., Alleyne, A. G., 2006. A survey o iterative learnin control: a learnin based method or hih-perormance tracin control. IEEE Control Systems Maazine, 26:96 114.

Cantarella, G. E., Velonà, P., Vitetta, A., 2012. Sinal settin with demand assinment: lobal optimization with day-to-day dynamic stability constraints. Journal o Advanced Transportation, vol 46(3), pp. 254-268. Cascetta, E., 2009. Transportation systems analysis: models and applications. Spriner, Second Edition. Cascetta, E., Gallo, M., Montella, B., 2006. Models and alorithms or the optimization o sinal settins on urban networs with stochastic assinment models. Ann. Operation Res. 144, pp. 301 328. Chachuat, B., Srinivasan, B., Bonvin, D., 2009. Adaptation strateies or -time optimization. Computers and Chemical Enineerin 33: 1557-1567. Chen, Y.Q., Moore, K.L., 2002. An optimal desin o PD-type iterative learnin control with monotonic converence. In Proceedins o the 2002 IEEE, International Symposium on Intellient Control, Vancouver, Canada, October 27-30, pp. 55-60. Cutler, C.R., Perry, R.T., 1983. Real time optimization with multivariable control is required to maximize proits. Computers and Chemical Enineerin, Vol. 7(5), 663-667. Dicson, T.J., 1981. A Note on Traic Assinment and Sinal Timins in a Sinal-Controlled Road Networ. Transportation Research Part B, Vol. 15, No. 4, pp. 267 271. Dinopoulou, V., Diaai, C., Papaeoriou, M., 2006. Applications o the urban traic control stratey TUC. European Journal o Operational Research, 175(3), 1652 1665. Fletcher, R., 1980. Practical Methods o Optimization, Vol. 1, Wiley Interscience, New Yor. Gao, W., Enell, S., 2005. Iterative set-point optimization o batch chromatoraphy. Computers and Chemical Enineerin 29, 1401-1409. Gartner, N.H., 1975. Area traic control and networ equilibrium. In: Florian M (ed) Proceedins o International Symposium on Traic Equilibrium Methods, Spriner-Verla, Berlin, pp 274-297. Gunnarsson, S., Norrlö, M., 2001. On the desin o ILC alorithms usin optimization. Automatica, vol. 37, no. 12, pp. 2011 2016. Han, D., Yan, H., Yuan, X., 2010. A practical trial-and-error implementation o marinal-cost pricin on networs. Journal o Industrial and Manaement Optimization, Vol. 6(2), 299-313. Head, K. L., Mirchandani, P. B., Sheppard, D., 1992. Hierarchical ramewor or -time traic control. In: Transportation Research Record: Journal o the Transportation Research Board, No. 1360, pp. 82-88. Heyi, A., De Schutter, B., Hellendoorn, J., 2005. Model predictive control or optimal coordination o ramp meterin and variable speed limits. Transportation Research Part C, 13(3), 185 209. Hu, T.Y., Mahmassani, H.S., 1997. Day-to-day evolution o networ lows under -time inormation and reactive sinal control. Transportation Research Part C 5(1), 51 69.

Huan, W., Viti, F., Tampère, C.M.J., 2013. An iterative learnin approach or sinal control in urban traic networs. In Proceedins o the 16th International IEEE annual conerence on Intellient Transportation Systems, the Haue, the Netherlands. Huan, W., Tampère, C.M.J., Viti, F., 2014. A daily adaptive low anticipatory control usin iterative learnin in urban traic networs. The 5th International Symposium on Dynamic Traic Assinment (DTA), Salerno, Italy. Hou, Z.S., Xu, J.X., Yan, J.W., 2008. An iterative learnin approach or density control o reeway traic low via ramp meterin. Transportation Research Part C 16, pp. 71-97. Lee, J. H., Lee, K. S., Kim, W. C., 2000. Model-based iterative learnin control with a quadratic criterion or time-varyin linear systems. Automatica, Vol. 36(5), 641 657. Lin, S., 2011. Eicient model predictive control or lare-scale urban traic networs. PhD thesis, Delt University o Technoloy, Delt, The Netherlands. Lo, H. K., Bie, J., 2006. Stability domains o traic equilibrium: directin traic system evolution to equilibrium. Proceedin: The First International Symposium on Dynamic Traic Assinment (DTA), 21-23 June 2006, Leeds, U.K.. Mansour, M., Ellis, J.E., 2003. Comparison o methods or estimatin process derivatives in online optimization. Applied Mathematical Modellin 27, 275-291. Marchetti, A., Chachuat, B., Bonvin, D., 2007. Real-time optimization o continuous processes via constraints adaptation. 8 th International IFAC Symposium on Dynamics and Control o Process Systems. Cancun, Mexico. Mauro, V., Di Taranto, C., 1990. UTOPIA. In Proceedins o the 6th IFAC/IFIP/IFORS symposium on control computers and communication in transportation, No. 12, pp. 245-252. Moore, K.L., 2001. An observation about monotonic converence in discretetime, P-type iterative learnin control. In Proc. IEEE Int. Symp. Intell. Contr., pp. 45 49. Osorio, C., Bierlaire, M., 2013. A simulation-based optimization ramewor or urban transportation problems. Operations Research 61(6), 1333-1345. Owens, D.H., Hätönen, J., 2005. Iterative learnin control an optimization paradim. Annual Reviews in Control 29, pp. 57-70. Par, K. H., Bien, Z., Hwan, D.H., 1999. A study on the robustness o a PID-type iterative learnin controller aainst initial state error. Int. J. Syst. Sci., vol. 30, no. 1, pp. 49 59. Roberts, P.D., 1995. Copin with model-ity dierences in industrial process optimization a review o interated system optimization and parameter estimation (ISOPE). Computers in Industry 26, 281-290. Roberts, P. D., 2000. Broyden derivative approximation in ISOPE optimizin and optimal control alorithms. In Proceedins o the 11 th IFAC worshop on control applications o optimisation: Vol. 1, pp.283 288.

Roberts, P. D., Williams, T. W. C., 1981. On an alorithm or combined system optimisation and parameter estimation. Automatica, 17(1), 199 209. Smith, M.J., 1979. Traic Control and Route-Choice: A Simple Example. Transportation Research Part B, Vol. 13, No. 4, pp. 289 294. Smith, M.J., Van Vuren, T., 1993. Traic Equilibrium with Responsive Traic Control. Transportation Science, Vol. 27, No. 2, pp. 118 132. Srinivasan, B., Bonvin, D., 2002. Interplay between identiication and optimization in run-to-run optimization schemes. In Proceedins o the American Control Conerence. May 8-10, Anchorae, AK. Taale, H., 2008. Interated Anticipatory Control o Road Networs. A ame theoretical approach. Ph.D. thesis, TRAIL thesis series T2008/15, Delt University o Technoloy, Delt, The Netherlands. Taale, H., Van Zuylen, H.J., 2003. The Eects o Anticipatory Control or Several Small Networs. In Proceedins o the 82nd Annual Meetin o the Transportation Research Board. Washinton D.C., U.S.A.. Vreeswij, J., Thomas, T., Van Berum, E., Van Arem, B., 2014. Perception bias in route choice. In: 93rd annual meetin o the Transportation Research Board, Washinton D.C.. Viti, F., Rinaldi, M., Corman, F., Tampère, C.M.J., 2014. Assessin partial observability in networ sensor location problems. Transportation Research Part B 70 pp. 65-89. Wan, Y., Papaeoriou, M., Messmer, A., 2006. RENAISSANCE A uniied macroscopic model-based approach to -time reeway networ traic surveillance. Transportation Research Part C 14, 190-212. Yan, H., Bell, M.G.H., 1998. Models and alorithms or road networ desin: a review and some new developments. TRANSPORT REVIEWS, VOL. 18, No. 3, 257-278. Yan, H., Yaar, S., 1995. Traic Assinment and Sinal Control in Saturated Road Networs. Transportation Research Part A, Vol. 29, No. 2, pp. 125 139. Yan, H., Men, Q., Lee, D.H., 2004. Trial-and-error implementation o marinal-cost pricin on networs in the absence o demand unctions. Transportation Research Part B 38, 477-493. Zhu, S., Levinson, D., Liu, H., Harder, K., 2010. The traic and behavioral eects o the I-35W Mississippi River bride collapse. Transportation Research Part A: Policy and Practice 44 (10), 771-784.