PARAMETER CHOICE IN BANACH SPACE REGULARIZATION UNDER VARIATIONAL INEQUALITIES

Transcription

1 PARAMETER CHOICE IN BANACH SPACE REGULARIZATION UNDER VARIATIONAL INEQUALITIES BERND HOFMANN AND PETER MATHÉ Abstract. The authors study arameter choice strategies for Tikhonov regularization of nonlinear ill-osed roblems in Banach saces. The effectiveness of any arameter choice for obtaining convergence rates deends on the interlay of the solution smoothness and the nonlinearity structure, and it can be exressed concisely in terms of variational inequalities. Such inequalities are link conditions between the enalty term, the norm misfit and the corresonding error measure. The arameter choices under consideration include an a riori choice, the discreancy rincile as well as the Leskiĭ rincile. For the convenience of the reader the authors review in an aendix a few instances where the validity of a variational inequality can be established. 1. Introduction In the ast years there was a significant rogress with resect to the error analysis including convergence rates results for regularized solutions to inverse roblems in Banach saces. Such roblems can be formulated as ill-osed oerator equations (1.1) F(x) = y with an (in general nonlinear) forward oerator F : D(F) X Y, with domain D(F), and maing between the Banach saces X and Y with norms X and Y, resectively. Equations of this tye frequently occur in natural sciences, engineering, imaging, and finance (see e.g. [19] and [20, Chater 1]). We denote by X and Y the corresonding dual saces and by, X X the dual airing betweenx andx. In this aer, for constructing stable aroximate solutions to (1.1) our focus is on the Tikhonov tye regularization based on noisy data y δ Y of the exact right-hand side y F(D(F)) under the deterministic noise model (1.2) y δ y Y δ. Date: Aril 11, Mathematics Subject Classification. 65J20, 47J06, 47A52, 49J40. 1

2 2 BERND HOFMANN AND PETER MATHÉ Precisely, we use for regularization arameters α > 0 regularized solutions x δ α D(F), which are minimizers of (1.3) T δ α(x) := 1 F(x) yδ Y +αω(x), subject to x D(F) X, with a convex enalty functional Ω : X [0, ] and some ositive exonent 1 < <. We suose in the sequel that the standard assumtions on F,D(F), and Ω, made for the Tikhonov regularization in [10] and in the recent monograhs [19, 20] are fulfilled. In articular, we assume that Ω is stabilizing, which means that for all c 0 the sublevel sets M Ω (c) := {x D(F) : Ω(x) c} are sequentially re-comact in a toologyτ X weaker than the norm toology of the Banach sace X. In this case minimizers x δ α D(F) of T α δ exist for all α > 0, and we refer to Section 2 for more details. The objective in the following study is to control a rescribed non-negative error functional, say E(x δ α,x ), measuring the deviation of the regularized solution x δ α from an Ω-minimizing solution, i.e., from a solution x to (1.1) for which we have Ω(x ) = min{ω(x) : x D(F), F(x) = y}. Tyical examles of error measures would be the norm misfit E(x,x ) = x x X or a ower E(x,x ) = x x q X of that with exonents1<q <. Within the resent context the Bregman distance (1.4) E(x,x ) = D Ω ξ (x,x ) := Ω(x) Ω(x ) ξ,x x X X. is often used, where we assume that x D(Ω) := {x X : Ω(x) < } and that x belongs to the Bregman domain x D B (Ω) := { x D(Ω) : Ω( x) }. Above, we denote by ξ Ω(x ) X the subdifferential of the convex functional Ω at the oint x, The Bregman distance was introduced into the regularization theory by the study [5] in 2004, and henceforth this concet was adoted, refined and develoed by many authors (cf., e.g., [9, 10, 14, 17, 18]). The goal of the resent aer is to study convergence rates of E(x δ α,x ) as δ 0 for several choices of the regularization arameter α = α(δ,y δ ). The quality of any arameter choice (in terms of rates of convergence) will deend on the interlay of the following four relevant ingredients, as these are (i) the smoothness of the solution x, (ii) the structure of the forward oerator F, and its domain D(F), (iii) roerties of the functional Ω, (iv) and the character of the error measure E(, ).

3 PARAMETER CHOICE UNDER VARIATIONAL INEQUALITIES 3 In this context, conditions are necessary that link the four factors. For E from (1.4) such conditions were resented in a rather general form in [10] as variational inequalities. Here we refer to the following variant (cf. [6, 7, 8]), which uses the concet of index functions. We call a function ϕ : (0, ) (0, ) an index function if it is continuous, strictly increasing, and satisfies the limit condition lim ϕ(t) = 0, see e.g. [11, 16]. t +0 Assumtion VI (variational inequality). We assume to have a constant 0 < β 1, and a concave index function ϕ such that (1.5) βe(x,x ) Ω(x) Ω(x )+ϕ( F(x) F(x ) Y ) for all x M. Remark 1. The set M of the validity of Assumtion VI must be large enough such that it contains x and all regularized solutions x δ α under consideration for 0 < δ δ max. This is for examle the case if M = M Ω (Ω(x )+c) for some c > 0. Moreover, there are good reasons to restrict in (1.5) to concave index functions. Namely, for index functions ϕ with lim ϕ(t) t +0 t = 0, including the family of strictly convex index functions, the variational inequality degenerates in the sense that Ω(x ) Ω(x) for all x M (see [7, Proosition 12.10], and ϕ(t) for a secial case [12, Proosition 4.3]). If 0 < lim t +0 t < then the situation is equivalent to the case ϕ(t) = c t, c > 0, in (1.5) (see [7, Proosition 12.11]), and for an index function ϕ with lim t +0 concave majorant index function that can be used in (1.5). ϕ(t) t ր + we can find a The outline is as follows. We resent the general methodology of our aroach in Section 2. Then we draw some consequences of the variational inequality (1.5) in form of inequalities in Section 3. Parts of these inequalities have been underestimated or even overlooked in ast work. However, they will be essentially used in Section 4 to derive error bounds for several arameter choices. In an aendix we shall indicate how (1.5) may be derived for some linear and nonlinear roblems. 2. Methodology and a fundamental error bound The existence and behavior of Tikhonov minimizers x δ α was analyzed in several studies (cf., e.g., [10, 19, 20]). Under natural assumtions, stated there, Ω-minimizing solutions x D := D(F) D(Ω) exist whenever (1.1) has a solution which belongs to D. Also, minimizers x δ α to the Tikhonov functional (1.3) exist for all data y δ Y and regularization arameters α > 0, and these are stable with resect to erturbations in the data for fixed α. Particularly relevant for our urose is the following: For

4 4 BERND HOFMANN AND PETER MATHÉ any arameter choice α = α (y δ,δ) satisfying (2.1) α 0 and we have convergence for both δ α 0 as δ 0 (2.2) Ω(x δ α ) Ω(x ) and F(x δ α ) F(x ) Y 0 as δ 0. Hence, all regularized solutions x δ α for sufficiently small δ > 0 belong to M Ω (Ω(x )+c), for some c > 0, and moreover if δ n 0 then the regularized solutions x δn α converge to (δ n) x in the weaker toology τ X of X. This is a weak convergence in the sense of subsequences if the Ω-minimizing solution x is not unique. For more details see, for examle, [20, Section 4.1.2]. This gives rise to the following methodology: In view of the convergence as stated in (2.2) and the variational inequality (1.5) the following region of stability is of interest. Definition (stability region).. Given δ > 0 and a concave index function ϕ we let { } F K,C (δ) := x D : Ω(x) Ω(x ) Kϕ(Cδ), F(x) F(x ) Y Cδ, be a stability region for the Ω-minimizing solution x D of (1.1) with constants K > 0, C 1. Notice that F K,C (δ) M Ω (Ω(x )+Kϕ(Cδ)), such that minimizers which are ushed towards F K,C (δ) belong to secified sublevel sets, in agreement with the outline in the beginning of this section. Here the constants C and K do not deend on δ, however on the exonent > 1, and on additional arameters used for the secific arameter choice. This methodology immediately allows for the following very elementary but fundamental error bound. Proosition 1. Let x obey Assumtion VI for some set M. If the aroximate solution x M belongs to a stability region F K,C (δ) for some K > 0, C 1 and δ > 0 then (2.3) E(x,x ) C K +1 β ϕ(δ). Above, we used the fact that ϕ(cδ) Cϕ(δ) is valid for all concave index functions ϕ and all δ > 0, C 1. The concet of stability region only controls the excess enalty Ω(x) Ω(x ), but not its modulus Ω(x) Ω(x ). In the roofs given below, we shall obtain the following strengthening. The arameter choices will ush the aroximate solutions towards the convergence region, given similarly to the Definition of the stability region as

5 PARAMETER CHOICE UNDER VARIATIONAL INEQUALITIES 5 Definition (convergence region).. For δ > 0 and a concave index function ϕ let { } FK,C conv (δ) := x D : Ω(x) Ω(x ) Kϕ(Cδ), F(x) F(x ) Y Cδ, be a convergence region for the Ω-minimizing solution x D of (1.1) with constants K > 0, C 1. Plainly, the inclusion F conv K,C (δ) F K,C(δ) holds. But the additional requirement rovides us with a rate of convergence for Ω(x δ α) Ω(x ), a surlus which we kindly areciate. Proosition 2. Let x obey Assumtion VI for some set M. If the aroximate solution x M belongs to a convergence region FK,C conv (δ) for some K > 0, C 1 and δ > 0 then, in addition to the assertion from Proosition 1, we have Ω(x) Ω(x ) KCϕ(δ) and F(x) F(x ) Y Cδ. The above bounds quantify the convergence assertions from (2.2) under Assumtion VI. We shall exhibit this methodology for a natural a riori arameter choice as well as for the discreancy rincile, and a variant of the Leskiĭ (balancing) rincile. 3. Preliminary estimates based on the variational inequality Before discussing arameter choice in detail we shall draw some first conclusions from the validity of the variational inequality. Here we neglect the secific structure of the error functional E(, ), and we only use its non-negativity. Let x δ α be any minimizer of the Tikhonov functional Tα δ from (1.3). The first observation is the following. Lemma 1. Under Assumtion VI we have for α > 0 and x δ α M that Ω(x ) Ω(x δ α) ϕ( F(x δ α) F(x ) Y ), and Ω(x δ α ) Ω(x ) δ α. Proof. The first assertion is an immediate consequence of (1.5) taking into account that β > 0 and E(x δ α,x ) 0. For the second we use the minimizing roerty to see that F(x δ α ) yδ Y +αω(x δ α) F(x ) y δ Y from which the assertion follows. +αω(x ) δ +αω(x ), Another conclusion is less obvious.

6 6 BERND HOFMANN AND PETER MATHÉ Lemma 2. Under Assumtion VI we have for α > 0 and x δ α M that F(x δ α ) F(x ) Y 2 δ +α2 1 ϕ( F(x δ α ) F(x ) Y ). Proof. We bound, by using the minimizing roerties of x δ α M, as 0 Ω(x δ α) Ω(x )+ϕ( F(x δ α) F(x ) Y ) 1 ( δ α 1 ) F(xδ α ) yδ Y +ϕ( F(x δ α ) F(x ) Y ). We also have the following lower bound F(x δ α ) yδ Y Inserting this we see that 1 F(x δ α ) F(x ) Y 2 1 δ. 0 2 δ 1 F(x δ α ) F(x ) Y 2 1 +αϕ( F(x δ α ) F(x ) Y ), which comletes the roof. The bound in Lemma 2 can be used on two ways. First, given a secific value of the arameter α > 0 we can bound the norm misfit from above. Secondly, assuming that the norm misfit is larger than δ we can bound the value of the arameter α from below. Both consequences will rove imortant. In this context we introduce the function (3.1) Φ (t) := t ϕ(t), t > 0, which is an index function for > 1, since ϕ is concave. Corollary 1. Let α be given from (3.2) α := Φ (δ). Then we have for α α and x δ α M that F(x δ α) F(x ) Y 2(2+) 1/( 1) δ.

7 PARAMETER CHOICE UNDER VARIATIONAL INEQUALITIES 7 Proof. If F(x δ α ) F(x ) Y > δ then we use the bound from Lemma 2 and the value for α to obtain F(x δ α) F(x ) Y 2 δ +α2 1 ϕ( F(x δ α) F(x ) Y ) 2 δ +α 2 1 ϕ( F(x δ α ) F(x ) Y ) = 2 δ +δ 2 1 ϕ( F(x δ α) F(x ) Y ) ϕ(δ) ( ) ϕ( F(x δ α) F(x ) Y ) ϕ(δ) δ 2 1 (2+) δ 2 1 (2+) δ ϕ( F(x δ α ) F(x ) Y ) ϕ(δ) δ 1 F(x δ α ) F(x ) Y ϕ(δ) ϕ(δ) = 2 1 (2+) δ 1 F(xδ α) F(x ) Y. Because of 2(2+) 1/( 1) > 1 the bound given in the corollary is also valid for F(x δ α ) F(x ) Y δ. Corollary 2. Let τ > 1. Suose that the arameter α > 0 is chosen such that x δ α M and the residual obeys F(xδ α ) yδ Y > τδ. Then we have (3.3) α τ 1 τ +1 Φ ((τ 1)δ). Proof. Using the first assertion in Lemma 1 and that x δ α M is a minimizer of T δ α we have under the assumtion made on α that τ δ Thus F(xδ α ) yδ Y δ +α ( Ω(x ) Ω(x δ α ) ) δ +αϕ( F(xδ α ) F(x ) Y ). δ 1 τ 1 αϕ( F(xδ α ) F(x ) Y ). We lug this into the bound in Lemma 2, and we temorarily abbreviate t α := F(x δ α) F(x ) Y. We thus obtain that t α 2 δ +α2 1 ϕ(t α ) 2 1 τ 1 αϕ(t α)+α2 1 ϕ(t α ) ( ) = 2 τ αϕ(t α ) = 2 1τ +1 τ 1 αϕ(t α). Since τδ t α +δ, we arrive, using the function Φ from (3.1), at and the roof is comlete. Φ ((τ 1)δ) Φ (t α ) 2 1τ +1 τ 1 α,

8 8 BERND HOFMANN AND PETER MATHÉ 4. Parameter choice The objective of this study is the error analysis of several arameter choice strategies, commonly used in regularization theory. This concerns a riori strategies, i.e., when α = α (δ) does not deend on the given data y δ, as well as a osteriori strategies, when α = α (y δ,δ) A natural a riori arameter choice. Several a riori arameter choices can be found in earlier studies (cf. [4, 6]). Here we resent an intuitive arameter choice, which was obtained in [8] by means of tools from convex analysis. Our aroach, however, is elementary and directly based on Assumtion VI. In addition we show that this arameter choice ushes the aroximate solution x δ α into a secific set FK,C conv (δ). We recall the index function Φ from (3.1). Theorem 1. Suose that x obeys Assumtion VI for some concave index function ϕ. Let α = α (δ) = Φ (δ) be chosen a riori. (i) If x δ α M then x δ α F conv K,C (δ) with K = 1 and C = 2(2+)1/( 1). (ii) If x δ α M for all 0 < δ δ max and some δ max > 0, then this a riori arameter choice yields the convergence rates E(x δ α,x ) = O(ϕ(δ)), F(x δ α ) F(x ) Y = O(δ), and Ω(x δ α ) Ω(x ) = O(ϕ(δ)) as δ 0. Proof. Corollary 1 rovides us with a bound of the norm misfit F(x δ α ) F(x ) Y. In view of the first assertion of Lemma 1 this also bounds Ω(x ) Ω(x δ α ), aroriately. Furthermore, from the second assertion of Lemma 1 we have that Ω(x δ α ) Ω(x ) δ /(α ) ϕ(δ), by the choice of α. The convergence rates in Item (ii) are a consequence of Proositions 1 & 2. Remark 2. An insection of the roofs in Section 3 shows that the first two convergence rates in Theorem 1 use only the imlication x δ α M = x δ α F K,C (δ), for sufficiently small δ > 0. Only for the third Ω-rate the membershi x δ α FK,C conv (δ) is required. The a riori arameter choice from (3.2) satisfies the condition (2.1). Moreover, it corresonds to the well-known a riori arameter choice for linear roblems in Hilbert saces, and this shows how ϕ is related to smoothness. Indeed, taking into account that E(x,x ) = x x 2 X measures the squared error, we obtain a rate x δ α x X = O( ϕ(δ)). Presuming that this is the otimal rate, we guess the relation ϕ(t) ψ(θ 1 ψ (t)) by setting Θ ψ (t) := tψ(t), t > 0, where ψ should be the smoothness in the source condition for x, meaning that x = ψ(a A)v, v X 1. Taking this for

9 granted we see that PARAMETER CHOICE UNDER VARIATIONAL INEQUALITIES 9 α δ ϕ(δ) = ( δ Θ ψ ψ(θ 1 ψ (δ)) = ) Θ 1 ψ (δ) ψ(θ 1 ψ (δ)) = Θ 1 ψ (δ), and hence Θ ψ (α ) = δ, which is the ordinary a riori arameter choice in linear roblems in Hilbert sace under given source condition x = ψ(a A)v, v X 1 (cf. [16]) A osteriori arameter choice. For the a osteriori arameter choice we restrict the selection of the regularization arameter to a discrete exonential grid. Precisely, we select 0 < q < 1, choose a largest arameter α 0 and consider the set (4.1) q := { α j : α j := q j α 0, j = 1,2,... }. Above, the arameter q determines the roughness of searching for the otimal arameter. If q is close to one than we scan for the otimal arameter accurately, however, many trials may be necessary to find the best candidate. If q is small than we roughly scan for the arameter, at a disense of loosing accuracy Discreancy rincile. Previous use of the discreancy rincile for nonlinear roblems in Banach sace was restrictive; a stronger version was used. Precisely, for two arameters 1 < τ 1 < τ 2 < the chosen arameter α was assumed to fulfill τ 1 δ F(x δ α ) y δ Y τ 2 δ (cf. [1, 7]). We shall call this the strong discreancy rincile. It is not clear that this is always ossible, and it was mentioned in [20, Chat. 4] that for nonlinear oerators F there may be a duality ga due to the non-convexity of the functional T δ α which revents the use of this strong discreancy rincile. Here we establish the use of the in general alicable classical discreancy rincile for which the variational inequality in Assumtion VI is strong enough to ensure convergence rates. Theorem 2. Let τ > 1 be given. Let α q, α < α 1 (no immediate sto) be chosen, according to the discreancy rincile, as the largest arameter within q for which F(x δ α ) yδ Y τδ. Suose that x obeys Assumtion VI for some concave index function ϕ. Then the following holds true. (i) If x δ α M, α α, then x δ α FK,C conv (δ) with K = max { 1 2q ( 2 τ 1) τ +1 τ 1,1 } and C = τ +1. (ii) If x δ α M for all 0 < δ δ max and some δ max > 0, then this a osteriori arameter choice yields the convergence rates E(x δ α,x ) = O(ϕ(δ)), F(x δ α ) F(x ) Y = O(δ), and Ω(x δ α ) Ω(x ) = O(ϕ(δ)) as δ 0.

10 10 BERND HOFMANN AND PETER MATHÉ Proof. We first bound F(x δ α ) F(x ) Y F(x δ α ) y δ Y + F(x ) y δ Y τδ+δ = (τ +1)δ. Under Assumtion VI this also gives Ω(x ) Ω(x δ α ) ϕ((τ + 1)δ), cf. Lemma 1. For bounding the negative Ω-difference we use Corollary 2 as follows. The (revious) arameter α /q fulfills the assumtion from Corollary 2, and we bound from below as α /q τ 1 τ +1 Φ ((τ 1)δ). This, together with the second assertion of Lemma 1, yields Ω(x δ α ) Ω(x ) ( τ δ α τ 1 qφ ((τ 1)δ) ( ) = 1 2 τ +1 2q τ 1 τ 1 1 2q ( 2 τ 1 ) δ ϕ((τ 1)δ) ) τ +1 τ 1ϕ((τ +1)δ). The convergence rates in Item (ii) are again consequences of Proositions 1 and 2. Remark 3. We emhasize that here we bounded Ω(x δ α ) Ω(x ) 1 ( ) 2 τ +1 2q τ 1 τ ϕ((τ +1)δ), 1 whereas the strong discreancy rincile yields Ω(x δ α ) Ω(x ) 0, which seems to be chicken hearted, and this oints at the limitations of this strong rincile. Remark 4. We shall conclude this section with a discussion on immediate sto of the discreancy rincile. Notice that in this rincile we start from the largest value α 1. In fact, it may haen that for α := α 1 the assumtion F(x δ α 1 ) y δ Y τδ is already fulfilled. In this case, the above roof will not work, and the question is, whether there is still the rate O(ϕ(δ)) to be observed. This can indeed be roved for linear roblems in Hilbert sace (cf. [3] for a recent treatment): For Tikhonov regularization in Hilbert sace this corresonds to the case of having small data, because (for linear roblems in Hilbert sace) x δ α 0 as α, and hence immediate sto refers to y δ Y τδ. Within the resent context we make the following observation. If the discreancy bound τδ holds, then the error bound E(x δ α 1,x ) holds if only Ω(x δ α 1 ) Ω(x ) is small. A look at the second bound given in Lemma 1 reveals that a bound δ /(α 1 ) is valid. So, the desired overall error bound holds rovided that δ /(α 1 ) Kϕ(δ), or equivalently, by using the index function Φ from (3.1), that (4.2) Φ (δ) Kα 1.

11 PARAMETER CHOICE UNDER VARIATIONAL INEQUALITIES 11 Plainly, for each solution x there is δ 0 such that Φ (δ) Φ (δ 0 ) α 1. Thus, for 0 < δ δ 0 we can bound Ω(x δ α 1 ) Ω(x ) δ α 1 = δ Φ (δ) = ϕ(δ). This shows that if the initial value α 0 is chosen large enough then immediate sto yields an error bound of the form O(ϕ(δ)). However, for any articular instance x at hand we cannot verify whether (4.2) holds, since ϕ is not known to us The Leskiĭ rincile. The Leskiĭ (balancing) rincile is studied here for the first time within the context of nonlinear equations in Banach sace regularization. However, it was used for nonlinear equations in Hilbert sace, and we refer to [2]. Actually, by its very construction this arameter choice is not sensitive to the roblem at hand, a generic formulation for this rincile was given in [15]. This rincile requires that the error functional is a metric, and we assume this within the resent section without further mentioning. We will need the following fact. Lemma 3. Suose that x obeys Assumtion VI, and that the arameter α AP is given as in Theorem 1. Then for all α m α α AP and for x δ α M we have that ( ) 1 δ βe(x δ α,x ) +2(2+)1/( 1) α. Proof. Under Assumtion VI, and using Lemma 1 and Corollary 1 we see that βe(x δ α,x ) Ω(x δ α) Ω(x )+ϕ( F(x δ α) x Y ) δ α +ϕ(c δ) δ α +C ϕ(δ) ) δ α, = δ α +C δ α = ( 1 +C where we abbreviated C := 2(2+) 1/( 1), the constant from Corollary 1. We want to use the Leskiĭ rincile, as this is outlined in [15], by using a multile of the decreasing function α δ /α, and we let (4.3) Ψ(α) := 1+C β δ α, α > 0. From [15, Pro. 1] we draw the following conclusion. Theorem 3. Fix m > 1 (large) and let α q be the largest arameter α for which E(x δ α,xδ α) 2Ψ(α ), for all α q, α m α < α.

12 12 BERND HOFMANN AND PETER MATHÉ Moreover, suose that x obeys Assumtion VI for some concave index function ϕ. If x δ α M for all α m α α and if E(x δ α m,x ) Ψ(α m ), then E(x δ α,x ) 3 1+C qβ ϕ(δ). Proof. As in [15, Pro. 1] we introduce the arameter { } α + := max α : E(x δ α,x ) Ψ(α ), α m α α. (Caution: the notation in [15] differs from here, and some care is needed to transfer the results.) Letα AP = Φ (δ) be the a riori choice from Theorem 1. If α AP q then Lemma 3 yields that α + α AP. Otherwise, we consider the index k m for which α k < α AP α k /q, which results in α + α k. Proosition 1 in [15] states that E(x δ α,x ) 3Ψ(α + ). Thus in either case this yields E(x δ α,x ) 3Ψ(α + ) 3Ψ(α k ) = 3 q Ψ(α k/q) 3 q Ψ(α AP) 1+C qβ ϕ(δ), which comletes the roof. Remark 5. The arameter choice à la Leskiĭ rovides us with an error bound, which is obtained regardless whether the aroximating x δ α belongs to some set F K,C (δ). In fact, the only information which can be deduced from Theorem 3 is the following lower bound for α : Actually, in [15, Pro. 2.1] the information is given that α α +, such that α α + qα AP = qφ (δ). This bounds the excess enalty Ω(x δ α ) Ω(x ) ϕ(δ)/q. However, it is not clear whether the discreancy F(x δ α ) y δ Y is of the order δ. Remark 6. For the Leskiĭ rincile we have to start with the smallest chosen value α m, and in the formulation of Theorem 3 we assumed that E(x δ α m,x ) Ψ(α m ). Since Ψ(α m ) is known to the user, some exogenous knowledge about the exected error size may allow to adjust for the choice of m, and hence of α m. However, we cannot verify this condition, based on information of δ and the given data y δ. So, if the Leskiĭ rincile stos immediately, one should decrease the initial value α m until this will not be the case. Remark 7. The above alication of the Leskiĭ rincile does not include the Bregman distance as error measure E(x,x ) := D Ω ξ (x,x ), because this is not a metric, in general. However, if the Bregman distance is q-coercive, D Ω ξ (x,x ) c x x q X q, then the validity of a variational inequality for D Ω ξ (x,x ) imlies the one for x x q X /q, and we can aly the Leskiĭ arameter choice to the differences x δ α xδ α, i.e., test whether x δ α xδ α X (2 1+C β ) 1/q δ /q /(α ) 1/q for α α.

13 PARAMETER CHOICE UNDER VARIATIONAL INEQUALITIES 13 Aendix A. Validity of Assumtion VI For the convenience of the reader we briefly sketch some aroaches to show how variational inequalities occur. We suose that the maing F : D(F) X Y with some convex domain D(F) has a one-sided directional derivative at x given as a bounded linear oerator F (x ) : X Y such that 1 ( ) (A.1) lim F(x +t(x x )) F(x ) = F (x )(x x ), x D(F). t +0 t For the alication of several variational inequalities of tye (1.5) that we will derive below, in the context of Tikhonov regularization, one still has to show that the minimizers x δ α of (1.3) belong to the domain M of validity of such inequality. This requires secial attention, and we do not tackle this question. We leave details to the indicated original references (see also [20, Sections 3.2 and 4.2]). A.1. Benchmark case. Here we assume that x D B (Ω) and the subdifferential ξ fulfills the benchmark source condition (A.2) ξ = F (x ) v Ω(x ), for some v Y. Such information allows us to bound ξ,x x X X = (F (x )) v,x x X X = v,f (x )(x x) Y Y v Y F (x )(x x ) Y. After adding the term Ω(x) Ω(x ) on both sides this yields that (A.3) D Ω ξ (x,x ) Ω(x) Ω(x )+ v Y F (x )(x x ) Y, x D(F). Remark 8. We highlight the secial case when X is a Hilbert sace and Ω(x) = x 2 X. Then DΩ (x,x ) = x x 2 ξ X (cf. [19, Examle 3.18]), and (A.3) imlies x x 2 X x 2 X x 2 X + v Y F (x )(x x ) Y, x D(F). It was emhasized in [6, Chater 13] that within the Hilbert sace setting solution smoothness can always be exressed by variational inequalities (1.5) with general index functions ϕ. Inequality (A.3) results in a variational inequality for bounded linear oerators F = A : X Y, because then A = F (x ), with β = 1, E(x,x ) = D Ω ξ (x,x ) and ϕ(t) = v Y t, t > 0 on M = X. If the maing F is nonlinear then we may use certain structure of nonlinearity to bound F (x )(x x ) Y in terms of F(x ) F(x) Y, and the validity of such structural conditions requires additional assumtions, which we will not discuss here. In its simlest form such condition is given as (A.4) F (x )(x x ) Y ησ( F(x) F(x ) Y ), x M,

14 14 BERND HOFMANN AND PETER MATHÉ for some concave index functionσ and constantη > 0 on some setm D(F) (cf. [4]). In this case (A.3) rovides us with a variational inequality on M with β = 1, E(x,x ) = D Ω ξ (x,x ) and ϕ(t) = η v Y σ(t), t > 0. An alternative structural condition is given in the form (A.5) F(x) F(x ) F (x )(x x ) Y ηd Ω ξ (x,x ), x M, again for some set M D(F), (cf., e.g., [10, 18]). This allows us to bound F (x )(x x ) Y ηd Ω ξ (x,x )+ F(x) F(x ) Y, x M. Then (A.3) imlies a variational inequality (1.5) under (A.6) η v Y < 1 with 0 < β = 1 η v Y 1, E(x,x ) = D Ω ξ (x,x ) and ϕ(t) = v Y t, t > 0 on M. This occurring smallness condition (A.6) indicates that (A.5) is a weaker nonlinearity condition comared with (A.4). A.2. Violation of the benchmark. If the assumtion (A.2) is violated then we may use the method of aroximate source conditions (cf. [4, 9]) to derive variational inequalities. To this end we need additionally that the distance function d ξ (R) := inf{ ξ ξ X : ξ = F (x ) v, v Y, v Y R}, R > 0, is nonincreasing and obeys the limit condition d ξ (R) 0 as R. As mentioned in [4] this is the case when F (x ) : X Y is injective. Additionally this aroach resumes that the Bregmann distance is q-coercive, i.e., that (A.7) D Ω ξ (x,x ) c q x x q X for all x M, is satisfied for some exonent 2 q < and a corresonding constant c q > 0. Such assumtion is for examle fulfilled if Ω(x) := x q X and X is a q-convex Banach sace. Then, for every R > 0 one can find elements v R Y and u R X such that ( vr ξ = F (x )) +u R with v R Y = R, u R X d ξ (R), and we can estimate for all R > 0 and x M as ξ,x x X X = ( F (x ) ) vr +u R,x x X X = v R,F (x )(x x ) Y Y + u R,x x X X R F (x )(x x ) Y +d ξ (R) x x X. Adding, as before, the difference Ω(x) Ω(x ) on both sides gives (A.8) D Ω ξ (x,x ) Ω(x) Ω(x )+R F (x )(x x ) Y +d ξ (R) x x X, x M.

15 PARAMETER CHOICE UNDER VARIATIONAL INEQUALITIES 15 Using the q-coercivity (A.7) we see that d ξ (R) x x X c 1/q q d ξ (R) An alication of Young s inequality yields ( D Ω ξ (x,x )) 1/q. d ξ (R) x x X 1 /q q DΩ ξ (x,x )+ c q q ( dξ q (R) ) q. Plugging this into (A.8) we obtain (with β = 1 1/q) that (A.9) /q βd Ω ξ (x,x ) Ω(x) Ω(x )+R F (x )(x x ) Y + c q q ( dξ q (R) ) q. The term F (x )(x x ) Y may be treated under structural conditions, used before in the benchmark case. To avoid this ste we confine ourselves to the linear case F (x ) = A, below. We equilibrate the second and the third term, deending of R and d ξ (R), resectively, by means of the auxiliary continuous and strictly decreasing function ( dξ (R) ) q (A.10) Φ(R) :=, R > 0, R which fulfills the limit conditions lim Φ(R) = and lim Φ(R) = 0, thus R 0 R it has a continuous decreasing inverse Φ 1 : (0, ) (0, ). By setting R := Φ 1( A(x x ) ) Y and introducing the index function ζ(t) := [ dξ (Φ 1 (t)) ] q, t > 0, we get from (A.9), with some constant ˆK > 0, a variational inequality of the form βd Ω ξ (x,x ) Ω(x) Ω(x )+ ˆKζ( A(x x ) Y ), x M. Remark 9. We observe, with t = Φ(R), R > 0, that t ζ(t) = [ Φ(R) dξ (R) ] q = 1 0 as t 0. R Thus the function t ζ(t) decreases to zero as t 0. In this case there is a concave majorant index function ϕ to ζ (cf. [13, Chat. 5]) such that βd Ω ξ (x,x ) Ω(x) Ω(x )+ϕ( A(x x ) Y ), x M, with the constant β = 1 1/q > 0, and an index function ϕ which is a multile of ϕ. References [1] S. W. Anzengruber and R. Ramlau. Morozov s discreancy rincile for Tikhonovtye functionals with nonlinear oerators. Inverse Problems, 26:025001, 17., [2] F. Bauer and T. Hohage. A Leskij-tye stoing rule for regularized Newton methods. Inverse Problems, 21(6): , 2005.

16 16 BERND HOFMANN AND PETER MATHÉ [3] G. Blanchard and P. Mathé. Conjugate gradient regularization under general smoothness and noise assumtions. J. Inverse Ill-Posed Probl., 18(6): , [4] R. I. Boț and B. Hofmann. An extension of the variational inequality aroach for obtaining convergence rates in regularization of nonlinear ill-osed roblems. Journal of Integral Equations and Alications, 22(3): , [5] M. Burger and S. Osher. Convergence rates of convex variational regularization. Inverse Problems, 20(5): , [6] J. Flemming. Generalized Tikhonov Regularization: Basic Theory and Comrehensive Results on Convergence Rates. Technische Universität Chemnitz, Det. Mathematics, Chemnitz, Germany, PhD thesis (Dissertation), htt://nbn-resolving.de/urn:nbn:de:bsz:ch1-qucosa [7] J. Flemming. Generalized Tikhonov Regularization and Modern Convergence Rate Theory in Banach Saces. Shaker Verlag, Aachen, [8] M. Grasmair. Generalized Bregman distances and convergence rates for non-convex regularization methods. Inverse Problems, 26(11):115014, 16., [9] T. Hein and B. Hofmann. Aroximate source conditions for nonlinear ill-osed roblems chances and limitations. Inverse Problems, 25(3):035003, 16., [10] B. Hofmann, B. Kaltenbacher, C. Pöschl, and O. Scherzer. A convergence rates result for Tikhonov regularization in Banach saces with non-smooth oerators. Inverse Problems, 23(3): , [11] B. Hofmann and P. Mathé. Analysis of rofile functions for general linear regularization methods. SIAM J. Numer. Anal., 45(3): , [12] B. Hofmann and M. Yamamoto. On the interlay of source conditions and variational inequalities for nonlinear ill-osed roblems. Al. Anal., 89(11): , [13] N. Korneĭchuk. Exact constants in aroximation theory, volume 38 of Encycloedia of Mathematics and its Alications. Cambridge University Press, Cambridge, [14] D. A. Lorenz. Convergence rates and source conditions for Tikhonov regularization with sarsity constraints. J. Inverse Ill-Posed Probl., 16(5): , [15] P. Mathé. The Leskiĭ rincile revisited. Inverse Problems, 22(3):L11 L15, [16] P. Mathé and S. V. Pereverzev. Geometry of linear ill-osed roblems in variable Hilbert scales. Inverse Problems, 19(3): , [17] E. Resmerita. Regularization of ill-osed roblems in Banach saces: convergence rates. Inverse Problems, 21(4): , [18] E. Resmerita and O. Scherzer. Error estimates for non-quadratic regularization and the relation to enhancement. Inverse Problems, 22(3): , [19] O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier, and F. Lenzen. Variational Methods in Imaging, volume 167 of Alied Mathematical Sciences. Sringer, New York, [20] T. Schuster, B. Kaltenbacher, B. Hofmann, and K.S. Kazimierski. Regularization Methods in Banach Saces. Walter de Gruyter, Berlin, Deartment of Mathematics, Chemnitz University of Technology, Chemnitz, Germany address: hofmannb@mathematik.tu-chemnitz.de Weierstraß Institute for Alied Analysis and Stochastics, Mohrenstraße 39, Berlin, Germany address: eter.mathe@wias-berlin.de