Foundations of Reproducing Kernel Hilbert Spaces Advanced Topics in Machine Learning
|
|
|
- Buck Boyd
- 7 years ago
- Views:
Transcription
1 Foundations of Reproducing Kernel Hilbert Spaces Advanced Topics in Machine Learning slides: teaching.html Gatsby Unit March 22, 2016
2 Overview 1 Concepts from functional analysis: normed-, inner product space, convergent-, Cauchy sequence, complete spaces: Banach-, Hilbert space, continuous/bounded linear operators.
3 Overview 2 RKHS: different views: 1 continuous evaluation functional, 2 reproducing kernel, 3 positive definite function, 4 feature view (kernel). equivalence, explicit construction.
4 Normed space We define the length of a vector. F: vector space over R. : F [0, ) is norm on F, if 1 f = 0 iff. f = 0 (norm separates points),
5 Normed space We define the length of a vector. F: vector space over R. : F [0, ) is norm on F, if 1 f = 0 iff. f = 0 (norm separates points), 2 λf = λ f λ R, f F (positive homogenity),
6 Normed space We define the length of a vector. F: vector space over R. : F [0, ) is norm on F, if 1 f = 0 iff. f = 0 (norm separates points), 2 λf = λ f λ R, f F (positive homogenity), 3 f + g f + g f, g F (triangle inequality).
7 Normed space We define the length of a vector. F: vector space over R. : F [0, ) is norm on F, if 1 f = 0 iff. f = 0 (norm separates points), 2 λf = λ f λ R, f F (positive homogenity), 3 f + g f + g f, g F (triangle inequality). Note: norm metric: d(f, g) = f g study continuity, convergence.
8 Normed space: examples (R, ), ( R d, x p = [ ) i x i p ] 1 p, 1 p. p = 1: x 1 = i x i (Manhattan), p = 2: x 2 = i x i 2 (Euclidean), p = : x = max i x i (maximum norm). ( C[a, b], f p = [ b a f(x) p dx ] 1 p ), 1 p.
9 Inner product space F: vector space over R., : F F R is an inner product on F if for α i R, f i, f, g F 1 α 1 f 1 +α 2 f 2, g = α 1 f 1, g +α 2 f 2, g (linearity),
10 Inner product space F: vector space over R., : F F R is an inner product on F if for α i R, f i, f, g F 1 α 1 f 1 +α 2 f 2, g = α 1 f 1, g +α 2 f 2, g (linearity), 2 f, g = g, f (symmetry),
11 Inner product space F: vector space over R., : F F R is an inner product on F if for α i R, f i, f, g F 1 α 1 f 1 +α 2 f 2, g = α 1 f 1, g +α 2 f 2, g (linearity), 2 f, g = g, f (symmetry), 3 f, f 0; f, f = 0 f = 0.
12 Inner product space F: vector space over R., : F F R is an inner product on F if for α i R, f i, f, g F 1 α 1 f 1 +α 2 f 2, g = α 1 f 1, g +α 2 f 2, g (linearity), 2 f, g = g, f (symmetry), 3 f, f 0; f, f = 0 f = 0. Notes: 1, 2 bilinearity. inner product norm: f = f, f. 1,2,3 ( f, f 0) is called semi-inner product.
13 Inner product space: examples ( R d, x, y = i x iy i ). (R d 1 d 2, A, B F = tr(a T B) = ij A ijb ij ). ( C[a, b], f, g = b a f(x)g(x)dx ).
14 Norm vs inner product Relations: f, g f g (CBS), 4 f, g = f + g 2 f g 2 (polarization identity), f + g 2 + f g 2 = 2 f g 2 (parallelogram law).
15 Norm vs inner product Relations: Notes: f, g f g (CBS), 4 f, g = f + g 2 f g 2 (polarization identity), f + g 2 + f g 2 = 2 f g 2 (parallelogram law). CBS holds for semi-inner products. parallelogram law = characterization of,.
16 Convergent-, Cauchy sequence F: normed space, {f n } n=1 F, f F, Convergent sequence: f n F f if ǫ > 0 N = N(ε) N, s.t. n N, f n f F < ǫ.
17 Convergent-, Cauchy sequence F: normed space, {f n } n=1 F, f F, Convergent sequence: f n F f if ǫ > 0 N = N(ε) N, s.t. n N, f n f F < ǫ. Cauchy sequence: {f n } n=1 is a Cauchy sequence if ǫ > 0 N = N(ε) N, s.t. n, m N, f n f m F < ǫ.
18 Convergent-, Cauchy sequence F: normed space, {f n } n=1 F, f F, Note: Convergent sequence: f n F f if ǫ > 0 N = N(ε) N, s.t. n N, f n f F < ǫ. Cauchy sequence: {f n } n=1 is a Cauchy sequence if ǫ > 0 N = N(ε) N, s.t. n, m N, f n f m F < ǫ. convergent Cauchy: f n f m F f n f F + f f m F.
19 Not every Cauchy sequence converges Examples: 1, 1.4, 1.41, 1.414, ,...: Cauchy in Q, but 2 / Q. ( ) C[0, 1], L 2 [0,1] : 1 f n n 1 n n But a Cauchy sequence is bounded.
20 Banach space, Hilbert space Complete space: Cauchy sequence converges.
21 Banach space, Hilbert space Complete space: Cauchy sequence converges. Banach space = complete normed space, e.g. 1 Let p [1, ), L p (X,A,µ) := { [ } 1/p f : (X,A) R measurable : f p = f(x) dµ(x)] p <. X 2 ( C[a, b], f = max x [a,b] f(x) ).
22 Banach space, Hilbert space Complete space: Cauchy sequence converges. Banach space = complete normed space, e.g. 1 Let p [1, ), L p (X,A,µ) := { [ } 1/p f : (X,A) R measurable : f p = f(x) dµ(x)] p <. ( 2 C[a, b], f = max x [a,b] f(x) ). Hilbert space = complete inner product space; L 2 (X,A,µ). X
23 Linear-, bounded operator F, G: normed spaces. A : F G is called linear operator: 1 A(αf) = α(af) α R, f F, (homogeneity), 2 A(f + g) = Af + Ag f, g F (additivity). G = R: linear functional.
24 Linear-, bounded operator F, G: normed spaces. A : F G is called linear operator: 1 A(αf) = α(af) α R, f F, (homogeneity), 2 A(f + g) = Af + Ag f, g F (additivity). G = R: linear functional. bounded operator: A is linear & A = sup f F Af G f F <. 1 F λ G A : F G {Af : f F 1}
25 Unbounded linear functional: example ( C 1 [0, 1], f := max x [0,1] f(x) ), A(f) = f (0) R: 1 A: linear differentiation & evaluation are linear, 2 f n (x) = e nx (n Z + ): f n 1, but A(f n ) = f n(0) = ne nx x=0 = n = n.
26 Continuous operator Def.: A is continuous at f 0 F: ǫ > 0 δ = δ(ǫ, f 0 ) > 0, s.t. f f 0 F < δ implies Af Af 0 G < ǫ. continuous: if it is continuous at f 0 F.
27 Continuous operator Def.: A is continuous at f 0 F: ǫ > 0 δ = δ(ǫ, f 0 ) > 0, s.t. f f 0 F < δ implies Af Af 0 G < ǫ. continuous: if it is continuous at f 0 F. Example: Let A g (f) := f, g F R, where f, g F. A g is Lipschitz continuous: Ag (f 1 ) A g (f 2 ), F : lin. = f 1 f 2, g F CBS g F f 1 f 2 F.
28 Continuous-bounded relations Theorem: A: linear operator. Equivalent: A is 1 continuous, 2 continuous at one point, 3 bounded.
29 Continuous-bounded relations Theorems: A: linear operator. Equivalent: A is 1 continuous, 2 continuous at one point, 3 bounded. Riesz representation (F: Hilbert, G = R): continuous linear functionals = {, g F : g F}.
30 Let us switch to RKHS-s!
31 View-1: continuous evaluation. Let H R X be a Hilbert space. Consider for fixed x X the δ x : f H f(x) R map.
32 View-1: continuous evaluation. Let H R X be a Hilbert space. Consider for fixed x X the δ x : f H f(x) R map. The (Dirac) evaluation functional is linear: δ x (αf +βg) = (αf +βg)(x) = αf(x)+βg(x) = αδ x (f)+βδ x (g) ( α,β R, f, g H).
33 View-1: continuous evaluation. Let H R X be a Hilbert space. Consider for fixed x X the δ x : f H f(x) R map. The (Dirac) evaluation functional is linear: δ x (αf +βg) = (αf +βg)(x) = αf(x)+βg(x) = αδ x (f)+βδ x (g) ( α,β R, f, g H). Def.: H is called RKHS if δ x is continuous x X.
34 Example for non-continuous δ x H = L 2 [0, 1] f n (x) = x n : 1 f n 0 H since lim f n 0 n 2 = lim n ( 1 0 1/2 x dx) 2n 1 = lim = 0, n 2n+1
35 Example for non-continuous δ x H = L 2 [0, 1] f n (x) = x n : 1 f n 0 H since lim f n 0 n 2 = lim n ( 1 2 but δ 1 (f n ) = 1 δ 1 (0) = /2 x dx) 2n 1 = lim = 0, n 2n+1 In L 2 : norm convergence pointwise convergence.
36 View-1: convergence In RKHS: convergence in norm pointwise convergence! Result: f n H f fn x f.
37 View-1: convergence In RKHS: convergence in norm pointwise convergence! Result: f n H f fn x f. Proof: For any x X, f n (x) f(x) δx def δx lin = δ x (f n ) δ x (f) = δ x (f n f) δ x: bounded δ x f }{{} n f H. }{{} < 0
38 View-2: reproducing elements, kernel trick. Let H be a Hilbert space of X R functions. k : X X R is called a reproducing kernel of H if for x X 1 k(, x) H ( generators ),
39 View-2: reproducing elements, kernel trick. Let H be a Hilbert space of X R functions. k : X X R is called a reproducing kernel of H if for x X, f H 1 k(, x) H ( generators ), 2 f, k(, x) H = f(x) (reproducing property).
40 View-2: reproducing elements, kernel trick. Let H be a Hilbert space of X R functions. k : X X R is called a reproducing kernel of H if for x X, f H 1 k(, x) H ( generators ), 2 f, k(, x) H = f(x) (reproducing property). Specifically: x, y X, k(x, y) = k (, x),k (, y) H.
41 View-2: reproducing elements, kernel trick. Let H be a Hilbert space of X R functions. k : X X R is called a reproducing kernel of H if for x X, f H 1 k(, x) H ( generators ), 2 f, k(, x) H = f(x) (reproducing property). Specifically: x, y X, k(x, y) = k (, x),k (, y) H. Questions Uniqueness, existence?
42 Reproducing kernel: uniqueness Reproducibility & norm definition uniqueness. Let k 1, k 2 be r.k.-s of H. Then for f H, x X f, k 1 (, x) k 2 (, x) H, H lin, k i r.k. = f(x) f(x) = 0.
43 Reproducing kernel: uniqueness Reproducibility & norm definition uniqueness. Let k 1, k 2 be r.k.-s of H. Then for f H, x X f, k 1 (, x) k 2 (, x) H, H lin, k i r.k. = f(x) f(x) = 0. Choosing f = k 1 (, x) k 2 (, x), we get i.e., k 1 = k 2. k 1 (, x) k 2 (, x) 2 H = 0, ( x X)
44 View-2 (r.k.) view-1 (RKHS) Result: H has a r.k. (k) H is a RKHS.
45 View-2 (r.k.) view-1 (RKHS) Result: H has a r.k. (k) H is a RKHS. Proof ( ): δ x (f) k(x, x) f H, i.e. δ x : H R is bounded (hence continuous).
46 View-2 (r.k.) view-1 (RKHS) Result: H has a r.k. (k) H is a RKHS. Proof ( ): δ x (f) k: r.k. = f(x) = f, k(, x) H k(x, x) f H, δx def i.e. δ x : H R is bounded (hence continuous).
47 View-2 (r.k.) view-1 (RKHS) Result: H has a r.k. (k) H is a RKHS. Proof ( ): δ x (f) δx def k: r.k. = f(x) = f, k(, x) H CBS k(, x) H f H k: r.k. = k(x, x) f H, i.e. δ x : H R is bounded (hence continuous).
48 View-2 (r.k.) view-1 (RKHS) Result: H has a r.k. (k) H is a RKHS. Proof ( ): δ x (f) δx def k: r.k. = f(x) = f, k(, x) H CBS k(, x) H f H k: r.k. = k(x, x) f H, i.e. δ x : H R is bounded (hence continuous). Convergence in RKHS uniform convergence! (k: bounded).
49 View-2 (r.k.) view-1 (RKHS):, existence of r.k. Proof ( ): Let δ x be continuous for all x X. 1 By the Riesz repr. theorem f δx H δ x (f) = f, f δx }{{} =k(,x)? H, f H.
50 View-2 (r.k.) view-1 (RKHS):, existence of r.k. Proof ( ): Let δ x be continuous for all x X. 1 By the Riesz repr. theorem f δx H δ x (f) = f, f δx }{{} =k(,x)? H, f H. 2 Let k(x, x) = f δx (x ), x, x X, then k(, x) = f δx H, f, k(, x) H = δ x (f) = f(x). Thus, k is the reproducing kernel.
51 View-3: positive definiteness. Let k : X X R be a symmetric function.
52 View-3: positive definiteness. Let k : X X R be a symmetric function. G := [k(x i, x j )] n i,j=1 : Gram matrix.
53 View-3: positive definiteness. Let k : X X R be a symmetric function. G := [k(x i, x j )] n i,j=1 : Gram matrix. k is called positive definite, if a T Ga 0 for n 1, a R n, (x 1,..., x n ) X n.
54 View-4: kernel as inner product view. Def.: A k : X X R function is called kernel, if 1 φ : X F, where F is a Hilbert space s.t. 2 k(x, y) = φ(x),φ(y) F. Intuition: k is inner product in F.
55 Reproducing kernel kernel positive definiteness Every r.k. is a kernel: φ(x) := k(, x), k(x, y) = k(, x), k(, y) H. Every kernel is positive definite: a T Ga = n i=1 j=1 k def,, F lin = F =, F = n a i a j k(x i, x j ) n a i φ(x i ), i=1 n a i φ(x i ) i=1 2 n a j φ(x j ) j=1 F 0. F
56 Until now Result-1 (proved): RKHS (δ x continuous) reproducing kernel. Result-2 (proved): reproducing kernel kernel positive definite.
57 Until now Result-1 (proved): RKHS (δ x continuous) reproducing kernel. Result-2 (proved): reproducing kernel kernel positive definite. Moore-Aronszajn theorem (follows) positive definite reproducing kernel.
58 Until now Result-1 (proved): RKHS (δ x continuous) reproducing kernel. Result-2 (proved): reproducing kernel kernel positive definite. Moore-Aronszajn theorem (follows) positive definite reproducing kernel. the 4 notions are exactly the same!
59 Moore-Aronszajn construction: high-level view Given: a k : X X R positive definite function. We construct a pre-rkhs H 0 : { } n H 0 = f = α i k(, x i ) : α i R, x i X {k(, x) : x X}, i=1 f, g H0 = k(x, y), where f = k(, x), g = k(, y).
60 Moore-Aronszajn construction: high-level view Given: a k : X X R positive definite function. We construct a pre-rkhs H 0 : { } n H 0 = f = α i k(, x i ) : α i R, x i X {k(, x) : x X}, f, g H0 = n i=1 i=1 j=1 m α i β j k(x i, y j ), where f = n i=1 α ik(, x i ), g = m j=1 β jk(, y j ).
61 Moore-Aronszajn construction: high-level view H 0 will satisfy: 0 linear space ( ); f, g H0 : well-defined & inner product. 1 δ x -s are continuous on H 0 ( x). 2 For any {f n } H 0 Cauchy seq.: f n x 0 f n H 0 0. From H 0 we construct H as: 1 H R X, for which 2 {f n } H 0 -Cauchy seq. such that f n x f.
62 Moore-Aronszajn construction: high-level view Let f, g H := lim n f n, g n H0, (1) x x where f n f, g n g H 0 -Cauchy sequences. H will satisfy: H 0 H: [f n f H 0 ]. H is a RKHS with r.k. k: -1 H: linear space ( ), 0 f, g H : well-defined & inner product. 1 H is complete. 2 δ x-s are continuous on H ( x). 3 H has r.k. k (used to define H 0 ).
63 , H0 : well-defined, k reproducing on H 0 Recall: if f = n i=1 α ik(, x i ), g = m j=1 β jk(, y j ), then f, g H0 = n m α i β j k(x i, y j ). i=1 j=1, H0 is independent of the particular {α i } and {β j }: f, g H0 = n m α i i=1 j=1 β j k(x i, y j ) = n m α i g(x i ) = β j f(y j ). i=1 j=1
64 , H0 : well-defined, k reproducing on H 0 Recall: if f = n i=1 α ik(, x i ), g = m j=1 β jk(, y j ), then f, g H0 = n m α i β j k(x i, y j ). i=1 j=1, H0 is independent of the particular {α i } and {β j }: f, g H0 = n m α i i=1 j=1 β j k(x i, y j ) = reproducing property on H 0 : n m α i g(x i ) = β j f(y j ). i=1 j=1 f, k(, x) H0 = n α i k(x i, x) = f(x). i=1
65 , H0 : inner product The tricky property to check: f H0 := f, f H0 = 0 = f = 0. This holds by CBS (for the semi-inner product, H0 ): x f(x) k r.k. on H 0 = CBS f, k(, x) H0 f H0 k(x, x) = 0. }{{} =0
66 Pre-RKHS: main property-1 δ x is continuous on H 0 ( x): Let f, g H 0, then δ x (f) δ x (g) δx def, k r.k.,, H lin = 0 f g, k(, x) H0 CBS, k r.k. k(x, x) f g H0.
67 Pre-RKHS: main property-2 f n : H 0 -Cauchy ( x) H 0 f 0 n 0: f n : Cauchy bounded, i.e. f n H0 < A. f n : Cauchy n, m N 1 : f n f m H0 < ǫ/(2a). Let f N1 = r i=1 α ik(, x i ). n N 2 : f n (x i ) < ǫ 2r α i For n max(n 1, N 2 ): (i = 1,...,r). f n 2 H 0 <ǫ.
68 Pre-RKHS: main property-2 f n : H 0 -Cauchy ( x) H 0 f 0 n 0: f n : Cauchy bounded, i.e. f n H0 < A. f n : Cauchy n, m N 1 : f n f m H0 < ǫ/(2a). Let f N1 = r i=1 α ik(, x i ). n N 2 : f n (x i ) < ǫ 2r α i (i = 1,...,r). For n max(n 1, N 2 ): f n 2 H 0 = f n, f n H0 f n f N1, f n + f H N1, f n 0 H 0 r f n f H0 N1 f n H0 + α i f n (x i ) <ǫ. }{{}}{{} i=1 <[ǫ/(2a)]a= ǫ ǫ < α i 2r α 2 i
69 , H : well-defined α n = f n, g n H0 is convergent by Cauchyness in R: α n α m <ǫ
70 , H : well-defined α n = f n, g n H0 is convergent by Cauchyness in R: α n α m = f n, g n H0 f m, g m H0 = f n, g n H0 f m, g n H0 + f m, g n H0 f m, g m H0 = f n f m, g n H0 + f m, g n g m H0 f n f m, g n H0 + fm, g n g m H0 g n H0 f n f m H0 + f m H0 g n g m H0 <ǫ. }{{}}{{}}{{}}{{} <A < ǫ <B < ǫ 2A 2B f n, g n : Cauchy bounded, i.e. f n H0 < A, g n H0 < B.
71 , H : well-defined The limit is independent of the Cauchy seq. chosen: let f n, f n x f ; g n, g n x g: H 0 -Cauchy seq.-s, α n = f n, g n H0, α n = f n, g n H0.
72 , H : well-defined The limit is independent of the Cauchy seq. chosen: let f n, f n x f ; g n, g n x g: H 0 -Cauchy seq.-s, α n = f n, g n H0, α n = f n, g n H0. Repeating the previous argument: α n α n g n H0 fn f n }{{} H0 + f H0 n gn g n H0. }{{}}{{}}{{} bounded 0 0 bounded
73 , H : well-defined The limit is independent of the Cauchy seq. chosen: let f n, f n x f ; g n, g n x g: H 0 -Cauchy seq.-s, α n = f n, g n H0, α n = f n, g n H0. Repeating the previous argument: α n α n g n H0 fn f n }{{} H0 + f H0 n gn g n H0. }{{}}{{}}{{} bounded : f n, f n similarly). x f f n f n x 0 f n f n bounded H 0 0 (g n g n
74 , H : inner product The tricky bit: f, f H = 0 f = 0. Let f n x f H 0 -Cauchy, and f, f H = lim n f n 2 H 0 = 0. Then f(x) = lim f n (x) = lim δ x (f n ) ( ) lim δ x f n n n }{{} n H0 = 0, }{{} < 0 ( ): δ x is continuous on H 0.
75 Until now:, H is well-defined & inner product. Remains: 1 δ x -s are continuous on H ( x). 2 H is complete. 3 The reproducing kernel on H is k.
76 δ x -s are continuous on H: lemma H 0 is dense in H. Sufficient to show: f n x f H 0 -Cauchy f n H f.
77 δ x -s are continuous on H: lemma H 0 is dense in H. Sufficient to show: f n x f H 0 -Cauchy f n H f. Proof: Fix ǫ > 0, f n : H 0 -Cauchy N m, n: f m f n H0 < ǫ. Fix n N, then f m f n x f f n. By the definition of H : i.e., f n H f. f f n 2 H = lim m f m f n 2 H 0 ǫ 2,
78 δ x -s are continuous on H Sufficient to show: δ x linear is continuous at f 0. Fix x X. We have seen: δ x is continuous on H 0, i.e. η g 0 H0 = g H0 < η δ x (g) δ x (0) = δ x (g) 0 = g(x) < ǫ/2.
79 δ x -s are continuous on H Sufficient to show: δ x linear is continuous at f 0. Fix x X. We have seen: δ x is continuous on H 0, i.e. η g 0 H0 = g H0 < η δ x (g) δ x (0) = δ x (g) 0 = g(x) < ǫ/2. Take f H: f H < η/2. Since H 0 H dense, f n H 0 -Cauchy, N f(x) f N (x) < ǫ/2 f f N H < η/2 [ f n x f], [ f n H f] f N H0 = f N H f }{{} H + f f N H < η. }{{} < η < η 2 2
80 δ x -s are continuous on H Sufficient to show: δ x linear is continuous at f 0. Fix x X. We have seen: δ x is continuous on H 0, i.e. η g 0 H0 = g H0 < η δ x (g) δ x (0) = δ x (g) 0 = g(x) < ǫ/2. Take f H: f H < η/2. Since H 0 H dense, f n H 0 -Cauchy, N f(x) f N (x) < ǫ/2 f f N H < η/2 [ f n x f], [ f n H f] f N H0 = f N H f }{{} H + f f N H < η. }{{} < η < η 2 2 With g = f N we get f N (x) < ǫ 2 f(x) f(x) f }{{ N(x) + f } N (x) < ǫ. }{{} < ǫ < ǫ 2 2
81 H is complete High-level idea: let {f n } H be any Cauchy seq., f(x) := lim n f n (x) since δ x cont. on H {f n (x)} R Cauchy seq. convergent.
82 H is complete High-level idea: let {f n } H be any Cauchy seq., f(x) := lim n f n (x) since δ x cont. on H {f n (x)} R Cauchy seq. convergent. Question: is the point-wise limit f H?
83 H is complete High-level idea: let {f n } H be any Cauchy seq., f(x) := lim n f n (x) since δ x cont. on H {f n (x)} R Cauchy seq. convergent. Question: is the point-wise limit f H? Idea: 1 H 0 dense in H g n H 0 s.t. g n f n H < 1 n. 2 We show x f; {g n} H 0 : Cauchy seq.} f H. g n f n H f.
84 g n x f : g n (x) f(x) g n (x) f n (x) + f n (x) f(x) = δ x (g n f n ) + f }{{} n (x) f(x). }{{} 0; δ x cont. on H 0; f def.
85 {g n } H 0 is Cauchy sequence: g m g n H0 = g m g n H g m f m H + f m f n H + f n g n H 1 m f m f n }{{ n H. }}{{} 0;f g m,g n def. n:h-cauchy
86 Finally, f n H f : f f n H f g n H + g n f n H g n def. {}}{{}}{ 1 f g n H + n. 0: shown at H 0 dense in H
87 Final property: the reproducing kernel on H is k Let f H, and f n x f H 0 -Cauchy sequence. Then, (a) (b) f, k(, x) H = lim f n, k(, x) n H0 = lim f n (x) (c) = f(x), n where (a): definition of, H, (b): k reproducing kernel on H 0, x (c): f n f.
88 Summary We have shown that RKHS (δ x continuous) reproducing kernel kernel (feature view) positive definite. Moore-Aronszajn theorem: RKHS construction for a k pos. def. function. Idea: 1 pre-rkhs: H 0 = span[{k(, x)} x X], 2 H := pointwise limit of H 0 -Cauchy sequences.
89 Thank you for the attention!
90 Vector space axioms (V,+,λ ) is vector space if [ v 1, v 2, v 3, v V, a, b R]: (v 1 + v 2 )+v 3 = v 1 +(v 2 + v 3 ), (associativity) v 1 + v 2 = v 2 + v 1, (commutativity) 0 : v+0 = v, v : v+( v) = 0, a(bv) = (ab)v, 1v = v, a(v 1 + v 2 ) = av 1 + av 2, (a+b)v = av+bv.
91 H is a vector space H R X Needed: x 1 f H λf H: {f n } H 0 -Cauchy, f n f. {λf n } H 0 ( H 0 : vector space), Cauchy, (λf n )(x) x (λf)(x).
92 H is a vector space H R X Needed: x 1 f H λf H: {f n } H 0 -Cauchy, f n f. {λf n } H 0 ( H 0 : vector space), Cauchy, (λf n )(x) x (λf)(x). 2 f, g H f + g H: {f n },{g n } H 0 -Cauchy, f n x f, g n x g {f n + g n } H 0 ( H 0 : vector space), Cauchy, (f n + g n )(x) x (f + g)(x).
93 , H0 : inner product Needed: for f = i α ik(, x i ), g = j β jk(, y j ) H 0 1 f, g H0 = g, f H0 : f, g H0 = α i β j k(x i, y j ) = β j α i k(y j, x i ) = g, f H0. i j j i
94 , H0 : inner product Needed: for λ R, f = i α ik(, x i ), g = j β jk(, y j ) H 0 1 f, g H0 = g, f H0 : f, g H0 = α i β j k(x i, y j ) = β j α i k(y j, x i ) = g, f H0. i j j i 2 λf, g H0 = λ f, g H0 : λf, g H0 = i,j (λα i )β j k(x i, y j ) = λ i,j α i β j k(x i, y j ) = λ f, g H0.
95 , H0 : inner product Needed: for λ R, f = i α ik(, x i ), g = j β jk(, y j ) H 0 1 f, g H0 = g, f H0 : f, g H0 = α i β j k(x i, y j ) = β j α i k(y j, x i ) = g, f H0. i j j i 2 λf, g H0 = λ f, g H0 : λf, g H0 = i,j (λα i )β j k(x i, y j ) = λ i,j α i β j k(x i, y j ) = λ f, g H0. 3 f 1 + f 2, g H0 = f 1, g H0 + f 2, g H0 [f 1 α i, x i, f 2 α i, x i ]: l.h.s = i,j α i β j k(x i, y j ) = i,j where f 1 + f 2 α i,α i, x i, x i. α i β jk(x i, y j)+ i,j α i β jk(x i, y j ) = r.h.s.,
96 , H0 : inner product Needed: for λ R, f = i α ik(, x i ), g = j β jk(, y j ) H 0 1 f, g H0 = g, f H0 : f, g H0 = α i β j k(x i, y j ) = β j α i k(y j, x i ) = g, f H0. i j j i 2 λf, g H0 = λ f, g H0 : λf, g H0 = i,j (λα i )β j k(x i, y j ) = λ i,j α i β j k(x i, y j ) = λ f, g H0. 3 f 1 + f 2, g H0 = f 1, g H0 + f 2, g H0 [f 1 α i, x i, f 2 α i, x i ]: l.h.s = i,j α i β j k(x i, y j ) = i,j where f 1 + f 2 α i,α 4 f = 0 f, f H0 = 0: i, x i, x i. α i β jk(x i, y j)+ i,j f = 0 k(, x) f, f H0 = 0 0 k(x, x) = 0. α i β jk(x i, y j ) = r.h.s.,
97 , H : semi inner product Needed: for f, f 1, f 2, g H 1 f, g H = g, f H : f, g H = lim n f n, g n H0 H 0 : = lim n g n, f n H0 = g, f H.
98 , H : semi inner product Needed: for f, f 1, f 2, g H,λ R 1 f, g H = g, f H : 2 λf, g H = λ f, g H : f, g H = lim n f n, g n H0 H 0 : = lim n g n, f n H0 = g, f H. λf, g H = lim n λf n, g n H0 H 0 : = lim n λ f n, g n H0 = λ lim n f n, g n H0 = λ f, g H.
99 , H : semi inner product Needed: for f, f 1, f 2, g H,λ R 1 f, g H = g, f H : 2 λf, g H = λ f, g H : f, g H = lim n f n, g n H0 H 0 : = lim n g n, f n H0 = g, f H. λf, g H = lim n λf n, g n H0 H 0 : = lim n λ f n, g n H0 = λ lim n f n, g n H0 = λ f, g H. 3 f 1 + f 2, g H = f 1, g H + f 2, g H : f 1 + f 2, g H = lim n f 1,n + f 2,n, g H0 H 0 : = lim n [ f 1,n, g H0 + f 2,n, g H0 ] = lim n f 1,n, g H0 + lim n f 2,n, g H0 = f 1, g H + f 2, g H.
100 , H : semi inner product Needed: for f, f 1, f 2, g H,λ R 1 f, g H = g, f H : 2 λf, g H = λ f, g H : f, g H = lim n f n, g n H0 H 0 : = lim n g n, f n H0 = g, f H. λf, g H = lim n λf n, g n H0 H 0 : = lim n λ f n, g n H0 = λ lim n f n, g n H0 = λ f, g H. 3 f 1 + f 2, g H = f 1, g H + f 2, g H : f 1 + f 2, g H = lim n f 1,n + f 2,n, g H0 H 0 : = lim n [ f 1,n, g H0 + f 2,n, g H0 ] 4 f = 0 f, f H = 0: Let f n 0 = lim n f 1,n, g H0 + lim n f 2,n, g H0 = f 1, g H + f 2, g H. f, f H = lim n 0, 0 H0 H 0 : = lim n 0 = 0.
Metric Spaces. Chapter 7. 7.1. Metrics
Chapter 7 Metric Spaces A metric space is a set X that has a notion of the distance d(x, y) between every pair of points x, y X. The purpose of this chapter is to introduce metric spaces and give some
BANACH AND HILBERT SPACE REVIEW
BANACH AND HILBET SPACE EVIEW CHISTOPHE HEIL These notes will briefly review some basic concepts related to the theory of Banach and Hilbert spaces. We are not trying to give a complete development, but
Chapter 5. Banach Spaces
9 Chapter 5 Banach Spaces Many linear equations may be formulated in terms of a suitable linear operator acting on a Banach space. In this chapter, we study Banach spaces and linear operators acting on
1 Norms and Vector Spaces
008.10.07.01 1 Norms and Vector Spaces Suppose we have a complex vector space V. A norm is a function f : V R which satisfies (i) f(x) 0 for all x V (ii) f(x + y) f(x) + f(y) for all x,y V (iii) f(λx)
MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets.
MATH 304 Linear Algebra Lecture 20: Inner product spaces. Orthogonal sets. Norm The notion of norm generalizes the notion of length of a vector in R n. Definition. Let V be a vector space. A function α
THE BANACH CONTRACTION PRINCIPLE. Contents
THE BANACH CONTRACTION PRINCIPLE ALEX PONIECKI Abstract. This paper will study contractions of metric spaces. To do this, we will mainly use tools from topology. We will give some examples of contractions,
Metric Spaces. Chapter 1
Chapter 1 Metric Spaces Many of the arguments you have seen in several variable calculus are almost identical to the corresponding arguments in one variable calculus, especially arguments concerning convergence
Inner Product Spaces
Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and
1. Let P be the space of all polynomials (of one real variable and with real coefficients) with the norm
Uppsala Universitet Matematiska Institutionen Andreas Strömbergsson Prov i matematik Funktionalanalys Kurs: F3B, F4Sy, NVP 005-06-15 Skrivtid: 9 14 Tillåtna hjälpmedel: Manuella skrivdon, Kreyszigs bok
University of Ostrava. Fuzzy Transforms
University of Ostrava Institute for Research and Applications of Fuzzy Modeling Fuzzy Transforms Irina Perfilieva Research report No. 58 2004 Submitted/to appear: Fuzzy Sets and Systems Supported by: Grant
1. Let X and Y be normed spaces and let T B(X, Y ).
Uppsala Universitet Matematiska Institutionen Andreas Strömbergsson Prov i matematik Funktionalanalys Kurs: NVP, Frist. 2005-03-14 Skrivtid: 9 11.30 Tillåtna hjälpmedel: Manuella skrivdon, Kreyszigs bok
FUNCTIONAL ANALYSIS LECTURE NOTES CHAPTER 2. OPERATORS ON HILBERT SPACES
FUNCTIONAL ANALYSIS LECTURE NOTES CHAPTER 2. OPERATORS ON HILBERT SPACES CHRISTOPHER HEIL 1. Elementary Properties and Examples First recall the basic definitions regarding operators. Definition 1.1 (Continuous
FUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES
FUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES CHRISTOPHER HEIL 1. Cosets and the Quotient Space Any vector space is an abelian group under the operation of vector addition. So, if you are have studied
3. INNER PRODUCT SPACES
. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.
0 <β 1 let u(x) u(y) kuk u := sup u(x) and [u] β := sup
![0 <β 1 let u(x) u(y) kuk u := sup u(x) and [u] β := sup](/thumbs/40/20844595.jpg "0 <β 1 let u(x) u(y) kuk u := sup u(x) and [u] β := sup")
456 BRUCE K. DRIVER 24. Hölder Spaces Notation 24.1. Let Ω be an open subset of R d,bc(ω) and BC( Ω) be the bounded continuous functions on Ω and Ω respectively. By identifying f BC( Ω) with f Ω BC(Ω),
Notes on metric spaces
Notes on metric spaces 1 Introduction The purpose of these notes is to quickly review some of the basic concepts from Real Analysis, Metric Spaces and some related results that will be used in this course.
Section 6.1 - Inner Products and Norms
Section 6.1 - Inner Products and Norms Definition. Let V be a vector space over F {R, C}. An inner product on V is a function that assigns, to every ordered pair of vectors x and y in V, a scalar in F,
MA651 Topology. Lecture 6. Separation Axioms.
MA651 Topology. Lecture 6. Separation Axioms. This text is based on the following books: Fundamental concepts of topology by Peter O Neil Elements of Mathematics: General Topology by Nicolas Bourbaki Counterexamples
n k=1 k=0 1/k! = e. Example 6.4. The series 1/k 2 converges in R. Indeed, if s n = n then k=1 1/k, then s 2n s n = 1 n + 1 +...
6 Series We call a normed space (X, ) a Banach space provided that every Cauchy sequence (x n ) in X converges. For example, R with the norm = is an example of Banach space. Now let (x n ) be a sequence
16.3 Fredholm Operators
Lectures 16 and 17 16.3 Fredholm Operators A nice way to think about compact operators is to show that set of compact operators is the closure of the set of finite rank operator in operator norm. In this
Tensor product of vector spaces
Tensor product of vector spaces Construction Let V,W be vector spaces over K = R or C. Let F denote the vector space freely generated by the set V W and let N F denote the subspace spanned by the elements
Metric Spaces Joseph Muscat 2003 (Last revised May 2009)
1 Distance J Muscat 1 Metric Spaces Joseph Muscat 2003 (Last revised May 2009) (A revised and expanded version of these notes are now published by Springer.) 1 Distance A metric space can be thought of
NOTES ON LINEAR TRANSFORMATIONS
NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all
Inner product. Definition of inner product
Math 20F Linear Algebra Lecture 25 1 Inner product Review: Definition of inner product. Slide 1 Norm and distance. Orthogonal vectors. Orthogonal complement. Orthogonal basis. Definition of inner product
Universal Algorithm for Trading in Stock Market Based on the Method of Calibration
Universal Algorithm for Trading in Stock Market Based on the Method of Calibration Vladimir V yugin Institute for Information Transmission Problems, Russian Academy of Sciences, Bol shoi Karetnyi per.
Fixed Point Theorems
Fixed Point Theorems Definition: Let X be a set and let T : X X be a function that maps X into itself. (Such a function is often called an operator, a transformation, or a transform on X, and the notation
Mathematics Course 111: Algebra I Part IV: Vector Spaces
Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are
1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
MATH 4330/5330, Fourier Analysis Section 11, The Discrete Fourier Transform
MATH 433/533, Fourier Analysis Section 11, The Discrete Fourier Transform Now, instead of considering functions defined on a continuous domain, like the interval [, 1) or the whole real line R, we wish
Mathematical Methods of Engineering Analysis
Mathematical Methods of Engineering Analysis Erhan Çinlar Robert J. Vanderbei February 2, 2000 Contents Sets and Functions 1 1 Sets................................... 1 Subsets.............................
Vector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty
Properties of BMO functions whose reciprocals are also BMO
Properties of BMO functions whose reciprocals are also BMO R. L. Johnson and C. J. Neugebauer The main result says that a non-negative BMO-function w, whose reciprocal is also in BMO, belongs to p> A p,and
Rate of convergence towards Hartree dynamics
Rate of convergence towards Hartree dynamics Benjamin Schlein, LMU München and University of Cambridge Universitá di Milano Bicocca, October 22, 2007 Joint work with I. Rodnianski 1. Introduction boson
Convex analysis and profit/cost/support functions
CALIFORNIA INSTITUTE OF TECHNOLOGY Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m
x a x 2 (1 + x 2 ) n.
Limits and continuity Suppose that we have a function f : R R. Let a R. We say that f(x) tends to the limit l as x tends to a; lim f(x) = l ; x a if, given any real number ɛ > 0, there exists a real number
Let H and J be as in the above lemma. The result of the lemma shows that the integral
Let and be as in the above lemma. The result of the lemma shows that the integral ( f(x, y)dy) dx is well defined; we denote it by f(x, y)dydx. By symmetry, also the integral ( f(x, y)dx) dy is well defined;
Differential Operators and their Adjoint Operators
Differential Operators and their Adjoint Operators Differential Operators inear functions from E n to E m may be described, once bases have been selected in both spaces ordinarily one uses the standard
Linear Algebra Notes for Marsden and Tromba Vector Calculus
Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of
Inner products on R n, and more
Inner products on R n, and more Peyam Ryan Tabrizian Friday, April 12th, 2013 1 Introduction You might be wondering: Are there inner products on R n that are not the usual dot product x y = x 1 y 1 + +
Numerical Analysis Lecture Notes
Numerical Analysis Lecture Notes Peter J. Olver 5. Inner Products and Norms The norm of a vector is a measure of its size. Besides the familiar Euclidean norm based on the dot product, there are a number
17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function
17. Inner product spaces Definition 17.1. Let V be a real vector space. An inner product on V is a function, : V V R, which is symmetric, that is u, v = v, u. bilinear, that is linear (in both factors):
Bindel, Spring 2012 Intro to Scientific Computing (CS 3220) Week 3: Wednesday, Feb 8
Spaces and bases Week 3: Wednesday, Feb 8 I have two favorite vector spaces 1 : R n and the space P d of polynomials of degree at most d. For R n, we have a canonical basis: R n = span{e 1, e 2,..., e
T ( a i x i ) = a i T (x i ).
Chapter 2 Defn 1. (p. 65) Let V and W be vector spaces (over F ). We call a function T : V W a linear transformation form V to W if, for all x, y V and c F, we have (a) T (x + y) = T (x) + T (y) and (b)
UNIVERSITETET I OSLO
NIVERSITETET I OSLO Det matematisk-naturvitenskapelige fakultet Examination in: Trial exam Partial differential equations and Sobolev spaces I. Day of examination: November 18. 2009. Examination hours:
Undergraduate Notes in Mathematics. Arkansas Tech University Department of Mathematics
Undergraduate Notes in Mathematics Arkansas Tech University Department of Mathematics An Introductory Single Variable Real Analysis: A Learning Approach through Problem Solving Marcel B. Finan c All Rights
CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e.
CHAPTER II THE LIMIT OF A SEQUENCE OF NUMBERS DEFINITION OF THE NUMBER e. This chapter contains the beginnings of the most important, and probably the most subtle, notion in mathematical analysis, i.e.,
INCIDENCE-BETWEENNESS GEOMETRY
INCIDENCE-BETWEENNESS GEOMETRY MATH 410, CSUSM. SPRING 2008. PROFESSOR AITKEN This document covers the geometry that can be developed with just the axioms related to incidence and betweenness. The full
Separation Properties for Locally Convex Cones
Journal of Convex Analysis Volume 9 (2002), No. 1, 301 307 Separation Properties for Locally Convex Cones Walter Roth Department of Mathematics, Universiti Brunei Darussalam, Gadong BE1410, Brunei Darussalam
Continuity of the Perron Root
Linear and Multilinear Algebra http://dx.doi.org/10.1080/03081087.2014.934233 ArXiv: 1407.7564 (http://arxiv.org/abs/1407.7564) Continuity of the Perron Root Carl D. Meyer Department of Mathematics, North
MEASURE AND INTEGRATION. Dietmar A. Salamon ETH Zürich
MEASURE AND INTEGRATION Dietmar A. Salamon ETH Zürich 12 May 2016 ii Preface This book is based on notes for the lecture course Measure and Integration held at ETH Zürich in the spring semester 2014. Prerequisites
Math 4310 Handout - Quotient Vector Spaces
Math 4310 Handout - Quotient Vector Spaces Dan Collins The textbook defines a subspace of a vector space in Chapter 4, but it avoids ever discussing the notion of a quotient space. This is understandable
Practice with Proofs
Practice with Proofs October 6, 2014 Recall the following Definition 0.1. A function f is increasing if for every x, y in the domain of f, x < y = f(x) < f(y) 1. Prove that h(x) = x 3 is increasing, using
Chapter 7. Continuity
Chapter 7 Continuity There are many processes and eects that depends on certain set of variables in such a way that a small change in these variables acts as small change in the process. Changes of this
ON SEQUENTIAL CONTINUITY OF COMPOSITION MAPPING. 0. Introduction
ON SEQUENTIAL CONTINUITY OF COMPOSITION MAPPING Abstract. In [1] there was proved a theorem concerning the continuity of the composition mapping, and there was announced a theorem on sequential continuity
1. Prove that the empty set is a subset of every set.
1. Prove that the empty set is a subset of every set. Basic Topology Written by Men-Gen Tsai email: b89902089@ntu.edu.tw Proof: For any element x of the empty set, x is also an element of every set since
Linear Algebra I. Ronald van Luijk, 2012
Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.
Metric Spaces. Lecture Notes and Exercises, Fall 2015. M.van den Berg
Metric Spaces Lecture Notes and Exercises, Fall 2015 M.van den Berg School of Mathematics University of Bristol BS8 1TW Bristol, UK mamvdb@bristol.ac.uk 1 Definition of a metric space. Let X be a set,
Matrix Representations of Linear Transformations and Changes of Coordinates
Matrix Representations of Linear Transformations and Changes of Coordinates 01 Subspaces and Bases 011 Definitions A subspace V of R n is a subset of R n that contains the zero element and is closed under
HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!
Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following
Inner Product Spaces. 7.1 Inner Products
7 Inner Product Spaces 71 Inner Products Recall that if z is a complex number, then z denotes the conjugate of z, Re(z) denotes the real part of z, and Im(z) denotes the imaginary part of z By definition,
Regression With Gaussian Measures
Regression With Gaussian Measures Michael J. Meyer Copyright c April 11, 2004 ii PREFACE We treat the basics of Gaussian processes, Gaussian measures, kernel reproducing Hilbert spaces and related topics.
PUTNAM TRAINING POLYNOMIALS. Exercises 1. Find a polynomial with integral coefficients whose zeros include 2 + 5.
PUTNAM TRAINING POLYNOMIALS (Last updated: November 17, 2015) Remark. This is a list of exercises on polynomials. Miguel A. Lerma Exercises 1. Find a polynomial with integral coefficients whose zeros include
MATH 425, PRACTICE FINAL EXAM SOLUTIONS.
MATH 45, PRACTICE FINAL EXAM SOLUTIONS. Exercise. a Is the operator L defined on smooth functions of x, y by L u := u xx + cosu linear? b Does the answer change if we replace the operator L by the operator
INTRODUCTORY SET THEORY
M.Sc. program in mathematics INTRODUCTORY SET THEORY Katalin Károlyi Department of Applied Analysis, Eötvös Loránd University H-1088 Budapest, Múzeum krt. 6-8. CONTENTS 1. SETS Set, equal sets, subset,
it is easy to see that α = a
21. Polynomial rings Let us now turn out attention to determining the prime elements of a polynomial ring, where the coefficient ring is a field. We already know that such a polynomial ring is a UF. Therefore
SOLUTIONS TO EXERCISES FOR. MATHEMATICS 205A Part 3. Spaces with special properties
SOLUTIONS TO EXERCISES FOR MATHEMATICS 205A Part 3 Fall 2008 III. Spaces with special properties III.1 : Compact spaces I Problems from Munkres, 26, pp. 170 172 3. Show that a finite union of compact subspaces
Finite dimensional topological vector spaces
Chapter 3 Finite dimensional topological vector spaces 3.1 Finite dimensional Hausdorff t.v.s. Let X be a vector space over the field K of real or complex numbers. We know from linear algebra that the
Introduction to Topology
Introduction to Topology Tomoo Matsumura November 30, 2010 Contents 1 Topological spaces 3 1.1 Basis of a Topology......................................... 3 1.2 Comparing Topologies.......................................
Adaptive Search with Stochastic Acceptance Probabilities for Global Optimization
Adaptive Search with Stochastic Acceptance Probabilities for Global Optimization Archis Ghate a and Robert L. Smith b a Industrial Engineering, University of Washington, Box 352650, Seattle, Washington,
THE KADISON-SINGER PROBLEM IN MATHEMATICS AND ENGINEERING: A DETAILED ACCOUNT
THE ADISON-SINGER PROBLEM IN MATHEMATICS AND ENGINEERING: A DETAILED ACCOUNT PETER G. CASAZZA, MATTHEW FICUS, JANET C. TREMAIN, ERIC WEBER Abstract. We will show that the famous, intractible 1959 adison-singer
Advanced Microeconomics
Advanced Microeconomics Ordinal preference theory Harald Wiese University of Leipzig Harald Wiese (University of Leipzig) Advanced Microeconomics 1 / 68 Part A. Basic decision and preference theory 1 Decisions
CONTINUOUS REINHARDT DOMAINS PROBLEMS ON PARTIAL JB*-TRIPLES
CONTINUOUS REINHARDT DOMAINS PROBLEMS ON PARTIAL JB*-TRIPLES László STACHÓ Bolyai Institute Szeged, Hungary stacho@math.u-szeged.hu www.math.u-szeged.hu/ Stacho 13/11/2008, Granada László STACHÓ () CONTINUOUS
Nonlinear Algebraic Equations. Lectures INF2320 p. 1/88
Nonlinear Algebraic Equations Lectures INF2320 p. 1/88 Lectures INF2320 p. 2/88 Nonlinear algebraic equations When solving the system u (t) = g(u), u(0) = u 0, (1) with an implicit Euler scheme we have
I. GROUPS: BASIC DEFINITIONS AND EXAMPLES
I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called
9 More on differentiation
Tel Aviv University, 2013 Measure and category 75 9 More on differentiation 9a Finite Taylor expansion............... 75 9b Continuous and nowhere differentiable..... 78 9c Differentiable and nowhere monotone......
INTRODUCTION TO DESCRIPTIVE SET THEORY
INTRODUCTION TO DESCRIPTIVE SET THEORY ANUSH TSERUNYAN Mathematicians in the early 20 th century discovered that the Axiom of Choice implied the existence of pathological subsets of the real line lacking
CHAPTER IV - BROWNIAN MOTION
CHAPTER IV - BROWNIAN MOTION JOSEPH G. CONLON 1. Construction of Brownian Motion There are two ways in which the idea of a Markov chain on a discrete state space can be generalized: (1) The discrete time
and s n (x) f(x) for all x and s.t. s n is measurable if f is. REAL ANALYSIS Measures. A (positive) measure on a measurable space
RAL ANALYSIS A survey of MA 641-643, UAB 1999-2000 M. Griesemer Throughout these notes m denotes Lebesgue measure. 1. Abstract Integration σ-algebras. A σ-algebra in X is a non-empty collection of subsets
We shall turn our attention to solving linear systems of equations. Ax = b
59 Linear Algebra We shall turn our attention to solving linear systems of equations Ax = b where A R m n, x R n, and b R m. We already saw examples of methods that required the solution of a linear system
DISINTEGRATION OF MEASURES
DISINTEGRTION OF MESURES BEN HES Definition 1. Let (, M, λ), (, N, µ) be sigma-finite measure spaces and let T : be a measurable map. (T, µ)-disintegration is a collection {λ y } y of measures on M such
Mathematics for Econometrics, Fourth Edition
Mathematics for Econometrics, Fourth Edition Phoebus J. Dhrymes 1 July 2012 1 c Phoebus J. Dhrymes, 2012. Preliminary material; not to be cited or disseminated without the author s permission. 2 Contents
Reference: Introduction to Partial Differential Equations by G. Folland, 1995, Chap. 3.
5 Potential Theory Reference: Introduction to Partial Differential Equations by G. Folland, 995, Chap. 3. 5. Problems of Interest. In what follows, we consider Ω an open, bounded subset of R n with C 2
Notes on Symmetric Matrices
CPSC 536N: Randomized Algorithms 2011-12 Term 2 Notes on Symmetric Matrices Prof. Nick Harvey University of British Columbia 1 Symmetric Matrices We review some basic results concerning symmetric matrices.
MATH PROBLEMS, WITH SOLUTIONS
MATH PROBLEMS, WITH SOLUTIONS OVIDIU MUNTEANU These are free online notes that I wrote to assist students that wish to test their math skills with some problems that go beyond the usual curriculum. These
Random graphs with a given degree sequence
Sourav Chatterjee (NYU) Persi Diaconis (Stanford) Allan Sly (Microsoft) Let G be an undirected simple graph on n vertices. Let d 1,..., d n be the degrees of the vertices of G arranged in descending order.
ON SUPERCYCLICITY CRITERIA. Nuha H. Hamada Business Administration College Al Ain University of Science and Technology 5-th st, Abu Dhabi, 112612, UAE
International Journal of Pure and Applied Mathematics Volume 101 No. 3 2015, 401-405 ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu doi: http://dx.doi.org/10.12732/ijpam.v101i3.7
Solutions of Equations in One Variable. Fixed-Point Iteration II
Solutions of Equations in One Variable Fixed-Point Iteration II Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011
Taylor and Maclaurin Series
Taylor and Maclaurin Series In the preceding section we were able to find power series representations for a certain restricted class of functions. Here we investigate more general problems: Which functions
v w is orthogonal to both v and w. the three vectors v, w and v w form a right-handed set of vectors.
3. Cross product Definition 3.1. Let v and w be two vectors in R 3. The cross product of v and w, denoted v w, is the vector defined as follows: the length of v w is the area of the parallelogram with
INTEGRAL OPERATORS ON THE PRODUCT OF C(K) SPACES
INTEGRAL OPERATORS ON THE PRODUCT OF C(K) SPACES FERNANDO BOMBAL AND IGNACIO VILLANUEVA Abstract. We study and characterize the integral multilinear operators on a product of C(K) spaces in terms of the
Date: April 12, 2001. Contents
2 Lagrange Multipliers Date: April 12, 2001 Contents 2.1. Introduction to Lagrange Multipliers......... p. 2 2.2. Enhanced Fritz John Optimality Conditions...... p. 12 2.3. Informative Lagrange Multipliers...........
Fixed Point Theorems For Set-Valued Maps
Fixed Point Theorems For Set-Valued Maps Bachelor s thesis in functional analysis Institute for Analysis and Scientific Computing Vienna University of Technology Andreas Widder, July 2009. Preface In this
Lecture Notes on Measure Theory and Functional Analysis
Lecture Notes on Measure Theory and Functional Analysis P. Cannarsa & T. D Aprile Dipartimento di Matematica Università di Roma Tor Vergata cannarsa@mat.uniroma2.it daprile@mat.uniroma2.it aa 2006/07 Contents
MA106 Linear Algebra lecture notes
MA106 Linear Algebra lecture notes Lecturers: Martin Bright and Daan Krammer Warwick, January 2011 Contents 1 Number systems and fields 3 1.1 Axioms for number systems......................... 3 2 Vector
(Quasi-)Newton methods
(Quasi-)Newton methods 1 Introduction 1.1 Newton method Newton method is a method to find the zeros of a differentiable non-linear function g, x such that g(x) = 0, where g : R n R n. Given a starting
GI01/M055 Supervised Learning Proximal Methods
GI01/M055 Supervised Learning Proximal Methods Massimiliano Pontil (based on notes by Luca Baldassarre) (UCL) Proximal Methods 1 / 20 Today s Plan Problem setting Convex analysis concepts Proximal operators
Extremal equilibria for reaction diffusion equations in bounded domains and applications.
Extremal equilibria for reaction diffusion equations in bounded domains and applications. Aníbal Rodríguez-Bernal Alejandro Vidal-López Departamento de Matemática Aplicada Universidad Complutense de Madrid,
I. Pointwise convergence
MATH 40 - NOTES Sequences of functions Pointwise and Uniform Convergence Fall 2005 Previously, we have studied sequences of real numbers. Now we discuss the topic of sequences of real valued functions.
THE CENTRAL LIMIT THEOREM TORONTO
THE CENTRAL LIMIT THEOREM DANIEL RÜDT UNIVERSITY OF TORONTO MARCH, 2010 Contents 1 Introduction 1 2 Mathematical Background 3 3 The Central Limit Theorem 4 4 Examples 4 4.1 Roulette......................................