Optimization of the HOTS score of a website s pages

Transcription

1 Optimization of the HOTS score of a website s pages Olivier Fercoq and Stéphane Gaubert INRIA Saclay and CMAP Ecole Polytechnique June 21st, 2012

2 Toy example with 21 pages Nodes = web pages Arcs = hyperlinks 21 : controlled page : non controlled page

3 Context A webmaster controls a given number of pages: May add links Must respect the content Wishes to maximize: - Sum of PageRank values of the site - HITS authority score of the home page - Sum of HOTS score values of the site PageRank: Brin and Page, 1998 HITS: Kleinberg, 1998 HOTS: Tomlin, 2003

4 Tomlin s HOTS algorithm: irreducible case A: adjacency matrix of the web graph (irreducible) ρ: web traffic max ρ 0 i,j [n] ρ i,j (log( ρ i,j A i,j ) 1) j [n] ρ i,j = j [n] ρ j,i, i [n] (p i ) i,j [n] ρ ij = 1 (µ) Optimal ρ while PageRank gives a specific ρ (uniform probability is arbitrary)

5 Minimize: Dual problem: irreducible case θ(p, µ) := i,j [n] θ is convex and differentiable. A ij e p i p j +µ µ. θ (p, µ) = 0 µ = log( A ij e p i p j ) µ i,j [n] θ (p, µ) = 0 p is a fixed point of f, where p f i (x) = 1 2 log(at e x ) i 1 2 log(ae x ) i g i (d) = ( (A T d) i (Ad 1 ) i ) 1/2 e p i is interpreted as the temperature of page i

6 The matrix balancing problem Given a n n matrix A, find a diagonal positive matrix D such that X = DAD 1 verifies i [n] j [n] X i,j = k [n] X k,i Proposition (Eaves, Hoffman, Rothblum, H Schneider, 1985) There exists v R n such that f (v) = v and i [n] v i = 0 if and only if A has a diagonal similarity scaling if and only if A is completely reducible. If in addition A is irreducible, then v is unique. v minimizes θ 0 (p) = θ(p, 0)

7 Algorithms for matrix balancing Convex optimization algorithms Coordinate descent: (M Schneider and Zenios, 1989) select a coordinate i and set ( (A T d) i d i (Ad 1 ) i ) 1 2 DomEig (Johnson, Pitkin, Stanford, 2000) { min λ max (A + diag(v)) } v i = 0 = min θ 0(p) p R n i [n]

8 Fixed point approach for matrix balancing Let f and g be the functions defined by f i (x) = 1 2 log(at e x ) i 1 2 log(ae x ) i ( ) (A T 1/2 d) i g i (d) = (Ad 1 ) i The solution of the matrix balancing problem verifies x = f (x) or d = g(d) Ideal HOTS algorithm (irreducible case): x k+1 = f (x k ) Does it converge? (Knight, 2008: not proved)

9 Convergence of the fixed point scheme The fixed point operator Theorem f (x) = 1 2 (log(at e x ) log(ae x )) is monotone, additively homogeneous. If A is irreducible and A + A T is primitive, then x R n, ( lim sup f k (x) v ) 1 k λ 2 (P) k P = 1 ( diag(a T e v ) 1 A T diag(e v ) + diag(ae v ) 1 Adiag(e v ) ) 2 Proof Nonlinear Perron-Frobenius theory (Nussbaum and followers)

10 Comparison of algorithms [ ] ɛ 1 CMAP NZ Uni A = 2 0 1,500 p 413,639 p λ 2 (P) Matlab s fminunc s 948 s Out of memory DomEig 0.5 s > 600 s > 600 s Coordinate descent s 0.03 s 6.06 s Fixed point (HOTS) s 0.02 s 7.52 s A is nearly imprimitive. CMAP website and surroundings NZ Uni dataset: New Zealand Universities Irreducibility by adding small positive values to all the entries

11 Tomlin s HOTS: handling reduciblity Network flow model with constraints on the modified network [ ] A 1 A = 1 T 0 max ρ i,j (log( ρ i,j ) 1) ρ 0 A i,j i,j [n+1] ρ i,j = ρ j,i, i [n + 1] (p i ) j [n+1] j [n+1] ρ ij = 1 (µ) i,j [n+1] ρ n+1,j = 1 α (a) j [n] 1 α = i [n] ρ i,n+1 (b)

12 θ(p, µ, a, b) = Dual function i,j [n] A ij e pi pj +µ + i [n] e b p n+1+p i +µ + j [n] e a+p n+1 p j +µ (1 α)a µ + (1 α)b 2α 1 µ(p) = log( i,j [n] A i,je p i p j ) a(p) = log( 1 α 2α 1 b(p) = log( 1 α 2α 1 We denote λ(p) := (µ(p), a(p), b(p)) i,j [n] A i,je p i p j j [n] ep n+1 p j ) i,j [n] A i,je p i p j i [n] ep i p n+1 )

13 Iterative scheme Define f λ to be the fixed point operator associated to the matrix balancing problem on the matrix [ ] e A = µ A e a+µ 1 e b+µ 1 T 0 Tomlin s HOTS algorithm: p k+1 = f λ(p k) (p k ) = F (p k ) F is homogeneous but not monotone Expansive in Thomson s metric (d(x, y) = max i x i y i min j x j y j )

14 Elements of the proof Theorem (Lyapounov function) The ideal HOTS operator verifies θ 0 (f (p)) θ 0 (p) Theorem (local contraction in projective space) Denote F (p) = f λ(p) (p) and p such that F (p ) = p. Then all the eigenvalues of F (p ) belong to ( 1, 1] and the eigenvalue 1 is simple.

15 Convergence of HOTS algorithm Theorem If there exists a primal feasible point with the same pattern as A, then the HOTS algorithm converges to the HOTS vector (unique up to an additive constant) with a linear rate of convergence equal to λ 2 ( F ). Proof Use the Lyapunov function Prove that all limit points ( p, λ) minimize θ(p, λ) Conclude thanks to the local contraction in projective space

16 Optimization of link-based rankings Effort concentrated on PageRank - Avrachenkov and Litvak, 2006, one page - Matthieu and Viennot, 2006, unconstrained problem - de Kerchove, Ninove, van Dooren, Ishii and Tempo, Csáji, Jungers and Blondel, F., Akian, Bouhtou, Gaubert, 2011 Perron vector optimization, HITS and HOTS optimization - Fercoq, 2011 (arxiv: )

17 Tomlin s HOTS optimization Obligatory links O, optional links F, prohibited links I N(p) := log( i [n] ep i ), J is the set of hyperlinks selected The HOTS optimization problem is: max J F,p R n{u(p) ; f λ(p) (A(J), p) = p, N(p) = 0, } Relaxed HOTS optimization problem: max A R n n,p R nu(p) f λ(p) (A, p) = p, N(p) = 0 A i,j = 1, (i, j) O A i,j = 0, (i, j) I 0 A i,j 1, (i, j) F

18 Matrix of partial derivatives Denote F (A, p) = f λ(p) (A, p). Proposition The derivative of U p is given by g i,j = l w l F l A i,j w = ( U T + ( U T e) N T )( p F I ) # Moreover, the matrix (g i,j ) i,j has rank at most 3. where Proposition Let z = U T + ( U T e) N T and v s.t. v T p F = v T. The fixed point scheme defined by k N, w k+1 = (z + w k p F )(I 1 v T e ev T ) converges in geometric speed to w.

19 Polak s Master algorithm model For any weighted adjacency matrix A, J(A) = U(p(A)) J n (A) = U(p kn ) where k n is the first nonnegative integer k such that p k+1 p k (n) B n (A) an approximate gradient iteration Let ω (0, 1), σ (0, 1), n 1 N and A 0 C, For i N, compute A i+1 and the smallest n i N s.t. n i n i 1 A i+1 = B ni (A i ) J ni (A i+1 ) J ni (A i ) σ ( (n i )) ω

20 Interrupted Armijo line search Let ( M n ) n 0 be a sequence diverging to +, σ (0, 1), α 0 > 0, β (0, 1) and γ > 0. Given n N, J n = U(p kn ) and g n is an approximate gradient. Let m n be the first m N such that J n ( PC (A β m α 0 g n (A)) ) J n (A) σ A P C(A β m α 0 g n (A)) 2 2 β m α 0 If m n M n, then Otherwise B n (A) =. B n (A) = P C (x β mn α 0 g n (A))

21 Convergence theorem for HOTS optimization Theorem Let (A i ) i 0 be a sequence constructed by the Master Algorithm Model for the resolution of the HOTS optimization problem such that B n (A) is the Interrupted Armijo line search Then every accumulation point of (A i ) i 0 is a stationary point of the optimization problem.

22 Web graph optimized for HOTS added links HOTS score sum:

23 Numerical results CMAP NZ Uni Gradient (linear system) 0.82 s/it out of memory Gradient (iterative scheme) 0.12 s/it 62 s/it Coupled iter. (master algo.) 0.04 s/it 2.9 s/it Comparison of algorithms for HOTS optimization CMAP website: 1,500 pages New Zealand universities websites: (413,639 pages) The iterative scheme formulation makes the problem scalable The Master algorithm model gives a speedup from 3 to 30

24 Locally optimal solution Local optimal solution of the HOTS optimization problem on CMAP website (1,500 pages) Adjacency matrix restricted to the set of controlled pages Pages sorted by w values Squares: obligatory links

25 Conclusion General convergence proof: nonexpansive maps, nonlinear Perron-Frobenius theory Low rank property of derivatives Iterative scheme to compute it Scalable optimization algorithm Better understanding of HOTS algorithm: - quality of the convergence rate - spamming techniques / search engine optimizations