# Part II Redundant Dictionaries and Pursuit Algorithms

Save this PDF as:

Size: px
Start display at page:

Download "Part II Redundant Dictionaries and Pursuit Algorithms"

## Transcription

1 Aisenstadt Chair Course CRM September 2009 Part II Redundant Dictionaries and Pursuit Algorithms Stéphane Mallat Centre de Mathématiques Appliquées Ecole Polytechnique

2 Sparsity in Redundant Dictionaries Bases are minimum set to decompose signals. Natural languages use redundant dictionaries. Use of larger dictionaries incorporating more patterns to represent complex signals f R N D = {φ p } p Γ with φ p = 1 and Γ = P > N. How to construct sparse representations in D? What is the impact of redundancy on the mathematics and applications?

3 Let D = {φ p } p Γ Dictionary Approximation be a redundant dictionary of P vectors. The best approximation of f from a sub-family orthogonal projection in V Λ = Vect{φ p } p Λ f Λ = a[p] φ p p Λ {φ p } p Λ is its Stability: {φ p } p Λ must be a Riez basis of V Λ : there exists 0 <A Λ B Λ such that a[p] R Λ, A Λ a[p] 2 a[p] φ p 2 B Λ a[p] 2. p Λ p Λ p Λ

4 Best M-Term Approximation The best M-term approximation support Λ minimizes f f Λ with Λ = M. A best M-term approximation minimizes a Lagrangian: L 0 = f f Λ 2 + T 2 Λ. If is an orthonormal basis then D Λ = {p : f,φ p T } In general, finding Λ is an NP-hard problem.

5 Compression Applications Compute a best M-term approximation f Λ = p Λ a[p] φ p with Λ = M. Compression with uniform quantization f Λ = p Λ Q(a[p]) φ p Total bit budget: R = log 2 ( P M ) + µm R M log 2 (P/M) Increasing P reduces D = f f Λ but increases R. If f f Λ 2 = O(M α ) then D(R) =R α log(p/r) α.

6 Wavelets for Cartoon Images Theorem: If f is uniformly approximation gives C α f f M 2 = O(M α ) then an M-term wavelet Theorem: If f is piecewise C α with finite length contours then f f M 2 = O(M 1 ) so D(R) =R 1 log(n/r) Result valid for all bounded variation functions. f j =-8 f, ψ H jn f h x j = -7 j = -6 y f, ψv jn f, ψjn D (b) Wavelet Transform (c) Zoom γ

7 Bandlets for Geometric Regularity Wavelet coefficients inherit the geometric regularity f j =-8 j = -7 j = -6 f, ψ H jn f h y f, ψv jn f, ψjn D (b) Wavelet Transform (c) Zoom γ 1. Segmentation of wavelet coefficients. b Regular square Edge square Small Edge square 2. Geometric flow in edge squares along the direction of regularity. w (a) (a ) Wavelet Transform Quadtree (a) 3. 1D wavelet S transform along 3 S the flow = 2 (b) (c) (d) bandlet transform Corner square S 1 S 2 (b ) (c ) (d ) (b ) (c ) ( S 1 (b) (c) ( S

8 Bandlet Approximations A bandlet dictionary is a union of orthonormal bases. The best bandlet approximation (best geometry) which minimizes L 0 = f f M 2 + T 2 M can be computed with O(N log N) operations. C α C α Theorem If f is piecewise with piecewise contours f f M 2 = O(M α ) so D(R) =R α log(p/r) α

9 Application of Photo ID Compression

10 just nice...

11 Xlets Beyond Wavelets Xlets take advantage of the image geometric regularity: bandlets, curvelets, contourlets, edglets, wedglets... Failure to improve wavelet approximations for natural images. Can wavelet approximations be asymptotically improved? I don t think so for static natural images.

12 Denoising in Redundant Dictionaries Measure a signal plus a Gaussian white noise: X[n] =f[n]+w [n] for 0 n < N. Orthogonal projection estimator in selected in a dictionary: V Λ = Vect{φ p } p Λ X Λ = p Λ a[p] φ p. Risk: If Λ is fixed then E{ W Λ 2 } = σ 2 Λ. Oracle choice: find Λ which minimizes E{ f X Λ 2 } = f f Λ 2 + E{ W Λ 2 }. E{ f X Λ 2 } = f f Λ 2 + σ 2 Λ

13 Penalized Estimation Choosing Λ is a model selection problem. Theorem: For satisfies T>σ 2 log e P ( ) Λ = arg min X X Λ 2 + T 2 Λ Λ Γ E{ f X Λ 2 } = 4 min ( f f Λ 2 + T 2 Λ + σ 2) Λ Γ Noisy image Wavelet thresholding Bandlet thresholding PSNR = 22db PSNR = 25.3db PSNR = 26.4db

14 Let D = {φ p } p Γ Greedy Matching Pursuits be a dictionary of P > N vectors. A best M-term approximation minimizes the Lagrangian: L 0 = f f Λ 2 + T 2 Λ Finding Λ is in general an NP complete problem. Greedy choice of the approximation vectors. The approximation of f over φ p0 D yields f = f,φ p0 + Rf f 2 = f,φ p0 2 + Rf 2 To minimize the residual error we choose φ p0 = arg max φ p D f,φ p

15 Matching Pursuit Iterations Initialize For each R 0 f = f m>0 φ pm = arg max φ p D Rm f,φ p R m f = R m f,φ pm + R m+1 f R m f 2 = R m f,φ pm 2 + R m+1 f 2 It results: f = f 2 = M 1 m=0 M 1 m=0 R m f,φ pm + R M f R m f,φ pm 2 + R M f 2

16 Time-Frequency Decompositions Dictionary of time-frequency atoms: D = { φ p (t) = 1 w(t u } 2 j 2 j ) e iξt (j,u,ξ) Γ It includes Dirac and Fourier bases, wavelets and window Fourier atoms. \$ # " ^ (\$)! % t % \$ " ( t )! 0 u t

17 Time Frequency Matching Pursuits f(t) ! 2" 250 t t 1

18 f(t) Speech Signals 1000 t ! 2" t 0 1 Applications: denoising, compression (video), pattern recognition, but instabilities.

19 Orthogonal Matching Pursuit Initialize For each R 0 f = f m>0 φ pm orthogonalize the projections: = arg max φ p D Rm f,φ p u m = φ pm m 1 l=0 φ pm,u l u l 2 u l. R m+1 f = Rm f, u m u m 2 + R m+1 f After N iterations, it gives decomposition in an orthogonal basis: f = N 1 m=0 R m f, u m u m 2.

20 Matching versus Basis Pursuit Greediness is not optimal Signal f Time-Frequency Coefficients Matching Pursuit Basis Pursuit Orthogonal Matching Pursuit

21 Basis Pursuit Find a sparse representation f = a[p] φ p ( ) p Γ 1/q by minimizing an l q norm : a q = a[n] q l 0 n Γ norm : number of non-zero a[n] Not convex for q < 1. l 1 norm : a 1 = a[n]. n Γ. l q balls q =0 q =0 q q =05.. q q =1 q = q =2

22 Basis Pursuit Approximation A basis pursuit computes a sparse approximation f = p Γ solution of a convex optimization problem: ã = arg min a R P a 1 ã[p] φ p Lagrangian formulation: find subject to f p Γ ã which minimizes L 1 (a) = 1 2 f p Γ a[p] φ p 2 + T a 1 a[p]φ p ɛ.

23 Basis Pursuit Denoising Original Image Noisy Image Wavelet Thresh. Basis Pursuit with Wavelet Cosine Diction.

24 Computation of l1 Minimization Many iterative algorithms to minimize L 1 (a) = 1 2 f p Γ a[p] φ p 2 + T a 1 Simplest one: iterative thresholding. Initialisation b = Φf = { f,φ p } p Γ and a 0 =0. For k>0 Gradient step: ã k = a k + γ(b ΦΦ a k ) ( Soft thresholding: a k+1 =ã k max 1 γt ) ã k, 0.

25 Exact Recovery Suppose that the signal is sparse f = p Λ can we recover Λ with pursuit algorithms? With matching pursuits, need that Exact Recovery Criteria a[p] φ p = p Λ f,φ p φ p V Λ. C(R m f,λ c )= max q Λ c R m f,φ q max p Λ R m f,φ p ERC(Λ) = sup C(h, Λ c ) = max h V Λ q Λ c p Λ < 1. φ p, φ q.

26 ERC Recovery Theorem: The approximation support Λ of f V Λ is exactly recovered by an orthogonal matching pursuit or a basis pursuit if ERC(Λ) < 1. Theorem (stability): If f is not exactly sparse, an orthogonal matching pursuit f M with M = Λ iterations satisfies: f f M ( 1+ Similar result for a basis pursuit. Λ ) A Λ (1 ERC(Λ)) 2 f f Λ.

27 Dictionary Coherence Dictionary mutual coherence µ(d) = sup φ p, φ q (p,q) Γ 2 Dirac-Fourier dictionary: µ(d) =N 1/2. Theorem: ERC(Λ) < 1 if Λ < 1 2 ( 1+ 1 ) µ(d) Exact recovery is possible for sufficiently sparse signals in incoherent dictionaries.

28 2nd. Conclusion Redundant dictionaries can improve approximation, compression, denoising. Finding optimal approximation is NP complete but can be approximated with matching or basis pursuits. May be used for pattern recognition but problems of instabilities. The stability depends upon the dictionary coherence. Major applications to inverse problems, superresolution and compress sensing.

### Cheng Soon Ong & Christfried Webers. Canberra February June 2016

c Cheng Soon Ong & Christfried Webers Research Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 31 c Part I

### Disjoint sparsity for signal separation and applications to hybrid imaging inverse problems

Disjoint sparsity for signal separation and applications to hybrid imaging inverse problems Giovanni S Alberti (joint with H Ammari) DMA, École Normale Supérieure, Paris June 16, 2015 Giovanni S Alberti

### Compressive Sensing. Examples in Image Compression. Lecture 4, July 30, Luiz Velho Eduardo A. B. da Silva Adriana Schulz

Compressive Sensing Examples in Image Compression Lecture 4, July, 09 Luiz Velho Eduardo A. B. da Silva Adriana Schulz Today s Lecture Discuss applications of CS in image compression Evaluate CS efficiency

### Morphological Diversity and Sparsity for Multichannel Data Restoration

Morphological Diversity and Sparsity for Multichannel Data Restoration J.Bobin 1, Y.Moudden 1, J.Fadili and J-L.Starck 1 1 jerome.bobin@cea.fr, ymoudden@cea.fr, jstarck@cea.fr - CEA-DAPNIA/SEDI, Service

### Wavelet analysis. Wavelet requirements. Example signals. Stationary signal 2 Hz + 10 Hz + 20Hz. Zero mean, oscillatory (wave) Fast decay (let)

Wavelet analysis In the case of Fourier series, the orthonormal basis is generated by integral dilation of a single function e jx Every 2π-periodic square-integrable function is generated by a superposition

### Low-Dimensional Models for Complex Structured Data

Low-Dimensional Models for Complex Structured Data Jackie Ma (Technische Universität Berlin) DEDALE Kick-off Meeting October 22-23, 2015 Jackie Ma (TU Berlin) WP2 CEA, Saclay/Paris 1 / 11 Low-Dimensional

### Stable Signal Recovery from Incomplete and Inaccurate Measurements

Stable Signal Recovery from Incomplete and Inaccurate Measurements EMMANUEL J. CANDÈS California Institute of Technology JUSTIN K. ROMBERG California Institute of Technology AND TERENCE TAO University

### Several Views of Support Vector Machines

Several Views of Support Vector Machines Ryan M. Rifkin Honda Research Institute USA, Inc. Human Intention Understanding Group 2007 Tikhonov Regularization We are considering algorithms of the form min

### Minimizing beam-on time during radiotherapy treatment by total-variation regularization

Minimizing beam-on time during radiotherapy treatment by total-variation regularization Xing Lab, Department of Radiation Oncology, Stanford. Zhu, L. & Xing, L. (2009) Search for IMRT inverse plans with

### Picture Perfect: The Mathematics of JPEG Compression

Picture Perfect: The Mathematics of JPEG Compression May 19, 2011 1 2 3 in 2D Sampling and the DCT in 2D 2D Compression Images Outline A typical color image might be 600 by 800 pixels. Images Outline A

### Stable Signal Recovery from Incomplete and Inaccurate Measurements

Stable Signal Recovery from Incomplete and Inaccurate Measurements Emmanuel Candes, Justin Romberg, and Terence Tao Applied and Computational Mathematics, Caltech, Pasadena, CA 91125 Department of Mathematics,

### Bag of Pursuits and Neural Gas for Improved Sparse Coding

Bag of Pursuits and Neural Gas for Improved Sparse Coding Kai Labusch, Erhardt Barth, and Thomas Martinetz University of Lübec Institute for Neuro- and Bioinformatics Ratzeburger Allee 6 23562 Lübec, Germany

### Clarify Some Issues on the Sparse Bayesian Learning for Sparse Signal Recovery

Clarify Some Issues on the Sparse Bayesian Learning for Sparse Signal Recovery Zhilin Zhang and Bhaskar D. Rao Technical Report University of California at San Diego September, Abstract Sparse Bayesian

### Solving Inverse Problems with Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity

1 Solving Inverse Problems with Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity Guoshen YU* 1, Guillermo SAPIRO 1, and Stéphane MALLAT 2 1 ECE, University of Minnesota,

### Lecture Topic: Low-Rank Approximations

Lecture Topic: Low-Rank Approximations Low-Rank Approximations We have seen principal component analysis. The extraction of the first principle eigenvalue could be seen as an approximation of the original

### Admin stuff. 4 Image Pyramids. Spatial Domain. Projects. Fourier domain 2/26/2008. Fourier as a change of basis

Admin stuff 4 Image Pyramids Change of office hours on Wed 4 th April Mon 3 st March 9.3.3pm (right after class) Change of time/date t of last class Currently Mon 5 th May What about Thursday 8 th May?

α α λ α = = λ λ α ψ = = α α α λ λ ψ α = + β = > θ θ β > β β θ θ θ β θ β γ θ β = γ θ > β > γ θ β γ = θ β = θ β = θ β = β θ = β β θ = = = β β θ = + α α α α α = = λ λ λ λ λ λ λ = λ λ α α α α λ ψ + α =

### Sparse recovery and compressed sensing in inverse problems

Gerd Teschke (7. Juni 2010) 1/68 Sparse recovery and compressed sensing in inverse problems Gerd Teschke (joint work with Evelyn Herrholz) Institute for Computational Mathematics in Science and Technology

### Lecture 5 Least-squares

EE263 Autumn 2007-08 Stephen Boyd Lecture 5 Least-squares least-squares (approximate) solution of overdetermined equations projection and orthogonality principle least-squares estimation BLUE property

### IK-SVD: Dictionary Learning for Spatial Big Data via Incremental Atom Update

IEEE COMPUTING IN SCIENCE AND ENGINEERING, VOL. XX, NO. XX, XXX 2014 1 IK-SVD: Dictionary Learning for Spatial Big Data via Incremental Atom Update Lizhe Wang 1,KeLu 2, Peng Liu 1, Rajiv Ranjan 3, Lajiao

### SPARSE REPRESENTATIONS APPLICATIONS ON COMPUTER VISION AND PATTERN RECOGNITION

SPARSE REPRESENTATIONS APPLICATIONS ON COMPUTER VISION AND PATTERN RECOGNITION Computer Vision Group Electronics Laboratory Physics Department University of Patras www.upcv.upatras.gr www.ellab.physics.upatras.gr

### CHAPTER 5 CORONARY ANGIOGRAM VIDEO COMPRESSION USING WAVELET BASED CONTOURLET TRANSFORM AND REGION OF INTEREST TECHNIQUE

123 CHAPTER 5 CORONARY ANGIOGRAM VIDEO COMPRESSION USING WAVELET BASED CONTOURLET TRANSFORM AND REGION OF INTEREST TECHNIQUE 5.1 INTRODUCTION Coronary angiograms play an important role in the diagnosis

### COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS16. Lecture 2: Linear Regression Gradient Descent Non-linear basis functions

COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS16 Lecture 2: Linear Regression Gradient Descent Non-linear basis functions LINEAR REGRESSION MOTIVATION Why Linear Regression? Regression

### Analysis of data separation and recovery problems using clustered sparsity

Analysis of data separation and recovery problems using clustered sparsity Emily J. King a, Gitta Kutyniok a, and Xiaosheng Zhuang a a Institute of Mathematics, Universität Osnabrück, Osnabrück, Germany

### c 2003 Society for Industrial and Applied Mathematics

SIAM J. SCI. COMPUT. Vol. 4, No. 4, pp. 148 143 c 3 Society for Industrial and Applied Mathematics WAVELET ALGORITHMS FOR HIGH-RESOLUTION IMAGE RECONSTRUCTION RAYMOND H. CHAN, TONY F. CHAN, LIXIN SHEN,

### A Multichannel Spatial Compressed Sensing Approach for Direction of Arrival Estimation

A Multichannel Spatial Compressed Sensing Approach for Direction of Arrival Estimation Aris Gretsistas and Mark D. Plumbley Queen Mary University of London Centre for Digital Music Mile End Road, E1 4NS,

### Level Set Framework, Signed Distance Function, and Various Tools

Level Set Framework Geometry and Calculus Tools Level Set Framework,, and Various Tools Spencer Department of Mathematics Brigham Young University Image Processing Seminar (Week 3), 2010 Level Set Framework

### Variational approach to restore point-like and curve-like singularities in imaging

Variational approach to restore point-like and curve-like singularities in imaging Daniele Graziani joint work with Gilles Aubert and Laure Blanc-Féraud Roma 12/06/2012 Daniele Graziani (Roma) 12/06/2012

### A VARIANT OF THE EMD METHOD FOR MULTI-SCALE DATA

Advances in Adaptive Data Analysis Vol. 1, No. 4 (2009) 483 516 c World Scientific Publishing Company A VARIANT OF THE EMD METHOD FOR MULTI-SCALE DATA THOMAS Y. HOU and MIKE P. YAN Applied and Computational

### Geometric Tools for Image Compression

Geometric Tools for Image Compression Michael Wakin, Justin Romberg, Hyeokho Choi, Richard Baraniuk Department of Electrical and Computer Engineering, Rice University 6100 Main St., Houston, TX 77005 Abstract

### Point Lattices in Computer Graphics and Visualization how signal processing may help computer graphics

Point Lattices in Computer Graphics and Visualization how signal processing may help computer graphics Dimitri Van De Ville Ecole Polytechnique Fédérale de Lausanne Biomedical Imaging Group dimitri.vandeville@epfl.ch

### Sparsity-promoting recovery from simultaneous data: a compressive sensing approach

SEG 2011 San Antonio Sparsity-promoting recovery from simultaneous data: a compressive sensing approach Haneet Wason*, Tim T. Y. Lin, and Felix J. Herrmann September 19, 2011 SLIM University of British

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Enhancement of scanned documents in Besov spaces using wavelet domain representations

Enhancement of scanned documents in Besov spaces using wavelet domain representations Kathrin Berkner 1 Ricoh Innovations, Inc., 2882 Sand Hill Road, Suite 115, Menlo Park, CA 94025 ABSTRACT After scanning,

### From Sparse Approximation to Forecast of Intraday Load Curves

From Sparse Approximation to Forecast of Intraday Load Curves Mathilde Mougeot Joint work with D. Picard, K. Tribouley (P7)& V. Lefieux, L. Teyssier-Maillard (RTE) 1/43 Electrical Consumption Time series

### Introduction to Convex Optimization for Machine Learning

Introduction to Convex Optimization for Machine Learning John Duchi University of California, Berkeley Practical Machine Learning, Fall 2009 Duchi (UC Berkeley) Convex Optimization for Machine Learning

### IN THIS paper, we address the classic image denoising

3736 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 12, DECEMBER 2006 Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries Michael Elad and Michal Aharon Abstract We

### Sparse Coding and Dictionary Learning for Image Analysis

Sparse Coding and Dictionary Learning for Image Analysis Part I: Optimization for Sparse Coding Francis Bach, Julien Mairal, Jean Ponce and Guillermo Sapiro ICCV 09 tutorial, Kyoto, 28th September 2009

### About the Ingham type proof for the boundary observability of a square and its generalization in N-d

About the Ingham type proof for the boundary observability of a square and its generalization in N-d M. Mehrenberger University of Strasbourg (France), INRIA TONUS team Rome 015 Rome, September 30, 015

### Compressed Sensing with Non-Gaussian Noise and Partial Support Information

IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 10, OCTOBER 2015 1703 Compressed Sensing with Non-Gaussian Noise Partial Support Information Ahmad Abou Saleh, Fady Alajaji, Wai-Yip Chan Abstract We study

### Some stability results of parameter identification in a jump diffusion model

Some stability results of parameter identification in a jump diffusion model D. Düvelmeyer Technische Universität Chemnitz, Fakultät für Mathematik, 09107 Chemnitz, Germany Abstract In this paper we discuss

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

### ELEC 4801 THESIS PROJECT

School of Information Technology and Electrical Engineering ELEC 4801 THESIS PROJECT Thesis: Speech Compression Using Wavelets Name: Nikhil Rao Student No: 33710745 Supervisor: Dr John Homer Submitted

### t := maxγ ν subject to ν {0,1,2,...} and f(x c +γ ν d) f(x c )+cγ ν f (x c ;d).

1. Line Search Methods Let f : R n R be given and suppose that x c is our current best estimate of a solution to P min x R nf(x). A standard method for improving the estimate x c is to choose a direction

### FFT Algorithms. Chapter 6. Contents 6.1

Chapter 6 FFT Algorithms Contents Efficient computation of the DFT............................................ 6.2 Applications of FFT................................................... 6.6 Computing DFT

### INPAINTING COLOR IMAGES IN LEARNED DICTIONARY. Bijenička cesta 54, 10002, Zagreb, Croatia

2th European Signal Processing Conference (EUSIPCO 212) Bucharest, Romania, August 27-31, 212 INPAINTING COLOR IMAGES IN LEARNED DICTIONARY Marko Filipović 1, Ivica Kopriva 1 and Andrzej Cichocki 2 1 Ruđer

### Evolutionary denoising based on an estimation of Hölder exponents with oscillations.

Evolutionary denoising based on an estimation of Hölder exponents with oscillations. Pierrick Legrand,, Evelyne Lutton and Gustavo Olague CICESE, Research Center, Applied Physics Division Centro de Investigación

### Image Compression Using Gabor Filter

Mr.B.H.Deokate, Dr. P. M. Patil and Mr.S.S. Majgaonkar 28 Image Compression Using Gabor Filter Mr.B.H.Deokate, Dr. P. M. Patil and Mr.S.S. Majgaonkar Abstract: The data is the basic component of information.

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### Regression Using Support Vector Machines: Basic Foundations

Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering

### Wavelet Analysis Based Estimation of Probability Density function of Wind Data

, pp.23-34 http://dx.doi.org/10.14257/ijeic.2014.5.3.03 Wavelet Analysis Based Estimation of Probability Density function of Wind Data Debanshee Datta Department of Mechanical Engineering Indian Institute

### Minimum-Volume Box Containing a Set of Points

Minimum-Volume Box Containing a Set of Points David Eberly Geometric Tools, LLC http://www.geometrictools.com/ Copyright c 1998-2016. All Rights Reserved. Created: July 25, 2015 Contents 1 Introduction

### FOURIER SERIES BASED PHASE MODEL FOR STATISTICAL SIGNAL PROCESSING

EE 602 TERM PAPER FOURIER SERIES BASED PHASE MODEL FOR STATISTICAL SIGNAL PROCESSING - Chong yung chi Instructor: Dr. Rajesh Hegde Presented by: Mounika Bopuddi (Y8104037) Richa Tripathi (Y3288) ABSTRACT

### An integrated data compression scheme for power quality events using spline wavelet and neural network

An integrated data compression scheme for power quality events using spline wavelet and neural network S.K. Meher a, A.K. Pradhan b,, G. Panda c a Department of Electronics and Communication Engineering,

### Applications to Data Smoothing and Image Processing I

Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is

### Coordinate Descent for L 1 Minimization

Coordinate Descent for L 1 Minimization Yingying Li and Stanley Osher Department of Mathematics University of California, Los Angeles Jan 13, 2010 Li, Osher (UCLA) Coordinate Descent for L 1 Minimization

### COMPRESSIVE SENSING WITH HIGHLY COHERENT DICTIONARIES

COMPRESSIVE SENSING WITH HIGHLY COHERENT DICTIONARIES SARA COHEN Abstract. Compressive sensing is an emerging field based on the discovery that sparse signals and images can be reconstructed from highly

### Robust principle component analysis (RPCA) for seismic data denoising

Robust principle component analysis (RPCA) for seismic data denoising Jinkun Cheng, Ke Chen and Mauricio D. Sacchi Department of Physics, University of Alberta Summary Principal component analysis (PCA)

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### Neural Networks. CAP5610 Machine Learning Instructor: Guo-Jun Qi

Neural Networks CAP5610 Machine Learning Instructor: Guo-Jun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector

### Sublinear Algorithms for Big Data. Part 4: Random Topics

Sublinear Algorithms for Big Data Part 4: Random Topics Qin Zhang 1-1 2-1 Topic 1: Compressive sensing Compressive sensing The model (Candes-Romberg-Tao 04; Donoho 04) Applicaitons Medical imaging reconstruction

### Image Super-Resolution via Sparse Representation

1 Image Super-Resolution via Sparse Representation Jianchao Yang, Student Member, IEEE, John Wright, Student Member, IEEE Thomas Huang, Life Fellow, IEEE and Yi Ma, Senior Member, IEEE Abstract This paper

### Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

### Linear Algebra Review. Vectors

Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length

### An Improved Reconstruction methods of Compressive Sensing Data Recovery in Wireless Sensor Networks

Vol.8, No.1 (14), pp.1-8 http://dx.doi.org/1.1457/ijsia.14.8.1.1 An Improved Reconstruction methods of Compressive Sensing Data Recovery in Wireless Sensor Networks Sai Ji 1,, Liping Huang, Jin Wang, Jian

### Numerisches Rechnen. (für Informatiker) M. Grepl J. Berger & J.T. Frings. Institut für Geometrie und Praktische Mathematik RWTH Aachen

(für Informatiker) M. Grepl J. Berger & J.T. Frings Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2010/11 Problem Statement Unconstrained Optimality Conditions Constrained

### Fast Compressed Sensing Based High Resolution Image Reconstruction

International Journal of New Technology and Research (IJNTR) ISSN:24-4116, Volume-2, Issue-3, March 16 Pages 71-76 Fast Compressed Sensing Based High Resolution Image Reconstruction Dr G Vijay Kumar Abstract

### From Maxent to Machine Learning and Back

From Maxent to Machine Learning and Back T. Sears ANU March 2007 T. Sears (ANU) From Maxent to Machine Learning and Back Maxent 2007 1 / 36 50 Years Ago... The principles and mathematical methods of statistical

### Cyber-Security Analysis of State Estimators in Power Systems

Cyber-Security Analysis of State Estimators in Electric Power Systems André Teixeira 1, Saurabh Amin 2, Henrik Sandberg 1, Karl H. Johansson 1, and Shankar Sastry 2 ACCESS Linnaeus Centre, KTH-Royal Institute

### NMR Measurement of T1-T2 Spectra with Partial Measurements using Compressive Sensing

NMR Measurement of T1-T2 Spectra with Partial Measurements using Compressive Sensing Alex Cloninger Norbert Wiener Center Department of Mathematics University of Maryland, College Park http://www.norbertwiener.umd.edu

### MPEG, the MP3 Standard, and Audio Compression

MPEG, the MP3 Standard, and Audio Compression Mark ilgore and Jamie Wu Mathematics of the Information Age September 16, 23 Audio Compression Basic Audio Coding. Why beneficial to compress? Lossless versus

### Noisy and Missing Data Regression: Distribution-Oblivious Support Recovery

: Distribution-Oblivious Support Recovery Yudong Chen Department of Electrical and Computer Engineering The University of Texas at Austin Austin, TX 7872 Constantine Caramanis Department of Electrical

### Exact shape-reconstruction by one-step linearization in electrical impedance tomography

Exact shape-reconstruction by one-step linearization in electrical impedance tomography Bastian von Harrach harrach@math.uni-mainz.de Institut für Mathematik, Joh. Gutenberg-Universität Mainz, Germany

### JPEG compression of monochrome 2D-barcode images using DCT coefficient distributions

Edith Cowan University Research Online ECU Publications Pre. JPEG compression of monochrome D-barcode images using DCT coefficient distributions Keng Teong Tan Hong Kong Baptist University Douglas Chai

### Chapter 6. Orthogonality

6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be

### Curvelets A Surprisingly Effective Nonadaptive Representation For Objects with Edges

Curvelets A Surprisingly Effective Nonadaptive Representation For Objects with Edges Emmanuel J. Candès and David L. Donoho Abstract. It is widely believed that to efficiently represent an otherwise smooth

### DSL Spectrum Management

DSL Spectrum Management Dr. Jianwei Huang Department of Electrical Engineering Princeton University Guest Lecture of ELE539A March 2007 Jianwei Huang (Princeton) DSL Spectrum Management March 2007 1 /

### Compressive Acquisition of Sparse Deflectometric Maps

Compressive Acquisition of Sparse Deflectometric Maps Prasad Sudhakar*, Laurent Jacques*, Adriana Gonzalez Gonzalez* Xavier Dubois +, Philippe Antoine +, Luc Joannes + *: Louvain University (UCL), Louvain-la-Neuve,

### Discrete Optimization

Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using

### FINITE ELEMENT : MATRIX FORMULATION. Georges Cailletaud Ecole des Mines de Paris, Centre des Matériaux UMR CNRS 7633

FINITE ELEMENT : MATRIX FORMULATION Georges Cailletaud Ecole des Mines de Paris, Centre des Matériaux UMR CNRS 76 FINITE ELEMENT : MATRIX FORMULATION Discrete vs continuous Element type Polynomial approximation

### Computer Networks 56 (2012) 2049 2067. Contents lists available at SciVerse ScienceDirect. Computer Networks

Computer Networks 56 (212) 249 267 Contents lists available at SciVerse ScienceDirect Computer Networks journal homepage: www.elsevier.com/locate/comnet Structural analysis of network traffic matrix via

### IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 5, MAY

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 5, MAY 2006 1071 Wavelet-Domain Approximation and Compression of Piecewise Smooth Images Michael B. Wakin, Student Member, IEEE, Justin K. Romberg, Member,

### Big Data & Scripting storage networks and distributed file systems

Big Data & Scripting storage networks and distributed file systems 1, 2, in the remainder we use networks of computing nodes to enable computations on even larger datasets for a computation, each node

### VOL. 2, NO. 4, May 2012 ISSN ARPN Journal of Science and Technology All rights reserved.

Analysis of Various Image Compression Techniques 1 G.M.Padmaja, 2 P.Nirupama 1 Senior Assistant Professor in CSE Dept, BVRIT 2 Associate Professor in CSE Dept, SIET 1 padmaja.gmp@gmail.com, 2 nirupama.cse1@gmail.com

### Computational Optical Imaging - Optique Numerique. -- Deconvolution --

Computational Optical Imaging - Optique Numerique -- Deconvolution -- Winter 2014 Ivo Ihrke Deconvolution Ivo Ihrke Outline Deconvolution Theory example 1D deconvolution Fourier method Algebraic method

### Couple Dictionary Training for Image Super-resolution

IEEE TRANSACTIONS ON IMAGE PROCESSING 1 Couple Dictionary Training for Image Super-resolution Jianchao Yang, Student Member, IEEE, Zhaowen Wang, Student Member, IEEE, Zhe Lin, Member, IEEE, Scott Cohen,

### α = u v. In other words, Orthogonal Projection

Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v

### Monte Carlo Simulation

1 Monte Carlo Simulation Stefan Weber Leibniz Universität Hannover email: sweber@stochastik.uni-hannover.de web: www.stochastik.uni-hannover.de/ sweber Monte Carlo Simulation 2 Quantifying and Hedging

### Pixel enlargement in high-speed camera image acquisition based on 3D sparse representations

Pixel enlargement in high-speed camera image acquisition based on 3D sparse representations A. Hirabayashi, N. Nogami, J. White Ritsumeikan University College of Information Science and Engineering Kusatsu,

### An Information-Theoretic Approach to Distributed Compressed Sensing

An Information-Theoretic Approach to Distributed Compressed Sensing Dror Baron, Marco F. Duarte, Shriram Sarvotham, Michael B. Wakin and Richard G. Baraniuk Dept. of Electrical and Computer Engineering,

### International Doctoral School Algorithmic Decision Theory: MCDA and MOO

International Doctoral School Algorithmic Decision Theory: MCDA and MOO Lecture 2: Multiobjective Linear Programming Department of Engineering Science, The University of Auckland, New Zealand Laboratoire

### BIL Computer Vision Mar 5, 2014

BIL 719 - Computer Vision Mar 5, 2014 Image pyramids Aykut Erdem Dept. of Computer Engineering Hacettepe University Image Scaling This image is too big to fit on the screen. How can we reduce it? How to

### Signal Processing for Speech Recognition

Signal Processing for Speech Recognition Once a signal has been sampled, we have huge amounts of data, often 20,000 16 bit numbers a second! We need to find ways to concisely capture the properties of

### Sparse Signal Recovery: Theory, Applications and Algorithms

Sparse Signal Recovery: Theory, Applications and Algorithms Bhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego Collaborators: I. Gorodonitsky, S. Cotter,

### Image Compression through DCT and Huffman Coding Technique

International Journal of Current Engineering and Technology E-ISSN 2277 4106, P-ISSN 2347 5161 2015 INPRESSCO, All Rights Reserved Available at http://inpressco.com/category/ijcet Research Article Rahul

### Sparsity & Co.: An Overview of Analysis vs Synthesis in Low-Dimensional Signal Models. Rémi Gribonval, INRIA Rennes - Bretagne Atlantique

Sparsity & Co.: An Overview of Analysis vs Synthesis in Low-Dimensional Signal Models Rémi Gribonval, INRIA Rennes - Bretagne Atlantique remi.gribonval@inria.fr http://www.irisa.fr/metiss/members/remi/talks

### A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION

1 A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION Dimitri Bertsekas M.I.T. FEBRUARY 2003 2 OUTLINE Convexity issues in optimization Historical remarks Our treatment of the subject Three unifying lines of

### 7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method