Part II Redundant Dictionaries and Pursuit Algorithms


 Barbra Adams
 2 years ago
 Views:
Transcription
1 Aisenstadt Chair Course CRM September 2009 Part II Redundant Dictionaries and Pursuit Algorithms Stéphane Mallat Centre de Mathématiques Appliquées Ecole Polytechnique
2 Sparsity in Redundant Dictionaries Bases are minimum set to decompose signals. Natural languages use redundant dictionaries. Use of larger dictionaries incorporating more patterns to represent complex signals f R N D = {φ p } p Γ with φ p = 1 and Γ = P > N. How to construct sparse representations in D? What is the impact of redundancy on the mathematics and applications?
3 Let D = {φ p } p Γ Dictionary Approximation be a redundant dictionary of P vectors. The best approximation of f from a subfamily orthogonal projection in V Λ = Vect{φ p } p Λ f Λ = a[p] φ p p Λ {φ p } p Λ is its Stability: {φ p } p Λ must be a Riez basis of V Λ : there exists 0 <A Λ B Λ such that a[p] R Λ, A Λ a[p] 2 a[p] φ p 2 B Λ a[p] 2. p Λ p Λ p Λ
4 Best MTerm Approximation The best Mterm approximation support Λ minimizes f f Λ with Λ = M. A best Mterm approximation minimizes a Lagrangian: L 0 = f f Λ 2 + T 2 Λ. If is an orthonormal basis then D Λ = {p : f,φ p T } In general, finding Λ is an NPhard problem.
5 Compression Applications Compute a best Mterm approximation f Λ = p Λ a[p] φ p with Λ = M. Compression with uniform quantization f Λ = p Λ Q(a[p]) φ p Total bit budget: R = log 2 ( P M ) + µm R M log 2 (P/M) Increasing P reduces D = f f Λ but increases R. If f f Λ 2 = O(M α ) then D(R) =R α log(p/r) α.
6 Wavelets for Cartoon Images Theorem: If f is uniformly approximation gives C α f f M 2 = O(M α ) then an Mterm wavelet Theorem: If f is piecewise C α with finite length contours then f f M 2 = O(M 1 ) so D(R) =R 1 log(n/r) Result valid for all bounded variation functions. f j =8 f, ψ H jn f h x j = 7 j = 6 y f, ψv jn f, ψjn D (b) Wavelet Transform (c) Zoom γ
7 Bandlets for Geometric Regularity Wavelet coefficients inherit the geometric regularity f j =8 j = 7 j = 6 f, ψ H jn f h y f, ψv jn f, ψjn D (b) Wavelet Transform (c) Zoom γ 1. Segmentation of wavelet coefficients. b Regular square Edge square Small Edge square 2. Geometric flow in edge squares along the direction of regularity. w (a) (a ) Wavelet Transform Quadtree (a) 3. 1D wavelet S transform along 3 S the flow = 2 (b) (c) (d) bandlet transform Corner square S 1 S 2 (b ) (c ) (d ) (b ) (c ) ( S 1 (b) (c) ( S
8 Bandlet Approximations A bandlet dictionary is a union of orthonormal bases. The best bandlet approximation (best geometry) which minimizes L 0 = f f M 2 + T 2 M can be computed with O(N log N) operations. C α C α Theorem If f is piecewise with piecewise contours f f M 2 = O(M α ) so D(R) =R α log(p/r) α
9 Application of Photo ID Compression
10 just nice...
11 Xlets Beyond Wavelets Xlets take advantage of the image geometric regularity: bandlets, curvelets, contourlets, edglets, wedglets... Failure to improve wavelet approximations for natural images. Can wavelet approximations be asymptotically improved? I don t think so for static natural images.
12 Denoising in Redundant Dictionaries Measure a signal plus a Gaussian white noise: X[n] =f[n]+w [n] for 0 n < N. Orthogonal projection estimator in selected in a dictionary: V Λ = Vect{φ p } p Λ X Λ = p Λ a[p] φ p. Risk: If Λ is fixed then E{ W Λ 2 } = σ 2 Λ. Oracle choice: find Λ which minimizes E{ f X Λ 2 } = f f Λ 2 + E{ W Λ 2 }. E{ f X Λ 2 } = f f Λ 2 + σ 2 Λ
13 Penalized Estimation Choosing Λ is a model selection problem. Theorem: For satisfies T>σ 2 log e P ( ) Λ = arg min X X Λ 2 + T 2 Λ Λ Γ E{ f X Λ 2 } = 4 min ( f f Λ 2 + T 2 Λ + σ 2) Λ Γ Noisy image Wavelet thresholding Bandlet thresholding PSNR = 22db PSNR = 25.3db PSNR = 26.4db
14 Let D = {φ p } p Γ Greedy Matching Pursuits be a dictionary of P > N vectors. A best Mterm approximation minimizes the Lagrangian: L 0 = f f Λ 2 + T 2 Λ Finding Λ is in general an NP complete problem. Greedy choice of the approximation vectors. The approximation of f over φ p0 D yields f = f,φ p0 + Rf f 2 = f,φ p0 2 + Rf 2 To minimize the residual error we choose φ p0 = arg max φ p D f,φ p
15 Matching Pursuit Iterations Initialize For each R 0 f = f m>0 φ pm = arg max φ p D Rm f,φ p R m f = R m f,φ pm + R m+1 f R m f 2 = R m f,φ pm 2 + R m+1 f 2 It results: f = f 2 = M 1 m=0 M 1 m=0 R m f,φ pm + R M f R m f,φ pm 2 + R M f 2
16 TimeFrequency Decompositions Dictionary of timefrequency atoms: D = { φ p (t) = 1 w(t u } 2 j 2 j ) e iξt (j,u,ξ) Γ It includes Dirac and Fourier bases, wavelets and window Fourier atoms. $ # " ^ ($)! % t % $ " ( t )! 0 u t
17 Time Frequency Matching Pursuits f(t) ! 2" 250 t t 1
18 f(t) Speech Signals 1000 t ! 2" t 0 1 Applications: denoising, compression (video), pattern recognition, but instabilities.
19 Orthogonal Matching Pursuit Initialize For each R 0 f = f m>0 φ pm orthogonalize the projections: = arg max φ p D Rm f,φ p u m = φ pm m 1 l=0 φ pm,u l u l 2 u l. R m+1 f = Rm f, u m u m 2 + R m+1 f After N iterations, it gives decomposition in an orthogonal basis: f = N 1 m=0 R m f, u m u m 2.
20 Matching versus Basis Pursuit Greediness is not optimal Signal f TimeFrequency Coefficients Matching Pursuit Basis Pursuit Orthogonal Matching Pursuit
21 Basis Pursuit Find a sparse representation f = a[p] φ p ( ) p Γ 1/q by minimizing an l q norm : a q = a[n] q l 0 n Γ norm : number of nonzero a[n] Not convex for q < 1. l 1 norm : a 1 = a[n]. n Γ. l q balls q =0 q =0 q q =05.. q q =1 q = q =2
22 Basis Pursuit Approximation A basis pursuit computes a sparse approximation f = p Γ solution of a convex optimization problem: ã = arg min a R P a 1 ã[p] φ p Lagrangian formulation: find subject to f p Γ ã which minimizes L 1 (a) = 1 2 f p Γ a[p] φ p 2 + T a 1 a[p]φ p ɛ.
23 Basis Pursuit Denoising Original Image Noisy Image Wavelet Thresh. Basis Pursuit with Wavelet Cosine Diction.
24 Computation of l1 Minimization Many iterative algorithms to minimize L 1 (a) = 1 2 f p Γ a[p] φ p 2 + T a 1 Simplest one: iterative thresholding. Initialisation b = Φf = { f,φ p } p Γ and a 0 =0. For k>0 Gradient step: ã k = a k + γ(b ΦΦ a k ) ( Soft thresholding: a k+1 =ã k max 1 γt ) ã k, 0.
25 Exact Recovery Suppose that the signal is sparse f = p Λ can we recover Λ with pursuit algorithms? With matching pursuits, need that Exact Recovery Criteria a[p] φ p = p Λ f,φ p φ p V Λ. C(R m f,λ c )= max q Λ c R m f,φ q max p Λ R m f,φ p ERC(Λ) = sup C(h, Λ c ) = max h V Λ q Λ c p Λ < 1. φ p, φ q.
26 ERC Recovery Theorem: The approximation support Λ of f V Λ is exactly recovered by an orthogonal matching pursuit or a basis pursuit if ERC(Λ) < 1. Theorem (stability): If f is not exactly sparse, an orthogonal matching pursuit f M with M = Λ iterations satisfies: f f M ( 1+ Similar result for a basis pursuit. Λ ) A Λ (1 ERC(Λ)) 2 f f Λ.
27 Dictionary Coherence Dictionary mutual coherence µ(d) = sup φ p, φ q (p,q) Γ 2 DiracFourier dictionary: µ(d) =N 1/2. Theorem: ERC(Λ) < 1 if Λ < 1 2 ( 1+ 1 ) µ(d) Exact recovery is possible for sufficiently sparse signals in incoherent dictionaries.
28 2nd. Conclusion Redundant dictionaries can improve approximation, compression, denoising. Finding optimal approximation is NP complete but can be approximated with matching or basis pursuits. May be used for pattern recognition but problems of instabilities. The stability depends upon the dictionary coherence. Major applications to inverse problems, superresolution and compress sensing.
Cheng Soon Ong & Christfried Webers. Canberra February June 2016
c Cheng Soon Ong & Christfried Webers Research Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 31 c Part I
More informationDisjoint sparsity for signal separation and applications to hybrid imaging inverse problems
Disjoint sparsity for signal separation and applications to hybrid imaging inverse problems Giovanni S Alberti (joint with H Ammari) DMA, École Normale Supérieure, Paris June 16, 2015 Giovanni S Alberti
More informationCompressive Sensing. Examples in Image Compression. Lecture 4, July 30, Luiz Velho Eduardo A. B. da Silva Adriana Schulz
Compressive Sensing Examples in Image Compression Lecture 4, July, 09 Luiz Velho Eduardo A. B. da Silva Adriana Schulz Today s Lecture Discuss applications of CS in image compression Evaluate CS efficiency
More informationMorphological Diversity and Sparsity for Multichannel Data Restoration
Morphological Diversity and Sparsity for Multichannel Data Restoration J.Bobin 1, Y.Moudden 1, J.Fadili and JL.Starck 1 1 jerome.bobin@cea.fr, ymoudden@cea.fr, jstarck@cea.fr  CEADAPNIA/SEDI, Service
More informationWavelet analysis. Wavelet requirements. Example signals. Stationary signal 2 Hz + 10 Hz + 20Hz. Zero mean, oscillatory (wave) Fast decay (let)
Wavelet analysis In the case of Fourier series, the orthonormal basis is generated by integral dilation of a single function e jx Every 2πperiodic squareintegrable function is generated by a superposition
More informationLowDimensional Models for Complex Structured Data
LowDimensional Models for Complex Structured Data Jackie Ma (Technische Universität Berlin) DEDALE Kickoff Meeting October 2223, 2015 Jackie Ma (TU Berlin) WP2 CEA, Saclay/Paris 1 / 11 LowDimensional
More informationStable Signal Recovery from Incomplete and Inaccurate Measurements
Stable Signal Recovery from Incomplete and Inaccurate Measurements EMMANUEL J. CANDÈS California Institute of Technology JUSTIN K. ROMBERG California Institute of Technology AND TERENCE TAO University
More informationSeveral Views of Support Vector Machines
Several Views of Support Vector Machines Ryan M. Rifkin Honda Research Institute USA, Inc. Human Intention Understanding Group 2007 Tikhonov Regularization We are considering algorithms of the form min
More informationMinimizing beamon time during radiotherapy treatment by totalvariation regularization
Minimizing beamon time during radiotherapy treatment by totalvariation regularization Xing Lab, Department of Radiation Oncology, Stanford. Zhu, L. & Xing, L. (2009) Search for IMRT inverse plans with
More informationPicture Perfect: The Mathematics of JPEG Compression
Picture Perfect: The Mathematics of JPEG Compression May 19, 2011 1 2 3 in 2D Sampling and the DCT in 2D 2D Compression Images Outline A typical color image might be 600 by 800 pixels. Images Outline A
More informationStable Signal Recovery from Incomplete and Inaccurate Measurements
Stable Signal Recovery from Incomplete and Inaccurate Measurements Emmanuel Candes, Justin Romberg, and Terence Tao Applied and Computational Mathematics, Caltech, Pasadena, CA 91125 Department of Mathematics,
More informationBag of Pursuits and Neural Gas for Improved Sparse Coding
Bag of Pursuits and Neural Gas for Improved Sparse Coding Kai Labusch, Erhardt Barth, and Thomas Martinetz University of Lübec Institute for Neuro and Bioinformatics Ratzeburger Allee 6 23562 Lübec, Germany
More informationClarify Some Issues on the Sparse Bayesian Learning for Sparse Signal Recovery
Clarify Some Issues on the Sparse Bayesian Learning for Sparse Signal Recovery Zhilin Zhang and Bhaskar D. Rao Technical Report University of California at San Diego September, Abstract Sparse Bayesian
More informationSolving Inverse Problems with Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity
1 Solving Inverse Problems with Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity Guoshen YU* 1, Guillermo SAPIRO 1, and Stéphane MALLAT 2 1 ECE, University of Minnesota,
More informationLecture Topic: LowRank Approximations
Lecture Topic: LowRank Approximations LowRank Approximations We have seen principal component analysis. The extraction of the first principle eigenvalue could be seen as an approximation of the original
More informationAdmin stuff. 4 Image Pyramids. Spatial Domain. Projects. Fourier domain 2/26/2008. Fourier as a change of basis
Admin stuff 4 Image Pyramids Change of office hours on Wed 4 th April Mon 3 st March 9.3.3pm (right after class) Change of time/date t of last class Currently Mon 5 th May What about Thursday 8 th May?
More informationα α λ α = = λ λ α ψ = = α α α λ λ ψ α = + β = > θ θ β > β β θ θ θ β θ β γ θ β = γ θ > β > γ θ β γ = θ β = θ β = θ β = β θ = β β θ = = = β β θ = + α α α α α = = λ λ λ λ λ λ λ = λ λ α α α α λ ψ + α =
More informationSparse recovery and compressed sensing in inverse problems
Gerd Teschke (7. Juni 2010) 1/68 Sparse recovery and compressed sensing in inverse problems Gerd Teschke (joint work with Evelyn Herrholz) Institute for Computational Mathematics in Science and Technology
More informationLecture 5 Leastsquares
EE263 Autumn 200708 Stephen Boyd Lecture 5 Leastsquares leastsquares (approximate) solution of overdetermined equations projection and orthogonality principle leastsquares estimation BLUE property
More informationIKSVD: Dictionary Learning for Spatial Big Data via Incremental Atom Update
IEEE COMPUTING IN SCIENCE AND ENGINEERING, VOL. XX, NO. XX, XXX 2014 1 IKSVD: Dictionary Learning for Spatial Big Data via Incremental Atom Update Lizhe Wang 1,KeLu 2, Peng Liu 1, Rajiv Ranjan 3, Lajiao
More informationSPARSE REPRESENTATIONS APPLICATIONS ON COMPUTER VISION AND PATTERN RECOGNITION
SPARSE REPRESENTATIONS APPLICATIONS ON COMPUTER VISION AND PATTERN RECOGNITION Computer Vision Group Electronics Laboratory Physics Department University of Patras www.upcv.upatras.gr www.ellab.physics.upatras.gr
More informationCHAPTER 5 CORONARY ANGIOGRAM VIDEO COMPRESSION USING WAVELET BASED CONTOURLET TRANSFORM AND REGION OF INTEREST TECHNIQUE
123 CHAPTER 5 CORONARY ANGIOGRAM VIDEO COMPRESSION USING WAVELET BASED CONTOURLET TRANSFORM AND REGION OF INTEREST TECHNIQUE 5.1 INTRODUCTION Coronary angiograms play an important role in the diagnosis
More informationCOMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS16. Lecture 2: Linear Regression Gradient Descent Nonlinear basis functions
COMPUTATIONAL INTELLIGENCE (INTRODUCTION TO MACHINE LEARNING) SS16 Lecture 2: Linear Regression Gradient Descent Nonlinear basis functions LINEAR REGRESSION MOTIVATION Why Linear Regression? Regression
More informationAnalysis of data separation and recovery problems using clustered sparsity
Analysis of data separation and recovery problems using clustered sparsity Emily J. King a, Gitta Kutyniok a, and Xiaosheng Zhuang a a Institute of Mathematics, Universität Osnabrück, Osnabrück, Germany
More informationc 2003 Society for Industrial and Applied Mathematics
SIAM J. SCI. COMPUT. Vol. 4, No. 4, pp. 148 143 c 3 Society for Industrial and Applied Mathematics WAVELET ALGORITHMS FOR HIGHRESOLUTION IMAGE RECONSTRUCTION RAYMOND H. CHAN, TONY F. CHAN, LIXIN SHEN,
More informationA Multichannel Spatial Compressed Sensing Approach for Direction of Arrival Estimation
A Multichannel Spatial Compressed Sensing Approach for Direction of Arrival Estimation Aris Gretsistas and Mark D. Plumbley Queen Mary University of London Centre for Digital Music Mile End Road, E1 4NS,
More informationLevel Set Framework, Signed Distance Function, and Various Tools
Level Set Framework Geometry and Calculus Tools Level Set Framework,, and Various Tools Spencer Department of Mathematics Brigham Young University Image Processing Seminar (Week 3), 2010 Level Set Framework
More informationVariational approach to restore pointlike and curvelike singularities in imaging
Variational approach to restore pointlike and curvelike singularities in imaging Daniele Graziani joint work with Gilles Aubert and Laure BlancFéraud Roma 12/06/2012 Daniele Graziani (Roma) 12/06/2012
More informationA VARIANT OF THE EMD METHOD FOR MULTISCALE DATA
Advances in Adaptive Data Analysis Vol. 1, No. 4 (2009) 483 516 c World Scientific Publishing Company A VARIANT OF THE EMD METHOD FOR MULTISCALE DATA THOMAS Y. HOU and MIKE P. YAN Applied and Computational
More informationGeometric Tools for Image Compression
Geometric Tools for Image Compression Michael Wakin, Justin Romberg, Hyeokho Choi, Richard Baraniuk Department of Electrical and Computer Engineering, Rice University 6100 Main St., Houston, TX 77005 Abstract
More informationPoint Lattices in Computer Graphics and Visualization how signal processing may help computer graphics
Point Lattices in Computer Graphics and Visualization how signal processing may help computer graphics Dimitri Van De Ville Ecole Polytechnique Fédérale de Lausanne Biomedical Imaging Group dimitri.vandeville@epfl.ch
More informationSparsitypromoting recovery from simultaneous data: a compressive sensing approach
SEG 2011 San Antonio Sparsitypromoting recovery from simultaneous data: a compressive sensing approach Haneet Wason*, Tim T. Y. Lin, and Felix J. Herrmann September 19, 2011 SLIM University of British
More informationLecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,
More informationEnhancement of scanned documents in Besov spaces using wavelet domain representations
Enhancement of scanned documents in Besov spaces using wavelet domain representations Kathrin Berkner 1 Ricoh Innovations, Inc., 2882 Sand Hill Road, Suite 115, Menlo Park, CA 94025 ABSTRACT After scanning,
More informationFrom Sparse Approximation to Forecast of Intraday Load Curves
From Sparse Approximation to Forecast of Intraday Load Curves Mathilde Mougeot Joint work with D. Picard, K. Tribouley (P7)& V. Lefieux, L. TeyssierMaillard (RTE) 1/43 Electrical Consumption Time series
More informationIntroduction to Convex Optimization for Machine Learning
Introduction to Convex Optimization for Machine Learning John Duchi University of California, Berkeley Practical Machine Learning, Fall 2009 Duchi (UC Berkeley) Convex Optimization for Machine Learning
More informationIN THIS paper, we address the classic image denoising
3736 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 12, DECEMBER 2006 Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries Michael Elad and Michal Aharon Abstract We
More informationSparse Coding and Dictionary Learning for Image Analysis
Sparse Coding and Dictionary Learning for Image Analysis Part I: Optimization for Sparse Coding Francis Bach, Julien Mairal, Jean Ponce and Guillermo Sapiro ICCV 09 tutorial, Kyoto, 28th September 2009
More informationAbout the Ingham type proof for the boundary observability of a square and its generalization in Nd
About the Ingham type proof for the boundary observability of a square and its generalization in Nd M. Mehrenberger University of Strasbourg (France), INRIA TONUS team Rome 015 Rome, September 30, 015
More informationCompressed Sensing with NonGaussian Noise and Partial Support Information
IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 10, OCTOBER 2015 1703 Compressed Sensing with NonGaussian Noise Partial Support Information Ahmad Abou Saleh, Fady Alajaji, WaiYip Chan Abstract We study
More informationSome stability results of parameter identification in a jump diffusion model
Some stability results of parameter identification in a jump diffusion model D. Düvelmeyer Technische Universität Chemnitz, Fakultät für Mathematik, 09107 Chemnitz, Germany Abstract In this paper we discuss
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical
More informationELEC 4801 THESIS PROJECT
School of Information Technology and Electrical Engineering ELEC 4801 THESIS PROJECT Thesis: Speech Compression Using Wavelets Name: Nikhil Rao Student No: 33710745 Supervisor: Dr John Homer Submitted
More informationt := maxγ ν subject to ν {0,1,2,...} and f(x c +γ ν d) f(x c )+cγ ν f (x c ;d).
1. Line Search Methods Let f : R n R be given and suppose that x c is our current best estimate of a solution to P min x R nf(x). A standard method for improving the estimate x c is to choose a direction
More informationFFT Algorithms. Chapter 6. Contents 6.1
Chapter 6 FFT Algorithms Contents Efficient computation of the DFT............................................ 6.2 Applications of FFT................................................... 6.6 Computing DFT
More informationINPAINTING COLOR IMAGES IN LEARNED DICTIONARY. Bijenička cesta 54, 10002, Zagreb, Croatia
2th European Signal Processing Conference (EUSIPCO 212) Bucharest, Romania, August 2731, 212 INPAINTING COLOR IMAGES IN LEARNED DICTIONARY Marko Filipović 1, Ivica Kopriva 1 and Andrzej Cichocki 2 1 Ruđer
More informationEvolutionary denoising based on an estimation of Hölder exponents with oscillations.
Evolutionary denoising based on an estimation of Hölder exponents with oscillations. Pierrick Legrand,, Evelyne Lutton and Gustavo Olague CICESE, Research Center, Applied Physics Division Centro de Investigación
More informationImage Compression Using Gabor Filter
Mr.B.H.Deokate, Dr. P. M. Patil and Mr.S.S. Majgaonkar 28 Image Compression Using Gabor Filter Mr.B.H.Deokate, Dr. P. M. Patil and Mr.S.S. Majgaonkar Abstract: The data is the basic component of information.
More informationStatistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
More informationRegression Using Support Vector Machines: Basic Foundations
Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering
More informationWavelet Analysis Based Estimation of Probability Density function of Wind Data
, pp.2334 http://dx.doi.org/10.14257/ijeic.2014.5.3.03 Wavelet Analysis Based Estimation of Probability Density function of Wind Data Debanshee Datta Department of Mechanical Engineering Indian Institute
More informationMinimumVolume Box Containing a Set of Points
MinimumVolume Box Containing a Set of Points David Eberly Geometric Tools, LLC http://www.geometrictools.com/ Copyright c 19982016. All Rights Reserved. Created: July 25, 2015 Contents 1 Introduction
More informationFOURIER SERIES BASED PHASE MODEL FOR STATISTICAL SIGNAL PROCESSING
EE 602 TERM PAPER FOURIER SERIES BASED PHASE MODEL FOR STATISTICAL SIGNAL PROCESSING  Chong yung chi Instructor: Dr. Rajesh Hegde Presented by: Mounika Bopuddi (Y8104037) Richa Tripathi (Y3288) ABSTRACT
More informationAn integrated data compression scheme for power quality events using spline wavelet and neural network
An integrated data compression scheme for power quality events using spline wavelet and neural network S.K. Meher a, A.K. Pradhan b,, G. Panda c a Department of Electronics and Communication Engineering,
More informationApplications to Data Smoothing and Image Processing I
Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is
More informationCoordinate Descent for L 1 Minimization
Coordinate Descent for L 1 Minimization Yingying Li and Stanley Osher Department of Mathematics University of California, Los Angeles Jan 13, 2010 Li, Osher (UCLA) Coordinate Descent for L 1 Minimization
More informationCOMPRESSIVE SENSING WITH HIGHLY COHERENT DICTIONARIES
COMPRESSIVE SENSING WITH HIGHLY COHERENT DICTIONARIES SARA COHEN Abstract. Compressive sensing is an emerging field based on the discovery that sparse signals and images can be reconstructed from highly
More informationRobust principle component analysis (RPCA) for seismic data denoising
Robust principle component analysis (RPCA) for seismic data denoising Jinkun Cheng, Ke Chen and Mauricio D. Sacchi Department of Physics, University of Alberta Summary Principal component analysis (PCA)
More informationApplied Algorithm Design Lecture 5
Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design
More informationNeural Networks. CAP5610 Machine Learning Instructor: GuoJun Qi
Neural Networks CAP5610 Machine Learning Instructor: GuoJun Qi Recap: linear classifier Logistic regression Maximizing the posterior distribution of class Y conditional on the input vector X Support vector
More informationSublinear Algorithms for Big Data. Part 4: Random Topics
Sublinear Algorithms for Big Data Part 4: Random Topics Qin Zhang 11 21 Topic 1: Compressive sensing Compressive sensing The model (CandesRombergTao 04; Donoho 04) Applicaitons Medical imaging reconstruction
More informationImage SuperResolution via Sparse Representation
1 Image SuperResolution via Sparse Representation Jianchao Yang, Student Member, IEEE, John Wright, Student Member, IEEE Thomas Huang, Life Fellow, IEEE and Yi Ma, Senior Member, IEEE Abstract This paper
More informationMarkov chains and Markov Random Fields (MRFs)
Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume
More informationLinear Algebra Review. Vectors
Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka kosecka@cs.gmu.edu http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 8F Linear Algebra review UCSD Vectors The length
More informationAn Improved Reconstruction methods of Compressive Sensing Data Recovery in Wireless Sensor Networks
Vol.8, No.1 (14), pp.18 http://dx.doi.org/1.1457/ijsia.14.8.1.1 An Improved Reconstruction methods of Compressive Sensing Data Recovery in Wireless Sensor Networks Sai Ji 1,, Liping Huang, Jin Wang, Jian
More informationNumerisches Rechnen. (für Informatiker) M. Grepl J. Berger & J.T. Frings. Institut für Geometrie und Praktische Mathematik RWTH Aachen
(für Informatiker) M. Grepl J. Berger & J.T. Frings Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2010/11 Problem Statement Unconstrained Optimality Conditions Constrained
More informationFast Compressed Sensing Based High Resolution Image Reconstruction
International Journal of New Technology and Research (IJNTR) ISSN:244116, Volume2, Issue3, March 16 Pages 7176 Fast Compressed Sensing Based High Resolution Image Reconstruction Dr G Vijay Kumar Abstract
More informationFrom Maxent to Machine Learning and Back
From Maxent to Machine Learning and Back T. Sears ANU March 2007 T. Sears (ANU) From Maxent to Machine Learning and Back Maxent 2007 1 / 36 50 Years Ago... The principles and mathematical methods of statistical
More informationCyberSecurity Analysis of State Estimators in Power Systems
CyberSecurity Analysis of State Estimators in Electric Power Systems André Teixeira 1, Saurabh Amin 2, Henrik Sandberg 1, Karl H. Johansson 1, and Shankar Sastry 2 ACCESS Linnaeus Centre, KTHRoyal Institute
More informationNMR Measurement of T1T2 Spectra with Partial Measurements using Compressive Sensing
NMR Measurement of T1T2 Spectra with Partial Measurements using Compressive Sensing Alex Cloninger Norbert Wiener Center Department of Mathematics University of Maryland, College Park http://www.norbertwiener.umd.edu
More informationMPEG, the MP3 Standard, and Audio Compression
MPEG, the MP3 Standard, and Audio Compression Mark ilgore and Jamie Wu Mathematics of the Information Age September 16, 23 Audio Compression Basic Audio Coding. Why beneficial to compress? Lossless versus
More informationNoisy and Missing Data Regression: DistributionOblivious Support Recovery
: DistributionOblivious Support Recovery Yudong Chen Department of Electrical and Computer Engineering The University of Texas at Austin Austin, TX 7872 Constantine Caramanis Department of Electrical
More informationExact shapereconstruction by onestep linearization in electrical impedance tomography
Exact shapereconstruction by onestep linearization in electrical impedance tomography Bastian von Harrach harrach@math.unimainz.de Institut für Mathematik, Joh. GutenbergUniversität Mainz, Germany
More informationJPEG compression of monochrome 2Dbarcode images using DCT coefficient distributions
Edith Cowan University Research Online ECU Publications Pre. JPEG compression of monochrome Dbarcode images using DCT coefficient distributions Keng Teong Tan Hong Kong Baptist University Douglas Chai
More informationChapter 6. Orthogonality
6.3 Orthogonal Matrices 1 Chapter 6. Orthogonality 6.3 Orthogonal Matrices Definition 6.4. An n n matrix A is orthogonal if A T A = I. Note. We will see that the columns of an orthogonal matrix must be
More informationCurvelets A Surprisingly Effective Nonadaptive Representation For Objects with Edges
Curvelets A Surprisingly Effective Nonadaptive Representation For Objects with Edges Emmanuel J. Candès and David L. Donoho Abstract. It is widely believed that to efficiently represent an otherwise smooth
More informationDSL Spectrum Management
DSL Spectrum Management Dr. Jianwei Huang Department of Electrical Engineering Princeton University Guest Lecture of ELE539A March 2007 Jianwei Huang (Princeton) DSL Spectrum Management March 2007 1 /
More informationCompressive Acquisition of Sparse Deflectometric Maps
Compressive Acquisition of Sparse Deflectometric Maps Prasad Sudhakar*, Laurent Jacques*, Adriana Gonzalez Gonzalez* Xavier Dubois +, Philippe Antoine +, Luc Joannes + *: Louvain University (UCL), LouvainlaNeuve,
More informationDiscrete Optimization
Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.14.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 20150331 Todays presentation Chapter 3 Transforms using
More informationFINITE ELEMENT : MATRIX FORMULATION. Georges Cailletaud Ecole des Mines de Paris, Centre des Matériaux UMR CNRS 7633
FINITE ELEMENT : MATRIX FORMULATION Georges Cailletaud Ecole des Mines de Paris, Centre des Matériaux UMR CNRS 76 FINITE ELEMENT : MATRIX FORMULATION Discrete vs continuous Element type Polynomial approximation
More informationComputer Networks 56 (2012) 2049 2067. Contents lists available at SciVerse ScienceDirect. Computer Networks
Computer Networks 56 (212) 249 267 Contents lists available at SciVerse ScienceDirect Computer Networks journal homepage: www.elsevier.com/locate/comnet Structural analysis of network traffic matrix via
More informationIEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 5, MAY
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 5, MAY 2006 1071 WaveletDomain Approximation and Compression of Piecewise Smooth Images Michael B. Wakin, Student Member, IEEE, Justin K. Romberg, Member,
More informationBig Data & Scripting storage networks and distributed file systems
Big Data & Scripting storage networks and distributed file systems 1, 2, in the remainder we use networks of computing nodes to enable computations on even larger datasets for a computation, each node
More informationVOL. 2, NO. 4, May 2012 ISSN ARPN Journal of Science and Technology All rights reserved.
Analysis of Various Image Compression Techniques 1 G.M.Padmaja, 2 P.Nirupama 1 Senior Assistant Professor in CSE Dept, BVRIT 2 Associate Professor in CSE Dept, SIET 1 padmaja.gmp@gmail.com, 2 nirupama.cse1@gmail.com
More informationComputational Optical Imaging  Optique Numerique.  Deconvolution 
Computational Optical Imaging  Optique Numerique  Deconvolution  Winter 2014 Ivo Ihrke Deconvolution Ivo Ihrke Outline Deconvolution Theory example 1D deconvolution Fourier method Algebraic method
More informationCouple Dictionary Training for Image Superresolution
IEEE TRANSACTIONS ON IMAGE PROCESSING 1 Couple Dictionary Training for Image Superresolution Jianchao Yang, Student Member, IEEE, Zhaowen Wang, Student Member, IEEE, Zhe Lin, Member, IEEE, Scott Cohen,
More informationα = u v. In other words, Orthogonal Projection
Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v
More informationMonte Carlo Simulation
1 Monte Carlo Simulation Stefan Weber Leibniz Universität Hannover email: sweber@stochastik.unihannover.de web: www.stochastik.unihannover.de/ sweber Monte Carlo Simulation 2 Quantifying and Hedging
More informationPixel enlargement in highspeed camera image acquisition based on 3D sparse representations
Pixel enlargement in highspeed camera image acquisition based on 3D sparse representations A. Hirabayashi, N. Nogami, J. White Ritsumeikan University College of Information Science and Engineering Kusatsu,
More informationAn InformationTheoretic Approach to Distributed Compressed Sensing
An InformationTheoretic Approach to Distributed Compressed Sensing Dror Baron, Marco F. Duarte, Shriram Sarvotham, Michael B. Wakin and Richard G. Baraniuk Dept. of Electrical and Computer Engineering,
More informationInternational Doctoral School Algorithmic Decision Theory: MCDA and MOO
International Doctoral School Algorithmic Decision Theory: MCDA and MOO Lecture 2: Multiobjective Linear Programming Department of Engineering Science, The University of Auckland, New Zealand Laboratoire
More informationBIL Computer Vision Mar 5, 2014
BIL 719  Computer Vision Mar 5, 2014 Image pyramids Aykut Erdem Dept. of Computer Engineering Hacettepe University Image Scaling This image is too big to fit on the screen. How can we reduce it? How to
More informationSignal Processing for Speech Recognition
Signal Processing for Speech Recognition Once a signal has been sampled, we have huge amounts of data, often 20,000 16 bit numbers a second! We need to find ways to concisely capture the properties of
More informationSparse Signal Recovery: Theory, Applications and Algorithms
Sparse Signal Recovery: Theory, Applications and Algorithms Bhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego Collaborators: I. Gorodonitsky, S. Cotter,
More informationImage Compression through DCT and Huffman Coding Technique
International Journal of Current Engineering and Technology EISSN 2277 4106, PISSN 2347 5161 2015 INPRESSCO, All Rights Reserved Available at http://inpressco.com/category/ijcet Research Article Rahul
More informationSparsity & Co.: An Overview of Analysis vs Synthesis in LowDimensional Signal Models. Rémi Gribonval, INRIA Rennes  Bretagne Atlantique
Sparsity & Co.: An Overview of Analysis vs Synthesis in LowDimensional Signal Models Rémi Gribonval, INRIA Rennes  Bretagne Atlantique remi.gribonval@inria.fr http://www.irisa.fr/metiss/members/remi/talks
More informationA NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION
1 A NEW LOOK AT CONVEX ANALYSIS AND OPTIMIZATION Dimitri Bertsekas M.I.T. FEBRUARY 2003 2 OUTLINE Convexity issues in optimization Historical remarks Our treatment of the subject Three unifying lines of
More information7 Gaussian Elimination and LU Factorization
7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method
More informationModelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches
Modelling, Extraction and Description of Intrinsic Cues of High Resolution Satellite Images: Independent Component Analysis based approaches PhD Thesis by Payam Birjandi Director: Prof. Mihai Datcu Problematic
More informationProbabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014
Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about
More information