LEAST-MEAN-SQUARE ADAPTIVE FILTERS

Size: px
Start display at page:

Download "LEAST-MEAN-SQUARE ADAPTIVE FILTERS"

Transcription

1 LEAST-MEAN-SQUARE ADAPTIVE FILTERS

2 LEAST-MEAN-SQUARE ADAPTIVE FILTERS Edited by S. Haykin and B. Widrow A JOHN WILEY & SONS, INC. PUBLICATION

3 This book is printed on acid-free paper. Copyright q 2003 by John Wiley & Sons Inc. All rights reserved. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) , fax (978) Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, New Jersey 07030, (201) , fax (201) , For ordering and customer service, call CALL-WILEY. Library of Congress Cataloging-in-Publication Data: Least-mean-square adaptive filters/edited by S. Haykin and B. Widrow p. cm. Includes bibliographical references and index. ISBN (cloth) 1. Adaptive filters Design and construction Mathematics. 2. Least squares. I. Widrow, Bernard, II. Haykin, Simon, TK7872.F5L dc21 Printed in the United States of America

4 This book is dedicated to Bernard Widrow for inventing the LMS filter and investigating its theory and applications Simon Haykin

5 CONTENTS Contributors Introduction: The LMS Filter (Algorithm) Simon Haykin ix xi 1. On the Efficiency of Adaptive Algorithms 1 Bernard Widrow and Max Kamenetsky 2. Traveling-Wave Model of Long LMS Filters 35 Hans J. Butterweck 3. Energy Conservation and the Learning Ability of LMS Adaptive Filters 79 Ali H. Sayed and V. H. Nascimento 4. On the Robustness of LMS Filters 105 Babak Hassibi 5. Dimension Analysis for Least-Mean-Square Algorithms 145 Iven M. Y. Mareels, John Homer, and Robert R. Bitmead 6. Control of LMS-Type Adaptive Filters 175 Eberhard Hänsler and Gerhard Uwe Schmidt 7. Affine Projection Algorithms 241 Steven L. Gay 8. Proportionate Adaptation: New Paradigms in Adaptive Filters 293 Zhe Chen, Simon Haykin, and Steven L. Gay 9. Steady-State Dynamic Weight Behavior in (N)LMS Adaptive Filters 335 A. A. (Louis) Beex and James R. Zeidler vii

6 viii CONTENTS 10. Error Whitening Wiener Filters: Theory and Algorithms 445 Jose C. Principe, Yadunandana N. Rao, and Deniz Erdogmus Index 491

7 CONTRIBUTORS A. A. (LOUIS) BEEX, Systems Group DSP Research Laboratory, The Bradley Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA Department of Mechanical and Aerospace Engineering, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA ROBERT R. BITMEAD, Technische Universiteit Eindhoven, Faculteit Elektrotechniek, EH 5.29, Postbus 513, 5600 MB Eindhoven, Netherlands HANS BUTTERWECK, ZHE CHEN, Department of Electrical and Computer Engineering, CRL 102, McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada L8S 4K1 DENIZ ERDOGMUS, Computational NeuroEngineering Laboratory, EB 451, Building 33, University of Florida, Gainesville, FL Acoustics and Speech Research Department, Bell Labs, Room 2D-531, 600 Mountain Ave., Murray Hill, NJ STEVEN L. GAY, Institute of Communication Technology, Darmstadt University of Technology, Merckstrasse 25, D Darmstadt, Germany PROF. DR.-ING. EBERHARD HÄNSLER, BABAK HASSIBI, Department of Electrical Engineering, 1200 East California Blvd., M/C , California Institute of Technology, Pasadena, CA Department of Electrical and Computer Engineering, McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada L8S 4K1 SIMON HAYKIN, JOHN HOMER, School of Computer Science and Electrical Engineering, The University of Queensland, Brisbane 4072 Stanford University, David Packard Electrical Engineering, 350 Serra Mall, Room 263, Stanford, CA MAX KAMENETSKY, IVEN M. Y. MAREELS, Department of Electrical and Electronic Engineering, The University of Melbourne, Melbourne Vic 3010 ix

8 x CONTRIBUTORS V. H. NASCIMENTO, Department of Electronic Systems Engineering, University of São Paulo, Brazil JOSE C. PRINCIPE, Computational NeuroEngineering Laboratory, EB 451, Building 33, University of Florida, Gainesville, FL YADUNANDANA N. RAO, Computational NeuroEngineering Laboratory, EB 451, Building 33, University of Florida, Gainesville, FL Department of Electrical Engineering, Room A Engineering IV Bldg, University of California, Los Angeles, CA ALI H. SAYED, Institute of Communication Technology, Darmstadt University of Technology, Merckstrasse 25, D Darmstadt, Germany GERHARD UWE SCHMIDT, BERNARD WIDROW, Stanford University, David Packard Electrical Engineering, 350 Serra Mall, Room 273, Stanford, CA JAMES R. ZEIDLER, Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA 92092

9 INTRODUCTION: THE LMS FILTER (ALGORITHM) SIMON HAYKIN The earliest work on adaptive filters may be traced back to the late 1950s, during which time a number of researchers were working independently on theories and applications of such filters. From this early work, the least-mean-square ðlmsþ algorithm emerged as a simple, yet effective, algorithm for the design of adaptive transversal (tapped-delay-line) filters. The LMS algorithm was devised by Widrow and Hoff in 1959 in their study of a pattern-recognition machine known as the adaptive linear element, commonly referred to as the Adaline [1, 2]. The LMS algorithm is a stochastic gradient algorithm in that it iterates each tap weight of the transversal filter in the direction of the instantaneous gradient of the squared error signal with respect to the tap weight in question. Let ^wðnþ denote the tap-weight vector of the LMS filter, computed at iteration (time step) n. The adaptive operation of the filter is completely described by the recursive equation (assuming complex data) ^wðn þ 1Þ ¼ ^wðnþþmuðnþ½dðnþ ^w H ðnþuðnþš*; ð1þ where uðnþ is the tap-input vector, dðnþ is the desired response, and m is the step-size parameter. The quantity enclosed in square brackets is the error signal. The asterisk denotes complex conjugation, and the superscript H denotes Hermitian transposition (i.e., ordinary transposition combined with complex conjugation). Equation (1) is testimony to the simplicity of the LMS filter. This simplicity, coupled with desirable properties of the LMS filter (discussed in the chapters of this book) and practical applications [3, 4], has made the LMS filter and its variants an important part of the adaptive signal processing kit of tools, not just for the past 40 years but for many years to come. Simply put, the LMS filter has withstood the test of time. Although the LMS filter is very simple in computational terms, its mathematical analysis is profoundly complicated because of its stochastic and nonlinear nature. Indeed, despite the extensive effort that has been expended in the literature to xi

10 xii INTRODUCTION: THE LMS FILTER (ALGORITHM) analyze the LMS filter, we still do not have a direct mathematical theory for its stability and steady-state performance, and probably we never will. Nevertheless, we do have a good understanding of its behavior in a stationary as well as a nonstationary environment, as demonstrated in the chapters of this book. The stochastic nature of the LMS filter manifests itself in the fact that in a stationary environment, and under the assumption of a small step-size parameter, the filter executes a form of Brownian motion. Specifically, the small step-size theory of the LMS filter is almost exactly described by the discrete-time version of the Langevin equation 1 [3]: Dn k ðnþ ¼n k ðn þ 1Þ n k ðnþ ¼ ml k n k ðnþþf k ðnþ; k ¼ 1; 2;...; M; ð2þ which is naturally split into two parts: a damping force ml k n k ðnþ and a stochastic force f k ðnþ. The terms used herein are defined as follows: M ¼ order (i.e., number of taps) of the transversal filter around which the LMS filter is built l k ¼ kth eigenvalue of the correlation matrix of the input vector uðnþ, which is denoted by R f k ðnþ ¼kth component of the vector mq H uðnþe* o ðnþ Q ¼ unitary matrix whose M columns constitute an orthogonal set of eigerivectors associated with the eigenvalues of the correlation matrix R e o ðnþ ¼optimum error signal produced by the corresponding Wiener filter driven by the input vector uðnþ and the desired response dðnþ To illustrate the validity of Eq. (2) as the description of small step-size theory of the LMS filter, we present the results of a computer experiment on a classic example of adaptive equalization. The example involves an unknown linear channel whose impulse response is described by the raised cosine [3] 8 1 h n ¼ 2 1 þ cos 2p < ðn 2Þ ; n ¼ 1; 2; 3; W ð3þ : 0; otherwise where the parameter W controls the amount of amplitude distortion produced by the channel, with the distortion increasing with W. Equivalently, the parameter W controls the eigenvalue spread (i.e., the ratio of the largest eigenvaiue to the smallest eigenvalue) of the correlation matrix of the tap inputs of the equalizer, with the eigenvalue spread increasing with W. The equalizer has M ¼ 11 taps. Figure 1 presents the learning curves of the equalizer trained using the LMS algorithm with the step-size parameter m ¼ 0:0075 and varying W. Each learning curve was obtained by averaging the squared value of the error signal eðnþ versus the number of iterations n over an ensemble of 100 independent trials of the experiment. The 1 The Langevin equation is the engineer s version of stochastic differential (difference) equations.

11 INTRODUCTION: THE LMS FILTER (ALGORITHM) xiii Figure 1 Learning curves of the LMS algorithm applied to the adaptive equalization of a communication channel whose impulse response is described by Eq. (3) for varying eigenvalue spreads: Theory is represented by continuous well-defined curves. Experimental results are represented by fluctuating curves. continuous curves shown in Figure 1 are theoretical, obtained by applying Eq. (2). The curves with relatively small fluctuations are the results of experimental work. Figure 1 demonstrates close agreement between theory and experiment. It should, however, be reemphasized that application of Eq. (2) is limited to small values of the step-size parameter m. Chapters in this book deal with cases when m is large. REFERENCES 1. B. Widrow and M. E. Hoff, Jr. (1960). Adaptive Switching Circuits, IRE WESCON Conv. Rec., Part 4, pp B. Widrow (1966). Adaptive Filters I: Fundamentals, Rep. SEL (TR ), Stanford Electronic Laboratories, Stanford, CA. 3. S. Haykin (2002). Adaptive Filter Theory, 4th Edition, Prentice-Hall. 4. B. Widrow and S. D. Stearns (1985). Adaptive Signal Processing, Prentice-Hall.

Statistics for Experimenters

Statistics for Experimenters Statistics for Experimenters Design, Innovation, and Discovery Second Edition GEORGE E. P. BOX J. STUART HUNTER WILLIAM G. HUNTER WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION FACHGEBIETSBGCHEREI

More information

Analysis of Financial Time Series

Analysis of Financial Time Series Analysis of Financial Time Series Analysis of Financial Time Series Financial Econometrics RUEY S. TSAY University of Chicago A Wiley-Interscience Publication JOHN WILEY & SONS, INC. This book is printed

More information

Stability of the LMS Adaptive Filter by Means of a State Equation

Stability of the LMS Adaptive Filter by Means of a State Equation Stability of the LMS Adaptive Filter by Means of a State Equation Vítor H. Nascimento and Ali H. Sayed Electrical Engineering Department University of California Los Angeles, CA 90095 Abstract This work

More information

Enhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm

Enhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm 1 Enhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm Hani Mehrpouyan, Student Member, IEEE, Department of Electrical and Computer Engineering Queen s University, Kingston, Ontario,

More information

Fundamentals of Financial Planning and Management for mall usiness

Fundamentals of Financial Planning and Management for mall usiness E REPRE EUR A F A CE Fundamentals of Financial Planning and Management for mall usiness M.J. Alhabeeb ENTREPRENEURIAL FINANCE The first effective form of investment was realized when the primitive man

More information

Adaptive Equalization of binary encoded signals Using LMS Algorithm

Adaptive Equalization of binary encoded signals Using LMS Algorithm SSRG International Journal of Electronics and Communication Engineering (SSRG-IJECE) volume issue7 Sep Adaptive Equalization of binary encoded signals Using LMS Algorithm Dr.K.Nagi Reddy Professor of ECE,NBKR

More information

MANAGEMENT OF DATA IN CLINICAL TRIALS

MANAGEMENT OF DATA IN CLINICAL TRIALS MANAGEMENT OF DATA IN CLINICAL TRIALS Second Edition ELEANOR MCFADDEN Frontier Science, Ltd. Kincraig, Inverness-shire, Scotland WILEY-INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION MANAGEMENT OF

More information

ADAPTIVE ALGORITHMS FOR ACOUSTIC ECHO CANCELLATION IN SPEECH PROCESSING

ADAPTIVE ALGORITHMS FOR ACOUSTIC ECHO CANCELLATION IN SPEECH PROCESSING www.arpapress.com/volumes/vol7issue1/ijrras_7_1_05.pdf ADAPTIVE ALGORITHMS FOR ACOUSTIC ECHO CANCELLATION IN SPEECH PROCESSING 1,* Radhika Chinaboina, 1 D.S.Ramkiran, 2 Habibulla Khan, 1 M.Usha, 1 B.T.P.Madhav,

More information

4F7 Adaptive Filters (and Spectrum Estimation) Least Mean Square (LMS) Algorithm Sumeetpal Singh Engineering Department Email : sss40@eng.cam.ac.

4F7 Adaptive Filters (and Spectrum Estimation) Least Mean Square (LMS) Algorithm Sumeetpal Singh Engineering Department Email : sss40@eng.cam.ac. 4F7 Adaptive Filters (and Spectrum Estimation) Least Mean Square (LMS) Algorithm Sumeetpal Singh Engineering Department Email : sss40@eng.cam.ac.uk 1 1 Outline The LMS algorithm Overview of LMS issues

More information

www.wileyglobalfinance.com

www.wileyglobalfinance.com Wiley Global Finance is a market-leading provider of over 400 annual books, mobile applications, elearning products, workflow training tools, newsletters and websites for both professionals and consumers

More information

Background 2. Lecture 2 1. The Least Mean Square (LMS) algorithm 4. The Least Mean Square (LMS) algorithm 3. br(n) = u(n)u H (n) bp(n) = u(n)d (n)

Background 2. Lecture 2 1. The Least Mean Square (LMS) algorithm 4. The Least Mean Square (LMS) algorithm 3. br(n) = u(n)u H (n) bp(n) = u(n)d (n) Lecture 2 1 During this lecture you will learn about The Least Mean Squares algorithm (LMS) Convergence analysis of the LMS Equalizer (Kanalutjämnare) Background 2 The method of the Steepest descent that

More information

Analysis of Mean-Square Error and Transient Speed of the LMS Adaptive Algorithm

Analysis of Mean-Square Error and Transient Speed of the LMS Adaptive Algorithm IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 48, NO. 7, JULY 2002 1873 Analysis of Mean-Square Error Transient Speed of the LMS Adaptive Algorithm Onkar Dabeer, Student Member, IEEE, Elias Masry, Fellow,

More information

Lecture 5: Variants of the LMS algorithm

Lecture 5: Variants of the LMS algorithm 1 Standard LMS Algorithm FIR filters: Lecture 5: Variants of the LMS algorithm y(n) = w 0 (n)u(n)+w 1 (n)u(n 1) +...+ w M 1 (n)u(n M +1) = M 1 k=0 w k (n)u(n k) =w(n) T u(n), Error between filter output

More information

HUMAN RESOURCES MANAGEMENT FOR PUBLIC AND NONPROFIT ORGANIZATIONS

HUMAN RESOURCES MANAGEMENT FOR PUBLIC AND NONPROFIT ORGANIZATIONS HUMAN RESOURCES MANAGEMENT FOR PUBLIC AND NONPROFIT ORGANIZATIONS Essential Texts for Public and Nonprofit Leadership and Management The Handbook of Nonprofit Governance, by BoardSource Strategic Planning

More information

International Marketing Research

International Marketing Research International Marketing Research Third edition C. SAMUEL CRAIG and SUSAN P. DOUGLAS Leonard N. Stern School of Business, New York University Allie International Marketing Research Third edition Allie International

More information

By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

By choosing to view this document, you agree to all provisions of the copyright laws protecting it. This material is posted here with permission of the IEEE Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology's products or services Internal

More information

Modeling, Analysis, and Control of Dynamic Systems

Modeling, Analysis, and Control of Dynamic Systems Modeling, Analysis, and Control of Dynamic Systems Second Edition William J. Palm III University of Rhode Island John Wiley Sons, Inc. New York Chichester Weinheim Brisbane Singapore Toronto To Louise.

More information

A STUDY OF ECHO IN VOIP SYSTEMS AND SYNCHRONOUS CONVERGENCE OF

A STUDY OF ECHO IN VOIP SYSTEMS AND SYNCHRONOUS CONVERGENCE OF A STUDY OF ECHO IN VOIP SYSTEMS AND SYNCHRONOUS CONVERGENCE OF THE µ-law PNLMS ALGORITHM Laura Mintandjian and Patrick A. Naylor 2 TSS Departement, Nortel Parc d activites de Chateaufort, 78 Chateaufort-France

More information

Component Ordering in Independent Component Analysis Based on Data Power

Component Ordering in Independent Component Analysis Based on Data Power Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, Signals

More information

Final Year Project Progress Report. Frequency-Domain Adaptive Filtering. Myles Friel. Supervisor: Dr.Edward Jones

Final Year Project Progress Report. Frequency-Domain Adaptive Filtering. Myles Friel. Supervisor: Dr.Edward Jones Final Year Project Progress Report Frequency-Domain Adaptive Filtering Myles Friel 01510401 Supervisor: Dr.Edward Jones Abstract The Final Year Project is an important part of the final year of the Electronic

More information

Graph Analysis and Visualization

Graph Analysis and Visualization Graph Analysis and Visualization Graph Analysis and Visualization DISCOVERING BUSINESS OPPORTUNITY IN LINKED DATA Richard Brath David Jonker Graph Analysis and Visualization: Discovering Business Opportunity

More information

Analysis of Filter Coefficient Precision on LMS Algorithm Performance for G.165/G.168 Echo Cancellation

Analysis of Filter Coefficient Precision on LMS Algorithm Performance for G.165/G.168 Echo Cancellation Application Report SPRA561 - February 2 Analysis of Filter Coefficient Precision on LMS Algorithm Performance for G.165/G.168 Echo Cancellation Zhaohong Zhang Gunter Schmer C6 Applications ABSTRACT This

More information

NEURAL NETWORKS A Comprehensive Foundation

NEURAL NETWORKS A Comprehensive Foundation NEURAL NETWORKS A Comprehensive Foundation Second Edition Simon Haykin McMaster University Hamilton, Ontario, Canada Prentice Hall Prentice Hall Upper Saddle River; New Jersey 07458 Preface xii Acknowledgments

More information

The Filtered-x LMS Algorithm

The Filtered-x LMS Algorithm The Filtered-x LMS Algorithm L. Håkansson Department of Telecommunications and Signal Processing, University of Karlskrona/Ronneby 372 25 Ronneby Sweden Adaptive filters are normally defined for problems

More information

Advanced Signal Processing and Digital Noise Reduction

Advanced Signal Processing and Digital Noise Reduction Advanced Signal Processing and Digital Noise Reduction Saeed V. Vaseghi Queen's University of Belfast UK WILEY HTEUBNER A Partnership between John Wiley & Sons and B. G. Teubner Publishers Chichester New

More information

Appendix H: Control System Computational Aids

Appendix H: Control System Computational Aids E1BAPP08 11/02/2010 11:56:59 Page 1 Appendix H: Control System Computational Aids H.1 Step Response of a System Represented in State Space In this section we will discuss how to obtain the step response

More information

Adaptive Variable Step Size in LMS Algorithm Using Evolutionary Programming: VSSLMSEV

Adaptive Variable Step Size in LMS Algorithm Using Evolutionary Programming: VSSLMSEV Adaptive Variable Step Size in LMS Algorithm Using Evolutionary Programming: VSSLMSEV Ajjaiah H.B.M Research scholar Jyothi institute of Technology Bangalore, 560006, India Prabhakar V Hunagund Dept.of

More information

Numerical Methods for Engineers

Numerical Methods for Engineers Steven C. Chapra Berger Chair in Computing and Engineering Tufts University RaymondP. Canale Professor Emeritus of Civil Engineering University of Michigan Numerical Methods for Engineers With Software

More information

Art Direction for Film and Video

Art Direction for Film and Video Art Direction for Film and Video This page intentionally left blank Art Direction for Film and Video SECOND EDITION Robert L. Olson Focal Press Taylor & Francis Croup NEW YORK AND LONDON First published

More information

Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Scientific Computing: An Introductory Survey Chapter 10 Boundary Value Problems for Ordinary Differential Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign

More information

COVERS ALL TOPICS IN LEVEL I CFA EXAM REVIEW CFA LEVEL I FORMULA SHEETS

COVERS ALL TOPICS IN LEVEL I CFA EXAM REVIEW CFA LEVEL I FORMULA SHEETS 2016 CFA EXAM REVIEW COVERS ALL TOPICS IN LEVEL I LEVEL I CFA FORMULA SHEETS Copyright 2016 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published

More information

Effective Methods for Software and Systems Integration

Effective Methods for Software and Systems Integration Effective Methods for Software and Systems Integration Boyd L. Summers CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 CRC Press is an imprint of Taylor

More information

NANOCOMPUTING. Computational Physics for Nanoscience and Nanotechnology

NANOCOMPUTING. Computational Physics for Nanoscience and Nanotechnology NANOCOMPUTING Computational Physics for Nanoscience and Nanotechnology NANOCOMPUTING Computational Physics for Nanoscience and Nanotechnology James J Y Hsu National Cheng Kung University, Taiwan National

More information

This page has been left blank intentionally

This page has been left blank intentionally Project Governance This page has been left blank intentionally Project Governance Ralf Müller PM Concepts, Sweden Ralf Müller 2009 All rights reserved. No part of this publication may be reproduced, stored

More information

Praise for Agile Contracts

Praise for Agile Contracts Agile Contracts Praise for Agile Contracts Agile development is starting to become popular in Japan, though Japanese companies have used all in one contracts for the last three decades. In this movement,

More information

NONLINEAR TIME SERIES ANALYSIS

NONLINEAR TIME SERIES ANALYSIS NONLINEAR TIME SERIES ANALYSIS HOLGER KANTZ AND THOMAS SCHREIBER Max Planck Institute for the Physics of Complex Sy stems, Dresden I CAMBRIDGE UNIVERSITY PRESS Preface to the first edition pug e xi Preface

More information

Contents. Preface. xiii. Part I 1

Contents. Preface. xiii. Part I 1 Contents Preface xiii Part I 1 Chapter 1 Introduction to Frequency-Modulated Continuous-Wave 3 Radar 1.1 Brief History 3 1.2 Examples of Use of FMCW Radar 5 1.2.1 Radio Altimeters 5 1.2.2 Level-Measuring

More information

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York

CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not

More information

COMPUSCIENCE. Subject Coverage. File Type. Features Thesaurus None. Record Content. File Size. Coverage 1972-2002. Updates.

COMPUSCIENCE. Subject Coverage. File Type. Features Thesaurus None. Record Content. File Size. Coverage 1972-2002. Updates. Subject Coverage File Type Algorithms Artificial intelligence Computers and education Computer graphics, image processing, pattern recognition Computer systems organization Information systems Software

More information

Programming Interviews Exposed: Secrets to Landing Your Next Job

Programming Interviews Exposed: Secrets to Landing Your Next Job Programming Interviews Exposed: Secrets to Landing Your Next Job Preface.... xxv Introduction....xxix Chapter 1 Before the Search... 1 Chapter 2 The Job Application Process....9 Chapter 3 Approaches to

More information

System Identification for Acoustic Comms.:

System Identification for Acoustic Comms.: System Identification for Acoustic Comms.: New Insights and Approaches for Tracking Sparse and Rapidly Fluctuating Channels Weichang Li and James Preisig Woods Hole Oceanographic Institution The demodulation

More information

Probability and Statistics

Probability and Statistics Probability and Statistics Syllabus for the TEMPUS SEE PhD Course (Podgorica, April 4 29, 2011) Franz Kappel 1 Institute for Mathematics and Scientific Computing University of Graz Žaneta Popeska 2 Faculty

More information

Data Visualization. Principles and Practice. Second Edition. Alexandru Telea

Data Visualization. Principles and Practice. Second Edition. Alexandru Telea Data Visualization Principles and Practice Second Edition Alexandru Telea First edition published in 2007 by A K Peters, Ltd. Cover image: The cover shows the combination of scientific visualization and

More information

AN INTRODUCTION TO OPTIONS TRADING. Frans de Weert

AN INTRODUCTION TO OPTIONS TRADING. Frans de Weert AN INTRODUCTION TO OPTIONS TRADING Frans de Weert AN INTRODUCTION TO OPTIONS TRADING The Securities & Investment Institute Mission Statement: To set standards of professional excellence and integrity

More information

Applied Linear Algebra I Review page 1

Applied Linear Algebra I Review page 1 Applied Linear Algebra Review 1 I. Determinants A. Definition of a determinant 1. Using sum a. Permutations i. Sign of a permutation ii. Cycle 2. Uniqueness of the determinant function in terms of properties

More information

The p-norm generalization of the LMS algorithm for adaptive filtering

The p-norm generalization of the LMS algorithm for adaptive filtering The p-norm generalization of the LMS algorithm for adaptive filtering Jyrki Kivinen University of Helsinki Manfred Warmuth University of California, Santa Cruz Babak Hassibi California Institute of Technology

More information

Software and Hardware Solutions for Accurate Data and Profitable Operations. Miguel J. Donald J. Chmielewski Contributor. DuyQuang Nguyen Tanth

Software and Hardware Solutions for Accurate Data and Profitable Operations. Miguel J. Donald J. Chmielewski Contributor. DuyQuang Nguyen Tanth Smart Process Plants Software and Hardware Solutions for Accurate Data and Profitable Operations Miguel J. Bagajewicz, Ph.D. University of Oklahoma Donald J. Chmielewski Contributor DuyQuang Nguyen Tanth

More information

Dynamic Process Modeling. Process Dynamics and Control

Dynamic Process Modeling. Process Dynamics and Control Dynamic Process Modeling Process Dynamics and Control 1 Description of process dynamics Classes of models What do we need for control? Modeling for control Mechanical Systems Modeling Electrical circuits

More information

Core Curriculum to the Course:

Core Curriculum to the Course: Core Curriculum to the Course: Environmental Science Law Economy for Engineering Accounting for Engineering Production System Planning and Analysis Electric Circuits Logic Circuits Methods for Electric

More information

AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS

AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS Revised Edition James Epperson Mathematical Reviews BICENTENNIAL 0, 1 8 0 7 z ewiley wu 2007 r71 BICENTENNIAL WILEY-INTERSCIENCE A John Wiley & Sons, Inc.,

More information

Praise for Launch. Hands on and generous, Michael shows you precisely how he does it, step by step. Seth Godin, author of Linchpin

Praise for Launch. Hands on and generous, Michael shows you precisely how he does it, step by step. Seth Godin, author of Linchpin Praise for Launch Launch is your road map to success in an ever-changing world. Stelzner shows you how to enchant your customers so that they ll want to help you change the world. Guy Kawasaki, author

More information

Introduction to Engineering System Dynamics

Introduction to Engineering System Dynamics CHAPTER 0 Introduction to Engineering System Dynamics 0.1 INTRODUCTION The objective of an engineering analysis of a dynamic system is prediction of its behaviour or performance. Real dynamic systems are

More information

LMS is a simple but powerful algorithm and can be implemented to take advantage of the Lattice FPGA architecture.

LMS is a simple but powerful algorithm and can be implemented to take advantage of the Lattice FPGA architecture. February 2012 Introduction Reference Design RD1031 Adaptive algorithms have become a mainstay in DSP. They are used in wide ranging applications including wireless channel estimation, radar guidance systems,

More information

Introduction to Industrial Organization

Introduction to Industrial Organization Introduction to Industrial Organization The MIT Press. Cambridge, Massachusetts London, England Second printing, 2002 2000 Massachusetts Institute of Technology All rights reserved. No part of this book

More information

Using quantum computing to realize the Fourier Transform in computer vision applications

Using quantum computing to realize the Fourier Transform in computer vision applications Using quantum computing to realize the Fourier Transorm in computer vision applications Renato O. Violin and José H. Saito Computing Department Federal University o São Carlos {renato_violin, saito }@dc.uscar.br

More information

Empirical Model-Building and Response Surfaces

Empirical Model-Building and Response Surfaces Empirical Model-Building and Response Surfaces GEORGE E. P. BOX NORMAN R. DRAPER Technische Universitat Darmstadt FACHBEREICH INFORMATIK BIBLIOTHEK Invortar-Nf.-. Sachgsbiete: Standort: New York John Wiley

More information

BEYOND 401(k)S SMALL BUSINESS OWNERS

BEYOND 401(k)S SMALL BUSINESS OWNERS BEYOND 401(k)S FOR SMALL BUSINESS OWNERS A Practical Guide to Incentive, Deferred Compensation, and Retirement Plans JEAN D. SIFLEET John Wiley & Sons, Inc. BEYOND 401(k)S FOR SMALL BUSINESS OWNERS BEYOND

More information

ADAPTIVE CHANNEL EQUALIZER FOR WIRELESS COMMUNICATION SYSTEMS

ADAPTIVE CHANNEL EQUALIZER FOR WIRELESS COMMUNICATION SYSTEMS International Journal of Electronics and Communication Engineering (IJECE) ISSN(P): 2278-9901; ISSN(E): 2278-991X Vol. 2, Issue 5, Nov 2013, 159-166 IASET ADAPTIVE CHANNEL EQUALIZER FOR WIRELESS COMMUNICATION

More information

NICK SMITH AND ROBERT WOLLAN WITH CATHERINE ZHOU. John Wiley & Sons, Inc.

NICK SMITH AND ROBERT WOLLAN WITH CATHERINE ZHOU. John Wiley & Sons, Inc. NICK SMITH AND ROBERT WOLLAN WITH CATHERINE ZHOU John Wiley & Sons, Inc. Copyright # 2011 by Accenture, LLP. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously

More information

Parameter identification of a linear single track vehicle model

Parameter identification of a linear single track vehicle model Parameter identification of a linear single track vehicle model Edouard Davin D&C 2011.004 Traineeship report Coach: dr. Ir. I.J.M. Besselink Supervisors: prof. dr. H. Nijmeijer Eindhoven University of

More information

Cloudonomics: The Business Value of Cloud Computing

Cloudonomics: The Business Value of Cloud Computing Additional praise for Cloudonomics: The Business Value of Cloud Computing It is a business imperative to do more with less and do everything faster. Cloudonomics offers a much-appreciated framework for

More information

Integrated Reservoir Asset Management

Integrated Reservoir Asset Management Integrated Reservoir Asset Management Integrated Reservoir Asset Management Principles and Best Practices John R. Fanchi AMSTERDAM. BOSTON. HEIDELBERG. LONDON NEW YORK. OXFORD. PARIS. SAN DIEGO SAN FRANCISCO.

More information

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10

Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 10 Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 10 Boundary Value Problems for Ordinary Differential Equations Copyright c 2001. Reproduction

More information

Univariate and Multivariate Methods PEARSON. Addison Wesley

Univariate and Multivariate Methods PEARSON. Addison Wesley Time Series Analysis Univariate and Multivariate Methods SECOND EDITION William W. S. Wei Department of Statistics The Fox School of Business and Management Temple University PEARSON Addison Wesley Boston

More information

Least-Squares Intersection of Lines

Least-Squares Intersection of Lines Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a

More information

INTELLIGENT SYSTEMS, CONTROL, AND AUTOMATION: SCIENCE AND ENGINEERING

INTELLIGENT SYSTEMS, CONTROL, AND AUTOMATION: SCIENCE AND ENGINEERING Robotics International Series on INTELLIGENT SYSTEMS, CONTROL, AND AUTOMATION: SCIENCE AND ENGINEERING VOLUME 43 Editor Professor S. G. Tzafestas, National Technical University of Athens, Greece Editorial

More information

D-optimal plans in observational studies

D-optimal plans in observational studies D-optimal plans in observational studies Constanze Pumplün Stefan Rüping Katharina Morik Claus Weihs October 11, 2005 Abstract This paper investigates the use of Design of Experiments in observational

More information

Computer exercise 2: Least Mean Square (LMS)

Computer exercise 2: Least Mean Square (LMS) 1 Computer exercise 2: Least Mean Square (LMS) This computer exercise deals with the LMS algorithm, which is derived from the method of steepest descent by replacing R = E{u(n)u H (n)} and p = E{u(n)d

More information

Forecasting of Economic Quantities using Fuzzy Autoregressive Model and Fuzzy Neural Network

Forecasting of Economic Quantities using Fuzzy Autoregressive Model and Fuzzy Neural Network Forecasting of Economic Quantities using Fuzzy Autoregressive Model and Fuzzy Neural Network Dušan Marček 1 Abstract Most models for the time series of stock prices have centered on autoregressive (AR)

More information

Mathematical Modeling and Methods of Option Pricing

Mathematical Modeling and Methods of Option Pricing Mathematical Modeling and Methods of Option Pricing This page is intentionally left blank Mathematical Modeling and Methods of Option Pricing Lishang Jiang Tongji University, China Translated by Canguo

More information

E-Commerce Operations Management Downloaded from www.worldscientific.com -COMMERCE. by 37.44.207.139 on 06/15/16. For personal use only.

E-Commerce Operations Management Downloaded from www.worldscientific.com -COMMERCE. by 37.44.207.139 on 06/15/16. For personal use only. -COMMERCE O p e r a t i o n s M a n a g e m e n t 2nd Edition This page intentionally left blank -COMMERCE O p e r a t i o n s M a n a g e m e n t 2nd Edition Marc J. Schniederjans University of Nebraska-Lincoln,

More information

Adaptive Sampling Rate Correction for Acoustic Echo Control in Voice-Over-IP Matthias Pawig, Gerald Enzner, Member, IEEE, and Peter Vary, Fellow, IEEE

Adaptive Sampling Rate Correction for Acoustic Echo Control in Voice-Over-IP Matthias Pawig, Gerald Enzner, Member, IEEE, and Peter Vary, Fellow, IEEE IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 58, NO. 1, JANUARY 2010 189 Adaptive Sampling Rate Correction for Acoustic Echo Control in Voice-Over-IP Matthias Pawig, Gerald Enzner, Member, IEEE, and Peter

More information

Social Services Administration In Hong Kong

Social Services Administration In Hong Kong Social Services Administration In Hong Kong Tneoretical Issues ana Case Studies This page is intentionally left blank Social Services Administration In Hong Kong Theoretical Issues ana Case Studies Editors

More information

250325 - METNUMER - Numerical Methods

250325 - METNUMER - Numerical Methods Coordinating unit: 250 - ETSECCPB - Barcelona School of Civil Engineering Teaching unit: 751 - ECA - Department of Civil and Environmental Engineering Academic year: Degree: 2015 BACHELOR'S DEGREE IN GEOLOGICAL

More information

IMPROVED NETWORK PARAMETER ERROR IDENTIFICATION USING MULTIPLE MEASUREMENT SCANS

IMPROVED NETWORK PARAMETER ERROR IDENTIFICATION USING MULTIPLE MEASUREMENT SCANS IMPROVED NETWORK PARAMETER ERROR IDENTIFICATION USING MULTIPLE MEASUREMENT SCANS Liuxi Zhang and Ali Abur Department of Electrical and Computer Engineering Northeastern University Boston, MA, USA lzhang@ece.neu.edu

More information

Experiment 7: Familiarization with the Network Analyzer

Experiment 7: Familiarization with the Network Analyzer Experiment 7: Familiarization with the Network Analyzer Measurements to characterize networks at high frequencies (RF and microwave frequencies) are usually done in terms of scattering parameters (S parameters).

More information

BINOMIAL OPTION PRICING

BINOMIAL OPTION PRICING Darden Graduate School of Business Administration University of Virginia BINOMIAL OPTION PRICING Binomial option pricing is a simple but powerful technique that can be used to solve many complex option-pricing

More information

Fundamentals of Actuarial Mathematics

Fundamentals of Actuarial Mathematics Fundamentals of Actuarial Mathematics S. David Promislow York University, Toronto, Canada John Wiley & Sons, Ltd Contents Preface Notation index xiii xvii PARTI THE DETERMINISTIC MODEL 1 1 Introduction

More information

SOLVING LINEAR SYSTEMS

SOLVING LINEAR SYSTEMS SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis

More information

A Reliability Point and Kalman Filter-based Vehicle Tracking Technique

A Reliability Point and Kalman Filter-based Vehicle Tracking Technique A Reliability Point and Kalman Filter-based Vehicle Tracing Technique Soo Siang Teoh and Thomas Bräunl Abstract This paper introduces a technique for tracing the movement of vehicles in consecutive video

More information

Tableau Your Data! Fast and Easy Visual Analysis with Tableau Software. Daniel G. Murray. Second Edition

Tableau Your Data! Fast and Easy Visual Analysis with Tableau Software. Daniel G. Murray. Second Edition Tableau Your Data! Tableau Your Data! Fast and Easy Visual Analysis with Tableau Software Second Edition Daniel G. Murray Tableau Your Data! Fast and Easy Visual Analysis with Tableau Software Published

More information

TAGUCHI APPROACH TO DESIGN OPTIMIZATION FOR QUALITY AND COST: AN OVERVIEW. Resit Unal. Edwin B. Dean

TAGUCHI APPROACH TO DESIGN OPTIMIZATION FOR QUALITY AND COST: AN OVERVIEW. Resit Unal. Edwin B. Dean TAGUCHI APPROACH TO DESIGN OPTIMIZATION FOR QUALITY AND COST: AN OVERVIEW Resit Unal Edwin B. Dean INTRODUCTION Calibrations to existing cost of doing business in space indicate that to establish human

More information

Introduction to Matrix Algebra

Introduction to Matrix Algebra Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

More information

Predictive Analytics for Human Resources

Predictive Analytics for Human Resources Predictive Analytics for Human Resources Wiley & SAS Business Series The Wiley & SAS Business Series presents books that help senior-level managers with their critical management decisions. Titles in the

More information

NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS

NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS NEURAL NETWORK FUNDAMENTALS WITH GRAPHS, ALGORITHMS, AND APPLICATIONS N. K. Bose HRB-Systems Professor of Electrical Engineering The Pennsylvania State University, University Park P. Liang Associate Professor

More information

Machine Learning in Statistical Arbitrage

Machine Learning in Statistical Arbitrage Machine Learning in Statistical Arbitrage Xing Fu, Avinash Patra December 11, 2009 Abstract We apply machine learning methods to obtain an index arbitrage strategy. In particular, we employ linear regression

More information

Contending Economic Theories

Contending Economic Theories Contending Economic Theories i ii Contending Economic Theories Richard D. Wolff and Stephen A. Resnick The MIT Press Cambridge, Massachusetts London, England iii 2012 Massachusetts Institute of Technology

More information

Optimal Design of α-β-(γ) Filters

Optimal Design of α-β-(γ) Filters Optimal Design of --(γ) Filters Dirk Tenne Tarunraj Singh, Center for Multisource Information Fusion State University of New York at Buffalo Buffalo, NY 426 Abstract Optimal sets of the smoothing parameter

More information

Linear Threshold Units

Linear Threshold Units Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

More information

SALEM COMMUNITY COLLEGE Carneys Point, New Jersey 08069 COURSE SYLLABUS COVER SHEET. Action Taken (Please Check One) New Course Initiated

SALEM COMMUNITY COLLEGE Carneys Point, New Jersey 08069 COURSE SYLLABUS COVER SHEET. Action Taken (Please Check One) New Course Initiated SALEM COMMUNITY COLLEGE Carneys Point, New Jersey 08069 COURSE SYLLABUS COVER SHEET Course Title Course Number Department Linear Algebra Mathematics MAT-240 Action Taken (Please Check One) New Course Initiated

More information

VOLATILITY AND DEVIATION OF DISTRIBUTED SOLAR

VOLATILITY AND DEVIATION OF DISTRIBUTED SOLAR VOLATILITY AND DEVIATION OF DISTRIBUTED SOLAR Andrew Goldstein Yale University 68 High Street New Haven, CT 06511 andrew.goldstein@yale.edu Alexander Thornton Shawn Kerrigan Locus Energy 657 Mission St.

More information

Classification of Bad Accounts in Credit Card Industry

Classification of Bad Accounts in Credit Card Industry Classification of Bad Accounts in Credit Card Industry Chengwei Yuan December 12, 2014 Introduction Risk management is critical for a credit card company to survive in such competing industry. In addition

More information

Monetary Theory and Policy

Monetary Theory and Policy Monetary Theory and Policy Third Edition Carl E. Walsh The MIT Press Cambridge Massachusetts 6 2010 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in

More information

US News & World Report Best Undergraduate Engineering Programs: Specialty Rankings 2014 Rankings Published in September 2013

US News & World Report Best Undergraduate Engineering Programs: Specialty Rankings 2014 Rankings Published in September 2013 US News & World Report Best Undergraduate Engineering Programs: Specialty Rankings 2014 Rankings Published in September 2013 Aerospace/Aeronautical/Astronautical 2 Georgia Institute of Technology Atlanta,

More information

Multivariate Statistical Inference and Applications

Multivariate Statistical Inference and Applications Multivariate Statistical Inference and Applications ALVIN C. RENCHER Department of Statistics Brigham Young University A Wiley-Interscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim

More information

Rashad Moarref 1/5. Rashad Moarref. Postdoctoral Scholar in Aerospace Graduate Aerospace Laboratories Phone: (626) 395 4459

Rashad Moarref 1/5. Rashad Moarref. Postdoctoral Scholar in Aerospace Graduate Aerospace Laboratories Phone: (626) 395 4459 Rashad Moarref 1/5 Rashad Moarref Postdoctoral Scholar in Aerospace Graduate Aerospace Laboratories Phone: (626) 395 4459 California Institute of Technology E-mail: rashad@caltech.edu 1200 E California

More information

Hedging Illiquid FX Options: An Empirical Analysis of Alternative Hedging Strategies

Hedging Illiquid FX Options: An Empirical Analysis of Alternative Hedging Strategies Hedging Illiquid FX Options: An Empirical Analysis of Alternative Hedging Strategies Drazen Pesjak Supervised by A.A. Tsvetkov 1, D. Posthuma 2 and S.A. Borovkova 3 MSc. Thesis Finance HONOURS TRACK Quantitative

More information

A Method for Measuring Amplitude and Phase of Each Radiating Element of a Phased Array Antenna

A Method for Measuring Amplitude and Phase of Each Radiating Element of a Phased Array Antenna Electronics and Communications in Japan, Vol. 65-B, No. 5, 1982 A Method for Measuring Amplitude and Phase of Each Radiating Element of a Phased Array Antenna Seiji Mano and Takashi Katagi, Regular Members

More information

Network quality control

Network quality control Network quality control Network quality control P.J.G. Teunissen Delft Institute of Earth Observation and Space systems (DEOS) Delft University of Technology VSSD iv Series on Mathematical Geodesy and

More information

Lecture 6. Artificial Neural Networks

Lecture 6. Artificial Neural Networks Lecture 6 Artificial Neural Networks 1 1 Artificial Neural Networks In this note we provide an overview of the key concepts that have led to the emergence of Artificial Neural Networks as a major paradigm

More information