Probabilistic and Statistical Methods in Bioinformatics

Transcription

1 Probabilistic and Statistical Methods in Bioinformatics Course Director: Rovshan Sadygov, Ph.D. (Dept. of Biochemistry & Molecular Biology) Schedule: 15 weeks course 2 classes/week (total of 30 classes) 1.0 hour/class format will include lectures, exercises, programming sessions in R Target students: graduate students medical students researches who work with/process large scale, "omics" datasets. Background Recommendation: basic linear algebra - properties and operations on matrices, vectors basic probability, binomial, Poisson, hypergeometric, normal distributions familiarity with coding concepts in one of programming language Overview: Applications of high-throughput technologies to biological samples produce large amounts of data characterizing the current state of samples. Statistical bioinformatics plays important role in planning experiments for testing specific hypotheses, analyzing the results, evaluating statistical significance of the conclusions and in accepting/modifying the original hypothesis. Examples of data processing will be provided from proteomics experiments and standard databases available in R. Learning Objectives: learn basics of statistical bioinformatics apply statistical methods to analyze large scale biological datasets/appreciate random effects in biological observations learn how to develop statistical models to describe biological evaluate statistical models Syllabus Part 1 Introduction to Probability Class 1: Definition of Probability

2 Experiments and Events Naive definition of probability, properties of probability. Four basic samples, factorial, power, binomial, Bose-Einstein. Inclusion-exclusion theorem, matching problem. Independence of events, conditional probability, Simpson's paradox. Bayes' Rule. Class 2: Random Variables and Their Distributions, Expectations Random variables, discrete vs. continuous, PMF, PDF, CDF Expectation of a Random Variable, indicator random variable Expectations of Bernoulli, Binomial, Geometric random variables Hypergeometric Distribution, proof of Vardemonde identity Class 3: Discrete Random Variables Gambler's ruin problem, difference equation. Linearity of the Expectation. Introduction to Conditional Probability, Polya's urn, the Hardy Law Functions of Random Variables Moment Generating Functions Class 4: Continuous Random Variables PDF, Normal, Cauchy, Exponential, Beta, Gamma distributions. Joint, Conditional, Marginal Distributions. Covariance and Correlation Conditional Expectation Class 5: Laws of Large Numbers The Central Limit Theorem The Law of Large Numbers Using R to Compute Classical Distributions The equalities and inequalities of probability Class 6: Markov Chains Transition matrix, transition graph stationary distribution, irreducibility,

3 recurrence, transient states, persistence, aperiodicity, ergodicity. Part II Classical Statistics Class 7: Estimation and Inference Statistical Inference Prior and Posterior Distributions Bayes Estimators Maximum Likelihood Estimators Score Statistic, Information matrix, Entropy Sufficient Statistics Class 8: Sampling Distributions of Estimators Sampling Distribution of a Statistic The Chi-Square, Wishart's and t Distributions Confidence Intervals Comparing the means of Two Normal Distributions The F distribution Class 9: Nonparametric Methods, Categorical Data Test of goodness-of-fit. Contingency Tables Sign and Rank Tests Kolmogorov-Smirnov Test Class 10: Brief Introduction into Experimental Design Fisher's exact test Associate vs. causative experimental design Causative: Polio vaccine example Associative case study: US vs. Kirsten Gilbert Class 11-12: Linear Statistical Models The Method of Least Squares, Regression Statistical and Bayesian Inferences in Linear Regression The General Linear Model and Multiple Regression

4 Analysis of Variance Generalized Linear Models Class 13: Review of Linear Algebra Fundamental Theorem of Linear Algebra Spectral Theorem, QR decomposition Singular Value Decomposition Class 14: Algorithm Design Dynamic Programming Linear Programming Metropolis-Hastings algorithm Class 15: Using R for Statistical Modeling Introduction to R programming Matrix Algebra in R Statistical modeling in R Generalized Linear Models, Mixed Effects. Class 16: Midterm Exam Part 3: Machine Learning Class 16-17: Supervised Learning Classification/Learning Theory Supervised Learning Generative Models: a)non-probabilistic, Fisher s Discriminant analysis; b) Probabilistic, Linear and Quadratic Discriminant Analysis, Naïve Bayesian model; Discriminative models, Logistic Regression model.

5 Class 18-19: Support Vector Machines Convex optimization Langrangian, duality. Jensen's inequality Vapnik-Chervonenkis Dimension Support Vector Machines Class 20-21: Hidden Markovian Models Probability of Sequence Occurence Backward algorithm, forward algorithm Viterbi algorithm, Baum-Welch algorithm. Class 22: Unsupervised Learning Clustering k-means clustering Principle Component Analysis Class 23: Expectation Maximization Expectation Maximization M-step E-step Part 4: Bioinformatics of Proteomics and Genomics Class 24-26: Applications in Mass Spectrometry/Proteomics Probability models to matching sequences to mass spectra Generating protein identifications, SVM, EM, multinomial models Multiple Hypothesis testing corrections for p-values Spectral counting for protein quantification, Generalized Linear Models

6 Class 27-29: Applications in Genomics Data Structures, Hashing, Lists Genome Sequencing Sequence alignment Class 30: Final Exam Recommended readings: Classical Probability: a) An Introduction to Probability Theory and Its Applications, 3 rd Edition, W. Feller, b) Introduction to Probability Models, S. Ross, 7 th Edition; Classical Statistics: Probability and Statistics, 3 rd Edition, M. H. DeGroot, M. J. Schervish, Linear Algebra: Linear Algebra and Its Applications, 3 rd Edition, G. Strang, Machine Learning: a) The Elements of Statistical Learning, 2 nd Edition, T. Hastie, R. Tibshirani, J. Friedman; b) Pattern Recognition and Machine Learning, C. M. Bishop, Applications in Bioinformatics: Statistical Methods in Bioinformatics, W. J. Ewens and G. R. Grand, b) Bioinformatics, 2 nd Edition, P. Baldi, S. Brunak; For Learning R: The R Book, M. J. Crawley. This course is an introduction to the ideas and tools of probability calculus, statistical methods and machine learning techniques for bioinformatics processing of large scale biological datasets. It consists of four parts: basic probability calculus, statistical models, machine learning and applications in proteomics and genomics. We will also provide the necessary introduction into linear algebra, R programming and algorithmic design techniques. The course build concepts for machine learning and statistical models from probabilistic and linear algebraic bases. Sample spaces, conditioning, Bayes Rule, random variables, distributions, expectation, and Markov chains will be covered in the 1rst part of the course. Interesting examples such as the Matching problem, variations of Birthday Problem, Gambler s Ruin, Simpson's paradox, St. Petersburg Paradox, and Markov Chain examples are discussed in the context of probability modeling. Linear algebraic (matrix based) view of modeling for linear least squares, support vector machines and other statistical tools will be provided. The discussed topics in probability calculus, statistics and machine learning are later applied in the example of bioinformatics data processing. Specific examples of applications are in mass spectrometry based proteomics, genomic sequencing and sequence alignments. Future opportunities and current limitations will be critically addressed. In addition to the regular lecture sessions, supplementary sections may be scheduled to address issues related to R.