Software Tools for Scientific Data Analysis and Visualization: And Bioconductor. Stowers Science Club

Size: px

Start display at page:

Download "Software Tools for Scientific Data Analysis and Visualization: And Bioconductor. Stowers Science Club"

Corey Hodge
8 years ago
Views:

1 Software Tools for Scientific Data Analysis and Visualization: And Bioconductor Stowers Science Club Earl F. Glynn Scientific Programmer Bioinformatics 20 May

2 Topics What is R? What is Bioconductor? Pros and Cons How to get R and Bioconductor? Why should a biologist care? 2

3 What is R? Calculator and Statistical Analysis Tool large number of built-in statistical and math functions Exploratory Data Analysis Tool descriptive statistics/graphics Graphics and Data Visualization Tool high-quality, customizable graphics Huge Library of Specialty Packages a growing number specifically for microarray data analysis Statistical Computing Language/Environment derivative of S and S-Plus languages from early 1980s 3

Data Analysis Tool descriptive statistics/graphics Graphics and Data Visualization Tool high-quality,

4 R Example R as a graphing calculator log(keq) Vs 1/T Temperature dependence of equilibrium constant > T <- c(478, 533, 588, 643) > Keq <- c(210, 73, 31, 16) > plot(1/t, log(keq), main = "log(keq) Vs 1/T") log(keq) /T Data Source: Hill & Petrucci, General Chemistry (2 nd ed), 1999, p

plot(1/t, log(keq), main = "log(keq) Vs 1/T") log(keq) 3.0 3.5 4.0 4.5 5.0 0.0016 0.

5 R Example R as a graphing calculator > fit <- lm(log(keq) ~ I(1/T)) > fit log(keq) Vs 1/T Call: lm(formula = log(keq) ~ I(1/T)) Coefficients: (Intercept) I(1/T) > abline(fit, col="red") log(keq) > coefficients(fit) (Intercept) I(1/T) > summary(fit) /T 5

111 > abline(fit, col="red") log(keq) 3.0 3.5 4.0 4.5 5.

6 R Example R as a graphing calculator Simple Model: log(keq) = a + b/t > fit <- lm(log(keq) ~ I(1/T)) Complex Model: log(keq) = a + b/t + c*log(t) + d*t + e*t 2 > T <- c(200, 225, 273, 300, 325) > Keq <- exp( /T *log(T) *T *T^2) > fit <- lm(log(keq) ~ I(1/T) + I(log(T)) + T + I(T^2)) > fit Call: lm(formula = log(keq) ~ I(1/T) + I(log(T)) + T + I(T^2)) Coefficients: (Intercept) I(1/T) I(log(T)) T I(T^2)

7 R Example Exploratory Data Analysis Use descriptive statistics to see big picture prior to formal analysis to examine data quality Need techniques that are robust to outliers Measures of Center: - Mean (normal distribution) - Median (skewed distribution) Measures of Spread: - Standard Deviation (SD) (appropriate with Mean) standardize: (X mean(x)) / sd(x) - Median Absolution Deviation (MAD) (appropriate with Median) standardize: (X median(x)) / mad(x) - Interquartile Range (appropriate with Median) 7

distribution) Measures of Spread: - Standard Deviation (SD) (appropriate with Mean) standardize: (X mean(x)) / sd(x) - Median

8 R Example Exploratory Data Analysis 12 Tukey s Five Number Summary Min Median Max Q Lower Hinge Interquartile Range (IQR) > x <- c(79,73,7,12,29,22,65,84,45,41,48,57,37) > fivenum(x) [1] Q3 79 Upper Hinge Source: John W. Tukey, Exploratory Data Analysis,

c(79,73,7,12,29,22,65,84,45,41,48,57,37) > fivenum(x) [1] 7 29 45 65 84 57 65

9 R Example Exploratory Data Analysis Five Number Summary Max Q3 Median IQR Q1 Min > x <- c(79,73,7,12, 29,22,65,84,45, 41,48,57,37) > boxplot(x, main= Five Number Summary ) box and whisker plot or simply a boxplot Visualize five-number summary with a boxplot: Minimum, Quartile 1, Median, Quartile 3, Maximum 9

main= Five Number Summary ) box and whisker plot or simply a boxplot Visualize

10 R Example Exploratory Data Analysis > RawData <- read.csv("complete_dataset.csv", as.is=true) > Expression <- log2( data.matrix(rawdata[,2:ncol(rawdata)])) > boxplot(data.frame(expression), main="bozdech 'Complete' Plasmodium Dataset", las=vertical<-3, cex.axis=0.7, ylab="log2 Expression Ratio") Bozdech 'Complete' Plasmodium Dataset TP1 TP2 TP3 TP4 TP5 TP6 TP7 TP8 TP9 TP10 TP11 TP12 TP13 TP14 TP15 TP16 TP17 TP18 TP19 TP20 TP21 TP22 TP23 TP24 TP25 TP26 TP27 TP28 TP29 TP30 TP31 TP32 TP33 TP34 TP35 TP36 TP37 TP38 TP39 TP40 TP41 TP42 TP43 TP44 TP45 TP46 TP47 TP48 Log2 Expression Ratio

7, ylab="log2 Expression Ratio") Bozdech 'Complete' Plasmodium Dataset TP1 TP2 TP3 TP4 TP5 TP6 TP7 TP8 TP9 TP10 TP11 TP12 TP13 TP14 TP15 TP16 TP17 TP18

11 R Example Exploratory Data Analysis > # Use Bioconductor package > library(arraymagic) > plot.imagematrix ( Expression, ylabels="", main="log2 Gene Expression in Plasmodium Dataset" ) 11

12 R Example Statistical Analysis: Evaluate Gene Expression for Periodicity > ShowSingleOligoProfileByName("i3518_1") i3518_1 Time Interval Variability Expression Frequency N = Time [hours] log10(delta T) Normalized Power Spectral Density Lomb-Scargle Periodogram Period at Peak = 45.7 hours p = 1e-06 p = 1e-05 p = 1e-04 p = p = 0.01 p = 0.05 Probability Peak Significance p = 1.48e-008 at Peak Frequency [1/hour] Frequency [1/hour] 12

0 Time [hours] log10(delta T) Normalized Power Spectral Density 0 5 10 15 20 25 Lomb-Scargle Periodogram Period at Peak = 45.

13 R Example Statistical Analysis: Evaluate Gene Expression for Periodicity > ShowSingleOligoProfileByName("j167_5") j167_5 Time Interval Variability Expression Frequency N = Time [hours] log10(delta T) Normalized Power Spectral Density Lomb-Scargle Periodogram Period at Peak = 17.8 hours Frequency [1/hour] p = 1e-06 p = 1e-05 p = 1e-04 p = p = 0.01 p = 0.05 Probability Peak Significance p = at Peak Frequency [1/hour] 13

0 Frequency 0 5 10 15 20 25 N = 35 0 10 20 30 40-1.0-0.5 0.0 0.5 1.0 Time [hours] log10(delta T) Normalized Power Spectral Density 0 5 10 15 20 25 Lomb-Scargle Periodogram Period at Peak = 17.

14 R Example Statistical Analysis: Multiple Hypothesis Testing Multiple Testing Correction Methods (Using R's p.adjust methods) p.adjust function in R stats package Log10(p) bonferroni holm hochberg fdr none α = Rank Order of Sorted p Values fdr = Benjamini and Hochberg s False Discovery Rate Method 14

adjust function in R stats package Log10(p) -8-6 -4-2 0 bonferroni holm hochberg fdr

15 R Example Statistical Analysis: Logic Regression Where L 1 and L 2 are Boolean expressions. Each L can be represented by logic tree. Logic Tree: L = (B C) A Ruczinksi, et al, (2003), Logic Regression, Journal of Computational and Graphical Statistics, 12(3), R Package: LogicRec 15

Logic Tree: L = (B C) A Ruczinksi, et al, (2003), Logic Regression,

16 R Example Data Visualization > library(scatterplot3d) > example(scatterplot3d) Volume 4 > example(layout) > set.seed(19) > x <- matrix(rnorm(200),10,20) > heatmap(x) scatterplot3d Girth Height > set.seed(19) > library(mass) > x <- rnorm(50) > y <- rnorm(50) > d <- kde2d(x,y) > image(d, col=terrain.colors(50)) > contour(d,add=t) > set.seed(19) > hist(rnorm(100),freq=f) > curve(dnorm(x), add=t, col="blue") Histogram of rnorm(100)

seed(19) > x <- matrix(rnorm(200),10,20) > heatmap(x) 11 8 7 3 5 1 4 2 10 6 9 10 20 30 40 50 60 70 80 scatterplot3d - 5 8 10 12 14 16 18 20 22 Girth 60 6570 75 8085

17 R Example concentration Customized Graphics ABC R plot subtitle Missing VeryLongName specimen Data Visualization XYZ # plot with error bars x <- c(1,2,3,4,5) y <- c(15,9,na,19,22) error <- c(3, 4, 1, 2.5, 0.5) plot(x,y, col="red", type="o", main="r plot", xaxt="n", xlab="specimen", ylab="concentration", ylim=c(0,25)) delta < * diff(par("usr")[1:2]) segments(x, y-error, x, y+error) segments(x-delta, y-error, x+delta, y-error) segments(x-delta, y+error, x+delta, y+error) names <- c("abc","12345","missing","verylongname","xyz") text(x, par("usr")[3] 0.01*diff(par("usr")[3:4]), srt=30, adj=1, labels=names, xpd=true) mtext("subtitle") 17

5) plot(x,y, col="red", type="o", main="r plot", xaxt="n", xlab="specimen", ylab="concentration", ylim=c(0,25)) delta <- 0.

18 R Example Data Visualization Graphics Notes R creates graphics as postscript, pdf, or in a variety of other formats. In Windows, copy and paste graphics as metafile to Word, PowerPoint, or other programs. In Windows, enable History, Recording in graphics window: Use PageUp/PageDown to step through graphics. In Word, save as Web page, filtered to make web page including GIF graphics with transparency. 18

In Windows, enable History, Recording in graphics window: Use PageUp/PageDown to step through

19 ~500 R Packages Most packages deal with data analysis, statistics, and visualization. Caution: Software quality varies. Validate first! 19

html Most packages deal with data analysis,

20 What is Bioconductor? Open Source Software for Bioinformatics Started in Fall 2001 at Harvard First Bioconductor Release in May 2002 ~100 R Packages Software categories: -Analysis (e.g., limma linear models for microarrays) -Annotation (e.g., Data packages ) -Database Interaction -Graphics & User Interface (e.g., limmagui ) -Graphs -Pre-processing -Ontologies (tools for working with gene ontologies) Web: 20

2002 ~100 R Packages Software categories: -Analysis (e.g., limma linear models for microarrays) -Annotation (e.

21 Bioconductor Example Limma: linear models for microarrays library(limma) # Adapted from?contrasts.fit # Simulate gene expression data: 6 microarrays and genes # with one gene differentially expressed in first 3 arrays. # contrasts.fit: Given a linear model fit to microarray data, # compute estimated coefficients and standard errors for a # given set of contrasts. set.seed(71) M <- matrix(rnorm(20000*6,sd=0.3),20000,6) M[1,1:3] <- M[1,1:3] + 2 # design matrix corresponds to oneway layout, # columns are orthogonal design <- cbind(first3arrays=c(1,1,1,0,0,0), Last3Arrays=c(0,0,0,1,1,1)) fit <- lmfit(m,design=design) # Would like to consider original two estimates plus # difference between first 3 and last 3 arrays contrast.matrix <- cbind(first3=c(1,0),last3=c(0,1), "Last3-First3"=c(-1,1)) fit2 <- contrasts.fit(fit,contrast.matrix) fit2 <- ebayes(fit2) # large values of eb$t indicate differential expression results <- classifytestsf(fit2) venndiagram( venncounts(results)) First3 Last Last3-First

22 Pros Powerful analysis tools Command line processing; Batch processing Graphics rich software Several revisions/year Fast (most tasks) Free and open source: UNIX/Windows/Apple Strong user community Help via mailing list R/Bioconductor Cons Can be quirky No GUI : Difficult to interact with data Graphics poor documentation Several revisions/year Slow (processing huge datasets) Correct way to ask One of the most intimidating things about R is the seeming endlessness of it. Paul E. Johnson, KU Political Science Dept, R-Help, 9 May

23 How to get R and Bioconductor? 23

24 How to get R and Bioconductor? 24

25 Resources Comprehensive R Archive Network (CRAN) R for Bioinformatics Nov 2005? SummeR Sessions? 25

26 Why should a biologist care? Excel has many limitations. R can serve as powerful graphing calculator. R can easily work with vectors and matrices with microarray data. State of the art analysis software often introduced in published papers using R. 26

27 Acknowledgements Bioinformatics us Arcady Mushegian Director Amy Ubben Admin Research Jie Chen Visiting Scientist Frank Emmert-Streib Galina Glasko Manisha Goel Piotr Kozbial Jing Liu Support Mike Coleman Scientific Programmer Malcolm Cook Database Applications Dan Thomasset UNIX Admin (IT) 27

28 Acknowledgements Microarrays Chris Seidel & Karen Zueckert-Gaudenz Pourquié Lab Mary-Lee Dequeant & Olivier Pourquié maps.google.com 28

STATS8: Introduction to Biostatistics. Data Exploration. Babak Shahbaba Department of Statistics, UCI

STATS8: Introduction to Biostatistics Data Exploration Babak Shahbaba Department of Statistics, UCI Introduction After clearly defining the scientific problem, selecting a set of representative members