Statistical Analysis of Gene Expression Data With Oracle & R (- data mining)

Transcription

1 Statistical Analysis of Gene Expression Data With Oracle & R (- data mining) Patrick E. Hoffman Sc.D. Senior Principal Analytical Consultant pat.hoffman@oracle.com

2 Agenda (Oracle & R Analysis) Tools Loading Data Statistical Analysis Most Important Genes MDL Correlations t statistics, etc R Visualizations Predictive models? Other?

3 Analysis Tools Tools For DB * PLSQL development ODM (Oracle Data Miner) TOAD (From Quest) Jdeveloper (Free from Oracle) does Java & plsql development Enterprise Manager Console ( Oracle client) Managing the whole DB SQLPlus (command line sql or plsql) R Project Open source clone of S-Plus TextPad (editor)

4 The data Affy gene expression (old AML/ALL) 7129 genes 72 patients (combined train/test) Can be expanded to the new chips (50,000 genes)

5

6 Loading Data into Oracle DB? From flat file? SQL*Loader Oracle Warehouse Builder Production (auto sql*ldr) All types of data Oracle Data Miner (ODM) Quick & Easy From R-Project tables

7 Screen Shots ODM r to load csv file

8

9

10

11

12

13

14 Load Data From R to DB Set up ODBC driver (in Windows) Download RODBC pkg (from cran) Load CSV file Sqlsave routine to store in DB

15 R code send csv to db # you must install(download) the RODBC first # r script to load a csv file to Oracle db # install packages(rodbc to connect to a database) from CRAN (menu option) library(rodbc) # use standard microsoft odbc connection # set up a dsn name to connect to correct service # setup the channel to database, ODBC must be set up to connect to Oracle DB chan1 <- odbcconnect(dsn="") odbcgetinfo(chan1) #make sure channel is ok # load in a csv file filename <- "D:\\clients\\affy\\gs_EXPRESSION.csv" csv <- read.csv(filename) sqldrop(chan1, "GS_EXPRESSION", errors = TRUE) #drop the old table # load the csv file to a table in the Oracle db sqlsave(chan1, csv, tablename = "GS_EXPRESSION", rownames = FALSE,colnames = T, fast = F) # fast did not work on 10g

16 7000 genes? 1000 column DB Limit? Gone! Transactional Format or Nested Columns Convert flat file table to one of these formats

17 Analysis Pl/Sql Package affy affy_to_trans Flat to Transactional affy_ai Calculate MDL Attrib. Import. trans_to_affy Convert top gene to flat form. corr_genes Correlate genes to genes corr_cases Correlate cases to cases corr_target Correlate genes to target T statistic Calculate t-statistics anova Stats for Multiple Classes

18 Convert gene expression table to transactional format Load data into table GS_EXPRESSION Use Affy package exec affy.affy_to_trans('gs_expression'); OUTPUT - will be in the table AFFY_TRANS

19 Normal format

20 Transactional Format (72*700)

21 PLSQL Affy_to_Trans

22 ODM will give stats

23 And Histograms

24 For Transactional & Flat Tables

25 Histogram for One Gene

26 Attribute Importance Target is ALL or AML This is Minimum Distance Length (MDL) algorithm

27 ODM GUI or plsql API

28 Top Genes by MDL (no normalization)

29 Classification & Clustering (ODM) Transactional or Flat tables SVM, Naïve Bayes, Adaptive Bayes Java, Plsql, API Java based GUI Advanced K-means, Orthogonal Clustering Lift and ROC curves

30 10g Statistics & SQL Analytics Ranking functions rank, dense_rank, cume_dist, percent_rank, ntile Window Aggregate functions (moving and cumulative) Avg, sum, min, max, count, variance, stddev, first_value, last_value LAG/LEAD functions Direct inter-row reference using offsets Reporting Aggregate functions Sum, avg, min, max, variance, stddev, count, ratio_to_report Statistical Aggregates Correlation, linear regression family, covariance Linear regression Fitting of an ordinary-least-squares regression line to a set of number pairs. Frequently combined with the COVAR_POP, COVAR_SAMP, and CORR functions. Descriptive Statistics average, standard deviation, variance, min, max, median (via percentile_count), mode, group-by & roll-up DBMS_STAT_FUNCS: summarizes numerical columns of a table and returns count, min, max, range, mean, stats_mode, variance, standard deviation, median, quantile values, +/- 3 sigma values, top/bottom 5 values Correlations Pearson s correlation coefficients, Spearman's and Kendall's (both nonparametric). Cross Tabs Enhanced with % statistics: chi squared, phi coefficient, Cramer's V, contingency coefficient, Cohen's kappa Hypothesis Testing Student t-test, F-test, Binomial test, Wilcoxon Signed Ranks test, Chi-square, Mann Whitney test, Kolmogorov- Smirnov test, One-way ANOVA Distribution Fitting Kolmogorov-Smirnov Test, Anderson-Darling Test, Chi- Squared Test, Normal, Uniform, Weibull, Exponential Pareto Analysis (documented) 80:20 rule, cumulative results table

31 Analytical Functions with TX format Correlating cases with cases Correlating genes with genes Correlating genes with TARGET (all/aml) T- statistics ANOVA

32 Correlating Cases -- put caseid s to correlate in table aftran2 create view aftran2 as select distinct(caseid) from affy_trans; create table pcor1 as select a.caseid p1, b.caseid p2, corr(a.attr_value, b.attr_value) corr from affy_trans a, affy_trans b where a.caseid < b.caseid and a.attrib = b.attrib and a.caseid in (select * from aftran2) group by a.caseid, b.caseid having corr(a.attr_value, b.attr_value) >.93 or corr(a.attr_value, b.attr_value) < -.93;

33 199 cases correlated >.93

34 Correlating Genes select a.attrib p1, b.attrib g2, corr(a.attr_value, b.attr_value) corr from affy_trans a, affy_trans b where a.attrib < b.attrib and a.seqnum = b.seqnum and a.attrib = 'X95735_at group by a.attrib, b.attrib --- zyxin having corr(a.attr_value, b.attr_value) >.5 or corr(a.attr_value, b.attr_value) < -.5;

35 208 genes correlate with zyxin across 72 patients

36 Correlating Genes with TARGET select a.attrib p1, b.attrib g2, corr(a.attr_value, b.attr_value) corr from affy_trans a, affy_trans b where a.attrib < b.attrib and a.seqnum = b.seqnum and a.attrib = TARGET group by a.attrib, b.attrib --- AML/ALL having corr(a.attr_value, b.attr_value) >.5 or corr(a.attr_value, b.attr_value) < -.5;

37 57 genes correlate with TARGET

38 R Correlations library(rodbc) chan1 <- odbcconnect(dsn="") odbcgetinfo(chan1) #make sure channel is ok # get gene expression data from db, but drop gene name gs1 <- sqlquery(chan1, query = "create table gs1 as SELECT * FROM GS_EXPRESSION") gs1 <- sqlquery(chan1, query = "alter table gs1 drop column gene") gs <- sqlquery(chan1, query = "SELECT * FROM GS1") # get correlation matrix cm <- cor(gs,use="pairwise.complete.obs") # Write the new table to a file. write.table( cm, file="d:\\clients\\affy\\gs_cm.csv", append = FALSE, quote = FALSE, sep = ",", eol = "\n", na = "", dec = ".", row.names = T, col.names = T ) heatmap(cm, Rowv=NA, Colv=NA,symm=TRUE)

39 Case Correlation Matrix

40 eatmap from R Correlation Matrix

41 Other Statistics Corr_s - Spearman s rho correlation coef. Corr_k - Kendall's tau-b correlation coef. stats_t_test_indep - equal variance stats_t_test_indepu - unequal variance stats_one_way_anova

42 statistics for each gene (tx format) Create table t_stats as SELECT a.attrib, count(*) cnt,avg(a.attr_value) avg_atr, avg(b.attr_value) avg_trg, STATS_T_TEST_INDEP(b.attr_value, a.attr_value, 'STATISTIC') t_observed, STATS_T_TEST_INDEP(b.attr_value, a.attr_value)*7130 two_sided_p_value FROM affy_trans A, affy_trans b WHERE a.attrib in (select * from aftran1) and b.attrib = 'TARGET' and a.caseid = b.caseid group by a.attrib;

43 T-statistic output table (Bonferroni correction)

44 F-distribution one way ANOVA drop table anova; create table anova as SELECT a.attrib, count(*) cnt,avg(a.attr_value) avg_atr, avg(b.attr_value) avg_trg, STATS_ONE_WAY_ANOVA(b.attr_value, a.attr_value, 'F_RATIO') f_ratio, STATS_ONE_WAY_ANOVA(b.attr_value, a.attr_value, 'SIG')*7130 p_value FROM affy_trans A, affy_trans b WHERE a.attrib in (select * from aftran1) and b.attrib = 'TARGET' and a.caseid = b.caseid group by a.attrib;

45 Top genes by ANOVA

46 R for Plotting and Visualization #### get data and plot all variables a1 <- sqlquery(chan1, query = "select * from anova") plot(a1)

47 Scatterplot Matrix in R

48 Histograms of Expression Distribution #Generate Histograms of gene expression cases, with labels and cut offs pam1 = 40 #Bars pam2 = 5000 #Ceiling pam3 = #floor wx = 1000 #width hx = 714 #floor N <- ncol(csv) # gene expression data is in csv N = 3 # do only 3 histograms R <- nrow(csv) nom<- attr(csv, "names") par(mfrow = c(n-1,1)) b <- 0:pam1 c = pam1/(pam2 - pam3) b <- b/c b <- b + pam3 for( num in 2:N) { h1 <- csv[,num] h1[ h1>pam2] <- pam2 h1[ h1<pam3] <- pam3 h<- hist(h1,nclass=pam1,breaks=b, main = paste("histogram of", nom[num],"clamp at",pam2,pam3), xlab = nom[num], col=5) }

49 3 columns of expression dist.

50 Histogram no labels N <- ncol(csv) # number of columns pam1 = 40 #Bars pam2 = 5000 #Ceiling pam3 = #floor R <- nrow(csv) #make the break points b <- 0:pam1 c = pam1/(pam2 - pam3) b <- b/c b <- b + pam3 par(mfrow = c(n,1),mar=c(0,0,0,0)) # space on graph for( num in 2:N) { h1 <- csv[,num] h1[ h1>pam2] <- pam2 h1[ h1<pam3] <- pam3 h<- hist(h1,breaks=b, main = "", xlab = "",axes=f, col=5) }

51 ll 72 cases of expression dist.

52

53 More? Many other applications of R Bioconductor Many other applications of Oracle Other code is available Oracle Informatics Consulting

54 Life Sciences DM Workshop A one day onsite technical session educating organizations on how to leverage one of their most valuable assets to provide insight in the operations of their business, the behavioral patterns of their customers and hidden relationships found deep within corporate data that can have direct impact to the bottom line. Life Sciences DM Blueprint Life Sciences DM Insight Life Sciences DMQuickstart A documented technical roadmap providing the organization with the strategy to integrate and deploy Life Sciences technology. This includes recommendations based on feedback from the Life Sciences workshop focusing on source data preparation, mining methodologies and supporting architecture. A five day onsite engagement focused on providing a detailed analysis of the business problem, data preparation, model build and analysis and knowledge deployment extending the analysis of the Life Sciences workshop culminating with a technical roadmap with a strategy to integrate and deploy Life Sciences technology. A thirty day engagement focused on taking a business problem and transforming into a Life Sciences solution. This includes transforming the business problem, preparing e data, creation of the mining model and knowledge deployment. Upon completion, results will be delivered mapped to the initial business problem. Life Sciences DM Services A series of custom services focused on delivering Life Sciences methodologies and solutions to provide insight in the operations of their business, the behavioral patterns of their customers and hidden relationships found deep within corporate data that can have direct impact to the bottom line.

55 Life Science Informatics experience Gene expression analysis Sequence Analysis (blast exon/intron prediction) Clinical/Medical data analysis QSAR/Cheminformatics Isis,Molconz, Predictive Tox Animal Studies Protein analysis (arrays, Mass spec) Ontology's and Text Mining

56 Data Mining & Informatics Services Contact Richard Solari Contact Patrick Hoffman