Next generation sequencing (NGS) Bioinformatics Challenges and strategies Urmi Trivedi Lead Bioinformatician urmi.trivedi@ed.ac.uk
Major Bottlenecks Data volume Data complexity Data noise Overview Solutions Data formats Levels of NGS Bioinformatics Analytical strategies 2
Imbalance in genome informatics ecosystem Stein Genome Biology 2010 11:207 doi:10.1186/gb-2010-11-5-207 3
Hierarchy of NGS Data Volume Individual features 1MB Variation Data 4-5 GB Alignment Data 600GB Sequence plus Quality Data 1TB Intensities and Raw Data 5 TB Typical output from a single flowcell of a hiseq run 5
Hierarchy of NGS Data Volume Scalable storage Maintenance Clear as you go Individual features 1MB Backup Network Variation Data 4-5 GB Streamed replication of the original data, with copies being stored at multiple location Alignment Data 600GB Deposit data in public data repositories like ENA (European Nucleotide Archive) High Speed Network Sequence plus Quality Data 1TB Intensities and Raw Data 5 TB High performance computing such as Cluster computing or cloud computing Typical output from a single flowcell of a hiseq run Analysis Paralysis 5
Large amount of intermediate files Bears no discernable relationship to experiments >5000 intermediate files after image processing 6
Multiplexing adds up to it LIMS (Laboratory Information Management System) Automatic pipelines for running Tracking Samples programs Running downstream analysis A flowcell with only 3 bacrodes in 2 lanes produces ~14000 files 7
Data Noise Base-pair quality score Adapter contamination Quality Control Uneven Amplification 8
Data Formats SFF A binary file containing information about flowgrams, sequences, qualities FASTQ Contains sequence by cycle and respective quality 9
Data Formats FASTA Fasta header and the sequence >HWI-EAS222_2093MAAXX GAAATATTAAGTCTTTCAAA QUAL Fasta header and phred scores >HWI-EAS222_2093MAAXX 40 40 40 40 40 40 40 40 40 40 FASTQ Sequence and ASCII coded phred qualities @HWI-EAS222_205JYAAXX GATTTCTTTGTCATTATTTA + IIIIIIIIIIIIIIIIIIIIIIIIIIIII 10
Levels of NGS Bioinformatics Production Bionformatics Advanced Bioinformactics Produce raw sequence reads and QC Map to genome and generate raw genomic features (e.g. SNPs), Assemble a genome de novo with existing tools, Bioinformatics Research Analyze the data; Uncover the biological meaning 11
Production Bioinformatics Vendor s pipeline Generation of fastq or similar files QC scripts L I M S >>Per sequence quality scores pass #Quality Count 2 96325.0 3 4392.0 4 7924.0 5 7229.0 6 12586.0 7 20861.0 8 22431.0 9 26053.0 10 35403.0 11 40341.0 12 46845.0 13 56089.0 14 63524.0 15 67926.0 FAIL PASS Offsite backup Further investigation Advanced Bioinformatics 12
Advanced Bioinformatics Existing Reference Sequence No Reference Sequence Short Read Alignment De novo Assembly Variant Calling Gene Expression sirna/microrna Analysis De novo Transcriptome Assembly Metagenomics Population Genomics 13
Advanced Bioinformatics Software/Tools Open source tools Free for use Mostly Linux based Runs on command line Complicated Installation at times Commercial software Tools for biologists Pretty interface and ease of use CLCBio, Geneious, DNAStar, Partek 14
Short Read Alignment: Challenges Speed Using tools like BLAST/BLAT would require 100 CPU hours Memory Read Errors Repetitive regions Sequencer Differences 15
ELAND MAQ BWA BOWTIE TOPHAT GSNAP SOAP-2 Novoalign Short Read Alignment: Software 16
Variant calling Reads Align Reference Genome SNP 17
Variant Calling Misalignment due to Indels 18
Variant Calling Indel Realignment GATK MSA 19
Variant Calling -Workflow Raw data Alignment (SAM/BAM format) Realignment to correct errors GATK ELAND, MAQ, BWA, BOWTIE, SSAHA2, SOAP-2 PICARD, SAMTOOLS Mark duplicates Validation, Visualization and Bioinformatics Research Annotations SAMTOOLS, GATK, VarScan SNPs/indel calling IGV, Savant, Tablet 20
S1 Gene Expression Analysis Reads (cdna fragments) S2 Align Aligned read counts as a measure of gene expression 21
Gene Expression Analysis -Workflow Raw data Alignment (SAM/BAM format) Mark duplicates? TOPHAT, GSNAP, STAMPY, BWA, BOWTIE PICARD, SAMTOOLS Validation & Bioinformatics Research Normalization and Differential Gene Expression Raw Counts EdgeR, DeSEQ 22
Velvet ABYSS ALLPATHS-2 SOAPDenovo SGA EDENA CLCbio Newbler De novo Genome Assembly Software 23
De novo Genome Assembly Workflow Short reads (Typically 100bp paired end) Filter Poor quality data, sequence adapters, etc Assemble and generate contigs QC Long reads (e.g. 454, Sanger) Mate pairs (3-10KB insert) Generate Scaffolds QC Data Visualization (GMOD, Gbrowse,Tablet) Annotation (Gene Prediction, etc.) (MAKER, Augustus) 24
Summary Nextgen Sequencing is still very rapidly moving field Plan for change Keeping our infrastructure flexible Keep disk space expandable Keep software agile NEVER proceed with the analysis without data QC Choose the right tool for the right job 25
Acknowledgements Professor Mark Blaxter Dr. Karim Gharbi Dr. Stephen Bridgett Timothée Cézard Gaganjot Kaur Stuart Taylor The Darwin Trust of Edinburgh 26