Data Integration and Data Retrieval

Transcription

1 Data Integration and Data Retrieval Curso práctico de base de datos e integración de informacion biológica Segovia 4 de Julio de2007 Alberto Labarga EMBL-EBI

2 Overview Data and services at the EBI Integration models EB-Eye SRS Biomart Integrative access Web services REST SOAP How do I use web services?

3 EBI databases Genomes Nucleotides Proteins Structures Other molecules Interactions Experiments Literature Ontologies

4

5 Challenges of Data Integration Different types of data (sequence, function, literature etc.) Different data formats (FASTA, EMBL, Genbank, tab delimited etc.) Different storage formats (ASCII flatfile, XML, RDBMS) No standard formats for common fields (citations, descriptions, dates etc.) Volume and size of data 2007/6/30 5

6 EBI Services

7 Challenges when using tools in unison Manually transfer data from one application to another Understand disparate data formats Convert file formats where appropriate Manage and understand disparate application environments e.g. web browser, desktop application

8 EB-Eye Curso práctico de base de datos e integración de informacion biológica Segovia 4 de Julio de2007 Alberto Labarga EMBL-EBI

9 EB-Eye Approach to Data Integration Parse original data Reformat data if necessary Create searchable indices Extrapolate links between datasets Standardise field formats Unified interface regardless of data 2007/6/30 9

10 What is the data available? > 20 domains Ligand >137M entries > 550 Gb of data

11 What is the data available formats ID... AC... DT... ID... AC... DT... ID :.. PARENT ID :.. RANK :..... Ligand <XML>... </XML> <XML>... </XML> <XML>... </XML> <XML>... </XML> <XML>... </XML> <XML>... </XML> <XML>... </XML>

12 What is the data available sizes 81M 43M 4.2G 1G 8.4G 57Gb, >500 files 25K 6.3G 374Gb, >600 files

13 Points to take into consideration Our World A large amount of data A variety of file formats A variety of file sizes Data formats are changing Our Quest Index the data as fast as possible Add and configure a new domain easily Detect errors in the data Being up to date No downtime

14 Parsing and indexing different formats Flat files ID AF030562; SV 1; linear; genomic DNA; STS; FUN; 852 BP. AC AF030562; DT 04-DEC-1997 (Rel. 53, Created) DT 03-MAR-2000 (Rel. 62, Last updated, Version 2) XX DE Fusarium venenatum clone VEN-A RAPD band generated using Operon primer DE OPW-03, sequence tagged site.... EMBL grammar Taxonomy grammar UniProt grammar... ID XML files ID AF030562; SV 1; linear; genomic DNA; STS; FUN; 852 BP. <MedlineCitationSet> XX <MedlineCitation ACOwner="NLM" Status="MEDLINE"> <PMID> </PMID> <MedlineCitationSet> ID AC AF030562; <DateCreated> <Year>2000</Year> XX<MedlineCitation Owner="NLM" Status="MEDLINE"> <Month>10</Month> Creation date / Modification date <PMID> </PMID> <Day>04</Day> DT 04-DEC-1997 (Rel. 53, Created) </DateCreated> DT <DateCreated> 03-MAR-2000 (Rel. 62, Last updated, Version 2) XX <Year>1965</Year> Creation Date DE <Month>02</Month> Fusarium venenatum clone VEN-A RAPD band generated using Operon primer DE <Day>01</Day> OPW-03, sequence tagged site. XX</DateCreated> KW <DateCompleted> STS. XX <Year>1996</Year> Organism species Modification Date OS <Month>12</Month> Fusarium venenatum Organism classes OC <Day>01</Day> Eukaryota; Fungi; Ascomycota; Pezizomycotina; Sordariomycetes; OC </DateCompleted> Hypocreomycetidae; Hypocreales; mitosporic Hypocreales; Fusarium. XX<DateRevised> RN <Year>2007</Year> [1] <Month>03</Month> References RP RA <Day>01</Day> Yoder W.T., Christianson L.M.; RT </DateRevised> "Species-specific primers resolve members of the section Fusarium. RT <Article Taxonomic PubModel="Print"> status of the edible 'Quorn' fungus re-evaluated"; RL <Journal> Fungal Genet. Biol. 0:0-0(1997). issn XX <ISSN IssnType="Print"> </ISSN> Medline grammar RN [2] <JournalIssue CitedMedium="Print"> RP <Volume>10</Volume> References <PubDate> Parser volume InterPro grammar RA Yoder W.T., Christianson L.M.; RT ; <Year>1964</Year> RL Submitted <Month>Jul</Month> (ANTLR) (21-OCT-1997) to the EMBL/GenBank/DDBJ Dump databases. file grammar RL Microbiology, </PubDate> Novo Nordisk Biotech, Inc., 1445 Drew Ave., Davis, CA 95616, </JournalIssue> name RL USA XX <Title>Clinica chimica acta; international journal of clinical.. chemistry</title>. FH Key <ISOAbbreviation>Clin. Location/Qualifiers Chim. Acta</ISOAbbreviation> FH </Journal> FT. source FT... /organism="fusarium venenatum" Indexer FT /strain="atcc20334 FT /db_xref="taxon:56646"... Lucene API Description Db <database> <name>intact.experiment</name> <description>experimental procedures that allowed to </description> <release>1.0</release> <release_date>2007-feb-16</release_date> <entry_count>5697</entry_count> <entries> <entry id="ebi-77680"> Parser (ANTXR) Dump file (XML) Uniprot Index Embl Index Taxonomy Index

15 Divide and Conquer the Indexing UniProt (>4M entries) Embl (>83M entries) Taxonomy (>0.37M entries) Medline (>16M entries) GO (>0.23M entries) Others (ArrayExpress Ensembl, Intact, ) Db XML XML 2 files, ~9.4G >600 files ~ 375G 1 file, ~81M >500 files ~ 57G 1 file ~ 27M XML dump XML dump XML dump XML XML XML dump XML dump XML dump 8 cpu 8 cpu 8 cpu 8 cpu Uniprot Index Embl Index Taxonomy Index Medline Index Embl Index ArrayExpress Index Ensembl Index Intact Index

16 Let s put some figures on it 80,000,000 70,000,000 60,000,000 50,000,000 40,000,000 30,000,000 20,000,000 10,000,000 09:36:00 08:24:00 07:12:00 06:00:00 04:48:00 03:36:00 02:24:00 01:12:00 Entries Time (hours) 0 00:00:00 emblnew_std iprmatches embldeleted uniprot emblnew_wgs emblcds medlinerelease emblrelease_wgs emblrelease_std Less than 18 hours to index all the EBI

17 Web side story Load balancer Tomcat 1 Tomcat 2 Tomcat 3 Tomcat 4 UniProt Index Embl Index Taxonomy Index Medline Index ArrayExpress Index Ensembl Index Intact Index

18 Libraries Indexing Lucene ( ANTLR ( ANTXR ( JGroups ( Web Tomcat ( Spring Framework (

19 EB-eye Global search mechanism Searches most of the EBI resources in one go Not specific to any resource Unified searches of the EBI resources Free-text search (unified semantic) Basic results display (Google-like) Simple cross reference navigation Available on all the EBI web pages

20 EB-eye results summary page Organized into categories called domains Number of results per domain Refine your search Expand/Collapse for more details

21 EB-eye domain result page Results for all the resources in a domain A domain can contain several resources First 3 entries displayed for each resource View more entries for a particular resource Hierarchy of domains Forward search (smaller set of resources) Backward search (wider set of resources) Refine your search Navigate the results pages

22 EB-eye domain result page (one resource) Basic information: ID, name, description Link to the main resource web site Additional links EB-eye internal references

23 EB-eye cross-references navigation Navigate inside the EB-eye References context Navigation Using resources explicit references Using resources implicit references

24 EB-eye Advanced Search Accessible from all the pages Simple search criteria Domain specific search Domain selection Fields selection References

25 Exercices Find about protease but not about protease inhibitors proteases are classified into 4 large groups. Find out about cysteine and/or serine proteases but not about aspartic or metallo peptidases Find all uniprot entries regarding SARS but not those that are incomplete (are fragments) or a putative (no evidence they are what they pretend to be) and represent the spike glycoprotein.

26 Queries with SRS Curso práctico de base de datos e integración de informacion biológica Segovia 4 de Julio de2007 Alberto Labarga EMBL-EBI

27 Overview What is SRS? Data Integration Subentries Analysis Tools Search Terms SRS Web Interface Quick Searches Library Specific Searching Query History Linking Analysis Tools Databank Information and Index Browsing URL Queries 2007/6/30 27

28 What is SRS? Central resource for biological data Data integration and retrieval system more than 200 databases Databases are linked via common data Data analysis applications server more than 150 applications Run tool on query results or user supplied data Results linked to related data in other databases 2007/6/30 28

29 SRS Approach to Data Integration Parse original data Reformat data if necessary Create searchable indices Extrapolate links between datasets Standardise field formats Unified interface regardless of data 2007/6/30 29

30 Main page Tabs to main sections Quick simple search Tips and Aids Latest News Search with a list of IDs 2007/6/30 30

31 Databank Information 2007/6/30 31

32 Databank Information Information Data Fields 2007/6/30 32

33 Databank Information Browse Indices 2007/6/30 33

34 Databank Information Search term entered 2007/6/30 34

35 Databank Information 2007/6/30 35

36 Databank Information 2007/6/30 36

37 Quick Text Search 2007/6/30 37

38 SRS query language: index search Index queries can be targeted to specific data sources and to specific fields in them, or full entries (AllText) [database-field:searchterm], e.g, search all proteins in SwissProt with term "kinase" in the description field [swissprot-description:kinase] applicable database and field names can be found from the web-interface SRS is completely case-insensitive, this applies to search terms, database names, and field names Searchterms can be plain strings strings with wildcards regular expressions numeric range queries Searchterms can be combined with boolean operators Queries return sets of accession numbers or ids

39 SRS query language: expressions Wildcards: * matches any string,? matches any character Regular expressions are given within slashes /expr/ syntax similar to regular expressions in python (character classes []; grouping (); alternatives, repetitions *,+,?; beginning and end of word ^,*) Example: pattern /^[OPQ][0-9][A-Z0-9][A-Z0-9][A-Z0-9][0-9]$/ matches UniProt accession numbers Numeric ranges x:y (use hash-mark instead of colon between field name and search term) [medline-year#1999:2003] boundaries can be excluded with! in front of the boudary, e.g., 100:!200 means 100 and <200 either boundary may be omitted

40 SRS query language: operators Boolean operators: and(&), or( ), butnot(!) [swissprot-description:kinase!inhibitor] Linking [swissprot-description:kinase] > PATHWAY Searching multiple data sources (only field names common to all selected databases can be used) : [{swissprot embl}-description:kinase]

41 Query expressions Index queries (e.g., [embl-des:kinase!inhib*]) and full databases (e.g., pdb) are query expressions Query expressions can be combined with operators and grouped with parentheses to form more complex query expression The operators are: set intersection (&), union ( ), and subtraction (!) link-operators < and >

42 Quick Text Search 2007/6/30 42

43 List Search 2007/6/30 43

44 List Search 2007/6/30 44

45 Library Search Quick search input box Search types Databases 2007/6/30 45

46 Quick Search Search term entered UniProtKB Selected 2007/6/30 46

47 Quick Search 2007/6/30 47

48 Quick Search Search term entered UniProtKB selected Group expanded IPI selected 2007/6/30 48

49 Quick Search Results from multiple databases 2007/6/30 49

50 Standard Query Form Click on standard query form UniProtKB selected 2007/6/30 50

51 Standard Query Form Search options Input fields Predefined views Creating a view 2007/6/30 51

52 Standard Query Form Search terms Choosing fields 2007/6/30 52

53 Standard Query Form 2007/6/30 53

54 Standard Query Form Changed boolean logic 2007/6/30 54

55 Standard Query Form Changed view 2007/6/30 55

56 Standard Query Form 2007/6/30 56

57 Standard Query Form Change to list view Include sequence Choose sequence format 2007/6/30 57

58 Standard Query Form 2007/6/30 58

59 Standard Query Form - Subentries Subentry field 2007/6/30 59

60 Standard Query Form - Subentries 2007/6/30 60

61 Standard Query Form - Subentries 2007/6/30 61

62 Standard Query Form - Subentries 2007/6/30 62

63 Standard Query Form - Subentries Change type 2007/6/30 63

64 Standard Query Form - Subentries 2007/6/30 64

65 Standard Query Form - Subentries 2007/6/30 65

66 Extended Query Form Click on Extended query form UniProtKB selected 2007/6/30 66

67 Extended Query Form Special date field type User chosen fields Choose view type 2007/6/30 67

68 Extended Query Form 2007/6/30 68

69 Extended Query Form - Subentries 2007/6/30 69

70 Extended Query Form - Subentries 2007/6/30 70

71 Linking Link options Choose query Click link button 2007/6/30 71

72 Linking Databases to link to 2007/6/30 72

73 Linking 2007/6/30 73

74 Linking 2007/6/30 74

75 Linking 2007/6/30 75

76 Linking 2007/6/30 76

77 Views

78 Views

79 Query History Direct input of an SRS query Result options List of queries View options 2007/6/30 79

80 Rerun Query Choose query Choose view Click to rerun query 2007/6/30 80

81 Rerun Query 2007/6/30 81

82 Combine Queries Choose boolean logic to combine Click to combine Choose multiple queries 2007/6/30 82

83 Combine Queries 2007/6/30 83

84 Save Results Choose output to file or browser Choose output format 2007/6/30 84

85 Save Results 2007/6/30 85

86 Subentries Subentries are used when there is repeated structured information within an entry. Need to be able to search inside this structured information as if they were each isolated entries E.g. Searching for author Smith,A. publishing in Nature 2007/6/30 86

87 Subentries 2007/6/30 87

88 Subentries Another example features E.g. searching for key ACT_SITE and description phosphoserine 2007/6/30 88

89 Subentries 2007/6/30 89

90 BioMart Curso práctico de base de datos e integración de informacion biológica Segovia 4 de Julio de2007 Alberto Labarga EMBL-EBI

91 BioMart A collaboration Aim European Bioinformatics Institute (EBI) Cold Spring Harbor Laboratory (CSHL) To develop a generic data management system that works well for biology

92 Data model reversed star FK FK FK FK PK PK FK FK FK FK

93 Deployment Source data 1 Transformation

94 Transformation Tool

95 Deployment Source data 1 2 Transformation XML XML XML Configuration

96 Configuration Tool

97 Deployment Source data 1 2 Transformation XML XML XML Configuration 3 BioMart software Querying

98 Datasets, Attributes and Filters Mart Dataset GENE gene_id(pk) gene_stable_id gene_start gene_chrom_end chromosome gene_display_id description Attribute Filter

99 Data federation Dataset 1 Exportable name = uniprot_id attributes = uniprot_ac Links Dataset 2 Importable name = uniprot_id filters = uniprot_ac

100 Data federation Dataset 1 Exportable name=genomic_region attributes=chr_name, chr_start, chr_end Links Dataset 2 Importable name=genomic_region filters=chr_name (=), chr_start (>=), chr_end (<=)

101 Data Flow Desktop GUI Mart XML XML XML XML XML JAVA PERL Command line Web GUI Web Service

102 MartView

103 Examples Retrieve all SNPs for novel human G-protein coupled receptor genes (GPCRs IPR000276) on chromosome 2. Retrieve the sequences of the exons of the human MEFV gene in FASTA format. Retrieve the gene structure (i.e. start and end coordinates of exons) of the mouse gene ENSMUSG Retrieve all human disease genes containing transmembrane domains located between p11.2 and q22. The file contains a list of probeset IDs from a microarray experiment using the Affymetrix array HG-U133 Plus 2.0 (human). Retrieve the 500 bp upstream of the transcripts matching these probeset IDs. Retrieve the sequences 5kb upstream of all human known genes between D1S2806 and D1S464. Retrieve all human SNPs that have an ID from The SNP Consortium (TSC), from chromosome 6 between 15 Mb and 15.2 Mb, with 200 bases flanking sequence. Retrieve the mouse homologues of Homo sapiens genes CASP1, CASP2, CASP3, and CASP4.

104 Distributed Annotation System Curso práctico de base de datos e integración de informacion biológica Segovia 4 de Julio de2007 Alberto Labarga EMBL-EBI

105 Distributed Annotation System Developed in 1999/2000 by Lincoln Stein (CSHL) and collaborators as a GFF-based web service. Biodas.org ( A client-server system in which a single client integrates information from multiple servers. It allows a single machine to gather up genome annotation information from multiple distant websites, collate the information, and display it to the user in a single view.

106 General Feature Format Genome annotations, especially structural annotations, are often stored in GFF (General Feature Format) files. GFF is a simple tab-delimited data format. Fields are: <seqname> <source> <feature> <start> <end> <score> <strand> <frame> [attributes] [comments] See also:

107 DAS Concept Annotation server A Annotation server B Annotation server C Client Reference server

108 DAS Registry

109 DAS Server DAS request to retrieve the data sources on a DAS server: Result:

110 DAS Server DAS request to retrieve features on a segment: segment=1:1, Result:

111 Das viewer

112 Links

113 Web services Curso práctico de base de datos e integración de informacion biológica Segovia 4 de Julio de2007 Alberto Labarga EMBL-EBI

114 RESTful web services GET, POST HTML,XML,PNG REST: REpresentational State Transfer

115 SOAP services SOAP: Simple Object Access Protocol fetchdata(uniprot,wap_rat,default,xml)

116 SOAP based architecture

117 WSDL structure <wsdl:definitions> <wsdl:types> <schema/> </wsdl:types> <wsdl:message> <wsdl:porttype> <wsdl:operation/> </wsdl:porttype> <wsdl:binding /> <wsdl:service /> </wsdl:definitions>

118 Messages <wsdl:message name="fetchdatarequest"> <wsdl:part name="query" type="xsd:string"/> <wsdl:part name="format" type="xsd:string"/> <wsdl:part name="style" type="xsd:string"/> </wsdl:message> <wsdl:message name="fetchdataresponse"> <wsdl:part name="fetchdatareturn" type="xsd:string"/> </wsdl:message>

119 PortType <wsdl:porttype name="wsdbfetchserver"> <wsdl:operation name="fetchdata" parameterorder="query format style"> <wsdl:input message="impl:fetchdatarequest" name="fetchdatarequest"/> <wsdl:output message="impl:fetchdataresponse" </wsdl:operation> </wsdl:porttype> name="fetchdataresponse"/>

120 Binding <wsdl:binding name="wsdbfetchsoapbinding" type="impl:wsdbfetchserver"> <wsdlsoap:binding style="rpc transport=" <wsdl:operation name="fetchdata"> <wsdlsoap:operation soapaction=""/> <wsdl:input name="fetchdatarequest"> <wsdlsoap:body encodingstyle=" namespace=" use="encoded"/> </wsdl:input> <wsdl:output name="fetchdataresponse"> <wsdlsoap:body encodingstyle=" namespace=" use="encoded"/> </wsdl:output> </wsdl:operation> </wsdl:binding>

121 Service <wsdl:service name="wsdbfetchserverlegacyservice"> <wsdl:port binding="impl:wsdbfetchsoapbinding" name="wsdbfetch"> <wsdlsoap:address location=" </wsdl:port> </wsdl:service>

122 Accessing EBI Web services Curso práctico de base de datos e integración de informacion biológica Segovia 4 de Julio de2007 Alberto Labarga EMBL-EBI

123 Overview Technologies REST DAS SOAP (rpc/enc, doc/lit) Services Dbfetch EB-Eye SRS Martservice EBI Tools (Blast, InterProScan, ClustalW)

124 Well, that is a RESTful web service GET, POST HTML,XML,PNG REST: REpresentational State Transfer

125 Any web page is a web service

126 Friendly URL and XML documents

127 Available databases EMBL Nucleotide Sequence Database (embl) EMBL CDS (emblcds) EMBL-Contig (emblcon) EMBL Annotated Contigs (emblann) EMBL Sequence Version Archive (emblsva) Genome Reviews (genomereviews) Human Genic Bi-Allelic Sequences Database (hgvbase) NCBI Reference Sequence refseq, refseqp International Protein Index (ipi) Uniprotkb (uniprot) UniProt Reference Clusters uniref100,uniref50, uniref90 UniProtKB Sequence/Annotation Version Archive (unisave) UniProtKB Archive (uniparc) Protein Data Bank (pdb) Patent Protein Sequences epo_prt, uspto_prt, jpo_prt Integrated Resource of Protein Domains and Functional Sites (interpro) Medline (medline)

128 REST client use LWP::UserAgent; use URI::URL; my $ua= LWP::UserAgent->new(); my $u= my $req = GET URI::URL->new($u); my $resp = $ua->request($req); print $resp->content();

129 MartService Overview information(get) Marts Datasets Configuration Queries (POST).

130 MartService Overview (GET) Marts?type=registry Datasets?type=datasets&mart=mymart Configuration?type=configuration&dataset=mydataset

131 Web service <Query virtualschemaname="central_server_1"> <Dataset name="hsapiens_gene_ensembl" > <Attribute name="ensembl_gene_id"/> <Attribute name="ensembl_transcript_id"/> <Filter name="chromosome_name" value="1"/> <Filter name="band_end" value= p36.33"/> <Filter name="band_start" value= q44"/> </Dataset> <Dataset name="msd"> <Attribute name="pdb_id"/> <Attribute name= experiment_type"/> <Filter name="experiment_type" value= NMR"/> </Dataset> </Query>

132 SRS wgetz Usage: wgetz [-ew] [-page wwwpagename] [-id userid] [-uname uname] [-gname gname] [-permuserid permuserid] [-l liblist] [-lib libname] [-entry entryname] [-i2f] [-dbg] [-dbr] [-dbs] [-dbt] [-dbw] [-lo] [- info libinfo] [-qfrom queryfrom] [-launchfrom launchfrom] [-lfrom linkfrom] [-from fromlibname] [-to tolibname] [-f fieldlist] [-bv viewentriesstartn] [-lv viewentrieschunksize] [-lset listsetnumber] [-sf seqformat] [-q querystring] [-mime mimetype] [-qnum querynumber] [-enum entrynumber] [-snum sessionnumber] [-vt] [-vn viewnumber] [- view viewname] [-rs viewrecordsep] [-cs viewcolumnsep] [-bf browseindex] [-blib browselibs] [-ifile icarusfiletype] [-newid] [-appl appl] [-package package] [-version] [-codestrings] [-off] [-bioscout bioscoutid] [-jobname jobname] [-debug] [-ascii] [-ldrname loadername] [-sort sorton] [-sortdir sortdir] [-sqlimage sqlimage] [- pman projman] [-fman fromman] [-dlout] [-dlkey dlkey] [-nosession] 'queryexpression'

133 SRS wgetz -e+[swissprot-acc:p32234] -e+[swissprot-acc:p32234]+-ascii -e+[swissprot-acc:p32234]+-vn+1 (get IDs) -e+[swissprot-acc:p32234]+-vn+2 (get entries) -e+[swissprot-acc:p32234]+-vn+4 (get sequence)

134 SRS wgetz +-page+cresult+-ascii (count results) +-view+sequencesimple (get sequences) -permuserid+segovia2007 (use saved queries and views)

135 Perl client use LWP::Simple; my $srs=" my $id = "SEGOVIA2007"; my $query = "[interpro:*]"; my $view= "FAMILIAS"; # permuserid+segovia2007+-ascii+-view+familias+[interpro-id:] my $url = $srs?-permuserid+ $id+-ascii+-view+$view+$query"; print get $url;

136 DAS Perl client use LWP::Simple; use XML::Simple; $s=" $f=" $xml = new XML::Simple; my $features = $xml->xmlin(get $f); my $sequence = $xml->xmlin(get $s); print $sequence->{sequence}->{content}; my $list = $features->{gff}->{segment}->{feature}; foreach $feature (keys %$list){ print "$feature: "; print "$list->{$feature}->{start} $list->{$feature}->{end}\n"; }

137 SOAP services SOAP: Simple Object Access Protocol fetchdata(uniprot,wap_rat,default,xml)

138 wsdbfetch entry fetchdata (db, id, format, style)

139 wsdbfetch fetchdata dbname:id <output format> <output style> fetchbatch dbname 'id1;id2' <format> <style> getsupporteddbs getdbformats dbname getformatstyles dbname format

140 Perl client use SOAP::Lite; my $WSDL=' my $soap = SOAP::Lite->service($WSDL); # fetchdata dbname:id <format> <style> my $result = $soap->fetchdata( uniprot, default, raw ); die $soap->call->faultstring if $soap->call->fault; foreach my $i (@$result) { print "$i\n"; }

141 Python from SOAPpy import WSDL wsdl =" dbfetch = WSDL.Proxy(wsdl) resultlist = dbfetch.fetchdata("uniprot:slpi_human", "fasta", "raw") for result in resultlist: print result Ruby require 'soap/wsdldriver' wsdl = ' dbfetchsoap = SOAP::WSDLDriverFactory.new(wsdl).create_rpc_driver puts dbfetchsoap.fetchdata("uniprot:slpi_human", "fasta", "default")

142 EBI web services (global search) getresultsids (db, query) ids getresults (db, query) results

143 Web Services API: Metadata List available domains (i.e. available for query in EB-eye) String[] listdomains() > listdomains() 2can arrayexpress-experiments arrayexpress-genes arrayexpress-repository biomodels chebi emblcds embldeleted emblnew_con emblnew_standard emblnew_wgs emblrelease_con List available fields in a domain (e.g. id, description ) String[] listfields(string domain) > listfields( msdpdb ) id acc_number name description

144 Web Services API: Metadata Get referenced domains in a domain or an entry (i.e. EB-eye domains referenced in a domain or an entry) String[] getdomainsreferencedindomain(string domain) String[] getdomainsreferencedinentry(string domain, > getdomainsreferencedindomain( msdpdb ) String entryid) go intenz interpro medlinenew medlinerelease msdpdb taxonomy uniprot List additional databases referenced in a domain (i.e. non EB-eye domains referenced in a domain) > listadditionalreferencefields( msdpdb ) String[] listadditionalreferencefields(string domain) CATH PFAM SCOP

145 Web Services API: Basic Full-text Search Get number of results for a simple query int getnumberofresults(string domain, String query) > getnumberofresults( msdpdb, dopamine ) 18 List result IDs for a simple query String[] getresultsids(string domain, String query, int start, int size) > getresultsids( msdpdb, dopamine, 0, 3) List result fields values for a simple query String[][] getresults(string domain, String query, String[] fields, int start, int size) > getresults( msdpdb, dopamine, [ id, acc_number, name, description ], 0, 3) "60734" "2a3r" "CRYSTAL STRUCTURE OF HUMAN " "TRANSFERASE XRAY entry at resolution 2.6" "14833" "5pah" "HUMAN PHENYLALANINE " "MONOOXYGENASE XRAY entry at resolution 2.1" "12412" "1i15" "DOPAMINE D2 RECEPTOR " "SIGNALING PROTEIN TMODEL entry at resolution "

146 Web Services API: Entry Search Get result fields values for a particular entry String[] getentry(string domain, String entryid, String[] > fields) getentry( uniprot, NR1H2_HUMAN, [ id, acc_number, description ]) NR1H2_HUMAN P55055 Oxysterols receptor LXR-beta (Liver X receptor beta) (Nuclear orphanreceptor LXR-beta) Get result fields values for a list a entries String[][] getentries(string domain, String[] entryids, > getentries( uniprot, [ NR1H2_HUMAN, Q8BP65_MOUSE, Q8AXU8_CHICK ], String[] fields) > [ id, acc_number, description ]) "NR1H2_HUMAN" "P55055" "Oxysterols receptor LXR-beta (Liver X receptor beta) (Nuclear..." "Q8BP65_MOUSE" "Q8BP65" "8 days embryo whole body cdna, RIKEN full-length enriched... " "Q8AXU8_CHICK" "Q8AXU8" "Liver X receptor alpha."

147 Web Services API: Cross-references Search Get referenced entries for a domain in a particular entry String[] getreferencedentries(string domain, String > entryid, getreferencedentries String ( uniprot, referenceddomain) NR1H2_HUMAN, medlinerelease )

148 Eb-Eye perl client use SOAP::Lite; my $namespace = ' my $endpoint = ' my $search = SOAP::Lite-> uri($namespace)-> proxy($endpoint); my $result = $search->getresultsids("uniprot", "kinase", 0, 100); = $result->valueof('//getresultsidsresponse//out//string'); foreach $id (@ids){ print "$id\n"; }

149 EBI web services (analysis tools) run(params, data) jobid checkstatus (jobid) status getresults (jobid) results available poll (jobid, type) result file

150 Perl use SOAP::Lite; my $WSDL = ' my $fasta_client = SOAP::Lite->service($WSDL); my %params=(); $params{'program'}='fasta3'; $params{'database'}='uniprot'; $params{' '}='your@ .com'; $params{ async'}= 1; $data={type=>"sequence", content=>"mrcsislvlgllalevalarnlqehvfnsvqsmcsddsfsedteci"}; # $data={type=>"sequence", # content=> uniprot:slpi_human"}; my $jobid = $fasta_client >runfasta( SOAP::Data->name('params')->type(map=>\%params), SOAP::Data->name( content => [$data])); print $fasta_client->poll($jobid);

151 Perl client (cont.) # set a loop for checking job submission status # RUNNING, NOT_FOUND, ERROR, DONE my $status = $fasta_client ->checkstatus($jobid); while (status eq "RUNNING") { sleep 10; $status = $fasta_client->checkstatus($jobid); } # when job is done, poll for the results my $result = $fasta_client ->poll($jobid) if ($status eq "DONE") ; print $result;

152 Perl client (cont.) my $results = $fasta_client->getresults($jobid); die $soap->call->faultstring if $soap->call->fault; for $result (@$results){ # tooloutput, toolxml, toolaln, toolpng $res= $fasta_client->poll($jobid,$result->{type}); write_file($outfile.".".$result->{ext},$res); }

153 REST client use LWP::UserAgent; use use use URI:URL; $URL=" my %params=(); $params{'tool'}="iprscan"; $params{'sequence'}="uniprot:slpi_human"; $ua= LWP::UserAgent->new(); $submit_url= URI::URL->new($URL); $resp = $ua->request( POST $submit_url, 'Content' => \@args); print $resp->content();

154 Available Services Ensembl Ensembl (Martservice Martservice) (Martservice Martservice) Data Data Retrieval Retrieval (WSDbfetch WSDbfetch) (WSDbfetch WSDbfetch) Uniprot Uniprot API API Search Search (EB-Eye) (EB-Eye) Sequence Sequence homology homology WSWUBlast WSWUBlast WSFasta WSFasta WSMPsrch WSMPsrch WSScanPS WSScanPS Expression Expression Profiler Profiler Integr8 Integr8 API API Wise, Wise, PromoterWise, PromoterWise, ScanWise ScanWise Protein Protein families, families, motifs motifs and and domains domains (WSInterProScan WSInterProScan) (WSInterProScan WSInterProScan) Literature Literature and and text text mining mining (Whatizit Whatizit, (Whatizit Whatizit, CiteXplore) CiteXplore) Ontologies Ontologies (OntologyLookup ( OntologyLookup) (OntologyLookup ( OntologyLookup) Protein Protein structure structure & & function: function: (MSD (MSD API) API) DaliLite, DaliLite, Maxsprout Maxsprout ChEBI ChEBI API API Sequence Sequence alignment alignment WSClustalW WSClustalW WSMuscle WSMuscle WSTCoffee WSTCoffee Sequence Sequence analysis analysis (WSEmboss) (WSEmboss) ID ID mapping mapping (Picr ( Picr, (Picr ( Picr, Martservice) Martservice)

155 Text output

156 XML results

157 Documentation