Information Integration

Transcription

1 Information Integration Assignment 3: Consistency of Annotations (NCBI UniProtKB) Ulf Leser

2 Overview Reuse all existing data We want to integrate and compare functional annotation from another data source the UniProtKB UniProtKB can be accessed through several interfaces Your choice We don t want to produce a DDoS restrict comparison to a small region of a chromosome Therefore, we need to clean MAP_LOCATION information Finally :-)

3 Task 1: Data Cleansing MAP_LOCATION So far, we left MAP_LOCATION as it was, though we have seen many problems We go for a quick-and-dirty solution to keep things simple Set the field to NULL if the value is - Remove all parts after a Remove all parts after a ; Remove all parts after a - Remove all parts after a (blank) Perform the same changes with CHROMOSOME We leave the few inconsistencies between CHROMOSOME and MAP_LOCATION (for now) Produces 1:1 relationship between a gene and a chromosomal region In varying levels of granularity

4 Task 2: NCBI UniProtKB Mapping??? VERSION! We need a mapping from GENE_ID (NCBI) to UniProtKB-ID (EBI) This is a little complicated GENE_ID 1:N PROTEIN_ACCESSION 1:N UniProtKB-ID Step 1 Create a 1:N relationship between NCBI genes and NCBI PROTEIN_ACCESSION This should have been done already in assignment 2 Remove from PROTEIN_ACCESSION the version part Everything after a. Step 2 Download the File gene_refseq_uniprot_collab from the NCBI web site Upload data into a database table (2 columns) Check that many PROTEIN_ACCESSIONs appear in the first column of this table

5 Task 3: Access UniProtKB Query all proteins whose genes are located in 21q22.1 (exact match) Should be around 50 You may use either of three ways (chose now! We want diversity!) Option 1: Use HTTP and parse flatfile See Option 1: Use HTTP and parse XML See Option 3: Use a Java library and parse nothing See : See also : Download/access/extract the following information per protein Taxonomy ID(s) All functional annotations with GO terms Update your database schema to add this information

6 Where is the data? XML <dbreference type="ncbi Taxonomy" key="2" id="10116"/> <dbreference type="go" key="32" id="go: "> Flatfile API OX NCBI_TaxID=10116; DR GO; GO: ; F:endopeptidase inhibitor activity; getncbitaxonomyids() getgoterms()

7 Which method? Group Method 1 API 2 API 3 Flatfile 4 XML 5 Flatfile 6 API 7 XML 10 Flatfile 11 API 12 XML 13 API 14 Flatfile 15 XML 16 API

8 Task 4: Compare Functional Annotation Knowledge about the function of a gene or protein is often highly diverging among data sources Let s see

9 Task 5: Queries Answer the following queries 1. How many different MAP_LOCATIONS did remain? 2. How many genes are located on each chromosome? 3. Compute a frequency table: How many NCBI genes are connected to 1,2, UNIPROTKB-IDs? 4. How many NCBI genes have a TAX_ID attached that is different from one of the TAX_IDs of at least one associated UniProtKB-ID? 5. How many UniProtKB GO associations did you collect for how many UniProtKB-ID s for how many NCBI genes (three numbers)? 6. How many gene-go associations exist in the NCBI data that are not in the UniProtKB data? 7. How many protein-go associations exist in the UniProtKB data that are not in the NCBI data? 8. How many NCBI genes have at least one diverging annotation 1. Either a GO term in NCBI / not in UniProt or vice versa

10 Competition Accessing a data source over the web can be slow or fast Access the requested information as fast as possible This includes HTTP connection, parsing, writing in your database, etc. Write a program that Takes as input a MAP_LOCATION Computes all NCBI genes at this location Retrieves all GO and TAX data for all associated UniProtKB proteins The program must be executable as is on gruenau2

11 Deliverables By Monday 8.12., or Wednesday 10.12, 23:59 o clock Three weeks Send by mail as ASCII An updated schema graph with the new table(s) for UniProt data Queries and answers for 8 questions For the competition: Executable code as specified on last slide