Marine Program Informatics A Success Story in Integrated Data/Information Partnering Richard Eisner- Maritime Science 27 November 2003
Theme/outline Who are we? Back Office versus front office Data/ Information- What do we know about it? Standardized Metadata and Data- Lead Data Managers- Geoconnection link Reliability- Infrastructure Sustainability Propagating perennial data partners and technology Threat Management- How safe is the data? Pest and disease management ( living with) We have been successful two examples - integrated data Management Scallop industry State of the Ecosystem New things Genomics Data Management and Canadian Earth Observation System (CEOS
Question(S)- Please put a $ in front of it Where is your data? Is it safe? What shape is it? Will it continue to be safe? Do you have to get a person from retirement /the grave to explain it to you?
Information- Questions How much do you know about the data? Can you mine it? How much did it cost to collect it and store it ( Hard copy?) Do you do annual data management or perennial data (gardening)
Governance-Lead Data Manager structure C/W Data Marts Richard Eisner Manager- PCC Chairperson-SOCC Activities-Internal Infrastructure Robert O'Boyle Associate Director RAP Coordinator Activities-GOL Stephen Smith Invertebrates Doug Gregory Oceans Sciences Phil Zamora Habitat Management Jennifer Hackett Oceans and Coastal Managment Carolyn Harvie Diadromous Robert Burke CHS Robert Branton Marine Fish Pierre Clement MESD Shawn Robinson Aquaculture
BIO Data and Information/Knowledge Management Includes all Science and Ocean programs Fisheries Science Oceanography Environmental Science Oceans and Coastal Management Aquaculture Hydrography Habitat Management Built on initiatives / existing structure- Y2K, MEDS, strong CHS Informatics Supported by Non core Informatics- Application Development, Applications Roll out, Client operations ( server support) Supported by Core Maritimes Informatics-National and Regional Network, Telephony, LAN Servers and Workstations, Help Desk, Security, Document Management, Library Linkages to Oceans and NRCAN at BIO plus Regional staff, all Sectors and external stakeholders
Scope of Science Data Management Service Delivery Analysis & Products Archival Processing & Analysis Data Gathering / Acquisition
Key Drivers and Trends Increasing demands: Census of Marine Life Ocean Biogeographical Information system- regional equivalents Metadata standards Metadata Info-Instrument types,data Editing, Data quality Energy development - Oil and Natural Gas Electronic Navigational Charting and improved Raster Nautical Charts Diversification of fishery - pressures on Wild Stocks Technological change - demand for real time information Marine Sciences Database development Expanding data management : Acoustic images ( example Multibeam) Video Internet and intranet portals Database Management Security HHcode and SDS
Recent new focus Increasing activity focus: Aquatic Invasive species Climate studies in the Labrador Sea and Eastern Arctic Fish Aquaculture /Environment Interactions Stock assessment for under- assessed, economically valuable species Science in support of Integrated Management Ecosystem Structure- descriptive Tools Discovery Corridors Genetic effects of Fishing Lobster Metapopulation structure Noise in the Aquatic Environment
Issues - can be grouped into a small number of broad topics. Application Development and Rationalization - new technology creates new datasets and data rescue has uncovered old ones. New applications are required and existing applications extended. Data integration requires that databases must "communicate". Accessibility - ranges from simply organizing small data sets that staff can use to making larger organized datasets available to a broader audience. Ownership and security also underlie this issue. Mass Storage - new technologies make our conventional data storage technologies completely inadequate. Data Rescue - major progress has been made - the important of data holding Training - retirements and new technology highlight the importance of both technical training and new employee mentoring.
Infrastructure- Challenges Storage capacity is not keeping pace with growth The security capacity for data and networks is dramatically improved as a result of 9/11- more work required/however not everyone can afford Recovery and business resumption strategies of most Science organizations are inadequate or non existence No wish by vendors to solve interoperability
Data Holdings Survey - Study Results Projected Science and Oceans Data Growth 16 14 12 10 Terabytes 8 6 4 2-1 2 3 4 5 Years
Related Documents marketing plan location or contact name/phone budget location or contact name/phone post mortem location or contact name/phone submit questions location or contact name/phone
Advanced Informatic architecture Lan- file requests first layer - clients( PC s) Second layer servers (Win2K- E-Mail,Unix/linux/,linux ( Web), Win 2K - file server San- - block requests central storage domain director - virtualization,backup/recovery,hsm/ Snapshots Managed Disk Assays and Nearline tape Library
Needed Advanced data management in support of monitoring the Ocean The use of Multi-beam and remote sensing technology Integration of the various groups through computer technology and multidimensional mathematics allows us to be able to do difference analysis or find similarities between two dissimilar measurements Change from ASCII ( a 7- bit binary code ) to 32 bit or 64 bit technology
Optimize Visual Interpretation
Water depth NEW BRUNSWICK < 200 m St. John 45 > 200 m MAINE Bangor 50 BAY OF FUNDY NOVA SCOTIA Halifax 44 100 Yar mout h GULF OF MAINE 50 43 100 100 100 500 1000 1500 42 CAPE COD 100 50 NORTHEAST CHANNEL GEORGES BANK 2000 2500 3000 3500 70 69 68 67 66 65 64 63 Brow
Browns Bank
Habitat Layer
Electronic charts environmental and economic impacts German Bank scallop quota :13,640 kg 1998 1999 bottom time 162 hr 43 hr bottom towed 1176 km 311 km hours lost 15 0 lost gear $10,000 0 fuel use 27,697 l 17,545 l equipment heavy light
What is a State of the Ecosystem Report? Assessment of a large area of the ocean based on the analysis on a wide variety of measurements made over many years A team of scientists working at BIO produced the report J. Choi, K. Drinkwater, B. Petrie, G. Harrison, A. Bundy, P. Yeats, S. Coffen-Smout, H. Breeze, R.N. O Boyle, K. Querbach, K. Frank
Components of a typical offshore marine ecosystem
Variety of measurements used 23 different types of measurements of the ocean and atmosphere» Air & water temperature, salinity, storms, etc. 12 types representing human activity» Fishery landings & landed value, oil and gas development, etc. 26 for marine life» Abundance: plankton fish marine mammals, composition, size
What next? Need to combine all of the information to search for patterns Look how each type of information has changed relative to its average level (e.g. are things greater or less than usual or bigger or smaller or higher vs. lower than normal, etc. Use colour coding for each year of observation (red less, smaller, lower than usual; green bigger, greater, higher than usual)
Colour display of 60+ indices for Eastern Scotian Shelf Grey seals - adults Pelagic fish - # s Pelagic:demersal # s Pelagic:demersal wt. Inverts - $$ Pelagics - wt Diatoms Grey seals pups Pelagics - $$ Greenness Dinoflagellates Fish diversity richness 3D Seisimic (km2) Gulf Stream position Stratification anomaly Diatom:dinoflagellate Sea level anomaly Volume of CIL source water Inverts landings Bottom water < 3 C Sable winds (Tau) SST anomaly (satellites) chlorophyll CPR Temperature of mixed layer NAO Bottom T Emerald basin Copepods Para/Pseudocal Shelfslope front position Storms Bottom T Misaine bank Groundfish landings Haddock length at age 6 Bottom area trawled (>150 GRT) Cod length at age 6 Average weight of fish Community similarity index PCB s in seal blubber Relative F Pollock length at age 6 Calanus finmarchicus Groundfish biomass Nov. RV Pelagics 27, 2003 landings Silver hake length at age Condition KF Depth of mixed layer Grey seals, pelagic fish abundance, value of shellfish landings, fish species richness, phytoplankton Bottom temp., exploitation, groundfish biomass & landings, growth-chp, avg. fish weight, copepods 1970 1975 1980 1985 1990 Red below average Green above average
What can Informatics Industry do for genomics Several Geomatic Informatic Industries are in the process of adapting their technology to the development of bioinformatic software packages, which will be used to provide a suite of tools that will allow for multi-faceted visual analysis of bioinformatic data. It will make storage, analysis, retrieval and updating of bioinformatic data more manageable through the use of the scaleable ISO based Self Defining Structure Archives. In addition, the software will be able to run on desktop PC's, making the analysis of genomic information more affordable for the average research facility.
Genomics Data Management The rapid pace of genomic data generation through the use of new acquisition technologies such as mass spectrometers, coupled with new computer technologies is challenging the biological community to build immensely vast bioinformatic data warehouses. Existing sequence databases are, for the most part, brute-force repositories using huge expanses of very expensive hardware to store and process bioinformatic information. Genomic information, at present, is stored as flat, dissociated ASCII streams with a moderate level of query capability due to limitations of software. There is a need, not only for more efficient storage capability to maintain the integrity of newly generated, potentially valuable proprietary sequences but also for advanced analysis and visualization tools for the creation of a bioinformatic knowledge base.
News Ideas Images example-medical images- Most hospitals are generating 5000-300, 000 images per year Books- Reducing library hard copy holdings 2 gigabytes is 20 meters of shelved books 2 terabyte is academic research library
Canadian Earth Observation System (CEOS The Group on Earth Observations was established following the Earth Observation Summit on July 31, 2003. This initiative was lead by the United States to promote the development of a comprehensive, coordinated and sustainable Earth Observation System(s) among governments and the international community to improve our ability to understand and address global environmental and economic challenges and meet international treaty obligations. The Canadian Group on Earth Observations is part of Canada's contribution to this initiative. DFO plan will be organised into deep water, coastal, Data management, modelling and products Capacity Building has to be an integral part of this plan as we need to focus on developing new instrumentation, new techniques for data assimilation and Modelling. AZMP example will be followed, focusing on variables from oceanographic up to and including lower trophic levels
The Way Ahead Some realignment of program resources to this high priority New Resources- capital, O & M and FTE s to focus on needs Risk Management- These are R & D - not operational R & D Data management- Enhanced partnering - Ocean Biogeographical Information System, Industry Partners on collection of data/ Applications development, data mangement,university partnerships