Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download ""

Transcription

1

2

3

4 Beamlines by Village Macromolecular Crystallography Soft Condensed Matter Spectroscopy Materials Engineering and Environment Surfaces and Interfaces

5 A National User Facility for Biological Electron Cryo-microscopy (ebic) Wellcome Trust Strategic Award/MRC/BBSRC, applicants: Helen Saibil, Kay Grünewald, David Stuart, Gerhard Materlik Funded initially by the Wellcome Trust, MRC and BBSRC at level of 15.6 M over 5 years, augmented to ~ 25 M by additional investment by the Trust in 2016 The facility currently includes: - 4 high-end 300kV automated cryo EMs (Titan Krios FEI) kev automated feeder instrument (Talos Arctica) - Cryo focussed ion beam instrument (SCIOS) - Sample prep incl. vitreous sectioning - Correlative fluorescence/em - FEI Oxford for CAT 3 samples

6 ebic 13 th May 2017

7 New ebic Facility Initially constructed with two large rooms for two Krios, remodel to house four - completed 9/16. Sample preparation, loading and general labs. + multiple rooms for smaller microscopes

8

9

10

11

12

13 Preparation Experiment Raw Data Processed Final Result

14 Scientific Computing User office Business IT STFC, CCPs, Universities, Collaborators Analysis Acquisition Controls

15

16 HPC / HTC Cluster (~3500 cores) X86, Nvidia GPU (K80, P100) High Performance Storage Lustre03, Lustre04, GPFS01, GPFS02 Network infrastructure 10Gb/s, 40Gb/s to some beamlines User Gateways, Visualisation, Data Transfer NX Service, Globus endpoint Support Predominantly Linux infrastructure, BUT also Windows support to beamlines/em/etc and VM platforms Relies on working with Corporate IT and other groups in Controls and Scientific Software

17 Target Available Used Performance XFS 50 TB 47 TB < 1GB/s Lustre TB 370 TB 6 GB/s Lustre TB 70 TB 2 GB/s GPFS01 1 PB 700 TB 15 GB/s GPFS PB 1.5 PB 40 GB/s STFC Archive n/a 9 PB 12 TB 50 TB per day ingest

18 Detector Computer 10 / 40Gbit/s Beamline Switch 10Gbit/s 1 Gbit/s Central Switch 1 Gbit/s 2 x 40 Gbit/s or 2x10 Gbit/s Storage GPFS Central computing 3500 CPU cores (between 4 GB and 12 GB RAM per core) 86 GPUs

19 x22 xn Beamline

20

21

22

23

24 Acquisition and Analysis in the Eclipse Framework (www.eclipse.org) Client server technology Experiment GUI Communication with EPICS and hardware Scan mechanism Acquisition Jython and Python Visualisation Communication with external analysis Analysis tools Data read, write, convert Metadata structure Workflows Analysis DAWN is a collection of generic and bespoke views collated into perspectives. The perspectives and views can be used in part or whole in either the GDA or DAWN. All core technologies open source

25 Interfaces

26 ~200GB Database, 2.5million connections/day

27

28 monitoring Standard Template? User Template files (GPFS) File discovery (message_source.py) Job Queue (ActiveMQ)? SPP Pipeline (Scipion) session (ISPyB) logging Tomo Pipeline (IMOD)

29 Standard Template? User Template SPP Pipeline (Scipion) Tomo Pipeline (IMOD) Drift Correction MotionCor2 Combine Frames (Xmipp) Drift Correction MotionCor2 IMOD Image aberration (CTFFind) Particle Picker (e.g. Ethan) session (ISPyB)

30

31 Diamond s experiment data policy...users conducting Peer Reviewed Research will own the Experimental Data that they produce. Following the initial 30 day storage period, Diamond will create a single archive copy of the Experimental Data on tape. Users of Diamond Facilities are responsible for meeting any third-party data management obligations that may be applicable.

32 Data is archived and metadata captured from every stage of the process Session generation Time and users https://icatproject.org/ DLS User Office Database (UAS) 15 min ICAT Information catalogue Web Pages (TopCAT) ISPyB Processing DATA Data Registration TAPE DLS hosted STFC hosted

33

34

35 Data capture at full rate on both direct electron detectors (e.g. 17 fps for Falcon II or 40 fps for the K2). Falcon II data rate ~ movies/hr K2 data rate ~25-75 movies/hr All data are directly written to high speed central computing/storage facility. All data are archived to tape and stored for the lifetime of the media. Diamond cluster available to external users particularly during beamline shut downs.

36

37

38

39

40

41

42

43

44

45 Cryo-EM & Cryo-Tomo The resolution revolution Single particle analysis requires similar high quality samples to X-ray crystallography Massive advances in microscope, detectors (movies), software. Still much to come with improved sample presentation, automation, hardware/software. Already pretty efficient (data output> MX beamline, computional demands circa 1000-fold more).

46

47

48

Linking raw data with scientific workflow and software repository: some early

Linking raw data with scientific workflow and software repository: some early Linking raw data with scientific workflow and software repository: some early experience in PanData-ODI Erica Yang, Brian Matthews Scientific Computing Department (SCD) Rutherford Appleton Laboratory (RAL)

More information

IT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez

IT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez IT of SPIM Data Storage and Compression EMBO Course - August 27th Jeff Oegema, Peter Steinbach, Oscar Gonzalez 1 Talk Outline Introduction and the IT Team SPIM Data Flow Capture, Compression, and the Data

More information

MX Data (& Sample) Handling (& Tracking) at the ESRF Gordon Leonard ESRF Macromolecular Crystallography Group

MX Data (& Sample) Handling (& Tracking) at the ESRF Gordon Leonard ESRF Macromolecular Crystallography Group MX Data (& Sample) Handling (& Tracking) at the ESRF Gordon Leonard ESRF Macromolecular Crystallography Group Slide: 1 Gordon Leonard, 3-Way Meeting, APS, March 2008 Data production at the ESRF Data Storage

More information

CHESS DAQ* Introduction

CHESS DAQ* Introduction CHESS DAQ* Introduction Werner Sun (for the CLASSE IT group), Cornell University * DAQ = data acquisition https://en.wikipedia.org/wiki/data_acquisition Big Data @ CHESS Historically, low data volumes:

More information

Likely Evolution of Computational Infrastructure for Bio Beam Lines Over the Next Five Years

Likely Evolution of Computational Infrastructure for Bio Beam Lines Over the Next Five Years Likely Evolution of Computational Infrastructure for Bio Beam Lines Over the Next Five Years Dieter K. Schneider BNL Biology and the PXRR at the NSLS BNL, April 21, 2010 with expert advise from: James

More information

Shibbolized irods (and why it matters)

Shibbolized irods (and why it matters) Shibbolized irods (and why it matters) 3 rd TERENA Storage Meeting, Dublin, February 12 th -13 th 2009 David Corney, for Jens Jensen, e-science centre, Rutherford Appleton Lab, UK Overview Introduction

More information

Building an Open Data Infrastructure for Science: Turning Policy into Practice

Building an Open Data Infrastructure for Science: Turning Policy into Practice Building an Open Infrastructure for Science: Turning Policy into Practice Juan Bicarregui Head of Services Division STFC Department of Scientific Computing Franco-British Workshop on Big in Science, November

More information

HPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk

HPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk HPC and Big Data EPCC The University of Edinburgh Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk EPCC Facilities Technology Transfer European Projects HPC Research Visitor Programmes Training

More information

Beamline Automation at the APS

Beamline Automation at the APS Beamline Automation at the APS John Maclean Group Leader, Beamline Controls and Data Acquisition, Advanced Photon Source Argonne National Laboratory A Laboratory Operated by The University of Chicago Introduction

More information

Unterstützung datenintensiver Forschung am KIT Aktivitäten, Dienste und Erfahrungen

Unterstützung datenintensiver Forschung am KIT Aktivitäten, Dienste und Erfahrungen Unterstützung datenintensiver Forschung am KIT Aktivitäten, Dienste und Erfahrungen Achim Streit Steinbuch Centre for Computing (SCC) KIT Universität des Landes Baden-Württemberg und nationales Forschungszentrum

More information

Emerging Trends: Cultural Heritage 3D Modelling

Emerging Trends: Cultural Heritage 3D Modelling Emerging Trends: Cultural Heritage 3D Modelling Preserving Computer Aided Design Institute of Mechanical Engineers, London 26 th July 2013 Stuart Jeffrey 1 The Digital Design Studio (DDS) Specialist part

More information

Overview of HPC Resources at Vanderbilt

Overview of HPC Resources at Vanderbilt Overview of HPC Resources at Vanderbilt Will French Senior Application Developer and Research Computing Liaison Advanced Computing Center for Research and Education June 10, 2015 2 Computing Resources

More information

DMBI: Data Management for Bio-Imaging.

DMBI: Data Management for Bio-Imaging. DMBI: Data Management for Bio-Imaging. Questionnaire Data Report 1.0 Overview As part of the DMBI project an international meeting was organized around the project to bring together the bio-imaging community

More information

Integrating Research Information: Requirements of Science Research

Integrating Research Information: Requirements of Science Research Integrating Research Information: Requirements of Science Research Brian Matthews Scientific Information Group E-Science Centre STFC Rutherford Appleton Laboratory brian.matthews@stfc.ac.uk The science

More information

HPC Growing Pains. Lessons learned from building a Top500 supercomputer

HPC Growing Pains. Lessons learned from building a Top500 supercomputer HPC Growing Pains Lessons learned from building a Top500 supercomputer John L. Wofford Center for Computational Biology & Bioinformatics Columbia University I. What is C2B2? Outline Lessons learned from

More information

Aaron Ponti. Single Cell Unit, D-BSSE ETHZ (Basel) aaron.ponti@bsse.ethz.c h

Aaron Ponti. Single Cell Unit, D-BSSE ETHZ (Basel) aaron.ponti@bsse.ethz.c h Aaron Ponti Single Cell Unit, D-BSSE ETHZ (Basel) aaron.ponti@bsse.ethz.c h Raw data annotated backed up centralized storage HRM 2 openbis open Biology Information System openbis is an extensible, open

More information

EREBOS: CosmoSim Database. CLUES Research Environment. Harry Enke (Kristin Riebe, Jochen Klar, Adrian Partl) CLUES Meeting 2015, Copenhagen

EREBOS: CosmoSim Database. CLUES Research Environment. Harry Enke (Kristin Riebe, Jochen Klar, Adrian Partl) CLUES Meeting 2015, Copenhagen EREBOS: CLUES Research Environment CosmoSim Database Harry Enke (Kristin Riebe, Jochen Klar, Adrian Partl) CLUES Meeting 2015, Copenhagen Collaborative Research Environment (CRE) Elements: - huge data

More information

CMIP6 Data Management at DKRZ

CMIP6 Data Management at DKRZ CMIP6 Data Management at DKRZ icas2015 Annecy, France on 13 17 September 2015 Michael Lautenschlager Deutsches Klimarechenzentrum (DKRZ) With contributions from ESGF Executive Committee and WGCM Infrastructure

More information

Scalable Services for Digital Preservation

Scalable Services for Digital Preservation Scalable Services for Digital Preservation A Perspective on Cloud Computing Rainer Schmidt, Christian Sadilek, and Ross King Digital Preservation (DP) Providing long-term access to growing collections

More information

NSLS-II Data Management Framework Arman Arkilic Brookhaven National Lab NSLS-II

NSLS-II Data Management Framework Arman Arkilic Brookhaven National Lab NSLS-II NSLS-II Data Management Framework Arman Arkilic Brookhaven National Lab NSLS-II Motivation Modern synchrotron experiments require frameworks that provide scientists with data mining/analysis tools Given

More information

The Mantid Project. The challenges of delivering flexible HPC for novice end users. Nicholas Draper SOS18

The Mantid Project. The challenges of delivering flexible HPC for novice end users. Nicholas Draper SOS18 The Mantid Project The challenges of delivering flexible HPC for novice end users Nicholas Draper SOS18 What Is Mantid A framework that supports high-performance computing and visualisation of scientific

More information

Globus Research Data Management: Introduction and Service Overview. Steve Tuecke Vas Vasiliadis

Globus Research Data Management: Introduction and Service Overview. Steve Tuecke Vas Vasiliadis Globus Research Data Management: Introduction and Service Overview Steve Tuecke Vas Vasiliadis Presentations and other useful information available at globus.org/events/xsede15/tutorial 2 Thank you to

More information

ArcGIS Pro: Virtualizing in Citrix XenApp and XenDesktop. Emily Apsey Performance Engineer

ArcGIS Pro: Virtualizing in Citrix XenApp and XenDesktop. Emily Apsey Performance Engineer ArcGIS Pro: Virtualizing in Citrix XenApp and XenDesktop Emily Apsey Performance Engineer Presentation Overview What it takes to successfully virtualize ArcGIS Pro in Citrix XenApp and XenDesktop - Shareable

More information

The ANKA Archiving System

The ANKA Archiving System The ANKA Archiving System Combining Tango, WinCC OA and the web front-end ADEI David Haas, KIT Universität des Landes Baden-Württemberg und nationales Forschungszentrum in der Helmholtz-Gemeinschaft www.kit.edu

More information

CEDA Storage. Dr Matt Pritchard. Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk

CEDA Storage. Dr Matt Pritchard. Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk CEDA Storage Dr Matt Pritchard Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk How we store our data NAS Technology Backup JASMIN/CEMS CEDA Storage Data stored as files on disk. Data is migrated

More information

Estonian Scientific Computing Infrastructure (ETAIS)

Estonian Scientific Computing Infrastructure (ETAIS) Estonian Scientific Computing Infrastructure (ETAIS) Week #7 Hardi Teder hardi@eenet.ee University of Tartu March 27th 2013 Overview Estonian Scientific Computing Infrastructure Estonian Research infrastructures

More information

HPC technology and future architecture

HPC technology and future architecture HPC technology and future architecture Visual Analysis for Extremely Large-Scale Scientific Computing KGT2 Internal Meeting INRIA France Benoit Lange benoit.lange@inria.fr Toàn Nguyên toan.nguyen@inria.fr

More information

Visualization Cluster Getting Started

Visualization Cluster Getting Started Visualization Cluster Getting Started Contents 1 Introduction to the Visualization Cluster... 1 2 Visualization Cluster hardware and software... 2 3 Remote visualization session through VNC... 2 4 Starting

More information

Virtualisation Cloud Computing at the RAL Tier 1. Ian Collier STFC RAL Tier 1 HEPiX, Bologna, 18 th April 2013

Virtualisation Cloud Computing at the RAL Tier 1. Ian Collier STFC RAL Tier 1 HEPiX, Bologna, 18 th April 2013 Virtualisation Cloud Computing at the RAL Tier 1 Ian Collier STFC RAL Tier 1 HEPiX, Bologna, 18 th April 2013 Virtualisation @ RAL Context at RAL Hyper-V Services Platform Scientific Computing Department

More information

Enhanced Research Data Management and Publication with Globus

Enhanced Research Data Management and Publication with Globus Enhanced Research Data Management and Publication with Globus Vas Vasiliadis Jim Pruyne Presented at OR2015 June 8, 2015 Presentations and other useful information available at globus.org/events/or2015/tutorial

More information

Globus Research Data Management: Introduction and Service Overview

Globus Research Data Management: Introduction and Service Overview Globus Research Data Management: Introduction and Service Overview Kyle Chard chard@uchicago.edu Ben Blaiszik blaiszik@uchicago.edu Thank you to our sponsors! U. S. D E P A R T M E N T OF ENERGY 2 Agenda

More information

The Trials and Tribulations and ultimate success of parallelisation using Hadoop within the SCAPE project

The Trials and Tribulations and ultimate success of parallelisation using Hadoop within the SCAPE project The Trials and Tribulations and ultimate success of parallelisation using Hadoop within the SCAPE project Alastair Duncan STFC Pre Coffee talk STFC July 2014 SCAPE Scalable Preservation Environments The

More information

Automated and Scalable Data Management System for Genome Sequencing Data

Automated and Scalable Data Management System for Genome Sequencing Data Automated and Scalable Data Management System for Genome Sequencing Data Michael Mueller NIHR Imperial BRC Informatics Facility Faculty of Medicine Hammersmith Hospital Campus Continuously falling costs

More information

Scientific Software & User- Centered Design. Cmpt 479 Jessie Helfrich December 3, 2015

Scientific Software & User- Centered Design. Cmpt 479 Jessie Helfrich December 3, 2015 Scientific Software & User- Centered Design Cmpt 479 Jessie Helfrich December 3, 2015 Context Experience: - Software developer at the Canadian Light Source. Problem: - Synchrotron facilities use software

More information

HPC Cloud. Focus on your research. Floris Sluiter Project leader SARA

HPC Cloud. Focus on your research. Floris Sluiter Project leader SARA HPC Cloud Focus on your research Floris Sluiter Project leader SARA Why an HPC Cloud? Christophe Blanchet, IDB - Infrastructure Distributing Biology: Big task to port them all to your favorite architecture

More information

Building Bioinformatics Capacity in Africa. Nicky Mulder CBIO Group, UCT

Building Bioinformatics Capacity in Africa. Nicky Mulder CBIO Group, UCT Building Bioinformatics Capacity in Africa Nicky Mulder CBIO Group, UCT Outline What is bioinformatics? Why do we need IT infrastructure? What e-infrastructure does it require? How we are developing this

More information

Scientific Computing Data Management Visions

Scientific Computing Data Management Visions Scientific Computing Data Management Visions ELI-Tango Workshop Szeged, 24-25 February 2015 Péter Szász Group Leader Scientific Computing Group ELI-ALPS Scientific Computing Group Responsibilities Data

More information

Lustre SMB Gateway. Integrating Lustre with Windows

Lustre SMB Gateway. Integrating Lustre with Windows Lustre SMB Gateway Integrating Lustre with Windows Hardware: Old vs New Compute 60 x Dell PowerEdge 1950-8 x 2.6Ghz cores, 16GB, 500GB Sata, 1GBe - Win7 x64 Storage 1 x Dell R510-12 x 2TB Sata, RAID5,

More information

Integrated Rule-based Data Management System for Genome Sequencing Data

Integrated Rule-based Data Management System for Genome Sequencing Data Integrated Rule-based Data Management System for Genome Sequencing Data A Research Data Management (RDM) Green Shoots Pilots Project Report by Michael Mueller, Simon Burbidge, Steven Lawlor and Jorge Ferrer

More information

MEDIA ASSET MANAGEMENT

MEDIA ASSET MANAGEMENT MEDIA ASSET MANAGEMENT D 3 -MAM The massive shift from analog to digital content, often referred to as the "digital revolution", has had a profound impact on many industries, yet the most efficient way

More information

TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 7 th CALL (Tier-0)

TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 7 th CALL (Tier-0) TECHNICAL GUIDELINES FOR APPLICANTS TO PRACE 7 th CALL (Tier-0) Contributing sites and the corresponding computer systems for this call are: GCS@Jülich, Germany IBM Blue Gene/Q GENCI@CEA, France Bull Bullx

More information

JASMIN Cloud ESGF and UV- CDAT Conference 09-11 December 2014 STFC / Stephen Kill

JASMIN Cloud ESGF and UV- CDAT Conference 09-11 December 2014 STFC / Stephen Kill JASMIN Cloud ESGF and UV- CDAT Conference 09-11 December 2014 STFC / Stephen Kill Philip Kershaw (1, 2), Jonathan Churchill (5), Bryan Lawrence (1, 3, 4), Stephen Pascoe (1, 4) and MaE Pritchard (1) Centre

More information

Optimizing Data Management at the Advanced Light Source with a Science DMZ

Optimizing Data Management at the Advanced Light Source with a Science DMZ Optimizing Data Management at the Advanced Light Source with a Science DMZ Eli Dart, Network Engineer ESnet Network Engineering Group GlobusWorld 2013 Argonne, Il April 17, 2013 Outline Science DMZ background

More information

Flow 3 Media Asset Management incorporating AirFlow and Automation

Flow 3 Media Asset Management incorporating AirFlow and Automation DATASHEET Media asset management Flow 3 Media Asset Management incorporating AirFlow and Automation EditShare s Flow products are all about adding efficiency and increasing productivity in your production

More information

SURFsara Data Services

SURFsara Data Services SURFsara Data Services SUPPORTING DATA-INTENSIVE SCIENCES Mark van de Sanden The world of the many Many different users (well organised (international) user communities, research groups, universities,

More information

Building a Top500-class Supercomputing Cluster at LNS-BUAP

Building a Top500-class Supercomputing Cluster at LNS-BUAP Building a Top500-class Supercomputing Cluster at LNS-BUAP Dr. José Luis Ricardo Chávez Dr. Humberto Salazar Ibargüen Dr. Enrique Varela Carlos Laboratorio Nacional de Supercómputo Benemérita Universidad

More information

Getting Started with HPC

Getting Started with HPC Getting Started with HPC An Introduction to the Minerva High Performance Computing Resource 17 Sep 2013 Outline of Topics Introduction HPC Accounts Logging onto the HPC Clusters Common Linux Commands Storage

More information

Purchase of High Performance Computing (HPC) Central Compute Resources by Northwestern Researchers

Purchase of High Performance Computing (HPC) Central Compute Resources by Northwestern Researchers Information Technology Purchase of High Performance Computing (HPC) Central Compute Resources by Northwestern Researchers Effective for FY2016 Purpose This document summarizes High Performance Computing

More information

Towards Elastic Application Model for Augmenting Computing Capabilities of Mobile Platforms. Mobilware 2010

Towards Elastic Application Model for Augmenting Computing Capabilities of Mobile Platforms. Mobilware 2010 Towards lication Model for Augmenting Computing Capabilities of Mobile Platforms Mobilware 2010 Xinwen Zhang, Simon Gibbs, Anugeetha Kunjithapatham, and Sangoh Jeong Computer Science Lab. Samsung Information

More information

Introducing ScienceCloud

Introducing ScienceCloud Zentrale Informatik Introducing ScienceCloud Sergio Maffioletti IS/Cloud S3IT: Service and Support for Science IT Zurich, 10.03.2015 What are we going to talk about today? 1. Why are we building ScienceCloud?

More information

CLS Office/Beam line Data Storage System Specification CLS 9.8.92.1 Rev. 0

CLS Office/Beam line Data Storage System Specification CLS 9.8.92.1 Rev. 0 CLS Office/Beam line Data Storage System Specification CLS 9.8.92.1 Rev. 0 2003-02-11 Copyright 2003, Canadian Light Source, Inc. This document is the property of Canadian Light Source Incorporated (CLS).

More information

Terabit Networking with JASMIN

Terabit Networking with JASMIN Terabit Networking with JASMIN Jonathan Churchill JASMIN Infrastructure Manager Research Infrastructure Group Scientific Computing Department STFC Rutherford Appleton Labs Terabit Networking with JASMIN

More information

The National Consortium for Data Science (NCDS)

The National Consortium for Data Science (NCDS) The National Consortium for Data Science (NCDS) A Public-Private Partnership to Advance Data Science Ashok Krishnamurthy PhD Deputy Director, RENCI University of North Carolina, Chapel Hill What is NCDS?

More information

VTrak G1100 Application and Performance Notes

VTrak G1100 Application and Performance Notes VTrak G1100 Application and Performance Notes Version 1.0 Date: 11/5/2014 Copyright 2014, Promise Technology, Inc. All Rights Reserved Revision History Revision Date Author Description 0.69 10/20/14 Gary

More information

Workflow Tools at NERSC. Debbie Bard djbard@lbl.gov NERSC Data and Analytics Services

Workflow Tools at NERSC. Debbie Bard djbard@lbl.gov NERSC Data and Analytics Services Workflow Tools at NERSC Debbie Bard djbard@lbl.gov NERSC Data and Analytics Services NERSC User Meeting August 13th, 2015 What Does Workflow Software Do? Automate connection of applications Chain together

More information

Image Data, RDA and Practical Policies

Image Data, RDA and Practical Policies Image Data, RDA and Practical Policies Rainer Stotzka and many others KIT University of the State of Baden-Württemberg and National Laboratory of the Helmholtz Association www.kit.edu Data Life Cycle Lab

More information

PES. Batch virtualization and Cloud computing. Part 1: Batch virtualization. Batch virtualization and Cloud computing

PES. Batch virtualization and Cloud computing. Part 1: Batch virtualization. Batch virtualization and Cloud computing Batch virtualization and Cloud computing Batch virtualization and Cloud computing Part 1: Batch virtualization Tony Cass, Sebastien Goasguen, Belmiro Moreira, Ewan Roche, Ulrich Schwickerath, Romain Wartel

More information

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect on AWS Services Overview Bernie Nallamotu Principle Solutions Architect \ So what is it? When your data sets become so large that you have to start innovating around how to collect, store, organize, analyze

More information

#jenkinsconf. Jenkins as a Scientific Data and Image Processing Platform. Jenkins User Conference Boston #jenkinsconf

#jenkinsconf. Jenkins as a Scientific Data and Image Processing Platform. Jenkins User Conference Boston #jenkinsconf Jenkins as a Scientific Data and Image Processing Platform Ioannis K. Moutsatsos, Ph.D., M.SE. Novartis Institutes for Biomedical Research www.novartis.com June 18, 2014 #jenkinsconf Life Sciences are

More information

ediscovery Journal Report: Digital Reef & BlueArc ediscovery Software Performance Benchmark Test

ediscovery Journal Report: Digital Reef & BlueArc ediscovery Software Performance Benchmark Test ediscovery Journal Report: Digital Reef & BlueArc ediscovery Software Performance Benchmark Test ediscoveryjournal was engaged to review the testing methodology, execution, and results of the Digital Reef

More information

GPU Renderfarm with Integrated Asset Management & Production System (AMPS)

GPU Renderfarm with Integrated Asset Management & Production System (AMPS) GPU Renderfarm with Integrated Asset Management & Production System (AMPS) Tackling two main challenges in CG movie production Presenter: Dr. Chen Quan Multi-plAtform Game Innovation Centre (MAGIC), Nanyang

More information

Deploying and managing a Visualization Farm @ Onera

Deploying and managing a Visualization Farm @ Onera Deploying and managing a Visualization Farm @ Onera Onera Scientific Day - October, 3 2012 Network and computing department (DRI), Onera P.F. Berte pierre-frederic.berte@onera.fr Plan Onera global HPC

More information

Remote & Collaborative Visualization. Texas Advanced Compu1ng Center

Remote & Collaborative Visualization. Texas Advanced Compu1ng Center Remote & Collaborative Visualization Texas Advanced Compu1ng Center So6ware Requirements SSH client VNC client Recommended: TigerVNC http://sourceforge.net/projects/tigervnc/files/ Web browser with Java

More information

Personalized Medicine and IT

Personalized Medicine and IT Personalized Medicine and IT Data-driven Medicine in the Age of Genomics www.intel.com/healthcare/bigdata Ketan Paranjape General Manager, Life Sciences Intel Corp. @Portlandketan 1 The Central Dogma of

More information

Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS

Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS Workshop on the Future of Big Data Management 27-28 June 2013 Philip Kershaw Centre for Environmental Data Archival

More information

NT1: An example for future EISCAT_3D data centre and archiving?

NT1: An example for future EISCAT_3D data centre and archiving? March 10, 2015 1 NT1: An example for future EISCAT_3D data centre and archiving? John White NeIC xx March 10, 2015 2 Introduction High Energy Physics and Computing Worldwide LHC Computing Grid Nordic Tier

More information

Data Analysis Sequencing - EDNA. Olof Svensson Data Analysis Unit ISDD ESRF

Data Analysis Sequencing - EDNA. Olof Svensson Data Analysis Unit ISDD ESRF Data Analysis Sequencing - EDNA Olof Svensson Data Analysis Unit ISDD ESRF Why do we need EDNA? EDNA is the best answer we (developers) have come up with so far for answering these questions : How can

More information

IT Infrastructure Management

IT Infrastructure Management IT Infrastructure Management Server-Database Monitoring An Overview XIPHOS TECHNOLOGY SOLUTIONS PVT LIMITED 32/3L, GARIAHAT ROAD (SOUTH) KOLKATA 700 078, WEST BENGAL, INDIA WWW.XIPHOSTEC.COM Xiphos Technology

More information

Parallels Plesk Automation

Parallels Plesk Automation Parallels Plesk Automation Contents Compact Configuration: Linux Shared Hosting 3 Compact Configuration: Mixed Linux and Windows Shared Hosting 4 Medium Size Configuration: Mixed Linux and Windows Shared

More information

Digital Forensics Tutorials Acquiring an Image with FTK Imager

Digital Forensics Tutorials Acquiring an Image with FTK Imager Digital Forensics Tutorials Acquiring an Image with FTK Imager Explanation Section Digital Forensics Definition The use of scientifically derived and proven methods toward the preservation, collection,

More information

Partek Flow Installation Guide

Partek Flow Installation Guide Partek Flow Installation Guide Partek Flow is a web based application for genomic data analysis and visualization, which can be installed on a desktop computer, compute cluster or cloud. Users can access

More information

Enabling multi-cloud resources at CERN within the Helix Nebula project. D. Giordano (CERN IT-SDC) HEPiX Spring 2014 Workshop 23 May 2014

Enabling multi-cloud resources at CERN within the Helix Nebula project. D. Giordano (CERN IT-SDC) HEPiX Spring 2014 Workshop 23 May 2014 Enabling multi-cloud resources at CERN within the Helix Nebula project D. Giordano (CERN IT-) HEPiX Spring 2014 Workshop This document produced by Members of the Helix Nebula consortium is licensed under

More information

Content. > How to ensure sustainability of RIs? > RIs generate knowledge > Socio-economic impact of RIs

Content. > How to ensure sustainability of RIs? > RIs generate knowledge > Socio-economic impact of RIs Long-term Sustainability of Accelerator-Driven Research Infrastructures in the Knowledge- Based Economy. Dr. Ute Krell Seminar on Role of RIs for a Competitive Knowledge Economy Brussels, 30 June 2009

More information

A pretty picture, or a measurement? Retinal Imaging

A pretty picture, or a measurement? Retinal Imaging Big Data Challenges A pretty picture, or a measurement? Organelles Dynamics Cells Retinal Imaging Physiology Pathology Fundus Camera Optical coherence tomography Fluorescence Histology High Content Screening

More information

Reference Architectures for Digital Libraries

Reference Architectures for Digital Libraries Reference Architectures for Digital Libraries Keith Rajecki Education Solutions Architect Sun Microsystems, Inc. 1 Agenda Challenges Digital Library Solution Architectures > Open Storage/Open Archive >

More information

Integration of Virtualized Worker Nodes in Batch Systems

Integration of Virtualized Worker Nodes in Batch Systems Integration of Virtualized Worker Nodes Dr. A. Scheurer, Dr. V. Büge, O. Oberst, P. Krauß Linuxtag 2010, Berlin, Session: Cloud Computing, Talk ID: #16197 KIT University of the State of Baden-Wuerttemberg

More information

Performance Testing in Virtualized Environments. Emily Apsey Product Engineer

Performance Testing in Virtualized Environments. Emily Apsey Product Engineer Performance Testing in Virtualized Environments Emily Apsey Product Engineer Introduction Product Engineer on the Performance Engineering Team Overview of team - Specialty in Virtualization - Citrix, VMWare,

More information

Where to use open source

Where to use open source EBU Open source seminar Geneva 1-2 October Where to use open source Giorgio Dimino RAI Research Centre g.dimino@rai.it Archive ingestion projects RAI Research Centre has been involved for many years in

More information

HPC @ CRIBI. Calcolo Scientifico e Bioinformatica oggi Università di Padova 13 gennaio 2012

HPC @ CRIBI. Calcolo Scientifico e Bioinformatica oggi Università di Padova 13 gennaio 2012 HPC @ CRIBI Calcolo Scientifico e Bioinformatica oggi Università di Padova 13 gennaio 2012 what is exact? experience on advanced computational technologies a company lead by IT experts with a strong background

More information

A New Scalable Multi-Node Event Logging System for BaBar

A New Scalable Multi-Node Event Logging System for BaBar A New Scalable Multi-Node Event Logging System for BaBar James A. Hamilton Steffen Luitz For the BaBar Computing Group Original Structure Raw Data Processing Level 3 Trigger Mirror Detector Electronics

More information

Software Engineering Support

Software Engineering Support Software Engineering Support Christopher Greenough, Alan Kyffin, Gemma Poulter Software Engineering Group Scientific Computing Department STFC Rutherford Appleton Laboratory christopher.greenough@stfc.ac.uk

More information

Let s Talk Digital An Approach to Managing, Storing, and Preserving Time-Based Media Art Works. DAMS Branch Manager Smithsonian Institution, OCIO

Let s Talk Digital An Approach to Managing, Storing, and Preserving Time-Based Media Art Works. DAMS Branch Manager Smithsonian Institution, OCIO Let s Talk Digital An Approach to Managing, Storing, and Preserving Time-Based Media Art Works DAMS Branch Manager Smithsonian Institution, OCIO 1 Love and Marriage Digital Asset Management?? Digital Diamonds

More information

Contents UNIFIED COMPUTING DATA SHEET. Virtual Data Centre Support. www.interoute.com

Contents UNIFIED COMPUTING DATA SHEET. Virtual Data Centre Support. www.interoute.com Contents Scope of this Document... 2 Product Overview... 2 Virtual Data Centre and VDC Dedicated Infrastructure... 2 Service Levels... 3 Severity and Support Response Times... 4 On-boarding... 5 Incident

More information

Evaluation of Enterprise Data Protection using SEP Software

Evaluation of Enterprise Data Protection using SEP Software Test Validation Test Validation - SEP sesam Enterprise Backup Software Evaluation of Enterprise Data Protection using SEP Software Author:... Enabling you to make the best technology decisions Backup &

More information

The Rise of Industrial Big Data. Brian Courtney General Manager Industrial Data Intelligence

The Rise of Industrial Big Data. Brian Courtney General Manager Industrial Data Intelligence The Rise of Industrial Big Data Brian Courtney General Manager Industrial Data Intelligence Agenda Introduction Big Data for the industrial sector Case in point: Big data saves millions at GE Energy Seeking

More information

The Extension of the DICOM Standard to Incorporate Omics

The Extension of the DICOM Standard to Incorporate Omics Imperial College London The Extension of the DICOM Standard to Incorporate Omics Data Richard I Kitney, Vincent Rouilly and Chueh-Loo Poh Department of Bioengineering We stand at the dawn of a new understanding

More information

Control Software at ESRF beamlines

Control Software at ESRF beamlines Control Software at ESRF beamlines BLISS group Vicente Rey Bakaikoa About the speaker ID card: Name: Vicente Rey Bakaikoa Born in: Pamplona / Spain Living in: Grenoble since 1990 Working in beamline control

More information

ParaVision 6. Innovation with Integrity. The Next Generation of MR Acquisition and Processing for Preclinical and Material Research.

ParaVision 6. Innovation with Integrity. The Next Generation of MR Acquisition and Processing for Preclinical and Material Research. ParaVision 6 The Next Generation of MR Acquisition and Processing for Preclinical and Material Research Innovation with Integrity Preclinical MRI A new standard in Preclinical Imaging ParaVision sets a

More information

Eliminate Dark and Dirty Data

Eliminate Dark and Dirty Data Eliminate Dark and Dirty Data ECM is dead Long Live ECM Why didn t they come? Deep, un navigable folder structures Application of Metadata and Classifications Find-ability Scope not well defined All Resulting

More information

Archival of raw and analysed radar data at EISCAT and worldwide

Archival of raw and analysed radar data at EISCAT and worldwide Archival of raw and analysed radar data at EISCAT and worldwide Carl-Fredrik Enell, EISCAT Scientific Association COOPEUS workshop and EGI-CC kickoff, 11 March 2015 C-F Enell, EISCAT Radar data archival

More information

Large-scale Research Data Management and Analysis Using Globus Services. Ravi Madduri Argonne National Lab University of Chicago @madduri

Large-scale Research Data Management and Analysis Using Globus Services. Ravi Madduri Argonne National Lab University of Chicago @madduri Large-scale Research Data Management and Analysis Using Globus Services Ravi Madduri Argonne National Lab University of Chicago @madduri Outline Who we are Challenges in Big Data Management and Analysis

More information

System Center 2012 Suite SYSTEM CENTER 2012 SUITE. BSD BİLGİSAYAR Adana

System Center 2012 Suite SYSTEM CENTER 2012 SUITE. BSD BİLGİSAYAR Adana 2013 System Center 2012 Suite SYSTEM CENTER 2012 SUITE BSD BİLGİSAYAR Adana Configure and manage apps, services, computers, and VMs... 1 Operations Manager... 3 Configuration Manager... 4 Endpoint Protection...

More information

SURFsara HPC Cloud Workshop

SURFsara HPC Cloud Workshop SURFsara HPC Cloud Workshop doc.hpccloud.surfsara.nl UvA workshop 2016-01-25 UvA HPC Course Jan 2016 Anatoli Danezi, Markus van Dijk cloud-support@surfsara.nl Agenda Introduction and Overview (current

More information

Virtual Server and Storage Provisioning Service. Service Description

Virtual Server and Storage Provisioning Service. Service Description RAID Virtual Server and Storage Provisioning Service Service Description November 28, 2008 Computer Services Page 1 TABLE OF CONTENTS INTRODUCTION... 4 VIRTUAL SERVER AND STORAGE PROVISIONING SERVICE OVERVIEW...

More information

BioHPC Web Computing Resources at CBSU

BioHPC Web Computing Resources at CBSU BioHPC Web Computing Resources at CBSU 3CPG workshop Robert Bukowski Computational Biology Service Unit http://cbsu.tc.cornell.edu/lab/doc/biohpc_web_tutorial.pdf BioHPC infrastructure at CBSU BioHPC Web

More information

U-LITE: a proposal for scientific computing at LNGS. S. Parlati, P. Spinnato, S. Stalio LNGS 13 Sep. 2011

U-LITE: a proposal for scientific computing at LNGS. S. Parlati, P. Spinnato, S. Stalio LNGS 13 Sep. 2011 U-LITE: a proposal for scientific computing at LNGS S. Parlati, P. Spinnato, S. Stalio LNGS 13 Sep. 2011 20 years of Scientific Computing at LNGS Early 90s: highly centralized structure based on VMS cluster

More information

Research Data Management PROJECT LIFECYCLE

Research Data Management PROJECT LIFECYCLE PROJECT LIFECYCLE Introduction and context Basic Project Info. Thesis Title UH or Research Council? Duration Related Policies UH and STFC policies: open after publication as your research is public funded

More information

Parallel Large-Scale Visualization

Parallel Large-Scale Visualization Parallel Large-Scale Visualization Aaron Birkland Cornell Center for Advanced Computing Data Analysis on Ranger January 2012 Parallel Visualization Why? Performance Processing may be too slow on one CPU

More information

STORNEXT PRO SOLUTIONS. StorNext Pro Solutions

STORNEXT PRO SOLUTIONS. StorNext Pro Solutions STORNEXT PRO SOLUTIONS StorNext Pro Solutions StorNext PRO SOLUTIONS StorNext Pro Solutions offer Post-Production and Broadcast Professionals the fastest, easiest, and most complete high-performance shared

More information

Data Management at UT

Data Management at UT Data Management at UT Maria Esteva, TACC, maria@tacc.utexas.edu Colleen Lyon, UT Libraries, c.lyon@austin.utexas.edu Angela Newell, ITS, anewell@austin.utexas.edu What is data management? systematic organization

More information