Instruments in Grid: the New Instrument Element

Size: px
Start display at page:

Download "Instruments in Grid: the New Instrument Element"

Transcription

1 Instruments in Grid: the New Instrument Element C. Vuerli (1,2), G. Taffoni (1,2), I. Coretti (1), F. Pasian (1,2), P. Santin (1), M. Pucillo (1) (1) INAF Astronomical Observatory of Trieste (2) INAF Informative Systems Unit INGRID 2007 S. Margherita Ligure Portofino 17 April

2 The Goal Gridification of Telescopes and attached astronomical instrumentation to allow remote monitoring and control in Quasi-Real Time Step 1. Remote monitoring of telemetry and scientific data under acquisition Step 2. Remote control: the astronomer interacts with night assistant by remotely sending commands and receiving feedback INGRID 2007 S. Margherita Ligure Portofino 17 April

3 The Origin: The GRID.IT Project GRID.IT: National multi-disciplinary project to validate the Italian Grid infrastructure INAF participates with 3 applications Access to Astronomical Databases via Grid Porting on Grid the VST Pipelines Monitoring/Control of astronomical instrumentation INGRID 2007 S. Margherita Ligure Portofino 17 April

4 Astro DBs in Grid: drivers Customer whishes : Transparent access to data sources (DBs); Transparent access to computing resources; Complex workflows; IVOA standards for data access; GRID computing and storing resources; Integration of the VObs with Grid technology is of key importance to provide users with proper computing power INGRID 2007 S. Margherita Ligure Portofino 17 April

5 INGRID 2007 S. Margherita Ligure Portofino 17 April

6 Our Grid It is the Grid middleware that we used in the framework of GRID.IT Project Set up by INFN (National Institute of Nuclear Physics) to run jobs of the LHC (Large Hadron Collider) experiments (CERN, Geneva) LCG (LHC Computing Grid) is based on Globus 2.4 native computing and data storage resources Tests are now in progress to migrate LCG to glite (WebServices enabled) No native access to non-computational resources G-DSE: a low level integration framework In Globus and not on Globus INGRID 2007 S. Margherita Ligure Portofino 17 April

7 Control systems in Grid Grid TeleControl Protocol (GTCP) JavaServices + GT4 container Xport (X-ray Cristallography) WebServices GridCC Web Services G-ICS: a low level integration framework on a production grid INGRID 2007 S. Margherita Ligure Portofino 17 April

8 Toward a Generalized Grid Element To have the Generalised Grid Element as a new Grid element embedded in its middleware some components of it must be revised and modified The new generalised element (and all of its specialisations) enhances the CE in terms of functionalities and intrinsic characteristics These intrinsic characteristics are expressed by means of a set of new metadata, that enhance the set of meta-data characterising traditional CE and a SE elements The new Grid middleware has to be able to Recognise this new set of meta-data Ingest them in its Information System Publish them to make users operating at client machines able to discover these new resources Some components of the middleware have then to be revised and modified. They are those related with the Grid Information System, namely the BDII (Berkeley Database Information Index), the GRIS (Grid Resource Information Server), the MDS (Monitoring and Discovery Service) and the RB (Resource Broker) INGRID 2007 S. Margherita Ligure Portofino 17 April

9 Recycling G-DSE to enable G-ICS Provide a suitable formal definition of a Grid Abstract Computing Machine using Formal specification language Review the architectural definition of a Data Source Engine (DSE), to build an Instrument Control System Provide ICS integrated in the existing Grid Resource Layer Framework (G-ICS) Extend GIS capabilities to understand and handle monitoring and control requests to ICS Extend GIS capabilities to describe ICS Resources Provide a new Manager for ICS integrated in Globus Extend semantic and provide it to RSL to use G-ICS Etc INGRID 2007 S. Margherita Ligure Portofino 17 April

10 ldif G-DSE and G-ICS gatekeeper GRAM GIS MDS GRIS JobManger Scheduler p-in QueryManger query plug-in ICSManger ICS plug-in Ldap Pbs/LSF JobProcess RDBMS Query DB specific driver QueryProcess ICS specific driver ICSProcess ICS Instrument Control System Grid Providers (snmp) RDBMS INGRID 2007 S. Margherita Ligure Portofino 17 April

11 LCG-2: DB + ICS enabled middleware Application level services App monitoring system User access User interfaces Resource Broker VDT (Condor, Globus, GLUE) + XMLDB + XMLICS Basic services Information system Collective services Information schema Applications Data management Data transfer EU DataGrid +Glite + G-DSE + G-ICS Security Scientific Linux Operating system NFS, File system System software Local scheduler PBS, Condor, LSF, RDBMS MySQL Ora, etc. ICS Hardware Computing cluster Network resources Data storage HPSS, CASTOR Instruments INGRID 2007 S. Margherita Ligure Portofino 17 April

12 From G-DSE to QE With the G-DSE enabled collective and resource Grid layers, the new QE Grid Element is introduced QE is a CE able to manage Queries and to connect RDBMS resources via Grid It speaks RSL to interact with Grid services, VOQL when exchanging data with RDBMS resources and processing query requests coming from UIs (Users) INGRID 2007 S. Margherita Ligure Portofino 17 April

13 From G-ICS to IE With the G-ICS enabled collective and resource Grid layers, the new IE of the Grid is introduced IE is a CE able to manage, monitor and control requests to interact with ICS resources via Grid It speaks RSL to interact with Grid services, and ICSL (Instrument Control System Language) when connecting an ICS INGRID 2007 S. Margherita Ligure Portofino 17 April

14 From G-ICS to IE What we need is: ICSP: a ICS Protocol to make IE and ICS able to communicate each other ICSL: a ICS Language used by IE to speak with ICS according to protocol ICSP ICSC: an Instrument Control System Connector, being it a driver able to speak ICSL and understand ICSP-compliant requests and map them in requests to the specific ICS sitting behind it handle answers to these requests INGRID 2007 S. Margherita Ligure Portofino 17 April

15 From G-ICS to IE ODBC client QE Odbc lang ODBC Driver Custom Oracle Mysql Postgres ICSC client IE ICSL ICSC Driver ICS 1 ICS 2 ICS 3 ICS 4 Mapping the DSE schema into ICS: The ODBC Driver translates standard SQL in the SQL (extensions included) of the specific DBMS to be contacted Similarly, the ICSC Driver translates standard ICSL in the specific ICSL (extensions included) of the ICS to be contacted Extensions retrieved from metadata repository via MDS INGRID 2007 S. Margherita Ligure Portofino 17 April

16 ICS Grid Access Enabled Grid Instrument Control System GRAM Protocol Job Manager Local Resource Manager ODBC ICSC Driver DSE ICS Instance snmpd ODBC ICSC Manager DSE ICS Instance Man snmpd GANGLIA Grid info system MDS LDAP Worker Node Tel UserCSDB DSE ICS Meta Machine Synchro Synchro Internal DB CS INGRID 2007 S. Margherita Ligure Portofino 17 April

17 VOQL External IVOA Resource (Registry) UI VOQL Long Term Goal: Grid and VObs RSL(VOQL) Data Grid Information and Brokering System RSL RSL Data not available in Grid RSL(VOQL) Data found in Grid QE MetaData Outside Grid (VObs) QE QE VOQL AstroDB VOQL VOQL AstroDB INGRID 2007 S. Margherita Ligure Portofino 17 April AstroDB

18 Long Term Goal: Grid and VObs MetaData UI Mon/Ctrl requests Data Grid Information and Brokering System RSL(VOQL) QE IE RSL RSL IE ICSL ICSL ICSC Instrument Control System Instrument Control System INGRID 2007 S. Margherita Ligure Portofino 17 April ICSC

19 In Summary: What is the IE? The IE inherits from the CE, so it has all the characteristics of a classic computing element The IE, moreover, makes possible to Publish on the Grid any kind of scientific instrumentation. Telescopes and related astronomical instrumentation are just an example of it Share on the Grid infrastructure any kind of scientific instrumentation Get access to any kind of scientific instrumentation shared on the Grid infrastructure Perform remote monitoring sessions in which the Grid technology only is used Perform remote control sessions in which the Grid technology only is used INGRID 2007 S. Margherita Ligure Portofino 17 April

20 Current Status Institutes that collaborated in the framework of GRID.IT Project INAF Trieste IE project coordination and development/test G-DSE development/test INAF Padova G-DSE development/test INAF Capodimonte (Naples) IE development/test INFN-CNAF G-DSE development/test INGRID 2007 S. Margherita Ligure Portofino 17 April

21 Current Status We are going on through subsequent refinement steps For the time being monitoring only implemented ICS not fully implemented on Grid yet At present the worker node (at Telescope Site) and the IE speak HTML. IE does not use G-ICS yet INGRID 2007 S. Margherita Ligure Portofino 17 April

22 Current Status Two test-bed ICS identified (at Trieste and Naples observatories) Tests successfully carried out on LCG implementation of the Grid middleware (no G-ICS extension) New project proposals and requests for funding under evaluation to carry out the implementation of the Instrument Element INGRID 2007 S. Margherita Ligure Portofino 17 April

23 Test-bed and the first prototype of UIF First prototype of UIF written in Java The Celestron C-14 located at the Basovizza branch station of INAF-OATs INGRID 2007 S. Margherita Ligure Portofino 17 April

24 The final goal: An Integrated System other IE SE QE DBMS Grid-UI IE The GRID IE DBMS QE Grid-GUI CE SE CE INGRID 2007 S. Margherita Ligure Portofino 17 April

25 End of Presentation Thank you for your attention INGRID 2007 S. Margherita Ligure Portofino 17 April

MIGRATING DESKTOP AND ROAMING ACCESS. Migrating Desktop and Roaming Access Whitepaper

MIGRATING DESKTOP AND ROAMING ACCESS. Migrating Desktop and Roaming Access Whitepaper Migrating Desktop and Roaming Access Whitepaper Poznan Supercomputing and Networking Center Noskowskiego 12/14 61-704 Poznan, POLAND 2004, April white-paper-md-ras.doc 1/11 1 Product overview In this whitepaper

More information

DAME Astrophysical DAta Mining Mining & & Exploration Exploration GRID

DAME Astrophysical DAta Mining Mining & & Exploration Exploration GRID DAME Astrophysical DAta Mining & Exploration on GRID M. Brescia S. G. Djorgovski G. Longo & DAME Working Group Istituto Nazionale di Astrofisica Astronomical Observatory of Capodimonte, Napoli Department

More information

Roberto Barbera. Centralized bookkeeping and monitoring in ALICE

Roberto Barbera. Centralized bookkeeping and monitoring in ALICE Centralized bookkeeping and monitoring in ALICE CHEP INFN 2000, GRID 10.02.2000 WP6, 24.07.2001 Roberto 1 Barbera ALICE and the GRID Phase I: AliRoot production The GRID Powered by ROOT 2 How did we get

More information

Sun Grid Engine, a new scheduler for EGEE middleware

Sun Grid Engine, a new scheduler for EGEE middleware Sun Grid Engine, a new scheduler for EGEE middleware G. Borges 1, M. David 1, J. Gomes 1, J. Lopez 2, P. Rey 2, A. Simon 2, C. Fernandez 2, D. Kant 3, K. M. Sephton 4 1 Laboratório de Instrumentação em

More information

An approach to grid scheduling by using Condor-G Matchmaking mechanism

An approach to grid scheduling by using Condor-G Matchmaking mechanism An approach to grid scheduling by using Condor-G Matchmaking mechanism E. Imamagic, B. Radic, D. Dobrenic University Computing Centre, University of Zagreb, Croatia {emir.imamagic, branimir.radic, dobrisa.dobrenic}@srce.hr

More information

The Grid-it: the Italian Grid Production infrastructure

The Grid-it: the Italian Grid Production infrastructure n 1 Maria Cristina Vistoli INFN CNAF, Bologna Italy The Grid-it: the Italian Grid Production infrastructure INFN-Grid goals!promote computational grid technologies research & development: Middleware and

More information

Sun Grid Engine, a new scheduler for EGEE

Sun Grid Engine, a new scheduler for EGEE Sun Grid Engine, a new scheduler for EGEE G. Borges, M. David, J. Gomes, J. Lopez, P. Rey, A. Simon, C. Fernandez, D. Kant, K. M. Sephton IBERGRID Conference Santiago de Compostela, Spain 14, 15, 16 May

More information

Data Management in an International Data Grid Project. Timur Chabuk 04/09/2007

Data Management in an International Data Grid Project. Timur Chabuk 04/09/2007 Data Management in an International Data Grid Project Timur Chabuk 04/09/2007 Intro LHC opened in 2005 several Petabytes of data per year data created at CERN distributed to Regional Centers all over the

More information

Concepts and Architecture of Grid Computing. Advanced Topics Spring 2008 Prof. Robert van Engelen

Concepts and Architecture of Grid Computing. Advanced Topics Spring 2008 Prof. Robert van Engelen Concepts and Architecture of Grid Computing Advanced Topics Spring 2008 Prof. Robert van Engelen Overview Grid users: who are they? Concept of the Grid Challenges for the Grid Evolution of Grid systems

More information

Integrating a heterogeneous and shared Linux cluster into grids

Integrating a heterogeneous and shared Linux cluster into grids Integrating a heterogeneous and shared Linux cluster into grids 1,2 1 1,2 1 V. Büge, U. Felzmann, C. Jung, U. Kerzel, 1 1 1 M. Kreps, G. Quast, A. Vest 1 2 DPG Frühjahrstagung March 28 31, 2006 Dortmund

More information

Analyses on functional capabilities of BizTalk Server, Oracle BPEL Process Manger and WebSphere Process Server for applications in Grid middleware

Analyses on functional capabilities of BizTalk Server, Oracle BPEL Process Manger and WebSphere Process Server for applications in Grid middleware Analyses on functional capabilities of BizTalk Server, Oracle BPEL Process Manger and WebSphere Process Server for applications in Grid middleware R. Goranova University of Sofia St. Kliment Ohridski,

More information

(RH 7.3, gcc 2.95.2,VDT 1.1.6, EDG 1.4.3, GLUE, RLS) Tokyo BNL TAIWAN RAL 20/03/2003 20/03/2003 CERN 15/03/2003 15/03/2003 FNAL 10/04/2003 CNAF

(RH 7.3, gcc 2.95.2,VDT 1.1.6, EDG 1.4.3, GLUE, RLS) Tokyo BNL TAIWAN RAL 20/03/2003 20/03/2003 CERN 15/03/2003 15/03/2003 FNAL 10/04/2003 CNAF Our a c t i v i t i e s & c o n c e rn s E D G - L C G t r a n s i t i o n / c o n v e r g e n c e p l a n EDG s i d e : i n t e g r a t i o n o f n e w m i d d l e w a r e, t e s t b e d e v a l u a t

More information

The CMS analysis chain in a distributed environment

The CMS analysis chain in a distributed environment The CMS analysis chain in a distributed environment on behalf of the CMS collaboration DESY, Zeuthen,, Germany 22 nd 27 th May, 2005 1 The CMS experiment 2 The CMS Computing Model (1) The CMS collaboration

More information

The GENIUS Grid Portal

The GENIUS Grid Portal The GENIUS Grid Portal (*) work in collaboration with A. Falzone and A. Rodolico EGEE NA4 Workshop, Paris, 18.12.2003 CHEP 2000, 10.02.2000 Outline Introduction Grid portal architecture and requirements

More information

EDG Project: Database Management Services

EDG Project: Database Management Services EDG Project: Database Management Services Leanne Guy for the EDG Data Management Work Package EDG::WP2 Leanne.Guy@cern.ch http://cern.ch/leanne 17 April 2002 DAI Workshop Presentation 1 Information in

More information

Introduction to Grid computing

Introduction to Grid computing Introduction to Grid computing The INFNGrid Project Team Introduction This tutorial has been implemented considering as starting point the DataGrid (EDG) tutorial Many thanks to the EDG tutorials team!

More information

Tier Architectures. Kathleen Durant CS 3200

Tier Architectures. Kathleen Durant CS 3200 Tier Architectures Kathleen Durant CS 3200 1 Supporting Architectures for DBMS Over the years there have been many different hardware configurations to support database systems Some are outdated others

More information

A Taxonomy and Survey of Grid Resource Planning and Reservation Systems for Grid Enabled Analysis Environment

A Taxonomy and Survey of Grid Resource Planning and Reservation Systems for Grid Enabled Analysis Environment A Taxonomy and Survey of Grid Resource Planning and Reservation Systems for Grid Enabled Analysis Environment Arshad Ali 3, Ashiq Anjum 3, Atif Mehmood 3, Richard McClatchey 2, Ian Willers 2, Julian Bunn

More information

Analisi di un servizio SRM: StoRM

Analisi di un servizio SRM: StoRM 27 November 2007 General Parallel File System (GPFS) The StoRM service Deployment configuration Authorization and ACLs Conclusions. Definition of terms Definition of terms 1/2 Distributed File System The

More information

Recommendations for Static Firewall Configuration in D-Grid

Recommendations for Static Firewall Configuration in D-Grid D-Grid Integrationsprojekt (DGI-2) Fachgebiet 3-3 Firewalls Recommendations for Static Firewall Configuration in D-Grid Version 1.5, 21. Mai 2008 D-Grid Integrationsprojekt (DGI-2) Autoren: Gian Luca Volpato

More information

Status and Integration of AP2 Monitoring and Online Steering

Status and Integration of AP2 Monitoring and Online Steering Status and Integration of AP2 Monitoring and Online Steering Daniel Lorenz - University of Siegen Stefan Borovac, Markus Mechtel - University of Wuppertal Ralph Müller-Pfefferkorn Technische Universität

More information

Monitoring Clusters and Grids

Monitoring Clusters and Grids JENNIFER M. SCHOPF AND BEN CLIFFORD Monitoring Clusters and Grids One of the first questions anyone asks when setting up a cluster or a Grid is, How is it running? is inquiry is usually followed by the

More information

Global Grid User Support - GGUS - in the LCG & EGEE environment

Global Grid User Support - GGUS - in the LCG & EGEE environment Global Grid User Support - GGUS - in the LCG & EGEE environment Torsten Antoni (torsten.antoni@iwr.fzk.de) Why Support? New support groups Network layer Resource centers CIC / GOC / etc. more to come New

More information

Heterogeneous Database Replication Gianni Pucciani

Heterogeneous Database Replication Gianni Pucciani LCG Database Deployment and Persistency Workshop CERN 17-19 October 2005 Heterogeneous Database Replication Gianni Pucciani A.Domenici andrea.domenici@iet.unipi.it F.Donno flavia.donno@cern.ch L.Iannone

More information

CMS Dashboard of Grid Activity

CMS Dashboard of Grid Activity Enabling Grids for E-sciencE CMS Dashboard of Grid Activity Julia Andreeva, Juha Herrala, CERN LCG ARDA Project, EGEE NA4 EGEE User Forum Geneva, Switzerland March 1-3, 2006 http://arda.cern.ch ARDA and

More information

GridICE: monitoring the user/application activities on the grid

GridICE: monitoring the user/application activities on the grid GridICE: monitoring the user/application activities on the grid Cristina Aiftimiei 1, Sergio Andreozzi 2, Guido Cuscela 3, Stefano Dal Pra 1, Giacinto Donvito 3, Vihang Dudhalkar 3, Sergio Fantinel 4,

More information

DBMS Project. COP5725 - Spring 2011. Final Submission Report

DBMS Project. COP5725 - Spring 2011. Final Submission Report DBMS Project COP5725 - Spring 2011 Final Submission Report Chandra Shekar # 6610-6717 Nitin Gujral # 4149-1481 Rajesh Sindhu # 4831-2035 Shrirama Tejasvi # 7521-6735 LINK TO PROJECT Project Website : www.cise.ufl.edu/~mallela

More information

Plateforme de Calcul pour les Sciences du Vivant. SRB & glite. V. Breton. http://clrpcsv.in2p3.fr

Plateforme de Calcul pour les Sciences du Vivant. SRB & glite. V. Breton. http://clrpcsv.in2p3.fr SRB & glite V. Breton http://clrpcsv.in2p3.fr Introduction Goal: evaluation of existing technologies for data and tools integration and deployment Data and tools integration should be addressed using web

More information

The ENEA-EGEE site: Access to non-standard platforms

The ENEA-EGEE site: Access to non-standard platforms V INFNGrid Workshop Padova, Italy December 18-20 2006 The ENEA-EGEE site: Access to non-standard platforms C. Sciò**, G. Bracco, P. D'Angelo, L. Giammarino*, S.Migliori, A. Quintiliani, F. Simoni, S. Podda

More information

GRIP:Creating Interoperability between Grids

GRIP:Creating Interoperability between Grids GRIP:Creating Interoperability between Grids Philipp Wieder, Dietmar Erwin, Roger Menday Research Centre Jülich EuroGrid Workshop Cracow, October 29, 2003 Contents Motivation Software Base at a Glance

More information

Database Services for Physics @ CERN

Database Services for Physics @ CERN Database Services for Physics @ CERN Deployment and Monitoring Radovan Chytracek CERN IT Department Outline Database services for physics Status today How we do the services tomorrow? Performance tuning

More information

Experiences with the GLUE information schema in the LCG/EGEE production Grid

Experiences with the GLUE information schema in the LCG/EGEE production Grid Experiences with the GLUE information schema in the LCG/EGEE production Grid Stephen Burke, Sergio Andreozzi and Laurence Field CHEP07, Victoria, Canada www.eu-egee.org EGEE and glite are registered trademarks

More information

JBoss Data Services. Enabling Data as a Service with. Gnanaguru Sattanathan Twitter:@gnanagurus Website: bushorn.com

JBoss Data Services. Enabling Data as a Service with. Gnanaguru Sattanathan Twitter:@gnanagurus Website: bushorn.com 1 Enabling Data as a Service with JBoss Data Services Prajod Vettiyattil Twitter: @prajods Gnanaguru Sattanathan Twitter:@gnanagurus Website: bushorn.com 2 What this session is about v The why and what

More information

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft. Global Grid User Support - GGUS - within the LCG & EGEE environment

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft. Global Grid User Support - GGUS - within the LCG & EGEE environment Global Grid User Support - GGUS - within the LCG & EGEE environment Abstract: For very large projects like the LHC Computing Grid Project (LCG) involving some 8,000 scientists from universities and laboratories

More information

Deploying a distributed data storage system on the UK National Grid Service using federated SRB

Deploying a distributed data storage system on the UK National Grid Service using federated SRB Deploying a distributed data storage system on the UK National Grid Service using federated SRB Manandhar A.S., Kleese K., Berrisford P., Brown G.D. CCLRC e-science Center Abstract As Grid enabled applications

More information

CERN local High Availability solutions and experiences. Thorsten Kleinwort CERN IT/FIO WLCG Tier 2 workshop CERN 16.06.2006

CERN local High Availability solutions and experiences. Thorsten Kleinwort CERN IT/FIO WLCG Tier 2 workshop CERN 16.06.2006 CERN local High Availability solutions and experiences Thorsten Kleinwort CERN IT/FIO WLCG Tier 2 workshop CERN 16.06.2006 1 Introduction Different h/w used for GRID services Various techniques & First

More information

Towards Heterogeneous Grid Database Replication. Kemian Dang

Towards Heterogeneous Grid Database Replication. Kemian Dang Towards Heterogeneous Grid Database Replication Kemian Dang Master of Science Computer Science School of Informatics University of Edinburgh 2008 Abstract Heterogeneous database replication in the Grid

More information

SEE-GRID-SCI. www.see-grid-sci.eu. SEE-GRID-SCI USER FORUM 2009 Turkey, Istanbul 09-10 December, 2009

SEE-GRID-SCI. www.see-grid-sci.eu. SEE-GRID-SCI USER FORUM 2009 Turkey, Istanbul 09-10 December, 2009 SEE-GRID-SCI Grid Site Monitoring tools developed and used at SCL www.see-grid-sci.eu SEE-GRID-SCI USER FORUM 2009 Turkey, Istanbul 09-10 December, 2009 V. Slavnić, B. Acković, D. Vudragović, A. Balaž,

More information

E-mail: guido.negri@cern.ch, shank@bu.edu, dario.barberis@cern.ch, kors.bos@cern.ch, alexei.klimentov@cern.ch, massimo.lamanna@cern.

E-mail: guido.negri@cern.ch, shank@bu.edu, dario.barberis@cern.ch, kors.bos@cern.ch, alexei.klimentov@cern.ch, massimo.lamanna@cern. *a, J. Shank b, D. Barberis c, K. Bos d, A. Klimentov e and M. Lamanna a a CERN Switzerland b Boston University c Università & INFN Genova d NIKHEF Amsterdam e BNL Brookhaven National Laboratories E-mail:

More information

Data Grids. Lidan Wang April 5, 2007

Data Grids. Lidan Wang April 5, 2007 Data Grids Lidan Wang April 5, 2007 Outline Data-intensive applications Challenges in data access, integration and management in Grid setting Grid services for these data-intensive application Architectural

More information

DataNet Flexible Metadata Overlay over File Resources

DataNet Flexible Metadata Overlay over File Resources 1 DataNet Flexible Metadata Overlay over File Resources Daniel Harężlak 1, Marek Kasztelnik 1, Maciej Pawlik 1, Bartosz Wilk 1, Marian Bubak 1,2 1 ACC Cyfronet AGH, 2 AGH University of Science and Technology,

More information

TUTORIAL. Rebecca Breu, Bastian Demuth, André Giesler, Bastian Tweddell (FZ Jülich) {r.breu, b.demuth, a.giesler, b.tweddell}@fz-juelich.

TUTORIAL. Rebecca Breu, Bastian Demuth, André Giesler, Bastian Tweddell (FZ Jülich) {r.breu, b.demuth, a.giesler, b.tweddell}@fz-juelich. TUTORIAL Rebecca Breu, Bastian Demuth, André Giesler, Bastian Tweddell (FZ Jülich) {r.breu, b.demuth, a.giesler, b.tweddell}@fz-juelich.de September 2006 Outline Motivation & History Production UNICORE

More information

Fast Innovation requires Fast IT

Fast Innovation requires Fast IT Fast Innovation requires Fast IT 2014 Cisco and/or its affiliates. All rights reserved. 2 2014 Cisco and/or its affiliates. All rights reserved. 3 IoT World Forum Architecture Committee 2013 Cisco and/or

More information

Welcome to the User Support for EGEE Task Force Meeting

Welcome to the User Support for EGEE Task Force Meeting Welcome to the User Support for EGEE Task Force Meeting The agenda is as follows: Welcome Note & Presentation of the current GGUS Support system Basic Support Model Coffee brake Processes Lunch Break Interfaces

More information

Introduction. Introduction: Database management system. Introduction: DBS concepts & architecture. Introduction: DBS versus File system

Introduction. Introduction: Database management system. Introduction: DBS concepts & architecture. Introduction: DBS versus File system Introduction: management system Introduction s vs. files Basic concepts Brief history of databases Architectures & languages System User / Programmer Application program Software to process queries Software

More information

Retrieval on the Grid Results from the European Project GRACE (Grid Search and Categorization Engine)

Retrieval on the Grid Results from the European Project GRACE (Grid Search and Categorization Engine) Retrieval on the Grid Results from the European Project GRACE (Grid Search and Categorization Engine) FRANK SCHOLZE, WERNER STEPHAN Stuttgart University Library Main parts of this paper are based on the

More information

Esqu Science Experiments For Computer Network

Esqu Science Experiments For Computer Network Vangelis Floros, GRNET S.A. 3 rd Open Source Software Conference March 22, 2008 NTUA, Athens Greece Introduction What is a Grid? What is escience? Large Scientific Grids The example of EGEE Building Grid

More information

File Transfer Software and Service SC3

File Transfer Software and Service SC3 File Transfer Software and Service SC3 Gavin McCance JRA1 Data Management Cluster Service Challenge Meeting April 26 2005, Taipei www.eu-egee.org Outline Overview of Components Tier-0 / Tier-1 / Tier-2

More information

Introduction: Database management system

Introduction: Database management system Introduction Databases vs. files Basic concepts Brief history of databases Architectures & languages Introduction: Database management system User / Programmer Database System Application program Software

More information

Grid Scheduling Architectures with Globus GridWay and Sun Grid Engine

Grid Scheduling Architectures with Globus GridWay and Sun Grid Engine Grid Scheduling Architectures with and Sun Grid Engine Sun Grid Engine Workshop 2007 Regensburg, Germany September 11, 2007 Ignacio Martin Llorente Javier Fontán Muiños Distributed Systems Architecture

More information

ATLAS job monitoring in the Dashboard Framework

ATLAS job monitoring in the Dashboard Framework ATLAS job monitoring in the Dashboard Framework J Andreeva 1, S Campana 1, E Karavakis 1, L Kokoszkiewicz 1, P Saiz 1, L Sargsyan 2, J Schovancova 3, D Tuckett 1 on behalf of the ATLAS Collaboration 1

More information

Oracle BI EE Implementation on Netezza. Prepared by SureShot Strategies, Inc.

Oracle BI EE Implementation on Netezza. Prepared by SureShot Strategies, Inc. Oracle BI EE Implementation on Netezza Prepared by SureShot Strategies, Inc. The goal of this paper is to give an insight to Netezza architecture and implementation experience to strategize Oracle BI EE

More information

Portals and Resource Scheduling at Imperial College

Portals and Resource Scheduling at Imperial College Portals and Scheduling at Imperial College Dr Steven Newhouse Technical Director, London e-science Centre Department of Computing, Imperial College Background UK Regional e-science Centre Sun Centre of

More information

Consolidate by Migrating Your Databases to Oracle Database 11g. Fred Louis Enterprise Architect

Consolidate by Migrating Your Databases to Oracle Database 11g. Fred Louis Enterprise Architect Consolidate by Migrating Your Databases to Oracle Database 11g Fred Louis Enterprise Architect Agenda Why migrate to Oracle What is migration? What can you migrate to Oracle? SQL Developer Migration Workbench

More information

MEGA Web Application Architecture Overview MEGA 2009 SP4

MEGA Web Application Architecture Overview MEGA 2009 SP4 Revised: September 2, 2010 Created: March 31, 2010 Author: Jérôme Horber CONTENTS Summary This document describes the system requirements and possible deployment architectures for MEGA Web Application.

More information

Distributed Database Access in the LHC Computing Grid with CORAL

Distributed Database Access in the LHC Computing Grid with CORAL Distributed Database Access in the LHC Computing Grid with CORAL Dirk Duellmann, CERN IT on behalf of the CORAL team (R. Chytracek, D. Duellmann, G. Govi, I. Papadopoulos, Z. Xie) http://pool.cern.ch &

More information

The GRID and the Linux Farm at the RCF

The GRID and the Linux Farm at the RCF The GRID and the Linux Farm at the RCF A. Chan, R. Hogue, C. Hollowell, O. Rind, J. Smith, T. Throwe, T. Wlodek, D. Yu Brookhaven National Laboratory, NY 11973, USA The emergence of the GRID architecture

More information

Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo

Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Sensor Network Messaging Service Hive/Hadoop Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Contents 1 Introduction 2 What & Why Sensor Network

More information

Virtualization Infrastructure at Karlsruhe

Virtualization Infrastructure at Karlsruhe Virtualization Infrastructure at Karlsruhe HEPiX Fall 2007 Volker Buege 1),2), Ariel Garcia 1), Marcus Hardt 1), Fabian Kulla 1),Marcel Kunze 1), Oliver Oberst 1),2), Günter Quast 2), Christophe Saout

More information

The ENEA gateway approach providing EGEE/gLite access to unsupported platforms and operating systems

The ENEA gateway approach providing EGEE/gLite access to unsupported platforms and operating systems EU-IndiaGrid Workshop Taipei, November 2nd 2007 The ENEA gateway approach providing EGEE/gLite access to unsupported platforms and operating systems G. Bracco, S.Migliori, A. Quintiliani, A. Santoro, C.

More information

Big Science and Big Data Dirk Duellmann, CERN Apache Big Data Europe 28 Sep 2015, Budapest, Hungary

Big Science and Big Data Dirk Duellmann, CERN Apache Big Data Europe 28 Sep 2015, Budapest, Hungary Big Science and Big Data Dirk Duellmann, CERN Apache Big Data Europe 28 Sep 2015, Budapest, Hungary 16/02/2015 Real-Time Analytics: Making better and faster business decisions 8 The ATLAS experiment

More information

ViSION Status Update. Dan Savu Stefan Stancu. D. Savu - CERN openlab

ViSION Status Update. Dan Savu Stefan Stancu. D. Savu - CERN openlab ViSION Status Update Dan Savu Stefan Stancu D. Savu - CERN openlab 1 Overview Introduction Update on Software Defined Networking ViSION Software Stack HP SDN Controller ViSION Core Framework Load Balancer

More information

Grid Computing With FreeBSD

Grid Computing With FreeBSD Grid Computing With FreeBSD USENIX ATC '04: UseBSD SIG Boston, MA, June 29 th 2004 Brooks Davis, Craig Lee The Aerospace Corporation El Segundo, CA {brooks,lee}aero.org http://people.freebsd.org/~brooks/papers/usebsd2004/

More information

Integraion und Steuerung von verteilten Sensorsystemen. Grid-Package for a Robotic Telescope Network (RTN)

Integraion und Steuerung von verteilten Sensorsystemen. Grid-Package for a Robotic Telescope Network (RTN) Integraion und Steuerung von verteilten Sensorsystemen Grid-Package for a Robotic Telescope Network (RTN) Potsdam 2010 October 20, Harry Enke, Thomas Granzer Astrophysikalisches Institut Potsdam Overview

More information

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing New Paradigms: Clouds, Virtualization and Co. EGEE08, Istanbul, September 25, 2008 An Introduction to Virtualization and Cloud Technologies to Support Grid Computing Distributed Systems Architecture Research

More information

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4 Concepts and Architecture of the Grid Summary of Grid 2, Chapter 4 Concepts of Grid Mantra: Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations Allows

More information

Deploying Business Virtual Appliances on Open Source Cloud Computing

Deploying Business Virtual Appliances on Open Source Cloud Computing International Journal of Computer Science and Telecommunications [Volume 3, Issue 4, April 2012] 26 ISSN 2047-3338 Deploying Business Virtual Appliances on Open Source Cloud Computing Tran Van Lang 1 and

More information

Run Control and Monitor System for the CMS Experiment

Run Control and Monitor System for the CMS Experiment Computing in High Energy and Nuclear Physics, La Jolla CA, March 24-28, 2003 1 Run Control and Monitor System for the CMS Experiment V. Brigljevic, G. Bruno, E. Cano, S. Cittolin, A. Csilling, D. Gigi,

More information

System Requirements. Version 2015.0

System Requirements. Version 2015.0 System Requirements Version 2015.0 Copyright Copyright 2000-2015, NICE s.r.l. All right reserved. We'd Like to Hear from You You can help us make this document better by telling us what you think of the

More information

Technical Guide to ULGrid

Technical Guide to ULGrid Technical Guide to ULGrid Ian C. Smith Computing Services Department September 4, 2007 1 Introduction This document follows on from the User s Guide to Running Jobs on ULGrid using Condor-G [1] and gives

More information

Profiling as a Service

Profiling as a Service Profiling as a Service Table of Contents 1. PraaS Overview 2 2. The Profiling Goal 2 3. What do you get from Profiling? 2 4. How PraaS Improves the Profiling Experience 2 5. What is the Profiling Process?

More information

PoS(EGICF12-EMITC2)110

PoS(EGICF12-EMITC2)110 User-centric monitoring of the analysis and production activities within the ATLAS and CMS Virtual Organisations using the Experiment Dashboard system Julia Andreeva E-mail: Julia.Andreeva@cern.ch Mattia

More information

The Italian Grid Infrastructure (IGI) CGW08 Cracow. Mirco Mazzucato Italian Grid Infrastructure Coordinator INFN CNAF Director

The Italian Grid Infrastructure (IGI) CGW08 Cracow. Mirco Mazzucato Italian Grid Infrastructure Coordinator INFN CNAF Director The Italian Grid Infrastructure (IGI) CGW08 Cracow Mirco Mazzucato Italian Grid Infrastructure Coordinator INFN CNAF Director IGI -> National Grid Initiative Applications: Archeology Astronomy Astrophysics

More information

DSA1.4 R EPORT ON IMPLEMENTATION OF MONITORING AND OPERATIONAL SUPPORT SYSTEM. Activity: SA1. Partner(s): EENet, NICPB. Lead Partner: EENet

DSA1.4 R EPORT ON IMPLEMENTATION OF MONITORING AND OPERATIONAL SUPPORT SYSTEM. Activity: SA1. Partner(s): EENet, NICPB. Lead Partner: EENet R EPORT ON IMPLEMENTATION OF MONITORING AND OPERATIONAL SUPPORT SYSTEM Document Filename: Activity: Partner(s): Lead Partner: Document classification: BG-DSA1.4-v1.0-Monitoring-operational-support-.doc

More information

CNR-INFM DEMOCRITOS and SISSA elab Trieste

CNR-INFM DEMOCRITOS and SISSA elab Trieste elab and the FVG grid Stefano Cozzini CNR-INFM DEMOCRITOS and SISSA elab Trieste Agenda/Aims Present elab ant its computational infrastructure GRID-FVG structure basic requirements technical choices open

More information

Client/Server Grid applications to manage complex workflows

Client/Server Grid applications to manage complex workflows Client/Server Grid applications to manage complex workflows Filippo Spiga* on behalf of CRAB development team * INFN Milano Bicocca (IT) Outline Science Gateways and Client/Server computing Client/server

More information

K@ A collaborative platform for knowledge management

K@ A collaborative platform for knowledge management White Paper K@ A collaborative platform for knowledge management Quinary SpA www.quinary.com via Pietrasanta 14 20141 Milano Italia t +39 02 3090 1500 f +39 02 3090 1501 Copyright 2004 Quinary SpA Index

More information

Oracle BI Publisher Enterprise Cluster Deployment. An Oracle White Paper August 2007

Oracle BI Publisher Enterprise Cluster Deployment. An Oracle White Paper August 2007 Oracle BI Publisher Enterprise Cluster Deployment An Oracle White Paper August 2007 Oracle BI Publisher Enterprise INTRODUCTION This paper covers Oracle BI Publisher cluster and high availability deployment.

More information

FROM RELATIONAL TO OBJECT DATABASE MANAGEMENT SYSTEMS

FROM RELATIONAL TO OBJECT DATABASE MANAGEMENT SYSTEMS FROM RELATIONAL TO OBJECT DATABASE MANAGEMENT SYSTEMS V. CHRISTOPHIDES Department of Computer Science & Engineering University of California, San Diego ICS - FORTH, Heraklion, Crete 1 I) INTRODUCTION 2

More information

A Service for Data-Intensive Computations on Virtual Clusters

A Service for Data-Intensive Computations on Virtual Clusters A Service for Data-Intensive Computations on Virtual Clusters Executing Preservation Strategies at Scale Rainer Schmidt, Christian Sadilek, and Ross King rainer.schmidt@arcs.ac.at Planets Project Permanent

More information

Data Services @neurist and beyond

Data Services @neurist and beyond s @neurist and beyond Siegfried Benkner Department of Scientific Computing Faculty of Computer Science University of Vienna http://www.par.univie.ac.at Department of Scientific Computing Parallel Computing

More information

Securing your business

Securing your business Securing your business Anders Askåsen Product Manager for OpenIDM * World Wide Coverage ForgeRock.com Enterprise Open Source Software ForgeRock Norway ForgeRock USA ForgeRock UK ForgeRock France Consulting

More information

Open Unified Data Protection and Business Continuity Framework

Open Unified Data Protection and Business Continuity Framework Open Unified Data Protection and Business Continuity Framework Presenter: Dr. Anupam Bhide Calsoft, Inc. Email: anupam@calsoftinc.com Phone: +91 (20) 4079 2900 Authors: Anupam Bhide (Calsoft) Parag Kulkarni

More information

Resource Management on Computational Grids

Resource Management on Computational Grids Univeristà Ca Foscari, Venezia http://www.dsi.unive.it Resource Management on Computational Grids Paolo Palmerini Dottorato di ricerca di Informatica (anno I, ciclo II) email: palmeri@dsi.unive.it 1/29

More information

The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets

The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets!! Large data collections appear in many scientific domains like climate studies.!! Users and

More information

A Web-based Portal to Access and Manage WNoDeS Virtualized Cloud Resources

A Web-based Portal to Access and Manage WNoDeS Virtualized Cloud Resources A Web-based Portal to Access and Manage WNoDeS Virtualized Cloud Resources Davide Salomoni 1, Daniele Andreotti 1, Luca Cestari 2, Guido Potena 2, Peter Solagna 3 1 INFN-CNAF, Bologna, Italy 2 University

More information

Accessing Your Database with JMP 10 JMP Discovery Conference 2012 Brian Corcoran SAS Institute

Accessing Your Database with JMP 10 JMP Discovery Conference 2012 Brian Corcoran SAS Institute Accessing Your Database with JMP 10 JMP Discovery Conference 2012 Brian Corcoran SAS Institute JMP provides a variety of mechanisms for interfacing to other products and getting data into JMP. The connection

More information

Mission-Critical Database with Real-Time Search for Big Data

Mission-Critical Database with Real-Time Search for Big Data Mission-Critical Database with Real-Time Search for Big Data February 17, 2012 Slide 1 Overview About MarkLogic Why MarkLogic Case Studies Technology and Features Slide 2 About MarkLogic 10 years in business

More information

GRID Stream Database Management for Scientific Applications Milena Ivanova (Koparanova) and Tore Risch. IT Department, Uppsala University, Sweden

GRID Stream Database Management for Scientific Applications Milena Ivanova (Koparanova) and Tore Risch. IT Department, Uppsala University, Sweden GRID Stream Database Management for Scientific Applications Milena Ivanova (Koparanova) and Tore Risch IT Department, Uppsala University, Sweden Outline Motivation Stream Data Management Computational

More information

Survey and Taxonomy of Grid Resource Management Systems

Survey and Taxonomy of Grid Resource Management Systems Survey and Taxonomy of Grid Resource Management Systems Chaitanya Kandagatla University of Texas, Austin Abstract The resource management system is the central component of a grid system. This paper describes

More information

ARC Computing Element

ARC Computing Element NORDUGRID NORDUGRID-MANUAL-20 15/7/2015 ARC Computing Element System Administrator Guide F. Paganelli, Zs. Nagy, O. Smirnova, and various contributions from all ARC developers Contents 1 Overview 9 1.1

More information

16th International Conference on Control Systems and Computer Science (CSCS16 07)

16th International Conference on Control Systems and Computer Science (CSCS16 07) 16th International Conference on Control Systems and Computer Science (CSCS16 07) TOWARDS AN IO INTENSIVE GRID APPLICATION INSTRUMENTATION IN MEDIOGRID Dacian Tudor 1, Florin Pop 2, Valentin Cristea 2,

More information

DBdeployer. About the Speaker: CJ Estel, Operations Database Engineer CoverMyMeds cj@cjestel.net

DBdeployer. About the Speaker: CJ Estel, Operations Database Engineer CoverMyMeds cj@cjestel.net DBdeployer About the Speaker: CJ Estel, Operations Database Engineer CoverMyMeds cj@cjestel.net Github: https://github.com/covermymeds/dbdeployer https://github.com/cjestel Company Tech Blog: https://www.scriptscribe.org/

More information

Evaluating Metadata access

Evaluating Metadata access Evaluating Metadata access strategies with the GOME test suite André Gemünd Fraunhofer SCAI www.eu-egee.org EGEE-II INFSO-RI-031688 EGEE and glite are registered trademarks Motivation Testing the test

More information

GEDDM - Commercial Data Mining Using Distributed Resources

GEDDM - Commercial Data Mining Using Distributed Resources GEDDM - Commercial Data Mining Using Distributed Resources Mark Prentice 1 st December 2004 Introduction Industrial partner Overview of GEDDM project Application areas Grid enabled implementation Current

More information

Das HappyFace Meta-Monitoring Framework

Das HappyFace Meta-Monitoring Framework Das HappyFace Meta-Monitoring Framework B. Berge, M. Heinrich, G. Quast, A. Scheurer, M. Zvada, DPG Frühjahrstagung Karlsruhe, 28. März 1. April 2011 KIT University of the State of Baden-Wuerttemberg and

More information

THE CCLRC DATA PORTAL

THE CCLRC DATA PORTAL THE CCLRC DATA PORTAL Glen Drinkwater, Shoaib Sufi CCLRC Daresbury Laboratory, Daresbury, Warrington, Cheshire, WA4 4AD, UK. E-mail: g.j.drinkwater@dl.ac.uk, s.a.sufi@dl.ac.uk Abstract: The project aims

More information

XSEDE Service Provider Software and Services Baseline. September 24, 2015 Version 1.2

XSEDE Service Provider Software and Services Baseline. September 24, 2015 Version 1.2 XSEDE Service Provider Software and Services Baseline September 24, 2015 Version 1.2 i TABLE OF CONTENTS XSEDE Production Baseline: Service Provider Software and Services... i A. Document History... A-

More information

Service Desk Intelligence 4.5.7 System Requirements

Service Desk Intelligence 4.5.7 System Requirements Service Desk Intelligence 4.5.7 System Requirements with Business Objects 6.5 Westbury 2007 The information in this document is subject to change without notice. No part of this document may be photocopied,

More information

OGSA - A Guide to Data Access and Integration in UK

OGSA - A Guide to Data Access and Integration in UK The OGSA-DAI Project Databases and the Grid Neil Chue Hong Principal Consultant EPCC, Edinburgh N.ChueHong@epcc.ed.ac.uk What is OGSA-DAI? 4It is a project: OGSA Data Access and Integration: funded by

More information