Instruments in Grid: the New Instrument Element C. Vuerli (1,2), G. Taffoni (1,2), I. Coretti (1), F. Pasian (1,2), P. Santin (1), M. Pucillo (1) (1) INAF Astronomical Observatory of Trieste (2) INAF Informative Systems Unit INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 1
The Goal Gridification of Telescopes and attached astronomical instrumentation to allow remote monitoring and control in Quasi-Real Time Step 1. Remote monitoring of telemetry and scientific data under acquisition Step 2. Remote control: the astronomer interacts with night assistant by remotely sending commands and receiving feedback INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 2
The Origin: The GRID.IT Project GRID.IT: National multi-disciplinary project to validate the Italian Grid infrastructure INAF participates with 3 applications Access to Astronomical Databases via Grid Porting on Grid the VST Pipelines Monitoring/Control of astronomical instrumentation INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 3
Astro DBs in Grid: drivers Customer whishes : Transparent access to data sources (DBs); Transparent access to computing resources; Complex workflows; IVOA standards for data access; GRID computing and storing resources; Integration of the VObs with Grid technology is of key importance to provide users with proper computing power INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 4
INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 5
Our Grid It is the Grid middleware that we used in the framework of GRID.IT Project Set up by INFN (National Institute of Nuclear Physics) to run jobs of the LHC (Large Hadron Collider) experiments (CERN, Geneva) LCG (LHC Computing Grid) is based on Globus 2.4 native computing and data storage resources Tests are now in progress to migrate LCG to glite (WebServices enabled) No native access to non-computational resources G-DSE: a low level integration framework In Globus and not on Globus INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 6
Control systems in Grid Grid TeleControl Protocol (GTCP) JavaServices + GT4 container Xport (X-ray Cristallography) WebServices GridCC Web Services G-ICS: a low level integration framework on a production grid INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 7
Toward a Generalized Grid Element To have the Generalised Grid Element as a new Grid element embedded in its middleware some components of it must be revised and modified The new generalised element (and all of its specialisations) enhances the CE in terms of functionalities and intrinsic characteristics These intrinsic characteristics are expressed by means of a set of new metadata, that enhance the set of meta-data characterising traditional CE and a SE elements The new Grid middleware has to be able to Recognise this new set of meta-data Ingest them in its Information System Publish them to make users operating at client machines able to discover these new resources Some components of the middleware have then to be revised and modified. They are those related with the Grid Information System, namely the BDII (Berkeley Database Information Index), the GRIS (Grid Resource Information Server), the MDS (Monitoring and Discovery Service) and the RB (Resource Broker) INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 8
Recycling G-DSE to enable G-ICS Provide a suitable formal definition of a Grid Abstract Computing Machine using Formal specification language Review the architectural definition of a Data Source Engine (DSE), to build an Instrument Control System Provide ICS integrated in the existing Grid Resource Layer Framework (G-ICS) Extend GIS capabilities to understand and handle monitoring and control requests to ICS Extend GIS capabilities to describe ICS Resources Provide a new Manager for ICS integrated in Globus Extend semantic and provide it to RSL to use G-ICS Etc INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 9
ldif G-DSE and G-ICS gatekeeper GRAM GIS MDS GRIS JobManger Scheduler p-in QueryManger query plug-in ICSManger ICS plug-in Ldap Pbs/LSF JobProcess RDBMS Query DB specific driver QueryProcess ICS specific driver ICSProcess ICS Instrument Control System Grid Providers (snmp) RDBMS INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 10
LCG-2: DB + ICS enabled middleware Application level services App monitoring system User access User interfaces Resource Broker VDT (Condor, Globus, GLUE) + XMLDB + XMLICS Basic services Information system Collective services Information schema Applications Data management Data transfer EU DataGrid +Glite + G-DSE + G-ICS Security Scientific Linux Operating system NFS, File system System software Local scheduler PBS, Condor, LSF, RDBMS MySQL Ora, etc. ICS Hardware Computing cluster Network resources Data storage HPSS, CASTOR Instruments INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 11
From G-DSE to QE With the G-DSE enabled collective and resource Grid layers, the new QE Grid Element is introduced QE is a CE able to manage Queries and to connect RDBMS resources via Grid It speaks RSL to interact with Grid services, VOQL when exchanging data with RDBMS resources and processing query requests coming from UIs (Users) INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 12
From G-ICS to IE With the G-ICS enabled collective and resource Grid layers, the new IE of the Grid is introduced IE is a CE able to manage, monitor and control requests to interact with ICS resources via Grid It speaks RSL to interact with Grid services, and ICSL (Instrument Control System Language) when connecting an ICS INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 13
From G-ICS to IE What we need is: ICSP: a ICS Protocol to make IE and ICS able to communicate each other ICSL: a ICS Language used by IE to speak with ICS according to protocol ICSP ICSC: an Instrument Control System Connector, being it a driver able to speak ICSL and understand ICSP-compliant requests and map them in requests to the specific ICS sitting behind it handle answers to these requests INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 14
From G-ICS to IE ODBC client QE Odbc lang ODBC Driver Custom Oracle Mysql Postgres ICSC client IE ICSL ICSC Driver ICS 1 ICS 2 ICS 3 ICS 4 Mapping the DSE schema into ICS: The ODBC Driver translates standard SQL in the SQL (extensions included) of the specific DBMS to be contacted Similarly, the ICSC Driver translates standard ICSL in the specific ICSL (extensions included) of the ICS to be contacted Extensions retrieved from metadata repository via MDS INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 15
ICS Grid Access Enabled Grid Instrument Control System GRAM Protocol Job Manager Local Resource Manager ODBC ICSC Driver DSE ICS Instance snmpd ODBC ICSC Manager DSE ICS Instance Man snmpd GANGLIA Grid info system MDS LDAP Worker Node Tel UserCSDB DSE ICS Meta Machine Synchro Synchro Internal DB CS INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 16
VOQL External IVOA Resource (Registry) UI VOQL Long Term Goal: Grid and VObs RSL(VOQL) Data Grid Information and Brokering System RSL RSL Data not available in Grid RSL(VOQL) Data found in Grid QE MetaData Outside Grid (VObs) QE QE VOQL AstroDB VOQL VOQL AstroDB INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 17 AstroDB
Long Term Goal: Grid and VObs MetaData UI Mon/Ctrl requests Data Grid Information and Brokering System RSL(VOQL) QE IE RSL RSL IE ICSL ICSL ICSC Instrument Control System Instrument Control System INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 18 ICSC
In Summary: What is the IE? The IE inherits from the CE, so it has all the characteristics of a classic computing element The IE, moreover, makes possible to Publish on the Grid any kind of scientific instrumentation. Telescopes and related astronomical instrumentation are just an example of it Share on the Grid infrastructure any kind of scientific instrumentation Get access to any kind of scientific instrumentation shared on the Grid infrastructure Perform remote monitoring sessions in which the Grid technology only is used Perform remote control sessions in which the Grid technology only is used INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 19
Current Status Institutes that collaborated in the framework of GRID.IT Project INAF Trieste IE project coordination and development/test G-DSE development/test INAF Padova G-DSE development/test INAF Capodimonte (Naples) IE development/test INFN-CNAF G-DSE development/test INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 20
Current Status We are going on through subsequent refinement steps For the time being monitoring only implemented ICS not fully implemented on Grid yet At present the worker node (at Telescope Site) and the IE speak HTML. IE does not use G-ICS yet INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 21
Current Status Two test-bed ICS identified (at Trieste and Naples observatories) Tests successfully carried out on LCG implementation of the Grid middleware (no G-ICS extension) New project proposals and requests for funding under evaluation to carry out the implementation of the Instrument Element INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 22
Test-bed and the first prototype of UIF First prototype of UIF written in Java The Celestron C-14 located at the Basovizza branch station of INAF-OATs INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 23
The final goal: An Integrated System other IE SE QE DBMS Grid-UI IE The GRID IE DBMS QE Grid-GUI CE SE CE INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 24
End of Presentation Thank you for your attention INGRID 2007 S. Margherita Ligure Portofino 17 April 2007 25