WP8: SA Multi-messenger Data Access Service (HEAP-way)

Size: px

Start display at page:

Download "WP8: SA Multi-messenger Data Access Service (HEAP-way)"

Augusta Taylor
8 years ago
Views:

1 WP8: SA Multi-messenger Data Access Service (HEAP-way) Javier Rico (ICREA/IFAE, Spain) + Giovanni Lamanna (CNRS, France) + Daniele Martello (INFN Lecce, Italy) + Uli Katz (Universität Erlangen-Nürnberg, Germany) +Werner Hofmann (MPG, Germany)

2 Introduction 1 HEAP provides new observational windows to the Universe: gamma-rays (MAGIC, HESS, CTA), cosmic rays (Auger) and neutrinos (Antares, IceCube, KM3NeT) Still pending going from experiments to observatories through integration among different facilities and with the rest of Astronomy through open data access So far, HEAP facilities keep privacy of data, accessible only by the members of the collaborations. Different experiments have adopted self-made, particular data representations and analysis tools. Astronomy, astrophysics, astroparticle physics, particle physics and cosmology communities would very much benefit from open data access to present and future HEAP facilities

3 MM Data Access Service Activity (I) The Multi-messenger Data Access Service Activity (HEAP-WAY) proposes to make data obtained by ground-based gamma-ray, neutrino and cosmic-ray existing (HESS, MAGIC, AUGER and ANTARES) and future (CTA and KM3NeT) observatories available to the astrophysics and particle physics communities, in standard formats, together to standard analysis tools and through a standard access portal, and to provide a framework to preserve these data beyond the lifetime of the instruments or respective collaborations. Data made available under this service activity include: Scientific published results (spectra, light curves, sky maps) integrated into the International Virtual Observatory Alliance (IVOA) Archival data accumulated by the instruments and made open after certain latency periods HESS and MAGIC data acquired under the transnational access activity, initially proprietary to the proposer and made open after a certain period (WP6) Open access to multi-messengers data, to data-processing software, services, training and documentation will be provided by a single access point, the High Energy AstroParticle gateway. 2

4 MM Data Access Service Activity (II) Serving data to the scientific community requires a preparation face including the definition and implementation of standard data formats and analysis tools and furnishing the existing data centers with the resources needed to provide the service. Data will be made available by the instruments at different levels. Common format and archiving of event-level data (particle directions, energies, associated errors, quality etc.) will be investigated and implemented. In addition to data, software, standard tools and data dissemination services will be provided for scientific analysis. Formats and tools should be defined such that they can be applied for future instruments such as CTA and KM3NET. Released data will reside in their parent data centers, which will provide access also through a virtual common portal HEAP-WAY, which will also serve the public analysis software, documentation and general support 3

5 MM Data Access Service Activity (III) Data level Data will HESS be released MAGIC on 3 different processing CTA ANTARES levels KM3NET AUGER A B C D E Observatory-level files, such as survey maps and catalogues Format: Common for all (FITS-BASED, VO). Routinely provided for all data Binned high-level scientific products, such as sky images, spectra, time series, upper limits, etc Format: Common for all (FITS-BASED, VO). Routinely provided for all data One record per particle event, with corresponding reconstructed parameters such as energy, direction, nature. Provided together with the corresponding auxiliary files needed for background estimation and collection areas Format: Common for all (FITS-BASED) Provided for all data with 2 year latency Provided for all data with 2 year latency Calibrated images, hits, Format: Common Provided for selected data sets Routinely provided for all data Routinely provided for all data, latency tdb Raw, uncalibrated data Format: Specific not provided Tbd Tbd Tbd 4

6 Task 1. WP Coordination Months 1-48 Management of the WP to guarantee a smooth inter-play between partners Provide progress reports of the WP activities to the project management. Report on progress to general coordination. Foster publicity of the provided services Public announcement of data release (month 24) Document: Half-term WP Progress Report (month 30) Document: WP Final Report (month 48) 5

7 Task 2. Data models Months 1-18 Define the standard data models of the field at the different processing levels Evaluate existing formats in similar fields Identify requirements for HEAP data Evaluate data format compatibility with existing software Science Tools Define scheme for data model evolution Document: Definition of data formats at different levels (month 18) 6

8 Task 3. Analysis tools Months 1-48 Produce open-source analysis software Define and document the analysis chain: input/output for different stages Define and document coding standards and dependencies Test the code in real working conditions and with real data Produce documentation (at users and developers levels) Maintenance and support Public release of open-source analysis software (month 24) Document: analysis software manual (for developers) (month 30) 7

9 Task 4. HEAP-WAY portal Months 1-48 Setup and maintain HEAP-WAY web portal Links with particular data centers for data access at different processing levels Services to access and download portable software Establish bases for HEAP data legacy Maintain general support and documentation Release of the HEAP-WAY web-portal (month 18) 8

10 Task 5. Community support Months Organize HEAP data analysis training sessions (theoretical background + hands one data analysis) Integrate and maintain users' documentation Users online support Startup of online support service (month 24) Document: data access and analysis users manual (month 24) First training session (month 30) Second training session (month 40) 9

11 Task 6. Data access interface Months 1-48 Setup data access interface at the different processing levels Produce and maintain data book-keeping systems, database, catalogues, etc Produce and maintain data search engines Setup access interface to proprietary data from guest observation program (WP 6) Release of web-based data access general user interface (month 24) 10

12 Task 7. Data pipelines Months 1-48 Define computing model at datacenters Write data converters to new formats Produce processing internal software (scripting, automatization, error handling, ) and databases Test data pipelines (software+hardware) Process public data into new formats Maintain automatic, routine processing Release of data translators from particular to common data formats (month 18) Certification of data pipelines (month 20) Startup of automatic data conversion service (month 22) 11

13 Partners + budget (preliminary) Participant IFAE CNRS (LAPP and APC) Observatoire de Paris Medon MPG (Munich and Heidelberg) FZK Karlruhe Erlangen University INFN (Padova, Lecce and Naples) INAF (Rome) Total Requested resources: 1FTE 4 years 6 observatories 1FTE 4 years 2 (MAGIC+HESS) datacenters Country Spain France France Germany Germany Germany Italy Italy Total 8 FTEs for 4 years 2 M 12

Observer Access to the Cherenkov Telescope Array

Observer Access to the Cherenkov Telescope Array IRAP, Toulouse, France E-mail: jknodlseder@irap.omp.eu V. Beckmann APC, Paris, France E-mail: beckmann@apc.in2p3.fr C. Boisson LUTh, Paris, France E-mail: