EGI Federated Cloud, a building block for the Open Science Commons

Transcription

1 EGI Federated Cloud, a building block for the Open Science Commons Yannick LEGRÉ Director, EGI.eu EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number

2 EGI today 620,000 CPU Cores 500 PB of storage 21 Cloud providers 350 data centres 40,000 users +3,000 research papers 2

3 An international endeavor Canada USA Latin America Africa Arabia Ukraine China India Asia Pacific 3

4 EGI Federated Cloud platform EGI-ENGAGE Production operations since May 2014 Currently: 21 providers from 14 NGIs cores in total 800k VMs, 9M CPUh in last 12 months Integrations in progress imarine VREs Canadian cloud for astronomical data an international federated cloud for distributed access to open astronomical data Cloud services for training and education 4

5 EGI Federated Cloud EGI Federated Cloud is a collaboration of communities developing, innovating, operating and using cloud federations for research and education. The collaboration facilitates innovation and sustainability of participating clouds and related communities. Public Cloud Open to any research community, maintained by EGI FC Task Force Open Standards: use of open standards to implement federation: OCCI, OVF, GLUE2, CDMI, Standards required. Stronger integration profile: Cloud Computing integrated into the existing production infrastructure. Community Clouds Available for specific communities, maintained by them Community choices: Services and APIs to implement federation is community choice. Standards encouraged. Looser Federation profile: Based on a subset of EGI components (accounting, monitoring, ) 5

6 Architecture of the EGI Federated Cloud EGI-ENGAGE Manage instances Share & endorse VM images (OVF) Image replication Uniform interfaces and behaviour Cloud Site Cloud Site Cloud Site (OpenStack) (OpenNebula) Cloud Providers (currently 21) (Synnefo) Endorsed VM images Appliances Marketplace (AppDB) Service registry Information system Accounting Operation services EGI Core Platform Monitoring AAI (VOs) 6

7 Collaboration Platform VM Image Catalogue of EGI endorsed VM images, Helpdesk EGI-ENGAGE Community Platforms Brokering, community-specific data, tools and applications Data-intensive computing HTC Platform Cloud compute and storage GPGPU Platform Open Data Platform EGI Core Infrastructure Platform AAI, Service Registry, Accounting, Monitoring Federated Service Management Physical Infrastructure 7

8 EGI Federator Services EGI-ENGAGE Single Sign-On Service Registry Accounting Service Availability and Reliability Monitoring Information System Virtual Machine Image catalogue Virtual Machine Image replication mechanism Helpdesk 8

9 EGI Solutions A combination of products, services, and intellectual property focused on solving a problem (opportunity) that creates and/or drives value (measurable) and can be significantly standardised High- Throughput Data Analysis Federated Cloud EGI.eu, NGIs (nationally)/eiros, resource centres (locally) EGI.eu SERVICES Community Driven Innovation & Support Federated Operations User communities DATA, KNOWLEDGE, EXPERTISE Collaborating projects and technology providers SOFTWARE 9

10 Open Science: a Complex Resource System Shared resources Integrated, easy and fair access Engaged communities Participating in the process Culture of sharing Collaborating in the management and stewardship Governance Rules to access Rules to resolve conflicts Rules to balance quality vs. openness Financial support For long-term availability Research Data Instruments Digital services and applications Knowledge & Expertise 10

11 Commons Institutionalised community governance of the production and/or sharing of a particular type of resource (from natural to intellectual) GÉANT: European Communications Commons Constructing Genome Commons Wikipedia e-infrastructure Commons Linux Internet 11

12 Open Science Commons: Definition A set of interrelated resource systems governed as commons that support the open creation and dissemination of scholarly knowledge An area of study in the commons theory applied to open science Research Data Digital services and applications Instruments Knowledge & Expertise website: - paper: 12

13 Developing an OSC (1): Governance structure and funding models Analyse governance structure of existing infrastructure/knowledge resource systems in open science Identify best practices and patterns for commoning Develop guidelines Define a multi-level governance International and national bringing together the different stakeholders including communities Identify funding models for sustainability and capacity building 13

14 Developing an OSC (2): Open Science Backbone The set of standardised generic ICT capabilities across countries and communities offering Compute/Data intensive capabilities Easy discovery, access, use and reuse of open data Shared capacity for RIs and long tail Supporting open standards (both API and data formats) Complemented by community-specific and communitymanaged services Research Infrastructures 14

15 Federated operations and support Service desk Monitoring and accounting Capacity management Service level management Developing an OSC (2): Shared Open Science Infrastructure Backbone Network of CSIRT Federated IdPs, Auth and Authz Management of different levels of assurance Research Infrastructures and long tail of science Research platform built on top of shared capabilities plus community owned resources Data products, tools, scientific gateways, virtual labs Multi-level governance with community participation Local National European Shared capabilities based on open standards Common national pools of resources From Member States Capacity dedicated to large RIs Free pools for long tail researchers Both pubiicly funded and commercial providers (all supporting open standards and no lock-in) Core capabilities Open Science Cloud (e.g., VM management, Data storage/access/disco very) PID Service registry and marketplace 15

16 Developing an OSC (3): Research data Stimulate the creation of public repositories of open research data Stimulate a culture of sharing and the right incentives to contribute and maintain Address legal and policy issues Prefer minimal IPR or non-exclusive licensing Many initiatives exist (e.g. RDA, CODATA) 16

17 Developing an OSC (4): Open Knowledge Hubs A coordinated network of competence centers Offering knowledge and expertise for scientific software, applications, tools Knowledge and expertise from a network of European training and education centres Scientific software is open, documented, discoverable, supported Support to access different capabilities (HTC, HPC, cloud, open research data, tools, applications, software ) Training and education from ESFRI RIs and VRE projects Distributed Competence Centres e- Infrastructure support and training Open Knowledge Hub 17

18 How can EGI contribute? Federate digital capabilities, resources and expertise Operate services across the federated infrastructure Co-create and integrate open and user-driven services and solutions Be a trusted adviser on data and compute intensive science 18

19 Thank you for your attention. Questions? EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number