Complex Problem Solving including GRID and Research Networking Infrastructures

Transcription

1 Complex Problem Solving including GRID and Research Networking Infrastructures Contributions from the FP6 Internal Reflection Group Draft report version th May

2 Summary...3 Excerpts from the Specific Programme...5 Vision...6 Complex Problems Challenges and Motivation...7 Grid and Peer-to-Peer...9 Distributed and Shared Infrastructure for e-learning...12 Research Network Infrastructures...13 Global Monitoring for Environment and Security...15 Annex I GRID Activities...16 Summary publicly funded R&D Grid Projects Summary of GRID / P2P Activities by IT Companies ANNEX II Complex Problems...26 Application Areas Classification of Complex Problems Annex III Draft - SWOT Analysis...31 Technology Areas Grid computing Peer-to-Peer Computing Research Network Infrastructures Mobile/Wireless IPv Application Areas Distributed and Shared Infrastructure for e-learning Complex Systems in Industrial Engineering (examples) Global Monitoring for the Environment and Security (GMES) Annex IV List of Background Documents...38 Annex V Members of the IRG...39 Annex VI Glossary and Acronyms

3 Summary Potential research activities in the field of complex problem solving in science, engineering, business and for the society, GRID and Peer-to-Peer (P2P) technologies able to solve those and the enabling networking infrastructure are topics of this report. The report highlights different research perspectives (both from a technical and application viewpoint) but does not pretend to be exhaustive. The report starts with an overview of what is meant by complex problems in science, industry and business. There is a wide variety of compute-intensive and/or data-centric problems and ways of solving them. The requirements these problems have on the underlying computational, communication and storage architecture can be quite different. The different types of players at all relevant levels of the value chain that are interested in those problems or can contribute to their solution can be considered as the potential drivers for Grid/P2P-like technologies that may become the basis for future ICT infrastructures. The GRID computing concept (a form of distributed computing) has shown considerable evolution since its inception a few years ago. Initially the term GRID was used for describing the pooling of distributed (local and/or remote) compute resources over the Internet. The massive integration of computer systems (processor and storage) offers performance unattainable by a single system. P2P technologies have been around in various forms (networking, computing, and applications) for more than a decade but have become famous through file sharing and distributed applications (e.g. seti@home). In P2P networks a group of computers communicate directly with each other rather than via central servers (the opposite of client-server model). The connectivity and diversity of the Internet is giving P2P computing a new role a role that allows the growing computing and data resources at the egdes of the network to achieve their full potential. P2P computing - forming a distributed information infrastructure, in which all information assets and resources of an organisation are brought together in a co-ordinated fashion - can be particularly effective in spanning geographical and organisational boundaries. P2P computing is intended to complement existing client/server applications and to open up new opportunities to take advantage of the computing assets and resources of both individuals and organisations. P2P and Grid computing are two collaborative computing technologies with different origins but many common applications and concerns. Grids are historically high-end, predominately e-science oriented. P2P is more low-end, strongly commodity oriented. Many middleware requirements, like security, are similar. More recently the term Grid evolved to mean also an inclusive structure linking people, communities, information, tools and facilities 1. Lately IT companies are picking up GRID/P2P concepts and see them as means to deploy utility computing and as enabler for webservices. Without widespread availability of fast and reliable Internet connectivity Grid and P2P applications would be restricted to resource sharing within a lab or organisation. Today research teams and research facilities, both in industry, business and universities are geographically dispersed, and only high speed networks (comparable to the bus of the computer used) will enable them to share data and computing infrastructure in real time. Global high-speed Internet connectivity assuring meaningful performance end-to-end is the first layer of useful Grid architecture. High-speed networks open up new possibilities for collaborative learning and researching. We have seen important shifts in the way we use networks: rather than simply transmitting lots of raw bits, we are sending information that is formatted in standard ways (html tags and data) that convey content and actions to be done (service protocols, executable content). These developments lead to information networking and the semantic web. 1 The term GRID refers to an emerging network-based computing infrastructure providing security, resource access, information and other services that enable the controlled and coordinated sharing of resources among virtual organisations formed dynamically by individuals and institutions with common interests. The Anatomy of the Grid: Enabling scalable virtual organizations, I. Foster, C. Kesselman and S. Tuecke, Int. J. Supercomputer Applic. 15, 3 (Fall 2001); see also IRG Complex Problem Solving including GRID and RNI 3 Draft V /05/02

4 The SWOT analysis in Annex III try to make an in depth critique of the technologies and assess opportunities for deployment. The EU and individual countries have already funded several substantial grid-related projects in the last two years, some for infrastructure and middleware development and some for applications that take advantage of the Grid concept. They are all addressing different research topics and fields spanning from High Energy Physics to bioinformatics and medical applications, yet using a de facto US developed open middleware standard (with one notable exception). Applications developed in academia today are often the basis for the commercial applications of tomorrow. The Member States and Commission must ensure that this potential for innovation is fully exploited. Europe should adapt a coherent strategy with respect to computing and communications, which, on one hand, takes an active role in the establishment of global standards and which, on the other, promotes the early development of commercial applications. Follow some recommendations from previous workshops: Grid technology must be developed for an open, mobile and connected environment. New developments need to focus on industrial applications both to establish a long-term sustainability and to stimulate the rapid development of an infrastructure, which responds to real commercial requirements. There is an urgent need to bridge the gap between technology push and application pull in the development of this infrastructure. Science-based testbeds will not deliver industrial-strength middleware. Early industrial applications are needed to prove and stimulate middleware development. The early involvement of EU software houses in the development of these new applications should be a priority. The development and integration of existing technologies into prototypes and early applications is the most important short-term issue. The development of applications using results, for example from the Globus project, is important to demonstrate both the validity and applicability of standards and models. The development of new technologies to enable the refinement of data into information into knowledge is central to the functioning of any global infrastructure and is an important mediumterm objective. Developments to support this infrastructure should be based on Open Source/Open Standards following models such as those of Linux, MPI and the Object Management Group. Joint US-EU-Japanese global projects are important to establish a global infrastructure which presents real business opportunities. It is important to complement rather than compete with other initiatives in the US and Japan. Integrated Projects (IP) and Networks of Excellence (NoE) are the primary instruments to implement this new disruptive paradigm in FP6. However, existing (industry) players are generally not motivated to develop or deploy disruptive technologies. The aim is to create critical mass and incentives compelling enough to attract early adopters not just in the academic and research sectors but in industry and business. Co-operation with national programmes is highly encouraged. Timely Public funding and private investment could lead to significant payoff. 4

5 Excerpts from the Specific Programme The group confirmed that the scope and nature of the Complex Problem Solving activities is well described in the modified proposal for the Specific Programme of FP6. Complex problem solving in science, engineering, businesses and for society: The objective is to develop technologies for harnessing computing and storage resources which are distributed in geographically dispersed locations, and for making them accessible, in a seamless way, for complex problem solving in science, industry, business and society. Application fields include environment, energy, health, transport, industrial engineering, finance and new media. Research will focus on new computational models, including computing and information GRIDs, peer-to-peer technologies and the associated middleware to make use of large scale highly distributed computing and storage resources and to develop scalable, dependable and secure platforms. It will include novel collaborative tools and programming methods supporting interoperability of applications and new generations of simulation, visualisation and datamining tools. European researchers already enjoy one of the world s fastest and most extensive research networks, developed through the GEANT project. The support for the research networking infrastructure in FP6 and the extension of GEANT will be provided through the Specific Programme: Structuring the ERA. The objective is to upgrade the research infrastructure to beyond 100 Gbit/s, enabling researchers across Europe to share knowledge and collaborate on solving complex scientific problems. Such networks will be a major cornerstone towards the realisation of the ERA. Research into the GRID technologies in the IST priority and the upgrade of the research infrastructure complement each other. In addition to applications in science, they will enable complex problems solving for societal (e.g. environment, health, ), engineering and business needs. These will pave the way for the full rollout of the next generation Internet. Close articulation is therefore needed between the work on the research infrastructure in the relevant specific programme and the IST priority. 5

6 Vision The continuing progress in computing and communication technologies (hardware and software) and the use of advanced applications allow - like never before - tackling complex problems across all domains of science and research and many dimensions of life (social, economical, political and personal) and performing simulations which begin to match the real world. Advanced computing and networking is becoming scalable and affordable and is no longer restricted to the realm of research or big science. Advanced (local and wide area, fixed and wireless) networks constitute the pervasive and persistent infrastructure to connect to information and instruments. Through progress in Web server and browser technology access to content has become easy and intuitive. Yet, the current computer is still based on the von Neumann stored programme architecture, networks still follow Shannon s store and forward concept. The above-described customary advances are merely achieved through incremental technical improvements in the current paradigm. The Grid and P2P computing model coupled with the concept of ambient intelligence is a step towards a new paradigm. As computers are becoming intelligent devices and ubiquitous, users will control their functioning and not vice-versa and be able to choose the resources they need to connect to. User-to-user and user-to-community networking will transform bit/data processing into information processing/sharing and ultimately knowledge processing. Users will be able to work across traditional boundaries and utilise the most up to date tools, to combine data and models from many resources, simulate complex interrelations. Access to information and knowledge will be via the Grid. The move is away from the server-centric web model to a many-to-many model with direct interaction between/amongst individuals. The Grid and P2P should not be limited to the deployment of distributed computing on a known set of computers, of static topology and a stand-alone network of computer nodes. Furthermore it must take into account the dynamics of the computations. Nodes may join and leave the grid at any time. The computational threads are allowed to travel. The nodes themselves may be mobile (as in the case of fleets of vehicles). It must be relying on existing infrastructures, such as the Internet. Grid and P2P technology raise a key challenge in terms of security. It requires a shift from the present algebraic approach (where A and B share a secret) to a new geometric approach where a group of subjects with dynamic links share parts of a secret. Another problem is trust (how to establish and how to maintain it) and getting companies to change their business processes to allow each to see the other s information. The challenge is to make them understand that by sharing they will gain. This new computing paradigm, grounded on open standards and sound architectural principles, has manifold implications and the potential to pave the way to an innovative replacement of the current omnipresent monopoly operating system. Monopolies are blocking innovation. Seminal contributions from virtually all major computer science disciplines, developed in collaboration with scientists and engineers, will be necessary. The final resting place of Grid and P2P as the next generation Internet, the architecture, should be in the operating system and not a separate add-on utility. 6

7 Complex Problems Challenges and Motivation As bandwidth capacity becomes ubiquitous and persistent, resources for computing and storage become available at geographically dispersed locations, creating a novel systems and application space available for complex problem solving in science, industry, business and society. There is a wide variety of such compute-intensive and/or data-centric problems and ways of solving them. The requirements these problems have on the underlying computational, communication and storage architecture can be quite different. The different types of players at all relevant levels of the value chain that are interested in those problems or can contribute to their solution can be considered as the potential drivers for Grid/P2P-like technologies that may become the basis for future ICT infrastructures. The Initial Drivers Two sets of applications have given a prominent visibility to respectively Grid and Peer-to-Peer applications. For the grid, this role was held by major scientific efforts, from high-energy physics, space and astrophysics, biotechnology, or climate change, that have reached a great visibility, and mobilise large resources. Peer-to-peer applications were made visible by grassroots efforts from the free / open source software and open contents movements addressing data sharing, co-operative computation, co-operative publishing, or distributed software development. During FP6, the prominent role of these driving Grid and Peer-to-Peer applications will remain very important. Commercial and Societal Applications The FP6 proposal texts situates the development of the basic technology for complex problem solving in the perspective of challenges in science, industry, business and society and of application fields such as environment, energy, health, transport, industrial engineering, finance and new media. One can list applications in these domains that can act as drivers for technology development and deployment: Distributed and Collaborative Product Development and Production: for instance, integrated development and manufacturing environments; collaborative development for just-in-time production, service and maintenance operations; multidisciplinary design and optimisation. Application Services Provision: scalable networking and computing infrastructure offered as a service for complex problem solving Virtual Enterprises (scalable dynamic virtual organisation): interoperability for inter-enterprise collaboration. Educational applications: large-scale peer-to-peer educational resources and educational Napsters. Co-operative publication or content creation networks: Open publishing and distributed content. Global information access: distributed searching for sound, image, 3D or other type of rich media. Health applications: the interactive simulations for clinical operations planning and real-time support, and the control of micro robotic operation facilities. Critical infrastructures security for information and communications, water and energy supply including electricity transportation, banking and finance and government services Global Monitoring for Environment and Security (GMES): making use of highly distributed large data sets and complex models and moving from data and map based systems towards fully operational systems and information. Intelligent Transport Systems: multi-source data fusion for prediction and decision support in multimodal traffic systems and integration with air pollution modelling. 7

8 Classification of Complex Problems If we now try to identify various classes of complex problems that arise in these domains, in view of identifying the potential underlying technical challenges, one can list: Very large data flow driven problems Large scale algorithmically complex problems Large scale heterogeneous (multi-physics) models Large-scale data with complex distributed analysis or interpretation Synchronous distributed co-operation around one model Managing co-operative production of entities by a large number of independent players Searching in a very large number of distributed sources A more detailed analysis of driving applications and taxonomy of complex problems can be found in Annex II of this report. 8

9 Grid and Peer-to-Peer Grid and Peer-to-Peer (P2P) computing can in the widest sense be considered a visible implementation of the concepts of ambient intelligence and pervasive computing. It empowers the individual user and fosters the creation of flexible dynamic virtual communities, initially in the academic and research sector but with pending uptake possibilities in industrial and business sectors. Public seed funding will encourage the stakeholders to engage and invest in Grid computing (technologies and applications) and revolutionise in the long term the way individuals and communities use the computer, computing as a utility. The Grid will really come into its own only when people learn to build virtual organisations. Virtual organisations are noteworthy because of their constantly shifting landscape of data and computer resources, and the authentication on which they rely. The aim is to go for the best, most exciting technology based on open standards and ensuring it truly serves human needs and create a robust Grid and P2P infrastructure at reasonable cost with high degree of functionality: evolvable, scalable, pervasive, accountable and competitive. In the business world this means taking ecommerce to next level by giving the customer a resilient, flexible, virtual IT infrastructure readily available on demand from any location. The emerging commercial support is going to accelerate the process of moving Grid and P2P technologies out of the labs and universities into the commercial mainstream. Grid, P2P have also something in common with web services. All lay claim to defining the next generation Internet. Web Services are enterprise applications that exchange data, share tasks, and automate processes over the Internet, solve the problem how to collaborate across multiple platforms and systems. Web services are driven by big companies and are mainly addressing businesses and enterprises. Priority is on interoperability: from their perspective what they are trying to accomplish is to get their business processes interconnected. Web services build on well-defined standards for communication between client and server, standards such as XML, SOAP, WDSL and UDDI. Grid, P2P and web services will become increasingly symbiotic, see the report on Open Grid Services Architecture (OGSA). Finally, perhaps the biggest question about P2P computing concerns the financial viability of the emerging business models. Where do we stand today? The EC and individual EU countries have already funded several substantial grid-related projects in the last two years, some for infrastructure and middleware development and some for applications that take advantage of the Grid concept. They are all addressing different research topics and fields spanning from High Energy Physics to bioinformatics and medical applications, yet using a de facto US developed open middleware standard (with one notable exception). See Annex I for details. Whilst the expectation is high, the number of organisation exploiting the technology is comparatively low. Partly this slow take up can be explained by the fact that the technologies (GRID, P2P, Webservices) are still a very young area and there is a lot of work being carried out developing standards and creating practical, usable and user-friendly tools. However, the biggest inhibitor to the uptake of these technologies is identified as a simple lack of security. Security is a major area of concern to all business today. Timely Public funding and private investment could lead to significant payoff. What research is required in FP6? The needs and problems of virtual organisations (in research, business and industry) and communities should define the Grid roadmap and research agenda. This is where the future Grid would have its greatest value. It is such problems that could lead to new business models just as the Internet created the conditions for e-commerce. Present EU funded grid projects are using in one way or the other existing (US developed) middleware toolkits and are strongly big-science oriented with rather shortterm goals. To fully exploit the capabilities of the Grid in the sense of utility computing one should aim at creating a truly open co-operative and competitive industrial-strength Grid-framework (collaboration with US and Japan to avoid diverging standards) considering the above mentioned research challenges. 9

10 Development of an open Grid architecture (OS, middleware) Development of interoperable security and directory architectures Development of Grid-aware applications in research, business, industry, health, transport, environment Key Challenges Since a Grid will consist of vastly heterogeneous sets of compute nodes, especially commodity clusters, storage resources, platform portability which enables programs to be downloaded from networks and to be executed is required. In addition, performance portability, which provides good performance irrespective of platforms, is also needed. To achieve this vision of the Grid as the next computing paradigm (render obsolete much of the computing world as it is today) a set of interrelated crucial challenges need to be tackled, at the network level, the computer level (OS, middleware, API), the application layer and the business layer. R&D for Grid is inspired by the confluence of several technological trends, application push and technology pull. Challenges at the network layer: Network capacity; bandwidth provision (QoS and flexible, scalable);latency issues; End-to-end issues (NATs and firewalls); Security and reliability issues; untrusted resources; Mobility aspects; Challenges at the OS and middleware layer: Start from open software OS kernel; Resource discovery, scheduling and management; Accounting (virtual brokerage of computer power) and billing issues; cross-organisational policy issues; Security issues (authentication, authorisation, integrity ); Directory and naming issues (LDAP, UDDI); APIs and open protocols; interoperability; Use of agent technologies; Standards; Challenges at the application layer: Develop grid-aware applications; Integrating Grids with existing applications; databases poorly matched to grids; Application fusion for improved performance; application portability; Simple visual user interface (hiding the underlying complexity); Standards; DRM; Challenges at the business layer: Grid business models; Integration of Grid/P2P with Web Services (Internet accessible software components) and databases; Regulatory, legal and social issues; Legacy issues; Integrating P2P in the supply chain and workflow control; 10

11 Building the Constituencies Research labs and universities already are well known actors in the field. Awareness rising is needed in business, industry, health, environment, transport, etc. 11

12 Distributed and Shared Infrastructure for e-learning Background Technology supported learning is a very broad area covering a number of different realities, depending on the target audience (schools, universities, vocational training, life-long learners), the pedagogical model and the socio-economical context. Following are some of the key issues where ICT is expected to play an essential role: Distance learning; Collaborative learning (e.g. problem solving by teams of learners); Use of virtual or mixed reality for learning in complex training environments; Sharing of educational material, using metadata and raising complex IPR issues; Personalised learning (learning path determined by the needs and background of the learner); On-the job learning (integration of training and work); Learner assessment; Motivation for GRID in e-learning Areas of e-learning suitable for GRID-based applications Some areas of e-learning can only really take off if a GRID type infrastructure is available. Advanced distance learning, implementing the concept of virtual classroom, where teachers and learners are geographically dispersed and where the use of interactive multimedia objects is required. Collaborative learning, which requires advanced communication facilities (e.g. video-conferencing) and application sharing facilities. The implementation of the concept of e-campus, where universities not only share course material but also research and computing facilities is only possible with a GRID infrastructure. The use of virtual and mixed reality simulations in a number of complex training areas (e.g. training of surgeons, training of maintenance staff of complex equipment, training of mechatronics engineers). Contribution of e-learning applications to GRID technology development The GRID technology is not only an enabler of e-learning applications, it could also benefit from the special requirements imposed by these applications on the GRID: Advanced multicasting facilities; High-class visualisation environments; Reliability and availability of the infrastructure; e-learning specific middleware (e.g. for content representation, specific IPR issues). 12

13 Research Network Infrastructures The Framework Programme text highlights that Research infrastructures address (amongst other issues): The support of existing research infrastructures to create a denser network between related initiatives, in particular by establishing a high-capacity and high-speed communications network for all researchers in Europe (GEANT) and specific high performance Grids and test-beds (GRIDs). Background on Research Infrastructures The two primary aims of GEANT are the provision of a high-speed state-of-the-art research network backbone infrastructure for European researchers and the extension of this network to the East- European candidate countries (associated to the Framework Programme). Both of these objectives have been fully achieved, as 10 Gb/s connectivity in the core of the network is fully operational since 1st November 2001 and more than 30 European countries are connected to the infrastructure. These achievements of GEANT have a significant impact on overall research in Europe. First and foremost as the GEANT network is the foundation of ERA. Second, because Europe starts attracting connectivity from other parts of the world and has established itself as a global player. Both are fundamental in keeping European researchers at the leading edge. The GEANT network is now recognised as an important global player. This is underlined by the attempts from the various sides (e.g. Mediterranean, Asia Pacific, Latin-America) to establish a direct interconnection between these areas and Europe. The new paradigm of Grids is to become the next revolution in networking: spectacular advances in both commodity computing power and network bandwidth have encouraged the belief amongst the scientific and the business communities around the globe that it will be possible to embrace new methods of global collaborative research and enterprise. Major projects have been launched into this area. This strong European interest has already been recognised by partners around the globe and collaboration has been established with similar initiatives in the US to ensure a coherent development of this new technology. The current Internet Protocol IPv4 (version 4) is reaching its limits mainly in terms of address space and network management. IPv4 can provide in theory 4 billion addresses that have to be compared to the current estimated world population of 6 billions people. The new version of the Internet Protocol, called IPv6, has been designed having the experience of IPv4 and provides solutions to the current IPv4 limitations as well as new features like plug and play auto-configuration. Considering especially the area of mobile communications (2.5 G and 3G systems), where Europe has a leading position, the limitations of IPv4 make the development of new applications either not possible or at least very difficult. IPv6 has furthermore the potential to bring amongst other things end to end security, end to end Quality of Service that are key enabler factors also for the implementation of the GRID concept. The 2 large pan-european infrastructure test-beds on IPv6 that started in January 2002, are seen as precursors for the Integrated Projects in FP6 as they constitute microcosms on their own (budgets beyond 15 M and more than 20 European participants each) and since research activities related to IPv6 are still needed (e.g. improvement of the Internet routing algorithms, large scale experimentation about security or Quality of Service). These test-beds will prepare the ground and provide the necessary know-how for the deployment of IPv6 in Europe. Work on Research Infrastructures in FP6 (GEANT and GRIDs) 2 The following research activities need to be carried out in order to foster and enhance Europe s position in the integrated area of GEANT and specific high performance Grids and test-beds: Exploit the opportunities of a liberalised telecommunication environment ( own fibre, own the network, etc.). Thus moving into a completely new and yet unexplored area of research 2 The Council of Minister proposed on its meeting on 10th December 2001 to allocate 300 M for the further development of GEANT and GRIDs. IRG Complex Problem Solving including GRID and RNI 13 Draft V /05/02

14 networking in Europe. Examples from Canada and US show that through this a paradigm shift might occur in the way networking services can be provided. Support and integrate research projects on top of the network infrastructure. The current growth in demand for bandwidth clearly shows the need to address the range of terabit/s communication capacities. However, it still needs to be demonstrated that these speeds scan be effectively managed and deployed in a production-class network. Enhance the inclusiveness of the research network infrastructure by taking into account the demands of various actors in the field (e.g. schools, educational networks, libraries, e-learning, etc.). This activity will widen the scope of the European research network backbone substantially and will have a profound impact on its structure. Strengthen Europe s position as a global player in the filed of networking by maintaining and extending the current international connectivity to the National Research and Education Networks (NRENs) in regions outside Europe (e.g. Mediterranean, Far-East and Pacific-Rim, Africa, South- America). Thus using this research infrastructure as a political instrument to support the development and foster cohesion in these countries. The future European research backbone network will need to go beyond the simple provision of bit-pipes. This infrastructure will have to provide advanced middleware services and GRID like services for the research community to facilitate their ever-growing needs. Build advanced test-beds to test, validate integrate and demonstrate new technologies and services (like Grids, GMPLS, new routing schemes, access technologies, photonics) in real-world settings. In particular in relation to Grids: Carry out advanced Grid experiments (on production level) to pave the way for the creation of a new class of infrastructure to enable next generation research and business over the internet based on global collaborations and on the sharing of information resources. As important step here is seen the creation of distributed tera-scale facilities (in terms of computing, storage, communication power) across Europe for the generic use across different research disciplines. Use advanced communication facilities (e.g. GEANT, broad scale utilisation of IPv6) to interconnect those facilities with an aim to lead to the creation of a virtual GRID-based research infrastructure across all Europe. Foster interoperability of solutions across many different disciplines in an effort to achieve broad scale uptake of Grid technology across numerous application areas (and user communities) and economies of scale; in the same context contribute to the creation of new standards; continue the strong contribution to open-source. Put particular emphasis on access facilities to the Grid infrastructure (e.g. wireless access); if Grids is to become a utility, a broad range of access devices need to be able to be connected to Grid infrastructures and user friendly interfaces are necessary to be in place. Interconnect major Grid-based testbeds/experiments in Europe with corresponding Grid testbeds/experiments in the world, e.g. US, Asia-Pacific region and others. Enable the smooth evolution of the current ICT-environments to pervasive computing ones based on flexible and co-ordinated schemes for the control and sharing of resources. 14

15 Global Monitoring for Environment and Security Background The initiative on Global Monitoring for the Environment and Security (GMES) is aimed at the provision of independent operational information for monitoring and management of the environment and security, in fulfilment of European Policies. GMES aims to support Europe s goal regarding sustainable development, global governance, by facilitating and fostering over the next decade the provision of enhanced quality data, information and knowledge. It will do so by paying particular attention to a better use of information technologies. The aim is to achieve a significant leap forward in the quality of information and services delivered. GMES s goal is to provide information in a transparent and user-friendly manner, allowing access to high quality services. The concept of an open information architecture will provide means for a more democratic participation in policy-making. A key emphasis is put on the integration of data and information gathered from space and from terrestrial sensor networks. Data acquisition from Earth observation will provide a critical component in building the type of information services which are needed. A transition from experimental, research type systems towards fully operational systems and services is needed. GMES is about exploiting Europe s industrial and technological competencies in information and communication technologies to respond to new challenges concerning sustainable development, crisis management, peace keeping, operations and humanitarian and development aid. GMES priority areas European Regional Monitoring A. Land cover change in Europe B. Environmental Stress in Europe Global Monitoring C. Global vegetation monitoring D. Global Ocean monitoring E. Global Atmosphere monitoring Security related Aspects F. Support to regional development aid G. Systems for risk management H. System for crisis management and humanitarian aid Horizontal Support Action I. Information management tools and contribution to the development of a European spatial data Infostructure Motivation and suitability for GRID related RTD on environmental applications (GMES) Most of the thematic priority areas in GMES require a combination of information and computing GRIDs, which is due to the complexity of the problems to be addressed, the large size of distributed data sets and the huge variety of scientific complex models to be deployed. Example 1: Global Climate change prediction requires the use and integration of different atmospheric, chemical, meteorological models which are to be fed by earth observation data from satellites, data from marine boys, sea temperature, etc. including terra-bits of historical data from large distributed data sets. Example 2: Risk modelling and management for flood prediction or forest fire management: Such information systems and services require the integration of mete-data, earth observation data with digital terrain data, flood or fire propagation models and appropriate decision support tools. 15

16 Annex I GRID Activities Summary publicly funded R&D Grid Projects Country EU funded Description EuroGrid: Application Testbed for European GRID computing. The EUROGRID project will demonstrate the use of GRIDs in selected scientific and industrial communities, address the specific requirements of these communities, and highlight the benefits of using GRIDs. DataGrid is a project funded by European Union. The objective is to enable next generation scientific exploration which requires intensive computation and analysis of shared large-scale databases, from hundreds of TeraBytes to PetaBytes, across widely distributed scientific communities. Damien: Distributed Applications and Middleware for Industrial use of European Networks. The project is coming from the are of High Performance Computing and deals with the problem of how to develop applications for computational grids. In High Performance Computing the application developer is used to some standards, like the Message Passing Standard MPI, and several tools, which ease the development and analysis of his application. The central objective of the project is therefore to provide these well-known and highly accepted tools to Computational Grids, too. To reach this goal, the tools have to be extended to the properties of Grids. Additionally, these tools will be tested in industrial environments with application used in every day production. DataTAG project will create a large-scale intercontinental Grid testbed that will focus upon advanced networking issues and interoperability between these intercontinental Grid domains, hence extending the capabilities of each and enhancing the worldwide programme of Grid development. The project will address the issues which arise in the sector of high performance inter-grid networking, including sustained and reliable high performance data replication, endto-end advanced network services, and novel monitoring techniques. The project will also directly address the issues which arise in the sector of interoperability between the Grid middleware layers such as information and security services. GRIA project will develop, apply and evaluate a Grid testbed, based on an existing open-source infrastructure but incorporating services for: end-to-end quality of service to provide reliable and manageable performance; support for secure, end-to-end business models and processes, enabling the Grid to be used for outsourcing computational services. The testbed will employ open standard interfaces to access Grid resources, and a toolkit will be included to make it easy to develop services and applications for the Grid. The results will be evaluated for two industrial applications (in structural analysis and in digital film post-production), and disseminated to promote further standardisation and take-up by European industry. IST Grip will develop interoperability software for EUROGRID and Globus and demonstrate the feasibility to exploit the specific strength of each implementation, the seamless User interface of EUROGRID and the protocol focus of Globus. Biomolecular and meteorological applications will be used as inter-grid examples in GRIP. Project resources will be dedicated to standards work in the Global Grid Forum. GridLab: A Grid Application Toolkit and Testbed. The GridLab project will develop a easy-touse, flexible, generic and modular Grid Application Toolkit (GAT), enabling todays applications to make innovative use of global computing resources. The project is grounded by two principles, (i) the co-development of infrastructure with real applications and user communities, leading to working scenarios, and (ii) dynamic use of grids, with self-aware simulations adapting to their changing environment. 16

17 EU funded UK escience ESGO will create a virtual archive by federating solar data centres scattered across Europe into a data Grid. This will dramatically enhance access to data for both the solar and non-solar communities and will for the first time make data available on demand to the user. The project will employ intuitive web-like interfaces and advanced search and visualisation tools to match observations made by numerous space- and ground-based observatories. A major innovation will be the provision of a Solar Feature Catalogue that will allow complex searches based on solar phenomena and events. The EGSO will also provide the facility for the evaluation of extended time series of large, complex data sets at source, without the need to download them. This Grid test bed will federate six data centres located in France, Italy and the United Kingdom, but the design of the EGSO will scale to include as many data archives world-wide as wish to participate. Cross Grid project will develop, implement and exploit new Grid components for interactive compute and data intensive applications like simulation and visualisation for surgical procedures, flooding crisis team decision support systems, distributed data analysis in high-energy physics, air pollution combined with weather forecasting. The elaborated methodology, generic application architecture, programming environment, and new Grid services will be validated and tested thoroughly on the CrossGrid testbed, with an emphasis on a user friendly environment. The work will be done in close collaboration with the Grid Forum and the DataGrid project to profit from their results and experience, and to obtain full interoperability. This will result in the further extension of the Grid across eleven European countries. GridStart: The GRID is widely seen as a step beyond the Internet, incorporating pervasive high bandwidth, high-speed computing, intelligent sensors and large-scale databases into a seamless pool of managed and brokered resources, available to industry, scientists and the man in the street. The potential benefits and social impact of the GRID are so great, that it is imperative to involve industry and the service-provision community at an early stage to ensure that the European economy and society can take full advantage of this revolution. The objective of the GRIDSTART Accompanying Measure is to maximise the impact of EU-funded Grid and related activities through the clustering of the currently funded projects and thereby enhance the potential of the new Grid technologies to benefit the people of the European Union. escience Initive: 120M 3 Year Programme to create the next generation IT infrastructure to support e- Science and Business 75M is for Grid Applications in all areas of science and engineering 10M for Supercomputer upgrade 35M for development of industrial strength Grid middleware Require 20M matching funds from industry Essential that UK plays a leading role in Global Grid development with the USA, EU and Asia Grand Challenge Projects: Equator: Technological innovation in physical and digital life AKT: Advanced Knowledge Technologies DIRC: Dependability of Computer-Based Systems. MIAS: From Medical Images and Signals to Clinical Information AstroGrid: links to EU AVO and US NVO projects escience projects Comb-e-Chem:Structure-Property Mapping Southampton, Bristol, Roche, Pfizer, IBM DAME: Distributed Aircraft Maintenance Environment York, Oxford, Sheffield, Leeds, Rolls Royce Reality Grid: A Tool for Investigating Condensed Matter and Materials QMW, Manchester, Edinburgh, IC, Loughborough, Oxford, Schlumberger My Grid: Personalised Extensible Environments for Data Intensive in silico Experiments in Biology: Manchester, EBI, Southampton, Nottingham, Newcastle, Sheffield, GSK, Astra- Zeneca, IBM, Sun GEODISE: Grid Enabled Optimisation and Design Search for Engineering: Southampton, Oxford, Manchester, BAE, Rolls Royce Discovery Net: High Throughput Sensing Applications, Imperial College, Infosense, Grid Support Centre. 17

18 NL DE FR GridPP will deliver the Grid software (middleware) and hardware infrastructure to enable testing of a prototype of the Grid for the Large Hadron Collider (LHC) project at CERN of significant scale. The GridPP project is designed to integrate with the existing Particle Physics programme within the UK, thus enabling early deployment and full testing of Grid technology and efficient use of limited resources. The project will disseminate GridPP deliverables in the multi-disciplinary e-science environment and will seek to build collaborations with emerging Grid activities both nationally and internationally. DutchGrid: a national Grid initiative in the Netherlands as a local testbed for the European Datagrid. VLAM-G: The Grid-based Virtual Laboratory AMsterdam (VLAM-G), provides a science portal for distributed analysis in applied scientific research. It offers scientists remote experiment control, data management facilities and access to distributed resources by providing crossinstitutional integration of information and resources in a familiar environment. The main goal is to provide a unique integration of existing standards and software packages. UNICORE (UNiform Interface to COmputing REsources) provides a science and engineering GRID combining resources of supercomputer centers and making them available through the Internet. Strong authentication is performed in a consistent and transparent manner, and the differences between platforms are hidden from the user thus creating a seamless HPC portal for accessing supercomputers, compiling and running applications, and transferring input/output data. Grid funding approved by French Government. JACO3 is a Grid environment that supports the execution of coupled numerical simulation. It is being developed jointly by EADS, INRIA, INTECS and KTH. It is a set of CORBA services that manage remote computing resources through the Internet.The PARIS research group at INRIA- Rennes is conducting research activities in Grid computing. A Computational Grid acts as a high-performance virtual computer to users to perform various applications such as for scientific computing or for data management. It is made of several computing resources interconnected together by various networks. Although Grid Computing is still an emerging academic research field, it is foresee that the industry will soon make a stronger demand on Grid environments. Indeed, today in the industry the trend is to replace prototyping as much as possible by simulation to reduce costs and time to market. IT Décrypthon est une nouvelle initiative de l'afm et d'ibm, qui permet de rassembler des milliers d'ordinateurs personnels pour participer activement à la recherche contre les maladies génétiques et les maladies rares. Un logiciel réalise des calculs complexes de comparaison de protéines, répartis sur des milliers d'ordinateurs personnels à travers le réseau Internet. Dans cet objectif, l AFM rassemblera, une nouvelle fois, le grand public autour d un projet scientifique exceptionnel pour le Téléthon 2001, les 7 et 8 décembre. There is increasing interest in Grid Computing in Italy. This is reflected by the growing number of projects in this area. Projects are underway in two major Italian research institutions: the Italian National Research Council (CNR) and the Italian Institute for Nuclear Physics (INFN). CNR is also involved in the DataGrid project. Grid for Remote Sensing: The Grid Resource Broker (GRB) is one of the current Globus projects of the HPC Lab of the University of Lecce, Italy). GRB is a grid portal that allows trusted users to create and handle computational grids on the fly exploiting a simple and friendly gui. We introduce GRB features and discuss its implementation status. INFN Grid will develop a Grid Infrastructure that will allow INFN users a transparent and effective use of computing and storage resources distributed in 26 INFN nodes of the Italian research network GARR-B. INFN is also involved in the DataGrid Project. 18

19 Nordic Countries Hungary Ireland Poland NorduGrid: the Nordic Testbed for Wide Area Computing project is a part of the Nordunet2 programme, aimed to develop networked applications with extensive usage of modern utilities and tools in the Nordic countries. The aim is to establish an inter-nordic test bed facility for implementation of wide area computing and data handling. The facility will provide the infrastructure for interdisciplinary feasibility studies of GRID-like computer structures. The project shall collect and document experience, as an input to the decision process on the future computer infrastructure strategy for sciences with distributed PByte storage requirements and processing power in the order of multi-teraflop. The Laboratory of Parallel and Distributed Systems (LPDS) of SZTAKI has a long and successful history in the research and development of distributed systems, tools and applications. A number of these activities and results are closely related to the current grid-oriented projects of the laboratory. The largest computer cluster in Hungary runs under the supervision of LPDS. SZTAKI participates in the EU DataGrid project. Grid-ireland: Initial seed funding has been provided by Enterprise Ireland to guarantee the establishment of a working grid between compute clusters at three partner sites in Ireland, that is, the Departments of Computer Science in Trinity College Dublin, University College Cork and NUI Galway. This project has one primary objective: to establish grid-ireland. This requires: Hardware: Compaq will donate a 4-way symmetric multiprocessing gateway machine per site Software: initially the Globus services Management: the Dept. of Computer Science at Trinity College will do this Interconnect: the Irish academic network services will be used. Poznan Supercomputing and Networking Center works on development of tools and methods for metacomputing. The tools include: MetaLearn - knowledge acquisition and rules generator system (knowledge about the state of the metacomputer, application behavior). MPI tools: MPIVIS, Visual MPI - integrated, knowledge based environment for developing MPI programs, PERVIS - knowledge based performance tunning and visualisation of metacomputing applications. The project is done within co-operation with APART working group. US KB Metascheduler - knowledge based scheduling of the jobs in a metacomputing (Globus) environment. All the PSNC metacomputing projects are being developed within the national metacomputer infrastructure, basing on POL-34 network Access Grid: Create and deploy persistent collaborative spaces on the Internet for group collaboration using commodity technologies. The advanced video-conferencing technology of the Access Grid, which was developed by the Argonne National Laboratory Futures Lab, enables wide-area collaborative computational science, large-scale distributed meetings and collaborative work sessions, and training. Each Access Grid node offers multimedia displays, presentations and interactions environments, and interfaces to Grid middleware and visualisation environments. Funding agencies DOE, NSF. ASCI DISCOM: Create operational Grid providing access to resources at three U.S. DOE weapons laboratories. Funding agency: DOE; Particle Physics Data Grid (PPDG): The Particle Physics Data Grid collaboration was formed in 1999 because its members were keenly aware of the need for Data Grid services to enable the worldwide distributed computing model of current and future high-energy and nuclear physics experiments. Initially funded from the NGI initiative and later from the DOE MICS and HENP programs, it has provided an opportunity for early development of the Data Grid architecture as well as evaluating some prototype Grid middleware. Funding agency: DOE. Earth Systems Grid: Funding agency: DOE. Turning Climate Model Datasets Into Community Resources. Delivery and analysis of large climate model datasets for the climate research community. 19

20 Science Grid: The DOE Science Grid's major objective is to provide the advanced distributed computing infrastructure based on Grid middleware and tools to enable the degree of scalability in scientific computing necessary for DOE to accomplish its missions in science. The vision for "Grids" is to revolutionise the use of computing in science by making the construction and use of large-scale systems of diverse resources as easy as using today's desktop environments. Funding agency: DOE. Fusion Collaboratory: Create a national computational collaboratory for fusion research. Funding agency: DOE. ß-Grid: A National Infrastructure for Computer Systems Research (ANL/NCSA). Information Power Grid (IPG): The IPG is NASA's high performance computational grid. Computational grids are are persistent networked environments that integrate geographically distributed supercomputers, large databases, and high-end instruments. These resources are managed by diverse organizations in widespread locations, and shared by researchers from many different institutions. Funding agency: NASA. National Technology Grid: Funding agency: NSF. Network for Earthquake Engineering Simulation (NEES): NEESgrid is an integrated network that will link earthquake engineering research sites across the country, provide data storage facilities and repositories, and offer remote access to the latest research tools. It was established in 2001 by a consortium of institutions led by the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign. Funding agency: NSF. Grid Application Development Software: Funding agency: NSF. Grid Physics Network (GriPhyN): Technology R&D for data analysis in physics experiments: ATLAS, CMS, LIGO, SDSS. Funding agencies: DOE and NSF. TeraGrid: is a multi-year effort to build and deploy the world's largest, fastest, most comprehensive, distributed infrastructure for open scientific research. When completed, the TeraGrid will include 13.6 teraflops of Linux Cluster computing power distributed at the four TeraGrid sites, facilities capable of managing and storing more than 450 terabytes of data, highresolution visualization environments, and toolkits for grid computing. These components will be tightly integrated and connected through a network that will initially operate at 40 gigabits per second and later be upgraded to gigabits/second 16 times faster than today's fastest research network. Funding agency: NSF. Distributed Tera-Scale Facility and Extensible Tera Scale Facility part of the NSF Cyberinfrastructure initiative. International Virtual Data Grid Laboratory (ivdgl): The ivdgl will provide a global computing resource for several leading international experiments in physics and astronomy, including the Laser Interferometer Gravitational-wave Observatory (LIGO), the ATLAS and CMS experiments at CERN, the Sloan Digital Sky Survey (SDSS), and the proposed National Virtual Observatory (NVO). Funding agency: NSF. The Brain Data Grid: National Scale Testbed for Federating Large Databases Using NIH High Field NMR Centers; Cyber Infrastructure Linking Tele-instrumentation, Data Intensive Computing, and Multi-scale Brain Databases. Grid Application Development Software (GrADS): The goal of the Project is to simplify distributed heterogeneous computing in the same way that the World Wide Web simplified information sharing over the Internet. The GrADS project will explore the scientific and technical problems that must be solved to make grid application development and performance tuning for real applications an everyday practice. 20