Computing Strategic Review. December 2015

Size: px
Start display at page:

Download "Computing Strategic Review. December 2015"

Transcription

1 Computing Strategic Review December 2015

2 Front cover bottom right image shows a view of dark matter at redshift zero in the Eagle simulation, by the Virgo Consortium using the DiRAC Facility (Durham Data Centric Cluster)

3 STFC Computing Strategic Review Contents Foreword... 1 Executive Summary... 2 Introduction... 4 Vision and Objectives... 7 STFC Programme Areas: Current Usage, Future Plans, Issues and Opportunities... 9 Particle Physics, Astronomy and Nuclear Physics Research... 9 The UK National Facilities Engineering Support Externally Funded Projects and Services Hartree Centre Cross-Programme Topics Recommendations Annex 1: The Strategic Review Development Process Annex 2: Glossary Annex 3: Summary of Responses to the Survey of current computing facilities and services Part 1 - Facilities, Scientific Computing Department and the Hartree Centre Part 2 Particle Physics, Astronomy and Nuclear Physics Research Programme... 50

4 Foreword The Science and Technology Facilities Council is one of Europe s largest multidisciplinary research organisations. Computing and data handling underpin everything that we do. The world-leading science we support through research at universities and our large-scale facilities depends on our computing facilities, services and internationally recognised expertise. Experiments need to transfer, analyse and store ever increasing amounts of data and this requires computing hardware, software, networks and skilled professional support. Computing also plays a central and growing role in our innovation activities. The Hartree Centre uses its software expertise and supercomputing power to deliver a competitive edge for UK business. In the future it is clear that the pace of technological change will continue and the demands placed on our computing facilities and services will dramatically increase. We need to ensure that we are in the best position possible to meet these challenges so that we can achieve our strategic goals of delivering world-class research, innovation and skills. This Strategic Review assesses our current computing capability and future requirements across our entire programme. It recommends actions to meet our future computing needs and improve efficiency. The Review provides the framework within which we will develop a strategic plan for computing. Following the publication of this Review, we will consult our communities to evaluate how we can best implement the recommendations within the funding available and decide our future priorities for computing. In the years ahead this should ensure we can provide the optimum computing support to our science and innovation communities. Professor John Womersley Chief Executive Officer 1

5 Executive Summary Computing is vital to deliver STFC s strategic goals of delivering world-class research, innovation and skills. The STFC Computing Strategic Review is an internal STFC review that aims to identify the current and future e-infrastructure requirements of STFC s scientific and industrial user communities and recommend how STFC can best support the provision of those needs. The review panel consulted with the users of STFC computing and other stakeholders to develop its recommendations. The Review covers hardware, data services, software, cyber security, networks and computing professionals. The vision for STFC Computing is to increase the scientific, technological or economic output of users of STFC computing by exploiting technology, developing skills and optimising the use of resources. The users of STFC computing comprise STFC-funded researchers; users of STFC National Facilities; other science communities supported by computing facilities or services hosted by STFC; and clients of the Hartree Centre. All STFC programme areas anticipate an order of magnitude increase over the next five years in data volumes, with implications for requirements for computing hardware, storage and network bandwidth. Meeting this challenge is essential to ensure the UK can continue to produce worldleading science, but in the current financial climate it is clear there will be funding limitations. Long-term planning will be critical, as will more efficient use of resources. The scientific disciplines supported by STFC should work more closely together to find ways of sharing computing infrastructure, with new projects encouraged to make use of existing expertise and infrastructure. STFC should play an active role in co-ordinating this, and maintain good communications with partner organisations such as the UK Space Agency on future project compute requirements. Consolidation is possible, while recognising that users require facilities ranging from desktops to the largest Tier 0 machines, for diverse specialities and at various locations. STFC needs to facilitate access to its compute resources, including professional expertise, by clearly communicating the resources available and providing appropriate user guidance and support. Interoperability and common authentication, software, metadata and file formats etc. will increase the effectiveness of the resources available. The growing diversity among emerging computing architectures will also affect all areas. STFC and its expert user communities should make full use of their computing capability to provide leadership in making the best use of developments in compute technology. STFC should also ensure that it maintains the skills and capabilities to support the large-scale scientific computing facilities that are necessary for a broad UK science base. Data scientists and software engineers are becoming vital to all research areas and need to be supported by ensuring there is a clear career path for these professionals both within STFC and at the universities. 2

6 Computing provision for National Facility users should be reviewed as an urgent priority, including high performance computing, software, expertise, data storage, data transfer to facility user sites, and access to archived data at the facilities (providing simple remote access for non-expert users). The Hartree Centre is set to play a crucial role in boosting UK business, with a government partnership with IBM providing a 315M investment in its data centric and cognitive computing capabilities. As plans for this exciting development progress over the coming months, the Hartree Centre s position in STFC s governance structure and corporate strategy, its role and remit, needs to be communicated clearly to all stakeholders. 3

7 Introduction Scope of the Strategic Review Computing is a cross-cutting activity that supports all STFC s strategic goals of delivering world-class research, innovation and skills. It is vital to all research supported by the Science and Technology Facilities Council (STFC). This includes the particle physics, nuclear physics and astronomy research programmes (both in the universities, our laboratories and at the international facilities); the experiments carried out by users of the STFC National Facilities; and the scientific projects funded by other Research Councils who rely on computing facilities and services hosted and operated by STFC. Computing also plays a key role in delivering STFC innovation, most notably at the Hartree Centre where government investment in high performance computing systems is intended to give UK business a key advantage over competitors. The scope of the Computing Strategic Review covers all of the above. STFC computing resources to support its own administrative needs are out of scope of this review. The Review addresses e-infrastructure in its widest sense: hardware, data services, software, cyber security, networks and computing professionals. Aims and purpose of the Strategic Review The Strategic Review aims to: identify current and future e-infrastructure requirements of STFC s scientific and industrial user communities recommend how STFC can best support the provision of these needs identify the relation to the wider UK and international setting consider how to maximise the impact of STFC e-infrastructure both for science and for industrial engagement. The Strategic Review will be used: to inform STFC s decision-making on e-infrastructure to inform STFC science communities, other Research Councils and stakeholders of STFC s priorities for e-infrastructure as a resource for preparing for spending reviews and responding to funding opportunities to inform the Scientific Computing Department s (SCD) strategy for the development of new compute infrastructure and service. The STFC Computing Strategic Review in context Computing is a key enabler of STFC science. The highest priority current and future scientific projects identified in the 2013 STFC Programmatic Review, such as the Large Hadron Collider (LHC) experiments, the European Extremely Large Telescope and gravitational wave experiments, will depend on computing to collect, process and store data. Access to state-of-the-art computing facilities will be vital to ensure UK science remains internationally competitive. 4

8 In recent years e-infrastructure has gained prominence on the government s science and innovation agenda: Big Data is one of the Eight Great Technologies; the E-Infrastructure Leadership Council was created in 2012; and the UK Data Capability Strategy was published in Big Data was identified as a key priority in the 2014 Government Response to Consultation on Proposals for Long-term Capital Investment in Science and Research. Recent years have seen substantial government capital investments in e-infrastructure projects, including most recently 115m in 2014 to expand cognitive computing capability at the Hartree Centre. This investment is leveraging access to a further 200m of intellectual property from IBM, giving a total value of 315m STFC computing is provided in concert with a range of organisational partners: funders such as government, other Research Councils, the UK Space Agency; commercial partners and collaborators; infrastructure support e.g. Jisc for the Janet network, universities; collaborators in research projects etc. The international nature of many projects means the computing also has an international dimension: project collaborations can span many countries and depend on international bodies such as the European particle physics laboratory (CERN) and the European Southern Observatory (ESO). For example, the UK particle physics computing grid, GridPP, is part of the Worldwide LHC Computing Grid, a global collaboration of 41 countries. STFC scientists use international facilities such as the Partnership for Advanced Computing in Europe (PRACE) supercomputing facility. Technology continues to evolve at a rapid pace, with the development of new computer architectures and models such as massively multi-core technologies and cloud computing. Future challenges include increased power costs as system performance increases, driving the need for more energy-efficient computing. As mathematical and scientific models grow in complexity they require much more complex codes. The massive increase anticipated in data volumes presents a significant challenge: the Square Kilometre Array (SKA) radio telescope predicts it will need to process 10,000 petabytes per day by However, developing the novel technology to meet such challenges also provides innovation opportunities: solutions may have applications for other research projects and there is potential for engagement with industry. The exploitation of large diverse data sets can also create knowledge and wealth, with the Hartree Centre s expertise and capability in big data analytics providing a competitive edge to a variety of industrial clients. The Review will address these opportunities, risks and challenges. STFC has a uniquely diverse remit, supporting research programmes in particle physics, nuclear physics, astronomy and exploitation of the space science missions funded by UKSA; providing National Facilities (Diamond Light Source (Diamond), Central Laser Facility (CLF), ISIS (neutron and muon source)) to support a broad spectrum of scientific communities; and developing the UK Science and Innovation Campuses at Harwell and Daresbury. The provision of computing support across the STFC programme is therefore complex, with some computing infrastructure being provided directly by STFC, and other infrastructure being funded by STFC but delivered by the universities and/or international organisations of which we are members. The Scientific Computing Department not only provides essential computing support for the National Facilities but also utilises its technical expertise to support projects and services of other Research Councils. This Review attempts to provide a cross-stfc overview of future computing needs and opportunities to increase efficiency and effectiveness of our computing provision. 5

9 The Computing Strategic Review development process The Strategic Review was carried out by an internal review panel with representation from the Computing Advisory Panel. The review process included a survey of STFC s current computing facilities and services and a consultation with key stakeholders to identify future requirements and opportunities for increased efficiencies and effectiveness. Documents such as strategies and policies from relevant organisations were reviewed to ensure alignment with national and international priorities. Annex 1 describes the review development process in more detail. Following publication of this review, STFC will consult our communities to consider how the review s recommendations should be implemented. This will require evaluating the implications of implementation and deciding priorities where funding is required. 6

10 Vision and Objectives The vision and objectives for STFC computing over the next five years ( ) are set out below. The Strategic Review is divided into two main sections: one covering the principal programme areas covered by the Review and one which describes topics and issues which cut across multiple programme areas. These sections contain detailed recommendations for actions that will contribute to the delivery of the vision and objectives. Vision To increase the scientific, technological or economic output of users of STFC computing by exploiting technology, developing skills and optimising the use of resources. The users of STFC computing comprise: STFC-funded researchers Users of STFC Facilities (CLF, Diamond, ISIS) Other science communities supported by computing facilities or services hosted by STFC Clients of the Hartree Centre. STFC computing covers e-infrastructure in the broad sense of hardware, data services, software, cyber security, networks and computing professionals. Objectives 1) Support the STFC strategic goals of delivering world-class research, innovation and skills 2) Develop a balanced approach to supporting computing requirements that vary from small-scale to large-scale, generic to specialised, local to centralised a) Provide clear, accessible information on the resources available and guide users to the most appropriate resource b) Equip users with appropriate tools, services, support or training to make the best use of STFC resources 3) Develop a strategic and co-ordinated approach to planning for future computing requirements a) Ensure the development of affordable planning provision for computing infrastructure requirements b) Ensure that planning for the computing requirements of projects starts at the earliest possible stage of project planning c) Put in place mechanisms to identify opportunities where sharing resources would be of benefit (between projects, science disciplines, funding agencies etc.) d) Provide an overview so that the impact of proposed changes can be assessed across the organisation e) Ensure that STFC s role as a provider of resources to scientific communities outside STFCfunded programmes and Facilities is appropriate in terms of meeting the needs of the communities and making the best use of STFC resources 7

11 4) Develop and make the best use of the computing expertise within STFC or the communities it supports a) Put in place mechanisms to facilitate the sharing of expertise and best practice b) Put in place measures to support the development of a career path for software engineers/data scientists in a research environment c) Provide guidance on the adoption of cutting-edge and future technologies 5) Improve support for data management, analysis, transfer, storage and publications, specifically: a) Promote the appropriate use of open data b) Prepare for increasingly challenging data requirements by ensuring optimal use of resources (code optimisation; investment in software; fast, robust networks etc.) c) Assist users to make use of the most appropriate software 8

12 STFC Programme Areas: Current Usage, Future Plans, Issues and Opportunities Particle Physics, Astronomy and Nuclear Physics Research a) Current usage STFC provides e-infrastructure resources across the particle physics, astronomy and nuclear physics (PPAN) programme. This includes computing hardware, storage, networks, and the support of computing personnel. STFC provides computing resources over a range of scales to service the needs of the community. The largest STFC resources are the Tier-1 particle physics compute centre, funded through GridPP (the UK particle physics computing grid), and hosted by the Rutherford Appleton Laboratory (RAL), and the various DiRAC (Distributed Research utilising Advanced Computing) facility sites providing high performance computing. These provide computing resources of tens of thousands of cores, tens of petabytes of data and/or tens of terabytes of shared memory. The next level is comprised of smaller, Tier-2 sites at universities, such as those funded through GridPP, or astronomy grants, which are typically an order of magnitude smaller. The smallest scale, Tier-3 computing, comprises desktop computing which is critical for interpretation and visualisation of results obtained at the Tier-1 and 2 centres. Although STFC does not provide any Tier-0 resource, other than through the CERN subscription, STFC funded researchers have been successful in securing time on the European PRACE infrastructure, which offers machines with hundreds of thousands of cores. Finally, several projects, such as Galaxy Zoo and LHC@home, have used citizen science and volunteer donated computing to fulfil their computing requirements. PRACE, CERN Tier 0 Tier 0 (no STFC-supported Tier 0 resource except Hartree Centre (for industrial users) and CERN Tier 0 via the CERN subscription) GridPP Tier 1, DiRAC Tier 1 GridPP Tier 2 sites, Astronomy grant-funded HPC at universities Tier 2 Grant-funded desktop computing Tier 3 Figure 1: The different scales of computing resources. 9

13 These resources currently provide the appropriate computing for the PPAN community. In cases where there is a concern about lack of resources, it is often due to a lack of readily available information: descriptions of available resources, their current usage and procedures to gain access. While available HPC facilities are adequate, they are significantly smaller than what is available in other countries, notably the United States. Nonetheless, by using these resources effectively, the UK has generated world-leading scientific output in numerous areas, including experimental and theoretical particle physics as well as astrophysics simulations of stars and galaxies. The majority of active users rely on access to Tier 1, 2 and 3 computing with large scale runs performed on Tier 1 centres, development and test runs performed on Tier 2 sites and data visualisation and interpretation performed on Tier 3 machines. The continued availability of all levels of resource, at an internationally competitive level, is required to maintain the UK s scientific excellence. Network connectivity between sites in the UK is provided by Jisc. This encompasses the national network (Janet) augmented by a handful of dedicated links, such as those from CERN to the Tier 1. International connectivity is routed through Geant. No current gaps in provision have been identified, although all areas of PPAN science highlighted the expected growth in e-infrastructure requirements over the coming years. b) Currents plans; long term requirements Most areas of PPAN research expect their computing needs to grow over the coming years as data sets get larger and more complex. The general view is that at least one of: processing requirements, size of data sets or network bandwidth will grow by an order of magnitude for existing projects. In addition, there are several new instruments in astronomy that will drive this rapid expansion, most notably the SKA, the Large Synoptic Survey Telescope (LSST) and the Cherenkov Telescope Array. These will begin to challenge the existing networks as well as the data and compute requirements. There are a small number of users who require Tier-0 scale computing. At present, this is not readily available to STFC researchers other than through applications to PRACE. The UK does have a Tier-0 scale machine, ARCHER, which is managed by the Engineering and Physical Sciences Research Council (EPSRC) and improved access for STFC researchers would help this group. There are plans in place to expand the Tier-1 level computing with proposed expansions to GridPP (GridPP-5) and DiRAC (DiRAC 3) which, if they are approved and fully funded, would go a long way towards meeting the increased computing requirements over the next three or four years. In the absence of government investment, DiRAC 3 may not be feasible. A number of the new projects will require computing that is close in scale to the existing Tier-1 centres. In many cases, this could be provided more effectively by an expansion of the existing centres rather than establishing new project specific centres. There is a continued need for the Tier-2 level machines where the smaller scale analyses are run. These typically provide a faster turnaround and more flexible service than can be achieved at the national facilities. The universities that host the Tier-2 machines often provide a significant amount of additional support and future computing models need to recognise the advantages of this additional leverage. Finally, there will continue to be the need for desktop scale computing. Even 10

14 within the High Throughput Computing (HTC) and High Performance Computing (HPC) communities, the final stage of data visualisation is typically performed on a local machine. Support for visualisation, as well as fast transfer of data products, continues to be a priority. The expected ten-fold increase in data will require an increase in network capacity. This will likely require a number of new, dedicated links (such as for SKA and LSST and also to/from the Tier-1 centres) as well as an overall improvement of the network. There are not any significant feasibility concerns regarding this expansion, as the evolution of Janet is expected to keep pace with these requirements. The majority of the science within PPAN is done in an international context and the computing resources that are provided must reflect this. International collaborations typically have developed a complex set of software and middleware that will need to be deployed on the UK provided resource. In addition, they will have requirements for authentication and authorisation for collaboration members. c) Issues and opportunities The continued growth of e-infrastructure requirements across the whole of PPAN will require a coherent approach to the provision of computing hardware, software and personnel. Scientific computing is a long-term project, whether it is in support of an experiment or in producing the next generation of simulations. Thus, stability in funding over the longer time-scale must be available to allow for long-term planning of the computing requirements of a project. Consequently, it is important that computing requirements are considered at the earliest stages of project planning, even if detailed, accurate costings are not available until the planning is at a more advanced stage. The existing computing tiers remain appropriate for provisioning of resources, and support for all levels of computing remains important. Issues with availability of computing are more often due to a lack of knowledge of existing systems rather than availability of hardware. STFC could assist by better publicising the availability of resources. Additionally, STFC should provide clear guidelines detailing the scale of computing that is appropriate for each tier, as well as better documentation on procedures to access resources. Greater transparency on funding bids and the process of time allocation on Tier-1 machines would also be beneficial. One of the major challenges facing HPC and HTC users is a change in available computing hardware. The speed of individual cores has reached a plateau and instead the number of cores per machine is rapidly increasing. Furthermore, we are witnessing the rise of specialist hardware such as Graphics Processor Units (GPUs) and Intel MiCs. To make optimal use of this hardware requires significant effort in porting the code to a new platform and undertaking detailed optimisation work. In many cases, this is not an area where the project scientists have expertise. There is an opportunity to coordinate this effort at the national scale. There is a continued pressure towards open data and this leads to significant requirements on data curation and preservation. Currently, data curation and preservation is carried out to some extent (for some data sets) at STFC sites, but it is not something that is handled by the DiRAC centres. A unified strategy to data preservation will be critical in the coming years. A key part of this will be the 11

15 generation and publication of the appropriate metadata that will allow researchers and the public to find and access the appropriate data sets. The provision of computing hardware, storage and networks is only useful if accompanied by expert computational support that allows researchers to make use of this e-infrastructure. In many cases, there are excellent software engineers working to ensure this is the case. However, the career path for software engineers and data scientists is by no means clear, and the long-term prospects for computing support staff need to be improved. Additionally, it is critical to situate these people appropriately. In some cases it is preferable to site computing staff close to researchers to allow for regular interaction. In other instances, a centralised hub is more appropriate with the computing specialists managing the centralised, specialised functions, and talking with the computing specialists embedded in the domain specific satellite. These two models work very effectively together, and both need supporting. Meeting the future computing needs of existing and new research projects will present major technological challenges. But overcoming these challenges will offer possibilities for industrial engagement and innovation opportunities. Some of the novel technology developed will almost certainly have applications to other research problems. STFC should facilitate partnerships between academia and industry that will enable both sides to benefit from each other s expertise. There will be business opportunities for UK companies to provide hardware, software and skills. d) Recommendations The PPAN research planned in the next five years requires an order of magnitude increase in computing hardware, storage and network bandwidth. STFC should provide a plan to ensure this is made available. This can be achieved by maintaining and strengthening the existing hierarchy of computing (from Tier-1 to Tier-3), whilst ensuring access to a diverse set of compute resources (high memory, data intensive, high throughput, massively parallel). STFC should ensure that access to Tier-0 scale computing remains available to those researchers who require it. The available resources should be better publicised and guidance provided to ensure that projects are making use of the most appropriate resources. When required, it is important to provide assistance to people moving up (or down) the tiers. Furthermore, STFC should take a more active role in co-ordinating use of e-infrastructure and investigating the opportunity to share resources, without sacrificing the variety and specialist resources required by various communities. This is particularly relevant for new projects, which should be strongly encouraged to make use of existing expertise and infrastructure such as data centres, although it is important to ensure this is a joint venture and projects are not pushed onto inappropriate infrastructure. Computing should be integrated into the long-term planning of projects. This will assist with the development of well-defined career paths in scientific computing. Improved authentication and authorisation infrastructure, which is standardised across the community, would facilitate the sharing of compute resources and access to data, applications and services. Cloud computing can provide researchers with simple, convenient access to compute resources. STFC should provide guidance on the use of cloud computing, including ensuring best value and appropriate security. 12

16 STFC should provide mechanisms to ensure optimisation of codes so that they make the best use of new hardware. This could be delivered through a mechanism similar to the current Collaborative Computational Projects (CCPs - see p21). Regardless of the mechanism, it is important that experts in new architectures are brought together with those with a detailed knowledge of computing requirements and codes for a specific community. To promote best practice in the sharing and management of research data, STFC should ensure the RCUK Common Principles on Data Policy and Guidelines on Best Practice in the Management of Research Data are observed by all data owners using facilities operated or funded by STFC. Research projects with significant computing requirements should be supported to explore opportunities for industrial engagement. See the Cross-Programme section for cross-programme recommendations, e.g. on data centres, longterm planning, authentication and authorisation solutions, and code optimisation. Recommendation 1: STFC should plan for an order of magnitude increase over the next five years in the computing hardware, storage and network bandwidth available to PPAN researchers. The existing hierarchy (from Tier-1 to Tier-3) should be maintained and strengthened whilst ensuring access to a diverse set of compute resources (high-memory, data intensive, high throughput, massively parallel). Recommendation 2: STFC should ensure that access to Tier-0 scale computing remains available to those PPAN researchers who require it. Recommendation 3: STFC should make it easier for the PPAN community to access computing resources. Resources should be better publicised and guidance provided to ensure that projects are making use of the most appropriate resources. When required, assistance should be provided to people moving up (or down) the tiers. Recommendation 4: STFC should ensure the RCUK Common Principles on Data Policy and Guidelines on Best Practice in the Management of Research Data are observed by all data owners using facilities operated or funded by STFC. 13

17 The UK National Facilities a) Current usage STFC provides support for computing at the UK national facilities, CLF, Diamond and ISIS, in the following categories: HPC/HTC support; data services (accounting/configuration data management, data storage, back-up and recovery services, databases and repositories, data management); data processing, visualisation and analysis; software development, training and community support; networks; and direct facility operations. The majority of these services are based at RAL, co-located with the facilities, although the Hartree Centre and the UK escience Certification Authority are based at Daresbury. The Scientific Computing Application Resource for Facilities (SCARF) cluster provides HPC resource for facility users and STFC staff, for such as molecular simulation work. SCARF is a Tier-2 scale cluster which has several hundred users. CLF operates a data management service that catalogues data and metadata from the Astra-Gemini laser system, enabling it to be displayed in near real-time. ISIS operates data storage software and has a data migration facility to migrate data from disk to tape library and for digital preservation of user data. All ISIS data is held in a data archive at RAL for long term curation. Both CLF and ISIS have internal database services that support user activities such as proposal management and user accounts. External access to scientific and technical output from ISIS and CLF is provided by an archive with metadata of research publications associated with the facilities, whilst Diamond operates similar archive access. Over 14,000 users have accessed this archive over the last 12 months. Diamond runs a data service, aiming eventually to give off-site access for all end users of the data archive. Underpinning all these activities is the STFC Network service, allowing transfer of data across the STFC sites and export from the facilities to user groups where necessary. No current gaps in provision have been identified; however significant trends and increasing requirements for the immediate future are evident due to the acquisition of ever larger data sets. b) Current plans; long term requirements Users of all three facilities identified significant changes in their e-infrastructure requirements that are expected to occur over the next five years. There is an increasing need to combine experimental data with theoretical modelling and simulations. Analysis of data sets and modelling of data are already the rate determining steps limiting throughput at the facilities and therefore investment in e-infrastructure will have significant benefits in output and impact from the facilities. Access to such real-time modelling and simulations will increase the productivity of the facilities by enabling targeted and better designed experiments. Coordinating access to HPC resource with facility access applications across the facilities will be needed to maximise the added value of providing HPC time. In some instances small amounts of HPC resource would be valuable as pump priming to facilitate better experimental design prior to formal applications for facility time. As the algorithms need to become more computationally expensive, there will be a need for significant new facilities for data reduction and modelling, including over the long term when the users will no 14

18 longer be on site. High capacity central resources will be needed to process data in real-time, with visualisation capability, and high throughput simulation computing will be needed. Fast and reliable network connections must be maintained across site and externally, to transfer data and to ensure transparent, fast and accurate access for users to archived data held at the facilities. Large datasets are difficult to transfer to home institutions so options for back-up data sets, (such as those being developed by Diamond [see section a above]) or remote access for users need to be developed further. In parallel, there is expected to be an increasing need for expert computing support, such as code developers and software support, as relatively few of the facility users will be expert users in terms of computing expertise. Provision of computing support through shared support across the facilities would foster best practice and promote coordination of provision, particularly for users working at more than one facility. c) Issues and opportunities The trends driven by the ever increasing data volumes show a growing need for computing provision and support at the facilities to gain maximum impact from the facilities. In some cases these are new costs but in other instances they represent the transfer of activity that was previously carried out at user institutions (such as data storage), to the facilities. It is important to seek the optimum balance between provision on and off-site, recognising that many users will not be experts in computing and will regard this as a support activity they require from the facility and something that they wish to be invisible to them. The increased demand for computing, in terms of hardware and staff support, will likely be reflected in increased operating costs at the facilities; however these are not all necessarily increased costs for the science base. The skills and expertise identified by the users from STFC focus on expertise in compute, algorithm/ code development and support, and user support for computing. Throughout the facilities the use of common platforms, and simple data formats is preferred, with common interfaces to simulation packages. STFC should take the lead to drive best and common practice across the facilities. Data storage for the facilities may offer opportunities for economies of scale with other activities across STFC although logistical issues such as security and access would need to be addressed. d) Recommendations STFC should consider how access to high performance computing capabilities can be made available to facility users, for pump priming experiments, and for modelling, simulation and visualisation. This could be through expansion of the SCARF facility on site or through collaboration with other providers such as ARCHER (the national HPC service) or the Hartree Centre. Provision of HPC resources should avoid the need for parallel applications to facility access and potential double jeopardy where the HPC access is an integral part of the preparations for the experiment. Networking provision should be reviewed to ensure transfer across the site is robust and STFC should undertake a detailed review of requirements for the transfer of data to user sites, and for access to archived data at the facilities. STFC should ensure that the requirements for network transfer and computing at the user sites is defined in collaboration with the users to ensure 15

19 successful transfer. The need for remote access to be straightforward for non-expert users must be taken into account. Due to the increasing size of data sets, the need for accessible data storage will likewise increase. Automated systems to store, access and analyse data will be essential. Working with representatives of the user community, STFC should develop a model for the optimum balance between storage on site, at the user institutions and through Cloud based solutions. This model should consider opportunities for sharing of resources such as data storage with non-facility activities within STFC and the user institutions. STFC should review the computing support (hardware, software and expertise) provided to users, focussing on the support required uniquely at the facilities, promoting coordination and best practice between the facilities. STFC computing support must recognise the differing needs of various research communities and be responsive to the needs of the users, whilst ensuring consistency. Such provision should be included in the operating costs of the facilities. See the Cross-Programme section for cross-programme recommendations, e.g. on support for users, interoperable software and interfaces, standard data formats and metadata etc. Recommendation 5: STFC should consider how access to high performance computing capabilities can be made available to facility users, for pump priming experiments, and for modelling, simulation and visualisation. Provision of HPC resources should avoid the need for parallel applications to facility access and potential double jeopardy where the HPC access is an integral part of the preparations for the experiment. Recommendation 6: Networking provision for the facilities at the RAL site should be reviewed to ensure that the provision keeps pace with capacity requirements and remains robust. The STFC campus network group should be supported to undertake developments not only for internal capacity, but for resilience and to improve the network management structure. Recommendation 7: STFC should undertake a detailed review of requirements for the transfer of data to facility user sites, and for access to archived data at the facilities, considering the need for remote access to be straightforward for non-expert users. Recommendation 8: Working with representatives of the facility user community, STFC should develop a model for the optimum balance between storage on site, at the user institutions and through Cloud based solutions. This model should consider opportunities for sharing of resources such as data storage with non-facility activities within STFC and the user institutions. 16

20 Engineering Support a) Current usage As well as operations and data analysis, the design and construction of STFC Facilities and experiments within the PPAN programme is already, and increasingly, heavily dependent upon access to advanced simulation, Computer Aided Design (CAD), Computer Aided Engineering (CAE) and Computer Aided Manufacturing (CAM). This is true across a wide range of engineering disciplines, from optics, through mechanical engineering and fluid dynamics, to electronics design, device simulations and satellite control. Much of the engineering design and simulation work is carried out with sophisticated commercial packages that increasingly integrate design and simulation. They are demanding applications, often with strict licensing restrictions and requiring high-end dedicated Windows or Unix workstations 1. However, there is also overlap with other aspects of computing such as, for example, the use of GEANT for modelling radiation effects, and the computational fluid dynamics developments of CCP12. The applications themselves are increasingly demanding in terms of computing power and data output. The nature of the broader market for the commercial products, however, tends to focus them on high-end dedicated workstations rather than shared HPC systems. As the simulation demands grow, we might expect to see this distinction blurred in the future. STFC supports access to advanced engineering primarily, though not exclusively, through its Laboratory sites. Sharing across sites is often restricted by license conditions. In some areas, such as micro-electronics, the laboratories also provide access to and support for advanced CAD tools for academic users in universities and research institutes. This includes the training for some of the microelectronics tools (e.g. FPGA programming). In this case STFC must meet stringent security requirements set by the software vendors. Failure to do this would jeopardise access to these advanced tools for HEI and laboratory staff. Underpinning all these activities is the STFC Network service, allowing transfer of data across the STFC sites and export from the facilities to user groups where necessary. No current gaps in provision have been identified beyond the general trend towards more provision driven by greater integration, more sophisticated simulation, and increasing data sizes. b) Current plans; long term requirements Over the next 5 years there will be a gradual increase in computing requirements, probably matched to the evolution of high-end windows/linux servers, although the increasing demands of ever more sophisticated simulation and modelling may open up new requirements for access to HPC machines. The current trend is for increased integration of design and modelling, leading to increased productivity. In micro-electronics the increasing complexity and cost of manufacture is driving a greater focus on simulation and verification before committing to costly fabrication runs. In 1 These different operating systems are not necessarily interchangeable; e.g. in the micro-electronics industry Unix is more important than Windows. 17

21 mechanical engineering, the growth of additive manufacturing (3-d printing) will also see the manufacturing itself increasingly integrated into the design process. In some areas, such as microelectronics and detector system design, there is an emerging trend towards further restricting access to software, requiring users to use remotely hosted services and data libraries, without direct access to the software. As trusted partners with a track record in supporting the academic community the National Laboratories are well placed to be able to offer these services to this community in situations where the software providers themselves do not wish to do so. This would be a means of maintaining UK academic access to advanced tools. If this trend continues it will naturally lead to increased demand for cloud like services such as those already provided by STFC. In parallel, there is expected to be an increasing need for expert computing support, as packages become more complex to operate and administer, fewer users will be expert users in terms of computing expertise. Provision of computing support through shared support across the facilities/departments/institutes would foster best practice and promote coordination of provision. Over recent years it has also been increasingly difficult to secure academic licenses for many products used within STFC Laboratories. This trend is likely to continue given the greater emphasis on industrial collaboration on the STFC campuses. Therefore, it will be important to make effective provision for the cost of commercial licenses, and to ensure efficient and effective use of licensed software. c) Issues and opportunities Many of the issues faced are common to the STFC facilities exploitation described elsewhere: The increased demand for computing, in terms of hardware and staff support Ever increasing data volumes and the data management tools required to support this Restrictive licensing often not well matched to distributed academic collaborations Access to HPC systems and dedicated emulators for hardware modelling and software development may be required in the near future. In some cases it is quite possible that the overall cost of ownership of the most sophisticated tools (including installation and management of advanced design tools), coupled with the security requirements (design tools and design data) will become so high (particularly in the microelectronic sector) as to be impractical for most, if not all, academic groups. It is possible that STFC National Laboratories can provide the critical mass required to offer a managed service to the academic community and their collaborators. This model is effectively already in operation in some areas such as calculating orbits of spacecraft, where the tool suites are sufficiently complex that consortium partners often look to STFC to do these calculations. d) Recommendations STFC should consider how access to high performance computing capabilities can be made available to engineers and technical staff for modelling, simulation and visualisation. In practice uptake will be dependent upon the relevant software availability for the platforms in question. 18

22 Networking provision should be reviewed to ensure that software can be accessed effectively and that data transfer across the site is robust. Due to the increasing size of data sets, the need for accessible data storage will likewise increase. STFC computing support must recognise the differing needs of various engineering communities, requiring access to best in class commercial software as well as providing externally facing support services in microelectronics that need to meet stringent requirements in authentication and authorisation set by the software vendors themselves. STFC should explore opportunities/requirements for managed 2 services to the academic community in order to access the most sophisticated tools. For networks see Recommendation 6. Recommendation 9: STFC should continue to make provision for the cost of engineering CAD/CAM/CAE licenses, both commercial and non-commercial, and to ensure efficient and effective use of licensed software. Recommendation 10: STFC should encourage sharing of expertise, support, and, where possible, licenses for commercial engineering software tools. Recommendation 11: STFC should explore opportunities/requirements for managed services to the academic community in order to access the most sophisticated engineering software tools. 2 In this context, managed is assumed to include training/expert support in exploiting the tools in question. 19

23 Externally Funded Projects and Services As well as providing computing for STFC programmes in Particle Physics, Astronomy and Nuclear Physics and the National Laboratories, STFC also provides computing for many other areas of the UK research base through services and joint research with academic and industrial organisations. Customers (and funders) for these services include other Research Councils, the European Commission (EC), universities and commercial clients. The nature of these services is very varied, but many make use of expertise within the Scientific Computing Department (SCD) and contribute to the STFC mission by delivering world-class research, innovation and skills in scientific computing. a) Current usage Current externally funded projects and services includes, for example: Collaborative Computational Projects JASMIN/Climate and Environmental Monitoring from Space Facility (CEMS) Emerald EC Projects and other grant funded work National Service for Computational Chemistry Software (NSCCS) Medical Research Council (MRC) Data Support Service Hartree Centre see next section Micro-electronics Support Centre (not using SCD) British Atmospheric Data Centre There are also a number of other similar collaborations under development including with the Culham Centre for Fusion Energy (CCFE), the Genome Analysis Centre, the European Centre for Medium Range Weatherforecasting and with individual universities. Collaborative Computational Projects The CCPs are software development projects that bring together researchers and developers in particular topics in computational science. By co-ordinating the work of many groups, the CCPs deliver leading research and high quality sustainable software. They also promote collaboration with industry. The CCPs are funded by the EPSRC, MRC and BBSRC and the US National Science Foundation and delivered and managed by the STFC SCD. JASMIN/CEMS The JASMIN facility is a "super-data-cluster" which delivers infrastructure for analysis of environmental data. It provides a globally unique computational environment that includes a high performance computing resource and a private cloud, both configured with access to a petascale data storage centre. CEMS is the Earth Observation activity which runs on the JASMIN infrastructure, alongside other services provided by JASMIN. JASMIN is funded by the Natural Environment Research Council (NERC) and delivered by the STFC SCD. 20

Collaborative Computational Projects: Networking and Core Support

Collaborative Computational Projects: Networking and Core Support Collaborative Computational Projects: Networking and Core Support Call type: Invitation for proposals Closing date: 16:00 07 October 2014 Related themes: Engineering, ICT, Mathematical sciences, Physical

More information

Building an Open Data Infrastructure for Science: Turning Policy into Practice

Building an Open Data Infrastructure for Science: Turning Policy into Practice Building an Open Infrastructure for Science: Turning Policy into Practice Juan Bicarregui Head of Services Division STFC Department of Scientific Computing Franco-British Workshop on Big in Science, November

More information

Computing Advisory Panel (CAP) Preliminary enquiry in respect of future PRACE membership. Jan- 2014

Computing Advisory Panel (CAP) Preliminary enquiry in respect of future PRACE membership. Jan- 2014 Computing Advisory Panel (CAP) Preliminary enquiry in respect of future PRACE membership Jan2014 To: STFC supported researchers who depend upon HPC facilities, including Cosmology, Particle Physics, Astrophysics,

More information

INFRASTRUCTURE PROGRAMME

INFRASTRUCTURE PROGRAMME INFRASTRUCTURE PROGRAMME programme remit Infrastructure s remit is to provide researchers in the Engineering and Physical sciences with access to the world class facilities necessary to enable internationally

More information

Nuclear Physics Technology Showcase Event 26 September 2013, IoP London. STFC External Innovations Programme Funding Schemes

Nuclear Physics Technology Showcase Event 26 September 2013, IoP London. STFC External Innovations Programme Funding Schemes Nuclear Physics Technology Showcase Event 26 September 2013, IoP London STFC External Innovations Programme Funding Schemes Dr Vlad Skarda DBA CPhys External Innovations Science and Technology Facilities

More information

STFC s Science and Technology Strategy

STFC s Science and Technology Strategy STFC s Science and Technology Strategy Director, Science Programmes Nuclear Physics and Particle Physics Town Meetings April 2008 Addressing the challenges of the twenty-first century The twenty-first

More information

Exploring the roles and responsibilities of data centres and institutions in curating research data a preliminary briefing.

Exploring the roles and responsibilities of data centres and institutions in curating research data a preliminary briefing. Exploring the roles and responsibilities of data centres and institutions in curating research data a preliminary briefing. Dr Liz Lyon, UKOLN, University of Bath Introduction and Objectives UKOLN is undertaking

More information

CYBERINFRASTRUCTURE FRAMEWORK FOR 21 st CENTURY SCIENCE AND ENGINEERING (CIF21)

CYBERINFRASTRUCTURE FRAMEWORK FOR 21 st CENTURY SCIENCE AND ENGINEERING (CIF21) CYBERINFRASTRUCTURE FRAMEWORK FOR 21 st CENTURY SCIENCE AND ENGINEERING (CIF21) Goal Develop and deploy comprehensive, integrated, sustainable, and secure cyberinfrastructure (CI) to accelerate research

More information

Shibbolized irods (and why it matters)

Shibbolized irods (and why it matters) Shibbolized irods (and why it matters) 3 rd TERENA Storage Meeting, Dublin, February 12 th -13 th 2009 David Corney, for Jens Jensen, e-science centre, Rutherford Appleton Lab, UK Overview Introduction

More information

Workprogramme 2014-15

Workprogramme 2014-15 Workprogramme 2014-15 e-infrastructures DCH-RP final conference 22 September 2014 Wim Jansen einfrastructure DG CONNECT European Commission DEVELOPMENT AND DEPLOYMENT OF E-INFRASTRUCTURES AND SERVICES

More information

Panasas High Performance Storage Powers the First Petaflop Supercomputer at Los Alamos National Laboratory

Panasas High Performance Storage Powers the First Petaflop Supercomputer at Los Alamos National Laboratory Customer Success Story Los Alamos National Laboratory Panasas High Performance Storage Powers the First Petaflop Supercomputer at Los Alamos National Laboratory June 2010 Highlights First Petaflop Supercomputer

More information

Executive summary. Key points

Executive summary. Key points Executive summary This report sets out the findings and recommendations of the e-infrastructure Advisory Group commissioned and chaired by BIS into the activities and recommendations put forward by the

More information

Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS

Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS Workshop on the Future of Big Data Management 27-28 June 2013 Philip Kershaw Centre for Environmental Data Archival

More information

Horizon 2020. Research e-infrastructures Excellence in Science Work Programme 2016-17. Wim Jansen. DG CONNECT European Commission

Horizon 2020. Research e-infrastructures Excellence in Science Work Programme 2016-17. Wim Jansen. DG CONNECT European Commission Horizon 2020 Research e-infrastructures Excellence in Science Work Programme 2016-17 Wim Jansen DG CONNECT European Commission 1 Before we start The material here presented has been compiled with great

More information

Integrating Research Information: Requirements of Science Research

Integrating Research Information: Requirements of Science Research Integrating Research Information: Requirements of Science Research Brian Matthews Scientific Information Group E-Science Centre STFC Rutherford Appleton Laboratory brian.matthews@stfc.ac.uk The science

More information

A Policy Framework for Canadian Digital Infrastructure 1

A Policy Framework for Canadian Digital Infrastructure 1 A Policy Framework for Canadian Digital Infrastructure 1 Introduction and Context The Canadian advanced digital infrastructure (DI) ecosystem is the facilities, services and capacities that provide the

More information

Experience of Data Transfer to the Tier-1 from a DIRAC Perspective

Experience of Data Transfer to the Tier-1 from a DIRAC Perspective Experience of Data Transfer to the Tier-1 from a DIRAC Perspective Lydia Heck Institute for Computational Cosmology Manager of the DiRAC-2 Data Centric Facility COSMA 1 Talk layout Introduction to DiRAC?

More information

Big Data Challenges in Bioinformatics

Big Data Challenges in Bioinformatics Big Data Challenges in Bioinformatics BARCELONA SUPERCOMPUTING CENTER COMPUTER SCIENCE DEPARTMENT Autonomic Systems and ebusiness Pla?orms Jordi Torres Jordi.Torres@bsc.es Talk outline! We talk about Petabyte?

More information

The Hartree Centre helps businesses unlock the potential of HPC

The Hartree Centre helps businesses unlock the potential of HPC The Hartree Centre helps businesses unlock the potential of HPC Fostering collaboration and innovation across UK industry with help from IBM Overview The need The Hartree Centre needs leading-edge computing

More information

School of Computing and Technology

School of Computing and Technology School of Computing and Technology We offer employmentfocused education and research in computing, the built environment and engineering, giving our students industry-relevant skills and knowledge. Applied

More information

EUDAT. Towards a pan-european Collaborative Data Infrastructure

EUDAT. Towards a pan-european Collaborative Data Infrastructure EUDAT Towards a pan-european Collaborative Data Infrastructure Damien Lecarpentier CSC-IT Center for Science, Finland EISCAT User Meeting, Uppsala,6 May 2013 2 Exponential growth Data trends Zettabytes

More information

IBM TotalStorage IBM TotalStorage Virtual Tape Server

IBM TotalStorage IBM TotalStorage Virtual Tape Server IBM TotalStorage IBM TotalStorage Virtual Tape Server A powerful tape storage system that helps address the demanding storage requirements of e-business storag Storage for Improved How can you strategically

More information

Creating a Data Management Plan for your Research

Creating a Data Management Plan for your Research Creating a Data Management Plan for your Research EPFL Workshop Lausaunne, 28 Oct 2014 Robin Rice, Laine Ruus EDINA and Data Library Course content What is a Data Management Plan? Benefits and drivers

More information

Information Services Strategy 2011-2013

Information Services Strategy 2011-2013 Information Services Strategy Issue 1 1 Introduction The States of Jersey public sector is facing significant pressure for efficiencies and savings. This has created the context to take a fresh look at

More information

Information Technology: Principles and Strategic Aims

Information Technology: Principles and Strategic Aims Information Technology: Principles and Strategic Aims As observed in the University of Cambridge IT Strategy, the University is a complex and diverse organization whose IT requirements vary depending on

More information

Archive Data Retention & Compliance. Solutions Integrated Storage Appliances. Management Optimized Storage & Migration

Archive Data Retention & Compliance. Solutions Integrated Storage Appliances. Management Optimized Storage & Migration Solutions Integrated Storage Appliances Management Optimized Storage & Migration Archive Data Retention & Compliance Services Global Installation & Support SECURING THE FUTURE OF YOUR DATA w w w.q sta

More information

Cambridge University Library. Working together: a strategic framework 2010 2013

Cambridge University Library. Working together: a strategic framework 2010 2013 1 Cambridge University Library Working together: a strategic framework 2010 2013 2 W o r k i n g to g e t h e r : a s t r at e g i c f r a m e w o r k 2010 2013 Vision Cambridge University Library will

More information

Industry. Head of Research Service Desk Institute

Industry. Head of Research Service Desk Institute Asset Management in the ITSM Industry Prepared by Daniel Wood Head of Research Service Desk Institute Sponsored by Declaration We believe the information in this document to be accurate, relevant and truthful

More information

Data sharing and Big Data in the physical sciences. 2 October 2015

Data sharing and Big Data in the physical sciences. 2 October 2015 Data sharing and Big Data in the physical sciences 2 October 2015 Content Digital curation: Data and metadata Why consider the physical sciences? Astronomy: Video Physics: LHC for example. Video The Research

More information

Asset Innovation. On The Move. www.cieam.com

Asset Innovation. On The Move. www.cieam.com Asset Innovation On The Move Infrastructure Asset Management CIEAM Infrastructure and Engineering Asset Management Asset management is the process of organising, planning and controlling the acquisition,

More information

Deploying a distributed data storage system on the UK National Grid Service using federated SRB

Deploying a distributed data storage system on the UK National Grid Service using federated SRB Deploying a distributed data storage system on the UK National Grid Service using federated SRB Manandhar A.S., Kleese K., Berrisford P., Brown G.D. CCLRC e-science Center Abstract As Grid enabled applications

More information

Dell s SAP HANA Appliance

Dell s SAP HANA Appliance Dell s SAP HANA Appliance SAP HANA is the next generation of SAP in-memory computing technology. Dell and SAP have partnered to deliver an SAP HANA appliance that provides multipurpose, data source-agnostic,

More information

University of Strathclyde: Information Services Directorate Operational Plan for 2014/15

University of Strathclyde: Information Services Directorate Operational Plan for 2014/15 University of Strathclyde: Information Services Directorate Operational Plan for 2014/15 Enabling the future: transforming our information systems and technology 2014/15 2014/2015 CONTENTS 1 DRIVERS, STRATEGIC

More information

Research Data Storage and the University of Bristol

Research Data Storage and the University of Bristol Introduction: Policy for the use of the Research Data Storage Facility The University s High Performance Computing (HPC) facility went live to users in May 2007. Access to this world-class HPC facility

More information

CHAPTER 1 INTRODUCTION

CHAPTER 1 INTRODUCTION CHAPTER 1 INTRODUCTION 1.1 Background The command over cloud computing infrastructure is increasing with the growing demands of IT infrastructure during the changed business scenario of the 21 st Century.

More information

What EDINA Does: EDINA Vision for 2020 A Community Report

What EDINA Does: EDINA Vision for 2020 A Community Report EDINA What Vision EDINA for Does: 2020 A Community Report Director s Introduction Welcome to our vision for EDINA in 2020. At EDINA, we develop and deliver shared services and infrastructure to support

More information

www.xenon.com.au STORAGE HIGH SPEED INTERCONNECTS HIGH PERFORMANCE COMPUTING VISUALISATION GPU COMPUTING

www.xenon.com.au STORAGE HIGH SPEED INTERCONNECTS HIGH PERFORMANCE COMPUTING VISUALISATION GPU COMPUTING www.xenon.com.au STORAGE HIGH SPEED INTERCONNECTS HIGH PERFORMANCE COMPUTING GPU COMPUTING VISUALISATION XENON Accelerating Exploration Mineral, oil and gas exploration is an expensive and challenging

More information

Data Management Planning

Data Management Planning DIY Research Data Management Training Kit for Librarians Data Management Planning Kerry Miller Digital Curation Centre University of Edinburgh Kerry.miller@ed.ac.uk Running Order I. What is Research Data

More information

Data Intensive Research Initiative for South Africa (DIRISA)

Data Intensive Research Initiative for South Africa (DIRISA) Data Intensive Research Initiative for South Africa (DIRISA) A Reinterpreted Vision A. Vahed 25 November 2014 Outline Background Data Landscape Strategy & Objectives Activities & Outputs Organisational

More information

LIBER Case Study: University of Oxford Research Data Management Infrastructure

LIBER Case Study: University of Oxford Research Data Management Infrastructure LIBER Case Study: University of Oxford Research Data Management Infrastructure AuthorS: Dr James A. J. Wilson, University of Oxford, james.wilson@it.ox.ac.uk Keywords: generic, institutional, software

More information

Benefits of managing and sharing your data

Benefits of managing and sharing your data Benefits of managing and sharing your data Research Data Management Support Services UK Data Service University of Essex April 2014 Overview Introduction to the UK Data Archive What is data management?

More information

Inside Track Research Note. In association with. Enterprise Storage Architectures. Is it only about scale up or scale out?

Inside Track Research Note. In association with. Enterprise Storage Architectures. Is it only about scale up or scale out? Research Note In association with Enterprise Storage Architectures Is it only about scale up or scale out? August 2015 About this The insights presented in this document are derived from independent research

More information

How To Speed Up A Flash Flash Storage System With The Hyperq Memory Router

How To Speed Up A Flash Flash Storage System With The Hyperq Memory Router HyperQ Hybrid Flash Storage Made Easy White Paper Parsec Labs, LLC. 7101 Northland Circle North, Suite 105 Brooklyn Park, MN 55428 USA 1-763-219-8811 www.parseclabs.com info@parseclabs.com sales@parseclabs.com

More information

Unterstützung datenintensiver Forschung am KIT Aktivitäten, Dienste und Erfahrungen

Unterstützung datenintensiver Forschung am KIT Aktivitäten, Dienste und Erfahrungen Unterstützung datenintensiver Forschung am KIT Aktivitäten, Dienste und Erfahrungen Achim Streit Steinbuch Centre for Computing (SCC) KIT Universität des Landes Baden-Württemberg und nationales Forschungszentrum

More information

The challenges of digital preservation to support research in the digital age

The challenges of digital preservation to support research in the digital age DRAFT FOR DISCUSSION WITH ADVISORY COUNCIL MEMBERS ONLY The challenges of digital preservation to support research in the digital age Lynne Brindley CEO, The British Library November 2005 Agenda UK developments

More information

Make the Most of Big Data to Drive Innovation Through Reseach

Make the Most of Big Data to Drive Innovation Through Reseach White Paper Make the Most of Big Data to Drive Innovation Through Reseach Bob Burwell, NetApp November 2012 WP-7172 Abstract Monumental data growth is a fact of life in research universities. The ability

More information

Research and Innovation Strategy: delivering a flexible workforce receptive to research and innovation

Research and Innovation Strategy: delivering a flexible workforce receptive to research and innovation Research and Innovation Strategy: delivering a flexible workforce receptive to research and innovation Contents List of Abbreviations 3 Executive Summary 4 Introduction 5 Aims of the Strategy 8 Objectives

More information

Linking raw data with scientific workflow and software repository: some early

Linking raw data with scientific workflow and software repository: some early Linking raw data with scientific workflow and software repository: some early experience in PanData-ODI Erica Yang, Brian Matthews Scientific Computing Department (SCD) Rutherford Appleton Laboratory (RAL)

More information

1 Executive Summary... 3. 2 Document Structure... 5. 3 Business Context... 6

1 Executive Summary... 3. 2 Document Structure... 5. 3 Business Context... 6 Contents 1 Executive Summary... 3 2 Document Structure... 5 3 Business Context... 6 4 Strategic Response... 8 4.1 Platform... 8 4.2 Network... 10 4.3 Unified Communications... 11 5 Implementation status...

More information

A Future Scenario of interconnected EO Platforms How will EO data be used in 2025?

A Future Scenario of interconnected EO Platforms How will EO data be used in 2025? A Future Scenario of interconnected EO Platforms How will EO data be used in 2025? ESA UNCLASSIFIED For Official Use European EO data asset Heritage missions Heritage Core GS (data preservation, curation

More information

THE CCLRC DATA PORTAL

THE CCLRC DATA PORTAL THE CCLRC DATA PORTAL Glen Drinkwater, Shoaib Sufi CCLRC Daresbury Laboratory, Daresbury, Warrington, Cheshire, WA4 4AD, UK. E-mail: g.j.drinkwater@dl.ac.uk, s.a.sufi@dl.ac.uk Abstract: The project aims

More information

EARSC Views on the. Procurement of the Copernicus Services

EARSC Views on the. Procurement of the Copernicus Services EARSC Views on the Procurement of the Copernicus Services EARSC, the European Association of Remote Sensing Companies represents the Earth Observation geoinformation services sector in Europe. Today EARSC

More information

MAC Consultation on the Review of the Shortage Occupation Lists for the UK and Scotland and Creative Occupations.

MAC Consultation on the Review of the Shortage Occupation Lists for the UK and Scotland and Creative Occupations. Response by the Research Councils UK To MAC Consultation on the Review of the Shortage Occupation Lists for the UK and Scotland and Creative Occupations. Background Research Councils UK (RCUK) is the strategic

More information

Customer requirements. Asset management planning Inspection and assessment Route asset planning Annual work plans Contracting strategy

Customer requirements. Asset management planning Inspection and assessment Route asset planning Annual work plans Contracting strategy Section 8 Output monitoring Inputs Customer requirements Safety standards Outputs and funding SRA and Government Policy Network stewardship strategy Asset and operational policies Maintenance & renewal

More information

ESRC Big Data Network Phase 2: Business and Local Government Data Research Centres Welcome, Context, and Call Objectives

ESRC Big Data Network Phase 2: Business and Local Government Data Research Centres Welcome, Context, and Call Objectives ESRC Big Data Network Phase 2: Business and Local Government Data Research Centres Welcome, Context, and Call Objectives Dr Fiona Armstrong, Professor Peter Elias, Dr Paul Meller Today s event This event

More information

ARCHITECTURE SERVICES. G-CLOUD SERVICE DEFINITION.

ARCHITECTURE SERVICES. G-CLOUD SERVICE DEFINITION. ARCHITECTURE SERVICES. G-CLOUD SERVICE DEFINITION. Table of contents 1 Introduction...3 2 Architecture Services...4 2.1 Enterprise Architecture Services...5 2.2 Solution Architecture Services...6 2.3 Service

More information

8970/15 FMA/AFG/cb 1 DG G 3 C

8970/15 FMA/AFG/cb 1 DG G 3 C Council of the European Union Brussels, 19 May 2015 (OR. en) 8970/15 NOTE RECH 141 TELECOM 119 COMPET 228 IND 80 From: Permanent Representatives Committee (Part 1) To: Council No. prev. doc.: 8583/15 RECH

More information

Building a Scalable Big Data Infrastructure for Dynamic Workflows

Building a Scalable Big Data Infrastructure for Dynamic Workflows Building a Scalable Big Data Infrastructure for Dynamic Workflows INTRODUCTION Organizations of all types and sizes are looking to big data to help them make faster, more intelligent decisions. Many efforts

More information

High Performance Computing (HPC)

High Performance Computing (HPC) High Performance Computing (HPC) High Performance Computing (HPC) White Paper Attn: Name, Title Phone: xxx.xxx.xxxx Fax: xxx.xxx.xxxx 1.0 OVERVIEW When heterogeneous enterprise environments are involved,

More information

S O U T H A F R I C A. Prepared by: Vinny Pillay Occasion: EUREKA - ITEA Co- Summit 2015 Date: 10 March 2015

S O U T H A F R I C A. Prepared by: Vinny Pillay Occasion: EUREKA - ITEA Co- Summit 2015 Date: 10 March 2015 S O U T H A F R I C A Prepared by: Vinny Pillay Occasion: EUREKA - ITEA Co- Summit 2015 Date: 10 March 2015 Overview South Africa s strategic context Framework for S&T cooperation with Europe ICT RDI Roadmap

More information

The Scottish Wide Area Network Programme

The Scottish Wide Area Network Programme The Scottish Wide Area Network Release: Issued Version: 1.0 Date: 16/03/2015 Author: Andy Williamson Manager Owner: Anne Moises SRO Client: Board Version: Issued 1.0 Page 1 of 8 16/04/2015 Document Location

More information

THE EFFECTS OF BIG DATA ON INFRASTRUCTURE. Sakkie Janse van Rensburg Dr Dale Peters UCT 1

THE EFFECTS OF BIG DATA ON INFRASTRUCTURE. Sakkie Janse van Rensburg Dr Dale Peters UCT 1 THE EFFECTS OF BIG DATA ON INFRASTRUCTURE Sakkie Janse van Rensburg Dr Dale Peters UCT 1 BIG DATA DEFINED Gartner in 2001 first coined the phrase Big Data Volume Velocity Big data is a popular term used

More information

Compute Canada Technology Briefing

Compute Canada Technology Briefing Compute Canada Technology Briefing November 12, 2015 Introduction Compute Canada, in partnership with regional organizations ACENET, Calcul Québec, Compute Ontario and WestGrid, leads the acceleration

More information

Research is everybody s business

Research is everybody s business Research and Development Strategy (DRAFT) 2013-2018 Research is everybody s business 1 Information Reader Box to be inserted for documents six pages and over. (To be inserted in final version, after consultation

More information

Data Analytics, Management, Security and Privacy (Priority Area B)

Data Analytics, Management, Security and Privacy (Priority Area B) PRIORITY AREA B: DATA ANALYTICS, MANAGEMENT, SECURITY AND PRIVACY ACTION PLAN Data Analytics, Security and Privacy (Priority Area B) Context Data is growing at an exponential rate; information on the web

More information

IT requirements for the next generation research infrastructures

IT requirements for the next generation research infrastructures IT requirements for the next generation research infrastructures CRISP Workshop: CERN - 1 February, 2013 Introduction This document provides a summary of the IT requirements for the next generation research

More information

The Road to Convergence

The Road to Convergence A UBM TECHWEB WHITE PAPER SEPTEMBER 2012 The Road to Convergence Six keys to getting there with the most confidence and the least risk. Brought to you by The Road to Convergence Six keys to getting there

More information

BBC Technology Strategy

BBC Technology Strategy BBC Technology Strategy Issued: January 2010 For more information contact: Spencer.piggott@bbc.co.uk Copyright 2010 BBC All Rights Reserved TECHNOLOGY STRATEGY Introduction This paper defines the BBC s

More information

Dr. Raju Namburu Computational Sciences Campaign U.S. Army Research Laboratory. The Nation s Premier Laboratory for Land Forces UNCLASSIFIED

Dr. Raju Namburu Computational Sciences Campaign U.S. Army Research Laboratory. The Nation s Premier Laboratory for Land Forces UNCLASSIFIED Dr. Raju Namburu Computational Sciences Campaign U.S. Army Research Laboratory 21 st Century Research Continuum Theory Theory embodied in computation Hypotheses tested through experiment SCIENTIFIC METHODS

More information

9360/15 FMA/AFG/cb 1 DG G 3 C

9360/15 FMA/AFG/cb 1 DG G 3 C Council of the European Union Brussels, 29 May 2015 (OR. en) 9360/15 OUTCOME OF PROCEEDINGS From: To: Council Delegations RECH 183 TELECOM 134 COMPET 288 IND 92 No. prev. doc.: 8970/15 RECH 141 TELECOM

More information

The NREN s core activities are in providing network and associated services to its user community that usually comprises:

The NREN s core activities are in providing network and associated services to its user community that usually comprises: 3 NREN and its Users The NREN s core activities are in providing network and associated services to its user community that usually comprises: Higher education institutions and possibly other levels of

More information

Scientific Cloud Computing Infrastructure for Europe Strategic Plan. Bob Jones,

Scientific Cloud Computing Infrastructure for Europe Strategic Plan. Bob Jones, Scientific Cloud Computing Infrastructure for Europe Strategic Plan Bob Jones, IT department, CERN Origin of the initiative Conceived by ESA as a prospective for providing cloud services to space sector

More information

HyperQ DR Replication White Paper. The Easy Way to Protect Your Data

HyperQ DR Replication White Paper. The Easy Way to Protect Your Data HyperQ DR Replication White Paper The Easy Way to Protect Your Data Parsec Labs, LLC 7101 Northland Circle North, Suite 105 Brooklyn Park, MN 55428 USA 1-763-219-8811 www.parseclabs.com info@parseclabs.com

More information

Virtualisation Cloud Computing at the RAL Tier 1. Ian Collier STFC RAL Tier 1 HEPiX, Bologna, 18 th April 2013

Virtualisation Cloud Computing at the RAL Tier 1. Ian Collier STFC RAL Tier 1 HEPiX, Bologna, 18 th April 2013 Virtualisation Cloud Computing at the RAL Tier 1 Ian Collier STFC RAL Tier 1 HEPiX, Bologna, 18 th April 2013 Virtualisation @ RAL Context at RAL Hyper-V Services Platform Scientific Computing Department

More information

Data Requirements from NERSC Requirements Reviews

Data Requirements from NERSC Requirements Reviews Data Requirements from NERSC Requirements Reviews Richard Gerber and Katherine Yelick Lawrence Berkeley National Laboratory Summary Department of Energy Scientists represented by the NERSC user community

More information

Impact of Big Data in Oil & Gas Industry. Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India.

Impact of Big Data in Oil & Gas Industry. Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India. Impact of Big Data in Oil & Gas Industry Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India. New Age Information 2.92 billions Internet Users in 2014 Twitter processes 7 terabytes

More information

The HP IT Transformation Story

The HP IT Transformation Story The HP IT Transformation Story Continued consolidation and infrastructure transformation impacts to the physical data center Dave Rotheroe, October, 2015 Why do data centers exist? Business Problem Application

More information

CYBERINFRASTRUCTURE FRAMEWORK FOR 21 ST CENTURY SCIENCE, ENGINEERING, AND EDUCATION (CIF21) $100,070,000 -$32,350,000 / -24.43%

CYBERINFRASTRUCTURE FRAMEWORK FOR 21 ST CENTURY SCIENCE, ENGINEERING, AND EDUCATION (CIF21) $100,070,000 -$32,350,000 / -24.43% CYBERINFRASTRUCTURE FRAMEWORK FOR 21 ST CENTURY SCIENCE, ENGINEERING, AND EDUCATION (CIF21) $100,070,000 -$32,350,000 / -24.43% Overview The Cyberinfrastructure Framework for 21 st Century Science, Engineering,

More information

Data Centric Systems (DCS)

Data Centric Systems (DCS) Data Centric Systems (DCS) Architecture and Solutions for High Performance Computing, Big Data and High Performance Analytics High Performance Computing with Data Centric Systems 1 Data Centric Systems

More information

JISC. Technical Review of Using Cloud for Research. Guidance Notes to Cloud Infrastructure Service Providers. Introduction

JISC. Technical Review of Using Cloud for Research. Guidance Notes to Cloud Infrastructure Service Providers. Introduction JISC Technical Review of Using Cloud for Research Guidance Notes to Cloud Infrastructure Service Providers May, 2010 Introduction Provisioning and maintenance of research computing facilities is a core

More information

NERC Data Policy Guidance Notes

NERC Data Policy Guidance Notes NERC Data Policy Guidance Notes Author: Mark Thorley NERC Data Management Coordinator Contents 1. Data covered by the NERC Data Policy 2. Definition of terms a. Environmental data b. Information products

More information

White paper: Unlocking the potential of load testing to maximise ROI and reduce risk.

White paper: Unlocking the potential of load testing to maximise ROI and reduce risk. White paper: Unlocking the potential of load testing to maximise ROI and reduce risk. Executive Summary Load testing can be used in a range of business scenarios to deliver numerous benefits. At its core,

More information

Taught Postgraduate programmes in the School of Physics and Astronomy

Taught Postgraduate programmes in the School of Physics and Astronomy Astrophysics (MSc) Theoretical Physics (MSc) Physics: Advanced Materials (MSc) Physics: Energy and the Environment (MSc) Physics: Global Security (MSc) Physics: Life Sciences (MSc) Physics: Nuclear Technology

More information

Jeff Wolf Deputy Director HPC Innovation Center

Jeff Wolf Deputy Director HPC Innovation Center Public Presentation for Blue Gene Consortium Nov. 19, 2013 www.hpcinnovationcenter.com Jeff Wolf Deputy Director HPC Innovation Center This work was performed under the auspices of the U.S. Department

More information

BUILDING A SCALABLE BIG DATA INFRASTRUCTURE FOR DYNAMIC WORKFLOWS

BUILDING A SCALABLE BIG DATA INFRASTRUCTURE FOR DYNAMIC WORKFLOWS BUILDING A SCALABLE BIG DATA INFRASTRUCTURE FOR DYNAMIC WORKFLOWS ESSENTIALS Executive Summary Big Data is placing new demands on IT infrastructures. The challenge is how to meet growing performance demands

More information

THE BRITISH LIBRARY. Unlocking The Value. The British Library s Collection Metadata Strategy 2015-2018. Page 1 of 8

THE BRITISH LIBRARY. Unlocking The Value. The British Library s Collection Metadata Strategy 2015-2018. Page 1 of 8 THE BRITISH LIBRARY Unlocking The Value The British Library s Collection Metadata Strategy 2015-2018 Page 1 of 8 Summary Our vision is that by 2020 the Library s collection metadata assets will be comprehensive,

More information

CiCS INFORMATION TECHNOLOGY STRATEGY 2008 2013 1. PURPOSE 2. CONTEXT. 2.1 Information Technology. 2.2 Changing Environment. 2.

CiCS INFORMATION TECHNOLOGY STRATEGY 2008 2013 1. PURPOSE 2. CONTEXT. 2.1 Information Technology. 2.2 Changing Environment. 2. CiCS INFORMATION TECHNOLOGY STRATEGY 2008 2013 1. PURPOSE This strategy replaces the previous IT strategy and is an integral part of the overall strategy being developed by Corporate Information and Computing

More information

HPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk

HPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk HPC and Big Data EPCC The University of Edinburgh Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk EPCC Facilities Technology Transfer European Projects HPC Research Visitor Programmes Training

More information

Big Workflow: More than Just Intelligent Workload Management for Big Data

Big Workflow: More than Just Intelligent Workload Management for Big Data Big Workflow: More than Just Intelligent Workload Management for Big Data Michael Feldman White Paper February 2014 EXECUTIVE SUMMARY Big data applications represent a fast-growing category of high-value

More information

and the world is built on information

and the world is built on information Let s Build a Smarter Planet Starting with a more dynamic and the world is built on information Guy England Storage sales manager CEEMEA englag@ae.ibm.com Tel: +971 50 55 77 614 IBM Building a Smarter

More information

IBM Enterprise Content Management Product Strategy

IBM Enterprise Content Management Product Strategy White Paper July 2007 IBM Information Management software IBM Enterprise Content Management Product Strategy 2 IBM Innovation Enterprise Content Management (ECM) IBM Investment in ECM IBM ECM Vision Contents

More information

City and County of Swansea. Human Resources & Workforce Strategy 2013-2017. Ambition is Critical 1

City and County of Swansea. Human Resources & Workforce Strategy 2013-2017. Ambition is Critical 1 City and County of Swansea Human Resources & Workforce Strategy 2013-2017 Ambition is Critical 1 Delivering quality services for a safer, greener, smarter, fairer, healthier, richer Swansea Executive Summary

More information

Software Engineering Support

Software Engineering Support Software Engineering Support Christopher Greenough, Alan Kyffin, Gemma Poulter Software Engineering Group Scientific Computing Department STFC Rutherford Appleton Laboratory christopher.greenough@stfc.ac.uk

More information

UK Government Information Economy Strategy

UK Government Information Economy Strategy Industrial Strategy: government and industry in partnership UK Government Information Economy Strategy A Call for Views and Evidence February 2013 Contents Overview of Industrial Strategy... 3 How to respond...

More information

PRACE the European HPC Research Infrastructure. Carlos Mérida-Campos, Advisor of Spanish Member at PRACE Council

PRACE the European HPC Research Infrastructure. Carlos Mérida-Campos, Advisor of Spanish Member at PRACE Council PRACE the European HPC Research Infrastructure Carlos Mérida-Campos, Advisor of Spanish Member at PRACE Council Barcelona, 6-June-2013 PRACE an European e-infrastructure & ESFRI-list item in operation

More information

Strategic Plan 2013 2017

Strategic Plan 2013 2017 Plan 0 07 Mapping the Library for the Global Network University NYU DIVISION OF LIBRARIES Our Mission New York University Libraries is a global organization that advances learning, research, and scholarly

More information

STORY: MASSIVE AFFORDABLE COMPUTING STORY OVERVIEW

STORY: MASSIVE AFFORDABLE COMPUTING STORY OVERVIEW STORY: MASSIVE AFFORDABLE COMPUTING STORY OVERVIEW Scientists and researchers at the University of the Witwatersrand are pioneering a new generation of generic supercomputers. The team of the High Energy

More information

PROGRAMME OVERVIEW: G-CLOUD APPLICATIONS STORE FOR GOVERNMENT DATA CENTRE CONSOLIDATION

PROGRAMME OVERVIEW: G-CLOUD APPLICATIONS STORE FOR GOVERNMENT DATA CENTRE CONSOLIDATION PROGRAMME OVERVIEW: G-CLOUD APPLICATIONS STORE FOR GOVERNMENT DATA CENTRE CONSOLIDATION 1. Introduction This document has been written for all those interested in the future approach for delivering ICT

More information

CEDA Storage. Dr Matt Pritchard. Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk

CEDA Storage. Dr Matt Pritchard. Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk CEDA Storage Dr Matt Pritchard Centre for Environmental Data Archival (CEDA) www.ceda.ac.uk How we store our data NAS Technology Backup JASMIN/CEMS CEDA Storage Data stored as files on disk. Data is migrated

More information

IBM Global Technology Services March 2008. Virtualization for disaster recovery: areas of focus and consideration.

IBM Global Technology Services March 2008. Virtualization for disaster recovery: areas of focus and consideration. IBM Global Technology Services March 2008 Virtualization for disaster recovery: Page 2 Contents 2 Introduction 3 Understanding the virtualization approach 4 A properly constructed virtualization strategy

More information