CHPC initiatives in Data and HPC in South Africa. An initiative of the Department of Science and Technology Managed by the CSIR Meraka Institute

Transcription

1 CHPC initiatives in Data and HPC in South Africa An initiative of the Department of Science and Technology Managed by the CSIR Meraka Institute

2 2 CHPC initiatives in Data and HPC in South Africa

3 The South African Department of Science and Technology (DST) established two pillars of cyberinfrastructure for the country. These are: 1. The South African National Research Network (SANReN): established to improve the network linking research institutions and academia in the country. 2. The Centre for High Performance Computing: established to support research and commerce with globally competitive facilities. The challenge became that there needed to be a system for the management of large physical data. The CHPC, through the support of the DST began to initiate the formation of the third pillar of the South African Cyberinfrastructure, the Very Large Data Base (VLDB). At the end of the 2011/12 financial year, the CHPC completed the first phase of the VLDB which involved putting in place the R17.5 Million, fully synchronised data storage system between the Pretoria and Cape Town sites. The infrastructure has a storage facility of two petabytes and is accessible through the SANReN network. It operates at a full capacity of 10 Gps/s and provides the latest technology based on open source software to enable CHPC users to define their own rules in data acquisition, sharing and curation. This first phase was funded by the DST, following a positive report on the 2007 feasibility study. The VLDB project has now been extended and renamed as the Data Intensive Research Infrastructure of South Africa (DIRISA); this was done to incorporate the curation of data that may not be large in nature, but still of national importance. DIRISA s objectives 1 e-research and e-education: Access to research outputs, the ability to find and collate data rapidly and the capability to commit one s own outputs to collaborative on-line platforms and publication channels, all form part of an emerging e-research and e-education trend. Because the architecture for this is mostly open, DIRISA will, by way of adherence to interoperability standards, fit into an infrastructure in support of these initiatives. These interventions are also referred to as Virtual Research Environments (VREs). 3

4 4 2 Data Publication: There is a strong international push towards the publication of data in its own right as a peer-reviewed research output. This is in response to the fact that many scientists are focused on the production of data sets and do not receive adequate professional recognition for their efforts. It is expected that this will lead, inter alia, to a need for the infrastructure to publish, curate and disseminate research outputs and not all institutions or relatively short-lived funded projects, will be in a position to provide such infrastructure. 3Leveraging Investment: The DST, through National Research Foundation and other mechanisms, invest around R 4.4 billion per annum in research development initiatives (RDI) and worldwide the expectation is that this investment should stimulate the knowledge economy. Other governmental departments and agencies in South Africa also commit a large portion of funding towards research and innovation interventions. The ability to leverage on the investment and prevent the duplication of effort for long-term benefits is strongly dependent on reliable, standardised access to previous RDI outputs.

5 4 Technology Development: Broad-based standardisation of meta-data (to improve discoverability and enhance future usability of RDI outputs) and the same trend in respect of data are strong drivers in support of the type of software infrastructure envisaged for DIRISA. The global interest and investment in research data infrastructures has a side benefit in clarifying workable approaches and architectures, discussing policy issues, best practice, sustainable funding models, and knowledge sharing. 5 World Data System and Trusted Digital Repositories: Some form of accreditation or certification is implied for initiatives such as DIRISA, and here international developments provide guidance. Both the ICSU Word Data System (WDS, aiming at dissemination of quality assured and peer reviewed scientific data) and the concept of a Trusted Digital Repository promote the good practices that will result in proper long-term curation of RDI outputs. 5

6 6 Broad-based Computing: The value of any data initiative such as DIRISA is to provide a broad-based data management environment. Stakeholders must be able to access, deposit, store and manage their data effortlessly through different devices, at any location with sufficient internet access. The idea of integrating work in progress, existing data and digital outputs of research, to cutting edge computing solutions such as cloud-based services and other novel technical solutions will be central to the impact of all research and the long-term sustainability of DIRISA. It is perhaps important to stipulate that while DIRISA will be working with stakeholders to provide flexible solutions through shared responsibilities, it will not claim any ownership of the data but will merely act as a custodian. 7 National learning repositories: Currently, knowledge such as in the form of training materials and tacit information, is scattered and dispersed cross the country. Normally, after training interventions are completed, particularly a training intervention which is supported through public funding, there is no 6

7 mechanism to retain the knowledge generated to ensure such knowledge can be openly accessed by other South Africans. It has been suggested that sharing some of the nonproprietary information/knowledge created by organisations supported through public funding, can strengthen the human capital development progress of South Africa and reduce the duplication of effort. 8 Foster innovation value chain: South Africa is committed to foster innovation. Having the ability to make use of the appropriate data can often underpin the success of a particular innovation pipeline, particularly for stakeholders from the private sector. DIRISA can add value to commercial organisations, especially new entities which were established as the spin-off from research interventions previously funded through the public investment schemes. Successes thus far The CHPC has completed the full migration of data portals in Climate Modelling, Astronomy and Earth Observation. Short term goals The Centre is currently migrating data portals in humanities, medical applicationson rheumatic heart disease data and bioinformatics data on genomic sequencing. Work is underway to drive the identification and curation of critical research data in order to make it accessible to a wider community. There are also plans to expand the current storage capacity of 2 petabytes to enable sufficient access by other strategic partners such as SKA, bioinformatics, Human Science Research Council (HSRC), medical applications; etc. DIRISA s storage infrastructure DIRISA utilises a (Web Object Storage) WOS 2.0 storage System. WOS delivers greater storage efficiency compared to conventional SAN/NAS file systems by utilising at least 25% more available disk platter space. 7

8 An initiative of: Managed by: Contact us: CSIR Campus, 15 Lower Hope Street, Rosebank, Cape Town, South Africa Tel: Fax: