RESEARCH DATA STORAGE INFRASTRUCTURE EDUCATION INVESTMENT FUND PROJECT FINAL REPORT

Transcription

1 RESEARCH DATA STORAGE INFRASTRUCTURE EDUCATION INVESTMENT FUND PROJECT FINAL REPORT 30 th March 2015

2 Table of Contents EXECUTIVE SUMMARY... 4 PURPOSE... 4 IMPACT OF THIS PROJECT... 4 HISTORY... 4 OBJECTIVES... 5 ACTIVITIES AND OUTCOMES... 5 FINANCE PROJECT BACKGROUND INTRODUCTION OBJECTIVES OF THE PROJECT INTENDED OUTCOMES OF THE PROJECT SCOPE ACTIVITIES UNDERTAKEN DURING NODE DEVELOPMENT (NODE) PROGRAMME RESEARCH DATA SERVICES (REDS) PROGRAMME DATA SHARING (DASH) PROGRAMME VENDOR PANEL (VEPA) PROGRAMME SECTOR ENGAGEMENT GOVERNANCE RISK MITIGATION MILESTONES FROM JULY 2014 TO JUNE PROJECT RESOURCES PROJECT OFFICE DESCRIPTION OF ROLES RDSI PROJECT FINANCE SUMMARY EXPECTED FUTURE USAGE OF THE INFRASTRUCTURE PROJECT KEY SUCCESSES AND LESSONS LEARNT THE PROJECT MANAGEMENT METHODOLOGY AND RESOURCES RDSI PROGRAMMES EXPECTED FUTURE TRENDS IN RESEARCH INFRASTRUCTURE USE PERFORMANCE INDICATORS PROVIDING RESEARCH INFRASTRUCTURE VALUE OF NEW INFRASTRUCTURE BY LOCATION VALUE OF ALL INFRASTRUCTURES MADE AVAILABLE UNDER EIF MEETING RESEARCHER NEEDS QUALITY OF RESEARCH INFRASTRUCTURE COLLABORATIVE INFRASTRUCTURE PROVISION FOSTERING COLLABORATIVE AND WORLD-CLASS RESEARCH APPENDIX A LARGE ALLOCATION COLLECTION PROPOSALS A.1 LARGE ALLOCATION COLLECTIONS RDSI Project Final Report Page 2 of 136

3 A.2 EVALUATION PROCESS APPENDIX B NODES ASSET REGISTER ADELAIDE (ERSA) SYDNEY (INTERSECT) CANBERRA (NCI) PERTH (PAWSEY CENTRE) BRISBANE (QCIF) TOWNSVILLE (QCIF) HOBART (TPAC) MELBOURNE (VICNODE) APPENDIX C RDSI RISK REGISTER APPENDIX D RDSI STORIES APPENDIX E RDSI PROJECT SUMMARY PUBLICATION APPENDIX F RDSI HISTORY APPENDIX G NODE S AUDITED FINANCE STATEMENTS ERSA INTERSECT NCI PAWSEY CENTRE QCIF (BNE) QCIF (TSV) VICNODE TPAC APPENDIX H NODE S PROGRESS APPENDIX I FIELDS OF RESEARCH STORED ON RDSI FUNDED NODES RDSI Project Final Report Page 3 of 136

4 Executive Summary Purpose This document is the RDSI Project Final report and includes the original project plan for completeness, describes the activities and overall Project progress for the period of July 2014 to June 2015 assessed against the approved Project Plan Updated 10 th July 2014 as well as the Project successes, lessons learnt, use and future trends. Impact of this Project The project has achieved all the objectives with which it was tasked and has expended the full $50m funding that it was provided together with an additional $3,778,449 of interest earned during the project. As a result of this investment from the Commonwealth, it is estimated that nodes have invested at least an additional $19,602,708 in the development of the RDSI data storage ecosystem, taking the total investment in this activity to $73,381,157. This represents a multiple of over the initial Commonwealth investment. As a consequence of this project, the research and education sector in Australia will ultimately have access to approximately 50 Petabytes (PB) of research data in over 100 PB of physical storage that was previously unavailable or difficult to access through the provision of data stored at RDSI Nodes. This storage is augmented by over 120 Gigabits/second of network capacity facilitating very fast transfers at primary nodes, together with software and tools for managing this infrastructure and a federated system for managing both the data and the authentication necessary to access it. The RDSI project was completed on time, on budget and met or exceeded all Performance Indicators (KPIs) described in section 9. There has been significant pro-active engagement with other Super Science Initiatives. These have involved joint projects and activities with the National Research Networks (NRN) and NeCTAR projects. There has also been significant joint work with both AARNet and the Australian Access Federation (AAF). This outreach from the project has been beneficial in enhancing engagement with project stakeholders and the eresearch community generally through joint activities, better alignment with RDSI nodes and the sharing of information. History A funding agreement between the Commonwealth represented by the then Department of Industry, Innovation, Science and, Research (superseded by the Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education (DIICCSRTE) and now the Department of Education and Training, henceforth referred to as the Department, and the University of Queensland (UQ) was executed on 23rd December 2010, with the Project Office established on 27th January A Deed of Variation was subsequently executed on 6th May 2013 extending the RDSI Project operational conclusion to 31st December 2014 and the contractual conclusion to 30th June 2015, with a requirement to submit the final report by 31st March The project was extended to strengthen the technical development of the data stores (aka Nodes) in a national network through the funding of specialised staff at each node to support the effective ingest of data into those stores and to provide access to collections to researchers. As a result the project developed improved associated access and data sharing capabilities to researchers, provided significantly increased capacity for research data holdings and enhanced the capability and capacity to RDSI Project Final Report Page 4 of 136

5 provide research data services to the sector through the Nodes. A total 53 participating organisations are involved in the operation and use of this national distributed set of facilities. Objectives The aim of the RDSI Project is that researchers will be able to use and manipulate significant collections of data that were previously either unavailable or difficult to access and that there will be a consistent means of accessing this data. This has been successfully achieved through the development of a national network of data stores, or nodes, where the content held in those stores is allocated capacity through a priority and merit process and where that content can be readily accessed, analysed and re-used in a coherently governed environment. This storage is augmented by over 120 Gigabits/second of network capacity facilitating very fast transfers at primary nodes, together with software and tools for managing this infrastructure and a federated system for managing both the data and the authentication necessary to access it. During the course of the project it became apparent that there was an expectation from many in the research community that RDSI would solve a number of problems that had existed in the eresearch sector for some time. This informed the development of many projects in the DaShNet Programme which have addressed these issues through activities such as the implementation of federated access to data across nodes (through the implementation of MediaFlux linked to the AAF) and effective high throughput data movement between participants (using Aspera software). Both of these are Australian firsts. Activities and Outcomes The project extension has demonstrated that it effectively accelerated the development of this national capability of research data storage and collaboration through the network of nodes by funding staff at the nodes to support and expedite deployment of their infrastructure, and accelerate data ingest. Currently over 39 PB of collections have been approved by the nodes Merit Allocation Committee with over 17 PB of data being made available to researchers. These figures are growing on a daily basis. The project continued to progress its objectives during the period in accordance with the relevant Annual Business Plan. During the period the RDSI project has achieved all milestones assigned to RDSI team and continues to support and monitor the nodes progress. The infrastructure, and the ecosystem that has been developed, places the sector in an excellent position to allow the Research Data Services (RDS) project to develop and deepen the use of this substantial shared resource through its work with research communities and eresearch Service Providers, such as the RDSI Nodes. As a broad overview of planned activities during the year , the project: Continued to manage the implementation of the ReDS acceleration activities with nodes. Continued to manage node contracts for distribution of data storage via the ReDS programme including the release of funds to nodes to enable storage infrastructure to be provisioned in advance of storage being allocated to data. Expanded the ReDS programme and continued to monitor data allocation and use. Continued to support nodes to purchase from the Vendor Panel. Maintained the ongoing Vendor Panel RFP and successfully transitioned it to CAUDIT. RDSI Project Final Report Page 5 of 136

6 Facilitated the entrance of more vendors to the Vendor Panel. Now a total of twenty four vendors and resellers. Continued to engage with vendors for solutions to node technical implementations. Continued and improved engagement with stakeholders. Finalised the DaSh programme requirements and activities with input from nodes. Continued to identify security related matters, developed and published the security guidelines and policies, ran a series of workshops to discuss security approaches with nodes. Managed the implementation of DaShNet with AARNet. RDSI Project Office continued to meet regularly with Node Steering Committee (Node SC). RDSI team continued to engage with stakeholders and had a strong presence at the eresearch Australasia Conference 2014 promoting awareness of the RDSI project to nodes, researchers and data custodians. Funded the Simple Markup Language (SAML) Provisioning for Independent Nodes (SPIN) Project delivered in collaboration between Intersect, QCIF and the AAF. Funded Aspera suite of tools license to nodes greatly improving nodes ability to transfer data between researchers, nodes and data custodians. Continued to monitor the Nodes implementation and publish node performance at the RDSI website. Managed, monitored and concluded RDSI subprojects. Transitioned RDSI project components. Provided Final project report to the Department. Detailed milestones achieved during July 2014 to June 2015 are described in section 3. Activities carried out outside the Annual Business plan with approval from the Department: The RDSI project went through an independent project review requested by the Department. This review was led by Professor Tom Cochrane and was conducted between February and May The series of key findings and recommendations contained in the review were provided to the Department and provided the project with an opportunity to reflect on progress and consider how future outcomes could be enhanced. The review recognised the valuable contribution that the project made to the eresearch environment. RDSI was invited to participate in the NCRIS eresearch Workshop on 5 December 2014, in Canberra. RDSI exhibited and showcased the outcomes of the project and the current use of the infrastructure built to store research data collections. The RDSI stand attracted interest from the community and government dignitaries. KPMG was engaged by the Department to conduct a review of each of the 27 National Collaborative Research Infrastructure Strategy (NCRIS) 2013 projects. The scope of their review was to undertake an objective assessment of performance by the Research Data Storage Infrastructure against their review framework which covered areas including governance, effectiveness, efficiency, integration, financial management and compliance and strategic policy alignment. Although the RDSI project was not funded by NCRIS 2013 the Project Office complied with KPMG s review as requested by the Department. We were informed by KPMG that they were satisfied with all aspects of the project. The project, in consultation with nodes and other eresearch projects, identified the need for funding the storage of large collections of research data which fell outside the scope of existing storage funding mechanisms within the project. Late March 2013 the RDSI advisory board and the Lead Agent have identified and approved a process The Board Allocation Process which addressed this requirement. It was supported by all nodes after extensive consultation and RDSI Project Final Report Page 6 of 136

7 included a workshop in which all stakeholders participated. This process is described in the RDSI website and it was approved by the Department on 31 July A call for Large Allocation Collections took place and were reviewed by the Resource Allocation Panel which made recommendations to the board. After considering these recommendations the board and the Department approved the large allocation collections proposals. Recipients of these funds were informed and contract variations provided for signature. All contracts have been signed and funds distributed. Further information is available in Appendix A. Expenditure on the project to date is within budget and it is anticipated that planned committed expenditure for the reminder of the project will also be within budget. The RDSI Advisory Board provided the Lead Agent with advice on the appropriate requests to fund based on approved principles and objectives defined by the project plan and approved business plans. Reports from nodes indicate that, at the end of February 2015, a total of 39 PB of Research Data had been approved by Merit Allocation Committees for storage and that 17 PB of this had been ingested into node stores and is available for research use. Figure 13 in 9.5, provides an excellent overall view of the data (defined by Fields of Research or FoRs) and this shows that there is data from all FoRs in the overall of the RDSI ecosystem. However, no two nodes are alike, and this is amply demonstrated, in Appendix I, by the highly diverse spread of data at each individual node. It is this diversity which contributes to the strength of the RDSI storage ecosystem. Finance The project has received $ 50,000,000 of investment funds from the Department; a total of $3,778,449 interest has been earned until 28 February 2015 on project funds and has been credited to the project. A total of $915,769 (including interest) provided as co-investment from the RDSI Nodes to the project, resulting in a total of $54,694,218 invested in building the researcher s data storage infrastructure. During the period from the start of the project until 28 February 2015, project funds expended on the DaSh, NoDE, VePa and ReDS programmes, and Project Office are $52,815,307. Net funds committed and allocated to 30 th June 2015 for the remainder of the project are $706,242. Further details regarding finances are described in section 5. The RDSI project anticipates it will have a total of $ of unspent funds by 30 June RDSI Project Final Report Page 7 of 136

8 1 Project Background This document is the Final Report for the Research Data Storage Infrastructure (RDSI) Project funded from the Education Investment Fund (EIF) under the Super Science Initiative. A funding agreement between the Commonwealth represented by the then Department of Industry, Innovation, Science and, Research (superseded by the Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education (DIICCSRTE) and now the Department of Education and Training, henceforth referred to as the Department, and the University of Queensland (UQ) was executed on 23rd December 2010, with the Project Office established on 27th January A Deed of Variation was subsequently executed on 6th May 2013 extending the RDSI Project operational conclusion to 31st December 2014 and the contractual conclusion to 30th June 2015, with a requirement to submit the final report by 31st March This document is the RDSI Project Final report and includes the original project plan in section 1 for completeness, describes the activities and overall Project progress for the period of July 2014 to June 2015 assessed against the approved Project Plan Updated 10 th July 2014 in section 2, as well as the overall Project successes, lessons learnt, use and future trends in the remaining sections. Section 1 is extracted from the Annual Business Plan and is included in this report for historical purposes and completeness. 1.1 Introduction The Super Science Initiative announced in the 2009 Budget included $50 million for Research Data Storage Infrastructure (RDSI) to enhance distributed data centre development to support nationally significant data assets. The Funding Agreement for the RDSI Project was signed with The University of Queensland (UQ) on 23 December The RDSI project is building a national network of distributed data stores, called nodes to enable research data to be readily accessed, analysed and re-used. The project will allow researchers and institutions to more effectively use, manage, share and preserve much larger amounts of research data and will support a national data environment at a scale that will enable new questions to be asked on topics and at scales not previously possible. This document is the fourth and final project report for the RDSI project covering the financial year July 2014 to June Note that the figures beyond 28 February are not available to RDSI until after the end of June The benefits from better-managed and more accessible research data are being sought everywhere across the research sector. At the same time, the acceleration in the generation of data is outstripping growth in data storage capacities. This is supported by the information provided to RDSI by the nodes in April 2014, estimating requests for funding research data storage at nodes would exceeded RDSI s capacity to fund such requests by a factor of 1.7. The RDSI Advisory Board provided the Lead Agent with advice on the appropriate requests to fund based on approved principles and objectives defined by the project plan and approved business plans. The RDSI Project therefore continues to represent a timely investment that is aimed at strengthening Australia s capabilities in data intensive research and data intensive research collaboration. In practical terms, this has been achieved through the nodes by providing significantly increased capacity for research data holdings; developing and improving associated access and data sharing capabilities; and enhancing the capability and capacity to provide research data services to the sector. The result is a data storage framework that will assist institutions and researchers to more effectively use, manage, share and conserve much larger holdings of research data. The expected benefits are to: RDSI Project Final Report Page 8 of 136

9 improve the availability of quality research data for sharing and re-use and, as a result, expand the scale and scope of problems that Australian researchers may seek to address; improve research efficiency; and reduce institutional data storage costs and enable more extensive collaboration. The infrastructure may also assist institutions to: sustain a quality of research in the digital age that includes the reproducibility of results; meet the storage requirements of key research activities undertaken at that institution; and comply with the research data provisions of Universities Australia s Australian Code for the Responsible Conduct of Research. Other related projects funded as part of the Super Science Initiative include investments in the improved management of data; a significant increase in the supply of high performance computing; enhancements to research tools and improved support for research applications; improved connectivity to the Australian Research and Education Network (AREN); and a range of developments specific to discipline or capability requirements. The intended role of the RDSI Project within this broader set of investments is to increase holdings of research data valued by the research community. Key components of this role were to support governance arrangements that underpinned ingest regimes, data quality and access policies and support access services required by those data collections. The implementation of the RDSI Project has enabled the provision of broader data sharing, movement and access services as required at a national level to support data intensive research and research collaboration that were difficult to access previously. This has been achieved with input from the nodes. Simultaneously, public infrastructure service providers developed new approaches to data infrastructure. The outcomes of the RDSI Project were positioned to assist the research sector to pursue the economies of scale available from these new approaches. 1.2 Objectives of the Project The aim of the RDSI Project is that researchers will be able to use and manipulate significant collections of data that were previously either unavailable or difficult to access and that there will be a consistent means of accessing this data. This will be achieved through the development of a national network of a limited number of data stores, or nodes, where the content held in those stores is allocated capacity through a priority and merit process and where that content can be readily accessed, analysed and re-used in a coherently governed environment. The Project will be realised through the creation and development of data storage infrastructure accessed through a common infrastructure layer and provided by agencies within the sector, and/or commercial providers. Specifically, the intention is to increase the sharing and re-use of research related data, which might be sourced from any sector, and the collaborative activity based on the sharing of data, and not merely to achieve improved data retention. A national data environment of the scale planned will also enable new questions to be asked on topics and at scales not previously possible. The data storage infrastructure created by the RDSI Project will be connected to the Australian Research and Education Network (AREN) by a high bandwidth connection, funded and constructed under the Super Science National Research Networks (NRN) Project. This will include dedicated high speed connections between major nodes. The resulting data environment is not intended to be exclusive to the use of RDSI Project stakeholders. Rather, the delivery of the RDSI Project is expected to contribute to the development of an open RDSI Project Final Report Page 9 of 136

10 infrastructure that institutions, disciplines and research facilities can expand or extend to meet other requirements and into which other sector interests can add capability over time. 1.3 Intended Outcomes of the Project The RDSI Project will deliver a national infrastructure able to hold significant collections of research data. That infrastructure will comprise: A small number of high capacity nodes where each node will contain multiple petabytes of storage configured to support the classes of access and retention appropriate to the research data held; A dedicated high speed inter-connection network with high bandwidth low latency connections that will support replication and data transfer between nodes; A common access infrastructure to provide a uniform experience for common researcher access to all the data held; and Appropriate specialist access infrastructure hosting specialist access tools appropriate to the disciplines closely related to these national collections. As a result of this infrastructure: Researchers will be able to use and manipulate significant collections of data that were previously either unavailable to them or difficult for them to access; Research outcomes will be improved as a result of improved access to this data; Researchers will be able to access the data in a consistent manner which will support a general interface as well as discipline specific access; Researchers will be able to use the consistent interface established by this project for access to data collections at participating institutions and other locations as well as data held at the nodes. In arriving at this infrastructure, the RDSI Project will leverage its purchase power to also develop a panel of vendors to help reduce storage costs across the Research and Education sector. It is important to note that all RDSI project outcomes were realised, with a high speed network between nodes and over 39PB of a single copy of approved data by the node s Merit Allocation Committees for ingest. This figure is almost eight times the amount of data expected to be stored when this project s outcomes were conceptualised by stakeholders in Scope The Project has undertaken activities across four key programme areas as follows: Programme Area Node Development (NoDe) Research Data Services (ReDS) Data Sharing Services (DaSh) Vendor Panel (VePa) Scope Assisted with the development of research data storage services and their supporting infrastructure. Accelerated the aggregation of data and the co-location of data with analysis capabilities. Provided a uniform data access, sharing and movement capability commensurate with the importance, volume and complexity of the large volumes of data the RDSI will hold. Established a vendor panel for use by RDSI Nodes and the sector. RDSI Project Final Report Page 10 of 136

11 From the original project plan, the scope of The Project was defined as follows: That via input from the nodes, the needs and expectations of institutions will be factored into overall strategy-setting, decision-making and management processes; That services enabled by the Project s infrastructure are tailored to meet the specific needs of different research contexts;that co-investors in the Project realise improved services for the researchers they support; That the performance of the infrastructure is measured against improvements to research outcomes; That Intellectual Property and other rights are managed in a way that permits the re-deployment of data from one jurisdiction to another, whenever possible; That Infrastructure is developed and installed for operation in enduring organisations, being the University of Queensland and the Project Sub-Contractors, as a basis for sustained operation of the nodes; That the scope of the Project will also account for the growing role of commercial providers, including public storage providers. It is likely that from time to time, commercial services will arise that can support or more effectively provide elements of the RDSI functionality; and That the Project will therefore ensure that investments are managed in a way that allows commercial services and software tools to be evaluated and adopted when appropriate over time. RDSI Project Final Report Page 11 of 136

12 2 Activities undertaken during Node Development (NoDe) Programme The NoDe programme has identified, strengthened and developed research data centres able to hold and process high data volumes. These research data centres are supported as six Primary nodes and two Additional nodes. The eight nodes involve a total of 53 organisations or collections of organisations. These comprise of 38 universities. Only two Australian universities have not joined a node, Charles Darwin University and Notre Dame University. Note that each of these had options to join a node. Location and Node type Australian Capital Territory Primary node South Australian Primary node New South Wales Primary node Western Australia Primary node Queensland Primary node Victorian Primary node Queensland Additional node Townsville Tasmanian Additional node Participant Organisations CSIRO, Australian National University, Bureau of Meteorology, Geoscience Australia eresearch SA, University of Adelaide, University of South Australia, Flinders University, SABRENet, State Government of South Australia Intersect members including all 10 Universities in NSW(Charles Sturt University, Macquarie University, Southern Cross University, University of New England, University of New South Wales, University of Newcastle, University of Sydney, University of Technology Sydney, University of Western Sydney, University of Wollongong) and Australian Catholic University as well as the University of Canberra in the ACT. CSIRO, Curtin University, Edith Cowan University, Murdoch University, The University of Western Australia. Central Queensland University (CQU), Griffith University (GU), James Cook University (JCU), Queensland University of Technology (QUT), The University Queensland (UQ), the University of Southern Queensland (USQ), University of the Sunshine Coast (USC), The Queensland Government, through three agencies: Department of Employment Economic Development and Innovation (DEEDI), Department of Environment and Resource Management (DERM) and the Department of Transport and Main Roads (DTMR), CSIRO, Ergon Energy, Terrestrial Ecosystem Research Network (TERN) and NICTA Eight Victorian Universities including Deakin University, Monash University, RMIT University, Swinburne University of Technology, University of Ballarat, University of Melbourne, La Trobe University, Victoria University and other Victorian research organisations. Central Queensland University (CQU), Griffith University (GU), James Cook University (JCU), Queensland University of Technology (QUT), The University Queensland (UQ), the University of Southern Queensland (USQ), University of the Sunshine Coast (USC) Tasmanian Partnership for Advanced Computing (TPAC), Australian Antarctic Division (AAD), Commonwealth Scientific and Industrial Research Organisation (CSIRO), Tasmanian Government (including Government Business Enterprises and State-Owned Companies) Legal entity signing agreements Australian National University University of Adelaide on behalf of eresearch South Australia Intersect CSIRO on behalf of Pawsey Centre Current Status Operational, researchers accessing collections Operational, researchers accessing collections Operational, researchers accessing collections Operational, researchers accessing collections Table 1- Illustrates the nodes, locations, participating organisations and progress status at the time of writing this report. The eight nodes collectively support disciplines from twenty two Fields of Research disciplines. This is an excellent outcome considering the complexity of developing the RDSI capability for researchers. At the time of writing this report 395 collections have been merit allocated by the nodes, based on the ReDS Merit Criteria. This is an increase of approximately 14% in allocations since the previous reported QCIF University of Melbourne QCIF University of Tasmania on behalf of TPAC Operational, researchers accessing collections Operational, researchers accessing collections Operational, researchers accessing collections Operational, researchers accessing collections RDSI Project Final Report Page 12 of 136

13 period. Information about these collections allocations is available at the RDSI website at The following activities were completed during period: As part of the NoDE programme the RDSI Project Office collates the monitoring information provided by nodes and delivers regular reports on node data collection performance which is published on the RDSI website Portal ( Additional reports include node availability information, up and down times, security breaches and the asset register which identifies the physical location of assets created and developed with funding from the NoDe, DaSh and ReDS programmes. The information collected by monitoring the nodes is also reported to the Department via this Report. Nodes collective reports regarding storage availability were close to 99.8% up time during this period, with some scheduled outages and very few unscheduled outages. In addition there were no reports of security breaches. This average is on par with performance of data centres in the higher education and research sector in Australia. Representatives of all nodes participated in regular group and individual meetings with RDSI team and actively partook in working groups to discuss and implement the network and infrastructure requirements in accordance with appropriate best security practices addressed under the various DaSh themes. Regular node Steering Committee (SC) meetings were an ongoing activity with meetings happening on a on an ad-hoc basis. As the nodes infrastructure developed further the need for the Technical Advisory Committee (TAC) diminished and evolved into the RDSI google node-tac mailing list. All nodes were able to use the Vendor Panel created under the VePa programme. A number of activities were subject to external influences, for example node availability or infrastructure and reporting mechanisms being in place to provide the required information. All milestones regarding this programme were completed by 31 December 2014 with the exception of node monitoring, ad-hoc SC meetings and node asset register reports to the Project Office. As these activities are either ad-hoc or ongoing and will continue until June 2015 to ensure nodes performance is reported to the community and the Department via the RDSI website. Milestones information are available in section 3. The information provided by the nodes with regards to the assets purchased using RDSI funds is reported in Appendix B - Nodes Asset Register. Node data collection ingest overview In May 2013 ersa, NCI, QCIF BNE, and TPAC nodes started to report that their Merit Allocation committees had approved data to be stored at their respective nodes. The remaining Nodes shortly followed suit, with all eight nodes being able to report both approved and ingested figures by December The allocation and ingest of collections continued to progress over 2014/2015. During this time RDSI provided two tranches of ReDS programme funds to advance storage procurement and fund two staff members at each node to support and expedite data ingest. In late 2014 the RDSI project also provided a third tranche of funds to support large or specialised collections that wouldn t otherwise be funded. Nodes that met ingest targets were also able to apply for ongoing ReDS programme storage funding in 2014 and early The graph below demonstrates the upwards trend of approved allocated collections and data ingest by the nodes. Collectively the RDSI funded nodes managed to ingest almost four times the data expected to need storage in 2010, with another 23 PB of approved collections to be ingested. This has a significant impact on the RDSI Project Final Report Page 13 of 136

14 research community supporting collaboration, sharing and re-use of data. Collectively the RDSI funded nodes store collections from the twenty two Fields of Research disciplines. TB Figure 1 The graph illustrates the combined approved and ingested collections Updated 25 March The graph is formulated from the end of month collective totals. Y axis figures are expressed in Terabytes (TB). A breakdown of approved and ingested data per node is available in Appendix H. Each node is undertaking a process of rapid ingestion of research data with the aim of having ingested all data relating the large board allocated collections by 30 June 2015 and the remainder by the end of RDSI Project Final Report Page 14 of 136

15 2.2 Research Data Services (ReDS) Programme ReDS programme identified research data holdings of lasting value and importance and contributed funding to their development at the most appropriate node. During the period the following ReDS programme activities were completed: Distributed funds for the ReDSII agreements and top-ups as per ABP Reviewed of Large Allocation Collection proposals Supported Resource Allocation Panel review of Large Allocation Collection proposals Implemented RDSI Advisory Board and Department decision on Large Allocation Collection funding and contractual arrangements Signed Large Allocation Collection (ReDSIII) contract variations with Intersect, NCI, Pawsey centre, QCIF and VicNode nodes and distributed funds accordingly Continued to fund ongoing ReDS programme to nodes ReDS node Merit Allocation Committees continued to allocate collections. Details on committees per node are available at The collection allocation process is described at The Merit Allocation Committee Checklist is available at Continued to fund the Data Infrastructure and Storage Infrastructure Specialists at eight nodes until 31 December These additional staff assisted with metadata creation, data normalisation, data input parsing, research and/or scientific data capture; and assisted in expediting the process of identifying and ingesting data for storage at the node. All milestones regarding this programme were completed by 31 December 2014 with all ReDS funds spent. 2.3 Data Sharing (DaSh) Programme The Data Sharing programme (DaSh) assessed the capability to support the sharing and re-use of research data by the nodes and, as a result, aimed at expanding the scale and scope of problems that Australian researchers may seek to address. In order to identify what high performance data sharing and data movement services were needed by the sector this programme ran substantial consultations with relevant research sector stakeholders, and with support from nodes evaluated existing services and products required by nodes. The approval from the Department allowing the RDSI project to fund two staff at each node from the ReDS programme has significantly facilitated the management and progress of the DaSh programme. The DaSh Programme encompassed two major elements: the DaSh Network (DaShNet), a high performance network between RDSI Nodes, together with edge devices provided high-speed data-movement capabilities, which was funded through the National Research Networks (NRN) programs led by UniSA, managed by RDSI, and implemented by AARNet. the DaSh Technical Architecture, developed the data sharing services that use DaShNet to provide services for the sector. Additional components required to support the DaSh Technical Architecture are illustrated in Figure 2. Detailed information about the DaSh programme is available at RDSI Project Final Report Page 15 of 136

16 Figure 2- DaSh Programme components these components were used to realise the DaSh Programme objectives. The DaSh Programme s complexity means that its components often overlap and have interdependencies, when this is the case it is demonstrated in sub-sections 2.3. All milestones under this programme have been completed or are scheduled to be finalised by June These have been implemented in stages in order for nodes, researchers and data custodians to take advantage of the infrastructure as it developed. The activities described in this section were pivotal to conclude the DaSh programme during the period. Milestones are described in section rex System The RDSI Researcher Identity Project (rex) is part of the Identity and Security theme. It developed the necessary infrastructure to enable researchers and those they collaborate with, to access RDSI Data Collections and Services. The project has partnered with the Australian Access Federation (AAF) to leverage the existing capabilities of the AAF and enhance the utility of the AAF to allow authentication and authorisation to non-web based resources. The rex Authentication and Authorisation phase one has been completed and delivered to nodes. The RDSI Advisory Board approved that the remaining funds from rex be used to further build on the identity and authentication activities at the nodes: Federated Identity and Services Programme Federated Access for Mediaflux (former LiveArc) These activities have been taken on by interested nodes and have improved the nodes ability to integrate their own IdPs and Service Providers (SP) with the AAF. RDSI Project Final Report Page 16 of 136

17 Since August, further work with nodes (Intersect and QCIF) who are using the test platform to integrate the Federated Identity and Authentication component of the DaSh Programme with Cloud Services, has identified a possible extension of the programme to integrate identities in the Australian Access Federation (AAF) with cloud services using the Simple Mark-up Language (SAML2) protocol. A project was established to develop the software for this integration such that it will be useable by any node and potentially any university. The project has been named the SAML Provisioning for Independent Nodes (SPIN) Project. The RDSI project team, Intersect, QCIF and the AAF have worked together to design the SPIN project. Intersect is leading this project and conclusion will occur by the end of March Test Platform The test platform theme was created to support meaningful evaluation of RDSI funded infrastructure and applications in a realistic environment spanning over RDSI funded nodes. To support this theme RDSI has established a test platform in conjunction with two nodes and this environment has being used to test RDSI-related components, change management and integration testing. The successful implementation of RDSI infrastructure and applications are dependent on the ability to undertake meaningful evaluation and testing. The test platform principally uses cloud based resources. The establishment of a test platform supported the evaluation of infrastructure and applications in a realistic environment. The Test Platform was used by nodes and currently undertaking the final evaluations including public cloud access authenticating using the AAF through the SPIN project Public Cloud Access This component was part of the Test Platform theme. It tested and integrated node access with Amazon Web Services (AWS) as the largest public cloud and could in future be extended to other clouds Workflow The workflow theme was created to support nodes with their reporting requirements and the RDSI project's reporting obligations to the Commonwealth. The workflow automation called Allocation Request Management System (ARMS) phase one has concluded and is being used by interested nodes. Phase two was delivered in March 2015 and will continue to be supported by QCIF until December DaShNet The DaShNet theme was developed to support the connection of nodes by a very fast network that is part of the Australian Academic Research and Education Network (AARNet). The project to establish these connections is also called DaShNet and was funded by the National Research Networks (NRN) project, managed by RDSI and implemented by Australia s Academic and Research Network (AARNet). This theme includes the Networks, Data Mover and Public Cloud interconnection components of the DaSh Technical Architecture. The data mover component has been incorporated into the DaShNet architecture, building on the high-performance Data Transfer Nodes (DTN) hardware has been configured at each node. It uses high-performance DTNs at each site running the Aspera software to provide an extremely highperformance and flexible data movement service. The Aspera software was deployed and tested by nodes and is being actively used. At the time of writing this report nodes have requested additional products from the Aspera product suite as RDSI Project Final Report Page 17 of 136

18 these improve the node performance and access to researchers. RDSI has procured additional licenses of the Aspera suite and has distributed to nodes as per their request. At the time of writing this report DashNet has progressed extremely well with all nodes having redundant fibre paths to AARNet installed with the exception of Tasmania that has a single fibre path which will be upgraded and finalised by AARNet in the coming weeks. The ScienceDMZ 1 hardware and associated software were delivered and installed at the nodes. As part of AARNet s development of DaShNet nodes are required to sign a service agreement with AARNet in order to continue DaShNet s development to nodes. By the end of February 2015 all nodes had signed the agreement with the exception of the University of Melbourne (one of the VicNode s operational data centres). Network connection to a Public Cloud Interconnection point is in place, provided by AARNet connecting the Amazon Web Services site in Sydney. Nodes are able to acquire a direct-connect link from their data centres to AWS at up to 10Gb/s. 2 Figure 3- DaShNet Components This diagram illustrates the DaShNet components. Note that this architecture applies to RDSI Primary Nodes which are directly connected to the AARNet backbone via new dedicated links; RDSI Additional Nodes access the AARNet NAP's via existing shared links currently operating at 10Gb/s or less. 1 Science DMZ (S-DMZ) is a portion of an institution's network, built at or near the network perimeter, that is designed such that the equipment, configuration, and security policies are optimized for high-performance data-intensive applications rather than for general-purpose business systems or "enterprise" computing: 2 Virtual Private Network (VPN); Data Transfer Nodes (DTN's); Border Edge switches (BES); Data Transfer Node (DTN); Network Access Points (NAP); National Research and Education Network (NREN) is a specialised internet service provider dedicated to supporting the needs of the research and education communities within a country. RDSI Project Final Report Page 18 of 136

19 KEY Not Started In Progress Complete TSV BNE NAP (Backbone) CPE (Access) Network Service Up S-DMZ Installed In Production Use Table 2 DaShNet Implementation Progress - this table provides an overview of the current deployment progress of DaShNet at each RDSI Node Security This project is part of the Identity and Security theme components of the DaSh Technical Architecture. The security project delivered a framework for security policy development for use by all RDSI funded nodes and the Information Security Model And Guidelines For RDSI-Funded Data Storage nodes and Institutions, both available at In addition the RDSI team directly assisted and advised RDSI funded nodes with possible enhancements or with the development of their security policy. This project was in-line with best security practices. A series of workshops took place at each node site to solicit feedback from nodes on the state of their security preparedness and policy development maturity. When appropriate this feedback has been incorporated into the Information Security Model And Guidelines For RDSI-Funded Data Storage nodes and Institutions document to disseminate knowledge and raise awareness with nodes and institutions. Detailed schedule regarding these workshops along with slides is available at All nodes participated in the workshops Data Access The Data access theme was created to help nodes to develop an overarching strategy or framework for coordination, as well as funding some implementation projects where data access commonality is important. The DaSh programme components were further improved to support the nodes and researchers demands. These modifications were supported by the nodes and RDSI Advisory Board. At the board meeting on 11th February 2014, it was agreed that RDSI would convene a workshop with nodes to develop the remaining options for the implementation of systems to support other data access methods, such as Programmatic Access Methods (OPeNDAP 3 ), file synchronisation or Dropbox like capability and extended licences for LiveArc. The board also agreed to an updated DaSh Budget. On the 17th February 2014 RDSI 3 OPeNDAP, an acronym for "Open-source Project for a Network Data Access Protocol", is a data transport architecture and protocol widely used by earth scientists. RDSI Project Final Report Page 19 of 136

20 ran the DaSh and ReDS Workshop with node proprietors and it was agreed that the nodes would undertake the following activities: The Programmatic Data Access Service project managed by NCI is working on the production version of the authorisation controls using their Lightweight Directory Access Protocol (LDAP) and ensuring that it is fit for the client-tools and across OPenDAP and Hyrax 4 with controls at the Apache layer. The last phase which would distribute and provide support of this service to all nodes will not be concluded in time and funds will be returned to the Lead agent. The Dropbox like Facility managed by QCIF produced a thorough report offering options for developing such facility. Its major recommendation was to use the Aspera dropbox like facility. DaShNet supports a friction-free path between the nodes storage and the wider area network paths to maximise data movement performance. To achieve maximum performance out of these links, the DaShNet project also licensed a set of tools from Aspera Software (now an IBM company) in the US. The Aspera tools provide a (mostly) UDP-based high-speed data transfer ecosystem between node storage and end-users, which is capable of reaching well over 95% of the available bandwidth capacity along any network path. That purchase provides each node with a baseline server from the Aspera suite together with 50,000 end-user licenses for the clients, with perpetual licenses for both the baseline server and the end-user clients. These licenses were purchased by AARNet on behalf of the nodes, and these assets are in the process of being transferred to each node. Subsequent to this investment, a number of nodes assessed and requested licenses for the wider suite of Aspera tools to provide additional functionality, greater integration with researcher workflows, as well as the software kit development (SDK) to allow developers to leverage the functionality of the Aspera tools and protocols within their own tools. In December 2014 RDSI (UQ), after approval from the board, was able to invest some of its remaining project funds to acquire licenses for almost the entire Aspera product suite for 2750 seats, for three years, shared across all nodes. These licenses were purchased and transferred to the nodes. This is the first time Aspera/IBM have sold such an all-tools license, in any sector, globally. In February 2015 the RDSI Project Office met with the RDSI Nodes to seek input on how the nodes would like to divide and use the Aspera product suite. An allocation distribution process was agreed and followed up by the RDSI Project Office with Intersect, VicNode, TPAC and Pawsey Centre already requesting and receiving their licenses. Distribution of remaining licenses will be agreed with nodes as well as a license holder on behalf of the nodes beyond June A three month scoping exercise was carried out with extensive nodes consultation. This dimensioning activity was extremely successful and led to all nodes agreeing to use a common infrastructure (Mediaflux), in addition to identifying customisation requirements. Mediaflux (formerly known as LiveArc Mediaflux) has been deployed at all RDSI Nodes as a collection management system. The central services contract has delivered many of the node requested enhancements to the Mediaflux software with remaining enhancements due before the end of June Some Mediaflux software training sessions have been delayed until 2015 to enable more effective learning to be undertaken. A project change request has been agreed by the Mediaflux Steering committee to enable the Aspera/Mediaflux integration to be completed in February Hyrax is the OPenDAP data server. RDSI Project Final Report Page 20 of 136

21 In order to assist researchers and nodes to understand the role Mediaflux plays in data access a short video was developed by SGI and RDSI and it is available at All milestones under this programme have been completed or were ongoing activities undertaken by external parties, i.e. nodes, AAF and AARNet. Milestones information are available in section 3. RDSI Project Final Report Page 21 of 136

22 2.4 Vendor Panel (VePa) Programme The Vendor Panel programme implemented in partnership with the Council of Australian University Directors of Information Technology (CAUDIT) was created to facilitate the procurement of storage related infrastructure, software and services. The purposes for the programme were twofold. Firstly to allow nodes, universities and other authorised users to avoid lengthy tendering processes by using an appropriately constructed panel and secondly to support volume pricing across nodes and the wider Higher Education and Research Sector. Late last year Xostor left the panel while the Australian Access Federation (AAF) joined. Aspera joined as part of IBM. The Vendor Panel has been handed over to CAUDIT to provide continuity of this service. The current Vendor Panel comprises of twenty two vendors as well as reseller partners. RDSI/CAUDIT Vendor Panel Table 3- Vendor Panel Participants This table illustrates the vendors that are part of the CAUDIT/RDSI vendor panel. A wide range of infrastructure and services are included on the panel and each Vendor has agreed to a Commonwealth Government Information Technology and Communications (GITC) compliant model contract (or its equivalent) which significantly streamlined the use of the panel across the university and research sector. Nodes and CAUDIT members are procuring from the Vendor Panel. All milestones under this programme have been completed. Milestones information are available in section 3. RDSI Project Final Report Page 22 of 136

23 2.5 Sector Engagement As part of the outreach and communications activities the project has significantly increased its exchanges with stakeholders as well as its footprint in social media. This approach has been successful regarding supporting researchers, data custodians and nodes. For a short list of RDSI being mentioned in the media please refer to section A number of activities were accomplished during this reporting period and are described herein RDSI website The new website went live in October The site has provided stakeholders and interested parties with a rich repository for project and node information. The objective of the site was to provide transparency and rich information to nodes, researchers, data custodians and the higher education in general, this objective has been fulfilled. Website statistics Google Analytics 5 November 2013 March 2015 Figure 4 Google Analytics RDSI Website This diagram illustrates the use of the RDSI website from November 2013 to March Averages per month Sessions 1027 Users 538 Number of pages per session 3.19 Page Views 3273 Natural session spikes are experienced immediately after the distribution of the RDSI monthly newsletter, which are released on the first business day of each month. The site has been accessed consistently from countries all over the world. Top visiting counties include Australia, United States, India, United Kingdom and New Zealand. 5 Google Analytics is a web analytics service offered by Google that tracks and reports website traffic. RDSI Project Final Report Page 23 of 136

24 Figure 5 RDSI website access from other countries This map represents countries currently accessing the RDSI website. Visitors are illustrated by a coloured sliding scale light blue representing small volumes of access and darker blue representing higher volumes of access DaShboard Portal The DaShboard contains information and reports regarding node performance, collections and the former Security Feeds. The node status page ( has been a great achievement, providing visitors with high level figures relating to the success of the nodes ingestion. In addition the 395 collections stored at RDSI funded nodes page ( provides comprehensive information at a glance. It provides detailed information about the collections stored at each node as well as graphical representation of Fields of Research disciplines stored at RDSI funded nodes collectively and individually in the form of pie charts ( These pie charts illustrate an aggregated representation of approved data broken down by the twenty two Fields of Research. The Fields of Research are in accordance with the Australian Research Council. The charts are also interactive and provide the ability to a viewer to isolate a FoR segment for visual effect. A breakdown of FoRs per node is illustrated in section 9.5 for completeness. The Dashboard Portal will continue to be updated until June 2015 when it will be supported by the Research Data Services (RDS) project. The Security Feed disseminated security issues/bulletins of relevance to the management and implementation of RDSI funded storage and has been harvested from various trusted news feeds and redistributed via twitter to node subscribers. The Security Feed concluded 31 December 2014 in accordance with the ABP RDSI story From July 2013, the RDSI Project began collecting use cases on how research groups across Australia use collections stored at RDSI funded nodes, and why RDSI funded storage is important to their research. From RDSI Project Final Report Page 24 of 136

25 high energy physics to the humanities, from climate to cancer research, researchers are discovering common needs around research data. They all need to preserve and store their data, access and share it with collaborators, bring disparate collections together to be analysed by common tools, and in many cases, reuse data that was collected by someone else or for a different purpose. In all of these cases, the data collections demand greater resources than were previously available to the researchers. RDSI funded storage is an integral part of the infrastructure needs of these research groups and a key enabler for their research. RDSI has also enabled many research groups to make data available to the wider research community where it was inaccessible before. In order to gain a sense of the impact RDSI funded infrastructure has had for research in Australia, the Project interviewed a number of researchers using the storage and the collections. The result is a set of case studies published on the RDSI website and in the RDSI monthly enewsletters. Several common themes have emerged from these case studies, providing an indication of the research benefits the infrastructure is helping to enable. These include: Generating new research data collections that could not have existed without such storage infrastructure. Examples include the Healthy Human Genome Collection--a reference collection of 4000 human genomes being sequenced by the Garvan Institute of Medical Research--and the weather@home Australia and New Zealand collection, used to understand more about severe weather events. Access to research data collections that were previously inaccessible. Examples include the ecosystem science datasets used by the TERN emast facility to produce carbon dioxide models; twenty years of film footage of the complex behaviours of ocean and terrestrial creatures contained in the Tropical Biological Video Library; and thirty years of research observations collected from the air by Airborne Research Australia. Opening the door to use large research datasets in a high performance computing environment or with the NeCTAR Research Cloud. Examples include the Geoscience Australia Data Cube, and the Drosophila and Aedes aegypti genome collections used to understand disease vectors. Providing key components for end-to-end research data management. Examples include the Australian Synchrotron and other instrument data used with NeCTAR's Characterisation Virtual Laboratory. Facilitating data reuse and new research collaborations. Examples include the CliMAS and Wallace Initiative collections that allow researchers to explore how climate change will affect animal populations, and the set of collections that comprise the Human Communication Science Corpora used with NeCTAR's Alveo Virtual Laboratory. To date we have published over 20 written and video recorded stories. Further information is available in Appendix D RDSI animated videos In order to simplify the complexity of the RDSI project and provide an overall understanding of its outcomes and benefits to the research sector in Australia two videos were developed. Released October 2013, the video was a high level view of the project objectives and outcomes. The video was promoted in outgoing communications, social media and the website. The target was researchers, data custodians and general public that had no previous knowledge of the project and its aims. This video was used at the RDSI booth at the NCRIS eresearch workshop and eresearch conferences in 2014 and at the Universities Australia 2015 conference. The video is available at RDSI Project Final Report Page 25 of 136

26 Released October 2014, as a joint effort between SGI and RDSI project, the Mediaflux (also known as LiveArc) video was a high level view of the benefits Mediaflux provides to RDSI funded nodes and researchers. The video explained how RDSI Nodes using Mediaflux assist researchers to find, use and query data, by simplifying how it s managed and making it easier to increase research publication rates, report on research and securely share data with colleagues. The video is available at RDSI monthly enewsletter Debuted December 2013, the enewsletter has been an opportunity to keep target audiences and distribution lists informed of node developments, use cases and overall project progress. The enewsletter has provided nodes with an additional outlet for raising awareness of their services and general promotion. The enewsletters publication will conclude in June Social media and news The RDIS project used on a regular basis social media to provide project updates, node updates, information relevant to researchers, data custodians and node operators, including vendors webinars. The tools used include Facebook, Twitter, Linkedin and YouTube. The RDSI project has been mentioned in a number of media releases including but not limited to: Arcitecta to supply Mediaflux platform for RDSI Data management platform to power collaborative data network Arcitecta's data management platforms selected to power nationwide shared research data project - RDSI RDSI Initiative Uses Aspera for Science DMZ Data Mover RDSI scheme - News, Features, and Slideshows AARNet deploys high-speed data connections for RDSI Two years of money does not compute: top scientists Another node announced for research big data project Universities mull public cloud for 100PB data network Two more partners announced for research big data project First partners announced for research 'big data' project UQ leads $50m national data storage project Print material RDSI produced printed materials on a need basis and released a series of materials for conferences including NCRIS eresearch workshop, eresearch, Theta and QuestNet conferences. The RDSI Project Report 2014 was developed to summarise various aspects of the project including outcomes and information about individual nodes. It is also available at Working groups and committees A number of working groups and steering committees were created to support the RDSI project and especially the DaSh programme and its sub-projects. These groups met on a regular basis and included the DaShNet Steering Committee, Identity and Authorisation Project Steering Committee, LiveArc Steering Committee, LiveArc Working Groups, Clearinghouse Working group, ARMS 2 Working Group, ARMS 2 Steering Committee. Except for the DaShNet Steering Committee the other committees have concluded 31 December RDSI Project Final Report Page 26 of 136

27 The RDSI funded staff at nodes met fortnightly with RDSI staff since January Discussions included main activities, challenges, wins and experience sharing across the sector. These meetings assisted in informally developing a community of technical and data management experts fostering sharing of knowledge and encouraging collaboration across nodes. The Node Clearinghouse group established in August 2013 to identify and provide solutions to scenarios requiring across node coordination. Scenarios requiring clearinghouse resolution were identified, and solutions proposed. Node head endorsement of the activities was achieved. Solutions implemented included creation of a contacts directory to assist in across node relationships and liaisons, and the automation of some workflows through software developments. In addition, end user processes for access to RDSI funded services at the nodes were aligned, to ensure a unified single experience across the sector. The Mediaflux steering committee was established to review and prioritise Mediaflux customisation scope of work, assign technical resources to acceptance and sign off of development activities. This group met fortnightly. The Dimensioning technical working group was established to determine suitability of Mediaflux as a collection management middleware to be employed by the nodes and to define requirements for enhancement and customisation scope of work required to tailor Mediaflux to node processes. The Mediaflux Contract Management Committee was established to ensure scope of enhancement work is delivered according to the agreed contract between QCIF and SGI. This group met weekly, led by QCIF. The ARMS Steering Committee was established to advise the development team on the priorities and requirements for development of ARMS 2, review progress and provide feedback on acceptance of deliverables. This group met monthly. In addition RDSI met regularly with ANDS colleagues to share experience, discuss approaches and identify areas for greater collaboration. RDSI participates on the AeRO regular meetings and met with Nectar on a need basis General activities summary Additional generic activities carried out by the project are listed here: Participation and organisation of Advisory Board meetings which met regularly. Participation and organisation of node Steering Committee meetings which met ad-hoc during this period. Participation on RDSI TAC google group on a regular basis and discussed and shared ideas as well as challenges. Participated at Node Heads and Nectar Platforms Steering Committee meetings. Participated in regular DaSh Sub project working groups and committee meetings. Facilitated Mediaflux System Administrator training for nodes organised by RDSI. Security framework workshops with individual nodes organised and ran by RDSI. RDSI ran a workshop with nodes to develop the remaining options for the implementation of systems to support additional data access methods. RDSI ran a workshop with nodes to discuss allocation of large collections. RDSI met with nodes to discuss Aspera licenses allocation. Participated at eresearch Australasia conference 2014 and the NCRIS eresearch workshop Continue to maintain and monitor each RDSI programme aliases to facilitate access to the RDSI project, programmes and node reports. Continue to maintain and update a public mailing list to provide information about the project and upcoming events. Subscription to this list can be done via the website. Maintains the RDSI twitter account to provide information about the project and upcoming events. RDSI Project Final Report Page 27 of 136

28 Continue to update the RDSI website regularly as information pertinent to RDSI project becomes available. Issues monthly RDSI enewsletters. These enewsletters are expected to evolve and continue to be supported by the RDS project. Continue to monitor and assist with node progress. 2.6 Governance The items listed in this section refer only to governance used during the project period being reported. Project Advisory Board All Project Advisory Board members were appointed for their outstanding abilities to guide the Project. They are expert individuals able to take a broad, national, collaborative perspective and includes individuals with previous experience in one or more of the following areas and collectively cover all of the areas: corporate governance; financial/business management; research and development activities; Information and Communications Technologies; international and national activities providing eresearch infrastructure and services; computational sciences; and data intensive research. RDSI Advisory board members from July 2014 to December 2014 included: Mr Brian Anker (Independent Chair) Dr Rhys Francis (ercp) Professor Anton Middelberg (The University of Queensland) Professor Doug McEachern Mr Peter Nikoletatos (Latrobe University) Mr John Shipp Professor Liz Sonenberg (The University of Melbourne) From January 2015 Dr Rhys Francis (ercp) and Professor Doug McEachern stood down from the Board. Independent Vendor Panel Evaluation Committee The Independent Vendor Panel Evaluation Committee was created to support the VePa programme by recommending a Vendor Panel for RDSI. The Independent Vendor Panel Evaluation Committee members included: Dr Paul Campbell, Executive Office of the ICT Industry Workgroup (Chair); appointed by RDSI Chris Bridge, Director of Information Technology Services, QUT; nominated by CAUDIT Peter Nikoletatos, former Director, Information Services and Chief Information Officer, ANU; nominated by RDSI Andrew Yeoh, IT Sourcing & Sustainability Manager, ITS, The University of Melbourne; nominated by CAUDIT Rick Van Haeften, Information Technology Directorate, ACU; nominated by RDSI Resource Allocation Panel (RAP) The Resource Allocation Panel (RAP) was created to support the ReDS programme by recommending to the board the assessment criteria for allocations, which proposals should be supported, to recommend funding allocations to data services and their implementation at Primary and/or Additional nodes, and to RDSI Project Final Report Page 28 of 136

29 advise the RDSI Advisory board on the scope of funding appropriate to research data holdings of lasting value and importance. Resource Allocation Panel (RAP) members included: Chair: John Shipp Professor Doug McEachern Activities undertaken by these committees are described under the RDSI programmes they support. The RDSI Advisory Board met regularly and continued to: Provide strategic guidance to RDSI project Monitor the overall strategic direction of the Project Receive and approve Annual Business Plans Provide other advice and input as required The RDSI Project Advisory board met five times during the period of this report. Table 4 illustrates the dates and main agenda items discussed during these meetings: Date 4 July September October December March 2015 RDSI project update Discuss Large Allocation Collections Main issues discussed/addressed RDSI project update Discussion eresearch conference participation Approved Annual Project Report ( ) RDSI project update Discussion response to RDSI project review Discuss communications and outcome dissemination RDSI project update and recommendations Discuss RDSI project wrap-up Node reconciliation activities Discuss RDSI project report analysis to be published in June RDSI project update Provide input to the RDSI Final Project Report Table 4 RDSI Board meetings this table illustrates the dates and main agenda items discussed during these meetings. RDSI Project Final Report Page 29 of 136

30 2.7 Risk Mitigation The project risks and mitigation strategies identified in Annual Business Plan have been updated and are described in Appendix C. The RDSI Project Office carries out regular reviews of project risks and raises any concerns with the Advisory board, as a result of such assessments, three major risks were acknowledged during this period: 1) It was identified that the demand for data storage exceeded the funds available from the RDSI project. In order to mitigate this risk the RDSI Advisory Board prioritised the expenditure of ReDS programme funds and acknowledged that it was a competitive programme. This culminated with the large allocation collection proposal approved by the Department. 2) After extensive consultation with nodes it was identified that a strategy needed to be developed with options for the implementation of systems to support other data access methods. The DaSh programme had sufficient funds to support the proposals and these were approved by the board. Further details are available in section ) In consultation with nodes it was identified that the rex system while useful could not provide for the granularity for authentication and authorisation required by the nodes. To minimise this risk the RDSI team proposed alternative approaches to overcome this challenge with nodes and a proposal was formulated to the Advisory board and approved. Details of the approach are described in section The RDSI Project Office will continue to actively monitor risks across the implementation of the project until the project end date rex Authentication and Authorisation Phase two After delivery of the most recently developed functionality from rex, and the adoption of Mediaflux by all nodes, it has become apparent that superior outcomes for nodes could be achieved through a change of emphasis for the remainder of this project by concentrating more on the expansion of Security Access Markup Language (SAML) based authentication rather than further extending directory (LDAP) based authentication. This change of approach supported by the nodes involves the integration of Mediaflux with SAML to allow it to interoperate with the Australian Access Federation (AAF) and funding further AAF integration at each node. This change had no effect on the overall budget for the rex project, which remained the same and was entirely in line with its project objectives. To achieve these objectives the RDSI project funded nodes to participate in any of the following programmes: Federated Identity and Services Programme Federated Access for LiveArc All nodes signed up for the above programmes and provided their proposals to the RDSI project for funding. RDSI Project Final Report Page 30 of 136

31 3 Milestones from July 2014 to June 2015 The project milestones from 1 July 2014 to 28 February 2015 were achieved and current status in the case of ongoing activities are described herein. Ongoing activities include node monitoring and reporting which were initially intended to conclude in December The allocation of the large collections to nodes in late 2014, coupled with nodes being contractually required to ingest data by June 2015 created a unique situation in which the Project Office is contractually required to continue to monitor and report on nodes ingest, and to continue to support the RDSI funded nodes until 30 June RDSI Project Final Report Page 31 of 136

32 Table 5- Milestones Milestones with an End date of December 2014 for Ongoing & on track activities relates to the large allocation collection monitoring, reporting and supporting obligations from nodes and RDSI. These End dates are June As the project progressed some milestones evolved to accommodate the nodes requirements with agreement from the RDSI Advisory Board and the Department. Note that some activities in Table 5 and sub-projects are carried out by nodes not the RDSI project team. The RDSI project infrastructure, and the environment that has been developed, now places the sector in an excellent position to transition to the Research Data Services (RDS) project. RDSI has made a substantial contribution to the eresearch environment which is fundamentally changing the way research is undertaken across institutions and organisations. RDSI Project Final Report Page 32 of 136

33 4 PROJECT RESOURCES The project resources described in this section represent all resources used in the project. Noting that from January to June 2015 only the Project Office and a part-time Communications role are required to fulfil the remaining obligations related to monitoring and reporting on nodes progress, finance, update of RDSI Dashboard and website, outreach and publishing of RDSI materials including the enewsletters. For completeness all project roles are described in this section and concluded by 31 December Project Office Co-investment In-kind co-investment to support the operations of the Project Office is provided by the University of Queensland. In addition, each Primary node was required to provide an annual cash co-investment of $50,000 to support the RDSI Project operational costs. The RDSI Project received $ 300,000 from six contracted Primary nodes. In addition RDSI from time to time requires expert matter support. During this reporting period RDSI required the roles of Solutions Specialist, Research Case Study expert and Vendor Engagement Specialist. RDSI Advisory Board and Resource Allocation Committees The members of the RDSI Advisory Board and Resource Allocation provide their invaluable expert advice on a voluntary basis, with the exception of the Board Chair which is remunerated by the project. 4.2 Description of Roles The following Roles are required to deliver the RDSI Project objectives. Project Director The role of the Project Director was to report to the University of Queensland delegate nominated in the Funding Agreement, support the RDSI Advisory Board in its role, undertake activities as required for the proper performance of the Project, receive advice from the RDSI Advisory Board with regard to the overall strategic directions of the Project and management and performance of the infrastructure in accordance with the project Funding Agreement and the longer-term national strategic goals outlined in the RDSI Annual Business Plan (ABP), develop, propose and oversee activities that implemented the objectives of the Project; and manage such other staff as may be appointed for purposes of the Project by the University of Queensland. Project Manager The role of the Project Manager was to develop, manage, monitor and report against the detailed RDSI Annual Business Plan and the RDSI project budget. This report details activities, resources and their allocation, risks and strategies of sub-contracted organisations where this was necessary to support the overall ABP. The role developed and managed reporting mechanisms for reporting to the RDSI Advisory Board, the Australian Government and interested parties. This role also provided significant support for the development of the vendor panels. Executive Officer The role of the Executive Officer was to provide administrative support to the Project Director, to staff within the RDSI project and to the RDSI Advisory Board and committees. This role managed the administrative functions of the RDSI office including the co-ordination of travel and events. Legal Resource Legal resources were required to develop sub-contracts and potential vendor contracts as well as providing legal advice to the RDSI project. RDSI Project Final Report Page 33 of 136

34 The Project Director, Project Manager, Executive Officer and the University Legal Resource are based at The University of Queensland. Communications Officer The communications officer engaged in 2012 continues to provide communications and outreach engagement activities and is funded from co-investment through subscriptions. This role is now working closely with RDS to assist in transition and the incumbent will move to the RDS project. Operations Resource Operations resources were required to support RDSI s consistent node monitoring methodology. This resource provided a coordinated approach to deliver information to update the RDSI portal information and support node performance transparency. In addition it assisted with the publication of data collection performance and use cases from researchers using RDSI funded storage. This role worked closely with the Project Director and Project Manager. Expenses related to this resource were funded from co-investment through subscriptions. Node Development Manager This role developed the first Call for Nodes proposal and undertook analysis of the responses leading to Primary node and any Additional node recommendations. The role also monitored implementation of the Primary nodes and Additional nodes and ongoing compliance with the conditions of sub-contracts. From April to June 2013 Dr Rhys Francis acted on this role to support the coordination of activities within and between nodes and stakeholders. In July 2013 Mr Richard Northam joined in this role. As the nodes progressed this role evolved to building and maintaining strong relationships with node operators, understanding the storage architecture at each node, establishing monitoring and reporting of RDSI services from all nodes, monitoring the implementation of nodes and ongoing compliance with the conditions of sub-contracts, contributing to the development of potential pricing regimes, facilitating collaboration between nodes and liaising with stakeholders throughout the research community. ReDS Programme Manager This role developed appropriate ReDS programme characteristics which led to the ReDS II interim and ReDS III proposals. The role developed and ran the subsequent rounds of ongoing ReDS funds distribution, undertook analysis of the responses leading to recommendations. The role monitored implementation and ongoing compliance. The ReDS Programme Manager also supported the arrangements entered between research communities, stewards of collections, institutions and node operators in consultation with key stakeholders. ReDS Research Data Manager This role identified research data holdings of lasting value and importance by consulting with research groups, institutions, government research organisations and other stakeholders in order to detect and document their data requirements. This role maintained documentation of these requirements and contributed to the development and ongoing refinement of the merit allocation process using the documentation collected and created. This role also tracked the emerging standards for digital repositories insofar as they would be appropriate to incorporate into the RDSI project, and managed the activities necessary to ensure alignment between the delivery of the RDSI Nodes and the use of ANDS infrastructure. This role worked closely with the ReDS Programme Manager. DaSh Technical Architect This role researched and developed the DaSh Technical Architecture after evaluating existing technical architectures. It identified DaSh characteristics relevant to nodes and recognised existing services and infrastructure that were relevant to integrate with nodes infrastructure and data access. Security Policy Manager RDSI Project Final Report Page 34 of 136

35 Due to the complexity of the node architecture, the networking requirements and the variety of data stored in the nodes this role identified potential security threats to define security policies across the nodes via consultation with each node and developing security framework guidelines and policies. This role also assisted with the integration and security of the Federated Authorisation and Identity components of the DaSh programme and provided expert security policy advice to the nodes as it relates to RDSI funded data storage. Storage Technical Architect As the project variance to accelerate the aggregation of data, and the co-location of data with analysis capabilities at each node was implemented this role became obsolete. Nodes required a different approach to accelerate their data ingest and new strategies to support this implementation were undertaken with support from the nodes and RDSI funding staff at the nodes to fulfil its objectives. Network Program Manager A Network Program Manager was required to assist in designing and managing the DaShNet theme. This expert role served as liaison between RDSI, nodes and subcontractors, developed the requirements for the DaShNet project and programme managed DaShNet. This role was funded by the DaSh programme. RDSI Project Final Report Page 35 of 136

36 5 RDSI project finance summary The total Commonwealth funding for the RDSI project was $50 million over three years: $24 million in , $13 million in and $13 million in As the funds were not all expended during the three year the project extension has provided the nodes with additional time to further develop their infrastructure, identify collections of national significance through their Merit Allocation committees, ingest and support these research collections. A request sent to the Department on the 10 October 2013 to extend the project until the end of 2015 to ensure nodes could ingest collections fully was not agreed by the Department. Table 6 provides information regarding received funds, investment, co-investment and actuals from the beginning of the project to February The projected expenditure for is provided for completeness. RDSI PROGRAMME INVESTMENT SUMMARY 2015 Node/Category Total Contracted Investment in RDSI Actuals to 17th March 2015 Table 6 RDSI Programme Investment Summary to 30 June Committed expenses March to June 2015 NODE DEVELOPMENT PROGRAMME $10,628,480 $10,570,452 $10,896 ReDS PROGRAMME $28,389,978 $32,037,251 $334,391 DaSh PROGRAMME $7,695,000 $7,193,031 $12,577 PROJECT OFFICE $2,986,542 $3,164,866 $418,723 Node's CO-INVESTMENT -$915,769 $693,547 $222,222 CONTINGENCY $300,000 REPORT PROJECT $40,000 REMAINING FUNDS INTEREST EARNED -$3,782,288 TOTALS $50,915,769 $53,659,147 $1,038,809 OPERATING POSITION $1,038,910 $101 It is anticipated that at the end of financial year the project operating position will be $ During this reporting period all funds were spent in accordance with the head agreement and a finance audit statement supporting this report will be sent to the Department directly by the University of Queensland finance department. RDSI Project Final Report Page 36 of 136

37 6 Expected future usage of the infrastructure All eight RDSI funded nodes have committed to continue to provide access for researchers to their operational infrastructure. Each node has their own funding structure and sustainability model. The original proposals received from each node provided information about how nodes would become sustainable after the RDSI project concluded and, in some instances, these include subscription charges. Nodes are not charging for usage during the term of the project. Sustainability beyond the project is outside the scope of this project. Reporting from RDSI funded nodes shows that the data being stored covers all twenty two FoR disciplines and there are indications of continuing demand for storage amongst researchers. Evidence for this can be seen from the use cases reported elsewhere in this document. It is, therefore, expected that significant future use of this infrastructure is anticipated. 7 Project key successes and lessons learnt In four years, as a result of the RDSI project, collaborative research data storage will have gone from none to Fifty Petabytes, overall research data storage will have increased tenfold and the use of shared storage is now commonly considered. The infrastructure, and the ecosystem that has been developed, now places the sector in an excellent position to allow the Research Data Services (RDS) project to develop and deepen the use of this substantial shared resource through its work with research communities and eresearch Service Providers, such as the RDSI Nodes. 7.1 The project management The RDSI project is unique as there is no similar national or international research storage infrastructure available to the higher education and research sector. This distinctiveness provided a clean canvas for the project to consult with the sector and achieve the best outcome for researchers in order to establish a national high speed collaborative research data storage system. It also posed many challenges that had to be addressed by the project team, stakeholders and sponsors and was closely watched by the community. At the time of writing this report nodes had made available over 17PB of collections to researchers and were expected to provide another 33 PB by June 2015, this is equivalent to over 100 PB of physical storage. The nodes also had a number of challenges to overcome while implementing their nodes effectively and some have not ingested as much data as contracted to the RDSI project. Successes RDSI is an exemplar for this type of project internationally, it addressed a unique challenge and in the process assisted in changing the mind set on how research data is stored in Australia Delivered Australia s research data cloud through eight nodes geographically dispersed and strategically positioned with high speed network access Established a Merit Allocation System (MAS) which ensured that Data stored through the ReDS programme met certain criteria that indicated or demonstrated that it formed part of a collection of national significance The Large collections allocation had a strategic view to support storage of research community level research by aggregating data from particular domains in ways that enable advanced research. This is an exceptional outcome of the ReDS programme RDSI project had a significant impact on the research capability development of nodes, in fact without the RDSI project Australia s research data cloud would not exist RDSI initiated a research data storage cultural shift improving collaboration and ultimately seeing new research discoveries in the future RDSI Project Final Report Page 37 of 136

38 RDSI funded storage hosts data from twenty two fields of research, from humanities to radio astronomy and much more providing an inclusiveness of disciplines Wide engagement and sector consultation was crucial to shaping the RDSI project priorities and outcomes, with the Strawman and Tinman documents prompting robust feedback from the community Support from RDSI Advisory Board and stakeholders was crucial to set the project in the right course Built a community of data storage experts that was not available in the sector prior to RDSI Addressed the big challenge of moving data fast by managing a high bandwidth network project, which was designed and lead by RDSI, implemented by AARNet and funded by NRN Discovered and implemented tools to manage and access data consistently across the sector as well as identity management across other methodologies besides web based access Node implementation of a secure accessible infrastructure Lessons learnt Some of the challenges the project, nodes, stakeholders and sponsors had to overcome include, but are not limited to: There was an expectation that decisions should be unanimous amongst nodes and this was difficult to achieve because nodes have competing as well as collaborating interests It was difficult to gain unanimous agreement on some aspects of the project consultation and implementation and this provided some delay to the project Upfront investment to align node understanding and expectations if implemented would have been beneficial to all parties involved Collaboration between nodes was time-consuming to establish and this had an impact on project timing The RDSI project was established very rapidly and this impacted clear definition of deliverables and realistic key performance indicators in the initial documents Although this was not a project objective per se it seems that there were expectations that a cultural change would be driven through the short term RDSI project. This was only partially successful and the constraints of the EIF funding model caused issues for the project and stakeholders RDSI project governance was clear however some nodes were eager to participate on the Advisory Board which could have been construed as a conflict of interest in many aspects of the project, in particular the distribution of funds A strong and responsive interaction between the Department and the project is essential to rapid delivery of a project of this type 7.2 Methodology and Resources The RDSI project management followed the Project Management Body of Knowledge (PMBOK ) guidelines using the fundamental practices needed to achieve the best results for the project as a whole and within its programmes and sub-projects. Agile project management, where applicable was also used to manage sub-projects that had an iterative and incremental nature to designing and building activities, especially the ones in which nodes participated actively. The risk register was updated regularly and the RDSI Advisory Board consulted in the event of potential issues. Regular video conferences and face to face meeting with the project team, nodes and various sub-project committees took place regularly. RDSI Project Final Report Page 38 of 136

39 Annual business plans and reports were provided to the RDSI Advisory Board and Department for approval after consultation with stakeholders and the project team. It was clear from the early stages of planning that the large and diverse scope of the project required recruitment of a number of expert staff with the ability to adapt to an evolving project to achieve its outcomes. Successes The RDSI project team was made up of twelve highly skilled staff, strategically located in major Australian cities, who were able to work successfully with stakeholders during a series of phases including, researcher and data custodian s consultations, disciplines and node consultations, node implementation and researcher satisfaction and feedback discussions. The governance of the project was established through an RDSI Advisory Board comprised of an experienced group of individuals prepared to share their knowledge and engage with the project team and sponsors to advise and ensure the project delivered its outcomes was indispensable to the success of the project. The additional two RDSI funded staff at each node, starting at different times at each node from July 2013 provided an enhancement for nodes to develop their infrastructure and for the project to fulfil its objectives and achieve its outcomes. The RDSI funded staff at the nodes created a group of highly qualified experts in the following areas to support nodes: data centres, infrastructure storage deployment, network service deployment and management, data storage and transmission efficiency analysis, data access interface design, or database design, and are expediting the implementation of the node architecture deployment. This is an excellent outcome for the sector. The RDSI project had a significant impact on the capability development of nodes not only financially but also promoting open discussions with node staff from technical to Chief Executive Officers (CEOs). Flexibility in the management of the project was beneficial to stakeholders as they had many opportunities to provide input and shape the outcomes. Lessons learnt Head agreement between the Department and Lead Agent (UQ) did not define the Governance artefacts in detail. In some instances, this lead to divergent views. Agreements between the Lead Agent (UQ) and nodes had to reflect the Head agreement which added significant complexity and created a level of discomfort for the nodes. A simpler Head Agreement with clear definitions of Governance requirements could streamline contractual agreements. The project would have benefited from greater interaction with the Government in later stages from the middle of The RDSI project review, which was outside the scope of the project was conducted at the end of the project when it had already achieved most of its outcomes. It caused some unplanned delay and did not publish outcomes. The flexibility, openness and inclusive nature of the project management may have given the impression that the project plan was not fully defined. Node expectations did not always accord with project objectives as defined in the overall project contract. This problem could have been managed more effectively through a more detailed definition of project governance and management in the original project contract. RDSI Project Final Report Page 39 of 136

40 7.3 RDSI Programmes The RDSI project was divided in four core programme areas to best deliver the project outcomes. Detailed information about each programme is available in section 1.4. Successes Stakeholders consultation for all programmes started in 2011 and concluded in Diagram 3 shows consultations during 2011 and 2012 as the programmes shaped up. Figure 6 RDSI Consultation This diagram provides an overview of consultations at the beginning of the project The Node Development programme funded the development of eight high capacity nodes strategically geographically positioned. Six Primary nodes located in Brisbane, Sydney, Canberra, Melbourne, Adelaide and Perth, with two additional nodes in Townsville and Hobart. Diagram 4 shows the dates nodes signed the first contract with the Lead Agent. Figure 7 Node s contract signing This diagram shows the progress of Node s contract signing with the Lead Agent RDSI Project Final Report Page 40 of 136

41 All nodes are operational, supporting the twenty two FoR disciplines and are being used by the researchers. Researcher s feedback was carried out to understand the impact RDSI funded infrastructure has on its users. Further information is available in Appendix D. The ReDS programme was designed to identify research data holdings of lasting value and importance and contribute funding to their development at the most appropriate node. ReDS delivered storage services in support of significant data collections, research data sets and associated access tools which are aggregated into related holdings that add value to each other through co-location. The ReDS programme through a series of consultations created a framework for allocating storage to collections of national significance. This was achieved through the formation of ReDS node Merit Allocation Committees. Details on committees per node are available at The Merit allocation process aided institutions to determine their own priorities. Collections per node are detailed at The ReDS programme also created a Merit Allocation Committee Checklist available at Merit allocation committees were diligent in supporting nodes and as a result twenty two FoRs are encompassed at the RDSI funded nodes which is an excellent outcome. The DaSh programme was designed to develop the DaSh Collaboration Network (DaShNet) and the DaSh Technical Architecture. The integration of these two major parts of the DaSh programme became the DaSh Technical Framework, which describes the network, data movement capabilities, security and identity matters, data access, cloud gateway access, test platform for the programme s components and workflow automation capabilities for the RDSI-funded nodes. The implementation of the DaSh programme was managed by the creation of sub-projects with its own steering committees to expedite development. All sub-projects have been delivered or will be finalised before 30 June Example of sub-project outcomes include: o The Mediaflux data management system was adopted by all nodes. o The Aspera suite of tools was negotiated by the RDSI project to make these available to all nodes. This arrangement with Aspera is a world first for Higher Education and Research o The ability to provide an Identity framework using Federated authentication through the AAF is a great success. Identity is an element many countries are still trying to sort out. The provisioning of funds for nodes to further develop their authentication and access mechanisms was a significant outcome for the nodes. o The security workshops ran tailored to each node were well received. RDSI developed Security Policy Guidelines as a framework for nodes and institutions. This activity improved the security awareness at nodes and empowered them to take action were necessary. o The Data Sharing Network (DaShNet) is a reliable high-speed network service built over the new AARNet4 backbone network. It connects RDSI-funded nodes to each other and researchers around Australia. It supports uploading, replication and distribution of data collections, network services delivered to the researchers at each node that are optimised for data-intensive research, and access to a public cloud infrastructure provider to support collaboration around research data. Nodes reported ability to achieve data transfer of up to 40Gb/s, faster than at the universities network. o Test platform to support meaningful evaluation of RDSI funded infrastructure and applications in a realistic environment including Public cloud interconnection point to Amazon Web Service (AWS). RDSI Project Final Report Page 41 of 136

42 Developed an automated storage allocation request management system (ARMS) for data custodians and researchers, available to all nodes. The RDSI project delivered its outcomes in a short period of time within the EIF funding restrictions. The complexity of the infrastructure built, coupled with the lack of any comparable endeavour of this magnitude internationally demonstrates a tangible success for the Australian researchers. The Vendor Panel programme, implemented in partnership with the Council of Australian University Directors of Information Technology (CAUDIT), was created to facilitate the procurement of storage related infrastructure, software and services. The purposes for the programme were twofold. Firstly to allow nodes, universities and other authorised users to avoid lengthy tendering processes by using an appropriately constructed panel, and secondly to support volume pricing across nodes and the wider Higher Education and Research Sector. The Vendor Panel was established in 2012 to assist RDSI Nodes and members of CAUDIT to procure storage related infrastructure, software and services using simple, consistent and pre-agreed terms and conditions to achieve economies of scale. It has allowed substantial procurements to be undertaken in less than a week. The current Vendor Panel comprises of twenty-two vendors as well as reseller partners. The management of the Vendor Panel has been transitioned to CAUDIT to ensure nodes and CAUDIT members continue to benefit from this panel. RDSI ensures transparency to researchers by publishing reports on its website about node data ingest, collections offered by each RDSI funded node and a breakdown of disciplines per fields of research stored at the nodes collectively and individually. Researchers and data custodian are able to decide which node best meets their needs. Outreach Community consultations for all NoDe, DaSh, ReDS and VePa programmes were extremely valuable as input into the project and to provide understanding of the research landscape at the start of the project. The NoDE programme call for participation was well received and supported by a number of organisations. Note that potential nodes had to provide co-contribution to the development of the RDSI funded infrastructure, clearly demonstrating engagement and commitment to the RDSI project outcomes. Input to the core elements of the plan for the ReDS programme were implemented through consultation activities carried out to inform and guide the direction of the ReDS programme, Collection Assessment activities and RDSI funded staff at nodes. This programme actively engaged the community. DaSh programme consultation with the technical stakeholders resulted in valuable feedback incorporated in the Strawman document. Consultations provided input in the shaping of the four programmes and used Strawman and Tinman documents to solicit feedback. Funding of nodes or third parties to develop elements of the programmes and the creation of subprojects with its own steering committees to develop such elements worked well as these activities had node buy in. Lessons learnt During the consultation period and throughout the project many stakeholders expected RDSI to solve many information and communications technology (ICT) collaboration issues that had been apparent in the higher education and research sector for many years, but were never specified as RDSI Project Final Report Page 42 of 136

43 project objectives or outcomes from the sponsor. However, a short term project would be unlikely to solve all such issues. Nodes were provided with funds and staff to support their infrastructure development, some nodes data collections ingest have been slower than expected. This occurred mostly with disciplines that were not well organised prior to the RDSI project, for example medical collections. On the other hand some nodes were able to ramp up ingest quickly as the disciplines they support already had the data catalogued and ready for ingest. What nodes attempted to do was complex and at the beginning they were not always certain of the context of the research data they were ingesting. Future attempts could take this into consideration before funding distribution. The complexity of developing eight nodes suggests that higher upfront investment would have facilitated greater focus in developing their infrastructure at an earlier stage in the project. There seems to be an expectation from some nodes that although contracts to develop specific activities included timelines and deliverables, it was not always necessary to meet these obligations. It was difficult to obtain regular availability reports even though this was a contractual obligations and this constrained the project in making this information public. There were expertise gaps regarding infrastructure specialists across the sector and this inhibited recruitment by nodes. It is advisable that the cohort of experts that has now emerged should continue to be engaged within the sector. Information provided at one level within the nodes were not always shared with other staff within the node and this required significant additional communication activity. 8 Expected future trends in research infrastructure use The RDSI project has been a catalyst in opening up access to research data by moving it to, or allowing it to be accessed through eight eresearch Service providers or nodes, which are widely dispersed throughout Australia. In achieving this, a level of data federation between nodes is becoming possible through the use of the Mediaflux application, federated identity access is being achieved through integration with the Australian Access Federation (AAF) and highly performing data transfers have been enabled through a combination of high bandwidth networks from AARNet, data transfer software from Aspera and the implementation of the Science DMZ research network design. This has established all the elements of a community cloud for research data. In addition links between this storage and the Amazon Web Services (AWS) Public Cloud have been established by extending network connections and developing identity integration between AWS and the AAF through the SPIN project. Integration has also occurred between RDSI storage and the compute capability provided by the NeCTAR cloud. It is anticipated that the cost of using the public cloud will significantly decrease over time to the point at which it becomes more cost effective to provision the majority of storage and compute services in public clouds. The RDSI project has positioned nodes to be able to take advantage of these expected future trends. The RDSI funded infrastructure, and the ecosystem that has been developed, places the sector in an excellent position to allow the Research Data Services (RDS) project to develop and deepen the use of this substantial shared resource through its work with research communities and eresearch Service Providers, such as the RDSI Nodes. RDSI Project Final Report Page 43 of 136

44 9 Performance Indicators 9.1 Providing Research Infrastructure The RDSI project funded research infrastructure that supports a wide range of disciplines within a wide institutional and geographic coverage accessible by end users/researchers. The geographical distribution of the current eight node operations involve a total of 53 organisations or collections of organisations. These comprise of 38 universities, Commonwealth Scientific and Industrial Research Organisation (CSIRO), Bureau of Meteorology, Geoscience Australia, Ergon Energy, Terrestrial Ecosystem Research Network (TERN), National ICT Australia (NICTA), Australian Antarctic Division (AAD), Queensland Cyber Infrastructure Foundation (QCIF), National Computational Infrastructure (NCI), Intersect, Tasmanian Partnership for Advanced Computing (TPAC), Pawsey Centre, eresearch South Australia (ersa), Queensland Government Organisations, Tasmanian Government Organisations, Integrated Marine Observing System (IMOS), International Centre for Radio Astronomy Research (ICRAR), Western Australia energy research alliance (wa:erc), Western Australian Marine Science Research Institution (WAMSRI), Western Australian Satellite Technology and Applications Consortium (WASTAC) and South Australian Government Organisations. Only two Australian universities are not using the nodes; Charles Darwin University and Notre Dame University. Both of these Universities have options to join a node. The eight nodes represent an excellent outcome as not only is there wide institutional coverage, but there is also wide geographic coverage with an RDSI Node in every state capital except Darwin and a node in Townsville, which is likely to also cover Darwin as part of a node concentrating on tropical research data. This meets the goals of the project for Institutional and Geographic Coverage. The following map shows the geographical location of each node. Figure 8- RDSI funded nodes this map shows the location of the RDSI funded nodes. RDSI Project Final Report Page 44 of 136

45 9.2 Value of new infrastructure by location The value of the total infrastructure investment by the RDSI project from the NoDE, DaSh and ReDS programmes is $44,866,264. Including $2,900,794 spent on RDSI funded staff at nodes. These staff assisted in the development of the nodes and collections ingest. The new infrastructure described here is an extrapolation of the investment provided to nodes. It is inaccurate to put a value on the new infrastructure until all RDSI Nodes have finalised ingesting the approved collections. The equipment purchased by each node is detailed in Appendix B Node Asset Register At this stage RDSI has received reports from the eight nodes indicating a total infrastructure investment of $29,521,283. The investment includes capital investment in storage but not operational costs. Node Total Asset register funded by RDSI Total NoDe/ReDS programmes funds Total DaSh programme funds RDSI funded nodes Total NoDe/ReDS /DaSh programmes and Staff funds Adelaide (ersa) * $2,309,946 $3,900,000 $255,592 $422,149 $4,577,741 Sydney (Intersect) * $2,919,492 $6,117,639 $1,001,392 $388,854 $7,507,884 Perth (Pawsey centre) * $5,917,630 $5,913,089 $382,208 $248,428 $6,543,726 Canberra (NCI) * $5,577,978 $7,633,858 $657,951 $370,658 $8,662,466 Brisbane (QCIF) * $5,392,691 $5,382,448 $1,829,746 $394,708 $7,606,902 Townsville (QCIF) ** $985,117 $986,000 $255,592 $367,500 $1,609,092 Hobart (TPAC) ** $1,625,797 $1,679,970 $255,592 $392,989 $2,328,550 Melbourne (VicNode) * $4,792,632 $5,458,804 $255,592 $315,508 $6,029,903 Total $29,521,283 $37,071,808 $4,893,662 $2,900,794 $44,866,264 Table 7- Comparison between RDSI investment and Node Asset register This table compares the total RDSI funds provided to nodes compared to the nodes Asset Register costs. Note: the difference in figures paid to each node relates to nodes being a Primary * or Additional ** node, date of signing agreements, invoicing UQ/RDSI and nodes collections ingest start. In addition some nodes that ingested faster and reached an agreed storage amount triggered payments of from the ongoing ReDS programme receiving additional funds for storage. These nodes were NCI, TPAC and Intersect. The funds above also include the Large Allocation distribution with all involved nodes having signed the agreements and all invoices paid at the time of writing this report. These nodes were Intersect, NCI, Pawsey Centre, QCIF and VicNode. The discrepancy on the amount of funds given to nodes and the amount spent on the asset register are due to nodes only being required to report assets above $50,000. The costs to store a TB of data varies per node. This happens because nodes provision classes of storage to suit the performance and cost requirements of their operations. For example some nodes use mostly tape to store their data, which is cheaper than disk and slower to access, others use a combination of tape and disk. The type of collections stored can also affect the cost to store a TB if the data collection is of a sensitive nature and requires extra layers of security, such as health research. The total of volume of storage aggregates all storage installed by each of the node operators. Within that total, each node has provisioned classes of storage to suit the performance and cost requirements of their operations. The architectural design and apportioning of volume across the storage classes is determined RDSI Project Final Report Page 45 of 136

46 by each node. A comparison of cost effectiveness of the RDSI Project investment, against reported cost per terabyte for storage already installed by each node is shown below. Node operator SSD storage ($/TB) Tier 1 disk storage ($/TB) Tier 2/3 disk storage ($/TB) Tape storage ($/TB) Adelaide (ersa) $ $ $ $ Sydney (Intersect) $1, $ $ Perth (Pawsey) $ $ $ Canberra (NCI) $ $ Brisbane (QCIF 1 ) $ $ $ Townsville (QCIF) $ $ $ Hobart (TPAC) $ Melbourne (Mon) $ $ $ (VicNode 2 ) (UoM) $1, $ $ Table 8 Node s cost per terabyte This table provides the reported cost per terabyte for storage already installed by each node 1 QCIF s Tier 3 disk offering. All disk storage classes are provided with a full tape backup incorporated into the costing. 2 VicNode operates two data centres at Monash (Mon) and University of Melbourne (UoM) with different architectures and cost structures. Appendix B provides the current reported Node Asset Register. 9.3 Value of all infrastructures made available under EIF The value of all infrastructures will be ascertained once all RDSI Nodes have ingested the collections approved by their merit allocation committees. Performance against EIF principles Principle 1: Projects should address national infrastructure priorities Principle 2: Projects should demonstrate high benefits and effective use of resources Principle 3: Projects should efficiently address infrastructure needs Principle 4: Projects should demonstrate they achieve established standards in implementation and management. All four RDSI programmes were designed to meet these principles and they have done so. Currently over $29,521,283 of RDSI funds has been used by nodes to build their infrastructure as described in section 9.2. RDSI has invested $44,866,264 in total on nodes. 9.4 Meeting Researcher Needs The RDSI Nodes are operational and provide information to the RDSI Project Office to report on the number and type of collections for each facility, including the percentage of utilisation and availability of collections stored at the RDSI Nodes. The information about collections and the status of data ingest at each node provides a mechanism for researchers and data custodians to decide on which node best suits their needs. Information on node data ingestion status is available at collection availability per node is available at along with an aggregated representation of approved data broken down by the twenty two Fields of Research at nodes collectively and individually The node Merit Allocation committee members that allocate storage at each node are available at RDSI Project Final Report Page 46 of 136

47 This graph shows the progress of each node as of 22 march 2015 with over 17PB of data available to researchers and over 39 PB of data approved. Figure 9 Node status This graph illustrates two data metrics that are defined as: TB Ready: The single-copy volume of data that has been ingested and is currently accessible to researchers, and TB Approved: The single-copy volume of data that has been approved for ingestion by the Node's Merit Allocation Committee. Last updated week ending 22 March ( Combined Nodes Ingest 40.25% 59.75% Figure 10 Combined Nodes Ingest This graph illustrates the proportion of ingested and approved data by all nodes. RDSI Project Final Report Page 47 of 136

48 ersa Intersect 31.39% 23.17% 68.61% 76.83% NCI Pawsey 43.04% 56.96% 25.85% 74.15% QCIF(BNE) VicNode 32.17% 35.70% 67.83% 64.30% QCIF(TSV) TPAC 83.16% 16.84% 48.42% 51.58% Figure 11 Proportion of ingested and approved data per node This graph illustrates the proportion of ingested and approved data per node. Note that TPAV at the end of February had 75% until their very recent ReDS II Topup. TPAC are probably the best performing node for ingestion of data. RDSI Project Final Report Page 48 of 136

49 ersa Intersect NCI Pawsey QCIF(BNE) VicNode QCIF(TSV) TPAC 31.39% 23.17% 56.96% 74.15% 32.17% 35.70% 16.84% 51.58% Table 9 Nodes ingest This table illustrates the Nodes ingest rates relative to their reported approved data. The amounts of approved and ingested data in varies in each node. All nodes have created a Merit Allocation Committee to assist them with collections allocation. Figure 12 Nodes FoRs representation - The graphs above illustrate the proportion of data each node represents as part of the combined total. Last updated week ending 22 March RDSI Project Final Report Page 49 of 136

50 Furthermore the eight nodes collectively support disciplines from the twenty two Field of Research, this represents an excellent range of disciplines within the sector. Mathematical Sciences Engineering Studies in Human Society Physical Sciences Technology Psychology and Cognitive Sciences Chemical Sciences Medical and Health Sciences Law and Legal Studies Earth Sciences Built Environment and Design Studies in Creative Arts and Writing Environmental Sciences Agricultural and Veterinary Sciences Language, Communication and Culture Biological Sciences Information and Computing Sciences History and Archaeology Education Economics Commerce, Management, Tourism and Services Philosophy and Religious Studies Table Fields of Research This table describes the fields of research supported by the RDSI Nodes. 9.5 Quality of Research Infrastructure The RDSI project has facilitated the creation of a robust, innovative and collaborative network of nodes to support world-class research in Australia. As Australia has been a leader in this area there are no obvious facilities elsewhere with which to undertake a comparison. However, the quality of the research infrastructure is evidenced by the reports received from nodes but even more so by the accounts from researchers themselves using this infrastructure. These accounts can be found at In addition the availability levels at the nodes are comparable or above international standards within the higher education sector. Figure 13 Nodes supported FoRs- This chart illustrates the combined nodes collections allocation by fields of research disciplines. Last updated week ending 22 March ( RDSI Project Final Report Page 50 of 136

51 Charts with fields of research breakdown per node are available in Appendix I. Although the combined view of the RDSI ecosystem, shown in Figure 13, clearly illustrates that the combined effects of the collections across all nodes is a spread across all 22 Fields of Research, Appendix I illustrates the very significant diversity between nodes. The patterns of storage at the nodes illustrates this point with some nodes storing data across 21 of the FoRs and others across only 4 or 5 FoRs. In this latter case, the nodes concerned have over 90% of their data within a single FoR. 9.6 Collaborative Infrastructure Provision The RDSI Project Office has ensured that each of the RDSI programmes was structured to satisfy this performance indicator. Each of the nodes has worked collaboratively with a wide range of stakeholders and co-investors, and have also worked with each other to best serve the researcher s needs. Nodes have and still are collaborating at a number of levels, from technical tests of infrastructure implementation, development of data access mechanisms and shared tools to joint proposals for large allocation collection storage. Nodes have collaborated significantly in the implementation of: DaShNet Data Transfer Network Mediaflux implementation Development of a common End User Licence Agreement (EULA) Common Application Processes Aspera license suite of tools Large allocation collections RDSI worked closely with associated eresearch infrastructure programs such as the National eresearch Collaboration Tools and Resources (NeCTAR), the Australian Access Federation (AAF), and the Australian National Data Service (ANDS), together with major high performance compute (HPC) providers such as NCI and Pawsey Centre. In addition the Board Approval Process was established to accommodate data that could not be hosted within the existing allocation mechanisms or required national coordination. Through this process, three collaborative Large Allocations were approved. Each of these has governance mechanisms appropriate for the respective disciplinary research and data management practices facilitating coordination of data storage infrastructure across participating RDSI Nodes. 9.7 Fostering Collaborative and World-class Research There is clear evidence of significant world class collaboration and research which utilises the research data stored at RDSI funded nodes. These are documented in the RDSI stories available on the RDSI website and at section The Large Allocations, approved by the RDSI Advisory Board and funded through the ReDS Programme, were specifically aimed at addressing disciplinary and domain needs at a national level. Each of the accepted proposals outlined the significant benefit to research that would result from the very large investment. Australian Coordinated Characterisation Data Space The ACCDS intends to underpin national-scale research programs, in particular two recently established characterisation-intensive ARC Centres of Excellence: the ARC Centre of Excellence in Advanced Molecular Imaging (CAMI), which will develop innovative imaging technologies to explore the immune system; and the ARC Centre of Excellence for Integrative Brain Function (CIBF), which is tackling the challenging problems involved in understanding how the human brain works. RDSI Project Final Report Page 51 of 136

52 The nature of the research programmes are collaborative and world-class, and depend upon the maturation, upscaling and integration of data imaging infrastructure with computation and digital toolsets. The participating RDSI Nodes are VicNode, Intersect and QCIF. Australian National Medical Research Data Storage Facility The ANMRDSF proposal was submitted on behalf of four RDSI Nodes after consultation with the NHMRC, AAMRI, and the NSW Office of Health and Medical Research and other major stakeholders. The proposal was developed in consultation with a significant number of independent medical research institutions nationally, as well as major universities with a strategic interest in health and medical research. The foundation data sets identified in the proposal represent major national assets supporting research into Australia s most significant diseases including heart disease, mental illness, the major cancers, as well as the increasing problems of lifestyle diseases such as diabetes and obesity. Importantly, children s health and the health of our aging population are both well supported in the foundation data sets of the ANMRDSF. Reflecting the concentration of health and medical research in Australia, the ANMRDSF will be located at Intersect, VicNode and QCIF. MWA Data Archive The Murchison Widefield Array project is funded for operations over a two year period, which commenced in July The use of RDSI funded infrastructure is expected to enable the MWA Data Archive to store in excess of 9PB of additional data output for scientific use. These data sets, of international importance, intend to assist a global user community to do research into the four main science goals of the MWA: exploration of the early Universe and the search for signals from the first stars and galaxies; exploration of the transient and dynamic Universe; studies of the Earth's ionosphere; and the study of astrophysics related to objects in our galaxy and in the distant galaxies. The MWA Data Archive is of considerable research interest to the Astronomy community in Australia, as well as internationally, and was the basis of an additional call for research proposals, in conjunction with additional NCRIS funding for MWA operations. National Environmental Research Data Centre The National Environmental Research Data Collection (NERDC) comprises international and national reference collections spanning five fields. This multidisciplinary confluence of collections: (a) spans the lithosphere, crust, biosphere, hydrosphere, troposphere, and the stratosphere, (b) encapsulates the complex interactions within, and amongst, these layers, and (c) will enable new, transdisciplinary approaches to research. The shared infrastructure platform of NCI provides a collaborative research environment that draws together data from national public research agencies, and purpose-designed computation, environmental modelling and analytics facilities. National Genomics Data Storage Facility Genomics is a critical and complex science for the understanding of living forms and the way they are impacted by the environment, for improving medicine, and understanding food amongst many other uses. The facility will store and make available large volumes of genome data generated at the leading national centres, as well as essential national and international genome libraries. RDSI Project Final Report Page 52 of 136

53 The participating RDSI Nodes support three major centres of genome science in Australia (Brisbane, Melbourne and Sydney) with strong links and interdependencies with international programs. The proposal was developed through consultation with Bioplatforms Australia (BPA), the national peak body for research infrastructure in genomics, and with the institution involved in the generation and use of genomics data. RDSI Project Final Report Page 53 of 136

54 Appendix A Large Allocation Collection Proposals The RDSI project, in consultation with nodes and other eresearch projects, identified the need for funding the storage of large collections of research data which fell outside the scope of existing storage funding mechanisms within the project. In late March 2013 the RDSI Advisory Board and the Lead Agent identified and approved a process The Board Allocation Process which proposed a way forward to address this requirement. The process was supported by all nodes after extensive consultation, which included a workshop with participation of all stakeholders. This process is described in the RDSI website and it was approved by the Department on 31 July A.1 Large Allocation Collections A call for Large Allocation Collections took place and in April proposals were received from Primary nodes: Australian Coordinated Characterisation Data Space (ACCDS) - VicNode, QCIF (BNE), Intersect Australian National Medical Research Data Storage Facility (ANMRDSF) - Intersect, QCIF (BNE), VicNode Murchison Widefield Array Data Archive (MWA) Pawsey Centre National Environmental Research Data Collection (NERDC) - NCI National Genomics Data Storage Facility (NGDSF) - VicNode, QCIF (BNE), Intersect The following summaries are extracted from the original proposals and other documentation provided by the proposers to RDSI. Note these proposals were not funded in full. Australian Coordinated Characterisation Data Space (ACCDS) Proponents VicNode, Intersect, QCIF (Brisbane) Overview VicNode, Intersect and QCIF, seek a combined allocation for data storage to serve the Australian characterisation research and infrastructure community. This comprises an existing 1PB plus an additional 2.78PB, obtained from key characterisation facilities. Instances of the Characterisation Virtual Laboratory will operate at each node to support collaboration and streamlined data access and processing. Imaging instruments generating significant volumes of data requiring storage can be found at many Australian universities and research institutions. The recently funded ARC Centre of Excellence in Advanced Molecular Imaging (CAMI) and the ARC Centre of Excellence for Integrative Brain Function (CIBF) will commence large multi-modal, multidisciplinary, and multi-institutional research programmes. These underpin demand for storage and provide the impetus for a distributed and coordinated data storage environment. Australian National Medical Research Data Storage Facility (ANMRDSF) Proponents Intersect, ersa, QCIF (Brisbane), VicNode Overview The proposed ANMRDSF is seeking an allocation to establish a distributed data storage facility for health and medical research, with up to 4PB primarily located at Intersect, VicNode and QCIF, with additional capacity at ersa. This facility will enable, for the first time, the aggregation of health and medical data sets generated, curated and managed by many of the dozens of universities, medical research institutes and other government funded organisations who contribute to health and medical research in Australia. There is a recognised need for task specific infrastructure and capacity to store, access, and use health and medical research data due primarily to the specific security and process considerations arising from stringent privacy requirements and RDSI Project Annual Report Page 54 of 136

55 the need to comply with the data management requirements of the NHMRC, health authorities and privacy legislation. The foundation data collections of the ANMRDSF represent an extraordinary aggregation of Australia s leading health and medical research covering Australia s main health priority areas. The ANMRDSF is a unique facility in that it aggregates high value data from a broad range of research, opening up enormous potential for the well understood benefits of data sharing, re-use and collaboration to be realised. As indicated by the NHMRC in their letter of support, the ANMRDSF is well aligned to the strategic intent of the sector and stands to stands to contribute strongly to maximising the value from Australia s health and medical research, and ultimately the wellbeing of Australians. Murchison Widefield Array Data Archive (MWADA) Proponents Pawsey Centre, VicNode Overview Pawsey Centre and VicNode are seeking to support the MWA Project for storage to accommodate a 7.5PB of instrument data (Pawsey Centre) and 150TB of secondary derived data (VicNode). MWA has a ReDS allocation of 1.5PB (Pawsey Centre) and a pending request for a further 1.5PB, totalling a potential request of 10.5PB of research data funded by the RDSI ReDS Programme from the Initial, Interim and Large Allocation Collection funds. The MWA Data Archive collection will be used for ongoing research into the main areas of radio astronomy. MWA data can be combined with large data sets from other Australian and international instruments such as the CSIRO Australian SKA Pathfinder, ANU's Skymapper, NASA's Fermi LAT and many others. The collection will be a primary and unique radio astronomy data set over the next decade, which may support ongoing significant research in this (and related) disciplines. National Environment Research Data Collection (NERDC) Proponents NCI Overview NCI is seeking to establish a National Environmental Research Data Collection (NERDC), comprising 2.8PB of collections already allocated storage through ReDS, and an additional 5.2PB of data identified in the proposal. This collection, with an initial volume of approximately 8.0 petabytes (PB), spans five areas: (a) weather, climate, and earth system science model simulations, (b) earth and marine observations, (c) geoscientific collections, (d) terrestrial ecosystems, and (e) water and hydrology. The confluence of these collections, hosted within the rich and highly integrated computational environment of NCI, will enable research not otherwise possible, and will be a powerful agent in supporting advances in economic forecasting, public safety, and public policy for sustainable development. This proposal is submitted with the strong endorsement of CSIRO, BoM, GA, ANU, IMOS, and the ARC Centre of Excellence for Climate System Sciences. This contribution, by completing the collection, will further develop national collaborative science, and will generate substantial, achievable outcomes, of national significance, that will be demonstrably greater than the outputs of any single participant or user community. National Genomics Data Storage Facility (NGDSF) Proponents QCIF (Brisbane), Intersect, VicNode Overview RDSI Project Final Report Page 55 of 136

56 The VicNode, QCIF and Intersect RDSI Nodes operators seek allocation to establish a facility that provides for the storage and sharing of data collections in the field of genomics, and the associated proteomics and metabolomics fields. The facility will store and make available up to 3.4PB of existing ReDS allocations and 4.19PB of new genome data generated at the leading national centres, as well as essential national and international genome libraries. Each of the nodes is experiencing demand for quantities of genomics data that exceeds the current allocation capacity of the nodes through the existing ReDS program. The proposed allocation will enable each node to store the data that is being generated, close to the instruments and the computation capacity that is being used and to share these collections nationally and internationally. The proposal has the support of BioPlatforms Australia (BPA) and in turn strongly supports the BPA national Framework Data Initiative (FDI). The FDI, including a new round of $3 million, funds data generation in important areas of genomics research. BPA plays a significant role in coordinating between the many organisations involved in genomics research, and in promoting standards, libraries and methods that can be adopted broadly to increase research productivity and excellence. A.2 Evaluation process The Resource Allocation Panel (RAP) was tasked by the board to review the five Large Allocation Collections proposals and additional information, and to make recommendations on the allocation of available funding. The process used at the time is described in this appendix. Since the board meeting on the 17th June 2014, all the principal proposers have responded to a series of questions formulated to seek further clarification regarding each proposal. All these responses were received and reviewed by the RAP. It was previously identified that the ReDS programme funding would be oversubscribed by a factor of up to 1.7 if all proposals were funded. After considering the original Large Collection proposals and the additional information, RAP formulated recommendations to the board. These recommendations took into account compliance with the criteria contained in the requests for proposals, capacity of nodes to ingest proposed Large Collections within the RDSI project timeframe, efficiency of nodes based on past performance and the effect of reduced funding being allocated to a proposal. In addition the RAP had to consider that the ReDS programme could not fund all proposals in full. After considering the recommendations from the RAP the RDSI project Advisory Board approved the recommendation and sought approval from the Department. The Department approval was received on 31 July The RDSI project notified the recipients of these funds and provided contracts to be signed. At the time of writing this report all node had signed the contracts and the invoices were paid. RDSI Project Final Report Page 56 of 136

57 Appendix B Nodes Asset Register Assets listed in this appendix were reported to the RDSI Project Office by 6 March 2015 and are listed in alphabetical order by the node name. The dates listed are provided by the nodes in relation to any changes to their infrastructure procurement. Adelaide (ersa) ersa 2-Mar-15 Reference Number Description Quantity Acquisition Date Total Cost 50050cc10a00208f EN-SASX6X /08/2012 $ 14, cc10a EN-SASX6X /08/2012 $ 14, cc10a108e31 EN-SASX6X /08/2012 $ 14, cc10a1090b8 EN-SASX6X /08/2012 $ 14, DBX /6/ DBX /6/ DBX /6/ DBX /6/ DBX /6/ DBX /6/ DBX /6/ DBX /6/2013 Location Contact person RDSI Project Final Report Page 57 of 136

58 DBX /6/ HUS /6/2013 ALM2543H02E Brocade /6/2013 ALM2543H020 Brocade /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/2013 RDSI Project Final Report Page 58 of 136

59 HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ HCP node /6/ DBX /6/ DBX /6/ DBX /6/ DBX /6/ DBX /6/ DBX /6/ DBX /6/ DBX /6/2013 RDSI Project Final Report Page 59 of 136

60 DBX /6/ HUS /6/2013 ASS2511J009 Brocade /6/2013 BWW2529H009 Brocade VDX /6/2013 BWW2522H010 Brocade VDX /6/2013 M2SEKW HNAS Node /6/2013 M2SEKW HNAS Node /6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/2013 RDSI Project Final Report Page 60 of 136

61 DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ DBS 1 30/6/ HUSVM 1 30/6/2013 CCD2517J00L Brocade /6/2013 CCD2519J039 Brocade /6/2013 S SMU /6/2013 B29Q9T1 DELL switch 1 30/6/ Hitachi CR210H 1 30/6/2013 ALM2543H01Y Brocade /6/2013 ALM2543H027 Brocade /6/ DBX 1 30/6/ DBX 1 30/6/ DBX 1 30/6/ DBX 1 30/6/ DBX 1 30/6/ DBX 1 30/6/2013 RDSI Project Final Report Page 61 of 136

62 ASS2511J00A Brocade /6/ Hitachi CR210H 1 30/6/2013 $ 2,250, Currie Street, City West Data Centre, University of South Australia, Adelaide Currie Street, City West Data Centre, University of South Australia, Adelaide 5000 Total $ 2,309,946 RDSI Project Final Report Page 62 of 136

63 Sydney (Intersect) Intersect Reference Number Description 27-Feb-15 Quantity Acquisition Date DH1-P-23 Market1 400TB 1 28/06/2014 $453,458 DH1-P-22 Market2 200 TB 1 20/12/2013 $145,606 DH1-P-21 Market3 400 TB 1 20/12/2013 $208,776 DH1-P-20 Market4 200TB 1 20/12/2013 $135,464 DH1-P-19 Vault1 1.3 PB 1 20/12/2013 $317,593 DH1-P-18 Vault2 1.5 PB 1 20/12/2013 $335,237 DH1-P-17 Vault3 1.3 PB 1 20/12/2013 $311,771 DH1-P-16 Vault4 100 TB 1 20/12/2013 $33,632 Total Cost Location Contact person Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2113 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2114 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2115 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2116 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2117 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2118 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2119 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2120 Shane Youl Shane Youl Shane Youl Shane Youl Shane Youl Shane Youl Shane Youl Shane Youl RDSI Project Final Report Page 63 of 136

64 DH1-P-15 Vault5 20/12/2013 $742,774 DH1-Q-14 DH1-Q-14 TapeLibrary 1.4 PB TapeLibrary 700 TB 1 20/12/2013 $156, /12/2013 $78,394 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2121 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2122 Macquarie Telecom IntelliCentre Talavera Road Macquarie Park NSW 2123 Shane Youl Shane Youl Shane Youl Total $2,919,492 RDSI Project Final Report Page 64 of 136

65 NCI Reference Number Canberra (NCI) RESEARCH DATA STORAGE INFRASTRUCTURE PROJECT FINAL REPORT 25-Feb-15 Description Quantity Acquisition Date Total Cost Location Contact person IS5500 (240 x4tb) $192, FC/SAS 1 1/3/2013 IS5500(180X3TB Mar-13 $124, NL/SAS) 1 IS5500(60x3TB 28/03/2013 $56, NL/SAS) 1 IS5500(60x3TB 28/03/2013 $56, NL/SAS) 1 Tape Lib /03/2013 $96, expansion frames 1 Tabe lib - no drives 5 28/03/2013 $344, frames + robot 1 SFA12K - dual $444, controller, 10 disk trays (/g/data2) 1 28/06/2013 NCI Data Centre, Australian National University NCI Data Centre, Australian National University NCI Data Centre, Australian National University NCI Data Centre, Australian National University Leonard Huxley Data Centre, Australian National University NCI Data Centre, Australian National University NCI Data Centre, Australian National University Allan Williams Allan Williams Allan Williams Allan Williams Allan Williams Allan Williams Allan Williams 7 SFA12k Disks 380x4 1 13/12/2013 $244, NCI Data Centre, Australian National University Allan Williams 8 Spectra Logic Tape Library 1 19/12/2013 $318, NCI Data Centre, Australian National University Allan Williams 9 SFA12K - dual controller, 10 disk trays (/g/data2) 1 1/09/2013 $754,924 NCI Data Centre, Australian National University Allan Williams 10 SFA12K - dual controller, 10 disk trays (/g/data2) 1 9/10/2014 $790,455 NCI Data Centre, Australian National University Allan Williams 11 TS1150 Tape drive x4 + media 1 17/11/2014 $499, NCI Data Centre, Australian National University Allan Williams 12 Metatdata store 1 17/11/2014 $74, NCI Data Centre, Australian National University Allan Williams 13 E5600 arrays 12 17/11/2014 $1,581, NCI Data Centre, Australian National University Allan Williams Total $5,577,978 RDSI Project Final Report Page 65 of 136

66 Perth (Pawsey Centre) Pawsey Centre 23-Feb-15 Reference Number Description Quantity Acquisition Date Total Cost Location DDN 2 x SFA12K 20E Arrays 2 PB (useable) General purpose resource (node) - Disk storage 1 20/01/2013 $1,500,000 DDN 2 x SFA 12K20 controllers (20 enclosures, 1440 x 4TB NL SAS, 3 racks, 4.5PB usable). Upgrade of 2 x 12K20E to 12K20 controllers (to covert from an appliance). 12K20 test system, 10 x NSD servers, 3 IB switches GRIDScaler External licences, installation & 5 years support. 1 24/11/2014 $2,132, x TS1150 Technology Fibre Channel, Drive/Sled, TFIN-J (Tape Drives), for the Pawsey Supercomputing Centre existing Spectralogic T-Finity Tape Library, located in the Pawsey Centre. The TS1150 Tape Drives operate with SGI s DMF (also in the Pawsey Centre). 1 24/11/2014 $1,355,431 The Pawsey Supercomputing Centre, Burvill Cresent, Kensignton. WA 6151 The Pawsey Supercomputing Centre, Burvill Cresent, Kensignton. WA 6151 The Pawsey Supercomputing Centre, Burvill Cresent, Kensignton. WA 6151 Contact person Jenni Harrison / Chris Schlipalius Jenni Harrison / Chris Schlipalius Jenni Harrison / Chris Schlipalius RDSI Project Final Report Page 66 of 136

67 4 5 Ibm System X3650 M4 2.5" Base Without Power Supply. Csiro_v c 18x900gb 6x800gb Ssd Easytier 5yrs. IBM Tivoli Storage Manager Extended Edition10 PROCESSOR VALUE UNITS (PVUS)LICENSE + SW SUBSCRIPTION & SUPPORT 12MONTHS 1 25/11/2014 $232, x Storagetek T10000 Tape Drive:1 T10000d 16 Gb Fc For Storagetek Sl x Storagetek T10000 T2 Tape Cartridge: Standard (Pack Of 20). Acsls Upgrade. Oracle Standard System Installation Service - Basic: Upgrade- Group I. 5 Years support. 1 26/11/2014 $676,615 The Pawsey Supercomputing Centre, Burvill Cresent, Kensignton. WA 6151 The Pawsey Supercomputing Centre, Burvill Cresent, Kensignton. WA 6152 Jenni Harrison / Chris Schlipalius Jenni Harrison / Chris Schlipalius 6 16 x Additional GPFS licences (00KE570 Ibm Gpfs X86 Advanced Ed V4.xfpo Per Socket With 3 Year Swsands). 1 9/12/2014 $16,951 The Pawsey Supercomputing Centre, Burvill Cresent, Kensignton. WA 6153 Jenni Harrison / Chris Schlipalius Total $5,913,606 RDSI Project Final Report Page 67 of 136

68 Brisbane (QCIF) QCIF BNE Reference Number 6-Mar-15 Description Quantity Acquisition Date Total Cost Location Contact person 1 QRIScloud 1 Storage Tier 2 Storage 712TB of (usable) storage 1 1/11/2012 $243,498 UQ Prentice DC2 Dr David Green QRIScloud1 Storage Network 1 1/11/2012 $41,090 UQ Prentice DC2 Dr David Green QRIScloud Stage 2 StorageExpansion of existing Stage 1 Tier 2 Storage with 2 additional shelves to add 390TB of (usable) storage 1 27/11/2013 $82,901 UQ Prentice DC2 Dr David Green Storage Arrays - Tier 1 Storage including SSD, 10k NL-SAS and SAS - total 1190 TB (usable), 5Y maintenance, shipping and professional services 1 27/11/2013 $446,687 Dell Polaris DC Dr David Green Storage Fileservers includes IO head nodes + Storage management server + 5Y maintenance 1 27/11/2013 $60,412 Dell Polaris DC Dr David Green Polaris Data Centre Development 1 20/01/2014 $500,000 Dell Polaris DC Dr David Green Mediaflux Hardware Infrastructure - SGI High Availability Server configuration + 3Y Maintenance 1 27/06/2014 $60,461 Dell Polaris DC Dr David Green Relocation of HSM SL8500 Hardware from Ipswich to Polaris 1 27/06/2014 $48,698 Dell Polaris DC Dr David Green RDSI Project Final Report Page 68 of 136

69 QRIScloud Stage 3 Storage and HSM Infrastructure including PB Tier 1 (usable), 1.34 TB Tier 2 (usable), 720 TB Tier 3 (usable) disk and 8.0 PB Tape Storage and 5Y maintenance, shipping and professional services 1 $1,903,915 Dell Polaris DC Dr David Green QRIScloud Stage 3 Off-site HSM Infrastructure upgrade including 4.8 PB of Tape Storage $387,028 UQ Prentice DC1 Dr David Green QRIScloud Stage 4 Storage and HSM Infrastructure including PB Tape 1 $1,303,228 Dell Polaris DC Dr David Green QRIScloud Stage 4 Off-site HSM Infrastructure upgrade including PB of Tape Storage 1 $314,772 UQ Prentice DC1 Dr David Green Total $5,392,691 RDSI Project Final Report Page 69 of 136

70 Townsville (QCIF) QCIF TSV Reference Number RESEARCH DATA STORAGE INFRASTRUCTURE PROJECT FINAL REPORT 6-Mar-15 Description Quantity Acquisition Date Total Cost Location Contact person 1 QRIScloud TSV Stage 1 including 540 TB General purpose storage disks, drive enclosures, HDD rack bays with FC connections including installation cost, 100TB DMF Licence capacity increase,nas Servers (including installation costs), Networking /c full care warranty, Shipping & Handling 1 18/09/2013 $220,867 JCU Townsville Data Centre (DA3-010) Wayne Mallett Ph Networking /c full care warranty, Cables, Ethernet switches, Power unit, fibre channel switch 1 18/09/2013 $14,347 JCU Townsville Data Centre (DA3-010) Wayne Mallett Ph QRIScloud TSV Stage 2 including 2280 TB general purpose storage disks, 9.6 TB high speed storage disks, drive enclosures, 2 x 42U racks with metered PDUs including installation cost, Networking /c full care warranty, Shipping & Handling. 1 19/08/2014 $701,571 JCU Townsville Data Centre (DA3-010) Wayne Mallett Ph x C2108-RP2 Dual Socket servers, each with 2 x Intel Xeon E5-2697v2 2.7 Ghz, 512 GB DDR3 RAM, 2 x 200 GB SSD, networking /c full care warranty, Shipping and Handling 1 19/08/2014 $39,701 5 Networking /c full care warranty, and cables 1 19/08/2014 $8,631 JCU Townsville Data Centre (DA3-010) JCU Townsville Data Centre (DA3-010) Wayne Mallett Ph Wayne Mallett Ph Total $985,117 RDSI Project Final Report Page 70 of 136

71 TPAC Hobart (TPAC) RESEARCH DATA STORAGE INFRASTRUCTURE PROJECT FINAL REPORT 23-Feb-15 Reference Number Description Quantity Acquisition Date Total Cost Location Contact person DDN Ref# SO DDN Core GPFS Based Storage Node 1 1/11/2012 $590,000 UTAS Sandy Bay DataCentre Nigel Williams Primary Storage with embedded GRIDScaler servers. 12K20ED-CX3 SFA12K-20E Embedded App Couplet with 16x QSFP FDR-IB/10GbE ports, ICL and power cables, rail kits, 16GB of mirrored and protected RAID cache, 256GB of AP memory, dual power BBUs, and licenses for SFA OS and Embedded App Engine 1 SS8K-SBOD SS slot 6Gb/s SAS/SATA enclosure for 3.5" drives. Includes 2x I/O Modules, redundant power supplies, power cables, rail kit for rack mounting and Cable Management Arms. All slots must be filled with drives or blanks. 10 CBL-HMSHMS-3M 3M 6Gb/s HD msas to HD msas Cable to connect SFA12K Singlet to SS8460 drive enclosures: FRU 40 8K-3T6TX3QJ 3TB 7,200 RPM 6Gb/s 512n SATA drive module for SS8460 enclosure with SAS Interposer 260 8K-600G6SC2QN 600GB 10K RPM 6Gb/s SAS drive module for SS8460 enclosure 20 MFM1T02A-SR SFP+ optical module for 10GBASE-SR; for use with ADPT-CX2-SFPP to convert SFP+ to Short Range Fibre Ethernet 8 RK AU-R 42U Rack with four PDUs for Australia - 9.9kW Max 2 SS8K-BLANK SS8460 enclosure empty slot filler 560 Hot Spares 8K-3T6TX3QJ 8K-600G6SC2QN 3TB 7,200 RPM 6Gb/s 512n SATA drive module for SS8460 enclosure with SAS Interposer GB 10K RPM 6Gb/s SAS drive module for SS8460 enclosure 2 RDSI Project Final Report Page 71 of 136

72 Primary Storage Clustered CIFS & Migration Services. SER20-1S 1U DDN Server - Single 2.0GHz Intel E (6 core), 32GB (upgradeable to 96GB max), 2x 300GB SAS, quad port 1GigE, 2x empty PCIe3 Slots, rails. No Operating System 2 MHQH29C-XTR ConnectX2 IB/10GE HCA, PCI-e2.0(x8), Dual-Port QSFP QDR (40Gb/s) IB/10GbE-PORT, 5GT/s, Tall Bracket, RoHS R6. FOR 10GE, E507USE CBL-QSFPP- XX OR ADPT-CX2-SFPP TO CONVERT FROM QSFP to SFP+ 10GbE 2 MFM1T02A-SR SFP+ optical module for 10GBASE-SR; for use with ADPT-CX2-SFPP to convert SFP+ to Short Range Fibre Ethernet 4 Secondary storage 30% replication with external GRIDScaler servers for installation in customer supplied rack. Note that 10 rack units are required. 6620R2-08FC-4 S2A6620 with dual storage managers, 12GB cache and 4x FC8 host interfaces with SFPs. Includes North America power cables SA02-NA SS Slot Dual-Port 3Gb/s SAS/SATA enclosure for 3.5" drives. Includes power cables, rail kit and Cable Management Arms. Do not mix with SS7000 enclosures. 1 OPT-6620-RS S/W, OPTION, S2A RAID-6 and SATAssure 1 KIT-CBL-6620E Cable kit to expand S2A6620 unit with additional SS6000 enclosure for total of 120 drive slots 1 KIT-CBL-ASOC Regional Power cable kit for non-ddn racks, for Central Asia, Eastern Asia, Southern Asia, South-East Asia, Western Asia, Australia and New Zealand, Melanesia, Micronesia, Polynesia 1 6K-3T6TX3QI 3TB 7,200 RPM 6Gb/s SATA drive module for SS6000 enclosure with SAS Interposer 80 RDSI Project Final Report Page 72 of 136

73 Hot Spares 6K-3T6TX3QI 3TB 7,200 RPM 6Gb/s SATA drive module for SS6000 enclosure with SAS Interposer 2 DirectMon - Single Pane of Glass Monitoring and Management. 1 DM-FSASTR-KIT DirectMon Starter Kit for "E" File Server Appliances. Includes Pre-loaded Primary Management Server, Licenses for Management Server, 1 "E" File Server Appliance, 2 File Server instances and 1 year support for Licenses. 1 Secondary Storage GRIDScaler Servers SER20-1S 1U DDN Server - Single 2.0GHz Intel E (6 core), 32GB (upgradeable to 96GB max), 2x 300GB SAS, quad port 1GigE, 2x empty PCIe3 Slots, rails. No Operating System 2 MHQH29C-XTR ConnectX2 IB/10GE HCA, PCI-e2.0(x8), Dual-Port QSFP QDR (40Gb/s) IB/10GbE-PORT, 5GT/s, Tall Bracket, RoHS R6. FOR 10GE, E507USE CBL-QSFPP- XX OR ADPT-CX2-SFPP TO CONVERT FROM QSFP to SFP+ 10GbE 2 MFM1T02A-SR SFP+ optical module for 10GBASE-SR; for use with ADPT-CX2-SFPP to convert SFP+ to Short Range Fibre Ethernet 4 I-239-2NFC8 QLE2562 8Gbps dual-channel multi-mode optical PCI-Express HBA, LC connector - includes 2x HSR- 807-SFP 2 CR-LC-LC-3 LC-LC FC cable optical to storage RoHS (3 Meter) 4 Software Licences GSS-T1-R DM-SRBA-LIC GRIDScaler Server v3.4: 1-Socket License + 1Yr Support REGISTERED PRICING: Quantity 1 50; 6 DirectMon Small Redundant Block Appliance Management License to manage a S2A6620 redundant block appliance. Includes 1 year support. 1 RDSI Project Final Report Page 73 of 136

74 DM-LFSA-LIC PSI-SME-BLK PSI-SME-BLK FREIGHT FRU-12K8K-KIT 8K-3T6TX3QJ 6K-3T6TX3QI 8K-600G6SC2QN FRU-60SKIT DM-RBA-SUP1 DirectMon Large File Server Appliance License to manage a 10K-E or 12K-20E Embedded File Server Appliance (GridScaler). Includes 1 year support. Installation & Training SME on-site time (four hours) for consultation and assistance. Time & Materials work to be done onsite and subject to yet to be agreed Statement of Work. Includes hardware and software installation and scrip for basic replication. 18 Allowance for on-site mentoring & training. SME on-site time (four hours) for consultation and assistance. Time & Materials work to be done onsite and subject to yet to be agreed Statement of Work. 2 Support, Freight & Cold spares Recommended allowance for freight, insurance and handling to Hobart 1 SFA12K Redundant Block Appliance w/ 5x SS8460 Enclosures FRU Kit. Includes 1x SFA12K Power Supply Module, 1x 12K NVC Protection Module, 12K chassis fan FRUs plus 1x SS8460 Power Supply and 1x SS8460 I/O-Module 1 3TB 7,200 RPM 6Gb/s 512n SATA drive module for SS8460 enclosure with SAS Interposer 7 3TB 7,200 RPM 6Gb/s SATA drive module for SS6000 enclosure with SAS Interposer 2 600GB 10K RPM 6Gb/s SAS drive module for SS8460 enclosure 2 SS6000 FRU Spares Kit. Includes 1x Power Supply, 1x I/O module and 1 Disk Expander Module 1 1 Year renewal/additional support for any Redundant Block Appliance License 2 1 RDSI Project Final Report Page 74 of 136

75 DM-FSA-SUP1 PFSS-L-HPC-T1S ReDS 1 9RVMG2S, 8RVMG25 1 Year renewal/additional support for a "E" File Server Appliance License 2 GRIDScaler Server v3.4: 1-Socket Support for Years 2+ Quantity 1 50; 12 TPAC SWITCH 1 1/10/2012 $9,134 SERVERS: Dell C6145, 128 core VM machines (openstack mini cluster), ReDS 2 1/06/2013 $32,468 4KSMG2S PowerEdge C6145 (Backend Service Support) 1 1/06/2013 $9,513 UPGRADE TO SO MT1243U01273 MT1315X02370 MCX354A-FCBT UTAS TPAC RDSI Upgrade 4TB option Quote ref /07/2013 $270,500 4TB 7,200 RPM 6Gb/s SAS drive module for SS8460 enclosure 350 3TB 7,200 RPM 6Gb/s SATA drive module for SS6000 enclosure with SAS Interposer 38 IB SWITCHING: $41,957 IB-MSX6036-M-36P SwitchX based FDR InfiniBand Switch, 36 QSFP ports, 1 1/06/2013 MSX6018F-1SFS SwitchX-2 based 18-port QSFP FDR 1U managed 1 1/06/2013 ConnectX -3 VPI adapter card, dual-port QSFP, FDR IB (56Gb/s) and 40GbE, PCIe3.0 x8 8GT/s, tall bracket, RoHS R6 4 1/06/2013 $4,440 SUP-ADPTR-3S Mellanox Technical Support and Warranty 4 1/06/2013 $592 MC Mellanox active fiber cable, VPI, FDR (56Gb/s) 8 1/06/2013 $3,872 ReDS 2 + Topup 1 UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre Nigel Williams Nigel Williams Nigel Williams Nigel Williams Nigel Williams Nigel Williams Nigel Williams Nigel Williams RDSI Project Final Report Page 75 of 136

76 Our Ref.:SGI/LH/UTAS- TPAC/Quote ref UTAS SFA12K RDSI Storage Expansion 16/09/2014 $466,000 Hardware SFA12KXE Embedded Application Couplet with 16x QSFP FDRIB/40GbE ports, ICL and power cables, rail kits, 64GB system memory,256gb of AP memory, dual UPS's and ATS's, and licenses for SFA OS and Embedded App Engine 1 SS slot 6Gb/s SAS/SATA/SSD enclosure. Includes 2x I/O modules, redundant power supplies, power cables, rail kit for rack mounting and cable management arms. 10 6TB 7,200 RPM 6Gb/s SAS drive module for SS8460 enclosure GB 10K RPM 6Gb/s SAS drive module for SS8460 enclosure 10 6Gb/s HD minisas to HD minisas cables, 6m 20 42U Rack with four PDUs for Australia - 9.9kW Max 1 DDN Rack Baying Kit - attaches two side-by-side racks - for 50U, 45U, 42U and 30U racks 1 DDN Datacenter Rack Bolt-Down Kit - for use with 50U, 45U, 42U and 30U racks 1 GRIDScaler Client: 1-Socket License + 1 year support: Quantity Unlimited; 1 Active Fibre Cable, 4X QSFP, 56Gb/s, 10m 8 Support Services One year SUP-BSPO-3- Basic Parts Only Support; storage systems, storage servers, disk drives and accessories; 7x24 remote support; parts only; replacement parts onsite NBD; does not include SW support for apps or file systems;annual pricing 1 UTAS Sandy Bay DataCentre Nigel Williams RDSI Project Final Report Page 76 of 136

77 SME/Consultant 8 hours onsite consultation and assistance. Travel and expense must be quoted separately, for countries: US, CA, AU, JP, NZ, CH, NO, SE, DK. 8 Premium Software Support; GRIDScaler client software; for unlimited - price is per socket; provides support for years 2 and above; 3-yr pricing 2 CEPH Expansion PowerEdge R720XD 4 19/09/2014 $34,424 PowerEdge R720XD 6 19/09/2014 $49,014 UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre Nigel Williams Nigel Williams Force10, S4810P, 1RU, 48 x 10GbE SFP+, 4 x 40GbE QSFP+, 1 x AC PSU, 2 x FM, PSU to IO Panels (Revrs) 2 19/09/2014 $19,730 UTAS Sandy Bay DataCentre Nigel Williams Kit - Dell Networking, Cable,QSFP+,40GbE,Active Fiber Optical Cable, 50 Meters (No optics required) 2 19/09/2014 $1,180 PowerConnect x1GbE Port Managed L3 Switch, optional module bays for dual 10Gb or stacking 1 19/09/2014 $2,787 Kit - 2.5" HDD Carrier 80 19/09/2014 $1,360 CS Intel DC GB Solid State Drive 80 19/09/2014 $37,920 UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre Nigel Williams Nigel Williams Nigel Williams Nigel Williams REDS top-up 2 APC Switched Rack PDU 2G Zero U - power distribution unit 2 19/09/2014 $3,160 DIMM,8GB,1600,2RX4,2G,DDR3L,R /10/2014 $22,400 UTAS Sandy Bay DataCentre UTAS Sandy Bay DataCentre Nigel Williams Nigel Williams 6TB 7.2K RPM NLSAS 6Gbps 512e 3.5in Hot-plug Hard Drive,CusKit 46 28/10/2014 $25,346 UTAS Sandy Bay DataCentre Nigel Williams Total $1,625,797 RDSI Project Final Report Page 77 of 136

78 VicNode Reference Number Melbourne (VicNode) 4-Mar-15 Description Quantity Acquisition Date Total Cost Location 540 TB General purpose 1 storage 1 NSP-QH2-SAS 2 Serial: / NSP-QH2-SAS 1 Serial: / NSP-NETAPP-C2-QH2 Serial: NSP-NETAPP-C1-QH2 Serial: NSP-QH2-SATA 1 Serial: SHM T65B1 6 NSP-QH2-SATA 2 Serial: SHM T6SAR 7 DCS3700 Storage System IBM Part C 8 DCS3700 Storage System IBM Part C 9 DCS3700 Expansion Unit IBM Part E 10 DCS3700 Expansion Unit IBM Part E 11 DCS3700 Expansion Unit IBM Part E 12 DCS3700 Expansion Unit IBM Part E 1 27/5/2013 $15,965 University of Melbourne 258 Queensberry Street 1 27/5/2013 $15,965 University of Melbourne 258 Queensberry Street 1 27/5/2013 $33,626 University of Melbourne 258 Queensberry Street 1 27/5/2013 $33,626 University of Melbourne 258 Queensberry Street 1 27/5/2013 $16,471 University of Melbourne 258 Queensberry Street 1 27/5/2013 $16,471 University of Melbourne 258 Queensberry Street 1 31/10/2013 $748, /10/2013 $748, /10/2013 $748, /10/2013 $748, /10/2013 $748, /10/2013 $748, Contact person David Lam David Lam David Lam David Lam David Lam David Lam RDSI Project Final Report Page 78 of 136

79 13 DCS3700 Expansion Unit IBM Part E 14 System Storage SONAS IBM Part 2851-RXA 15 System Storage SONAS IBM Part 2581-I /10/2013 $748, /10/2013 $748, /10/2013 $748, David Lam David Lam David Lam 16 System Storage SONAS IBM Part 2581-I /10/2013 $748, David Lam 17 System Storage SONAS Interface Nodes IBM Part 2851-SI2 1 31/10/2013 $748, David Lam 18 System Storage SONAS Interface Nodes IBM Part 2851-SI2 1 31/10/2013 $748, David Lam 19 System Storage SONAS Storage Nodes IBM Part 2851-SS2 1 31/10/2013 $748, Noble Park Fujitsu DC David Lam 20 System Storage SONAS Storage Nodes IBM Part 2851-SS2 1 31/10/2013 $748, Noble Park Fujitsu DC David Lam 21 System Storage SONAS Interface Nodes IBM Part 2851-SI2 1 31/10/2013 $748, Noble Park Fujitsu DC David Lam 22 System Storage SONAS Interface Nodes IBM Part 2851-SI2 1 31/10/2013 $748, Noble Park Fujitsu DC David Lam 23 TS3500 Tape Library IBM Part 3584-L /10/2013 $748, Noble Park Fujitsu DC David Lam 24 TS3500 Expansion Frame IBM Part 3584-D /10/2013 $748, Noble Park Fujitsu DC David Lam RDSI Project Final Report Page 79 of 136

80 25 TS3500 High Availability Frame IBM Part 3584-HA1 1 31/10/2013 $748, Noble Park Fujitsu DC David Lam 26 TS3500 HD Frames Enterprise Drive IBM Part 3584-S /10/2013 $748, Noble Park Fujitsu DC David Lam 27 TS3500 HD Frames Enterprise Drive IBM Part 3584-S /10/2013 +E2 $748, Noble Park Fujitsu DC David Lam 28 TS1140 Tape Drive IBM Part 3592-E /10/2013 $748, Noble Park Fujitsu DC David Lam 29 TS1140 Tape Drive IBM Part 3592-E /10/2013 $748, Noble Park Fujitsu DC David Lam 30 TS1140 Tape Drive IBM Part 3592-E07 31 TS1140 Tape Drive IBM Part 3592-E07 32 TS1140 Tape Drive IBM Part 3592-E07 33 TS1140 Tape Drive IBM Part 3592-E /10/2013 $748, /10/2013 $748, /10/2013 $748, /10/2013 $748, Noble Park Fujitsu DC Noble Park Fujitsu DC Noble Park Fujitsu DC Noble Park Fujitsu DC David Lam David Lam David Lam David Lam Labelled Data cart IBM Part /10/2013 $748, Noble Park Fujitsu DC David Lam 35 IBM POWER 750 (TSM SERVER) IBM Part 8408-E8D 1 31/10/2013 $748, Noble Park Fujitsu DC David Lam 36 IBM Flashsystem Storage IBM Part 9830-AE1 37 Sonas Software Interface 3 year REG Serial Number:66AYZIQ Part number:5639-nn3 1 31/10/2013 $748, /10/2013 $748, Noble Park Fujitsu DC Noble Park Fujitsu DC David Lam David Lam RDSI Project Final Report Page 80 of 136

81 38 Sonas Software Interface Node Lic Serial Number:66AY5IQ Part number:5639-sn1 39 AIX operating System Serial Number:66DZNIQ Part number:5765-g /10/2013 $748, /10/2013 $748, Noble Park Fujitsu DC Noble Park Fujitsu DC David Lam David Lam 40 AIX Power VM Serial Number:66DZO1Q Part number:5764-pve 1 31/10/2013 $748, Noble Park Fujitsu DC David Lam 41 SW maintenance for AIX 3 years Serial Number:66AX81Q Part number:5773-sm3 1 31/10/2013 $748, Noble Park Fujitsu DC David Lam 42 NetApp Storage Shelf (DS2246) serial number NetApp Storage Shelf (DS2246) serial number NetApp Storage Shelf (DS2246) serial number NetApp Storage Shelf (DS2246) serial number NetApp Storage Shelf (DS2246) serial number NetApp FAS3250 Controller serial number /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne 258 Queens berry Street University of Melbourne 258 Queens berry Street University of Melbourne 258 Queens berry Street University of Melbourne 258 Queens berry Street University of Melbourne 258 Queens berry Street University of Melbourne 258 Queens berry Street 48 NetApp FAS3250 Controller serial number /2/2014 2,108, University of Melbourne 258 Queens berry Street RDSI Project Final Report Page 81 of 136

82 49 NetApp Storage Shelf (DS4246) serial number SHJSG A47 50 NetApp Storage Shelf (DS4246) serial number SHJSG A4A 51 NetApp Storage Shelf (DS4246) serial number SHJSG A49 52 NetApp Storage Shelf (DS4246) serial number SHJSG NetApp Storage Shelf (DS4246) serial number SHJSG A3C 54 NetApp Storage Shelf (DS4246) serial number NetApp Storage Shelf (DS4246) serial number /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne 258 Queens berry Street University of Melbourne 258 Queens berry Street University of Melbourne 258 Queens berry Street University of Melbourne 258 Queens berry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street 56 NetApp Storage Shelf (DS4246) serial number NetApp Storage Shelf (DS4246) serial number SHJSG /2/2014 2,108, /2/2014 2,108, University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street RDSI Project Final Report Page 82 of 136

83 58 NetApp Storage Shelf (DS4246) serial number SHJSG A3F 59 NetApp Storage Shelf (DS4246) serial number SHJSG A3E 60 Supermicro server 1027R serial number S /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street 61 Supermicro server 1027R serial number S /2/2014 2,108, University of Melbourne 258 Queensberry Street 62 NetApp E5400 Controller and Storage Shelf serial number NetApp E5400 Storage Shelf serial number NetApp E5400 Storage Shelf serial number /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street 65 Supermicro server 1027R serial number S Supermicro server 1027R serial number S NetApp E5400 Controller and Storage Shelf serial number NetApp E5400 Storage Shelf serial number NetApp E5400 Storage Shelf serial number /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street RDSI Project Final Report Page 83 of 136

84 70 Supermicro server 1027R serial number S Supermicro server 1027R serial number S NetApp E5400 Controller and Storage Shelf serial number NetApp E5400 Storage Shelf serial number NetApp E5400 Storage Shelf serial number NetApp Storage Shelf (DS4246) serial number SHJSG A36 76 NetApp Storage Shelf (DS4246) serial number SHJSG A35 77 NetApp Storage Shelf (DS4246) serial number NetApp Storage Shelf (DS4246) serial number SHJSG A39 79 NetApp Storage Shelf (DS2246) serial number NetApp Storage Shelf (DS2246) serial number /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne 258 Queensberry Street University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park RDSI Project Final Report Page 84 of 136

85 81 NetApp Storage Shelf (DS2246) serial number NetApp FAS3250 Controller serial number NetApp FAS3250 Controller serial number /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park 84 NetApp Storage Shelf (DS4246) serial number SHJSG NetApp Storage Shelf (DS4246) serial number SHJSG NetApp Storage Shelf (DS4246) serial number SHJSG BA0 87 NetApp Storage Shelf (DS4246) serial number SHJSG A3D 88 NetApp Storage Shelf (DS4246) serial number SHJSG A43 89 Supermicro server 1027R serial number S Supermicro server 1027R serial number S /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park RDSI Project Final Report Page 85 of 136

86 91 NetApp E5400 Controller and Storage Shelf serial number /2/2014 2,108, University of Melbourne Noble Park 92 NetApp E5400 Storage Shelf serial number /2/2014 2,108, University of Melbourne Noble Park 93 NetApp E5400 Storage Shelf serial number Supermicro server 1027R serial number S /2/2014 2,108, /2/2014 2,108, University of Melbourne Noble Park University of Melbourne Noble Park 95 Supermicro server 1027R serial number S /2/2014 2,108, University of Melbourne Noble Park 96 NetApp E5400 Controller and Storage Shelf serial number /2/2014 2,108, University of Melbourne Noble Park 97 NetApp E5400 Storage Shelf serial number NetApp E5400 Storage Shelf serial number 99 Supermicro server 1027R serial number S /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park 100 Supermicro server 1027R serial number S /2/2014 2,108, University of Melbourne Noble Park 101 NetApp E5400 Controller and Storage Shelf serial number NetApp E5400 Storage Shelf serial number NetApp E5400 Storage Shelf 103 serial number /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne Noble Park University of Melbourne Noble Park University of Melbourne Noble Park RDSI Project Final Report Page 86 of 136

87 NetApp Cluster Switch serial number NetApp Cluster Switch serial number NetApp Cluster Switch serial number NetApp Cluster Switch serial number /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, /2/2014 2,108, University of Melbourne 258 Queens berry Street University of Melbourne 258 Queens berry Street University of Melbourne Noble Park University of Melbourne Noble Park 108 Mellanox 36 Port 40/56Gbps switch Serial number: MT1431U00313 Part number: SX /10/ Mellanox 36 Port 40/56Gbps switch Serial number: MT1431U00314 Part number: SX /10/ Mellanox 36 Port 40/56Gbps switch Serial number: MT1431U00307 Part number: SX /10/2014 Mellanox 36 Port 40/56Gbps switch Serial number: MT1431U00311 Part number: SX /10/2014 Mellanox 36 Port 40/56Gbps switch Serial number: MT1431U00308 Part number: SX /10/2014 RDSI Project Final Report Page 87 of 136

88 113 Mellanox 36 Port 40/56Gbps switch Serial number: MT1431U00315 Part number: SX /10/ Dell R720XD server Serial number: 2YHDWW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: 5XHDWW22 Part number: R720XD 1 1/10/2014 Dell R720XD server Serial number: 7YHDWW22 Part number: R720XD 1 1/10/2014 Dell R720XD server Serial number: DXHDWW22 Part number: R720XD 1 1/10/2014 Dell R720XD server Serial number: RWHDWW22 Part number: R720XD 1 1/10/2014 Dell R720XD server Serial number: 1DHDWW22 Part number: R720XD 1 1/10/2014 Dell R720XD server Serial number:46hdww22 Part number: R720XD 1 1/10/2014 Dell R720XD server Serial number: 58HDWW22 Part number: R720XD 1 1/10/2014 Dell R720XD server Serial number: 6BHDWW22 Part number: R720XD 1 1/10/2014 RDSI Project Final Report Page 88 of 136

89 123 Dell R720XD server Serial number: JBHDWW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: 16KDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: 35KDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: B6KDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: C5KDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: C6KDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: 48HDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: C9HDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: F6HDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: F7HDW22 Part number: R720XD 1 1/10/2014 RDSI Project Final Report Page 89 of 136

90 133 Dell R720XD server Serial number: J8HDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: 4ZJDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: 8ZJDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: 9YJDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: CZJDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number: FYJDW22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:2dhdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:4chdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:5bhdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:cbhdw22 Part number: R720XD 1 1/10/2014 RDSI Project Final Report Page 90 of 136

91 143 Dell R720XD server Serial number:g8hdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:3zhdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:4zhdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:8yhdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:hyhdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:jyhdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:40jdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:50jdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:90jdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:f0jdw22 Part number: R720XD 1 1/10/2014 RDSI Project Final Report Page 91 of 136

92 153 Dell R720XD server Serial number:fzhdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:3chdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:fchdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:g9hdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:fz6dw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:81hdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:90hdw22 Part number: R720XD 1 1/10/ Dell R720XD server Serial number:2z6dw22 Part number: R720XD 1 1/10/ Dell R320 server Serial number:62ydw22 Part number: R /10/ Dell R320 server Serial number:92ydw22 Part number: R /10/2014 RDSI Project Final Report Page 92 of 136

93 163 Dell R320 server Serial number:d2ydw22 Part number: R /10/ Dell R720 server Serial number:84hdw22 Part number: R /10/ Dell R720 server Serial number:1yhdw22 Part number: R /10/ Dell R720 server Serial number:6xhdw22 Part number: R /10/ Dell R720 server Serial number:byhdw22 Part number: R /10/ Dell R720 server Serial number:hwhdw22 Part number: R /10/ Dell R720 server Serial number:jxhdw22 Part number: R /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:2sfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:8rfdw22 Part number: MD /10/2014 RDSI Project Final Report Page 93 of 136

94 172 Dell MD1200 Disk Array (12*4TB drives) Serial number:jrfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:22gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:82gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:f2gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:50gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:60gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:fzfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:7zfdw22 Part number: MD /10/2014 RDSI Project Final Report Page 94 of 136

95 180 Dell MD1200 Disk Array (12*4TB drives) Serial number:8zfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:bzfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:64gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:c3gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:g4gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:6nfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:9mfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:cnfdw22 Part number: MD /10/2014 RDSI Project Final Report Page 95 of 136

96 188 Dell MD1200 Disk Array (12*4TB drives) Serial number:14gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:43gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:d3gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:92gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:b1gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:j1gtdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:11gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:32gdw22 Part number: MD /10/2014 RDSI Project Final Report Page 96 of 136

97 196 Dell MD1200 Disk Array (12*4TB drives) Serial number:91gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:34gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:83gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:h3gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:1nfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:7nfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:dnfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:12gdw22 Part number: MD /10/2014 RDSI Project Final Report Page 97 of 136

98 204 Dell MD1200 Disk Array (12*4TB drives) Serial number:72gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:d2gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:3tfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:7sfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:9tfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:41gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:b0gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:d1gdw22 Part number: MD /10/2014 RDSI Project Final Report Page 98 of 136

99 212 Dell MD1200 Disk Array (12*4TB drives) Serial number:bnfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:hnfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:jmfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:30gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:80gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:c0gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:71gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:f0gdw22 Part number: MD /10/2014 RDSI Project Final Report Page 99 of 136

100 220 Dell MD1200 Disk Array (12*4TB drives) Serial number:f1gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:66gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:c6gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:f6gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:46gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:76gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:g6gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:20gdw22 Part number: MD /10/2014 RDSI Project Final Report Page 100 of 136

101 228 Dell MD1200 Disk Array (12*4TB drives) Serial number:40gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:gzfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:1sfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:4tfdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:fsfdw22 Part number: MD /10/2014 Dell MD1200 Disk Array (12*4TB drives) Serial number:96gdw22 Part number: MD /10/2014 Dell MD1200 Disk Array (12*4TB drives) Serial number:d6gdw22 Part number: MD /10/ Dell MD1200 Disk Array (12*4TB drives) Serial number:h6gdw22 Part number: MD /10/2014 RDSI Project Final Report Page 101 of 136

102 236 IBM TS3500 Tape Library High Density Expansion Frame Serial number:78s1898 Part number: 3584-S24 1 1/10/ TS1140 (Jag 4) Tape Drive Serial number:78dad0d Part number: 3592-E07 1 1/10/ TS1140 (Jag 4) Tape Drive Serial number:78dad5d Part number: 3592-E07 1 1/10/ TS1140 (Jag 4) Tape Drive Serial number:78dad57 Part number: 3592-E07 1 1/10/ TS1140 (Jag 4) Tape Drive Serial number:78dad56 Part number: 3592-E07 1 1/10/ TS1140 (Jag 4) Tape Drive Serial number:78dad44 Part number: 3592-E07 1 1/10/ TS1140 (Jag 4) Tape Drive Serial number:78dad0b Part number: 3592-E07 1 1/10/2014 IBM FlashSystem 840 (6TB) Serial number: Part number: 9843-AE1 1 1/10/ IBM TSM pseries - IO Expansion Serial number:9k8v494 Part number:8408-e8d 1 1/10/2014 RDSI Project Final Report Page 102 of 136

103 Labelled Data cart (4TB Capacity) Part number: /10/ IBM Tivoli Storage Manager Extended Edition PVU with 1yr Support Subscription Part Number: D56FELL 672 2/10/ IBM Tivoli Storage Manager Extended Edition PVU with 3yr Support Subscription Renewal (Software support) Part Number: E029ELL 672 3/10/2014 Total $4,792,632 RDSI Project Final Report Page 103 of 136

104 Appendix C RDSI Risk Register The original project plan identified the broad risks for the RDSI project. The tables below expanded on the risks from the project plan and have been evaluated at the end of the RDSI project deployment phase concluded 31 December The project risks were managed adequately and addressed when necessary. The Risk rating has been updated as appropriate and an explanation added when appropriate. Nomenclature: Likelihood: A- Almost Certain HL - Highly Likely L - Likely P - Possible R - Rare Consequence: C - Catastrophic Mjr - Major Mod - Moderate Min - Minor I - Insignificant Risk rating: E - Extreme H - High M - Medium L Low None - Eliminated Risk Area 1: Financial including sustainability Ref No. Risk description and impact Potential consequences Likelihood rating Consequence rating Risk rating Risk mitigation strategy / Treatment plan Responsibility 1.1 Potential for cost overruns: - Funds allocated to each programme may not be sufficient or realistic depending on demand. - Storage and data centre maintenance costs may go up. - Sub-contractor costs may go up. - Project cost overrun. - Need for project scope reduction. - Need for reduction on project deliverables. - Loss of trust from the sector. - Inability to manage and/or finalise the project. P Mjr None - The Funding Agreement limits total EIF funding, and each Sub-Contract will limit EIF funding available to each Sub- Project. - Monitor closely funds expenditure. - Reallocate funds from programmes if necessary providing the Department agrees. RDSI Advisory board Project Director/Manager RDSI Project Final Report Page 104 of 136

105 1.2 Potential for funds to not be spent by the end of the project - Return funds to the Department - Potential to invest funds in other project programmes and expand programmes - Extend project life which will extend infrastructure sustainability - Interest earned goes into ReDS programme for storage: will have 2 years to allocate funds for storage HL Mjr None Opportunity to use remaining funds to fund activities identified by nodes in December 2014 as well as top-ups - Ensure funds are spend adequately in each programme - Monitor project expenses to ensure all programmes are managed appropriately - Re-allocate funds to programmes as required if approved by the Department - RDSI may be able to fund more than 100PB at today s prices and over 2 years prices may drop hence there is the potential to fund all data and still have storage space left. - Monitor collections/nodes and assist them to ramp up - De-allocation process to be used if collection/node does not perform. - Extend project to support sustainability and provide additional time to spend all ReDS funds. RDSI Advisory board Project Director/Manager RDSI Project Final Report Page 105 of 136

106 1.3 Limitations of EIF funds to infrastructure development - Inability of some nodes to commit as operational costs may be high and not sustainable over long term -Need for project scope reduction. - Re-evaluate programmes priorities and funds allocation. - Reduction on project deliverables. P Mjr None Addressed by RDS project - The RDSI Advisory Board to ensure that the structures and governance put in place will support the operational use and availability of the infrastructure in consideration of the limitations of the EIF funds. - Utilise other funds (coinvestment) to assist with non-eif expenditures such as funding the communications role. RDSI Advisory board Project Director/Manager 1.4 Inability to attract key coinvestors. - Jeopardise project outcomes. - Loss of reputation for the Department, Lead institution, Project Board and Project Team. P Mjr None - Co-investment will be actively encouraged through the pursuit of open control and access policies operating the infrastructure. - Co-investment is a requirement on Primary nodes. RDSI Advisory board Project Director/Manager RDSI Project Final Report Page 106 of 136

107 - Manage the ReDS programme to strengthen the ongoing role of nodes regarding the data they hold. 1.5 RDSI project activities do not assist nodes to become sustainable - The infrastructure created by the RDSI project is not sustained after the project concludes resulting in the loss of shared data storage capability. P Mjr (few nodes) C (all nodes) L (few nodes) None (all nodes) - Encourage node's RDSI funded facilities can be and where possible are constructed as an element of larger research data infrastructures. - Ensure Dash connects nodes to institutions as well as other nodes at very high speeds in a combined data movement fabric. - Note that not all nodes must satisfy all requirements for RDSI to succeed. Project Director Project Manager RDSI Advisory board 1.6 Sustainability of non-node based Project elements - Failure of any of the project programmes elements such as DaSh P Mjr None - The Project Board will work with stakeholders and Sub-Project participants to seek solutions to contribute to the sustainability of the Project elements during the life of the Project RDSI Advisory board Project Team Stakeholders (AAs) RDSI Project Final Report Page 107 of 136

108 1.7 Inability to construct an attractive vendor panel to nodes - Inability to attract the most value for money vendor panel. P Mod None - The RDSI project will work with CAUDIT and other stakeholders to leverage existing expertise in the construction of vendor panels - CAUDIT will work with the project to assist in the construction of the panel - consultation with vendors and community will be sought Project Team Stakeholders CAUDIT 1.8 Vendors currently used by the sector are not on the vendor panel causing a problem for nodes - Inability to include currently used vendors in the panel P Mod None - The RDSI project is aware of the major established vendors and will take this into account when negotiating the panel - Leverage CAUDIT's existing knowledge of the sector vendor use - Seek vendor consultation on the process to be used to form the vendor panel to ensure vendors are included - Inclusion on the vendor Project Team CAUDIT RDSI Project Final Report Page 108 of 136

109 panel will be an ongoing activity. 1.9 Interest rate variability is a risk to overall funding Interest rate could impact: - project funds may earn more interest than predicted. - Ability to extend the amount of storage funded by the project. - Ability to extend the length of the project to ensure expenditure of ReDS funds and coverage of a larger number of collections. - Project cost overrun. - Need for project rescope. - Reduction on project deliverables. - Inability to finalise the project. HL Mjr None - The interest earned is identified as an additional source of funding for the ReDS programme rather than as a critical component of project funding. - Extend the project lifetime to allow for greater storage allocation due to interest earned and reduction in storage prices over the lifetime of the project. - Reallocation of interest earned to other project programmes if needed may be considered. - Observe interest rate fluctuation and employ risk mitigation measures if there is a large interest rate fluctuation that impacts the project negatively. RDSI Advisory board Project Team RDSI Project Final Report Page 109 of 136

110 1.10 Inability of some nodes to attract collections - Some nodes may not be perceived as sustainable in the long term. P Min None - Project to ensure at least some nodes is supported and monitored to increase their chances of attracting collections of significance and large volumes of collections. - Ensure node and collection owners are aware of each other to increase the node's sustainability in the long term. RDSI Advisory board Project Team Risk Area 2: Suitability for Purpose Ref No. Risk description and impact Potential consequences Likelihood rating Consequence rating Risk rating Risk mitigation strategy / Treatment plan Responsibility 2.1 Adequacy of access arrangements - DaSh infrastructure fails to deliver adequate access options P Mjr None - access policies and arrangements will be determined with sector consultation and modelled on those already in use to support data collections within the research sector. - Input from vendors in many aspects of DaSh themes will be sought. Project Team RDSI Project Final Report Page 110 of 136

111 - support nodes to build own access requirements 2.2 nodes not selected in time - Cannot get the Technical Advisory Committee (TAC) and decisions made in time P Mjr None - Persuade the Department to select nodes - Assist potential nodes to adhere to the NoDe programme and hence have a chance to be selected by the IAP, Board and the Department. the Department RDSI Advisory board Project Manager/Director 2.3 Adequacy of data movement - Network is not fast enough to support data movement - Data movement tools are inadequate P Mjr None - Ensure network players are engaged and reach agreement - Monitor negotiations and progress Project Manager/Director RDSI Advisory board Project Team 2.4 Nodes reject collections - Collections of significance will not be accessible P Mjr None - Ensure nodes are strongly encouraged to accept collections via the ReDS programme. - Contractual arrangements will be in place to minimise this risk. - Monitor collection allocation progress closely Project Manager/Director RDSI Advisory board Project Team - Encourage nodes to talk to researchers RDSI Project Final Report Page 111 of 136

112 2.5 Incomplete agreement between TAC and project on DaSh technical architecture - Non-delivery of elements of DaSh causing low accessibility of data, project delays and possibly failure L Mjr None - Involve TAC early on the project and ensure nodes understand the DaSh proposal. - Facilitate technical discussions and stage DaSh to identify functional elements. Project Manager/Director RDSI Advisory board Project Team Stakeholders (nodes) 2.6 Incomplete implementation of DaSh architecture - Nodes fail to implement DaSh. - Project fails to negotiate an agreed approach with the majority of nodes. P Mjr None - Encourage nodes to agree on a reasonable architecture. - If some nodes do not implement the DaSh architecture they'll not receive further ReDS funds. Project Manager/Director RDSI Advisory board Project Team Stakeholders (nodes) 2.7 DaShNet is not adequately implemented for all nodes - Low ingest rate causing project delays - Inability to replicate between some nodes P Mjr None - Ensure a fast network will operate between nodes. - Contractual arrangements should be in place to support this outcome. Project Manager/Director RDSI Advisory board Project Team 2.8 Data holders may consider ReDS allocations as free storage for use at their discretion. - The intended purpose of the storage is not met due to ineffective data storage and management practices P Min None - Reinforce performance expectations and metrics Project Manager/Director RDSI Advisory board Project Team RDSI Project Final Report Page 112 of 136

113 Data owners might minimise their engagement because of uncertainty about longevity of the project Location of collections may become political, with RDSI only willing to fund a reasonable number of copies for holding purposes - Fewer collections are registered and stored, and funding is underspent P Mod None - Stakeholders may be dissatisfied with the limits of reasonable storage Note: Node sustainability is outside the scope of this project. P Mod None - Promote relationship between data owner and node(s) as enduring - Discuss with data owner, offer options for extra services by nodes, and allow data owner to prioritise distribution of funding for a ReDS allocation amongst nodes Project Manager/Director RDSI Advisory board Project Team Project Manager/Director RDSI Advisory board Project Team Risk Area 3: Project management Ref No. Risk description and impact Potential consequences Likelihood rating Consequence rating Risk rating Risk mitigation strategy / Treatment plan Responsibility 3.1 Adequacy of Project leadership and management - Project delivery failure P C None - Ensure experienced personnel are employed. - Ensure project leadership is adequate. - Ensure project team brings any issues to the project Director/Manager's attention for mitigation. - Ensure project team meets regularly. RDSI Advisory board/the Department - Ensure steps to increase engagement with RDSI Project Final Report Page 113 of 136

114 governance, technical and expert advisory structures are in place - Engage nodes in technical architecture implementation as soon as practicable. 3.2 Delays in programmes implementation - Reduced number of deliverables. - Project overrun - Project failure - Loss of sector trust in the project P Mjr None Project was extended due to inability of nodes to ingest at the start of the project. - Employ key personnel with relevant seniority and experience to work under the direction of the Project Director and within the policies of the University of Queensland. - Experienced and dedicated personnel will be deployed as needed to ensure sufficient attention and effort is available to progress the Project programmes in a timely manner. - Develop staged delivery milestones for all programmes. Project Manager/Director RDSI Advisory board Project Team - Review programmes regularly - Ensure delays are monitored and problems escalated in a timely RDSI Project Final Report Page 114 of 136

115 manner. - Ensure project implementation deploys standardised and proven technologies as much as possible, which will limit the need to test solutions and the danger of project overruns. - Ensure project team brings any issues to the project Director/Manager's attention for mitigation. - Monitor programmes to avoid and/or address any delays. 3.3 Lack of adequate accommodation for project staff - May impact in staff productivity P Mod None - Ensure appropriate accommodation and resources are available to staff. Project Director 3.4 Loss of resources - Staff may leave the project - Inability to source experienced staff replacements in the short lifetime of the project P Mjr None Addressed as needed - Ensure staff are engaged in the project - Ensure there are alternative options for replacement with experienced staff - Monitor staff morale levels - Monitor staff burnout Project Manager/Director RDSI Project Final Report Page 115 of 136

116 3.5 Adequacy of sub-contracting arrangements - Potential for participant organisations to decline participation - Potential for the Department to reject nodes P Mjr None - The University of Queensland will ensure appropriate project management methodologies and policies are applied to the Project. - Maintain an open dialogue with stakeholders. - Provide business information to legal to facilitate contract negotiations. Project Manager/Director RDSI Advisory board 3.6 NRN project does not agree with the definition of DaShNet - DaShNet wouldn't be delivered and therefore movement of data and replication between nodes would be extremely slow and in some cases impossible. P Mod None - Engage fully with NRN SC to ensure alignment between DaShNet and NRN - NRN to assign implementation of DaShNet to RDSI project - Monitor discussion and implementation progress Project Director 3.7 Inability to develop DaShNet at high speed - Data movement and replication between some nodes will be impacted. P Mod None - Actively engage in a technical dialogue with AARNet to identify the element to implement a high speed network. - Monitor discussions and implementation progress Project Director/Manager RDSI Project Final Report Page 116 of 136

117 3.8 AARNet can't deliver the network upgrades within agreed timeframes - DaShNet wouldn't be delivered and therefore movement of data and replication would not be possible P Mod None - Actively engage in a technical dialogue with AARNet - monitor discussion and implementation progress Project Director/Manager 3.9 Inability to deliver RDSI Portal and other DaSh infrastructure tools defined in the DaSh themes - Impact project progress and delivery. - No data movement or collaboration would be established. P Mod None - Actively engage stakeholders to support Dash architecture development - Monitor discussion and implementation progress - Form TAC with participation from nodes and key stakeholders. Project Director/Manager Dash tech architect 3.10 Inability to discover collections - Collections are difficult to discover P Mod None Nodes have canvased their member institutions for collection successfully - Ensure dialogue with ANDS is consistent and proactive - Engage with researchers to discover collections - Monitor discussions and data collections discovery progress ReDS programme 3.11 Misuse of published collection performance data - Loss of project credibility. P Mod None - Publish collection methodology and disclaimers for appropriate use ReDS programme 3.12 Misuse of published information in RDSI Portal - Loss of project credibility. P Mod None - Publish methodology and disclaimers for appropriate use Project team RDSI Project Final Report Page 117 of 136

118 Appendix D RDSI Stories The use cases, or RDSI stories are developed on a monthly basis (minimum), published on the website and released across multiple distribution channels. These stories demonstrate the use of the RDSI funded infrastructure from the researchers perspective. The use cases are available from a summary is provided herein: RDSI Stories Democratising access to ecosystem science datasets Contributing insect genomic data for global research Australia's healthy human genome reference collection Cinematography meets science Weather@home Impact of climate change on Australian Weather How climate change will affect animal populations Observing Earth from space Global research on greenhouse gases Airborne Research Australia Making ocean change visible to everyone CoEPPUsing the cloud to understand the universe Store.Synchrotron Collecting and visualising stem cell datastemformatics Understanding how cancer develops The Lambert Ancient and Modern DNA Sequence Collection The history of a data collection Adapting to climate change Terra Nova Structure instrumentation at QUT's Science and Engineering Centre Paper MinerBig Questions in History Video Stories CoEPP: Using the cloud to understand the universe Store.Synchrotron Stories published from Nodes RDSI supports researchers to understand disease transmission among lizards RDSI helps to unlock the DNA secrets of cancer RDSI Project Final Report Page 118 of 136

119 Appendix E RDSI Project Summary publication The RDSI Project Summary report publication was developed to provide an overview of the outcomes achieved by October This report was distributed at the eresearch Australasia conference 2014 and the Universities Australia conference in March To access this publication click on the image below or download it from the website in pdf format RDSI Project Final Report Page 119 of 136

120 Appendix F RDSI History It has been quite a journey over the past four years as the RDSI project and nodes created this extraordinary infrastructure for the research sector. This publication takes you on an expedition through the history of the project and provides comments from the people who made this remarkable project possible. To access this publication click on the image below or download it from the website RDSI Project Final Report Page 120 of 136

121 Appendix G Node s audited finance statements The audited finance statements as received from the nodes by RDSI are included in this section. Some statements were large documents, to open these as pdf files double click on the image for each node. At the time of writing this report RDSI received finance statements from the following nodes: Node operator Adelaide (ersa) Finance statement received Funds to be returned to RDSI Sydney (Intersect) Nil Perth (Pawsey) Uncertain Canberra (NCI) $ 103, Brisbane (QCIF) $ 7, Townsville (QCIF) $ 10, Hobart (TPAC) Nil Melbourne (VicNode) Nil ersa An audited finance statement has not been received at the time of writing this report. RDSI Project Final Report Page 121 of 136

122 Intersect RDSI Project Final Report Page 122 of 136

123 NCI A draft finance statement was provided by NCI at the time of writing this report. RDSI has been informed that an updated finance statement will be provided. RDSI Project Final Report Page 123 of 136

124 Pawsey Centre An audited finance statement has not been received at the time of writing this report. RDSI was informed on the 27 March 2015 that Pawsey Centre will return $7, of unspent project funds. RDSI Project Final Report Page 124 of 136

125 QCIF (BNE) RDSI Project Final Report Page 125 of 136

126 QCIF (TSV) RDSI Project Final Report Page 126 of 136

127 VicNode An audited finance statement has not been received at the time of writing this report. TPAC RDSI Project Final Report Page 127 of 136

128 April May June July August September October November December January Feburary March April May June July August September October November December January February March April May June July August September October November December January Feburary March April May June July August September October November December January February March RESEARCH DATA STORAGE INFRASTRUCTURE PROJECT FINAL REPORT Appendix H Node s progress The nodes Merit Allocation committee approvals for data collections and the data ingest are summarised here in a monthly basis from 2013 to 2015, including an overview of its progress and challenges. These graphs were last updated week ending 22 March Graphs are formulated from the last report of each month. In cases where no update was received within a month, the previous month has been carried forward. Y axis figures are expressed in Terabytes (TB). Overview August 2014 event approval decline event: The CoEPP data collection was limited from 2PB to 500TB because ersa s Merit Allocation committee decided it to be unwise to hand over that much storage to a single allocation approved ersa available The graph indicates ingest has been very slow with a slight increase mid-2014, with approximately 31% of approved data ingested. Overview The graph indicates that the Intersect s Merit Allocation committee data allocation approvals started slow and ramped up by December Intersect Ingest started off slow with activities ramping up over the last 12 months due to the availability of the Aspera products approved available The graph indicates ingest of approximately 23% of the approved data. RDSI Project Final Report Page 128 of 136

129 April May June July August September October November December January Feburary March April May June July August September October November December January February March April May June July August September October November December January Feburary March April May June July August September October November December January February March RESEARCH DATA STORAGE INFRASTRUCTURE PROJECT FINAL REPORT Overview This graph was formulated from amended NCI figures received in late February 2015 and may not refect figures reported to RDSI on corresponding dates NCI The graph indicates that NCI s Merit Allocation approved available committee data allocation started in July 2013 and increased its approval again in October Ingest has steadly gone up, with approximately 54% of the approved data ingested. Overview April 2014 event: The reduction in approved data was due to an over allocation in relation to the amount of data funded. The data approved returned to its previous figure when the funds were received by Pawsey Centre Pawsey approved available Ingest started very slow ramping up in February 2014 and since has steadly gone up, with approximately 70% of the approved data ingested. RDSI Project Final Report Page 129 of 136

130 April May June July August September October November December January Feburary March April May June July August September October November December January February March April May June July August September October November December January Feburary March April May June July August September October November December January February March RESEARCH DATA STORAGE INFRASTRUCTURE PROJECT FINAL REPORT Overview The graph shows variations in regards to Merit Allocation of storage, this is due to QCIF s diligence to review requests and ingest rates with collection owners aligning the approved ingest quantities to represent near term dates. QCIF indicated it will continue with this approach QCIF BNE approved available Ingest started relativelly slow and since has steadly gone up, with approximately 40% of the approved data ingested. Overview VicNode s Merit allocation scheme was launched in September Early pilot data custodians were identified and where possible data was ingested onto available infrastructure VicNode The sharp increase of data approved in approved available July 2014 is due to VicNode preparing data custodians prior to going live to ensure their datasets were ready once both operating centres were fully operational. VicNode experienced a plateau of ingest during September/October 2014 due to the eresearch Australasia Conference and the Openstack conference in Paris where many data custodians, technical team members and VicNode staff attended. The graph shows ingest started very late and since has steadily ramped up, with approximately 34% of the approved data ingested. RDSI Project Final Report Page 130 of 136

131 April May June July August September October November December January Feburary March April May June July August September October November December January February March RESEARCH DATA STORAGE INFRASTRUCTURE PROJECT FINAL REPORT Overview This graph was formulated from amended QCIF figures received in early March and may not refect figures reported to RDSI on corresponding dates QCIF TSV During October/December approved available 2013 there was a rapid ingest of collections. At that time, there was insufficient capacity to ingest any additional collections as the remaining capacity (around 70 TB) was inadequate for the collections being proposed. This capacity shortfall resulted in a loss of momentum with ingestion and a move to building more bespoke tools and related data services. The commissioning of additional storage took time and pulled resources away from important data cleansing tasks. Delays have also occurred in some data sources (such as the Daintree Rainforest Observatory), IT and networking systems being 100% functional. QCIF informs that all of these areas are now being actively addressed to assist users to start ingesting data into RDSI funded storage. Additional capacity was planned for and implemented, enabling a significant increase in approved allocations in November The rate of ingest has not yet increased due to staff capacity and timing issues. While disk capacity came on-line from Q4 2014, significant researchers were away from campus over the summer break on field work and preparing 2015 grant proposals. Acceleration and staging of data for 2015 has now commenced. The graph shows a sharp ingest in early days followed by a plateau in ingest and an increase in Merit Allocated storage, with approximately 17% of the approved data ingested. RDSI Project Final Report Page 131 of 136

132 April May June July August September October November December January Feburary March April May June July August September October November December January February March RESEARCH DATA STORAGE INFRASTRUCTURE PROJECT FINAL REPORT Overview TPAC s Merit Allocation committee started approving data collections early on and continued to ramp up. In July/August 2014 the collections were re-submitted though ARMS resulting in a reduction of approved data TPAC approved available Ingest started slow and and since has steadly gone up, with approximately 72% of the approved data ingested reported at end of February. TPAC has since sought approval from their Merit Allocation Committee to increase the size of stored collections prior to proposing a ReDSII top-up request to RDSI, which has been accepted. RDSI Project Final Report Page 132 of 136

133 Appendix I Fields of Research stored on RDSI funded Nodes The graph below illustrates the FoRs disciplines stored on the RDSI funded Nodes, demonstrating the richness and diversity of disciplines supported by some nodes compared to the niche disciplines of few nodes. RDSI Project Final Report Page 133 of 136

134 RDSI Project Final Report Page 134 of 136