HEAnet s Data Storage Services

Transcription

1 HEAnet s Data Storage Services Context & Environment Brian Boyle Network Services Manager HEAnet

2 Storage Services Overview International environment National environment HEAnet s Storage services & plans Current offerings New offerings Planned activity

3 International Terena TF-Storage Working Group Open to all Mainly NREN and IT Services Appearances by Storage Vendors Cloud Providers, Hardware people, etc Many examples of: National Data Storage IaaS by NRENs for IT Services

4 International Summary: Data Storage: Is very complex & expensive Not usually a core activity but could easily take up this role! Needs a lot of work Procurement System Administration End user support

5 International Leads to outsourcing To a cloud storage provider To the NREN Examples: Poland: PSNC NDS WebDAV/SSH interface Hungary: NIIF NSN iscsi

6 International Czech Republic Distributed Data Repository HSM with NAS frontend Sweden SNIC s SWESTORE UK Petabyte STC - Rutherford labs

7 National Environment Many projects, with separate repositories Normal response to distributed funding, expertise, administration, etc

8 National Environment Many projects, with separate repositories Normal response to distributed funding, expertise, administration, etc One exception: E-INIS

9 The e-inis initiative

10 e-inis data storage A pilot service Open to all researchers Access based on scientific merit Distributed across three sites

11 HEAnet s Storage Services Data Centre Storage Data Centre users Very highly scaleable High Functionality High Quality Today: Applications FileSender Media Hosting Tomorrow: Storage as a Service

12 The Solution Spectrum

13 FileSender Simple ad hoc Large file transfer HEAnet, UNINETT, AARNet project A really simple web service Send (big) files to another user that s really hard to do Browser limits (2GB) Interoperability limits Java? Adobe Air? Flash? Silverlight? Google Gears? HTML5?

14 FileSender And there s not many alternatives ? 10MB? 20MB? FTP Who s server shall we use? Dropbox, YouSendIt etc? Costs money Limited in size Not federated

15 Media Hosting Simple Video Sharing web service Accepts any format video Automatically transcoded into H.264 Two outputs: Embedded Video for web pages Link that can be shared

16 Media Hosting There are many alternatives! YouTube, Vimeo, DailyMotion etc Limitations on media size Focused on public sharing Gives control of media away Can cost for high end users Not federated

17 Storage as a Service Generic block/bulk storage service Multi site implementation Rich service menu e.g Snapshots, backups Access via IP or ethernet p2p Service user creates/disposes of resources (Edugate) Equivalent to medium performance locally attached storage

18 Service Overview

19 Storage Services Target Audience: High end MIS users Research Groups Administration Applications Target Applications: Virtualisation Research databases MIS Application storage Backup data location

20 Operational Model Service Users contract for storage space over time, e.g. 20 TB for 6 months 45 TB for 3 years Up to 500TB initially available over 2 POPs Blanchardstown & Parkwest Capable of >2 PB within 3 years

21 Our environment July 2010: 1 of 3 Questionnaires October 2010: Report & Feedback ata_storage September 2010: Proposal to HEAnet s Board Initiation of Storage Steering Group

22 Data Storage Questionnaire Data Storage Service Proposal 3 inter-related threads Extent of storage hosting activity Likely growth from Applicability of HEAnet s proposed service in this context

23 Storage Questionnaire 58 recipients 43% Response rate 70% of the University sector 45% Institute of Technology sector Combined current capacity: 1 PetaByte

24 Storage Questionnaire 80% * interested in HEAnet s proposed storage service Forecast 500TB initial demand 1000TB within 3 years Support for a usage based charge

25 Storage Procurement Necessary for HEAnet s service But lets make it useful for everyone Data Storage Procurement Framework Focused on 3 goals Enterprise Storage Hardware Campus Storage Middleware appliances Cloud Storage Service Providers

26 Status May 2010: Service proposal July 2010: Questionnaire to seek requirements October 2010: Invitation to RFT workshop November 3 rd : RFT Workshop Ongoing: Steering Group updates

27 Next Steps Agree procurement requirements RFT specification Outline vendor contract Framework model Publication, evaluation, award Ongoing framework model Lasts for 4 years Support starts from delivery date

28 FileSender

29 FileSender

30 FileSender

31 FileSender

32 FileSender

33 FileSender

34 FileSender

35 FileSender

36 FileSender

37 FileSender

38 FileSender

39 FileSender

40 FileSender

41 Media Hosting

42

43

44 Media Hosting

45 Media Hosting

46 Media Hosting

47 Storage Services Overview International environment National environment HEAnet s Storage services & plans Current offerings New offerings Planned activity Questions?