Michał Jankowski Maciej Brzeźniak PSNC

Transcription

1 National Data Storage - architecture and mechanisms Michał Jankowski Maciej Brzeźniak PSNC

2 Introduction Assumptions Architecture Main components Deployment Use case Agenda

3 Data storage: The problem needs considerable resources (human, software, hardware ) is complex and expensive exceeds the abilities of many institutions is not their core business Outsourcing the process may be the only or at least the most reasonable solution to that problem!

4 Our project KMD (NDS) National Data Storage R&D project that implemented the software PLATON-U4 Popular backup/archival service Deployment project Target: scientific and academic institutions

5 Primary aim: Aims To support scientific and academic community in protecting and archiving the data Secondary aims: Physical protection of the data Assuring logical consistency of the data Long-term data archival Tools supporting backup

6 Our potential ti customers Digital libraries Virtual laboratories Academic computer centres and network operators Research institutions Universities iti Clinical hospitals ~500 > 50 > 600 orga anization ns hundreds d of TB/year

7 Design assumptions I Hi h il bilit d li bilit High-availability and reliability Geographically distributed storage system with data replication Additional profit: scalability (performance, storage capacity, number of users) Challenges: consistency, fault tolerance and high performance

8 Design assumptions II Focus on specific system features and functionality (pointed by the potential users in a survey) Secondary data storage Data durability and service availability Geographical data replication No data sharing or exchange capabilities Confidentiality of the data -> dedicated name spaces Automatic replication according to preferred policy: Number of replicas Synchronous/asynchronous mode Allowed physical localizations

9 Design assumptions III Realism about what we are actually able to provide stable production-level service budget and the time limitations

10 Overall architecture t User Metadata DB Database Node Access MethodsServers (SSH, HTTPs, WebDAV...) Virtual file systems for data and meta data NDS system logic Access Node Users DB Accounting & limits it DB Replica access methods servers Storage Node file system Replication Storage Node

11 Mt Metacatalog tl Logical structure of the virtual file system Attributes and other metadata of files Mapping logical files replicas History of operations

12 Logical separation of namespaces Each customer s contract is connected with separate virtual file system (namespace) Data sharing is not expected by the users Confidentiality is improved Logically separate namespaces mean physically separated metacatalogs Improved performance and scalability

13 Ditibti Distribution of metadata tdt Each metacatalog is replicated asynchronously in master-slaves mode (Slony-I) Number of MC replicas refers number of replicas of user files In case of failure of master MC some slave one is (manually) selected as master

14 Semi-synchronous metadata tdt replication Used in synchronous mode of replication of user data All operations on metadata are synchronously logged to a number of distributed logs In case of failure: all operations logged between the update of the slave MC and the failure of the master MC are performed on the new master Solution safe as synchronous database replication, but much lighter

15 Users database Institutions -customers Contracts and profiles (parameters of services) Required number and localization of replicas Mode of replication (synchronic, asynchronic) Users (certificates)

16 Accounting database Resource usage Statistics Billing Limits it (quota)

17 Data Daemon Emulates virtual file system with logical files and directories on AN Enforces security policies and replication policies Takes into account output of monitoring and prediction modules Produces accounting data The virtual FS can be accessed in a standard way or via a portal (universal interface!) Based on FUSE

18 Metadata Daemon Emulates virtual file system with metadata on AN Metadata is placed in special files located in directories i corresponding to logical files and directories The virtual FS can be accessed in a standard way or via a portal Based on FUSE

19 System interfaces Low level: Virtual file systems for data and metadata High level: Standard protocols: SSH, HTTPS, WebDAV, GridFTP Limitation: authorization - keyfs

20 Data access Typical client software Specialized ed portal

21 Monitoring i and prediction Monitoring of all important elements allows avoiding unaccessible SNs by Data Daemon and quick reaction of the administrators Prediction helps optimal selection of replica to read or selection of node to write a new replica

22 Data Storage HSM (Hierarchical Storage Management)

23 From the user s point of view

24 Scalability Storage Nodes Access performance System capacity Data transmission throughput (distributed data traffic) Access Nodes Responsiveness to I/O requests Data transmission throughput Metacatalogs Potential bottleneck Maximum number of files/directories Perform file system level operations Operation time depends on the database size Transactions -> limited parallel access Sensitive to meta-data-intensive operations (backup/archive applications are rather throughput-intensive)

25 System instantiation ti ti Separate metacatalogs t allows for easy division i i of the system into many instances Pools of access nodes and storage nodes may be assigned to the instances The instances and their elements (metacatalogs, virtual file systems ) may be located on dedicated physical or virtualized servers or coexist The configuration depends on user requirements against data and metadata processing efficiency

26 System deployment infrastructure t for PLATON 12,5 PB tape storage in 5 localizations 2 PB disc storage in 10 localizations 70 servers, SANs and10gbit Ethernet t

27 Use case storage of PIONIER network traffic Store protocol headers for legal l requirements 168TB/year -> 5,5 MB/sec Data collected in >20 geographically distributed PIONIER nodes 5-year durability (replication required) Frequent data writes, rare metadata operations Fits well to PLATON environment: Many data sources -> multiple virtualized ANs Replicas -> multiple SNs Little metadata processing -> may use shared DBMS

28 dcache Distributed, ib t heterogeneous, visible ibl as single virtual file system Designed for big number of users (scientists) accessing the same file system exchange of data is important Persistence model: defines how the data should be exchanged with trirtrary storage, replicated, migrated to hot spots, recovered Access: dedicated protocol, NFS, FTP, HTTP, WebDAV, GridFTP Metadata stored in relational DB, metadata services may be organized hierarchically Suitable for grid environment

29 irods Software for data-grids, digital it libraries, i persistent t archives, real-time systems Data managed using set of rules: replication, modes, security assumptions, access time, load balancing, failure recovery (flexible, but complex) Metacatalog: central, based on relational database Implemented as a set of services Object oriented Approved by NASA for commerce, but used mainly by R&D

30 Thank you! Questions?

31

32 Some facts on massive dt data storage 1800 EB globally ll / 260 GB per person to be produced in 2010 * Data produced in computer systems exceed the storage capacity available Managing, classifying, storing, short-term and long-term protecting data is complex and expensive! *) source: IDC analysis.