KMD National Data Storage

Transcription

1 KMD National Data Storage in the PIONIER network Maciej Brzezniak, Norbert Meyer, Rafał Mikołajczak Maciej Stroiński

2 Location igrid2005, Sept. 27th, 2005

3 SZCZE KOSZAL P GDAŃSK IN BYD GOSZCZ OZNAŃ TORUŃ ŁÓDŹ O LSZTYN CIN GUBIN ZIELONGÓR A W ARSZAW ASIE DLCE PUŁAW Y W ROCŁAWCZ ĘSTOCH OWA RADOM LU BLIN OPOLE KIELC E G LIWICE KATOW ICE KRAK ÓW RZE SZÓW CIESZYN BIELSK O-BIAŁA 10 GElink s 10 GEno des BIA ŁYSTOK igrid2005, Sept. 27th, 2005 PIONIER - Available infrastructure GDAŃSK Installed fibers SZCZECIN KOSZALIN BYDGOSZCZ TORUŃ POZNAŃ OLSZTYN BIAŁYSTOK PIONIER nodes Fibers+pipes (end of 2005) Fibers+pipes (end of 2006) GUBIN ZIELONA GÓRA WARSZAWA SIEDLCE ŁÓDŹ PUŁAWY WROCŁAW RADOM LUBLIN OPOLE CZĘSTOCHOWA KIELCE GLIWICE KATOWICE KRAKÓW RZESZÓW CIESZYN BIELSKO-BIAŁA

4 PIONIER transmission 4Q2005 GDAŃSK KOSZALIN OLSZTYN BASNET 34 Mb/s SZCZECIN BYDGOSZCZ TORUŃ BIAŁYSTOK GÉANT Gb/s DFN 10 Gb/s Gorzów POZNAŃ GÉANT/TELIA 2x2,5 Gb/s PIONIER S FIBERS 2 x 10 Gb/s GTS 1,6 Gb/s WROCŁAW ZIELONA GÓRA OPOLE ŁÓDŹ CZĘSTOCHOWA RADOM KIELCE WARSZAWA PUŁAWY LUBLIN 10 Gb/s (1 lambda) CBDF 10 Gb/s KATOWICE Bielsko-Biała KRAKÓW RZESZÓW 1 Gb/s MAN CESNET, SANET

5 Project Partners Academic Computing Center CYFRONET AGH, Kraków Academic Computing Center in Gdańsk (TASK) Częstochowa University of Technology Marie Curie-Skłodowska University in Lublin Poznań Supercomputing and Networking Center (PNSC) Technical University of Białystok Technical University of Łódź Wrocław Supercomputing and Networking Center (WCSS)

6 National Data Storage - goals Reliable and distributed data storage systems national level The system includes: hardware infrastructure and Backup/Archive management system (server and client) The hardware infrastructure: hierarchical storage systems, file servers, application servers Network: uses PIONIER communication channels and the infrastructure of city

7 The target infrastructure: 4 KMD nodes (thick) 4 application nodes Embedded in PIONIER Features: High security level Confidentiality and integrity of data (advanced B/A client) Higher reliability System and data availability Fault tolerance (hardware and software) Wide service access National (PIONIER+city net) Many SAPs Efficiency Fast backbone networks, broadband network access High-end systems (tape libr.,raids)

8 Added Value Flexible policy definition High level of security and reliability Geographical replications many localisations Coherency of distribued data KMD management software Data coding used in some scenarios (for advanced client configurations) High accessibility thanks to appropriate hardware redundancy and the fault tolerance features in KMD management software User interface integrated with the new system features Data encryption, data coherency Encryption software and/or hardware based (cert. equipments) before moving the data from the customer into KMD Ensuring data coherency by MD5, HMAC High availability ensured by the management software The target data location can be chosen The data replica location can be chosen Confidentiality of the archived data The encryption key is known only by the data owner The KMD infrastructure is storing raw data Accounting of used resources Equiped with accounting system (average monthly amount of data,...)

9 Architecture igrid2005, Sept. 27th, 2005

10 Example of usage - FTP/SCP clients Moving data from client location to KMD

11 Example of usage - FTP/SCP clients Data movement KMD-> client Original location

12 Example of usage - FTP/SCP clients Data Replication Done off-line Depends on the predefined management policy

13 Example of usage - FTP/SCP clients Reading replicated data from KMD Data moved to another node

14 Supporting industrial standards TSM (Tivoli) client Read/Copy data from/to KMD Features: - without data replications - compatible with industrial standards - access to any location -loadbalancing

15 Example advancedb/a client Read/Copy data from/to KMD Features: - DATA REPLICATION! -basedon B/A Tivoli - API interface - compatible with industrial standards

16 End users Video Monitoring System cameras cameras finally TB now TB backup window 23 days only! igrid2005, Sept. 27th, 2005

17 PIONIER POLICE GDAŃSK City Network MAN KMD node POLICE ŁÓDŹ CZESTOCHOWA

18 Summary User point of view Reliable + Secure + User friedly It must be cheap Service Provider Scalable + efficient Easy to manage Network demands: Nowadays: dedicated channels Future: bandwidth on demand, qos» Backbone infrastructure (p2p)» User access

19 Thank YOU!