Data Movement in Distributed Science Environments. Raj Kettimuthu Argonne National Laboratory and The University of Chicago

Transcription

1 Data Movement in Distributed Science Environments Raj Kettimuthu Argonne National Laboratory and The University of Chicago

2 Outline Science Environments Data movement problem GridFTP New features Challenges Globus.org Hosted Data Movement Service

3 Distributed Science Distributed community of users to access and analyze large amounts of data

4 Advanced Photon Source APS Beam line APS HPC APS DMZ Public Server Public Network Beam line Controls Data Acquisition GridFTP Server Beam line Storage Infiniband Lustre Parallel FS GridFTP Server Beam line controls pushes data to HPC GridFTP ANL Server Tier1 Firewall Myproxy APS Server Tier2 Firewall User gets credentials from Myproxy server User pulls data from public GridFTP server

5 Bandwidth Requirements

6 ESNET 09/15/2010 San Diego Supercomputer Center

7 TeraGrid

8 End-to-End Problem

9 GridFTP High-performance, reliable data transfer protocol optimized for high-bandwidth widearea networks Based on FTP protocol - defines extensions for high-performance operation and security Standardized through Open Grid Forum (OGF) GridFTP is the OGF recommended data movement protocol

10 Standards Interoperability Big selling point for adoption Multiple independent implementations Globus provides a reference implementation Server Client tools Development libraries Fermi Lab and U. Virginia have home grown servers that work with ours

11 Robustness It has to work ALL the time Hard to get a solid stable code base Harder to extend it Race conditions Recover from errors Ability to check point transfers A session crash can't be a service crash Fork()/setuid()/exec()

12 GridFTP Servers Around the World Created by Tim Pinkawa (Northern Illinois University) using MaxMind's GeoIP technology (

13 GridFTP Usage

14 Throughput It had to be fast GridFTP was sold on speed Fast varies with the environment LANs, WANs, Long Fat Pipe Parallel TCP streams, optimal TCP buffer Non TCP protocol such as UDT Striping Multi-node data movement

15 FTP transfer pattern Tradi&onal transfer pa0ern Receiver Data Sender Client

16 Lots of Small Files Large files are easy (but less prevalent) Overhead is low Datasets partitioned into many small files Overlap control overhead with data payload Data channel caching Pipelining Concurrent sessions On-the-fly tar

17 Pipelining Traditional Pipelining File Request 1 File Request 1 DATA 1 ACK 1 DATA 1 File Request 2 File Request 3 ACK 1 File Request 2 DATA 2 ACK 2 File Request 3 DATA 2 ACK 2 DATA 3 ACK 3 DATA 3 ACK 3

18 On-the-fly tar Tar Tar Client GridFTP Server

19 Performance

20 Security GridFTP provides strong security using GSI Protection vs. Ease of use GSI and CAs were hard for many users Speed vs. protection Users area happy with a minimal amount of data channel protection GridFTP over SSH Use SSH credentials for authentication A big win for many users

21 Past success Standards Robustness Throughput Future Security Fire and Forget Firewalls Challenges

22 New features Sync feature GT Size, mod time, checksum Checkpoint across globus-url-copy sessions GT 5.0 Load balancing in globus-url-copy GT 5.0 GridFTP for cygwin GT 5.0 Chroot GridFTP GT 5.0.3

23 Security SSH GridFTP limitations Requires SSH access to GridFTP servers No security on the data channel Myproxy online CA Use login credentials to obtain certificates These certificates typically not trusted elsewhere Incommon?

24 Firewalls DATA GridFTP GridFTP Source TCP 2811 TCP 2811 Dest Server Server Client

25 Fire and Forget transfers Data movement is not a fun activity for users Minimize time spent Reduce user intervention as much as possible Automatic retries on failures Handle server, network and client failures Server provides checkpoint information Support partial transfer

26 Globus.org hosted data movement service

27 Globus.org Enable users to focus on domain-specific work Manage technology failures Notifications of interesting events Provide users with enough information to resolve problems Ease the infrastructure providers support burden Hosted and supported by Globus team

28 Globus.org hosted services: Data replication as an example

29

30

31 More Information at Questions