Home or Cloud? Where To Go for HPC

Transcription

1 Home or Cloud? Where To Go for HPC Gordon Springer, Prasad Calyam MU CI Day, October 10, 2013

2 Scaling Science for Performance: Implementing a Cost-Effective "Big Data" Environment for Research Analysis Gordon K Springer, PhD Scientific Director, UM Bioinformatics Consortium Associate Professor, Computer Science University of Missouri

3 Building Infrastructure Support u The University has built cyberinfrastructure (CI) to support researchers and minimize duplication of expensive resources like computational and storage resources, and advanced networking services. u Building our own data analysis tools as a strategy has served well up until now. u BUT, the explosive increase in infrastructure and analysis demands has led to a re-evaluation of the strategy and possible modification of the approaches.

4 The Dawn of a New Era u Fabric computing" is enabling the University s bioinformatics consortium (UMBC) to scale automated pipelines and alleviate the data management burden associated with traditional high-performance computing architectures. u The environment facilitates research collaborations by enabling scientists to rapidly and cost effectively develop custom pipelines using their preferred analysis tools locally and at other institutions.

5 What s the Problem? u Providing research infrastructure to support increasing numbers of scientists with diverse interests is a major issue. u Data is getting produced at an ever increasing rate and data analysis of the flood of data is increasing faster than the current staff can handle. No additional people have been added to the analysis efforts despite the significant increase in usage and expectations of the researchers.

6 Advantages of Design The MU/OSU Collaboration at 100G u A single instance of the system can span multiple data centers, mirroring and distributing files across all cloud storage servers so that the loss of any one data center does not limit access to data. u We can incrementally scale the capacity of a deployed system by adding new servers and storage, without downtime or loss of file availability. Data replication policies are fully customizable, enabling a high level of fault tolerance, regardless of system configuration.

7

8 Proposed UM 100G CC-NIE Computer (September 2013) [Operational 10G - next month or 100G] MLX16 VDX8770 MLX4 I2 Innovation Fiber (8.8 Terabits)

9

10 Switches in the Infrastructure Brocade MLXe-4 Brocade MLXe-16 Brocade VDX G Ports SDN/OF RSCT1 Rnet Router

11 High Level Layer 2/3 Diagram of MU Science DMZ Ohio State U Texus3 MX-480 Vlans 1751 Vlans 1751 AL2S KC Brocade MLX-E 100gig Vlans 1750, 1751 et-10/2/0 GPN KC MX-960 xe-11/1/0 10 gig Vlans 1750, 1751 Ten 2/6 RSCT M AL2S CLEV Brocade MLX-E AL2S ATLA Brocade MLX-E 100gig Vlan gig vlan 1750 Vlan 1750 GENI Mesoscale Switch Telecom MX-960 vlan 1751 Science DMZ Resources GENI rack

12 Federated Identity can Overcome Multi-institutional Collaboration Barriers

13 SSO Authentication Using Federated Identity Services via InCommon

14 The Story Continues SECURITY DATA RESEARCH & DISCOVERY NETWORK STORAGE COMPUTE

15 Wide-area Overlay ing to Manage Science DMZ Accelerated Flows Prasad Calyam, Ph.D. Assistant Professor, Department of Computer Science October 10th, 2013

16 -as-a-service u u u Need for -as-a-service for inter-institutional collaborations agnostic scientific researcher has to play the role of a network admin! u u Design a Science DMZ architecture that can enable: Deep network programmability according to researcher s requirements with a simple intuitive application dashboard Real-time policy enforcement

17 Science DMZ Concept Normal Application Campus Science Application Instrument Site on Campus Campus Access IP Gatekeeper Proxy Middleware Extended VLAN Overlay Software-Defined Direct Connect Campus Access Web Application Science Application Public Cloud Science Application Remote Collaborator

18 Science DMZ Design at OSU (MU has a mirror!) Two layered network architecture with connectivity to both Layer 3 IP network for public data access and Advanced Layer 2 Service for accelerated big data transfers!

19 OSU-MU Dual-ended Science DMZ Case Study Researcher, Imaging Microscope Lab Instrument Site Campus Access Science Drivers Management Tools Application Layer Gatekeeper Proxy Middleware Extended VLAN Overlay HTC Services Service Engine Measurement Engine Routing Engine Software-Defined Backbone Campus IP Direct Connect Campus Access Amazon Web Services, GENI, Dropbox Public Cloud Researcher, HPC Applications Lab Remote Collaborator End-to-end overlay network based performance acceleration over wide-area networks and across dual-ended Science DMZs

20 Gatekeeper Proxy Middleware maintained by a Performance Engineer Performance Engineer Service Engine 1. Define application end-points and monitoring objectives Authenticated Researcher Gatekeeper Proxy Middleware Routing Engine 2. Provision policy-directed flow rules OpenFlow Controller 3. Install HTC flow 3. Install HTC flow 3. Install measurement flow Measurement Engine Legend: Data Flow Control Flow An intelligent Gatekeeper proxy middleware that automatically programs network based on application requirements and researchers inputs 4. Authorized HTC flow Campus-A Edge Non-IP Campus-B Edge 4. Authorized measurement flow Imaging Microscope Extended VLAN Overlay 4. Non-Science DMZ flow Image Processing Cluster Campus-A Firewall IP Campus-B Firewall

21 Researcher -as-a-service Dashboard u Intuitive application dashboard as a simple user input form to define endpoints and monitoring objectives u Fine-grain control of application requirements as Send Data to Host X with Bandwidth 4 Gbps, Priority 1 and No Delay u Real-time software defined network monitoring of critical network parameters for bottleneck identification and troubleshooting Researcher Federated Identity Login Application Dashboard Real-time Monitoring

22 Ke rne lspac e TCP Streams (Layer3) TCP Soc ket s TCP/IP Reliabili ty Flow Control Congest ioncont rol EthernetDriver EthernetAdapt er Ke rne lspac e TCP Streams (Layer3) TCP Soc ket s TCP/IP Reliabili ty Flow Control Congest ioncont rol EthernetDriver EthernetAdapt er Science DMZ Use Case between MU and OSU Imaging Microscope Render Display Kernel Space TCP Streams TCP Sockets TCP/IP Flow Reliability Control Ethernet Driver Congestion Control ADTS Receiver/ Sender RDMA Support? No Yes User Space RDMA Streams Verbs API RDMA RoCE Adapter TCP/IP Flow Reliability Control Congestion Control Real-time processing of Neuroblastma cell images, has to happen in the order of less than seconds per magnification at OSU and sent back to MU Microscope facility! Ethernet Adapter RoCE Driver IP Non-IP Legend: Raw Image Processed Image MPI Traffic No Yes RDMA Support? ADTS Receiver/ Sender Image Processing Cluster

23 Questions?