Molecular Structure Determination on a Computational & Data Grid

Transcription

1 Molecular Structure Determination on a Computational & Data Grid Mark Green & Russ Miller Center for Computational Research, SUNY-Buffalo Computer Science & Engineering, SUNY-Buffalo Hauptman-Woodward Medical Inst NSF ITR ACI University at Buffalo The State University of New York

2 Biomedical Advances PSA Test (screen for Prostate Cancer) Avonex: Interferon Treatment for Multiple Sclerosis Artificial Blood Nicorette Gum Fetal Viability Test Implantable Pacemaker Edible Vaccine for Hepatitis C Timed-Release Insulin Therapy Anti-Arrythmia Therapy Tarantula venom Direct Methods Structure Determination Listed on Top Ten Algorithms of the 20 th Century Vancomycin Gramacidin A High Throughput Crystallization Method: Patented NIH National Genomics Center: Northeast Consortium Howard Hughes Medical Institute: Center for Genomics & Proteomics

3 Bioinformatics in Buffalo A $360M Initiative New York State: $121M Federal Appropriations: $13M Corporate: $146 Foundation: $15M Grants & Contracts: $64M

4 Bioinformatics Partners Lead Institutions University at Buffalo (UB) Hauptman-Woodward Medical Research Inst. Roswell Park Cancer Institute Corporate Partners Amersham Pharmacia, Beckman Coulter, Bristol Myers Squibb, General Electric, Human Genome Sciences, Immco, Invitrogen, Pfizer Pharmaceutical, Wyeth Lederle, Zeptometrix Dell, HP, SGI, Stryker, Sun AT&T, Sloan Foundation InforMax, Q-Chem, 3M, Veridian BioPharma Ireland, Confederation of Indian Industries

5 Center for Computational Research Snapshot High-Performance Computing and High-End Visualization 110 Research Groups in 27 Depts 13 Local Companies 10 Local Institutions External Funding $111M External Funding $13.5M as lead $97.5M in support $41.8M Vendor Donations $360M Bioinformatics Initiative Deliverables 350+ Publications Software, Media, Algorithms, Consulting, Training, CPU Cycles Raptor Image

6 Major Resources (12TF & 80TB) Dell Linux Cluster: #22 fi #25 fi #38 IBM BladeCenter Cluster 600 P4 Processors (2.4 GHz) 532 P4 Processors (2.8 GHz) 600 GB RAM; 40 TB Disk; Myrinet 5TB SAN Apex Bioinformatics System Dell Linux Cluster: #187 fi #368 fi off 4036 Processors (PIII 1.2 GHz) 2TB RAM; 160TB Disk; 16TB SN Restricted Use (Skolnick) Sun V880 (3), 6800, 280R (2), PIIIs Sun 3960: 7 TB Disk Storage HP/Compaq SAN 75 TB Disk; 190 TB Tape SGI Origin Alpha Processors (400 MHz) 32 GB RAM; 400 GB Disk IBM RS/6000 SP: 78 Processor Sun Cluster: 80 Processors SGI Intel Linux Cluster 150 PIII Processors (1 GHz) Myrinet

7 Visualization Resources Fakespace ImmersaDesk R2 Portable 3D Device Tiled-Display Wall 20 NEC projectors: 15.7M pixels Screen is 11 7 Dell PCs with Myrinet2000 Access Grid Node Group-to-Group Communication Commodity components SGI Reality Center 3300W Dual Barco s on 8 4 screen

8 Network Connections 1000 Mbps 100 Mbps FDDI 1.54 Mbps (T1) - RPCI 44.7 Mbps (T3) - BCOEB 100 Mbps Medical/Dental 155 Mbps (OC-3) I2 OC-3 - I1 BCOEB 1.54 Mbps (T1) - HWI NYSERNet NYSERNet 350 Main St 350 Main St Abilene 622 Mbps (OC-12) Commercial

9 Advanced Data Center (ACDC) Computational Grid Overview Joplin: Compute Cluster 300 Dual Processor 2.4 GHz Intel Xeon RedHat Linux TB Scratch Space Young: Compute Cluster 16 Dual Sun Blades 47 Sun Ultra5 Solaris GB Scratch Space Expanding RedHat, IRIX, Solaris, WINNT, etc Computer Science & Engineering 25 Single Processor Sun Ultra5s Nash: Compute Cluster 75 Dual Processor 1 GHz Pentium III RedHat Linux TB Scratch Space ACDC: Grid Portal 4 Processor Dell GHz Intel Xeon RedHat Linux GB Scratch Space Fogerty: Condor Flock Master 1 Dual Processor 250 MHz IP30 IRIX 6.5 School of Dental Medicine 9 Single Processor Dell P4 Desktops Mama: Compute Cluster 9 Dual Processor 1 GHz Pentium III RedHat Linux GB Scratch Space Crosby: Compute Cluster SGI Origin MHz IP35 IRIX m 360 GB Scratch Space T1 Connection 19 IRIX, RedHat, & WINNT Processors Hauptman-Woodward Institute 13 Various SGI IRIX Processors Note: Network connections are 100 Mbps unless otherwise noted.

10 ACDC Data Grid Overview 182 GB Storage Joplin: Compute Cluster 300 Dual Processor 2.4 GHz Intel Xeon RedHat Linux TB Scratch Space 56 GB Storage Young: Compute Cluster 16 Dual Sun Blades 47 Sun Ultra5 Solaris GB Scratch Space Nash: Compute Cluster 4 Processor Dell GHz Intel Xeon RedHat Linux GB Scratch Space 75 Dual Processor 1 GHz Pentium III RedHat Linux TB Scratch Space ACDC: Grid Portal 100 GB Storage 136 GB Storage Mama: Compute Cluster 9 Dual Processor 1 GHz Pentium III RedHat Linux GB Scratch Space Crosby: Compute Cluster SGI Origin MHz IP35 IRIX m 360 GB Scratch Space 70 GB Storage 100 GB Storage CSE Multi-Store 2 TB Network Attached Storage 480 GB Storage Area Network 75 TB Note: Network connections are 100 Mbps unless otherwise noted.

11 WNY Grid Highlights Heterogeneous Computational & Data Grid Currently in Beta with Shake-and-Bake WNY Release in 2H04 Bottom-Up General Purpose Implementation Ease-of-Use User Tools Administrative Tools Back-End Intelligence Backfill Operations Prediction and Analysis of Resources to Run Jobs (Compute Nodes + Requisite Data)

12 X-Ray Crystallography Objective: Provide a 3-D mapping of the atoms in a crystal. Procedure: 1. Isolate a single crystal. 2. Perform the X-Ray diffraction experiment. 3. Determine molecular structure that agrees with diffration data.

13 X-Ray Data & Corresponding Molecular Structure Underlying atomic arrangement is related to the reflections by a 3-D Fourier transform. FFT Reciprocal Space FFT -1 Real Space X-Ray Data Molecular Structure Phases lost during the crystallographic experiment. Phase Problem: Determine phases of the reflections.

14 Shake-and-Bake Method: Dual-Space Refinement Trial Structures Shake-and-Bake DeTitta, Hauptman, Miller, Weeks Structure Factors Trial Phases Tangent Formula Phase Refinement Parameter Shift FFT FFT -1 Density Modification (Peak Picking) (LDE)? Solutions Reciprocal Space Shake Real Space Bake

15 Ph8755: SnB Histogram Atoms: 74 Phases: 740 Space Group: P1 Triples: 7,400 Rmin range: Trials: 100 Cycles:

16 Phasing and Structure Size Se-Met with Shake-and-Bake? Se-Met 190kDa Multiple Isomorphous Replacement? Shake-and-Bake Conventional Direct Methods Vancomycin ,000 10, ,000 Number of Atoms in Structure

17 Grid-Based SnB Objectives Install Grid-Enabled Version of SnB Job Submission and Monitoring over Internet SnB Output Stored in Database SnB Output Mined through Internet-Based Integrated Querying Tool Serve as Template for Chem-Grid & Bio-Grid Experience with Globus and Related Tools

18 Grid Services and Applications Applications ACDC-Grid Computational Resources Globus Toolkit MPI Shake-and-Bake MPI-IO Apache High-level Services and Tools C, C++, Fortran, PHP MySQL NWS globusrun Oracle Metacomputing Directory Service Core Services Globus Security Interface GRAM GASS ACDC-Grid Data Resources Condor Stork MPI Local Services RedHat Linux WINNT LSF PBS Maui Scheduler TCP UDP Irix Solaris Adapted from Ian Foster and Carl Kesselman

19 ACDC-Grid Portal Login Grid Portal login screen

20 Data Grid Capabilities Browser view of miller group files published by user rappleye

21 Grid Portal Job Status Grid-enabled jobs can be monitored using the Grid Portal web interface dynamically. Charts are based on: total CPU hours, or total jobs, or total runtime. Usage data for: running jobs, or queued jobs. Individual or all resources. Grouped by: group, or user, or queue.

22 Grid Portal Job Status

23 ACDC-Grid Portal User Management Administrator based user based

24 ACDC-Grid Portal Resource Management Administrator grants a user access to ACDC-Grid resources, software, and web pages.

25 Data Grid Resource Info

26 Data Grid Resource Info Both platforms have reduced bandwidth available for additional transfers

27 Data Grid File Age File age, access time, and resource id denote: the amount of time since a file was accessed, when the file was accessed, and where the file currently resides respectively.

28 ACDC-Grid Development/Maintenance Development Requirements 7 Person months for Grid Services Coordinator Including Grid and Database conceptual design and implementation 5 Person months for Grid Services Programmer Web portal programming 5 Person months for System Administrator Globus, NWS, MDS, etc. installations 3 Person months for Database Administrator Grid Portal Database implementation Minimum Maintenance Requirements 1 Grid Services Coordinator 100% level of effort 1 Grid Services Programmer 100% level of effort 1 System Administrator 50% level of effort 1 Database Administrator 10% level of effort

29 Acknowledgments Steve Gallo Jason Rappleye Jeff Tilson Martins Innus Cynthia Cornelius George DeTitta Herb Hauptman Charles Weeks Steve Potter National Science Foundation, National Institutes of Health, Oishei Foundation, Wendt Foundation, Sloan Foundation, Verizon, NYS Gov Pataki, Congressman Reynolds, Senator Clinton, Senator Schumer, Congressman Quinn

30