Science Gateways and scalable application tools with QosCosGrid (QCG) for large communities in Grids and Clouds

Transcription

1 Science Gateways and scalable application tools with QosCosGrid (QCG) for large communities in Grids and Clouds Tomasz Piontek, Krzysztof Kurowski and Dawid Szejnfeld [piontek, krzysztof.kurowski,

2 Science Gateways evolution at PSNC Vitrall/DaaS + In-situ vis + AV/AR Gridportlets Java+HTML Vine Toolkit Vine Toolkit Java+HTML+Flash Java+HTML+Flash Gridsphere Java+HTML Liferay Java+HTML Django Django + JS+ Python+HTML5 GRMS QCG middleware QCG & tools QCG & SG will be supported by PSNC at least till 2020

3 PL-Grid domain-specific Science Gateways Domain-specific solutions were created for scientific communities representing the strategic areas and important topics for the Polish and international science: AstroGrid-PL, HEPGrid, Nanotechnologies, Acoustics, Life Science, Quantum Chemistry and Molecular Physics, Ecology, SynchroGrid, Energetics, Bioinformatics, Health, Materials, and Metallurgy. QCG based Science Gateways were used by all marked domain-specific grids. New domain-specific Science Gateway is delivered for Mathematics in scope of PLGrid NG project.

4 PL-Grid statistics [CPUhours] 120, ,000 glite UNICORE 80,000 60,000 40,000 20,000 0

5 PL-Grid statistics with QCG [CPUhours] 3,500,000 3,000,000 Previous slide glite UNICORE QCG 2,500,000 2,000,000 1,500,000 1,000, ,000 0

6 PL-Grid statistics [Users] 1,200 Long tail? 1,000 glite UNICORE QCG

7 QCG (Quality in Clouds & Grids) middleware l l l l l Set of integrated lightweight services and tools for automa@c steering of various types of complex compu@ng experiments in HPC/HTC and Cloud (work in progress) environments. Compu9ng management & advance reserva9on capabili@es (Quality of Service and co- alloca@on of compu9ng and network (BoD) resources) Applica@on progress monitoring and mul@- protocol (SMTP, XMPP, SOAP, JSON) no9fica9ons capabili@es + Python SDKs Various Scien9fic Gateways and web apps supported External in- situ visualiza9on apps available

8 QCG Architecture & open standards JSDL DRMAA 1.0 OGSA-BES HPC-BasicProfile WS-Notification XMPP/Jabber

9 Users tools (Old good command line UIs ;-) Tailored for advanced users and use cases Batch system like look & feel: set of basic commands: qcg-sub, qcg-list, qcg-info, qcgpeek, qcg-cancel job can be described as bash script annotated with #QCG directives

10 Users tools (QCG-Icon & QCG-Icon 2 ) A simple desktop application integrated tightly with the system (systray, context menu, etc.) Platforms: Windows, Mac OSX and Linux Supported applications: MATLAB R Mathematica Gaussian (integrated with Gaussview) GAMESS Molpro plus any bash script! Demo movies:

11 Users Tools (Vine Toolkit with QCG) Job Management, File Management, Credential Manager, SSH Client, Certificate Manager Application, Science Gateways for ABINIT, Quantum Espresso, NAMD, NWChem, nanomd, LAMMPS, SIMPL, Anelli, SpaceGroup Builder, 3D Editor, and more

12 Shows all tasks and reservations in PL-Grid clusters (used data from QCG-Computing ) Allows to see occupancy of all nodes Shows nodes temperatures in the server room view 2 modes: Matrix view to show current values for nodes in server room Gantt view to show reservations and tasks in time. QCG-Monitoring for admins in data center

13 QCG-Monitoring for users

14 PL-Grid Application Database & Catalogue (more at:

15 Vitrall/DaaS Advanced Rendering service Multi-GPU remote rendering service optimized for web sockets/html5 Suport for long-distance connections: complex 3D scene rendered in HD/4K/ 8K quality in real-time with users interactions high interactivity even over long Geant & GLIF (cross-atlantic) network distances images and traffic control via TCP/UDP connections

16 In-situ & VAPOR/ParaView with QCG Visual data discovery environments Tailored towards astro and geosciences CFD community Capable of handling terascale size data sets Advanced 3D Integrated with QCG & Vitrall for remote web in- situ

17 Thank You! Poznań Supercomputing and Networking Center affiliated to the Institute of Bioorganic Chemistry of the Polish Academy of Sciences, ul. Noskowskiego 12/14, Poznań, POLAND, Office: phone center: (+48 61) , fax: (+48 61) ,