Science Gateways and scalable application tools with QosCosGrid (QCG) for large communities in Grids and Clouds Tomasz Piontek, Krzysztof Kurowski and Dawid Szejnfeld [piontek, krzysztof.kurowski, dejw]@man.poznan.pl
Science Gateways evolution at PSNC Vitrall/DaaS + In-situ vis + AV/AR Gridportlets Java+HTML Vine Toolkit Vine Toolkit Java+HTML+Flash Java+HTML+Flash Gridsphere Java+HTML Liferay Java+HTML Django Django + JS+ Python+HTML5 GRMS QCG middleware QCG & tools 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 QCG & SG will be supported by PSNC at least till 2020
PL-Grid domain-specific Science Gateways Domain-specific solutions were created for 13 + 11 scientific communities representing the strategic areas and important topics for the Polish and international science: AstroGrid-PL, HEPGrid, Nanotechnologies, Acoustics, Life Science, Quantum Chemistry and Molecular Physics, Ecology, SynchroGrid, Energetics, Bioinformatics, Health, Materials, and Metallurgy. QCG based Science Gateways were used by all marked domain-specific grids. New domain-specific Science Gateway is delivered for Mathematics in scope of PLGrid NG project.
PL-Grid statistics [CPUhours] 120,000 100,000 glite UNICORE 80,000 60,000 40,000 20,000 0
PL-Grid statistics with QCG [CPUhours] 3,500,000 3,000,000 Previous slide glite UNICORE QCG 2,500,000 2,000,000 1,500,000 1,000,000 500,000 0
PL-Grid statistics [Users] 1,200 Long tail? 1,000 glite UNICORE QCG 800 600 400 200 0
QCG (Quality in Clouds & Grids) middleware www.qoscosgrid.org l l l l l Set of integrated lightweight services and tools for automa@c steering of various types of complex compu@ng experiments in HPC/HTC and Cloud (work in progress) environments. Compu9ng management & advance reserva9on capabili@es (Quality of Service and co- alloca@on of compu9ng and network (BoD) resources) Applica@on progress monitoring and mul@- protocol (SMTP, XMPP, SOAP, JSON) no9fica9ons capabili@es + Python SDKs Various Scien9fic Gateways and web apps supported External in- situ visualiza9on apps available
QCG Architecture & open standards JSDL DRMAA 1.0 OGSA-BES HPC-BasicProfile WS-Notification XMPP/Jabber
Users tools (Old good command line UIs ;-) Tailored for advanced users and use cases Batch system like look & feel: set of basic commands: qcg-sub, qcg-list, qcg-info, qcgpeek, qcg-cancel job can be described as bash script annotated with #QCG directives
Users tools (QCG-Icon & QCG-Icon 2 ) A simple desktop application integrated tightly with the system (systray, context menu, etc.) Platforms: Windows, Mac OSX and Linux Supported applications: MATLAB R Mathematica Gaussian (integrated with Gaussview) GAMESS Molpro plus any bash script! Demo movies: http://www.qoscosgrid.org/trac/qcgicon
Users Tools (Vine Toolkit with QCG) Job Management, File Management, Credential Manager, SSH Client, Certificate Manager Application, Science Gateways for ABINIT, Quantum Espresso, NAMD, NWChem, nanomd, LAMMPS, SIMPL, Anelli, SpaceGroup Builder, 3D Editor, and more
Shows all tasks and reservations in PL-Grid clusters (used data from QCG-Computing ) Allows to see occupancy of all nodes Shows nodes temperatures in the server room view 2 modes: Matrix view to show current values for nodes in server room Gantt view to show reservations and tasks in time. QCG-Monitoring for admins in data center
QCG-Monitoring for users
PL-Grid Application Database & Catalogue (more at: https://aplikacje.plgrid.pl)
Vitrall/DaaS Advanced Rendering service Multi-GPU remote rendering service optimized for web sockets/html5 Suport for long-distance connections: complex 3D scene rendered in HD/4K/ 8K quality in real-time with users interactions high interactivity even over long Geant & GLIF (cross-atlantic) network distances images and traffic control via TCP/UDP connections
In-situ & VAPOR/ParaView with QCG Visual data discovery environments Tailored towards astro and geosciences CFD community Capable of handling terascale size data sets Advanced 3D visualiza@on Integrated with QCG & Vitrall for remote web in- situ visualiza@on
Thank You! Poznań Supercomputing and Networking Center affiliated to the Institute of Bioorganic Chemistry of the Polish Academy of Sciences, ul. Noskowskiego 12/14, 61-704 Poznań, POLAND, Office: phone center: (+48 61) 858-20-00, fax: (+48 61) 852-59-54, e-mail: office@man.poznan.pl, http://www.psnc.pl