Integration of Virtualized Worker Nodes in Batch Systems
|
|
|
- Thomas Bryant
- 8 years ago
- Views:
Transcription
1 Integration of Virtualized Worker Nodes Dr. A. Scheurer, Dr. V. Büge, O. Oberst, P. Krauß Linuxtag 2010, Berlin, Session: Cloud Computing, Talk ID: #16197 KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association
2 Outline General Motivation High Energy Physics (HEP) / Particle Physics Introduction to Current Experiments Challenges of HEP Computing HPC Cluster Models & Virtualization Motivation & Overview Performance Benchmarks Virtual Cluster Concept Overview Common API libvirt Implementation Conclusion & Outlook Dr. Armin Scheurer
3 What is High Energy Physics? Dr. Armin Scheurer
4 What is High Energy Physics? Dimensions Macrocosm Crystal Molecule Atom 10 1 m 10-2 m 10-9 m m Energy / Frequency Object Size / Wavelength Elementary Particles Proton Nucleus < m m m Dr. Armin Scheurer
5 What is High Energy Physics? Instruments Energy / Frequency Object Size / Wavelength The smaller an object of interest, the larger the necessary energy to resolve Resolution of optical instruments (magnifying glass, microscope) > 10-8 m Other microscopes (electron, force, tunneling) reach scales of ~ m Smaller scales require particle accelerators and detectors. Resolution is only limited by the energy of the accelerated particles. Current HEP detectors ( microscopes ) operate on scales < m Dr. Armin Scheurer
6 Current HEP Experiments Dr. Armin Scheurer
7 The Large Hadron Collider (LHC) ALICE CMS Lake Geneva LHCb ATLAS CERN Proton-Proton Collider Location: CERN Circumference: 27 km Beam Energy: 7 TeV/c² Below Surface: 100 m Temperature: -271 C Energy Use: 1 TWh/a 4 Large Experiments CMS (General-Purpose) Atlas (General-Purpose) LHCb (Physics of b-quarks) Alice (Lead Ion Collisions) 2 Smaller Experiments LHCf and Totem Dr. Armin Scheurer
8 The Compact Muon Solenoid (CMS) Experiment Muon Chambers Hadronic Calorimetry Magnetic Retrun Yoke Technical Details: Total Weigh: Diameter: Total Length: Magnetic Field Solenoid: Yoke: t 15 m 21,5 m 4 Tesla 2 Tesla Silicon Tracker Readout Channels Detector: 100 Mio. Collission Rate: 40 MHz Superconducting Solenoid Electromagnetic Calorimetry Forward Calorimetry Imagine a digital camera with 100 MPixel taking 40 Mio. pictures per second! Dr. Armin Scheurer
9 Pictures of the CMS Detector Dr. Armin Scheurer
10 Collision of Two Proton-Beams Dr. Armin Scheurer
11 Collision of Two Proton-Beams In each proton-proton collision, more than 1000 particles are created. The decay products allow to conclude on underlying physics processes of the collision Dr. Armin Scheurer
12 Collision & Data Rate 60 TB/sec Level 1 Trigger Reduction with ASICs (Hardware) Collision Rate: 40 MHz Event-Size: 1,5 MB 150 GB/sec Tape & HDD Storage for Offline- Analysis 300 MB/sec High Level Trigger Software Data Reduction Recorded Events: 200 per second (1.5 PB / year) Dr. Armin Scheurer
13 What are the computing challenges? Dr. Armin Scheurer
14 LHC / CMS Computing Model Challenges The LHC collaborations are large, e.g. CMS: ~ 40 Nations, > 180 Institutions, > 2000 Scientists and Engineers LHC experiments use distributed computing and storage resources The World-wide LHC Computing Grid (WLCG) User analysis jobs go to where the data is Grid services are based on the glite (and OSG) middleware Computing centres are arranged in a four-tiered hierarchical structure Distributed around the globe Huge datasets must be accessible by thousands of physicists, fast and reliable (ideally 24/7) Grid middleware and experiment specific analysis software is only validated and certified on a dedicated operating system (Scientific Linux)! Dr. Armin Scheurer
15 Peculiarities of HEP Analyses Usual data analysis: (e.g. meteorology, geography, finance) Static program code, dynamic data: Set of fixed programs used to analyze new data in short intervals Same approach for HEP data analysis: Static data, dynamic program code: Data acquisition is very expensive Events independent! Jobs can be processed serially, split & distributed to many machines (trivial parallelisation) Data is analysed repeatedly with iteratively improved code 1000 publications from 500 people with 1TB data Dr. Armin Scheurer
16 Why do we aim for virtualization? Dr. Armin Scheurer
17 HPC Cluster Models Isolated Computing Cluster Each group/institution has sep. cluster Administration overhead Can not cover peak loads Group A Group B Group C Dr. Armin Scheurer
18 HPC Cluster Models Isolated Computing Cluster Each group/institution has sep. cluster Administration overhead Can not cover peak loads Group A Group B Group C Shared Computing Cluster All groups share one cluster Setup compromise not always possible Load-balancing by fair-share Dr. Armin Scheurer
19 HPC Cluster Models Isolated Computing Cluster Each group/institution has sep. cluster Administration overhead Can not cover peak loads Group A Group B Group C Shared Computing Cluster All groups share one cluster Setup compromise not always possible Load-balancing by fair-share Dynamic Partitioned Cluster Configure cluster in real-time with VMs Allows any software/os configuration Virtualization layer hidden Load-balancing by fair-share Dr. Armin Scheurer
20 Is the performance loss acceptable for our purpose? Dr. Armin Scheurer
21 Performance: Benchmarks Synthetic Benchmarks: e.g. SPEC2000, Bonnie++, NetPerf Test CPU, disk I/O, network performance individually Hard to measure correlations VMBench: use standard HEP workflows to monitor the performance VMBench uses well defined default CMS analysis workflows Modularized core, could load several Watchers to get data from different Sources, e.g.: Memory, CPU load Current network speed Performance tests have been done on KVM and XEN virtualized systems Dr. Armin Scheurer
![Running Time (Absolute) Running Time (Relative) Performance: Results (KVM) Performance: Virtual vs. Native [# VMs] 120% 100% 80% 60% 40% 20% 0% 1 2 4 8 Performance: Virtual vs.](/docs-images/29/13847522/images/22-0.png "Native [Workflow Step] 12000 10000 8000 Native Virtual Benchmark runs stable and at good performance: Overall performance loss between 7% to 10% Main reason for performance loss due to parallel")
22 Running Time (Absolute) Running Time (Relative) Performance: Results (KVM) Performance: Virtual vs. Native [# VMs] 120% 100% 80% 60% 40% 20% 0% Performance: Virtual vs. Native [Workflow Step] Native Virtual Benchmark runs stable and at good performance: Overall performance loss between 7% to 10% Main reason for performance loss due to parallel filesystem (Lustre) No paravirtualized Lustre driver available Scalable solution: Host exports Lustre via NFS to guest VMs Native Virtual Performance Loss: OK HEP can benefit from shared clusters with virtualization Dr. Armin Scheurer
23 The ViBatch Concept & Implementation Dr. Armin Scheurer
24 Virtual Cluster Concept: ViBatch Dr. Armin Scheurer
25 Virtual Cluster Concept: ViBatch Dr. Armin Scheurer
26 Virtual Cluster Concept: ViBatch Use prologue & epilogue scripting functionality of standard batch system No modification of batch system Minor configuration changes Tested with: Maui/Torque, SGE Batch system treats VM as user job VM creation/deployment very small compared to running time of standard HEP jobs No virtualized batch client necessary Dr. Armin Scheurer
27 Interoperability of ViBatch ViBatch uses libvirt libvirt is a library providing an API to basically all common virtualizers Command line interface: virsh GUI: virt-manager Provides bindings to: C/C++, Python, Perl, Ruby, Java, C# and others Security aspects: Encrypted (TLS, x509 cert) remote management Authentication with Kerberos & SASL Local access control, PolicyKit libvirt provides everything needed to manage and control a dynamic virtual cluster independent from the used hypervisor Dr. Armin Scheurer
28 Implementation: KIT IC1 Cluster IC1 Cluster Shared between 9 institutions of KIT Very different setup requirements, mostly parallel HPC Without virtualization, no possibility to take part due to restriction on Scientific Linux IC1 Hardware & Software Setup 1600 CPU cores 200 x 8 core Intel Xeon X5355 (VT-x, 64bit) 3200 GB memory 2 GB memory / core 400 TB Lustre filesystem Exported via NFS to VMs Operating system Host: SUSE Linux Enterprise Server 10.2 (Kernel ) Guest: Scientific Linux CERN 5 (patched to Kernel ) Hypervisor KVM Version Dr. Armin Scheurer
29 Implementation: Schema 1. Create VM image 2. Start VM 3. Wait for VM to finish booting (check lockfile) 4. Pipe job to VM 1. Stop VM 2. Delete VM image Simple layout easy to integrate: Prologue & epilogue do VM creation & destruction Can be used besides native cluster functionality Queue determines if native host or which VM is used Dr. Armin Scheurer
30 Implementation: Security Aspects Epilogue & Prologue scripts run as user Root privileges required (start/stop XEN and KVM guests) Necessary scripts in sudoers list To start a job inside a VM, password-less login required Currently each user brings his own SSH key Future plans: Creation of one-time user/host keys Grid usage user@vm user@ce (CE: Grid Compute Element) User can be identified in case of misbehaviour (Grid certificate) No other connections needed/allowed. User has no possibility to run code outside of VM Dr. Armin Scheurer
31 Implementation: Current Status Integrated in existing production system (SLES10) Two VM queues (SLC5 and Debian Lenny) VM templates/images stored on local HDD of worker nodes Automatic image deployment in development Stable operation user jobs Jobs with high I/O, CPU load and network traffic Robustness Simulation of node or batch system crashes ViBatch preliminary code & documentation available on our Trac-server: Dr. Armin Scheurer
32 Outlook Future plans: Automatic image deployment P2P, cluster FS, etc. Scalability? More diffrent VM images and setups User defined images? Parallel usage of different hypervisors on the same cluster Integration of glite Grid middleware Security: Automated creation of WNs with one-time SSH keys (user/host) Dr. Armin Scheurer
33 Conclusion HEP has high demands on computing Distributed resources: CPU and especially storage complicates setup HEP needs validated and certified OS and software environment Native usage of worker nodes in shared clusters is not possible Virtualization offers the possibility to benefit from shared clusters Flexibility Easy administration Security Load-balancing Performance loss small compared to benefit ViBatch Dynamic Cluster Concept Easy to integrate Light-weight & transparent No batch system modification necessary, only pro/epilogue functionality Independent from hypervisor Dr. Armin Scheurer
34 Thank you! ;-) Dr. Armin Scheurer
Betriebssystem-Virtualisierung auf einem Rechencluster am SCC mit heterogenem Anwendungsprofil
Betriebssystem-Virtualisierung auf einem Rechencluster am SCC mit heterogenem Anwendungsprofil Volker Büge 1, Marcel Kunze 2, OIiver Oberst 1,2, Günter Quast 1, Armin Scheurer 1 1) Institut für Experimentelle
Integration of Virtualized Worker Nodes in Standard-Batch-Systems CHEP 2009 Prague Oliver Oberst
Integration of Virtualized Worker Nodes in Standard-Batch-Systems CHEP 2009 Prague Oliver Oberst Outline General Description of Virtualization / Virtualization Solutions Shared HPC Infrastructure Virtualization
Integration of Virtualized Workernodes in Batch Queueing Systems The ViBatch Concept
Integration of Virtualized Workernodes in Batch Queueing Systems, Dr. Armin Scheurer, Oliver Oberst, Prof. Günter Quast INSTITUT FÜR EXPERIMENTELLE KERNPHYSIK FAKULTÄT FÜR PHYSIK KIT University of the
Dynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012
Dynamic Extension of a Virtualized Cluster by using Cloud Resources CHEP 2012 Thomas Hauth,, Günter Quast IEKP KIT University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz
Computing in High- Energy-Physics: How Virtualization meets the Grid
Computing in High- Energy-Physics: How Virtualization meets the Grid Yves Kemp Institut für Experimentelle Kernphysik Universität Karlsruhe Yves Kemp Barcelona, 10/23/2006 Outline: Problems encountered
Virtualization Infrastructure at Karlsruhe
Virtualization Infrastructure at Karlsruhe HEPiX Fall 2007 Volker Buege 1),2), Ariel Garcia 1), Marcus Hardt 1), Fabian Kulla 1),Marcel Kunze 1), Oliver Oberst 1),2), Günter Quast 2), Christophe Saout
Options in Open Source Virtualization and Cloud Computing. Andrew Hadinyoto Republic Polytechnic
Options in Open Source Virtualization and Cloud Computing Andrew Hadinyoto Republic Polytechnic No Virtualization Application Operating System Hardware Virtualization (general) Application Application
Operating Systems Virtualization mechanisms
Operating Systems Virtualization mechanisms René Serral-Gracià Xavier Martorell-Bofill 1 1 Universitat Politècnica de Catalunya (UPC) May 26, 2014 Contents 1 Introduction 2 Hardware Virtualization mechanisms
Virtualization of a Cluster Batch System
Virtualization of a Cluster Batch System Christian Baun, Volker Büge, Benjamin Klein, Jens Mielke, Oliver Oberst and Armin Scheurer Die Kooperation von Cluster Batch System Batch system accepts computational
Solution for private cloud computing
The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details Use cases By scientist By HEP experiment System requirements and installation How to get it? 2 What
Long term analysis in HEP: Use of virtualization and emulation techniques
Long term analysis in HEP: Use of virtualization and emulation techniques Yves Kemp DESY IT First Workshop on Data Preservation and Long Term Analysis in HEP, DESY 26.1.2009 Outline Why virtualization
Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases
NASA Ames NASA Advanced Supercomputing (NAS) Division California, May 24th, 2012 Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases Ignacio M. Llorente Project Director OpenNebula Project.
A quantitative comparison between xen and kvm
Home Search Collections Journals About Contact us My IOPscience A quantitative comparison between xen and kvm This content has been downloaded from IOPscience. Please scroll down to see the full text.
www.see-grid-sci.eu Regional SEE-GRID-SCI Training for Site Administrators Institute of Physics Belgrade March 5-6, 2009
SEE-GRID-SCI Virtualization and Grid Computing with XEN www.see-grid-sci.eu Regional SEE-GRID-SCI Training for Site Administrators Institute of Physics Belgrade March 5-6, 2009 Milan Potocnik University
Module I-7410 Advanced Linux FS-11 Part1: Virtualization with KVM
Bern University of Applied Sciences Engineering and Information Technology Module I-7410 Advanced Linux FS-11 Part1: Virtualization with KVM By Franz Meyer Version 1.0 February 2011 Virtualization Architecture
Virtualization. (and cloud computing at CERN)
Virtualization (and cloud computing at CERN) Ulrich Schwickerath Special thanks: Sebastien Goasguen Belmiro Moreira, Ewan Roche, Romain Wartel See also related presentations: CloudViews2010 conference,
RED HAT ENTERPRISE VIRTUALIZATION
Giuseppe Paterno' Solution Architect Jan 2010 Red Hat Milestones October 1994 Red Hat Linux June 2004 Red Hat Global File System August 2005 Red Hat Certificate System & Dir. Server April 2006 JBoss April
Cloud Computing with Red Hat Solutions. Sivaram Shunmugam Red Hat Asia Pacific Pte Ltd. sivaram@redhat.com
Cloud Computing with Red Hat Solutions Sivaram Shunmugam Red Hat Asia Pacific Pte Ltd sivaram@redhat.com Linux Automation Details Red Hat's Linux Automation strategy for next-generation IT infrastructure
Intro to Virtualization
Cloud@Ceid Seminars Intro to Virtualization Christos Alexakos Computer Engineer, MSc, PhD C. Sysadmin at Pattern Recognition Lab 1 st Seminar 19/3/2014 Contents What is virtualization How it works Hypervisor
Automated deployment of virtualization-based research models of distributed computer systems
Automated deployment of virtualization-based research models of distributed computer systems Andrey Zenzinov Mechanics and mathematics department, Moscow State University Institute of mechanics, Moscow
(Possible) HEP Use Case for NDN. Phil DeMar; Wenji Wu NDNComm (UCLA) Sept. 28, 2015
(Possible) HEP Use Case for NDN Phil DeMar; Wenji Wu NDNComm (UCLA) Sept. 28, 2015 Outline LHC Experiments LHC Computing Models CMS Data Federation & AAA Evolving Computing Models & NDN Summary Phil DeMar:
Virtualization @ Google
Virtualization @ Google Alexander Schreiber Google Switzerland Libre Software Meeting 2012 Geneva, Switzerland, 2012-06-10 Introduction Talk overview Corporate infrastructure Overview Use cases Technology
Pros and Cons of HPC Cloud Computing
CloudStat 211 Pros and Cons of HPC Cloud Computing Nils gentschen Felde Motivation - Idea HPC Cluster HPC Cloud Cluster Management benefits of virtual HPC Dynamical sizing / partitioning Loadbalancing
Enabling Technologies for Distributed Computing
Enabling Technologies for Distributed Computing Dr. Sanjay P. Ahuja, Ph.D. Fidelity National Financial Distinguished Professor of CIS School of Computing, UNF Multi-core CPUs and Multithreading Technologies
Efficient Load Balancing using VM Migration by QEMU-KVM
International Journal of Computer Science and Telecommunications [Volume 5, Issue 8, August 2014] 49 ISSN 2047-3338 Efficient Load Balancing using VM Migration by QEMU-KVM Sharang Telkikar 1, Shreyas Talele
Deploying Business Virtual Appliances on Open Source Cloud Computing
International Journal of Computer Science and Telecommunications [Volume 3, Issue 4, April 2012] 26 ISSN 2047-3338 Deploying Business Virtual Appliances on Open Source Cloud Computing Tran Van Lang 1 and
Full and Para Virtualization
Full and Para Virtualization Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF x86 Hardware Virtualization The x86 architecture offers four levels
CS 356 Lecture 25 and 26 Operating System Security. Spring 2013
CS 356 Lecture 25 and 26 Operating System Security Spring 2013 Review Chapter 1: Basic Concepts and Terminology Chapter 2: Basic Cryptographic Tools Chapter 3 User Authentication Chapter 4 Access Control
Red Hat enterprise virtualization 3.0 feature comparison
Red Hat enterprise virtualization 3.0 feature comparison at a glance Red Hat Enterprise is the first fully open source, enterprise ready virtualization platform Compare the functionality of RHEV to VMware
Developing a dynamic, real-time IT infrastructure with Red Hat integrated virtualization
Developing a dynamic, real-time IT infrastructure with Red Hat integrated virtualization www.redhat.com Table of contents Introduction Page 3 Benefits of virtualization Page 3 Virtualization challenges
Integrating a heterogeneous and shared Linux cluster into grids
Integrating a heterogeneous and shared Linux cluster into grids 1,2 1 1,2 1 V. Büge, U. Felzmann, C. Jung, U. Kerzel, 1 1 1 M. Kreps, G. Quast, A. Vest 1 2 DPG Frühjahrstagung March 28 31, 2006 Dortmund
2972 Linux Options and Best Practices for Scaleup Virtualization
HP Technology Forum & Expo 2009 Produced in cooperation with: 2972 Linux Options and Best Practices for Scaleup Virtualization Thomas Sjolshagen Linux Product Planner June 17 th, 2009 2009 Hewlett-Packard
KVM, OpenStack, and the Open Cloud
KVM, OpenStack, and the Open Cloud Adam Jollans, IBM Southern California Linux Expo February 2015 1 Agenda A Brief History of VirtualizaJon KVM Architecture OpenStack Architecture KVM and OpenStack Case
Virtualization on Linux Using KVM and libvirt. Matt Surico Long Island Linux Users Group 11 March, 2014
Virtualization on Linux Using KVM and libvirt Matt Surico Long Island Linux Users Group 11 March, 2014 What will we talk about? Rationalizing virtualization Virtualization basics How does it work under
NT1: An example for future EISCAT_3D data centre and archiving?
March 10, 2015 1 NT1: An example for future EISCAT_3D data centre and archiving? John White NeIC xx March 10, 2015 2 Introduction High Energy Physics and Computing Worldwide LHC Computing Grid Nordic Tier
GUEST OPERATING SYSTEM BASED PERFORMANCE COMPARISON OF VMWARE AND XEN HYPERVISOR
GUEST OPERATING SYSTEM BASED PERFORMANCE COMPARISON OF VMWARE AND XEN HYPERVISOR ANKIT KUMAR, SAVITA SHIWANI 1 M. Tech Scholar, Software Engineering, Suresh Gyan Vihar University, Rajasthan, India, Email:
Work Environment. David Tur HPC Expert. HPC Users Training September, 18th 2015
Work Environment David Tur HPC Expert HPC Users Training September, 18th 2015 1. Atlas Cluster: Accessing and using resources 2. Software Overview 3. Job Scheduler 1. Accessing Resources DIPC technicians
Enabling Technologies for Distributed and Cloud Computing
Enabling Technologies for Distributed and Cloud Computing Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Multi-core CPUs and Multithreading
Virtualization, Grid, Cloud: Integration Paths for Scientific Computing
Virtualization, Grid, Cloud: Integration Paths for Scientific Computing Or, where and how will my efficient large-scale computing applications be executed? D. Salomoni, INFN Tier-1 Computing Manager Davide.Salomoni@cnaf.infn.it
A Web-based Portal to Access and Manage WNoDeS Virtualized Cloud Resources
A Web-based Portal to Access and Manage WNoDeS Virtualized Cloud Resources Davide Salomoni 1, Daniele Andreotti 1, Luca Cestari 2, Guido Potena 2, Peter Solagna 3 1 INFN-CNAF, Bologna, Italy 2 University
Virtualization. Michael Tsai 2015/06/08
Virtualization Michael Tsai 2015/06/08 What is virtualization? Let s first look at a video from VMware http://bcove.me/x9zhalcl Problems? Low utilization Different needs DNS DHCP Web mail 5% 5% 15% 8%
HTCondor at the RAL Tier-1
HTCondor at the RAL Tier-1 Andrew Lahiff, Alastair Dewhurst, John Kelly, Ian Collier, James Adams STFC Rutherford Appleton Laboratory HTCondor Week 2014 Outline Overview of HTCondor at RAL Monitoring Multi-core
COS 318: Operating Systems. Virtual Machine Monitors
COS 318: Operating Systems Virtual Machine Monitors Kai Li and Andy Bavier Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall13/cos318/ Introduction u Have
Batch and Cloud overview. Andrew McNab University of Manchester GridPP and LHCb
Batch and Cloud overview Andrew McNab University of Manchester GridPP and LHCb Overview Assumptions Batch systems The Grid Pilot Frameworks DIRAC Virtual Machines Vac Vcycle Tier-2 Evolution Containers
Dynamic Load Balancing of Virtual Machines using QEMU-KVM
Dynamic Load Balancing of Virtual Machines using QEMU-KVM Akshay Chandak Krishnakant Jaju Technology, College of Engineering, Pune. Maharashtra, India. Akshay Kanfade Pushkar Lohiya Technology, College
Shoal: IaaS Cloud Cache Publisher
University of Victoria Faculty of Engineering Winter 2013 Work Term Report Shoal: IaaS Cloud Cache Publisher Department of Physics University of Victoria Victoria, BC Mike Chester V00711672 Work Term 3
Virtual Machine Monitors. Dr. Marc E. Fiuczynski Research Scholar Princeton University
Virtual Machine Monitors Dr. Marc E. Fiuczynski Research Scholar Princeton University Introduction Have been around since 1960 s on mainframes used for multitasking Good example VM/370 Have resurfaced
International Journal of Computer & Organization Trends Volume20 Number1 May 2015
Performance Analysis of Various Guest Operating Systems on Ubuntu 14.04 Prof. (Dr.) Viabhakar Pathak 1, Pramod Kumar Ram 2 1 Computer Science and Engineering, Arya College of Engineering, Jaipur, India.
Securing your Virtual Datacenter. Part 1: Preventing, Mitigating Privilege Escalation
Securing your Virtual Datacenter Part 1: Preventing, Mitigating Privilege Escalation Before We Start... Today's discussion is by no means an exhaustive discussion of the security implications of virtualization
Data Centers and Cloud Computing
Data Centers and Cloud Computing CS377 Guest Lecture Tian Guo 1 Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Case Study: Amazon EC2 2 Data Centers
CMS Tier-3 cluster at NISER. Dr. Tania Moulik
CMS Tier-3 cluster at NISER Dr. Tania Moulik What and why? Grid computing is a term referring to the combination of computer resources from multiple administrative domains to reach common goal. Grids tend
Status and Evolution of ATLAS Workload Management System PanDA
Status and Evolution of ATLAS Workload Management System PanDA Univ. of Texas at Arlington GRID 2012, Dubna Outline Overview PanDA design PanDA performance Recent Improvements Future Plans Why PanDA The
Installing & Using KVM with Virtual Machine Manager COSC 495
Installing & Using KVM with Virtual Machine Manager COSC 495 1 Abstract:. There are many different hypervisors and virtualization software available for use. One commonly use hypervisor in the Linux system
Building a Volunteer Cloud
Building a Volunteer Cloud Ben Segal, Predrag Buncic, David Garcia Quintas / CERN Daniel Lombrana Gonzalez / University of Extremadura Artem Harutyunyan / Yerevan Physics Institute Jarno Rantala / Tampere
Introduction to Virtualization & KVM
Introduction to Virtualization & KVM By Zahra Moezkarimi ICT Research Institute Software Platform Laboratory Outline Virtualization History Overview Advantages and Limitations Types of virtualization Virtualization
Science+ Large Hadron Cancer & Frank Wurthwein Virus Hunting
Shared Computing Driving Discovery: From the Large Hadron Collider to Virus Hunting Frank Würthwein Professor of Physics University of California San Diego February 14th, 2015 The Science of the LHC The
Virtualization with Windows
Virtualization with Windows at CERN Juraj Sucik, Emmanuel Ormancey Internet Services Group Agenda Current status of IT-IS group virtualization service Server Self Service New virtualization features in
PARALLELS SERVER 4 BARE METAL README
PARALLELS SERVER 4 BARE METAL README This document provides the first-priority information on Parallels Server 4 Bare Metal and supplements the included documentation. TABLE OF CONTENTS 1 About Parallels
Week Overview. Installing Linux Linux on your Desktop Virtualization Basic Linux system administration
ULI101 Week 06b Week Overview Installing Linux Linux on your Desktop Virtualization Basic Linux system administration Installing Linux Standalone installation Linux is the only OS on the computer Any existing
wu.cloud: Insights Gained from Operating a Private Cloud System
wu.cloud: Insights Gained from Operating a Private Cloud System Stefan Theußl, Institute for Statistics and Mathematics WU Wirtschaftsuniversität Wien March 23, 2011 1 / 14 Introduction In statistics we
Virtualization Performance on SGI UV 2000 using Red Hat Enterprise Linux 6.3 KVM
White Paper Virtualization Performance on SGI UV 2000 using Red Hat Enterprise Linux 6.3 KVM September, 2013 Author Sanhita Sarkar, Director of Engineering, SGI Abstract This paper describes how to implement
NoSQL Performance Test In-Memory Performance Comparison of SequoiaDB, Cassandra, and MongoDB
bankmark UG (haftungsbeschränkt) Bahnhofstraße 1 9432 Passau Germany www.bankmark.de info@bankmark.de T +49 851 25 49 49 F +49 851 25 49 499 NoSQL Performance Test In-Memory Performance Comparison of SequoiaDB,
Virtualization: Know your options on Ubuntu. Nick Barcet. Ubuntu Server Product Manager nick.barcet@canonical.com
Virtualization: Know your options on Ubuntu Nick Barcet Ubuntu Server Product Manager nick.barcet@canonical.com Agenda Defi nitions Host virtualization tools Desktop virtualization tools Ubuntu as a guest
StACC: St Andrews Cloud Computing Co laboratory. A Performance Comparison of Clouds. Amazon EC2 and Ubuntu Enterprise Cloud
StACC: St Andrews Cloud Computing Co laboratory A Performance Comparison of Clouds Amazon EC2 and Ubuntu Enterprise Cloud Jonathan S Ward StACC (pronounced like 'stack') is a research collaboration launched
U-LITE Network Infrastructure
U-LITE: a proposal for scientific computing at LNGS S. Parlati, P. Spinnato, S. Stalio LNGS 13 Sep. 2011 20 years of Scientific Computing at LNGS Early 90s: highly centralized structure based on VMS cluster
SEP Disaster Recovery and Backup Restore: Best
SEP Disaster Recovery and Backup Restore: Best Practices Who We Are Established, HW Development Development of Logistics SW Logistic Software Relag Develop Backup and Data Protection Software -- VMS (92)
Das HappyFace Meta-Monitoring Framework
Das HappyFace Meta-Monitoring Framework B. Berge, M. Heinrich, G. Quast, A. Scheurer, M. Zvada, DPG Frühjahrstagung Karlsruhe, 28. März 1. April 2011 KIT University of the State of Baden-Wuerttemberg and
Big Data and Storage Management at the Large Hadron Collider
Big Data and Storage Management at the Large Hadron Collider Dirk Duellmann CERN IT, Data & Storage Services Accelerating Science and Innovation CERN was founded 1954: 12 European States Science for Peace!
HPC performance applications on Virtual Clusters
Panagiotis Kritikakos EPCC, School of Physics & Astronomy, University of Edinburgh, Scotland - UK pkritika@epcc.ed.ac.uk 4 th IC-SCCE, Athens 7 th July 2010 This work investigates the performance of (Java)
Investigation of storage options for scientific computing on Grid and Cloud facilities
Investigation of storage options for scientific computing on Grid and Cloud facilities Overview Context Test Bed Lustre Evaluation Standard benchmarks Application-based benchmark HEPiX Storage Group report
IaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures
IaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Introduction
Enterprise-Class Virtualization with Open Source Technologies
Enterprise-Class Virtualization with Open Source Technologies Alex Vasilevsky CTO & Founder Virtual Iron Software June 14, 2006 Virtualization Overview Traditional x86 Architecture Each server runs single
Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH
Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH CONTENTS Introduction... 4 System Components... 4 OpenNebula Cloud Management Toolkit... 4 VMware
PES. Batch virtualization and Cloud computing. Part 1: Batch virtualization. Batch virtualization and Cloud computing
Batch virtualization and Cloud computing Batch virtualization and Cloud computing Part 1: Batch virtualization Tony Cass, Sebastien Goasguen, Belmiro Moreira, Ewan Roche, Ulrich Schwickerath, Romain Wartel
Cloud Computing PES. (and virtualization at CERN) Cloud Computing. GridKa School 2011, Karlsruhe. Disclaimer: largely personal view of things
PES Cloud Computing Cloud Computing (and virtualization at CERN) Ulrich Schwickerath et al With special thanks to the many contributors to this presentation! GridKa School 2011, Karlsruhe CERN IT Department
QuickSpecs. HP Integrity Virtual Machines (Integrity VM) Overview. Currently shipping versions:
Currently shipping versions: HP Integrity VM (HP-UX 11i v2 VM Host) v3.5 HP Integrity VM (HP-UX 11i v3 VM Host) v4.1 Integrity Virtual Machines (Integrity VM) is a soft partitioning and virtualization
Virtual Environments for Prototyping Tier-3 Clusters
Virtual Environments for Prototyping Tier-3 Clusters M Mambelli, R Gardner University of Chicago, 5735 S Ellis Ave, Chicago, IL 60637, USA marco@hep.uchicago.edu Abstract. The deployed hierarchy of Tier-1
Solution for private cloud computing
The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details System requirements and installation How to get it? 2 What is CC1? The CC1 system is a complete solution
CERN Computer & Grid Security
CERN Computer & Grid Security Dr. Stefan Lüders (CERN Computer Security Officer) ITU SG17 Tutorials, Geneva, September 5 th 2012 CERN in a Nutshell Tim Berners-Lee Overview CERN s security footprint Operational
IMPLEMENTING GREEN IT
Saint Petersburg State University of Information Technologies, Mechanics and Optics Department of Telecommunication Systems IMPLEMENTING GREEN IT APPROACH FOR TRANSFERRING BIG DATA OVER PARALLEL DATA LINK
HEP Data-Intensive Distributed Cloud Computing System Requirements Specification Document
HEP Data-Intensive Distributed Cloud Computing System Requirements Specification Document CANARIE NEP-101 Project University of Victoria HEP Computing Group December 18, 2013 Version 1.0 1 Revision History
Virtual Switching Without a Hypervisor for a More Secure Cloud
ing Without a for a More Secure Cloud Xin Jin Princeton University Joint work with Eric Keller(UPenn) and Jennifer Rexford(Princeton) 1 Public Cloud Infrastructure Cloud providers offer computing resources
Xen @ Google. Iustin Pop, <iustin@google.com> Google Switzerland. Sponsored by:
Xen @ Google Iustin Pop, Google Switzerland Sponsored by: & & Introduction Talk overview Corporate infrastructure Overview Use cases Technology Open source components Internal components
IOS110. Virtualization 5/27/2014 1
IOS110 Virtualization 5/27/2014 1 Agenda What is Virtualization? Types of Virtualization. Advantages and Disadvantages. Virtualization software Hyper V What is Virtualization? Virtualization Refers to
Basics of Virtualisation
Basics of Virtualisation Volker Büge Institut für Experimentelle Kernphysik Universität Karlsruhe Die Kooperation von The x86 Architecture Why do we need virtualisation? x86 based operating systems are
OGF25/EGEE User Forum Catania, Italy 2 March 2009
OGF25/EGEE User Forum Catania, Italy 2 March 2009 Constantino Vázquez Blanco Javier Fontán Muiños Raúl Sampedro Distributed Systems Architecture Research Group Universidad Complutense de Madrid 1/31 Outline
Virtualization for Cloud Computing
Virtualization for Cloud Computing Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF CLOUD COMPUTING On demand provision of computational resources
HPC and Big Data. EPCC The University of Edinburgh. Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk
HPC and Big Data EPCC The University of Edinburgh Adrian Jackson Technical Architect a.jackson@epcc.ed.ac.uk EPCC Facilities Technology Transfer European Projects HPC Research Visitor Programmes Training
Assignment # 1 (Cloud Computing Security)
Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual
Virtualization Technologies and Blackboard: The Future of Blackboard Software on Multi-Core Technologies
Virtualization Technologies and Blackboard: The Future of Blackboard Software on Multi-Core Technologies Kurt Klemperer, Principal System Performance Engineer kklemperer@blackboard.com Agenda Session Length:
PARALLELS CLOUD SERVER
PARALLELS CLOUD SERVER An Introduction to Operating System Virtualization and Parallels Cloud Server 1 Table of Contents Introduction... 3 Hardware Virtualization... 3 Operating System Virtualization...
Distributed Computing for CEPC. YAN Tian On Behalf of Distributed Computing Group, CC, IHEP for 4 th CEPC Collaboration Meeting, Sep.
Distributed Computing for CEPC YAN Tian On Behalf of Distributed Computing Group, CC, IHEP for 4 th CEPC Collaboration Meeting, Sep. 12-13, 2014 1 Outline Introduction Experience of BES-DIRAC Distributed
Tools and strategies to monitor the ATLAS online computing farm
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Tools and strategies to monitor the ATLAS online computing farm S. Ballestrero 1,2, F. Brasolin 3, G. L. Dârlea 1,4, I. Dumitru 4, D. A. Scannicchio 5, M. S. Twomey
IT of SPIM Data Storage and Compression. EMBO Course - August 27th! Jeff Oegema, Peter Steinbach, Oscar Gonzalez
IT of SPIM Data Storage and Compression EMBO Course - August 27th Jeff Oegema, Peter Steinbach, Oscar Gonzalez 1 Talk Outline Introduction and the IT Team SPIM Data Flow Capture, Compression, and the Data
Cloud Computing through Virtualization and HPC technologies
Cloud Computing through Virtualization and HPC technologies William Lu, Ph.D. 1 Agenda Cloud Computing & HPC A Case of HPC Implementation Application Performance in VM Summary 2 Cloud Computing & HPC HPC
OSG Hadoop is packaged into rpms for SL4, SL5 by Caltech BeStMan, gridftp backend
Hadoop on HEPiX storage test bed at FZK Artem Trunov Karlsruhe Institute of Technology Karlsruhe, Germany KIT The cooperation of Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) www.kit.edu
Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS
Big Data and the Earth Observation and Climate Modelling Communities: JASMIN and CEMS Workshop on the Future of Big Data Management 27-28 June 2013 Philip Kershaw Centre for Environmental Data Archival
With Red Hat Enterprise Virtualization, you can: Take advantage of existing people skills and investments
RED HAT ENTERPRISE VIRTUALIZATION DATASHEET RED HAT ENTERPRISE VIRTUALIZATION AT A GLANCE Provides a complete end-toend enterprise virtualization solution for servers and desktop Provides an on-ramp to
KVM, OpenStack and the Open Cloud SUSECon November 2015
KVM, OpenStack and the Open Cloud SUSECon November 2015 Adam Jollans Program Director, Linux & Open Virtualization Strategy IBM Agenda A Brief History of Virtualization KVM Architecture OpenStack Architecture