Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011
Open Cloud System University Private Cloud is deployed with Cloud Open Source Systems (Enterprise Cloud infrastructure and platform framework ) (Data storage and large data processing framework) (Platform of Google App Engine implementation framework)
Open Cloud System: Purposes To take advantages of benefits of open source technologies to deploy University Private Cloud. To integrate on different cloud vendors such as Google, Amazon, Yahoo into University Private Cloud. To deploy Low Cost Data Center with desktop p y p computers into University Private Cloud.
Overview Architecture of University Private Cloud System Staffs Faculties Researchers Students University Private Cloud HDFS Map/Reduce Hadoop Open Source Open Source Storage Cloud Infrastructure t OS Network Services Eucalyptus Open Source Infrastructure
Private Cloud Infrastructure Elastic Utility Computing Architecture Linking Your rprograms To Useful Systems
Physical Components of University Private Cloud System with Eucalyptus Front-end Servers Layer Back-end dservers Layer Virtualized Servers Layer
Cloud User s s Request to Private Cloud Cloud user request from client nanohub infrastructure Physical Machine Virtual Machine Cloud provider physical servers pool Internal Network nanohub.org Web site NMI Cluster Remote login with SSH Elastic Storage Save user s into data center iscsi Storage Save application data Hadoop DFS
Type of s Provided by Univesity it Private Cloud Instruction Set s (Amazon EC2, Eucalyptus) Ubuntu s CentOS s Windows Server 2003 s Framework s 8
Eucalyptus Components on University Private Cloud
What Is? Google s MapReduce inspired Yahoo s Hadoop. Distributed large data computing framework by Mr. Doug Cutting. For clusters of computers Thousands of Compute Nodes Fault tolerant and scalable storage of very large datasets across machines in a cluster. Now part of Apache group Consists of two components File Store (Hadoop Distributed File System(HDFS)) A distributed processing system (Map/Reduce Model)
HDFS (Hadoop Distributed File System)
Map/Reduce Mode
Hadoop Storage System Specification Name node 4GB (RAM) 320 GB (HDD) 2 Cores CPU Data node 128 MB (RAM) or low 80 GB (HDD) or low
Google App Eingine (GAE) Overview Write user application in Python or Java Test locally Deploy on Google public cloud dinfrastructure t Automatic Scaling Pay-as-you-go Free for limited quotas Pay for additional i scale: CPU, BW, data
Open source GAE Implementation framework on private cloud Distributed and scalable API implementations Infrastructure K/Xen Eucalyptus Amazon EC2 Design and implement by UCBS
and GAE in the Cloud Stack Software-as-a-Service (SaaS) Platform-as-a-Service (PaaS) If Infrastructure-as-a-Service (IaaS) On Public Cloud On Private Cloud
Integrated Components of University Private Cloud System Cloud Client Backbone Network NameNode Data Node Data Node Data Node Data Node Cloud Controller Walrus Storage Linux Linux Data Node Data Node Hadoop Storage System Cluster Controller (K/Xen) Storage Controller 1 Storage Controller 2 Cluster Controller () Windows Windows Node Controller1 Linux Linux Windows Windows Node Controller2 Node Controller3 V Node Controller4 Eucalyptus Private Cloud Infrastructure Node Controller 3 Framework and Platform
Thank you very much