Campus Clouds: Effec%vely U%lizing Central Storage and Compute Clusters. Dan debeaubien Michigan Technological University

Transcription

1 Campus Clouds: Effec%vely U%lizing Central Storage and Compute Clusters Dan debeaubien Michigan Technological University

2 Background General, IT Org(s), Technical Outline: Brief DescripDon Current Systems Storage, CompuDng, [CollaboraDon] Process of Change Technology, Trust, and Timing Goals

3 Michigan Tech Facts Public University Total Enrollment: 6,550 Graduate Enrollment: Majors Carnegie FoundaDon Doctoral II status 400 Faculty, 1000 staff Ranked programs in Environmental, Mechanical, and Metallurgical Engineering

4 Where? U.P. of Michigan Lots and lots of snow Far from everywhere

5 IT OrganizaDonal EvoluDon We have had many (of course) 80 s: Mainframe (Central CompuDng, Central Network ) Early 90 s: Distributed compudng, in room networks, small centralized coordinadon (uid, dns, , etc.) Mid 90 s: Heavily distributed compudng, emerging central network, small centralized coordinadon

6 Y2k: Totally distributed compudng, strong central network, centralized coordinadon (+policies) Early 00 s: recentralized , central security, datacenter Mid 00 s: Some centralized storage/backup, early VM work Now: Nearly fully centralized storage, strong central compute cloud (also security, compliance, web, , DckeDng, database, directory, CC processing, and a few more systems)

7 MTU IT Org Structure

8

9 Important ObservaDon: Each shib in organizadon was both driven by, and, subsequently caused: Needs changes Technology changes Resource availability and models Campus percepdons (especially)

10 Technically Speaking Early adopters of PC TCP/IP via SUN PC NFS (late 80 s) Heavy SUN influence (3/50 >Sparc >Enterprise >SunFire systems) Around 2000, moved sharply toward Linux (RedHat that is), real datacenters and storage technology.

11 Where we are at (with VM/Storage) Storage systems: Roughly 90% of campus data* is stored on central storage systems. It is all secure, it is all backed up, it is all redundant. Servers: Roughly 80% of campus servers are centralized, 30% are virtualized.

12 Central Storage Cloud Remember: Technology, Trust, and Timing We looked for systems which were just short of Enterprise Class, but were sdll affordable, scalable, and reliable. We worked with a subset of Campus users, big enough to be not purely central but far from University wide. We found a few seed projects and combined their storage needs; AdministraDve systems upgrade, large academic department, central ITSS storage needs ( , courseware, etc.)

13 Storage Cloud In 2003, we built support, took bids, etc. Purchased a EMC CX600, 30 TB or disk in 3 flavors, with 3+1 headed Celerra NAS front end DB access via HBA s NEO head FC LTO2 Tape backup system w/500 tape capacity Redundant storage processors, NAS heads, HBA s, fiber switching and fiber paths Took on all Database, Services, 15 20% of campus home directory space Design tweak in Upgraded to CX700 for faster back end ( driven)

14 Results: Great performance, high availability, data recoverability, data security Many campus players observed: It worked as adverdsed It was very complicated and expensive Reduced storage client s workload We are able to support 24x7 service with an 8x5 staff

15 In other words Our storage clients were able to get out of the storage and backup business completely. Tape backup units were shut off, staff were repurposed, roles changed. The bosom line: Campus costs were significantly reduced.

16 and then Others bought in, storage capacity was increased, momentum was gained Added SATA drives, muldprotocol file system support, central authendcadon In 2007, upgraded to EMC CX3 80, 3+1 NAS (NSX), 80 TB disk, new FC switches CIO proclaimed Free Storage!

17 Next. By 2010, we will house all home directory space, departmental shares, support shares, etc. We will operate the only significant backup system on campus Our model is for centrally funded, liberally granted storage space. It includes radonal quotas (which are essendally unlimited) We are aggressively working on offerings for large research storage requirements

18 Central Compute Cloud By 2005, ITS owned well over 150 dual proc. Intel based Linux servers. The College of Engineering (COE) also had dozens of servers. Two separate virtualizadon efforts begun around 2006, one in ITSS, one in East Engineering. East went live in 2006, and soon took over server support for the endre college on a 7 node VMware cluster. What happened next is interesdng

19 COE compudng support structure West Engineering CompuDng Network Director Customer Service Technical Service Server Support Cent. Engineering CompuDng Network User Support Director User Support Sobware Sobware Customer Service User Config User Config Service Config Technical Service Service Config OS Config OS Config VM/OS Server Support VM/OS Hardware Hardware Datacenters Datacenters East Engineering CompuDng Network Director User Support Sobware Customer Service User Config Technical Service Service Config OS Config Server Support VM/OS Hardware Datacenters

20 COE compudng support structure West Engineering CompuDng Network Cent. Engineering CompuDng Network East Engineering CompuDng Network Director Customer Service Technical Service Server Support Director Customer Service Technical Service Server Support Director Customer Service Technical Service Server Support Datacenters Hardware VM/OS OS Config Service Config User Config Sobware User Support Datacenters Hardware VM/OS OS Config Service Config User Config Sobware User Support Datacenters Hardware VM/OS OS Config Service Config User Config Sobware User Support Director Customer Support Director Technical Services Director Systems Services

21 COE compudng support structure Director Director Director Customer Support Engineering InformaDon Technology Technical Services Systems Services ITSS

22 Cloud CompuDng? New Engineering Group (EIT) is Serverless ITSS took ownership of VM support staff Many appropriate ITSS services migradng to VMs now Cloud offerings to any campus group (no cost or low cost) ITSS now has a wide range of offerings: COLO Virtual Bare metal Pre built Virtual Server Managed Service

23 Current Cloud Architecture Datacenter Support: Redundant UPS, power source, ECUs, etc. Network: ~7 ports/server. Redundant. RSTP (yuck) 2GBx(VM Services + ISCSI + service) Mb drac Storage: Shared EMC CX3 80 [~2TB FC 146GB@15k rpm] Servers: 9 Dell 2950 [3 GHz, Dual Quad Core w/32 GB Ram] So@ware: VMware Virtual Infrastructure 3.5 Enterprise [HA, Dynamic Resource Scheduling, VmoDon] Guest OS: You name it.

24 Summer 2009 (now) Storage: Fully redundant CX4 120 [~8TB SAS rpm] Servers: adding 7 nodes (total of 16) So@ware: VMware vsphere 4.0 Enterprise Plus [adds fault tolerance, host config profiles, storage vmodon, thin provisioning, Hot Add, vnetwork Distributed Switch, 3 rd party storage muldpathing] VMware Site Recovery Manager [Fall 2009] vkernel Capacity Analyzer with cost modeler

25 Goals Move 50% of ITSS servers (~175) to VMs by 2010, 100% by 2011* CriDcal Redundancy now, expand to full offsite redundancy by 2011 Asract further departmental investment from other campus players shut down datacenters Expand into research compudng

26 Results We have eliminated many servers, disk farms, and backup systems, several datacenters, and a few posidons (through asridon); performance, reliability, security and consistency have all been enhanced. We have de silo ed several large organizadons and built expert pools; customers and staff are happier.

27 Conclusion Combined clouds for storage and compudng make a very powerful one two punch for a campus looking for cost savings, but they do require real buy in.

28 QuesDons? Dan debeaubien Director, InformaDon Technology Services and Security Michigan Technological University