Building(a(( Fault(Tolerant(Network(

Transcription

1 Building(a(( Fault(Tolerant(Network( Donnie&Grimes,(Vice(President(of(Information(Services( Justin&Hensley,(Support(Administrator,(Department(of(Information(Technology(

2 Diagnosing(the(Problem(! University(of(the(Cumberlands(Department(of(Information( Technology(implemented(a(risk(analysis(to(review(potential( risks(to(technology(! The(results(of(the(analysis(revealed(that(our(datacenter(did( not(contain(enough(redundant(equipment(to(mitigate(the( high(risk(of(downtime(! The(results(also(pointed(out(several(key(areas(in(which(our( datacenter(needed(the(most(attention(

3 A( B( C( Risk(Analysis(Plot(Graph( Two(factor(analysis(that(includes(frequency(of(issues(and(severity((risk)(of(outage(

4 Key(Areas(of(Risk( ISP(Failure( Datacenter( Failure( Risk( Power/ HVAC( Failure( Core( Network( Equipment( Failure(

5 Project(Goals(! Although(we(realized(this(project(would(take(multiple(years,( we(created(the(following(goals(for(completion:(! ISP(redundancy(! Full(power(and(HVAC(redundancy(! Ability(to(quickly(rectify(core(network(hardware(failures(! Offsite(backups(to(cloud(! Offsite(replication(to(site(at(least(2(hours(away(! C0mpliance(with(industry(standard(disaster(recovery(procedures(

6 Mitigation( ISP(Failure(! The(BGP((Border(Gateway(Protoc0l)(solution(was(chosen(to( mitigate(isp(failures(! The(solution(included(the(following(acquisitions:(! Additional(100Mbps(circuit(from(Windstream(to(supplement(the( existing(100mbps(circuit(from(at&t(! Two(Cisco(2900(Series(routers(! Two(Cisco(5500(Series(ASA(firewalls(! Bandwidth(upgrade(for(Exinda(packet(shaping(device(

7 UC(ISP(Infrastructure( UC(utilizes(two(separate(ISP s(with(diverse(paths(for(bgp(routing.(

8 Mitigation( Power/HVAC(Failure(! High(power(UPS(units,(an(enterprise(backup(generator,(and( redundant(hvac(units(were(implemented(to(mitigate(power( and(hvac(failures(! The(solution(included(the(following(acquisitions:(! Enterprise(class(Generac(generator(! All(Datacenter(and(IT(facility(power(on(generator(! APC(220(volt(UPS(backup(systems(for(all(racks(! Datacenter(class(HVAC(unit(with(separate(redundant(unit(

9 Mitigation( Core(Network(Equipment(Failure(! Redundant(core(network(equipment(and(additional(stock( replacement(devices(were(implemented(to(mitigate(core( network(equipment(failures(! The(solution(included(the(following(acquisitions:(! Cisco(Catalyst(4510(with(redundant(supervisor(engines(! Additional(Cisco(Catalyst(backup(chassis(and(line(cards(in(stock(! Two(Cisco(5500(Series(ASA(firewalls(! Two(Exinda(packet(shaping(devices(

10 Mitigation( Datacenter(Failure( (Backups(! Historically,(backups(have(moved(from(diskatoatape(to(diska toadisk(due(to(increasing(backup(file(sizes(! In(order(to(comply(with(disaster(recovery(standards,(the(need( to(store(backups(in(the(cloud(was(realized(! The(solution(included(the(following(acquisitions:(! VEEAM(Backup(and(Replication(with(Cloud(Edition(! BlazePod(storage(units(! Amazon(Glacier(cloud(storage(

11 Cloud(Architecture(in(Veeam( Veeam(utilizes(Amazon(Glacier(to(store(onsite(backups(in(the(cloud(for(further( redundancy.(

12 Mitigation( Datacenter(Failure( (Disaster(Recovery(! University(of(the(Cumberlands(recently(expanded(to(a( remote(campus(in(florence,(ky(! This(campus(and(its(network(meet(our(criteria(for(a(disaster( recovery(site,(therefore(offsite(replication(will(target(the( Florence(location(! The(solution(included(the(following(technology(appropriated( from(main(campus:(! 3(Dell(ESXi(hosts(! HP(Lefthand(storage(units(

13 Dual(Site(Deployment(of(Veeam( Site(A((Main(Campus)(and(Site(B((Florence(KY(Campus)(each(have(their(own( replication(management(server(to(facilitate(full(site(replication.(

14 Conclusion( Issues(and(Results( Issues&! BGP(did(not(function(correctly( the(first(time(due(to(isp(routing( issue(! BGP(is(not(50/50(balanced(due( to(difference(in(tiers(of(isp s(! Generator(was(switched(off( during(a(maintenance(test(and( not(brought(back(online(before( a(power(outage( Results&! BGP(functioned((perfectly( when(an(outage(occurred(! Generator(alarm(panel(was( installed(to(notify(our(staff(of( issues(! Power(blackout(occurred(and( our(it(facility(served(teachers( as(they(accessed(online(class( sessions(