Preparing for Next Generation Data Center Infrastructure: Rick Dallmann: Sr. Data Center Architect CABLExpress

Transcription

1 Preparing for Next Generation Data Center Infrastructure: Rick Dallmann: Sr. Data Center Architect

2 Over Thirty Years Data Center Experience RCDDs (Registered Communications Distribution Designer ); DCDCs on staff Subject matter expert presenter at AFCOM, FOCE, BICSI, DCD conferences Expert infrastructure data center design, spec and implementation according to TIA standards Large-scale, high reliability data center specialization ISO 00:00 process and design certified qualification documented and verified by third party Brocade Authorized Partner and Solutions Provider

3 0 & Beyond Challenges 0.ba/m 0/00/00G Ethernet, FCOE, Infiniband Standard // Gigabit Fibre Channel Standards Contradictions..Just meeting the standards will make your infrastructure Obsolete. Tight Loss budgets, Infrastructure management Lead times Design; Product Sets; Exceeding the Standards

4 The Cabling Plant

5 Cabling Methodologies Point-tothe wrong way

6 Point-to-Point wiring is difficult to maintain over time Sub-floor cable archaeology layers develop jumpers become tangled

7 History has shown (hundreds of visited data centers) that loss is based on the design, installation, and products choices In switch and patch archaeology, layers develop This mess results in high losses in jumpers under stress

8 Cable stress loss events The results

9 Infrastructure Standards for Data Centers TIA- In 00, the TIA responded: The TIA/EIA- standard addresses an industry-recognized need for infrastructure requirements that can be applied to any data center, independent of size or usage, including small to large single-tenant enterprise data centers and small to large scale multi-tenant Internet hosting data centers.

10 TIA-: Distributed Data Center Topology Centralized/TOP/EOR/Leaf &Spine Main Distribution Area (MDA) Analogy: Equipment Room Horizontal Distribution Area (HDA) Analogy: Telecom Room Zone Distribution Area (ZDA) Analogy: Consolidation Point Equipment Distribution Area (EDA) Analogy: Work Area Tier Data Center!!!

11 R R A B C D E F G H J K L M FC FC FC FC USB Active CP MGMT SERVICE USB Active CP MGMT SERVICE A B C D E F G H J K L M FC FC FC FC USB Active CP MGMT SERVICE USB Active CP MGMT SERVICE R R A B C D E F G H J K L M R A B C D E F G H J K L M R A B C D E F G H J K L M 0 R A B C D E F G H J K L M R R A B C D E F G H J K L M R A B C D E F G H J K L M R A B C D E F G H J K L M R A B C D E F G H J K L M CB TIA-: Distributed Data Center Topology SAN A() port Cards SAN B() port Cards MTP-MTP Fiber (qty PER U ENCLOSURE) A & B Fabrics.,.,.(x) DA.,.,.(x) DD.,.,.(x) CB.0,.,.(x) CH.0,.,.(x) Skinny Trunk Harness MTP-mLC port QTY per director 0 ()u H-Series Loaded () mtp position Fee-thrus;() blanks Recommended Cable Managers 0 0 FAB A RESERVED Space FAB B RESERVED Space () u Hseries Loaded() mtp-lc port Cassettes ( ports each FAB) A & B Ports To floor CK.0,.,.(x) CQ.0,.,.(x) CT.0,.,.(x) CH.,.,.,.,.,.(x) CQ.,.,.,.,.,.(x) () u Replicater Loaded() port mtp-lc Cassettes () Blanks SAN A & B Replication MTP-MTP port trunks; Connect u to u Replicator QTY per Director NOTE: FAB A AND FAB B Ports will be divided in same cassette module in each u enclosure: IE: MTP & will be FAB A( ports) MTP & will be FAB B( ports) NOTE: all glass is Corning Clear Curve Bend Insensitive Om spec;

12 Cisco MDS 0 Cisco 0 Director Chassis Loaded with Eight -Port Line Cards MDA () Skinny-Trunk MTP-MTP fiber OM/OM B C F G H J M R L K E D OR A B C D R E A *U H-Series Enclosure with ()MTP Feed thru Adapter panels F G H J K U Cable Managers L M R 0U H-Series loaded with () Port LC/MTP modules and () blanks U H-Series loaded with () Port LC/MTP modules ()Skinny-Trunk Fiber Harnesses LC-MTP PSU FAN SUP FAB Cisco MDS 0 IOM u Cable Managers Status ID Status ID Status ID Status ID Status ID Status ID Cisco 0 Director Chassis Eight -Port Line Cards Diagram u design /0 Status ID Status ID PS PS PS PS PS MDS 00 GRID A PS PS SERIES PS GRID B logo U Blank

13 TIA- The right way!!! Skinny-Trunk Harnesses

14 Plan Design Execute

15 Fiber Connector Generations Generation Connector Type Density fibers / inch I ESCON /. I ST-Type /. II SC Duplex /.0 III MT-RJ / 0. III LC Duplex / 0. IV MTP / 0.

16 The Optics OFL (Overfilled Launch) LEDs, not lasers Power distributed over 00% of the fiber core PRE GIG Transceivers EMB (Effective Modal Bandwidth) VCSEL s Power distributed in a narrow region More accurate indication of performance in highspeed laser-based systems (,,,, 0 Gb/s and higher) Different VCELS fill a different set of modes in each fiber, which can affect pulse spreading.

17 Generation IV The MT Ferrule LC Duplex Connector Historically, fiber connectors had discrete ferrules for each fiber. Now, fiber MT ferrules are standard in high density connectors. fiber MT ferrule MTP connectors can have to fibers in one MT (mass termination) ferrule. fiber MT ferrules

18 Fibre Channel Speedmap Product Naming Throughput (MBps) Line Rate (GBAUD) T Spec Technically Completed (Year) Market Availability (Year) GFC 00.0 GFC GFC GFC GFC GFC GFCp 00 x GFC 00 TBD 0 Market Demand GFC 00 TBD 0 Market Demand GFC 00 TBD 0 Market Demand GFC 000 TBD 0 Market Demand TFC 000 TBD 0 Market Demand

19 The 0.ba 0/00G Ethernet Standard Standard 0.ba Standards Document Ratified 0//00 Covers Physical Layer OSI Model Electrical characteristics And much more

20 IEEE 0.bm 00G (revised)/00g Ethernet - Publication Planned for CY 0-00GBase-SR to be released - 00G to be released Translation: transceiver speeds will jump from 0 gigs per fiber or lane to gigs on multimode fiber. fibers X G = 00G. This revision will change the required MPO/MTP connector interface from a fiber to fiber for 00G

21 IEEE 0.bm 00G (revised)/00g Ethernet SFP+ (G to 0G) QSFP-BiDi (0G to 00m on OM) *BiDi cannot breakout into 0G links QSFP+ SR (0G to 00m on OM) QSFP+ CSR (0G to 00m on OM) QSFP+ SR0 (00G to 00m on OM) *00G moving to above f MPO/MTP

22 The MTP ferrule: 0 and 00G For data rates of 0G and 00G Ethernet over OM fiber and OM, parallel optics will be the transmission method & fiber MT ferrules Demand Documentation Infrastructure must be SKEW tested

23 FAIL OK N00-PAC-00W NK-C0-FAN NM- ESW- x x x x x x x x FASTETHERNET PORTS x x x x 0x x x x x 0x x 0x 0/ 00/ 000 EN EXT PWR -V GE BASE TX FAN STAT FAIL OK N00-PAC-00W MGMT ID 0 CONSOLE STAT PSU FAN SUP FAB IOM Cisco Nexus 000 Series 0 NK-MGS- 0 Discrete SERIAL vs //? Discrete 0G VS 0G H-Series u 0GBASE-SR SFP: $ 0GBASE-SR QSFP : $ Cisco 0 MTP/MTP Panel ($)- $ = $(%) Savings 0 Gigabit QSFP+ Module MTP FEMALE MTP MALE MTP FEMALE Skinny Trunk MTP-MTP FEMALE Skinny Trunk Harness MTP MALE-LC PORT Cisco NEXUS 0 GIGABIT Ethernet to () Discrete 0 Gigabit Diagram

24 GFCp Replication Parallel to Serial Conversion u H-series Replicator GFCp MTP Parallel Optics Staggered Trunk Ports of QSFP MTP MTP MTP MTP MTP MTP MTP MTP DCX Duplex Ports of /GFC SFP+

25 The Hard Facts % of system outages are related to cabling AFCOM: Data Center Institute / Gartner estimate % SAN Trouble Tickets are caused by cabling Brocade

26 Performance/Reliability EIA/TIA A&B Industry baseline: EIA/TIA A&B; TIA Makes Testing and Durability recommendations No certification requirement or program Left up to the vendor No required documentation Are you thinking downtime?

27 Performance/Reliability Telcordia GR standards Standards Level requirement of many companies and government agencies in order to supply optics to Requires large monetary investment to become certified Approximately month process, outside lab certification, documented testing, and process line control Less than % of the optics companies are certified

28 OEM World Class Qualification Testing Skinny Trunk brand products are the worlds only that are design to and pass qualification testing equal to the highest level OEM s OEM qualification testing and documentation forces connectors, cabling and enclosures products to go through over exhaustive tests that meet and FAR exceed all industry standards Tensile load, core diameter, compressions and impact tests Ship - Shock Temperature & Humidity Crush Mating repeatability, insertion/withdrawal forces Off axis pull And dozens of other tests. Documentation available Maximum Insertion losses far exceed industry standards with STANDARD PRODUCT

29 A Look at Loss Budget Loss Budget Scary? Year Application Data Rate Standard Loss Budget (db) Ethernet 0 Mbps IEEE 0.. Fast Ethernet 00 Mbps IEEE 0..0 Short Wavelength Fast Ethernet 0/00 Mbps TIA/EIA G Ethernet,000 Mbps IEEE 0.z. 00 &0G FC &0G Ethernet 0,000 Mbps IEEE 0.ae.0 00 G FC & 0G Ethernet 0,000 Mbps IEEE 0.ba G Ethernet 00,000 Mbps IEEE 0.ba.

30 A Look at Loss Budget Insertion loss is a critical performance parameter in current data center cabling deployments. Total connector loss impacts the ability to operate over the maximum supportable distance. As total connector loss increases, the supportable distance at that data rate decreases. /0/00G standard OM ; 00m distance; maximum channel loss of. db, which includes. db total connector loss budget. OM; 0m distance maximum channel loss of. db, which includes.0 db total connector loss budget. The maximum cabled fiber attenuation is. db/km at 0 nm. Thus, the insertion loss specifications of connectivity components should be evaluated when designing data center cabling infrastructures. With low-loss connectivity components.

31 A Look at Loss Budget Ethernet channel lengths over multimode fiber G 0G 0G 00G 00G 0 OM m m OM 0 m m OM 00 m 00 m 00 m 00 m 0m* OM 00 m 0 m 0 m 0 m 0m* Distances in red are specified by manufacturers but not in IEEE standards. *IEEE 0.bm Standard for lane 00G target Q0

32 How can I maintain TIA- structure? The Industry Standard EIA/TIA LC 0.dB/mated pair SC 0.dB/mated pair MTP 0.dB/mated pair

33 TIA-Link Challenges - connection points typical

34 Loss and Attenuation Performance (Standard Product) Clear Curve Bend Insensitive Glass Max Loss LC 0. db/mated pair MTP 0. db/mated pair** **WORLD Industry Leader Rest of World Standard R.O.W pay PREMIUM LC 0.dB LC 0. to.db MTP 0.dB MTP 0. to.0db

35 Demand Documentation Standard Specs

36 Manufacturing Capacity 00% USA Manufacturing,000 sq. ft. facility,000 sq. ft. enclosure/assembly testing 0 enclosures per week (current capacity),000 sq. ft. testing/cable labeling 0,000 sq. ft. Copper/Fiber inventory

37 Skinny-Trunk offerings Reduce Installation Costs CLEAR CURVE GLASS o or - plug factor vs discrete o Modular and scalable o Factory performance and guarantee o Low loss optical components Increase Your R.O.I. o Lower energy usage o Reduce installation labor o High reliability B.I.F Standard for multi-mode and single-mode - Industry first Skinny-Trunk assemblies ensure that your optical cabling system will last through generations of hardware.

38 Brocade ICL Cables Made in the USA Available in Standard, 0, 00 meters and Custom lengths ICL stands for Inter-Chassis Link

39 H-Enclosures Managing Increasing Density o &u o &u o 0&u o ZERO u oladder Rack

40 H- World-leading Density & Flexibility Industry s ONLY port director class switch Replication

41 Skinny-Trunk Harnesses, Blade Specific LC Staggers Skinny-Trunk Harnesses MTP Coupler Faceplates and ZERO-U

42 Product Solutions: Fiber Jumpers Easily managed Field Reversible LC Built on mm zipcord Uniboot design Built on mm round Mini LC Built on mm zipcord Maintain loss budgets Low loss optical components Factory termination Increase your R.O.I. High repeatability Reduce installation labor High reliability All jumpers are not created equal. Ensure uptime, our jumpers are the most reliable in the industry. Side Release clip Uniboot Mini LC

43 MINI-CAT Copper Trunking - Half the diameter (0. inches) of standard Cat - Staggered or Even breakouts - Customizable lengths from -0 feet

44 TAP Modules Network Tap Modules Creates a passive split of fibers for use with active tap hardware Integrates into H or RSD Series footprint space savings MM OM/OM and SM 0/0 or 0/0 split

45 Product Solutions: Clean-Inspect

46 Any Questions???