10G and Beyond in the Data Center with OM3 Fiber

Transcription

1 10G and Beyond in the Data Center with OM3 Fiber Doug Coleman, Manager Technology and Standards

2 Overview Optical Trends in Data Center Networks The time is NOW for 10G optical connectivity 10GBASE-SR compared to 10GBASE-T A 10G Link Connectivity Comparison What s next? Ethernet Fibre Channel Infiniband MTP Connectivity Solutions For Today and the Future Summary

3 Data Center Environment - Today Higher Speeds Higher Density Higher Reliability Lower CAPEX Lower OPEX

4

5 Optical Trends in Data Center Networks Standards have evolved to meet increasing bandwidth requirements with cost-effective solutions Standards Roadmap 10Gb/s required new OM3 fiber specifications and measurement methods Data rate (mb/s) Below 10 Gb/s, application standards used then-current multimode fibers to design network solutions 10M Ethernet Token Ring FDDI M Fast Ethernet ATM LEDs 1G GbE Fiber Channel Lasers 2G FC 10G GbE Sonet/SDH InfiniBand 4G FC 20/40G InfiniBand 8G FC 40/100G IEEE We expect next step in data rates to primarily use current fiber specifications

6 Optical Trends in Data Center Networks Multimode Fiber Nomenclature TIA-568 Rev C adoption

7 Optical Trends in Data Center Networks (=< 300m) Expect lower-cost solutions in data center networks will remain at 850 nm 850 nm VCSELs have won the 1G premises market 850 nm 10G VCSELs just entering high-volume manufacturing cycle Will continue to be low-cost solution for 10G 1300 nm solutions will capture some market share in premises with legacy systems (10GBASE-LX4 and10gbase-lrm) Relative Cost G Transceivers 850 nm optics 1300 nm optics

8 Optical Trends in Data Center Networks 80% 70% 60% 50% 40% 30% 20% OM3 OM2 OM1 SMF 10% 0% Source: CCS

9 Optical Trends in Data Center Networks Laser-Optimized 50 Micron Fiber (OM3) Core Size: Attenuation: Bandwidth: 50 Micron 3.0/ /1300 nm OFL 1500/500 MHz 850/1300 nm EMBc 2000 MHz 850 nm Distance: Gigabit Ethernet 1000/ /1300 nm Serial 10 Gigabit Ethernet: nm CWDM 10 Gigabit Ethernet: nm

10 The Time Is NOW for 10G Optical Connectivity 10 Gigabit Ethernet 10GBASE-S Multimode fiber, serial transmission at 850 nm Lowest cost for new installs (=<300/550 m) Data centers and building/campus backbones 10GBASE-LX4 Multimode or single-mode fiber, WWDM transmission in the 1300 nm region (300 m) Multimode fiber solution intended for legacy systems 10GBASE-L Single-mode fiber, serial transmission at 1300 nm (10 km) Campus backbones 10GBASE-E Single-mode fiber, serial transmission at 1550 nm (30-40 km) Metro area networks

11 The Time Is NOW for 10G Optical Connectivity 10 Gigabit Ethernet 10GBASE-CX4 Twin axial coax cable (15m) 10GBASE-LRM Multimode fiber, EDC, FDDI fiber, 1300 nm, 220 m Multimode fiber solution intended for legacy systems

12 The Time Is NOW for 10G Optical Connectivity!!!!! 10GBASE-S optical connectivity is the choice solution when compared to 10GBASE-T copper connectivity Premier transmission performance Data rate scalability Pathway and space utilization Electronics port density, power and cooling efficiencies Ease of installation and testing

13 The Time Is NOW for 10G Optical Connectivity!!!!! Copper Cable Raw Material Costs Increasing Raw Materials Copper $ 3.86/lb (3/31/08) Hydrocarbon base plastic raw materials $100+ per barrel of oil Copper $$$/pound Copper $$$/pound Drives higher copper connectivity cable $$$$ Optical fiber cable prices trending down 4 day average 9 day average 18 day average

14 The Time Is NOW for 10G Optical Connectivity!!!!! Lower the total cost of network ownership Fiber optic cabling is less expensive to operate Less power consumption Decreased cooling requirements Smaller transceiver size Higher electronic port densities Fiber ~1-4 W 10 Gb/s Operating Cost Fiber v. Copper Power Consumption Cooling Requirements Transceiver Size Copper ~8-15W Some estimates blame up to 60% of all data center downtime on heat-related issues -- IDC $ Data Center Area $$$$

15 The Time Is NOW for 10G Optical Connectivity!!!!! Bandwidth headroom for what s now and what s next Design & Installation Complexity versus Bandwidth Cost and Complexity Bandwidth Demand and Time Copper Fiber Optics Copper cabling Increasingly complex engineering and installation to keep up with bandwidth demands Fiber optic cabling is trending the other way Becoming easier and easier to install while effortlessly keeping up with bandwidth demand

16 The Time Is NOW for 10G Optical Connectivity 10GBASE-T Transmission Issues Operates across 500 MHz frequency spectrum Increased insertion loss Extensive data encoding and signal processing required to achieve acceptable BER Complex electronic digital signal processing (DSP) for internal noise impairments Contributes significantly to inherent time delay known as latency External noise such as alien cross-talk and EMI cannot be corrected with electronics due to random nature

17 The Time Is NOW for 10G Optical Connectivity 10GBASE-T Transmission Issues 10G optical has 1000 times better latency performance than copper Typical 10G optical physical (PHY) latency in the nanosecond range (10-9 sec.) Typical10G copper PHY latency in microseconds (10-6 sec.) A one-millisecond advantage in trading applications can be worth $100 million a year to a major brokerage firm, Information Week, April 2007 Copper s Road LATENCY!!! Fiber s Road

18 The Time Is NOW for 10G Optical Connectivity 10GBASE-T Electronics Issues Significant switch power requirements 10G copper watts per port Major silicon chip development required to reduce power Expect 3-4 watts to be lowest achievable power level per port independent of distance 10G optical switches 1-4 watts per port Significant server adapter card power requirements 10G copper 25 watts per server adapter card 30m service distance 10GBASE-SR optical <9 watts per server adapter card 300m service distance

19 The Time Is NOW for 10G Optical Connectivity 10GBASE-T Power Requirements High power requirements result in higher generated heat, contributing to higher cooling needs EPA states that for every KW of electronics power, an equal unit of power is required for cooling Higher power requirements result in higher CO2 emissions 1.6 lbs CO2 per KW-Hr The result is significantly higher energy costs operation and cooling -- with 10GBASE-T electronics Yearly Energy Cost Total Electronics and Cooling Energy Cost and Savings Comparison for 10GBASE-SR and 10GBASE-T $80,000 $70,000 $60,000 $50,000 $40,000 $30,000 $20,000 $10,000 $0 76% 82% 84% 84% 85% 86% Port Count 87% 85% 83% 81% 79% 77% 75% 73% 71% 69% Copper Switch Fiber Switch % Energy Reduction OPTICAL CONNECTIVITY Enables The Green Data Center!!! Energy Savings

20 The Time Is NOW for 10G Optical Connectivity 10GBASE-T Reduced Switch Port Density Projected 4-8 ports per 10G copper card Maximum 100m distance Low density drives need for more line cards and chassis, driving up power and space utilization 10G optical switch density X2: 16 ports, XFP: 36 ports, SFP+ 48 ports per line card 10GBASE-SR 300m distance One Fiber Line Card = 48 Ports Six Copper Line Cards = 48 Ports

21 The Time Is NOW for 10G Optical Connectivity Emerging Optical Electronics 10G Transceiver Trends Move signal processing from module to the line card Reduce module size Reduce power Reduce price Source: Intel Source: CMP Media XENPAK & X2 XFP SFP+

22 The Time Is NOW for 10G Optical Connectivity 10GBASE-T Pathway and Space Issues Larger cable OD and heavier 1 CAT6A UTP inch diameter and 46 lbs/1000ft weight A 216-fiber ribbon cable, 0.76 inch OD, 200 lbs/1000ft 108 circuits at 200ft 108 CAT6A cables, 1000lbs, typical effective diameter 5.0 inch 216-fiber optical cable, 40 lbs, effective diameter 0.76 inch

23 Let s Build a Link! Scenario: CAT6A UTP Copper and 10G Electronics Scenario: G Copper Ports This will require 6U of rack space per rack dedicated to patch panels 14 copper 10G line cards per rack, with a maximum port line card density of 8 Next, 108 jumpers per rack And finally, 108 CAT6A UTP cables to connect it all (at 200 ft, that s 1000 lbs!) Cross-section of 12 x4 cable tray (to scale) 40% filled

24 Now a 10G Optical Link! Scenario: OM3 Fiber and 10G Electronics Scenario: G Optical Ports This will require 1U of rack space per rack dedicated to patch panels 3 optical 10G line cards per rack with a maximum port line card density of 48 Next, 18 MTP -to-single-fiber connector harnesses per rack And finally, fiber cable to connect it all (at 200 ft, that s 40 lbs!) Cross section of 12 x4 cable tray (to scale) 2% filled

25 Let s Compare: 10G Copper UTP Network v. 10G Optical Network CAT6A UTP 10G Network Electronics operating power OM3 10G Network Electronics operating power 96,500 kw-hr per year 20,100 kw-hr per year $14,184 per year / rack $2,950 per year / rack Electronics cooling power Electronics cooling power 96,500 kw-hr per year 20,100 kw-hr per year $14,184 per year / rack $2,950 per year / rack Total power cost Total power cost $28,369 per year / rack $5,910 per year / rack Total CO2 emissions Total CO2 emissions 155 tons 32 tons 79% overall energy savings!

26 Ethernet What s Next? IEEE HSSG Approved Motions 40 and 100 Gbps At least 100 m on OM3 multimode fiber At least 10 km on single-mode fiber At least 40 km on single-mode fiber (100G only) At least 10 m on copper cable assembly Key project dates Study group formed in July 2006 Project authorization in December 2007 Task force formed in January G standard complete mid 2010

27 Ethernet What s Next? Recent history suggests that standards (and initial fiber sales) will lead actual port sales by ~3 years Given port sale historical trends, we can project initial applications ~2011 Most applications not expected until >2013 Ethernet and LAN Port Sales (MM units) Mb/s Standard 100Mb/s Standard 1Gb/s Standard Gb/s Standard Year Gb/s Standard '100Gbps' 10Gbps 1Gbps 100Mbps 10Mbps 10Mb-10Gb data from Dell Oro

28 Premises Cable Market Length Distribution of Cable Supplied to Customers % Relative Frequency % 60% 40% 20% Cumulative Frequency Cable Length(m) 0% Length Distribution Cumulative Frequency

29 Ethernet What s Next? Several possible optical solutions are all currently being discussed in IEEE Parallel CWDM DSP Hybrid MTP Optical Connectivity

30 40G Ethernet Parallel Optics 12F MTP Interface

31 QSFP 40G Optical Transceiver Uses standard ribbon fiber cable with MTP Connector Source: Zarlink

32 100G Ethernet Parallel Optics 24F MTP Or Two 12F MTP Interface Source: Sumitomo

33 Fiber Skew Certain optical cables and terminations may not be suitable for parallel optics applications IEEE 40/100G skew requirement not defined to date. InfiniBand QDR 0.75ns skew requirement now exists CCS has performed skew testing demonstration compliance to 0.75 ns up to 300m Need to understand skew capability today for existing and future cable deployments Skew = difference in propagation time between lanes of a parallel transmission system. Δ propagation speed or distance Infiniband Skew Requirements:

34 SAN Fibre Channel Road Map Product Naming Throughput (MBps) Line Rate (GBaud) T11 Spec Technically Completed (Year) Market Availability (Year) Base2 Parallel Optics 1GFC GFC GFC GFC GFC GFC GFC GFC Market Demand Market Demand Market Demand

35 SAN Fibre Channel Road Map Product Naming Throughput (MBps) Line Rate (GBaud) T11 Spec Technically Completed (Year) Market Availability (Year) Base10 10GFC GFC GFC TBD Market Demand Parallel Optics 80GFC TBD Market Demand 160GFC TBD Market Demand

36 Fiber Channel over Ethernet (FCoE) Activity initiated at T11 Fiber Channel, April 2007 Encapsulate Fiber Channel Packet into a Ethernet Frame Lossless Packet Performance Supports utilization of low cost Ethernet electronics up to the SAN switch FCoE Server Adapter Card FCoE Gateway Line Card Designed to operate at 10G Large data centers focus Standard completion mid-2008 Commercial products 2009

37 InfiniBand Applications HPC Supercomputers Financial data center focus Electronic trading and algorithm modeling High BW with low latency Media Types Optical fiber Multimode and single-mode fiber MTP Connectivity Twin axial copper cable Factory-terminated only 15-20m distance capability Not adequate for structured wiring

38 InfiniBand Architecture: Server Area Network (HPC)

39 InfiniBand Data Rates Parallel Optics!!!

40 InfiniBand Electronics

41 Optical Media Converter Copper to optical media converter (OMC) Converts a power-enabled InfiniBand copper port to an optical port Uses standard ribbon fiber cable with MTP Connector Data rate specific (4X-SDR,DDR) Perfect match to fixed-port CX4 socket Emcore OMC Active optical cable (Zarlink, Intel) Integrated media converter Specified distances (10, 25, m) Data rate specific (4X-SDR,DDR) Perfect match to fixed-port CX4 socket Zarlink

42 QSFP Optical Transceiver 2008 Hot pluggable 12 Fiber MTP Interface Data rate specific (4X SDR, DDR, QDR (10G, 20G, 40G)) Zarlink, Luxtera, QLOGIC and other manufacturers Source: Zarlink

43 MTP Connectivity Solutions in the Data Center Server Racks Server Main Distribution Area (MDA) Server Ports Switch Ports Storage Switch Racks Storage Ports

44 Ribbon Cable High-density ribbon design in a small-form-factor package 12-fiber ribbons Data center and LAN backbones MTP connectivity 12 fibers connectorized simultaneously Riser and plenum flame ratings Interlocking armor 96-Fiber Cable with eight MTP Connectors

45 MTP Connectorization Factory or field termination Terminate 12 fibers in one step with MTP connectors Expedites cable installation and MACs Minimizes errors Reduces congestion in patch panels Footprint similar to SC Connector

46 Ribbon Cable Field Termination No-Epoxy/No-Polish Connector TIA/EIA (FOCIS) Installs in less than 4 minutes Fiber type OM1 62.5/125 µm (beige housing) OM2 50/125 µm (black housing) OM3 LOMMF (aqua housing) SMF OS2 (green housing)

47 MTP Connector Modules Used to break out the 12-fiber MTP connectors terminated on trunk cables into simplex or duplex style connectors SC, LC, ST compatible MDA cross-connect and EDA interconnect Support easy migration to parallel optics

48 Main Distribution Area Housing Back Panel PnP training4

49 Main Distribution Area Housing Front Panel LAN490

50 MTP Connector Harnesses Hardware interconnection to backbone cable Break out 12-fiber MTP connector into simplex or duplex connectors Transitions plug & play system MTP connector trunks straight into electronics Support easy migration to parallel optics

51 MTP Connector Harness SAN Director Termination

52 The Time Is NOW for Optical Connectivity Silicon and electronic industry focused on optical solutions 10GBASE-SR OM3 fiber optical connectivity is the choice solution when compared to 10GBASE-T copper connectivity OM3 fiber supports migration to established Fibre Channel, Ethernet, and InfiniBand roadmap high data rates MTP Connectivity Solutions available today for legacy serial and future parallel optics transmission schemes 1G, 10G, 16G 40G 100G

53 Contact Info Doug Coleman Phone: Fax: Address: th Street NW Hickory, NC 28601