High Performance (Copper) Cable Technology Jay Diepenbrock October, 2013 September, 2013 IEEE 1
Outline What and where are High Performance cables? Cable types Differential links Cable assembly construction Cables and EMI Cable EMI mitigation Measuring EMC properties of Cables References September, 2013 2
High Performance Cables Where? Everywhere What? Big Data servers, networks Ethernet, InfiniBand, SAS, PCI-Express PCs SAS, USB 3.0 Multimedia devices USB 3.0, Thunderbolt TVs, entertainment Coax (!), HDMI September, 2013 3
Aggregate throughput, Gb/s High Performance Cables 350 300 250 200 150 100 I/O Interface Data Rates PCI-Express Gen. 1 PCI-Express Gen. 2 60 50 40 40 30 24 18 12 12 6 6 10 0.012 0.48 4.95 0 5 10.2 0 0.4 4 1990 1995 2000 Year 2005 2010 2015 120 168 80 300 100 128 PCI-Express Gen. 3 SAS 2.1 SAS 3 S-ATA 1.0 S-ATA 2.0 S-ATA 3.0 InfiniBand SDR InfiniBand DDR InfiniBand QDR InfiniBand FDR InfiniBand EDR Thunderbolt USB 1.1 USB 2 USB 3.0 HDMI 1.0 HDMI 1.3 HDMI 1.4 HDMI 2.0 Ethernet (100 Mb) Ethernet (Gb) Ethernet (802.3ba) Ethernet (SFF-8431) Ethernet (802.3bj) September, 2013 4
Cable types Passive or Active Copper or Fiber Bulk wire construction Shielded or not Single or multiconductor + Ground Round or ribbonized Flex Laminated coax Hybrid misc. mixes (signals + power, etc.) Connectors Coax (F, SMA, N) Direct attach multi-pin Paddle card (soldered) multi-pin Backplane style Pluggable transceiver September, 2013 5
Cable Types connector Cu bulk wire connector Passive Half active (Tx or Rx end) connector Cu bulk wire connector connector Cu bulk wire connector Full active connector Optical fiber connector Active Optical = electrical amp or eq. = O/E or E/O converter September, 2013 6
Single-conductor Cable (coax) Construction Many sizes, materials Majority are 50 or 75 Ohms Single signal conductor Dielectric PE, PTFE, etc. Shield (braid or foil+braid) Jacket D dielectric Z 0 = 60 e r D d Applications TV, radio broadcasting Cable TV Commercial, amateur radio Military Cell phones Anything RF (audio?) Center Cond. d shield September, 2013 7
Differential Pair Cables Majority of high speed interfaces now differential On chip, between functional islands Memory On-card I/O Why Differential signaling? Higher system noise margin Power supply voltages decreasing -> lower voltage swing Lower noise immunity (crosstalk) Reduced EMI September, 2013 8
Differential Pair Bulk wire Construction Two signal lines, many geometries Typically 100 Ohms impedance Twisted or parallel pair Dielectric air, PE, PTFE, etc. Shielded (braid or foil+braid) or not Jacketed or not Applications Networking ( Category ) UTP, STP HPC, Supercomputing, I/O (Fibre Channel, PCI-e, SAS, S-ATA, InfiniBand, Ethernet, etc.) Computer storage SAS, S-ATA, USB Consumer HDMI, USB, Thunderbolt September, 2013 9
Twisted Pair bulk wire Inexpensive Various performance grades ( Category 5, 5e, 6, 6a, 7 cables) Some shielded Can be field terminated Susceptible to crosstalk Application Lane speed Ethernet 1-1000 Mb/s # lanes (pairs) Cable Type 4 Cat. 3, 5, 5e UTP* Ethernet 10 Gb/s 4 Cat. 6a STP PCI-e 2.5-16 Gb/s FC, Enet, IB, * 2-32 SPP 2-25 Gb/s 2-24 SPP September, 2013 10
Shielded Parallel Pair ( Twinax ) bulk wire Higher performance than TP Individually Shielded Pairs Various dielectrics - dielectric PE, PTFE, etc. Foil and/or bulk braid shield Outer jacket per application Flammability Abrasion, chemical resistance Applications - I/O, networking (FC, PCI-e, SAS, S-ATA, InfiniBand, Ethernet, etc.) Bulk shield s D d Drain wire September, 2013 11
Shielded Parallel Pair ( Twinax ) Advantages Good performance Low crosstalk Pitfalls Symmetry important Non-uniform materials Geometric structure Common Mode generation Skew System asymmetries Manufacturing good bad September, 2013 12
Shielded Parallel Pair shield topology 0 EXD versus Standard Spiral Shield 24 AWG 100 Ohm -10-20 EXD Spiral 10 meter data, fixture not removed Longitudinal shield -30 SDD21 db / 10 meter -40-50 -60-70 thru fixture EXD 1 EXD 2 EXD 3 EXD 4-80 -90 Optimized for High Frequency 1 Optimized for High Frequency 2 Optimized for High Frequency 3 Optimized for High Frequency 4 Spiral shield -100 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 Frequency MHz September, 2013 13
Quad Construction Four signal lines (two pairs), but smaller Dielectric PE, PTFE Unshielded quad Bulk shield Jacket Applications HPC, Supercomputing Limited usage Expensive, hard to make orthogonality critical to CM, xtalk perf. Hard to terminate 1+ 2+ 2-1- shield September, 2013 14
Connectors SFF-8487 internal mini SAS SFF-8088 external mini SAS SFF-8470 SAS HDMI 7-pin Serial ATA right-angle 7-pin Serial ATA straight SFF-8482 SAS 29 pin w /power September, 2013 15
Connectors PCI-Express x16 (QSFP) SFF-8038 (SFP+) September, 2013 16
Tear-down QSFP (SFF-8088) Screw Cover Shell Spring Latch Base Raw Cable Spacer Insert Molding PCBA Cover September, 2013 17
Wire termination QSFP SFF-8088 (12X InfiniBand) September, 2013 18
Differential Links Each signal transmitted by a pair of conductors, driven Signal conductors Dielectric 180 degrees out of phase + - card wire cable + - Considerations: greater common mode noise immunity than single-ended less EMI radiation than single-ended must consider and measure differential quantities analysis, simulation methods test equipment, fixtures additional propagation modes are possible Drain wire Foil shield September, 2013 19
Differential Impedance Modes" are now possible Case 1 L, C, Z L, C, Z common mode C 11 C 12 C 21 C 22 L 11 L 12 L 21 L 22 L/C September, 2013 20
Differential Impedance Modes" are now possible Case 1 L, C, Z L, C, Z common mode Case 2 L, C, Z L, C, Z differential mode the modes have different impedances, and different propagation delays! C 11 C 12 L 11 L 12 C 21 C 22 L 21 L 22 It's still L/C, but now C= and L= September, 2013 21
Differential Measurements Options Make multiple single-ended measurements and do the math yourself Z 11 Z 12 Z 22 see Carey, Scott, and Weeks: "Characterization of Multiple Parallel Transmission Lines," IEEE Trans. Instr. and Meas., Sept. 1969 Buy differential test equipment, build differential fixtures Differential TDR - measure M1=C1-C2 Four port VNA or two port with external test set - measure sdd21, not s21, and sdd11, not s11 Provides additional information over use of baluns (no common mode data) September, 2013 22
Differential Pair Skew Two types: in-pair (between legs of pair) Due to difference in propagation delay between legs of pair Manifested as "excess attenuation" Spec. limits pretty tight - causes differential imbalance, and can cause EMI problems due to common mode energy not uniform with length! pair to pair (between pairs) difference in propagation delay between pairs modern interfaces relatively insensitive to it (500 ps limit) - it's corrected in the design September, 2013 23
Skew September, 2013 24
Skew Small amounts of skew create significant common mode noise As little as 1% of bit width for skew can have significant EMI effects As little as 10% of bit width skew creates CM signal of equivalent amplitude to initial signals September, 2013 25
Skew 0.6 Individual Channels of Differential Signal with Skew 2 Gb/s with 50 ps Rise and Fall Time (+/- 1.0 volts) 0.4 0.2 Voltage 0-0.2-0.4 Channel 1 No Skew 10 ps 20 ps 50 ps 100 ps 150 ps 200 ps -0.6 5.0E-10 1.0E-09 1.5E-09 2.0E-09 2.5E-09 3.0E-09 Time (seconds) September, 2013 26
Skew Common Mode Voltage on Differential Pair Due to In-Pair Skew 2 Gb/s with 50 ps Rise and Fall Time (+/- 1.0 volts) 0.6 0.4 Amplitude (volts) 0.2 0.0-0.2-0.4 10 ps 20 ps 50 ps 100 ps 150 ps 200 ps -0.6 5.0E-10 1.0E-09 1.5E-09 2.0E-09 2.5E-09 3.0E-09 3.5E-09 4.0E-09 4.5E-09 5.0E-09 Time (seconds) September, 2013 27
Rise/fall time mismatch Small amounts of mismatch create significant CM noise Not as significant as skew, but harder to control! Telltale is significant 2 nd harmonic content September, 2013 28
Rise/fall time mismatch Example of Effect for Differential Signal with Rise/Fall Time Mismatch 2 Gb/s Square Wave (Rise/Fall = 50 & 100 ps) 0.6 0.4 Channel 1 Channel 2 T/R=50/100ps 0.2 Voltage 0-0.2-0.4-0.6 0.0E+00 2.0E-10 4.0E-10 6.0E-10 8.0E-10 1.0E-09 1.2E-09 1.4E-09 1.6E-09 1.8E-09 2.0E-09 Time (Seconds) September, 2013 29
Rise/fall time mismatch 0.2 0.15 Common Mode Voltage on Differential Pair Due to Rise/Fall Time Mismatch 2 Gb/s with Differential Signal +/- 1.0 Volts T/R=50/100ps T/R=50/150ps T/R=50/200ps 0.1 0.05 Level (volts) 0-0.05-0.1-0.15-0.2 0 5E-10 1E-09 1.5E-09 2E-09 2.5E-09 3E-09 3.5E-09 4E-09 4.5E-09 5E-09 Time (seconds) September, 2013 30
Eye opening and Jitter Measures time domain performance of link Measured using PRBS or application-specific data pattern (e. g., CJTPAT) Eye opening - vertical "black space" in middle of many overlaid bits minimum opening needed for receiver to distinguish between "1" and "0" Jitter - horizontal width of zero crossing of overlaid waveforms eye opening jitter September, 2013 31
Eye Opening and Jitter test setup Pattern or BERT Gen. Sampling or real-time oscilloscope Clock PRBS7, 9,..31 pattern Vout ~= 1 Vpp Trise ~= 30 ps xx Gb/s Color-graded display Infinite persistence x Histogram hits Asynch. Crosstalk Source Test card Cable Test card (terminate unused ports with 50 Ohms to Ground) September, 2013 32
Sources of EMI in Cables Skew in system coupled to cable shield, due to Asymmetric differential pairs Unequal rise/fall time of signals Common mode in signals Cable construction Common mode conversion in bulk wire Poor connection from Chassis to Cable plug backshell Leaky backshell Skew in plug/paddle card/bulk wire Poorly shielded bulk wire September, 2013 33
Common mode conversion September, 2013 34
Cable Assembly Construction Influence on EMC Shielding Pair shields foil in or out? Shielded or not? Drain wire handling Bulk shield Foil (high freq.) Braid (low freq.) shield coverage (typ. 80-90%), weave angle, etc. Backshell design Seams, leakage potential Latches, jack screws Grounding Backshell-chassis connection springs, gaskets, drain wires Don t forget the system influence! In-pair skew Mismatched rise/fall tmes Common mode September, 2013 35
Cable EMI sources HDMI cable shield connection From Bergey and Altland, EMI Shielding of Cable Assemblies, DesignCon 2008 September, 2013 36
Cable EMI sources USB cable shield connection (or not!) From Bergey and Altland, EMI Shielding of Cable Assemblies, DesignCon 2008 September, 2013 37
Cable EMI sources From Bergey and Altland, EMI Shielding of Cable Assemblies, DesignCon 2008 September, 2013 38
Measuring Cable EMI Key parameters Transfer Impedance Shielding Effectiveness Measurement methods EM 52022 (CISPR 22) semi-anechoic chamber Tube fixture (IEC 62153-4-7) Measures transfer impedance Max. frequency ~1 GHz Reverb chamber (no standard yet) Measures shielding effectiveness Usable ~300 MHz 20 GHz September, 2013 39
Tube Fixture September, 2013 40
Tube Fixture Sample Results September, 2013 41
Reverb Chamber Closed, conductive-walled room Usable frequency range ~300 MHz-20 GHz, depending on room size and antennae used Don t dampen resonance, celebrate it! CUT is driven with differential or common mode signal, radiated energy is measured No system hardware required Tuner used to stir resonances, either stepped or continuously from external controller Much work on reverb chambers at OK State Univ. (C. Bunting, et. al.) September, 2013 42
Reverb Chamber September, 2013 43
Reverb Chamber Measurement Antennas Tuner CUT Stepper motor CUT support (non-conductive) September, 2013 44
September, 2013 45
EMI Mitigation in Cables Many paths to EMC cleanliness Reduce system in-pair skew Match signal rise/fall times Reduce common mode energy coupling to cable shield Improve cable shield connection to cable backshell Reduce connection inductance Better shield coverage Utilize absorbing material in cable jacket Utilize Band Gap devices on host card September, 2013 46
EMI Absorbing Material Available from ARC Technologies, Inc. for extrusion in cable jacket Molded enclosures (replace metal can) Covers over connectors Frequency selective suppression range depends on formula used Doesn t need to be used on whole cable just ends are enough September, 2013 47
EMI Absorbing Material Motivation - Eliminate Ferrite Cores on Cables September, 2013 48
EMI Absorbing Material Ethernet Cable Emission Reduction (When Drive Signal at Same End of Cable) ARC Lossy Material Covers Partial Length 20 18 Reduction in Emissions (db) 16 14 12 10 8 6 Ethernet Sample #1 w/ 11" Covered Ethernet Sample #1 w/ 23" Covered Ethernet Sample #1 w/ 37" Covered Ethernet Sample #1 Full Cable Covered 4 2 0 0.0E+00 1.0E+09 2.0E+09 3.0E+09 4.0E+09 5.0E+09 6.0E+09 7.0E+09 8.0E+09 9.0E+09 1.0E+10 Frequency (Hz) September, 2013 49
References Diepenbrock, J.: Measurement and Analysis of Shielding Effectiveness and Transfer Impedance of High Speed Data Cables, DesignCon 2012 Archambeault, B., Connor, S., Diepenbrock, J., and Knight, A.: Developing Limits for Common Mode Noise on High Speed Differential Signals, DesignCon 2011 Hill. D.: Electromagnetic Theory of Reverberation Chambers, Natl. Inst. of Standards and Technology Tech Note 1506, 1998 Vignesh Rajamani, Charles F. Bunting and James C. West, Calibration of a Numerically Modeled Reverberation Chamber, IEEE Symposium on Electromagnetic Compatibility 2009 Archambeault, B., Chikando, E., Connor, S., and Diepenbrock, J.: High Speed Cables with Lossy Material Coating, IEEE 2010 Symposium on Electromagnetic Compatibility 2010 September, 2013 50
Standards Other References Code of Federal Regulations Title 47, Telecommunications, part 15 (US) EN 55022, Information Technology Equipment Radio Disturbance Characteristics Limits and Methods of Measurement (Europe) ANSI/EIA/ECA 364-66A EMI Shielding Effectiveness of Electrical Connectors IEC 61000-4-21 Reverb chamber test methods IEC 61276 Screening attenuation measurement by the reverberation chamber method IEC 62153-4-7 Transfer impedance and screening, tube in tube method IEC 62153-4-9 Coupling attenuation of screened balanced cables, triaxial method IEEE 802 InfiniBand Specification, volume 2 PCI-Express Cabling Specification Other Agilent Technologies: Understanding the Fundamental Principles of Vector Network Analysis," AN 1287-1, available at http://www.agilent.com Bogatin, E: "Differential Impedance Finally Made Simple, available at http://www.ewh.ieee.org/r5/denver/rockymountainemc/archive/2000/diffimp.pdf Carey, Scott, and Weeks: "Characterization of Multiple Parallel Transmission Lines," IEEE Trans. Instr. and Meas., Sept. 1969 Deutsch, A., "Electrical Characteristics of Interconnections for High-Performance Systems," IEEE Proceedings vol. 86 No. 2, Feb. 1998 September, 2013 51
Conferences DesignCon February, in Santa Clara, CA IEEE Electrical Performance of Electronic Packaging (EPEP) IEEE EMC Symposium (EMCS) in Raleigh, NC in August, 2014 Embedded SI conference http://www.emcs.org IEEE ECTC, ED, ISSCC IEEE SPI workshop (Europe) 10/30/2013 IEEE 52