AMC13 T1 Rev 2 Preliminary Design Review. E. Hazen Boston University E. Hazen - AMC13 T1 V2 1

Transcription

1 13 T1 Rev 2 Preliminary Design Review E. Hazen Boston University E. Hazen - 13 T1 V2 1

2 Scope of this Review Background: 13 T1 board is being revised to support 10 GbE per request from CDAQ group These boards will be installed during LS1 in HCAL Prefer to keep compatibility with existing T2 For Today: Presentation of schematic-level changes Discussion of open technical issues which may affect T1 Agreement on what needs to be done to proceed with prototyping Not under discussion today: T2 or T3 board design Firmware / Software E. Hazen - 13 T1 V2 2

3 Documentation Please see (Rev2 link at the top) Schematic and layout preview posted N.B. Not all recent changes are included in the preview documents posted E. Hazen - 13 T1 V2 3

4 CMS MicroTCA Crate DAQ optical fibers (5/10 Gb/s) Fiber links from detector (i.e. GBT) TTC / TTS (160.xxx Mb/s) Power 13 Power MCH1 Ethernet Fiber links to trigger E. Hazen - 13 T1 V2 4

5 utca Ports Use for CMS MCH Fabric Port MCH 13 Category CMS Use Tongue 0 Yes GbE A 1 1 Yes Common DAQ [1] 2 Yes Options B 3 Yes Fast controls (TTC) [2] 2 TCLKA CLK1/2 Clock Clocks FCLKA CLK1/2 LHC Clock 4-7 Yes D-G Fat Pipes 8-11 Yes[2] 3, 4 User Extended H-K Fat Pipes Notes: 1. Fabric A (DAQ link) will be operated at 2.5 or 5.0Gb/s in standard 13 firmware This link is AC-coupled, CML level 2. Fabric B (TTC) is a non-mgt link operated at 80Mb/s in standard 13 firmware This link is DC-coupled, LVDS level E. Hazen - 13 T1 V2 5

6 13 Rev 2 Changes Virtex 6 to Kintex 7 for 10G link support Two-way GbE switch removed, GbE to Spartan chip only (Never used Virtex GbE option) SDRAM size increased from 128MB to 512MB speed to 800MHz DDR (1600MT/s * 16 bits) Clock source changes: Rev 1 used 2x Si570 programmable XO Rev 2 uses Si5338B Quad programmable clock generator TTC Recovered clock to clock-capable input on Kintex 7 Changes to T1-T2 board connector pinout Hope to maintain plug compatibility with existing connector Add 1-2 pins for additional 12V to be routed to T3 (agree, pending) There are two diff pairs unused (GPIO pairs) unused Is this sufficient? Furhter enlarging connector further is possible but disruptive, may not be compatible with existing T E. Hazen - 13 T1 V2 6

7 13 Module Rev 2 TTC in TTS out SFP CDS IO IO CLK F/O 40.xx CLK To s DAQ 10 Gb/s DAQ 10 Gb/s SFP+ SFP+ GTX GTX Kintex 7 GTX GTX GTX Fabric A (DAQ) 12 ports Gb/s Spare 10 G b/s SFP+ GTX GTX (2) 128Mx16 DDR3 MCH1 Front Panel via T3 GbE IPMI JTAG LEDs (4x GPIO) MMC uc IO GTP GTP Spartan 6 SPI Flash 1600MT/s (6.4 GB/s) Fabric B 80 Mb/s (TTC) DC LVDS E. Hazen - 13 T1 V2 7

8 SDRAM Update Rev 1: Rev 2: (1) MT41J64M16LA 64M x 16 DDR3 Tested at 1600 MT/s (800 MHz) (2) MT41J128M16JT M x 16 DDR3 Speed rated to 1866 MT/s (933 MHz) Plan to operate at 1600 MT/s (800 MHz) This part promised long-term availability by Micron factory rep E. Hazen - 13 T1 V2 8

9 New clock source E. Hazen - 13 T1 V2 9

10 T1-T2 connectors MMC T2 T1 SPI Bus Switch, LEDs Propose to add 1-2 pins PWR12V Connector can expand this way only GbE to T2 Spare GPIOs (4) 2.5Gb/s T1 T2 link T1 T2 TTC Data 40MHz clk JTAG / Config JTAG / Config E. Hazen - 13 T1 V2 10

11 Clocks: Rev 1 Tongue 1 Tongue 2 SFP 1300nm receiver (ATM type) Compatible with TTC fiber data ADN2814 Clock/data Recovery IC Recovered Clock 160MHz SY89832 Fanout T1 U3 160MHz x 4 Div Reset SY89872 Divide by 2/4 MGT CLK MGT CLK IO_L9 40MHz IO_L10 LVDS SY89832 Fanout T2 U23 DS91M125 1:4 Fanout 40MHz Clock To utca backplane M-LVDS T1 U2 Recovered Data 80 Mb/s Virtex 6 LVDS GCLK Spartan 6 LX130T 80Mb/sec TTC data To utca Backplane (Fabric B) E. Hazen - 13 T1 V2 11

12 Clocks: Rev 2? Tongue 1 Tongue 2 SFP 1300nm receiver (ATM type) Compatible with TTC fiber data ADN2814 Clock/data Recovery IC Recovered Clock 160MHz SY89832 Fanout T1 U3 160MHz x 4 Div Reset SY89872 Divide by 2/4 MGT CLK MGT CLK CC input IO_L10 40MHz LVDS Alternative T3 Clock input (solder SMT jumper) SY89832 Fanout T2 U23 DS91M125 1:4 Fanout 40MHz Clock To utca backplane M-LVDS T1 U2 Recovered Data 80 Mb/s Kintex-7 LVDS GCLK Add clock, TTC signals from T2 to T1 (existing, unused pairs) Spartan 6 LX130T 80Mb/sec TTC data To utca Backplane (Fabric B) E. Hazen - 13 T1 V2 12

13 Design Issue for Discussion... Fabric B is DC driven LVDS by 13 (TTC) Possible damage to FPGAs could result if an card is powered down while drivers are active Power up should be OK as drivers come up tri-stated, so it is the power-down case (for hot swap, i.e) at issue This was pointed out by T. Gorsky some time ago. Xilinx claims it is not a problem, but we would like to be conservative... Options: 1. Change to AC-coupling on, redefine protocol 2. Add discrete LVDS buffer on card* 3. Add series i.e. 10 ohm resistors to limit current on card This does not in principle affect the T1 board rev * my preferred solution E. Hazen - 13 T1 V2 13

14 Layout Snapshot Two 128Mx16 DDR3 SDRAM High-speed T1-T2 connector GbE, MGT Link JTAG / Config SFP (TTC) SFP (10Gb) SFP (10Gb) SFP (10Gb) XC7K325T-2 FPGA Low-speed T1-T2 connector (power, MMC signals) E. Hazen - 13 T1 V2 14

15 Top Components E. Hazen - 13 T1 V2 15

16 Bottom Components E. Hazen - 13 T1 V2 16

17 Other Design Notes MGT clock sources on Kintex-7 TTC 160MHz recovered clock 2 outputs from 5338 Quad clock generator (other two are global logic clock and SDRAM clock) This allows any configuration of link speeds we can conceive of E. Hazen - 13 T1 V2 17

18 Reserve Slides E. Hazen - 13 T1 V2 18

19 13 Rev 1 Hardware Atmel AVR-32 uc MMC Functions Tongue 3 PCB (optional, for initial programming) (4) SFP+ Sites 1 for TTC (160Mb) 3 for DAQ/etc 6.2Gb Tongue 2 PCB Micro USB MMC serial console Spartan 6 FPGA Fabric B TTC distribution Firmware management interface to MMC Virtex-6 LX130T FPGA DAQ Functions, buffering 6Gb links to backplane, SFP Tongue 1 PCB JTAG Headers MMC programming E. Hazen - 13 T1 V2 FPGA programming 19