Advancements in Beamformer and Correlator Optical Backplane. Dr. Grant Hampson 11 th January 2013 URSI Boulder

Transcription

1 Advancements in Beamformer and Correlator Optical Backplane Technology Dr. Grant Hampson 11 th January 2013 URSI Boulder CSIROASTRONOMY AND SPACE SCIENCE

2 Australian Square Kilometre Array Pathfinder Sensitive wide field of view telescope for L Band radio astronomy 1. Array Front of page 36 three axis 12m dishes located in Western Australia 2. Each ASKAP dish overview/specs/photo equipped with a 192 port Phased Array Feed (PAF) 3. Frequency The task range beamformer,correlator,tiedarray, of 700 to 1800MHz with 300MHz cross connects processed BW 4. Existing backplanes (atca, switches, etc.) Each PAF equipped with a 36 beam dual pol 5. beamformer Large Distributed systems large DSP tasks multi rack, big data, large electronics, enabling optical sensitive trend fast sky surveys (30deg 2 FoV) 6. Locations rear air, front IO, sideplane 7. The technology fpgas, avago, molex photos 8. Risks and Challenges of optical backplanes safety, clean, replace 9. Multiple uses same problem, also tied array 10. SKA power, size, cost, RFI, BW 11. Summary and conclusions 12. Thankyou slide 2 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

3 ASKAP Big Data Challenge Two distinct signal processing areas: Antenna contains analog RF signal processing for the PAF RF over Fibre connects the PAF to the central site ( ~ 7000 fibres) Central site building contains all DSP (where RFI shielding is more efficient) 1 1 Phased PAF Antenna Array Receiver Cross Beamformer Cross Feed Connect Connect Correlator Central Site ports antennas Big Data challenge both in communications and processing! 192 PAF Ports, >300MHz dual stage filterbanks, 36xbeams, 666x11GHz correlator, tied Array and many other features ~ 2Petaflops, ~ 300Terabits per second (digital optical comms) Note that data flow is unidirectional 3 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

4 300Tbps Cross Connect Challenge Cross connects can be implemented using fixed PCB routing backplanes (mesh, orthogonal) cabled backplane (copper or optical) 10 packet switched networks (Ethernet, etc.) If for example two Ethernet switches per antenna then 300T/36/2 = ~ 4Tbps per switch (and 36x2=72 required) using 10G ports requires a 400 port switch using 100G ports this is a 40 port switch These are large, expensive and power hungry Alternatively consider an ATCA backplane operating at 3Gbps it has a total BW of ~3Tbps in the same ballpark as the switch and uses significantly less power However it restricts the electronics to one board may not be applicable to large distributed processing systems 4 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

5 Large Distributed Systems g y Trend is growing systems big data has large distributed electronics g g For systems that grow beyond a single processing board Larger processing tasks 1U chassis, subrack, or even multi cabinets Optical trend both analog and digital Leads to optical circuitry! Photos courtesy of Molex 5 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

6 ASKAP Cross Connect Technology ASKAP uses a combination of technologies FPGAs capable of direct 10Gbps digital it dt data communications Unidirectional parallel optics modules (12 x 10Gbps, ribbon fibre) Connectorised cross connect optical circuitry using 12 fibre ribbons 12x12x10G=1.5Tbps Two required to achieve 3Tbps Example smaller 4x4 cross connect each output ribbon contains a fibre from the 4 input ribbons ASKAP Dragonfly3 board ADC then FPGA (filterbank) Optics 6 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

7 Backplane Locations Backplanes typically at rear of processing electronics Cooling drives the design in high performance systems typically air cooled although systems migrating to liquid id e.g., front to rear air cooling A possible solution to avoid blocking air flow Side panel is an ideal out of the way location Sideplane optical circuit Cabinet Front 19 Cabinet Cabinet Side Panel Sideplane Planar Optical Circuit 7 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

8 ASKAP Receiver Receiver 1U chassis houses: 8 dual port RFOF receivers ADC clock synthesizer/distributor Dragonfly 3 processing board Bullant control/monitoring board Cross connect hierarchy Internal to FPGAs Board level between 4 FPGAs Externally via optical cross connect (between FPGAs at destination board) Systems are connected via ribbon fibre Ribbon fibre is much lower cost than single fibre (and breakouts) Parallel optics modules allow electronics to be located anywhere on PCB Compact bulkheads and cabling 8 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

9 Multi use Mesh Routing Original concept for the optical backplane designed for PAF beamformer cross connect However, it was found that the 36 antenna correlator could also use this mesh without modification Beyondthe ASKAP correlator is the tied array hardware (sums beams from each antenna) once again the same optical mesh can be applied Sothe same fixed optical mesh has multiple astronomy uses The SKA telescope may find applications for this as it addresses power, size, cost, RFI, and BW 9 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

10 ASKAP Beamformer Beamformer 1U chassis houses: Redback 3 pocess processing gboard Bullant control/monitoring board No analog modules Redback 3 dual purpose beamformer and correlator Board data routing and signal processing is different External cross connect identical Beamformer Array Covariance Matrix Coarse Delay CRAFT Fine 1 Filter Bank Fringe Beamformer Firmware 1 36 Antennas Antenna Cross Connect Correlator Firmware Correlator Tied Array Long Term Accumlator 10 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

11 Optical Circuit Pros/Cons Feature Description High performance BW scales with FPGA/optics performance optical circuit passive. Next generation FPGAs may directly operate up to 60Gbps Low power Low RFI Low cost Independent Signal integrity Long distance Growth area Cleaning Low thermal generation. Also, VCSEL optical drivers low power. Optics don t emit RFI (termination equipment will though) NRE and production costs reasonable compared to PCB backplanes (l (also short production times) Compatible with other FPGAs and opto electronic vendors. Electronics can be upgraded independent of optical circuitry. Short copper traces between FPGA/optics. Minimal crosstalk. No long varying length backplane traces, or backplane connectors Not limited by PCB losses, MM optics travel several 100metres Optical ribbon technology moving ahead in leaps and bounds Keeping optics clean is a challenge/risk. Safety needs to be managed at 850nm interlocks 11 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

12 Summary Optical circuitry, as well as opto electronics, technology is advancing rapidly and has numerous advantages over copper Enables large systems to be connected efficiently with low risk/cost The speed at which h datarates are going up is much faster than copper backplanes/switches can keep up with ASKAP demonstrates that a fixed mesh can fit multiple astronomy applications Well down the road of advancing PAF technology for SKA cigar planar 12 Optical Backplanes for ASKAP Grant Hampson 11 Jan 2013 URSI Boulder

13 Thank you Dr. Grant Hampson ADE Team Leader t e grant.hampson@csiro.au w Mr. Andrew Brown ADE Design Engineer t e andrew.brown@csiro.au w CSIRO ASTRONOMY AND SPACE SCIENCE