DiFX at IRA: cluster, network and dbbc testing

Transcription

1 ISTITUTO DI RADIOASTRONOMIA DiFX at IRA: cluster, network and dbbc testing

2 vlbi-mgr DiFX software server infiniband infiniband infiniband tank-mc tank-nt tank-srt tank-?? Cluster architecture Dedicated storage node for each antenna

3 OS CPU RAM Motherboard 3WARE PCI- Express GT/s RAID Card Scientific Linux 6.4 2x XEON 2.4 GHz 12 GB X8DTH-IF 7x x 9750DR Storage nodes architecture

4 Cluster view Infiniband connection 10 Gbit network 24 disks bay Now 72 TB available

5 correlation time (sec) antennas 3 antennas 4 antennas Correlation benchmark 300 GB per antenna / 5 hrs rec 30 cores 24 min to correlate 3 antennas

6 Antenna Now Mc - 32 mt 10 Gbit/s 10 Gbit/s 10 Gbit/s Nt - 32 mt 1 Gbit/s 10 Gbit/s 10 Gbit/s SRT - 64 mt - 10 Gbit/s 10 Gbit/s IRA - HQ 10 Gbit/s 10 Gbit/s 3 x 10 Gbit/s

7 Juniper POP and Huawei Lambda DWDM devices

8 IPERF transfer tests Bologna -> Metsahövi time data speed latency packets lost lost % 30 sec 22.0 GB 6.28 Gbit/s ms 2399/ (0.16%) Metsahövi -> Bologna time data speed latency packets lost lost % 30 sec 13.8 GB 3.96 Gbit/s ms 38/ (0.0023%)

9 dbbc VSI VSI XFP Fila10 XFP XFP Glapper CX4 SFP+ MARK-5C SFP+ SFP+ SFP+ SFP+ SWITCH Correlator WAN Storage Antenna station network layout as set in Medicina

10 dbbc & Fila10G board upon installation

11 Fila 10G board test Medicina -> Bologna recording : 4 Gbit/s

12 Issues fixed and not fixed

13 Issues fxmanager.cpp & dedisperse_difx.cpp had something missing... #include<string.h> has helped a lot. Thanks Chris and Adam!

14 SIGNS!!! Apparently no antenna follows a directive about delay signs. gps-fmout (or fmout-gps) in antenna logs can differ from what is written in the EVN db As far as we know, Geodesy likes delay sign - whereas Astronomy likes delay sign + Only way to make things work... Try both signs. MC and NT have positive signs on delays (on the.v2d file)

15 En route to fringes First test got wrong - snowing in Medicina Second test cables inverted due to previous observation setup Finally got fringes on April 15th 2013

16 Mk4/DiFX fourfit 3.8 rev 756 Amp. and Phase vs. time for each freq., 60 segs, 2 APs / seg (4.00 sec / seg.), time ticks 10 sec BLLAC.wwdeyt, No0010, MS MEDICINA - NOTO, fgroup C, pol LL Fringe quality 5 Error code G SNR Int time Amp Phase 63.2 PFD 0.0e+00 Delays (us) SBD MBD Fringe rate (Hz) Ion TEC 0.00 Ref freq (MHz) AP (sec) Exp. Exper # Yr:day Start Stop FRT apr : Corr/FF/build 2013:150: :151: :074: RA & Dec (J2000) 22h02m s " U Validity L M S Freq (MHz) Phase Ampl Sbd box U/L 119/ / / /119 APs used M PC freqs S PC freqs M:S -174: : : :152 PC phase M:S 0:0 0:0 0:0 0:0 Manl PC M PC amp S M C03U,C01L 26,28,30,32,10,12,14,16 C07U,C05L 27,29,31,33,11,13,15,17 C11U,C09L 26,28,30,32,10,12,14,16 C15U,C13L 27,29,31,33,11,13,15,17 Chan ids Tracks S C03U,C01L C07U,C05L C11U,C09L C15U,C13L Chan ids 26,28,30,32,10,12,14,16 27,29,31,33,11,13,15,17 26,28,30,32,10,12,14,16 27,29,31,33,11,13,15,17 Tracks Group delay (usec) E+03 Apriori delay (usec) E+03 Resid mbdelay (usec) E-02 Sband delay (usec) E+03 Apriori clock (usec) E+01 Resid sbdelay (usec) E-02 Phase delay (usec) E+03 Apriori clockrate (us/s) E-12 Resid phdelay (usec) E-05 Delay rate (us/s) E-01 Apriori rate (us/s) E-01 Resid rate (us/s) E-07 Total phase (deg) 21.8 Apriori accel (us/s/s) E-06 Resid phase (deg) /- +/- +/- +/- +/- All E E E E RMS Theor. Amplitude / Pcal mode: NORMAL, NORMAL Pcal period (AP s) 9999, 9999 ph/seg (deg) Search (256X16) Pcal rate: 1.301E-08, 8.509E-08 (us/s) sb window (us) amp/seg (%) Interp Bits/sample: 2 SampCntNorm: disabled mb window (us) ph/frq (deg) Inc. seg. avg Sample rate(msamp/s): 16 dr window (ns/s) amp/frq (%) Inc. frq. avg Data rate(mb/s): 256 nlags: 128 t_cohere infinite ion window (TEC) M: az el 36.4 pa 59.6 S: az el 30.4 pa 65.8 u,v (fr/asec) iterative interpolator Control file: default Input file: /space2/fullvlbit/1234/no0010/ms..wwdeyt Output file: Suppressed by test mode

17 dbbc bitstream The vex track assignment differs completely from the MARK-5B format which supposedly takes after. bit-stream (2-33) Astronomy mode 18 1LS 19 1LM 26 5LS 27 5LM 2 1US 3 1UM 10 5US 11 5UM 20 2LS 21 2LM 28 6LS 29 6LM 4 2US 5 2UM 12 6US 13 6UM 22 3LS 23 3LM 30 7LS 31 7LM 6 3US 7 3UM 14 7US 15 7UM 24 4LS 25 4LM 32 8LS 33 8LM 8 4US 9 4UM 16 8US 17 8UM Table 2: dbbc bitstream assignment

18 VDIF tests on dbbc The VDIF header recorded from the Medicina dbbc differs significantly from the one generated with DiFX tool vlbi_fake Tools used to record : NeXPRES s vlbistreamer and DiFX s captureudpvdif Endianess problem! VDIF vlbi_fake \0 \0 \0 \0 * \0 325 f \ \0 \0 \ $ 003 \ t T \ \0 \0 \0 \0 * A A A A VDIF Md \0 \r [ \ \ \0 M d * \ \0 \

19 VSM Vlbi Storage Manager

20 Recording Which tool is best? What is going to be supported?