CAIM Correlator Monitoring Server

Transcription

1 CAIM Correlator Monitoring Server Roelof Anne Schoenmaker, January Introduction & motivation Several correlators are currently in operation or under development at JIVE, the Join Institute for VLBI in Europe. As the number of correlator hours at JIVE is expected to increase significantly in the next years, unattended operation of these correlators will have to be implemented. To enable this, correlator monitoring software has been developed as part of the CAIM system (described in the Continuous Automated Intelligent Monitoring system overview document by Eldering and Schoenmaker). The CAIM Correlator Monitoring Server software uses modules for the different correlators and monitoring parameters. The current software contains modules for capturing output data from both hardware and software correlators at JIVE. Two monitoring modules are also available, namely a weight and a fringe monitoring module. Using modules has the advantage that new correlators or different monitoring parameters can be added readily to the server software. For communication between a correlator and monitoring modules, a generic communication-protocol has been designed. For both module types, an abstract class has been created. 2. Software fundamentals & communication The modular built CAIM Correlator Monitoring Server software contains three obligatory classes, all written in the programming language C++. These three are: Server class: checks at regular intervals, if new experiments should be monitored. When an experiment is ready to be monitored (for example, when an e-vlbi experiment starts), this class will set-up the corresponding correlator and monitoring modules and their intercommunication. Multiple experiments running on one or more correlators can be monitored simultaneously. When an experiment has finished, this class will terminate the monitoring process. Correlator abstract class: contains the necessary methods, which are needed to establish a generic communication with the server and monitor classes. This class dictates also the datastructure, which is used between one correlator and one or more monitoring modules. Monitor abstract class: contains the necessary methods, which are needed to establish a generic communication with the server and correlator classes. Both internal and external communications are handled by this class. Currently the results are sent to a network monitoring tool, but other output format can be easily implemented. All above classes and derived modules are in fact threads, making it possible to monitor multiple experiments parallel. Nevertheless, multi-threading must be implemented with care. Therefore the exchange of the data-structure between correlator and monitoring modules has been made thread-safe.

2 3. Correlators 3.1 Abstract class As previously mentioned, the abstract correlator class dictates the data-structure to its derived classes and the monitoring modules. This data-structure contains: seconds_since_subjob_start (float): Location of current data-structure within a sub-job subjob (string): Name of sub-job source_name (string): Name of observed source stations_object (vector): Vector of station names, with e.g. start/stop times datalines (vector): Vector of available correlation products The last vector describes each correlation product with the following attributes: weight (float, 0...1): Normalized weight of data-line station1 (integer): Location in stations_object station2 (integer): Location in stations_object polarization1 (integer): Polarization of station1 (left or right) polarization2 (integer): Polarization of station2 (left or right) subband (integer): Number of frequency sub-band sky_freq (integer): Sky frequency sideband (integer): Lower or upper sideband datapoints (vector): Vector of complex data-points It should be noted that the monitoring modules expect the complex data-points in lag-space and not in frequency space. The current data-structure uses the name sub-job to indicate to which sub-job it belongs, but a scan name can also be used. This has the advantage that the monitoring modules can output results per scan. The choice is made in the derived correlator class, as one correlator works on a per-sub-job base and another on a per-scan base. 3.2 Derived classes Two classes are derived from the abstract correlator class. These correspond to the MarkIV hardware correlator and the SFXC software correlator. Data are collected from the correlators by; the reading of correlator output-files. In the case of the software correlator these files are read directly by the software correlator data retrieval class, but the derived hardware correlator data retrieval class had to use an adopted version of J2MS to retrieve all needed information.

3 4. Monitoring modules 4.1 Abstract class The abstract monitoring class provides the derived classes with: Generic communication with the central server class A general approach to communicate with a correlator through the data-structure Default communication with the outside world, such as the current network tool 4.2 Derived classes Currently there are two monitoring modules derived from the abstract monitoring class. The first checks the weights of the data-structure, the second the quality of the fringes. 4.3 Derived class: weights Weights are calculated as the ratio between the number of valid data points and total possible number of data points. Low weights can indicate a failure in the data transport from one of the stations to the correlator, or a failure of the recording equipment itself. Weight are transferred to the network monitoring tool, which applies a threshold and sends a warning to the user in the case of low weights. 4.4 Derived class: fringes The second monitoring module collects the data from the data-structure and computes a fringe Signalto-Noise Ratio (SNR) over a certain amount of time for each correlated product (for example, at the end of each scan). The source name attribute in the data-structure is used to determine whether a source is a calibrator or not. Calibrators are bright, unresolved sources that, depending on several factors such as observing frequency, baseline length and integration time, usually will produce a fringe. Therefore the current fringe SNR monitor module only computes SNR values for calibrators. The calibrators will be selected by an experienced VLBI support scientist before a correlation. To compute a SNR value of the correlation function averaged over a certain amount of time, the following approach has been used: 1. Sum each complex data-point (real and imaginary part separately) over K integrations, then calculate N values of r n : K r n = ( i=0 2 real i) +( K 2 imag j= 0 j) K 2. Find in the N data-points, the maximum value m. 3. To determine the background level, two options have been tested: 1. Find the nearest neighbors A (left of maximum) and B (right-side) with half or lower maximum value. Consider all points outside the range A to B as part of the background.

4 2. A more simple approach: exclude all points within plus or minus 5 lags of the maximum from the calculation of the background. When the maximum value lies within 5 lags from positions 0 or N-1, the total number of excluded points will of course be less than Compute the mean of the background level as follow: μ noise =( A 1 n= 0 N 1 r n) + ( n=b+1 A+ ( N 1 B ) r n) 5. The standard deviation is computed as: A 1 2 σ noise =( n=0 N 1 (r n μ noise ) 2) + ( n=b+ 1 A+ (N 1 B) (r n μ noise ) 2) 6. Finally the SNR value of the fringe can be computed as: SNR= m μ noise 2 σ noise The computed SNR values are transferred to the network monitoring tool, where a threshold is applied. 4.5 Results of fringe SNR calculations To determine which of the two approaches performs best, a number of tests have been done. Table 1 shows the results of applying both approaches to an auto-correlation (Figure 1): LL RR LL RR SNR MAX MEAN STDEV Table 1: Fringe SNR calculations on auto-correlation The numbers in black result from the nearest neighbors approach, the red from the 11-point window approach. It is clear that the nearest neighbor approach overestimates the background level by including too much of the actual fringe peak.

5 Fringe plot auto-cor Bd sub-band LL RR Figure 1: Fringe plot of auto-correlation Fringe plot cross-cor Ys Zc subband 6 LL LR RL RR Figure 2: Fringe plot of the cross-correlation of a weak source

6 The fringe plot shown in Figure 2, contains four different polarizations and is clearly very noisy, although RR (green) seems to show a peak. The results of applying both methods are listed in Table 2. LL LR RL RR LL LR RL RR SNR MAX MEAN STDEV Table 2: Fringe SNR results of cross-correlation of weak source The results are almost equal, which was to be expected as the source is obviously very weak. Fringe plot cross-cor Bd Mc sub-band 6 LL LR RL RR Figure 3: Fringe plot of the cross-correlation of a strong source Figure 3 shows a clear fringe of a calibrator. Table 3 shows the same effect as with the auto-correlation, namely that the nearest neighbor method overestimates the background noise level by including too much of the peak. Note that the cross-polarizations are much weaker. LL LR RL RR LL LR RL RR SNR MAX MEAN STDEV Table 3: Fringe SNR results of cross-correlation of strong source Clearly the second method of calculation the background level of fringe plots does a better job at determining the actual level. Consequently this method has been implemented in the software.