A Methodology for Enhancing Legacy TPS/ATS Sustainability via Employing Synthetic Instrumentation Technology

A Methodology for Enhancing Legacy TPS/ATS Sustainability via Employing Synthetic Instrumentation Technology Dr. David Carey Tobyhanna Army Depot Tobyhanna, PA Christopher Antall Tobyhanna Army Depot Tobyhanna, PA Robert Wade Lowdermilk BAE Systems San Diego, CA Alexis Allegra BAE Systems San Diego, CA Abstract This paper presents a methodology for mitigating Test Program Set (TPS) & Automatic Test System (ATS) obsolescence and enhancing TPS/ATS sustainability via employing Synthetic Instrumentation (SI) technology. The methodology and the associated sub- processes described within this paper represent a major paradigm shift in current support equipment hardware & software sustainability approaches and will have a profound impact on the process of supporting and maintaining legacy automated test systems (ATS) and TPSs now and into the future. The subject methodology was validated employing the U.S. Army Repair Depot at Tobyhanna (TYAD) RF Test Platform as the demonstration vehicle test bed. The proof-of-concept demonstration validated the concept of emulating and replacing several legacy Commercial Off-the-Shelf (COTS) instruments with synthetic instrument technology. The primary goals of validating this technology paradigm were to provide an environment that would reduce TPS rework costs, decrease ATS maintenance and repair costs, and enhance the sustainability of legacy ATSs/TPSs going forward. During the course of this project the synthetic instrument technology insertion paradigm was demonstrated to the at large DOD maintenance community. Keywords-Synthetic Instrumentation, Automatic Test Systems I. INTRODUCTION The electronics industry and the Department of Defense (DOD) have thousands of obsolete legacy Automatic Test Systems (ATS). Under Title 10 of the United States Code (USC) 2464, it is a requirement that maintenance depots retain maintenance core capabilities. In order to comply with Title 10 responsibilities, the DOD must continually strive to ensure capability to maintain critical. This compliance is necessary to ensure the following Maintain critical Command, Control, Communications, Computer, Intelligence, Surveillance, and Reconnaissance (C4ISR) components and systems in a state of readiness to support operations required in support of our national security interests Have the ability to sustain the critical components and systems while conflicts are underway Have the ability to rapidly reconstitute that readiness once operations are complete. There are many ATS, with different hardware, that require obsolescence mitigation and upgrades through technology insertion. The inability to reliably test products is having an effect on mission readiness [1]. This challenge is similar to the medical and manufacturing industries where they must keep their test, diagnostic, and robotic manufacturing equipment operational in order to continue to do business. The difference is that the responsibility for maintaining a state of mission readiness falls on the DOD depots. The cost of maintaining the support equipment cannot be transferred to the customer in the depot. In industry, that responsibility is ultimately transferred to the customer through price increases. Obsolescence of Commercial Off-the-Shelf (COTS) test equipment, commonly referred to as legacy test equipment, has threatened the longevity of ATS and the ability for manufacturers and repair depots to support their workload. The solution to this problem is ATS modernization. Modernization may be realized in several ways Modify the system while keeping the identical capabilities to the current system Develop a system with a few enhancements/upgrades, such as better speed or accuracy Develop/acquire a completely new system. For decades ATS modernization has been a hot topic and has attracted attention from the ATS maintenance personnel as well as ATS equipment suppliers. Traditionally, legacy ATSs have been upgraded or refurbished to extend their life. The end result has been aging but working ATSs and a requirement for the ATSs to remain in service much longer than originally

Host and planned and a test equipment obsolescence issue that must be addressed. There has been a substantial investment in Test Program Sets (TPSs) over many years, and whenever an instrument within a given ATS is replaced there is a risk of requiring TPS updates [2]. This paper describes an integration process that is an attempt to recover the efforts of the original TPS developer and reduce the amount of TPS rewrite or updates that must be affected by the maintenance community. In order to reduce the amount of TPS rework, it has been proposed that the synthetic instrument would be a drop in replacement in which the test execution is identical to the original legacy equipment [3]. ATSs are application-specific systems that increase the reliability and productivity of testing activities in different stages of product development, manufacturing and maintenance. These test systems operate on a wide range of products: Radar, guidance systems, radio communications, jamming systems, automotive, satellite communications, and nuclear control systems to name a few. Different test strategies are necessary to validate a product's performances. These strategies have been instantiated in the development of hardware and software as it is associated with the multitude of legacy TPSs. Many companies have tried to solve the TPS portability challenge via system upgrades [4]. To prevent the software and TPS obsolescence, several companies in the industry are working to develop both the test executive software and programming languages that are intended to be fully backwards compatible. From the perspective of TPS portability the implication is that any given TPS developed with this software suite will be 100% transportable across software upgrades. In practice, however, TPSs must be rewritten by varying amounts and then revalidated in order to perform the upgrade. The industry has not yet provided a stable, backwards compatible, software solution [5][6]. Insertion of synthetic instrumentation technology into DOD ATS addresses primary concerns in ATS: sustainability and obsolescence. The systems and Units under Test (UUT) may have a life cycle exceeding twenty-five years, whereas the software and hardware that define a test system typically become obsolete in less than five years. Historically, ATS designers combat this obsolescence by using COTS hardware along with obsolescence contracts to ensure the availability of the hardware (both test interfaces as well as instrumentation) after the commercial life cycle of the legacy instruments expires. However, the quantity of hardware is fundamentally limited, and if the support software is upgraded it must utilize the obsolete drivers and software interfaces to the obsolete hardware (to ensure portability) [3][7]. The continued use of aging legacy equipment has resulted in a deterioration of mission readiness, which has resulted in investigating insertion of synthetic instrumentation technology into existing and next-generation ATS. Synthetic instrumentation technology may improve the performance of ATS and reduce the associated acquisition and maintenance costs. As part of this investigation (and validation), several primary integration objectives were identified as part of inserting synthetic instruments into DOD ATS: 1. Replace traditional rack and stack instruments with synthetic instrumentation technology and instrument Personality Modules (PMs). 2. Maximize the portability of existing TPS software. 3. Develop an integration process to facilitate integration of synthetic instrumentation based instruments. 4. Evaluate the performance of synthetic instrumentation based instruments for ATS applications. 5. Validate the synthetic instrument functions. As part of the initial evaluation described in this paper, the synthetic instrument will replace three legacy rack and stack instruments: the Spectrum Analyzer, the Digital Storage Oscilloscope (DSO) and the Vector Network Analyzer (VNA). II. ATS DESCRIPTION A. ATS Plaform Description The ATS selected for integration with the synthetic instrument is the U.S. Army Repair Depot at Tobyhanna (TYAD) Radio Frequency Test Platform, or RFTP, which consists of two primary racks of equipment containing RF test equipment used for testing several RF systems and subsystems. Figure 1 depicts the RFTP configuration which includes a spectrum analyzer, RF power meter, digital storage oscilloscope (DSO), frequency counter, vector network analyzer (VNA), arbitrary waveform generator (ARB), and several signal generators. The planned integration of the synthetic instrument into the RFTP will be to interface the synthetic instrument to the RFTP station controller via a standard Local Area network (LAN) interface. This configuration allows the user to monitor the synthetic instrument s Graphical User Interfaces (GUIs) and measurement performance as well as the existing instruments. Station Controller GPIB / LAN Interfaces LAN Synthetic Instrument Figure 1. Rack Configuration of RF Test Platform B. Synthetic Instrument Description A high-level block diagram of the synthetic instrument that will be integrated into the RFTB is shown in Figure 2, and is composed of a PXI Express (PXIe) hybrid chassis, PXI and PXIe hardware modules, vector RF down converter module, and signal conditioning module. Note that the signal conditioning module allows for standalone operation of the

synthetic instrument and would be absorbed in the RFTB RF interface panel as part of follow-on integration. The synthetic instrument is configured with a dedicated embedded Microsoft Windows based instrument controller allowing stand alone operation that can be configured for both local and remote control. The synthetic instrument provides connections to a standard external monitor, keyboard, and mouse allowing for local control of the synthetic instrument via a Graphical User Interface (GUI) for each Personality Module (PM). In remote control mode the synthetic instrument provides a remote interface via standard Gigabit LAN Ethernet. Depending on the Personality Module, the synthetic instrument configures the signal paths and may use RF down conversion, RF vector down conversion, or direct digitization of the input signal(s). Once the input signals are digitized, specific measurements are performed by DSP-based or numeric processing techniques. Figure 2. Synthetic Instrument Functional Block Diagram III. INTEGRATION PROCESS To facilitate integration of the synthetic instrument based instrumentation into the RFTP ATS and maximize TPS portability, an integration procedure (depicted in Figure 3) was developed to establish a baseline for future integration. As shown in the figure, the process of integrating a synthetic instrument based instrument consists of four separate processes; a system-level analysis of the integration environment, followed by a driver system analysis, a driver function analysis and lastly a TPS analysis and validation. The following sections describe each process. A. System-Level Analysis The objective of the system analysis is to determine the method of integrating the synthetic instrument into the existing ATS, addressing Form, Fit, and Function (FFF) with an emphasis on the data and control communication interfaces and the existing host controller software. The standard communication interface for legacy instruments is the General Purpose Interface Bus (GP-IB). However, in addition to GP-IB, modern instruments include Ethernet, USB, and, Serial (RS232) interfaces. The available communication interfaces on the host controller will determine the interface that should be used for integrating the synthetic instrument into the ATS. Depending on the decisions of the ATS designer, the new instrument may utilize the same communication interface as the legacy instrument in the final integrated system or leverage new technology. After the communication interfaces between the host controller and the synthetic instrument has been established, the host controller s resource management software must be verified. The resource management tool is a software tool that allows the user to communicate to the instruments installed in the ATS from a central location. An example of this kind of software is National Instruments Measurement and Automation Explorer (NI-MAX), or the Agilent Connection Expert. The resource manager assumes that the communication protocol is based on the Virtual Instrument Software Architecture (VISA) and that the instruments conform to the Standard Commands for Programmable Instrumentation (SCPI). The host-to-instrument software architecture is depicted in Figure 4. Figure 3. Integration Process Flow Diagram Figure 4. Host-to-Instrument Software Architecture The resource management software provides the interface required to initialize and configure the synthetic instrument and provides a method to send commands and verify results. Once the instruments and their designators are configured in the resource manager a basic communication test can be performed. This is the lowest level of instrument integration and testing in the ATS and is a powerful tool to verify functionality and identify any communication problems or command problems that may arise during the driver integration phases. The end user can go back to this sequence of test panels to verify the functionality of a specific command, isolated from any timing or driver constraints. If any issues

arise during this portion of integration, the synthetic instrument manufacturer can modify the remote interface to adapt to the ATS command set before initiating the driver system analysis phase of integration. B. Driver System Analysis Each instrument generally provides two different communication paths; the first is through low-level instrument SCPI commands and the second is through a driver which provides functions that send the SCPI commands eliminating the need for the developer to know the SCPI commands. The ATS designer may have chosen to use only the low-level instrument commands in their TPS or they may have constructed a series of intermediate drivers. These intermediate drivers would utilize the instrument drivers (or the SCPI commands) to provide the required functionality to the TPS developer, inferring that many instrument functions can be provided to the developer in a single function call. Integration of these drivers involves first assessing if the driver can communicate to the instrument. If the instrument is found and communication established between the instrument and the host controller within the resource management software, the low-level drivers should also be able to communicate. However, because there are different kinds of VISA writes and reads (synchronous vs. asynchronous), a lowlevel test program written in the same language as the lowlevel drivers will facilitate the debug process and ensure that the drivers will work with the synthetic instrument. This lowlevel program will mimic what was done in the VISA test panel. Once it is verified that this level of program is working properly, the same function call utilizing the existing intermediate driver should be used. The choice of a simple intermediate driver function, after verification of basic functionality, will ease the integrators into the process of driver function analysis. If there is a communication problem at this level the synthetic instrument manufacturer can adjust their Control and Support (C&S) software to interface with the existing ATS interface. Once the full communication path is verified, the driver function analysis can begin. C. Driver Function Analysis TPS portability is typically the primary goal when replacing the legacy instruments SCPI command set with the synthetic instrument inferring that maintaining the existing drivers and function calls are important. The synthetic instrument C&S software can be modified to match the existing commands resulting with a subset of supported functions to encompass all of the ATS s required functions. At this stage in the integration process it is important to document the command set being transmitted by the driver to the synthetic instrument. This can be accomplished by utilizing tools such as NI SPY to verify the VISA commands being utilized. The command set is compared with the driver documentation to verify instrument functionality. The command set is then mapped to the set of synthetic instrument commands (in the synthetic instrument C&S software) in order to duplicate the synthetic instrument functionality. If any disparities between the legacy instrument s command and the how the synthetic instrument has interpreted the command, the synthetic instrument s C&S software can be modified to accommodate the command correctly. This modification may include incorporating a remote command, or modifying the synthetic instrument personality module s functionality to incorporate and/or modify a function. D. Integration Process Discussion One of the primary concerns in ATE is sustainability and obsolescence. The systems and units under test (UUT) may have a life cycle exceeding 25 years, whereas the software and hardware that define a test system typically become obsolete in less than 5 years. In order to combat obsolescence ATS designers have begun using COTS hardware along with obsolescence contracts to ensure the availability of the hardware (both test interfaces as well as instrumentation) after the commercial life cycle. However, the quantity of hardware is fundamentally limited, and if the support software is upgraded it must utilize the obsolete drivers and software interfaces to the obsolete hardware (to ensure portability). To prevent the software obsolescence, several companies in the industry are working to develop both the test executive software and programming languages, that are intended to be fully backwards compatible. From the perspective of TPS portability the implication is that any given TPS developed with this software suite will be 100% transportable across software upgrades. In practice, however, TPSs must be rewritten by varying amounts and then revalidated in order to perform the upgrade. The industry has not yet provided a stable, backwards compatible, software solution. A synthetic instrument is designed to be agnostic to hardware and software obsolescence from the standpoint of TPS portability. A synthetic instrument will require software and hardware upgrades throughout its lifecycle, however the changes that are made internally to the synthetic instrument are not reflected in the software and hardware interfaces. This implies that the synthetic instrument making up the core set of instrumentation in an ATS will minimize the need to rewrite any TPSs due to instrumentation changes. In fact the only significant integration work for a synthetic instrument should be in the initial replacement phase of legacy stand-alone instrumentation. A synthetic instrument, because it can be thought of as an instrument emulator will be able to perform to the required specifications and measurement speeds. In TPS sustainment, the objective is to maintain measurement accuracy and speed. A TPS that passes on the legacy instrument must still pass on the new instrument. Due to the performance improvements inherent in the synthetic instrument (technology evolution), TPSs will often fail due to the faster processing speeds and higher quality signal capture. The synthetic instrument can be tailored to perform in the same manner as the legacy instrument. The software architecture of the synthetic instrument is such that it can add, as needed, legacy software modules to reduce bandwidths and increase (or decrease) measurement speeds.

IV. INTEGRATION RESULTS TPS portability was the primary objective of integrating synthetic instrumentation into the TYAD RFTP with no/minimal changes to the RFTP C&S software. The RFTP utilizes National Instruments (NI) TestStand as the test executive and LabVIEW 2009 to host the instrument drivers for the legacy instruments being replaced by the synthetic instrument. The instrument drivers utilized manufacturerprovided drivers to the maximum extent possible which made the integration process simpler because the remote interfaces are well documented and conform to industry standards. TYAD followed the integration process described in this paper. A. System-Level Analysis The TYAD RFTP is a communications tester for RADAR testing applications. The RFTP consists of two primary racks of equipment containing RF test equipment used for testing several RF systems, including a spectrum analyzer, RF power meter, digital storage oscilloscope (DSO), frequency counter, vector network analyzer (VNA), arbitrary waveform generator (ARB), and several signal generators. The planned integration of the BAE Systems SIU5230 synthetic instrument into the RFTP was to interface the synthetic instrument to the RFTP station controller via a standard LAN interface. This configuration allows the user to monitor the synthetic instrument s Graphical User Interfaces as well as the existing legacy instruments. The RFTP s legacy instruments being replaced by the synthetic instrument are considered modern instrumentation, and it is assumed that synthetic instrument does not require tailoring of the measurement speeds to reduce performance or measurement speed to match obsolete instrument performance. The RFTP ATS architecture provides multiple communication interfaces and GPIB is used for primary hostto-instrument communications. However, the LAN interface was selected for the synthetic instrument integration. By utilizing this configuration the resource manager software was able support both the legacy instrument and the synthetic instrument allowing commands to be sent to each instrument for functional comparisons. The RFTP host software configuration which uses Windows XP as an operating system and the following National Instruments software tools: NI TestStand for TPS development and execution NI-MAX for resource management. NI LabVIEW 2009 for driver development. This choice of software tools works particularly well from an update and TPS portability standpoint as each layer is, supposedly, agnostic to changes at the other layers. The synthetic instrument is compatible with this development environment because it is standards-compliant and the RFTP uses software that also utilizes these standards. The synthetic instrument fits in at the driver communication layer. The synthetic instrument remote communication interface is independent of the drivers and test executive because it interfaces to the host controller at the instrument layer. The resource manager creates a link to the synthetic instrument and the commands already being sent by the driver(s) are automatically sent to the synthetic instrument. This configuration satisfies the requirement of not altering the drivers to successfully communicate with the synthetic instrument. B. Driver System Analysis Once the communication channels were verified the process of assessing the driver system was performed. This process defined how the instrument drivers were written and is the next level of communication between the host and the synthetic instrument. Verification that these drivers can communicate to the synthetic instrument allowed the team to progress to the driver function analysis. All of the intermediate drivers were developed in LabVIEW 2009. The RFTP is a modern ATS and therefore many of the instruments used are modern instruments with modern drivers. The original instrument manufacturer supplied a LabVIEW-based driver with their instruments. The low-level drivers (i.e., the function calls from NI TestStand) are a combination of vendor-supplied LabVIEW blocks, direct VISA I/O, and Instrument I/O Assistant blocks in LabVIEW. The commands are well documented and adhere to IEEE standards. The goal in this driver-level analysis was to ensure that all of the individual driver components successfully communicate to the SIU5230 SMS. If the low-level elements communicate with the SIU5230 SMS, it was assumed that the higher level intermediate drivers will also communicate with the synthetic instrument. C. Driver Function and TPS Analysis The functional integration of the synthetic instrument was accomplished by systematically evaluating the existing RFTP drivers and their functions for each synthetic instrument Personality Module (PM). Because all the drivers were written in LabVIEW each of the driver functions were systematically tested separately before allowing TestStand to combine the functions together in a TPS. Each driver function was evaluated to define the function applied to the legacy equipment. The driver command was transmitted to the legacy instrument to visually understand how the command(s) was being interpreted. Once the function was verified on the legacy equipment the functionality was verified manually on the synthetic instrument by using the GUI. When the functionality was verified on the GUI, the host controller would send the same command it sent to the legacy equipment to the SMS. This methodology facilitated discovery of any faults in an isolated and precise manner. 1) Instrument Control. The synthetic instrument supports local and remote control of the instrument functions, which can be selected from the synthetic instrument s GUI s instrument control panel. The synthetic instrument defaults to local control after completion of the power up sequence allowing the user to control the personality modules via the GUI. If the synthetic instrument is in Remote Mode (and the GUI is no longer user controllable), selecting the Local button in the GUI s instrument control field configured the instrument to local mode.

Remote control of the synthetic instrument was accomplished through a VXI-11 compliant server/client Ethernet-based interface. Verification of the synthetic instrument remote mode capability was a primary objective in the integration effort in order to evaluate TPS portability. As part of the remote mode validation, each of the intermediate driver functions was tested individually. These functions are implemented in LabVIEW VIs which can be modified (during testing) to reference the synthetic instrument `over the legacy instrument. This process for the spectrum analyzer is described in the following sections. A specific intermediate driver function is chosen and the testing process is explained. 2) Integration of Bandwidth Settings Intermediate Driver Settings. The bandwidth settings intermediate driver bundled together the legacy instrument manufacturer s low level drivers. It provides to the test executive, the ability to setup manual versus automatic bandwidth settings, set the desired bandwidth, number and type of averages etc. All of the LabVIEW blocks making up this driver, were vendor supplied. Because of this, the code in this particular function was very well documented and straight forward. This piece of code was run by changing the instrument reference used in the code to the reference used for the synthetic instrument instead of the legacy instrument. These references are setup and managed in NI-MAX. By running this code in a standalone manner, any faults with the instrument communication system could be isolated and fixed. The intermediate drivers utilized by TYAD also contained different kinds of blocks which took advantage of different kinds of instrument writes and reads. After the bandwidth settings intermediate driver was verified to work identically to the legacy instrument, the remaining intermediate drivers were systematically tested. Any discrepancies found in the spectrum analyzer were resolved by altering the code in the synthetic instrument, not in the TYAD driver. Each driver call, for each of the three modules (DSO, Spectrum Analyzer, and VNA) was tested and verified to be functional with the synthetic instrument, during the integration period. Once the drivers were verified, the TPSs utilizing the driver functions were run. Again the integration team worked slowly at first to isolate any problems in handshaking. Any issues that arose were resolved by altering the code in the synthetic instrument, not in the TPS. D. Synthetic Instrument Calibration and Alignment The synthetic instrument provides an internal power vs. frequency alignment module to improve the accuracy of the synthetic instrument and to provide the capability to align and calibrate the synthetic instrument. The synthetic instrument alignment can be mapped to the RF input port on the synthetic instrument s front panel or to an arbitrary RF interface (such as the RFTP RF interface panel). The alignment module utilizes the integrated RF source (see Figure 2) and a USB-based integrated average power sensor to provide a traceable RF source for aligning the synthetic instrument s RF receiver chain. Figure 5 depicts the functional block diagram of the synthetic instrument configuration required to perform alignment of the synthetic instrument. The RF source s output is divided into two identical signals via a resistive power splitter. One signal is applied to a USB average power meter (or equivalent) and the second to the synthetic instrument s RF Input Port. As shown in the figure, the RF receiver chain consists of the RF input connector, cable and signal conditioning, RF down converter and the analog-to-digital converters (ADC). 3 db SPLITTER Figure 5. Synthetic Instrument Alignment Functional Block Diagram. Figure 6 shows the flow diagram for the synthetic instrument alignment process. This is an automated process and requires no user intervention other than the initial hardware setup. Figure 6. Synthetic Instrument alignment process flow diagram. V. SUMMARY This paper describes a methodology for mitigating TPS & ATS obsolescence and enhancing TPS/ATS sustainability via inserting test equipment functionality employing synthetic instrumentation technology. Synthetic instrumentation represents a major paradigm shift in current support equipment hardware & software sustainability approaches and will have a profound impact on the process of supporting and maintaining legacy ATS and TPSs now and into the future.

The subject methodology was validated employing the U.S. Army Repair Depot at Tobyhanna (TYAD) RF Test Platform as the demonstration vehicle test bed. The proof-of-concept demonstration validated the concept of replacing legacy COTS instruments with synthetic instrument technology. REFERENCES [1] David R. Carey, Tobyhanna Army Depot Automated Test System Modernization, IEEE Autotestcon 2010 Proceedings [2] Dr. Ion A. Neag, Supporting Hardware Independence in the Next Generation of Automatic Test Equipment, IEEE Autotestcon 2005 Proceedings [3] Michael N. Granieri and Robert Wade Lowdermilk, Synthetic Instrumentation: The Road Ahead, IEEE Autotestcon 2010 Proceedings [4] Steven Wegner, Test Forensics: A Guide to Evaluating TPS Transportability, IEEE Autotestcon 2010 Proceedings [5] Ian Roberts and Chris Gorringe, A Practical Solution to Instrument Obsolescence in Test Systems, IEEE Autotestcon 2002 Proceedings [6] Jeff Hulett, Re-Hosting Measurements in the Real World Overcoming Implicit Design Factors that Influence Measurements, IEEE Autotestcon 2004 Proceedings [7] James a. Orlet, Key Characteristics of Synthetic Instrumentation to Facilitate TPS Transportability, IEEE Autotestcon 2005 Proceedings [8] Cathleen Kennedy, Sustainment of Legacy Automated Test systems: Lessons Learned on TPS Transportability, IEEE Autotestcon 2004 Proceedings