A real-time satellite system based on UNIX

Behavior Research Methods & Instrumentation 1980, Vol. 12 (2),126-131 A real-time satellite system based on UNIX SHARON MURREL and TED KOWALSKI BellLaboratories, Murray Hill, New Jersey07974 One solution to the difficulty of running real-time applications under UNIX is to develop the application programs on an available UNIX system and execute them on a dedicated satellite processor. This combines the advantages of a powerful timesharing operating system with the real-time capabilities of a single-process system. PARASITE, a real-time satellite system, provides tools for developing the application program on the host and executing it on a satellite. A host utility serves to invoke the standard UNIX C compiler and link its output with the PARASITE library. The PARASITE library consists of routines that mimic the standard library and routines that read and write the real-time peripherals. PARASITE currently supports digital inputs and outputs, asynchronous serial-line interfaces, programmable real-time clocks, and analog-to-digital and digital-to-analog converters. Another PARA SITE utility downloads the object code of the user program and PARASITE support code into the satellite, where it runs independently. Once the application routine is executing on the satellite, it controls the satellite processor at all times and is continuously available for servicing hardware-generated interrupts. When a real-time peripheral interrupt routine is invoked by a hardware interrupt, it sends a software signal to the user program, in addition to processing the interrupt. This allows the user program to perform additional tasks that are specific to the application. All data to be permanently stored must be transferred to the host. Since the satellite has no direct access to the resources of the host, a process running on the host receives the data and manages files. PARASITE provides packet driver routines on boththe hostand the satellite, which together handle thedata transmission protocol. A wealth of behavioral experiments throughout experimental psychology require real-time processing. Experiments from many diverse areas, ranging from animal learning to human information processing to mathematical psychology, require the monitoring of digital and analog inputs (e.g., pushbuttons and joysticks) and the control of digital and analog outputs (e.g., lights, buzzers, and tones). Many laboratories have satisfied these real-time requirements for many happy years using racks of relay equipment and solid state circuitry. The advent of the microprocessor and inexpensive solid state memory has enabled many labs to increase the power and flexibility of their experimental set-ups. A computerized real-time data acquisition facility alleviates much of the work involved in running experiments. It enables the experimenter to collect and analyze large amounts of data quickly and efficiently. The ease of running subjects is attractive to those among us who have spent many hours copying counters in the correct order and zeroing them before the next session. Most important, the experimental results are in machine-readable form and are therefore immediately available for analysis. The capabilities of experimental set-ups vary widely. Each lab is inevitably torn between the need for inexpensive, single-process systems dedicated to real-time applications and the desire of all experimenters to rush to a terminal and perform just one more analysis at whatever time they have new insights. A minimal data acquisition set-up requires a microcomputer system, interfaces appropriate to the task, and a terminal. Supporting analysis capabilities on a timesharing system, which allocates the processor and its resources among a number of users, requires a larger, more expensive computer system. Such a system, by its very nature, does not support real-time processing as well as a dedicated processor. In order to run a real-time application on a timesharing system, the application must be able to gain control of the processor when it needs to sample, process, or output data. To monitor and respond to external events, an application must be polling the device at the time of the event or must be invoked by an interrupt generated by the event. There are myriad solutions that satisfy both the realtime and timesharing requirements. One solution is to use a small laboratory computer for data acquisition and to use a large, centrally located and supported computer for analyses. Despite the complaints about turn-around time, missing outputs, and the wasted time and effort required to transfer data between the processors, this is a simple solution. When the need for increased on-site capabilities exceeds the disadvantages of supporting a large computer system, there are two approaches. The first is to use the timesharing system to support both real-time and timesharing applications. In this case, the operating system may be adequate for both or may be modified to improve real-time performance. The second approach is to use the timesharing system to develop the real-time application programs, which then execute on a dedicated processor. The dedicated processor may be either the halted host, in which case timesharing is Copyright 1980 Psychonomic Society, Inc. 126 0005-7878/80/020126-06$00.85/0

A REAL-TIME SATELLITE SYSTEM 127 unavailable while an experiment is running, or a satellite processor. In the latter case, the timesharing host is capable of supporting a number of minimally configured satellite systems. We have chosen this last path. PARASITE, a real-time satellite system, is a software support system that runs under a host computer's UNIX 1 operating system and serves as a framework for the development of real-time application programs, which then run independently on a satellite processor. Readers unfamiliar with the UNIX operating system may be interested in the overview presented by Ritchie and Thompson (1978); although many specific details presented below will escape those unfamiliar with UNIX, the essential points should be clear. PARASITE is designed for applications that monitor external events by receiving and processing data from peripheral devices. A representative application is a psychological latency experiment, which measures the time elapsed between a stimulus and the selection by the subject of one of the available responses. A user program running under UNIX cannot satisfy real-time requirements because it cannot control when it is memory resident, it cannot affect when it has control of the processor, and it cannot receive hardware-generated interrupts. On a satellite, however, rapid response to interrupts and availability for monitoring is assured because the program is in memory and active at all times. Real-time response is further improved since the user program does not invoke indirect system calls in order to interact with peripherals, as in a timesharing system, but rather has direct access to these devices. This allows a real-time program, which could not run under UNIX, to flourish in the satellite. PARASITE uses the utilities and resources available on the host to minimize the satellite software and hardware required. Currently, PARASITE does not support a resident file system or any mass storage devices. It supports only the PDP-11 2 architectural family as processors, enabling it to incorporate important standard utilities. These serve to develop and invoke the specialpurpose utilities, to create and maintain the specialpurpose libraries, to configure the satellite system, and to edit, compile, and load the application programs. Since an understanding of the host-satellite relationship is critical, a summary of the important points is presented in Table 1. All source for the PARASITE system and the application routines resides on the host. The host compiles the real-time application program, links it with the PARASITE software modules, and downloads the object code into the satellite. The satellite processor executes the application program, including all library routines and I/O functions. Only the object code and data downloaded from the host and the data being collected ever reside in the satellite. The satellite has no resident file system and must transfer all data to be permanently stored to the host. Since it has no direct access to the resources of the host, a process running Table 1 Host-Satellite Relationship I. Source code for the application is stored, edited, compiled, and loaded on the host system. 2. The object code is downloaded to the satellite, where it is executed independently. The satellite does not support a resident file system or any mass storage devices. Only the object code and the data being collected ever reside on the satellite. 3. All data are transferred to and stored on the host. 4. The satellite has no direct access to the resources of the host; a host process running in parallel receives the data and manages files. there on behalf of the satellite receives data packets and manages files. This technique of partitioning the realtime data acquisition task provides an efficient interface between the processors. PARASITE provides five functions for the user (see Table 2). It supplies utilities that invoke the C language compiler and link the compiled code with subroutine modules from the library designed for the satellite. The satellite library contains, in addition to the standard functions, routines that read from and write to the realtime peripheral interfaces available on the satellite, including interrupt service routines. PARASITE includes host utilities that download the absolute binary code of the application program into the satellite processor. It supplies an interactive debugger based on the UNIX adb and provides routines that allow the processors to exchange data. PARASITE HARDWARE With the advent of the microprocessor and inexpensive solid state memory, a minimal satellite system can be purchased for $3,000. 3 The minimal hardware configuration for a satellite includes an LSI-II /2 CPU, 16K bytes of memory, two serial lines, backplane, power supply, and the peripherals appropriate to the monitoring task. Any PDP-II family member whose instruction set includes the extended instruction set (EIS) can be used as a satellite (e.g., LSI-II/23, PDP-I 1/40, etc.). All peripherals used in the application must be interfaced to the satellite. Currently, there are two serial lines between the host and the satellite (see Figure 1). One serves as the satellite's console and is used for downloading and for communication between the user and the application program. The second serial line is used by the data transmission routines and is necessary only if data are being transferred using the data transmission routines. This configuration is convenient, since all terminals are hard-wired to the host and are cormected to the satellite through a standard software utility only when appropriate; it is necessary when using the debugging routine. However, it requires two serial ports on the host system when the data transmission routines are used. We are now developing a monitor program that will require only a single link between the two processors

128 MURREL AND KOWALSKI Library Utilities: Table 2 Functions of PARASITE Software 1. Host utility for compiling and linking the user's C application program with the PARASITE satellite library. Shell Utility: Icc 2. Satellite library and routines which read and write real-time peripherals (device drivers), including interrupt routines. _tolower, _toupper, a641, abort, abs, assert, atoi, atoi, break, brk, calloc, clearerr, close, edata, end, errno, etext, exit, fclose, fdopen, feof, ferror, fflush, fgetc, fgets, fileno, fopen, fprinif, fputc, fputs, fread, free, freopen, fscanf; fwrite, getc, getchar, gets, getw, gtty, isalnum, isalpha, isascii, iscntrl; isdigit, islower, isprint, ispunct. isspace, isupper, Btol, 164a, longjmp, Ito13, mal/oc, open, perror, printf; putc, putchar, puts, putw, qsort, rand, read, real/oc, sbrk, scarf; setbuf, setimp, signal, sleep, sprintf, srand, sscanf, strcat, strchr, strcmp, strcpy, strlen, strncat, strncmp, strncpy, strrchr, stty, swab, syserrlist, sys_nerr, toascii, tolower, toupper, and ungetc. Device Utilities: AtoD, line clock, digital input, digital output, DtoA, serial line, packet, and programmable clock Operating System Utilities: C Support Utilities: 3. Host utility for downloading. Shell Utility: Prologue code: _init and start. Run-time code: _trap, _sendsig, and _stray. Exit code: exit and dump. Register handling: csv and cret. Arith opers: aldiv, almul, alrem, ldiv, lrem, and lmul. dload: abload, cvboot, cvlda, and connect. 4. Interactive debugging utility based upon UNIX adb. C Utility: ladb 5. C routines on both processors for exchanging data. Host Utility: Satellite Utility: pkt packet driver (see Figure 2). In this configuration, one serial line serves as the satellite's console. The satellite monitor, which is stored in PROM memory in the satellite, enables the user to interact with the host or the satellite, as appropriate. It may also be used to store the application program. The second is the communication link between the two processors. This link is used for downloading routines, all data transmission, and communication between the user, at the satellite console, and the host computer. An important advantage of this configuration is that the satellite system may be used at a remote location. Thus the console, its modem, the satellite system, and its stimulus and response devices may run the experiment at another location, connecting to the host system over phone lines. The command sequences to download and execute a program on the satellite are identical in the two configurations. The power of PARASITE depends upon the speed of the satellite processor and the link to the host system. The satellite must transfer data to the host as fast as the data collection task can fill its local memory; very highspeed data collection can therefore be handled only

TT SATELLITE HOST Interfaces: Console Peripherals: Serial Lines Digital Inputs Digital Outputs Real-Time Clocks AID, DIA Converters Terminal Terminals Disks Line Printers Tapes I/O Devices: Subject Terminals Stimulus Devices Response Manipulanda Figure 1. Host-satellite configuration requiring two links. SATELLITE Modem //» Modem 0- HOST Interfaces: Console Peripherals: Serial Lines Digital Inputs Digital outputs Real-Time Clocks AID, D/A Converters Terminal Terminals Disks Line Printers Tapes I/O Devices: Subject Terminals Stimulus Devices Response Manipulanda Figure 2. Host-satellite configuration requiring a satellite monitor.

130 MURREL AND KOWALSKI in bursts, if at all. The seriousness of this restriction depends upon the task. The system's capabilities may be extended, if necessary, by either increasing the speed of the data link or increasing the amount of storage available on the satellite. The user may implement a highspeed parallel data link between the two processors. Currently, PARASITE supports 64K bytes of memory. With the introduction of the LSI-11/23 and a minor modification to PARASITE, the amount of memory supported by the satellite system will rise to 256K bytes. Note that the application routine will still be limited to 64K bytes; the additional memory is available only for data storage. A valuable enhancement would be a disk driver to temporarily store large amounts of data on contiguous disk regions. We do not propose to implement a UNIX file system on the disk, but rather to use the disk as an extension of the satellite's memory. PARASITE SOFTWARE Special-purpose software written for the PARASITE system includes both the utilities that run under UNIX and the code that is downloaded with the application program into the satellite. The five functions provided by PARASITE and listed in Table 2 reflect functional software modules, which are described in detail below. Compiling and Loading The Icc command invokes the standard C compiler, defining the location of the include files. It then links the compiler output with the low-core interrupt vectors and the satellite library routines. In order to execute that file on a satellite, utilities for loading it into the satellite memory are provided. Alternatively, this loader output file can be executed on the host by halting the processor and booting that file. PARASITE Library The code written for the satellite includes routines that are directly called by the user and support routines that are used by the system. User library and I/O routines. The application program must be written in C and reference the PARASITE library header files. It is compiled by the standard UNIX C compiler and linked with the PARASITE library by the standard UNIX loader. The library includes functions to perform input and output, to convert between ASCII and binary, to allocate memory, to catch signals, and to manipulate strings. It provides routines for generating random numbers and taking absolute values, but it does not support the mathematical routines that require floating point operations. The floating point instruction set on the LSI-I 1/2 is not compatible with the code generated by the C compiler, and therefore no C-generated floating point instructions can be executed on the satellite. Note, however, that the mathematical library may be implemented on satellites such as the LSI-l 1/23. Calling parameters and return codes are identical to those used under UNIX, except that minor device numbers" are passed instead of file names. Both library routines and device drivers use minor device numbers to communicate directly with the devices, rather than referencing an underlying file system. Similarly, system calls are replaced by subroutine calls, which allow the user direct access to the real-time peripherals and reduce the software overhead needed to access resources. These subroutine calls closely mimic the standard open, close, read, write, gtty, and stty UNIX system call. Note that these changes in implementation are transparent to the user. Device drivers have an additional feature under PARASITE: When the device driver interrupt routine is invoked by a hardware-generated interrupt, it sends a software signal to the user, in addition to processing the interrupt. Each signal may be ignored or may invoke a subroutine specified by the user. Two values are passed to that subroutine: the minor device number and a value appropriate to the device. For example, the interrupt routine of an analog-to-digital converter passes the converted value. The library subroutine signal allows the user to specify the action to be taken when a signal occurs. PARASITE supports asynchronous serial line interfaces, digital inputs and outputs, programmable real-time clocks, analog-to-digital converters, and digitalto-analog converters. Operating system support routines. The library routines that are invisible to the user are the C language support routines and the operating system routines. The C compiler does not produce in-line code for certain operations. It assumes that it has routines to save and restore registers during subroutine calls and that there exists either a hardware or a software implementation of multiplication, division, and modulus of longs. These operations are provided by routines included in the PARASITE library. The operating system library consists of three functional modules: code that initializes the system, code used during execution of the user program, and code that properly terminates the user program. The prologue code sizes memory, sets up the standard stack frame, zeroes uninitialized data, opens standard input and output files, and starts the user program upon a signal from the user. The user may start and terminate the program in the satellite from the terminal. During execution, the user program uses machine language assist routines to set the program's priority level, to interface hardware interrupts to C routines, and to catch stray interrupts. The exit code prints a termination message, flushes I/O buffers, and returns to the monitor. Downloading The shell file dload performs a two-step bootstrapping process that loads the file into the satellite across a serial line link. First, a load command, available in the DEC-supplied LSI-II/2 monitor, reads the absolute loader into the satellite's memory and jumps to the start

A REAL-TIME SATELLITE SYSTEM 131 location given. This command cannot be used to download the application program directly, as it can only load 114 l o-bit words. The absolute loader then reads the object of the application program into memory and passes control to that program. Interactive Debugger An interactive debugging program, ladb, runs on the host computer and enables the user to interactively debug an application program executing on the satellite processor. The user may examine the satellite's memory after a program has aborted or trace execution of the satellite program by embedding breakpoints. In any case, the user is able to modify the satellite memory and display it in a variety of formats (e.g., as octal values or instructions sequences). Ladb command syntax is identical to the UNIX debugging program adb. Data Transmission Between Processors Once the application routine is downloaded, it requires a host process running in parallel to access the resources of the host. There are packet driver routines on both the host and the satellite that together handle the data transmission protocol. Depending on the volume and speed of data collection, these routines may be invoked during or after a particular application program. The two types of transmissions used are control packets, which are handshaking signals, and data packets. Control packets are used to open and close transmission sequences and to acknowledge or reject data packets. Both types of packets begin with as-byte header: a synchronization byte, a control byte indicating the packet type, a data-packet sequence number, and a header checksum. Data packets also include a data block and its checksum. If a data packet is rejected by the receiving processor, it is retransmitted. Each routine sends error messages to its console if it detects a breakdown in the communication link. Thus the standard host packet driver routine serves to exchange data with the satellite and to store the data received in a file. If this satisfies the requirements of the user, he may simply invoke the packet driver using the utility pkt with arguments specifying the serial line device name, the baud rate, and the file name. Otherwise, the user may write a more sophisticated routine that processes the data received from the satellite and conditionally opens files on the host and transfers the contents to the satellite. The host process may interact with the timesharing system like any other process; it must be available to receive and acknowledge data packets, however, and complex support tasks may be best implemented as separate processes. CONCLUSIONS Supporting a satellite on a host running UNIX combines all the advantages of the UNIX timesharing system with the real-time capabilities of a dedicated singleprocess system. PARASITE defmes the relationship between the two processors and provides a set of utilities for developing satellite programs that maintain the standard UNIX-user interface. It is designed to minimize the effort required to implement a given real-time application. The system itself is organized so as to minimize the satellite hardware and special-purpose software required. Moreover, it is modeled after UNIX so as to facilitate both system mastery and modification. The system may be easily augmented with the addition of device driver routines. It is a powerful solution to the difficulties of implementing real-time experiments. REFERENCE RITCHIE, D. M., & THOMPSON, K. The UNIX time-sharing system. The Bell System Technical Journal 57, 6 (July-August 1978, Part 2), pp. 1905-1929. NOTES 1. UNIX is a trademark of Bell Laboratories. 2. DEC and PDP are registered trademarks of Digital Equipment Corporation. 3. This includes a four dual-slot backplane and power supply available from DEC or Automated Computer Systems, an LSI 11/2 CPU card, and a multifunction module, MXVn-AC, available from DEC, that includes 32K bytes of RAM, two serial lines, and PROM slots. 4. A major device number specifies a device type; minor device numbers uniquely select a single device from each type.