Adaptive pacing of visual stimulation for fmri studies involving overt speech

Transcription

1 Technical Note NeuroImage 29 (2006) Adaptive pacing of visual stimulation for fmri studies involving overt speech Thomas J. Grabowski, a,b, * Matthew D. Bauer, a Derek Foreman, b Sonya Mehta, a Brent L. Eaton, a William W. Graves, a Dori L. Defoe, a and Lizann Bolinger c a Department of Neurology, University of Iowa, 200 Hawkins Dr./2155F RCP, Iowa City, IA 52242, USA a Department of Radiology, University of Iowa, Iowa City, IA 52242, USA c NRC Institute for Biodiagnostics, Winnipeg, MB R3B 1Y6, USA Received 21 March 2005; revised 12 July 2005; accepted 16 August 2005 Available online 21 November 2005 We report the development of an interactive approach to single-word language production studies in fmri. The approach, adaptive pacing, involves real-time adjustment of stimulus presentation times based on individual subject performance timing and content. At the same time, it maintains a stochastic distribution of interstimulus intervals to avoid confounding task covariates with speech-related signal variance. Adaptive pacing of overt speech production is an example of a new class of paradigms that require an observational approach to data acquisition and benefit from a time-aware acquisition and processing environment. The advantages of adaptive pacing in fmri of impaired subjects are expected to be the acquisition of more informative data per unit time, less contamination of data by correlates of non-language processes such as emotion, and facilitation of experiments that combine normal and impaired subjects. D 2005 Elsevier Inc. All rights reserved. Keywords: fmri; Language; Real-time analysis; Aphasia; Efficiency; Experimental design; Speech Introduction Event-related fmri has the potential for assisting diagnosis, planning treatment, and monitoring disease and treatment course in clinical populations. When subjects with impairments are studied, the fmri methodology should be able to accommodate variable performance success, failure to conform to task protocol, and reduced tolerance for the physical demands of an imaging session. An impaired subject may succeed on some trials, fail on others, and have a pathologically long response latency still on others. In the past, such performance variability has presented severe problems for block-design studies (e.g. Grabowski et al., 2003) because * Corresponding author. Fax: address: thomas-grabowski@uiowa.edu (T.J. Grabowski). Available online on ScienceDirect ( correlates of blocks of impaired performance reflect mixed degrees of success and therefore cannot be compared simply to measurements in normal subjects. For example, the physiologic correlates of language processing in aphasia remain largely unexplored because of the fundamental difficulty of interpreting measurements that are contaminated by abnormal or failed performance. The problem of mixed performance is solved in principle by using event-related design, in which successful trials can be identified post hoc and analyzed selectively. This project seeks to further reduce the technical obstacles for using event-related fmri to study language-impaired subjects. Besides taking care to compare the successful component of an impaired performance to a normal control subject s performance, another issue to respect when comparing normal and abnormal subjects is that the rate at which they are able to perform tasks may differ. Choosing a performance rate involves a trade-off. In general, more rapid performance (i.e., a shorter mean interstimulus interval [ISI]) leads to more efficiency in the experimental design (Dale, 1999), and therefore studies in normal subjects often use rapid performance rates. However, an impaired subject may not be able to support a rapid rate. One problem in this situation will be collisions of trials, that is, one trial is not completed before the subsequent stimulus appears. Collisions confound successful performance with increased working memory demands, arousal, and emotion, and therefore these other factors will influence the physiologic correlates of the affected trials. One solution would be to decrease the stimulus presentation rate for both normal and abnormal subjects to accommodate the (slowest) impaired subjects. Implementation of this approach is problematic, however, because it assumes that the appropriate performance rate is known a priori and does not vary over time. Furthermore, it makes the dubious assumption that decreasing performance schedules will not lead to differential emotional and arousal demands in normal subjects as compared to impaired ones. Consequently, the rate at which the groups are required to perform a task may need to be different. For studies of impaired subjects, it is /$ - see front matter D 2005 Elsevier Inc. All rights reserved. doi: /j.neuroimage

2 1024 T.J. Grabowski et al. / NeuroImage 29 (2006) desirable to explore flexible performance rates, which might facilitate better performance and better equate the performance demands across impaired and normal subjects. In this connection, an advantageous characteristic of event-related design is that a rigidly prescribed performance schedule is not essential as long as the actually performed ( de facto ) schedule varies sufficiently so as to maintain an efficient experimental design (Dale, 1999). These considerations have led some investigators to implement and demonstrate self-paced task paradigms in fmri (Macotta et al., 2001; Daselaar et al., 2001, 2003). The self-paced studies just cited implemented self-pacing using subject button press responses. Single-word language production paradigms are also candidates for a self-pacing approach, but the implementation is more complicated because responses are spoken. This leads to two problems. The first is that response onsets are obscured by gradient switching noise. Secondly, response classification requires expert oversight to determine if an utterance is a response (as opposed, for example, to a prenomial utterance or a false start) and possibly, depending on the paradigm, to determine whether a response is correct. Finally, speech production produces signal intensity artifacts that may confound data analysis if they correlate with BOLD responses of interest. De-correlation of speech artifacts and BOLD responses may be achieved by using variable inter-trial intervals because hemodynamic responses to neural events are delayed and linearly additive (Hajnal et al., 1994; Birn et al., 1999; Barch et al., 1999; Huang et al., 2001). Fortunately, this constraint is compatible with efficient designs. Any pacing strategy for language production studies must be constrained by this requirement. The self-paced approaches reported to date have exploited variable response latencies to jitter the inter-trial interval. It is unclear whether this strategy will also suffice to de-correlate speech artifacts and BOLD responses. The contribution of this article is the demonstration of the use of a time-aware fmri acquisition and processing environment to implement an interactive approach to single-word language production studies in fmri, capable of adapting stimulus delivery rates in real time to individual subject performance. The delivery of the next trial is not triggered directly by the subject or the investigator, but by an interplay between subject performance, investigator oversight, and a stochastic distribution of ISIs. Because the delivery of the next stimulus is contingent on all these factors, we refer to this arrangement as adaptive pacing rather than self-pacing. The advantages of this approach, over and above a simple voice key approach to pacing, are robustness to erroneous utterances, broadly defined, and preservation of a stochastic ISI. Methods Paradigm implementation The adaptive pacing algorithm (Fig. 1) was implemented with I/OWA (Input Output time aware Architecture, Smyser et al., 2001). I/OWA utilizes a streaming data model and a unified, timedimensioned data file format to support time-aware software applications that implement real-time processing of response data and control of stimulus delivery. The basic paradigm of adaptive pacing is the simultaneous satisfaction of paradigm timing constraints that requires an interaction of performance monitoring and a requirement for variable inter-trial intervals. For the language production application discussed here, performance monitoring involves real-time evaluation of speech response content. The approach currently depends on preserving a scheduled stochastic distribution of ISIs insofar as it is compatible with waiting for confirmed successfully completed responses, together with randomization of stimulus order, to maintain an efficient design. Fig. 1. Diagram of information flow in the adaptive pacing paradigm. Control is interactive, involving the subject and a monitoring investigator. I/OWA information streams are represented by bold lines, and I/OWA time-aware processes by rounded boxes. All I/OWA information streams are recorded during the study. See text for detailed explanation.

3 T.J. Grabowski et al. / NeuroImage 29 (2006) Sound from the subject (contaminated by magnet gradient switching noise) is collected with a pneumatic system (cf. Barch et al., 1999) and digitized at 16 khz using I/OWA. The raw sound stream is processed with a streaming speech filter to remove scanner noise. The filter involves simple subtraction of samples of the raw sound stream obtained from the beginning of a run (before the first utterance) from later samples, aligning the samples on the basis of the time-stamps of the receiver unblank data stream (also obtained at 16 khz temporal resolution). Because of the fidelity of the time-stamps, this approach works remarkably well and yields easily intelligible speech, permitting the investigator to monitor the performance in real-time. In parallel, the subtracted scanner noise is thresholded to identify the onset times of spoken responses, which we refer to as the response event stream. The investigator pushes a button when a response has been successfully spoken. Then, the visual stimulator delivers the next stimulus, not immediately, but after a specified stochastic interval, contingent on receiving permission to do so by the investigator s button push. In this study, the same stochastically distributed interstimulus intervals (ISIs) are used in adaptively paced and scheduled conditions. The prescribed ISI may be timed either from the onset of the subject s response (the response event stream) or from the onset of the prior stimulus (the stimulus event stream). In this study, timing was generated from the stimulus event stream. It is not timed directly from the investigator button push because the investigator reaction time may depend systematically on trial type. Thus, stimulus delivery timing incorporates the required stochastic component, but response content is endorsed by a human monitor before stimulation is allowed to proceed. Normal subjects performing the task completely successfully will obtain the same performance schedule whether or not the adaptive pacing algorithm is turned on. However, in practice, even normal subjects make some false starts and repairs, and, whenever a correct response is delayed, the system effectively waits for the response and, if necessary, delays the presentation of the next stimulus. If no response is generated, the system times out after a user-specified interval (8 s for this study) and delivers the next stimulus. Simulations Using a representative language production paradigm schedule, we generated simulations of the performance of a subject and investigator in order to evaluate the following issues: (1) the effect of adaptive pacing on the efficiency of the experimental design for detecting the target task covariate or the contrast between two task covariates; (2) the effects of progressively longer and more variable response latencies on efficiency for the same effects; and (3) the trade-offs between running in adaptively paced mode for a prescribed number of trials vs. a prescribed duration of time. The simulator was developed with custom software embedded in a UNIX shell script (Grabowski et al., 2003). The custom software included general purpose image processing programs (Frank et al., 1997) and time-aware image processing programs (Smyser et al., 2001). The simulations modeled a target task ( naming ) and a baseline task that involved overt responses to a series of stimuli, and null events, presented in random order in a 2:1:1 ratio, with a scheduled ISI of 1.5 s, for a total of 10 min. (This schedule is the same as that used for the fast event-related schedule in the human experiment described below.) We used 2:1:1 instead of the more efficient 1:1:1 ratio because the paradigm is intended to be used with impaired subjects who may have a high proportion of failures on the naming task. The response latency of the naming task was varied systematically from 700 to 1800 ms, with standard deviation 30% of the mean latency. The response latency of the baseline task was modeled as 700 ms with a 100 ms standard deviation. The investigator s button push verification was modeled as requiring 500 ms (T100 ms), consistent with our experience with this task. For the purpose of the simulation, all trials resulted in subject response, but responses that overlapped the subsequent stimulus presentation were labeled as unsuccessful because of the collision. In this case, both trials affected by the collision were scored as unsuccessful. For adaptive pacing, no collisions occurred because stimulus presentation times were delayed with respect to the a priori schedule if the investigator button push had not yet occurred. The task reference functions for naming and the baseline task were generated by convolving the observed scheduled trial schedules with a literature BOLD response function (Cohen, 1997). In addition, for successful naming trials, the word frequency effect was modeled by convolving the mean-centered log-transformed frequency of each response with the same BOLD response function. Incorporation of the word frequency effect allowed us to assess the impact of adaptive pacing on a parametric variable, as well as the task covariates and the contrast between the task covariates. Finally, speech envelope epochs lasting 250 T 75 ms per syllable, were simulated as containing additional noise (Mehta et al., in press), and were modeled in the analyses with a boxcar regressor. The resulting functions were sampled at the modeled TR of 2 s (600 observations per scheduled run) to form the design matrix. Design efficiency was calculated for successful naming, for the contrast between successful naming and the baseline task, and for the word frequency effect, using the variance covariance matrix and the appropriate contrast matrices (Dale, 1999; Friston et al., 1999; Mechelli et al., 2003). Error terms were assumed to be uncorrelated. Image acquisition We illustrate the approach in an fmri study conducted in a 20- year-old normal right-handed (+95) female subject at 1.5 T in a GE LX CV/i scanner using a standard clinical head coil. Time series images were acquired using a T2* gradient echo single-shot EPI sequence (TE 40ms, TR 2 s, FOV 24 cm, matrix 64 64) time points (depending on task condition) composed of 24 contiguous oblique axial slices (5-mm thick), parallel to the intercommissural line and covering nearly the entire brain, were scanned. An 8-shot echoplanar T2*-weighted volume (matrix ) and T1-weighted acquisition (SPGR, flip angle 30, TR 24 ms, TE 7, FOV 24 cm, matrix ) were also acquired in the same oblique axial orientation as the EPI time series. Finally, a high resolution anatomical scan (SPGR, flip angle 30, TR 24 ms, TE 7, , slices thickness 1.5 mm) was also obtained. Task paradigm The subject was randomly presented either images of concrete non-unique entities (target task) or images of objects that had been tiled and scrambled. The latter pictures served as the baseline task and were comparable in visual complexity and intensity to pictures in the target task but were not recognizable. Images were presented on a rear-projection screen for 1.0 s. For the baseline task, the subject was instructed to name the objects as rapidly and accurately as possible and to say light or dark for scrambled images, depending on the intensity of the tiled pieces.

4 1026 T.J. Grabowski et al. / NeuroImage 29 (2006) The target task was visual confrontation naming task using a rapid event-related design with a variable ISI. The subject was not informed that some runs were adaptively paced. The following performance schedules were employed: (A) a conventional eventrelated schedule, with mean ISI 3.3 s; (B) mean planned ISI 3.3, but with adaptive pacing turned on; (C) a fast event-related schedule, with mean ISI 2.0 s; (D) mean planned ISI 2.0, but with adaptive pacing turned on. The conventional schedule was generated by scheduling naming, baseline, and null trials in random order, in a 2:1:1 ratio, separated by an ISI drawn from a Chi-squared distribution with 5 degrees of freedom, truncated such that the minimum ISI was 1.5 s and the maximum ISI was 3.5 s. The fast event-related schedule was constructed similarly, except that the ISI among events (including null events) was always 1.5 s. This schedule was identical to that employed in the simulations described above. These schedules were used twice each, for a total of eight experimental runs. (E) Finally, there was one run with a very rapid schedule (fixed planned ISI 1.0s), with adaptive pacing turned on. It was included in order to demonstrate system response characteristics using a schedule that subject was not expected to be able to perform successfully at the nominal rate. The same number of stimuli was delivered in each run, that is, run length was flexible rather than predetermined for the paced conditions. The conditions were run in counterbalanced order: ABCDEDCBA. Ancillary data acquisition Five ancillary data streams were acquired, digitized, and timestamped with the I/OWA system: (1) a sound channel; (2) the receiver unblank signal (to identify image acquisition times); (3) a stimulus photodetector channel; (4) an investigator button push; and (5) pulse oximeter (to analyze heart rate). Sound was gathered from the subject in the magnet using a plastic funnel attached to a foam-encased stiff plastic tube that extended outside the magnet and had a microphone affixed to its distal end. The sound and receiver unblank stream were sampled at 16 khz, while the stimulus photodetector, investigator button push, and pulse oximeter were sampled at 1 khz. Speech processing The raw sound stream from the scanner was re-analyzed off-line to confirm the content of subject responses, determine the successful events, and demarcate speech envelopes. The raw sound data (speech contaminated with acoustic noise from gradient switching) were filtered using a spectral subtraction algorithm (Bolls, 1979; Nelles et al., 2003) with several modifications to incorporate explicit slice timing information (Mehta et al., 2004a,b). Word boundaries defining the timing and duration of the speech envelopes (time-span from the onset to the offset of acoustic phonation) were automatically delineated using local variance estimates of the filtered sound data. Aural and visual review of the results ensured that all speech events were correctly identified. Preliminary results from a validation study indicate a speech boundary precision of 29 ms T 53 ms for onsets and 1 T 105 ms for offsets (Mehta et al., 2004a,b). Dependent measures The data from this study were evaluated for: (1) effects of adaptive pacing on subject performance quality; (2) effects of adaptive pacing on the design efficiency and the correlation between task covariates and speech envelopes; (3) the technical implementation of the algorithm; and (4) the effect of adaptive pacing on subject stress and emotion. We evaluated the impact of adaptive pacing on subject performance quality by comparing paced and unpaced runs for (1) the latency of successful responses; (2) the proportion of naming trials that were successful; (3) the rate of successful naming events per unit time. We evaluated the technical impact of the adaptive pacing algorithm by ascertaining: (1) the efficiency, E, of the emergent schedule of successful naming; and the efficiency of the contrast between the successful naming and baseline trials; and (2) the correlation, R, of the speech envelopes with the task reference functions. We evaluated the technical implementation of the algorithm in the following ways: (1) the proportion of speech events that were detected by the real-time system; (2) the amount of processing time overhead for pacing control (i.e., when the subject is behind schedule, how much time elapses between the investigator button push and the photodetection of the next stimulus). We evaluated the effect of adaptive pacing on subject stress and emotion by (1) a post-run questionnaire; and (2) the heart rate of the subject, as a function of paced v, unpaced run. The post-run questionnaire was administered orally after each run and included six questions: (1) how was the speed of this task (1 too slow 3 = appropriate 5 = too fast); (2 5) how difficult/stressful/mentally tiring/hurried did you find this task (1 5, 1 = not at all, 3 = somewhat, 5 = very); and (6) how would you estimate your performance on the naming task, in terms of percent correct? Image processing Finally, we evaluated the effect of adaptive pacing on activation images obtained during the imaging session. Image registration operations were performed using Automated Image Registration, AIR (Woods et al., 1998a,b). The first three images in the time series were discarded to avoid saturation effects, and images within a run were aligned to the 4th image of the time series using a 3D 6- parameter rigid body model. Data from all runs were then aligned to the average image of the first run, smoothed with a mm Gaussian kernel, and analyzed in this orientation. The average EPI time series image was also registered to the 3D structural MRI, using the 8-shot EPI and the T1-weighted images as intermediaries. These registration parameters allowed the statistical images obtained from the analysis to be transformed and overlaid on the anatomical scan. Images from all but the very fast paced condition were analyzed using the general linear model in a single regression analysis. The design matrix included the following sets of covariates: (1) global mean, (2) run effects, (3) low frequency regressors nested within run, (4) speech envelope boxcar function, and (5 8) task reference functions for successful naming, failed naming, and the baseline task nested within the 4 conditions (slow scheduled, slow paced, fast scheduled, and fast paced). Test statistics were computed for paced and scheduled successful naming, as well as for the contrast between the two. Results Effect of pacing on task performance Error rates were 5 10% and were the same for paced and unpaced conditions (Table 1). The error rate was higher for runs

5 T.J. Grabowski et al. / NeuroImage 29 (2006) Table 1 Effect of pacing status and mean ISI on task performance success and response latency Condition ISI (seconds) Success rate (%) Success per time Mean latency, milliseconds (max) Scheduled ISI ( ) (1640) Paced ISI ( ) (1330) 1.1% Scheduled ISI ( ) (1420) Paced ISI ( ) (1390) 11.0% Paced ISI ( ) (1390) 95.1% % trials behind schedule with shorter mean ISI. Successful performance per time was greater for runs with shorter ISI. There was a significant effect of mean ISI on naming latency (t(478) = 4.74, P < 0.001), but there was no significant difference in naming latency between scheduled and paced runs (t(478) = 1.12 P = 0.27). There was also no significant difference when the data were analyzed separately for long and short ISI runs. Technical implementation As expected, the subject performed slower than the prescribed schedule on many of the trials for the very rapid run. On 95% of trials in this run, the investigator button push endorsing the subject response did not occur before the next scheduled stimulus (on average 0.19 s late, with a maximum of 0.94 s). On these behind schedule trials, presentation of the next stimulus was delayed until completion of the previous trial. On the conventional (nominal ISI 3.3s) and rapid paced (nominal ISI 2.0s) runs, 1.1% and 11.0% of trials were behind schedule, respectively. On behind-schedule trials, the mean time delay due to control processes (e.g. the time interval between the investigator s button push and the photodetection of the next stimulus) was 0.22 s. This processing overhead was found to be due principally to reading the next stimulus from disk. We have subsequently reduced this system delay to 0.07 s by reading stimuli from disk into memory ahead of the investigator button push. Across the five paced runs, the realtime speech filter and speech thresholding processes detected 96.4% of naming responses. Simulations The rate of invalid trials due to collisions was 13% for mean response latency 700 ms, 45% for latency 1000 ms, 67% for latency 1300 ms, 75% for latency 1500 ms, and 80% at the 1800 ms time point. The invalid rate is about twice the collision rate since each collision invalidated two task trials. The results of the adaptive pacing schedule simulations on design efficiency, as a function of mean response latency, are displayed in Fig. 2. Three schedules are depicted in these graphs: (1) adaptive pacing for the scheduled number of stimuli (i.e., running up to 50% longer than the nominally scheduled 10 min) (PACED-TRIALS); (2) adaptive pacing for the scheduled run duration (i.e., stopping at 10 min, before all scheduled stimuli have been delivered) (PACED-time); and (3) no pacing (UNPACED). The effects of adaptive pacing on efficiency differed, depending on whether the parameter of interest was the naming covariate, contrast, or word frequency effect, a result that accords with prior publications (Friston et al., 1999; Mechelli et al., 2003) that have emphasized that design efficiency is dependent on the question being asked. For successful naming trials, efficiency declined more rapidly with response latency (i.e., collision rate) for UNPACED relative to PACED conditions, but this effect was not apparent until the 1200 ms point (60% invalid trials in the UNPACED condition). Moreover, up to that point, the UNPACED condition was more efficient than the PACED conditions. It may be that the invalidation of the shortest ISI trials changed the distribution of stimulus delivery times such that the UNPACED successful naming covariate returns to baseline more often, whereas the PACED condition remains semi-saturated. With longer latencies, more trials are missed in the UNPACED condition, and the naming task reference function is eventually disproportionately weighted to 0 and does not achieve the same maximization of magnitude through linear summation as the PACED condition. For the contrast of naming and the baseline task, clear efficiency advantages emerged for the PACED conditions. PACED-time diverged from UNPACED beyond 1000 ms (45% of trials invalid in the UNPACED condition), whereas PACED- TRIALS had superior efficiency at all time points. Pacing was especially advantageous for the word frequency effect, for which efficiency was robust to increases in response latency. PACED-TRIALS showed no loss of efficiency at all with respect to mean response latency. In the simulations, the mean (and max) absolute correlation coefficients for the speech envelope with the successful naming task reference function were (0.140) for UNPACED, (0.089) for PACED-time, and (0.070) for PACED-TRIALS. Similar results were obtained when speech envelopes were correlated with a combined reference function for the naming and baseline tasks. Thus, adaptive pacing did not increase the correlation of speech envelopes with task reference functions. Effect of pacing on experimental design metrics For the human subject study, the correlation coefficients between speech envelopes and the reference functions for all naming events, or all speech events, were consistently less than 0.1 and were not related to pacing status (Table 2). The efficiency of the de facto experimental design (i.e., as actually performed), for successful naming events, or the contrast between naming and the baseline task, was greater (as expected) for runs with shorter ISI but was not significantly affected by pacing status (Table 2). Effect of pacing on subject stress and emotion The post-run questionnaire results (Table 3) indicated that the subject found the task to be challenging. She estimated her success rate on the naming items as about 80%. She rated the paced and unpaced runs as having the same degree of difficulty and stressfulness. She consistently reported the paced runs to be less

6 1028 T.J. Grabowski et al. / NeuroImage 29 (2006) Table 2 Effect of pacing status and mean ISI on correlation of task reference functions (R) with speech envelopes, and on design efficiency (E), normalized for run length Condition R, succ nam R, all tasks E, succ nam E, contrast Scheduled ISI Paced ISI Scheduled ISI Paced ISI tiring and that she felt less hurried during them. Her responses regarding the speed of the task reflect the fact that she found the ISI 3.3 runs to be too slow. She said after the study that waiting for stimuli made the task seem tiring. She found the paced ISI 2.0 runs to be the least tiring run. Mean heart rate did not vary significantly across the conditions. Activation images The activation results for the demonstration subject are depicted in Fig. 3, which displays the naming covariate for paced runs, scheduled runs, and the contrast between the covariates for paced and scheduled runs. Paced and scheduled designs gave very similar activations. This result accords with the result of the simulation, which showed that, given a normal performance quality (latency 780 ms, success rate 93%), PACED-TRIALS has a minimal efficiency advantage over UNPACED. Discussion Fig. 2. Efficiency of task covariates and contrasts in simulations. (A) Efficiency of the naming covariate, defined over all naming trials in which collisions of adjacent trials did not occur. (B) Efficiency of the contrast between naming and baseline covariates, defined over all naming and baseline task trials in which collisions of adjacent trials did not occur. (C) Efficiency of the word frequency effect, defined for all naming trials in which collisions of adjacent trials did not occur. For each panel, three simulated task schedules are displayed at each time point: (1) paced for the scheduled number of stimuli (denoted PACED-TRIALS in the text); (2) paced for the scheduled duration of run (PACED-time); and (3) not paced (UNPACED). The pacing simulations show that adaptive pacing of tasks involving speech production, as implemented here, may successfully improve the number of uncontaminated task trials while avoiding the confounds of speech-related variance and resisting the loss of design efficiency, especially for task contrasts and parametric effects. The latter effects in particular show preserved efficiency. Maximizing the number of successful trials maximized the efficiency for a parametric effect. The demonstration experiment shows that adaptive pacing and real-time monitoring of response content are technically feasible in fmri using time-aware methods. Response detection was robust, efficiency was preserved, and temporal correlation of task reference functions with speech envelopes was avoided. The processing time requirement for the control processes is 0.07 s per trial and therefore too small to interfere with an experimental design. A schedule delay due to investigator reaction time necessarily occurs, but, in practice, it overlaps substantially with the subject s speech envelope and is of little practical consequence. In this normal subject, performance success was high across all the task conditions. She was not aware that the stimulus presentation schedule was varying between scheduled and adaptive pacing conditions but was aware of the changes in nominal mean ISI. The subject found slow presentation rates to be tiring and indicated on post-run questionnaires that she preferred the paced ISI 2.0 s condition. In this condition, 11% of trials were behind schedule, but, because of adaptive pacing, no trial collisions occurred, and she maintained a 93% performance success rate. This condition led to a much improved rate of successful performance

7 T.J. Grabowski et al. / NeuroImage 29 (2006) Table 3 Effect of pacing status and mean ISI on subject stress and emotion Condition Subject ratings, post-run questionnaire Mean Difficult Stressful Tiring Hurried Summed index Subjective % correct HR (bpm) Scheduled ISI Paced ISI Scheduled ISI Paced ISI Paced ISI per unit time and improved de facto design efficiency, relative to the ISI 3.3 cases (paced and unpaced). Although accuracy was similar across all runs, response latencies were significantly briefer when nominal mean ISI was shorter, despite the instruction to name the items as quickly and accurately as possible in all runs. This observation points out the sensitivity of task performance (and presumably the fmri correlates of the performance) to the perceived experimental demand and underlines the importance of minimizing differences in perceived demand across subjects with different abilities, especially when comparing data from normal and impaired subjects. The methods presented here point the way to finding a performance rate that will optimize efficiency for subjects with different performance capacities. The purpose of this technical report is to establish the feasibility and utility of this approach. We will be turning next to experiments in impaired populations. The issue arises whether the adaptive pacing method could introduce between-subject differences that might confound analyses. By design, this approach might result in differing numbers of trials per subject, but regression coefficients estimated by the general linear model are not biased by differences in the numbers of trials per subject. There may however differences in the precision of these estimates across subjects. In return, adaptive pacing helps ensure that the desired processes are being studied, by fostering successful performance, tending to better equate perceived performance demands, and minimizing subject frustration. It is probably more desirable to develop ways to equate the precision of the estimates across subjects (i.e., standardize the standard error of regression coefficients) than it is to attempt to standardize the rate and/or number of performances across subjects. Finally, the simulations suggest that the precision of the estimates of parametric effects may be less sensitive to differences in stimulus schedules. The innovative aspects of the scheme presented here are the use of data streaming architecture to implement real-time interactive paradigm control and the implementation of timing constraints to foster design efficiency in the face of real-time adjustments in paradigm schedules. While novel, the inclusion of the investigator is not central to the logic of adaptive pacing but instead is specific to its application to a paradigm requiring overt speech production. There is currently no other practical way to evaluate spoken response content in real-time. For some paradigms with narrow response options (such as naming entities with high name agreement), it might be possible in the future to use speech recognition algorithms to replace the investigator in the evaluative role. Nevertheless, the ability to have the investigator act as a gatekeeper is advantageous since it allows a broader range of experiments (e.g. interactive paradigms in which different response types trigger alternate stimulation options). Maintaining a stochastic ISI currently depends (1) on preserving an a priori stochastic distribution of ISIs, insofar as is possible; and (2) taking advantage of normal response latency variability (see Macotta et al., 2001) in combination with random trial order. In the future, a real-time, dynamic calculation of efficiency might be used to better adapt the paradigm schedule to an impaired subject. Fig. 3. Activation results for the demonstration subject. The pseudocolor scale displays the t statistic for successful naming trials, superimposed on the subject s T1-weighted anatomic image (left), for (center left) paced runs; (center right) scheduled runs; and (right) the contrast between these covariates between paced and scheduled runs.

8 1030 T.J. Grabowski et al. / NeuroImage 29 (2006) This system described here has been implemented and is now in routine use in our laboratory. It is suitable for application to patients with language production disorders (anomia, stuttering, aphasia), where it can reduce frustration with task performance rate and accommodate improvement or worsening of impairments over time. The benefits of adapting fmri to impaired subjects, rather than expecting them to conform to rigid paradigm designs, are expected to be the accrual of a greater proportion of the data during performances of interest, better efficiency, and less contamination of data by correlates of other processes, notably emotion. The adaptively paced overt visual naming paradigm described above is an example of a new class of paradigms that require an observational approach to data acquisition and that benefit from time-aware acquisition and processing architecture. Although we demonstrate this approach in the context of a specific language task dedicated to the assessment of aphasia, the methodology it establishes is flexible and extensible. For example, it is straightforward to introduce a more elaborate contingency on the content of the subject s response, e.g. selection of the next stimulus type based on the response to the current stimulus. It is applicable to the study of a wide range of cognitive impairments. It is particularly suitable for longitudinal studies. We envision applications in rehabilitation planning and evaluation, wherein imaging techniques like those proposed here serve as surrogate markers to predict and assess efficacy of treatment interventions. Acknowledgments This work was supported in part by R33 EB and R21 EB The authors thank Vincent Magnotta for technical assistance with data collection. The I/OWA software is available to interested investigators under a BSD-style public license. Please contact the corresponding author. References Barch, D.M., Sabb, F.W., Carter, C.S., Braver, T.S., Noll, D.C., Cohen, J.D., Overt verbal responding during fmri scanning: empirical investigations of problems and potential solutions. NeuroImage 10 (6), Birn, R.M., Bandettini, P.A., Cox, R.W., Shaker, R., Event-related fmri of tasks involving brief motion. Hum. Brain Mapp. 7 (2), Bolls, S., Suppression of acoustic noise in speech using spectral subtraction. IEEE Trans. Acoust. Speech Signal Process. 27 (2), Cohen, M., Parametric analysis of fmri data using linear systems methods. NeuroImage 6, Dale, A.M., Optimal experimental design for event-related fmri. Hum. Brain Mapp. 8 (2 3), Daselaar, S.M., Rombouts, S.A., Veltman, D.J., Raaijmakers, J.G., Lazeron, R.H., Jonker, C., Parahippocampal activation during successful recognition of words: a self-paced event-related fmri study. Neuro- Image 13, Daselaar, S.M., Veltman, D.J., Rombouts, S.A., Raaijmakers, J.G., Jonker, C., Neuroanatomic correlates of episodic encoding and retrieval in young and elderly subjects. Brain 125, Frank, R.J., Damasio, H., Grabowski, T.J., Brainvox: an interactive, multimodal, visualization and analysis system for neuroanatomical imaging. NeuroImage 5, Friston, K.J., Zarahn, E., Josephs, O., Henson, R.N., Dale, A.M., Stochastic designs in event-related fmri. NeuroImage 10, Grabowski, T.J., Damasio, H., Cooper, G.E., Tranel, D., Frank, R.J., Ponto, L.L.B., Watkins, G.L., Hichwa, R.D., Residual naming after damage to left temporal pole: a PET activation study. NeuroImage 19, Hajnal, J.V., Myers, R., Oatridge, A., Schwieso, J.E., Young, I.R., Bydder, G.M., Artifacts due to stimulus correlated motion in functional imaging of the brain. Magn. Reson. Med. 31 (3), Huang, J., Carr, T.H., Cao, Y., Comparing cortical activations for silent and overt speech using event-related fmri. Hum. Brain Mapp. 15 (1), Macotta, L., Zacks, J.M., Buckner, R.L., Rapid self-paced eventrelated functional MRI: feasibility and implications of stimulus- vs. response-locked timing. NeuroImage 14, Mechelli, A., Price, C.J., Henson, R.N., Friston, K.J., Estimating efficiency a priori: a comparison of blocked and randomized designs. NeuroImage 18, Mehta, S., Ballard, K.J., Moon, J.B., Graves, W.W., Grabowski, T.J., 2004a. Validation of detecting speech boundaries in fmri. NeuroImage 22 (S1), 239. Mehta, S., Grabowski, T.J., Razavi, M.R., Rudrauf, D., Cole, J., Bolinger, L., 2004b. Identifying and modeling the time-course of speech-related signal variance in fmri. NeuroImage 22 (S1), WE 240. Mehta, S., Grabowski, T.J., Razavi, M.R., Eaton, B., Bolinger, L. Analysis of speech-related variance in rapid event-related fmri using a timeaware acquisition system. NeuroImage (in press). Nelles, J.L., Lugar, H.M., Coalson, R.S., Miezin, F.M, Petersen, S.E., Schlaggar, B.L., Automated method for extracting response latencies of subject vocalizations in event-related fmri experiments. NeuroImage 20 (3), Smyser, C., Grabowski, T.J., Frank, R.J., Haller, J.W., Bolinger, L., Real-time multiple linear regression for fmri supported by time-aware acquisition and processing. Magn. Reson. Med. 45 (2), Woods, R.P., Grafton, S.T., Holmes, C.J., Cherry, S.R., Mazziotta, J.C., 1998a. Automated image registration: I. General methods and intrasubject, intramodality validation. J. Comput. Assist. Tomogr. 22 (1), Woods, R.P., Grafton, S.T., Watson, J.D., Sicotte, N.L., Mazziotta, J.C., 1998b. Automated image registration: II. Intersubject validation of linear and nonlinear models. J. Comput. Assist. Tomogr. 22 (1),