CRITICAL REVIEW AND DISCUSSION

Transcription

1 CRITICAL REVIEW AND DISCUSSION Industrial-Organizational Psychology and Organizational Behavior Management: An Objective Comparison Barbara R. Bucklin Alicia M. Alvero Alyce M. Dickinson John Austin Austin K. Jackson ABSTRACT. This article compares traditional industrial-organizational psychology (I-O) research published in Journal of Applied Psychology (JAP) with organizational behavior management (OBM) research published in Journal of Organizational Behavior Management (JOBM). The purpose of this comparison was to identify similarities and differences with respect to research topics and methodologies, and to offer suggestions for what OBM researchers and practitioners can learn from I-O. Articles published in JAP from were reviewed and compared to articles published during the same decade in JOBM (Nolan, Jarema, & Austin, 1999). This comparison includes Barbara R. Bucklin, Alicia M. Alvero, Alyce M. Dickinson, John Austin, and Austin K. Jackson are affiliated with Western Michigan University. Address correspondence to Alyce M. Dickinson, Department of Psychology, Western Michigan University, Kalamazoo, MI ( wmich.edu.) Journal of Organizational Behavior Management, Vol. 20(2) 2000 E 2000 by The Haworth Press, Inc. All rights reserved. 27

2 28 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT (a) author characteristics, (b) authors published in both journals, (c) topics addressed, (d) type of article, and (e) research characteristics and methodologies. Among the conclusions are: (a) the primary relative strength of OBM is its practical significance, demonstrated by the proportion of research addressing applied issues; (b) the greatest strength of traditional I-O appears to be the variety and complexity of organizational research topics; and (c) each field could benefit from contact with research published in the other. [Article copies available for a fee from The Haworth Document Delivery Service: address: <getinfo@haworthpressinc.com> Website: < KEYWORDS. Industrial-organizational psychology, organizational behavior management, Journal of Applied Psychology, Journal of Organizational Behavior Management, objective comparison In 1989, Balcazar, Shupert, Daniels, Mawhinney and Hopkins (1989) reviewed and analyzed the articles that were published in the first 10 years of the Journal of Organizational Behavior Management (JOBM) to assess whether the Journal was meeting its original objectives. Nolan, Jarema and Austin (1999) recently analyzed JOBM articles from as a follow-up assessment. Data collected by Nolan et al. was used in this study to compare current research topics and methodologies in Organizational Behavior Management (OBM) to those in traditional Industrial-Organizational (I-O) psychology. To make this comparison, we reviewed and analyzed articles published in the Journal of Applied Psychology (JAP) for the same ten-year period ( ). The purpose of the comparison was to identify similarities and differences with respect to research topics and methodologies used in OBM and I-O psychology. To provide context for the comparison of these two fields, we briefly describe the history, topics of interest, and conceptual and theoretical underpinnings of both I-O psychology and OBM, after which we describe the primary publication outlets for these fields, JAP and JOBM, respectively. We then compare the (a) author characteristics, (b) authors published in both journals, (c) topics addressed, (d) types of articles, and (e) research characteristics and methodologies. We conclude with a general discussion about similarities and differences, relative strengths and weaknesses, suggestions for what OBM can learn from I-O, and questions regarding the future relationship between OBM and I-O psychology.

3 Critical Review and Discussion 29 Industrial-Organizational Psychology History In a recent issue of JAP, Katzell and Austin (1992) provided an extensive history of the development of I-O psychology. We summarize the major events and influences here, but interested readers should see Katzell and Austin for a more detailed account. I-O psychology was shaped by many events in the early 1900s including the publication of Psychology and Industrial Efficiency, by Hugo Munsterberg in 1913, the first department of applied psychology at Carnegie Institute of Technology (now Carnegie-Mellon University) in 1915, and the initiation of JAP in During this time, applied psychologists began to address two major areas of application in work settings: personnel selection and placement, and productivity improvement (Aamodt, 1991; Hilgard, 1987). The development and validation of selection instruments for military personnel in World War I resulted in further progress and recognition for I-O psychology (Scott, 1920). The first Ph.D. in I-O psychology was awarded to Bruce Moore from the Carnegie Institute in By the end of the 1920s, there were approximately 50 Ph.D. level I-O psychologists in the country, and major universities were adding increasing numbers of I-O faculty. The Hawthorne studies, conducted at Western Electric Company in the 1930s, are often cited as one of the most salient developments in the field (e.g., Aamodt, 1991; Hilgard, 1987; Katzell & Austin, 1992). These studies were among the first scientific experiments conducted in an organizational setting and, in addition, expanded the topics that I-O psychologists examined to variables such as the work environment, wage incentives, and employee attitudes. Prior to that time, I-O psychologists were mainly involved in personnel issues. In essence, the O in I-O psychology can be attributed to the Hawthorne studies and the new research they fostered. During the past sixty years, I-O psychology has grown tremendously. Current membership in the Society for Industrial-Organizational Psychology (SIOP) (established as the Society for Industrial Psychology in 1945) is over 5,000, and more than 90 universities now offer Ph.D. s in I-O psychology. Although personnel selection and placement remains one of the largest areas in the field, research and application now cover a wide array of topics that will be detailed later in this article.

4 30 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT Theory and Concepts In the first handbook of I-O psychology, published in 1976, England stated, Industrial and Organizational Psychology possesses no unified and generally accepted theoretical or conceptual base (p. 15). In a more recent version of the handbook, Dunnette (1990) explained that this has not changed; no one unifying underlying theory exists today. He stated, however, that a primary purpose of the I-O handbook was to describe the dozens of theories that do exist and consider the relevance of each to the field. Individual traits and individual differences were the primary theoretical concepts underlying the initial development of I-O psychology. Early I-O psychologists developed mental tests, measurement tools and statistical analyses to identify individual differences in order to select employees and identify performance differences on various work tasks (Ackerman & Humphreys, 1990; Guion, 1976). Selection and placement has been described as the hallmark of traditional I-O psychology and, prior to the 1970s, situational or environmental variables were not typically considered in the selection of individuals for employment (Guion, 1976). In addition to a general theoretical focus on individual differences, more specific theories were also responsible for shaping the field. For example, as one explanation for the lack of behavior analytic influence on traditional I-O, Weiss (1984, 1990) described the influence of Kurt Lewin and explained that Lewin used Galilean models to develop his own explanation for behavior. Weiss explained, He [Lewin] concluded that the best way to conceptualize the causes of behavior was in terms of the immediate relationship between the person and the environment (1990, p. 174). Weiss also stated that Lewin disdained behaviorist explanations as well as any theory that sought causes for current behavior in past history. Weiss maintained that Lewin s influence remains strong, indicating that Lewin s theoretical concepts can be seen in expectancy theory, goal setting theory, leadership theory and several organizational development theories. Starting in the 1960s, I-O psychology became heavily influenced by cognitive psychology as well. This influence resulted in an emphasis on mental processes to explain work-site measures such as supervisory performance ratings, skill acquisition, transfer of training, and leadership (Katzell & Austin, 1992; Lord & Maher, 1990).

5 Critical Review and Discussion 31 These theoretical influences (i.e., trait theory, Lewinian theory, and cognitive psychology) are evident in the research questions and methodology used by mainstream I-O psychologists and also account for many of the differences between I-O and OBM research topics. Not surprisingly, much of traditional I-O research is theory-driven, and designed to test hypotheses derived from various theories. Journal of Applied Psychology In the I-O Handbook, Dunnette (1990) stated that JAP is one of the key journals that serve as publication outlets for I-O psychologists. Other authors have cited JAP as the premier publication journal for I-O researchers and practitioners (e.g., Darley, 1968; Katzell & Austin, 1992; Lowenberg & Conrad, 1998). In 1917, JAP was the first journal to publish I-O psychology research. The editors of the first volume of JAP described the purpose of the journal as an outlet for the publication of (a) the application of psychology to law, art, public speaking, industrial and commercial work, and business problems, (b) studies of individual differences, (c) the influence of environmental conditions, and (d) the application of psychology to everyday activities (Hall, Baird, & Geissler, 1917). These first editors explained that, the most strikingly original endeavor to utilize the methods and the results of psychological investigation has been in the realm of business (p. 5). The purpose of the journal does not appear to have changed significantly over the past 80 years. When the current JAP editor, Kevin Murphy, became editor in 1997 he stated JAP s mission was devoted primarily to original investigations that contribute knowledge or understanding to the fields of applied psychology other than clinical and applied experimental human factors (Murphy, 1997, p. 3). Murphy explained that papers published in JAP should contribute to the interaction between basic and applied psychology in settings where applied research is conducted (e.g., organizations, military and educational settings). However, he clarified that it is not necessarily the setting (e.g., field or laboratory) that makes an article relevant to applied psychology; rather it is its contribution to the field.

6 32 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT ORGANIZATIONAL BEHAVIOR MANAGEMENT Theory and Concepts The history and theoretical basis of OBM are so intertwined that it is difficult to explain one without the other. Unlike I-O psychology, OBM has one consistent theoretical basis. OBM began as the application of behavior analysis to organizational settings and retains the philosophical and methodological principles of behavior analysis. In an overview of behavior analysis, Michael (1993) explained that many have considered behavior analytic work to be a sub-class of learning theory, while others have viewed it as anti-theoretical. Michael argued that it does not fit the learning theory description, because in addition to learning (i.e., in the sense of skill acquisition), behavior analysts are concerned with the maintenance of skills following acquisition (e.g., schedules of reinforcement). Although Michael contended that behavior analysis is not anti-theoretical, the purpose of behavior analytic research is not specifically to test theories and furthermore, it is not an application of the hypothetical deductive model. Rather than antitheoretical (except with respect to inferred mental events) it is a deterministic view that sees human behavior as the inevitable product of innate endowment and environmental events taking place during the person s lifetime (Michael, 1993, p ). In a recent discussion of behavioral principles in OBM, Hopkins (1999) described behavior analysts reluctance to call the behavioral principles a theory. Hopkins suggested that the principles be called an empirical theory to differentiate this type of theory from most cognitive psychology theory that makes use of untestable mental events to causally explain behavior. However, whether or not behavior analytic principles are referred to as theory, one of the fundamental differences between OBM and I-O is the descriptive, empirical influence that behavior analysis has had on OBM, and the hypothetical, theory testing influence that cognitive psychology has had on traditional I-O. Early theoretical influences that shaped the foundation of behavior analysis (i.e., OBM) also differ from those that influenced early I-O psychology. As previously mentioned, early I-O was partially shaped by Lewin s theory, which was influenced by Galilean models derived from the laws of physics (Weiss, 1990), whereas, OBM s behavior analytic theory was derived from Darwin s influence on Skinner (Donahoe & Palmer, 1994; Michael, 1993) and the principle of behavioral

7 Critical Review and Discussion 33 selection by consequences. In other words, this difference is between modeling causal laws of behavior from the laws of physics (i.e., Galilean s a historic approach) versus modeling them from biology (i.e., Darwin s selectionist approach) (Donahoe & Palmer, 1994; T. Mawhinney, personal communication, December, 1999). Although Darwin s approach was a major influence on Skinner and the field of behavior analysis, there were others that helped shape Skinner s intellectual repertoire. In a chapter entitled, Historical Antecedents to Behavior Analysis, Michael (1993) provided a detailed description of those influences. In addition to Darwin, he noted the contributions of Francis Bacon, Ivan Sechenov, Ernst Mach, Edward Thorndike, Ivan Pavlov, John Watson, Betrand Russell, Jacques Lobe, and W. J. Crozier. Readers are referred to that chapter for a detailed description of those contributions. Duncan and Lloyd (1982) contended that an understanding of the theory and philosophy of behaviorism was necessary for successful OBM practitioners. In other words, practitioners should view behavior as naturally-occurring, scientific subject matter, and understand that orderly relations between behavior and the environment allow for the prediction and control of behavior. In addition to a theoretical understanding, knowledge of the experimental principles of behavior (e.g., reinforcement, punishment, stimulus control, discrimination and generalization) is necessary for successful application of behavior analysis to organizational problems. Analyses of work behavior in terms of the principles of behavior analysis are provided in many sources (e.g., Brown, 1982; Daniels, 1989; O Brien & Dickinson, 1982; Mawhinney, 1984). For particularly detailed analyses, including the role of rules and establishing operations, readers are referred to Johnson, Redmon, and Mawhinney (in press), Mawhinney and Mawhinney (1982) Mawhinney and Fellows-Kubert (1999) and Poling and Braatz (in press). History An extensive history of the field of OBM is beyond the scope of this paper, thus for a more detailed account, readers are encouraged to see Frederiksen s (1982) introduction to the Handbook of Organizational Behavior Management, or Dickinson s (in press) article, The Historical Roots of OBM in the Private Sector: The 1950s-1970s. Although the history of OBM is short compared to the history of I-O psycholo-

8 34 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT gy, some of the early influences on the field of OBM also influenced the field of I-O psychology. For example, in their historical account of I-O psychology, Katzell and Austin (1992) cited John B. Watson and E. L Thorndike as important contributors to early I-O. The work of Watson and Thorndike also influenced B. F. Skinner and subsequently the field of behavior analysis on which OBM is based (Frederiksen, 1982; Michael, 1993). Frederiksen cited other individuals and events as important precursors to both fields including Fredrick Taylor and his approach to scientific management, the Hawthorne studies, and Munsterberg s application of psychology to industrial settings. Dickinson (in press) did not include these influences as precursors in her history of OBM. Rather she restricted her account to events and individuals within the behavioral community, such as Skinner s development of programmed instruction and the advent of behavior modification in other settings, contending that while the early events in I-O were chronological precursors, they were not causal precursors. She maintained that OBM developed in relative isolation from traditional I-O events, and that those events influenced OBM only after the field expanded in the late 1970s and the 1980s. Nonetheless, it is certainly the case that the application of psychology to the work site predated behavioral involvement and that this earlier work subsequently, if not immediately, helped shape OBM. OBM did not emerge as a separate field until the 1960s (e.g., Andrasik, 1979; Daniels, 1989; Dickinson, 1995, in press; Frederiksen & Johnson, 1981; O Brien & Dickinson, 1982). Frederiksen (1982) stated the decade of the seventies was a period of accelerated growth and integration of the field (p. 8). Early OBM interventions primarily addressed small-scale organizational problems, but OBM was considered to be a promising approach to performance improvement in a large range of settings (Frederiksen, 1982). Through the 1970s OBM became much more widely researched and applied, with a substantial increase in the volume of publications. Whereas fewer than 10 or so articles were published in the 1960s, more than 45 had been published by 1977, the year that JOBM was initiated (Dickinson, in press). JOBM was begun by Behavioral Systems, Inc., a behavioral consulting firm, to disseminate OBM applications and was the first journal devoted solely to the publication of OBM interventions (Dickinson, in press). The Journal quickly became the flagship journal of the field. At approximately the same time that JOBM was first published, the

9 Critical Review and Discussion 35 OBM Network was formed as a professional association for OBM researchers and practitioners (Dickinson, in press; Frederiksen, 1982). In Frederiksen s (1982) account, he described four features that define the field of OBM. These features include the purpose, the subject matter, the theoretical and conceptual basis, and the methodology. Three of these four features are unique to OBM. However, the purpose of OBM as a method to improve performance and satisfaction and to make organizations more effective in achieving their goals, is similar to the purpose of other approaches that study and apply interventions in work settings (e.g., I-O psychology). Although improved performance and organizational effectiveness are undeniably purposes of OBM, increased satisfaction as a purpose is debatable. In 1984, Mawhinney provided some evaluative feedback to OBM, and stated that although OBM researchers and practitioners professed a concern for both productivity and satisfaction, satisfaction was rarely measured. He argued that it should be: If we are seriously committed to the values of improved productivity and job satisfaction we must come to grips with the satisfaction issue. Our theory is clear on this point. We can achieve high productivity and high satisfaction. But we can also achieve high productivity with low satisfaction. Unless we measure Eden-actual value received discrepancies (dissatisfaction) we cannot hope to achieve our equally worthy objectives of high productivity and high work satisfaction. (p. 23) Later in this comparison we will discuss the current frequency of social validity (i.e., satisfaction) measures in OBM and in I-O psychology and raise questions about whether OBM should adopt methods for social validity assessment from traditional I-O. The other features of OBM, such as the behavior of individuals and groups in organizational settings as the primary subject matter, clearly differentiates it from other approaches that tend to rely on self-reports and mentalistic constructs as subject matter. Furthermore, OBM s theoretical and conceptual basis, behavior analysis, results in a clear difference between OBM and traditional I-O psychology. Traditional researchers often infer underlying mental processes and use these to explain behavior, rather than analyzing the relationship between behavior and the environment. OBM relies on direct observation of

10 36 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT behavior as its main dependent variable rather than survey data, which is often used in I-O research (Frederiksen, 1982). In addition, when I-O psychologists study direct measures of behavior, these observations are typically collected during a one-time cross-sectional event. OBM measures, on the other hand, are usually collected repeatedly over time and assessed using a within-subject design. This latter methodology results in an emphasis on practical significance as a measure of successful OBM interventions. Conversely, statistical significance is often used as the measure of successful interventions in I-O research. These and other features that differentiate OBM from traditional approaches will be compared and discussed later in this review. Journal of Organizational Behavior Management In the first issue of JOBM, Aubrey Daniels (1977), then editor, described the purpose of the Journal as three-fold: (a) research should meet the criteria described a decade earlier by Baer, Wolf and Risley (1968) for applied behavior analysis; (b) these behavioral methodologies should be applied to organizational settings; and (c) in addition to the value of the Journal to OBM researchers and practitioners, it should also have practical value for managers. In their 1989 review of JOBM, Balcazar et al. found that the Journal was clearly meeting the first two stated objectives, but perhaps not the third. In Nolan et al. s (1999) recent review of the second decade ( ), they agreed that the Journal was meeting the first objective but stated, However, the remaining objectives are not directly addressed by the data collected in the current review (and neither were they, we feel, in that of Balcazar et al., 1989) (p. 109). They offered suggestions for data collection that would address the remaining objectives as well as suggestions that could result in increased dissemination of OBM to the general business public. Readers are referred to that article for further detail. HISTORICAL SIMILARITIES AND DIFFERENCES While there are clearly differences in the concepts and theories that formed and underlie the fields of OBM and I-O, the purpose for both

11 Critical Review and Discussion 37 fields is essentially the same: to improve the performance and satisfaction of individuals in business settings in order to ensure the efficiency and effectiveness of organizations. From the time of their respective inceptions, both fields have faced similar dilemmas, such as the distribution of resources devoted to research and practice, and the extent to which research findings have been used to solve practical problems (Frederiksen, 1982; Katzell & Austin, 1992). In our comparison, we will address the respective emphases on theoretically oriented research (I-O) versus practical research (OBM). Katzell and Austin (1992) explained that OBM had some influence on traditional I-O psychology as OBM emerged in the 1970s; however, OBM has remained largely outside the mainstream of I-O. The use of different conceptual and methodological approaches to guide and explain research has no doubt resulted in reluctance from both fields to adopt methods and ideas from the other. However, despite the previous lack of cross-fertilization between fields, Katzell and Austin stated that some OBM influence has returned to I-O psychology. For example, Katzell and Austin cited Komaki s work on supervision and teams (e.g., Komaki, Desselles, & Bowman, 1989) as evidence that OBM research has been making a reappearance in JAP. Behavioral research and methods have also appeared in recent I-O text books (e.g., Lowenberg & Conrad, 1998; Muchinsky, 1997). The purpose of this paper is to identify and discuss the similarities and differences between fields during the most recent decade. METHOD The first author reviewed every article published in JAP between 1987 and 1997, including short notes, research reports and monographs (N = 997). The second author independently reviewed every article in volumes published in even years (N = 452). The categories and operational definitions used to classify the articles were derived from those developed by Nolan et al. (1999). Nolan et al. based their categories on those used by Balcazar et al. (1989) in their review of the first ten years of JOBM, however, they added sub-categories to some of variables for a more detailed analysis. Some of the categories in Nolan et al. s review were not used in the present review because they would not have resulted in relevant comparisons (e.g., number of pages published). One category (correlational research versus experimental research) was

12 38 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT added for purposes of the current comparison. For this new category, the first and second authors evaluated the articles in JOBM using the same method they used to review the JAP articles. The categories and definitions used to classify articles are detailed in the following section. The data-recording sheet used by the first two authors to classify the articles listed all of the relevant categories and sub-categories and authors circled the appropriate classification. The Appendix identifies a random sample of the JAP articles that were reviewed and indicates how they were classified with respect to each of the relevant categories. Data are provided for 45 articles, which represents 4.5% of the articles that were reviewed. It was not feasible to publish all of the data due to the large number of articles (N = 997). The complete JAP data base is, however, available from the third author. Interobserver agreement was calculated for every article that was reviewed by both authors. The following formula was used: # of agreements for categories and sub-categories used to classify the article/total number of categories used [i.e., (number of agreements)/(number of agreements plus disagreements)]. This figure was then multiplied by 100 to obtain the percentage of agreement. Initial agreement ranged from 86.1% to 95.4%, with a mean of 92.1%. All disagreements were discussed until the two reviewers arrived at a unanimous decision; thus, ultimate agreement was 100% for the articles (N = 452) that were reviewed by both authors. The results of the present classification were compared to the results reported by Nolan et al. (1999) who reviewed the articles from JOBM for the same years (N = 119 articles). The comparative data are presented in terms of the percentage of articles classified according to each variable (i.e., the percentage of JAP articles that were research articles versus the percentage of JOBM articles that were research articles). When Nolan et al. presented percentage data, they did not specify the numbers of articles used to calculate them. The first author contacted Nolan et al. to obtain these raw data (J. Austin, personal communication, September, 1999). Most, but not all, of these data were available. Thus, in the present comparison, the numbers used to calculate the percentages for the JOBM articles are reported when they were available; otherwise only the percentages are reported.

13 Critical Review and Discussion 39 CATEGORIES AND OPERATIONAL DEFINITIONS As indicated earlier, we adopted the categories and operational definitions used by Nolan et al. (1999). This was done so that the results of the present classification could be compared to those reported by Nolan et al. Author Characteristics For each article, one of the following affiliations was recorded for each author (if more than one affiliation was listed for an author, the first one to appear was used as the classification): (a) academic (college or university), (b) company (private business, organization or consulting firm), or (c) agency (government or public agency). Nolan et al. (1999) did not assess author gender in JOBM; however, Jarema, Syncerski, Bagge, Austin, and Poling (1999) did assess author gender for the same years we used in our analysis, Thus, we used those data to make our comparisons. To classify author gender, author names that were typically male (e.g., John, Brad, Alan) were recorded as male, and author names that were typically female (e.g., Jennifer, Susan, Melissa) were recorded as female. Additional information (e.g., author gender known by a data recorder, or some indication of gender in the author note or article) was also used for classification. An unknown category was used for authors with gender-neutral names, and no additional information available. Authors Published in Both Journals To assess the relationship between I-O psychology and OBM, the data recorders identified the authors who published in both journals between 1987 and The first author recorded the names of each author appearing on articles published in JOBM during this decade. The second author reviewed this list for accuracy (i.e., to ensure all names were spelled correctly and no author was excluded). Using this list as a data-recording sheet, the first and second authors independently reviewed all tables of contents in JAP from , comparing author names against the list compiled from JOBM and recording references for all JAP articles published by those authors. Agreement was 100%.

14 40 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT Topics Addressed The data recorders used the list of topics that Nolan et al. (1999) used to classify JOBM articles. This list included: Productivity and quality, customer satisfaction, training and development, safety/ health, accuracy, rate of performance, sales, labor, timeliness, novelty, management, material, and other. However, due to the variety of additional topics addressed in JAP, the data recorders labeled the topics that were recorded as other. Some of these additional topics included: Selection and placement, statistical analyses, performance appraisals, attitudes, cognitive processes, legal issues, turnover/absenteeism/attendance, gender and minority issues, group performance, leadership, and decision making. Type of Article: Research versus Discussion/Review To be classified as research, the article must have contained, at minimum, empirical data and a description of the methodology for collecting and analyzing data (Nolan et al., 1999, p. 86). All other articles were classified as discussion/review articles. Statistical metaanalyses were classified as research; however, they were not classified with respect to the research article sub-categories that follow. They were excluded because they analyzed extant data from a variety of different types of studies, and thus could not be appropriately classified. Research Article Sub-Categories When articles were classified as research, they were further evaluated with respect to the following two sub-categories: (a) Type of research article: Experimental versus correlational, and (b) Field versus laboratory research. Experimental research articles were further reviewed according to the following categories: (a) Applied versus theoretical research; (b) Type of dependent variable(s): Behavior or product of behavior; (c) Participant characteristics; (d) Types of independent variables; (e) Research designs and analyses; and (f) Additional relevant categories (whether they contained cost-benefit analyses, follow-up data, program continuation information, social validity data, and reliability data for dependent and independent variables).

15 Critical Review and Discussion 41 With the exception of the first sub-category, the preceding categories were examined and defined by Nolan et al. (1999). As indicated above, it should be noted that the correlational research articles were reviewed only to determine whether they were conducted in field or laboratory settings. They were excluded from further sub-classification because the JOBM and JAP data could not be validly compared. Only three JOBM research articles were classified as correlational, therefore there were insufficient data to make comparisons. Nonetheless, this exclusion should certainly be taken into account when reviewing the results. Type of Research Article: Experimental versus Correlational To be classified as experimental, articles contained at least one independent variable that was manipulated by the researchers. Articles classified as correlational contained analyses of variables that already existed in the environment. Field versus Laboratory Research Articles were classified as field research if they (a) contained data collected in an applied (non-laboratory) setting for analysis in that article, (b) re-analyzed data collected in an applied setting at an earlier time (excluding meta-analyses), or (c) collected survey or observational data that applied directly to the population observed/surveyed (e.g., drug use among employees). Experimental or correlational articles were classified as laboratory when data were collected in a laboratory or simulated setting, or if survey questions were not relevant to current setting (e.g., college students asked about preference for management style). Applied versus Theoretical Experimental Research Although the distinction between applied and theoretical research could arguably be operationalized in a number of different ways, because we compared our data to those reported by Nolan et al. (1999), we retained their definition. Experimental research articles were classified as applied if the interventions addressed specific problems in organizations (e.g., to increase productivity or decrease

16 42 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT absenteeism). All articles classified as applied were field studies, however, not all research studies conducted in field settings were classified as applied. In other words, research that was conducted to specifically solve an organizational problem was classified as applied ; all other research was classified as theoretical. Theoretical research was therefore defined as research conducted to answer more basic questions, or bridge research questions. Types of Dependent Variables in Experimental Research: Behavior or Product of Behavior If researchers reported having directly observed behavior (Nolan et al., 1999, p. 96), dependent variables were classified as behavior. If researchers examined permanent products of behavior (Nolan et al., 1999, p. 96), dependent variables were classified as products of behavior. Articles that reported both types of dependent variables were classified as both. Products of behaviors were further sub-categorized as outcomes, defined as directly measured behavioral outcomes such as number of errors, or self-report, defined as answers to survey or test questions. Participant Characteristics in Experimental Research Experimental research participants were classified as: (a) nonmanagement (those supervised or managed and not themselves in any position of formal authority), (b) management (those in any position of recognized authority over other individuals), (c) executive (those identified as top level management), (d) college student (participants identified in the article as students, college students, university students, or students enrolled in a specific course), or (e) other (those not fitting any of the other operational definitions in this category). Types of Independent Variables in Experimental Research The independent variables were classified using the following categories: (a) feedback (information about past performance provided to the participant), (b) praise (positive verbal consequence following performance), (c) goal-setting (performance standard set and commu-

17 Critical Review and Discussion 43 nicated to the participant, or set by the participant, before performance was measured), (d) monetary rewards (any monetary consequence), (e) non-monetary rewards [any positive consequence that was not monetary or verbal (i.e., praise)], (f) training (any intervention called training and/or that included information or exercises to teach new skills to participants), (g) antecedents (any intervention implemented prior the behavior of interest, excluding training and goal-setting), and (h) punishment (any aversive, or negative consequence, designed to reduce or terminate behavior). If a study examined more than one independent variable, all of the independent variables that were examined were recorded. Research Designs and Analyses in Experimental Research An experimental research article was classified as having used a within-subject design if each participant (or group) was exposed to all experimental and control conditions, and data were analyzed across conditions for each participant (or group). Between-group design was recorded when comparisons were made between groups of participants who were exposed to different conditions. Designs with and without randomization procedures (i.e., quasi-experimental) were included in the between-group classification. If inferential statistics (e.g., ANOVA, ANCOVA, etc.) were used to analyze the data for within-subject or between-group designs, the name of that test was recorded. Additional Experimental Research Sub-Categories All of the following sub-categories appeared on the data-recording sheet under experimental research, and were circled if the article contained the relevant measure or description: (a) cost-benefit for any description of a cost/benefit analysis (e.g., dollar amount spent and dollar amount saved), (b) follow-up data for articles with a description of data collected any time after termination of the intervention, (c) program continuation if they described any continuation of the intervention following completion of the study, (d) social validity for articles that reported participant opinions regarding the nature of the intervention or the results obtained, (e) reliability of the dependent variable for articles that provided a description of inter-rater reliability

18 44 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT or inter-observer agreement, (f) reliability of the independent variable for articles that described any provisions taken to ensure that the intervention was implemented as planned. To remain consistent with the definition of reliability used by Nolan et al. (1999), articles that reported the reliability of the data collection instrument were excluded from analysis. While a good argument can be made for including these articles from a theoretical and conceptual perspective, their inclusion would prohibit a comparison with the data from Nolan et al. Author Characteristics RESULTS AND DISCUSSION Nolan et al. (1999) classified the affiliation of the authors (academic, company, or agency) who published in JOBM over the past decade. We compared these data to the affiliations of the authors who published in JAP. Comparisons were also made with respect to the gender of authors publishing in JOBM and JAP. As indicated earlier, Nolan et al. did not assess author gender in JOBM; however, Jarema et al. (1999) did assess author gender for the same years we used in our analysis, Thus, we used those data to make our comparisons. As discussed below, author characteristics from both sources were very similar. Percentages of author affiliation and author gender are displayed in Figure 1. Author Affiliation The author affiliation was determined for all authors whose names have appeared on JOBM and JAP articles over the past decade. In both publications, the majority of authors were affiliated with academic institutions, 79% in JOBM (209 of 264 authors) and 87% in JAP (2,041 of 2,346 authors). Author Gender A majority of the articles in both journals were authored by men (JOBM = 68%, 179 of 264 authors; JAP = 68%, 1,603 of 2,346 authors). Just over thirty percent (30.5%) of the JOBM authors were

19 Critical Review and Discussion 45 women (Jarema et al., 1999) and 25% (n = 577) of the JAP authors were women.thegenderof7%(n=166)ofjap authors and 1.5% (n = 4) of JOBM authors could not be determined because the names were gender neutral and the data recorders did not personally know the authors. The percentages for first authorship were similar to the overall percentages: For JOBM, 73% (n = 87) were men, 27% (n = 32) were women and 0% were unknown (Jarema et al., 1999); for JAP, 70.1% (n = 699) were men, 22.3% (n = 222) were women, and 7.6% (n = 76) were unknown. Rodgers and Maranto (1989) reviewed gender issues with respect to publication in I-O psychology and used correlational analyses to determine the relationship between gender and publishing productivity. They reported that the quality of publications for men and women was equal; however, the quantity was significantly higher for men. Jarema et al. made the following conclusions about the role of women in OBM, Things, it appears, are looking up for female researchers. Nonetheless, progress has been slow and there is a need to recognize, as well as encourage, productive female researchers (p. 90). Authors Published in Both Journals Table 1 presents an alphabetical list of the authors who have published articles in both JAP and JOBM over the past decade. JOBM articles are listed for each of these authors first, followed by JAP articles. This table also includes author names, article titles and type of article published in both journals. Only nine authors have published in both sources during the past decade, and none of these authors had multiple publications in both journals. That is, while some authors had multiple publications in one journal, they had only one in the other. Due to this limited sample, it is difficult to compare the methodology used by the same authors in each journal. Moreover, three of the nine authors published discussion articles in JOBM (Latham & Huber 1992; Notz, Boschman, & Tax, 1987) and research articles in JAP (Cole & Latham, 1997; Frayne & Latham, 1987; Huber, 1991; Huber & Neale, 1987; Latham, Erez, & Locke, 1988; Latham & Frayne, 1989; Notz & Starke, 1987; Scarpello, Huber, & Vandenberg, 1988) making comparisons more difficult. However, a comparison of articles published by Ludwig and Geller (1997, 1999) in both sources demonstrates the different methodology and data display between journals. The JOBM article published by Ludwig and Geller (1999) was too recent to be included in the current

20 46 JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT FIGURE 1. Author characteristics in JOBM and JAP from : The percentage of authors with academic, organizational or governmental affiliations; and the percentage of male, female and undetermined authors. Affiliation Gender Percentage of Authors JAP (N = 2346) JOBM (N = 263) JAP (N = 2346) JOBM (N = 263) comparison which was restricted to 1997, however, due to the similar topics addressed, and methodologies used in these articles, they provide a clear illustration of the differences. These publications examined the effects of goal setting and feedback (Ludwig & Geller, 1997) and participants serving as change agents (Ludwig & Geller, 1999) on a targeted safe driving behavior, while also measuring additional non-targeted safe driving behaviors. The data analysis in the JAP article focused on statistical analysis of the means across conditions (i.e., 3 3 repeated measures ANOVA) with no display of the time series data. Conversely, the JOBM article displayed the entire time series (i.e., each data point) to evaluate the impact of the intervention. This comparison of articles by Ludwig and Geller supports the comparison of research designs and analyses favored by OBM versus traditional I-O, presented later. Types of Problems Addressed Table 2 identifies the problems most frequently addressed in JAP and JOBM (Nolan et al., 1999). The topics are rank ordered, starting