GRADUATE RECORD EXAMINATIONS PROGRAM

Save this PDF as:
Size: px
Start display at page:

Download "GRADUATE RECORD EXAMINATIONS PROGRAM"

Transcription

1 VALIDITY and the GRADUATE RECORD EXAMINATIONS PROGRAM BY WARREN W. WILLINGHAM EDUCATIONAL TESTING SERVICE, PRINCETON, NEW JERSEY

2 Vaidity and the Graduate Record Examinations Program

3 Vaidity and the Graduate Record Examinations Program by Warren W. Wiingham Educationa Testing Service, Princeton, New Jersey

4 Copyright by Educationa Testing Service. A rights reserved.

5 FOREWORD In recognition of the importance of questions of vaidity to the Graduate Record Examinations program, the GRE Board asked Dr. Warren W. Wiingham of the staff of Educationa Testing Service to prepare a paper on the subject of vaidity and the GRE that coud provide a basis for further Board discussion and decisions. After reviewing Dr. Wiingham s paper, members of the Board agreed that it was an exceent document, which might we be of interest to others concerned with graduate admissions and the transition from undergraduate to graduate study. Accordingy, the GRE Board asked that the paper be pubished, and we are peased to make it avaiabe. Richard H. Armitage Chairman, GRE Board

6 INTRODUCTION... 1 BACKGROUND OF THE PROBLEM Nature of the GRE program... 2 Previousresearch... 3 The current importance of research on vaidity... 4 THE SCOPE AND MEANING OF VALIDITY OF THE GRE... 7 Conventiona interpretations of vaidity... 7 Socia interpretations of vaidity... 8 Construct vaidity of the program SIX PROPOSED OBJECTIVES FOR RESEARCH ON VALIDITY.. 11 Objective I: To encourage and faciitate institutiona vaidity studies Objective II: To dea effectivey with methodoogica issues concerning vaidity that require the GRE program s initiative Objective III: To deveop improved criteria of success in graduate study Objective IV: Popuation vaidity: How to improve it and enhance understanding of it Objective V: To increase institutiona use of summary programdata Objective VI: To systematicay insure the vaidity of revised or new measures resuting from programrenewa STATUS OF RESEARCH RELEVANT TO THE SIXOBJECTIVES Objective I: To encourage and faciitate institutiona vaidity studies Objective II: To dea effectivey with methodoogica issues concerning vaidity that require the GRE program s initiative Objective III: To deveop improved criteria of success in graduate study Objective IV: Popuation vaidity: How to improve it and enhance understanding of it Objective V: To increase institutiona use of summary program data Objective VI: To systematicay insure the vaidity of revised or new measures resuting from programrenewa REFERENCES AND BIBLIOGRAPHY

7 Vaidity is necessariy a major concern of any testing program. It is in the interest of the user that a test measure what it is supposed to measure and that it bear a reasonabe reationship to the criteria it is intended to predict. It is the responsibiity of a test sponsor to insure that these quaities prevai in the testing program. This principe of responsibiity appies with a specia force to a nationa program such as the Graduate Record Examinations (GRE), which affects arge numbers of peope. The stakes are high for both the individua and society. Because of these considerations, the vaidity of examinations sponsored by the GRE program has aways received cose attention. It is suggested in this paper, however, that current issues concerning their vaidity are critica to the immediate future of graduate admissions. Therefore, it is aso sug gested that research on vaidity shoud have high priority in the GRE program over the next severa years. The main purpose of this paper is to faciitate discussion of important issues concerning vaidity and to work toward a framework that the GRE Board Research Committee wi find usefu in assigning priorities and initiating projects. Toward that end, subsequent sections of the paper provide background, define the scope of the probem, and outine six major objectives that might guide the Board s efforts in this important area of research.

8 BACKGROUND OF THE PROBLEM To appreciate the various aspects of vaidity that appy to the GRE program, it is usefu to consider the functions and roe of the program, the kinds of research on vaidity undertaken to date, and why research on vaidity is especiay important at this point in the ife of the program. Nature of the G RE program The meaning one attaches to vaidity, as regards the GRE program, depends on how one perceives the roe of the program. The GRE program is identified primariy as a series of examinations for use by graduate schoos in seecting their students. The program has, however, a variety of functions and a variety of constituents. The foowing functions can be identified: to hep with the admission of students to graduate schoo through pubications, research, advisory services, and forum activities; to provide examination programs as measures of students potentia for success in graduate education; to provide an objective nationa basis for understanding the nature and distribution of academic taent by anayzing and describing characteristics of reevant student groups; to faciitate educationa and career guidance by providing information to students and facuty; to inform undergraduate institutions what the graduate community considers adequate preparation for graduate study. The foowing constituencies can be distinguished: students of different age, sex, background, and so forth; administrators and their institutions; different academic discipines and fieds; masters as we as doctora programs. There is considerabe variation in the extent to which the GRE program serves these functions and constituencies, and there is, no doubt, considerabe difference of opinion regarding their priority. Some have a recognized and expicit roe in the program; the roe of others may be argey impicit. For exampe, as the program has operated thus far, the roes of the ast two functions isted above are more accuratey characterized as potentiay important than as of primary concern. Aso, it seems ikey that students comprise a more important constituency of the program in their own minds than in the minds of many institutiona sponsors. Nonetheess, each of these functions and constituencies heps to define the responsibiities of the program. 2

9 In discussing the program s functions, it is necessary to distinguish between the GRE program and the examination program. The GRE program denotes the entire program structure-its organization, governance, staffing, financing, research activities, major operationa components, and so on. The examination program has a narrower meaning: the test or group of tests that a candidate may take or an institution require and the directy reated services such as directions to examinees and information for interpreting their scores, guideines for institutiona use of the tests, anaytic reports, and so forth. Uness stated otherwise, program is used hereafter to denote this narrower meaning. Previous research Research directy reated to the vaidity of the examination program has faen into one of two partiay overapping categories: (A) a variety of research carried out at ETS under GRE sponsorship and (B) vaidity studies carried out at other institutions. The overview which foows indicates the genera types of GRE research that have a bearing on vaidity and the main concusions concerning predictive vaidity that seem warranted on the basis of institutiona studies. For reviews of methodoogica or other issues, see the reports cited beow. The reevant GRE research over the past 10 to 15 years has been party concerned with traditiona vaidity studies, but aso it has treated a variety of other topics bearing upon vaidity. Part A of the attached bibiography ists 25 GRE pubications that report pertinent research. They concern the foowing topics: methodoogica issues in the conduct of vaidity studies (Bodt, 1975; Reiy and Jackson, 1974; Rock, 1974, 1975) studies of the vaidity of the GRE in the seection of foreign graduate students (Harvey and Pitcher, 1963; Sharon, 1974) test bias and the use of the GRE in seecting minority appicants for graduate study (Echternacht, 1974; Faugher, 1974) criterion probems and the anaysis of what constitutes success in graduate study (Campbe, Freund, and Lannhom, 1965; Carson, Evans, and Kuykenda, 1974; Reiy, 1974a, ) studies concerning test use in seective admissions (Bums, 1970; Campbe, Hiton, and Pitcher, 1967; Bums, Dremuk, and others, 1971; Lannhom, 1962, 1968a; Madaus, 1966) specia prediction studies and summaries of institutiona vaidity studies (Lannhom, 1960, , 1972; Lannhom, Marco, and Schrader, 1968; Lannhom and Schrader, 1951; Osen, 1955; Rock, 1972) The institutiona studies report on statistica anayses of the reationship between GRE tests and other predictors to various criteria of success in 3

10 graduate study. Wiingham (1974) has provided the most recent anaysis of a pubications and reports of vaidity studies invoving the GRE. Part B of the attached bibiography ists the 43 studies reported between 1952 and 1972 that Wiingham anayzed in that artice. These studies were based on 138 independent sets of data and 616 vaidity coefficients. The data indicated that: Vaidity coefficients for various predictors of graduate grade-point average (GPA) tend to be somewhat ower than corresponding coefficients at the undergraduate eve. This is not surprising considering the restricted range of taent frequenty encountered at the graduate eve (see, for exampe, Dawes, 1975). The undergraduate GPA is a moderatey good predictor of graduate GPA and facuty ratings; it is a poor predictor of whether a student wi attain the Ph.D. Depending upon the success criterion used, the GRE composite of Verba and Quantitative Abiity scores is either sighty or substantiay more vaid than the undergraduate GPA. The GRE Advanced Test is the most generay vaid predictor among those reviewed. It was typicay more vaid than the GRE Aptitude Test and had a higher vaidity than the undergraduate GPA in eight of the nine academic fieds represented in the review. Recommendations are a fairy poor predictor of whether a student wi successfuy compete a doctora program. A weighted composite incuding undergraduate GPA and one or more GRE scores typicay provided a vaidity coefficient in the.40 to.45 range. This was somewhat higher than the vaidity of GRE scores aone and substantiay higher than the vaidity of undergraduate GPA aone. This was the case for each success criterion and practicay every academic discipine represented. In addition to these empirica resuts, a variety of methodoogica and conceptua probems were cited that tend to create unusua difficuty in demonstrating the vaidity of entrance examinations at the graduate eve. On the basis of these probems, the data avaiabe, and other considerations, Wiingham concuded that (1) the efficiency of prediction is not ikey to be enhanced merey through the deveopment of improved predictors, and (2) the main hope for improved effectiveness in predicting success in graduate education ies in better definitions of what constitutes success, i.e., more reiabe criteria that are more ceary differentiated with respect to training objectives. The current importance of research on vaidity Considering the number of graduate programs in the country and the far reaching importance of their admissions poicies and procedures, few studies 4

11 have been made of the vaidity of the GRE for seecting graduate students. That fact and the fact that vaidity shoud aways be a prime responsibiity in any testing program are sufficient reasons for emphasizing research in this area. Furthermore, severa current circumstances make vaidity a specia concern of the GRE program. These circumstances foow three genera themes. First, the seection of graduate students is of greater concern now than in the past for the simpe reason that many more students are invoved. Seection often cannot be handed on a persona basis and, at the same time, the process is fragmented (typicay aong departmenta ines) so that the statistica technoogy of seection frequenty cannot be appied effectivey. Concurrenty, other trends are causing facuties to question the adequacy of seection practices. Undergraduate grades are assumed to be infated and ess trustworthy than in the past. New reguations to protect individua privacy give further reason to doubt the usefuness of persona recommendations. These deveopments suggest to some that GRE Verba and Quantitative Abiity scores shoud perhaps have greater weight in seection. At the same time there is increasing interest in the assessment of competence as opposed to aptitude. For exampe, it is now argued by some that seection in higher education shoud pace more emphasis on traits that come coser to the rea requirements of professiona work (Hodgkinson, 1975). In support of this view, the modest reationship between coege grades and adut success is frequenty cited (Hoyt, 1965). A these deveopments and considerations contribute uncertainty as to what constitutes a vaid basis for seecting students. Second, these educationa and methodoogica concerns are confounded by socia and ega issues that have gained great importance in the ast few years. To a considerabe extent there is de facto acceptance of an egaitarian phiosophy of admission in many institutions at the undergraduateeve-at east in the pubic sector. In arge part, admission to graduate study is sti based upon merit, but this genera rue is sharpy conditioned by the widey perceived necessity to represent fairy those groups that constitute minorities in graduate education. This necessity raises compex questions concerning what constitutes unbiased seection when prediction is, as aways, imperfect. The socia issue becomes an important ega issue when the courts are asked to decide what constitutes a vaid test and whether an institution must aways seect the student with the highest probabiity of success. Ironicay, a decision either way is ikey to raise questions of impementation that wi require far greater sophistication concerning the vaidity of admission practices than presenty exists. Whatever the resoution, when admission to priviege is treated as a ega issue, those responsibe for the process must be abe to defend its equity. The third reason vaidity is currenty such an important issue for the GRE program is that the Board is sponsoring a systematic research and deveopment effort toward program renewa, i.e., shortening the Aptitude Test, deveoping additiona modues for optiona use, and examining ways 5

12 to make the program more usefu to students and to institutions. Each of these efforts wi require carefu attention to the vaidity of proposed program changes. Not ony must presenty vaid tests and procedures be maintained; the soundness of any new conceptions regarding vaid measures and procedures in graduate admission must be demonstrated. 6

13 THE SCOPE AND MEANING OF VALIDITY OF THE GRE Often the term vaidity is conceived narrowy-simpy as a reationship between a test score and some measure of success in a subsequent activity. In considering what sorts of research on vaidity the GRE Board might want to undertake, it is necessary to take into account not ony severa conventiona conceptions of vaidity, but aso the fact that the program has various parts and various socia impications. Conventiona interpretations of vaidity The most common forms of vaidity are generay referred to as content vaidity, criterion-reated vaidity, and construct vaidity. The definitions quoted in the foowing paragraphs are taken from Standards for Educationa and Ps ychoogicu Z es ts (American Psychoogica Association, 1974). The emphasis beow has been added. Evidence of content vaidity is required when the test user wishes to estimate how an individua performs in the universe of situations the test is intended to represent. Content vaidity is most commony evauated for tests of ski or knowedge; it may aso be appropriate to inquire into the content vaidity of personaity inventories, behavior checkists, or measures of various aptitudes. Thus, content vaidity has specia reevance to the Advanced Tests since these examinations must represent subject fieds accuratey and produce appraisas of knowedge that are fair regardess of the fact that undergraduate curricuums vary from institution to institution. Criterion-reated vaidities appy when one wishes to infer from a test score an individua s most probabe standing on some other variabe caed a criterion. Statements of predictive vaidity [for exampe] indicate the extent to which an individua s future eve on the criterion can be predicted from a knowedge of prior test performance.... For many test uses, such as for seection decisions,... predictive vaidity provides the appropriate mode for evauating the use of a test or test battery. Predictive vaidity is centra to the GRE program not ony because the examinations are used to seect students ikey to succeed in graduate study, but aso because there is increasing socia and ega pressure against using tests for such purposes uness there is cear pubic evidence of such a reationship. Evidence of construct vaidity is not found in a singe study; rather, judgments of construct vaidity are based upon an accumuation of research resuts. In obtaining the information needed to estabish construct vaidity, the investigator begins by formuating hypotheses about the characteristics of those who have high scores on the test in contrast to those who have ow scores. Taken together, such hypotheses form at east a tentative theory 7

14 about the nature of the construct the test is beieved to be measuring. In considerabe part, the construct vaidity of the GRE rests upon decades of psychometric research, indicating that verba and quantitative abiity pay a critica roe in most types of inteectua work, and upon even more extensive educationa experience which indicates that frequenty the best predictor of future success in an academic fied is eary competence indicated by a subject-matter test. Construct vaidation requires constant attention, however, to insure that a test is actuay measuring the construct intended. For exampe, it is necessary to insure that a reading comprehension test is not so compicated in content as to stress reasoning instead of reading, or that a mathematics test does not use anguage that paces a premium upon knowedge of vocabuary. Naturay, construct vaidation is even more demanding and important in the case of new measures in areas ike cognitive stye and creativity. Socia interpretations of vaidity A number of broad interpretations of vaidity are associated with the socia impications of test use. For exampe, there are such questions as thevaidity of a test for different groups of peope, test vaidity as refected in the ways test use affects the users, and onger range effects of using a particuar test in a arger socia context. These are more recent interpretations of vaidity. They deserve specia consideration because the GRE program operates in an unusuay broad socia context. First, a vaid test must be fair and appropriate for a individuas taking the test. That is, it must be free of systematic bias and distortion vis-&is the various popuations or subgroups taking the test, and the test shoud not have different meanings for such groups. Messick (1975) makes the important point that one vaidates not a test, but an interpretation of data derived from a specific procedure. Whether that procedure (test) has the same properties and patterns of reationships in different popuation groups is an important empirica question. From a somewhat different ange, Thorndike and Hagen (1969) refer generay to vaidity as whether the test measures what we want it to measure. Thus, whie a test may be intended to measure knowedge of American history, there are a number of things it is not intended to measure; e.g., reading speed, cutura disadvantage, sex, age, anguage spoken in the home, and so on. In this sense there are an indefinite number of ways in which a test may be biased and an indefinite number of subgroups for which a test may not be appropriate. There is no way to guard against a such possibiities, and it can easiy happen that making a test fairer for one person may make it ess fair for another. It is evident, however, that bias and ap-

15 propriateness are important aspects of vaidity that require constant attention. A second socia interpretation of vaidity concerns the use and usefuness of a test in the context in which it is actuay appied. The casua phrase test vaidation seems to impy that the score one interprets comes from a naked instrument. The instrument however is ony one eement in a procedure and a vaidation study examines a procedure as a whoe (Cronbach, 1971). Tests are constructed with a purpose in mind and they are used in a context of instructions, interpretative materias, supporting research, expected effects, and constanty changing conditions of use. Again, vaidity does not reside in the test itsef, but depends upon appropriate outcomes resuting from the use of the test. The GRE Board cannot be responsibe for every conceivabe instance of test misuse, but it shoud assume responsibiity for making known the conditions of proper use, for advancing understanding of the socia and educationa impications of different uses of tests, and for insuring that tests are not presented in ways that might encourage inappropriate use. There is another important reationship between vaidity and test use. It is especiay true in a context ike the GRE program that a test or measure does not stand aone. It competes for time and attention with other parts of the examination program that may be equay vaid for the same purpose or more vaid for other purposes. This eads to a third socia interpretation of vaidity. Cronbach (1971) distinguishes educationa importance as a form of vaidity equa in stature, and parae, to content vaidity and construct vaidity. He defines this form of vaidity as foows: Does the test measure an important educationa outcome? Does the battery of measures negect to observe an important outcome. 7 The choice and content of measures incuded in the Board s examination program are significant because those measures constitute an important socia communication. Regardess of whether they are intended as such, the GRE do to some extent communicate to coeges what the graduate community thinks coeges shoud teach and what students shoud earn. It is aso argued that the GRE program needs to refect the important earning outcomes of undergraduate education; i.e., it must foow the curricuum instead of eading it. From either point of view, the content of the program constitutes a message that has a bearing on education far beyond the admissions process. Consequenty the inherent reevance, significance, and vaue of the traits measured deserve cose attention. These atter interpretations of vaidity bear especiay upon the usefuness and appropriateness of different components of the examination program, both as they serve the immediate purposes for which they are intended and as they may be justified in some broader educationa sense. Obviousy such interpretations do not appy to isoated measures, but to the program as a whoe. These considerations and the foregoing discussion suggest a broader notion-one of program vaidity as distinct from test vaidity. 9

16 Construct vaidity of the program Principes concerning vaidity can be appied to tests and other individua measures, to batteries or groups of measures, or to an integrated program that may have a variety of pieces (e.g., a centra core, a variety of test options, biographica information, specia measures, and so on). Each of those pieces shoud have a rationae concerning its egitimate and usefu function. But the pieces are not free-standing. The various components are used in a context of reated procedures, materias, services, and so on. Each part of the program is to some extent dependent upon other parts and upon an overa rationae as to how the program serves its functions and its constituents. These considerations suggest that the Board shoud be guided by an overarching sense of the construct vaidity of the program. In this context, a vaid test is a defensibe test; i.e., an accurate and fair measure of what you want to measure and aso one that is usefu for its purpose. More specificay, the notion of the construct vaidity of a program suggests that a test or measure is a vaid component of a program if it meets these conditions: It represents fairy what is intended. That is, it satisfies concerns such as content vaidity, construct vaidity, educationa importance, and appropriateness for the examinees, both as one group and as subgroups. Its use is demonstraby effective. It meets the requirements of criterion reated vaidity, predictive bias, characteristics of the program that affect test use, and ega issues concerning test use. It serves a distinctive purpose in reation to other tests and measures in the program, that is a purpose not served by the other tests and measures. 10

17 The previous discussion provides background for the probem. In this section six research objectives are suggested in order to provide for the GRE Board Research Committee s consideration specific proposas for action. The objectives are as foows: I. To encourage and faciitate institutiona vaidity studies II. To dea effectivey with methodoogica issues concerning vaidity that require the GRE program s initiative II I. To deveop improved criteria of success in graduate study IV. Popuation vaidity: How to improve it and enhance understanding of it V. To improve institutiona use of summary program data VI. To systematicay insure the vaidity of revised or new measures resuting from program renewa In the foowing paragraphs an initia statement of each objective is foowed by a brief rationae and discussion of severa issues reevant to the objective. These issues are discussed either as genera research needs or, in some cases, as more specific possibe projects. But the main purpose is to suggest a framework for thinking about vaidity research that is needed. Objective I: To encourage and faciitate institutiona vaidity studies The American Psychoogica Association (1974) outines a variety of responsibiities of test sponsors for examining and estabishing the vaidity of measures they offer for use. In this paper we give specia attention to these responsibiities of the GRE program, but the conditions under which tests may be vaid or invaid are essentiay unimited because appications vary so widey with respect to purpose, academic fied, criteria, oca conditions, and so on. The GRE Board cannot hope to estabish vaidity in even a significant minority of the possibe situations in which the tests may be used. Consequenty, it is important for users to recognize their own responsibiity for examining the vaidity of a test for the purpose and circumstances they have in mind. As Cronbach (1971) states, In the end, the responsibiity for vaid use of a test rests on the person who interprets it. The pubished research merey provides the interpreter with some facts and concepts. He has to combine these with his other knowedge about the persons he tests and the assignments or adjustment probems that confront them, to decide what interpretations are warranted. But users confront many probems in carrying out institutiona vaidity studies. In most cases the appropriate ocae is the individua department where, however, the number of students may be sma and the facuty may ack sufficient interest or expertise to 11

18 pursue the question of vaidity. It is important, therefore, that the GRE program find ways to encourage and faciitate institutiona vaidity studies. One possibiity woud be to deveop a imited program of cooperative vaidity studies. This might invove identification of individua departments or institutions where there is the interest and possibiity of carrying out a study of more than routine interest. With technica advice from staff at Educationa Testing Service (ETS), the institution might organize and suppy appropriate data; a researcher in Princeton might anayze the data and prepare a report, probaby in some mode format. This process might aso invove identification and announcement of priority areas of interest; e.g., particuar academic fieds, interesting possibiities for criterion deveopment, specia popuations of students, and institutionwide use of program information. Periodicay such group studies might be coected and reported. Another possibe approach woud be to deveop a vaidity study kit that might consist of a step-by-step notebook for doing oca studies, usefu references and forms, a coection of reevant reprints, and so on. This mode for encouraging oca studies has the virtue of cost-effectiveness, but it aso paces most of the responsibiities for initiative on the institution. ETS s main responsibiity woud be to produce the kit. That itsef wi require experience and good ideas if such a kit is to be usefu and cope effectivey with the variation of oca circumstances. Another research need somewhat reated to those above is the desirabiity of deveoping effective reationships with institutions in order to faciitate work in this area. On the one hand, both of the above possibiities can be greaty faciitated by working intensivey with one or two institutions over a reasonabe period of time to expore the probems of conducting institutiona vaidity studies at the graduate eve. Furthermore, there wi ikey be a need to deveop a cooperative reationship with a variety of institutions in order to vaidate experimenta modues that may be considered for incusion in the GRE program over the next severa years. This needis directy reated to Objective VI, though the deveopment of the necessary institutiona reationships wi profit from groundwork prior to the time when the need actuay arises. Objective II: To dea effectivey with methodoogica issues concerning vaidity that require the GRE program s Initiative A variety of traditiona issues typicay referred to as technica probems make vaidity studies exceptionay difficut at the postgraduate eve. Perhaps the most serious is the criterion probem, though it is a transcendent issue concerning conception as we as methodoogy and deserves separate 12

19 discussion as Objective III beow. The most famiiar technica issues concern very sma sampes, often severe restriction in the range of taent due to seection, and ack of confidence in the meaning and reiabiity of undergraduate grades. The foowing paragraphs outine some of these issues and suggest some possiby fruitfu ines of research. Vaidity studies carried out in individua departments are often based upon sma sampes and a very restricted range of test scores and undergraduate grades. These conditions often combine to produce ow and erratic vaidity coefficients. A reated but different view of this probem is the fact that there has been very itte attention given to the vaidity of the GRE among departments within individua discipines or fieds. It woud appear desirabe to give additiona attention to ways of pooing data across departments, thereby mitigating the technica probems and aso demonstrating vaidity in a arger context. This requires use of some common criterion, Perhaps the ony one that woud make sense is some genera notion of success in graduate education, such as competion of the degree or overa facuty ratings. Another genera possibiity for deaing with these issues is a retrospective nomination study; i.e., asking facuty in a number of departments within a fied to nominate outstanding and poor students over a period of severa years. It may be possibe to deveop substantia sampes for studies within seected fieds. This type of study woud aso require a common criterion of success, such as obtaining the Ph.D. A specia advantage of these two types of studies is the fact that they can be carried out over a imited time period. But, in addition to narrowy conceived vaidity studies, many especiay interesting research questions require ongitudina study. Serious consideration shoud be given to the deveopment of a ongitudina study that foows a carefuy structured sampe of students through and beyond graduate education. Students from severa fieds might be incuded with oversamping of specia groups of interest. With severa spaced foow-ups, a group of cooperating students coud provide a vauabe data base for a variety of studies in addition to specific investigations designed at the outset. Topics of specia interest in such a ongitudina study woud incude foow-up of minority students through and beyond graduate education, studies of patterns of attendance and career choice, and anaysis of the cost of graduate education and how financing aternatives affect students decisions. The quaity of undergraduate grades as a predictor is another famiiar probem. It is commony understood and accepted that a B at one institution is not necessariy equivaent to a B at another institution. A good dea of research at the undergraduate eve_$as indicated that there is no vaue in trying to adjust grades from different high schoos if admissions decisions are made on the basis of grades and-an entrance examination. At the graduate eve, however, it seems that corrections for variations in grading standards from one undergraduate coege to another can sometimes improve predictions sighty. Pitcher and Schrader (1972) report that mutipe 13

20 correations predicting success in graduate business schoos are increased on the average about.02 when the quaity of the undergraduate institution is taken into account. A simiar resut has been found in the case of aw schoo admissions. This matter may be worth an exporatory investigation in connection with the GRE, though it is not cear whether the potentia gain woud make such an investigation worthwhie, nor whether it woud be easy to impement the resuts in any event. Furthermore, appying institutiona corrections may invove difficut poitica probems in graduate admissions. A more serious question may be the status of undergraduate grades in genera. Informa opinions suggest a mounting ack of confidence in the reiabiity and dependabiity of undergraduate grades as a predictor due to the fact that grades are severey infated and the fact that some facuty are opposed to competitive grading as a matter of principe. It is difficut to judge the extent to which undergraduate grades have actuay been seriousy compromised as a measure of accompishment and predictor of subsequent academic success. A carefu study of coege grading, perhaps with resuting recommendations from the Board, coud possiby be of great vaue in reversing an undesirabe trend or, at the very east, in reveaing the character and scope of the probem and suggesting ways of deaing with it. A reated issue is the probem of interpreting the credentias of undergraduate students when they do not come in the form of grades at a An increasing number of institutions are recognizing various forms of nontraditiona earning, converting to competency-based curricuums, awarding credit for experientia earning, and experimenting with narrative transcripts. A study of how such credentias are evauated and to what extent they forecast success in graduate study coud provide a vauabe service by heping to infuence evoving nontraditiona practices in sound directions. Objective III: To deveop improved criteria of success in graduate study It is commony acknowedged that a principa difficuty in estabishing the vaidity of predictors and procedures for admitting graduate students is the ack of cear-cut reiabe criteria of success. Grades are widey suspect; methods of evauating students performance in graduate education vary greaty from department to department; and the utimate criterion of degree attainment takes many years to ascertain and depends upon many intangibe and fortuitous events. To a considerabe extent the criterion probem is due to the very common tendency of those responsibe for educationa seection programs to focus upon predictors and to ignore the measures of achievement that those predictors are intended to forecast. The need to give specific attention to the rationae and measurement (i.e., construct vaidity) of criteria is we-stated by Cronbach (1971): The 14

21 r asymmetric conception of the test as a predictor of a certain performance has been discarded in favor of a symmetric view. According to this, persons are observed in situations. Some are artificia occasions for observations, which are caed tests, and some are situations arising in the natura course of the person s work or schooing. Reating these observations to each other tes one about the situationa demands and about the resources individuas bring to bear. To study the vaidity of a test interpretation is to study how behavior in one situation is reated to behavior in another. Both observations revea characteristics of,the individua, and both types of behavior shoud be understood. Defensibe and reiabe criteria of success are ikey to become more important in the face of agging confidence in grades and the increasing need to justify administrative actions, both with respect to admitting and dropping students. Furthermore, admission standards are ikey to come under increasing scrutiny, party by those speaking for underrepresented groups who question the socia equity of current practices and party by those who assert there is undue reiance upon aptitude tests and objective measures in genera. Shoud ega action require empirica justification of admission decisions, the need to deveop sound criteria wi immediatey become critica. Wiingham (1974) has urged much greater attention to the probem of criteria, especiay in the context of a broader view of predictor-criterion reationships and aternate strategies of seection. The aternate strategies depicted in Figure 1 impy that different departments or programs within departments may emphasize different training objectives, which in turn shoud be reated to the way students are seected and the way their performance is evauated. PREDICTORS specia accompishments. undergraduate grades abtty tests achievement tests creatwty cogntve styes background characterstcs PROGRAM OBJECTIVE INTERMEDIATE ON-THE-JOB (TO TRAIN THE) CRITERIA CRITERIA PRACTITIONER - F TEACHER e demonstrated sk and Interest I practica probems eary invovement m professiona afaw Intern performance academic competency teaching skis demonstrated interest and sku m heppg students Invovement m nsttutona affairs academic competency I r 9 Independent r professtona eadershp student and aum171 judgment of coeagues facuty eadership pubcatons accomphshments ctatons SCHOLAR/SCIENTIST - novat ve work ) awards pub\cat!ons eminence academic competency nventons Figure 1: L Aternate prediction strategies in graduate education. Reproduced with permission from Science, 1974, 183, (4122), 277. Copyright 1974 by the American Association for the Advancement of Science. 1 15

22 Another aspect of Figure 1 deserves specia attention. Moving from eft to right in the figure, predictors, intermediate criteria, and on- the-job criteria form a prediction chain. Preadmission predictors such as test scores or specia achievements are intended to predict success in graduate schoo (an intermediate criterion). Performance in graduate schoo shoud, in turn, predict success in subsequent career-reated activity. Since utimate career success depends upon many circumstances and deveoped skis not necessariy reated to performance in graduate schoo or to earier predictors, it is improper to think of a graduate admission test as a predictor of job success. On the other hand, if a test or other measure is used to screen students, it ought to be hepfu in predicting success in schoo and aso shoud have a ogica reationship to ong-range criteria. That is, the test shoud have construct vaidity in the sense that it represents an abiity of demonstrabe importance (perhaps one of many) in determining ong-range success. Deveoping a better conception of intermediate criteria of success in graduate schoo is an important step in examining the ogica reationships in the prediction chain. There are severa strategies that might be hepfu in deveoping improved criteria. It seems especiay important to encourage systematicay the deveopment of better criteria in the routine execution of vaidity studies as we as the routine evauation of students performance. This might be accompished party by working with individua institutions to carry out and pubish mode studies that can hep to iustrate the deveopment of different types of criteria. For exampe, it woud be desirabe to iustrate and encourage the use of reiabe criteria ike rating scaes, such as those deveoped through GRE research (Carson, Reiy, Mahoney, and Cassery, 1976), or comprehensive examinations, which may aready be avaiabe within departments but are not normay used in vaidity studies. Considering the importance of criteria and the critica roe they coud pay given additiona ega interest in graduate admissions, it may be desirabe to undertake a fairy systematic anaysis of how graduate departments view success. A usefu foow-up to the Carson, Evans, and Kuykenda (1974) survey woud be an intensive anaysis of the rationae and basis upon which departments evauate their students, the evauation procedures actuay empoyed, and the psychometric properties of the resuting criteria. This sort of anaysis coud have considerabe vaue in describing how high eve taent is currenty assessed and in suggesting opportunities and possibe road bocks in the deveopment of improved criteria. Another generay desirabe strategy woud be to foster the deveopment of such intermediate criteria as depicted in Figure 1. This might invove facuty ratings of particuar types of accompishments, specia means of coecting outside judgments, or whatever procedures may be required to obtain information that is reevant to the most important training objectives. One way of encouraging the deveopment of such criteria is through the cooperative institutiona studies described under Objective I. Another approach is a specia deveopmenta project which may be required in the 16

23 case of an unusuay compex criterion such as scientific creativity. The present GRE research on scientific creativity is concerned specificay with the deveopment of an intermediate criterion. Another potentiay usefu approach starts with the observation that practicay a vaidity studies incorporate criteria based upon student performance in graduate schoo. This typica design has two shortcomings if one is interested in confirming the utimate socia reevance (i.e., construct vaidity) of the GRE. First, typica vaidity studies are not directy reevant to the question of whether the GRE are effective in seecting for graduate study peope who are ikey to reach the highest eves of professiona success. Whie screening prospective professionas from a group of graduate students is primariy the responsibiity of the graduate schoos, tests used for earier screening shoud not be counterproductive in that process; that is, one woud ike to know that very successfu professionas have typicay scored we so that screening out students with ow scores is both efficient and defensibe. Second, studies that use success criteria reative to the standards of individua institutions may seriousy underestimate the usefuness of the examinations because the range of taent is typicay restricted at individua institutions, but ranges widey from one institution to another. Thus, the GRE may be reativey poor predictors of graduate performance in a singe prestigious history department, but may provide a reasonaby good indication of differentia competence among graduates of a history departments. It might be worthwhie, therefore, to examine the feasibiity of determining Aptitude Test score eves for pertinent groups of individuas who have achieved some forma measure of success in their fied. These might incude such ad hoc groups as feows of earned societies, officers of professiona organizations, facuty of prestigious departments, individuas isted in honorific biographies, and so on. Comparison of the scores of such individuas with appropriate normative groups woud be interesting, even admitting the possibiity that some individuas may achieve prestigious status party because at one time they were known to have scored highy on the test, or in spite of having scored poory. Objective IV: Popuation vaidity: How to improve it and enhance understanding of it Messick and Barrows (1972) used the term popuation vaidity in referring to the generaizabiity of research findings across different popuations. A simiar notion appies to the vaidity of tests and other psychometric measures. If a test eads to incorrect inferences about a particuar popuation, then the test is to that extent invaid. Incorrect inferences may resut from the fact that the test itsef is not a good measure for that popuation or that the test does not have the same reationship with the criterion for that popu- 17

24 ation. We can refer to these two sources of error as appropriateness and predictive bias. Research on test appropriateness has been imited argey to item-group interaction studies and has not been especiay productive as yet. More attention has been directed to predictive bias and most such studies have compared predicted and actua performance of back and white students. Linn s (1973) review of that research suggests a sma but fairy consistent tendency for tests to overpredict the coege grades of back students. Recenty, attention has been directed to technica factors such as reiabiity that might expain that finding (Linn, Note 1). A significant recent deveopment has been the systematic anaysis of the psychometric characteristics and socia impications of different seection modes (Petersen and Novick, 1974). Somewhat to the surprise of many peope working in this fied, it is now apparent that there are different seection approaches with rather different impications that are compex both technicay and poiticay. For exampe, it can be demonstrated that a seection mode that is fair to an individua may not be fair to a socia group to which that individua beongs. These different seection modes may give rise to ega compications and, if so, perhaps even greater compications for the GRE program and for admission committees. The resoution of these issues may be argey poitica, but it seems important for the Board to pursue reevant research and other activities in this area that wi hep to iuminate the issues and carify the impications for the program. Severa possibiities can be suggested. The program has undertaken severa studies to anayze item-group interactions in an effort to identify types of items that might be unfair to some popuations of examinees. That work has not been fruitfu in discovering significant numbers of such items or in deveoping effective ways of identifying an unfair item prior to its use. More recent anaysis of these types of data revea some sma but interesting tendencies for certain types of items to be harder or easier for different subgroups of examinees. In most cases it seems doubtfu that these differences can be termed bias in any reasonabe sense of the word, but the ine of inquiry is usefu from a research standpoint and desirabe from the standpoint of monitoring the fairness of the GRE. A promising and cosey reated ine of research is the possibiity of a more genera attack on the question of the appropriateness of a test for the various popuations of peope who might take it. The Board has underway a deveopmenta project that may succeed in creating appropriateness indices that woud permit study of this probem from a different vantage point. The work is argey mathematica and, if successfu, wi ikey take severa years to improve our understanding of popuation vaidity. Whie the Board must be concerned about the possibiity of predictive bias in reation to any popuation of examinees, a specia probem exists with respect to individuas who have been out of the educationa system for an extended period of time. There is a common assumption that an objective examination is to some extent unfair to an individua of 35 who has been 18

25 away from coege for a dozen years. In considerabe part this probem paraes the issue of test bias with respect to minorities and can be studied in a simiar fashion. A substantia iterature has deveoped concerning the technica, educationa, and ega issues surrounding the genera matter of popuation vaidity. There are reviews of various aspects of test bias, ega decisions, and impications for admissions committees. These probems are of considerabe concern to departmenta facuties, though the iterature is ikey not we known or avaiabe to them. The Board might consider whether the pubication of a imited group of reprints with commentary woud be ausefu service beyond its present activities in this area. Objective V: To improve institutiona use of summary program data Vaidity is often conceived of ony in reation to the individuas who take a test. But test performance data are often reported about groups of examinees and inferences are drawn concerning those groups and the educationa programs in which they have taken part. As the GRE program seeks to serve better the interests and needs of institutiona sponsors, such summary data shoud be reported more frequenty and systematicay. The prospect of more systematic reporting of program data intensifies the need to insure that such data serve the usefu purposes intended and do not foster erroneous inferences. Two exampes iustrate the opportunity and the need. The GRE program generates biographica and test data that can hep institutions to understand the dynamics of the admissions process and possiby ater the process to serve their institutiona purposes. The fow of taent into and through graduate education can be conceived of as a diminishing poo of individuas; viz., the poo of appicants, the group offered admission, the group who actuay enro, and finay the group who obtain a degree. Data from the program can be used not ony to describe the characteristics of these successivey diminishing groups, but aso to identify the persona and demographic characteristics that operate most strongy in the seection process preceding each stage. Such anaysis shoud hep educationa panners make usefu connections between resource aocations, program panning, and encouragement of taent. As a second iustration: the Board has taken steps to reate the GRE program to the Undergraduate Assessment Program at ETS. One aspect of this new service is to deveop additiona subscores for the GRE Advanced Tests and report them in summary form for groups of students in undergraduate departments. This sort of reporting can be quite usefu to departments if it emphasizes the comparison of departmenta objectives with student performance on corresponding parts of the examinations. The design of an interpretive framework of this sort coud be an important contribution of 19

Teamwork. Abstract. 2.1 Overview

Teamwork. Abstract. 2.1 Overview 2 Teamwork Abstract This chapter presents one of the basic eements of software projects teamwork. It addresses how to buid teams in a way that promotes team members accountabiity and responsibiity, and

More information

Australian Bureau of Statistics Management of Business Providers

Australian Bureau of Statistics Management of Business Providers Purpose Austraian Bureau of Statistics Management of Business Providers 1 The principa objective of the Austraian Bureau of Statistics (ABS) in respect of business providers is to impose the owest oad

More information

Qualifications, professional development and probation

Qualifications, professional development and probation UCU Continuing Professiona Deveopment Quaifications, professiona deveopment and probation Initia training and further education teaching quaifications Since September 2007 a newy appointed FE ecturers,

More information

l l ll l l Exploding the Myths about DETC Accreditation A Primer for Students

l l ll l l Exploding the Myths about DETC Accreditation A Primer for Students Expoding the Myths about DETC Accreditation A Primer for Students Distance Education and Training Counci Expoding the Myths about DETC Accreditation: A Primer for Students Prospective distance education

More information

Business schools are the academic setting where. The current crisis has highlighted the need to redefine the role of senior managers in organizations.

Business schools are the academic setting where. The current crisis has highlighted the need to redefine the role of senior managers in organizations. c r o s os r oi a d s REDISCOVERING THE ROLE OF BUSINESS SCHOOLS The current crisis has highighted the need to redefine the roe of senior managers in organizations. JORDI CANALS Professor and Dean, IESE

More information

PREFACE. Comptroller General of the United States. Page i

PREFACE. Comptroller General of the United States. Page i - I PREFACE T he (+nera Accounting Office (GAO) has ong beieved that the federa government urgenty needs to improve the financia information on which it bases many important decisions. To run our compex

More information

CERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY. Course Offered By: Indian Environmental Society

CERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY. Course Offered By: Indian Environmental Society CERTIFICATE COURSE ON CLIMATE CHANGE AND SUSTAINABILITY Course Offered By: Indian Environmenta Society INTRODUCTION The Indian Environmenta Society (IES) a dynamic and fexibe organization with a goba vision

More information

Degree Programs in Environmental Science/Studies

Degree Programs in Environmental Science/Studies State University Memorandum of New York to Presidents Date: June 30, 2000 Vo. 00 No. I From: Office of the Provost and Vice Chanceor for Academic Affairs SLbject: Guideines for the Consideration o New

More information

3.3 SOFTWARE RISK MANAGEMENT (SRM)

3.3 SOFTWARE RISK MANAGEMENT (SRM) 93 3.3 SOFTWARE RISK MANAGEMENT (SRM) Fig. 3.2 SRM is a process buit in five steps. The steps are: Identify Anayse Pan Track Resove The process is continuous in nature and handed dynamicay throughout ifecyce

More information

Learning from evaluations Processes and instruments used by GIZ as a learning organisation and their contribution to interorganisational learning

Learning from evaluations Processes and instruments used by GIZ as a learning organisation and their contribution to interorganisational learning Monitoring and Evauation Unit Learning from evauations Processes and instruments used by GIZ as a earning organisation and their contribution to interorganisationa earning Contents 1.3Learning from evauations

More information

Internal Control. Guidance for Directors on the Combined Code

Internal Control. Guidance for Directors on the Combined Code Interna Contro Guidance for Directors on the Combined Code ISBN 1 84152 010 1 Pubished by The Institute of Chartered Accountants in Engand & Waes Chartered Accountants Ha PO Box 433 Moorgate Pace London

More information

Vital Steps. A cooperative feasibility study guide. U.S. Department of Agriculture Rural Business-Cooperative Service Service Report 58

Vital Steps. A cooperative feasibility study guide. U.S. Department of Agriculture Rural Business-Cooperative Service Service Report 58 Vita Steps A cooperative feasibiity study guide U.S. Department of Agricuture Rura Business-Cooperative Service Service Report 58 Abstract This guide provides rura residents with information about cooperative

More information

Practicing Reference... Learning from Library Science *

Practicing Reference... Learning from Library Science * Practicing Reference... Learning from Library Science * Mary Whisner ** Ms. Whisner describes the method and some of the resuts reported in a recenty pubished book about the reference interview written

More information

Overview of Health and Safety in China

Overview of Health and Safety in China Overview of Heath and Safety in China Hongyuan Wei 1, Leping Dang 1, and Mark Hoye 2 1 Schoo of Chemica Engineering, Tianjin University, Tianjin 300072, P R China, E-mai: david.wei@tju.edu.cn 2 AstraZeneca

More information

Quality Assurance in Initial Teacher Education. The Standard for Initial Teacher Education in Scotland Benchmark Information

Quality Assurance in Initial Teacher Education. The Standard for Initial Teacher Education in Scotland Benchmark Information Quaity Assurance in Initia Teacher Education The Standard for Initia Teacher Education in Scotand Information OCTOBER 2000 Convener's introduction This document is a statement of the Standard for Initia

More information

Leadership & Management Certificate Programs

Leadership & Management Certificate Programs MANAGEMENT CONCEPTS Leadership & Management Certificate Programs Programs to deveop expertise in: Anaytics // Leadership // Professiona Skis // Supervision ENROLL TODAY! Contract oder Contract GS-02F-0010J

More information

Let s get usable! Usability studies for indexes. Susan C. Olason. Study plan

Let s get usable! Usability studies for indexes. Susan C. Olason. Study plan Let s get usabe! Usabiity studies for indexes Susan C. Oason The artice discusses a series of usabiity studies on indexes from a systems engineering and human factors perspective. The purpose of these

More information

Preschool Services Under IDEA

Preschool Services Under IDEA Preschoo Services Under IDEA W e don t usuay think of Specific Learning Disabiities in connection with chidren beow schoo age. When we think about chidren age birth to six, we think first of their earning

More information

HEALTH PROFESSIONS PATHWAYS

HEALTH PROFESSIONS PATHWAYS T heoffic eofcommuni t yco egeres ea r c ha ndl ea der s hi p Co egeofe duc a t i ona ti i noi s The Heath Professions Pathways (H2P) Consortium is a nationa consortium comprised of nine coeges in five states

More information

Undergraduate Studies in. Education and International Development

Undergraduate Studies in. Education and International Development Undergraduate Studies in Education and Internationa Deveopment Wecome Wecome to the Schoo of Education and Lifeong Learning at Aberystwyth University. Over 100 years ago, Aberystwyth was the first university

More information

STRATEGIC PLAN 2012-2016

STRATEGIC PLAN 2012-2016 STRATEGIC PLAN 2012-2016 CIT Bishopstown CIT Cork Schoo of Music CIT Crawford Coege of Art & Design Nationa Maritime Coege of Ireand Our Institute STRATEGIC PLAN 2012-2016 Cork Institute of Technoogy (CIT)

More information

Human Capital & Human Resources Certificate Programs

Human Capital & Human Resources Certificate Programs MANAGEMENT CONCEPTS Human Capita & Human Resources Certificate Programs Programs to deveop functiona and strategic skis in: Human Capita // Human Resources ENROLL TODAY! Contract Hoder Contract GS-02F-0010J

More information

Niagara Catholic. District School Board. High Performance. Support Program. Academic

Niagara Catholic. District School Board. High Performance. Support Program. Academic Niagara Cathoic District Schoo Board High Performance Academic Support Program The Niagara Cathoic District Schoo Board, through the charisms of faith, socia justice, support and eadership, nurtures an

More information

Using School Leadership Teams to Meet the Needs of English Language Learners

Using School Leadership Teams to Meet the Needs of English Language Learners Nationa Center on Response to Intervention Information Brief: Using Schoo Leadership Teams to Meet the Needs of Engish Language Learners M. Movit, I. Petrykowska, and D. Woodruff May 2010 Leadership teams

More information

A Description of the California Partnership for Long-Term Care Prepared by the California Department of Health Care Services

A Description of the California Partnership for Long-Term Care Prepared by the California Department of Health Care Services 2012 Before You Buy A Description of the Caifornia Partnership for Long-Term Care Prepared by the Caifornia Department of Heath Care Services Page 1 of 13 Ony ong-term care insurance poicies bearing any

More information

(David H T Lan) Secretary for Home Affairs

(David H T Lan) Secretary for Home Affairs Message We sha make every effort to strengthen the community buiding programme which serves to foster among the peope of Hong Kong a sense of beonging and mutua care. We wi continue to impement the District

More information

Older people s assets: using housing equity to pay for health and aged care

Older people s assets: using housing equity to pay for health and aged care Key words: aged care; retirement savings; reverse mortgage; financia innovation; financia panning Oder peope s assets: using housing equity to pay for heath and aged care The research agenda on the ageing

More information

DECEMBER 2008. Good practice contract management framework

DECEMBER 2008. Good practice contract management framework DECEMBER 2008 Good practice contract management framework The Nationa Audit Office scrutinises pubic spending on behaf of Pariament. The Comptroer and Auditor Genera, Tim Burr, is an Officer of the House

More information

No longer living together: how does Scots cohabitation law work in practice?

No longer living together: how does Scots cohabitation law work in practice? Centre for Research on Famiies and Reationships Briefing 51 October 2010 No onger iving together: how does Scots cohabitation aw work in practice? crfr In response to the greater diversity of famiy ife

More information

Program Management Seminar

Program Management Seminar Program Management Seminar Program Management Seminar The word s best management schoos are noted for their superior program execution, high eves of customer satisfaction, and continuous program improvements.

More information

UCU Continuing Professional Development

UCU Continuing Professional Development UCU Continuing Professiona Deveopment Cassroom management The background Good cassroom and behaviour management is one of the key eements of successfu teaching and earning, and wi be crucia to your success

More information

Frequently Asked Questions

Frequently Asked Questions Community Heathcare Organisations Report & Recommendations of the Integrated Service Area Review Group Frequenty Asked Questions 1. What are Community Heathcare Services? Community Heathcare Services are

More information

Integrating Risk into your Plant Lifecycle A next generation software architecture for risk based

Integrating Risk into your Plant Lifecycle A next generation software architecture for risk based Integrating Risk into your Pant Lifecyce A next generation software architecture for risk based operations Dr Nic Cavanagh 1, Dr Jeremy Linn 2 and Coin Hickey 3 1 Head of Safeti Product Management, DNV

More information

Vendor Performance Measurement Using Fuzzy Logic Controller

Vendor Performance Measurement Using Fuzzy Logic Controller The Journa of Mathematics and Computer Science Avaiabe onine at http://www.tjmcs.com The Journa of Mathematics and Computer Science Vo.2 No.2 (2011) 311-318 Performance Measurement Using Fuzzy Logic Controer

More information

Creative learning through the arts an action plan for Wales

Creative learning through the arts an action plan for Wales Creative earning through the arts an action pan for Waes 2015 2020 Audience The entire teaching workforce and government and nationa partners, incuding regiona education consortia, oca authorities, governing

More information

Introduction the pressure for efficiency the Estates opportunity

Introduction the pressure for efficiency the Estates opportunity Heathy Savings? A study of the proportion of NHS Trusts with an in-house Buidings Repair and Maintenance workforce, and a discussion of eary experiences of Suppies efficiency initiatives Management Summary

More information

SELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH. Ufuk Cebeci

SELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH. Ufuk Cebeci SELECTING THE SUITABLE ERP SYSTEM: A FUZZY AHP APPROACH Ufuk Cebeci Department of Industria Engineering, Istanbu Technica University, Macka, Istanbu, Turkey - ufuk_cebeci@yahoo.com Abstract An Enterprise

More information

Chapter 3: e-business Integration Patterns

Chapter 3: e-business Integration Patterns Chapter 3: e-business Integration Patterns Page 1 of 9 Chapter 3: e-business Integration Patterns "Consistency is the ast refuge of the unimaginative." Oscar Wide In This Chapter What Are Integration Patterns?

More information

Delhi Business Review X Vol. 4, No. 2, July - December 2003. Mohammad Talha

Delhi Business Review X Vol. 4, No. 2, July - December 2003. Mohammad Talha Dehi Business Review X Vo. 4, No. 2, Juy - December 2003 TREATMENT TMENT OF GOODWILL IN ACCOUNTING Mohammad Taha GOODWILL is usuay ony recorded in an accounting system when a company purchases an unincorporated

More information

CONTRIBUTION OF INTERNAL AUDITING IN THE VALUE OF A NURSING UNIT WITHIN THREE YEARS

CONTRIBUTION OF INTERNAL AUDITING IN THE VALUE OF A NURSING UNIT WITHIN THREE YEARS Dehi Business Review X Vo. 4, No. 2, Juy - December 2003 CONTRIBUTION OF INTERNAL AUDITING IN THE VALUE OF A NURSING UNIT WITHIN THREE YEARS John N.. Var arvatsouakis atsouakis DURING the present time,

More information

STUDY MATERIAL. M.B.A. PROGRAMME (Code No. 411) (Effective from 2009-2010) II SEMESTER 209MBT27 APPLIED RESEARCH METHODS IN MANAGEMENT

STUDY MATERIAL. M.B.A. PROGRAMME (Code No. 411) (Effective from 2009-2010) II SEMESTER 209MBT27 APPLIED RESEARCH METHODS IN MANAGEMENT St. PETER'S UNIVERSITY St. Peter s Institute of Higher Education and Research (Decared Under Section 3 of the UGC Act, 1956) AVADI, CHENNAI - 600 054 TAMIL NADU STUDY MATERIAL M.B.A. PROGRAMME (Code No.

More information

Early access to FAS payments for members in poor health

Early access to FAS payments for members in poor health Financia Assistance Scheme Eary access to FAS payments for members in poor heath Pension Protection Fund Protecting Peope s Futures The Financia Assistance Scheme is administered by the Pension Protection

More information

Fixed income managers: evolution or revolution

Fixed income managers: evolution or revolution Fixed income managers: evoution or revoution Traditiona approaches to managing fixed interest funds rey on benchmarks that may not represent optima risk and return outcomes. New techniques based on separate

More information

Business Banking. A guide for franchises

Business Banking. A guide for franchises Business Banking A guide for franchises Hep with your franchise business, right on your doorstep A true understanding of the needs of your business: that s what makes RBS the right choice for financia

More information

Key Questions to Ask About

Key Questions to Ask About everychid. onevoice. Every time i have to go to the bathroom at schoo, there s a ine. And sometimes they reay need to be ceaned. Maybe the schoo can get some more hep to open up a of the restrooms again.

More information

Are Leaders Born or Made? Perspectives from the Executive Suite. QuickView Leadership Series. Helping you navigate the leadership landscape

Are Leaders Born or Made? Perspectives from the Executive Suite. QuickView Leadership Series. Helping you navigate the leadership landscape QuickView Leadership Series Heping you navigate the eadership andscape Are Leaders Born or Made? Perspectives from the Executive Suite By: Wiiam Gentry, Ph.D., Jennifer J. Dea, Ph.D., Sarah Stawiski, Ph.D.,

More information

Addressing the Leadership Gap in Healthcare

Addressing the Leadership Gap in Healthcare A White Paper Addressing the Leadership Gap in Heathcare What s needed when it comes to eader taent? Issued June 2010 / Revised June 2011 CONTENTS 3 6 10 13 15 18 19 19 19 Introduction Prioritizing Competencies

More information

READING A CREDIT REPORT

READING A CREDIT REPORT Name Date CHAPTER 6 STUDENT ACTIVITY SHEET READING A CREDIT REPORT Review the sampe credit report. Then search for a sampe credit report onine, print it off, and answer the questions beow. This activity

More information

Order-to-Cash Processes

Order-to-Cash Processes TMI170 ING info pat 2:Info pat.qxt 01/12/2008 09:25 Page 1 Section Two: Order-to-Cash Processes Gregory Cronie, Head Saes, Payments and Cash Management, ING O rder-to-cash and purchase-topay processes

More information

Corporate Governance f o r M a i n M a r k e t a n d a i M C o M p a n i e s

Corporate Governance f o r M a i n M a r k e t a n d a i M C o M p a n i e s Corporate Governance f o r M a i n M a r k e t a n d a i M C o M p a n i e s 23. Corporate governance towards best-practice corporate reporting John Patterson, PricewaterhouseCoopers LLP Reporting is

More information

THE IMPACT OF AN EXECUTIVE LEADERSHIP DEVELOPMENT PROGRAM

THE IMPACT OF AN EXECUTIVE LEADERSHIP DEVELOPMENT PROGRAM Leadership THE IMPACT OF AN EXECUTIVE LEADERSHIP DEVELOPMENT PROGRAM n Jay S. Grider, DO, PhD, Richard Lofgren, MD and Raph Weicke In this artice A eadership deveopment program at an academic medica center

More information

Diploma Decisions for Students with Disabilities. What Parents Need to Know

Diploma Decisions for Students with Disabilities. What Parents Need to Know Dipoma Decisions for Students with Disabiities What Parents Need to Know Forida Department of Education Bureau of Exceptiona Education and Student Services Revised 2005 This is one of many pubications

More information

The Web Insider... The Best Tool for Building a Web Site *

The Web Insider... The Best Tool for Building a Web Site * The Web Insider... The Best Too for Buiding a Web Site * Anna Bee Leiserson ** Ms. Leiserson describes the types of Web-authoring systems that are avaiabe for buiding a site and then discusses the various

More information

AN APPROACH TO THE STANDARDISATION OF ACCIDENT AND INJURY REGISTRATION SYSTEMS (STAIRS) IN EUROPE

AN APPROACH TO THE STANDARDISATION OF ACCIDENT AND INJURY REGISTRATION SYSTEMS (STAIRS) IN EUROPE AN APPROACH TO THE STANDARDSATON OF ACCDENT AND NJURY REGSTRATON SYSTEMS (STARS) N EUROPE R. Ross P. Thomas Vehice Safety Research Centre Loughborough University B. Sexton Transport Research Laboratory

More information

COASTLINE GROUP HUMAN RESOURCES STRATEGY 2015 2017. Great homes, great services, great people.

COASTLINE GROUP HUMAN RESOURCES STRATEGY 2015 2017. Great homes, great services, great people. COASTLINE GROUP HUMAN RESOURCES STRATEGY 2015 2017 Great homes, great services, great peope. Contents Foreword 2 Executive summary 1. Achievements 5 2. Context 7 3. Our peope 9.Objectives 11 5. What we

More information

INDUSTRIAL PROCESSING SITES COMPLIANCE WITH THE NEW REGULATORY REFORM (FIRE SAFETY) ORDER 2005

INDUSTRIAL PROCESSING SITES COMPLIANCE WITH THE NEW REGULATORY REFORM (FIRE SAFETY) ORDER 2005 INDUSTRIAL PROCESSING SITES COMPLIANCE WITH THE NEW REGULATORY REFORM (FIRE SAFETY) ORDER 2005 Steven J Manchester BRE Fire and Security E-mai: manchesters@bre.co.uk The aim of this paper is to inform

More information

Professional Kingston

Professional Kingston Professiona Kingston Organisationa and workforce deveopment soutions n Facuty of Business and Law Aways improving 0/1 Contents Wecome 2 Why Kingston? 4 Course portfoio 6 Course detais 8 Work-based earning

More information

Bite-Size Steps to ITIL Success

Bite-Size Steps to ITIL Success 7 Bite-Size Steps to ITIL Success Pus making a Business Case for ITIL! Do you want to impement ITIL but don t know where to start? 7 Bite-Size Steps to ITIL Success can hep you to decide whether ITIL can

More information

Management Accounting

Management Accounting Management Accounting Course Text Professiona, Practica, Proven www.accountingtechniciansireand.ie Tabe of Contents FOREWORD...v SYLLABUS: MANAGEMENT ACCOUNTING...vii PART 1 INTRODUCTION Chapter 1: Introduction

More information

History of Stars and Rain Education Institute for Autism (Stars and Rain)

History of Stars and Rain Education Institute for Autism (Stars and Rain) History of Education Institute for Autism () Estabished:: March 15. 1993 in Beijing Founder:: Ms. Tian Huiping (mother of a boy with autism) STARS AND RAIN was founded in 1993 by a parent and is China

More information

The Advantages and Disadvantages of Different Social Welfare Strategies

The Advantages and Disadvantages of Different Social Welfare Strategies The Advantages and Disadvantages of Different Socia Wefare Strategies by Lawrence H. Thompson* The foowing was deivered by the author to the High Leve American Meeting of Experts on The Chaenges of Socia

More information

Good assessment in secondary schools HMI 462

Good assessment in secondary schools HMI 462 Good assessment in secondary schoos HMI 462 Good assessment in secondary schoos March 2003 HMI 462 Good assessment in secondary schoos Crown copyright 2003 Document reference number: HMI 462 To obtain

More information

Investigating and Researching HR Issues

Investigating and Researching HR Issues part 1 INTRODUCTION A sampe chapter from Research Methods in Human Resource Management: Investigating a Business Issue, by Vaerie Anderson. Pubished by the CIPD. Copyright 2013. A rights reserved. No part

More information

The Impact of Parental Involvement on Children s Education

The Impact of Parental Involvement on Children s Education The Impact of Parenta Invovement on Chidren s Education 2 The Impact of Parenta Invovement on Chidren s Education The Impact of Parenta Invovement on Chidren s Education Key findings Parenta invovement

More information

FINANCIAL ACCOUNTING

FINANCIAL ACCOUNTING FINANCIAL ACCOUNTING Part 1 A conceptua framework: setting the scene 1 Who needs accounting? 2 A systematic approach to financia reporting: the accounting equation 3 Financia statements from the accounting

More information

Asia Pacific Disability Rehabilitation Journal

Asia Pacific Disability Rehabilitation Journal DEVELOPMENTAL ARTICLES Asia Pacific Disabiity Rehabiitation Journa PRIMARY HEALTH CARE AND COMMUNITY BASED REHABILITATION: IMPLICATIONS FOR PHYSICAL THERAPY Tracy Bury* ABSTRACT The Word Confederation

More information

Education Quality Improvement Framework

Education Quality Improvement Framework Education Quaity Improvement Framework for Undergraduate and Postgraduate Medica Education and Training in the UK The duties of a doctor registered with the Genera Medica Counci Patients must be abe to

More information

The Comparison and Selection of Programming Languages for High Energy Physics Applications

The Comparison and Selection of Programming Languages for High Energy Physics Applications The Comparison and Seection of Programming Languages for High Energy Physics Appications TN-91-6 June 1991 (TN) Bebo White Stanford Linear Acceerator Center P.O. Box 4349, Bin 97 Stanford, Caifornia 94309

More information

Contingency Planning Guidelines A PRACTICAL GUIDE FOR FIELD STAFF. UNHCR Division of Operations Support. August 1996

Contingency Planning Guidelines A PRACTICAL GUIDE FOR FIELD STAFF. UNHCR Division of Operations Support. August 1996 Contingency Panning Guideines A PRACTICAL GUIDE FOR FIELD STAFF UNHCR Division of Operations Support August 1996 Contents The Purpose This section covers the "when" and "why" of contingency panning. It

More information

APPENDIX 10.1: SUBSTANTIVE AUDIT PROGRAMME FOR PRODUCTION WAGES: TROSTON PLC

APPENDIX 10.1: SUBSTANTIVE AUDIT PROGRAMME FOR PRODUCTION WAGES: TROSTON PLC Appendix 10.1: substantive audit programme for production wages: Troston pc 389 APPENDIX 10.1: SUBSTANTIVE AUDIT PROGRAMME FOR PRODUCTION WAGES: TROSTON PLC The detaied audit programme production wages

More information

Pay-on-delivery investing

Pay-on-delivery investing Pay-on-deivery investing EVOLVE INVESTment range 1 EVOLVE INVESTMENT RANGE EVOLVE INVESTMENT RANGE 2 Picture a word where you ony pay a company once they have deivered Imagine striking oi first, before

More information

professional indemnity insurance proposal form

professional indemnity insurance proposal form professiona indemnity insurance proposa form Important Facts Reating To This Proposa Form You shoud read the foowing advice before proceeding to compete this proposa form. Duty of Discosure Before you

More information

Art of Java Web Development By Neal Ford 624 pages US$44.95 Manning Publications, 2004 ISBN: 1-932394-06-0

Art of Java Web Development By Neal Ford 624 pages US$44.95 Manning Publications, 2004 ISBN: 1-932394-06-0 IEEE DISTRIBUTED SYSTEMS ONLINE 1541-4922 2005 Pubished by the IEEE Computer Society Vo. 6, No. 5; May 2005 Editor: Marcin Paprzycki, http://www.cs.okstate.edu/%7emarcin/ Book Reviews: Java Toos and Frameworks

More information

Speech, language and communication. Information for managers and school staff

Speech, language and communication. Information for managers and school staff Speech, anguage and communication Information for managers and schoo staff Introduction The Communication Trust has deveoped this short guide to hep schoos support chidren s speech, anguage and communication.

More information

SCHOOL OF BUSINESS MANAGEMENT Information Brochure 2014

SCHOOL OF BUSINESS MANAGEMENT Information Brochure 2014 WANT MORE INFO? If you have any questions, contact us at the address, phone numbers or mai ids mentioned beow. We aso invite you to visit our institute to see the faciities and discuss the courses on offer.

More information

Advanced ColdFusion 4.0 Application Development - 3 - Server Clustering Using Bright Tiger

Advanced ColdFusion 4.0 Application Development - 3 - Server Clustering Using Bright Tiger Advanced CodFusion 4.0 Appication Deveopment - CH 3 - Server Custering Using Bri.. Page 1 of 7 [Figures are not incuded in this sampe chapter] Advanced CodFusion 4.0 Appication Deveopment - 3 - Server

More information

What makes a good Chair? A good chair will also: l always aim to draw a balance between hearing everyone s views and getting through the business.

What makes a good Chair? A good chair will also: l always aim to draw a balance between hearing everyone s views and getting through the business. Chairing a meeting An important job of the Chairperson is chairing meetings. Prior House 6 Tibury Pace Brighton BN2 0GY Te. 01273 606160 Fax. 01273 673663 info@resourcecentre.org.uk www.resourcecentre.org.uk

More information

TMI ING Guide to Financial Supply Chain Optimisation 29. Creating Opportunities for Competitive Advantage. Section Four: Supply Chain Finance

TMI ING Guide to Financial Supply Chain Optimisation 29. Creating Opportunities for Competitive Advantage. Section Four: Supply Chain Finance TMI171 ING info pat :Info pat.qxt 19/12/2008 17:02 Page 29 ING Guide to Financia Suppy Chain Optimisation Creating Opportunities for Competitive Advantage Section Four: Suppy Chain Finance Introduction

More information

READY TO LEARN. White Paper on Early Childhood Education. Pn.7939 5.00 a6.35

READY TO LEARN. White Paper on Early Childhood Education. Pn.7939 5.00 a6.35 READY TO LEARN White Paper on Eary Chidhood Education Pn.7939 5.00 a6.35 Government of Ireand 1999 ISBN 0-7076-6411-X Designed by Metaphor Printed by Cahi Printers Ltd. Dubin Pubished by The Stationery

More information

Strengthening Human Resources Information Systems: Experiences from Bihar and Jharkhand, India

Strengthening Human Resources Information Systems: Experiences from Bihar and Jharkhand, India Strengthening Human Resources Information Systems: Experiences from Bihar and Jharkhand, India Technica Brief October 2012 Context India faces critica human resources (HR) chaenges in the heath sector,

More information

effect on major accidents

effect on major accidents An Investigation into a weekend (or bank hoiday) effect on major accidents Nicoa C. Heaey 1 and Andrew G. Rushton 2 1 Heath and Safety Laboratory, Harpur Hi, Buxton, Derbyshire, SK17 9JN 2 Hazardous Instaations

More information

Chapter 2 Traditional Software Development

Chapter 2 Traditional Software Development Chapter 2 Traditiona Software Deveopment 2.1 History of Project Management Large projects from the past must aready have had some sort of project management, such the Pyramid of Giza or Pyramid of Cheops,

More information

Graduate Program Prioritization Criteria and Questions/Elements. 6th Year Professional Diploma in Clinical Mental Health or School Counseling

Graduate Program Prioritization Criteria and Questions/Elements. 6th Year Professional Diploma in Clinical Mental Health or School Counseling Graduate Program Prioritization Criteria and Questions/Eements 6th Year Professiona Dipoma in Cinica Menta Heath or Schoo Counseing 1. History, Deveopment and Expectations of the Program a. Provide, to

More information

Structural Developments and Innovations in the Asset- Backed Commercial Paper Market

Structural Developments and Innovations in the Asset- Backed Commercial Paper Market Structura Deveopments and Innovations in the Asset- Backed Commercia Paper arket ark H. Adeson anaging Director, Asset-Backed Commercia Paper oody s Investors Service Strategic Research Institute 1997

More information

The eg Suite Enabing Rea-Time Monitoring and Proactive Infrastructure Triage White Paper Restricted Rights Legend The information contained in this document is confidentia and subject to change without

More information

The guaranteed selection. For certainty in uncertain times

The guaranteed selection. For certainty in uncertain times The guaranteed seection For certainty in uncertain times Making the right investment choice If you can t afford to take a ot of risk with your money it can be hard to find the right investment, especiay

More information

We are XMA and Viglen.

We are XMA and Viglen. alearn with Microsoft 16pp 21.07_Layout 1 22/12/2014 10:49 Page 1 FRONT COVER alearn with Microsoft We are XMA and Vigen. Ca us now on 0115 846 4900 Visit www.xma.co.uk/aearn Emai alearn@xma.co.uk Foow

More information

Conflict analysis. What is conflict analysis and why is it important? Purpose of chapter. Who should read it. Why they should read it CHAPTER 2

Conflict analysis. What is conflict analysis and why is it important? Purpose of chapter. Who should read it. Why they should read it CHAPTER 2 CHAPTER 2 Confict anaysis Purpose of chapter This chapter expains: what confict anaysis is and why it matters how to undertake an anaysis Who shoud read it The chapter is aimed at practitioners in governments,

More information

Serving the Millennial Generation - The Challenge and Opportunity for Financial Services Companies

Serving the Millennial Generation - The Challenge and Opportunity for Financial Services Companies Serving the Miennia Generation - The Chaenge and Opportunity for Financia Services Companies May 2015 Christopher J. Perry, CFA Equity Research Anayst Today, the Miennia Generation (or Generation Y), broady

More information

A guide to understanding Childcare Proceedings

A guide to understanding Childcare Proceedings A guide to understanding Chidcare Proceedings About this guide Care Proceedings are one of the most traumatic and emotiona episodes that can happen in anyone s ife. When Chidren s Services (formery Socia

More information

Network/Communicational Vulnerability

Network/Communicational Vulnerability Automated teer machines (ATMs) are a part of most of our ives. The major appea of these machines is convenience The ATM environment is changing and that change has serious ramifications for the security

More information

SABRe B2.1: Design & Development. Supplier Briefing Pack.

SABRe B2.1: Design & Development. Supplier Briefing Pack. SABRe B2.1: Design & Deveopment. Suppier Briefing Pack. 2013 Ros-Royce pc The information in this document is the property of Ros-Royce pc and may not be copied or communicated to a third party, or used

More information

Accreditation: Supporting the Delivery of Health and Social Care

Accreditation: Supporting the Delivery of Health and Social Care Accreditation: Supporting the Deivery of Heath and Socia Care PHARMACY E F P T O L P E D P E C M F D T G L E F R Accreditation: Supporting the Deivery of Heath and Socia Care June 9, 2015 marks Word Accreditation

More information

David Eccles S chool of Business. PhD Program. The University of Utah

David Eccles S chool of Business. PhD Program. The University of Utah David Ecces S choo of Business PhD Program The University of Utah ENGAGE IN RESEARCH For the curious, committed, intense schoar, a doctorate from the University of Utah s David Ecces Schoo of Business

More information

Distribution of Income Sources of Recent Retirees: Findings From the New Beneficiary Survey

Distribution of Income Sources of Recent Retirees: Findings From the New Beneficiary Survey Distribution of Income Sources of Recent Retirees: Findings From the New Beneficiary Survey by Linda Drazga Maxfied and Virginia P. Rena* Using data from the New Beneficiary Survey, this artice examines

More information

With the arrival of Java 2 Micro Edition (J2ME) and its industry

With the arrival of Java 2 Micro Edition (J2ME) and its industry Knowedge-based Autonomous Agents for Pervasive Computing Using AgentLight Fernando L. Koch and John-Jues C. Meyer Utrecht University Project AgentLight is a mutiagent system-buiding framework targeting

More information

The Productive Therapist and The Productive Clinic Peter R. Kovacek, MSA, PT

The Productive Therapist and The Productive Clinic Peter R. Kovacek, MSA, PT The Productive Therapist and The Productive Cinic Peter R. Kovacek, MSA, PT Format Interactive Discussions among equa peers Constructive argument Reaity Oriented Mutua Accountabiity Expectation Panning

More information

ICAP CREDIT RISK SERVICES. Your Business Partner

ICAP CREDIT RISK SERVICES. Your Business Partner ICAP CREDIT RISK SERVICES Your Business Partner ABOUT ICAP GROUP ICAP Group with 56 miion revenues for 2008 and 1,000 empoyees- is the argest Business Services Group in Greece. In addition to its Greek

More information

Certified Once Accepted Everywhere Why use an accredited certification body?

Certified Once Accepted Everywhere Why use an accredited certification body? Certified Once Accepted Everywhere Why use an accredited certification body? Third party management systems certification is a frequenty specified requirement to operate in the goba market pace. It can

More information

Best Practices for Push & Pull Using Oracle Inventory Stock Locators. Introduction to Master Data and Master Data Management (MDM): Part 1

Best Practices for Push & Pull Using Oracle Inventory Stock Locators. Introduction to Master Data and Master Data Management (MDM): Part 1 SPECIAL CONFERENCE ISSUE THE OFFICIAL PUBLICATION OF THE Orace Appications USERS GROUP spring 2012 Introduction to Master Data and Master Data Management (MDM): Part 1 Utiizing Orace Upgrade Advisor for

More information