An Undergraduate Curriculum Evaluation with the Analytic Hierarchy Process

Transcription

1 An Undergrdute Curriculum Evlution with the Anlytic Hierrchy Process Les Frir Jessic O. Mtson Jck E. Mtson Deprtment of Industril Engineering P.O. Box University of Albm Tuscloos, AL Abstrct - An undergrdute curriculum committee hs developed the use of the Anlytic Hierrchy Process (AHP) for the evlution of lterntive curriculum designs. The hierrchy consists of four levels of interction - from the top most objective through ffected prties (students, fculty, employers, etc.), curriculum components (design, science, mth, etc.), to curriculum lterntives t the bottom. An internet web site hs been designed nd is being implemented to collect AHP judgments from the ffected prties. This collected informtion cn then be used to rnk the vrious curriculum lterntives generted by the committee nd others. AHP cn be chrcterized s multi-criteri decision technique in which qulittive fctors re of prime importnce. A model of the problem (undergrdute curriculum design) is developed using hierrchicl representtion. At the top of the hierrchy is the overll gol or prime objective one is seeking to fulfill. The succeeding lower levels then represent the progressive decomposition of the problem. Knowledgeble prties complete pir-wise comprison of ll entries in ech level reltive to ech of the entries in the next higher level of the hierrchy. The composition of these judgments fixes the reltive priority of the entities t the lowest level (curriculum lterntives) reltive to chieving the top-most objective. A description of AHP development for this curriculum design problem is provided. The implementtion of n internet web site to collect the AHP judgments is detiled. Finlly the combintion of the vrious AHP inputs for the rnking of the curriculum lterntives is discussed. Introduction The orgniztion for this pper is tht first the Anlytic Hierrchy Process (AHP) is described in sufficient detil so tht the reder cn understnd the motivtion nd methodology of this technique. Next the curriculum committee's use of AHP to evlute severl different curriculum designs is detiled. This section lso describes the methodology followed in generting the curriculum lterntives. In ending, this pper describes severl complicting fctors ssocited with this experiment, some tenttive conclusions nd recommendtion for continued investigtion of the use of AHP for such evlutions. The Anlytic Hierrchy Process (AHP) This technique is especilly suited for ppliction to problem evlutions in which qulittive fctors dominte. It cn be chrcterized s multi-criteri decision technique tht cn combine qulittive nd quntittive fctors in the overll evlution of lterntives. This section provides n introduction to AHP with n emphsis on the presenttion of the generl methodology. No ttempt is mde to provide the mthemticl foundtions for AHP; rther the interested reder is referred to [1] nd [2]. AHP determines the priority ny lterntive hs on the overll gol of the problem of interest. The nlyst/user cretes model of the problem by developing hierrchicl decomposition representtion. At the top of the hierrchy is the overll gol or prime objective one is seeking to fulfill. The succeeding lower levels then represent the progressive decomposition of the problem. The nlyst, or other knowledgeble prty, completes pir-wise comprison of ll elements in ech level reltive to ech of the progrm elements in the next higher level of the hierrchy. The composition of these judgments fixes the reltive priority of elements in the lowest level (usully solution lterntives) reltive to chieving the top-most objective. Four Steps re used to solve problem with the AHP methodology: 1. Build decision "hierrchy" by breking the generl problem into individul criteri. (User/Anlyst Modeling Phse)

2 2. Gther reltionl dt for the decision criteri nd lterntives nd encode using the AHP reltionl scle [see following exmple]. (User/Anlyst pirwise comprison input) 3. Estimte the reltive priorities (weights) of the decision criteri nd lterntives (AHP softwre [3], [4] or PC-spredsheet [5]). 4. Perform composition of priorities for the criteri, which gives the rnk of the lterntives (usully lowest level of hierrchy) reltive to the top-most objective (AHP softwre or spredsheet). Mny exmple pplictions of AHP cn be found in the literture. See, for instnce, [1] nd [2]. The AHP steps described bove cn be best understood through discussion of n exmple ppliction. Consider the exmple hierrchy of figure 1. In this exmple the decision problem is to determine the best contrctor for the procurement of prticulr system. Such considertion my hve been necessitted by the ssocited cost proposls being judged to be equl. The non-quntittive considertions (s displyed in the bove figure) re sfety, performnce nd relibility/mintinbility. AHP will be utilized to determine the highest-rnking contrctor bsed on these non-quntittive considertions. This determintion will be bsed on the subjective judgment/experience of the decision mker(s). below: Detils of the AHP methodology re presented Step 1. Develop the hierrchicl representtion of the problem. At the top of the hierrchy is the overll objective nd the decision lterntives re t the bottom. Between the top nd bottom levels re the relevnt ttributes of the decision problem, such s selection criteri nd the vrious "ctors " (individuls, gencies nd orgniztions), if pproprite, tht provide significnt input on the decision process. The number of levels in the hierrchy depends on the complexity of the problem nd the nlyst/decision mker s model of the problem hierrchy. Step 2. Generte reltionl dt for compring the lterntives. This requires the nlyst (decision-mker) to mke pirwise comprisons of elements t ech level reltive to ech ctivity t the next higher level in the hierrchy. In the system exmple the importnce of ech criterion reltive to system cceptnce needs to be estblished. In AHP reltionl scle of rel numbers from 1 to 9 is used to systemticlly ssign preferences. When compring two ttributes (or lterntives) A nd B, with respect to n ttribute U, in higher level, the following numericl reltionl scle is used: 1 - A hs the sme importnce s B with respect to U 3 - A hs slightly more importnce thn B with respect to U. 5 - A hs more importnce thn B with respect to U. 7 - A hs lot more importnce thn B with respect to U. 9 - A totlly domintes B with respect to U. 1/3 - B hs slightly more importnce thn A with respect to U. 1/5 - B hs more importnce thn A with respect to U. 1/7 - B hs lot more importnce thn A with respect to U. 1/9 - B totlly domintes A with respect to U. Intermedite numbers re used for finer resolution. Step 3 Utilizing the pirwise comprisons of step 2 n eigenvlue method (mthemticl pproch used by AHP-see [1]) is used to determine the reltive priority of ech ttribute to ech ttribute one level up in the hierrchy. In ddition, "consistency rtio" is clculted nd displyed. According to Sty [1], smll consistency rtios (less thn 0.1 is the suggested rule-of-thumb) do not drsticlly ffect the rtings. The user hs the option of redoing the comprison mtrix if desired. Step 4 In this step, the priorities (or weights) of the lowest level lterntives reltive to the top most objective re determined nd displyed. For the exmple system hierrchy (bove) the AHP vlues re given in figure 2. We see from the overll priorities (figure 2) tht contrctors 1 nd 2 re pproximtely tied for best wheres contrctor 3 does not pper to be competitive. AHP fcilittes comprehensive nd logicl nlysis of problems for which considerble uncertinty exists. If fct, the power of AHP (nd to lrge degree its uniqueness) is being ble to consider qulittive gols nd ttributes within its frmework. The method of pirwise comprisons is systemtic nd comprehensive. One might wnt to repet set of pirwise comprisons if the consistency rtio is lrmingly high. The finl output from the AHP softwre is the reltive priorities of the bottom most (in the hierrchy) lterntives reltive to the overll objective (top level of hierrchy).

3 AHP Use in Curriculum Design During the cdemic yer the uthors, s members of n undergrdute curriculum committee, were chrged with developing new Industril Engineering (IE) curriculum. This new curriculum should stisfy ABET 2000 criteri, University core curriculum requirements s well s providing contemporry tretment of pproprite subjects within "resonble number" of required credits. The following ctions were tken to comply with this chrge: 1. The IE curriculum gols nd objectives were developed utilizing the gols nd objectives we hd developed for our lst ABET visit (1995), ABET 2000 criteri, review of other progrms gols nd objectives nd input from fculty nd lumni. 2. A detiled survey of the composition of IE curriculums t 29 different schools ws conducted. Detils on curriculum composition t these 29 schools in the generl res of mth, nturl sciences, written nd orl communiction, humnities nd socil sciences, generl engineering topics, nd industril engineering courses were tbulted nd summrized. 3. Armed with the informtion from steps 1-2 bove this curriculum committee estblished the gol of producing n effective IE curriculum tht required less thn 130 semester hour credits. Our current curriculum requires 133 credit hours. This decision ws lso influenced by other UA engineering deprtments recently estblishing curriculums requiring less thn 130 credit hours (e.g. Mechnicl Engineering chnged this yer to curriculum of 128 credit hours). 1. After delibertion the committee decided to generte t lest three different curriculum lterntives for evlution. Tht is, we would consider curriculums tht hd the following orienttions: mnufcturing engineering engineering mngement generl industril engineering The mnufcturing engineering focused curriculum would require considerbly more mnufcturing engineering courses from the mechnicl nd industril engineering deprtments. The engineering mngement lterntive not only would require dditionl courses from the college of business dministrtion but lso would result in the students being wrded minor in business dministrtion long with the BSIE. The generl IE curriculum is most like our existing curriculum but does reflect the findings of 1-3 bove. The next step ws to develop the hierrchy for the lterntive curriculum evlutions. Figure 3 presents this hierrchy. In figure 3 we hve s: Level 1: Top Most Objective---An IE undergrdute curriculum recognized s excellent by ll ffected prties Level 2: Affected Prties Students 2. Fculty 3. Alumni 4. ABET 5. University 6. Employers 7. IE Community Level 3: Curriculum Components Mth 2. Bsic Sciences 3. Engineering Topics (other thn IE) 4. IE Topics 5. Business Topics 6. Humnities, Socil Science & Contemporry Issues 1. Design Opportunities 2. eserch Opportunities 3. Multidisciplinry Opportunities 4. Teming Experiences 5. Engineering Tools 6. Communiction 7. Ethics nd Professionlism 8. Professionl Prctice 9. Lifelong Lerning Level 4: Curriculum Alterntives IE Mnufcturing Alterntive 2. IE Engineering Mngement Alterntive 3. IE Generl Alterntive At the time of submittl of this pper (Mrch 1998) we hve initited the web-bsed implementtion to conduct the AHP ssocited with this problem. We nticipte tht we will hve the softwre in plce by the end of the spring term nd will collect judgments from the ffected prties (listed in level 2 of figure 3) during the summer. Anlysis will be performed during the summer nd this pper updted to reflect these results by

4 erly fll. The results of ll of these ctivities will be reported t presenttion time of this pper. Summry This pper hs described the use of decision-mking technique for curriculum design. The finl effectiveness of this innovtive pproch is yet to be determined but its ppliction hs ssisted the uthors in their job of curriculum design nd nlysis. Just the exercise of constructing the AHP model ws instructive with respect to how curriculum might be evluted from vrious ffected prties points of view. Becuse of the nture of AHP we should be ssured of receiving individully consistent dt tht cn be exmined to determine the reltive priority of the three curriculum designs. In ddition, the nlysis should serve s sensitivity tool s we combine judgments from the vrious ffected prties. eferences 1. Sty, Thoms, L., The Anlytic Hierrchy Process, 1980,McGrw-Hill Co., New York, NY. 2. Sty, Thoms, L., "How to Mke Decision: The Anlytic Hierrchy Process", Europen Journl of Opertions eserch 48 (1990), pp Expert Choice softwre vilble from Expert Choice Inc., Pittsburgh, PA. 4. HIPE 3+ softwre vilble from Snt Monic Softwre, Mlibu, CA. 5. Winston, Wyne, L., Opertions eserch: Applictions nd Algorithms, pp , 2nd Edition, Duxbury Press, System Acceptnce Level 1 Sfety elibility & Mintennce Performnce Level 2 Contrctor 1 Contrctor 2 Contrctor 3 Level 3 Figure 1. An Exmple AHP

5 FOCUS: Acceptnce Pirwise Comprison Mtrix /5 1 1/ FOCUS: Sfety Priorities Pirwise Comprison Mtrix Priorities 1 1/5 1/ / OVEALL PIOITIES t i o FOCUS: elibility & Mintennce FOCUS: Performnce t Pirwise t Pirwise t i Comprison i Comprison i o Mtrix Priorities o Mtrix Priorities o /2 1 1/ / / Figure 2. AHP esults for Exmple System An IE undergrdute curriculum recognized s excellent by ll ffected prties Students Fculty Alumni ABET University Employers IE Community

6 IE -Mnufcturing Curriculum IE- Engineering Mgt Curriculum IE- Generl Curriculum Figure 3 AHP Hierrchy for Curriculum Design Evlution