Artificial Intelligence Approach to Evaluate Students Answerscripts Based on the Similarity Measure between Vague Sets

Transcription

1 Wan, H.-Y., & Chen, S. M. (007). rtfcal Intellence pproach to Evaluate Students nswerscrpts ased on the Smlarty Measure between Vaue Sets. Educatonal Technoloy & Socety, 0 (), -. rtfcal Intellence pproach to Evaluate Students nswerscrpts ased on the Smlarty Measure between Vaue Sets Hu-Yu Wan Department of Educaton, Natonal Chench Unversty, Tawan // 955@@nccu.edu.tw Shy-Mn Chen Department of Computer Scence and Informaton Enneern, Natonal Tawan Unversty of Scence and Technoloy, Tawan // Tel: // smchen@mal.ntust.edu.tw STRCT In ths paper, we present two new methods for evaluatn students answerscrpts based on the smlarty measure between vaue sets. The vaue marks awarded to the answers n the students answerscrpts are represented by vaue sets, where each element u n the unverse of dscourse U belonn to a vaue set s represented by a vaue value. The rade of membershp of u n the vaue set à s bounded by a subnterval [t à (u ), f à (u )] of [0, ]. It ndcates that the exact rade of membershp μ à (u ) of u belonn the vaue set à s bounded by t à (u ) μ à (u ) f à (u ), where t à (u ) s a lower bound of the rade of membershp of u derved from the evdence for u, f à (u ) s a lower bound of the neaton of u derved from the evdence aanst u, t à (u ) + f à (u ), and u U. n ndex of optmsm λ determned by the evaluator s used to ndcate the deree of optmsm of the evaluator, where λ [0, ]. ecause the proposed methods use vaue sets to evaluate students answerscrpts rather than fuzzy sets, they can evaluate students answerscrpts n a more flexble and more ntellent manner. Especally, they are partcularly useful when the assessment nvolves subjectve evaluaton. The proposed methods can evaluate students answerscrpts more stable than swas s methods (995). Keywords Smlarty functons, Students answerscrpts, Vaue rade sheets, Vaue membershp values, Vaue sets, Index of optmsm Introducton In recent years, some methods have been presented for students evaluaton (swas, 995; Chan & Sun, 99; Chen & Lee, 999; Chen & Yan, 998; Chan and Ln, 99; Frar, 995; Echauz & Vachtsevanos, 995; Hwan, Ln, & Ln, 006; Kaburlasos, Marna, & Tsoukalas, 00; Law, 996; Ma & Zhou, 000; Lu, 005; McMartn, Mckenna, & Youssef, 000; Nykanen, 006; Pears, Danels, erlund, & Erckson, 00; Wan & Chen 006a; Wan & Chen, 006b; Wan & Chen, 006c; Wan & Chen, 006d; Weon & Km, 00; Wu, 00). Chan and Sun (99) presented a method for fuzzy assessment of learnn performance of junor hh school students. Chen and Lee (999) presented two methods for evaluatn students answerscrpts usn fuzzy sets. Chen and Yan (998) presented a method for usn fuzzy sets n educaton radn systems. Chan and Ln (99) presented a method for applyn the fuzzy set theory to teachn assessment. Frar (995) presented a method for student peer evaluatons usn the analytc herarchy process method. Echauz and Vachtsevanos (995) presented a fuzzy radn system to translate a set of scores nto letter rades. Hwan, Ln and Ln, (006) presented an approach for test-sheet composton wth lare-scale tem banks. Kaburlasos, Marna, and Tsoukalas (00) presented a software tool, called PRES, for computer-based testn and evaluaton used n the Greek hher educaton system. Law (996) presented a method for applyn fuzzy numbers n educaton radn systems. Lu (005) presented a method for usn mutual nformaton for adaptve tem comparson and student assessment. Ma and Zhou (000) presented a fuzzy set approach for the assessment of student-centered learnn. McMartn, Mckenna and Youssef (000) used scenaro assnments as assessment tools for underraduate enneern educaton. Nykanen (006) presented nducn fuzzy models for student classfcaton. Pears, Danels, erlund, and Erckson (00) presented a method for student evaluaton n an nternatonal collaboratve project course. Wan and Chen (006a) presented two methods for students answerscrpts evaluatons usn fuzzy sets. Wan and Chen (006b) presented two methods for evaluatn students answerscrpts usn fuzzy numbers assocated wth derees of confdence. Wan and Chen (006c) presented two methods for students answerscrpts evaluaton usn vaue sets. Weon and Km (00) presented a leann achevement evaluaton stratey n student s learnn procedure usn fuzzy membershp ISSN 6-5 (onlne) and (prnt). Internatonal Forum of Educatonal Technoloy & Socety (IFETS). The authors and the forum jontly retan the copyrht of the artcles. Permsson to make dtal or hard copes of part or all of ths work for personal or classroom use s ranted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantae and that copes bear the full ctaton on the frst pae. Copyrhts for components of ths work owned by others than IFETS must be honoured. bstractn wth credt s permtted. To copy otherwse, to republsh, to post on servers, or to redstrbute to lsts, requres pror specfc permsson and/or a fee. Request permssons from the edtors at knshuk@eee.or.

2 functons. Wu (00) presented a method for applyn the fuzzy set theory and the tem response theory to evaluate learnn performance. swas (995) ponted out that the chef am of educaton nsttutons s to provde students wth the evaluaton reports reardn ther test/examnaton as suffcent as possble and wth the unavodable error as small as possble. Therefore, swas (995) presented a fuzzy evaluaton method (fem) for applyn fuzzy sets n students answerscrpts evaluaton. He also eneralzed the fuzzy evaluaton method to propose a eneralzed fuzzy evaluaton method (fem) for students answerscrpts evaluaton. In (swas, 995), the fuzzy marks awarded to answers n the students answerscrpts are represented by fuzzy sets (Zadeh, 965). In a fuzzy set, the rade of membershp of an element u n the unverse of dscourse U belonn to a fuzzy set s represented by a real value between zero and one, However, Gau and uehrer (99) ponted out that ths snle value between zero and one combnes the evdence for u U and the evdence aanst u U. They ponted out that t does not ndcate the evdence for u U and the evdence aanst u U, respectvely, and t does not ndcate how much there s of each. Gau and uehrer (99) also ponted out that the snle value between zero and one tells us nothn about ts accuracy. Thus, they proposed the theory of vaue sets, where each element n the unverse of dscourse belonn to a vaue set s represented by a vaue value. Therefore, f we can allow the marks awarded to the questons of the students answerscrpts to be represented by vaue sets, then there s room for more flexblty. In ths paper, we present two new methods for evaluatn students answerscrpts based on the smlarty measure between vaue sets. The vaue marks awarded to the answers n the students answerscrpts are represented by vaue sets, where each element belonn to a vaue set s represented by a vaue value. n ndex of optmsmλ(chen and Yan, 998) determned by the evaluator s used to ndcate the deree of optmsm of the evaluator, whereλ [0, ]. If 0 λ< 0.5, then the evaluator s a pessmstc evaluator. Ifλ= 0.5, then the evaluator s a normal evaluator. If 0.5 <λ.0, then the evaluator s an optmstc evaluator. ecause the proposed methods use vaue sets to evaluate students answerscrpts rather than fuzzy sets, they can evaluate students answerscrpts n a more flexble and more ntellent manner. Especally, they are partcularly useful when the assessment nvolves subjectve evaluaton. The proposed methods can evaluate students answerscrpts more stable than swas methods (995). In ths paper, we present two new methods for students answerscrpts evaluaton based on the smlarty measure between vaue sets. The vaue marks awarded to the answers n the students answerscrpts are represented by vaue sets. n ndex of optmsmλ(chen and Yan, 998) determned by the evaluator s used to ndcate the deree of optmsm of the evaluator, whereλ [0, ]. If 0 λ< 0.5, then the evaluator s a pessmstc evaluator. If λ= 0.5, then the evaluator s a normal evaluator. If 0.5 <λ.0, then the evaluator s an optmstc evaluator. The proposed methods can evaluate students answerscrpts n a more flexble and more ntellent manner. Especally, they are partcularly useful when the assessment nvolves subjectve evaluaton. The proposed methods can evaluate students answerscrpts more stable than swas s methods (995). asc Concepts of the Vaue Set Theory Gau and uehrer (99) presented the theory of vaue sets. Chen (995a) presented the arthmetc operatons between vaue sets. In (Chen, 995b) and (Chen, 997), Chen presented smlarty measures between vaue sets. vaue set à n the unverse of dscourse U s characterzed by a truth-membershp functon t à and a false-membershp functon f Ã, where t à : U [0, ], f à : U [0, ], t à (u ) s a lower bound of the rade of membershp of u derved from the evdence for u, f à (u ) s a lower bound of the neaton of u derved from the evdence aanst u, t à (u ) + f à (u ), and u U. The rade of membershp of u n the vaue set à s bounded by a subnterval [t à (u ), f à (u )] of [0, ]. The vaue value [t à (u ), f à (u )] ndcates that the exact rade of membershp μ à (u ) of u s bounded by t à (u ) μ à (u ) f à (u ), where t à (u ) + f à (u ). n example of a vaue set à n the unverse of dscourse U s shown n F.. If the unverse of dscourse U s a fnte set, then a vaue set à of the unverse of dscourse U can be represented as follows: n =. () à = [ t ( u ), f ( u )]/ u 5

3 If the unverse of dscourse U s an nfnte set, then a vaue set à of the unverse of dscourse can be represented as t ( u ), ( ) / U f u u u U, () à = [ ], where the symbol denotes the unon operator. t à (U), - f à (U).0 - f à (U) - ƒ à (u ) t à (U) t à (u ) 0 u U Fure. vaue set Defnton : Let à be a vaue set of the unverse of dscourse U wth the truth-membershp functon t à and the falsemembershp functon f Ã, respectvely. The vaue set à s convex f and only f for all u, u n U, t à (λ u + ( λ )u ) Mn(t à (u ), t à (u, () f à (λ u + ( λ ) u ) Mn( f à (u ), f à (u, () whereλ [0, ]. Defnton : vaue set à of the unverse of dscourse U s called a normal vaue set f u U, such that f à (u ) =. That s, f à (u ) = 0. Defnton : vaue number s a vaue subset n the unverse of dscourse U that s both convex and normal. Chen (995b) presented a smlarty measure between vaue values. Let X = [t x, f x ] be a vaue value, where t x [0, ], f x [0, ] and t x + f x. The score of the vaue value X can be evaluated by the score functon S shown as follows: X) = t x f x, (5) where X) [-, ]. Let X and Y be two vaue values, where X = [t x, f x ], Y = [t y, f y ], t x [0, ], f x [0, ], t x + f x, t y [0, ], f y [0, ], and t y + f y. The deree of smlarty M(X, Y) between the vaue values X and Y can be evaluated by the functon M, X ) Y ) M ( X, Y ) =, (6) where X) = t x f x and Y) = t y f y. The larer the value of M(X, Y), the hher the deree of smlarty between the 6

4 vaue values X and Y. It s obvous that f X and Y are dentcal vaue values (.e., X = Y), then X) = Y). y applyn Eq. (6), we can see that M(X, Y) =,.e., the deree of smlarty between the vaue values X and Y s equal to. Table shows some examples of the deree of smlarty M(X, Y) between X and Y. Table. Some examples of the deree of smlarty M(X, Y) between the vaue values X and Y X Y M(X, Y) [, ] [0, 0] 0 [, ] [, 0] [, 0] [, ] [0, ] [0, ] Let X and Y be two vaue values, where X = [t x, f x ], Y = [t y, f y ], t x [0, ], f x [0, ], t x + f x, t y [0, ], f y [0, ], and t y + f y. The proposed smlarty measure between vaue values has the follown propertes: Property : Two vaue values X and Y are dentcal f and only f M(X, Y) =. Proof: () If X and Y are dentcal, then t x = t y and f x = f y (.e., f x = f y ). ecause X) = t x f x and Y) = t y f y = t x f x, the deree of smlarty between the vaue values X and Y s calculated as follows: X ) Y ) M ( X, Y ) = ( t x f x ) ( t y f y ) = = ( t =. () If M(X, Y) =, then x f ) ( t X ) Y ) M ( X, Y ) = ( t x f x ) ( t y f y ) = x x f x ) =. It mples that t x = f x and t y = f y (.e., t y = f y ). Therefore, the vaue values X and Y are dentcal. Q. E. D. Property : M(X, Y) = M(Y, X). Proof: ecause and M ( X, Y ) = M ( X, Y ) = X ) Y ), Y ) X ), 7

5 we can see that X ) Y ) Y ) X ) =, X ) Y ) = Y ) Y ) and M(X, Y) = M(Y, X). Q. E. D. Let à and be two vaue sets n the unverse of dscourse U, U = {u, u,, u n }, where à = [ t ( u ), f ( u )] / u + [ t ( u ), f ( u )] / u + + [ t ( un ), f ( un )] / u n, and = t ( u ), f ( )] / u + t ( u ), f ( )] / u + + t ( u ), f ( u )]/ u n. [ u [ u [ n n Let V ( ) = [ t ( ), f ( ) ] be the vaue membershp value of u n the vaue set Ã, and let V ( u ) = u u u [ t ( u ), f ( u ) ] be the vaue membershp value of u n the vaue set. y applyn Eq. (5), we can see ( V ( u that the score S of the vaue membershp value V ( ) can be evaluated as follows: S V ( u = t ( ) f ( ), ( u u and the score S V ( u of the vaue membershp value V ( u ) can be evaluated as follows: ( u S V ( u = t ( u ) f ( u ), ( where n. Then, the deree of smlarty H(Ã, ) between the vaue sets à and can be evaluated by the functon H, n H (, ) = M ( V ( u ), V u ( n = n V ( u V ( u, = = (7) n where H(Ã, ) [0, ]. The larer the value of H(Ã, ), the hher the smlarty between the vaue sets à and. Let à and be two vaue sets n the unverse of dscourse U, U = {u, u,, u n }, where à = [ t ( u ), f ( u )] / u + [ t ( u ), f ( u )] / u + + [ t ( un ), f ( un )] / u n, and = [ t ( u ), f ( u )] / u + [ t ( u ), f ( u )] / u + + [ ( un ), f ( un )] The proposed smlarty measure between vaue sets has the follown propertes: Property : Two vaue sets à and are dentcal f and only f H(Ã, ) =. Proof: () If à and are dentcal, then t / u n. 8

6 t ( u ), f ( u )] = t ( u ), f ( u )], where n. [ That s, ( ) u [ ( u f ( u t = t ( u ), f ) = ), and n. ecause and S V ( u = t ( ) f ( ) ( u u S V ( u = t ( u ) f ( u ) = t ( ) f ( ) = S V ( u. ( Therefore, we can see that u u ( H(Ã, ) = n V ( u V ( u = n () If H(Ã, ) =, then n V ( u V ( u = n = =. H(Ã, n V ( u V ( u ) = =. n = It mples that S V ( u = S V ( u, where n. ecause S V ( u = ( ( S V ( u and S V ( u = t ( u ) f ( u ), where n, we can see that ( ( ( u t ( ) = t ( u ) and f ) = f ( u ) (.e., f ( ) = ( u ) u u f ), where n. Therefore, the vaue sets à and are dentcal. Q. E. D. Property : H(Ã, ) = H(, Ã). Proof: ecause H(Ã, n V ( u V ( u ) = n = and H( n V ( u ) V ( u, Ã) = n = and because n V ( u V ( u = n, n V ( u ) V ( u = = n, we can see that H(Ã, ) = H(, Ã). Q. E. D. ( 9

7 Example : Let à and be two vaue sets of the unverse of dscourse U, U = {u, u, u, u, u 5 }, à = [0., 0.]/u + [0., 0.5]/u + [0.5, 0.7]/u + [0.7, 0.9]/u + [0.8, ]/u 5 = [0., 0.5]/u + [0., 0.6]/u + [0.6, 0.8]/u + [0.7, 0.9]/u + [0.8, ]/u 5, where V ( u ) = [0., 0.], V u ) = [0., 0.5], ( V ( u ) = [0., 0.5], u ) V = [0., 0.6], ( V ( u ) = [0.5, 0.7], u ) V = [0.6, 0.8], ( V ( u ) = [0.7, 0.9], u ) V = [0.7, 0.9], ( V ( u 5 ) = [0.8, ], V ( u ) 5 = [0.8, ]. y applyn Eq. (5), we can et = = 0., V ( u S V ( u S S S ( ( V ( u ( V ( u ( V ( u 5 ( V ( u = = 0., = = 0., = = 0.6, = = 0.8, S = = 0., S V ( u = = 0, ( S V ( u = = 0., ( S V ( u = = 0.6, ( S V ( u = = 0.8. ( 5 y applyn Eq. (7), the deree of smlarty H(Ã, ) between the vaue sets à and can be evaluated, shown as follows: H(Ã, 5 V ( u V ( u ) = = 5 0. ( 0.) = ( = ( ) 5 = 0.9. It ndcates that the deree of smlarty between the vaue sets à and s equal to

8 Revew of swas Methods for Students nswerscrpts Evaluaton swas (995) used the matchn functon S to measure the deree of smlarty between two fuzzy sets (Zadeh, 965). Let and be the vector representaton of the fuzzy sets and, respectvely. Then, the deree of smlarty, ) between the fuzzy sets and can be calculated as follows (Chen, 988):, ) =, (8) Max(, ) where, ) [0, ]. The larer the value of, ), the hher the smlarty between the fuzzy sets and. swas (995) presented a fuzzy evaluaton method (fem) for evaluatn students answerscrpts, based on the matchn functon S. He used fve fuzzy lnustc hedes, called Standard Fuzzy Sets (SFS), for students answerscrpts evaluaton,.e., E (excellent), V (very ood), G (ood), S (satsfactory) and U (unsatsfactory), where X = {0%, 0%, 0%, 60%, 80%, 00%}, E = {(0%, 0), (0%, 0), (0%, 0.8), (60%, 0.9), (80%, ), (00%, )}, V = {(0%, 0), (0%, 0), (0%, 0.8), (60%, 0.9), (80%, 0.9), (00%, 0.8)}, G = {(0%, 0), (0%, 0.), (0%, 0.8), (60%, 0.9), (80%, 0.), (00%, 0.)}, S = {(0%, 0.), (0%, 0.), (0%, 0.9), (60%, 0.6), (80%, 0.), (00%, 0)}, U = {(0%, ), (0%, ), (0%, 0.), (60%, 0.), (80%, 0), (00%, 0)}. He used the vector representaton method to represent the fuzzy sets E, V, G, S and U by the vectors E, V,G, S and U, respectvely, where E = <0, 0, 0.8, 0.9,, >, V = <0, 0, 0.8, 0.9, 0.9, 0.8>, G = <0, 0., 0.8, 0.9, 0., 0.>, S = <0., 0., 0.9, 0.6, 0., 0>, U = <,, 0., 0., 0, 0>. swas ponted out that,, C, D and E are letter rades, where 0 E < 0, 0 D < 50, 50 C < 70, 70 < 90 and Furthermore, he presented the concept of md-rade-ponts, where the md-radeponts of the letter rades,, C, D and E are P(), P(), P(C), P(D) and P(E), respectvely, P() = 95, P() = 80, P(C) = 60, P(D) = 0 and P(E) = 5. ssume that an evaluator evaluates the frst queston (.e., Q.) of the answerscrpt of a student usn a fuzzy rade sheet as shown n Table. Table. fuzzy rade sheet (swas, 995) Queston No. Fuzzy mark Grade Q Q. Q. Total mark = In the second row of Table, the fuzzy marks 0., 0., 0., 0.6, 0.8 and 0.9, awarded to the answer of queston Q., ndcate that the derees of the evaluator s satsfacton for that answer are 0%, 0%, 0%, 60%, 80% and 00%, respectvely. In the follown, we brefly revew swas method (995) for students answerscrpt evaluaton as follows: Step : For each queston n the answerscrpt repeatedly perform the follown tasks: () The evaluator awards a fuzzy mark F to each queston Q. and flls up each cell of the th row for the frst seven

9 columns shown n Table, where n. Let F be the vector representaton of F, where n. () ased on Eq. (8), calculate the derees of smlarty E ), V ), G ), S ) and U ), respectvely, where E, V, G, S and U are the vector representatons of the standard fuzzy sets E (excellent), V (very ood), G (ood), S (satsfactory) and U (unsatsfactory), respectvely, and n. () Fnd the maxmum value amon the values of E ), V ), G ), S ) and U ). ssume that V ) s the maxmum value amon the values of E ), V ), G ), S ) and U ), then award the letter rade to the queston Q. due to the fact that the letter rade corresponds to the standard fuzzy set V (very ood). If E ) = V ) s the maxmum value amon the values of E ), V ), G ), S ) and U ), then award the letter rade to the queston Q. due to the fact that the letter rade corresponds to the standard fuzzy set E (excellent). Step : Calculate the total mark of the student as follows: n Total Mark = 00 [ T ( Q. ) P( )], (9) = where T(Q.) denotes the mark allotted to Q. n the queston paper, denotes the rade awarded to Q. by Step of the alorthm, P( ) denotes the md-rade-pont of, and n. Put ths total score n the approprate box at the bottom of the fuzzy rade sheet. swas (995) also presented a eneralzed fuzzy evaluaton method (fem) for students answerscrpts evaluaton, where a eneralzed fuzzy rade sheet shown n Table s used to evaluate the students answerscrpts. Table. eneralzed fuzzy rade sheet (swas, 995) Derved letter Queston No. Fuzzy mark rade Mark F F F Q. F m F F F Q. F m Total mark = In the eneralzed fuzzy rade sheet shown n Table, for all j =,,, and for all, j denotes the derved letter rade by the fuzzy evaluaton method fem for the awarded fuzzy mark F j and m denotes the derved mark awarded to the queston Q., where T (Q.) P ( j ), (0) m = 00 and the Total Mark = m. n = j=

10 New Method for Evaluatn Students nswerscrpts ased on the Smlarty Measure between Vaue Sets In ths secton, we present a new method for evaluatn students answerscrpts based on the smlarty measure between vaue sets. Let X be the unverse of dscourse. We use fve fuzzy lnustc hedes, called Standard Vaue Sets (SVS), for students answerscrpts evaluaton,.e., E (excellent), V (very ood), G (ood), S (satsfactory) and U (unsatsfactory), where X = {0%, 0%, 0%, 60%, 80%, 00%}, E = [0, 0]/0% + [0, 0]/0% + [0, 0]/0% + [0., 0.5]/60% + [0.8, 0.9]/80% + [, ]/00%, V = [0, 0]/0% + [0, 0]/0% + [0, 0]/0% + [0., 0.5]/60% + [, ]/80% + [0.7, 0.8]/00%, G = [0, 0]/0% + [0, 0]/0% + [0., 0.5]/0% + [, ]/60% + [0.8, 0.9]/80% + [0., 0.5]/00%, S = [0, 0]/0% + [0., 0.5]/0% + [, ]/0% + [0.8, 0.9]/60% + [0., 0.5]/80% + [0, 0]/00%, U = [, ]/0% + [, ]/0% + [0., 0.5]/0% + [0., 0.]/60% + [0, 0]/80% + [0, 0]/00%. ssume that,, C, D and E are letter rades, where 0 E < 0, 0 D < 50, 50 C < 70, 70 < 90 and ssume that an evaluator evaluates the frst queston (.e., Q.) of a student s answerscrpt, usn a vaue rade sheet as shown n Table. Table. vaue rade sheet Queston No. Vaue mark Q. [0, 0] [0., 0.] [0., 0.] [0.6, 0.7] [0.7, 0.8] [, ] Q. Q. Derved letter rade Q.n Total mark = In the second row of the vaue rade sheet shown n Table, the vaue marks [0, 0], [0., 0.], [0., 0.], [0.6, 0.7], [0.7, 0.8] and [, ], awarded to the answer of queston Q., ndcate that the derees of the evaluator s satsfacton for that answer are 0%, 0%, 0%, 60%, 80% and 00%, respectvely. Let the vaue mark of the answer of queston Q. be denoted by F. Then, we can see that F s a vaue set of the unverse of dscourse X, where X = {0%, 0%, 0%, 60%, 80%, 00%}, F = [0, 0]/0% + [0., 0.]/0% + [0., 0.]/0% + [0.6, 0.7]/60% + [0.7, 0.8]/80% + [, ]/00%. The proposed vaue evaluaton method (VEM) for students answerscrpts evaluaton s presented as follows: Step : For each queston n the answerscrpt repeatedly perform the follown tasks: () The evaluator awards a vaue mark flls up each cell of the th row for the frst seven columns shown n Table, where n. () ased on Eq. (7), calculate the derees of smlarty H( E ), H(V ), H(G ), H( S, H(U ), respectvely, where E (excellent), V (very ood), G (ood), S (satsfactory) and U (unsatsfactory) are standard vaue sets. F represented by a vaue set to each queston Q. by hs/her judment and F ) and

11 () Fnd the maxmum value amon the values of H( E ), H(V ), H(G ), H( S ) and H(U ). If H(W ) s the larest value amon the values of H( E ), H(V ), H(G ), H( S ) and H(U ), wherew { E,V,G, S,U }, then translate the standard vaue set W nto the correspondn letter rade, where the standard vaue set E s translated nto the letter rade, the standard vaue set V s translated nto the letter rade, the standard vaue set G s translated nto the letter rade C, the standard vaue set S s translated nto the letter rade D, and the standard vaue set U s translated nto the letter rade E. For example, assume that H(V ) s the maxmum value amon the values of H( E ), H(V ), H(G ), H( S ) and H(U ), then award rade to the queston Q. due to the fact that the letter rade corresponds to the standard vaue set V (very ood). If H( E ) = H(V ) s the maxmum value amon the values of H( E ), H(V ), H(G ), H( S ) and H(U ), then award the letter rade to the queston Q. due to the fact that the letter rade corresponds to the standard vaue set E (excellent). Step : Calculate the total mark of the student as follows: Total Mark = n [ T ( Q. ) K( ) H ( w )], () 00 = where T(Q.) denotes the mark allotted to the queston Q. n the queston paper, denotes the letter rade awarded to Q. by Step, K( ) denotes the derved rade-pont of the letter rade based on the ndex of optmsm λ determned by the evaluator, where λ [0, ], H(W ) s the maxmum value amon the values of H( E, H(V ), H(G ), H( S ) and H(U ), W { E,V,G, S,U }, such that the derved letter rade awarded to the queston Q. s, and n. If 0 λ < 0.5, then the evaluator s a pessmstc evaluator. If λ = 0.5, then the evaluator s a normal evaluator. If 0.5 < λ.0, then the evaluator s an optmstc evaluator. ssume that the derved letter rade obtaned n Step wth respect to the queston Q. s, where {,, C, D, E} and 0 y y 00, then the derved rade-pont K( ) shown n Eq. (8) s calculated as follows: K( ) = ( λ) y + λ y, () where λ s the ndex of optmsm determned by the evaluator, λ [0, ], and 0 y K( ) y 00. Put the derved total mark n the approprate box at the bottom of the vaue rade sheet. Example : Consder a student s answerscrpt to an examnaton of 00 marks. ssume that n total there are four questons to be answered: TOTL MRKS = 00, Q. carres 0 marks, Q. carres 0 marks, Q. carres 0 marks, Q. carres 0 marks. ssume that an evaluator awards the student s answerscrpt usn the vaue rade sheet shown n Table 5, where the ndex of optmsmλ determned by the evaluator s 0.60,.e.,λ = ssume that,, C, D and E are letter rades, where 0 E < 0, 0 D < 50, 50 C < 70, 70 < 90 and Table 5. Vaue rade sheet of Example Queston No. Vaue mark Derved letter rade Q. [0, 0] [0, 0] [0, 0] [0., 0.5] [, ] [0.5, 0.6] Q. [0, 0] [0, 0] [0, 0] [0., 0.5] [0.8, 0.9] [, ] Q. [0, 0] [0., 0.5] [, ] [0.6, 0.7] [0., 0.5] [0, 0] Q. [0.8, 0.9] [0.5, 0.6] [0., 0.] [0, 0] [0, 0] [0, 0] Total mark = F ),

12 From Table 5, we can see that the vaue marks of the questons Q., Q., Q. and Q. represented by vaue sets are F, F and F, respectvely, where F = [0, 0]/0% + [0, 0]/0% + [0, 0]/0% + [0., 0.5]/60% + [, ]/80% + [0.5, 0.6]/00%, F = [0, 0]/0% + [0, 0]/0% + [0, 0]/0% + [0., 0.5]/60% + [0.8, 0.9]/80% + [, ]/00%, F = [0, 0]/0% + [0., 0.5]/0% + [, ]/0% + [0.6, 0.7]/60% + [0., 0.5]/80% + [0, 0]/00%, F = [0.8, 0.9]/0% + [0.5, 0.6]/0% + [0., 0.]/0% + [0, 0]/60% + [0, 0]/80% + [0, 0]/00%. [Step ] ccordn to the standard vaue sets E,V,G, S,U and the vaue marks F, F, F, we can et the vaue values, as shown n Table 6. Table 6. Vaue values of Example t V ( t E ) V V ( t) V G ( t) V S ( t) V U ( t) V F ( t) V ( t) F V F ( t) V ( t) F 0 % [0, 0] [0, 0] [0, 0] [0, 0] [, ] [0, 0] [0, 0] [0, 0] [0.8, 0.9] 0 % [0, 0] [0, 0] [0, 0] [0., 0.5] [, ] [0, 0] [0, 0] [0., 0.5] [0.5, 0.6] 0 % [0, 0] [0, 0] [0., 0.5] [, ] [0., 0.5] [0, 0] [0, 0] [, ] [0., 0.] 60 % [0., 0.5] [0., 0.5] [, ] [0.8, 0.9] [0., 0.] [0., 0.5] [0., 0.5] [0.6, 0.7] [0, 0] 80 % [0.8, 0.9] [, ] [0.8, 0.9] [0., 0.5] [0, 0] [, ] [0.8, 0.9] [0., 0.5] [0, 0] 00 % [, ] [0.7, 0.8] [0., 0.5] [0, 0] [0, 0] [0.5, 0.6] [, ] [0, 0] [0, 0] y applyn Eq. (5), we can et scores of the vaue values, as shown n Table 7. Table 7. Scores of the vaue values of Example t S ( V ( E V V ( t V G ( t V S ( t V U ( t V F ( t V ( t F V F ( t V ( F 0 % % % % % % Table 8. The derees of smlarty between the vaue sets H(X, Y) Y F F F F X E V G S U

13 y applyn Eq. (7), we can et the deree of smlarty H(X, Y) between the vaue values X and Y, where X { E,V,G, S,U } and Y { F }, as shown n Table 8. ecause H(V ) s the maxmum value amon the values of H( E ), H(V ), H(G ), H( S ) and H(U ), we award rade to the queston Q. due to the fact that the letter rade corresponds to the standard vaue set V (very ood). ecause H( E ) s the maxmum value amon the values of H( E ), H(V ), H(G ), H( S ) and H(U ), we award rade to the queston Q. due to the fact that the letter rade corresponds to the standard vaue set E (excellent). ecause H( S ) s the maxmum value amon the values of H( E ), H(V ), H(G ), H( S ) and H(U ), we award rade D to the queston Q. due to the fact that the letter rade D corresponds to the standard vaue set S (satsfactory). ecause H(U ) s the maxmum value amon the values of H( E ), H(V ), H(G ), H( S ) and H(U ), we award rade E to the queston Q. due to the fact that the letter rade E corresponds to the standard vaue set U (unsatsfactory). [Step ] ecause 90 00, 70 < 90, 0 D < 50 and 0 E < 0, where,, D and E are letter rades, and the ndex of optmsmλ determned by the evaluator s 0.60 (.e.,λ = 0.60), based on Eq. (), we can et the follown results: K() = ( 0.60) = 96, K() = ( 0.60) = 8, K(D) = ( 0.60) =, K(E) = ( 0.60) = 8. ecause the questons Q., Q., Q. and Q. carry 0 marks, 0 marks, 0 marks and 0 marks, respectvely, and because H(V ) = 0.967, H( E ) =.000, H( S ) = and H(U ) = 0.85, based on Eq. (), the total mark of the student s evaluated as follows: ( ) 00 = ( ) 00 = 6.68 = 6 (assumn that no half mark s ven n the total mark). Generalzed Method for Evaluatn Students nswerscrpts ased on the Smlarty Measure between Vaue Sets In ths secton, we present a eneralzed vaue evaluaton method (GVEM) for students answerscrpts evaluaton based on the smlarty measure between vaue sets, where a eneralzed vaue rade sheet shown n Table 9 s used to evaluate the students answerscrpts. Table 9. eneralzed vaue rade sheet Queston No. Sub-questons Vaue mark Derved letter rade Mark Q. F Q. Q. F m 6

14 Q. Q. Q. Q. Q. Q. Q. F F F F F F m Q.n Q.n n Q.n n Q.n n Q.n n F n F n F n F n Total mark = m n In the eneralzed vaue rade sheet shown n Table 9, each queston Q. conssts of four sub-questons,.e., Q., Q., Q. and Q.. For all j =,,, and for all, j s the derved letter rade by the proposed vaue evaluaton method VEM of the awarded vaue mark F wth respect to the sub-queston Q.j, and m s the derved mark j awarded to the queston Q., m = T (Q.) [ K ( ) (, )], 00 j H w Fj () j = and Total Mark = m. n = where T(Q.) denotes the mark allotted to Q. n the queston paper, j denotes the derved letter rade awarded to Q., and K( j ) denotes the derved rade-pont of the letter rade j based on the ndex of optmsm λ determned by the evaluator, whereλ [0, ], H(W j ) s the maxmum value amon the values of H( E j ), H(V j ), H(G j ), H( S j ) and H(U j ), W { E,V,G, S,U }, such that the derved letter rade awarded to the queston Q.j s j, j, and n. If 0 λ < 0.5, then the evaluator s a pessmstc evaluator. Ifλ = 0.5, then the evaluator s a normal evaluator. If 0.5 <λ.0, then the evaluator s an optmstc evaluator. ssume that the derved letter rade wth respect to the sub-queston Q.j s j, where j {,, C, D, E} and 0 y j y 00, then the derved rade-pont K( j ) shown n Eq. () s calculated as follows: K( j ) = ( λ ) y +λ y, () whereλ s the ndex of optmsm determned by the evaluator,λ [0, ], and 0 y K( j ) y 00. Put the derved total mark n the approprate box at the bottom of the eneralzed vaue rade sheet. Expermental Results We have made an experment to compare the evaluatn results of the proposed method wth the swas method (995) for dfferent days. In our experment, there are four questons to be answered n a student s answerscrpt, where TOTL MRKS = 00, Q. carres 0 marks, 7

15 Q. carres 5 marks, Q. carres 5 marks, Q. carres 0 marks. ssume that the ndex of optmsmλ of the evaluator s 0.60 (.e., λ = 0.60). The evaluator uses swas method (995) and the proposed method to evaluate the student s answerscrpt on dfferent days, respectvely. The results are shown n F. and F., respectvely. comparson of the evaluatn results of the student s answerscrpt s shown n Table 0. From Table 0, we can see that the proposed method s more stable to evaluate students answerscrpts than swas method (995). It can evaluate students answerscrpts n a more flexble and more ntellent manner. July, 006 Queston No. Satsfacton Levels Grade Q Q Q Q Total Mark = July, 006 Queston No. Satsfacton Levels Grade Q Q Q Q Total Mark = July, 006 Queston No. Satsfacton Levels Grade Q Q Q Q Total Mark = July, 006 Queston No. Satsfacton Levels Grade Q Q Q Q Total Mark = Fure. Evaluatn the student s answerscrpt at dfferent days usn swas method (995) July, 006 Queston No. Vaue marks Grade Q. [0, 0] [0, 0] [0, 0] [0.6, 0.7] [0.8, 0.9] [0.8, 0.9] Q. [0, 0] [0, 0] [0.6, 0.7] [0.8, 0.9] [0.8, 0.9] [0, 0] Q. [0, 0] [0, 0] [0, 0] [0.6, 0.7] [0.8, 0.9] [0.8, 0.9] Q. [0, 0] [0.5, 0.6] [0.8, 0.9] [0.7, 0.8] [0., 0.] [0, 0] Total mark = July, 006 Queston No. Vaue marks Grade 8

16 Q. [0, 0] [0, 0] [0, 0] [0.7, 0.8] [0.8, 0.9] [0.9,.0] Q. [0, 0] [0, 0] [0.6, 0.7] [0.8, 0.9] [0.8, 0.9] [0, 0] Q. [0, 0] [0, 0] [0, 0] [0.7, 0.8] [0.8, 0.9] [0.8, 0.9] Q. [0, 0] [0.5, 0.6] [0.8, 0.9] [0.7, 0.8] [0, 0] [0, 0] Total mark = July, 006 Queston No. Vaue marks Grade Q. [0, 0] [0, 0] [0, 0] [0.6, 0.7] [0.8, 0.9] [0.7, 0.8] Q. [0, 0] [0, 0] [0.6, 0.7] [0.8, 0.9] [0.7, 0.8] [0, 0] Q. [0, 0] [0, 0] [0, 0] [0.5, 0.6] [0.7, 0.8] [0.8, 0.9] Q. [0, 0] [0.5, 0.6] [0.8, 0.9] [0.6, 0.7] [0, 0] [0, 0] Total mark = July, 006 Queston No. Vaue marks Grade Q. [0, 0] [0, 0] [0, 0] [0.6, 0.7] [0.8, 0.9] [0.8, 0.9] Q. [0, 0] [0, 0] [0.5, 0.6] [0.8, 0.9] [0.7, 0.8] [0, 0] Q. [0, 0] [0, 0] [0, 0] [0.7, 0.8] [0.8, 0.9] [0.8, 0.9] Q. [0, 0] [0.6, 0.7] [0.8, 0.9] [0.7, 0.8] [0, 0] [0, 0] Total mark = Fure. Evaluatn the student s answerscrpt at dfferent days usn the proposed method Table 0. comparson of the evaluatn results for dfferent methods Methods Total mark Days swas method (995) The proposed method July, July, July, July, The Merts of the Proposed Methods The proposed methods have the follown advantaes: () The proposed methods are more flexble and more ntellent than swas methods (995) due to the fact that we use vaue sets rather than fuzzy sets to represent the vaue mark of each queston, where the evaluator can use vaue values to ndcate the deree of the evaluator s satsfacton for each queston. Especally, the proposed methods are partcularly useful when the assessment nvolves subjectve evaluaton. () The proposed methods are more stable to evaluate students answerscrpts than swas methods (995). They can evaluate students answerscrpts n a more flexble and more ntellent manner. Conclusons In ths paper, we have presented two new methods for evaluatn students answerscrpts based on the smlarty measure between vaue sets. The vaue marks awarded to the answers n the students answerscrpts are represented by vaue sets, where each element belonn to a vaue set s represented by a vaue value. n ndex of optmsmλ determned by the evaluator s used to ndcate the deree of optmsm of the evaluator, whereλ [0, ]. ecause the proposed methods use vaue sets to evaluate students answerscrpts rather than fuzzy sets, they can evaluate students answerscrpts n a more flexble and more ntellent manner. The expermental results show that 9

17 the proposed methods can evaluate students answerscrpts more stable than swas methods (995). cknowledements The authors would lke to thank Professor Jason Chyu Chan, Department of Educaton, Natonal Chench Unversty, Tape, Tawan, Republc of Chna, for provdn very helpful comments and suestons. Ths work was supported n part by the Natonal Scence Councl, Republc of Chna, under Grant NSC 95--E-0-7-MY. References swas, R. (995). n applcaton of fuzzy sets n students evaluaton. Fuzzy Sets and Systems, 7 (), Chan, D. F., & Sun, C. M. (99). Fuzzy assessment of learnn performance of junor hh school students. Paper presented at the Frst Natonal Symposum on Fuzzy Theory and pplcatons, June 5-6, 99, Hsnchu, Tawan. Chen, S. M. (988). new approach to handln fuzzy decsonmakn problems. IEEE Transactons on Systems, Man, and Cybernetcs, 8 (6), Chen, S. M. (995a). rthmetc operatons between vaue sets. Paper presented at the Internatonal Jont Conference of CFS/IFIS/SOFT 95 on Fuzzy Theory and pplcatons, December 7-9, 995, Tape, Tawan. Chen, S. M. (995b). Measures of smlarty between vaue sets. Fuzzy Sets and Systems, 7 (), 7-. Chen, S. M. (997). Smlarty measures between vaue sets and between elements. IEEE Transactons on Systems, Man, and Cybernetcs-Part : Cybernetcs, 7 (), Chen, S. M. (999). Evaluatn the rate of areatve rsk n software development usn fuzzy set theory. Cybernetcs and Systems, 0 (), Chen, S. M., & Lee, C. H. (999). New methods for students evaluatn usn fuzzy sets. Fuzzy Sets and Systems, 0 (), Chen, S. M., & Wan, J. Y. (995). Document retreval usn knowlede-based fuzzy nformaton retreval technques. IEEE Transactons on Systems, Man, and Cybernetcs, 5 (5), Chen, C. H., & Yan, K. L. (998). Usn fuzzy sets n educaton radn system. Journal of Chnese Fuzzy Systems ssocaton, (), Chan, T.T., & Ln C. M. (99). pplcaton of fuzzy theory to teachn assessment. Paper presented at the 99 Second Natonal Conference on Fuzzy Theory and pplcatons, September 5-7, 99, Tape, Tawan. Frar, L. (995). Student peer evaluatons usn the analytc herarchy process method. Paper presented at the Fronters n Educaton Conference, November -, 995, tlanta, G, US. Gau, W. L., & uehrer, D. J. (99). Vaue sets. IEEE Transactons on Systems, Man, and Cybernetcs, (), Echauz, J. R., & Vachtsevanos, G. J. (995). Fuzzy radn system. IEEE Transactons on Educaton, 8 (), Hwan, G. J., Ln,. M. T., & Ln, T. L. (006). n effectve approach for test-sheet composton wth lare-scale tem banks, Computers & Educaton, 6 (), -9. Kaburlasos, V. G., Marna, C. C., & Tsoukalas, V. T. (00). PRES: software tool for computer-based testn 0

18 and evaluaton used n the Greek hher educaton system. Paper presented at the 00 IEEE Internatonal Conference on dvanced Learnn Technoloes, uust 0 September, 00, Joensuunland. Law, C. K. (996). Usn fuzzy numbers n educaton radn system. Fuzzy Sets and Systems, 8 (), -. Lu, C. L. (005). Usn mutual nformaton for adaptve tem comparson and student assessment. Educatonal Technoloy & Socety, 8 (), Ma, J., & Zhou, D. (000). Fuzzy set approach to the assessment of student-centered learnn. IEEE Transactons on Educaton, (), 7-. McMartn., Mckenna,., & Youssef, K. (000). Scenaro assnments as assessment tools for underraduate enneern educaton. IEEE Transactons on Educaton, (), -9. Nykanen, O. (006). Inducn fuzzy models for student classfcaton. Educatonal Technoloy & Socety, 9 (), -. Pears,., Danels, M., erlund,., & Erckson, C. (00). Student evaluaton n an nternatonal collaboratve project course. Paper presented at the Frst Internatonal Workshop on Internet-Supported Educaton, January 08 -, 00, San Deo, C, US. Wan, H. Y., & Chen, S. M. (006a). New methods for evaluatn the answerscrpts of students usn fuzzy sets. Paper presented at the 9 th Internatonal Conference on Industral, Enneern & Other pplcatons of ppled Intellent Systems, June 7-0, 006, nnecyrance. Wan, H. Y., & Chen, S. M. (006b). New methods for evaluatn students answerscrpts usn fuzzy numbers assocated wth derees of confdence. Paper presented at the 006 IEEE Internatonal Conference on Fuzzy Systems, July 6-, 006, Vancouver, C, Canada. Wan, H. Y., & Chen, S. M. (006c). New methods for evaluatn students answerscrpts usn vaue values. Paper presented at the 9 th Jont Conference on Informaton Scences, October 8-, 006, Kaohsun, Tawan. Wan, H. Y., & Chen, S. M. (006d). Evaluatn students answerscrpts based on the smlarty measure between vaue sets. Paper presented at the th Conference on rtfcal Intellence and pplcatons, December 5-6, 006, Kaohsun, Tawan. Weon, S., & Km, J. (00). Learnn achevement evaluaton stratey usn fuzzy membershp functon. Paper presented at the st SEE/IEEE Fronter n Educaton Conference, October 0-, 00, Reno, NV, US. Wu, M. H. (00). research on applyn fuzzy set theory and tem response theory to evaluate learnn performance, Master Thess, Department of Informaton Manaement, Chaoyan Unversty of Technoloy, Tawan. Zadeh, L.. (965). Fuzzy sets. Informaton and Control, 8, 8-5.