Learning Outcomes Based Approach in Higher Education System of Georgia Nino Javakhishvili 1 Tbilisi State University Georgia

Transcription

1 Learning Outcomes Based Approach in Higher Education System of Georgia Nino Javakhishvili 1 Tbilisi State University Georgia Higher education system of Georgia has been in the process of extensive reforms since 2005, for that time two major innovations, directly related to each other, were carried out: The new Higher Education law was adopted by Georgia s Parliament in The components of the law are mainly driven by western European and USA achievements and experience in the field ( Georgia joined the Bologna process in May 2005, expressing its desire to become a member of the European community and be able to exchange students as well as specialists within the united European space. Institutional accreditation decreased number of universities from about 250 to 60. Out of the current universities, 20 are public and 34, private 71% of these institutions are located in Tbilisi, capital of Georgia. There are three cycles of education 240 ECTS for bachelor, 120 ECTS for master and 180 ECTS for PhD levels. Major nationwide achievement is eradication of corruption (Glonti, 2001; Kachkachishvili 2001; Rositashvili, 2004; Hotlge, 2005) prospering in Soviet Union times as for entering higher education institutions, as during study at these institutions. In general, level of research and teaching is constantly improving and all Georgian universities aim at getting closer to the European space of education; although the process often goes on with deviations, difficulties and problems (Machabeli, 2007; Glonti, 2007; Maisuradze, 2007; Glonti, Chitashvili, 2006) One of the most problematic spheres is employment of graduates and relations of higher education institutions with employers/labor market 2. Based on the 2008 Fall study (EPPM, 2008) 60% of universities report to have collected data on employment of their students, but much lower percent can provide actual database. 85% of universities report to have special activities dedicated to employing their graduates, out of them, 53% have contracts with employers on employment and internship issues, 27% of universities organizes meetings with potential employers, 20% of universities carry out regularly inquires about vacancies on job market and makes the information available for their students. However, interviews 3 with experts in the field of higher education show that most of these activities are carried out only superiorly and just as a formality for accountancy reasons. Nearly no university has student placement, only few private universities with business administration programs report to have business sector involved in development of curricula 1 All data and results provided in the article are retrieved from the research in the frames of TUNING-Georgia project, thanks to the courtesy of Pablo beneitone. Beneitone, P. Identifying Key Competences: The Outcomes of the Stakeholders Survey. Presentation made at the working seminar in Gronningen, According to Georgia Department of Labor, unemployment rate is 10.3% in Georgia: and according to department of statistics, it is -13.3% 3 Interviews were carried out in the frames of the study on student-centered learning at Georgian universities with academics and administrators from Minidstry of Education and Science of Georgia (in total 18 interviews). 1

2 Most of the relations with employers then are dedicated to supporting graduates in entering job marker after they have already been educated, while one of the main TUNING principles tells higher education institutions to bring market interests and demands into university curricula and develop needed job skills during study. Carried out in , the Tempus project Application of Tuning approaches in Georgian higher education system (TUNING-Georgia) was important to bring content changes and support development of student-centered higher education system in Georgia. The project was co-coordinated by professors Robert Wagenaar (University of Groningen) and Julia Gonzales (University of Deusto), 10 experts from European countries and about 100 representatives of 7 Georgian universities took part in the project. Student-centered learning is one of the main principles, while learning outcomes based curricula and syllabi are the milestones of the TUNING approach (Beneitone, Esquenti, Gonzalez, Maleta, Siufi, Wagenaar, 2007). Provided that any higher education program is based on learning outcomes in the form of competencies, participants of the TUNING Georgia project carried out a survey of 2500 stakeholders to develop a list of generic and subject specific competences in 10 subject areas included in TUNING project 4. I will first discuss generic competences for Georgia and compare these to the European generic competencies, than I will discuss subject specific competencies for education sciences and teacher education for Georgia, and finally, I will compare Georgian results on subject specific and generic competencies. Generic competencies Georgia 5 The list of Generic competences for Georgia was made based on the European one and consists of 31 competences. We asked stakeholders to rate the list on a 4 point scale (1 none, 2- weak, 3 - considerable, 4 strong) according to perceived importance of these competences and level of development by university degree in educational programs provided by the respondents respective universities. Besides, respondents were asked to rank five most important competences. The stakeholders who filled up the questionnaire were university professors, students, graduates and employers. The survey was carried out on the basis of 7 universities participating in Tuning-Georgia project. All 4 groups were unanimous in their views: by importance, 29 out of 31 competences were rated over 3: means varied from 3.83 to 3.11 and respective standard deviations were small enough from 0.01 to Overall the importance scores varied from 2.76 to In terms of achievement, fewer competences rated over 3: from 13 - for students and employers through 14 for professors to 18 for graduates. Means varied from 3.00 to 3.38 and standard deviations varied from 0.04 to Not surprisingly, mean scores 4 The ten subject areas are: History, European Studies, Nursing, Education Sciences and Teacher Education, Earth Sciences, Mathematics, Physics, Engineering, Chemistry, Business Administration. 5 I was not able to obtain information about differences among 10 subject groups regarding importance of generic competences. Most likely, there should have been variances in answers, let us say, of business administration and history representatives to entrepenership and some other competences. 2

3 for achievement are lower than those for importance, but positively enough the gap is quite small Most of these competences are rated over 3 and the scores are very close to each other, however, it is very interesting to look at their hierarchical order. Importance: In terms of the top five competences, all 4 groups named 3 common competences: Ability to apply knowledge in practical situations Capacity to learn and stay up-to-date with learning Ability to work autonomously All 3 groups, but students, had one more competence in common: Ability to communicate both orally and through the written word in native language Two groups: professors (3d from the top) and students (5 th from the top) had one more competence Knowledge and understanding of the subject area and understanding of the profession in common, and two other groups graduates and employers had the same competence rated as sixth from the top. Employers and graduates instead had another competence - Determination and perseverance in the tasks given and responsibilities taken in their respective lists of the top five competences. This skill probably shows itself to be needed more after students start working. Academics, students, employers and graduates were also unanimous with the least important competences: Ability to show awareness of equal opportunities and gender issues Ability to communicate with non-experts of one s field These competences were given the lowest rates by all 4 groups. Then the groups started to diverge: students and graduates were the closest to each other with only one competence difference. If we take graduates as a basis for comparison, each group then had one less important competence of its own: The least important competences graduate s employe rs acade mics studen ts Mean and SD Ability to show awareness of equal opportunities and gender issues 2.94 (0.05) 2.98 (0.09) (0.1) Ability to communicate with non-experts of one s field Were rated the lowest by all 4 groups (0.06) 2.90 (0.04) 2.76 (0.07) 3

4 Appreciation of and respect for diversity and multiculturality 3.15 (0.05) 3.20 ( (0.06) 3.20 Ability to work in an international context 3.07 (0.1) 3.12 (0.06) 3.19 (0.07) 3.19 (0.06) Commitment to safety Were rated the lowest by graduates 3.16 (0.1) 3.35 (0.09) 3.17 (0.10) 3.13 (0.1) Spirit of enterprise, ability to take initiative employers and academics 3.17 (0.05) 3.18 (0.02) 3.11 (0.10) 3.26 (0.04) Ability to act with social responsibility and civic awareness - students Ability to communicate in a second language employers 3.18 (0.09) 3.21 (0.09) (0.04) (0.1) Ability to motivate people and move toward common goals academics Were rated lowest either by just one, or two groups 3.26 (0.07) 3.30 (0.07) 3.13 (0.03) 3.30 Table 1: least important generic competencies Based on the data from the table we can conclude the following: 1. No wonder commitment to safety is rated low; there is lack of culture of safety at jobs in general, although it is slowly entering Georgian market. While employers rate it the highest 13 th from the bottom, students rated it the 3d, graduates 5 th and academics - 6 th. The gap here among employers and the rest of three groups is quite large. 2. Ability to motivate people and move toward common goals is least important for academics (rated 4 th from the bottom), then to employers (rated 10 th from the bottom) and then less important for grads (rated 11 th from the bottom) than for students (rated 14 th from the bottom), although the scores are close enough. So, comparing these 4 groups show that students rate this ability the highest among all the groups, which is surprising. Apparently, this competency should be less interesting for students and the most interesting for employers, but employer s rate it lower than graduates and students. 3. Ability to communicate in a second language is the least important for Georgian employers (rate 4 from the bottom) and it is probably because there is not much of such context in Georgia, while for graduates, professors and students it is equally important (rate 8 from the bottom). And indeed, ability to work in an international context is also rated very low by all 4 groups, although, surprisingly enough it received the highest rating from academics (7th from the bottom) probably because they foresee such environment in the nearest future to come? It is especially interesting because competence spirit of enterprise received the 4

5 lowest rating from academics and this is understandable it is not important skill for professors at all. 4. Ratings for Spirit of enterprise, ability to take initiative yield interesting results it is scored the lowest by professors, which is not surprising but followed by employers (rated 5 th from the bottom), which is very surprising, also it is rated the highest by students (12 th from the bottom). Employers might not like their employees to take initiatives, but they should definitely appreciate spirit of enterprise! The largest gap among the 4 groups studied is with the competence Ability to undertake research at an appropriate level. While academics rate it the highest, 10 th from the top; all 3 other groups rate it relatively lower: 19 th for students, 20 th for graduates and 26 th for employers. Correlations among the 4 groups surveyed showed quite high agreement, the smallest association is found between academics and students 0.88 and the highest association is found between employers and graduates 0.97, however, all correlation coefficient values are very high and again show that the 4 groups studied are unanimous in rating importance of generic competences from the list. Academics Employers Students Graduates Academics 1,000 Employers 0,902 1,000 Students 0,875 0,912 1,000 Graduates 0,939 0,966 0,954 1,000 Table 2: correlations by importance of generic competencies Achievement: All 4 groups think that competences Ability to work autonomously and Knowledge and understanding of the subject area and understanding of the profession are achieved well, these are among the top five on the list of achievements. Three groups but students equally rate Ability to communicate both orally and through the written word in native language as the top achieved capacity. Students rate achievement of this ability as 8 th from the top - 7 steps lower, this means universities are not good enough in development of this competence. Three groups but graduates consider Ability to apply knowledge in practical situations and Capacity to learn and stay up-to-date with learning among the top five achieved ones; however, graduates have the latter capacity just one step behind (6 th from the top). Graduates have brought into the list of top five achievements one unique capacity: Ability to work in a team (4 th from the top) - 7 th from the top for students, 8 th for - employers, 12 th for professors. Graduates and students have one more capacity in common: Determination and perseverance in the tasks given and responsibilities taken - respectively 3d and 2nd from the top, it is rated 7 th for professors and 9 th for employers. 5

6 All 4 groups think that competences Ability to communicate with non-experts of one s field and Ability to show awareness of equal opportunities and gender issues are badly achieved, these are among the bottom five competences on the list of achievements. Correlations among 4 groups showed lower associations than in case of importance scores: the highest correlation coefficient value is between graduates and employers 0.95 and the lowest 0.84 between students and academics. Academics Employers Students Graduates Academics 1,0000 Employers 0,9455 1,0000 Students 0,8403 0,8936 1,0000 Graduates 0,9161 0,9501 0,9433 1,0000 Table 3: correlations by achievement of generic competencies The largest gap between importance and achievement of competences is with the competence Ability to apply knowledge in practical situations. While all four groups rate this competence as the top in terms of importance, it gets scores in range of 3 to 7 in terms of achievement. Another gap is found with the competence - Ability to make reasoned decisions all four groups rates it 7 th and 8 th in terms of importance and 10 th to 13 th (by academics) in terms of achievement. One more competence Capacity to generate new ideas (creativity) is rated as important to the extent of 16 through 19 th and showed achievement with ratings of st. For employers the largest gap was found with two competences: Ability to apply knowledge in practical situations and Ability to plan and manage time; graduates add Ability to work in a team and students share the large gap with all these three competences. To sum up, the difference between importance and achievement of generic competences is the following: the range for importance is and the range of achievement is Comparing European and Georgian results: Based on rankings, three generic competencies are equally important for Georgian and European respondents. These competences are: Ability for abstract thinking, analysis and synthesis Ability to apply knowledge in practical situations Knowledge and understanding of the subject area and understanding of the profession Georgian and European experts also have similar lists of the least important competencies: Appreciation of and respect for diversity and multiculturality Ability to work in an international context Ability to communicate with non-experts of one s field Commitment to safety Ability to show awareness of equal opportunities and gender issues There is one competence that is highly important for Europeans - 3 and 4 th range; and less important for Georgians 8-14 ranges: 14. Ability to identify, pose and resolve problems 6

7 Correlations among the four groups of experts were quite high in European as well as in Georgian sample that shows enough cohesiveness among these four groups in views about competences. However, there were slight differences. In European group, correlations among these groups ranged from 0.92 to 0.97, except correlation between professors and employers In Georgian group, correlations were in the range of 0.92 and a relatively low correlation was between professors and students The following tendency can be traced: in Europe, professors diverge from employers in opinions regarding importance of competences and level of achievement, while In Georgia, they diverge from students. The latter might be result of soviet legacy of professor-centered approach versus student-centered approach, where professors are the sole proprietors of knowledge and students are considered as non-mature adolescents, need to be taken by hand and lead to specific goals. In this approach, therefore, the goals and the ways to reach them are set and designed by professors. If this is the case, then, probably, professors and students have different opinions on many aspects of teaching and learning, including importance of competences and the level of their achievement (Telia, Niauri, 2004). Subject specific competences: Georgian list of 25 subject-specific competences for education sciences and teacher education was developed on the basis of surveying 130 stakeholders. Group of professors (30) consisted of academics from Tbilisi State University and Ilia Chavchavadze University; students (40) from the same universities representing the first and the second cycles of education were involved in the students group, graduates 6 (30) of the same universities who are currently employed in education sector, were included in the graduates group and the group of employers (30) consisted of directors of daycare centers, elementary, primary and secondary schools, as well as representatives of non governmental organizations working in the education sphere. The mean scores given to importance of competences are quite close to each other, minimal mean score was 3.00, maximal mean score was m 3.76 and SD varied from 0.02 to Mean scores of the top five competences ranged from 3.76 to 3.70; the difference , and the overall range 0.76 are very small. According to the level of achievement, there is not a large difference among competences; the scores range from 1.96 to 3.23, the range here is 1.27 and larger by 0.51 than in case of importance of competences. The distribution of answers here is also slightly larger, SD = this means that those studied have more diverge opinions about how well the competences are taught at our universities than about how important these competences are. Also, according to respondents, level of achievement of competencies is lower than their importance. When importance of competencies ranges from scores 3 and 4, which means that all competences are almost equally important and even very important, according to respondents, the educational programs of higher education institutions are not able to develop these competences to the desired level. Scores here range from 2 to 3, meaning that the experts make one step backwards in assessing achievement of 6 3 years after graduation was requirement to qualify as a graduate 7

8 competencies. Only one competence, based to experts, is developed relatively better, with the mean score of 3.23: knowledge of the subject to be taught. If we consider traditional higher education system in Georgia (as one of the Soviet republics) and its approach to occupation in general and occupation of a teacher specifically, the result will not look surprising. Traditionally, the main thing was to provide students with knowledge and not with skills. Regarding an occupation of a teacher, he/she should very well know the subject to be taught and this is proved by the survey results, the smallest gap between importance and level of achievement 0.37 is with this very competence. The largest gap between importance and achievement scores - is shown by two competences: Ability to understand trends in education and be able to recognize the potential implications Ability to improve the teaching and learning environment According to the importance and level of achievement, there is a large gap in two more competences from the list of top five: Ability to create a climate conducive to learning Ability to recognize and respond to the diversity of learners and the complexities of the learning process According to those surveyed, these four competences are very important, all of them are in the top five of the list, but, are not worked out correspondingly. From the top five important competences, only one competence - Commitment to learners progress and achievement - is relatively better developed - the gap is Overall, all top five competences are less developed by our higher education institutions. Thus, important information for our educators - they should improve development of these competences. From the list of bottom five competences, competence 14 stands alone: Ability to lead or coordinate multidisciplinary educational team is the gap The data on importance and achievement of 5 top and 5 bottom competences are provided in the table below: Subject specific competencies importance achievement 5 most important competences Ability to recognize and respond to the diversity of learners and the complexities of the learning process Ability to create a climate conducive to learning 3.76 (0.07) (0.16) 2.66 (0.16) 8

9 Ability to understand trends in education and be able to recognize the potential implications Commitment to learners progress and achievement Ability to improve the teaching and learning environment (0.14) 2.76 (0.13) 2.56 (0.14) 5 least important competences Ability to design and implement varied strategies to evaluate learning based on specific criteria Awareness of the different roles of participants in the learning process Ability to identify potential connections between aspects of educational theory and educational policies and contexts Awareness of the different contexts in which learning can take place Ability to lead or coordinate multidisciplinary educational team 3.26 (0.14) 3.23 (0.14) 3.13 (0.10) 3.06 (0.13) 3.00 (0.10) 2.46 (0.12) 2.60 (0.13) 2.26 (0.15) 2.30 (0.14) 1.96 (0.15) Table 4: subject specific competencies by their importance and level of achievement It is interesting to know, that data from professors and employers are very close to each other; the largest gap between importance and achievement of competencies was shown by students, while the smallest gap was shown by graduates. Probably, after graduation and starting to work, the graduates revise their position and think that learned and gained more, than they assumed so before, as students 7. These data are very important for universities as the students think they are not given chance to correspondingly develop subject specific competencies necessary to make a good professional in education sciences and teacher education. Correlations according to scoring importance of subject-specific competencies are still high among all four groups, but lower than in case of generic competences. The highest association here is between graduates and students, and the lowest association is between employers and students. Academics Employers Students Graduates Academics 1 Employers 0, Students 0, , Graduates 0, , , Table 5: correlations by importance of subject specific competencies 7 Those who graduated 3 years ago were included in the category of alumni; therefore, there is not much difference in teaching provided by the higher institution programs between current students and alumni. 9

10 Correlations according to scoring achievement of subject specific competences is again lower than those of generic ones, the highest correlation is between employers and academics 0.86 and the lowest correlation is between graduates and students. Academics Employers Students Graduates Academics 1 Employers 0, Students 0, , Graduates 0, , , Table 6: correlations by achievement of subject specific competencies To compare scores for importance and achievement of subject specific competencies, the range for importance is and the range for achievement is Finally, comparing scores for generic and subject specific competencies yields the following picture: competencies importance achievement generic Subject specific Table 7: score ranges for generic and subject specific competencies The lists of generic and subject specific competences for Georgia are very similar to their European counterparts. The Georgian rankings are also quite close to those gained from the European stakeholders. In general, stakeholders in Georgia and Europe emphasize similar competences, although there is one global difference most probably caused by the traditional European and traditional soviet understanding. Soviet system put larger accent on professor-centeredness, while European system put larger accent on student-centeredness. Results of our survey support the finding in the following: In general, employers show strange trends: the competences that should be important for employers are not as important. Three competences, in my opinion, should be perceived as very important market requirements - Awareness of the different roles of participants in the learning process; Ability to identify potential connections between aspects of educational theory and educational policies and contexts; Awareness of the different contexts in which learning can take place but are not considered as such by stakeholders, including employers and according to the same stakeholders are not developed enough by university degrees. Stakeholders recognize importance of 4 other competencies - Ability to 10

11 improve the teaching and learning environment; Commitment to learners progress and achievement; Ability to understand trends in education and be able to recognize the potential implications; Ability to create a climate conducive to learning however, they think these are not developed enough by universities All surveyed groups show that ability to work autonomously is well developed by university degrees; however, reality does not fully comply with these results. Another study of employers in the frames of the dissertation thesis showed that employers are mainly concerned with employees having possessed superficial skills, like to behave one and to realize orders (Nadareishvili, 2009). Employers say that their employees lack Ability to apply knowledge in practical situations and Ability to plan and manage time, which should be an alert for educators. It is very important, that competencies gained by students fit demands of labor market. This is the reason to ask opinion of employers and alumni they provide information on demands and expectations of labor market, especially that, currently, there is no other research data available for Georgian labor market. Learning outcomes in the forms of competences worked out by students should be based on requirements of both: labor market and the discipline (Gonzalez, Wagenaar 2008; Gonzalez, Wagenaar, 2003); Gonzalez, Wagenaar, 2005). The survey provided very interesting results and the ordered lists of generic and subject specific competencies in ten subject areas are an asset to higher education institutions in Georgia. Tbilisi State University, for example, decided to move to TUNING approach step by step on the level of programs only in ten subject areas and on the level of syllabi in all curricula (Glonti, Chitashvili, 2006; Glonti, Bakradze, 2006; Javakhisvhili, Aslanishvii, Glonti, 2007). From now on, while designing or redesigning any educational program, its authors can use these lists of competences. However, the survey results, supported by results of other studies (both: quantitative and qualitative) show that there is a lack of connections between employers and academics. According to 2008 EPPM study university curricula do not reflect market demands partly because universities do not have effective dialogue with employers and partly because employers are not able to articulate their own needs (EPPM, 2008). References: 1. Beneitone, P. Identifying Key Competences: The Outcomes of the Stakeholders Survey. Presentation made at the working seminar in Groningen, Beneitone, P., Esquenti, C., Gonzalez, J., Maleta, M., Siufi, G., Wagenaar, R. (eds.) Tuning America Latina. Reflections on and outlook for higher education in Latin AMERICA. (2007). University of Deusto, University of Groningen 3. EPPM Study of Bologna Process Requirements, Tbilisi, 2009 (in Georgian) 4. K. Glonti Higher Education and Labor Market in Georgia Tbilisi 5. Glonti, L., Chitashvili, M. (2006). The Challenge of Bologna: The Nuts and Bolts of Higher Education reform in Georgia. In V. Tomusk (ed.) Creating the European Area of Higher Education: Voices form the Periphery. Springer, Glonti, L. (2007). The Way to Europe: Problems and Perspectives. In: Proceedings of 4 th International Silk Road Symposium New Trends in Higher Education, Tbilisi, pp

12 7. Gonzalez, J. and Wagenaar R (eds) (2008) Universities contribution to the Bologna Process Tuning Education Structures in Europe. General brochure. University of Deusto. 8. Gonzalez, J. and Wagenaar R (eds) (2003) Tuning educational Structures in Europe. Final Report. Phase I University of Deusto, University of Groningen 9. Gonzalez, J. and Wagenaar R (eds) (2005) Tuning educational Structures in Europe. Final Report. Phase II University of Deusto, University of Groningen 10. Hotlge, K. (2005). Governance in Transition: What Makes Georgia s Higher Education System so Corrupt? Kassel University Press 11. Javakhishvili, N., Aslanishvili, T., Glonti, L., (2007). Higher Education Reform at TSU. In: Proceedings of 4 th International Silk Road Symposium New Trends in Higher Education, Tbilisi, pp Kachkachishvili, I. (2001). Study of Private Higher Educational Institutions in Georgia Tbilisi (in Georgian) 13. Machabeli I.(2007) Implementation of the Bologna Principles in the Georgian Higher Education System In: Proceedings of 4 th International Silk Road Symposium New Trends in Higher Education, Tbilisi, pp Glonti, Bakradze Main Directions and Action Plan for Implementing the Bologna Process in Georgia Until 2010 (2006) last retrieved, Maisuradze, L. (2007) Looking beyond 2010 Perspectives for Georgia in the European Higher Education Area. In: Proceedings of 4 th International Silk Road Symposium New Trends in Higher Education, Tbilisi, pp Nadareishvili, V. (2009) Strategic Planning for Postsecondary Education: Aligning the Education Sector and the Labor Market. Unpublished dissertation thesis for Rand Corporation 17. Rositashvili, Ketevan: Problems of Corruption in Higher Education System of Georgia. Tbilisi TRACCC Telia, R. Niauri, N. Assessment of the Ongoing Reforms in the National System of Higher Education in the Movement of The Lisbon Convention. Final Report, EPPM,