MID-TERM EVALUATION REPORT NICHE PROJECT 102, March, 2014 Introduction: The NICHE/GHA/ 102 started in 2012; it is to run till 2015. (cf.?) The Year 2013 serves as the mid-term of the project. A mid-term review of the project was part of the initial plan of the project. (cf.?) In keeping with the work plan, an evaluation team was appointed to perform a mid-term review of the project so far. The NICHE/GHA/102 project aims to provide quality health care services meeting the needs of the three Northern Regions of Ghana and help meet MDGs 4&5 by strengthening the capacity of the NTCs/MTCs to train sufficient number of nurses and midwifes at Diploma level. (cf. Inception report) The project envisages to do this through strengthening the capacity of medical post-secondary education and training in Northern Ghana to deliver quality, gender sensitive, education and training, and to do relevant research, that meet changing needs and demands on the (health) labor market. The central objective of the project in practical terms is: To upscale and upgrade the training of nurses and midwives in North Ghana This the projects proposes to achieve through three main specific ways, namely: improve the management of the colleges, help run cost-effective programs and the use of innovative teaching and learning methodologies all aimed at achieving the overall objectives of the project. The vehicles through which the project will carry out a well selected and carefully planned set of activities are all outlined. (cf. Inception report) Flowing from the model of implementation are four major activities with three main themes in view. They are: Management of Colleges, Curriculum innovation and course design, implementation of curriculum, teaching and learning and facilities and investments all surrounds the three thematic of gender, labour market and quality assurance. Through study visits, needs assessment and planning, seminars, meetings and workshops the first half of the project life has been rolled out. A mid-term review aimed at making an impact assessment of the project with a view to making recommendations for further improvement where necessary is partly in line with the initial plan of the project. A team was therefore put in place to undertake the review. Membership of the Team: A representative of the Faculty of Allied Health Sciences of the Kwame Nkrumah University of Science and Technology together with a professional Midwife, were members of the evaluation team. Terms of Reference: (i) (ii) (iii) To review the relevant documents so that a conceptual framework for the evaluation could be formulated. Make site visits to assess the impact of the project through individual and group discussions. Submit a report at the end of the review using 13 days in total.
The team was charged with the responsibility of assessing the following: The extent to which the planned outputs have been implemented Whether outcomes are visible in the day to day practices of the colleges Staff and students benefit from the project Whether outcomes and outputs contribute to the achievement of the overall project objectives The objective of the team in the evaluation process was: 1. To assess the successes and failures so that the areas for improvement can be identified and corrections made. 2. Examine responsible use of resources and their justification from the outcomes with suggestions for better utilization of same if needed. 3. To explore the limits of the joint cooperation for better understanding of all participants 4. Make suggestions for better implementation of the project within the remaining time frame Ultimately the team aimed at exploring the state of affairs of the practical implementation of the project on the ground to see if the planned activities have been adequately implemented, whether achievable goals have been accomplished and therefore visible at the implementing sites, namely the Colleges. Methodology: The mid-term evaluation aimed at assessing the impact of the NICHE/GHA/102 project in six (6) Nurses and Midwifery Training Colleges in the Northern Regions of Ghana and was conducted through site visits, face to face discussion and administration of a questionnaire to elicit effects of the project so far. The questionnaire was designed, after a conceptual framework had been formulated, to assess interviewees rating of the outputs of the project and as well as their views of the outcomes at study point on a scale of 1 to 5. At the initial stage, a meeting of all the key people to be interviewed was held to discuss the evaluation tool, namely the questionnaire and an opportunity to ask questions and seek clarification was offered in each College. After this a one on one face to face discussion took place where interviewees were requested to scale their opinions of the outputs as well as the outcomes of the key areas for evaluation, namely: Gender, Management of the Colleges, Curriculum development and learning materials, Skills Laboratory, Capacity development, Management of the project in general, Trainer of Trainers workshops, Transfer and dissemination of knowledge and the Visibility of the project activities. Interviewees were requested to discuss their views openly and to offer reasons and explanations for their choices in the rating of the key themes. At the end of the discussion of each theme they could tick their rating of the output as well as the outcome of the theme under discussion. At the end of the discussion of the key themes, interviewees were given the chance to choose any theme that in their view carried the greatest impression of the project and rate that as well. In other words interviewees had to give their general impression of the project with a specific theme in view and
rate that as the single most important idea carrying their view of the project, also on a scale of one to five. In each College a total five to 8 people were interviewed. After the interviews, we looked at their facilities such as the skills lab or whatever other facility they considered important enough to be looked at as resulting from the project. Our visits always ended with this cursory look at facilities and a chat with the staff about the project in general. Two sets of data were gathered: a set of qualitative data and answers from the questionnaire that was administered to generate quantitative data for analysis. Data Analysis: Responses on the questionnaire (data collection form) from each respondent were entered into the Census and Survey Processing System version 4.1 (CSPro 4.1) which runs only on Microsoft Windows operating systems. CSPro is a public domain software package developed by the United States Census Bureau and ICF International for data entry, editing, tabulation and dissemination of census and survey data. The final data capturing responses from all respondents was exported from CSPro to Microsoft excel and then onwards to Stata as a single file. Statistical analyses were performed using Stata (Stata version 11). Tabulations showing frequencies and percentages for the different level of responses were done for each question theme. Responses were on a scale of 1 to 5 where 1 = very poor, 2 = poor, 3 = good, 4 = very good and 5 = excellent. A descriptive analysis was done and the results are here presented. Results and Findings: A total of 34 people were interviewed covering six colleges. The distribution of the interviewees was done according to colleges as follows: Name of college Number of people interviewed Bawku NTC 5 Bolga MTS 5 Bolga NTC 5 Jirapa NTC 5 Jirapa MTC 5 Project Coordinator 1 Tamale NMTC 8
Respondents were requested to express their views on the major themes of Gender, Management of Colleges, Curriculum Development and Learning Materials, Skills Lab, Capacity Building, Project Management in General, Trainer of Trainers Workshops, Transfer and Dissemination of Knowledge as well as the Visibility of the project as a whole. The following picture emerged: Gender - Output: n=26. 42.31% (n=11) expressed that the output was good whilst 57.69% (n=15) expressed the view that output was very good. Total 26 = 100% Gender Outcome: n=26. 38.46% (n=10) said the outcome was good, 57.69% (n=15) said it was very good and 3.85% (n=1) said it was excellent totaling 100%. With regard to the management of the colleges, 26 people expressed views about the theme, all were tutors or principals; students were excluded. They scored the output as follows: 3.70% (n=1) said output was poor, 18.52% (n=5) scored output as good and 77.78% (n=21) thought it was very good. When it came to outcome, 33.33% (n=9) said it was good whilst 66.67% (n=18) thought it was very good, making 100% (n=27). A total of 28 people expressed views about curriculum development and learning materials. 10.71% (n=3) thought the output was poor, 42.86% (n=12) thought it was good whilst 46.43% (n=13) thought it was very good. When it came outcome 17.86% (n=5) expressed the view that the outcome was poor, 46.43% (n=13) thought it was good and 35.71% (n=10) thought it was very good; total of 100% where n=28. Regarding skills lab an enthusiastic number of 27 scored their opinions about the subject. As far as output was concerned 3.70% (n=1) said it was poor, 33.33% (n=9) thought it was good, 55.56% (n=15) said it was very good whist 7.41% (n=2) said it was excellent making a total of 100%. When it came to outcome, 7.41% (n=2) said it was poor, 48.15% (n=13) said it was good whilst 44.44% (n=12) said it was very good making a total of 100%. Another theme that attracted a divergent opinion was capacity development. The opinions ranged from very poor to very good. The opinions were as follows: 8.33% (n=2) thought the output was very poor, 33.33% where n=8 thought it was poor, 50.00% (n=12) thought the output was good and 8.33% (n=2) said it was very good. As far as outcomes were concerned the following were representative of the views expressed: 8.70% (n=2) thought the outcome was very poor, 39.13% (n=9) said the outcome was poor and 52.17% (n=12) said it was good. In all a total of 24 and 23 people expressed their opinions on the output and outcomes respectively, all of them were tutors; none of them was a student. Project management in general had 17 people commenting on the outputs and outcomes, these were all tutors who felt more closely related to the project enough to express some opinion. These ranged from good to excellent. 17.65% made up of 3 people thought output was good, 70.59% (n=12) thought it was very good whilst 11.76% (n=2) thought it was excellent. As far as outcomes were concerned, 41.18% (n=7) said the outcome was good whilst 58.82% (n=10) thought it was very good, making a total of 100% where n=17.
What people had learnt from the Trainer of Trainers workshops attracted an equally enthusiastic response from almost all the respondents. In all 28 people responded with 28.57% (n=8) responding that the output was good, 67.86% (n=19) said it was very good and 3.57% (n=1) saying that the output was excellent. Regarding the outcome, 10.71% (n=3) responded that it was poor, 32.14% (n=9) thought it was good and 57.14% (n=16) thought it was very good. The disparity in the view of the output and outcome is noteworthy. The students were excluded. How people assessed their transfer and dissemination knowledge was clearly explained to mean their own activity and the outcome the impact they felt they had been made. Again total number of 27 people responded, none of them was a student. One person making 3.70% responded that in their opinion output was poor in this area, 25.93% comprising 7 persons thought output was good and 70.37% where n=19 felt output was very good. As far as the outcome was concerned 33.33% (n=9) responded that outcome was good. Again 59.26% (n=16) responded that the outcome was very good and 7.41% (n=2) responded that the outcome was excellent. Opportunity was given to everybody to express their views about the visibility of the project activities. A total of 33 people responded and the results are as follows: In terms of output, 3.03 (n=1) responded that output was poor, 33.33% (n=11) thought it was good, 63.64% (n=21) thought it was very good. Regarding the outcome, meaning the visibility of the effects of the project in terms of changes, learning etc. of the project, 3.03% (n=1) thought it was poor, 33.33% (n=11) thought it was good and 63.64% (n=21) thought it was very good. Students were excluded. We requested the respondents to formulate their own category that in their view captured their single most important aspect of the project and to rate it as with previous themes. The following picture emerged: Overall, all thirty-three respondents gave their rating with 3.03% (n=1) thought the=overall project output was poor; 30.30% (n=10) thought it was good whilst 66.67% (n=22) rated it as very good. They rated the outcome as follows: 6.06% (n=2) rated it as poor, 36.36% (n=12) rated it as good, 54% (n=18) rated it as very good whilst 3.03% (n=1) rated it as excellent. In this same regard, the table below shows what categories they believed were the single most important aspect of the project.
Themes/Categories No. of persons Activating teaching and learning method 1 Almost all tutors have attended a Niche workshop 1 Awareness creation 1 Capacity building in terms of workshop seminars 2 Creation of awareness about gender issues 1 Human capacity development 1 Human resource development 1 Knowledge 1 Project Management Information 1 Skills lab and capacity building 1 So far so good but capacity building is problem area 1 Student centered learning 1 Teaching and learning 1 The knowledge acquisition is very good and I have learnt a lot from the program 1 There is institutional improvement in terms of staffing and teaching methodology 1 Workshops organized 1 Exposure 1 Classroom learning and gender 1 Knowledge acquisition 1 Exposure has been very good 1 The themes above have been mentioned as important effects of the project presented in an unfiltered manner.
Themes Output and outcome Rating (%) Total (%) Very poor poor good Very good Excellent Gender Output 0 0 42.31 (n=11) 57.69 (n=15) 0 100 (n=26) Outcome 0 0 38.46 (n=11) 57.69 (n=15) 3.85 (n=1) 100 (n=26) Management of colleges Curriculum Development Output 0 3.70 (n=1) 18.25 (n=5) 77.78 (n=21) 0 100 (n=26) Outcome 0 0 33.33 (n=9) 66.67 (n=18) 0 100 (n=27) Output 0 10.72 (n=3) 42.68 (n=12) 46.43 (n=13) 0 100 (n=28) Outcome 0 17.86 (n=5) 46.43 (n=13) 35.71 (n=10) 0 100 (n=28) Skills lab Output 0 3.70 (n=1) 33.33 (n=9) 55.56 (n=15) 7.41 (n=2) 100 (n=27) Outcome 0 7.41 (n=2) 48.15 (n=13) 44.44 (n=12) 0 100 (n=27) Capacity development Project management in general Trainer of trainers Transfer and dissemination of knowledge Output 8.33 (n=2) 33.33 (n=8) 50.00 (n=12 8.33 (n=2) 0 100 (n=24) Outcome 8.70 (n=2) 39.13 (n=9) 52.17 (n=12) 0 0 100 (n=23) Output 0 0 17.65 (n=3) 70.59 (n=12) 11.76 (n=2) 100 (n=17) Outcome 0 0 41.18 (n=7) 58.82 (n=10) 0 100 (n=17) Output 0 0 28.57 (n=8) 67.86 (n=19) 3.57 (n=1) 100 (n=28) Outcome 0 10.71 (n=3) 32.14 (n=9) 57.14 (n=16) 0 100 (n=28) Output 0 3.70 (n=1) 25.93 (n=7) 70.37 (n=19) 0 100 (n=27) Outcome 0 0 33.33 (n=9) 59.26 (n=16) 7.41 (n=2) 100 (n=27) Visibility Output 0 3.03 (n=1) 33.33 (n=11) 63.64 (n=21) 0 100 (n=33) Outcome 0 3.03 (n=1) 33.33 (n=11) 63.64 (n=21) 0 100 (n=33) Most important aspect of project Output 0 3.03 (n=1) 30.30 (n=10) 66.67 (n=22) 0 100 (n=33) Outcome 0 6.06 (n=2) 36.36 (n=12) 54.00 (n=18) 3.03 (n=1) 100 (n=33)
Observations and Recommendations: Our observations were in two categories. The first category was the non-tangible effects of the project whilst the second are the tangibles. The non-tangible but observable outcome of the project can also be categorized into two. Here the observable but non-verbalized effects as well as the described effects such as attitudes, behavioral changes that have taken place as a result of the project. 1. The non-tangibles: a. Management of the Colleges Many of the respondents reported that there was an observable change in the manner in which colleges are managed and they attributed this to the management courses that the principals attended. Some mentioned that the management had become more transparent and inclusive and democratic. Some mentioned that management had become more resourceful and are seeking new ways of sourcing for funds outside the traditional ways of waiting for government funds that were not forthcoming. This had created an atmosphere of cooperation and tutors had become more enthusiastic about their work. b. Curriculum development and learning materials Some of the respondents reported that even though the curriculum is set, as a result of new insights brought about by the project, innovative ways of handling the same curriculum has been found and tutors are trying new things. PowerPoint projectors are commonly in use in many of the colleges and both students and tutors are encouraged to participate. Tutors mentioned student-centered learning as an innovation that has become commonplace in the colleges as a result of tutors participation in ToT workshops. c. Skills Lab In the course of our visit to the colleges it was reported that new arrangements had taken place in skills labs of some of the colleges due the participation of the focal persons who had learned new things from their participation in some workshops on skills labs. They also reported that the little manikins and other things are better managed. Also students participation and interest in the use of the skills labs was on the increase and more learning was taking place through the use of the skills labs now. d. Capacity building This is an area where many reported had seen some improvement. Attention was drawn to the fact that a lot of training has taken place through participation in trainer of trainers workshops, through debriefing sessions back at the colleges when participants in the workshops have come back to train their colleagues and some exposure as a result of visits to the Netherlands affording them opportunity to learn new things. The local capacity is being built for better realization of the goals and the fulfillment of the mandate of the colleges. e. Gender mainstreaming Of the themes discussed, the one that had generated the most non-tangible but great impact was gender. In all the colleges respondents mentioned the changes that had taken place in the attitudes in the colleges as a result of the gender sensitization as most effective. Examples upon examples of changes in attitudes, female participation, the new awareness etc. as a result of the gender mainstreaming were given in all the colleges.
2. Tangibles: a. Structures As result of the new changes that had taken place in some of the colleges following adoption of the student-centered problem based learning they needed to build more classrooms for the effective implementation of the innovative approaches that had been introduced. Consequently new classrooms had been built in some colleges to cater for more student-teacher contact at the lectures. b. Expansion Some more space has been provided for by way of expansion of skills labs in some colleges to prepare for the arrival of manikins and other implements that are expected to arrive. In others, the library had seen a face-lift. c. Beds, books etc. some colleges had received beds, some improvements in the Library environment such as books etc. Many of the Colleges had received laptop computers as well as pointer projects for use in classrooms. Recommendations: 1. Need for better understanding of roles: Reportedly there is the need for a better understanding of the roles of the key people in the project. The impression is created that the Ghanaian is not trusted to purchase local things, even made in Ghana goods, such as locally manufactured beds. It is recommended that Ghanaian Project Coordinator be made responsible for purchases of locally acquired goods whilst those goods purchased in Holland or elsewhere in Europe are made the responsibility of the Dutch counterpart. 2. Closely associated with this is the procedure for obtaining exemption of goods imported from overseas. It recommended that the University of Development Studies should get more and early information so that they can take their responsibility for the getting of exemption permits to facilitate the clearing of goods from the harbor. 3. Proper follow-up at the Colleges: It is clear that there is the need for strengthening the role of focal persons and a support system be put in place so that there will be a further follow-up of all initiatives so that the ideas do not die after the initial enthusiasm wanes. 4. Capacity Building (Masters and PhD): There is the need to re-examine how the program is going to be implemented. 5. Development of Learning Materials: There seems to be a disconnection between the initial first steps and the necessary follow-up leading to the realization of the goals, namely the arrival of the developed materials. 6. Curriculum Development: the method of Problem Based Learning is not completely understood by the Colleges. More information and understanding will improve the sustainability of this innovation. Conclusion: The objectives of the project are achievable and on course. The number of planned activities in the year is ambitious, but they have all been implemented very impressively and the outcomes are laudable and visible in most cases. The general atmosphere in the colleges is one of enthusiasm and this is largely due to the innovations introduced through the project activities such as
new methods of teaching, introduction of teaching aids such as power point projectors, library books, etc. Teaching and learning is taking place in an atmosphere that is conducive for such activities and this is reflected in the attitudes of teachers and students. There are 3 areas where attention is needed for better implementation. The manikins and other things for the skills labs should be delivered as soon as possible so the new enthusiasm does not wane. The selected tutors for Masters Degrees should be helped to achieve their goals of getting further training before they get disappointed or their hopes dashed. And the teaching aids that they are looking forward to should be developed quickly for use within the nearest possible time frame. Over all, the achievements of the project are admirable and it should continue with the suggested adjustments.