M-Tool SET CONTENTS 1 / 35

Transcription

1 CONTENTS Sectin I NA Cntext t Develp Mnitring Visits t LLP Partnerships 2 Sectin II M-Tl Strategy fr Mnitring Educatinal Partnerships 4 Sectin II. A M-Tl Strategy - Methdlgy 4 Sectin II. B M-Tl Strategy - Tls 5 1. NA Mem 5 2. Letter t the institutin 6 3. Qualitative techniques fr cllecting data 7 4. Summary f the Interview Guide General interview Guide Guide fr analysis f mnitring data 31 Example f the analysis grid fr the interview 32 Example f the analysis grid fr bservatin NA Mnitring reprt Fllw-up reprt 35 1 / 35

2 Sectin I NA CONTEXT TO DEVELOP MONITORING VISITS TO LLP PARTNERSHIPS 1. Fllw the Guide fr Natinal Agencies (NA) established by the Eurpean Cmmissin (EC) Main tpics: 1.1. Eurpean definitin f cncepts in the Lifelng Learning Prgramme (LLP) LLP Glssary On-ging mnitring f prjects - NA will mnitr prject implementatin at any mment during the prject lifetime (in situ prject visits) Pre-defined criteria t select beneficiaries in a balanced spread: institutin size, institutin type, gegraphical criteria, grant level. 1 Mnitring: Mnitring is the regular bservatin and recrding f activities taking place in a prject r prgramme. It is a prcess f rutinely gathering infrmatin n all aspects f the prject. T mnitr is t check n hw prject activities are prgressing. It is bservatin; systematic and purpseful bservatin. Mnitring als invlves giving feedback abut the prgress f the prject t the dnrs, implementers and beneficiaries f the prject. Reprting enables the gathered infrmatin t be used in making decisins fr imprving prject perfrmance. in LLP Guide fr Applicants Fllw-up f perfrmance and realisatins f a beneficiary and prvisin f feedback and guidance in view f imprving the supprted activity and management f the grant as needed. in Guide fr Natinal Agencies implementing the LLP Mnitring (at prject level): The prcess invlves cntinuus and systematic cntrl f the prject s prgress. The intentin is manage and if necessary crrect any deviatin frm the peratinal bjectives and thus imprve the perfrmance. Every prject shuld be mnitred thrughut its duratin in rder t ensure its success. Mnitring cnsists f supervisin f activities, cmparisn with the wrk plan and using the infrmatin btained fr the imprvement f the prject. During the mnitring prcess disseminatin and explitatin activities must be carefully checked, verified and, if necessary - reriented and adapted. in LLP Guide fr Applicants Thematic Mnitring: Thematic Mnitring is a qualitative prcess put in place t increase the impact f the LLP prgramme. The main elements are: 1) Clustering f prjects int thematic grups t gain verview n specific cntents and utcmes; 2) Facilitating exchange f experience between prject actrs with a view t imprving quality and impact at individual prject level; 3) Facilitating the netwrking f prjects, practitiners and decisin makers with a view t the future rientatin f plitical pririty and strategy. in LLP Guide fr Applicants 2 / 35

3 2. Establish a natinal mnitring strategy accrding t the EC rientatins: 2.1. Structured apprach cnsidering, at natinal level, the bjectives and target grups f the LLP sectral prgramme (natinal criteria t select prjects fr mnitring) NA Purpse: Cllecting infrmatin n the quality in management f the LLP Sectrial Prgramme and the efficiency f the implementatin f the prject in the institutin / rganizatin and in the educatinal cmmunity NA Aims: T cllect infrmatin regarding the quality in management f the prjects. T supprt the beneficiaries in prject implementatin. T lead the beneficiaries t pssible slutins f the prblems detected in the implementatin. T be familiar with the real resurces (human, material and institutinal) applied t the prjects. T assess the prgress f the prjects. T disseminate examples f gd practices. T establish a clse relatin between the beneficiaries and the NA by recgnising their prject wrk Use f the mnitring results (NA Reprts, Institutin Fllw-up Reprts and NA staff experience): T enlarge and imprve the pedaggical and technical supprt given by the NA t beneficiaries and ptential prject makers in rder t enhance the partnerships quality. T identify success stries and examples f best practices useful fr valrisatin purpses (disseminatin and explitatin f results). T establish impact assessment indicatrs. 3 / 35

4 Sectin II. M-TOOL STRATEGY FOR MONITORING EDUCATIONAL PARTNERSHIPS Sectin II. A M-TOOL STRATEGY - METHODOLOGY WORK PHASE PROCEDURE TOOL / OUTCOME Preparatin f the field mnitring prcess Field wrk / Cllecting data Data analysis and interpretatin 1 Mem with infrmatin abut the mnitring strategy. NA Mem 2 Letter t the institutin Letter t the (At least ne week befre the visit) t infrm abut the aim and methdlgy f institutin the visit. 3 Mnitring visit in agreement with the principles f qualitative techniques fr cllecting data (interview and bservatin). Using the General Interview Guide during the visit and carrying ut direct bservatin f dcuments and premises. One day r half-day visit. 4 Guide fr analysis/interpretatin f mnitring data: Recrd the cllected data in written and rganize them subsequent t the visit. Emply a cmmn methd t analyse and t interpret the data (Cntent Analysis). General Interview Guide Guide fr analysis f mnitring data Reprting 5 NA Reprt t the institutin with feedback n the mnitring visit. NA Mnitring Reprt Fllw-up 6 Fllw-up Reprt n the implementatin f the NA recmmendatins. Fllw-up Reprt 4 / 35

5 Sectin II.B M-TOOL STRATEGY - TOOLS 1. NA MEMO TOPICS General selectin criteria f the institutin (regarding the NA Guide rientatins). Specific selectin criteria regarding a cncrete prject: Defining the purpse f the visit (due t difficulties detected, using the criterin visits t crdinatrs, t chse a specific thematic, etc). Selecting specific prject lcal activities and mbilities emphasized in the wrk plan / cntract (ne r several main activities; e.g. the mre recent, the nes that invlved mre persns, the ne with mre visibility, the nly nes implemented ), by using a principle f purpsive sampling. Basic Data abut the prject: Institutin data (identificatin, type, level, nature, cntacts); Prject basics (cde, type, title, cuntries); Prject cntent (cncrete aims, target-grups, thematic areas); Quality assessment (scre and verall cmments). Definitin f NA staff. Letter t the institutin. 5 / 35

6 2. LETTER TO THE INSTITUTION TOPICS Cmmunicating the bjectives, bth general (regarding the General Interview Guide) and specific (cncerning the specific selectin criteria related t the case study i.e. this cncrete prject). Infrming abut the methd f the visit, regarding the use f the grup interview. Defining the participants in the mnitring visit and their rle (institutin management, prject crdinatr, ther staff directly invlved, pupils/learners directly invlved). Request f dcumentatin fr bservatin regarding ne r several finalized activities f the wrk plan (lcal activities and agenda f the mbilities). Guidance t gather quantifiable data abut the level f participatin in activities (lcal, mbilities and disseminatin/valrisatin). Request fr a written cnfirmatin f the availability f the participants n the day f the visit (t ensure the presence f the relevant participants invlved in the prject activities and t have an agreement in advance n the agenda and cnditins f the visit). 6 / 35

7 3. QUALITATIVE TECHNIQUES FOR COLLECTING DATA TOOL FOR COLLECTION DATA Type f interview Purpse Strategy INTERVIEW Face-t-face verbal interchange by grup interviewing. Semi-structured (using a general interview guide in a flexible apprach): - Interview questins are planned but nt standardized in advance because specific questins will emerge during the interview; - There are n pre-set respnse categries. Grup interview: - Several peple simultaneusly (management, prject crdinatr, prject team, pupils/learners, ther participant staff); - Definitin f wh will be interviewed, why and hw many persns. Understanding the prject prgress. Gathering cumulative data with the grup situatin. Stimulating peple t make their views, mtives and reasns explicit. One-time interview (variable duratin). Use f the General Interview Guide: - Use the pre-set questins (Summary f the Interview Guide) t knw hw t ask and gather relevant infrmatin (related t the bjectives); - Flexible use and rder f questins, adapted t the mnitring bjectives and prject life cycle; - Make use f the Guiding Tpics t manage the answer and avid deviatins frm the bjectives. The rle f NA staff: interviewer as a facilitatr, wh supplies the questins and tpics fr answers and grup interactin. Managing the interview - In the NA: 7 / 35

8 TOOL FOR COLLECTION DATA INTERVIEW preparing and scheduling the interview; reading the applicatin frm, the cntract, reprts (if any) r ther dcuments related t the prject. - In the institutin: having the applicatin frm and the cntract available during the visit; beginning the interview with their stated bjectives; giving ral feed-back during the visit; clsing the interview with an verall view f the strng and weak pints, including recmmendatins; writing dwn ntes abut the participants' speech (during and after the interview). Data frm the grup interviews Written recrds f the NA staff TOOL FOR COLLECTION OF DATA Apprach Purpse Scpe OBSERVATION Qualitative apprach, nt using predetermined categries and classificatins. Observatin in a natural pen-ended way. Triangulatin between bservatinal data and the interviewees discurse t ensure different angles. Making wider links related t the prject implementatin. Direct bservatin f: - Institutin premises; - Space used fr prject activities and disseminatin; - Dcumentary data, like prject materials and (final) prducts. 8 / 35

9 TOOL FOR COLLECTION OF DATA OBSERVATION Observing and listening t the institutin s envirnment. Reading different types f dcuments: - f different authrship (teachers/trainers, pupils/learners); - and different availability (clsed, restricted r penly published). Definitin f what will be bserved and why, taking int accunt the bjectives f the Strategy mnitring visit. Categries fr describing and analysing the bservable data (descriptive and fcus categries). Key categries t analyse dcumentary data: credibility, representativeness and meaning (wh writes them, fr what purpse, wh read them, what is mitted, what is accmplished). Data frm the bservatin Written recrds f bservable data. 9 / 35

10 4. SUMMARY OF THE INTERVIEW GUIDE CATEGORIES QUESTIONS Hw did yu knw abut Cmenius/Grundtvig/Lenard Prgrammes? Hw did yu cme acrss the partners fr yur prject? Cnceptin Was there a prject team t begin the wrk? Hw did yu cllabrate within the partnership t cnceive the cmmn prject? When and hw did yu infrm the institutin abut the prject? Hw d yu cmmunicate amng partners? Hw d yu share the infrmatin with all partners? Cmmunicatin in the partnership What tls d yu use t cmmunicate? Which are the cmmunicatin languages used during the prject? Was there any linguistic preparatin needed? Hw did yu satisfy thse needs? Partnership Lcal activities Hw is the prject included in the institutin s wrk plan? Which prject activities are included in the curriculum f pupils/learners? Which activities have yu already carried ut? Presently, in what activity are yu wrking? What was the mst crwded/ppular prject activity? What changed in the partnership planned activities? Will yu be able t cmplete the planned activities? 10 / 35

11 CATEGORIES Mbilities QUESTIONS What kind f cmmn activities have yu already carried ut with yur partners? Which mbilities have yu already carried ut? What changed in yur mbility plan? Can yu cmplete the planned mbilities? What kind f preparatry activities were develped befre the mbility? Culd yu quantify the participating staff? Hw did yu chse staff fr mbility? Culd yu quantify the participant pupils/learners? Hw did yu chse pupils/learners fr mbility? Hw did yu twin pupils during their freign stay/ when hsting partners? M-Tl SET Hw did yu invlve yur pupils/learners in the implementatin f the activities during mbility and /r when hsting partners? Which was the cntributin given by the parents/families t the mbility activities? Which was the cntributin given by the Assciated Partners t the mbility activities? Which activities were develped during mbility and/r partner s hsting? Hw much time was spent in each kind f activity during mbility and/r partner s hsting? What kind f fllw-up activities were develped after mbility? Institutin s invlvement Hw many persns are presently invlved in the prject, including the prject team? Hw is the prject team frmed at the mment? Hw ften des the prject team meet? 11 / 35

12 CATEGORIES Cmmunity invlvement Outcmes QUESTIONS Which channels des the prject team use t infrm / guide the ther institutin s participants? Hw is the institutin s management invlved in the prject? Which respnsibilities were taken by the relevant staff regarding the prject? In which tasks were pupils/learners invlved? In which tasks were parents / families invlved in the prject? Have yu invlved ther institutin(s)/rganizatin(s) frm the lcal cmmunity? In which tasks were the institutin(s)/rganizatin(s) f the lcal cmmunity invlved? Did they assume the rle f assciated partners? Are yu planning t invlve ther institutin(s)/rganizatin(s) frm the lcal cmmunity? Which final prducts have yu already finalized? In what stage are yu, cncerning the cmmn final prduct(s)? M-Tl SET What are the tasks and rles fr each partner institutin cncerning the final cmmn prduct(s)? Prject results What knwledge did yu gain thrugh the prject regarding its thematic area? What knwledge did yu achieve thrugh the prject regarding Eurpean issues? What changed in yur language skills due t the prject? What changed in yur ICT skills due t the prject? What changed cncerning the implementatin f pedaggical methds due t the prject? What changed in the rganizatin f the institutin due t the prject? What changed cncerning mtivatin in learning / teaching? What changed in the relatinships within the institutin s participants due t the prject? 12 / 35

13 CATEGORIES QUESTIONS M-Tl SET What changed in the establishment f links with ther institutins f the lcal cmmunity due t the prject? What changed cncerning pupils/learners schl/training success mtivated by the prject? Hw d yu evaluate the prgress f the prject? Wh participates in the evaluatin prcess? Evaluatin What kind f feedback reached the participants? Which are the results frm yur evaluatin? What have yu changed as a cnsequence f yur evaluatin results? In which events / initiatives did/will yu participate t present /make knwn yur prject activities/ experiences and r results? What kind f infrmatin was prduced t use in the disseminatin activities? Disseminatin What activities did/will yu start and implement, n yur wn initiative, t present / make public the prject? Have yu develped any activity in cmmn with ther institutins t present/publicize the prject? Hw many participants were present in each disseminatin activity? Hw d yu intend t cntinue t spread/share infrmatin abut yur prject? Which activities have yu develped t prmte the applicatin f yur prject results by ther institutins / individuals? Which prject results were applied by ther institutins and /r individuals? Valrisatin What kind f institutins / grup f individuals have directly benefited frm yur prject results? What prject result(s) turned t be the mst significant ne(s) in terms f being applicable in similar educatinal cntexts? D yu cnsider that yur prject results are in harmny with the LLP Prgramme? 13 / 35

14 CATEGORIES Prject influences Cntacts with the NA QUESTIONS Which prblems/bstacles did yu encunter during the prject? Did yu share the prblems with all the peple invlved in the prject? Hw did yu slve the prblems detected during the prject s implementatin? Hw did the mentined prblems change, in sme way, the riginal prject? Which were the mst favurable cnditins t the prject implementatin? Are yu satisfied with the cntacts yu had with yur NA? Hw d yu think they culd be imprved? M-Tl SET 14 / 35

15 5. GENERAL INTERVIEW GUIDE CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T expse the aims f the visit. T describe the methdlgy used in the mnitring visit. Legitimating the interview T explain the prcess f a mnitring visit. T reinfrce the imprtance f the institutin s cllabratin. T mtivate the participants in the prject n the imprtance f the mnitring prcess. T infrm abut the feedback prcess regarding the mnitring results. Internet / NA website Cnceptin T recgnise hw the public has access t the Sectrial Prgramme. T identify the channel(s) used t btain infrmatin abut the Prgramme Hw did yu knw abut Cmenius/Grundtvig/Lenard Prgrammes? Persnnal / Prfessinal cntacts Previus participatin in ther Eurpean Prgrammes Participatin in NA events 15 / 35

16 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T be familiar with the partner finding prcedure. T understand hw the cntacts are established t begin a partnership. T cmprehend the criteria t select the partners. Hw did yu cme acrss the partners fr yur prject? Participatin in a VP r Cntact Seminar Use f an website fr partner finding Netwrks Previus partners in a different prject T recgnize hw the institutin rganized internally the prcess f prject cnceptin. T understand the initial mtivatin fr the prject. T knw wh is the staff invlved in the prject team. Was there a prject team t begin the wrk? Hw, whm T identify the tasks f each partner institutin during the cnceptin prcess. T be aware f the cmmn cnceptin f the partnership s prject. T knw the measures taken by the partnership members t elabrate the prject. T identify the articulatin between partner institutins. Hw did yu cllabrate within the partnership t cnceive the cmmn prject? Cmmn fill-in f the jint sectins f the applicatin frm 16 / 35

17 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T knw the awareness f the institutin cmmunity abut the prject. T identify the pprtunities given t the institutin cmmunity t be invlved in the prject cnceptin. T recgnise the invlvement f ther persns besides the prject team. When and hw did yu infrm the institutin abut the prject? In teacher/trainer meetings, thrugh institutin s psters r newspaper articles, n a website, etc Cmmunicatin in the partnership T be acquainted with the cmmunicatin strategy implemented amng partners. T identify a flwing cmmunicatin thrughut the prject. T identify a regular cmmunicatin thrughut the prject. Hw d yu cmmunicate amng partners? When (every day, every week, every mnth) Whm (prject team, teachers/trainers, management, pupils/learners, ther staff) T identify the measures undertaken t share infrmatin amng the partnership. Hw d yu share the infrmatin with all partners? Written cnfirmatin f activities and decisins T knw the tls used t cmmunicate with the partners. What tls d yu use t cmmunicate? Fax, phne, , regular mail, etc T be acquainted with the languages used t cmmunicate with the partners. Which are the cmmunicatin languages used during the prject? Curricular, nncurricular 17 / 35

18 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS Why T recgnize the need fr linguistic preparatin. Was there any linguistic preparatin needed? Wh (teachers/trainers, management, pupils/learners, ther staff, parents/families) T identify resurces used fr linguistic preparatin. Hw did yu satisfy thse needs? Using audivisual materials, making a language curse, hsting a Cmenius Assistant, etc Partnership Lcal activities T recgnize the integratin f the prject in the institutin. T be aware f the ways t integrate the prject in the institutin s wrk plan. T identify the measures taken t integrate the prject activities in the curriculum f pupils/learners. Hw is the prject included in the institutin s wrk plan? Which prject activities are included in the curriculum f pupils/learners? Which subjects T verify the implementatin f the lcal activities at the institutin and/r in the lcal cmmunity. T identify the degree f accmplishment f the planned activities. Which activities have yu already carried ut? Presently, in what activity are yu wrking? Where, when and with whm (teachers/trainers, management, pupils/learners, parents/families, assciated partners, ther participants) 18 / 35

19 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T be aware f the mst visible prject activity. What was the mst crwded/ppular prject activity? Hw many participants f each kind (teachers/trainers, management, pupils/learners, parents/families, assciated partners, ther participants) T identify changes in the partnership s wrk plan. What changed in the partnership planned activities? Why T be aware f the capacity t accmplish the partnership s wrk plan. T identify the cnditins that influence the prject s implementatin. Will yu be able t cmplete the planned activities? Why (enugh resurces, time, etc) T understand the cnsistency f the cperative wrk dne in the partnership. T identify the cmmn activities in the partnership. What kind f cmmn activities have yu already carried ut with yur partners? When and hw Mbilities T verify the implementatin f the mbility activities. T identify the degree f accmplishment f the mbility plan. Which mbilities have yu already carried ut? When, where and wh (teachers/trainers, management, pupils/learners, parents/families, assciated partners, ther participants), hw many T identify changes in the partnership s mbility plan. What changed in yur mbility plan? Why 19 / 35

20 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T be aware f the capacity t accmplish the mbility plan. T be aware f the cnditins that influence the implementatin f mbilities. Can yu cmplete the planned mbilities? Why (enugh resurces, time, etc) T understand the strategy t implement mbility activities. T be aware f the planning prcess regarding mbility activities. What kind f preparatry activities were develped befre the mbility? Wh T knw abut the cmpsitin f the participant staff in mbilities. Culd yu quantify the participating staff? Number f teachers/trainers, management, assciated partners, ther participants T identify the criteria fr chsing the staff fr mbility. Hw did yu chse staff fr mbility? Staff frm different subjects, staff with disabilities, etc T knw abut the cmpsitin f the participant pupils/learners in mbilities. Culd yu quantify the participant pupils/learners? Number f pupils/learners, schl level/type f training, age T identify the criteria fr chsing the pupils/learners fr mbility. Hw did yu chse pupils/learners fr mbility? Pupils/learners age, schl level/type f training, pupils/learners with disability, etc T be familiar with the criteria t twin pupils. Hw did yu twin pupils during their freign stay/ when hsting partners? Applicable t Cmenius Bilateral Partnerships Hst families and reciprcal twining 20 / 35

21 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T knw the direct cntributin f pupils/learners t the implementatin f mbility activities. Hw did yu invlve yur pupils/learners in the implementatin f the activities during mbility and /r when hsting partners? T knw the direct cntributin f parents/families t the implementatin f mbility activities. Which was the cntributin given by the parents/families t the mbility activities? T knw the direct cntributin f the Assciated Partners t the implementatin f mbility activities. Which was the cntributin given by the Assciated Partners t the mbility activities? Wh Visiting ther institutins, time spent as trainer r trainee, attending lessns, wrking tgether n the prject, time spent fr leisure T recgnize the cnsistency f the mbility planned activities. T knw the kind f activities develped during mbility. T be aware f the cherence f the wrk agenda f the mbility activities. Which activities were develped during mbility and/r partner s hsting? Hw much time was spent in each kind f activity during mbility and/r partner s hsting? T identify the fllw-up activities cnsequently t the mbility. What kind f fllw-up activities were develped after mbility? 21 / 35

22 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T verify the degree f invlvement in prject activities. T spt the vlume f institutin s participants. Hw many persns are presently invlved in the prject, including the prject team? Teachers/trainers, management, pupils/learners, ther staff participants. Institutin s invlvement T recgnize the dynamics f the institutin s prject team. T be aware f the management s supprt t the prject. T knw the cmpsitin f the institutin s prject team. T knw the wrk regularity f the prject team. T identify the channels used t infrm / guide the institutin s participants abut the prject. T identify measures regarding management supprt. Hw is the prject team frmed at the mment? Hw ften des the prject team meet? Which channels des the prject team use t infrm / guide the ther institutin s participants? Hw is the institutin s management invlved in the prject? Wh Hw many What subjects Cmenius Crner Meetings Visit ther partner institutins Links t ther institutins (lcal cmmunity, educatinal authrities, etc) T be acquainted with the staff s invlvement in the prject. T categrize the main respnsibilities f the staff invlved in the prject. Which respnsibilities were taken by the relevant staff regarding the prject? Teachers/trainers and ther institutin staff T be aware f the pupils /learners invlvement in the prject. T classify the main tasks f the pupils/learners invlved in the prject. In which tasks were pupils/learners invlved? Pupils assciatin Cmmunity invlvement T recgnize the parents /families invlvement in the prject. T identify synergies between the institutin and families, accmplished thrugh the prject. In which tasks were parents / families invlved in the prject? Parents Assciatin 22 / 35

23 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T identify synergies between the institutin and the lcal cmmunity, accmplished thrugh the prject. Have yu invlved ther institutin(s)/rganizatin(s) f the lcal cmmunity? Type f institutins/ rganizatins T recgnize the fficial relatinships between the institutin and the lcal cmmunity. T categrize the main tasks f the lcal cmmunity s invlvement. T identify the existence f an assciated partners netwrk. In which tasks were the institutin(s)/rganizatin(s) f the lcal cmmunity invlved? Did they assume the rle f assciated partners? Named in the applicatin frm T knw abut the pssible enlargement f cmmunity invlvement. Are yu planning t invlve ther institutin(s)/rganizatin(s) frm the lcal cmmunity? T identify the degree f accmplishment f the planned utcmes. Which final prducts have yu already finalized? Final prducts with Eurpean Lg Outcmes T verify the implementatin f the prject utcmes. T cllect evidence f the existence f the cmmn final prduct(s). T recgnize the cmplementary f tasks within the partnership. In what stage are yu, cncerning the cmmn final prduct(s)? What are the tasks and rles fr each partner institutin cncerning the final cmmn prduct(s)? What Whm Prject results T understand the results f the prject s implementatin. T knw the imprvements in knwledge in the thematic area f the prject. What knwledge did yu gain thrugh the prject regarding its thematic area? Teachers/trainers, pupils/learners, management, ther institutin staff, parents/families, etc 23 / 35

24 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T recgnize the imprvements made in the Eurpean Dimensin f Educatin/Training. T knw the imprvements in language skills achieved thrugh the prject s implementatin. T identify the imprvements accmplished in ICT skills thrugh the prject s implementatin. T knw the imprvements in pedaggical methds with the prject s implementatin. T identify the implementatin f new pedaggical methds due t the prject. T identify the imprvements in the management apprach. T knw the new rganizatin strategies implemented due t the prject. What knwledge did yu achieve thrugh the prject regarding Eurpean issues? What changed in yur language skills due t the prject? What changed in yur ICT skills due t the prject? What changed cncerning the implementatin f pedaggical methds due t the prject? What changed in the rganizatin f the institutin due t the prject? Teachers/trainers, pupils/learners, management, ther institutin staff, parents/families, etc Teachers/trainers, pupils/learners, management, ther institutin staff, parents/families, etc Teachers/trainers, pupils/learners, management, ther institutin staff, parents/families, etc Teachers/trainers, management, ther institutin staff Teachers/trainers, management, ther institutin staff 24 / 35

25 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T identify the imprvements n mtivatin f the prject participants. T knw the imprvements in interpersnal relatinships stimulated by the prject. T identify the reinfrcement f the institutin s internal chesin. T recgnize the imprvements n linking the institutin t the lcal cmmunity due t the prject. T knw the imprvements in pupils/learners schl/training perfrmance due t the prject. What changed cncerning mtivatin in learning / teaching? What changed in the relatinships within the institutin s participants due t the prject? What changed in the establishment f links with ther institutins f the lcal cmmunity due t the prject? What changed cncerning pupils/learners schl/training success mtivated by the prject? Pupils/learners / teachers/trainers Between teachers/trainers and pupils/learners, amng pupils/learners, amng teachers/trainers, between staff in general, between teachers/trainers and parents/families 25 / 35

26 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS Evaluatin T be aware f the evaluatin strategy adpted in the prject. T distinguish the implementatin f a twlevel evaluatin prcess (cmbining the institutin level and the partnership level). T identify the evaluatin phases. T recgnize the existence f an nging evaluatin prcess. T knw the methds used t cllect, analyse and interpret evaluatin data. Hw d yu evaluate the prgress f the prject? At institutin level and at the partnership level Phases Méthdes Techniques (interviews, questinnaires, bservatin, etc) What T identify the level f participatin in the evaluatin prcess. Wh participates in the evaluatin prcess? Teachers/trainers, management, pupils/learners, ther staff, parents/families, etc Hw many T identify the prcedure t reprt n evaluatin results. What kind f feedback reached the participants? At institutin level and at the partnership level 26 / 35

27 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T knw the institutin s evaluatin results (selfevaluatin). T be aware f the selfevaluatin results. T be familiar with the strng/weak pints f the prject accrding t the institutin s view. T knw the recmmendatins resulting frm the self-evaluatin prcess. Which are the results frm yur evaluatin? At institutin level and at the partnership level T identify the adptin f recmmendatins made at evaluatin reprting stage. What have yu changed as a cnsequence f yur evaluatin results? At institutin level and at the partnership level Disseminatin T recgnize the develpment f the disseminatin strategy. T identify the degree f accmplishment f the planned disseminatin activities. In which events / initiatives did/will yu participate t present /make knwn yur prject activities/ experiences and r results? Cnferences, seminars, meetings, exhibitins Where, when, hw, t whm Wh presented T knw the tls used t disseminate the prject. T be familiar with the criteria used t select the infrmatin fr disseminatin purpses. What kind f infrmatin was prduced t use in the disseminatin activities? Cntent f: a webpage, a newspaper, a PwerPint presentatin, an exhibitin, a perfrmance, etc 27 / 35

28 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T distinguish the nature f the initiative regarding the disseminatin activities. T identify the measures taken t disseminate the prject. T identify the existence f cperative wrk regarding the implementatin f disseminatin activities. T be aware f the level f public knwledge abut the prject. T identify the channels used t disseminate infrmatin abut the prject. T knw the kind f institutin s participants taking part in the disseminatin activities. What activities did/will yu start and implement, n yur wn initiative, t present / make public the prject? Have yu develped any activity in cmmn with ther institutins t present/publicize the prject? Hw many participants were present in each disseminatin activity? Hw d yu intend t cntinue t spread/share infrmatin abut yur prject? Aut-initiative C-authrship with ther institutins (wh) Cmmn rganizatin at partnership level Shared rganizatin with the assciated partners r ther rganizatins Teachers/trainers, management, pupils/learners, ther staff, representatives frm the educatinal authrities, frm the assciated partners, etc Spread experiences, activities and prject results What, when, where, hw, by wh and t whm 28 / 35

29 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T identify the degree f accmplishment f the planned valrisatin activities. Which activities have yu develped t prmte the applicatin f yur prject results by ther institutins / individuals? Training curses, seminars, cnferences, pedaggical materials ffers T understand the transfer pssibilities f the prject results (apprenticeships and/r final prducts). T identify the kind f prject results mre valuable t be transferred t ther educatinal cntexts. T knw the extent f the prject results at natinal / internatinal scale. Which prject results were applied by ther institutins and /r individuals? When, where, hw and by whm At individual, lcal, reginal r natinal level. Valrisatin T identify the target grup(s) that culd benefit frm the prject results. What kind f institutins / grup f individuals have directly benefited frm yur prject results? Distinctin between the participants (internal target grup) and the external individuals Hw many T be acquainted with the cntributin f the prject results t achieve Lisbn bjectives and/r the peratinal bjectives and pririties f the LLP Prgramme. T recgnize hw the prject results are linked t the expected impact. T identify the achievement f the prject results in face f the bjectives and pririties f the LLP Prgramme What prject result(s) turned t be the mst significant ne(s) in terms f being applicable in similar educatinal cntexts? D yu cnsider that yur prject results are in harmny with the LLP Prgramme? Target grup(s) f the prject Hw Why Prject influences T be aware f the difficulties in prject develpment. T identify specific factrs that negatively influenced the prject s implementatin. Which prblems/bstacles did yu encunter during the prject? At the institutin and at the partnership level 29 / 35

30 CATEGORIES OBJECTIVES SPECIFIC OBJECTIVES QUESTIONS GUIDING TOPICS T knw the slving prblem prcedures implemented in the partnership. T spt the effectiveness f the cmmunicatin strategy. T identify the level f cperatin between the prject participants. Did yu share the prblems with all the peple invlved in the prject? Wh (prject team, management, partnership crdinatr, all partners, NA, authrities, etc) T identify cncrete slutins in face f the prject s difficulties. Hw did yu slve the prblems detected during the prject s implementatin? Wh, when T be aware f the level f deviatin frm the initial prject plan. Hw did the mentined prblems change, in sme way, the riginal prject? Applicatin frm T be familiar with the factrs in favur f the prject develpment. T knw specific factrs that psitively cntribute t the prject implementatin. Which were the mst favurable cnditins t the prject implementatin? At the institutin and at the partnership level Cntacts with the NA T be acquainted with the degree f satisfactin f the institutin cncerning the NA. T spt the strng /weak pints in the supprt given by the NA t the beneficiary institutin. T identify ways t imprve NA best supprt and guidance. Are yu satisfied with the cntacts yu had with yur NA? Hw d yu think they culd be imprved? 30 / 35

31 6. GUIDE FOR ANALYSYS OF MONITORING DATA METHODOLOGICAL FRAMEWORK TO QUALITATIVE ANALYSIS Purpse Strategy CONTENT ANALYSIS Establishing hw the NA reaches the findings abut the mnitring visit. Organizatin, categrizatin and interpretatin f the infrmatin cllected (thrugh interview and bservatin). Cding the data cllected in the interview t d their labelling: - The pre-cded categries will be used t validate the interview respnses, because these categries are based n a clear definitin f aims (included in the General Interview Guide). - Fr cmplementing the use f the categries pre-specified (n the General Interview Guide), ther data categries (either descriptive r higher inference) will be develped if necessary. - The aim f rganizing the data by categries is t reduce data withut significant lss f infrmatin, summarizing the data by pulling tgether themes and identifying patterns and regularities. - The data cllected is rganized by small r large sentences labelled by the categries and bjectives (units f sense), filled in the clumn Mnitring Recrd f the Analysis Grid. Develping prpsitins based n the patterns and cnnectins evidenced with the cding, requiring sme degree f inference beynd the data. - Drawing the findings t establish a meaningful and cherent picture f the prject implementatin, putting in evidence the strng and weak pints and defining recmmendatins (filled in the clumn Cmments fr the Reprt f the Analysis Grid accrding t the items f the NA Mnitring Reprt). - Crssing the data cllected in the interview with the bservatin data t create a crrespndence between verbal respnses (what peple say) and what they d and what they say they d. - The findings will be supprted by crssing the bservatin evidence (frm direct bservatin and frm the interview) with the specific bjectives established fr each categry in the General Interview Guide. 31 / 35

32 EXAMPLE OF THE ANALYSIS GRID FOR THE INTERVIEW Categries Objectives Mnitring Recrd Cmments fr the Reprt Cnceptin Cmmunicatin in the Partnership T recgnise hw the public has access t the Sectrial Prgramme. T be familiar with the partner finding prcedure. Filled in with tpics cnstituted by units f sense frm the ntes cllected by the NA staff, during and after the mnitring visit. ( ) ( ) ( ) ( ) Filled in with inferences (findings) btained by crssing the bservatin evidence with the specific bjectives defined in the General Interview Guide. ( ) EXAMPLE OF THE ANALYSIS GRID FOR OBSERVATION Observed Dimensins Objectives Mnitring Recrd Cmments fr the Reprt 32 / 35

33 7. NA MONITORING REPORT Cmbinatin f results frm data cllected directly (bservatin) and indirectly (interview). Pssibility given t the institutin t refute matters f fact r interpretatins stated in the cnclusins f the NA Reprt (existence f a 1 st draft). Reprt structure (final versin) Identificatin f the institutin and the prject Objectives f the reprt phase: Mnitring visit: Gd practices emphasis; Cunselling and advice. Intrductin (abut the visit): Observed data: Findings: Date f visit, identificatin f the persns interviewed (management, crdinatin, ther staff and pupils/learners), place; Cntext and ther relevant circumstantial infrmatin; Identificatin f the NA mnitring team. Dcuments, materials and ther aspects bserved in situ; Evidence f prject activities (artefacts, reprts, bklets, lg bks, multimedia prducts, etc). Results f the cntent analysis f the interview(s); Reference t imprtant bservatin evidence, stressing the strengths and the 33 / 35

34 Reprt structure (final versin) M-Tl SET weaknesses identified with reference t the pre-defined parameters/categries; Identifying prblems and their causes; Identifying seeds f innvatin r excellence; Cnclusin (general appreciatin f the state f prject implementatin; lessns learned). Recmmendatins What shall be dne t slve the identified prblems? T adjust activities and/r resurces? T imprve actin? T make the prject better? Cntributin f the prject t the LLP Prgramme (bjectives and pririties). Institutin cmments abut NA cnclusins (in case there are any regarding the 1 st draft). Request fr a Fllw-up Reprt: Infrmatin regarding its minimum requirements; Establishing a deadline t its submissin t the NA. Date f next visit (ptinal). 34 / 35

35 8. FOLLOW-UP REPORT Aims: T ensure that the recmmendatins have practical cnsequences in prject implementatin. T make certain that recmmendatins are implement during the prject lifecycle. T guarantee imprvements in the quality f the prject. Cntent: Different frm institutin fr institutin; Open frmat, chsen by each institutin. Minimum requirements: The institutin has t prpse cncrete fllw-up measures fcused n the NA recmmendatins; These cncrete measures have t be established within a cncrete timeframe during prject implementatin. 35 / 35