I 1. The IDOS Practice Sheet addresses a broad audience, including persons who do not consider themselves educational

Transcription

1 A tool for the description, planning, and analysis of educational measures and learning systems. The Practice Sheet addresses a broad audience, including persons who do not consider themselves educational experts. is intended as a pragmatic tool for structuring tasks and discussions in the context of planning, development, or evaluation of learning systems. The way the tool is used always depends on its practical applicability and usefulness in the given case. Experience with in consulting mandates has shown that the tool is readily understood by all participants and actors and enjoys broad acceptance. Note: Where the flyer uses only the generic masculine for better readability, it refers to both male and female addressees. The four dimensions of a learning system is a tool for the description, planning, analysis, and evaluation of educational measures or learning systems. The acronym stands for: I = Content (in German: Inhalt) D=Didactics O=Organisation S = Significance People involved in the discussion of basic and advanced training or qualification or in the planning, implementation, evaluation and funding of training events or courses are always confronted with these 4 dimensions, because in most cases four fundamental questions need to be addressed: What do participants learn? Content Organised learning refers to specific learning contents, describable targets of knowledge and expertise to be conferred to trainees. The learning content is usually defined by training objectives. These comprise cognitive knowledge, (manual) skills, as well as values and attitudes to be acquired by the trainees. No training event can be effective without describable contents. How does learning and teaching take place? Didactics Learning is an individual process that often occurs in leaps and bounds and below the level of consciousness. Learning processes are probably often chaotic in nature and only partly controllable. Nevertheless, organised learning consists of a managed process where trainees are offered a learning environment which makes it possible for them to absorb information, gain and reflect on experience, ask questions and receive answers, rehearse and perform actions etc. These processes are organised in the particular learning system such that trainees can use a variety of methods: lectures, presentations, writing tasks, group work, case studies, simulations, role playing, etc. Under what conditions does learning take place? Organisation Organised learning requires an organisational framework which may vary in the degree of flexibility and formalisation. The spectrum ranges from ad-hoc training courses within a company or organisation to the formal curriculum of a school. Ideally, learning processes are fostered by appropriate organisation and plausible rules that enable trainees to follow their individual learning style and to benefit from a variety of learning arrangements. I 1

2 What is the relationship to the education-policy context? Significance Every learning system is placed within a context shaped by educational policies, job markets and, increasingly, competitive educational markets. The significance of the training is determined by its position in the hierarchy of the educational system (secondary level II, tertiary level B, tertiary level A, academic / non-academic), the degree of recognition the certificate enjoys in the job market, its reputation in the particular trade, as well as the recognition of course providers and occupational titles. Statements, assessments, and observations regarding a particular learning system may be more or less correct, more or less significant, rely on a more or less adequate basis and an individual configuration of interests. Never- theless, all such statements, observations and evaluations can be associated unambiguously with one of the four dimensions named above: «The course is very sophisticated.» Content «I have to read a lot at home.» Didactics «It is an expensive course.» Organisation «I think this course will bring me great benefit.» Significance The actors in a learning system and their role A particular learning system involves different actors. In the actual learning process, which is always in the focus, these actors are the instructors and the trainees. But in the background, representatives of both the operating and the funding agencies play an essential role unless the training activity is fully funded by the participants themselves. All four actors and actor groups view the learning system from their own perspectives. The perspective is determined in each case by the specific interest in the particular learning system. The students and trainees expect a personal benefit from participation in a learning system. Their aim is to achieve the greatest possible learning success with a minimum investment in time and money in order to enhance their education, improve their professional skills, and thus increase their job market opportunities or acquire a diploma. They must be willing to make a personal effort and to incur expenses. The training period should be diversified and inspiring, and the training place and environment should be as pleasant as possible. For instructors and teachers, the learning system represents a workplace and a source of income. They want to foster learning success, keep their own workload within acceptable limits, have an optimal infrastructure at their disposal, and work in a pleasant job environment at the highest possible wage. As representatives of a particular discipline or as subject lecturers or trade representatives, they also aim at developing the competence of specific staff groups and increase their public reputation. For most providers (schools, course centres etc.), to provide training is a core task. But today there also exist training facilities offered by other, non-school entities such as trade associations, private service providers, and companies. For these providers, training is an additional product in their overall portfolio. Those who fund the learning system are interested in the effectiveness of the training. For them, it is important that successfully qualified trainees are capable of utilising the acquired knowledge and skills in a way that best serves the aims and objectives of the funding institution or the trade. This implies an interest in an optimal training efficiency (cost-benefit ratio) as well as in the best possible applicability of the acquired qualifications. These aspects reach beyond the actual training process and include, for example, the performance and opportunities of the trainees in the job market. These different perspectives are inherent in every organised learning scheme and determine its design. Therefore, a differentiated consideration of these perspectives is very important when describing and evaluating learning systems: I 2

3 Participants, trainees, beneficiaries Lecturers, educators, trainers Training suppliers Donors Contents Curiosity: Are the learning contents interesting? Learning objectives: What needs to be taught? Qualification: Do the contents advance qualification? Relevance: Are the contents adequately selected, are they important? Didactics Method: How and in which way do I learn? Learning mode: What is the best way to convey the contents? Attractiveness: Are the learning modes up to date and userfriendly? Effectiveness: Are the learning objectives attained? Organisational Environment: In which environment do I learn? Resources: Which teaching aids are available? Adequacy: Are conditions favorable to the learning processes? Efficiency: Is there a favorable relationship between costs and benefits? Significance Significance: What is my benefit from the training? Learning success: How many beneficiaries attain the learning objective? Profile: Is the training labour market relevant and/or image-promoting? Legitimacy: What relationship exists with the educational system? Two practical examples Certification process for a post-graduate study Initial situation Within the scope of the certification process, the actors involved are required to discuss the quality of the training course. The three essential actor groups comprise 1. Trainees, students 2.Teachers, lecturers 3. Representatives and staff of school and funding agency Preparation of the workshop Four pin boards with headings and moderation box with cards, markers etc. Content: Selection of subjects, structure, weight etc. Didactics: Knowledge transfer, forms of instruction, practical relevance etc. Organisation: Time management, information etc. Significance: Course accreditation, labour market relevance I 3

4 3. Implementation Step 1: The moderator explains the grid / System and the procedure (time: approx. 5 minutes). Step 2: The three actor groups circulate from one pin board to the next, write keywords on the cards, and attach the cards to the fields on the pin boards. Each actor group uses its own card colour. (Time for working a single pin board: approx. 12 to 15 minutes. Maximum total time for step 2: 1 hour). Step 3: The participants discuss all four pin boards in a plenary session time: 10 to 15 minutes per pin board, maximum 1 hour). 4. Result Four key aspects of the course have been evaluated from different perspectives (card colours!). The assessments are known to all actor groups, and a first discussion has taken place. An overview of the evaluation of the course is available: Based on the material developed and the discussion, an overview of the in-depth evaluation of the course, its strengths and weaknesses etc. has been produced. Agreement and dissent in the evaluation become transparent: conflicting interests can be identified and possible solutions can be assessed. The need for action has been discussed and identified: a first analysis is available to the decision makers, and development steps can be planned. Planning of educational activities in development cooperation (DC) Programmes and projects of international cooperation consist to a considerable extent of activities to facilitate and foster learning and know-how transfer. Particularly in bilateral cooperation, there are hardly any projects that do not involve measures to develop capacities. For the funding agencies of development cooperation (DC), planning and preliminary evaluation of basic and advanced training activities involves the following questions: Content: Are the training contents appropriately selected and are they significant for (a) the development project or (b) the development process in the target country? Response / indicators: The justification of the appropriate selection and the necessity of a specific training are clearly recognisable in the project proposal. Or: There is a proven relevance to the job market or to a previously developed job market analysis. Didactics: Does the project or the project organisation ensure that the learning contents are conveyed to trainees in an adequate and learning-friendly way? Response / indicators: The primary question is whether professional educators should be employed and whether the learning scheme has already been successfully implemented in another place, and/or whether consulting and coaching would ensure the use of adequate and attractive learning methods. Organisation: Is the training or learning process organised such that the available resources translate into a commensurate benefit? Response / indicators: Basically, the principle that participant costs per hour or other indicators can be used to asses the cost-benefit ratio also applies to educational measures in international cooperation. However, educational measures in DC are often tailor-made, one-off events for which no comparative cost-benchmarking is available. A simple way to express the organisational effectiveness of learning processes is to see whether local training offers exist or, respectively, whether existing local training offers can be acquired (e.g. from another funding agency or DC project). Significance: How does the planned educational measure fit into the educational landscape of the target country? In many DC projects, new courses are introduced with the intent to integrate them later into the regular educational system. The key question is: Is it possible, for example, to institutionalise the course locally so that the know-how developed in and through the project, the facilities, the curriculum, the trainers and the multipliers can be utilised in future and made available to a larger target group? I 4

5 checklist for preliminary evaluation of training courses or modules in development cooperation projects Answer questions, assess facts and circumstances Questions Content Are the training contents adequately justified? Are the training contents relevant to the job, market opportunities or to the status of development in the target country? Are the contents up to date, do they correspond to the state of the art? Questions Didactics Are professional trainers employed? Is it assured that adequate, user-friendly and modern learning methods are applied? Questions Organisation Is the training organised efficiently? Have alternative solutions (existing training offers in the target country, acquisition of training offers) been considered and assessed? Are the training services systematically evaluated through feedback from trainees? Is information on the costs available, and can the costs be compared with similar projects and activities? I 5

6 Questions Significance: Is there a reasonable or observed demand for the planned training in the job market? Is it possible to institutionalise the training locally? Do the responsible actors intend to integrate the training into the regular education system at a future date? Is an indirect benefit to be expected? Is an indirect benefit described and operationalised in the planning documents?. Analysing the results The result from the answers to these questions requires interpretation. For example, conditions for a positive overall evaluation of the project may be stipulated in advance. In the case presented here, it was decided that a positive evaluation would require at least 10 green points. In the case of more than 4 red points, the project would need to be either redesigned, or its planning improved, or its funding abandoned. AG KEK-CDC AG is a consulting firm founded in 1983; eight to ten partners supported by scientific collaborators offer consulting services, concept development, evaluation, moderation, and advanced training in the fields of Organisational development International co-operation Education Social issues Public administration and non-profit organisations, Universitätstrasse 69, CH-8006 Zurich, Tel , info@kek.ch, Rue Hugi 3, CH-2502 Biel-Bienne, Tel , info@kek.ch June 2010 I 6