1 G Program Development and Evaluation Collecting Evaluation Data: Direct Observation Ellen Taylor-Powell and Sara Steele We often become so involved in developing questionnaires to secure information from people that we overlook the fact that considerable data can be collected by just observing. Direct observation 1 is an underused and valuable method for collecting evaluation information. ÒSeeingÓ and ÒlisteningÓ are key to observation. Observation provides the opportunity to document activities, behavior and physical aspects without having to depend upon peoplesõ willingness and ability to respond to questions. When observation is useful Observation is an essential element in good teaching and program development. In evaluation, it can be used to secure benchmark and descriptive data during program initiation and to document program activities, processes and outcomes. Observation is appropriate in the following conditions: When you want direct information. For example: Ñ Paying random visits to farms, homes, gardens (or any sites) may be a more reliable indicator of whether people are using practices recommended by Extension than by simply asking people. Ñ Observing the management operations and procedures of small firms may provide better information than relying on reports or key informants. Ñ Unobtrusively recording the ethnic diversity of a group provides direct information on participantsõ characteristics. When you are trying to understand an ongoing behavior, process, unfolding situation or event. For example: Ñ Observing and describing what is done in each phase of a community development project. Ñ Observing parents dealing with interruptions from their toddlers. Ñ Observing officers conducting a business meeting before and after an officer training program. When there is physical evidence, products or outcomes that can be readily seen. For example: Ñ Observing materials set out for recycling along the roadsides. Ñ Periodically observing along the coastline of a lake involved in the ÒAdopt a LakeÓ program. Ñ Having a team of experts inspect the quality of grasses and legumes in a pasture. Ñ Inspecting gardens, newsletters, project books, etc. When written or other data collection procedures seem inappropriate. For example: Ñ You have several participants volunteer to observe and report on your program delivery rather than having all participants fill out a questionnaire. Ñ A perceptive observer records dynamics and concerns during a workshop for new immigrants. Ñ Trainers observe each othersõ financial management classes with low-income women, noting dynamics, questions raised, and level of participation. 1 Direct observation as used here should not be confused with participant observation as used in ethnography and anthropological research. The latter involves much greater time, training and immersion in the setting than direct observation as discussed here.
2 2 P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N The question isñwhat do you want to know? Then, is observation the most appropriate and costeffective method to collect that information? Another question isñwhat will the users/stakeholders view as credible and useful information? This may not be a concern if you use the results internally for program improvement or professional development. But, if the data are to be used with or by others outside the organization, you might assess how they will react to observations as evidence of program processes and results and who will be seen as credible observers. Being thoughtful and systematic will help establish observation as a credible evaluation method. Do people need to know that they are being observed? Observation can be overt (obtrusiveñeveryone knows that they are being observed) or covert (unobtrusiveñpeople do not know). The extent to which people need to be informed depends upon the situation and your evaluation purpose. People may or may not be asked for their full and informed consent. Given that people often behave differently when they know they are being observed, covert observation is appealing. However, it is essential that neither the observation nor the resulting report harms the people observed. This is seldom a concern in Extension program evaluation. While there is no consensus in social science research about how explicit to be about observations, the ethics and morality of evaluation must always be considered. What to observe The possibilities for observation are almost limitlessñ people, behaviors, reactions, physical settings, environmental features, record keeping systems, project reports, and more. Even within an educational event, a variety of elements can be observed to evaluate the delivery and potential outcomes of the event. For example, consider the topics in the sidebar when you observe an Extension program. Often, it will be impossible to observe all people, sites, or program documents. In these cases, you will need to sample. The same principles of sampling apply to observations as to other forms of data collection. If you want to generalize your findings to the total population from which you took your sample, you will need a probability sample (which provides for random selection). If generalizability is not important or if your purpose is to observe certain cases for a specific reason, then nonprobabilty or purposeful sampling is appropriate. Refer to the publication on sampling for further help. Program components to observe Characteristics of participants (individually and as a group) Gender, age, profession/vocation, dress, appearance, ethnicity Attitude toward subject, toward others, about self Skill and knowledge levels Statements about commitments, values, changes to be made Interactions Level of participation, interest Power relationships, decision-making, current issues General climate for learning, problem-solving Levels of support, cooperation Nonverbal behavior (learners, presenters) Facial expressions, gestures, postures Interest and commitmentñ initial impacts Program leader(s), presenters Clarity of communication, access to questions Group leadership skills, encouraging full participation Awareness of group climate Flexibility, adaptability Knowledge of subject, use of aids, other teaching/learning techniques Sequence of activities Physical surroundings The roomñspace, comfort, suitability AmenitiesÑbeverages, etc. Seating arrangements Products of a program Demonstrations, facility, plansé Brochures, manuals, newslettersé Source: Cloutier et al., 1987, section III, p. 50 A common occurrence is to observe the most successful sites or to observe the most convenient people. This is understandable, but state clearly what your observations represent. DonÕt suggest that they represent the total population.
3 C O L L E C T I N G E V A L U A T I O N D A T A : D I R E C T O B S E R V A T I O N 3 Looking for or looking at: Structured and unstructured observations Looking for. In evaluation, we are often looking for certain thingsñhow many kids are wearing their riding helmets, what leadership skills are being demonstrated, how farmers are practicing grazing management. Such structured observations are directed by a preset guide or checklist of what to observe. For example, to pass a CPR examination a trained instructor watches how you give CPR. The instructorõs observation guide probably lists each step of the process with a place to indicate if it does or does not occur, and/or a rating scale to indicate how well it is carried out. There is also probably a place to make notes. Structured observations are used when we want to standardize information and do a numerical summary of how many people are doing certain things. They provide quantitative data from frequency counts, rankings and ratings. Looking at. In other instances, you may wish to observe people, activities, or physical aspects as they naturally exist, recording whatever you see that relates to your evaluation question(s). You do not confine yourself to looking for preset items. This approach comes from anthropology. It seeks to be inclusive and to see things within the participantõs context. For example, when evaluating a program to improve fathersõ parenting skills, you may observe fathers with their preschoolers. You will observe the full situationñeverything that is going on in the settingñand not limit your observations to preidentified areas. With an open-ended, unstructured format, you can pick up things which you might not have thought about in advance. Less structured observations produce qualitative data. In some situations, observing what does not happen may be as important as observing what happens. Often, it is useful to combine elements of the structured and unstructured formats in the observation, including the preidentified items that you want to pay attention to as well as space for recording the unexpected or unique happenings. Recording your observations To be useful and credible for program evaluation, observations need to be recorded. Sometimes this is done on the spot; at other times, you may record your observations after youõve left the situation. It is important to record the date and location of the observation. A short description of the context in which the observation occurred is helpful. Remember, observations pertain to what you see as well as what you hear. Recording observations is done through one or more of the following means. You will find examples of these methods in the appendix. Observation guides. These are printed forms that provide space for recording observations. They are particularly useful when several observers are involved or when you wish to obtain comparable information from several sites/observation points or observations of many people. The more structured the guide, the easier it will be to tally the results, but the less opportunity there will be to record variations or unexpected occurrences. Recording sheets or checklists. These forms are used to record observations as in a YES-NO option (present-not present) or on a rating scale to indicate extent or quality of something. Checklists are used when there are specific, observable items, actions or attributes to be observed. In Extension, we are familiar with such checklists when we are judging contests or fairs and looking at the products that people have produced or the performances they are giving. Field notes. This is the least structured way to record observations. Originating from the field of anthropology, using field notes in Extension evaluation means recording observations in a narrative, descriptive style as you notice or hear something of import. There is no predetermined item or behavior to observe and, thus, no printed recording form. Often a notebook is used to record these observations. Leave a wide margin on one side of the notebook paper to facilitate later analysis of the text. Remember to carefully record the date, location and any relevant contextual information for all field note entries. You may select certain sites/programs/situations to visit or record observations spontaneously when you observe programs, participants, or physical products and outcomes. You may write down observations at the time, take brief notes and write a more descriptive narrative later. Or you may commit your observations to memory and make notes later. In some instances, you may want to dictate your observations as soon as possible into a tape recorder for later analysis. Small pocket tape recorders are particularly handy for recording field notes.
4 4 P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N Pictures. Photographs and video tapes can be used to record observations. The camera is an extension of your eye. The camera makes a record that can be analyzed later and may be used to illustrate your evaluation report. Combinations. There are no pure observation recording methods. Any of the above recording options may be combined or altered to meet your particular evaluation needs. Often observation forms include a combination of checklists, scales, number lines, etc. as well as an open space to record Òother observations.ó Preparing yourself How many times have you looked at a scene with another person and ended up saying, ÒI didnõt see that!ó or sat in an audience but not really listened. If you are going to use observation as a means of collecting information, you need to watch and listen attentively. One element is to be able to capture detail. Another is discerning what is important. Once the observations are made, you will need to interpret the meaning of what youõve observed. And sometimes you will need to validate major impressions. If possible or feasible, using a team to carry out the observations provides a more complete picture and avoids individual biases. Capturing detail. Artists develop skill in observation by studying and trying to replicate what they see. You might want to start increasing the accuracy of your observation by studying a complex picture for a few seconds. Then move on to describing everything you can see out of your window. Finally, try describing the detail in a dynamic scene that is not repeated. Discerning. There is likely to be much more in what you are observing than you can or need to take in. What is important? You need to be discerning in looking at all of the detail in an observation and select that which is important to your purpose. Interpreting. Just as survey and interview data can be interpreted differently, what you see and hear can be interpreted in various ways by different people or even by yourself at different times. Therefore, a good observer takes care in interpretation. You may wish to include others in the interpretation or use research to help make sense of the observational data. Validating. When decisions about what is observed have major consequences, it may be important to validate oneõs observations. That may mean going back and repeating the observation. It may mean watching several people before forming a conclusion. It may mean involving others to validate the observation. Recording observations on video tape may be necessary when instantaneous observations are difficult. Tapes can be played several times and examined by more than one person to validate the conclusion. Who are the observers? You. You are the most likely person to be the observer. Arm yourself with an observation guide, a checklist or a notebook for jotting down observations and comments. Keep field notes during the course of a program and use those notes as evidence. Participants. Program participants, whether randomly chosen and asked to do some special observing or a purposefully selected steering panel, are in a good position to observe a program. Either singly or as a group, you might develop a list of things to look for or you may decide to record anything deemed important. Through a debriefing, the observers can share their information and implications of what they observed. Or each observer might write up a brief report with or without recommendations. Participants are asked not only to participate as they ordinarily would in the session, but to be more aware of the reactions and interactions of their fellow participants. Participants can also be asked to observe someone elseõs program. For example, a 4-H project leader might be asked to attend a project meeting in another club, and, upon leaving the meeting, to fill in a check sheet or write or tape a description following a guide. A secondary benefit is that the project leader may well see things to change in her or his own meeting by spotting them somewhere else. Stakeholders. If a key stakeholder can take the time, involving him or her as an ÒofficialÓ observer can serve as a public relations effort, can increase the officialõs awareness of the depth and value of the program, and can provide you with evaluative data. You will probably want to orient the official to the ÒobserverÓ role and perhaps, provide him/her with a guide or checklist of things to observe. Arrange for a debriefing or discussion afterwards. Colleagues. Fellow agents, specialists and/or collaborators could be useful observers of your programs. You might work out a reciprocal relationship. Other members of the program team could provide this input and feedback, intruding little, if at all, on the atmosphere of the educational program. Volunteers. Volunteers can be engaged to carry out observations, or you might interest exisiting program volunteers in the task. For example, Master Gardeners sometimes carry out inspections/observations of gardening practices, including inspections of shelves for dangerous chemical products. A team of Master Volunteers could be recruited to visit club meetings and evaluate club strengths and weaknesses. Or, video tape sessions you present and have volunteers help you analyze your performance. As suggested earlier, a team approach to collecting observation data provides a more complete picture and helps avoid individual biases in data collection and interpretation.
5 C O L L E C T I N G E V A L U A T I O N D A T A : D I R E C T O B S E R V A T I O N 5 Training observers When involving other people as observers, some training may be necessary. The level and amount will depend upon the complexity of the observations and individual capabilities. There might be some orientation to the purpose of the evaluation, how the observations are to be used and what is involved in collecting observational data: what to look for and how to record the observations. Training will be necessary if you want to standardize the observations to aggregate comparable data across sites. All observers need to see and record as similarly as possible. It will be important that all the observers look for the same things, use similar meanings for what they see and record their observations in like manner. If rating scales are used, a clear understanding of the ratings (what each rating means and how to tell one rating from another) is required. Practice will help to ensure that observers are rating the same observation in like fashion. The Urban Institute, in particular, has used trained observers to evaluate community services such as street and alley cleanliness, road and traffic control maintenance, housing conditions, community appearance and building maintenance. Basic to this system is a set of written and photographic guidelines that enable the observers to consistently assign a grade or numerical rating to what is observed. It requires clear written definitions and/or benchmark photographs to ensure consistent ratings. After appropriate training, different persons acting at different times and using the same photographic and/or written guidelines and rating procedures are able to produce comparable ratings (see Wholey et al., 1994 for further detail). A variety of possibilities exist for using observation as a credible data collection method in Extension program evaluation. Being thoughtful and systematic is fundamental. With a little planning, we can build observation techniques into our programs, collecting and interpreting information as we go along rather than leaving evaluation until the end. References Cloutier, Dorothea, Bill Lilley, Devon Phillips, Bill Weber, David Sanderson A Guide to Program Evaluation and Reporting. Orono, Maine: University of Maine Cooperative Extension Service. Kumar, Krishna Rapid, Low-Cost Data Collection Methods for A.I.D. U.S. Agency for International Development. Washington, D.C. Patton, M.Q Qualitative Evaluation and Research Methods. 2nd ed. Newbury Park, Calif: Sage Publications. Spradley, James P Participant Observation. New York: Holt, Rinehart and Winston. Wholey, Joseph S., Harry P. Hatry, Kathryn E. Newcomer Handbook of Practical Program Evaluation. San Francisco: Jossey-Bass. Worden, Phyllis E A Source Book for Program Evaluation and Accountability. Fort Collins, CO: Colorado State Cooperative Extension. Numerous books and articles on observation in social science research and participant observation in anthropology.
6 6 Appendix: Observation guides Examples Field Day Evaluation Location Date 1. Number of people who stopped and looked at exhibit (make a mark for each) 2. Number of people who asked a question to agent supervising exhibit 3. Number of people who actively used the interactive materials 4. Number of people who took brochures 5. Did people seem to struggle to see the exhibit? yes no 6. Were staff always available? yes no 7. Comments made or questions asked: Family Financial Management: How is the trainer doing? Location Date 1. Were all materials distributed to participants? yes no 2. Were the learning objectives stated? yes no 3. Did the instructor speak clearly and distinctly? yes no 4. Did participants arrive and return from breaks on time? yes no 5. Was time allotted for questions and feedback? yes no 6. Did instructor answer participantsõ questions? yes no Father-Child Workshop Location Date [Instructions to observer: Write down actions you see and comments you hear under the following headings] 1. Disciplinary actions fathers demonstrate 2. Physical contact fathers have with child 3. Ways in which fathers praise their children 7. Did instructor demonstrate a thorough knowledge of the topic? yes no 8. Did instructor engage the participants in problem solving? yes no 9. Did participants seem to listen attentively? yes no 10. To what extent did participants willingly participate in activities? Not at all A great deal Comments heard: Other observations:
7 C O L L E C T I N G E V A L U A T I O N D A T A : D I R E C T O B S E R V A T I O N 7 Individual Shopper Observation Recording Sheet 1. Sex: M F 2. Estimated age: 3. Race: Caucasian Black Hispanic Other or unsure 4. Hurried Unhurried 5. Major shopping Minor shopping 6. Using shopping list: yes no uncertain 7. Product area shopped a. Reads container label b. Examines for freshness date c. Selects generic brand d. Obviously compares brands e. Selects Òbest buyó among brands available f. Uses coupons 8. Other observations about shopping practices: Soybean Impact Study Recording Sheet Date Location and site Name/ Mile marker number of road NW quadrant of county NE quadrant of county SW quadrant of county SE quadrant of county Approximate size of bean field: acres Tillage method used: No-till In stubble In grass Conventional Other things observed: (equipment being used, row spacings, approximate planting date, weed contamination, seed bed preparation, general appearance of beans, etc.) Source: Worden, Example from Florida Source: Worden, Example from Florida Field Notes (excerpt) Date March 5, 1996; 8Ð9:30 p.m. Location Church meeting room; third meeting of strengthening youth collaborative Judy, Stan and Colleen talking when I arrive. Judy said she thinks weõre spinning our wheels; Stan agreed. George arrives 10 mins lateñapologizes. HeÕs been traveling so didnõt accomplish what he promised at last meeting. Spent most of time talking about what to do next. Judy wants to do something; George wants to include more people in our group. Each to bring recommendations to next meeting. I suggested that we spend some time at the next meeting talking about vision and what we each hope to accomplishñiõll facilitate. Date March 25, 1996; 8Ð10 p.m. Location Church meeting room; fourth meeting of strengthening youth collaboratives I facilitated discussion about visions and individual expectations; seemed to work well. Used visioning process and scenario building. Colleen agreed to draft mission statementñall to review. Emerged that we need representation from the business community in our group and more public visibility.
8 8 Youth Development Skills: Communications Skill Recording Sheet Instructions to the observer: Observe youth at program start-up and again at the end of the program. Key Needs Beginning Developing Advanced/ 1st 2nd Change concept to improve Leader evaluation evaluation score Self- Resists sharing Wants to Expresses Expresses expression of thoughts, express thoughts, thoughts, thoughts, feelings; shy feelings feelings in feelings even directly, but some safe in difficult struggles situations situations to do so Listening Routinely Listens to Listens Questions ignores input friends but carefully to others to from others ignores input input from get other from others everyone viewpoints Impulse control Routinely mis- Identified Routinely Turns negative handles conflict impulses manages impulse (verbal/physical that tend conflict by into positive abuse or to promote themselves actions avoidance) trouble Source: Bonnie Herrmann, Milwaukee County The authors wish to thank Mohammad Douglah for his thoughtful review and Tom Parslow, Laurie Boyce and Bonnie Herrmann for their additions to the observation recording forms. Authors: Ellen Taylor-Powell is a program development and evaluation specialist for Cooperative Extension, University of WisconsinÐExtension. Sara Steele is a professor of continuing and vocational education at the University of WisconsinÐMadison. An EEO/Affirmative Action employer, University of WisconsinÐExtension provides equal opportunities in employment and programming, including Title IX and ADA requirements. Requests for reasonable accommodation for disabilities or limitations should be made prior to the date of the program or activity for which they are needed. Publications are available in alternative formats upon request. Please make such requests as early as possible by contacting your county Extension office so proper arrangements can be made by the Board of Regents of the University of Wisconsin System doing business as the division of Cooperative Extension of the University of WisconsinÐExtension. Send inquiries about copyright permission to: Director, Cooperative Extension Publications, 201 Hiram Smith Hall, 1545 Observatory Dr., Madison, WI This publication is available from Cooperative Extension Publications, Room 170, 630 W. Mifflin Street, Madison, WI Phone: (608) G Program Development and Evaluation, Collecting Evaluation Data: Direct Observation (August 1996) I M-200
G3658-4 Program Development and Evaluation Collecting Evaluation Data: An Overview of Sources and Methods Ellen Taylor-Powell Sara Steele June 1996 Acknowledgements The authors wish to express their sincere
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this
G3658-6 Program Development and Evaluation Analyzing Quantitative Data Ellen Taylor-Powell Statistical analysis can be quite involved. However, there are some common mathematical techniques that can make
And Organization Building GOALS By the end of this module, participants should be able to understand: The concept of self-interest and how it is used to recruit and retain volunteers. The Six-Step Process
Career Service Authority Clinical Social Work Team Leader Page 1 of 6 GENERAL STATEMENT OF CLASS DUTIES Performs supervisory duties over professional social workers engaged in performing a variety of social
G3658-11 University of Wisconsin-Extension Collecting Evaluation Data: End-of-Session Questionnaires Ellen Taylor-P ylor-powell ell Marcus Renner Program Development and Evaluation University of Wisconsin-Extension
Career Service Authority Contact Center Operations Manager Page 1 of 6 GENERAL STATEMENT OF CLASS DUTIES Supervises customer service agents working in a contact center environment while managing continuous
Steps for Conducting Focus Groups or Individual In-depth Interviews 1. Plan the study Plan your study with the input of your strategy team, stakeholders, and research experts so that collaboratively you
13. 3and Creating mutual trust respect Organisations that thrive are those where the company culture promotes mutual trust and respect of colleagues, and this is as true in PR as it is elsewhere. In this
Leadership D e v e l o p m e n t Your Guide to Starting a Program This publication is produced by the Leadership Education and Training Division of Rotary International. If you have questions or comments,
2008 Community and College Student Development 2009 PRACTICUM HANDBOOK The College of Education & Human Development UNIVERSITY OF MINNESOTA Department of Educational Psychology Counseling and Student Personnel
County of Yuba Customer Service Examination Study Guide The following study guide will familiarize and assist you with preparing for a written examination containing multiple-choice customer service items.
Effective teaching and classroom management is about whole child - and whole school development for knowledge, skills and human values During the past years as an outcome of the UN Study on Violence against
General Guidelines for Conducting Structured Interviews The Interview Board When developing an interview board the following principles should always apply. Interview boards should consist of three people.
10 Steps for Planning Educational Programs Guidelines to Help You Complete the Worksheet and Develop Quality Programs 1. Situation Study your county/area to assess the needs for educational programming.
Sergeant at Arms CLUB OFFICER TRAINING TOASTMASTERS INTERNATIONAL P.O. Box 9052 Mission Viejo, CA 92690 USA Phone: +1 949-858-8255 Fax: +1 949-858-1207 www.toastmasters.org/members 2015 Toastmasters International.
St. Cloud State University Department of Residential Life Student Leadership Development Model Mission of the Department of Residential Life The Department of Residential Life works to provide a student
Customer Service and Communication Bringing service to the next level 1 Park Authority Philosophy & Goals Before focusing on customer service, it is first important to understand and reinforce the Park
Career Service Authority Therapist Supervisor Page 1 of 6 GENERAL STATEMENT OF CLASS DUTIES Performs supervisory duties over professional mental health therapists engaged in performing a variety of social
UCLA CENTER FOR HEALTH POLICY RESEARCH Section 4: Key Informant Interviews Purpose Key informant interviews are qualitative in-depth interviews with people who know what is going on in the community. The
Project on Improvement of Local Administration in Cambodia Five Steps of Training Evaluation Step 1: Identify the Purposes of Evaluation Step 2: Select Evaluation Method Step 3: Design Evaluation Tools
EMPORIA STATE UNIVERSITY THE TEACHERS COLLEGE CLASSROOM MANAGEMENT: ED334 Spring, 2011 Three Hours Instructor: Steve Neill, Associate Professor Office: Visser Hall 203B Address: Box 4037 Emporia, Kansas
V. Course Evaluation and Revision Chapter 14 - Improving Your Teaching with Feedback There are several ways to get feedback about your teaching: student feedback, self-evaluation, peer observation, viewing
Collier County Public School District Principal Mentor Handbook Masterful mentors inspire people by helping them recognize the previously unseen possibilities that lay embedded in their existing circumstances.
Chapter 7 Qualitative methods for effectiveness evaluation: When numbers are not enough 7.1 Introduction 7.2 Methods of collecting qualitative information 7.2.1 Interviews and focus groups 7.2.2 Questionnaires
Page 1 of 11 REVISED JOB TITLE: REPORTS TO: SUPERVISES: DIRECTOR OF EARLY CHILDHOOD EDUCATION Assistant Superintendent for Curriculum and Instruction Students and staff assigned to the Early Childhood,
Interests, Aptitudes, and Abilities... 1 Entrepreneurship... 2 Taking an Interest Inventory... 3 TSA... 5 Keys to a Positive Attitude... 6 Teamwork... 7 Work Habits... 9 Personal Employability Traits...
JOB FAIRS AND INTERVIEWS: TIPS FOR SUCCESS 1. You may land a job! Why Attend a Job Fair? 2. The Employers Want to Meet You: these employers will be prepared to conduct onthe-spot interviews with the candidates
Building a Training Program A Workshop presented to [Insert audience s name here] [Insert trainer s name here] [Insert date/year of training here] [Insert location of training here] Learning Goals At the
One-On-One Interviews: A Qualitative Assessment Approach by Cristin Bates (Wellesley College 08) Claire Droste (Wellesley College 10) Lee Cuba (Wellesley College) Joseph Swingle (Wellesley College) TABLE
Group Volunteering TOOLS AND RESOURCES Funded in part by the Government of Canada s Social Development and Partnerships Program Contents What is group volunteering?... 3 Finding group volunteering opportunities...
Data collection is collecting specific information about a student s academic or behavioral performance. Collecting data helps an instructor determine a program s effectiveness. By collecting and analyzing
The Challenge of Helping Adults Learn: Principles for Teaching Technical Information to Adults S. Joseph Levine, Ph.D. Michigan State University firstname.lastname@example.org One of a series of workshop handouts made
NORTHERN VIRGINIA COMMUNITY COLLEGE PSYCHOLOGY 211 - RESEARCH METHODOLOGY FOR THE BEHAVIORAL SCIENCES Dr. Rosalyn M. King, Professor DETAILED TOPICAL OVERVIEW AND WORKING SYLLABUS CLASS 1: INTRODUCTIONS
1. Behavioural Interviewing What is the Behavioural Interview? The behavioural interview is based on the premise that the best way to predict future behaviour is to determine and evaluate past behaviour.
Professional Development Coordinator Competencies and Sample Indicators for the Improvement of Adult Education Programs A Publication of Building Professional Development Partnerships for Adult Educators
Substance Abuse and Mental Health Services Administration Women, Children, and Family Treatment Program The Service Grant for Pregnant and Postpartum Women INSTRUCTIONS Organization Date: Father Friendly
Guidelines for Recruitment Screening Committees I. How to serve as a screening committee member a. Screening committee meetings II. How to reading and evaluating applications a. How to use the applicant
Career Service Authority Workers Compensation Claims Supervisor Page 1 of 6 GENERAL STATEMENT OF CLASS DUTIES Supervises and directs the work of a unit of claims adjusters. DISTINGUISHING CHARACTERISTICS
Career Service Authority Page 1 of 6 Heating, Ventilating, and Air Conditioning (HVAC) Supervisor GENERAL STATEMENT OF CLASS DUTIES Performs supervisory duties over HVAC Mechanics engaged in the operation,
Quick Guide For Administrators Based on TIP 52 Clinical Supervision and Professional Development of the Substance Abuse Counselor Contents Why a Quick Guide?...2 What Is a TIP?...3 Benefits and Rationale...4
Performance Management WORKSHOP HANDOUTS Facilitated by: Tara Kemes, Vantage Point Knowledge Philanthropist June 2013 Page 1 of 16 Handout 1 Performance Management System Overview What is performance management?
Methodology Brief: Introduction to Focus Groups Barry Nagle Nichelle Williams 1 This methodology brief outlines a five-stage process for conducting focus groups and reporting on the results. The five stages
Interpersonal Communication: Key Competencies for Enterprise Project Management Steven Flannes, Ph.D. Principal, Flannes Associates 2080 Mountain Blvd., Suite 205 Oakland, California 94611 USA Telephone:
Module 9 Building Communication Skills Essential Ideas to Convey To apply a facilitative approach to supervision, supervisors have to approach the people they manage in a different way, by using certain
About The IIA Global Internal Audit Competency Framework The IIA Global Internal Audit Competency Framework (the Framework) is a tool that defines the competencies needed to meet the requirements of the
G3658-11 Collecting Evaluation Data: End-of-Session Questionnaires Ellen Taylor-Powell Marcus Renner Program Development and Evaluation University of Wisconsin-Extension Cooperative Extension www.uwex.edu/ces/pdande
MEDICAL ASSISTANT I LOCATION/DEPARTMENT: Back Office DATE DEVELOPED: 9/27/2012 DATE OF THIS REVISION: 02/27/14 REPORTS TO: Lead Medical Assistant SUPERVISORY RESPONSIBILITIES: No supervisory responsibilities.
68 S ECTION 5: DEVELOPING AND IMPLEMENTING AN EVALUATION PLAN EVALUATION METHODS Although indicators identify what you will look at to determine whether evaluation criteria are met, they do not specify
TABLE OF CONTENTS INTRODUCTION... 1 WHAT HAPPENS IF MY CHILD IS HAVING TROUBLE LEARNING IN SCHOOL?... 2 STEPS TO GETTING SERVICES... 3 ANSWERS TO FREQUENTLY ASKED QUESTIONS... 9 REQUEST FOR ASSISTANCE...
Identification of Quality Characteristics for Technology Education Programs: A North Carolina Case Study Aaron C. Clark and Robert E. Wenig Since its beginning, technology education has consistently pursued
Illinois Professional Teaching Standards Preamble: We believe that all students have the potential to learn rigorous content and achieve high standards. A well-educated citizenry is essential for maintaining
Agricultural Occupational Health Services and Delivery: Alternative Strategies to Deliver/distance Learning Dr. Carol J. Lehtola, Ph.D. Extension Agricultural Engineering Safety Specialist Department of
In general, people want to feel that they have been treated fairly and feel that they have been understood and respected, regardless of what is being communicated. The ability to listen respectfully can
Metropolitan State University of Denver Master of Social Work Program Evaluation Date: Agency/Program Task Supervisor Faculty Liaison Total Hours Completed To Date for this semester: s will not receive
GUIDELINES FOR THE IEP TEAM DATA COLLECTION & Progress Monitoring Decisions about the effectiveness of an intervention must be based on data, not guesswork. Frequent, repeated measures of progress toward
Service Delivery Guideline for Service Providers: Creating Effective Welcoming Spaces March 2012 Table of Contents Background... 2 1. Service Standards... 4 Guiding Principles Welcoming Clients Needs Assessment
EVALUATION METHODS TIP SHEET QUANTITATIVE METHODS: Quantitative data collection methods consist of counts or frequencies, rates or percentages, or other statistics that document the actual existence or
SALEM AREA ONE COMMUNITY INITIATIVE Mission Statement To develop a safe, respectful, all-inclusive community through education, advocacy, and coordinated actions that include an effective response to prejudice
Self Study and Training for Members and Staff of Agricultural Cooperatives A Guidance Manual for Advisers and Trainers This manual offers guidance to field workers involved in advising and providing training
Job Search Resources for International Students Returning Home The differences from a United States job search Are you considering returning home after your studies? In addition to preparing for your relocation,
Department of Educational Leadership & Policy College of Education & Human Services University of Wisconsin Oshkosh Critical Analysis of Systemic Inequity: Social Justice Education 3 credits EDL 820 Course
IMPROVING QUALITY Quality criteria for global education school visits Organisations that have worked on these quality criteria: Pro Ethical Trade Finland Kepa Service Centre for Development Cooperation
TRAINING AND RESOURCES TOOLKIT Welcome to the training and resource toolkit. The Training Overview recommended a five step process to develop training and resources. In addition to following the process,
6 Maintaining Positive Relations with Vendors This chapter will enable you to: Be aware of the role of the manager and the market in the success of vendors. Be aware of actions you can take to maintain
Interview Questions for Nursing Posts CC 33 Careers Centre Always remember that an interview is a formal process. Even if you have had a clinical placement on a ward where you are being interviewed for
Module 6; Managing Large Classes Approaches to Language Teaching: Extension; Video Length: Approximately 11 Minutes; Notes to the Trainer; For best results, have participants go through the readings for
JOMC 279: Advertising and Public Relations Research Methods Fall 2015 Class Time: 8:00am to 9:15am, Tuesdays and Thursdays Room: Carroll Hall 33 Instructor: Joseph Cabosky, J.D., Ph.D. Office Hours: 9:30am
Prepared Public Speaking Purpose: The purpose of the Prepared Public Speaking CDE is to allow students to develop a speech pertaining to a topic in agriculture. It aims to provide the opportunity for personal
TRAINING NEEDS ANALYSIS WHAT IS A NEEDS ANALYSIS? It is a systematic means of determining what training programs are needed. Specifically, when you conduct a needs analysis, you Gather facts about training
Administrative Support Professionals Competency Framework The Centre for Learning and Development Table of Contents 01. Acknowledgements...3 02. Introduction...4 03. Background...5 04. Competency Assessment
HOW TO ORGANISE AND RUN FOCUS GROUPS BACKGROUND Focus groups can form an important part of the process of using the Management Standards to assess and control risks related to work-related stress. Focus
Planning and Conducting an Effective Meeting The Top Ten Reminders for Running an Effective Meeting 1. First decide if you need to have a meeting. Often times there are more efficient ways to meet objectives
Copyright 2005 by Washington Office of the Superintendent of Public Instruction. All rights reserved. Educational institutions within the State of Washington have permission to reproduce this document.
Quality Care: Foster Care Training - Orientation Trainer s Notes Time Resources Method of delivery Learning outcomes Assessment 3 hours Handouts Paper/felt pens Lecture, activity. This module can be presented
The Directive Supervision Employee Handbook Agency Name Patricia Miles Table of Contents Table of Contents... 1 Introduction:... 2 History of the Directive Supervision System:... 2 Defining the Directive
Guide to Using Results Contact Information For assistance, call ACT Engage Customer Service at 319.337.1893, 8:30 a.m. 5:00 p.m., central time, Monday through Friday, or email email@example.com. Resources
PERFORMANCE APPRAISAL WHAT IS A PERFORMANCE APPRAISAL? The Appraisal Process PHASE 1 Identification and Observation of Behaviour PHASE 2 Measurement of Performance PHASE 3 Development of future Performance