Assessing an Organization s in Health Communication: A Six Cs Approach [11/12 Version] Name & location of organization: Date: Scoring: 0 = no capacity, 5 = full capacity Category Assessment Indicator As an As a training Means of Measurement As a research implementing organisation I. COMPETENCE: An organization s collective level of available technical expertise needed to design and implement communication programs achieving a consistent level of quality. 1. The organization s ability to follow a systematic framework in designing and implementing communication 1.1.1 Are you aware of any formal behavior change and health communication theories or models such as the Theory of Reasoned Action, the Social Change Theory, Diffusion of Innovation, etc. 1.1.2 Are these theories or models used to guide program and intervention design? Degree of consistency following a systematic framework. [A written expression and checklist based on the framework to follow.] Instructions Composite score for use of a systematic framework is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section If so, which models or theories are you using (please list here) 1.1.3 Are you aware of any planning tools or frameworks, if so, which ones: 2. The organization s ability to conduct an analysis before developing and implementing communication 1.2.1 Do you do a problem analysis to understand what social and behavioral issues can support or block your communication interventions during design and planning? (e.g. social norms, behavioral patterns, policy, legislative or economic factors) Use of data in analysis; performing needs assessments; analyzing resources; familiarity with DHS. [A written analysis document.] Composite score for analysis is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section 1
1.2.2 Do you gather and analyze secondary data (e.g. desk and literature reviews of social, behavioural or epidemiological data, studies)? 1.2.3 Do you collect and analyze your own qualitative data (e.g. focus groups, interviews, observations, or other participatory methods such as social and community mapping, participatory learning for action methodologies)? 1.2.4 Do you collect and analyze your own quantitative data (e.g. surveys and large baseline studies)? 1.2.5 Are you usually satisfied with the collection and analysis of qualitative and quantitative data? 3. The organization s ability to develop a strategic design for communication 1.3.1 Do you set clear social or behavioural targets during planning and design? 1.3.2 Do you select your intended audiences and conduct audience segmentation? 1.3.3 Do you include plans for materials and dissemination channels as part of strategic planning? 1.3.4 Do these plans reflect your communication objectives and do they address obstacles to change and how to communicate key benefits? 4. The organization s ability in development, pretesting, revision, and production of messages and materials in communication SMART objectives; positioning; including an evaluation plan in the design; organizational partners. [A written strategy with an appropriate BCC theory.] Evidence of pre-testing, recording of results, and changes. Composite score for strategic design is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section Composite score for development, pre-testing, revision and production is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section 2
1.4.1 Do you use multiple communication channels in your programs (e.g. interpersonal communication, group approaches, mass media, etc.) to reach the target audience? 1.4.2 Do your communication interventions seek to influence different societal levels (e.eg. individual, family, group, community, and workplace, regional national)? 1.4.3 If you develop your own materials, do you use key elements of SBCC materials and message development (e.g. creative briefs, communication objectives, addressing obstacles to change, communicating key benefits, observing literacy levels, etc.) and do you use those in the design of your materials? 1.4.4 If you develop your own messages and materials, are they reviewed by technical staff and stakeholders to ensure accuracy of health information? If you use materials from other organizations, have you reviewed them for accuracy? 1.4.5 If you develop your own materials, are they developed and pretested with members of the target audience and feedback incorporated into final messages and materials? If you use materials from other organizations, do you pre-test them with members of your target audience? 1.4.6 Are material formats and channels of communication (individual, group, radio, drama, etc) in your program selected with input from target audiences, and are their preferences and access taken into consideration? 5. The organization s ability to manage, implement, and monitor communication Develops/follows a workplan; maintains financial accountability; creates a positive organizational climate; monitors process of dissemination, transmission, reception of program outputs. [Written reports at stated intervals]. Composite score for management, implementation and monitoring is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section 3
1.5.1 Do you conduct periodic (e.g. yearly) reviews of your communication approaches and materials to ensure that they are up to date and relevant to the context and realities? 1.5.2 Do you monitor that the job aids and tools you produce (e.g. manuals, activity guides, flip charts, counseling cards, drama scripts) are used by SBCC implementers to support communication interventions? 1.5.3 Are the data generated by M&E analyzed and fed back to the SBCC implementers? 1.5.4 Are the data generated by M&E used to improve program interventions? 6. The organization s ability to carry out or arrange for an impact evaluation of communication 1.6.1 Does your program have process indicators for SBCC activities that are clearly defined (e.g. numbers of people reached by the program, numbers trained)? 1.6.2 Are there monitoring tools used to track program process and impact? 1.6.3 Do you have indicators to measure changes in SBCC outcomes (e.g., number of clients who are using a long-lasting and permanent family planning method, numbers who sleep under an ITN every night, numbers who use condoms correctly and consistently)? 1.6.4 Is there a system in place to collect outcome data (i.e. short-term or intermediate results of the program through execution of its activities)? Conducts or contracts for evaluations; understands data sources and research design; uses information to improve future [Written evaluation report, publications based on evaluation.] Composite score for evaluation is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section 4
1.6.5 Are quantitative research methods (e.g. surveys) used to measure the social and behavioral outcomes of SBCC interventions? 1.6.6 Are qualitative research methods (e.g. focus groups, interviews, observations) or participatory research methods (e.g. diagramming, community dialogue, participatory learning for action methodologies) used to measure the social or behavioral outcomes of SBCC interventions (e.g. community involvement)? 1.6.7 Are you usually satisfied with the quality (e.g. collection and analysis) of your qualitative and quantitative data that measure outcomes? 7. The organization s ability to achieve objectives in developing and implementing communication activities. 1.7.1 Are project results communicated to key stakeholders, donors and partners? 1.7.2 Are lessons learned and best practices systematically documented and disseminated? Completes projects/programs; reaches objectives, documents results. (COMPETENCE) Subtotal: [MAXIMUM SCORE = 35] Composite score for achieving objectives is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section Add all the composite scores from the gray boxes above Overall for Category 1: Competence Component 1. The organization s ability to follow a systematic framework in designing and implementing communication 2. The organization s ability to conduct an analysis before developing and implementing communication 3. The organization s ability to develop a strategic design for communication Composite as an implementing organisation Composite as a training Composite as a research 4. The organization s ability in development, pretesting, revision, and production of messages and materials in communication 5. The organization s ability to manage, implement, and monitor communication 5
6. The organization s ability to carry out or arrange for an impact evaluation of communication 7. The organization s ability to achieve objectives in developing and implementing communication activities. Overall for Competence Strengthening needs in competence Plans of Action or change in competence Suggestions for technical support in competence II. Category COMMITMENT: An organization s degree of belief in, and responsibility for supporting the use of strategic communication for social change and maintaining acknowledged standards of technique and product. Assessment Indicator 1. The organization s level of adherence to an acknowledged set of standards or guidelines related to the quality of materials and activities. 2.1.1 Does the technical staff have knowledge of and experience in the production of quality SBCC products, based on using a set of standards and guidelines? 2.1.2 Does management staff have the capacity to implement SBCC, either through training or previous experience? As an implementing organisation As a training As a research Means of Measurement Management support for health communication; adoption of local or international communication guidelines. [Mission statement] Instructions Composite score for use of a standards and guidelines is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section 6
2. The organization s human resource commitment to communication for health. 2.2.1 Does technical supervisory staff have the capacity to implement SBCC, either through training or previous experience? 2.2.2 Have SBCC implementers or volunteers been trained in SBCC? (note: the terms implementers or volunteers refer to any person carrying out SBCC activities; similar terms used include peer educators, prevention officers, or community health workers) 2.2.3 Do you conduct an ongoing assessment of staff SBCC competencies? 2.2.4 Is there a plan to strengthening SBCC competencies of staff (basic SBCC training, on-the-job training, etc.)? Is the capacity strengthening plan being implemented? 3. The organization s structural importance placed on communication within overall operations. 2.3.1 Are SBCC implementers or volunteers selected according to clear criteria? Do volunteers have experience with or are they members of the target audience? (Note: the terms implementers or volunteers refer to any person carrying out SBCC activities; similar terms used include peer educators, prevention officers, or community health workers) 2.3.2 Is there a written job description (or list of duties and responsibilities) for SBCC implementers or volunteers? A critical mass of personnel dedicated to communication. [The number of people specializing in various aspects of communication., e.g. mass media, interpersonal, evaluation, etc.] An established communication division or body at a high level within the organization; degree of integration such a body has within the overall organization and access to leadership. Composite score for human resource commitment is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section Composite score for structure reflecting importance of SBCC is calculated by averaging each score for the elements of the indicator and putting the composite score in the gray section 7
2.3.3 Is the program structured in a way that allows for SBCC implementers or volunteers to have repeated contact with the same clients to reinforce messages? 2.3.4 Are there incentives for SBCC implementers or volunteers to encourage them to stay in the program (e.g. transport, SBCC/IEC materials, training, cash incentives, supportive supervision, etc)? 2.3.5 Does the program structure allow technical advisors or supervisors to visit SBCC implementers regularly to identify strengths or needs for further support? 2.3.6 Do the advisors /supervisors visits include visits to the SBCC implementer at his or her site and observation of SBCC service delivery? 2.3.7 Is there a checklist to guide advisor/supervisor visits? Does it include indicators related to the observation of SBCC service delivery? 4. The level of the organization s overall resource base dedicated to communication for health. 2.4.1 Is the level of budget for the organization appropriate to accomplish the SBCC implementation needed? 2.4.2 Is the level of non-material resources appropriate? (COMMITMENT) Subtotal: Level of financial and physical resources devoted to communication. [MAXIMUM SCORE = 20] Overall for Category 2: Commitment Component 1. The organization s level of adherence to an acknowledged set of standards or guidelines related to the quality of materials and activities. 2. The organization s human resource commitment to communication for health. Composite as an implementing organisation Composite as a training Composite as a research 3. The organization s structural importance placed on communication within overall operations. 4. The level of the organization s overall resource base dedicated to communication for health. 8
Overall for Commitment Strengthening needs in commitment Plans of Action or change in commitment Suggestions for technical support in commitment 9
III. CLOUT: Includes the degree to which an organization is able to influence the environment in which it works and access resources from outside. Overall for Clout Strengthening needs in clout 1. The organization s ability to influence key decision-makers in its operating environment. 2. The organization s ability to mobilize resources from outside. (CLOUT) Subtotal: Links/contacts with government and NGO leaders; past successes. The number and extent of donor support. [MAXIMUM SCORE = 10] Plans of Action or change in clout Suggestions for technical support in clout 10
Category Assessment Indicator As an implementing organisation As a training As a research Means of Measurement IV. COVERAGE & COST EFFECTIVENESS: An organization s reach in terms of geographic coverage, subject matter, audience segments, use of multi-media, and ability to track associated costs per person. 1. The organization s reach in terms of geographic distribution. 2. The organization s reach in terms of different subject areas. 3. The organization s reach in terms of different audiences. 4. The organization s reach in terms of multi-media. 5. The extent to which the organization has a budgeting system that allows for disaggregation of expenditures and resources so that inputs can be compared with outputs. 6. The organization s ability to measure the effect of communication (COVERAGE & Cost Effectiveness) Subtotal: Overall for Coverage and Cost Effectiveness Strengthening needs in Coverage and Cost Effectiveness Number of areas and people reached, rural/urban. Number of health subjects addressed through communication projects/programs; e.g. FP, RH, HIV/AIDS, MCH. Number of different audience segments reached; or segments within the primary clients served (e.g. women, men, adolescents, influentials); and are less advantaged groups and individuals most immediately affected by an issue included? Number of media used in communication Keeps track of expenditures; has code systems for projects. Measurement system to obtain data/information related to effects; if not capable of generating such data, has access to source of data/information. [MAXIMUM SCORE = 30] Plans of Action or change in Coverage and Cost Effectiveness 11
Suggestions for technical support in Coverage and Cost Effectiveness V. CONTINUITY: An organization s historical record and ability to perform in the future, adapt to changes in its environment, and sustain its ability to generate internal and outside resources. 1. Past history. Number of years in operation; reputation; awards; and recognition. 2. Potential to continue and expand operations (CONTINUITY) Subtotal: [MAXIMUM SCORE =10] Leadership, staffing and resource development potential, existence of a strategic manual. Overall for Continuity Strengthening needs in Continuity Plans of Action or change in Continuity Suggestions for technical support in Continuity 12
Category Assessment Indicator As an implementing organisation As a training As a research Means of Measurement VI. COLLABORATION: An organization s ability to establish, build and maintain positive working relationships with qualified partner organizations 1. The organization s ability to establish positive working relationships with other organizations. 6.1.1 Are relevant local and national stakeholders (e.g. MOH, NGOs, CBOs, other donor programs) for your program clearly identified, and their activities reviewed for overlap during planning and design? 6.1.2 Do you collaborate and establish partnerships with these stakeholders during planning and design? 6.1.3 Are relationships with these stakeholders continued and nurtured over the course of the program? Number of partnerships; duration of established partnerships; level of involvement/quality of integration of partner organizations (Do their employees serve as committee members for the partner organization or are they irreplaceably involved in the organization on multiple levels serving an integral role in the partner organization?) 2. The organization s ability to mobilize allies. Reputation in the field; ability to form and guide coalitions. (COLLABORATION) Subtotal: [MAXIMUM SCORE =10] Overall for Category 6: Collaboration Component 1. The organization s ability to establish positive working relationships with other organizations. Composite as an implementing organisation Composite as a training Composite as a research 2. The organization s ability to mobilize allies. Overall for Collaboration 13
Strengthening needs in Collaboration Plans of Action or change in Collaboration Suggestions for technical support in Collaboration 14
Summary Table Domain Implementation Training Research Recommendations 1 Competence 2 Commitment 3 Clout 4 Coverage and Cost Effectiveness 5 Continuity 6 Collaboration Organization s TOTAL SCORE: [MAXIMUM TOTAL SCORE = 115] Please complete the following: Date of Assessment: Name of Interviewer(s): Names and titles of individuals present during assessment: 15
16