AES Queensland Branch Seminar 22 October Ellen Vasiliauskas Director d-sipher pty ltd. ph:

Transcription

1 AES Queensland Branch Seminar 22 October 2009 Ellen Vasiliauskas Director d-sipher pty ltd ph:

2 What is modelling? Who uses modelling? Why use modelling? Problems of not using modelling Types of models - learnings from other sectors & ways models can be developed Using your RAIDAR case study Application of models to evaluation Cautions using modelling Final word from Alfred Korzybski

3 At its simplest, modelling is the use of visual representations to communicate.

4 Visualization through visual imagery has been an effective way to communicate both abstract and concrete ideas since the dawn of man. Used in mathematics, architecture, computer sciences, engineering, & business process modelling as well as social sciences.

5 ..uses mathematical language to describe a system. 'a representation of the essential aspects of an existing system (or a system to be constructed) which presents knowledge of that system in usable form. Eykhoff s definition of a mathematical model (1974) downloaded 21/10/2009

6 A model is a simplified abstract view of the complex reality. A scientific model represents empirical objects, phenomena, and physical processes in a logical way. Attempts to formalize the principles of the empirical sciences in the same way logicians axiomatize the principles of logic use an interpretation to model reality. [the] aim is to construct a formal system for which reality is the only interpretation. The world is an interpretation (or model) of these sciences, only insofar as these sciences are true. a way in which the human thought processes can be amplified. downloaded 21/10/2009

7 process of generating abstract, conceptual, graphical and/or mathematical models. a way to read elements easily that have been broken down to the simplest form. an essential and inseparable part of all scientific activity little general theory about scientific modelling new fields like knowledge visualization. downloaded 21/10/2009 Modelling part of the science of evaluation

8 The Chunking Limit We understand and remember 7+/- 2 chunks From Harvard psychologist George A. Miller s work on short term memory

9 The Chunking Limit Implications recall and comprehension: Recoding using modelling is a powerful tool for increasing the amount of useable information takes into account reader's comprehension and ability to access and retrieve information. Practical recommendations make it < 9: > nine bullet points on a slide - order information into smaller, related groups > nine objects - use flow diagrams to reflect a model's complexity & divide the diagram into subcomponents.

10 The ESOMAR model shows the move from research findings to marketing intelligence European Society for Opinion and Marketing Research Models [using] software allow [us] to simulate, visualize, manipulate and gain intuition about the entity, phenomenon or process being represented. g downloaded 21/10/2009 Wisdom & Creativity Vision/strategy development Information & Knowledge Decision-making Data & Information From Book-ending the Engine Room: Embedding market research in strategic management and corporate communication. E. Vasiliauskas & P. Bycroft 14 October 1998 presentation to the Australian Market and Social Research Society Annual Conference

11 Brief I want an evaluation of our program The Engine Room Proposal What questions would you like? Data gathering Users are respondents not resources Analysis Static, descriptive Output Basic reporting, mystification of results, few actionable recommendations Adapted from Book-ending the Engine Room: Embedding market research in strategic management and corporate communication. E. Vasiliauskas & P. Bycroft 14 October 1998 presentation to the Australian Market and Social Research Society Annual Conference

12 Strategic research model WISDOM AND INTELLIGENCE WISDOM Intuition, understanding, creativity, value, foresight INTELLIGENCE Contextual knowledge, intellectual capital, innovation From Book-ending the Engine Room: Embedding market research in strategic management and corporate communication. E. Vasiliauskas & P. Bycroft 14 October 1998 presentation to the Australian Market and Social Research Society Annual Conference

13 Strategic research model STRATEGIC POSITION Applied wisdom and intelligence STRATEGIC ALIGNMENT KNOWLEDGE MANAGEMENT Information, integrated modelling, communication From Book-ending the Engine Room: Embedding market research in strategic management and corporate communication. E. Vasiliauskas & P. Bycroft 14 October 1998 presentation to the Australian Market and Social Research Society Annual Conference

14 Strategic research model WISDOM AND INTELLIGENCE STRATEGIC ALIGNMENT Brie f Propos al The Engine Survey Room Analysis Output Upstream value adding From Book-ending the Engine Room: Embedding market research in strategic management and corporate communication. E. Vasiliauskas & P. Bycroft 14 October 1998 presentation to the Australian Market and Social Research Society Annual Conference

15 Strategic research model COMMUNICATION STRATEGIC RE-ALIGNMENT CULTURAL CHANGE PERFORMANCE IMPROVEMENT IMPROVED BUSINESS PERFORMANCE Downstream From Book-ending the Engine Room: Embedding market research in strategic management and corporate communication. E. Vasiliauskas & P. Bycroft 14 October 1998 presentation to the Australian Market and Social Research Society Annual Conference

16 Strategic research model WISDOM AND INTELLIGENCE STRATEGIC ALIGNMENT Brief Proposal The Engine Survey Room Analysis Output PERFORMANCE IMPROVEMENT FEEDBACK Requires high level models of organisational dynamics From Book-ending the Engine Room: Embedding market research in strategic management and corporate communication. E. Vasiliauskas & P. Bycroft 14 October 1998 presentation to the Australian Market and Social Research Society Annual Conference

17 S.W.O.T. Program logic Value chain analysis (M. Porter) Communication models e.g. Diffusion of Innovation (E. M. Rogers) Process mapping internal systems & mapping the experience from the user s perspective Hierarchy of ideas models Hierarchy of effect models - AIDA and RAIDAR

18 Creative Expression Painting Sculpture Ceramics Film Writing Medium Oil Acrylic Watercolour Style Landscape Abstract Still life Religious Portraiture Type Individuals Groups of people Couples Self-portraits Animals

19 ASK: Why are we doing this? For what purpose? What is the intention? CHUNK UP Move from specific to general category For agreement and alignment CHUNK DOWN Move from to general category For specificity and How to ASK: What or who specifically? How do we do this? What are examples of this?

20 Developed in 1898, by E. St. Elmo Lewis - American advertising and sales pioneer. AIDA funnel model created to explain the mechanisms of personal selling. Assumes buyers go through four cognitive phases when accepting a new idea or purchasing a new product. Showed importance of timing to leverage the mental state or values of users as they progress through the AIDA stages (or over time). Basis for measuring effects of advertising...foundation for 'hierarchy of effect' models.

21 R.A.I.D.A.R. Research and Results component of AIDA developed by Peter Bycroft RESEARCH AWARENESS INTEREST DECISION (DESIRE) ACTION RESULTS (OUTCOME) demographic trends, intelligence about our user/client, other programs, our program and service how do we make them aware of our presence, the program and services? relevant to user needs, catches their eye, what gets them interested in it? what triggers their decision to purchase/ engage? what happens once we have established a user/ provider relationship the user lifecycle? repeat use, user expectations met, cost effective, program success, outcomes achieved

22 R.A.I.D.A.R. RESEARCH Useful as guideline for developing communication strategies and user engagement. AWARENESS INTEREST DECISION (DESIRE) ACTION If the program is an innovation, it requires building awareness. If the program is established, it may require building desire. Users may require different information at each stage to be able to move to the next stage. RESULTS

23 Study in Australia Website Evaluation By d-sipher and Corporate Diagnostics

24 RESEARCH Who is using the SIA website? AWARENESS INTEREST Gender Age Country/Continent Student or other type of user Which study opportunities in Australia are users most interested in? DECISION ACTION RESULTS

25 RESEARCH AWARENESS INTEREST DECISION ACTION RESULTS How do users find out about the SIA website? What is the indicated level of awareness of the site? How did users find out about the SIA Website? Why are users using the SIA Website? How many times have users visited the SIA Website?

26 RESEARCH AWARENESS INTEREST DECISION ACTION RESULTS What attracts and holds the user s interest? What study opportunities are users interested in? Which countries are users considering/have studied in? Do study interests vary by country? Do study destinations vary by country? How well does the website hold their interest?

27 RESEARCH AWARENESS INTEREST DECISION ACTION RESULTS What influences students in their decision making? What issues influence students choice of studying in Australia? Do these differ by country? What drives user satisfaction with the Study in Australia website? Do these drivers differ by country?

28 RESEARCH AWARENESS INTEREST DECISION ACTION RESULTS What happens as a result of using the SIA Website? How easy is it to use the site? What do users think of the Website tools? How effective is the Website in meeting users needs? Does the Website help students make a decision about study in Australia?

29 RESEARCH AWARENESS What outcomes has the SIA website achieved? INTEREST DECISION ACTION How well does the website position Australia as a study destination? How does the website compare with other sites? Would users recommend this website to others? RESULTS

30 Study-in-Australia Website: Ten User-based Performance Indicators Quality of Content Ease of use SIA Information Overall satisfaction The Look Clarity of Communication General Information about Australia Overall Satisfaction with the SIA Website Navigability User Engagement Effectiveness of the Website Comparative Performance Positioning Australia

31 Evaluation inception Evaluation implementation Evaluation conclusion A way to conceptualise and simplify complex elements. Show links to corporate logic Often developed from a user, client or staff perspective. Post-Evaluation

32 Evaluation inception Evaluation implementation Evaluation conclusion Post-Evaluation To conceptualise and frame up the evaluation. To simplify complex evaluations. To inform the evaluation plan. To ensure all aspects & issues to be evaluated are captured. To develop a hierarchy of evaluation issues e.g. what must be evaluated & what s unnecessary detail? To gain agreement. For clarity of steps and details. To avoid the shopping list evaluation. To show preliminary links to strategic plans: Why are we doing this? How are we doing it?

33 Evaluation inception Evaluation implementation Evaluation conclusion Post-Evaluation Identifies internal processes. Maps a user based experience: e.g. at which points and how users/clients engage with or flow through the process or program e.g. engagement points or doorways the user needs to or can open to engage with the program. Identifies how the user is channelled to desired doorways and through pathways. Complements and aids understanding to the input element of program logic. Refines & tests the model, concepts and assumptions through: stakeholder and user feedback multivariate analysis techniques in quantitative surveys.

34 Evaluation inception Evaluation implementation Evaluation conclusion Post-Evaluation Delivers the refined model. Highlights strengths and weaknesses. Flags issues under key thematic areas. Assists with reporting and communicating results. Focuses recommendation areas.

35 Evaluation inception Evaluation implementation Evaluation conclusion Post-Evaluation Simplifies communication. Assists with internal change, rollout and organisational ownership. Focuses action planning and where accountabilities should lie. Assists with refining management thinking and understanding of clients/program/staff. Basis for developing monitoring frameworks and performance indicators.

36 A picture is worth a thousand words. Potential to communicate powerfully. Simplifies the complex. Provides the basis for change and action planning. Provides a logic for monitoring and performance indicators.

37 All models are imperfect not all things fit, not all things may be able to be incorporated. Models can be applied too strictly & result in rigid adherence rather than innovation. There may be more than one way of modelling the experience or process. Can be done poorly boxes with arrows! Steps are not necessarily sequential.

38 Polish-American scientist and philosopher The map is not the territory

39 AES Queensland Branch Seminar 22 October 2009 Thank you Ellen Vasiliauskas Director d-sipher pty ltd ph: