Teaching Problem-Based Data Analysis to Public Administration Students: A Reinforcement to Statistics and Research Methods in the MPA Program

Network of Asia-Pacific Schools and Institutes of Public Administration and Governance (NAPSIPAG) Annual Conference 2005 BEIJING, PRC, 5-7 DECEMBER 2005 THEME: THE ROLE OF PUBLIC ADMINISTRATION IN BUILDING A HARMONIOUS SOCIETY Workshop on the Teaching of Public Administration and Policy Teaching Problem-Based Data Analysis to Public Administration Students: A Reinforcement to Statistics and Research Methods in the MPA Program Ester L. Raagas, Ph. D. Dean, Graduate School & Coordinator, Public Administration Programs Xavier University-Ateneo de Cagayan Cagayan de Oro City, Mindanao Philippines

I. Introduction The potentials of our MPA (Master of Public Administration) students in undertaking research in policy development, public management, governance, citizen participation, delivery of basic services, program evaluation and the like will remain undeveloped unless our MPA program empowers them. True enough our MPA Curriculum requires instruction in the application of quantitative techniques of analysis in policy and program formulation, implementation and evaluation, in decision making and in problem solving. Most of these areas require the collection and analysis of quantitative data. These are being addressed by such MPA offerings as Statistics for Public Administration (Stat-PA) and Research Methods for Public Administration (RM-PA). However, after taking these two foundation courses along with the major public administration offerings, MPA students are still found wanting in their preparedness to undertake research. It cannot be argued that for decades our tools and methods in teaching statistics and research in the MPA program had been coherent and potent, but they could be hard for beginners to grasp, and they could be difficult for non-experts to use effectively. In my own university for example, STAT-PA is taught with the implicit assumption that it is usually a student s first course in graduate level statistics, possibly with a gap of more than a few years between any course in statistics taken at the graduate level, if at all. The general approach taken to teach STAT-PA and RM-PA courses is more of a lecture type focusing on the important practical implications and interesting applications of these two courses, ensuring that students are exposed to the tools essential to conduct research in public administration. However, fear of research due to inadequacies in statistics continue to persist among MPA students especially those who are working on their thesis requirement. These are all evident to me from my perspective of years as a statistics consultant and coordinator of our public administration program. Most MPA students have forgotten all but a few catch phrases from their research and statistics courses within a year or two of its completion. What needs to be done? In my meeting with other research and statistics professors, we all agreed on the merits of the lecture method to familiarize the MPA students on the basic concepts of statistics and research. These are not enough though. We need to offer a course that would reinforce the current statistics and research offerings of the MPA program where students can put it all together as Hill (2003) pointed out. This paper then describes an MPA problem-based data analysis course that would reinforce students understanding of research and statistics in public administration. This course gives students practical experience by focusing on actual public administration studies and the data analysis aspect. II. A Problem-Based Data Analysis Course 2.1 An Overview of Problem-Based Data Analysis Course This course is primarily anchored on Problem-Based Learning (PBL). Here, a course starts with research studies in public administration rather than with the exposition of disciplinary knowledge. These studies move students towards the acquisition of knowledge and skills through a staged sequence of problems in context, together with the associated learning materials and support from the faculty (Boud & Feletti, 1991). These studies drive the process of learning, hence the starting point of teaching problembased data analysis is a real research study on public administration where students wish to learn more about. The PBL approach is expected to develop among students a set of competencies such as adapting to and participating in change, dealing with problems, making reasoned decisions in unfamiliar situations and reasoning critically and creatively. MPA students can also develop more their critical thinking skills, teamwork skills, time management skills, information gathering skills and even communication skills over and above their computing skills (Hollingshed, 2004). These are vital in undertaking research in public administration not just in one nation s context but also considering global perspectives. These are summarized in Figure 1.

Stat - MPA Teaching Learning Materials (Manual/Sample Research Studies) Computer Technology Self Directed Learn ing Group-Based Learning Research - MPA Problem-Based Data Analysis Course Figure 1. The Problem-Based data Analysis Course As Reinforcement To Statistics And Research Methods In The MPA Program In many ways, PBL can be considered as an implementation of the constructivist model of learning. PBL is a change of understanding of learning from a transfer of information from teachers to students towards social interaction and individual constructing of knowledge. This model asserts that people construct knowledge by making sense in terms of what they already know. According to constructivism, people can only understand what they have constructed themselves (Leidner and Jarvenpaa, 1995). Learning is an active process when learners develop their own mental models. In fact, in PBL, the students own questions, experience, formulations, and conceptions of problems serve as the basis for learning. Furthermore, in PBL, the group is another important resource for the learning process, extending constructivism to include social interaction and collaboration among the students in a group. The success of the design in teaching problem-based data analysis crucially depends on the accompanying teaching materials, the computer technology and the educator s capability. We created inhouse materials to fit the course design in tandem with the studies conducted on policy implementation. Teaching materials such as the Operation Manual For Data Analysis were prepared by the professors who are assigned to teach the course. The manual had an advantage of combining in a single material exposure to a research problem on public administration, its instrument, instruction to encode the responses, computer commands on the statistics, approaches to analyze the data sets according to the research problem, and interpretation of results. Thus, in Problem-Based Data Analysis course, most of the time and efforts of the faculty take place prior to the beginning of the course which is on the development of the teaching/learning materials such as the sample research studies and the major reference material that will guide both the MPA students and the faculty to achieve the objectives of the course.

2.2 Development of Cases and Course Materials In the preparation of this course, I opted to use public administration research studies which were under my supervision as MPA program coordinator. In this way, access to the original data is easy as well as insights into the investigator s reasons for conducting the study and the problems encountered in the preparation and conduct of the study. Another consideration in the choice of the studies includes their relevance and timeliness to public administration in the Philippine setting. These research studies have also been rated above average in terms of overall quality by the panel of examiners in our MPA program. This adds credibility and importance of these studies to the public administration students. The preparation of the major reference material which is in a manual form blends one sample study in public administration and the use of spreadsheet in the computations of common statistical procedures. A problem-based data analysis course when based on spreadsheets makes it more accessible to different types of students and deepens students understanding of statistical tools needed for research in public administration. The manual introduces the sample study (considering its framework, problems to be answered, instruments used in the gathering of data and the like) and the steps involve in data analysis and how to arrive at the required statistics in a very detailed manner including the mouse clicks and/or keystrokes. 2.3 Spreadsheet and the Statistics In Data Analysis It should be noted that this course is also heavily reliant on computer technology which includes accessing on-line data bases of the university and more specifically the spreadsheet manipulation. The spreadsheet itself is a laboratory, one that presents the stages of a calculation, showing lots of details. If the calculation is incorrect, the spreadsheet provides clues to what is wrong. This makes a spreadsheet a vehicle for interactive learning. One benefit of a spreadsheet is that it automates calculation. A spreadsheet provides easy access to descriptive statistics and common statistical tests. For example, just a few mouse clicks and the statistics are all ready by using the Excel Add-In. Instead of spending much time on statistical computations, the students can now focus on the research itself and grapple with the implications of the statistics at hand to the realities of public administration and governance. It is also true that students would then be eager to sharpen their spreadsheet skills looking forward to use spreadsheets not only for their theses but in other courses, improvement in their respective offices and ultimately be part of the research activities along their line of work. Learning statistical computations through spreadsheet allows students to remove the basic barrier in conducting research. With success come positive reinforcement, self-confidence, enthusiasm and the ability to apply the new skills and knowledge in novel ways. 2.4 Organization of the Course The practical implementation of PBL does of course vary. The model adopted for the data analysis course is a combination of self-directed learning and base-group session where the students work in groups of five to seven to finish the whole course. As students pursue solutions to their assigned research studies, each one tends to increase responsibility for his/her own learning. Before each student or group of students start to work, a lecture, or equivalent, is held by the faculty to set the borders of the session to start or to work on the required tasks. This is done to make sure the students do not get lost in working with the research study. This lecture, demonstration on how to manipulate the spreadsheet, or the delicate facilitation is to guarantee that the students cover, for the course, relevant material. The problem-based data analysis course also follows the prescribed 48 class hours per course in a semester. The course is then divided into 16 sessions with 3 hours per session. More than half of the

time, the students are left to themselves, however the faculty and students do meet for a total of 7 sessions distributed in the following activities; Orientation on the course and introduction to public administration research studies. This is to familiarize the students by presenting the objectives, background and other pertinent information of the research study. (1 session) Spreadsheet demonstration on data encoding and generation of descriptive statistics and test statistics ( 2 sessions) Evaluation of students (1 at the middle of the term & 1 at the end of the semester) Demonstration on presentation of research outputs using power point, hyperlinks and accessing on-line data bases to enhance implications and insights to research outputs ( 1 session) Student presentation of research outputs and critiquing ( 1 session) The initial discussion involves understanding the research study. Students are led to recognize how the research questions, the kind of data gathered and the research design influence the selection of statistical methods. The assumptions underlying various statistical methods and ways of testing these assumptions are reviewed and considered. Furthermore, the faculty makes the demonstration on spreadsheet manipulation with the mouse clicks and keystrokes that facilitate statistical computations. The remaining 9 sessions are used by the students to analyze the research study assigned to them. They have the options to do the task initially by themselves and then by groups so deadlines are met punctually. If short questions arise, they can either e-mail or text the faculty for response. III. Experiences with the Course 3.1 Evaluation This course has been presented for six semesters in our MPA program where the class size vary from 10 to 14 students. All of the students had taken the required Stat-MPA and Research-MPA prior to the course. Almost all of them had very minimal knowledge to navigate the spreadsheet although average in internet and word processing. The kind of MPA students enrolled in our university are those who work full time in government agencies and local government units, and like other professionals working toward a graduate degree have considerable personal and professional constraints on their time spent in school. With the problembased approach of teaching data analysis, students can have time options for learning experience. They can even work at home since the computer technology requirement is very minimal. Results of a brief evaluation of the course are shown in Table 1. There is a general trend in their appreciation of the course, most particularly in the integration of Statistics, Research and the Table 1. Results of Evaluation On the Problem Based Data Analysis Course (n=34) Areas Comments 1. Objectives of the Course 97% of the respondents indicated that the objectives of the course were achieved. 2. On the manual very useful; empowers us to analyze our own data provides basic steps in a way that even neophytes can learn fast computer applications specially spreadsheet operations simple and direct in instructions; user friendly very useful but not perfect; statistical interpretations of test results should be included

tailored to computer illiterates ; thanks to the authors! 3. On the research study assigned 4. On Statistics and Research Continuation of Table 1 5. On Group-Based Sessions the research study is a learning experience; the research process, content, statistical results, & insights are very interesting and revealing; enables us to be open-minded to different results of the study hence minimizing biases gave us insights on the government s policies on merit promotion learned a lot especially on organizing the data learned a lot from the research study; gave me a feel on how to go about thesis writing especially on analyzing the data a little only, I have difficulty looking for the theoretical framework and insights to results; I have difficulty using on-line data bases. we are now trained how to handle the data gathered and interpret accordingly stat computations and data analysis are no longer intimidating and threatening complements and integrates concepts in Stat and Research I can now relate my learning in Stat with Research enhanced learning on basic concepts of research; learn more on the use of statistics in research learned how to do research reinforced by on-line data basis for non-math minds, statistics become doable; learned how to conduct research that can lead to sound and valid findings learned a lot, but I have limited time to establish the theoretical framework strengthened my stat capability; results can be checked easily my first time to access research journals on-line work cooperatively and learn to take care of group mates best ideas come from the group learned more through my group mates; they helped me understand concepts/applications I can t manage working with my group enhanced my learning very much; this is where I have the most overwhelming experience since coming back to school after long years of not in school Established interdependence/sharing with others I would suggest individual work; other group members tend to depend on other group mates especially in the preparation of group presentation

6. On Self-Directed Learning 7. On Computer Technology 8. Other comments on the Course learn to develop patience, discipline, resourcefulness and perseverance while going through the assigned research study I have to practice the use of computer allows for personal learning through trial and error I have to learn by myself because my group mates are busy; need to prepare myself for the group session made me aware of the statistical features of Excel enhanced my knowledge of Excel; I was introduced to on-line data bases of the university on-line data bases reinforced my insights to the findings; research efforts in other countries can inspire similar studies in the Philippine setting realize the importance of Excel in analyzing data; but I have difficulty using the electronic journals before I can only open Excel but did not know how to use it, now I can use Excel in many papers that I do; printed references are now reinforced with electronic materials; found out Power Point is easy in a scale of 1 to 10, I would give an 8 rating because the course is useful, informative, challenging and interesting; however, it demands more time from us, very difficult to give what we do not have time the professor has reduced students fears and biases towards statistics, research and computer usage; very good rating to the course, it has empowered us very helpful in our own research study and personal activities this course is a vehicle in making our graduate student life easier Computer Technology. However, the results would indicate difficulty in achieving better time management with respect to the course. This is very evident in their difficulty in using on-line data bases of the university. This needs time to sift through the materials while the statistical computations posed no problem because most have PC s at home or notebooks. Most of the MPA students later consulted with me concerning the analysis of data for their theses. They displayed less statistics anxiety compared to the previous students who have not taken the course. For the latter group, consultations would either begin by this question Madam, I am very poor in statistics, do you think I can finish the thesis requirement? or worst Is there anybody you can recommend who will analyze my data for a fee?. With those who had taken the problem-based data analysis course, consultations are more on the research itself and confirmation that the statistical tools used are most appropriate. This is a much improvement to the very extensive and time consuming research and statistics consultation prior to the offering of the data analysis course. I guess this is the area where the course has helped us most in the MPA program. Other than promoting MPA students capability for self-directed learning, the students preparedness to undertake researches on public policy and governance is greatly enhanced. There is also better appreciation of the computer technology in facilitating research activities.

3.2 Challenges Challenges were also encountered in teaching the problem-based data analysis course. One was that most of the students need more than the prescribed time to assume complete responsibility for investigating all learning issues related to each research study which can include any or all of the following; public administration theories involved in the study, computer manipulation for the required statistics, graphs and tables, and the time to access and examine the materials in the university s on-line data bases. The faculty may also spend time answering short queries of the students sent through e-mail or text messages. Otherwise the faculty had to meet a group at a time to provide essential supplementary information without encouraging passive student behavior. IV. Summary and Conclusions Our experience suggests that the use of well prepared teaching materials and actual studies in public administration in conjunction with common statistical software/program helps overcome the traditional barriers that challenge the teaching and learning of statistics and research methods in public administration. Learning in Problem-Based Data Analysis appeared to permit a more thoughtful and detailed examination of the concepts taught in Stat-PA and RM-PA. These lead to better preparedness of MPA students to conduct researches in public administration (Figure 2) and management in general and researches in their respective agencies in particular. References Boud, D. and Feletti, G. (1991). The Challenge of Problem-Based Learning. New York: St. Martin s Press. Bridges, Edwin M. and Philip Hallinger. Using Problem-Based Learning To Prepare Educational Leaders, Retrieved July 15, 2005, Wilson, from: http://library.xu.edu.ph/index.htm. 1997 Hill, Carolyn (2003). Can They Put It All Together? A Project for Reinforcing What Policy Students Learn In A First-Semester Quantitative Methods Course. Journal Of Policy And Management, pp. 473-481. Hollingshed, Maria Terese M. Students Perceptions Of Problem-Based Learning Within An Introductory Computer Application Course, Retrieved August 20, 2005, ProQuest Digital Dissertations, from: http://library.xu.edu.ph/index.htm. 2004 Nulden, Urban and Helana Scheepers. Interactive Multimedia And Problem-Based Learning: Experiencing Project Failure, Retrieved July 8, 2005, Wilson, from: http://library.xu.edu.ph/index.htm. 1999

Stat - MPA Research - MPA Teaching Lear ning Materials (Manual/Sample Res earch Studi es) Computer Technology Self Directed Learning Group-Based Learning Other MPA Offerings Problem-Based Data Analysis Course Research Preparedness of MPA students [ Thesis Writing Capabilities ] Figure 2. Major Objective of Problem-Based Data Analysis Course