Running head: RESEARCH PROSPECTUS 1 RESEARCH PROSPECTUS. Sylvester Ngoma. Information Technology Educator. Harding University High School

Transcription

1 Running head: RESEARCH PROSPECTUS 1 RESEARCH PROSPECTUS Student Success: An Investigation into the Link between Student Information Systems and Student Academic Achievement Sylvester Ngoma Information Technology Educator Harding University High School Ph.D. Candidate nngoma@carolina.rr.com Date: December 10, 2011

2 RESEARCH PROSPECTUS 2 Table of Contents Introduction. 3 Problem statement...4 Purpose statement...5 Conceptual framework...5 Research questions.6 Case significance...6 Potential contributions...7 Literature review 8 Methodological approach Sources of information and measurement plan.10 Data collection plan..10 Data analysis plan and coding scheme...10 Credibility, generalizability, and bias...14 Data management plan...14 Computer application strategy..15 Assumptions and limitations.16 Ethical considerations...16 Expected outcome.17 Conclusion.17 References.19 Appendices: Data collection instrument...22

3 RESEARCH PROSPECTUS 3 Introduction With the growing demand for accountability brought about by different state and federal government initiatives, many school districts have adopted a number of student information systems (SISs) to manage student academic data. Educational experts contend that proper usage of student academic data subsequently leads to better decision-making. McIntire (2004) argues that the increasing demand for accurate, timely data means schools and districts are relying heavily on SIS technologies (p. 9). Once data are gathered, their analysis is mission-critical for informed decisions. Bernhardt (2006) posits that School districts that gather, analyze, and use information about their organizations make better decisions, not only about what to improve, but also how to institutionalize systematic improvement (p. 2). Admittedly, Student Information Systems have become essential pedagogical tools in K-12 schools. SIS technologies have brought significant changes in the way schools handle student data and interact with the community. Over the last two decades, Student Information Systems have been linked to improved student academic achievement and increased parent-school collaboration. A vast body of literature supports the view that SISs affect student performance (Visscher and Bloemen, 1996; Bernhardt, 2006). Essentially, the use of SISs in K-12 schools has a positive impact on educational outcomes. Visscher and Bloemen (1999) observed that worldwide, the use of computerized student information systems has become very important for the management of educational institutions (p. 172). However, the beneficial effects of SISs have been reportedly relative. School districts in the United States have implemented a number of SIS technologies with varying degrees of success.

4 RESEARCH PROSPECTUS 4 Over the last decade, Charlotte-Mecklenburg Schools (CMS) the second largest school district in North Carolina has been serving an increasing number of students. With increasing student enrollment came the challenge of managing a significant amount of student data. In 2004, CMS adopted and deployed a state-mandated Student Information System (SIS) known as North Carolina Window on Student Education, commonly referred to as NCWISE. NCWISE is an electronic student information system that is based on esis, a web-based software package that provides student and school information management capabilities. It is a statewide information management and reporting system that provides teachers, principals, counselors, nurses, central office staff and parents with immediate access to data on students in the North Carolina school system. NCWISE was designed as a support to accountability management for federal and state reporting requirements associated with No Child Left Behind, and the ABCs Accountability program. Problem Statement The importance of SISs in education has received much attention over the last three decades (Keen, 1981; Barrett, 1999; and McIntire, 2004). Research has shown that an effective SIS can enhance student learning in all areas by engaging parents and students in a collaborative partnership. Studies have examined the design and effectiveness of SISs, including factors affecting the adoption of an SIS (Visscher and Bloemen, 1999; Barrett, 1999; Bernhardt, 2006). However, there has been a dearth of scholarship that investigates scientifically the systematic link between an electronic Student Information Systems and student academic achievement. The proposed qualitative study seeks to be distinctive in that regard by depicting the scientific association between an SIS and student attainment.

5 RESEARCH PROSPECTUS 5 Purpose Statement Designed to be an exploratory qualitative study, the main purpose of the research is to determine the relative effectiveness of student information systems in improving student academic achievement and boosting parental involvement in education. The ultimate goal of an SIS is to increase school-parent collaboration, thereby, improving student academic performance and student conduct. Conceptual Framework The conceptual theoretical framework guiding this study rests on the interplay between a successful planning and implementation of an SIS and student academic achievement. It is conceptualized as a strategic planning framework that should guide the selection, adoption, and deployment of an effective and cost-efficient SIS capable of improving student learning throughout Charlotte-Mecklenburg Schools. The framework will combine all existing SISs into one consolidated and integrated Student Information System. The three main points of the theoretical framework include (1) SIS Initiation and Planning, (2) SIS Implementation, and Student Academic Achievement. Figure 1: Ngoma s conceptual framework. SIS Initiation and Planning Current Situation Review SIS Requirements Expected Outcomes S I End- Users SIS Integration Experience and Knowledge Sharing SIS Developers Academic Achievement S SIS Testing District CTO/CIO/ Tech Team Review of Architecture SIS Implementation

6 RESEARCH PROSPECTUS 6 It is assumed that a properly designed and developed SIS tailored to the needs of CMS students may result in increased student achievement. It is imperative that end-users, the Chief Information Officer (CIO), the Chief Technology Officer (CTO), Technology Team, SIS Developers, and educational planners engage in a constant give and take with regards to the design and effectiveness of the SIS. Research Questions The central research question (RQ) that this study will address is: How does a student information system, as form of student data management system, improve student academic achievement? This question will be broken into five investigative questions (IQ): IQ1: What factors prevent the design of efficient and effective Student Information Systems? IQ2: What, if any, barriers exist that prevent student information systems from increasing parental involvement? IQ3: What, if any, Information Systems management impediments exist to improving the adoption process of SIS? IQ4: What processes, policies, or standards are in place to facilitate the adoption of Student Information Systems in schools? IQ5: How effective is a student information system in improving student academic achievement? These questions are designed to capture the essence of users perceptions of current SISs used in CMS schools. Case Significance As indicated earlier, school districts that use data management systems effectively make better informed educational decisions. As McIntire (2004) suggests, Today, the SIS is more central than ever to schools and districts. The changing technology landscape, coupled with the

7 RESEARCH PROSPECTUS 7 data reporting demands of No Child Left Behind has compelled districts to evaluate all of the major software systems, including the SIS (P. 9). Scholarship has shown that SISs play an essential part in increasing parent-school collaboration, and thereby, affecting student performance. Bernhardt (2006) contends that It is simply no longer an option not to have one (p. 358). Barrett (1999) observes that one of the problems facing school districts is that they may be installing elaborate student information systems without adequate strategies or knowledge about how to use them effectively or the extent of their effects on the functioning of the school system" (p. 5). Hence, a cost-effective SIS will serve as an aid that helps teachers and educational leaders implement instructional strategies that can enhance student learning. This study suggests that the use of an effective and efficient SIS is positively associated with positive educational outcomes in Charlotte-Mecklenburg School district. Potential Contributions There is scarcity of scientific evidence in the literature regarding the effects of SISs on educational outcomes in secondary schools (Visscher and Bloemen, 1999). The proposed study endeavors to elicit holistic understanding of the scientific link between a SIS and student academic achievement. It will prove that an effective SIS can increase parental involvement, improve student conduct, and can help explain the achievement gap in student performance. Moreover, this study will help diagnose causes of failure and success in student achievement. With the growing strategic importance of SIS, school districts are increasingly adopting and implementing electronic student information systems. As of , nearly all school districts maintained at least some student data electronically (U.S. Department of Education, 2008). Despite the widespread use of SISs, scientific studies have yet to measure the extent of the relationship between SISs and student achievement and parental involvement. Most studies

8 RESEARCH PROSPECTUS 8 tout the value of SISs in boosting student achievement but fail to account for the existence of the link. This study will examine the link between student information systems and student performance and parent involvement, and will show how SISs affect learning outcomes. Literature Review A Student Information System is commonly defined in various ways. Barrett (1999), Assistant Superintendent of Technology for Conroe Independent School District in Conroe, Texas, views a student information system (SIS) as an integrated software package that maintains, supports, and provides inquiry, analysis, and communication tools that organize student accountability data into information to support the educational process (p. 4). Simply stated, an SIS is a secure, centralized data system where public school information is stored, accessed and analyzed (The IDANET Steering Committee, 2003, p. 2). The above definitions emphasize the role of an SIS as an organizational and analytical tool of student data. The first generation of SIS technologies were mainframe programs that managed student academic records. They were very static. Today s SIS technologies provide client-server solutions (McIntire, 2004, p. 9) and are more central than ever to schools and districts. The author notes that the new centralized Web-based student information systems can share data with other critical administrative applications such as transportation, special education, or food service. He adds that the latest generation of systems provides options for communicating important academic information to parents and students via the Internet (p. 9). Ultimately, a student information system is expected to perform seven key functions: 1) collect student data, 2) increase parental involvement, 3) analyze and measure comprehensive student data, 4) make informed decisions based on results of data analysis, 5) identify learning problems, 6) create personalized education plans, and 7) diagnose student learning styles.

9 RESEARCH PROSPECTUS 9 Recently, school districts have been under considerable amount of pressure to adopt student data systems which promote student learning and parental involvement. Wayman (2005) points out that accountability mandates have drawn attention to the practical use of student data for school improvement. A familiar example is the 2002 No Child Left Behind (NCLB) legislation, which mandates a significant increase in the gathering, aggregation, and upward reporting of student-level data (Wayman, 2005, p.3). Today s technology landscape coupled with federal accountability legislations No Child Left Behind and Race to the Top has compelled several school districts to adopt SIS technologies. Methodological Approach & Research Design A Grounded theory orientation was chosen to attempt to generate theoretical explanations about subjective meanings and experiences of student information systems usage from the respondents perspectives. As Strauss and Corbin (1990) noted, grounded theory can be used to uncover and understand what lies behind any phenomenon about which little is yet known. It can be used to gain novel and fresh slants on things about which quite a bit is already known (p. 19). The study will target individuals who have experienced the Student Information System (SIS) phenomenon. The driving premise of the study is that improvement of student performance can be achieved mainly through the use of an effective and efficient student information system. The research design for the proposed study will encompass semi-structured interviews. Phone and onsite semi-structured interviews will be conducted because they allow some flexibility to the researcher. Essentially, data will be constructed through observations, interactions, and data gathered about Student Information Systems.

10 RESEARCH PROSPECTUS 10 Sources of Information & Measurement Plan Data will be collected from a stratified purposeful sample drawn from a population of Charlotte-Mecklenburg Schools (CMS) teachers, administrators, and parents with past or present experience with student information systems (SISs). A stratified purposeful sample, as Creswell (2007) puts it, illustrates subgroups and facilitates comparisons (p. 127). Thirty informants from Charlotte-Mecklenburg Schools will be randomly sampled for this study: 10 teachers, 10 administrators, and 10 parents. Socio-demographic characteristics such as age group, gender educational level, education level, years of teaching, years of experience with SIS, and college major will be considered. Since Charlotte-Mecklenburg School district is subdivided into four learning communities, schools will be selected evenly to ensure representation of each learning community and to reduce the potential of researcher-interviewer bias and interviewer effect. Data Collection Plan As indicated earlier, in-depth semi-structured interviews will be employed as the method of data collection. Each interview should last an hour. Most interviews will be conducted on the phone but a few will be conducted on site. They will be recorded in a digital voice recorder and stored electronically in WMA format. They will then be transcribed in a word processing program (MS Word) and coded. Study participants will be CMS end-users who use the student information systems mandated by the school district. Data Analysis Plan and Coding Scheme This study will adopt a thematic analysis of data. It will focus on recurring themes and patterns. Coding will play a critical role in the analysis stage of the data collected from the study participants. Data coding will follow Creswell s (2007) linear, hierarchical approach. It is a three-step process that consists in (1) organizing and preparing data for analysis, (2) reading

11 RESEARCH PROSPECTUS 11 through all data, and (3) coding the data. A preliminary coding scheme will be developed and refined. Emerging and recurring themes or ideas will be grouped in different categories. A typology of SIS users perception will emerge from this categorization. An interview code list will be an important tool for data collection. Figure 2 graphically depicts the relationships between preliminary codes, categories, and themes. Glover (2005) describes a web-based SIS as a path to parental involvement. Figure 2: Data Analysis Process Preliminary codes SIS Type School Strategy SIS Management Challenges SIS Adoption SIS Development SIS Grade Management SIS Behavior Management SIS End-Users SIS Architecture Requirements Refined Core Codes features & Themes Design Effectiveness Integration SIS Management Maintenance Improvement Upgrade Fixing Testing Student Records Behavior Management Grades School Leadership Student Learning End-Users Training Feedback Evaluation

12 RESEARCH PROSPECTUS 12 The simplest form of the model of the effects of an SIS on student learning is presented in figure 3. The approach taken for this causal network is deductive or conceptual. Following Miles and Huberman s (1994) guidelines, the constructs were enumerated, and then matched with the body of field data. A logical chain of evidence will be used to confirm or test the proposed study conclusions. It shows how various factors are interrelated and how they can lead to a successful implementation of a student information system. The causal network presented in figure 4 seems most appropriate for data analysis because it displays the relationships among the most important independent variables and dependent variables (Miles and Huberman, 1994). It is assumed that a student information system increases parental involvement. Parental involvement increases student motivation, which in turn, increases student academic achievement (Vogt, 2010, p. 46). Figure 3: Ngoma s model of the effects of an SIS on student learning Motivation Student Information System Parental Involvement Student Academic Achievement

13 RESEARCH PROSPECTUS 13 Figure 4: Ngoma s Causal Network for SISs in Charlotte-Mecklenburg Schools Identify problem with SIS prototype Revise and enhance SIS SIS Implementation Expected outcomes Prototype development Test SIS Prototype SIS Adoption Impact on student conduct SIS Review District/CMS endorsement Student access to academic information Student motivation Need for SIS Parent social support Improved performance State legislators recommend new SIS State/NCDPI mandate Progress Tracking End-users feedback End-users not consulted SIS Upgrade SIS Overall effectiveness review

14 RESEARCH PROSPECTUS 14 Figure 5: Ngoma s Chain of Evidence Supporting the success of an SIS. 1. High SIS Adoption 2. High Implementation Process Features -Amount of training -Period of SIS operation 3. High SIS Flexibility and usability 5. Low Web access to SIS by students and parents 6. High SIS compatibility with platform 7. Low SIS maintenance 4. High SIS Use Features 1. Use to support parent-school communication 2. Extent of use by parents and teachers 8. High Student Performance (Improvement) The chain of evidence helps a researcher gain an integrated understanding of the relationships between different variables. It will be used to validate the tactics adopted for the study. Credibility, Generalizability, and Bias The use of triangulation multiple sources of data, multiple methods, and multiple investigators to confirm the emerging findings (Swanson and Holton (2005) will aid in ensuring the validity, reliability, and dependability of the study. The external validity or transferability of the findings depends upon the integrity of the study. Jones and Kottler (2006) note that A major influence on reliability in all types of data is the amount of information gathered (p. 91). Thus, ensuring that interviews are long enough and the sample is large enough and representative of the target population can provide more reliability in the data.

15 RESEARCH PROSPECTUS 15 As an end-user of current Student Information Systems used in Charlotte-Mecklenburg Schools, there is real temptation to only select study participants who agree with my premise that current SISs are ineffective and need to be integrated. Any conclusions reached in such conditions will be negated if, as Yin (2009) argues, an investigator seeks only to use a case study to substantiate a preconceived position (p. 72). Thus, reducing systematic or substantive biases is important for the success of the proposed study. As Creswell (2009) recommends, transcripts will be checked to make sure they do not contain obvious mistakes (p. 190). This researcher will also check and test the trustworthiness, authenticity, and credibility of the study participants at all stages of data collection. As Miles and Huberman (1994) suggest, this researcher will include informants with different points of view from the mainstream (p. 266). Data Management Plan The popularity of to commercial-off-the-shelf (COTS) software programs and matrices for analyzing qualitative data is undeniable. Lu and Shulman (2008) touted the value, possibilities, and flexibility of computer-assisted data analysis software (CAQDAS) programs: CAQDAS preserves and arguably enhances avenues for flexibility in the coding process (p. 106). They noted that Researchers across disciplines with qualitative data analysis problems increasingly turn to commercial-off-the-shelf (COTS) software for solutions (p. 105). Despite the popularity of computer-assisted qualitative data analysis software, generally referred to as CAQDAS, the role of the researcher is critically important. Glesne (2011) notes: the researcher remains the decision maker and the interpreter. He or she needs to be intimate with the data in order to know what to ask the software to do (p. 205). This researcher will play a fundamentally crucial part with any CAQDAS program that will be chosen for qualitative data analysis.

16 RESEARCH PROSPECTUS 16 Computer Application Strategy There is general consensus that CAQDAS programs are important in qualitative data analysis. Weitzman (2000) argues that CAQDAS programs can provide assistance in searching, marking up, linking, and reorganizing the data, and representing and storing your own reflections, ideas, and theorizing (p. 806). Triangulation of data coding, data displays and CAQDAS will be the chosen avenue for this qualitative inquiry. Manual matrix displays will be supplemented by QSR NVivo for data analysis. Walsh (2003) contends that NVivo is a useful teaching tool that works like an old loose-leaf binder. According to Walsh (2003), NVivo offers the following advantages: Many different kinds of documents can be kept in one place, and they are linked together for easy access. Also, one can quickly trace the progression of an idea from its earliest stages using NVivo (Walsh, 2003, p. 253). Arguing for the benefits of CAQDAS programs, the author further adds, software-based research allowed more freedom to play with ideas, because researchers can link and compare patterns within and across documents and the results can be saved, printed, or undone at will (Walsh, 2003, p. 253). Choosing the right CAQDAS program is critically crucial for a successful completion of this study. Assumptions and Limitations It is assumed that an electronic SIS is positively associated with parental involvement, which, in turn, increases student academic achievement. The proposed study will demonstrate the extent of the causal link between SIS, parental involvement, and student academic achievement. The study participants will be sampled from Charlotte-Mecklenburg Schools. And CMS is only one out of 116 school districts in North Carolina. Furthermore, the fact that this researcher works for a CMS school may negatively affect the results of the study.

17 RESEARCH PROSPECTUS 17 Ethical Considerations It is generally accepted that a study of Student Information Systems can be highly delicate because it involves sensitive student data. The type of student data these systems store may vary. NCES (2000) describes the contents of an SIS as follows: Typical contents may include family information, courses taken and grades, special program participation information, immunization records, assessment scores, extracurricular activities, and other information that is used by the education system to promote student success and provide appropriate services (p. 2). It is incumbent upon this researcher to protect student data as well as the confidentiality of study participants. Once data are collected, they will be stored on a secure computer and destroyed after analysis. Participants will be required to consent to participate in the study through Informed Consent. In addition, a Capella s Institutional Review Board (IRB) will review and approve of the scientific, educational, and/or societal value of the research topic before it is conducted. Furthermore, it is important that a researcher explains the nature, the scope, and the purpose of the study to the participants. This puts participants at ease, lets them know to whom they are speaking, and motivates them to answer questions truthfully (Cooper and Schindler, 2011, p ). One of the benefits of doing this is improved cooperation. In fact, Cooper and Schindler (2011) note that knowing why one is being asked questions improves cooperation through honest disclosure of purpose (p. 34). Mishandling of participants personal demographic information and responses may lead to trouble. As Creswell (2007) suggests, a research study must employ a rigorous approach to data collection, data analysis, and report writing (p. 46). Hence, the proposed study will abide by these strict ethical guidelines. Expected Outcome

18 RESEARCH PROSPECTUS 18 Bernhardt (2006) argues that all school districts need data warehouses. But it is not enough for a student information system to store student data. Learning Point Associates (2006) posit that Data fuel the process of change. Schools and districts should have electronic data systems that store data and make it easy to extract useful information (p. 4). An SIS should allow users to analyze the data stored and make informed educational decisions. By its design, a student information system needs to meet a number of core requirements: Interactivity, user-friendliness, access to statistics, and data visualization. The conceptual framework of this study suggests the design of an SIS that is interactive, user-friendly, and easily accessible by users. Such an SIS should be able to store and analyze data as effectively as possible. Additionally, it should increase school-parent collaboration, shed light on the decision-making process, and improve instruction. Conclusion The proposed study will demonstrate the need for the Charlotte-Mecklenburg Schools district to review the adoption process of its student information systems. The lack of integration limits the effectiveness of existing student information systems. The current SISs can be unified and consolidated into one integrated SIS that can meet the needs of the users. This study will suggest an integration framework for all SISs used in CMS, including the newly launched data portals.

19 RESEARCH PROSPECTUS 19 References Auerbach, C. F. and Silverstein, L.B. (2003). Qualitative data: An introduction to coding and analysis. New York: New York University Press. Barrett, S. (1999). Information systems: An exploration of factors influencing effective use. Journal of Research on Computing in Education, 32(1), ISTE. Bernhardt, V.L. (2006). Using data to improve student learning in school districts. Eye on Education, Inc. Larchmont, NY. Cooper, D. R., & Schindler, P. S. (2011). Business Research Methods (11th ed.). New York: McGraw-Hill/Irwin. Creswell, J. W. (2007). Qualitative inquiry & research design: Choosing among five approaches (2nd ed.). Thousand Oaks, CA: Sage. Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. (3rd ed.). Los Angeles, CA: SAGE Publications, Inc. ISBN Idanet Steering Committee. (2003). Idaho student information management system. The Journal: Transforming Education Through Technology. Retrieved September 13, 2011 from Glesne, C. (2011). Becoming qualitative researchers: An introduction. (4th ed.). Boston, MA: Pearson. Jones, W.P. & Kottler, J.A. (2006). Understanding research: Becoming a competent and critical consumer. Upper Saddle River, NJ: Pearson Keen, P. G. W. (1981). Information systems and organizational change. Communications of the ACM, 24(1), Learning Point Associates. (2006). Effective use of electronic data systems: A readiness guide for school and district leaders. Learning Point Associates and the Educational Service Agency (ESA) Alliance of the Midwest. Retrieved August 27, 2011 from

20 RESEARCH PROSPECTUS 20 McIntire, T. (2004). Eight buying tips: Data warehouses. Technology & Learning, 25(1). Retrieved August 26, 2011, from Miles, M.B. & Huberman, A.M. (1994). Qualitative data analysis. (2 nd ed.). Sage Publications, Inc. National Center for Education Statistics. (2000, October). Building an automated student record system. Office of Educational Research and Improvement. U.S. Department of Education. NCES Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage. Swanson, R. A., & Holton, E. F. (2005). Research in organizations: Foundations and methods of inquiry. San Francisco: Berrett-Koehler Publishers, Inc. Visscher, A.J. (1996). The implications of how school staff handle information for the usage of school information systems. International Journal of Educational Research, 25(4), Retrieved August 3, 2011 from Visscher, A.J. and Bloemen, P. P. (1999-Fall). Evaluation of the use of computer-assisted management information systems in Dutch schools. Journal of Research on Computing in Education, International Society for Technology in Education, 32(1), Vogt, W. P. (2010). OM8020: Quantitative research methods for professionals. Boston, MA: Pearson Custom Publishing. Custom edition. Walsh, M Teaching qualitative analysis using QSR NVivo. The Qualitative Report, 8(2), Wayman, J.C. (2005, Summer). Involving teachers in data-driven decision-making: using computer data systems to support teacher inquiry and reflection. Journal for Education of Students Placed At Risk, 10(3). Weitzman, E.A. (2000). Software and qualitative research. In N. Denzin & Y.S. Lincoln (Eds.), Handbook of qualitative research. Thousand Oaks, CA: Sage.