Examining Science and Engineering Students Attitudes Toward Computer Science

Examining Science and Engineering Students Attitudes Toward Computer Science Abstract Concerns have been raised with respect to the recent decline in enrollment in undergraduate computer science majors. Women are one subpopulation that is severely underrepresented. To better understand the factors that discourage students, both males and females, from pursuing degrees in computer science, a valid and reliable survey is needed. This type of instrument would support the quantitative tracking of attitudinal changes with respect to the field overtime as well as attitudinal comparisons across various subpopulations. This paper describes a survey which is being developed based on current research in computer science education at the Colorado School of Mines through support of the National Science Foundation. Based on the results of a factor analysis and with respect to the pilot population (Colorado School of Mines undergraduate students), there is evidence to support the assertion that this instrument is accurately measuring the five constructs that it was designed to assess. Index Terms Assessment, Attitudes Survey, Computer Science, First Year Students INTRODUCTION During an era of technological and scientific advancement in the United States, students are increasingly selecting not to major in computer science at the undergraduate level [1]. The origins of this decline dates back to the collapse of many dotcom businesses [2] which occurred between 2000 and 2001. Failure to prepare more students in computer science is expected to have lasting repercussions on the U.S. [3]. In order to maintain an international competitive edge, more students need to be trained in this field. Women, as a subpopulation, are severely underrepresented in the field of computer science. The failure of computer science to attract and retain women has been a long standing problem and existed prior to the recent challenges in dotcom businesses. For example, in 1995 only 28% of awarded computer science bachelor s degrees throughout the nation were awarded to women [4] and this percentage remained approximately constant until 2001 [5]. Despite many efforts to increase the participation of women in this field, by 2006, it had decreased to 21% of awarded bachelor degrees [5]. Research [6] in computer science education has provided insight into the problems that need to be overcome to increase the participation of our nation s students in computer science, especially women. Many students lack Andrew Hoegh, Barbara M. Moskal Colorado School of Mines, bmoskal@mines.edu confidence in their abilities to learn computer science concepts and this has been found to be especially true for women [6, 7]. Furthermore, computer science is often perceived as a field that is more appropriate for males than it is for females, discouraging female participation [6, 8]. The vast array of applications of computer science to many career paths is also unknown to many students [6]. In other words, students may not recognize that computer science skills are both necessary and useful. Another factor that discourages student participation is the belief that computer scientists are geeks or nerds who select to interact with computers rather than people [9]. In combination, these incorrect perceptions of computer science as a field have little appeal to either men or women. The purpose of this research is to develop a valid and reliable survey that measures the prevalence of these beliefs in a college student population who are enrolled at a school of science and engineering. There have been prior efforts to develop surveys in computer science that address the issues described here. For example, Wieb, Williams, Yang and Miller [10] created a survey that seeks to measure many of these same issues. Their instrument was based on one which was originally designed to measure students attitudes in mathematics [11]. A major assumption of their work is that the same factors that discourage participation in mathematics also discourage participation in computer science; no evidence is presented by these authors to support the accuracy of this assumption. Other instruments [12] have also been developed to examine students attitudes with respect to computer science once students have enrolled in a computer science course. These instruments are limited in that a major obstacle is getting students to enroll in that first computer science experience. RESEARCH FOCUS The focus of this research is the development and validation of a survey that measures undergraduate student perceptions of computer science as a field of study in a school of science and engineering. This instrument was developed based on a literature review of students perceptions of computer science as a field of study. The target population is college students who are enrolled in a school of science and engineering but who are not currently pursuing a degree in computer science. Using prior research and expert review, the following five constructs were identified as areas of interest: 1. students confidence in their own ability to learn computer science skills; W1G-1

2. students perceptions of computer science as a male field; 3. students beliefs in the usefulness of learning computer science; 4. students interests in computer science; and 5. students beliefs about professionals in computer science. As part of this investigation, an attitude survey has been developed that includes a subset of questions that address each of the constructs described above. This survey was administered to first year college students attending the Colorado School of Mines (CSM), a school primarily of science and engineering. This paper reports the results of a factor analysis which was completed on that data. This work was supported by the National Science Foundation (NSF), DUE-0512064. The opinions expressed here are that of the authors and do not necessarily reflect that of the NSF. METHODS Participants Data was collected during the spring semester of 2007 in a freshman level course, Calculus for Scientists and Engineers II. This course was selected because it is part of the first year required mathematics course for all students at CSM and students typically complete this course before selecting a major. Therefore, the students in this course should be representative of first year students across the institution. A total of 276 students signed the consent forms and agreed to participate in this investigation. These students primarily represent future majors in science and engineering because this is the population that CSM serves. Instrument Development Based on prior research in computer science education, the following five constructs were identified as the focus of this survey: Confidence Construct (C): students confidence in their own ability to learn computer science skills; Interest Construct (I): students interests in computer science; Gender Construct (G): students perceptions of computer science as a male field; Usefulness Construct (U): students beliefs in the usefulness of learning computer science; and Professional Construct (P): students beliefs about professionals in computer science. A team of investigators, that included experts in computer science education and assessment, generated a list of survey questions designed to measure each construct. The goal was to generate five questions that were positively phrased and five that were negatively phrased for each construct. These questions were then administered to three computer science novices (individuals who had no background or training in computer science) and directed to think aloud as they read and responded to each question. The think aloud was used to fine tune the phrasing of the questions and ensure that respondents were interpreting the questions in the intended manner. Next, the instrument was administered to a small sample of students in Probability and Statistics for Engineers. Probability and Statistics for Engineers is a junior level course which is required for the majority of science and engineering majors at CSM. This course was selected due to convenience. One of the investigators was coordinating the efforts of the instructors and graduate students that were teaching the various sections of this course, allowing for ease of access. Descriptive statistics were used to analyze and to determine whether the questions appeared to be measuring the intended constructs; further refinements were made to the instrument based on these results. Current Instrument The administered version of the resultant survey is contained in Figure 1. Although the questions were administered in random order, the questions in Figure 1 are organized according to the construct that they were designed to examine. Questions were designed using a four point Likert scale with students responding to each question by selecting from the following options: strongly disagree, disagree, agree, and strongly agree. A neutral category was not included in order to encourage respondents to make a positive or negative decision. The survey also included a set of demographic questions concerning, gender, age, ethnicity, college major and undergraduate level. Analysis For analysis purposes, the selected response questions were re-coded to a numerical scale which ranged from 1 to 4. Negatively phrased questions were reversed coded such that a high score always reflected a positive attitude. A Cronbach s alpha was used to examine the reliability of the proposed questions within each construct. Examination of the reliability, however, does not establish the validity of the questions. Although a high Cronbach s alpha supports that a set of questions measure the same construct, it does not suggest the nature of that construct. A general rule of thumb is that Cronbach values of 0.7 or higher indicates an acceptable level of reliability [13]. The next step in the analysis process was the completion of a factor analysis. A factor analysis is a data reduction technique that takes a collection of observed random variables and groups them into common factors. The factor loadings reflect the strength of the association of the particular variable, or in this case a particular question, has with respect to the given factor. It is possible for variables to load highly on multiple factors or not load highly on any W1G-2

Confidence construct (C): C1. I am comfortable with learning computing concepts. C2. I have little self-confidence when it comes to computing courses. C3. I do not think that I can learn to understand computing concepts. C4. I can learn to understand computing concepts. C5. I have a lot of self-confidence when it comes to computing courses. C6. I can achieve good grades (C or better) in computing courses. C7. I am confident that I can solve problems by using computer applications. C8. I am uncertain that I can achieve good grades (C or better) in computing courses. C9. I am not comfortable with learning computing concepts. C10. I doubt that I can solve problems by using computer applications. Interest construct (I): I1. I would not take additional computer science courses if I were given the opportunity. I2. I think computer science is boring. I3. I hope that my future career will require the use of computer science concepts. I4. The challenge of solving problems using computer science does not appeal to me. I5. I like to use computer science to solve problems. I6. I do not like using computer science to solve problems. I7. The challenge of solving problems using computer science appeals to me. I8. I hope that I can find a career that does not require the use of computer science concepts. I9. I think computer science is interesting. I10. I would voluntarily take additional computer science courses if I were given the opportunity. Gender construct (G): G1. I doubt that a woman could excel in computing courses. G2. Men are more capable than women at solving computing problems. G3. Computing is an appropriate subject for both men and women to study. G4. It is not appropriate for men to study computing. G5. Women are more capable than men at solving computing problems. G6. Women are more likely to excel in careers that involve computing than men are. G7. Women produce higher quality work in computing than men. G8. Women and men can both excel in careers that involve computing. G9. I doubt that a man could excel in computing courses. G10. It is not appropriate for women to study computing. G11. Men produce higher quality work in computing than women. G12. Men are more likely to excel in careers that involve computing than women are. G13. Women produce the same quality work in computing as men. G14. Men and women are equally capable of solving computing problems. G15. Men and women can both excel in computing courses. Usefulness construct (U): U1. Developing computing skills will not play a role in helping me achieve my career goals. U2. Knowledge of computing will allow me to secure a good job. U3. I use computing skills in my daily life. U4. My career goals do not require that I learn computing skills. U5. Developing computing skills will be important to my career goals. U6. Knowledge of computing skills will not help me secure a good job. U7. I do not use computing skills in my daily life. U8. I expect that learning to use computing skills will help me achieve my career goals. Professional construct (P): P1. Doing well in computer science does not require a student to spend most of his/her time at a computer. P2. A student who performs well in computer science will probably not have a life outside of computers. P3. To do well in computer science, a student must spend most of his/her time at a computer. P4. A student who performs well in computer science is likely to have a life outside of computers. P5. Being good at computer science is a negative quality. P6. Students who are skilled at computer science are less popular than other students. P7. Being good at computer science is a positive quality. P8. Students who are skilled at computer science are just as popular as other students. P9. Students who are skilled at computer science are more popular than other students. Note: Strike through indicates questions that were removed based on the results of the statistical analysis. FIGURE 1 CONSTRUCTS AND SURVEY QUESTIONS W1G-3

factors. The desire in the validation of a survey instrument is that all of the questions designed to measure a given construct will group into the factor that represents that construct. Factor analysis is used here to identify questions that are not measuring the intended construct. A generally accepted threshold for a load in a factor analysis is 0.4 [14]. RESULTS Cronbach s Alpha Table I summarizes the results of the calculation of the Cronbach s Alpha for each of the constructs. As this table indicates, the questions for the first four constructs each had an original Cronbach s alpha which was greater than 0.70. Therefore, no adjustments were made to the questions associated with the first four constructs based on this analysis. In the Professional construct, however, the original alpha fell below the accepted cut-off of 0.70. Questions were systematically removed from the analysis and their impact was examined on the resultant alpha level. Removal of question P9 resulted in an acceptable alpha level of 0.77. Based on this analysis, only one adjustment was made to the instrument and this was the removal of question P9. TABLE I CRONBACH S ALPHA FOR EACH CONSTRUCT Construct n Original Α Questions Removed Adjusted Α Confidence (C) 276 0.84-0.84 Interest (I) 276 0.93-0.93 Gender (G) 276 0.89-0.89 Usefulness (U) 276 0.85-0.85 Professional (P) 276-0.05 P9 0.77 Factor Analysis The next analysis was the completion of a factor analysis on the questions that remained on the instrument after completing the Cronbach s alpha analysis. In other words, the factor analysis was completed on the student data with P9 removed. The purpose of the factor analysis was to determine, based on students responses, whether the questions grouped into the five intended constructs. Based on an exploratory factor analysis with the five prior factors, it was determined that the data contained more than five factors. Analysis within each construct indicated that the gender factor was measuring two constructs and the two groupings are displayed in Table II. A qualitative examination of questions indicated that Factor A s questions were measuring the desired information while Factor B s questions were measuring an unintended construct. The questions that comprised Factor B appeared to be measuring whether women are superior to men in computer science rather than students perceptions of computer science as a male field. The questions that grouped into Factor B were removed from the survey. After removing Factor B from the Gender Construct, the factor analysis suggests that five factors are appropriate for the data. The loading for each factor in the analysis are displayed in Table III. Loadings under 0.2 are considered irrelevant and are suppressed by the software package; in this case computations were completed using R. Based on this analysis, some of the questions failed to meet the accepted load threshold of 0.4 and therefore, were also removed from the survey. The questions that were removed were C8, U7, P1, P3, and P7. Other questions were removed because they loaded on unintended constructs. Questions C5, U3, and P5 were removed because their load indicated that they measured a different, unintended construct. The final instrument contains eight, ten, ten, six and four questions that address the following constructs, Confidence, Interest, Gender, Usefulness, and Professional, respectively. These questions are displayed in Figure 1 without a strike through. CONCLUSIONS This paper reports on the initial research efforts that have been used in the development and validation of a computer science attitude survey. Using the current literature in computer science education, five constructs were identified as the survey s targets. Experts in computer science education and assessment then collaborated in the development of the initial set of questions which were designed to measure the specified constructs. These questions were pilot tested using think aloud protocols with computer science novices and using an analysis of descriptive statistics collected from a convenience sample of students. All of these efforts resulted in revisions to the instrument and to its questions. TABLE II QUESTION LOADINGS WITHIN GENDER CONSTRUCT Question Factor A Factor B G1 0.713 G2 0.738 G3 0.458 G4 0.238 0.545 G5 0.800 G6 0.847 G7 0.836 G8 0.612 G9 0.232 0.641 G10 0.615 0.275 G11 0.762 0.241 G12 0.725 G13 0.659 G14 0.662 G15 0.635 0.223 W1G-4

It should be noted that a factor analysis does not indicate what constructs are being measured, but rather how responses to questions group based on a statistical load. In order to determine the nature of the construct that is being measured, qualitative methods are used. In this analysis, the constructs being measured were defined before questions TABLE III RESULTS OF THE FACTOR ANALYSIS Question Factor 1 (C) Factor 2 (I) Factor 3 (G) Factor 4 (U) Factor 5 (P) C1.471.447 C2.618.327 C3.674.232 C4.542.249 C5.486.480 C6.521 C7.467.427 C8.318 C9.477.369 C10.586.223 I1.656.238 I2.613 I3.662.243 I4.321.618.281 I5.747 I6.248.719.228 I7.831 I8.243.631.287 I9.743 I10.734 G1.703 G2.758 G3.475 G8.575.231 G10.207.622 G11.807 G12.756 G13.644 G14.614 G15.260.616 U1.222.238.582 U2.359.536 U3.416 U4.233.702 U5.382.640 U6.267.531 U7.256.290.261.389 U8.286.609 P1.230.233 P2.494 P3 P4.441 P5.252.403.392.208 P6.667 P7.202.350.305.231 P8.882.338 W1G-5

were developed by experts in the computer science education and assessment and through a review of the literature. The use of experts and the literature contributes to the evidence base that the sets of questions are likely to measure the intended constructs. Think aloud protocols were used to further confirm that novice respondents were interpreting the questions in the intended manner. The factor analysis was used to examine whether there was consistency in students responses with respect to the sets of questions designed to measure a given construct. The current version of the instrument, including the revisions that resulted from each phase of this process is shown in Figure 1 with the strike through questions removed. As was discussed in this paper, there is statistical evidence that this instrument is reliable and there is qualitative evidence that it is valid. The factor analysis lends further support to our qualitative validation efforts. A great deal of research is currently underway to attract and retain students to the field of computer science [15]- [18]. A major challenge to these efforts is the limited number of assessment tools that are available to support these efforts. If students are to eventually pursue a career in computer science, they first need to have positive attitudes toward computer science. Therefore, an initial step in reversing the decline in computer science majors is likely to be understanding students attitudes toward the field. A goal of this research is to develop and validate an assessment tool that accurately measures students attitudes toward the five identified constructs in the field of computer science. This paper reports the results of these efforts with first year students attending a school of science and engineering. It is our intention to continue this research and examine the appropriateness of this instrument for measuring students attitudes beyond the current institution and academic level. [9] Jepson, A. and Perl, T., Priming the pipeline, SIGCSE Bulletin, Vol. 34, No. 2, 2002, 36-39. [10] Wiebe, E., Williams, L., Yang, K. & Miller, C., Computer Science Attitude Survey, Technical Report, Department of Computer Science, NC State University, Raleigh, NC, TR-2003-01, 2003. [11] Fennema, E., and Sherman, J.A., Fennema-Sherman mathematics scales, JSAS: Catalog of Selected Documents in Psychology, Vol. 6, No. 31, 1976. [12] Palaigeorgiou, G., E., Siozos, P.D., Konstantakis, N.I., and Tsoukalas, I. A., A computer attitudes scale for computer science freshman and its educational implications, Journal of Computer Assisted Learning, Vol. 21, No., 5, 2005, 330-342. [13] Santos, J. R. Cronbach's Alpha: A tool for assessing the reliability of scales, Journal of Extension [On-line], Vol. 37, No. 2, 1999. [14] Op t Eynde, P, and De Corte, E. Students Mathematics-Related Beliefs: Design and Analysis of a Questionnaire, paper presented at the annual meeting of the American Educational Research Association, Chicago, IL, 2003. [15] Carter, L. Why students with an apparent aptitude for computer science don't choose to major in computer science, SIGCSE 2006. ACM, New York, NY, 2006, 27-31. [16] Goode, J., If You Build Teachers, Will Students Come? The Role of Teachers in Broadening Computer Science Learning for Urban Youth, Journal of Educational Computing Research, Vol. 36, No. 1, 2007, pp. 65-88 [17] Mahmoud, Q, H, "Revitalizing Computing Science Education, Computer, Vol. 38, No. 5, 2005, pp. 100, 98-99. [18] Williams, L., Debunking the Nerd Stereotype with Pair Programming, Computer, Vol. 39, No. 5, 2006, pp. 83-85. REFERENCES: [1] Patterson, D. A., Restoring the popularity of computer science, Communication of the ACM, Vol 48, No. 9, 2005, pp. 25-28. [2] Reges, S., Back to basics in CS1 and CS2. SIGCSE Bulletin, Vol. 38, No. 1, 2006, pp. 293-297. [3] Foster, A. L., Student interest in computer science plummets, The Chronicle of Higher Education, May 27, 2005, pp. A31 A32. [4] National Science Foundation, Table C-4 Bachelor s Degrees, by sex, and field: 1995-2004, 2006. Retrieved January 17, 2008, from http://www.nsf.gov/statistics/wmpd/pdf/tabc-14.pdf [5] National Science Foundation, Table C-4 Bachelor s Degrees, by sex and field: 2006, 2006. Retrieved March 12, 2010, from http://www.nsf.gov/statistics/wmpd/pdf/tabc-4.pdf [6] Margolis, J. and Fisher, A., Unlocking the Clubhouse: Women in Computing, MIT Press: Cambridge, MA., 2002. [4] Gurer, D and Camp, T., An ACM literature review on women in computing, SIGCSE Bulletin, Vol. 34, No. 2, 2002, pp. 121 127. [8] Prey, J. and Treu, K. What do you say?: Open letters to women considering a computer science major, SIGCSE Bulletin, Vol. 34, No. 2, 2002, pp. 18-20. W1G-6