The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education

Volume 2, Number 2, Special Issue 2011 The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education Nathalie Duval-Couetil *, Teri Reed-Rhoads, and Shiva Haghighi Abstract - This paper describes the Engineering Entrepreneurship Survey (EES), an assessment instrument designed to examine engineering student involvement in entrepreneurship education and related outcomes. It was developed as part of an NSF-funded study Entrepreneurship Education and its Impact on Engineering Student Outcomes: The Role of Program Characteristics and Faculty Beliefs. It has been used successfully over the past two years to gather data from senior-level students at multiple institutions that have entrepreneurship courses or programs that are available to engineers. The 135- item survey examines: 1) engineering student attitudes towards entrepreneurship, 2) engagement in entrepreneurial activities and behaviors, 3) knowledge of entrepreneurship-related terms and concepts, 4) entrepreneurial self-efficacy, 5) student perceptions of engineering faculty attitudes toward entrepreneurship. Item categories and subscales show high reliability. The validated survey collects baseline data that can be valuable in program development and evaluation. 1. Introduction Entrepreneurship courses and programs may be one of the fastest growing curricular areas within engineering schools. Economic realities such as global competition, downsizing, decentralization, re-engineering, mergers, and new technologies have made career paths more complex and uncertain for graduates in all sectors (Gibb, 1996). In this new environment, it is widely accepted that entrepreneurial ventures are key to innovation and economic growth. Therefore, students who are able to identify opportunities, understand market forces, commercialize new products, communicate, and lead teams, in addition to having strong science and technical skills, are likely to have more value in the marketplace for jobs. To meet this need, colleges of engineering are increasingly developing entrepreneurship courses and/or are making those offered through business schools or multidisciplinary programs accessible to their students. This movement has garnered support from influential publications and professional organizations such as the National Academy of Engineering (NAE) and the American Society for Engineering Education (ASEE) (Dabbagh and Menascé, 2006; Rover, 2005). The characteristics of entrepreneurship programs offered to engineers can vary widely as do the beliefs and practices of the faculty who teach them. A multi-institution, NSF-funded study, Entrepreneurship Education and its Impact on Engineering Student Outcomes: The Role of Program Characteristics and Faculty Beliefs, was undertaken to address how these factors influence student outcomes. The purpose of the study was to explore the relationship between *Associate Professor, Technology, Leadership, and Innovation Director, Certificate in Entrepreneurship and Innovation Program Purdue University 35

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi program models, faculty beliefs, and student outcomes to help inform program development, improve educational experiences, and evaluate their success (Figure 1). Consequently, a primary component of this study was to investigate the engineering student involvement in entrepreneurship education within their engineering programs and its impact on them. This required the development of a comprehensive assessment instrument capable of collecting baseline data on a broad range of attitudes toward and outcomes of entrepreneurship education on engineering students across multiple institutions. Entrepreneurship Program models Faculty beliefs and practices Student outcomes Engineering School-based Business Schoolbased Multidisciplinary Engineering faculty Business School Faculty Lecturers or Parttime faculty Practitioners Behaviors Attitudes Knowledge Self-efficacy Explore relationships to help inform program development, improve educational experiences, and evaluate their success Figure 1. Components and objectives of the NSF-funded study Entrepreneurship Education and its Impact on Engineering Student Outcomes: The Role of Program Characteristics and Faculty Beliefs Somewhat surprisingly, although an increasing number of students in all disciplines are being exposed to entrepreneurship education (Streeter et a., 2002), minimal published research has examined related learning outcomes or their influence on student attitudes, behaviors, career goals, or professional competence. Pittaway et al. (2009) observed that most of the research on entrepreneurship pedagogy focused on program design and implementation, and that assessment practice was a major gap in the field. One of the major reasons contributing to the lack of assessment in entrepreneurship education pointed to within the management literature is a lack of consensus on what entrepreneurship or being entrepreneurial is. Another reason is the wide range of learning objectives that exist across courses and programs. This led Henry et al. (2005) to state that the "content of syllabi of courses developed by entrepreneurship scholars differs to such an extent that it is difficult to determine if they even have a common purpose" (p. 103). A comprehensive review of the entrepreneurship education assessment literature uncovered few validated instruments available to faculty and administrators to assess the impact of entrepreneurship education at the course or program level (Duval-Couetil et al., 2010a). The review identified three primary types and categories of assessment: 1) course level evaluations which measure student reactions to a particular class or specific activities; 2) focused instruments which measure very specific aspects of, or constructs related to, entrepreneurship; and 3) program evaluations which assess a wider range of outcomes. Most published work appeared to be related to developing entrepreneurship-related constructs that are most appropriate for management or 36 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education professional contexts. Of the instruments reviewed, only those by Shartrand et al. (2008)and Lucas et al. (2009) were developed to be used primarily with engineering students. The purpose of this paper is to describe the development, use, and preliminary validation of the Engineering Entrepreneurship Survey (EES) (Appendix A). This survey instrument has been used successfully to collect data among engineering students enrolled at universities that have entrepreneurship programs accessible to engineering students. The data gathered from the administration of the survey over the past two years has served as the basis for several conference presentations and papers (Duval-Couetil et al., 2010b, 2011a, b & c). Given the breadth of data collected, it can serve as a valuable tool for engineering programs or faculty interested in gathering student data as part of their program development and/or evaluation processes. 2. Development of the Engineering Entrepreneurship Survey 2.1. Research Questions The EES instrument was developed to capture the data necessary to investigate how entrepreneurship education impacts a broad range of attitudes toward and outcomes of entrepreneurship education for engineering students. The primary focus of the research study was to examine the extent and nature of changes in behaviors, knowledge, attitudes, and self-efficacy that result from exposure to entrepreneurship education. It was important to provide a context for the research by providing background data related to the extent to which engineering students were involved in or being exposed to entrepreneurial activities within their academic programs. Therefore, the instrument addressed the following research questions: To what extent do engineering students participate in entrepreneurship education and related activities? To what extent is entrepreneurship addressed in their engineering programs? What are engineering student attitudes toward entrepreneurship as a career? Why are students interested or not interested in entrepreneurship? How familiar are engineering students with entrepreneurship terms and concepts? What are engineering student perceptions of their entrepreneurship-related abilities? What are the characteristics of engineering students participating in entrepreneurship education? Student demographic and program data were collected to investigate differences within and across the following groups: Engineering students who had and who had not participated in entrepreneurship courses or programs Demographic characteristics (e.g., sex, culture, ethnicity, familial entrepreneurial background) Engineering disciplines Individual programs and universities 2.2. Selection of the Appropriate Assessment Method and Administration Period Given the varying program models, the first step in creating the assessment instrument was to review the requirements of the entrepreneurship programs across the institutions. This was necessary in order to determine the type of assessment that would be most appropriate to capture Volume 2, Number 2 Special Issue2011 37

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi involvement in entrepreneurship education, and the period within students academic programs when it would be most appropriate to administer the survey. The review found that the programs involved in the study could vary significantly in their length and when they began and ended. Consideration was given to administering pre- and post-tests to engineering students involved in entrepreneurship education given their value in establishing benchmarks against which changes in attitudes, behaviors, knowledge, or self-efficacy can be measured. It was determined, however, that they could not be used in this study given the varying program models and lengths. This was based on the principle that to be valid, pre- and post-assessments should be used with highly structured curriculum where learning objectives are addressed to all subjects in a systematic manner at equivalent times within and across academic programs. Another factor that precluded the use of pre- and post-tests was the three-year grant period, which limited our ability to collect data for more than three semesters. Given the time necessary to develop the assessment instrument and analyze results, three semesters was not sufficient to capture exposure to entrepreneurship education in an adequate sample of students across the three institutions. As a result, it was decided that a self-report instrument administered to students involved in senior-level capstone design courses would provide the necessary sample homogeneity across institutions. It would capture involvement in entrepreneurship and innovation-related activities, such as formulating an idea for a product/business, developing prototypes, presenting, researching markets, and preparing business plans, since they typically occur at the capstone level (Dabbagh and Menascé, 2006). It would also allow for the investigation of differences across the desired demographic characteristics and groups. To minimize logistics and challenges related to survey administration across multiple institutions, it was decided that a web-based version would be used (Qualtrix). 2.3. Review of Existing Assessment Instruments As stated in the introduction to this paper, a comprehensive literature review of entrepreneurship education assessment was conducted both within and outside the domain of engineering. The results of this review were published in a conference paper (Duval-Coutil et al., 2010a) and disseminated in a workshop on entrepreneurship assessment (Duval-Coutil et al., 2010b). The literature review identified two existing scales developed that were integrated into the EES instrument with permission from their authors and with minor modifications: Venturing and Technology Self-efficacy (Lucas et al., 2009): This scale is comprised of 15 items to measure engineering students confidence in their venturing and technology application skills based on authentic tasks they might encounter in the workplace. It emphasizes the leadership of innovation rather than a narrow focus on the formation of companies. Items consist of short task descriptions that involve self-confidence in one s ability to recognize opportunities, write business plans, manage projects and human resources, estimate the costs of, and market a new product, as well as items that examine self-efficacy related to moving from science concepts to applications. The scale was validated with a sample of approximately 400 third and fourth year engineering students. The category of items uses a response scale reflecting 0-100% in intervals of 10, which the authors state is easily understood by engineers. Motivations and Barriers to Starting a Business (Shinner et al., 2009): Two scales were selected from a larger survey instrument designed to investigate 38 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education attitudes toward entrepreneurship and its education in order to assess demand for a multidisciplinary entrepreneurship course. The survey was administered to 317 students and 87 faculty members in a variety of disciplines. Original survey items were modified slightly and the original 5-point Likert scale based on relative importance was changed to level of agreement (1=strongly disagree, 5=strongly agree). Consideration was given to using the NCIIA s Entrepreneurship Inventory as a source for a subset of survey items related to familiarity with entrepreneurial terms and concepts (Shartrand, et al., 2008). The instrument was developed to measure engineering students familiarity with 105 terms and concepts, and it provides information on validated scales related to five categories; becoming an entrepreneur, finance and accounting, people and human resources, sales and marketing, and product ideation and development. Its integration was not feasible, however, due to its length. The need to minimize the number of response scales in the EES was another factor in the decision. In addition to the literature review, course- and program-level instruments used at each institution were reviewed. This was done to determine the extent to which existing assessments used at each institution could be leveraged and to evaluate the degree to which there might be overlap or interference with historical assessments still taking place at each university. Each program had used assessments internally for assessment and/or program development purposes, and they were in various stages of development and validation. Of the internal instruments reviewed, the research team drew most heavily on pre- and post-survey instruments used at their own institution, Purdue University, as part of its multidisciplinary Certificate in Entrepreneurship and Innovation Program. The entry survey has 74 items and the exit survey 61 items focused on work experience, post-graduation plans, motivations for taking an entrepreneurship course, familiarity with entrepreneurial topics, entrepreneurial self-efficacy, family/parental history in entrepreneurship, and demographics. It has been used with hundreds of students primarily for program development and reporting purposes. A number of items and scales from these instruments were modified and/or expanded upon to better address the needs of engineering students. Where no assessments were found to address research topics, the research team created new survey items. This occurred primarily in categories having to do with entrepreneurial activity or behaviors occurring within engineering programs: Engineering student participation in entrepreneurial activities: This category of ten items was developed to examine the extent to which students had been exposed to entrepreneurship education or related experiential learning activities. It was generated from the literature review and knowledge of entrepreneurial activities available to engineering students either through or external to their engineering programs. Students were asked to respond yes or no to whether they participated in activities such as taking a course, interning with a startup, conducting market research for a new product, developing a product for a real client, writing a business plan, delivering an elevator pitch, protecting intellectual club, participating in an entrepreneurship-related competition, joining an entrepreneurship student organization, or participating in non-credit workshops. Extent to which entrepreneurship is addressed in engineering programs: This category of nine items examined student perceptions of the extent to which entrepreneurship was addressed within their engineering programs. Items Volume 2, Number 2 Special Issue2011 39

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi investigated the degree to which students agreed that they were taught or encouraged to develop entrepreneurial skills, take entrepreneurship courses, participate in experiential learning opportunities related to entrepreneurship, and interact with entrepreneurs. It also addressed the degree to which engineering faculty discussed entrepreneurship. 2.4. Format and Structure Given the number of sources of existing and newly created items, it was necessary to structure the survey in a way that was user-friendly for the subjects. Effort was made to minimize the number and types of response scales used. Most had 5-point Likert scales with endpoints either strongly disagree (1) strongly agree (5) or poor (1) to excellent (5) with the exception of Lucas and Cooper s Venturing and Technology Self-Efficacy Scale. So that it would not interrupt the flow of the survey, the scale was placed near the end. Survey codes were developed for each of the subscales or categories of questions, and these are presented in Table 1. Attention was paid to the length of the survey; it had to be comprehensive enough that faculty would recognize its value yet short enough so students would complete it. It was estimated that it would take students approximately 15 minutes to complete. Feedback from the faculty who participated was positive. Over 90 percent of the students who started the survey completed it. 3. Sample Over a period of three semesters, the EES was administered to engineering students enrolled at senior-level engineering capstone design courses at three large public universities with established entrepreneurship programs available to engineering students. Two of the entrepreneurship programs were predominantly engineering-based. The other was a multidisciplinary program, whereby engineering students took core entrepreneurship courses with students in a variety of majors and complemented them with approved engineering courses to complete program requirements. Students received the survey via the faculty members teaching these courses. Contact with faculty members was made either through personal contacts at each institution, or through Internet searches to identify department heads and others able to approve and/or facilitate survey administration. Faculty members were sent an email describing the intent of the study and what would be required of them. If faculty agreed to participate, they were sent an email to be forwarded to students, which included a brief explanation of the survey and its URL. Over the course of the month that followed initial distribution of the survey to students, faculty members were asked to remind students to complete it. Over three semesters, 501 engineering students completed the survey. It was distributed to approximately 30 courses across the three institutions. Response rates per course ranged from 3 to 58 percent, with a mean of 21 percent. Preliminary results from an initial sample of 345 subjects and final sample of 501 have been reported (Duval-Couetil, 2011a &b). 40 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education Table 1. EES Survey Item Categories and Scales, Topics Addressed, and Item Sources Category, Code, No. of Items Specific Topic Addressed by EES Item Categories and Scales Source of Items BEHAVIORS ACTIVITIES (10) Extent to which engineering students participate New items in entrepreneurship education and related activities POSTGRAD (7) Students post-graduation career plans Purdue surveys BUSINESS (1) VENTURE (1) PROGRAM (9) INTEREST (7) STARTBUS (12) NOTSTART (14) FAMILIAR (37) EFFICACY (15) SKILLS (6) ABILITY (1) BUSABILITY (1) Multiple codes (14) Number of students who had, have, or intend to have a business Type of businesses students are interested in starting ATTITUDES Extent to which entrepreneurship addressed in engineering programs Nature of engineering student interest in entrepreneurship Reasons students would be interested in entrepreneurship Reasons students would not be interested in entrepreneurship KNOWLEDGE Student familiarity with entrepreneurship terms and concepts SELF-EFFICACY Student perceptions of their technology venturing and entrepreneurship-related abilities Student perceptions of their skills in areas related to entrepreneurship Student perceptions of their entrepreneurship ability overall Student perceptions of their ability to start a business immediately DEMOGRAPHICS Characteristics of engineering students participating in entrepreneurship education New items Purdue surveys New items Purdue surveys Shinnar et al. Shinnar et al. New items Purdue surveys Lucas et al. New items Purdue surveys Purdue surveys Purdue surveys New items Purdue surveys Volume 2, Number 2 Special Issue2011 41

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi 4. Establishing Validity and Reliability To be effective, a survey instrument should be both reliable and valid. Validity refers to the ability of an instrument to measure the real concept under consideration. Reliability refers to whether a particular technique applied repeatedly to the same object would yield the same result each time (Babbie, 1990). Since reliability is an indication of measurement error, an instrument must be reliable in order for validity to have meaning. Therefore, both are necessary and individually are not sufficient. 4.1. Validity The first type of validity to be considered in survey instrument development is content validity, which refers to the degree to which a survey covers the range of meanings and content within a specific domain (Babbie, 1990). EES content validity was evaluated through a comprehensive literature review, as well as evaluation by a panel of approximately 20 experts. This panel was comprised of engineering and entrepreneurship faculty at the institutions involved in the study, members of the external advisory board for the grant and educational assessment specialists. Feedback was integrated into revisions of the survey. Face validity is a form of content validity and may be of importance in determining its acceptability and reasonableness to those who will be tested (Messick, 1989). If a survey is considered to be meaningless or irrelevant by subjects, they may be less willing to participate. To determine whether the EES would be considered valid from the subject s point of view, engineering students who had and who had not been exposed to entrepreneurship education, participated in a think-aloud protocol. This protocol allowed members of the research team to observe subjects completing the online survey to evaluate comprehension, the instrument s relevance, and its user-friendliness. Subjects were asked to identify any ambiguities or difficulties they encountered as they completed the survey. The think-aloud protocol resulted in feedback that led to the revision of items, scales, survey instructions and length. Construct validity refers to evidence of the extent at which a measure relates to other variables within a system of theoretical relationships or constructs (Babbie, 1990). Often, experts in the field are used to verify construct validity as was the case with the EES. Preliminary evaluation of construct validity also consisted of data analyses to examine differences in the results of students who had and who had not been exposed to entrepreneurship education. These were detected in larger populations as data collection progressed. Future analyses will examine predictive validity, which refers to the extent to which a given instrument can be used to predict desired future performance, such as a student s future success within a given field. In the short term, this could be achieved by examining performance in entrepreneurial activities while they are on campus (e.g., competitions). Longer term, longitudinal research could examine engineering students post-graduation career choices, income, and net worth as reported. A survey of entrepreneurship alumni of a business school conducted by Charney and Libecap (2003) explored the impact of entrepreneurship education on these factors. 4.2. Reliability In this study, reliability was assessed through internal consistency to determine the extent to which similar survey items give consistent responses. Cronbach s coefficient alpha was used to measure the overall internal reliability within the categories of survey items that addressed 42 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education attitudes, beliefs, or perceptions that were most appropriate for such analysis (Table 2). Categories of items that measured behaviors and past or future participation in activities were excluded from the analyses. Table 2. Cronbach s alpha for EES survey item categories and scales (see Table 1 for details of items and scales) Category of Items/Scale Number of items PROGRAM (9) 9.89 INTEREST (7) 7.92 STARTBUS (12) 12.83 NOTSTART (14) 14.86 FAMILIAR (37) 37.96 EFFICACY (15) 15.96 SKILLS (6) 6.74 Cronbach s Coefficient Alpha Notable results are the following: All but one category, SKILLS, met the standard of 0.8, which is considered acceptable for group comparisons (Nunnally and Bernstein, 1991). Items in this category addressed a variety of skills including analytical, communication, and presentation skills, as well as the ability to evaluate business ideas, level of risk tolerance and ability to deal with uncertainty. These items were not written in a way that tied them to an entrepreneurial context, and this will be addressed in future revisions. The category FAMILIAR is comprised of 37 terms and concepts that are relevant to entrepreneurship. Students were asked to rate their familiarity with each. Cronbach s alpha of 0.96 was high for the group, which is to be expected with such a large number of items. Given the large number of items, this category was analyzed independently for a conference presentation (Duval-Couetil, 2011)). This consisted of dividing 35 of the 37 terms into six categories based on content and face validity and calculating the internal consistency for each. They included the following: engineering (α =.88); general entrepreneurship (α =.86); 3); general business (α =.84); marketing (α =.93); finance (α =.93); and professional skills (α =.75). Future work will address the refinement of these scales. 5. Discussion The Engineering Entrepreneurship Survey (EES) was developed based on a need to be able to measure student attitudes toward entrepreneurship, their level of involvement and interest in entrepreneurship education, and its impact on their entrepreneurial knowledge and self-efficacy. The instrument was designed to collect a wide range of baseline data that would be useful to make comparisons based on student characteristics and program characteristics both within and across institutions. It was developed after conducting a comprehensive review of the entrepreneurship education assessment literature both within and outside the field of engineering. This review identified validated scales that were used in the development of the EES and identified assessment gaps that were filled with new items and scales developed by the research team. Attention was given to ensure that the survey would be administered at a period in a Volume 2, Number 2 Special Issue2011 43

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi student s academic program when exposure to entrepreneurship education would be detected and that the administration would not interfere or overlap with assessments used at institutions involved in the study. Using conventional analytical approaches, we have demonstrated that the EES is acceptable as a self-report instrument to be used with engineering students. It was used over a period of two years to collect data that, thus far, has been primarily used to measure levels of student interest in and involvement in entrepreneurship education and to investigate differences between groups of students who have and have not had exposure to entrepreneurship education. Future analyses will explore differences in attitudes based on demographic characteristics such as sex, ethnicity, and parent involvement in entrepreneurship. They will also explore differences in institutions to examine the extent to which program characteristics and faculty beliefs and practices impact student attitudes and outcomes, which is the overarching objective of the NSF-funded study. As it stands, the EES can be used as an effective assessment tool for needs assessment, program development, and research purposes. It can be used alone to measure differences in groups, as has been described in this manuscript. There are, however, opportunities to refine the instrument as a whole and to further explore the validation of subscales focused on the following areas: Demand for entrepreneurship education: Items addressing student interest in taking entrepreneurship courses, as well as their post-graduation career plans can be used to estimate potential demand for entrepreneurship courses or related activities. Entrepreneurial climate: Items related to student perceptions to the extent that entrepreneurship is addressed within engineering programs or by engineering faculty can assess whether a particular environment is conducive to and supportive of entrepreneurial education or activities. Entrepreneurial mindset: Items pertaining to student involvement in entrepreneurial education, activities, courses, and whether students have ideas for ventures; the nature of student interest in entrepreneurship can be used to measure the degree to which students have an entrepreneurial mindset. Gains in entrepreneurial knowledge and self-efficacy: Used as a pre- and post-test in contexts with homogeneous conditions, it can measure gains related to exposure to entrepreneurship education or related activities. It can also be used to compare knowledge or gains across courses, engineering departments or institutions. Body of knowledge for engineering entrepreneurs: Items pertaining to familiarity with terms and entrepreneurial self-efficacy can identify the emphases, strengths and weaknesses of programs being offered to engineering students. Items related to barriers to entrepreneurship can identify areas that must be addressed in an educational program designed to make entrepreneurship a desirable and/or accessible choice. As with any instrument, it is important to examine the sample on which it is tested. To date, the external validity of the survey, or degree to which the results can be generalized to populations beyond the sample, is limited. The instrument has been used primarily within large, public universities with established engineering and entrepreneurship programs where interest and participation in entrepreneurship may be higher than in the general population of engineering schools. Purposive sampling has been used to ensure adequate representation of students who had exposure to entrepreneurship education in order to make comparisons between groups. The representation of underserved populations and certain engineering disciplines in the sample was too low to draw conclusions about these groups. Improving the external validity will require administration of the survey to a more diverse group of institutions, with varying program models and student populations. 44 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education Internal validity refers to the recognition of bias that could be introduced by the researchers themselves through their beliefs or actions. This can be manifested in the research design or the way the sample was obtained. For this particular instrument, and the way it has been administered to date, bias could arise from self-selection of faculty and students who chose to participate. Senior design faculty members from three institutions were asked to participate in the study; however, those who chose to do so may have a positive bias toward entrepreneurship education. Since the survey was voluntary for students, it is possible that those who chose to complete it had a positive bias toward entrepreneurship as well. Furthermore, its administration as an online survey may have resulted in lower response rates since students completed them outside of class instead of in-class. Future work must address these sampling limitations. 6. Conclusion The Engineering Entrepreneurship Survey (EES) can be an effective tool as universities and departments develop entrepreneurship courses and programs for engineering students. The literature review on which it is based can serve as a resource for researchers developing assessment instruments in the field of entrepreneurship education. Thus far, it has been used effectively to detect differences across groups of students who have and who have not been exposed to entrepreneurship education. Future work will examine differences by program and demographic characteristics. The EES is certain to undergo refinement, and future work will explore developing individual instruments that focus on specific constructs or content. However, in its current form, it can provide valuable baseline data for faculty and administrators who are involved in teaching, developing, or evaluating the impact of entrepreneurship courses delivered to engineering students. References Babbie, E. (1990). Survey Research Methods. Belmont, CA: Wadsworth. Charney, A. and Libecap, G. (2003). The contribution of entrepreneurship education: An analysis of the Berger program. International Journal of Entrepreneurship Education, 1(3):276-288. Dabbagh, N. and Menascé, D. (2006). Student perceptions of engineering entrepreneurship: An exploratory study. Journal of Engineering Education, 95(2): 153-163. Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2010a). Development of an assessment instrument to examine outcomes of entrepreneurship education on engineering students. Proceedings of the 40th ASEE/IEEE Frontiers in Education Conference. Washington, DC. Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2010b). Assessing the outcomes of entrepreneurship education on engineering students. Workshop of the NCIIA Annual Meeting. San Francisco, CA. Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2011a). Investigating the impact of entrepreneurship education on engineering students. Proceedings of the NCIIA Annual Meeting. Washington, DC. Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2011b). Engineering students and entrepreneurship education: Involvement, attitudes and outcomes. International Journal of Engineering Education, in press. Duval-Couetil, N., Reed-Rhoads, T., and Haghighi, S. (2011c). Entrepreneurship literacy for engineering students. Proceedings of the 41st ASEE/IEEE Frontiers in Education Conference. Rapid City, South Dakota. Volume 2, Number 2 Special Issue2011 45

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi Gibb, A. (1996). Entrepreneurship and small business management: Can we afford to neglect them in the twenty-first century business school? British Journal of Management, 7(4): 309-321. Henry, C., Hill, F., and Leitch, C. (2005). Entrepreneurship education and training: Can entrepreneurship be taught? Part 1. Education & Training, 47(2): 98-111. Lucas, W., Cooper, S., Ward, T., and Cave, F. (2009). Industry placement, authentic experience and the development of venturing and technology self-efficacy. Technovation, 29(11): 738-752. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement. New York, NY, England: Macmillan Publishing. 60-68. Nunnally, J. and Bernstein, I. (1991). Psychometric Theory. New York: McGraw-Hill. Pittaway, L., Hannon, P., Gibb, A., and Thompson, J. (2009). Assessment practice in enterprise education. International Journal of Entrepreneurial Behaviour & Research, 15(1): 71-93. Rover, D. (2005). New economy, new engineer. Journal of Engineering Education, 94(4): 427-428. Shartrand, A., Weilerstein, P., Besterfield-Sacre, M., and Olds, B. (2008). Assessing student learning in technology entrepreneurship. Proceedings of the 2008 ASEE/IEEE Frontiers in Education Conference. Saratoga Springs, New York. Shinnar, R. Pruett, M., and Toney B. (2009). Entrepreneurship education: Attitudes across campus. Journal of Education for Business. 84(3):151-159. Streeter, D., Jaquette, P. & Hovis, K. (2002). University-wide entrepreneurship education: Alternative models and current trends. WP 2002-02 March 2002 Working Paper - Department of Applied Economics and Management. Ithaca, NY: Cornell University. 46 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education Appendix A Engineering Entrepreneurship Survey (EES) This survey is to gather information about your experiences and involvement in entrepreneurship activities and education. Entrepreneurship refers to the practice of starting business ventures based on the development of new products and/or services. This survey is voluntary and you may skip any questions. If necessary, you can stop the survey and complete it at a later time. Use the same link used to access the survey to pick up where you left off. You must be 18 years or older to participate. Personal information, such as your name and email, is requested so that we may conduct follow-up interviews with a small sample of students. At the end of the survey, you will be given the option to opt-out of being contacted for a follow-up interview. Identifying information will not be released in any way. Please select "yes" indicating that you have read the above statement and agree to be included in this survey. If you do not wish to continue, you may close the Web browser now. I agree to participate in this survey. Yes (1) No (AGREE) INSTRUCTIONS: Please take your time to answer each question as honestly and as accurately as possible. You will need to click on the arrow button at the end of each page to save it and move on to the next. Please be sure to click the 'Submit' button on the last page to complete the survey. The survey should take approximately 15 minutes. If you need assistance or have questions, please contact: Survey Administrator University Name Email Address Phone Thank you for your time. Volume 2, Number 2 Special Issue2011 47

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi What university do you attend? University A University B University C (UNIV) What is your major? (MAJOR1) Aeronautics and Astronautics Engineering Agricultural and Biological Engineering Architectural Engineering Biomedical Engineering Biomolecular Engineering Chemical Engineering Civil Engineering Computer Engineering Computer Science Construction Engineering Electrical Engineering Environmental/Ecological Engineering Industrial/Systems Engineering Interdisciplinary Engineering Manufacturing Engineering Materials/Materials Science Engineering Mechanical Engineering Multidisciplinary Engineering Nuclear Engineering Textile Engineering Other (please list) If you have a double major, please list it (if not, leave blank). (MAJOR2) If you have one or more minors, please list it/them (if not, leave blank). Minor 1 Minor 2 Minor 3 What is your sex? (SEX) Male Female Please enter your age (in years). (AGE) Please indicate your residence status. In-state student Out-of-state student International student (RESIDENCE) If you are an international student, what is your home country? (HOMECOUNTRY) 48 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education What is your ethnic/racial background? Select all that apply. American Indian or Alaska Native Asian Black or African American Hispanic or Latino Native Hawaiian or Other Pacific Islander White Multiracial Other (please specify) Decline (RACE) Are either of your parents entrepreneurs? Yes No I m not sure (PARENTS) Is anyone in your family (not including your parents) an entrepreneur? Yes No I m not sure (FAMILY) How many entrepreneurship courses have you taken in your college career, outside of engineering? (COURSES) None 1 2 3 or more If you have taken any entrepreneurship courses outside of engineering (see previous question), please list the course name(s) and number(s). (COURSE NAME) Have you participated in any of the following types of academic entrepreneurship programs? (ESHPROGRAM) Certificate in Entrepreneurship Major in Entrepreneurship Minor in Entrepreneurship Other Volume 2, Number 2 Special Issue2011 49

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi While in college, have you done any of the following: Taken an entrepreneurship course within the College of Engineering at your university Interned or worked for an entrepreneurial or start-up company Conducted market research and analysis for a new product or technology Developed a product or technology for a real client/customer Given an elevator pitch or presentation to a panel of judges about a product or business idea Been involved in patenting a technology or protecting intellectual property Been involved in entrepreneurship- or businessrelated student organizations Written a business plan Participated in an entrepreneurship-related competition (e.g., product development, business plan) Participated in entrepreneurship-related workshops (extra-curricular, non-credit) (ACTIVITIES) No Yes Rate your level of agreement with the following: In general, in my engineering courses (PROGRAM) Strongly Strongly Disagree Undecided Agree disagree agree Faculty discuss entrepreneurship Students are taught entrepreneurial skills Students are encouraged to develop entrepreneurial skills Students are encouraged to take entrepreneurship courses Students are encouraged or required to participate in entrepreneurship-related activities Students are encouraged to consider starting their own companies Entrepreneurship is presented as a worthwhile career option There are opportunities to interact with entrepreneurs Students should learn more about entrepreneurship 50 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education Consider your post-graduation options and please rate your level of agreement with the following: I plan to (POSTGRAD) Strongly Strongly Undecided Agree disagree Disagree agree Start my own business or be self-employed Work for a small business or start-up company Work for a medium- or large-size business Work for the government Serve in the military Work for a non-profit organization Attend graduate/professional school Please rate your level of agreement with the following: (INTEREST) Strongly Disagree Undecided Agree disagree I have a general interest in the subject of entrepreneurship I want to become an entrepreneur I have an idea for a business product or technology I would like to know if I have what it takes to be an entrepreneur I am interested in taking entrepreneurship classes Entrepreneurship education can broaden my career prospects and choices I would like to learn about entrepreneurship in my engineering courses Strongly agree Please check the answer that best fits your current situation. I had my own business I have my own business now I would like to start a business in the next year I would like to start a business in the next 5 years I would like to start a business in the next 10 years I don t have any plans to start a business at this time (BUSINESS) Volume 2, Number 2 Special Issue2011 51

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi If you are interested in being an entrepreneur, what type of business are you interested in starting? Please describe the product or service and industry or market. (VENTURE) Please rate your level of agreement with the following: I would start a business in order to (STARTBUS) Strongly Strongly Disagree Undecided Agree disagree agree Focus on a technology that interests me Satisfy a need in a market Solve a social problem Create something of my own Have more flexibility and independence Have more free time Make more money Be at the head of an organization Manage people Create jobs Follow a family tradition Gain high social status If there are any reasons unlisted above for why you would start a business, please list them here. (STARTBUS2) 52 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education Please rate your level of agreement with the following: I would NOT start a business in order to (NOTSTART) Strongly Strongly Disagree Undecided Agree disagree agree Lack of ideas regarding what business to start Lack of assistance available to assess business viability Excessively risky Lack of initial capital for start-up Lack of legal assistance or counseling Lack of knowledge of the business world and the market Lack of experience in management and finance Current economic situation Irregular income Lack of support from people around me (family, friends, etc) Fear of failure Doubts about personal abilities Having to work too many hours Problems with employees and colleagues If there are any reasons unlisted above for why you would start a business, please list them here. (NOTSTART2) Volume 2, Number 2 Special Issue2011 53

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi Please rate your level of knowledge or skill in the following areas related to entrepreneurship. (FAMILIAR) Below Poor Average average Characteristics of entrepreneurs Role of entrepreneurship in the world economy Business ethics Risk management Legal structures for ventures Intrapreneurship Social entrepreneurship Leadership Managing teams Project management Negotiation Product development Product life cycle Economies of scale Feasibility study Prototype Intellectual property Technology commercialization Patents Finance and accounting Venture capital Equity Company valuation Balance sheet Income statement Break even Market research Competitive analysis Target market Product positioning Product distribution Advertising and promotion Sales and selling Executive summary Business plan Business models Business incubator Above average Excellent 54 The Journal of Engineering Entrepreneurship

The Engineering Entrepreneurship Survey: An Assessment Instrument to Examine Engineering Student Involvement in Entrepreneurship Education For each statement select a number from 0 (0% Not at all confident) to 10 (100% Completely confident) to indicate how confident you are that you could perform that skill or ability now. (EFFICACY) 0 1 2 3 4 5 6 7 8 9 10 Know the steps needed to place a financial value on a new business venture Pick the right marketing approach for the introduction of a new service Work with a supplier to get better prices to help a venture become successful Estimate accurately the costs of running a new project Recognize when an idea is good enough to support a major business venture Recruit the right employees for a new project or venture Convince a customer or client to try a new product for the first time Write a clear and complete business plan Convert a useful scientific advance into a practical application Develop your own original hypothesis and a research plan to test it Grasp the concept and limits of a technology well enough to see the best ways to use it Design and build something new that performs very close to your design specifications Lead a technical team developing a new product to a successful result Understand exactly what is new and important in a groundbreaking theoretical article Translate user needs into requirements for a design so well that users will like the outcome Overall, how would you rate your entrepreneurial ability? Poor Below average Average Above average Excellent (ABILITY) Volume 2, Number 2 Special Issue2011 55

N. Duval-Couetil, T. Reed-Rhoads, and S. Haghighi How would you rate your ability to start a business now? Poor Below average Average Above average Excellent (BUSABILITY) Rate your skill levels in the following areas: (SKILLS) Below Poor Average average Communication skills Presentation skills Analytical skills Ability to evaluate business ideas Level of risk tolerance Ability to deal with uncertainty Above average Excellent Please enter your first name. (FIRSTNAME) Please enter your last name. (LASTNAME) Please enter your university assigned email address. (EMAIL1) Please list an alternate email address, if applicable (optional). (EMAIL2) Are you willing to be contacted for a follow-up interview? (FOLLOWUP) Yes No 56 The Journal of Engineering Entrepreneurship