Tampereen teknillinen yliopisto. Ohjelmistotekniikan laitos. Raportti 7 Tampere University of Technology. Department of Software Systems.

Transcription

1 Tampereen teknillinen yliopisto. Ohjelmistotekniikan laitos. Raportti 7 Tampere University of Technology. Department of Software Systems. Report 7 Antti Puhakka & Kirsti Ala-Mutka Survey on the Knowledge and Education Needs of Finnish Software Professionals

2 Tampereen teknillinen yliopisto. Ohjelmistotekniikan laitos. Raportti 7 Tampere University of Technology. Department of Software Systems. Report 7 Antti Puhakka & Kirsti Ala-Mutka Survey on the Knowledge and Education Needs of Finnish Software Professionals Tampereen teknillinen yliopisto. Ohjelmistotekniikan laitos Tampere 2009

3 ISBN (PDF) ISSN X

4 Survey on the Knowledge and Education Needs of Finnish Software Professionals Antti Puhakka & Kirsti Ala-Mutka Abstract One of the greatest challenges in planning education for future and present software professionals is knowing what skills people working in the field need and how they learn these skills. With the aim of providing such first-hand information, this article presents the results of a survey of 212 Finnish software professionals about their perceptions on 72 topics, related to or frequently taught in computer science and software engineering education. Among the findings of the survey was that basic programming skills, data structures and software specification and design skills were considered very important among all respondent groups. Databases, documentation skills, fault tolerance, and practical topics related to the software development process, such as planning, requirements engineering and testing, were not learned during education to the extent necessary at work. Typically, classical engineering mathematics, physics and chemistry had been learned far more during education than was needed in practice. Discrete mathematics and theoretical computer science were also considered of low average importance, but respondents with the highest education considered them more important than others. Presentation and writing skills as well as software quality related issues were among topics on which the respondents felt they would need more education, e.g., in the form of extension studies. However, respondents area of work had some effect on the skills considered necessary. 1 Introduction One of the most important tasks in planning education is finding the right combination of topics to teach. Frequently this involves finding the right balance between more theoretical topics that develop general thinking and problem solving skills, and more specific technical skills, needed in everyday practical tasks. This challenge is especially acute in computer science and software engineering education, as there has been formal education in this field for a much shorter time than in other, more traditional fields. Therefore, first-hand information about the educational background, knowledge and needs of professionals currently working in the software field would be extremely useful. Such data would provide help in planning a curriculum for educating future software professionals, as well as supplementary education for those already in work life. Unfortunately, there is relatively little such data available. For these reasons, the Department of Software Systems at the Tampere University of Technology decided to conduct a survey about the knowledge and education needs of Finnish Software Professionals. The survey was modelled on a survey conducted in 1998 in Canada by T. C. Lethbridge [2, 3, 4]. In that study, 76% of respondents were from North America. T. C. Lethbridge granted his permission for using the survey as a basis for a new survey in Finland. Apart from the Lethbrige study, other related surveys in the area include [8] and [1]. In the former, 60 information systems professionals from the USA estimated the importance of different skill areas three years in the future for programmers, analysts and end-user support personnel. The latter was a web-based survey where employees of IT companies from Arkansas, USA, estimated the importance of various skills. The results of these two studies emphasized interpersonal and organisational skills, business knowledge, and adaptability to change. [5, 6, 7] describe a number of studies related 1

5 to the education and usefulness of skills for software professionals. This includes an interview of 11 hand-picked Finnish expert level software developers, with the aim of identifying the technical and cognitive skills that are important for expert-level professionals. Among the other studies was a trend analysis of job advertisments in an USA-based professional magazine, and an analysis of detailed technical skills mentioned in a USA-based web recruiting service. The main result of the former study was that the variety of required technical skills for software professionals had increased over the years. The survey described in this article, carried out by the TUT Software Systems Laboratory, was implemented as a voluntary questionnaire answered through a www-form in May Respondents were invited by delivering personal anonymous answering codes by to all the members registered to the Finnish Information Processing Association, which is the national association for ICT professionals and companies, and a member of the International Federation for Information Processing. Altogether, 212 persons answered the survey. The survey asked the respondents questions about 72 topics, which were selected from the following 7 topic categories (an abbreviation in parenthesis): 1. Computer Science Core (CSC) 2. Theoretical Computer Science (TCS) 3. Software Engineering (SoE) 4. Applications of Software (ApS) 5. Other Information Technology (OIT) 6. Mathematics (Mat) 7. Other Topics (Oth) All the topics and the categories they belong to are shown in Table 1. Associated with each of the 72 topics there were four questions: 1. How much did you learn about this topic during your formal education? 2. How much do you know about this topic now? 3. How useful has the detailed knowledge about this topic been to your software-related work? 4. How much has studying this topic affected your way of thinking? The respondents answered the questions by using a scale from 1 ( not at all ) to 5 ( very much ). A lot of information in the form of tables and diagrams, produced by analysing the data from the survey, is available on the www-page of the survey This article is a summary of those results. One well-known drawback of a voluntary questionnaire of this type is that the willingness of respondents to answer the questionnaire could be different for different groups of people, so that the sampling of respondents may not be entirely random. However, since there does not appear to be strong reasons why some types of software professionals would be systematically more or less willing to respond than others, we believe that the sampling is at least close enough to random to provide useful information. Also, detailed background information about the respondents was gathered in the survey. This included questions about their formal education, how long they have been working on software, the types of software they are working on, how they divide their time among various types of tasks, and so on. On the one hand, the resulting distributions show that the respondents do seem to represent a rather wide cross-section of people working in the software field. On the other hand, this background information allows us to study the perceptions of different types of respondent groups, as we will do later in this article. The rest of the article is organised as follows. In Section 2 we present the averages of answers to all questions of all topics. In Section 3 we make a summary of the average data by classifying the topics into just a handful of different profiles. This overview should be readable by even those who do not wish to read the more detailed numerical analysis. In Section 4 we look in detail at the numerical differences of different questions, and what information they convey about the topics. In Section 5 we look at how the perceptions on each topic differ between those who have learned the topic during their formal education and those who have not. In Section 6 we look at the background data gathered about the respondents and how it affects the answers to questions. Finally, in Section 7 we summarize our findings. 2

6 Ctg. Topic Learned in Educ. Rank Current Knowl. Rank Usefulness in Work Rank Infl. on Thinking 1 CSC A Programming Language D 2 CSC Basic Programming Skills D 3 CSC Algorithm Design D 4 CSC Data Structures D 5 CSC Object-Oriented Programming D 6 CSC Programming Language Theory CSC Parsing and Compiler Design C 8 CSC Databases E 9 CSC Operating Systems D 10 CSC Assembly Progr. and Embedded Syst C 11 CSC Reliability and Fault Tolerance E 12 CSC Parallel and Distributed Computing C 13 TCS Performance Evaluation C 14 TCS Computability & Comput. Complexity B 15 TCS Automata Theory & Formal Lang B 16 TCS Formal Program Verification B 17 TCS Formal Specification Methods C 18 TCS Information Theory B 19 SoE Requirements Gathering and Analysis E 20 SoE Specification and Design Methods E 21 SoE Software Architecture E 22 SoE Software Design and Patterns E 23 SoE Software Processes E 24 SoE Testing, Verification and Validation E 25 SoE Softw. Maintenance & Reengineering E 26 SoE Documentation E 27 SoE Config. & Release Management E 28 SoE Usability and User Interfaces E 29 SoE Software Metrics SoE Project Management E 31 SoE Software Cost Estimation E 32 SoE Software Process Standards C 33 ApS Software in Digital Networks D 34 ApS Internet and World Wide Web ApS Other Hypermedia and Multimedia C 36 ApS Mobile and Wireless Software E 37 ApS Security and Cryptography E 38 ApS Computer Graphics ApS Artificial Intelligence B 40 ApS Pattern Recogn. & Image Processing B 41 ApS Comput. Methods for Numerical Probl B 42 ApS Simulation C 43 OIT Computer System Architecture C 44 OIT Microprocessors OIT Microprocessor Design B 46 OIT Digital Systems B 47 OIT Analog Electronics B 48 OIT Digital Communication Technologies C 49 OIT Digital Signal Processing B 50 OIT Mechatronics, Robotics B 51 OIT Measurement Technology B 52 Mat Differential and Integral Calculus A 53 Mat Linear Algebra and Matrices A 54 Mat Probability and Statistics A 55 Mat Logics Mat Sets, Functions and Relations A 57 Mat Combinatorics B 58 Mat Graph Theory B 59 Mat Queuing Theory B 60 Mat Laplace and Fourier Transforms A 61 Mat Control Theory B 62 Oth Physics A 63 Oth Chemistry A 64 Oth Philosophy Oth Psychology E 66 Oth Business Economics C 67 Oth Marketing C 68 Oth Leadership E 69 Oth Giving Presentations and Negotiation E 70 Oth Technical Writing E 71 Oth Foreign Languages E 72 Oth Software-related Law E Table 1: The averages for all 72 topics and 4 questions, including the rank of each average among the values in the same column, and the profile of each topic Rank Profile 3

7 Figure 1: Averages for all topics and questions 2 The Averages Table 1 shows the averages of the answers to the four questions for each of the 72 topics. Table 1 also shows the rank of each value among the values in the same column/question. Here and later we have shortened the question names as Learned in Education, Current Knowledge, Usefulness in Work and Influence on Thinking. The same data is also shown in Figure 1, where each row represents one topic, and the four bars on each row represent the averages to the four questions for this topic. If we combine the averages for the topics within each topic category, we obtain the results shown in Figure 2. Considering first the question that is probably the most interesting of all, namely Usefulness in Work, we can see that the highest averages are in the Software Engineering (3.2) and Computer Science Core (3.1) categories. Other Topics (2.6) and Applications of Software (2.3) position themselves in the middle, while Theoretical Computer Science (1.8), Mathematics (1.8) and Other Information Technology (1.7) are the lowest categories. As for individual topics, the very highest averages are found in the Computer Science Core category, namely Basic Programming Skills (4.1), A Programming Language (4.0) and Databases (3.9), followed by the topics Documentation (3.9), Requirements Gathering and Analysis (3.9) and Software Maintenance and Reengineering (3.7) from the Software Engineering category. 4

8 Figure 2: Averages for all topic categories and questions Immediately after these comes Foreign Languages (3.7) from Other Topics, while Giving Presentations and Negotiation (3.6) from the same category is positioned two places lower. The two highest scores for the Applications of Software category are Internet and World Wide Web (3.2) and Security and Cryptography (3.0), for the Mathematics category Logics (2.7) and Sets, Functions and Relations (2.2), for the Other Information Technology category Computer System Architecture (2.6) and Digital Communication Technologies (2.3), and for the Theoretical Computer Science category Performance Evaluation (2.1) and Formal Specification Methods (1.9). If we look at the topics with the very lowest scores for this question, many are in the Other Information Technology category, such as Microprocessor Design (1.1), Mechatronics, Robotics (1.2), Analog Electronics (1.2), Measurement Technology (1.3), Digital Signal Processing (1.3), and Digital Systems (1.4), and in the Mathematics category, such as Laplace and Fourier Transforms (1.2) and Control Theory (1.2). Among the very lowest averages are also Chemistry (1.2) from Other Topics and Information Theory (1.4) from Theoretical Computer Science. If we look at the Learned in Education question, measuring how well each topic has been learned during formal education, we find that Computer Science Core topics occupy the three top positions, with Basic Programming Skills (3.5) and A Programming Language (3.2) having again the two highest scores. While Foreign Languages (3.0) is again in a high position, namely fourth, a clear difference to the Usefulness in Work question is that Mathematics topics now occupy positions 5 9, the two highest being Probability and Statistics (2.9) and Differential and Integral Calculus (2.9). In fact, Mathematics (2.3) is now the second highest among the topic categories, while Software Engineering (2.0) has fallen to fourth place. Physics (2.6) and Business Economics (2.5) from Other Topics are now also placed near the top. If we look at the very lowest scores, that is, the least well learned topics, the two lowest are again Mechatronics, Robotics (1.1) and Microprocessor Design (1.3) from the Other Information Technology category. Measurement Technology (1.5) from the same category is the 10th lowest, and Information Theory (1.4) from Theoretical Computer Science is again 9th lowest. Control Theory (1.4) from Mathematics is again very low in the list, but among the lowest are now also Mobile and Wireless Software (1.3) and Pattern Recognition and Image Processing (1.4) from Applications of Software category, as well as Software Metrics (1.3) and Software Cost Estimation (1.4) from Software Engineering category and, perhaps unsuprisingly, Psychology (1.4) from Other Topics. 5

9 Figure 3: All 72 topics in a scattergram, with the area occupied by profile A E topics shown 3 Profiles There are 72 topics and 4 questions for each, so even if we only consider the averages of answers, there are altogether 288 values to consider. However, there are certain patterns that are clearly repeated in the data. In this section we will give an easily readable overview of the data of the previous section. We will classify the topics into just a handful of different profiles, based on the averages of the answers to the four questions. The individual numerical differences between the question averages will be analysed in more detail in Section Classifying Topics The classification of topics into profiles is based on Figures 3 and 4. Figure 3 shows a scattergram of all topics, where each topic is represented by a point and the number of the topic as given in Ta- 6

10 Figure 4: The same topics in a scattergram with a different vertical axis ble 1. The vertical axis of the diagram is the difference between the averages to the questions Current Knowledge and Learned in Education. Thus, topics which have been forgotten after studies are low in the picture, and topics which have been learned after studies are high in the picture. The horizontal axis of the diagram is the difference between the averages to the questions Usefulness in Work and Current Knowledge. Thus, topics on which respondents have a lot of knowledge compared to their usefulness are towards the left in the picture, and those on which respondents have too little knowledge compared to their usefulness are towards the right in the picture. Profiles are denoted by letters A to E, and the topics belonging to each profile are shown in the diagram. It should be noted, however, that some topics are not classified to any of these profiles; this is discussed further below. Figure 4 shows a scattergram, where the horizon- 7

11 tal axis is the same as in Figure 3, but the vertical axis now simply the average for Current Knowledge. Thus, Figure 4 can be seen as an alternative projection of the same 3-dimensional scattergram as Figure 3. Topics about which the respondents know very little are now towards the bottom of the picture, and topics about which they know a lot are towards the top of the picture. Individual topics and profiles are shown in the same way as in Figure 3. Table 1 also lists the profile to which each topic has been classified. We will next look into each profile in turn. 3.2 Profile A Typical for profile A topics is that question Learned has the highest average, Knowledge has a lower average, and Usefulness is the lowest of all; Influence, however, has typically a slightly higher average. A typical example is given in Figure 5. In Figure 3, profile A topics are in the lower left corner. Thus, a topic with this profile has been learned reasonably well during formal education, but has been partly forgotten after that, and is not found very useful for practical work. However, it has had some amount of influence on the thinking of the respondents. Learned in Education Current Knowledge Usefulness in Work Influence on Thinking Mat: Differential and Integral Calculus Figure 5: Example of profile A: averages for topic Differential and Integral Calculus (from category Mathematics) Most clearly, this profile includes the classical Mathematics topics often taught to engineering students, namely Differential and Integral Calculus, Linear Algebra and Matrices, Probability and Statistics and Laplace and Fourier Transforms (which has, however, very low averages), as well as Sets, Functions and Relations. Just as clearly, this profile includes Physics and Chemistry from the Other Topics category. 3.3 Profile B This profile is characterised by a rather uniform and low average for all four questions, with sometimes a slightly lower value for Usefulness than for the other questions. A typical example is given in Figure 6. In Figure 3, these topics are near and slightly to the left of the origin (0, 0) of the scattergram. In Figure 4, these topics are near the bottom of the picture. One potential interpretation for the shape of the profile is that these are specialized topics that few people have studied and that few people are perhaps even fully aware of their content and only those same few people consider these topics important. Other analysis based on the effect of learning on perceived usefulness, described in Section 5, provides support for this interpretation for the Mathematics and somewhat for the Theoretical Computer Science topics, while less for the other topics in this profile. Learned in Education Current Knowledge Usefulness in Work Influence on Thinking TCS: Automata Theory & Formal Lang Figure 6: Example of profile B: averages for topic Automata Theory and Formal Languages (from category Theoretical Computer Science) Most Theoretical Computer Science topics exhibit this profile, the only exceptions being Performance Evaluation and Formal Specification Methods, which are classified to profile C. From the Applications of Software category, profile B includes Artificial Intelligence, Pattern Recognition and Image Processing and Computational Methods for Numerical Problems. From the Other Information Technology category this profile includes Microprocessor Design, 8

12 Learned in Education Current Knowledge Usefulness in Work Influence on Thinking CSC: A Programming Language Figure 7: Example of profile D: averages for topic A Programming Language (from category Computer Science Core) Learned in Education Current Knowledge Usefulness in Work Influence on Thinking CSC: Parsing and Compiler Design Figure 8: Example of profile C: averages for topic Parsing and Compiler Design (from category Computer Science Core) Mechatronics, Robotics and Measurement Technology as well as Digital Systems, Analog Electronics and Digital Signal Processing, typically with very low scores for all four questions, and the last three with somewhat lower Usefulness and Influence than Learned and Knowledge. From Mathematics this profile includes the computer science -related topics Combinatorics, Graph Theory and Queuing Theory, as well as Control Theory (the last with especially low averages for all questions). 3.4 Profiles C and D In Figure 3 profiles C and D are located above and to the right from profile B. Let us first consider profile D. Characteristic for profile D is a relatively high rating for Learned in Education, high rating for both Current Knowledge and Usefulness in Work, and typically a slightly less high rating for Influence on Thinking. A typical example of profile D is given in Figure 7. The shape of the profile indicates that the respondents find these topics important for practical work, they feel they have sufficient skills in them, and these topics are, in comparison, also taught reasonably well in formal education. Most of the topics in the Computer Science Core category exhibit this profile, namely A Programming Language, Basic Programming Skills, Algorithm Design, Data Structures, Object-Oriented Programming, and Operating Systems. Software in Digital Networks from Applications of Software category is also best classified to this profile. Some other topics exhibit a similarly shaped profile, but have lower overall averages. These topics form profile C. The difference between profiles C and D is seen especially clearly in Figure 4, where profile C is located below profile D. A typical example of profile C is given in Figure 8. From Computer Science Core this profile includes Parsing and Compiler Design, Assembly Programming and Embedded Systems and Parallel and Distributed Computing. From Theoretical Computer Science category profile C includes Performance Evaluation and Formal Specification Methods, and from Software Engineering category Software Process Standards. From Applications of Software category profile C includes Other Hypermedia and Multimedia and Simulation, from Other Information Technology category Computer System Architecture and Digital Communication Technologies, and from the Other Topics category Business Economics and Marketing. 3.5 Profile E Characteristic for this profile is a medium value for Learned, significantly higher scores for Knowledge and Usefulness with Usefulness typically the highest while Influence on Thinking typically has a slightly lower value than these two. An example is given in Figure 9. In Figure 3 profile E is located at the top right of the diagram. A potential interpretation is that these topics are important specific skills for practical work that, in comparison to their importance, are taught relatively little in formal education, and that the re- 9

13 spondents may still not have entirely sufficient skills in these topics. Learned in Education Current Knowledge Usefulness in Work Influence on Thinking SoE: Testing, Verification and Validation Figure 9: Example of profile E: averages for topic Testing, Verification and Validation (from category Software Engineering) Most importantly, almost all Software Engineering topics fall into this profile. The only exceptions are Software Process Standards and Software Metrics; the former is classified to profile C, and the latter is unclassified, as discussed below. Several topics from the other categories are also classified to profile E. Databases and Reliability and Fault Tolerance from Computer Science Core clearly exhibit this pattern. Also, from the Applications of Software category this profile includes Security and Cryptography as well as Mobile and Wireless Software, the last with somewhat lower averages. Profile E also includes many topics from the Other Topics category, namely Psychology, Leadership, Giving Presentations and Negotiation, Technical Writing, Foreign Languages and Softwarerelated Law. 3.6 Other Profiles There are some topics that cannot be classified to any of the above profiles. One example is Philosophy from the Other Topics category, which has the averages shown in Figure 10. Another example is Computer Graphics from Applications of Software category, which has a clearly higher score for Current Knowledge than for the other questions. This seems to indicate that this topic has usually been learned by the respondents themselves and for mainly other purposes than work. Yet another example is Logics from Mathematics category, which has reasonably high and uniform values for all questions. Thus, in a way, it represents an intermediate form of profiles D and B. The remaining unclassified topics are Programming Language Theory from Computer Science Core category, Software Metrics from Software Engineering category, Internet and World Wide Web from Applications of Software category, and Microprocessors from Other Information Technology category. Learned in Education Current Knowledge Usefulness in Work Influence on Thinking Oth: Philosophy Figure 10: An unclassified profile: averages for the topic Philosophy (from category Other Topics) 4 Differences Between Questions There are clearly some skills that are acquired more in practice than through formal education, while for some others the opposite is true. Similarly, some important topics may receive little attention during education, while others are emphasized more, even though they may not be used a great deal in practice. We can investigate these and similar questions by looking at the differences between answers to the four questions for each topic. We already looked at this information in the previous section when we classified the topics to different profiles. However, we will now look at individual differences one at a time and in terms of absolute numbers. 4.1 Difference between Knowledge and Learned We will first consider the difference between the average of answers to the questions Current Knowledge and Learned in Education for each topic. A potential interpretation for a resulting positive number is that the topic has been, to some degree, 10

14 Figure 11: The difference between Current Knowledge and Learned in Education for each topic: the centre line is zero, left is negative and right is positive learned in practice after studies, and for a negative number that the topic has been somewhat forgotten after studies. Figure 11 shows a bar diagram of the differences for each topic. As we would expect, the topics from the E profile, described in Section 3, have the highest positive differences. Software Engineering topics, such as Software Maintenance and Reengineering (1. pos./difference +1.5), Configuration and Release Management (3./+1.2) and Testing, Verification and Validation (4./+1.2), occupy 9 out of the 12 highest places; the remaining three consist of two other profile E topics, namely Databases (8./+1.1) and Reliability and Fault Tolerance (10./+1.0) from Computer Science Core, as well as the unclassified Internet and World Wide Web (2./+1.3) from Applications of Software. As for the topics with the largest negative differences, these are, unsurprisingly, from the A profile. Mathematics topics take 8 out of the 10 lowest val- 11

15 ues, with the classical mathematics topics such as Differential and Integral Calculus ( 0.7), Linear Algebra and Matrices ( 0.6), Probability and Statistics ( 0.4), and Laplace and Fourier Transforms ( 0.3) having the lowest values. The two other topics among the 10 lowest are Physics ( 0.3) and Chemistry ( 0.2). 4.2 Difference between Usefulness and Learned We next consider the difference between the answers to the questions Usefulness in Work and Learned in Education. A potential interpretation for a positive number is that in comparison to its usefulness, the topic may be taught too little during formal education, and for a negative number that it may be taught too much. Figure 12 shows a bar diagram of the differences for each topic. Table 2 shows the 20 highest values. Pos. Topic Diff. 1 SoE: Softw. Maintenance & Reengineering SoE: Testing, Verification and Validation SoE: Requirements Gathering and Analysis SoE: Documentation SoE: Software Cost Estimation SoE: Config. & Release Management CSC: Reliability and Fault Tolerance Oth: Giving Presentations and Negotiation SoE: Project Management Oth: Technical Writing CSC: Databases ApS: Internet and World Wide Web SoE: Usability and User Interfaces SoE: Software Processes ApS: Security and Cryptography Oth: Software-related Law Oth: Leadership SoE: Specification and Design Methods SoE: Software Design and Patterns SoE: Software Architecture +0.9 Table 2: Difference between Usefulness in Work and Learned in Education: the 20 highest values As we would except, profile E topics exhibit the highest values. Software Engineering topics occupy 12 out of the 20 top positions including positions 1 6, accompanied mainly by other profile E topics. Table 3 shows the 20 lowest values for the difference between Usefulness and Learning. Again, as we would expect, it is profile A topics that occupy this list. Pos. Topic Diff. 72 Mat: Differential and Integral Calculus Oth: Physics Mat: Linear Algebra and Matrices Mat: Probability and Statistics Oth: Chemistry Mat: Laplace and Fourier Transforms Mat: Sets, Functions and Relations OIT: Digital Systems TCS: Automata Theory & Formal Lang TCS: Computability & Comput. Complexity OIT: Analog Electronics Mat: Graph Theory OIT: Microprocessors Mat: Combinatorics Mat: Control Theory Mat: Queuing Theory OIT: Digital Signal Processing OIT: Measurement Technology ApS: Artificial Intelligence CSC: Programming Language Theory 0.2 Table 3: Difference between Usefulness in Work and Learned in Education: the 20 lowest values 4.3 Difference between Usefulness and Knowledge Studying the difference between Usefulness in Work and Current Knowledge gives us information about the present balance between knowledge and needs of software professionals already working in the field, and may thus give us hints for the need for supplementary education. A potential interpretation for a positive number is that software professionals may be in need of further education, and for a negative number that they may have excessive knowledge already. Pos. Topic Diff. 1 Oth: Giving Presentations and Negotiation SoE: Software Cost Estimation CSC: Reliability and Fault Tolerance Oth: Technical Writing SoE: Requirements Gathering and Analysis SoE: Testing, Verification and Validation Oth: Psychology SoE: Documentation SoE: Specification and Design Methods Oth: Leadership ApS: Security and Cryptography Oth: Foreign Languages SoE: Project Management Oth: Software-related Law SoE: Config. & Release Management SoE: Softw. Maintenance & Reengineering CSC: Databases SoE: Usability and User Interfaces SoE: Software Processes SoE: Software Architecture +0.1 Table 4: Difference between Usefulness in Work and Current Knowledge: the 20 highest values 12

16 Figure 12: The difference between Usefulness in Work and Learned in Education for each topic Figure 13 shows the differences for each topic, and Table 4 shows the 20 highest values. It is probably unsurprising that Software Engineering topics are again well-presented; however, they are not as dominating as in the previous two subsections. Instead, there are now many topics from the Other category in the highest places, such as Giving Presentations and Negotiation (1st place), Technical Writing, Psychology, Leadership, Foreign Languages and Software-related Law. The Computer Science Core topics Reliability and Fault Tolerance and Databases, and the Applications of Software topic Security and Cryptography are again high in the list. If we look at the largest negative values, it is again the same profile A topics, namely classical Mathematics topics, Physics (1st place, 0.9) and Chemistry ( 0.6) as in the previous two subsections that are highest in the list. Philosophy ( 0.4), Computer Graphics ( 0.4), Programming Language Theory ( 0.3), Computer System Architecture ( 0.3) and Other Hypermedia and Multime- 13

17 Figure 13: The difference between Usefulness in Work and Current Knowledge for each topic dia ( 0.3) are new topics on the negative list, and Microprocessors ( 0.5) and Artificial Intelligence ( 0.3) are higher in the negative list than before. 4.4 Difference between Influence and Usefulness There is at least one more way in which we can obtain useful information from the differences between the averages for the questions. Some topics provide necessary technical skills that are needed frequently in everyday practical tasks, while other topics affect the way of thinking and develop general problem solving skills, even if the detailed knowledge they provide is not needed as often in everyday work. In order to study the placement of different topics on this axis, we can look at the difference between the questions Influence on Thinking and Usefulness in Work. A potential interpretation for a positive number in this case is that the topic is more important in developing general problem solving skills than as a specific technical skill, 14

18 Figure 14: The difference between Influence on Thinking and Usefulness in Work for each topic and for a negative number the opposite. Figure 14 shows the differences for each topic. The lists of positive and negative values for this difference are in many ways the opposite of the lists in the previous subsections. Many of the profile A and B topics that were high on the previous negative lists are now high on the positive list, such as Physics (1st place, +0.7) and Chemistry (+0.4), and the classical Mathematics topics, for example Differential and Integral Calculus (+0.4) and Probability and Statistics (+0.3), as well as Artificial Intelligence (+0.3), Automata Theory and Formal Languages (+0.2) and Information Theory (+0.2). The list of largest negative values is rather unsurprising, occupied mainly by profile D and E topics. This includes practical topics from the Other category, Foreign Languages ( 0.8), Giving Presentations and Negotiation ( 0.8) and Technical Writing ( 0.8) being the three highest topics in the list, as well as topics from the Software Engineering and Computer Science Core categories, together with Internet and World Wide Web ( 0.5) from the Applications of Software category. 15

19 5 Effect of Learning There are clearly some topics whose importance is recognized by everyone working in the software field, regardless of their educational background. It is often the case that software professionals have learned these skills in their work if not already during formal education. However, in some other cases, only those that have studied a topic during their education have good knowledge of the topic and find the topic important in their work. These may be specialized topics only needed for very specific tasks. Sometimes it may even be the case that only those with sufficient education will know how to take advantage of the involved knowledge in their work. 5.1 Dividing Respondents by Level of Learning To understand the results from the above point of view, we have calculated how the answers to the questions about each topic change when we consider only those respondents who have learned the topic during their studies at least to a reasonable degree. More precisely, for each topic we have selected the subgroup of respondents whose answer to the Learned in Education question is at least 3, and we have calculated how the averages for the other questions about this topic change. In each case, we have also considered the statistical significance of the change. Because the answers are in many cases not normally distributed, we have calculated statistical p values by using the non-parametric Mann-Whitney-U-test, which compares the medians of an ordinal scale variable for two groups; those two groups are here the learned and not learned groups. The result of the statistical tests was that all but one of the changes to Current Knowledge, and by far most of the changes to Usefulness in Work, were statistically significant at the p 5% level. In the latter case there were even a few small negative changes, but none of these were statistically significant. 5.2 Effect of Learning on Current Knowledge Let us first look at the differences in Current Knowledge for the learned subgroups, revealing to what extent learning a topic during education has increased respondents perceived current knowledge of that topic. The topics from Theoretical Computer Science as well as the computer science -related Mathematics topics exhibit the highest differences, for example Information Theory (1st place with absolute change +1.7 and relative change +112%), Automata Theory and Formal Langages (+1.3/+77%), Combinatorics (+1.3/+77%), Formal Specification Methods (+1.2/+66%), Computability and Computational Complexity (+1.2/+66%), Performance Evaluation (+1.2/+56%), Graph Theory (+1.1/+64%), and Formal Program Verification (+1.1/+66%). Somewhat surprisingly, many of the other Mathematics topics are placed quite low in the list, for example Probability and Statistics (63rd place/+0.3/+14%), Sets, Functions and Relations (+0.4/+16%) and Differential and Integral Calculus (+0.5/+23%). Thus, those who have studied these topics more do not seem to have much more knowledge about them than others. As might be expected, almost all of the Software Engineering topics are placed in the lower half of the list, with the striking exception of Software Metrics (+1.5/+98%), the second highest of all topics. Topics from Computer Science Core are spread very widely, with some topics quite high on the list, such as Parallel and Distributed Computing (+1.5/+79%) and Assembly Programming and Embedded Systems (+1.3/+68%), while most are near the low end, including the three very lowest topics, A Programming Language (+0.2/+5%), Databases (+0.1/+4%) and Basic Programming Skills (+0.1/+2%). Thus, the perceived skills on these topics do not depend on whether people have studied them or not. Applications of Software and Other Information Technology topics are also spread quite widely, with the Other Information Technology topics on the average slightly higher of the two, with nearly all of its topics in the top half of the list. Interestingly, Computer Graphics (+1.0/+44%) is reasonably high on the list, even though it also has a reasonbly high value (+0.5) for the Current Knowledge Learned in Education difference (Section 4.1), implying that the same people who have studied the topic during their formal educa- 16

20 Figure 15: The distribution of the number of years in the field tion have also learned much more on their own. 5.3 Effect on Perceived Usefulness We next look at the differences in Usefulness in Work for the learned subgroups, revealing how much learning the topic changes the perceived usefulness of the topic. The results are very similar to the ones in Current Knowledge in terms of the relative placement of different topics. In particular, Theoretical Computer Science, Mathematics and Software Engineering topics are distributed in the way described above. The absolute and relative changes in the numbers are smaller than in the above case, however. The three topics with the very highest absolute changes are now Parallel and Distributed Computing (+1.1/+58%), Software Metrics (+0.9/+60%) and Combinatorics (+0.9/+57%), while the three with the very lowest absolute values are Software Process Standards ( 0.1/ 7%), Security and Cryptography ( 0.1/ 2%), and Software Maintenance and Reengineering (+0.0/+1%). Computer Science Core topics are no longer at the very bottom of the list, but are still in the lower half, for example A Programming Language (+0.2/+4%), Databases (+0.2/+4%), Operating Systems (+0.1/+3%) and Basic Programming Skills (+0.1/+3%). Thus, people recognize the importance of these topics regardless of whether or not they have studied them. 6 Respondent Backgrounds As described in the introduction, the survey asked the respondents various questions about their educational and professional background. We will next look at some of this background information. In cases where the possible answers to a question can be arranged on an ordinal scale, we have calculated the correlations between the answer and the answers to other questions. As a correlation measure we have used Spearman s non-parametric correlation, and in each case we have also calculated the statistical significance of the correlation. In several cases we have also divided the respondents to two groups based on the answer to a background question, and calculated the difference in answers to other questions, as well as the statistical significance of the difference. Because many variable distributions are non-normal, we have calculated the statistical difference by using the nonparametric Mann-Whitney U-test, which compares the medians of an ordinal scale variable for two groups. 6.1 Number of Years in the Field The respondents were asked how many years they have been working in the software field. The distribution of the anwers to this question is shown in Figure 15. Let us first look at the correlations of this background question to the questions about the topics. We find that there are a number of topics that those who have been longer in the field have studied less, especially in the Applications of Software and Software Engineering categories (correlations to category averages being 0.31 and 0.14, respectively). The strongest negative correlations with Learned in Education were for Object-Oriented Programming ( 0.46), Internet and World Wide Web ( 0.45), Other Hypermedia and Multimedia ( 0.43), Software Processes 17

21 ( 0.30), Mobile and Wireless Software ( 0.28), Usability and User Interfaces ( 0.27), Software in Digital Networks ( 0.25), Digital Communication Technologies ( 0.24), Digital Signal Processing ( 0.24), Security and Cryptography ( 0.22) and Specification and Design Methods ( 0.20). There were virtually no topics that those longer in the field had learned significantly better; the highest positive correlation was 0.16 with p 4% for Sets, Functions and Relations. It is notable that the perceived Current Knowledge had for almost all of the above-mentioned topics no statistically significant correlation to the number of years in the field. This would seem to indicate that those who have worked longer have somehow closed the cap in the knowledge of these topics. In fact, there are several topics that those longer in the field feel they master better, for example Project Management (0.27), Software Maintenance and Reengineering (0.24), Data Structures (0.24), Psychology (0.23) and Software Cost Estimation (0.21). 6.2 Most Important Degree The respondents were asked about their most important degree regarding their work in the software field. Out of those who answered the question, 3.6% had a doctor s degree, 38.9% had a master s degree and 52.3% had a bachelor level degree. For calculating correlations we used an ordinal scale of degrees, other being the lowest and doctor being the highest. Unsurprisingly, the level of degree had a positive correlation with the Learned in Education and Current Knowledge questions for the majority of topics, and correlation was usually higher for more theoretical topics. The highest correlations were found among Theoretical Computer Science and Mathematics topics, certain Computer Science Core topics such as Algorithm Design and Parallel and Distributed Computing, and certain Applications of Software topics such as Artificial Intelligence and Computational Methods for Numerical Problems. For example, the highest individual correlations for the Current Knowledge question were Automata Theory and Formal Languages (0.52), Graph Theory (0.52), Computability and Computational Complexity (0.51), Combinatorics (0.47), Linear Algebra and Matrices (0.43) and Performance Evaluation (0.41). There were also a few topics that those with a higher-level degree had learned less, although these negative correlations did not exist for the Current Knowledge of these topics. These were all businessrelated Other topics, such as Marketing ( 0.28) and Giving Presentations and Negotiation ( 0.25). As for the correlation to the perceived usefulness of topics, probably the most clear pattern was that those with a higher education considered most Theoretical Computer Science as well as computer-science related Mathematics topics more useful than others. The correlation to the Theoretical Computer Science category Usefulness average was 0.25; the three very highest correlations were Automata Theory and Formal Languages (0.42), Combinatorics (0.40) and Graph Theory (0.38). There were also correlations to other Mathematics topics and Physics (0.20), but most of these were less strong. A more detailed analysis reveals that a large proportion of this correlation is due to the respondents with a doctor s degree, whose answers differ most clearly from those of other respondents. For example, if we divide the respondents to two groups based on whether they have a universitylevel degree or not, the ones with a university-level degree find Graph Theory 64% more useful, and Automata Theory and Formal Languages 54% more useful than others. On the other hand, if we divide the respondents to the group with a doctor s degree and the others (including those with some other universitylevel degree), the doctors find Graph Theory 132% more useful, and Automata Theory and Formal Languages 82% more useful than the others. Other very high proportional differences in the perceived usefulness between doctors and others are, for example, Linear Algebra and Matrices (+148%), Information Theory (+120%), Combinatorics (+116%) and Sets, Functions and Relations (+102%). The respondents were also asked about the year of their most important degree. As one would expect, these results were mostly the reverse of what was described above in Section 6.1 for the number of years in the field. One difference, however, was that those with a later year of degree found certain Theoretical Computer Science topics more useful in their work, for example Automata Theory and 18

22 Figure 16: The distribution of the portion of time spent on project management and other management Formal Languages (0.32) and Computability and Computational Complexity (0.27); there was also a stronger correlation for learning these topics in education. 6.3 Time Spent on Various Tasks The survey asked the respondents how much time they spend on each of seven types of tasks: project management and other management requirements and software specification, specification inspections software and architecture design, design inspections working at the software code level (writing and inspecting code, etc.) testing software made by other people installation, training, customer support maintaining software and documentation made by other people (modifying and getting acquainted with) From the data it is clear that people are typically involved with several different types of tasks. Also, certain patterns emerge about the typical combinations of tasks. For example, those that spend a lot of time working at the software code level also work at the software and architecture design level; they may or may not work at the requirements and software specification level, but they typically do not do very much project management. Project managers, on the other hand, also typically work at the requirements level, and they may or may not work at the design level, but they do not work so much at the code level. The distribution for the proportion of time that the respondents spend on management is shown in Figure 16, and the distribution for the time that the respondents spend at the code level in Figure 17. On the other hand, the time people spend on testing software made by other people is, perhaps surprisingly, statistically independent of any of the previously mentioned tasks, so people doing the above tasks may or may not do also software testing. The distribution of time spent on testing is shown in Figure 18. A lot of time spent on testing, however, does correlate with spending time on installation, training and customer support, as well as on maintaining software and documentation. These two are not correlated to each other, but the latter has a large negative correlation to project management, so project managers do not typically spend a lot of time on maintenance; nevertheless, they may or may not spend time on installation, training and customer support, since these two are independent. Interestingly, the level of education, as measured by the most important degree, has a relatively modest correlation, 0.16, to the time spent on project management, and no statistically significant (positive or negative) correlation to any other type of task. The number of years in the field, however, has a slightly higher correlation of 0.23 to project management work. Correlations to Topic Questions Unsurprisingly, those who do more project management work find several practical, project and business-related topics more important in their work, the highest correlations being Project 19

23 Figure 17: The distribution of the portion of time spent on working at the software code level Figure 18: The distribution of the portion of time spent on testing software made by other people Management (0.52) and Software Cost Estimation (0.36) from the Software Engineering category, and Leadership (0.49), Business Economics (0.34), Marketing (0.33), Giving Presentations and Negotiation (0.31), Psychology (0.30) and Softwarerelated Law (0.27) from the Other category. The next two topics on the list are, however, Mathematics topics, namely Probability and Statistics (0.25) and Combinatorics (0.24). Based on the observations in the previous subsection, it is unsurprising that project managers find less useful A Programming Language ( 0.30) and Basic Programming Skills ( 0.28). Working on requirements and software specification has a statistically significant correlation to the Usefulness of almost all Software Engineering topics (the only exception being Software Metrics with correlation 0.16 at p 9% level); these correlations range from 0.20 (Software Process Standards) to 0.40 (Software Cost Estimation). There were significant correlations also to, for example, Formal Specification Methods (0.25), Databases (0.25) and Giving Presentations and Negotiation (0.23). Working on software and architecture design has similar correlations as working on specification to the perceived usefulness of Software Engineering topics except for the cost, project management, standards and usability related topics. Working at the design level has, additionally, significant correlations to most Computer Science Core topics, except for the operating system, low-level programming and compiler related topics; the significant correlations for this category range from 0.19 (Parallel and Distributed Computing) to 0.37 (A Programming Language). Working at the software code level has very similar correlations to the Usefulness of Computer Science Core topics as working at the design level, except that correlation to the basic programming topics is extremely high (for example, correlation for Basic Programming Skills is 0.67), and there is no significant correlation to Parallel and Distributed Computing. Correlation to the Usefulness of Software Engineering topics is weaker and more varied, with Software Maintenance and Reengineering (0.42) being significantly stronger than any of the others. Working at the code level also has a number of statistically significant negative correlations to various business- and management-related Other top- 20

24 Figure 19: The portion of respondents who have worked on a type of software ics, an example being Leadership ( 0.24). Rather surprisingly, testing software made by other people has hardly any significant positive or negative correlations to the usefulness of any topics, not even to the Software Engineering topic Testing, Verification and Validation. In fact, the only statistically significant correlations at the p 5% level are the negative ones to Software Processes ( 0.17) and A Programming Language ( 0.15). The amount of time spent on installation, training and customer support has a significant positive correlation to various Other Information Technology topics, such as Computer System Architecture (0.38), Microprocessors (0.22), Digital Communication Technologies (0.21); Applications of Software topics such as Computer Graphics (0.25), Pattern Recognition and Image Processing (0.25), Security and Cryptography (0.24) and Internet and World Wide Web (0.21); and Computer Science Core topics, such as Operating Systems (0.31). There are also a number of statistically significant negative correlations to Software Engineering topics and the basic programming topics from Computer Science Core. The amount of time spent on maintaining software and documentation has very few significant positive correlations to the usefulness of topics. Rather surprisingly, it does not have nearly as strong correlation (0.18) to Software Maintenance and Reengineering as does working at the code level (0.42). Working on maintenance has, however, negative correlations to most of the topics from the Other category, especially the practical and businessoriented ones, the strongest being Foreign Languages ( 0.23), Philosophy ( 0.21), Psychology ( 0.21), Leadership ( 0.20), Giving Presentations and Negotiation ( 0.19) and Marketing ( 0.19). 6.4 Types of Software The respondents were asked about the types of software they had worked on during the previous two years. The respondents could select one or several choices from six types of software: software as integrated part of consumer goods, such as cellular phones or television sets telecommunications, networking and internet services other real-time, embedded software business and management oriented applications, systems tailored for companies software for consumer and mass market, such as word processing and games special software in none of the above categories The portions of respondents who have worked on each type of software are shown in Figure 19. For each software type we have divided the respondents to two groups based on their answer: those who have worked on that type of software, and others. We have then calculated the difference in averages to the topic questions between the two groups. Effect on Answers Working on software integrated into consumer goods creates only few statistically significant differences in the perceived usefulness of topics, the highest differences being, unsurprisingly, Mobile and Wireless Software (+1.5) and Assembly Programming and Embedded Systems (+1.2). 21

25 Figure 20: The distribution of company size The highest differences in perceived usefulness of those working on telecommunications, networking and internet services are for networking and mobility related Applications of Software topics such as Software in Digital Networks (+0.9), Mobile and Wireless Software (+0.8) and Internet and World Wide Web (+0.7); for Other Information Technology topics such as Digital Communication Technologies (+0.7) and Microprocessors (+0.6); and for several Computer Science Core topics such as Object-Oriented Programming (+0.6) and Operating Systems (+0.5). Working on other real-time, embedded software changes the perceived usefulness of topics more than working on any other type of software. The highest changes are in Other Information Technology topics such as Microprocessors (+1.6), Measurement Technology (+1.2), Analog Electronics (+1.0), Digital Systems (+1.0), and Mechatronics, Robotics (+0.8); Computer Science Core topics such as Assembly Programming and Embedded Systems (+2.4), Algorithm Design (+1.1), Programming Language Theory (+0.9), and Operating Systems (+0.8); and in Applications of Software topics such as Mobile and Wireless Software (+1.0), and Simulation (+0.9). There are also significant changes in Control Theory (+1.1) and Physics (+1.1). Working on business and management oriented applications and systems tailored for companies creates statistically significant changes in the usefulness of only two topics which are, unsurprisingly, Databases (+0.7), to which all the previous software types are negatively correlated, and Business Economics (+0.5). The respondents working on software intended for the consumer and mass market consider more useful the Applications of Software topics Computer Graphics (+1.3), Security and Cryptography (+1.4) and Software in Digital Networks (+1.0), as well as Foreign Languages (+1.0) from Other Topics. Working on other, unclassified types of software did not cause any statistically significant positive changes in the usefulness of topics. 6.5 Company and Group Size Figure 20 shows the distribution of the size of the companies the respondents have been working in. Probably the most interesting result is that company size has an almost uniformly negative, although mostly modest, correlation to the Current Knowledge of topics. This correlation is either statistically significant, or nearly significant (with p 6%), for the category averages of five of the seven topic categories (all but Other Information Technology and Other Topics). A similar negative correlation is also present, but somewhat less uniformly, for Learned in Education and Usefulness in Work. An analysis of distributions shows that this correlation is not caused by the very smallest or the very largest companies alone. An even more detailed analysis reveals that for some topics, especially from the Theoretical Computer Science category, the main immediate cause for the correlation is that in larger companies there are proportionally much more respondents who know nothing about the topic. However, for many other topics, for example most of the Software Engineering topics, this is not true; instead, the distribution just tends to shift to slightly lower values for larger companies. Also, a partial correlation analysis reveals that the correlation is not caused, e.g., by the education level of the respondents. Some potential explanations for this correlation could be that people working in smaller companies 22

26 Figure 21: The distribution of group and software size are, on the average, involved with a greater variety of different tasks and therefore develop skills in a greater variety of topics, or that smaller companies are, on the average, slightly more selective in hiring emloyees. The respondents were asked to select the most descriptive choice from a given list of group and software sizes. Figure 21 shows the distribution of answers to this question. Contrary to the company size, group and software size has a modest positive correlation to the Current Knowledge and also to the Usefulness of the majority of topics. These positive correlations are strongest in the Software Engineering, Theoretical Computer Science, Other Topics and Applications of Software categories, and part of the Computer Science Core category. 7 Conclusions In this article we have presented the results of a survey where 212 Finnish software professionals gave their perceptions about 72 topics by answering four questions about each topic. We presented the averages of the answers to the questions, and how the topics are ordered with respect to each question. We divided the 72 topics into a small number of different profiles based on the averages to the four questions. We also supplemented this analysis by a detailed numerical analysis of differences between the questions. This showed us to what extent topics are learned at work or forgotten after studies, how well topics are taught in formal education compared to their usefulness, about which topics software professionals have sufficient skills, and whether topics develop thinking and general problem solving skills or detailed technical abilities. We also analysed for each topic how the perceptions of those who have learned the topic in their education differ from the average. Finally, we looked at the background information gathered about the respondents, such as their educational and professional history, how they divide their time among various tasks, the types of software they work on, and so on. We looked at the distributions of these background variables and how the answers of different respondent groups differ. The main purpose of presenting these results is to provide first-hand information that can serve as a basis for evaluating and planning education for present and future software professionals. There are some trends in the data that are especially clear. Certain core skills such as basic and object-oriented programming skills and data structures, were both learned reasonably well and found almost universally important, even though their importance, like for most topics, was slightly different for people doing different types of work. On the other hand, there were topics that had not been sufficiently learned during education. This includes a few of the core skills such as reliability and fault tolerance and databases, a number of topics related to the software development process such as planning, requirements engineering and testing, and several general topics needed in professional life, such as presentation, negotiation and writing skills, leadership and economics. Among these topics were also found the most potential candidates for supplementary education. On the other hand, several classical mathematics topics frequently taught in engineering education, such as linear algebra, differential and integral calculus and statistics and probability, as well as physics and chemistry, were studied excessively compared to their usefulness. It should be noted, however, that these topics had had some influence on the thinking for the respondents, and those working on real-time software found physics somewhat more useful than others. 23