!!!!!!!!! Report on the survey Mikkel Hjorth, Ole Sejer Iversen, Rachel Charlotte Smith, Kasper Skov Christensen & Paulo Blikstein

Transcription

1 !!!!!!!!! Report on the survey Mikkel Hjorth, Ole Sejer Iversen, Rachel Charlotte Smith, Kasper Skov Christensen & Paulo Blikstein 1

2 Table of Contents REPORT ON THE SURVEY 1! REPORT ON THE SURVEY 4! 1! THE SURVEY 5! 1.1! HYPOTHESIS 5! 1.2! THE QUESTIONNAIRE 5! 1.2.1! TYPES OF QUESTIONS 6! 1.2.2! ORDERING OF QUESTIONS 7! 1.3! SURVEY ADMINISTRATION 8! 1.4! RECRUITMENT OF SCHOOLS 9! 2! DATA TREATMENT 11! 2.1! BLANKS 11! 2.2! DUPLICATE ENTRIES 11! 2.3! EXCLUDED BECAUSE OF AGE 11! 2.4! COMPLETION 11! 2.5! CONTENT AND LIMITATIONS OF THIS REPORT 10! 3! PARTICIPATING SCHOOLS 12! 4! RESPONDENTS 14! 4.1! AGE AND GRADE 14! 4.2! FABLAB AND NON FABLAB SCHOOLS 15! 5! ABILITIES TO USE, MASTER AND UNDERSTAND DIGITAL TECHNOLOGIES 18! 5.1! CONSUMERS OF DIGITAL TECHNOLOGY 18! 5.2! LEISURE TIME SPENT ON IT COMPARED TO OTHER TYPES OF LEISURE 18! 5.3! USE OF INFORMATION TECHNOLOGY OUTSIDE OF SCHOOL 19! 5.4! SELF-PERCEIVED KNOWLEDGE OF IT 21! 5.5! FABRICATION TECHNOLOGIES 23! 6! OFFICE LITERACY: IT IN SCHOOLS? 25! 6.1! WHAT DO THEY LEARN IN SCHOOL? 26! 6.2! DO THEY LEARN DIGITAL FABRICATION IN SCHOOLS? 28! 7! ABILITIES TO ACTIVELY ENGAGE IN HETEROGENEOUS COMMUNITIES OF PRACTICE AND TO THINK AND ACT INNOVATIVELY (WITH TECHNOLOGY) ON SOCIETAL CHALLENGES 30! 7.1! CREATIVITY, IMAGINATION AND COOPERATION 30! 7.2! INVENTIONS 31! 2

3 7.3! THE DEMENTIA TASK 33! 7.3.1! TRYING TO CREATE A SITUATION IN WHICH THE DEMENTED DO NOT TRY TO LEAVE/GET LOST 34! 7.3.2! PREVENTING THE DEMENTED FROM LEAVING THE NURSING HOME 34! 7.3.3! TRACKING AND FINDING THE DEMENTED, ONCE THEY HAVE LEFT 35! 7.3.4! DATA ANALYSIS 36! 7.3.5! IMPORTANCE OF STAKEHOLDERS 38! 7.3.6! THE DESIGN PROCESS 39! 7.4! SUMMING UP 41! 8! SUMMARY AND CONCLUSION 42! 9! LITERATURE 44! APPENDIX I: THE QUESTIONNAIRE 46! 3

4 Report on the survey This report contains findings from a survey on Danish adolescents aged years in the fall of It is a part of the FabLab@school.dk research program, which investigates the use of digital fabrication technologies in Danish schools. We would like to thank all the participating schools and the municipalities of Aarhus, Silkeborg, Vejle and Favrskov. 4

5 1 The survey is a Danish research project at Aarhus University supported with a grant from The Danish Industry Foundation. It is part of the global FabLab@school initiative, started by the Transformative Learning Technologies Lab at Stanford University. The Danish research project focuses on Fablabs as a hybrid learning laboratory, which combines digital fabrication, design thinking, collaborative idea generation and creating in solutions to complex societal challenges. The survey reported here has been done in cooperation with Stanford University, and parts of the survey are run in various countries around the world in order to establish the possibility for comparisons on a global scale. Aarhus University is cooperating with Aarhus, Vejle and Silkeborg municipalities in the FabLab@School.dk project. Some of the measures in the report have been validated in the FabLab@School project running at Stanford University. Others are tentative measures, which are guiding our further investigations, but which are not yet established as valid measures of the concerned traits. This report is mainly descriptive in its approach to the collected data, and it serves the purpose of presenting these data in a way, that lends itself to further exploration. Apart from the survey reported here, the research project consists of Observations, Interviews with teachers and students and on interventions. 1.1 Hypothesis The OECD has introduced the term of 21 st century skills and defined a range of such skills. Among these are: Abilities to use, master and understand digital technologies Abilities to actively engage in heterogeneous communities of practice Abilities to think and act innovatively (with technology) on societal challenges It is a central hypothesis of this research project, that adolescents aged years through hands-on education with digital fabrication technologies can improve these abilities significantly compared to existing offers in the Danish school system. 1.2 The questionnaire The research program investigates this hypothesis from different angles one being this survey, which will be compared to a second round of the same survey after two years. The above three 21 st century skills have been operationalized into 227 questions in 6 themes, which are: 5

6 1. Personal information 2. School & Leisure 3. Media and technology in everyday use 4. Technology in school 5. Design and Creativity 6. Hacking and repair of technology The following table gives an overview on how the 6 themes were used for investigating the initial 3 areas of interest Areas of interest Themes Number of questions Personal background and interests Abilities to use, master and understand digital technologies Abilities to actively engage in heterogeneous communities of practice Abilities to think and act innovatively (with technology) on societal challenges Personal information School & Leisure Media and technology in everyday use Technology in school Hacking and repair of technology Hacking and repair of technology Design and Creativity Three main areas of interest 6 main themes 227 questions Types of questions In the survey, four types of questions were used to investigate the themes. Multiple-choice questions with a scale from 1 (strongly disagree) to 6 (strongly agree) were used to gain insight into the students views and perspectives on technology and design process related topics. In order to gauge self-perceived abilities within the areas of interest, multiple choice questions with scales ranging from 1 (I know nothing about it) to 6 (I could teach others about it) were used. Time used on leisure activities and different types of IT-use was measured through multiple choice questions with different ranges. Finally, open-ended questions and tasks were used in order to evaluate students' abilities and mindsets with regards to the aforementioned areas of interest. The latter method 6

7 involves a coding of the responses with regards to categories found in the coding process. This type of questions affords opportunities for comparing self-perceived ability with scores or categories on a type of performance. An example of this is that the students were asked how creative they are, and how good they are at coming up with new ideas. In this way, insight into their self-views was gained, while the students on the other hand were asked for ways to solve a problem of demented elderly disappearing from nursing homes and sometimes dying before they are found. An evaluation of their responses on the open-ended questions suggested, that the students think of creativity as something different than abilities to think and act innovatively (with technology) on societal challenges. It is important to note, that in many questions, we ask the students to evaluate themselves. This method is of course prone to different types of biases. For one, students do not always know their level of competence. A testament to this, is that the recent Danish report on International Computer and Information Literacy Study (Bundsgaard et al. 2014) found, that Danish male students have more unrealistic images of their abilities with regards to IT abilities, than do their female counterparts. Another problem are so-called demand characteristics: Students answers are often influenced by their wish to find the "right" answer, that is to answer, what they think, we as researchers or teachers want to hear. They experience the survey as a test, and they want to do well on this test. When using scales in multiple choice questions, it is common to recommend using scales with an uneven number of response possibilities because respondents whose views are genuinely in the middle of the scale are otherwise forced to answer to one of the sides and are thus misrepresented in the data (Marsden and Wright 2010). On the other hand by taking away the easiest category, even those respondents who are prone to satisficing in order to finish quickly will at least have an extra incentive to reflect on if they are on one side of the middle or the other Ordering of questions The themes were ordered in the abovementioned sequence of: 1. Personal information 2. School & Leisure 3. Media and technology in everyday use 4. Technology in school 5. Design and Creativity 6. Hacking and repair of technology Though it is quite common to have background and demographic characteristics at the end of a questionnaire (Marsden and Wright 2010), we chose to start off with these same questions. We did this for two reasons: One was that we were a bit unsure, if all students would reach the end of the questionnaire (and the data would be useless without correct background information), and the other was for 7

8 motivational reasons: We did our best to communicate to the students that we were very interested in them and their particular answers, and thus that this questionnaire was indeed not a test. The questions were grouped by content in order to facilitate respondents'' cognitive processing (Marsden and Wright 2010). In a sense the ordering of groups of questions represents an increasing complexity with regards to our foci. We have had two aims with our ordering of the questions and themes. One was for the respondents to get a sense of a common thread running throughout the questionnaire to make the questions make sense to the respondents. Another was not to reveal too much about our views on technology and design before the students answered questions about their use of technology and their takes on designing and creativity. This was done in order to minimize demand characteristics, which could potentially lead students to try to give us the answer, they thought we wanted the most. The wording of the questionnaire was carefully selected with a goal of having as little text as possible in order to speed up the reading process and of being as precise and easy to understand as possible in order to secure valid data and minimize fatigue and satisficing resulting from this fatigue. The questions and the questionnaire was tested on small groups of adolescents within the age group on four occasions until we did not seem to get much new information out of these tests. The tests were carried out by asking the students to read the questions aloud, which revealed which words and wordings were difficult to understand. Furthermore, the students were asked to discuss their answers, and this discussion revealed how the students interpreted the questions. At the end, the researchers interviewed the students about filling out the questionnaire. 1.3 Survey administration The survey was done among year olds in 39 schools in 4 municipalities (Vejle, Silkeborg, Aarhus and Favrskov). It was carried out by researchers from Aarhus University in the period of August 25 th September 12 th, In the classroom, the survey was in most cases administered by two researchers (although in a few instances towards the end of the administration period only one researcher was present), who were present in the classroom during the entire test in order to help with questions and technical problems. Studies suggest, that this might help in the sense that fatigue-effects are slower to set in (Marsden and Wright 2010). The session started with the researcher introducing the project (2 minutes) and the survey (2 minutes). To increase reliability, the introduction was done based on the same document, and hence every classroom was given the same information in the introduction. The response time varied from approximately 30 minutes and up to an hour. The teachers and the students were in general very flexible with the students' time, which allowed us to get data from most students (see below). 8

9 The survey was administered as an online survey through SurveyXact. We had the students enter the webpage fremtid.eu, on which we had a pop-in link to the survey. In a lot of cases, this worked flawlessly, but we did experience major technical problems with everything from internet connection problems to server issues. The survey worked on computers, tablets and phones, and therefore some technical issues were solved by letting the students use their own phones/devices on the mobile network, while others were solved by tricking school computers to enter other networks than the faulty one, which they had been locked to. Because of technical problems, some students, who had started answering the survey before the problems arose, had to start over again (some even had to wait to a different day), and this lead to two entries in our database. We had a total of 1433 entries, but we ended up with 1156 responses (see data treatment below). 1.4 Recruitment of schools 21 of the participating school were designated future fablab-schools by the municipalities. This means, that there were plans to let these schools be a part of the FabLab@School.dk-project - and that some of them already were. Being part of the FabLab@school.dk project means different things in the three participating municipalities, but common to all three is of course the use of digital fabrication technologies. We asked the municipalities to find us corresponding groups of schools, which were not planning to be a part of the project. Ideally these schools should match the first group. The survey was carried out on 18 schools outside the project. We made sure, that we had as diverse a group of schools as possible. We have included rural and urban, low-achieve and high-achieve, low- and high socioeconomic status schools in order to be able to screen for effects from school type. Since we did not have the possibility of creating a representative sample of schools, we do not claim to be able to conclude on Danish adolescents as such, but we make all our claims with regards to our sample. 1.5 Data collection When nothing else is noted, we do not distinguish between fablab and nonfablab schools. We specifically asked the project schools to assign a class, which had not been a part of the fablab@school.dk-project yet, to this test. We did this as we were trying to establish a baseline against which it would be possible to measure the outcome of fablab in education after two years time. This means, that the two groups of students should not differ in terms of exposure to digital fabrication, and thus we have pooled the data from the two groups into one big group in this report. On most schools we only tested one class or group. We did this in order to get responses from as many schools as possible in order to get as 9

10 much diversity with regards to background factors, such as e.g. socio-economic status, as possible. Initially, the contact to the schools was made through a letter to the principals asking them for 45 minutes with a grade 6, 7 or 8 class of their choice. With most of the schools, it was necessary to call several times in order to make an arrangement. In the beginning, most principals chose to go with grade 7, and when this became apparent, we did our best to get 6 th and 8 th graders on board. Our original aim was for 1200 students with 200 in each group consisting of a specific grade of students in either fablab or non-fablab schools. Since we do not distinguish between the schools within the project and schools outside the project in this report however, this goal was unnecessarily high for the results described here. After the data-collecting period had ended, the data was exported from SurveyXact to Microsoft Excel. In the preparation of the data, only respondents who had reached the end of question 199 were kept. Deleted from the data were also entries with an age outside our range of year olds. Furthermore entries before the administration period were deleted, since these were test runs. As stated earlier, this left us with 1156 responses. For statistical analysis, both SAS and R have been used. 1.6 Content and limitations of this report This report is ordered by themes which have been revealed during the analysis of the data. It describes the frequency of answers on questions, and the report further tentatively explores composite measures of e.g. self-perceived creativity. These measures have not been thoroughly validated in previous work, and it is not the scope of this report to so. In chapter two of this report, treatment of the data is discussed with a focus on which records were removed from the dataset and why. Chapter 3 compares participating schools with the population of schools in general and in chapter 4, the group of respondents is analysed. Chapter 5 concerns the students' abilities to use, master and understand digital technologies and in chapter 6 is an analysis of what the students claim to learn about IT in schools (Office Literacy). Chapter 7 is a comparison of the students self-perceived creativity compared to their experiences with working with ideas and relating to complex societal challenges. Chapter 8 is the conclusion, which is followed by a list of references and an appendix containing the questionnaire. 10

11 2 Data treatment The data was downloaded from SurveyXact as a Microsoft Excel file. In order to facilitate easy data calls from SAS(r) and R (r), the variable names were changed to s1-s227. A document matching these new variable names to the original variable names and to the variable labels, was created and called labels.xlsx. In the original data file, there were 1433 entries. 2.1 Blanks When a person creates a questionnaire without filling in anything at all, it is counted as a blank. There can be several reasons for blanks. One such cause of blanks was students, who got confused and decided to start over, and another one was when the researchers/administrators of the test wanted to see, if the system was up and running. For this reason, the blanks should not be counted as responses. 112 Blanks were deleted from the dataset, which means, that there were 1321 responses to the survey. 2.2 Duplicate entries Due to the many technical problems, several students needed to start over on the survey thus creating duplicate entries. In each case, the entry with the most answers was kept in the data set, and the others were deleted. There were 163 records of multiple entries. Of these, 5 persons had three entries, and 74 had duplicates. A total of 84 entries was deleted from the 163, which left Excluded because of age In this survey, we are researching year olds, and therefore any entries out of this range were deleted. 6 entries had put something un-age related in the age field, and they were deleted (1231 left), whereas 10 entries had stated an age above 15 and were deleted as well (1221 left). 2.4 Completion It was decided to keep all responses that were either completed or were only missing the last task of answering what was inside a key fob. Of the remaining 1221, 65 had not answered the last question before the key fob task (s199). Since respondents were prompted to answer all questions, it was not possible to be missing this item, if the subsequent items were answered. It was also not possible to have answered this item without answering the items before it (except for open ended items). To sum it up, if there was an answer to item s199, there would also be answers on the items before it. If item s199 was not answered, neither would the subsequent items be. For this reason, the responses, which were blank with regards to item s199, were deleted (and the rest were kept), which meant that the total number of respondents ended up being A number of index variables have been added during the analysis of the data. These variables will be introduced, in the groups of items they refer to. 11

12 3 Participating schools The 1156 responses came from 39 schools in the municipalities of Vejle, Silkeborg, Aarhus and Favrskov. The sample includes a wide range of schools, though all except one are public schools. 1 The schools are positioned in both rural and urban locations and recruit from a wide range of socioeconomic groups. As a crude comparison of our sample of schools to the population of Danish public schools in general, the average score for grade 9 students from these schools on the national examinations in the years 2011/2014 has been calculated (see below). Three numbers are publicly available on each school: The average score, the expected average score (based on socioeconomic status of the students), and the difference between the first two, which is supposed to indicate the performance of the school. In the table below is both the average of these numbers based on the schools in our sample and the weighted average, in which the average is calculated based on the number of respondents from each school. Table 1: Average marks of participating schools and their expected average marks. Both un-weighted and weighted. 39 schools 1156 resp. Average Weighted Weighted Exp. Exp. Average Average Aver. 6,95% 6,98% 6,84% 6,87% The average score of all students in Denmark is evaluated in different ways. First of all, it can be split into different school types such as public schools, private schools etc. With one exception (N. Kochs skole with 17 respondents) our sample consisted solely of public schools. 2 The national average in these same three years among all school types in this age group is 6,6. As can be seen, the schools we visited, were on average placed higher than this mean, both when considering actual scores and when looking at expected scores, which can be seen as a measure of average socioeconomic status of pupils in the given schools. The same goes for the weighted averages. Comparing the average 1 As of October 1st, 2014, 78% of all pupils of grade 0-10 in Denmark attended public schools. According to the Statistics Denmark: (retrieved April 14., 2015) 2 Removing N. Kochs skole gives the averages 6.93, 6.97, 6.81 and 6.85 respectively. This does not change the conclusion, that our sample is above the national average with regards to expected average scores and realized average scores. 12

13 score between the fablab and the non-fablab groups, the schools designated as future FabLab schools on average have higher average grades than the other schools. This difference is however not statistically significant 3 and thus we cannot conclude that the two groups differ with regards to average score on examinations in the 9 th grade. Fab% Respondent Non%fab! Weighted! Respondents! Weighted! average! s! average! 7,06% 595% 7,01% 6,81% 527% 6,95% As stated, we do not claim to have a representative sample, and our goal was to get two groups as close to each other as possible with regards to background factors. We have no power over which schools are designated future FabLab schools, and which are not, thus our Non-FabLab schools should differ from the national population, when the FabLab schools do. In conclusion, our sample seems to differ from the population in terms of grades and socioeconomic status. Both of which are at a statistically significant higher level in our sample than in the population in general. 3 See appendix. The t-test was done on un-weighted scores for which the difference is between the two groups is largest. 13

14 4 Respondents 4.1 Age and grade As stated, we ended up with 1156 useful responses. The age ranged from years old, with the majority between 12 and 14, as can be seen from the following chart: 500! 450! 400! 350! 300! 250! 200! 150! 100! 50! Age$ 0! 11! 12! 13! 14! 15! Figure 1: Respondents in our sample. Listed by age. These students all attended grade 6, 7, 8 or 9, with the majority in grades 6 through 8: 14

15 Grade$ 500! 450! 400! 350! 300! 250! 200! 150! 100! 50! 0! 6! 7! 8! 9! Figure 2: Respondents in our sample. Listed by grade. 4.2 Fablab and Non Fablab schools We asked the municipalities of Vejle, Silkeborg and Aarhus to give us a list of existing or future schools in the Fablab@School.dk project. We also asked the municipalities to supply us with a group of reference schools matching the FabLab schools as well as possible. When this turned out to be difficult, we at least did our very best to ensure as diverse a group of schools as possible. The distribution of designated FabLab and Non FabLab schools with regards to the age of the students shows the difficulties we had in securing an even spreading out of the different groups of students: 15

16 Age$ 500! 450! 400! 350! 300! 250! 200! 150! 100! 50! 0! 11! 12! 13! 14! 15! FabLab! Non!FabLab! Figure 3: Respondents in our sample. Listed by age. The same is clear, when viewed based on the grades attended by the students. Ideally, we would have wanted the 6 groups of grade 6, 7 and 8 in both FabLab and reference group to be of equal size, but this turned out to be very difficult. Unless otherwise noted, we treat the FabLab and Non Fablab groups as one during our data analysis, and therefore we are satisfied with the number of respondents in each of the groups of 6 th, 7 th and 8 th graders when testing for the effect of grade. We had specifically asked for classes, which had not yet been a part of FabLab@School activities and as such there should be no difference between the two groups with regards to FabLab background. There is however a chance that the two groups differ on background variables such as a schools tendency to participate in projects in general. 16

17 500! 450! 400! 350! 300! 250! 200! 150! 100! 50! 0! Grade$ 6! 7! 8! 9! FabLab! Non!FabLab! Figure 4: Respondents in our sample. Listed by grade. As can be seen on the chart, we have the highest participation in grade 7, which as stated in a former chapter is due to the choice, which we gave the school principals. Given the choice between 6 th, 7 th and 8 th graders, most schools chose 7 th grade. We had not expected this, and when it became clear, we did our best to ensure, that we had enough in the other two groups as well. As can be seen, it is mostly the designated FabLab schools which are causing this problem. This is partly due to the fact, that the FabLab schools were much more responsive and thus got to pick the desired grades, whereas the reference group mostly had to be contacted several times, which gave us the chance to influence the choice of grade and thus spread the participating schools out more evenly. 17

18 5 Abilities to use, master and understand digital technologies The results from the survey has been ordered in 4 themes, which are the main findings here 5.1 Consumers of digital technology In the recent International Computer and Information Literacy Study (ICILS) report it is concluded, that Danish adolescents are heavy users of Information Technology but that their use is not very advanced (Bundsgaard et al. 2014). This conclusion resonates well with our findings, although we ask with regards to different technologies. Whereas ICILS based the conclusions on tests of advanced functions within software programs, the present survey focuses more on use of different types of technologies some of which we have chosen to designate as more advanced than others. These assumptions have not been tested. 5.2 Leisure time spent on IT compared to other types of leisure In order to better understand our respondents, we asked them, what they spent their time on outside of school. In the question, we asked them to tell us how many hours they spent on a given activity in the past week. To ensure, that the students did not spend too much time on this item, we had made the categories rather large. By far, the most popular activities were using a phone or computer or being with friends. The students also spent a great deal of time on sports. When spending time with computers and phones is the most popular leisure activity, it is tempting to talk about the students in our sample as digital natives: Adolescents who were born in a digital world and to whom all things digital are easy and comes naturally (Prensky 2001). This image changes however, when looking deeper into what exactly the adolescents use their computers for. 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! Computer/phone! With!friends!! Sports! In!nature! Householding! Studying! Caring!for! A!job!! Create!things! Singing/playing! Home!improvement!! Repairing! Volunteering! 0N2!hours/week! 2N5!hours/week! 5N10!hours/week! >!10!hours/week! 18

19 Figure 5: Time consumption on activities outside of school during the week leading up to the test. The chart is ordered with regards to the 0-2 hours/week category. 5.3 Use of information technology outside of school The talk about digital natives could be taken to implicate, that theses natives do not need to learn about IT in school, since they live it already. In order to find out what kinds of activities with IT and what kinds of technologies, the students were spending most time on. Corresponding well with the question about their leisure time, the students answered, that they use computers/tablets a lot (More than 70% use them several times a day). What is also apparent, is that they primarily use them for gaming, surfing, chatting, liking/upvoting and commenting that is: They are digitally consuming, what others produce. These are all, what we have chosen to call less advanced uses of technology. One could argue, that by commenting on the posts of other, students are actually involved in the production of information, but here we have chosen to include commenting as closer to consuming than to producing information. 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! Don't!know! Never! Less!than!once! per!month! At!least!once! per!month! At!least!once! per!week! At!least!once! per!day! Several!times!a! day! Figure 6: It use outside of school. Ordered by the sum of "at least per hour" and "Several times per day" In the following chart, the categories have been collapsed into the three categories of several times a day, at least once per week and less than once per week or don t know. To gain a better visualization, the chart has been split into two one for the activities the students spend a lot of time on, and one for activities which the students spend little time on (relatively speaking). We have chosen to make the cut-off at the point, where more than 50% report, that they do the activity less than once per week. 19

20 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! Less!than!once!a!week!or! "don't!know"! At!least!once!a!week! At!least!once!per!day! Figure 7: The Danish students spend a lot of time on computers/tablets, and they use them mostly for consuming. Ordered by "several times a day" Incidentally, most of these activities are more advanced uses of IT. In line with the results from ICILS, the students in our sample don t spend much time on the more advanced tasks: 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! Less!than!once! a!week!or! "don't!know"! At!least!once!a! week! At!least!once! per!day! Figure 8: The students in our sample don't spend much time on advanced use of it. Ordered by "several times a day" Programming sticks out with more than 10% of students claiming to do this several times a day. It is a surprising result to us, and it would be interesting to 20

21 probe it further gaining insight into what the students mean by programming, what they program and how they go about it. It was our impression, that some students thought programming was akin to downloading apps from an appstore, but we have not had the chance to investigate this. In the International Computer and Literacy Study, 9% of the Danish 8 th grade students reported that they programmed at least once per week, and only 3% claimed to be doing this every day Questions probing the production of content in general such as image- and video editing, writing on wikis or blogs are not popular among the students in our survey. A result, that may surprise some, is that "writing status updates" seems to be quite uncommon more than 75% of the students claim to do this less than once a week or not knowing (less than 10 percentage points). If this has to do with Facebook not permitting children to open a profile before the age of 13, parental control or simply that teenagers use other social media than Facebook, is not possible to see from our data. We asked about hosting a server, communicating on IRCs or hacking, because it from our own experience looks to be good predictors of very advanced computer skills, and we wanted to test this hypothesis. Very few students engage in these activities, which could point to them not being in the category of very advanced IT users/producers, but since the measure has not been validated, it is still too early for this conclusion. To conclude, it seems from our data, that the students, we have asked, use a lot of time consuming digital media and digital technology, but that they spend little time producing content and/or technology. 5.4 Self-perceived knowledge of IT In the survey, we investigated the technological and digital literacy of the students through different instruments. One such instrument was a list of technologies towards which the students were asked to evaluate themselves on a scale of 1 to 6 (1 being I know nothing about it and 6 being I could teach others about it ). The list of technologies is long, and therefore is was split up into two pages. Here, it is likewise split in two the second consisting mainly of makertechnologies, whereas the first consist of purely digital technologies. The scale has been collapsed into 3 categories. It is apparent, that the students in our sample are confident with using Smartphones (Less than 10% claim poor knowledge), Tablets (11% claim poor knowledge) and Computers (Less than 10% claim poor knowledge). As stated above, the students spend a lot of time outside class using phones, tablets and computers, and here we can see that only very few of them report to have little or poor knowledge of these technologies. 21

22 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! Smartphones! Tablets! Privacy!settings! Computers/laptops! Software/App! Presentation!software! Word!processing! Image!editing! Video!editing! BackNup! Spreadsheets! Webpage!creation! Blogs! Poor!Knowledge! Knowledge! Good!knowledge! Figure 9: Self-perceived knowledge of technologies. Scale collapsed from 6 to 3 categories (1+2, 3+4, 5+6). Ordered by "Good knowledge" (5+6). They are also relatively confident in changing privacy settings (18% claim poor knowledge) and installing apps/software (26% claim poor knowledge, but 33% claim good knowledge). Both of these can be said to be important skills in an age of social media. Presentation software and word processing have relatively few with poor self-perceived knowledge (17% and 20% respectively). The students are not as confident in their knowledge of tools for creating other kinds of digital content than texts and presentations, however. Image and video editing both have less than 20% of the students claiming good knowledge and more than 40% claiming poor knowledge. This corresponds well with the more than 80% of the students, who answer that they use these tools less than once per week (53% never use or don t know). It also corresponds well with the very similar responses on sharing videos, music and pictures on the internet (78% do it less than once per week, 57% never do or don t know). This adds further to the picture of the adolescents as digital consumers, who do not produce with advanced technologies. More than 50% claim to have little knowledge of doing back-ups, which could be worrying given the importance of digital data in our time. Both webpages and blogs seem to be something, they know very little about. Wikis and blogs is also something, which only 5% of the students claim to do weekly. One could speculate that what we see here is that the technology has shifted away from webpages, wikis and blogs. If you want express yourself and reach an audience, you can do so through Instragram, Tumblr or by making a Facebook -page. You no longer need to set up a homepage or even what used to be called a blog. 22

23 In conclusion. the students we surveyed are heavy users of computers, tablets and phones. They feel confident using these technologies, but in their time outside of class, they use them for consuming rather than producing content. The students are pretty confident however in using tools such as word processing and presentation software. We have not inquired into for what purposes and how advanced functions, they use, but according to the ICILS report, grade 8 students in Denmark do not use these programs in advanced ways. In line with this report, the students in our survey report, that they have relatively poor knowledge of tools for producing images, videos and webpages, which again would be considered more advanced tasks than consuming content. Finally we conclude, that the students in our survey cannot be termed digital natives in the sense, that they themselves acquire the skills needed for being part of a digital 21 st century. 5.5 Fabrication technologies In the instrument to measure self-perceived technological and digital literacy, the students were asked to evaluate themselves on a range of technologies associated with FabLabs and digital fabrication. As described above, they were asked to do so on a scale of 1 to 6 (1 being I know nothing about it and 6 being I could teach others about it ). When comparing responses on the items regarding (digital) fabrication technologies, it is obvious, that the students are much less familiar with those, than they are with the purely digital technologies. Again, the categories have been collapsed in order to gain a clearer overview: 23

24 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! Poor!Knowledge! Knowledge! Good!knowledge! 0%! Figure 10: Self-perceived knowledge of technologies. Scale collapsed from 6 to 3 categories (1+2, 3+4, 5+6). Ordered by "Good knowledge" (5+6). Very few students claim to have good knowledge of any of these technologies, but tools for woodworking and metalwork stand out as the most familiar. Except for Wood/metal tools and Multi tester, these technologies are all digital fabrication technologies, of which very few students claim to have good or even medium knowledge. More students claim to have knowledge about programming (8% good knowledge, 27% some knowledge) and programmable robots (8% good knowledge, 19% some knowledge) than the rest of the technologies. One could speculate, that perhaps these students have participated in some of the LEGO Mindstorms competitions which are run in the area every year. As described earlier, the students claim to spend more of their time programming, than what was expected by the researchers, and thus it would be interesting to investigate, what the students mean by programming, how they do it, and what they use programming for. All in all, it is clear, that the students are much less familiar with digital fabrication technologies, than they are with other digital technologies. Thus they are as stated before heavy consumers, but they are not producers neither of content, of digital products or of physical products using digital tools. 24

25 6 Office Literacy: IT in schools? When running a factor analysis 4 on the data of self-reported knowledge with regards digital technologies, it becomes clear, that especially one factor stands out even under different initial assumptions. Three variables all draw heavily on this factor, and that is presentation-, spreadsheet- and word processing software. What this means, is that there seems to be an underlying factor, which is explanatory with regards to these three variables. Since these three types of software are the core of any office-package of software, we have chosen to call the ability to use the programs for an Office Literacy. The Danish findings do not match findings from identical questions used by the TLTL group at Stanford University in California, which makes them even more interesting. Why do these three software types single out in a factor analysis? In order to investigate the Office Literacy further, we have developed an index variable, which consists of the students self-perceived knowledge with regards to word processing-, spreadsheet- and presentation software averaged and rounded off to nearest whole number. This gives the following chart: 400! 350! 300! 250! 200! 150! 100! 50! 0! 1! 2! 3! 4! 5! 6! Figure 11: Average of students responses on self-perceived knowledge about office software on a scale of 1 ( I know nothing about it) to 6 ( I could teach others about it). The mean value of the scale is 3,5, and as can be seen, this coincides with the mean value of the Office Literacy score of the students. 4 Principal Components on a polychoric correlation matrix with an orthogonal rotation. A factor analysis is a statistical method, which searches for common factors underlying answers on several questions. 25

26 Taking the Office Literacy components one by one, it becomes clear, that the students are pretty familiar with both presentation software and word processing, and that spreadsheet software appears to be more difficult. As seen in the index variable, the middle categories are very populated, indicating that office programs is something most know about, but also something, that fewer claim to be able to teach others about than e.g. phones, computers and tablets. This could suggest, that teaching Office Literacy in schools has the aim of getting as many students as possible to reach a certain level of literacy in the office programs and thus not on teaching the more advanced skills, which only few of the students are ready for. The data is visualized with the categories collapsed into three (Good=5+6, Some=3+4, Poor=1+2): 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! 194! 226! 474! 622! 649! 546! 340! 281! 136! Presentations! Word!proc.! Spreadsheets! Poor!knowledge! Some!knowledge! Good!knowledge! Figure 12: Self-perceived abilities in office software. Scale collapsed into 3 categories. 6.1 What do they learn in school? After the students are asked to rate their knowledge of different technologies, they are asked, where they think they have acquired these skills. The answers are either primarily in school, primarily at home or I haven t learned it. Primarily at home was meant to capture all the instances, where the students acquire IT skills outside of school, but it is possible, that the wording has lead the students to think otherwise. However, we focus on how many students are 26

27 reporting to have acquired the skills primarily in school. For the first part of the questions, the chart looks like this: 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! Haven't!learned! Out!of!school! In!school! Figure 13: Where have the students primarily learned their digital skills? Ordered by "in school" Here, it is clear, that according to the students the main digital technologies that they primarily learn in school are presentation software (81%), spreadsheets (74%) and word processing (72%). This corresponds with the result from the aforementioned factor analysis on self-perceived digital literacy. Here, the office software stood out as having a common explanatory factor. On top of this, very few students state, that they haven t learned these office programs. Thus the underlying factor explaining the results in Office Literacy could well have to do with these programs being taught to a great extent in Danish schools. One could speculate, that this means, that schools have done a significant job at teaching office-software. With regards to Tablet, phones and to a certain degree computers, it seems, that most students think they acquire skills for using these technologies outside of school. This could mean, that schools are free to assume, that students don't need to be taught about these in schools, and thus focus their resources other aspects of digital technologies. On the other hand, there is a group of students claiming to learn how to use especially computers (18%) in schools. The question is, what would happen to their digital literacy, if they were not taught to use computers in school? 27

28 At the same time, it seems, that image (12%) and video (35%) editing skills are not in the same way the focus of the schools. With regards to both types of software, there is a group of about 30% claiming not to know how to use them in this instrument, and in the instrument asking for self-perceived technological and digital literacy, more than 40% of the answers on each item, is in the category of poor knowledge of these two skills. The scope of this report is not to claim, that video- and image editing are important skills, but it is clear from the data, that if they are indeed deemed important, there seems to be an opportunity for schools to make an impact. In conclusion, it seems, that the main kind of digital literacy, which the students in our sample acquire in schools, is Office Literacy. This corresponds well with factor analysis showing that these skills group together under a common factor. The results suggest, that schools have an important impact on the digital literacy of the students in our sample, who do not seem to enter school as digital natives. It raises the question about which other (more advanced) technologies schools should consider focusing on in the future. 6.2 Do they learn digital fabrication in schools? Just as with the digital technologies such as office software mentioned above, the students were also asked to answer where they had acquired their digital fabrication skills (if they had acquired such skills). Again, the possible answers were: primarily in school, primarily at home or I haven t learned it. The questions regarding fabrication technologies focus on digital fabrication, but we have included a metal-/woodwork item and an item about a multi-tester because these technologies are often used in connection with digital fabrication. The one that is mostly learned in school according to the students in our sample is wood- and metalworking. Woodworking is taught as a subject in Danish schools, and metalwork is often offered as an elective course. Thus the result is not surprising. The item, that has the second-most students claiming to have learnt it in school, is the multi-tester, which some of them will have been introduced to in the subject Nature&Technology (grade 1-6) and everyone should be introduced to in the subject Physics&Chemistry (grade 7-9). We see the rest of the items as being about digital fabrication, and except for Programmable robots (such as LEGO Mindstorms), which 22% of the students state, they have learned to use in school, none of the items have more than 15% of the students claiming to have learned it in school. The lowest are 3D printers (8%), Programming (8%) and Laser cutters and CNC routers (5%). The conclusion is unsurprisingly, that the students in our sample do not seem to have been taught much about digital fabrication in school. Again, programming seems to be a much more common technology to know, than what the researchers expected. 28

29 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! Haven't!learned! Out!of!school! In!school! Figure 14: Where have the students primarily acquired their fabrication skills? Ordered by "in school" In conclusion, it seems that most students are taught to use office programs in school, and that they acquire some form of Office Literacy. It also seems, that they are not taught how to use image/video editing which can be seen as more advanced tools for production of content. 29

30 7 Abilities to actively engage in heterogeneous communities of practice and to think and act innovatively (with technology) on societal challenges The ability to actively engage in heterogeneous communities of practice and to think and act innovatively on societal challenges are part of what the OECD has coined as 21 st century skills. It is also an important part of design thinking (Cross 2011). These skills are difficult to measure in the limited timespan of this survey and in a survey at all. We have set out to do so through a range of different types of questions. Firstly, we have asked the students how good they are at working in groups, and how they perceive their own creative, imaginative and idea generating abilities on a scale of 1 to 6. Then we ask for their experiences with getting ideas for products or inventions and acting on these ideas. After first probing the self-views of the students, and asking for their experiences, the students were given an open-ended problem of solving a societal challenge with dementia. Following up on the open-ended question, the students were asked to rate the importance of different types of stakeholders and different parts of a problem-solving (design-) process. By thus probing their views on stakeholders and processual parts, it is possible to get a rather broad image on the students' theoretical view on solving complex, societal challenges. Finally, the questionnaire has a range of questions regarding students' views on certain aspects of technology and data, and what innovative possibilities, technology gives, to gain insights into the technology side of acting innovatively on societal challenges. 7.1 Creativity, imagination and cooperation The students were asked about self-perceived abilities in creativity, imagination and cooperation by stating to which degree, they agreed with a range of statements. This was done on a scale of 1 (strongly disagree) to 6 (strongly agree). The students were furthermore asked, whether or not they think creativity is fixed (genetically), whether they are interested in the creative (crafts) subjects in school, and whether inventing is important to them. They seemed to have a pretty good confidence in their own cooperative abilities, creativity and imagination, and they find creative subjects interesting, but inventing stuff is not important to most of them. Apparently this is not a part of being a creative person according to many of the students. In the chart, the six categories of strongly disagree to strongly agree have been collapsed into the two categories of either agree or disagree. 30

31 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! Disagree! Agree! 0%! Figure 15: Attitudes and self-evaluation with regards to creativity, imagination, cooperation and creative subjects. Values collapsed from 6 to 2 categories. Ordered by Agree. Based on the four variables of I am good at coming up with new ideas, I have a good imagination, I have a lot of good ideas and I am a creative person, an index variable has been made, which measures the students self-perceived creativity, imagination and idea generation. 7.2 Inventions The kids were asked if they ever had an idea for an invention and if the answer was yes, they were asked to describe it. They were then asked whether they had created the invention. 47% of the students stated, that they had indeed had an idea for a product or an invention. 31

32 700! 600! 500! 400! 300! 200! 100! 0! Yes! No! Figure 16: Have you ever had an idea for a product or an invention? 12% of the total amount of students have build or created their idea or product. 34% have had an idea that they didn t create or build. 700! 600! 500! 400! 300! 200! 100! 0! Yes! No! Figure 17: Did you create or build this product or invention? 32

33 1200! 1000! 800! 600! Hasn't!had!an!idea! Has!had!an!idea! 400! 200! 0! Has!created!the!product! Hasn't!created!the!product! Figure 18: Have you had an idea and created the product or invention? In conclusion the students perceive of themselves as good at collaborating in heterogeneous groups and at imagining, generating ideas and being creative, but they are lacking either the skills or the interest to act on their ideas. 7.3 The dementia task As stated in the introduction, the survey consists of different types of instruments. More specifically the instruments range from closed items asking about opinions or self-perceived abilities through multiple choice tasks and to open-ended problems. One such problem, is the demented task, which is an example of a socalled wicked problem (Buchanan 1992). It was a real-world-problem being talked about in Danish media, and the number of elderly refers to a Danish context (population approx. 5.7 million) : In the beginning of the year 2014, 9 grandparents disappeared from their nursing home because of their loss of memory (dementia). The problem for the nursing home is to create security for the elderly without taking away their freedom. If you were asked to solve this problem, what would you do? We had the hypothesis, that the students did not have any tools to approach this task with. That is, that they would jump to solutions instead of painting the picture of a process, they would set up in order to reach a better understanding and a possible solution. Such tools of getting to know the problem through 33

34 investigations, sketching, mocking-up, testing and prototyping are all integral parts of design thinking (Cross 2011). In order to score this open-ended question, the answers were coded in a grounded theory method (Strauss and Corbin 1990) the researcher trying to approach the answers with an open mind and letting the answers speak for themselves. Each time a new type of answer appeared, a new code was created. When the amount of codes seemed to have stabilized after about 220 respondents, there were 18 codes. These were then categorized into the three categories of (1) trying to create a situation in which the demented do not try to leave or get lost, (2) preventing the demented from leaving the nursing home, and (3) keeping track of and finding the demented, once they have left the nursing home. Within each of these categories, a taxonomy of 4 levels was created, and in the end this gave a total of 12 codes. Level four in each category was one of starting an investigation into the problems at hand. Two of these had not been used by any of the first 200 responses, that were coded, but they were created for the sake of showing, what could also have been answered (level 4 in preventing and tracking the demented): Trying to create a situation in which the demented do not try to leave/get lost 1. Talk to the elderly about not leaving/ letting the elderly stay in their own home/ cure the demented 2. Make sure, the demented have a good time (get a lot of visits/ participate in activities/ have better surroundings/ have more caretakers/ more freedom) 3. Make the demented feel more at home by making their rooms/apartments look the homes, they stayed in, while younger. Thus starting by understanding the individual demented person 4. Investigate, what makes the demented leave and get lost, and then act on this. Thus starting by understanding the situation of demented in general Preventing the demented from leaving the nursing home 1. Fence in the demented/lock the doors/surveillance/guards at the doors 2. Putting a chip on/in the demented and a sensor at the door, which will not let them pass or using other kinds of technology with the purpose of preventing the demented from leaving, without reflections about how it is placed or how it should function 3. Putting a chip on/in the demented or using other kinds of technology with the purpose of preventing them from leaving the home. Including reflections on where to place it or how it should function 4. Investigate, how the demented could be prevented from leaving and what this would affect their lives, and then act on this 34

35 7.3.3 Tracking and finding the demented, once they have left 1. Look for the demented/ Call the police to have them look for the demented 2. Put a tracking device on/in the demented without reflecting upon how to place it or how it should function or put up signs in the neighborhood for the demented to follow back to the nursing home 3. Put a tracking device on/in the demented including reflections about how to place it and how it should function 4. Investigate, why the demented get lost, how to best find them and how this possible solution would affect their lives, and then act on the findings This one-dimensional coding system does by no means capture all the information in answers. There was e.g. no way of distinguishing between those students who only had their own grandparents in mind, and those who thought of the population of elderly people in general. There was also no way of coding their level of empathy for the elderly and their situation. The coding system, however, captures the range of their ideas and whether or not they have opted to set up a design process. The numbering system was made like this: 0. Don t know/not readable/nonsensical/blank 1. Talk to the elderly about not leaving/ letting the elderly stay in their own home/ cure the demented 2. Fence in the demented/lock the doors/surveillance/guards at the doors 3. Look for the demented/ Call the police to have them look for the demented 4. Make sure, the demented have a good life (get a lot of visits/ participate in activities/ have better surroundings/ have more caretakers/ more freedom) 5. Putting a chip on/in the demented and a sensor at the door, which will not let them pass or using other kinds of technology with the purpose of preventing the demented from leaving, without reflections about how it is placed or how it should function 6. Put a tracking device on/in the demented without reflecting upon how to place it or how it should function or put up signs in the neighborhood for the demented to follow back to the nursing home 7. Make the demented feel more at home by making their rooms/apartments look the homes, they stayed in, while younger. Thus starting by understanding the individual demented person 8. Putting a chip on/in the demented or using other kinds of technology with the purpose of preventing them from leaving the home. Including reflections on where to place it or how it should function 9. Put a tracking device on/in the demented including reflections about how to place it and how it should function 10. Investigate, what makes the demented leave and get lost, and then act on this 35

36 11. Investigate, how the demented could be prevented from leaving and what this would affect their lives, and then act on this 12. Investigate, why the demented get lost, how to best find them and how this possible solution would affect their lives, and then act on the findings When more codes were relevant to a response, the one with the highest number was chosen. As code 11 and 12 were only used once, they were collapsed into code 10. Looking at the results, the two most popular categories are I don t know (31%) and Improve their lives (27%). The next in line was to lock up the demented (12%). Only 3% responded, that they would start a process. 400! 350! 300! 250! 200! 150! 100! 50! 0! Figure 19: Different kinds of responses to the open-ended dementia problem. Ordered by frequency Data analysis The coding system did still present some challenges. Often, it was difficult to distinguish between some of the codes within a category (e.g. reflection or not), and for that reason, the scale within these was collapsed somewhat. This also eliminated the problem with arguing that locking the doors is at a different taxonomical level than preventing the demented from leaving by using chips and sensor or codes at the (locked) doors. In many ways, this is the same thing. Two groups of responses, that still posed a challenge, were the ones where the students wanted staff to follow the demented on walking tours outside the home. This could be done in order to improve the happiness of the demented and thus, it is the idea, make them less likely to want to leave, or it could be as a form of surveillance. Thus, responses in which the demented are escorted on walking trips could be seen as both a limitation of their freedom and an increase in their 36

37 freedom. The responses were coded based on an interpretation of the intention of the respondent. The codes 11 and 12, which were made to highlight these possibilities even though they were not found in the first 220 responses, were only used once in total, and for this reason, they were combined with code 10 in the data analysis. The new collapsed coding system was made in this way: 0: I don t know etc. 1: Persuade them not to leave or cure them 2 (2+5+8): Prevent them from leaving/escaping 3: Find them 4 (4+7): Improve the life of the demented 6 (6+9): Track the demented (GPS) 10 ( ): investigative, processual approach to the problem 400! 350! 300! 250! 200! 150! 100! 50! 0! Figure 20: Distribution of answer in final version of the coding of the open-ended dementia problem. Ordered by frequency. Again, what is very noticeable is that a processual approach is not common (3%), whereas the idea of improving the lives of the demented resonates well with the students (31%), and there is 31% who don t have an answer. The following pie chart is another representation of the same data. 37

38 3%! 4%! 3%! 16%! 12%! 31%! Don't!know! Improve!their!lives! Prevent!them!leaving! Track!them! Persuade! Find!them! Investigate/process! 31%! Figure 21: Pie chart of the distribution of responses on the open-ended dementia problem. Overall, it is concluded, that when posed a complex, societal challenge, the students try to provide a solution rather than describe a process which could lead to a solution. There can be many reasons for this, and one of them perhaps has to do with the wording of the question, but it is nevertheless an interesting finding, which it would be interesting to investigate further Importance of stakeholders After being asked an open-ended question about how to solve the dementia problem, the students were asked to rate the importance of specific possible stakeholders on a scale of 1 (not important al all) to 6 (very important). A large group of students answered "I don't know" to the possible stakeholders of NASA (23%) and industrial designers (13%). Thus they appear to be uncertain as to who these are or what their possible role should be. The same applies to some extent to tech companies (9%) and interest groups for the elderly (10%) as well. A chart with the values collapsed into three (1+2, 3+4, 5+6) 5, paint a picture of, who the students think are the most important stakeholders. Most students identify nursing staff (54%) and demented elderly people (38%) as important stakeholders followed by interest groups for the elderly (33%) and municipal representatives 5 We have chosen to call these categories Unimportant, neither/nor and important respectively. 38

39 (30%). On the other hand, most students identify the fire brigade and NASA as the most unimportant (64% and 75%). Around 16% of the students seem to prefer to solve the problem themselves, while 54% has rated this option 1 or 2. It seems as though the students are rather good at pointing out important stakeholders, when they are prompted to do so, but as mentioned earlier, they did not by themselves mention these stakeholders in an open-ended task. Thus the conclusion is, that when confronted with a complex real-world challenge, the students rush to offer a possible solution without thinking about involving stakeholders, but when asked about the relevance of different stakeholders, they have a good idea about who to involve. 1200! 1000! 800! 600! 400! 200! 0! Unimportant! Neither/nor! Important! Figure 22: Who would be important for you to collaborate with? Scale collapsed from 6 to 3 categories. Ordered by "important" The design process It was our hypothesis, that the students did not have an adequate understanding of a design process. Therefore, the students were asked a series of questions about how the process, they would set up to solve the dementia problem, would be. Again they were asked to score the items on a scale of 1 to 6. 6 being very important and 1 being non-important. In the following charts, the values are collapsed into three possibilities, and the above chart is split in two, the first being a chart of all the variables, to which the more students have chosen it as important than unimportant. The most important parts of the process according to the students would be making all the 39

40 stakeholders agree on a solution (57%), setting up a meeting with staff and relatives (54%), visiting nursing home to explore the problem (54%), creating a thorough plan (43%) and test a possible solution with the elderly (46%). 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! Uimportant! Neither/nor! Important! Figure 23: What parts of the process would be most important to you? Scale collapsed from 6 to 3 categories. Ordered by average score. The second chart on the setup of the process shows the suggested parts of the process, which more students found unimportant than important. It is clear, that very few students mark the three suggestions that entail building a cardboard model (6%), testing it (10%) and repeating this (11%) as important. In line with this, sketching on paper is important to only 21% of the students. When considering that sketching and using mock-ups are essential parts of design thinking (Cross 2011), this is an interesting result that suggests that the students do not on average have good knowledge of design thinking. 40

41 100%! 90%! 80%! 70%! 60%! 50%! 40%! 30%! 20%! 10%! 0%! Uimportant! Neither/nor! Important! Figure 24: What parts of the process would be most important to you? Scale collapsed from 6 to 2 categories. Ordered by average score. 7.4 Summary When asking Danish students to rate different stakeholders and parts of design processes, they are good at identifying important stakeholders and they do value some of the typical parts of a design process high. The students do not, however, have a good understanding of the value of externalizations such as sketching and building mock-ups which are typically used with the purpose of gaining insights and knowledge which can inform the production of a possible solution. This might also be reason, why when asked how they would solve the dementia problem, only 3% of the students reply that they would investigate the issue further. If this is true, it points to a lack of understanding with regards to design processes. 41

42 8 Summary and Conclusion This is the report on a baseline survey for the FabLab@school.dk research project. The 1156 respondents were students of years of age, and they answered 227 questions about knowledge and use of digital technologies in and outside of school and of design and creativity. The sample is not representative, which means, that the claims are valid for the sample only. Below is a summary of the most important findings. The students are consumers of digital media and technology The students use their phones and computer a lot. More than 90% of the students in our survey reported to use phones and computers at least 2 hours per Week, while 34% of them reported to spend more than 10 hours per week on the same. On the other hand, very few students report to use their phones and computers for producing and sharing digital content e.g. videos or pictures. This goes well with the finding, that their knowledge of content creation tools such as imageand video editing software, blogs and webpage creation is pretty poor. Less than 20% claim to have good knowledge of any of these tools. All of this leads us to conclude, that the students use their phones, tablets and computers primarily for consuming digital content and not so much for producing content, and that their knowledge of IT is not very advanced. Only very few students have knowledge of digital fabrication Less than 5% of the students have good knowledge of technologies such as 3Dprinters, laser cutters or microcontroller boards such as the Arduino and Makey Makey, and less than 10% of the students claim to have some knowledge of laser cutters and microcontroller boards. Thus we conclude, that the students as a whole are not really exposed to digital fabrication yet. The schools focus on Office Literacy When asked where they have acquired skills in the given technologies, more than 70% of the students report, that they have acquired their word processing, spreadsheet or presentation software skills in school. At the same time, around half of students have placed themselves the middle categories with regards to their knowledge of office software. This suggests, that schools have focused on giving the students an Office Literacy, and that they have to some degree succeeded in this. Creativity is not linked to producing When asked about their self-view, more than 70% of the students claim to be creative, to have a good imagination and to be good at coming up with new ideas. About half of the students have furthermore tried to have an idea for a product or an invention, but only 12% has actually tried to produce their idea. Thus it does not seem as if creativity is linked to producing in the minds of the students. 42

43 The students have poor knowledge of design processes The students do not value externalizations e.g. sketches and cardboard models. When they are given a complex, societal challenge (the demented who leave the nursing home and get lost), it seems as though the students tend to jump to a possible solution instead of initiating a design process which can qualify, reject or develop their ideas. This is still a rather tentative measure of design proces knowledge, but only 3% of the students answered, that they would investigate or in other ways gain further insights 43

44 9 Literature Buchanan, R. (1992). Wicked problems in design thinking. Design issues, Accessed 4 February 2015 Bundsgaard, J., Rasmus Puck, M., & Petterson, M. (2014). Digitale kompetencer: It i danske skoler i et internationalt perspektiv. Aarhus Universitetsforlag. Cross, N. (2011). Design thinking: Understanding how designers think and work. Oxford. UK: Berg. Hansen, K. Y., & Munck, I. (2012). Exploring the measurement profiles of socioeconomic background indicators and their differences in reading achievement: A two-level latent class analysis. In Issues and Methodologies in Large-Scale Assessments (Vol. 5). Educational Testing Service and International Association for the Evaluation of Educational Achievement. Marsden, P. V., & Wright, J. D. (Eds.). (2010). Handbook of survey research (2. ed.). Bingley: Emerald. Prensky, M. (2001). Digital natives, digital immigrants part 1. On the horizon, 9(5), 1 6. Accessed 25 April 2015 Strauss, A., & Corbin, J. M. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Sage Publications, Inc. 44

45 Accessed 28 April

46 Appendix I: The questionnaire In the following pages is a print out of the original questionnaire. It is included in the original wordings (in Danish) 46

47 Teknologi i skole og fritid Velkommen til Aarhus Universitets spørgeskema om dig og dit forhold til teknologi i skole og fritid. Personlig information Vi vil først gerne vide noget om dig og din skole Hvad er dit UNI-login (brugernavn)? Hvor gammel er du? Hvad hedder din skole? Hvilket køn er du? Dreng Pige Hvilket klassetrin går du på? Hvor mange bøger er der ca. i dit hjem? (du skal ikke tælle blade, aviser eller dine skolebøger med) 0-25 bøger bøger bøger bøger Over 200 bøger Skole og fritid I løbet af den sidste uge, hvor mange timer brugte du på... Lektier derhjemme 0-2 timer pr. uge 2-5 timer pr. uge 5-10 timer pr. uge Over 10 timer pr. uge Passe dine søskende, familiens kæledyr osv. Hjælpe til i hjemmet (rengøring, madlavning osv.) 1 of 12 09/04/15 11:45

48 Arbejde på gør-det-selv byggeprojekter Reparation af f.eks. møbler, cykler eller elektriske apparater i dit hjem At arbejde kreativt med f.eks. træ, maling eller stof. Fritidsjob (med løn) Være på computer, TV, mobil osv. Sportsaktiviteter At synge, spille et instrument eller i band At være sammen med dine venner (fysisk/online) Frivilligt arbejde (f.eks. foreningsarbejde) At være i naturen Her spørger vi dig om dit forhold til skolen og din fremtid. Hvor enig er du? Meget uenig Uenig Lidt uenig Lidt enig Enig Meget enig Jeg klarer generelt mig godt i livet Jeg stoler på mine egne faglige evner Jeg klarer mig godt i skolen Jeg lærer hurtigt nye ting Jeg vil have en fremtid indenfor et kreativt fag (f.eks. håndværk, film eller musik) Jeg vil have en fremtid indenfor teknologi og design Jeg vil have en fremtid som ingeniør eller indenfor naturvidenskab Jeg vil starte min egen virksomhed Teknologi i hverdagen Her spørger vi om din brug af teknologi og deltagelse i online fællesskaber. Hvor ofte gør du følgende? Bruger computer eller tablet derhjemme Spiller computer-, mobil- eller videospil Surfer eller ser video på Internettet Uddeler f.eks. likes eller upvotes på Facebook, Instragram, Reddit, osv. Kommenterer andres posts og opdateringer Skriver statusopdateringer Kommunikerer med billeder, video og tekst (f.eks. Facebook, Instagram, Snapchat) Deler filer som film, musik og billeder (f.eks. YouTube, Soundcloud, Pinterest) Ændre app indstillinger (f.eks. sikkerhed eller layout) Skriver på Wikis, blogs, egen hjemmeside, osv. Arbejder med video eller billeder (f.eks. i Photoshop) Deltager på forum, IRC kanaler eller mailing lister Min. hver time Flere gange om dagen Min. en gang om dagen Hvor ofte? Min. en gang om ugen Min. en gang om måneden Højst en gang om Aldrig måneden Ved ikke 2 of 12 09/04/15 11:45

49 Deltager i hacking, tilpasning, piratkopiering, osv. Programmerer hjemmesider, spil, apps, osv. Deler filer på egen server (f.eks. FTP) eller P2P netværk (f.eks. Bittorrent) Bidrager til en modeblog, en Minecraft server, filmprojekter, osv. Teknologi i skolen Hvad du ved om teknologi og hvordan du bruger den i skolen? Skriv kort om din bedste oplevelse med teknologi i skolen. Skriv hvad og hvorfor det var spændende. Hvor godt kender du disse teknologier? Bedøm dig selv på en skala fra 1 til 6, hvor 1 er "Det ved jeg ikke noget om" og 6 er "Jeg kunne undervise andre om det." Computer / bærbar Smartphones Tablets eller ipads Blogs Tekstbehandling (f.eks. Word, Google Docs) Regneark (f.eks. Excel, Google Spreadsheets) Præsentationsprogrammer (f.eks. Powerpoint, Prezi) Produktion eller redigering af digitale film/videoer Lave en hjemmeside Redigering af digitale billeder Installation af software eller apps Back-up af dokumenter, kontakter, mails, osv. Ændring af privatindstillinger på f.eks. Facebook, Gmail, Instagram...forsat fra sidste side 3 of 12 09/04/15 11:45

50 Hvor godt kender du disse teknologier? Programmering (f.eks. kodning af apps) Bygge elektroniske dimser eller simple maskiner fra bunden Bygge programmérbare robotter (f.eks. Lego Mindstorms) Arbejde med træ- og metalværktøj Lasercutters eller CNC fræsere 3D-printere Elektronik og lodning (f.eks. dioder og modstande) Microcontroller boards (f.eks. MakeyMakey og Arduino) Multimeter (f.eks. volt eller ohm måler) Hvor har du lært at bruge disse teknologier? Primært i skolen Primært hjemme Har ikke lært det Computer / bærbar Smartphones Tablets eller ipads Blogs Tekstbehandling (f.eks. Word, Google Docs) Regneark (f.eks. Excel, Google Spreadsheets) Præsentationsprogrammer (f.eks. Powerpoint, Prezi) Produktion eller redigering af digitale film/videoer Lave en hjemmeside Redigering af digitale billeder Installation af software eller apps Back-up af dokumenter, kontakter, mails osv. Ændring af private indstillinger på f.eks. Facebook, Gmail, Instagram Hvor har du lært at bruge disse teknologier? Primært i skolen Primært hjemme Har ikke lært det Programmering (f.eks. kodning af apps) Bygge elektroniske dimser eller simple maskiner fra bunden Bygge programmérbare robotter (f.eks. Lego Mindstorms) Arbejde med træ- og metalværktøj Lasercutters eller CNC fræsere 3D-printere Elektronik og lodning (f.eks. dioder og modstande) Microcontroller boards (f.eks. MakeyMakey og Arduino) Multimeter (f.eks. volt eller ohm måler) Har du nogensinde arbejdet med digital fabrikationsteknologi på din skole f.eks. i et FabLab eller 4 of 12 09/04/15 11:45

51 værksted? Digital fabrikationsteknologi er f.eks. MakeyMakey, loddekoble eller 3D printer Ja Nej Ved ikke Beskriv kort, hvad du har lavet, hvilken teknologi brugte du og til hvad? 1. projekt 2. projekt Hvordan var det at arbejde med digital fabrikation i skolen/fablab? Jeg kedede mig Jeg kan godt lide at være i værkstedet/fablab Undervisningen er interessant Undervisningen er spild af min tid Jeg vil gerne bruge teknologierne til mine egne projekter udenfor skolen Det, vi lærer i værkstedet/fablab, kan jeg bruge i fremtiden Jeg elsker at arbejde på digital fabrikations projekter Jeg lærer meget i værkstedet/fablab Jeg tænker på det, vi har lært, når jeg er derhjemme Meget uenig Uenig Hverken enig/uenig Enig Meget enig Har du nogensinde modtaget undervisning i elektronik, robot-teknologi eller programmering uden for skolen - f.eks. på en workshop eller sommerlejr? Ja Nej Ved ikke Skriv navnet på begivenheden, og hvor lang tid den varede (f.eks. 5 dage): Beskriv kort hvad du lavede til begivenheden 5 of 12 09/04/15 11:45

52 I hvor høj grad synes du at skolen har hjulpet dig til... Bedøm på en skala fra 1 til 6, hvor 1 er "Slet ikke" og 6 er "I høj grad". At arbejde kreativt med teknologi At lære at løse svære eller komplekse udfordringer At lære at forholde dig til samfundsmæssige problemer At forestille dig, hvordan du kan forandre ting, f.eks. med teknologi At samarbejde med mennesker med forskellig baggrund og evner At lære hvordan teknologi påvirker den måde, vi lever på At lære hvordan nye ideer, ting og teknologier bliver skabt...forsat fra sidste side I hvor høj grad synes du at skolen har hjulpet dig til... At forholde dig kritisk til din egen og andres brug af teknologi At kommunikere med forskellige mennesker over Internettet At bruge teknologi til at arbejde systematisk med opgaver (i f.eks. fysik/kemi, natur/teknik) At ville have en videregående uddannelse At ville have en kreativ eller håndværksmæssig uddannelse At ville starte din egen virksomhed Design og kreativitet De næste spørgsmål handler om at få nye idéer, arbejde kreativt og skabe nye ting med teknologi. Hvor enig eller uenig er du i følgende... Jeg kan lide alt, der har med teknologi at gøre Jeg er god til at lave eller bygge ting Det jeg lærer i natur/teknik eller fysik/kemi interesser mig Jeg kan godt lide at arbejde på natur/teknik eller fysik/kemi projekter Jeg keder mig, når vi har natur/teknik eller fysik/kemi Jeg er ikke en naturfags-person Meget uenig Uenig Lidt uenig Lidt enig Enig Meget enig 6 of 12 09/04/15 11:45

53 Jeg er god til natur/teknik eller fysik/kemi Hvor enig eller uenig er du i... Meget uenig Uenig Lidt uenig Lidt enig Enig Meget enig Jeg er god til at finde på nye idéer Jeg har en god fantasi Jeg har en masse gode idéer Jeg er en kreativ person Nogle personer er født kreative, mens andre aldrig lærer det Det jeg lærer i de kreative fag interesser mig At opfinde ting er vigtigt for mig Jeg er god til at bygge videre på andres tanker og ideer Jeg er god til at samarbejde med forskellige mennesker i grupper Jeg er god til at finde på ideer sammen med andre på min alder Har du nogensinde haft en idé til et nyt produkt eller opfindelse? Ja Nej Beskriv kort din idé Har du skabt eller bygget din idé eller opfindelse? Ja Nej Hvorfor ikke? Hvordan og med hvem? Designopgave: Plejehjemmets udfordring I begyndelsen af 2014 forsvandt 9 bedsteforældre fra deres plejehjem pga. hukommelsestab (demens). Plejehjemmets problem er at skabe tryghed for de ældrene uden at tage deres frihed fra dem. Hvis du blev bedt om at løse dette problem, hvad ville du så gøre? 7 of 12 09/04/15 11:45

54 Hvilke stikord ville du søge med på Internettet, for at få idéer til at løse problemet? Hvem ville være vigtige for dig at samarbejde med, om at finde en løsning? Vælg et tal fra 1 til 6 (1 = slet ikke vigtigt, 6 = virkeligt vigtigt) Plejehjemspersonale Andre ældre med demens Politiet Brandvæsenet Folk fra kommunen NASA Et teknologi firma En industriel designer Interessegrupper for ældre (f.eks. Ældresagen) Jeg vil hellere tænke på en løsning selv Andet. Skriv det i tekstboksen. Ved ikke Hvordan ville du finde den rigtige løsning på problemet med de demente ældre, som bliver væk? Hvilke dele af processen ville være vigtigst for dig? Vælg et tal fra 1 til 6 (1 = slet ikke vigtigt, 6 = virkeligt vigtigt) Jeg ville lave en grundig plan for hele projektet Jeg ville vente til at en god idé dukkede op Jeg vil besøge et plejehjem for at udforske problemet nærmere Jeg ville finde ud af, hvad de gør i andre lande Jeg vil skitsere mulige løsninger på et stykke papir Jeg ville bygge min idé i pap Jeg vil teste min pap-model på et plejehjem Jeg vil gentage mine tests med en ny skitse eller pap-model flere gange 8 of 12 09/04/15 11:45

55 Jeg vil afprøve min løsning sammen med ældre plejehjemsbeboerne...forsat fra sidste side Vælg et tal fra 1 til 6 (1 = slet ikke vigtigt, 6 = virkeligt vigtigt) Jeg vil afholde et møde med plejehjemspersonale, pårørende, for at diskutere min løsning Jeg vil sørge for, at alle er enige om løsningen Jeg vil bruge uenigheder mellem personer/grupper til at udvikle nye idéer Jeg vil tage patent på min idé Jeg vil starte et firma til at markedsføre min løsning og tjene penge Så snart min løsning er færdig, stopper jeg helt med at arbejde på problemet Jeg vil bruge min viden fra dette projekt, i fremtidige projekter Andet du ville gøre? Beskriv dem her. Hacking, data og teknologi Her handler det om dit forhold til hacking og reparation af teknologi i din hverdag. Hvor enig eller uenig er du... Når jeg ser en ødelagt ting, tænker jeg straks på en måde at reparere Jeg er ligeglad med hvordan mine digitale dimser fungerer, bare de virker Jeg har en god idé om, hvad der er inde i en mobiltelefon, og hvordan den virker Jeg er interesseret i at vide, hvordan mine digitale dimser fungerer, og jeg forbedre dem ofte Meget uenig Uenig Lidt uenig Lidt enig Enig Meget enig Hvad gør du, hvis noget ikke virker på f.eks. din computer eller mobil? Markér tre muligheder. Ringer til en ven Læser i en manual Spørger en af mine forældre Ringer til support Søger på problemet på Internettet Søger efter hjælp på specifikke hjemmesider Starter en diskussion på en f.eks. et forum Roder med forskellige indstillinger, kommandoer osv., som jeg kender Ved det ikke Andet. Skriv venligst her: Har du nogensinde skilt din telefon eller andre digitale dimser ad? 9 of 12 09/04/15 11:45

56 Ja Nej Ved ikke Hvorfor åbnede du den? Var det f.eks. for at fikse/forbedre noget? Hvorfor ikke? Hvorfor skulle jeg? Det kan jeg ikke finde ud af Så ville jeg bryde garantien Ved ikke Andet. Skriv det venligst her: Hvor enig er du i disse udsagn om teknologi og data? Teknologi, data og information bør være åbne og tilgængelige for alle Meget uenig Uenig Lidt uenig Lidt enig Enig Meget enig Ved ikke Staten skal gemme alles personlige data og information Jeg går op i hvem der ejer mine data og informationer, f.eks. billeder og musik Hacking er kun noget kriminelle gør på internettet...forsat fra sidste side Hvor enig er du i disse udsagn om teknologi og data? Hacking er noget alle gør Meget uenig Uenig Lidt uenig Lidt enig Enig Meget enig Ved ikke Teknologi giver mig frihed til at udfolde mine interesser Jeg kan se hvordan teknologi kan kombineres med andre materialer (f.eks. stof, træ eller papir) Teknologi giver mig mulighed for at forstå nye sammenhænge og muligheder Opgave: Hvilke dele er der i bilnøglen? 10 of 12 09/04/15 11:45

57 Hvis du har andet at fortælle om dit forhold til teknologi, eller ideer til hvordan fremtidens skole kan bruge teknologi i undervisningen, så skriv dem gerne her: Mange hilsner Ole, Rachel, Kasper og Mikkel Aarhus Universitet 12 of 12 09/04/15 11:45