TINKERING WITH POCKET CODE, A SCRATCH-LIKE PROGRAMMING APP FOR YOUR SMARTPHONE Wolfgang Slany 1 ) Abstract Pocket Code allows you to create and execute Catrobat programs in a visual, "LEGO-style" programming environment. Pocket Code and Catrobat are inspired by MIT s Scratch but independently developed and entirely running on smartphones and tablets no Laptop or PC whatsoever is needed to create or execute Catrobat programs. With Pocket Code you can develop your own games, animations, or music videos, taking advantage of multi-touch, built-in sensors, and the full display resolution of your device. It also allows wirelessly controlling external hardware such as Arduino boards, Lego Mindstorms robots, or Parrot s AR.Drone. Pocket Code allows you to download Catrobat programs that others have made and upload programs to the Web, all under free open source licenses to foster learning, sharing, and remixing. All major mobile platforms are supported. Google is promoting Pocket Code through their Google Play for Education platform, and Catrobat has been chosen as the official programming language for OLPC tablets. Pocket Code encourages tinkering and allows you to iteratively create your own apps using just one finger. A first app can be running in less than one minute. I will demo Pocket Code live during my presentation and report about our experiences in physics project at school. Keywords visual programming, kids, children, teenagers, smartphones, tablets, Arduino, robots, games, mobile app, remixing, schools, computer science education, STEM 1. About the Catrobat project Whether for good or not, most kids starting from a very young age and everywhere on the globe will soon have their own smartphones. Our non-profit free open source Catrobat project aims at unleashing their creativity by encouraging them to create their own apps and content. As of April 2014, more than 260 volunteers from all over the world have contributed to the project. Our Pocket Code and Pocket Paint apps are freely available on Google s Android market. More of Catrobat s apps and web services, for other smartphone platforms, for other age and interest groups, and with different functionality (e.g., oriented towards music creation), are currently in various alpha and beta stages. Our project thrives on user experience, with a large dedicated team uniquely focussing on usability and design issues, its members continuously testing our software with real kids and advanced recording and analysis systems, e.g., SMI s mobile eye-tracking glasses. Another focus is software quality, e.g., the automated test suite of Pocket Code currently would run for 160 minutes, were it not parallelized on several test-servers and virtual as well as real test devices. Catrobat has been chosen for the fourth time as a Google Summer of Code mentoring organization and has won the Austrian National Innovation Award in the category Multimedia in 2013. 1 wolfgang.slany@catrobat.org www.catrobat.og 1
2. Your first app in one minute A few pictures will say more than 1000 words! Figure 1: After installing Pocket Code, create a new empty program and name it compass (you have to take the numbers on the Google Play page with a grain of salt). Figure 2: Then create a new object by drawing a needle for the compass using our Pocket Paint app. 2
Figure 3: After cropping the image, name the new object needle and start to add a script to it. Note that tapping the play triangle button at the bottom right would already show a static, non-moving needle. Figure 4: The needle of a compass at all times has to keep pointing in the direction of the magnetic north, thus we need a Forever loop. Note the similarity with Scratch s blocks. 3
Figure 5: We then need a brick that points the needle in the correct direction. Figure 6: Drag and drop the Point in direction brick to insert it into the Forever loop. Using the formula editor, replace the default value 90 with the value of the magnetic field sensor that is built into the phone. 4
Figure 7: Select the compass sensor value, confirm the formula, and press the play button to start your app. Figure 8: Turn your phone: the needle keeps pointing to the north. Voilà, your first app! Exit your app by pressing your device s back button, and press the back button again to return to Pocket Code s main screen. 5
3. Share your program, or explore and remix programs that others have made Figure 9: Upload your new program to www.pocketcode.org, or download programs that other users have contributed to play and also to learn from them. All programs are available under free open source licenses, and remixing projects of others is encouraged as it helps to quickly understand how others have created, e.g., a moving background as used in the Air fight program shown above among the most downloaded programs. 4. Tinkering with Pocket Code in a middle school physics course When Google decided to feature Pocket Code on its Google Play for Education platform, it sent a camera team to Austria and gave 30 Nexus 7 tablets to Graz Bilingual International School (GIBS) to create a promotion video 2 with GIBS s teachers and kids using Pocket Code. Since then members of the Catrobat team located in Graz have continued to work together with the teachers and pupils of GIBS as well as with 16 computer science pedagogy students of Graz University of Technology to explore ways how to use Pocket Code in middle and high school curricula. Computer Science courses are compulsory in Austrian schools only in 9 th grade (age range 14 15), but teachers of other courses were interested in using Pocket Code for projects relating to their own courses. Indeed, with Pocket Code it is easy to let students create games or animations that include spoken foreign languages (recording one s voice from inside Pocket Code is straightforward), interactive geography quizzes with clickable maps, historical animated stories, animated mathematical proofs (see the Pythagorean Theorem program on the right of Figure 9), interactive art, and much more. One physics teacher immediately said she wanted to use Pocket Code with her 8 th graders (age range 13 14), who just had learned about forces, velocities, acceleration, and gravity in class. Together with the teacher we decided to let her pupils develop their own games or animations that, 2 http://youtu.be/75i10o_uv0u 6
as the sole condition, would have to somehow include simulated gravity. While a dedicated version of Pocket Code that includes a 2D physics engine, which greatly facilitates the programming of Angry Birds or Jump and Run type of games, is in alpha development stage, in the physics course at GIBS we wanted kids to focus on understanding the mathematics behind gravity, and thus we used the normal, non-physics version of Pocket Code. None of the 24 kids had any previous experience with Scratch or any other kind of programming. Though all of them owned smartphones, for the physics project we had them using the Nexus 7 tablets that Google had donated to GIBS. Of course some of the kids continued exploring Pocket Code at home on their own devices. Instead of teaching the class all details about Pocket Code, I first showed them, projecting from my tablet s screen to the classroom s wall through a beamer, some apps that were created with Pocket Code, including the Gossip Girl music animation and the Air fight 0.5 game that are both listed under Most Downloaded in Figure 9, as well as a live demo of how to program a Parrot AR.Drone using a special version of Pocket Code 3. We then gave the tablets to the pupils and I walked them through the programming of the compass (Figures 1 to 8), again projecting each step from my tablet to the wall. Three computer science pedagogy students from Graz University of Technology were present and were additionally supporting pupils on a one to one basis. After this stage, all kids had successfully created their own first smartphone app, namely the compass, had seen the drone being programmed live to fly in one meter height in front of them while the video from the drone s camera with themselves was streaming live to the beamer through Pocket Code on the classroom s wall using only five bricks of Catrobat code, and had seen an amazing music video animation that really was made by a girl of their age, as well as a graphically appealing game with airplanes that could explode spectacularly when shot down. Seeing is believing, but creating a program oneself allows to really acquire these essential skills The kids were thrilled and hooked! Since the pupils had absolutely no previous knowledge of programming but we had only a very limited amount of time for the physics project with Pocket Code, we had decided in advance that I would quick-start the kids on the physics project by programming a very simple game in a highly iterative way, together with them, while continuing to project my tablet s screen to the wall. Pupils who wanted to follow their own ideas were free to do so, and could at any time ask for help from the three university students who were also present. My first step was to have a green ball (with a red spot to indicate the direction of its movement) speed around on the screen while bouncing from the walls, without considering gravity yet (script of the ball in Figure 10). Note that Forever loops have a built-in delay (as in MIT s Scratch) so that games have the same speed on all devices, independently of the underlying hardware. Instead of just presenting the solution, I always asked them to first try to find the next step themselves before showing them one possible way to achieve whatever was next, e.g., by asking them to find and add the right brick to move the ball. Drawing the ball (or something more creative ) and letting it move in various directions, with various speeds, with and without the ball bouncing from the walls, all kept us busy for some time ;-) Once all kids had this successfully running in various forms, I suggested adding a paddle at the bottom of the screen. The paddle would have to be moving with the inclination of the tablet as held in one s hands. This required experimenting: Where to initially place the paddle, which inclination sensor to use, how to make sure that the paddle would not leave the screen, and with or without a minus in front of the sensor value. Kids who had already finished ahead of others were invited to, 3 See http://youtu.be/350-8fqgb1c for a video of how to program the drone using an earlier version of Pocket Code. Note that the version of Pocket Code supporting Parrot s AR.Drone is still in alpha stage. 7
in the mean time, decorate their ball and/or paddle, or to add a background, either by drawing the background themselves, or by taking photos with the built-in camera and then editing them, or to continue otherwise exploring Pocket Code and Pocket Paint (Pocket Paint allows to make parts of images transparent as well as to zoom up to pixel level, thus allowing to easily create and edit in a detailed way the looks of objects in Pocket Code). After a few minutes all kids had some kind of moving ball and a paddle controlled via the tablet s inclination (see Figure 10 below). Figure 10: First version of the ball and the paddle, without gravity yet. 4 The next step was to let the ball react to when it was touching the paddle, at first by simply stopping the ball s movement for some time using a Wait 1 second brick. Once this worked nicely, we added a small change of direction of the ball bouncing from the paddle, the change being proportionate to the current inclination of the tablet, i.e., allowing to cut the ball according to the movement of the paddle. All this took quite some time to get right (e.g., we had to decide whether to add or to subtract the inclination from the direction of the ball), but after some tinkering the ball started to react correctly when touching the paddle for everyone, so we removed the wait brick and finally could move on to adding gravity to the ball in the next step. First we added a speed variable to be used in the move brick. Next, in each iteration of the Forever loop, we first memorized the current x and y coordinates by storing them in appropriately named variables, then moved the ball in its current direction for speed steps, 4 Note that, while the Nexus 7 tablets have a resolution of 1920 1200, the program shown in the screenshots was made on an HTC Sensation phone with a smaller resolution of 960 540. The coordinate numbers given in the screenshots correspond to this smaller resolution. A Catrobat program always keeps the coordinate system of the device on which it was originally created, notwithstanding of having been remixed or being played on a device with a different resolution. 8
followed by a displacement of the ball on the y-axis by a gravity constant. We then computed the velocity vector by subtracting the memorized x and y coordinates from the new x and y coordinates of the ball, and finally computed the new speed by applying the Pythagorean Theorem to the x and y values from the thus updated velocity vector. The kids had just learned all of this in their math and physics classes, so it was straightforward for them. The next step might have been quite a challenge for adults, though: The kids had not yet learned how to use trigonometric functions, to say nothing of inverse trigonometric functions, which would normally be used to get the necessary new angular direction from the updated velocity vector. This value is needed to set the new orientation of the ball so that it moves in the correct direction when the loop is repeated in the next iteration. Luckily, Catrobat (and Scratch) provides a brick that allows one object to point towards another object. The solution thus was easy: Create a new pointer object (its look can be deleted once it has been created) and always place it at a position so that its x coordinate is the sum of the new x coordinate value of the ball and the x component of the ball s velocity (this can also be expressed as the sum of the previous x coordinate value of the ball and two times the x component of the ball s velocity), and similar for the y coordinate, and then pointing the ball towards this pointer object. And that s it: The ball would then move in parabolic curves, correctly bouncing from all walls as one would expect from a real physical ball! Parts of the final program 5 can be seen in Figure 11. Figure 11: Parts of the final program 5 showing the calculations of the new speed value using the Pythagorean Theorem ( Set variable speed brick in the screenshot on the left) and the new direction computed without any inverse trigonometric function from the new velocity vector including gravity. Kids then had time for extending this basic program on their own, e.g., by showing increasing life points whenever the paddle touches the ball, by adding more balls by copying the ball object, 5 The complete Catrobat program is available under the name gravity pong at https://pocketcode.org/details/1655 9
by converting the game to landscape mode, or, alternatively, time for creating an entirely new game. At the end of the class, I was amazed to see, e.g., a very nice looking Badminton game based on our basic gravity ball program on one of the tablets. Kids also clearly had caught on with the idea of iteratively creating more and more complex games or animations, always tinkering with some new aspect or feature, one small step after the other, with immediate feedback on whether it would work the way they had imagined, sometimes trying different approaches, and always learning from each other as well as from exploring the programs available on Pocket Code s sharing site. 5. Success factors of Pocket Code The following aspects contribute in a unique way (compared to similar programs) to sustainably motivating kids to create their own programs with Pocket Code: Kids can create their own programs as well as run and remix programs made by others on their own private and personal smartphones, which they have readily available in their pockets, at any place and whenever they have time, independently of schools and parents. Kids can easily show their own programs and those created by others to their friends, e.g., during school breaks, as well as family members, and also share these programs with them. Smartphones and tablets allow using various sensors (inclination, camera, etc) and effectors (sound, vibration, flash light) of the devices in one s programs. The multitouch nature of the smartphone and tablet displays makes it possible, e.g., to program a piano allowing to play chords with multiple keys at the same time. The high resolutions of modern smartphones and tablets, the ease of taking snapshots from, e.g., arbitrary web pages, the high resolution photos that can immediately be taken with the built-in camera at any place, the possibility to precisely edit each pixel s RGB+alpha values of the looks of objects with our Pocket Paint companion app, as well as the high speed of the built-in GPUs allows creating high quality, visually highly appealing programs. The small size of smartphones and the built-in Bluetooth capability allows to readily mount them directly on the front of Lego Mindstorms robots, thereby making these robots much more interesting through colorful and complex animation of their face, allows using the camera to locate and turn to human faces, sound and speech playback as well as speech recognition, and use of acceleration or self-localization sensors to make the robots really autonomous. And of course Arduino boards and Parrot s AR.Drone can be programmed The non-profit and multilingual character of our Catrobat project as well as the free open source character of programs contributed by kids themselves makes creating one s own apps widely accessible to kids in their own language on a worldwide scale. Our free open source approach and the high quality of our development process guarantee the long term availability and future extendibility of our apps and services. Many fascinating Catrobat subprojects that already are in an advanced development stages will soon allow, e.g., creating real Android apps 6 or Android live wall papers. The platform agnostic approach of the Catrobat project will soon allow kids to use whatever smartphone or tablet operating systems they use, independently on which platform a Catrobat program was originally created. Acknowledgements: See http://catrob.at/credits and http://developer.catrobat.org/special_thanks 6 The Tic-Tac-Toe-Master program https://pocketcode.org/details/817 created with Pocket Code, e.g, is available also as a regular Android app (http://catrob.at/gptttm downloaded 16,000 times from Google Play as of April 2014). 10