CLASSIFICATION OF TOOLS FOR USE IN INTRODUCTORY PROGRAMMING COURSES

Transcription

1 Research Report 91 Tutkimusraportti 91 CLASSIFICATION OF TOOLS FOR USE IN INTRODUCTORY PROGRAMMING COURSES Timo Rongas, Arto Kaarna and Heikki Kälviäinen Lappeenranta University of Technology Department of Information Technology P.O. Box 20 FIN Lappeenranta ISBN ISSN Lappeenranta

2 CLASSIFICATION OF TOOLS FOR USE IN INTRODUCTORY PROGRAMMING COURSES Timo Rongas*, Arto Kaarna**, Heikki Kälviäinen* *Laboratory of Information Processing **Laboratory of Communications Engineering Department of Information Technology Lappeenranta University of Technology P.O.Box 20 FIN Lappeenranta FINLAND {timo.rongas, arto.kaarna, ABSTRACT Teaching and learning programming are difficult tasks since programming requires new concepts in thinking and creative skills, especially for first year students. A number of tools have been built to assist teachers and students in introductory programming courses. These tools can be roughly divided into four categories: A) integrated development interface, B) visualisation, C) virtual learning environments, and D) systems for submitting, managing and testing of exercises. In this paper, a classification of the tools is presented. The classification is based on a review of the existing tool, both commercial and freely available tools. Guidelines for the selection of a suitable tool are discussed. Keywords: learning environments, programming, computer assisted learning 1 INTRODUCTION A number of tools have been created to support teachers in introductory programming courses. Tools used at Lappeenranta University of Technology (LUT) have been useful, but it has been questioned whether some additional benefit could be gained by using some other systems. Therefore, we started to search for systems that could help the teacher to teach and the student to learn programming. This document has been written to clarify the availability and usability of such tools, according to information gathered. It aims to be a guideline on what kinds of systems are available, and for what purposes they are suitable. The reader should get an overview of the types of systems available, and typical features in them. Systems like WebCT and BulletinBoard are widely used course management systems which could also possibly be used in programming courses. However, they are general tools, designed for managing any kind of course. This document tries to introduce systems created especially for programming, and therefore WebCT and BulletinBoard are not discussed in more detail. 2

3 The paper is organized as follows: First in Section 2 some of the problems in introductory programming courses are presented, from a teachers and a students point of view. Next, the description of different models for systems aiming to solve these problems is presented in Section 3. Some examples are given on each of the categories defined in Section 4, and finally discussion and conclusion are given in Sections 5 and 6. 2 PROBLEMS CONCERNING INTRODUCTORY PROGRAMMING COURSES 2.1 Introduction Programming is basically problem solving. First, the problem is analysed and then a detailed description of the problem is provided. The detailed description contains problem definition, input and output data description, basis steps of the solution of the problem and finally assumptions proposed and possibly an example case of a problem occurrence. The basis steps of the solution are then gradually developed such that finally the program source code can be written from the steps of the algorithm. The algorithms are displayed as flowcharts, structured flowcharts or as pseudo-code. In addition, the data structures required should be described. Governing the programming language assumes knowledge of syntax, semantics and pragmatics of the language. Thus, a basic course in programming includes techniques for problem solving and extensive knowledge of the programming language. Similarly, these requirements should be fulfilled by the tools implemented as computer assisted learning systems. The number of students on introductory programming courses has been increased in recent years, and the tightening resources do not allow hiring more teaching staff. As the number of students increases, the students have more are more different backgrounds. This leads to the problem of choosing the level of teaching. If we choose the ones without any prior experience as our primary target group, there is risk that students with some experience get unmotivated and skip the course and probably miss some vital information. On the other hand, focusing on the ones with some experience might lead to situation where others are unable to learn. [1] Therefore, tools used on programming courses should provide some motivation for the experienced students and the inexperienced ones. All the students should feel that engaging in tasks is a good idea, not just a mandatory thing to be done. [2] Ideally the tools should also be of profit to both students and teacher [3], support self-study, allow for possibilities to observe progress at all times and offer new and different pedagogical ways of learning [4]. Although most of the tools available lack some features that either students or teacher would welcome, it seems that the direction of development is going towards multipurpose systems. 3

4 2.2 Students point of view When a student is faced with her first programming course, she or he might encounter a number of different problems with learning. Students have the tendency to concentrate on the problem they first encounter which on introductory programming courses is usually syntax. For a number of students the idea of how the program works is unclear, and as such will lead to correct instructions written in wrong order. The student should first understand control structures, create an algorithm for solving the given task, and only after that start the process of turning the algorithm into program code. [5] These things can be problematic for some students, while others who can master the idea of program flow and syntax might hold the mastering of complex development environment as the harder task [6]. As it will be discussed later in this document, the tools for professionals are not usually suitable for newcomers in programming. One of the problems is definitely the need for a student to write a complete program before getting any output or feedback. The process of writing a complete program before being able to know whether it is correct so far or not, can be frustrating. Systems have been designed for precisely this purpose, providing program code skeleton, where the student just fills in the gaps. This way concentration is more focused on the topic at hand. [7] Although compiler messages are informative, they are directed towards professional programmers [8] and might scare the beginning student with their complexity. At some point it is of course necessary to introduce students to compiler messages, but should it be done with the first programming exercise, or only when moving to a more professional environment? Getting feedback and personal assistance from the teacher might be impossible, or at least slow if there are a lot of students the teacher has to take care of. The lack of feedback leaves students less time to improve their solutions, and might lead to breaks in the learning process. [9] 2.3 Teachers point of view The more the students actually do in practice and the more they get response to the solutions and guidance to the problems, the more they learn. Unfortunately, giving feedback to numerous programming exercises for a large group of students is nearly impossible, and without tools it is a very time-consuming task. [3]. If feedback and corrections for simple errors were given by an automatic system, the teacher could concentrate more on teaching the subject more deeply, instead of doing the timeconsuming task of checking a student s code for simple mistakes. [10] Even if there were be time to consult and give feedback to each student separately, checking the source codes and doing test-runs manually would be error-prone even with small programs, and might lead to incorrect advice. This could be made easier, if the students were required to submit test results with the data, but this gives the student a chance to make the program such that it provides right answers to just the 4

5 selected test cases. It might also be too difficult for some beginning students to handle testing properly. [11] On the other side, using the traditional classroom model for teaching, including lectures and exercises, does not give very strong contact with the students. Problems like the difficulty of getting feedback and general comments from the students and assigning projects for them arise could be solved with a system where such things would be handled automatically. [12] 3 SOLUTION MODELS Different kind of approaches for solving the aforementioned problems exist, but mainly they tackle just one or two of the problems, leaving some things unsolved which slows down the learning and teaching process. By combining tools suited for different purposes, one might be able to build a system that automates the managing of tasks and projects as far as possible, while steering the student to work on the tasks to focus on the right things. Some of the systems are even designed to take care of the whole process on both the students and teachers side. The systems found can be coarsely divided into four categories: A) IDE (integrated development interface), B) tools using visualization such as AAPS (Animation Aided Problem Solving) and mini-languages, C) virtual learning environments for programming covering a whole course, and finally D) tools for submitting and testing student assignments. This section presents a brief description on each of these categories, and examples of the systems belonging to certain category can be found in Section 4. A few systems not belonging to any of these categories are described after the description of the categories. 3.1 Integrated development interface IDE (Integrated Development Interface) is a system where editor, compiler and/or interpreter, tools for build-automation, and often a debugger are all found within the limits of the same system [13]. Typically an IDE is quite a large system, holding a lot more tools than necessary for a student who is just beginning to learn the concepts of programming. Therefore the IDEs designed for professional users are not really suitable to be used as learning environments, as the amount of available tools can easily overwhelm the student and distract her or him from the actual learning process. [6] As a result, a number of IDEs have been created to suit to the needs of introductory programming courses. These are simplified in such way that they are as easy to use and understand as possible, but still offer the required functionality for the student to be able to produce complete programs [14]. A common problem with these minimized development environments is the fact that they rarely offer any use for more advanced students. When the students learns more it might be that the features of minimized IDE fall short [3], and she has to learn to 5

6 use another IDE. Learning to use many environments in the beginning can be considered as a waste of time, but starting out with a simplified IDE could focus the students on right things, although it requires the adoption of another IDE later on. 3.2 Visualization One problem to consider is whether students comprehend how a program flow proceeds. Many students have trouble to understand the states the program goes through, which will easily lead to mistakes done by writing syntactically correct sentences, but giving them in wrong order. This can be helped by showing to the student some visual output that demonstrates what is happening during the execution of the program what are the values of certain variables at certain moments, and when and how they change. [1] The comments given by a few students at LUT, by no means a large enough population to give general results, supported this idea: simple statements are understandable, but the structure and run-time behaviour of the program are harder to realize. Although mini-languages, programming languages with small and simple command sets [15], could have been given their own chapter in this document, they are explained here because of the similarities they have to other visualization systems concerning the way of learning. Most of them provide output through some kind of graphical entity that is controlled by coding instructions. The line between visualization (or AAPS, Animation Aided Problem Solving) and mini-language based tools is thin enough to consider them as one group. 3.3 Virtual learning environments Systems, called virtual learning environments are tools created specifically for programming. They are created with the thought that all the tools required on a programming course, for both the student and teacher, should be found inside one system. In addition to being able to edit, submit, and get feedback on assignment, students should also be able to find a necessary theory on the subject through the system. Having all the data in the same place can also ease the teacher s burden to manage the course. [3] Virtual learning environments should also include tools for interaction with other students and teacher, such as instant messenger or message forum. These features are often brought to courses through a separate system. [16] Having all the communication tools available while accessing the learning environment, instead of separate tools or no tools at all, should make it easier for students to give feedback and interact with each other and the teacher. 3.4 Submission systems There are many ways for students to return their assignments to be evaluated. The old way of printing the code on paper with output, or returning assignments by are 6

7 quite out of the question these days with the volumes of students taking part in introductory programming courses. In interactive learning environments, submitting an exercise is a basic feature. Submitting systems could therefore be said to exist as a subset of these learning environments. The returning of a student task consists of three parts: submitting, evaluating, and marking. There are systems available that take care of all the three parts, but even then some user interaction is required in the evaluation phase [17]. In addition, writing tasks that can be evaluated automatically as far as possible is an extra effort for the teacher. The difficulty of creating such a task depends on the system used. However, comparing this to the trouble of handling many student tasks evaluations entirely by hand, the use of testing system should be an easy choice. As there are quite many ways to create this kind of functionality the approaches are naturally quite different. Some systems include only submission, others only testing. Some systems are created keeping in mind easy usability, others use the Unix command lines [17]. Mainly testers are helpful tools for teachers, but they are commonly of profit to students as well. Automatic testing makes it possible to give instant feedback without any human checking, so that students can be immediately alarmed to possible errors in their solutions [17]. 3.5 Summary The systems described above provide various properties. In Table 1 below we have collected the most essential properties from the programming point of view. The grading is no, low, medium, and high describing the quality of the support for the property in each category. Environment Table 1. Support of properties in each category IDE Visualization Virtual Learning system Property Support for no no Possible no Problem Solving Syntax low low high low Semantics low low medium low Pragmatics no high high low Support for Self-Study Support for Teacher Suitability for a newcomer low medium high low no no high high medium medium high low Submission Systems 7

8 4 AVAILABLE SYSTEMS This section gives a brief description on some of the systems belonging to the aforementioned categories. Other systems not described here will also be pointed out to give a wider scale of information of systems in a certain category. Appendix A will hold links to information on the all of the systems, categorized in the same way as elsewhere in this document. An interesting point to notice was that most of the systems found during the creation of this document were free. Commercial products were available on the specific tasks of programming, but there was few information available about them. Asking for a presentation would probably have given more information on the insides of commercial systems, but as the scope of this study was limited, the information mainly originates from user comments and web pages. The use of commercial systems is usually thought to provide the guarantee of service and quality. However, a lot of effort has been put in to the creation of many of free systems, and as such it is not quite clear whether commercial system is always the better one. Evaluating the cost of supported commercial system versus free system, possibly requiring more administrative work, could lead to very different results in different environments and institutions. 4.1 IDE This chapter has a brief description on three IDEs: BlueJ, DrJava, and Eclipse. There are lots of commercial and free IDEs around, but most of them are targeted for some other group than beginning students. Probably there are also some IDEs available for our target group in addition to the three discussed here, but DrScheme, the model for DrJava, was the only one that came up during the search. Considering Eclipse and its philosophy, there could be lots of helpful tools for beginning programmers built on top of it. Appendix A lists all the systems belonging to this category with links to additional information BlueJ BlueJ is a simplified integrated development environment for Java, which has been developed having especially teaching of object-orientation in mind. It was developed at a university, to be used in a university. BlueJ and all the accompanying tools are absolutely free. It is written in Java, so it should work on all platforms that have a JDK new enough installed. [6] The development of BlueJ first began at Monash University, Melbourne, Australia. It is based on the Blue system, which is an integrated teaching environment and language also developed at Monash. Now BlueJ is being developed and maintained by a joint research group at Deakin University, Melbourne, Australia, the Mærsk Institute 8

9 at the University of Southern Denmark, and the University of Kent in Canterbury, UK. The project is also supported by Sun Microsystems. [6] BlueJ is a capable IDE, containing its own editor, compiler, and a debugger. It has a graphical display of the classes and their relations, and new classes can be inserted into the program by drawing them on the display. The editor of BlueJ is displayed in figure 1 and class display in figure 2. When a new class is created BlueJ automatically creates a template for the code, so that all the basic functions needed (e.g., in the case of applet init(), start(), stop()) are already given, with comments telling what they are there for. Figure 1. Editor screen of BlueJ. One of the strongest features of BlueJ is the capability to interact with single objects and execute their public methods in addition to executing the whole program. The student does not have to write a complete program to know if a certain function does what it should. There is also a possibility to do unit testing with JUnit. [6] The possibility of creating extensions for BlueJ exists, but so far the only extension available is a submitter. This extension is also automatically included in the package of BlueJ. This is a nice addition to BlueJ as it gives a possibility to create a list of exercises for the whole course. Submitting can then be done straight from the IDE, and there is no need for the student to worry about sending all the required files. The submitter takes care that all the files belonging to the tasks are sent, and it can also be configured to require files with certain names. All the student has to do is follow the naming rules given in the exercise, select the right exercise, and submit. [6] 9

10 Figure 2. Class display of BlueJ. The configuration for the submitter can be obtained from the Internet, so all the students have always up to date, common settings. The automation of task deadline settings would have been a nice feature since now tasks have to be set on/off manually in the configuration file [18]. An extension that would add the possibility to include some theory and the exercise descriptions to the BlueJ environment would turn the system into a virtual learning environment. Now theory and exercise definition have to be given through some other route. Instructions for creating extensions are available at the BlueJ website. The system is also missing a tool for evaluating the submitted exercises, but it is possible to combine the submitter with another tool. Returning the output of the check to the student is already implemented in BlueJ. BlueJ is in use at LUT in a course dealing with object-orientation and Java (course description on www-page index_eng.html?cid= ) In Finland at least the universities of Kuopio and Joensuu also use BlueJ on some of their programming courses [19, 20]. Worldwide, there are over 300 universities or other educational institutes using BlueJ [6]. However, the number is probably somewhat larger as registration is not required for download. A list of institutions using BlueJ can be found from the BlueJ website. The BlueJ website also holds some crossword puzzles which might be a nice thing to point out on a introductory Java programming course. They include subjects on object-oriented programming and Java, so there is no need to be using BlueJ itself to get some advantage of these. 10

11 4.1.2 DrJava DrJava is a lightweight IDE under active development in Rice University, USA by the JavaPLT group. It is mainly designed for students with little experience on programming, but also includes some features for more advanced users. DrJava is free as it is available under DrJava Open Source License. Another similarity to BlueJ is platform independency. DrJava is created with Java and it is available for any platform with a recent enough JDK. [21] DrJava tries to be easy to use and understand by having a simple user interface with just a few panels and a legible toolbar [21]. The interface of DrJava really is clear, and locating all the different tools is easy. It is of course a personal opinion whether a single window with all the tools is better or worse than multiple windows, but DrJava and BlueJ are fairly equal in simplicity, although DrJava looks a more professional product. Like BlueJ, DrJava also offers the availability to test and run single methods directly, without running the whole program. This is done through the interactions pane, where student can execute any valid Java statement or evaluate expressions. The resulting output or error will be displayed on the interactions pane. [21]. Interactions pane can be seen in figure 3. A plugin for IBM s Eclipse development platform is also being created (see Section 4.1.3). So far a fully functional interactions pane and a simplified user interface (Eclipse perspective ) have been created. Integrating the debugger from Eclipse with the Interactions Pane will be the next major step. [21] There is currently no way to convert DrJava in to a virtual learning environment directly, but through using the Eclipse and DrJava plugin the necessary features can probably be implemented. At the moment the use of DrJava on a programming course is of no profit to the teacher, besides having a simple environment to familiarize the students with. On the other hand, a student might profit a lot from the different features of DrJava, as the system is well designed for the purpose. 11

12 Figure 3. DrJava with interactions pane visible Eclipse Eclipse is an open extensible IDE for anything and nothing in particular [22]. This means that Eclipse is a platform for building IDEs for different purposes. There is a lot of generic, basic functionality in Eclipse itself, but to get the real use out of it, one should have plugins to fit the task at hand. [23] Especially considering beginning students, the interface should be made simpler with a fitting plugin. There are even ready tools for supporting the learning situations, but features like giving course material nicely available are missing. [24] There is a large scope of plugins available for Eclipse, starting from C/C++ and Java development to UML modelling to playing mp3 [25]. As it is possible to get a lot of functionality from Eclipse it might be possible to learn basics of programming with some simplified plugin (e.g. DrJava) and when more features are required, switch to a more advanced plugin. This way the student could continue to do more professional programming in the same environment. Figure 4 displays Eclipse without any plugins in use. 12

13 Nine large founding companies formed Eclipse in 2001, including IBM, Borland, Rational, and six others. Today the number of companies committing to the development of Eclipse is 41, and is still increasing. A complete list can be found from the eclipse website. IBM itself uses Eclipse as the platform for its WebSphere Studio family of products. [26] On the whole, Eclipse is too large a system to be described here the way it should. There is a lot of potential on it, and some of the tools described in this document are built on the Eclipse platform as plugins. Taking Eclipse to use gives the possibility to use it on several courses on different programming topics by just changing the plugin used, and selecting a fitting plugin for a course from the wide variety should not be a problem. Eclipse is still a fairly new platform, and the number of plugins for different uses will probably still increase for a long time. Figure 4. Eclipse screen without external views or plugins. 4.2 Visualization Since LOGO, the task of programming robot motion has been used in many systems [27]. The idea is present at many of the systems found during the background work for this document, but also different kinds of approaches through graphics are available. Two systems called Jeliot and Kara will be described here. The visualization tools are mainly just supportive tools, and could easily be added to a normal lecture-exercise method of teaching. Alone they are insufficient at teaching 13

14 the student, but could help the process of understanding different concepts of programming by lowering the level of abstraction. Appendix A lists all the systems belonging to this category with links to additional information Jeliot Jeliot, an Interactive Algorithm Theater, was originally a product of the AAPS (Animation Aided Problem Solving) project working at the Department of Computer Science of the University of Helsinki. It was developed by researchers of the project and two groups of students, based on the ideas of Eliot. It soon became too complex for only the students to develop as part of their studies, and since then it has evolved into a usable and stable product. [28, 29] The latest version, called Jeliot 3 has been developed at University of Joensuu. This version is based on the Jeliot 2000 which was developed at Weizmann Institute, Israel. [29] The idea of Jeliot is to provide run-time animation of what is going on inside a program. User provides Jeliot with source code, which is automatically turned into an animation. The program code is interpreted as a script of a play, and the variables are the roles that change values and act according to the script. [28] The first version of Jeliot had the feature of selecting which data structures are shown on the screen. However, this feature does not exist in Jeliot 3 anymore, probably because there is no need for it due to a much improved display of interactions. Although in the first version of Jeliot it was a bit hard to follow the program flow, Jeliot 3 is very clear and easy to observe. Now a program code is visible on the side of the animation, and the part being played is always highlighted in the code. Jeliot running algorithm showing recursive Fibonacci is show in figure 5. 14

15 Figure 5. Jeliot running recursive algorithm displaying Fibonaccis numbers. The animation can be run step-by-step, or automatically with adjustable speed. The settings can even be altered while the animation is running. [28] If the program needs user interaction Jeliot prompts the user with a dialog asking for certain type of input All output is printed on output box, located visibly under the animation. Jeliot is really easy to use, as the controls are clearly visible and easy to understand. [29] Jeliot was originally created with a client/server model, and as such required the user to only have a java-capable browser to be able to run it. This also decreased the need for heavy calculations on the client side. [28] Jeliot 3 is a standalone application which needs JDK to be installed. All the calculation is done on the local machine [29], but it should not be a problem with the computers mostly in use these days. Actually, being able to use the program without network connection is more of an advantage. Jeliot 2000, which was especially designed for introductory programming students, understood only a small subset of the Java-language [30]. Jeliot 3 looks the same as Jeliot 2000, so no compromises with the usability to beginning students have been done. However, Jeliot 3 already understands most of the Java constructs and is able to animate them. The next step in development will be to add capability to animate object oriented features, such as inheritance. [29] Kara 15

16 Kara, the programmable ladybug, is another kind of system for learning-byvisualization. It is being maintained and developed by the Department of Computer Science at Swiss Federal Institute of Technology, Zurich. It can be freely downloaded from at [31]. Kara is based on the concept of finite state machines. It creates animations of Kara the programmable ladybug living in its own flat world according to these state machines. [27]. Figure 6 shows the execution window of Kara. Figure 6. Execution window of Kara. Finite state machines can be seen in every-day devices, such as traffic lights, which make them a great means of explaining the concept to beginning students. An example solution modelled with a finite state machine can be seen in figure 7. Comments on Kara from students have generally been positive. It has been a source of motivation towards programming, especially for those who have had no prior experience on the subject. [27]. Kara comes in many forms. There are five different views of the program, all targeted for slightly different purposes. The basic model is Kara where the ladybug works alone in its environment. Second it comes MultiKara where there are multiple ladybugs inhabiting the same world. After the student learns how to guide the ladybug with finite state machines drawn with mouse she or he can switch to learning Java with JavaKara. There the student is provided with a library of functions to handle the ladybugs behavior. There is also TuringKara which provides the possibility to experiment with turings machines. Here the world naturally consists of only ones, zeros, and marks instead of leaves, trees, and mushrooms. Finally, there is LegoKara which is meant for programming a robot built out of Lego Mindstorms Set. [31]. 16

17 Kara has been programmed in Java, so it can be run where ever there is a new enough JRE available. Running JavaKara requires a compiler in addition to JRE, so JDK must be installed if one wishes to use it. [31] Kara has a bunch of exercises for all the different views. They are an easy way to start learning, as well as a nice introduction to the systems behaviour. The exercises are given in a clear way, with sample solutions nicely available. The variety of exercises is large, ranging from simple pick-a-leaf-if-it-exists tasks to calculation of Pascal triangles and sorting. JavaKara even had an example exercise on how to draw Mandelbrot sets. On the whole, Kara is a well-implemented and interesting tool for playing around even if you have some programming skills. The system is extremely clear, and the usability of the interface is high. For teaching state machines, there should be no better and more illustrative way. JavaKara also offers a nice transition from state machines to Java programming within the same environment. Although there are limits to the tasks that can be taught with Kara, it is a nice tool for introducing students to the world of programming. Figure 7. A sample solution modelled with a state machine in Kara. 17

18 4.3 Virtual learning environments A lot of research referred in this document seems to point on the fact, that having all the tools in the same package is a good thing. Virtual learning environments are created with this philosophy. They are a fairly new branch of development, and as such there were not many ready environments available. This document describes three of them: Viope, GILD, and ELP. There were systems that were pretty close to being categorized here, but they were designed in the way that they were better suited for some other group. Again, appendix A lists all the systems belonging to this category with links to additional information. All three systems described here are fairly new, and only Viope contains the status of a complete product. Even though then it lacks some essential functionality which makes the staff complain about its use. GILD and ELP are still under development, and in general it could be guessed that the number of systems in this category will probably increase in the near future. Again, appendix A lists all the systems belonging to this category with links to additional information Viope Viope is a commercial virtual learning environment created by Viope Solutions. They offer basic courses on different programming topics, such as C, C++, and Java. A course on SQL is also provided. The courses are provided through a web interface which can either be run from Viope Solutions server or installed at the institute s own server. [32] The main target group for Viope courses is the ones with no or very little prior experience on programming. The courses are also said to be suitable for more experienced students for testing and practising already learned skills. [32] Viope courses are a true virtual learning environment, as they include theory, exercises, and means of communication with teacher and other students. Communication is done through a message forum, so it is possible to post a message that everyone has access to. Theories on the courses are organised in chapters, leading the student with little steps from the basics towards the end of the course. The exercises are also grouped according to the theory chapters, and are available right after reading the theory on the subject. There are two types of exercises: multiple choice and programming. Both types are usually included with every theory chapter. It is possible to deny the student from accessing more difficult topics, in case they have not completed the more simple ones first. An example of this can be seen from figure 8, where in the upper-left corner there are all the exercises listed, but only the first one is available. This way student can be forced into going through the material in right order. [32] Exercises in Viope are formed in such a way that the student is not required to answer with a complete program, but with only a part of it. This way learning is concentrated 18

19 on the topic at hand, not on other things taught at a different time. If a full program is required, the student needs information on things like main-function and return values which are probably not the first things taught on a course. Figure 8. First exercise of Viope C, labeled first touch to programming. Viope gives instant feedback from the exercises. Multiple-choice questions are immediately answered with the information whether the answer was correct or not. If it was not, the student is provided with the right answer, without having to make another try. Programming exercises is a more complicated issue. Viope Solutions has developed automatic checking and feedback system (ART), which checks the solution and gives feedback on it. If the program is correct, the student is informed and allowed to proceed to the next exercise. If the solution gives a wrong answer or does not compile, the student is provided with information on what went wrong. Compiler messages are never visible to the student, so she has to rely entirely on the messages provided by ART in natural language. [32] This might be somewhat disturbing for the students who know some programming as they might be used to reading compiler given messages. In addition, the messages and tips given by ART were quite uninformative, but there has been some improvement, with more coming. 19

20 One of the major problems in the system concerns testing. The output of the answer must be exactly like the output of the model solution, with all the tabs and spaces printed equally. This might be frustrating for the students, as all the typos will lead to failure. Some improvement to this has also been promised to be coming in the future. [33] The teacher has the ability to add material and assignments to the course. Theoretical information can be added through links, and multiple-choice questions can be created and combined as a quiz, which can be evaluated as an exam. [32] Inserting programming tasks is also possible. The teacher can define the exercise, give a sample solution, and some test-cases to check the students assignments against. The first test case is used to create a sample output for the exercise definition by running it through the solution provided by the teacher. This has the same problem as the basic exercises: checking the output is done character by character, so typos will lead to the failure. However, on the positive side, it is even possible to require a task that needs user interaction. LUT has been using Viope C on the first programming course, with the final task requiring the creation of an ASCII-based menu, which can be navigated by numbers. Testing is done according to a predefined set of commands, with output compared to a model solution with the same input. [33] One more problem with the interface of Viope exists. It is only possible to see one screen at a time, so viewing a theory while solving an exercise is impossible. Of course, in the middle of exercise it is possible to go back to the theory, and then continue, but simultaneous viewing of both screens could make the system more usable. [33] GILD GILD stands for Groupware enabled Integrated Learning and Environment. It is a plug-in for Eclipse, created in the Department of Computer Science at University of Victoria, Canada, with the participation of IBM Centre of Advanced studies. It can be freely downloaded for use at [34]. As GILD is run as a plug-in on top of Eclipse, it should work on all platforms Eclipse does. [34] A lot of effort has been put to the design and development of GILD. Especially, the requirements analysis has been done deeply, taking into consideration the features of already existing systems like BlueJ and DrJava [3]. The system is not yet complete, and the development is currently at version [34]. Figure 9 shows a screen capture of GILD. The approach of GILD to the learning situation is a bit different from others. It is emphasized that all communication and work is done in the same environment, but currently only a small portion of functionality is implemented. The goal is to build an environment for learning situation where students could take part more actively. A scenario describing this goal is given by Peggy Storey in the GILD application for an Eclipse grant. [16] 20

21 In case the scenario described is such that it could be applied at the institute, then GILD should be something to wait for. At the moment functionality that have been created make it possible to use GILD as an IDE, but not much more. Development is still at quite an early stage, so getting to take part in building the system should still be possible. Figure 9. GILDs IDE and web-browser screen open in Eclipse ELP ELP stands for an Environment for Learning to Program. It is an online, active, collaborative and constructive environment developed at Queensland University of Technology, Australia. The project was motivated by the poor success of beginning programming students. [7] Currently ELP is being rearchitectured and the next version should be available early in 2004, with some significant changes done. Most likely it will be set under a free license. [35] The following text is based on a document written on an old version of ELP, and as such might be outdated already. The idea of ELP is to provide the students with an easily accessible environment, which supports the learning process of Java programming through providing problems and exercises the students can work upon at their own pace. The programming tasks usually include a program skeleton, to allow the students to focus on certain subject. [7] Focusing the student on learning to program instead of learning how to use and set up an environment is also considered important in ELP. The system is used through the 21

22 web, so the student is only required to have JRE installed. ELP detects whether it is installed, and will lead the student to a page where it can be downloaded and installed in case it is found missing. [7] Currently the student is able to use ELP for reading subject study guides, tutorial question,s and exercises. Exercises can also be answered by writing the solution on a field provided by ELP. Features for saving, resetting, and viewing exercise results are implemented, as well as compiling and running the solution codes. At the moment the only feedback on solutions is the compilation errors, if such exist. However, it is already possible to view a sample solution given by the teacher. [7] The weekly tutorials and exercises are written in XML, so even the definition files are easily readable and modifiable. XML also allows the creation of ones own tags. This way the teacher can format the information in any way he wishes. Solutions and hints to the exercises are marked where the student is supposed to fill her code in the given skeleton. [7]. This can be seen in figure 10, where an example exercise is being displayed. Future work on ELP will include automatic, both static and dynamic, analyses of students programs. The results created by the analysis will be returned to the student immediately, and will also be saved on the disk. Another thing not yet developed is collaboration: the students will be able to work alone or in groups. [7] ELP seems to be a promising system, with some crucial development still missing. When completed, it probably is an easily manageable environment for the instructor, as well as a useful learning environment for students. 22

23 Figure 10. Code skeleton and options on it in ELP. 4.4 Submitters & testers Submitting and testing systems were by far the widest category. All kinds of solutions have been created the easy the burden of the teacher in various tasks. Even basic tools meant for the general use such as Expect and DejaGNU could be taken into use in this category, as the students are not required to handle them directly. This category could also have included a number of general-purpose course management systems, but as no specific tools for them in the field of programming assignment were found, they were left out of the scope. Nevertheless, such widely spread systems as WebCT, Bulletin Board and WebAssign are worth mentioning here. This chapter describes two systems, BOSS and CourseMarker. Links to information on the systems described will be given in Appendix A BOSS BOSS is an assignment submitting, testing, and grading system developed by the Department of Computer Science at the University of Warwick, UK. It is developed 23

24 under the Gnu General Public License (GPL) and as such is freely distributable with the source codes available for modification. It is intended to provide a secure method for students to return their tasks online when most of the other submitting systems don not consider security as an important requirement. For the teachers it offers assistance in the testing and grading process of the students work. The testing features can also be set to be available for the students, so they can test their work before submitting it for evaluation. [36] BOSS has been written in Java to make it usable on many platforms. It has been tested on several platforms, including MacOS X, Windows 95/98/NT/2000, Linux, and SunOS/Solaris. There are two interfaces for BOSS, a web client done with JSP and a locally run client written in Java. The updating of the software in case of the locally run Java client is managed by having a small bootstrap program installed on the students machine, which downloads the actual client software from the server. [36] All submissions end up on a database on the server where they can be taken for examination. Saving the submitted solutions in a database provides an easy way to calculate statistics on the course. Comparing different versions of solutions always allow for analysing of students progress. [36] Originally BOSS was designed to handle only Java programming assignments. Development has taken a step forward, and at the moment it can be run in two different modes, making it possible to submit and test also non-programming exercises. Testing non-programming tasks uses the method previously also used for programming assignments where the programs output with certain input was compared character by character to a model answer. By simplifying this process to testing the submitted file against a model solution, questions requiring numerical answer etc. can be tested. [36] Testing of Java programming assignment is a lot more sophisticated task. BOSS is integrated with the JUnit framework, which makes it possible to use objects as input and output. Given a certain object as input, the students program should return an object matching the given input according to the exercise definition. The students can be provided with I/O code to make the task easier, but this is up to the teacher to decide. If program skeleton is given, the students modules can be inserted into possibly even complex environment where the behaviour of the module can be easily observed and checked. [36] The teacher is able to categorize different assignments, and set weights on them. Availability dates and deadlines can be set individually for each assignment. After the student submits her solution, BOSS runs automatic tests on it, and possibly assigns some kind of marking for the student s information. Afterwards assigned markers go through the submissions, assign final marks on them, and leave the authorization of the final marking grid for the teacher. There is nothing to prevent the teacher from working as a marker also, but usually markers could be assistants or post-graduate students taking care of the exercises for the course. [36] Sherlock Plagiarism Detector is also included with BOSS. It can be run in separate modes for natural language assignments and source-code, making it possible to 24

25 control plagiarism on all assignments. More information on Sherlock can be found at [36]. On the whole, BOSS seems to be a capable system for handling students exercise submission and marking. Programming assignments in Java are already taken care of, but as BOSS is open source it should be possible to implement a checking system for any language, and just replace the one for Java. The marking and submission part should not lose any of their features, but this is of course dependant on the way BOSS has been coded. [36] CourseMarker CourseMarker is a student work assessment and administration system based on 10 years of experience with the Ceilidh system being used in several institutions. It is commercial software, developed by School of Computer Science and IT in University of Nottingham, UK. For a while the system was called CourseMaster, but as another system with that name already existed, the name was changed to CourseMarker. [37] CourseMarker is created with client/server architecture. The client provides the students with an environment for managing their exercises, with features including leafing through the tree-view of exercises, submitting and receiving feedback on the solutions. The selected IDE for developing the exercises can be spawned directly from the CourseMarker client. The process of installing and learning to use an IDE is left entirely on the student. [37] The teacher is provided with automatic marking and archiving of students submissions, plagiarism reports. and capability of monitoring student and class progress. A lot of information is held in the system, such as logs of all submissions, all the versions of student s submissions etc. Due to this the teacher can have a good view on the status of the students at all times. The views showing different statistics are clearly formatted, and easily readable. The teacher is also responsible for opening and closing the exercises on given dates, and for assigning different weights depending on the difficulty of the exercise. CourseMarker is also able to analyse even the typographic style of students code. For example, too many or too few comments, wrong type of indentation and variable name lengths are things that can be detected and used in the evaluation. [37] The system supports at least C and C++ programming exercises [37]. Some information to which links were given on the website was not available, as it was restricted only for viewing inside the domain of University of Nottingham, or protected behind a login. Therefore a really good picture of the systems current state was not obtained. CourseMarker seems like a capable system for handling exercises on a programming course. It has a lot of functionality, including extensive checks done on students submissions in addition to maintaining the course. 4.5 Other Systems 25

26 In addition to these, there are some tools for certain purposes that do not exactly belong to any of the previous categories. CodeLab for example has quite similar idea to Viope, but lacks all the functionality around the exercises. Therefore it is listed here with CodeSaw which also lacks the functionality to be listed in the previous categories. Again, more information on them can be found through the links found from Appendix A CodeLab CodeLab is a commercial programming exercise product created by Turings Craft. It a collection of short, focused programming exercises available on the web with a checking system to give feedback from the solutions. Supported programming languages so far are C, C++, and Java with some questions touching the basic knowledge on procedural and object-orientated programming. [38] CodeLab is accessed through the web from the TuringsCrafts server. It is not possible to get CodeLab to run on institutions own server. This way TuringsCraft ensures good support, liability, and ancillary environmental software to be set as they should be. The teacher has to do no installation, and more importantly, neither does the student. All that is required is a common browser. However, Flash plug-in and Java plug-in are required from the teacher to be able to run the course management system. [38] The exercises are not complete programs, just parts of them, always focusing on a certain topic. There are more than 200 exercises available, but as they are only short, specific tasks, it is possible to go through them during a course. At the moment it is not possible for teachers to add their own questions, but suggestions on what is missing and should be added are welcome to the TuringsCraft company. [38] CodeLab provides the student with exercise definition and a textbox to write the solution in. After submitting the solution, CodeLab gives immediate feedback telling what was wrong in the code, suggesting possible solutions to fix it or just says the solution was correct. The feedback on the exercises is clear and extensive. First verbal information is given, than the compiler message, and finally the code is printed inside the exercise skeleton. After seeing this output, fixing the error is easy. [38]. Figure 11 shows Codelab giving feedback. Model solutions are available after the deadline has passed. Exercises also contain hints, through which the student can acquire some information and reminders on the topic at hand. The theory is supposed to be taught on lectures, but an active student might be able so solve most of the exercises just with the help of provided hints. [38] CodeLab gathers information on student progress, so that the teacher always has upto-date information on the status of the students available. Setting deadlines for a certain group of tasks is possible, so the student always has to complete exercises for a certain topic after the topic has been taught in theory. [38] 26

27 TuringsCraft also claims CodeLab being a good helper for switching from one language to another. Knowing the basics of programming in general, it is easy to learn the syntax of the language with CodeLab. [38] Figure 11. Codelab showing feedback on an exercise CodeSaw CodeSaw is a text-book supporting product from Liqwid Crystal. It provides the examples presented in Addison-Wesley text books for viewing, compilation, and running. This way the student can test the examples she sees in the book immediately without having to write them. [39] Many exercises in books focus on some details, and the writing of a whole program structure to test some changed structure solution can be frustrating. With all the examples provided, motivation to trying them out increases. An example exercise in C is shown in figure 12. CodeSaw is free for use after registration. Currently it has exercises from books on topics C, C++, Java, Perl, Python, Ruby, XML, HTML, PHP, SQL, PL/SQL, and JavaScript. It works by providing the student with a small program, that she can use whenever she comes across an exercise in the book. She goes to the CodeSaw website, selects the book she is reading, and clicks on invoke CodeSaw-button found next to the book. The code will automatically be downloaded on the students computer, and opened on the CodeSaw program. Then the code can be compiled and 27

28 the executable run. At the moment it is not possible to do modifications on the code, but a chance to run own code and modify the given codes will be provided in the future versions of CodeSaw. [39] CodeSaw cannot take care of exercises for a course, but it is a great support while the student is reading a textbook. As it is free for use, there should be no harm in pointing the students to CodeSaw website. Of course, to be of any use, the students should have Addison-Wesley books included in CodeSaw. CodeSaw only works on Windows, which could be a limitation for some students. [39] Figure 12. CodeSaw with a C example. 5 DISCUSSION The classification of the environments was presented in Chapter 3. IDE-systems support mostly learning the syntax. Visualization systems are useful as soon as the student is capable of producing full programs. Then they support the pragmatics of the language. Virtual learning systems are still under development, they have capabilities 28

29 to provide most broad range of support for both the student and the teacher. Submission systems are good in reducing the routine work of the instructor. The overview of the properties for each environment was compiled in Table 2. The properties selected are such that they at least in some dimension differentiate the systems. Still one can get an overview of the systems properties by viewing the table. Property Table 2. Properties of the Environments Support for self study Support for teacher Partial compi ling Free/ Commercial Language Suppoted Networked required Ease of Setup Newcomer/ Experienced Operating system Environment BlueJ Free Java Medium Low No In setup Easy Newcomer Any DrJava Free Java Medium No Yes In setup Easy Newcomer Any Eclipse Free Several NA NA NA In setup NA. Experienced Any Jeliot Free Java Medium No No In setup Easy Newcomer Any Kara Free Comp. Medium No No In setup Fair Newcomer Any Science Viope Comm. C, C++, High High No Yes Nil Newcomer Browser Java GILD Free Java NA NA NA Yes Easy Newcomer Any ELP Free Java High High NA In setup Easy Newcomer Browser BOSS Free Java Low High No In setup Easy Newcomer Any Course- Comm. C, C++ Low Medium No In setup NA Newcomer Browser Marker CodeLab Service Several Yes Yes Yes Yes Fair Both Browser CodeSaw Literatu re related Several Low No No In setup Easy Both Windows NA: information Not Available or depends on the plugin The table contains only some properties and more detailed information about the environments is given in Chapter 4. 6 CONCLUSION We have reviewed a set of environments that can be used in learning programming skills. An optimal environment would support problem solution, algorithm design, data structure design and finally coding in a selected programming language. The environments support mostly learning the syntax of the selected language. The syntax becomes important as soon as the semantics and the pragmatics of the language are known. In general, program flow contains three control structures: a sequential flow, a conditional structure, and a loop structure. Now the semantics and the pragmatics are somewhat independent of the programming language and thus, they can be learned with simpler flowchart or pseudo code presentations of the program flow. The classification presented in this study helps the instructor to select an appropriate system matching her requirements. The classification was established based on the 29

30 systems used in teaching basic programming in several university level courses around the world. A good solution for a learning system would provide the following properties: Support for a newcomer in programming o Problem solving, algorithm design, data structure design o Learning syntax, semantics and pragmatics of the language Enhancing understanding of the program flow Partial compiling for fast checking of a certain operation Administrative properties to support the instructor Communication properties for team-work None of systems reviewed fulfilled the requirements presented. A combination of several systems would fulfill the requirements but the whole system would be too complicated to operate and to learn. ACKNOWLEDGEMENTS The authors would like to thank the Eastern Finland Virtual University (ISVY), for financial support. REFERENCES [1] Cooper, S., Dann, W., Pausch, R.: Alice: A 3-D Tool For Introductory Programming Concepts, available from < [retrieved ]. [2] Jenkins, P.: The Motivation of Students of Programming, Proceedings of the 6 th annual conference on Innovation and technology in computer science education, Canterbury, United Kingdom, 2001, pp [3] Storey, M-A. & al: Adopting GILD: An Integrated Learning and Development Environment for Programming, 2003, [e-document], from < [retrieved ]. [4] Silkelä, R.: Mielekäs ja merkityksellinen verkko-opiskeluympäristö, University of Joensuu Bulletins of Educational Technology Centre 5, Joensuu, Finland, 2003, pp [5] Crews, T., Ziegler, U.: The Flowchart Interpreter for Introductory Programming Courses, FIE 1998, Tempe, Arizona, [e-document] available from < [retrieved ]. [6] BlueJ, 30

31 [7] Truong, N., Bancroft, P., Roe, P.: ELP A Web Environment for Learning to Program, Proceeding of ASCILITE2002, Auckland, New Zealand. [e-document] available from < [retrieved ]. [8] Widera, M.: Testing Scheme Programming Assignments Automatically, proceedings of TFP 2003, [e-document], available from < [retrieved ]. [9] Pisan, Y. & al: Submit! A Web-Based System for Automatic Program Critiquing, Proceedings of ACE2003, Adelaide, South Australia, pp [10] Testimonials for TuringsCrafts CodeLab, [www-document], available from < [retrieved ]. [11] Luck, M., Joy, M.: A Secure On-line Submission System, Software Practice and Experience, 29, pp , [12] Voracek, J.: Different Aspects of Virtual Education: Personal Experience and Conclusions, University of Joensuu Bulletins of the Educational Technology Centre 5, Joensuu, Finland, 2003, pp [13] Wikipedia the free encyclopedia: Integrated Development Environment, available from: < [retrieved ]. [14] Kölling, M., & al: The BlueJ system and its pedagogy, Journal of Computer Science Education, Special issue on Learning and Teaching Object Technology, Vol 13, No 4, Dec 2003, < [15] Brusilovsky, P. & al: Mini-languages: A Way to Learn Programming Principles. Education and Information Technologies 2(1), pp , [www-document], available from < [retrieved ]. [16] Storey, P., Gild: An Integrated Learning and Development plug-in for Eclipse, 2003, [e-document], available from: < [retrieved ]. [17] Jones, E.: Grading student programs a software testing approach, Proceedings of the fourteenth annual consortium on Small Colleges South-eastern conference, Virginia, United States, 2000, pp [18] The BlueJ Submitter Extension, [www-document], available from < [retrieved ]. 31

32 [19] University of Kuopio,: mikroluokkien ohjelmistot (software listing of computer classes), [www-document], available from < [retrieved ]. [20] University of Joensuu, Homepage of course Object-Oriented Programming, [www-document], available from < [retrieved ]. [21] DrJava, [22] Eclipse, [23] Object Technology International, Inc., Eclipse Platform Technical Overview, 2003, [e-document], available from < [retrieved ]. [24] Storey, M-A. & al: Improving the Usability of Eclipse for Novice Programmers, [e-document], available from < [retrieved ]. [25] Eclipse plugins: plugin overview, [www-document], available from < [retrieved ]. [26] IBM: WebSphere software platform IBM WebSphere and Eclipse, [wwwdocument], available from < 3.ibm.com/software/info1/websphere/index.jsp?tab=eclipse/about&S_TACT=10 3BGW01&S_CMP=campaign>, [retrieved ]. [27] Hartmann, W., Nievergelt, J., Reichert, R.: Kara, finite state machines, and the case for programming as part of general education, Proceedings of the 2001 IEEE Symposia on Human Centric Computing Languages and Environments, Stresa, Italy, pp [28] Jeliot Interactive Algorithm Theater, 1997, [www-document], available from < [retrieved ]. [29] Jeliot, [30] Ben-Bassat, R.: Jeliot 2000 User s guide, [e-document], available from < [retrieved ]. [31] Kara, [32] Viope Solutions, [33] Jani Rönkkönen, M.Sc.(Techn), private communication, [34] Gild, 32

33 [35] from Mr. Paul Roe, [36] BOSS, [37] CourseMarker, [38] CodeLab, [39] CodeSaw, 33

34 APPENDIX A This appendix shows a list of tables, including links to additional information on systems described in this document, as well as some of those left out. Even if the system was described, it is recommendable to check the documents or homepages where the information has been taken from to get a better picture. Some of the links point to a homepage of the system, others to documents such as reports of or introductions to a system. The locations have all been present at the 8 th of December 2003 through HTTP. Table 1. Links to information on IDEs. System Described More information BlueJ Yes DrJava Yes DrScheme No Eclipse Yes Table 2. Links to information on visualisation systems. System Described More information Jeliot Yes Alice No Kara Yes Karel No RobotWar3 No D AlgoArena No icle002.html CeeBot No Sivil No delivery.acm.org/ /380000/378678/p74- masterson.pdf?key1=378678&key2= &coll= GUIDE&dl=ACM&CFID= &CFTOKEN= Table 3. Links to information on virtual learning environments. System Described More information Viope Yes GILD Yes gild.cs.uvic.ca ELP Yes df ELM-ART No trier.de:8000/projects/elm/papers/um97- WEBER.html 34

35 Table 4. Links to information on submitting/testing systems. System Describ More information ed BOSS Yes CourseMarker Yes Ceilidh No NESS No The CUCS No Online Submission System TRY system No ugweb.cs.ualberta.ca/try/ Praktomat No Expect No expect.nist.gov/ DejaGNU No Table 5. Links to information on uncategorized systems. System Described More information An Online No crpit.com/confpapers/crpitv20roberts.pdf Programming Assesment Tool CodeSaw Yes CodeLab Yes Visual Paradigm No WebEX No www2.sis.pitt.edu/~peterb/papers/webnet01.ht ml 35