PLAYING WITH CLOUDS: MAKING WEB APPLICATION ASSESSMENTS MORE REALISTIC Bruce Scharlau University of Aberdeen Computing Science University of Aberdeen, Aberdeen AB24 3UE b.scharlau@abdn.ac.uk http://www.csd.abdn.ac.uk~/bcharla ABSTRACT This paper describes the reasoning behind introducing more of a problem-based learning approach to the delivery of an advanced computing module, and the lessons learned from this change. In addition a key component of the change was the use of Amazon Web Services and their cloud computing tools, which allow for more adventurous applications to be developed by the students during the term. The basic lesson is open up the assessment to let the students imagination guide their work, and to have a plan b ready for when plan a does not go as expected due to technology issues, or other external factors. Keywords Assessment, cloud computing, problem-based-learning. 1. INTRODUCTION CS5302: Enterprise Programming at the University of Aberdeen is the core module of an advanced MSc computing science degree, which all students on the programme take. This module aims to provide them with the basic knowledge of enterprise level Java web applications that will see them through the programme including what they will need for their group and individual projects. The module includes the nuts and bolts of Java web applications, plus the theoretical underpinnings for building such applications. This is reflected in the 50/50 weighting between the coursework and the exam. Originally the module had two assessments of the build a web application and the build additional components to support or use the web application variety. This produced lots of work to be marked, and mostly unexciting web sites, although some were quite creative and inventive. Nor did these assessments address the deeper issues of building such sites, which would make the students think more fully about the issues around large complex web applications. Three years ago the approach to the coursework was changed to present more of a challenge to the students. Now the coursework harnesses the imagination of the students. It also more efficiently pulls together the theory and the nuts and bolts of larger web application development. The course is now more professional, and aids development of graduate skills. 2. THE OLD APPROACH WAS PROBLEMATIC The coursework in CS5302 is worth 50% of the total course mark, which reflects its emphasis on programming. This was formerly broken down into two submissions. The first assessment was a build a web site, which was due half-way through the term. The second assessment was build a mobile front-end using Java ME for the site plus integrate appropriate web services for the site, which was due at the end of term. Both components had required technical details, but were generally open to interpretation to cover a wide variety of sites. However, the results were generally unsatisfactory. This worked fine for a number of years as the students worked to understand the concepts and learn from the examples, but had become less challenging as an increasing number of students started the programme knowing this material already. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. 2011 Higher Education Academy Subject Centre for Information and Computer Sciences
First, students would put minimal effort into the content of their sites. This meant e-commerce sites with ten items for sale and not much effort was required for the organization of the content on the web pages. Small sites mean most of the work was trivial. Search, browsing, navigation and pagination are easy when there is not much content to manage. Lists of items were retrieved from the database with select * from tablename statements. This type of SQL statement was safe with no repercussions as so few items were in the database that the returned result would not break the application. Similarly, most students managed to code up the Java classes managing the models, controllers and views without too much trouble, and made good use of the basic design pattern. Second, the students had a hard time seeing any benefit in using more professional software development approaches that will be expected of them after graduation such as testing, iteration planning, source control, and similar processes. According to them it was all too short of a timescale for these tasks. The level of work required for the assessment meant that there was no immediate payoff for the students to deploy professional approaches. Third, there was a short window in which the first assessment needed to be marked and returned in order to be helpful as an input into the second assessment. The students already had a good idea of how well they did on the coursework, so the value to them was less than the effort involved in the marking. Yes, they liked to have the official comments and feedback on their coursework, but also realized that informal feedback, which they had already received during practical sessions was just as useful for them to continue on with their second assessment. 3. ENTER THE NEW APPROACH Three years ago the approach to the coursework was changed. The coursework became more unified in an effort to introduce more of the skills students will need to use as software developers after graduation. Instead of two assessments, there is now one large assessment due at the end of term. Now the students put more effort into their web sites, they see the benefit of source control, iteration planning, and some limited testing while doing the work. Instead of doing this individually, the students also have the option of submitting the coursework as a pair of students. If they do this as a pair, then both of them sign a form agreeing on how much each of them contributed to the whole. Usually they claim to have done equal amounts of work. Most students now do this as a pair, while some prefer to work on their own. Whether someone works on a team or individually does not seem to have a bearing on their coursework mark, and both teams and individuals have submitted exceptional pieces of work. The coursework pulls together the theory and the practicalities of larger web application development so that it is more professional, and aids development of graduate skills. The aim is to have the students do what they would be doing if they were building this application in a small development company. This means familiarizing them with the nuts and bolts of everyday development. With the changed coursework the students need to deal with navigation, pagination, search strategies and content planning if the site is to work correctly. The students need to start this assessment early and continue to develop it as they progress through the term. The new coursework is moving towards a problem-based learning (PBL), or enquiry-based learning approach, as it is also known. (Schanck, 2005; O Kelly, 2005; Kahn and O Rourke, 2004) PBL aims to let the students choose how to tackle the problem and learn what is needed for the solution. This assignment differs from a pure PBL approach in that the course still has most of the content set. However, the course does not proscribe a set solution for the students. The students can choose the solution they prefer to resolve the problems of their particular scenario. Indeed, there are little, but useful things, which are not covered in detail, such as shopping carts, and pagination, which are mentioned, but for which they need to find a solution that works with their application themselves. This goal was to remove as much of the proscriptive details of the assignment as possible. This allows more room for freedom of choice by the students to build something that they find personally challenging within a larger framework of options. They should feel free to let their imagination roam and create imaginative applications. If they are inspired by their application, then they will put more effort into it too. 4. PLAYING WITH CLOUDS A centrepiece of this change has been using the tools and components of Amazon s web services at http://aws.amazon.com/ to build a finished application required to have 10,000 or more items in the database. The scale of the application forces students to contend with issues required in real-world applications. Many
tend to find novel and useful solutions to the needs of their application. The students are guided in supervision meetings during practical sessions where their plans and problems are discussed and evaluated. Use of Amazon s web services (AWS) computing cloud is only possible through grants which they make available to academics for use in teaching. AWS has no free sandbox for developers to use. Yes, you can make calls using REST and SOAP to retrieve data from Amazon s product catalogue for free, but if you want to start up your own instances running in the cloud, and deploy your application there, then you need to pay; and the best way for academics to do this is to obtain a grant through the Amazon Web Service education office (http://aws.amazon.com/education/). The prices for the hourly rates are low for each of the services, but the grant provides cushion for the time a student forgets to shut down an instance and the fates prevent anyone from checking that all instances have been stopped. AWS provides storage for objects in its Simples Storage Service (S3) service, remote MySQL instances with Relational Database Service (RDS), message queue systems with Simple Queue Service (SQS), and a basic server instance running Windows, Linux, or Unix, and which you can install what you need, with its Elastic Compute Cloud (EC2). You can just run your web server and database inside an EC2 instance, or separate them out to use EC2 with an RDS instance for more stability, and then back up your application to S3 so that you can replicate it again next time you need it, or if you wanted to start up multiple instances. The new assessment allows the students to run their application inside of Amazon s cloud. While they are encouraged to do so, and a range of tools can make it trivial or not so trivial to do so, they do not have to deploy their application into AWS. Setting up the tools and fighting with the proxy, as well as possible problems with other components of the assessment mean that some students do not deploy their application into the cloud. Most students take advantage of this chance to use it and show how they pulled the components together for their application. 5. THE NEW ASSESSMENT The assessment is conceived of as a mini-project that the students can do either individually or in pairs. They are told about the details in the first class and that it is due on the last day of class. As noted earlier there are three parts to complete prior to the final submission. Each of the three stages has a part describing what is intended for that stage, which is due a week later, and the application components detailed in the description, which are due about a fortnight later. The students are not obliged to follow the description of what they intend to do blindly. They are free to submit something different if they find that their intended approach would not work, but that they needed to pursue an alternative instead. The new assessment is flexible in interpretation for most parts and comes down to a few simple required parts: an application providing a transaction for users, who can either register for a service, or purchase a product from a choice of at least 10,000 items, which they can find through browsing or searching through the site. In addition there are a few extras, which would be desirable such as using the Spring Framework, deploying the application in the AWS cloud, and enabling and consuming web services, or enabling suitable mobile browsing of your application. Most students achieve at least several of the extras, while two or three achieve them all. As these are not design students, we do not require an aesthetically pleasing site, only that it provides complete and consistent navigation. 6. ITERATIVE AND INCREMENTAL DELIVERY The students use iterative delivery during the term. This ties in with three stages during the term to ensure that the students know where they should be for each stage of development. These are to (a) gather data and build a prototype, (b) deploy to Amazon web services, and (c) optimise the application for mobile users, and resolve potential bottlenecks in scalability. The final deadline is a fortnight after the third stage is due. Building the site using multiple iterations has a number of benefits for both staff and students. The process ensures that the students have something to submit for their final deadline. Each additional iteration adds new functionality. Therefore the students will always have something that works which can be submitted for assessment. In addition, this is similar to the working patterns in software houses, so understanding the process adds to their professional skills. The staff have benefited from this approach by having a number of places where they can intervene to point out problems that might occur if the team follows their proposed path, and suggest alternative solutioins. It is also useful for staff to see the application develop over time, as this provides a number of teaching points about the student s design decisions, and the implication that arise from their design. The result is that the students are more likely to have a successful application. In addition, instructors are more familiar with the application at marking time, which means that the whole application will be seen more clearly and better understood.
7. PROVIDE GUIDES AND ROUTES The lectures and practical exercises align along the functionality required for the coursework assessment. The lectures provide the context for the work, while the practicals provide details for a specific scenario. Between the lectures and the practicals the students need to see how they can accomplish the what of the assessment. They need to see how the 10,000 records in the database requirement can be achieved using RESTful web services, and why this is not as difficult as it might appear. Similarly, using frameworks to build applications is shown to be useful and less difficult than it appears. The lectures cover the distributed web application concepts and discuss solutions while pointing out the background. Alternative solutions are illustrated too, and differences between them discussed. The lectures provide the context for their coursework and guide them to why something is useful, and what they need to know in order to implement the solution. The practicals allow the students to practice solutions in isolation, before integrating the components into their own application. Implementation details of alternatives can be provided here too, so that they are easy for the students to use and try as needed. Alternative solutions are needed because the preferred solution might be unworkable due to lack of documentation on a combination of frameworks or APIs. Sometimes the solution planned for inclusion this term is discovered to be impractical in the time you have available. When this happens students need to be pointed to an alternative solution that does work, and provided with a working example for them to use. Working with technology such as AWS means that you might have new possibilities each week. This means the practical used last year will need updating, and might even be redundant as new solutions are possible. For example, last year RDS was available the week we were to explore configuring MySQL in an EC2 instance. Instead of doing that then, we could use RDS. This required a re-write of the practical and lectures. Yes, it is fun and exciting to use the bleeding edge, but be aware that it takes time and you will need to know the work arounds available to use proxy servers in the labs, which are usually only found in the fine print of the documentation. 8. MONITOR PROGRESS The deadlines for the iterations are all soft deadlines. It does not matter if the students miss them. The final deadline at the end of term is the one that matters. Instead the deadlines provide a way for students to gauge whether or not they are on schedule to meet the final deadline. In addition staff discuss progress with each team or student during the weekly practical sessions. This provides another chance to uncover issues the students might have encountered, and recommend alternatives if required. Sometimes it is necessary to suggest alternative solutions as the path the students have chosen will not work as they expect. This needs to be done so that the students can argue their position carefully and critically, while the alternatives are laid out clearly, and they are left to find that they made a poor decision. They should always have the final choice, but be made aware of fallback positions, which can mean they still submit suitable coursework. Monitoring progress also means that those students who are reluctant to ask for help, still receive help. There is at least one student like this each year, who is behind schedule, but does not ask for help, until spoken to by someone else about their progress. It is important not to let these students be overlooked. They need to be directed to workable solutions that will mean they submit something for their coursework. Problems faced by one group might also be indicative of what the other teams will face, as usually one team is ahead of the others, so this also provides an early warning system. Alternatives can then be found and brought to the attention of the other students. Therefore, a plan b is always necessary for this approach as sometimes the tools and processes do not work as expected. The fun new tool may not work as expected so you need to have a fallback of what was done the previous year available and working as an alternative. The other possibility is that while your course is progressing, other courses are causing difficulties for the students. The symptom appears when everyone is having difficulty achieving the desired functionality required for better assessments by the deadline, because they are busy working on competing assessments for other courses. There are several solutions to this problem. First, where possible, reduce this risk at the start of term by agreeing with the other course organisers that the students can use the same basic assessment for both courses. While we have not introduced synoptic assessments (SA) and synoptic learning per se, we do allow for a loosely linked version of SA. (Kyaw and Drummond, 2007) For example if one course is about web applications, and another covers security for web applications, then the security components can be assessed in that course, while the general web application is assessed in the other. This allows students to spend their time working on both simultaneously to a large extent. They can also use what they learn in one course in another course quite openly and not have to build
similar applications twice. This is something, which we should look at more closely in the future as a means to reduce the work required of the students. Second, if aligning the assessment with what is needed in other courses does not work, then the rubric, or marking scheme may need to be changed. This could be due to students having problems completing parts of the assessment due to issues with the components, or due to work in other courses causing problems. In any case, be aware that the rubric may need to be changed due to external events beyond your control. This happens and you need to have plans in place to accommodate these eventualities. 9. CONCLUSION This approach works and will be continued in the future. This approach pulls together a real-world approach to coursework, while pushing the students to stretch the boundaries of what they have done previously. They leave as changed students more aware of what they can do with their abilities, and how to resolve technical and design problems that will be useful in other courses as well as after they graduate. All students find CS5302: Enterprise Programming difficult and challenging, but they also think it is one of the best courses on the programme. 10. REFERENCES [1] Kahn, P and O Rourke, K (2004) Guide to Enquiry-Based Learning, University of Manchester) [2] Kyaw P., and Drummond S., (2007) Synoptic learning and assessment: An experience report, in the 8th Annual Conference of the Higher Education Academy, University of Southampton, 28th - 30th August 2007. [3] O Kelley (2005) Designing A Hybrid Problem-Based Learning(Pbl) Course: A Case Study Of First Year Computer Science in Nui Maynooth, NUI, Maynooth [4] Schank, Roger (2005) Lessons in Learning, e-learning and Training. Pfeiffer.