Open Source Learning Management System (OSS)

Transcription

1 DEPLOYING MISSION CRITICAL LMS USING OPEN SOURCE SOFTWARE Leigh Jin San Francisco State University OMG Standards for FOSS Governance Workshop Santa Clara, California December 11, 2013

2 OUTLINE Introduction Research Questions Methodology Research Findings Discussions and Conclusions

3 INTRODUCTION Open Source Learning Management System an attractive alternative: as institutions of higher education confronting budget cuts and financial crisis Low cost (+): no licensing fee Flexibility (+): open source code means customizability Deploying and maintenance cost (-) Hardware People with Technical Skills

4 INTRODUCTION: NACUBA SURVEY NACUBA survey (2004) finds members perceive open source software as a viable alternative to commercial software due to: Open standards and interoperability with other application systems (61%) Freedom to modify the code (58%) Software designed by and for the industry (58%) Lower cost of ownership (55%) NACUBA refers to National Association of College and University Business Officers

5 OSS DEVELOPMENT VS. IMPLEMENTATION OSS Development: two assumptions: Developers are users with itch to scratch OSS movement is unique because of the way software is being developed OSS Implementation: Fitzgerald and Kenny (2003) study OSS implementation in an Ireland hospital leads to cost savings of 13 million over five years Free access to source code is not the primary motivation for adoption

6 INTRODUCTION OSS IMPLEMENTATION: BENEFITS VS. BARRIERS: OSS Implementation Benefits Low licensing cost Open standard and interoperability, avoid vender lock-in Freedom to modify the code Greater choice and control for end-users Innovation Barriers Lack of documentation and support service from the OSS provider Technical skills required to implement, customize, and maintain the software Steep learning curve and end user training and support

7 CASE STUDY RESEARCH QUESTIONS How OSS LMS is implemented at an University in California The motivation, benefits, and challenges associated with adopting open source LMS in the public education sector How its impacts are perceived by the stakeholders?

8 INTRODUCTION: IMOODLE IMPLEMENTATION FRAMEWORK OSS LMS Community Frequent releases on weekly/monthly basis Global audience: sites from 203 countries 186 Moodle developers (with write access) from all over the world University Deployment New release per semester Designed specifically to suit the local needs for a University in California Developed and supported by Academic Technology Group within the university A Course Using imoodle Contents are updated throughout the semester Features of imoodle are selectively chosen to meet the pedagogy needs for a specific class Designed and managed by the instructor of the class

9 OSS IMPLEMENTATION: STAKEHOLDER VIEWS Technologists Educators Administration Ideology towards Common Good Software Development Methodology Security & Risk Management Adoption of Innovation Constructivist Learning Environments Online Pedagogy Commercial LMS Limitations Cost Saving Strategic Benefits User Satisfaction Total Cost of Ownership Easy to Use Policy

10 METHODOLOGY Interpretive Case Study Participant Observation Longitudinal Study Interviews with Administrators IT Staffs Faculty Focus Group study with 40 students (college of business)

11 IMOODLE IMPLEMENTATION MOTIVATION: Limitations of Commercial LMS years experience with Blackboard Earliest adopters of version 6 on 2002 We had a class that went over 1000 students Blackboard forced us to break a class into two sections We ended up having blackouts, we had 13 hours outage during the finals week we find that they are not on our timeline they said we would fix it in six months, when we release that new version. But for us, we need the answer now Officially move to Moodle on 2007, currently having 26,000 (out of 30,000) students uses imoodle (customized Moodle) in their classes.

12 IMOODLE INITIAL IMPLEMENTATION: SCALABILITY AND STABILITY ISSUES We have some major slow time, where people would login and it would take a minute for the page to load We found that there are a number of issues that cause this, part of it was some of the ways that people were using the tool, part of it was how it was configured, part of it was our hardware but when you have 1,500 students in one class, with the same deadline of midnight on Wednesday, and they are all taking a quiz, we would see 16,000 quiz attempts in a 12 hours period, nobody else in the whole [Moodle] community has seen this.

13 ADMINISTRATOR S TAKE ON OSS So I didn t have anything to do with that decision. I came from a really structured project environment for a proprietary software There wasn t a project plan that I m used to seeing, there wasn t a scope of the project What is this leap of faith into this open source technology without this documentation? That scared me, I m still scared. And I don t know if I would have made the decision to go open source given the situation that I came into.

14 UPGRADING IS AN ISSUE The tipping point I think was 1.8 based on the manpower that we have and the resources, we can't test this out thoroughly. But the [manager] really wanted to upgrade to 1.8, and it was just a nightmare The automated process would not complete, we were manually downloading the files for each course, manually takes a 1.5 courses and make it become 1.8 compatible and uploaded into 1.8 and restore it There were thousands of courses there. And our team stayed until 11pm.

15 WHEN TO UPGRADE? First of all there were no procedures in place to define what should be the process of maintaining and upgrading these systems Everybody said, well, this is open source and when an open-source bug is fixed we implement it and we keep going But you have to follow the natural timeline for your business processes, which in this case is the academic process. The academic process has a beginning of the academic year and an ending of the academic year... So every academic year we would do a major change. Minor changes like bug fixes we would allow if it s something critical, but we wouldn t go from a Moodle, let s say, 1.84 to 1.92 or something. Only on an annual basis, where there would be major changes.

16 ISSUE OF TESTING They had no testing, but for ongoing work you have to decide when you are going to implement changes, test them and then put them into production. So you have to distinguish between development, testing and production So our second task was to instill a process for upgrades and testing and we hired student assistants to write test case scenarios.

17 ISSUE OF VIRTUALIZATION You ve got to remember, we ve got Linux upgrades, we ve got Apache upgrades, PHP upgrade, and on top of that you ve got the MySQL. So all four components plus Moodle are changing. It's a lot of different pieces that have to be synchronized So what we would do is to create a virtual system which consisted of the Linux operating system, that was the platform, Apache, PHP and MySQL of a particular version, married to a particular Moodle version with its content database. And we would create a new virtual system for every new semester. So we would do this on an annual basis Once the concept of virtual machine is accepted, then it becomes a much more stable environment and also a scalable environment

18 PROJECT MANAGEMENT SKILLS IS NECESSARY It doesn t matter whether it s open-source or private or in an academic institution, it has to be treated like any other piece of software that goes into production And these people don t know the ABCs of project management. That s the problem, they grow up as developers, from developers they became sort of supervisors and they don t have any skills in project management Open source gets hurt more because they don t have the discipline of project management I took what we taught in project management class and implemented it step-by-step. This is what had to be done

19 THE MOODLE COMMUNITY The Moodle core team members, it s a very interesting mix of younger developers who always want to just try to implement new features without thinking about usability, without thinking about scalability or how easy it can be maintained Also, it is very different from Linux kernel, where most of the developers really know the code and also they are the users of the code. The Moodle community is different in the way that most of the users might be teachers and students and they might not know all the technical details So whoever will be paying the Moodle core team a lot of money and say, hey, you do this, they would just do it. And after they do it, most likely the newly written code will be added into the core code and got released despite that it might not be really helpful for most of the university

20 CONTRIBUTING BACK TO MOODLE COMMUNITY The way that I have kind of established my own voice is to volunteer a lot in the community. Either help out with testing or with documentation or with reports and things like that We have taken ownership of certain modules or blocks as they are called. So now we maintain the particular QuickMail and Gradebook Plus for the whole community. So when an upgrade happens, we test them to make sure they works with the upgrade and make any changes necessary and then release them back to the community

21 ESTABLISHING RELATIONSHIP WITH THE MOODLE CORE TEAM Because of our size, right now I think we have 26,000 out of the 30,000 students use it for one or more classes, some of the things that we ve been experiencing that other campuses have not We ve been in communication with the core team: here are some issues with scalability that, when [an university] gets big, they are going to face these things too so it s to your advantage to kind of learn from us and make the fixes They already recognized us as [a national leader]. They ve had me be the keynote speaker last year. We co-organized the Moodle conference this year. And then people call me from around the country to ask questions. And some from Canada. I ve had one from Israel. Santa Barbara. All over the place. In 2008 Moodle conference, the core team actually came to our campus to visit every one of us, and that s because they recognized us.

22 CONTRIBUTIONS TO EDUCATION COMMUNITY When I consult with different campuses, I give them strategies for finding answers: find a small group of local campuses that are either a little bit ahead of you or at the same place [in Moodle implementation] and get their phone numbers. So I said, I can be one of those phone numbers We have the Quick Guides project which is how-to documents for people, to learn how to use imoodle. And because of this consortium with other Moodle campuses, we modified the quick guides tool to allow multiple campuses to use the same information We worked with [some community colleges], so we gave them all of our training materials and our Moodle course for instructors which has lots of pedagogical information and technical how-tos And then a month later they said we made podcasts from all of your documents would you like those?

23 BENEFITS TO CAMPUS Revenue Generation: We could be hired by other campuses with zero programmers to make a widget. We can provide help desk for a campus that has no help desk. We can bring money in through support, through development or even through consulting. We can start charging for me to go to campuses or something. Curriculum Innovation: we have been able to encourage many more faculty to move beyond just storing their documents for students to download. Now they are engaging students in different types of activities like discussion forums or self-assessment quizzes or chats or wikis Administrative Use: we have over 250 organizations and committees using imoodle for non-coursework. We have HR who wants to use it for all their training sessions. I m trying to push the defensive driver training because I don t want to go to that room for three hours... Other OSS Projects on Campus: I think the campus has gained a whole lot more in terms of those commonalities that we have across projects than just paying a fee to some service provider to provide us with a service.

24 TECHNOLOGIST: WHY WE NEED TO CONTRIBUTE BACK Because the more you customize, the next time you want to upgrade, when you re into a new version of Moodle, you can t just upgrade to what the new Moodle community says is Moodle. You ve got all these dependencies and customizations that you have to make sure they work. Now you have to recustomize all of these things.

25 ADMINISTRATOR S TAKE ON CONTRIBUTION OUTSIDE OF CAMPUS In the early days, I participated in Moodle forum for outreach reasons, we encountered a lot of stability problems and we really need help. So it started for work purposes, not for personal gain. But later on, people noticed that I gained too much visibility in Moodle community, so now we had a policy that doesn t allow forum participation during office hours, so I have to use my spare time to do it instead. Speaking of revenue, one of our staff members is extremely generous and was making a lot of visits to other campuses to spread the word of Moodle. And you can put a price on that. At $300 a day it looks like you just gave away $4000 worth of work. And some of our needs on our own campus are not being met. So I said, we have to stop that, and if they want you to come then we will set up a costing structure for what we will charge internally for consultations and what we charge externally.

26 STAFF RETENTION ISSUE We lost seven people in seven weeks this summer. And the other manager and I, who oversees this group, we talked about having posttraumatic stress disorder because when anybody would ask to meet with us privately, we were afraid that they were going to resign I mean, one person that left was making $27,000 a year when I came in. And I got him up to $50,000 but then he still left for a $72,000 a year job which is what his market rate is. He deserves to make $70,000. I think that a lot of the people who left had a certain loyalty to [the manager who left]. And I think that is interesting to see the wave that is still hanging on that I'm really fighting to keep are a wave that came in with [the new manager]. So I think there is a certain loyalty there.

27 OPEN SOURCE LMS: MOTIVATIONS AND BENEFITS Flexibility and ability to customize our own implementation is important Usability and Accessibility Technical innovation and rich features Pedagogical innovation and sharing I have seen just as many contributions to the moodle community pedagogically derived, so people say this is how I am teaching math with this tool, nobody asks them to supply that, they just happy to share. Ideology and Value Leadership role and Reputation

28 CHALLENGES Technical skills and know-how Technical support related issues Upgrades Scalability Testing Staff Retention User Training and Education Unrealistic expectations from users Concerns people have are not only cost related issues, but the safety net, if something goes wrong, they want to yell at somebody else, now when something goes wrong, they yell at me

29 COST OF OWNERSHIP Cost-: Free licensing Cost+: Pay for hosting and other costs associated with customization and support Cost-: Engaging other local campus who are interested in adopting Moodle, establishing collaborative relationships Cost-, Revenue+: developing expertise surrounding Moodle support, potential consulting and revenue generating opportunities

30 ACTIVELY PARTICIPATING IN MOODLE COMMUNITY At developer level Contributing and maintaining the quickmail block Maintaining tools like gradebook+ in the previous release At system administrator level Communicating with core teams regarding scalability issues Training, support and documentation level Making contributions to forum discussions and documentation

31 FROM MOODLE TO IMOODLE Initially, upgrades following Moodle release cycle caused stability and scalability issues Measures taken to solve the problem: Carefully evaluate features of Moodle and customize it to meet needs of a specific campus Establish testing environment, thoroughly test new build before releasing to the campus Adjust the upgrade cycle of imoodle to match the academic semester Virtualization, instead of attempting to upgrade all courses across all semesters, isolating upgrades within each semester More effective project management, focusing resources to stabilize and support the system As the result, the current imoodle implementation has been quite stable, no major outages and disruptions

32 FROM IMOODLE TO THE CLASSROOM THE FACULTY PERSPECTIVE imoodle is very effective and easy to set up comparing to Blackboard Faculty computer skills vary across the campus Faculty don t have time to attend training sessions System stability is really important Do we really need new upgrade every semester? Online course is a whole different beast to support: my department benefit a lot when I get 1300 students in my online class, but AT people are not paid for supporting me. I think online course should be treated differently because in this case, the technology is the classroom

33 FROM IMOODLE TO CLASSROOM: STUDENTS PERSPECTIVE imoodle is easy and intuitive to learn and use How effective imoodle is really depends on how instructor engage and set up the course Biggest complaint: instructors do not access students work on time The instructor uses imoodle actively tend to be better instructors, students tend to perform better in the class Students do expect instructor to be the problem solver when they encounter technical problems If a feature is available in imoodle, students do expect instructors to use them Feature Requests: Deadline for instructor update grades? Automatic notification and reminding s before homework due Who is logged in now feature, dynamically forming work groups

34 POLICIES AND GUIDELINES Service Level Agreements Let campus know this is what we can provide now with our limited staff and what we can do, and then this is what we ask you to do to help us out. Incentives for faculty to make effort in curriculum innovation: Tenure and Promotion policy Providing venues for faculty to learn and share best practices of using technology effectively in the classroom Providing effective and fair funding mechanism to support technology innovation