Quantitative Risk Assessment in Moodle Learning Management System

Transcription

1 , pp Quantitative Risk Assessment in Moodle Learning Management System HyunChul Joh Dept. of Computer Eng., College of IT Convergence, Kyungil Univ., Korea Abstract For the past decade, in many educational institutes, learning management systems have become essential parts to deliver class materials and to provide communicational channels between course instructors and students. When institutes are adopting a learning management system, perhaps the two most concerning points would be how much a system costs to deploy and how secure a system is. Moodle, which is the most popular open software learning management system, satisfies a budget-related issue. Thus, as a low-cost solution, educational institutes often adopt Moodle. However, there have been few studies investigating the security aspects of Moodle, which might be more important than a budget problem. Here, we quantitatively investigate how secure Moodle system is. First, vulnerabilities discovered in Moodle are speculated with respect to the CVSS score. Then we apply a well-known vulnerability discovery model into the vulnerability discovery process. Also, we investigate whether there are seasonal variations in the discovery process. The result shows that a reasonably modified datasets are well followed the discovery model, and there is indeed a seasonal pattern in the Moodle vulnerability dataset. Keywords: LMS, Moodle, vulnerability discovery model, CVSS, seasonality 1. Introduction Nowadays, an online learning management system (LMS) is considered as a musthave item for any educational institutes, regardless of their sizes. They are using LMSes for various purposes such as delivering course materials, tracking students activities, assigning homework, announcing course notifications, etc. In the LMS market, currently, hundreds of LMS software systems are available ( Among those, there are some of the top players, such as Moodle ( Blackboard ( or Sakai ( According to Research and Markets [14], the LMS market will grow to $7.83 billion in More and more institutes are getting intensified to depend on LMS with digitized class materials and students private class information. Needless to say, the LMS market has even more prosperous future. When schools are planning to adopt optimal LMS software for their educational environments, the two most concerning points are budge and security. Based on the number of users, the most popular LMS is Moodle ( and that is because Moodle software system is open source software, which means users are able to study, modify and distribute Moodle to anyone and for any purpose [16]. However, at the same time, many security vulnerabilities have been found in Moodle continually from its first release. Consequently, schools mull over tradeoff between budge and worries about probabilities of security compromises. Although only with the number of known vulnerabilities does not represent the entire risk factors, a LMS having more security vulnerabilities has higher possibilities of occurring security breaches than that of having less number of ISSN: IJSEIA Copyright c 2015 SERSC

2 vulnerabilities in general. If institutes do not pay attention to the possible risks caused by careless management of LMS, they will pay tremendous costs to compensate their damaged reputations. As a result, it is getting more obvious that we need to investigate the security perspective of LMS. Lord Kelvin, who is widely known for realizing that there was a lower limit to temperature, which is absolute zero, once said that If you cannot measure it, then you cannot improve it, and in our case, the it is the software security, which could be understood in a degree of how much a system is free from the vulnerabilities. A software vulnerability can be defined as software defects or weaknesses in the security system which might be exploited by malicious users causing loss or harm [13]. If we can measure the software security, then we are able to compare the risks among the software systems. And it will assist for optimal software deployment, patch management and risk remediation in advance. There could be largely two kinds of risk analyses: one is qualitative and the other is quantitative. A qualitative analysis is heavily depending on the experts opinions which tend to be subjective by its nature. Hence, some people [8] compare it, as an Art which does not need to depend on predefined definitions. On the other hands, a quantitative analysis is depending on actual data-driven empirical studies, and usually followed by statistical tests. To conduct the quantitative analysis, researchers first need to collect datasets. People [8] compare the quantitative analysis to Science which most of the time depends on the predefined definitions. Our study is more like a quantitative analysis since we need to collect the datasets, and apply the datasets into the predefined formula. These days, fortunately, vulnerability datasets have been accumulated enough to be analyzed for meaningful results, and researchers are able to collect the datasets from publically available vulnerability databases on the Web, such as NVD ( or OSVDB ( Usually, vulnerability information in the Web based databases is overlapped and complement each other, so there is no the best vulnerability database. Usually the databases provide CVE ( identifiers, which are the identification numbers for the known vulnerabilities, CVSS scores, and vulnerability published dates. If it is available, they also provide vulnerability patch dates, discovery dates, vulnerability types, etc. There are literally hundreds of vulnerability databases and security advisories on the Web. Some of them are freely available and others are proprietary resources. Some of them are managed by governments and others are run by private security companies or open security communities. In this paper, we investigate security vulnerabilities in open sourced Moodle LMS in a quantitative manner. We mined Moodle vulnerability dataset from the NVD on December The rest of the paper is composed as follows. In section 2, we are walking through some of the related works. In section 3, we first examine the Moodle security vulnerabilities with respect to CVSS score. After that, in section 4, vulnerability discovery process in Moodle will be observed to see whether the discovery pattern could be estimated. In section 5, the vulnerability dataset is speculated for checking a seasonal fluctuation. Finally, in section 6, the conclusion will be given. 2. Related works In [5], the authors, first, try to explain LMS security concept with respect to the 10 security domains defined by the international information systems security certification consortium ( The 10 security domains are {risk management, access control, cryptography, physical security, security architecture and design, business continuity and disaster recovery planning, network security, application security, operation security, and security related regulations}. As a result, this paper could be a simple quick guide for new LMS system admins who 254 Copyright c 2015 SERSC

3 want to have an overall picture of security issues about LMSes. Then, the paper presents analysis of survey results about e-learning students perspectives of how secure their LMSes are. The analysis shows that about half of the participants are confident that their LMSes are secure enough. In [15], the authors investigate to see if online media with high media richness would be more effective than those with low media richness to enhance the progress of learning from the perception to the comprehension, and then to the projection of security risks. The results show that the existence of positive correlations between the degree of media richness and the improvement of security awareness levels. Also the research found that hypermedia-based instruction is the most effective approach to enhance security awareness levels. Based on what they found, the authors say that the observations from the paper could be used by educators, and training designers to create meaningful information security awareness materials. In [12], the author tries to answer the two research questions of, first, what factors constitute e-learning system adoption outcomes and how do those factors related to each other and, second, how do beliefs about usefulness and ease of use held by students influence adoption of e-learning for end-users. This study is based on 249 university students who are using Moodle. The research is collecting numerical data using survey questionnaires and analyzing it with statistical methods. The result was presented as a structural equation model using partial least squares to evaluate the research results and hypotheses. This study reveals that perceived usefulness had a significant influence on e-learning system use, and perceived ease of use also influences e-learning system use significantly. Further, it shows that the predictive strength of perceived ease of use is weaker than that of perceived usefulness. The perceived usefulness strongly influences both perceived learning assistance and perceived community building assistance. Authors in [9] indicate that educational institutes are tend to use free and open source software in their infrastructures, and present analysis of evolution in Moodle to gain insight into the freely available LMS project. They analyzed the LMS with the four criteria of {growth of lines of code, number of updates in codes per month, similarity between releases, and growth of cyclomatic complexity [10]}. Based on what they had observed, they concluded that the architecture of Moodle has become more stable over time because versions differ less in recent releases than older versions although update commit activities have been higher in recent months, and the calculated cyclomatic complexity versus lines of code has shown a noticeable decrease. 3. Moodle Secureness with respect to CVSS Software vulnerabilities are created by unsafe or careless coding habits. According to mitre.org ( information security vulnerabilities are mistakes in software system which can be directly used by malicious hackers to gain access to a system or network. These software vulnerabilities are systemically recorded and managed by some of the standards in the field. Among them, there is a Common Vulnerability Scoring System (CVSS) [11] which measures how much a given vulnerability is critical. When privacy for an individual is a top priority, institutes should continually estimate their LMSes with respect to historical security reputations, vendor s proper patch distribution policy, and total cost they need to pay when their system is compromised. CVSS provides answers for many of those questions in some degrees. This section, first gives an overview of CVSS, and then examines the secureness of Moodle with respect to the CVSS. Copyright c 2015 SERSC 255

4 Figure 1. CVSS structure; Letters in Smaller Rectangular Represent Abbreviations 3.1. Common Vulnerability Scoring System CVSS has been adopted by many IT vendors to measure security vulnerabilities since its first launch in The CVSS scores for known vulnerabilities are readily available on the majority of public vulnerability databases on the Web, such as NVD ( The scoring system provides vendor independent framework for communicating the characteristics and impacts of the known vulnerabilities. When security analysts estimate risk assessment, they do not need to think about qualitative aspects because CVSS is largely designed to assess the severity of vulnerabilities in quantitative manners. It is now on the second version which was finalized its design in June 2007, and for the time being, its third version is ready to be released ( The CVSS is composed of three metric groups: base, temporal and environmental. Figure 1 shows the structure of the score system. It attempts to evaluate the degree of risk posed by a vulnerability, so that mitigation efforts can be prioritized. The score ranges [0.0, 10.0]; scores close to 0.0 indicate more stable whereas scores close to 10.0 mean more vulnerable to exploitation and cause more serious outcome. The base metric group, ranges of [0.0, 10.0], represents the intrinsic and fundamental characteristics of a vulnerability, and the score does not change over time. The base metric has two sub-scores of exploitability and impact sub-scores. The two sub-scores are also ranges of [0.0, 10.0]. The exploitability sub-score captures how a vulnerability is accessed and whether extra conditions are required to exploit a vulnerability while the impact sub-score measures how a vulnerability will directly affect IT assets as the degree of losses in confidentiality, integrity, and availability. The exploitability sub-score is composed by three elements of access vector (AV), access complexity (AC), and authentication (Au). The AV reflects how the vulnerability is exploited in terms of local (L), adjacent network (A), or network (N). The AC measures the complexity of the attack required to exploit the vulnerability once an attacker has gained access to the target system in terms of High (H), Medium (M), or Low (L). The Au counts the number of times an attacker must authenticate in order to exploit a vulnerability in terms of Multiple (M), Single (S), or None (N). On the other hand, the impact sub-score is composed by the three key aspects in information security components: confidentiality (C), integrity (I) and availability (A). The impact attributes are all assessed in terms of None (N), Partial (P), or Complete (C). 256 Copyright c 2015 SERSC

5 The temporal and environmental metrics are used to augment the base metrics, and they are depending on the target system and changing circumstances. In this paper, these two metrics are not utilized, so they are not explained here Quantitative analysis of Moodle with CVSS (a) Moodle (b) The entire NVD database Figure 2. Number of each Value in Exploitability and Impact Sub-Score Groups Figure 2 shows the number of each value in exploitability and impact sub-score groups. Specifically, Figure 2 (a) and Figure 2 (b) are representing Moodle and the entire software systems respectively from the NVD. For the AV element, both Moodle and the entire software systems tend to be compromised from the outside network (N). There have been 254 vulnerabilities have been found in Moodle by now. So, when we add all the numbers in each element, we achieve 254. For example, in AV from Figure 2(a), = 254. However, in Figure 2(b), it is possible that the number is not matching due to the incomplete database information. For Au, both have the most of incidences with no authentications. However, in Moodle, while single authentication has about 42% or 108 incidents are occurred, only about 6% or 3822 incidents are occurred in Figure 2 (b). This indicates that even though a Moodle user protected by a password, a single access password seems not secure enough compare to other software systems. Furthermore, this shows that exploitations are usually from remote networks, and if we have at least one authentication process in our systems, it is a lot safer than systems having zero authentication. For AC, both Moodle and the others have about the same proportions among H, M and L. Most of the time, incidents are occurred in M and L. There are very small incidents of Hs. It implies that more complex systems have a lot less probabilities to be targeted. Figure 3. Relationship between Software Age and Vulnerabilities Represented by AML Copyright c 2015 SERSC 257

6 For the Impact sub-score, P (partial) takes place the highest numbers for the all three categories (Confidentiality, Integrity and Availability) in Figure 2(b). In Moodle, P has two highest numbers out of three. For the A (Availability), C (Complete) is the most common values in Moodle, which indicates that once the LMS is compromised, then users are not able to access to the Moodle properly for about 70% of chances. 4. Moodle vulnerability discovery process In this section, we investigate vulnerability discovery process in Moodle by a model fitting method. This will let us know how well the discovery process will be estimated in advance. For the model fitting, AML (Alhazmi-Malaiya Logistic) vulnerability discovery model [2] is utilized. The AML model is originally proposed for the operating systems, and later, applied to general software systems such as HTTP Servers [17] and browsers [7]. Figure 3 shows the AML model representing the relationship among software age, cumulative number of vulnerabilities and vulnerability discovery rate. The AML assumes that during the initial learning phase, very few vulnerabilities are found because of a small number of users. In the next phase, termed the linear phase, a steady stream of vulnerabilities is discovered. In the final saturation phase, the vulnerability discovery rate declines. The durations implicitly depend on factors such as market share or undetected number of vulnerabilities remaining. In the figure, the bell-shaped dashed line shows the vulnerability discovery rate whereas the S-shaped solid line expresses the cumulative number of vulnerabilities along with the timeline. Market share is a significant factor impacting the effort expended in exploring potential vulnerabilities for a target system. A higher market share provides more incentive for malicious hackers to explore vulnerabilities. The effect of the market share rise and fall is implicit in the AML model. Equation (1) gives us the three-parameter AML model where A, B and C are empirical parameters and Ω(t) represents the total number of known vulnerabilities at time t. This equation represents the S-shaped line in Figure 3. (a) AML Model fitting on the original growth (b) AML Model fitting on the modified growth Figure 4. AML Model Fittings on the Vulnerability Discovery Process in Moodle Table 1. χ 2 Goodness of Fit Test Results from Figure 4 A B C 2 χ s 2 χ c P-value Figure. 4(a) 1.39E E-43 Figure. 4(b) 9.93E B ( t ) (1) ABt BCe Copyright c 2015 SERSC

7 In Equation (1), notice that when time t goes to the infinity, B becomes the eventual number of vulnerabilities discovered in a software system. Figure 4 shows the AML model fittings on the vulnerability discovery process for the Moodle. Figure 4(a) shows the original Moodle vulnerability discovery process while Figure 4(b) is a modified discovery process with a reasonable assumption. In each sub-figure, black dots represent the number of vulnerabilities on the corresponding year-month time while the red solid lines are AML model fittings. In Figure 4(a), the growth trend shows a slow growing linear pattern until the middle of Between July 2010 and June 2012, only 2 vulnerabilities had been reported. However, on July 2012, 83 new vulnerabilities were announced. Because of the sudden ascent, the discovery process does not follow the linear pattern anymore. Moodle version 2 was released on November 2010, and 28 major and minor versions were released during the flat period of October 2010 to June This should have had produced more security flaws due to the active code evolutions. Furthermore, Moodle have expanded its market share by supporting more language selections. In Table I, to see whether model fittings are statistically significant, we conducted Chisquare (χ 2 ) goodness of fit test. For the fit to be acceptable, the test statistic value (χ s 2 ) should be less than the corresponding critical value (χ c 2 ) for the given alpha level and the degrees of freedom. In this paper, p-values need to be greater than 0.05 to be statistically significant since we are using the alpha level of Goonatilake et al. [4] provide a material how to apply χ 2 test in the computer science subject. As the table shows, the model fitting result with the original discovery pattern is not statistically significant. Its p- value is less than 0.05 and χ s 2 > χ c 2. Figure 4(b) shows the number of vulnerabilities with AML model fitting, but the number of vulnerabilities found, between July 2010 and June 2012, is modified, so that the sudden uprising in Figure 4(a) can be eliminated. During the flat period, our assumption is that the discovery process is linear since AML model indicates that the first transition point 1 (T1) from Figure 3 occurs at September 2012 in Figure 4(b). Table 2. Seasonal Index JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC χ 2 s : χ 2 c : p-value: 9.5E-148 Figure 5. Autocorrelation Function Values. The Dashed Lines Represents 80% of Confidence Intervals. Legs are in Month Table 3. Autocorrelation Function Values in Number Copyright c 2015 SERSC 259

8 For your information, the T1 value in Figure 4 (a) is November The time point can be achieved by T1= ln[bc ((2+ 3))]/AB [1]. Based on the model fitting, we could expect continuous vulnerability reports for a while in Moodle since the second transition point will be falling into April However, if newer versions, having new codes, are released in the future, then the transition points could be changed accordingly. As shown in Table 1, the model fitting on the modified growth is statistically significant, p-value greater than 0.05 and χ s 2 < χ c 2. Thus, to estimate the future trend in vulnerability discovery for Moodle, eliminating the flat period is more suitable. In the previous work [6], the model fitting is conducted with the dataset collected by October 2012, and the result showed that the reasonably modified vulnerability dataset represents more suitable to estimate the future trend of the growth rate. The fitting result from this paper also suggests that caulking the flat area in the vulnerability discovery process is necessary for better model fittings and possibly better model prediction capabilities. 5. Seasonal Variation in Vulnerability Discovery Process A time series analysis with the periodic trends allows the developers to predict future needs more accurately. In this section, we examine Moodle for annual seasonality in its vulnerability discovery process. Statistical methods, that we are using to investigate whether there is a seasonal pattern in the discovery process, are seasonal index ( and the autocorrelation function (ACF) [3]. A seasonal index states to what extent the average for a particular period tends to be above (or below) the expected value. Time series data is not uniformly distributed and periodic patterns are present if certain periods have significantly more instances of reported vulnerabilities than other time periods. Table 2 shows seasonal indices for the twelve months. March, July, September and November have the expected seasonal index value greater than 1.0, which is an expected value. To evaluate the significance of the non-uniformity of the distribution among the seasonal indices, we applied the Chi-square test to the grand total of each month against the mean value (total vulnerabilities divided by 12). In this paper, the level of alpha chosen is Hence, when the p-value of the Chi-square test is below 0.05, the null hypothesis that there is no seasonality in the dataset, will be rejected. In Table 2, we can see that the systems yield extremely small p-values, thereby providing a statistical evidence of the non-uniformity, contrary to the null hypothesis. Now, the autocorrelation function (ACF) in time series analysis is calculated by computing the correlation between a variable value and the successive values of the same variable with some time lags. Thus, ACF measures the linear relationship between time series observations separated by a lag of k time units [3]. When an ACF value is located outside of defined confidence intervals at a lag k, there is a significant relationship associated with that time lag. Figure 5 shows the ACF values from the Moodle vulnerability dataset, and Table 3 shows the detail numbers for Figure 5. In the figure, lags of six and twelve tend to be outside the 80% confidence intervals ( , ) shown by the dashed lines. This demonstrates strong autocorrelations with lags that are multiples of six, which confirms a six-month periodicity in the Moodle vulnerability dataset. This six-month up and down pattern should be related to the school semester systems, adapted by many educational institutes. 6. Conclusion This paper has focused on the quantitative analysis of vulnerabilities in Moodle LMS. Specifically, our analysis is focused on the three points, namely, CVSS, discovery trend, 260 Copyright c 2015 SERSC

9 and seasonality in the Moodle vulnerability dataset. Results demonstrate the followings. First, it has been found that exploitations are usually from the remote networks, and if we have at least one authentication process in our systems, it is a lot safer than systems having zero authentication. However, a single access password method seems not secure enough, compared to other types of software systems. Further, the result implies that more complex systems are a lot less chances to be targeted, and once Moodle is compromised, there are about 70% of chances that users are not able to access to the Moodle. Second, the result shows that, to estimate the future vulnerability discovery process more accurately in Moodle system, modifying the certain part of the period is necessary. Eliminating the flat area in Figure 4 (a) makes fitting result statistically significant. It might help to improve model prediction capability also. Measuring prediction capability with modified datasets could be one of the future works. Lastly, we have been observed that there is indeed a six-month periodicity in the Moodle vulnerability dataset. Based on this information, system administrators should patch their systems before the seasonal indices hit the high values, indicated in Table 2. Acknowledgment This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (2015R1C1A1A ) References [1] O. H. Alhazmi and Y. K. Malaiya, Prediction capabilities of vulnerability discovery models, in Proceedings of the '06 Annual Reliability and Maintainability Symposium, Washington, DC, USA: IEEE Computer Society (2006), pp [2] O. H. Alhazmi and Y. K. Malaiya, Application of Vulnerability Discovery Models to Major Operating Systems, IEEE Transactions on Reliability, (2008), vol. 57, no. 1, pp [3] B. L. Bowerman and R. T. O'connell, Time Series Forecsting: Unied concepts and computer implementation. (2nd ed.). Boston: Duxbury Press, (1987) [4] R. Goonatilake, A. Herath, S. Herath, S. Herath, and J. Herath, Intrusion detection using the chi-square goodness-of-fit test for information assurance, network, forensics and software security, Journal of Computing Sciences in Colleges, (2007), vol. 23, no. 1, pp [5] M.F. Hilmi, S. Pawanchik, and Y. Mustapha, Exploring security perception of learning management system (LMS) portal, in Proceedings of the 3rd International Congress Engineering Education, (2011), pp [6] H. Joh, Modeling Security Vulnerabilities in Learning Management Systems, International Journal of Learning Management systems, (2013), vol. 1, no. 2, pp.1-12 [7] H. Joh, Assessing Web Browser Security Vulnerabilities with respect to CVSS, Journal of Korea Multimedia Society, (2015), vol. 18, no. 2, pp [8] J.A. Jones, An introduction to factor analysis of information risk (FAIR), A framework for understanding, analyzing, and measuring risk, Technical Report, Risk Management Insight Inc., (2007) [9] H. J. Macho and G. Robles, Preliminary lessons from a software evolution analysis of Moodle, in Proceedings of the 1st International Conference on Technological Ecosystem for Enhancing Multiculturality, Francisco José García-Peñalvo (Ed.). ACM, New York, NY, USA, (2013), pp [10] T. J. McCabe, A Complexity Measure, IEEE Transactions on Software Engineering, (1976), vol. SE-2, no. 4, pp [11] P. Mell, K. Scarfone, and S. Romanosky, CVSS: A complete Guide to the Common Vulnerability Scoring System Version 2.0, Forum of Incident Response and Security Teams (FIRST), Technical Report, (2007) [12] A.K.M. Najmul Islam, Investigating e-learning system usage outcomes in the university context, Computers & Education, (2013), vol.69, pp [13] C. P. Pfleeger and S. L. Pfleeger, Security in Computing. (3rd ed.). New Jersey: Prentice Hall PTR, (2003) [14] Research and Markets, Learning Management Systems (LMS) Market by Products & Users: Worldwide Copyright c 2015 SERSC 261

10 Market Forecasts and Analysis ( ), Technical Report, (2013) [15] R.S. Shaw, C. C. Chen, A. L. Harris and H. J. Huang, The impact of information richness on information security awareness training effectiveness, Computers & Education, (2009), vol. 52, no. 1, pp [16] A. M. St. Laurent, Understanding Open Source and Free Software Licensing. Sebastopol, CA: O Reilly Media, (2008) [17] S.W. Woo, H. Joh, O. H. Alhazmi and Y. K. Malaiya, Modeling Vulnerability Discovery Process in Apache and IIS HTTP Servers, Computers & Security, (2011), vol. 30, no. 1, pp Author HyunChul Joh, he is an assistant professor in department of computer engineering at Kyungil University. From 2012 to 2014, he was a GIST college laboratory instructor in division of liberal arts and sciences at Gwangju Institute of Science and Technology. His research focuses on modeling the discovery process for security vulnerabilities and risk metrics. He received his Ph.D. and M.S. in computer science from Colorado State University in 2011 and 2007 respectively. He also received a B.E. in Information and Communications Engineering from Hankuk University of Foreign Studies in Copyright c 2015 SERSC