1 In Proceedings of EDUCON Remote Security Labs in The Cloud ReSeLa Anders Carlsson BTH Sweden Rune Gustavsson BTH Sweden Leo Truksans IMCS LU Latvia Martens Balodis IMCS LU Latvia Abstract The paper describes on-going work on a configurable network based experimental platform ReSeLa. The platform is a key component of the ongoing EU funded TEMPUS project ENGENSEC. The project is aiming at providing courses and training material to educate future generation of Cyber security experts. Our project is based on the educational Framework; Conceive, Design, Implement, Operate (CDIO 1 ). The paper outlines the background of earlier efforts on similar platforms in Sweden and Latvia. In the paper we also compares our approach with lessons learned in international projects such as PlanetLab, EmuLab, and GENI. Our approach of Cloud based environments is based on recent advanced in distributed computing such as OpenStack. Some identified and addressed challenges care presented. We also present the architecture of ReSeLa as well as its basic functionalities. The paper ends with a section on Future work and a short list of references Keywords Cloud Based Security Engineering, Open Stack, learning and training, international cooperation I. BACKGROUND We are partners in the on going EU Project ENGENSEC 2 Educating the Next generation experts in Cyber Security (Engensec.eu). The main objective of the ENGENSEC project is to create new Master s programs in areas of IT Security as a response to current and emerging cyber security threats. The goal of ENGENSEC is to develop courses, environments and training material to educate the next generation experts in this area. Thus supporting development of e-based economics and other security critical societal services in partner countries. The ENGENSEC project addresses challenges in information systems development related to ensuring Cyber Security and privacy. The project thus enables advanced Master s curricula in Cyber Security supporting education and training of the next generation experts. These suggested curricula are based on previous best practices in double degree diploma among EU universities, ECTS grading and mutual degree recognition. A crucial component in the project is to design, develop and implement sustainable experimental environments. That is, our Cloud based Remote Security Labs (ReSe-La). The ReSeLA architecture is a result of earlier efforts by the partners, for example at Blekinge Institute of Technology (BTH) and Institute of Mathematics and Computer Science of University of Latvia (IMCS LU). In 2006, Blekinge Institute of Technology (BTH) deci-ded to create a remotely accessible laboratory supporting advanced security exercises and training (OpenLabs Security laboratory). In the laboratory, students were able to experiment with insecure protocols, software vulnerabilities and other harmful software, e.g. viruses, in a safe and isolated environment. This remotely accessible security laboratory was the next logical step of the BTH campus security laboratory that had been used for advanced security experiments since The remote security laboratory introduced new challenges related to network security and remote control. To be able to provide remote controlled experi-ments in security, isolation of the experiments must be guaranteed. No unwanted information should be able to reach the experiment from the outside world and, more importantly, no information should ever be able to escape the experiment environment. Providing isolation is crucial when experimenting with self-replicating code, e.g. viruses, which must not be allowed to spread outside the laboratory environment. Good experiments should also be reproducible, i.e. it should be possible to restart the experiment from a known state. When we designed and implemented OpenLabs we considered
2 to use virtualization to increase flexibility. However, at that time this technology had some known limitations. Virtualization platforms can utilize the hardware resources better than emulated platforms, and will most certainly be cheaper to set up and run, depending on what hardware and software platform that is used. The startup of a Virtual Machine (VM) is faster that doing physical writing to a hard drive, so the startup time of an experiment will be shorter. After the experiment is over, the VMs disk content can be discarded, which means you do not need to wipe the complete hard drive. It will also be easier to prepare installations, as the teacher can do that at his/her own machine in the virtualized environment, without the worry for hardware incompatibility. On the downside, using virtual machines can lead to unwanted behavior. If many experiments are going to run on the same machine, they have to share the hardware resources between each other, which means that they also influence each other. Experiments which are resource intensive will take longer to perform and results from performance measurements will not always be reliable. Competing on resources can also influence results from experiments depending on concurrency, one example from the security domain is race conditions (See ). Running several virtual environments on the same machine will also lead to penalties on hard drive access. In a virtualized environment the hardware is emulated. Direct hardware access can be done be in form of device delegation, however this mechanism is not well automated in existing IaaS implementations. The emulation could introduce new bugs related to the emulation software and will probably hide bugs in the real hardware. Experiments that require direct hardware access will therefore be impossible in the first version of RSeLa, as would experiments depending on hardware bugs. For many experiments, these negative aspects may not be a problem and virtualization can still serve a good purpose as a remote experiment solution. Particularly, the experiments on software functionality that disregard properties of performance or hardware are expected to be well suited for the virtualized environments like Infrastructure as a Service (IaaS) clouds. In this case the machine will be prepared with the virtualization software and the teacher or the students can themselves provide the virtual machine image and take advantage of the simpler preparation procedure given by the virtualized environment. In Proceedings of EDUCON service provided para-virtualized virtual machines (VMs) based on Xen hypervisor, virtual networks to isolate groups of machines belonging to a separate research lab or project. The service was a clustered, high availability, automated system to host high performance VMs with real life OSes (Linux, Windows) dedicated to scientists and/or projects. It was envisioned, designed, implemented and operated at IMCS UL. It was also a new approach with no existing demand and it took time for the users to get comfortable with concept that getting a new computer for calculations is not getting a physical equipment but rather an access to a dedicated VM somewhere in a data center. Less than 3 years later after starting system operations in 2011 it hit first resource deficit. That reflects high acceptance of the system that by then was called a Cloud on par with the industry trend. At that time the Cloud had been extended to host multiple infrastructure and platform services (IaaS, PaaS), VPN access and tunnel services, real time monitoring, etc. The Cloud was recognized by its users to be reliable and well performing resource, used by tens of labs and projects. Today the IMCS UL Cloud continues to evolve and operate. In 2013 OpenStack development was skyrocketing, it was clear to IMCS UL Cloud team that OpenStack will be the next Linux of clouds (open source platform project embraced and developed by the industry). When IMCS UL got to design and deploy an automated platform for Faculty of Physics and Mathematics of University of Latvia (FPM UL) that year, it was decided to deploy an OpenStack installation because of its ease of use. Yet one concept was carried from IMCS UL Cloud into the FPM UL Cloud. The latter had InfiniBand cards for MPI calculations. It was made sure these cards are available to certain type of VMs through device delegation. And that made the Cloud usable for HPC. Some of the mentioned challenges will be addressed and some of the mentioned concepts will be further investigated in design, implementation and maintenance of ReSeLa. While ReSeLa does dedicate and give full control of the operating systems to any student, as the OpenLabs did, ReSeLa is designed to integrate and take advantage of the modern infrastructure automation solutions and concepts like IaaS clouds and software defined networking (SDN) as the IMCS UL Cloud did. Another interesting aspect of virtualization is security. The main contributor to software vulnerabilities are complexity and by introducing a virtualization layer the complexity is increased even more. As no software is perfect, there may be ways for users to break out of the virtualized environment . This would lead to undefined behavior of the whole laboratory. In 2008 Institute of Mathematics and Computer Science of University of Latvia (IMCS UL) decided to create a virtual infrastructure service for Latvian scientific community. The The remaining part of the paper is organized as follows. The educational Framework of ENGENSEC is described in Section II ENGENSEC Educational Framework. The ReSeLa configurable platform is an important part of the ENGENSEC project being the link between, courses and training material The ReSeLa architecture is described in Section III. Section IV ReSeLa functionality has focus on the architecture and needed functionality. Section V Challenges addressed identifies critical challenges that have been identified and mitigated so far in our development. Section VI Other approaches presents the main ideas behind the international efforts on PlanetLab,
3 Emulab and GENI as well as recent EU funded project based on OpenStack technologies with comparisons with the ReSeLa approach. Finally, Section VII Future Work describes some key areas presently under investigation in development and maintenance of ReSeLa. The paper ends with a short list of references, Section VIII. In Proceedings of EDUCON The educational framework of ENGENSEC adopts the ideas behind the CDIO 3 Conceive, Design, Implement, and Operate Framework. The CDIO project started in 2000 with a first set of standards adopted The CDIO book version 2 came out The Lab environments at BTH (Section I) were also heavily influenced by the CDIO ideas. II. ENGENSEC EDUCATIONAL FRAMEWORK The goal of ENGENSEC is to develop, implement and validate a Joint MSc Frame Program in Cyber Security. The Frame program can be bilaterally implemented with Double Degree Diploma to enable cooperation and exchange of students and staff between involved institutions. The first step was to identify challenges and needs of a joint MSc Frame program in the different Information societies. The second step was to identify suitable contents and processes to develop and implement selected courses. The third, on going, step is to develop and evaluate he study program with related resources and education environments. The next step is to implement, train teachers, and evaluate selected pilot projects. The results of the initial survey to 52 companies are related to security threat models and existing tools as well as expected education skills developed in the MSc programs. The following set of seven courses where identified to be part of the ENGENSEC Frame program: Advanced Network & Cloud Security Wireless & Mobile Security Secure Software Development Malware Analysis Web Security Penetration tests and Ethical Hacking Digital Forensic Furthermore a configurable training environment Remote Security Lab (ReSeLa) was identified to be the training backbone of the program. ReSeLa is further described in this paper. Development and integration of the eight work packages is done in groups of peers from different organizations. The selection criteria of people were: Advanced skills in the specific topic Practical skills in the specific topic Teaching experiences on master level Interest and administrative skills in development of master courses AdobeConnect meetings and Owncloud facilitate collaboration, within and between teams. The suggested courses of ENGENCEC, above, are also a context dependent selection of courses recommended by, for instance, NICE National Initiative for Cybersecurity Education 4. Specifically, the NICE Framework 5. To foster a common view of the goals and means of the ENGENSEC project a workshop Train the Trainer was organized October 20 th 24 th in Schloss Waldhausen in Budeshem, Germany. The workshop was organized by BKA Bundeskriminalamt, Germany. A follow up of this workshop is planned as a Summer School during A key component of the CDIO framework is development of suitable training and education material bringing together students and teachers in learning situations. The ReSeLa will be a fundamental component to that end. An early version of ReSeLa OpenStack was used to perform simulations related to our research on Methods of Slow-Attack Detection . Our experiments illustrate that ReSeLa, as many other web-based systems, are vulnerable to HTTP-Slow-Attacks. Future work on ReSeLa will address this vulnerability as well. Traditional Intrusion Detection Systems (IDS) are based on statistical models and methods. For example, wavelet analysis, signature analysis, cluster analysis and so on. Those methods can be used to configure effective methods for detection and response of attacks on servers. For example flooding DDoSattacks and transport layer attacks. The attacking DDoSattack methods are typically aiming to/at filling the channel capacity (Smurf, DDP-flood, etc.) and increase the normal load of individual nodes (SYN-flood, Teardrop, Ping of death, etc.). At the same time, those IDS approaches are ineffective for the detection and protection of low-intensity DoS attacks on application level. Those attacks are typically characterized by the absence of anomalies in the traffic patterns. For instance, smart phones and tablets operating in areas with weak signals are generating traffic patterns that are very similar to Slow- HTTP attack patterns! e_framework_03_2013_version1_0_interactive.pdf
4 The classes of low-intensity attacks are relatively recent but are probably growing recently. Typically low intensity attacks lead to failure of web-servers but can also be adapted to influence any application layer system. From an attackers point of view the main advantages of lowintensity attacks are: The connections appear as legitimate user connections Traditional IDS usually don not detect those attacks Existing signature based IPS/IDS techniques typically do not detect those attacks Those attacks require few resources and low bandwidth to be implemented Such attacks can drop web-servers regardless of attacker s hardware capabilities Different stages of queries can be used to implement different types of low intensity attacks Our proposed solutions for detection of slow-http attacks are based on assessment of web-server utilization and prediction of the time of transition to a state of overloading . Analysis of identified conditions of slow-attacks has resulted in a set of traffic patterns that are specific for Slow-HTTP attacks on application layer . Our request handling process by a we-server is modeled as a queuing system without a queue due to the fact that a presence of a waiting buffer in a web-server has no effect on the availability of service during a slow-http attack. The purpose of the model is detection rather than duration of an attack. In Proceedings of EDUCON ReSeLa is an integrated system that consists of following three components on the conceptual level: ReSeLa Panel front end; OpenStack IaaS backend; SDN router firewall and VPN backend. We chose OpenStack as a back-end for IaaS automation because it is well supported by the industry, has wide functionality, is modular in its architecture and has well defined and stable API. The integration is done through OpenStack API. Currently we use RouterOS based Mikrotik router appliance because it has API for firewall and VPN management, rich functionality, high performance and low cost. Other solutions might be used but currently only the RouterOS driver for ReSeLa has been developed. ReSeLa also uses MySQL database for its internal data structures. The ReSeLa integration model is shown below in Figure 1. A model to detect Slow-HTTP-attacks is designed and implemented using Markov chains and queuing system theory 6. The model can estimate the transition time to an overload condition of the attacked system. This value can be used for attack preventions algorithms and to develop appropriate tools to protect web-servers. III. RESEALA ARCHITECTURE The ReSeLa architecture takes advantage of recent advancements in distributed computing, e.g., Cloud Computing 6, such as Open Stack 7. Earlier examples include Emulab 8, Planet-lab 9, and GENI 10. The test beds Emulab, Planetlab and GENI will be further discussed in Section VI Other approaches Fig. 1. Resela integration model Open Stack is an Open source Cloud Computing software system environment, while Emulab focuses on the links between virtual topologies and their emulations on hardware. Selected technologies for Emulab virtual nodes are evaluated against four criteria, two from an application perspective, two from a system-wide perspective : Application transparency. The extent to which virtual name spaces (e.g., process, network, file system) are isolated from each other. (Can the application run unchanged?) Application fidelity. The extent to which virtual node resources (e.g., CPU, memory, IO bandwidth) are isolated from each other. (Does the application get the resources it needs to function correctly?) System capacity. The amount of virtualization overhead. (How many virtual nodes can we host per physical one?)
5 System flexibility. The level of which virtualization take place (can we run multiple OSes?) and the degree of portability (can we run on a wide range of hardware?) The following Figure 2 captures present architecture concepts from OpenStack provisionally selected for ReSeLa. In Proceedings of EDUCON Hardware Controller Node: 1 processor, 2 GB memory, and 5 GB storage Network Node: 1 processor, 512 MB memory, and 5 GB storage Compute Node: 1 processor, 2 GB memory, and 10 GB storage Figure 3 illustrates our Three node Architecture: Controller, Compute and Network. Those architecture models are also subject to changes that will be further elaborated in Section VI. Software Centos 6.5 desktop version OpenStack Icehouse - Current stable release, security-supported!! Horizon! IV. RESEALA FUNCTIONALITY Provides!UI! Neutron! Nova! Network!connection! VM! Provisions! Provides! Images! Provides!Auth! Keystone! Glance! Stores! Images! Swift! ReSeLa provides the following high-level functionality: Web access to Lab management and usage; On demand Lab provisioning; A concept of Virtual Lab as a singular object; Isolated Lab networks, customizable network policies; VPN access points that allow direct connections to Lab instances. Fig. 2. OpenStack components for ReSeLa ReSeLa Panel is the front end for students, teachers and administrators, Figure 4. Fig. 4. The ReSeLa student front page ReSeLa students and teachers are commonly referred to as Users. The Panel supports the life cycle of the Virtual Lab, Figure 5. Fig. 3. ReSeLa three node architecture Minimum hardware requirements for the three-node configuration are at present, but are subject for changes in the future: Fig 5. The ReSeLa Virtual Lab life cycle
6 The Lab s object is a set of the following virtual objects: Virtual machines (VMs) with some CPU/RAM/HDD resources; References to templates of VM disk images; Nets with connections to VM interfaces; Network policy (access to Internet or specific Ips). When provisioned a Lab object forms a sandbox internal network and one or more VMs in it. In Proceedings of EDUCON Network policy defines the inbound/outbound access of the Lab network. All users and administrators have user accounts with their role in the ReSeLa and can use those for ReSeLa functionality allowed to their role. A subset of basic functions of ReSeLa and the role access matrix is shown in Table 1. Fuinction Student Teacher Admin Lab life cycle List labs and access consoles ot the instances Number of simultaneous labs Manage Lab templates Manage accounts users Access OpenStack management Use VPN account for network access to instances (all enrolled courses) (all assigned courses) All courses Only own For all assigned students For all users 1 1 per course unlimited (only own) /only own) Table 1. Selected functions and role access of ReSeLa The ReSeLa system will be configured to support security training and education by selecting appropriate configurations of courses from the ENGENSEC program and related training material and manuals. Lessons learned will be inputs to coming generations of the ReSeLa environment. The functional architecture of ReSeLa is presented in Figure 6 below. Fig. 6. Functional architecture of ReSeLa V. CHALLENGES ADDRESSED We have identified and addressed several challenges of different types in the ENGENSEC project. The challenges are related to education policies, suitable solutions to technological problems and challenges related to learning and practice security measures to mitigate different types of security threats. Examples related to technological issues are, for instance, related to that our partner countries Ukraine and Russia can only provide limited Internet speed connectivity. Availability of IP v.4 addresses is extremely limited implying that they have no possibility to use IPv.6 addresses. However, at both BTH in Sweden and at University of Latvia in Lithuania we can use IPv.6 and we have available IPv.4 resources. Consequently, we will implement and evaluate an IP - tunnel testing of OpenVPN/PPTP connections. This means that students at those institutions can have full access from their homes or university computer to the available Lab-set of computers. Another challenge is to assure that only limited traffic is allowed, The protocols ssh, VNC, Virtual desktop or X.11 are at the student s computers and we have to assure that there is no possibilities for other foreign traffic, inbound or outbound from the lab-sets or between different student labsets to exist. One of the strengths of Resela is it's simple and clean user interface. We believe this is an important feature of a remote Lab system and follow this feature as a guideline when designing the architecture and functionality. The access to consoles of VMs is provided through the same Resela Panel Web interface, which eliminates need to install and use other 3rd party software. VI. OTHER APPROACHES Examples of other international approaches towards shared environments supporting experiments with distributed computing are, as we earlier mentioned:
7 PlanetLab Emulab GENI PlanetLab. PlanetLab is an open platform for developing, deploying, and accessing planetary scale services. A first version of the system was developed 2002 by Princeton, UC Berkeley, and Intel Research. In October 2008: A prototype implementation of the GENI interfaces to PlanetLab functionality is made available. PlanetLab is network testbed distributed among a large number of universities and research organizations and it gave researcher a possibility to analyze network functionality, but with the important limitation; it was to only one special Linux dialect allowed as operating system. Other limitation is reliability of results of performance or timing. We have uncertain result because there is no suitable control function of those parameters. The test/ experiments could be executed simultaneously potentially intervening with other experiments without being able to take into account the workloads of network connections and computations of the experiments. In Proceedings of EDUCON and limitations leading to guidelines for Emulab users to take into account when setting up and evaluating experiments. GENI. GENI (Global Environment for Network Innovations) provides a virtual laboratory for networking and distributed system research and education. It is well suited for exploring networks at scale, thereby promoting innovations in network science, security, services and applications. GENI has a GENIStack which is build around EmuLab components and hence closed systems. GENI allows experimenters to: Obtain compute resources from locations around the United States; Connect compute resources using Layer2 networks in topologies best suited to their experiments; Install custom software or even custom operating systems on these compute resources; Control how network switches in their exoeriments handle traffic flows; Run their Layer 3 and above protocols by installing protocol software in their compute resources and by providing flow controllers for their switches. PlanetLab has proven to be an invaluable platform for learning about network-wide phenomena, creating new network protocols, evaluating new and existing network services, gaining experience with network systems running at global scale, and in the end, deploying novel network services that enhance the capabilities of the Internet. Emulab. Emulation testbeds are increasingly used to study the Internet in order to improve protection and response mechanisms. These are frequently considered more adequate than software simulators to realistically recreate the complex behaviour of networks. A recent paper On the use of Emulab Testbeds for scientifically Rigorous Experiments  sheds some light on insights of strengths and weaknesses of emulation based experiments. Scientifically rigorous experiments should meet requirements in terms of: Fidelity; refers to how accurately an experimental platform reproduces a real system Repeatability; represents the ability to repeat an experiment and obtain the same or statically consistent result Measurement accuracy and Interference; experiments should be accurately monitored and measurements should not interfere with the experiment because the might alter the outcome of the experiments The findings of the paper is that Emulab based testbeds are representative of real systems in terms of emergent behaviour (qualitative) and that repeatable experiments are possible. Based on experimental results have furthermore provided caveats and insights to significant configuration parameters Recent EU funded projects based on OpenStack are XIFI 11 and FIware 12, aiming at simplifying design and implementation of business support systems. Some comparisons of PlanetLab and EmuLab with ReSeLa: Planetlab s major limitations it only support one special Linux dialect as operating system Planetlab s other imitations is the limited reliance on result in performance and timing Both EmuLab and DistSeclab from BTH have high demands on computing resources, operating systems or network components and need a computer box for each object (limited distribution) ReSeLa use SDN Software Defined Network technologies and principles to create objects and Virtual Machines (VMs) to simulate OS or Services. A Hypervisor that could support, for instance, setting up and monitoring scientifically rigorous behaviors, by providing trustworthy mappings and scheduling of VM on hardware equipment. VII. FUTURE WORK Future work will be on testing behavior and components and improving functionality
8 Testing of functionality and performance. Specifically related to flexible and reliable allocation of resources and networking. Extending the learning and teaching environments by tools to directly report completed tasks by the students. Some results of lessons learned from experiments on the BTH platform are presented in  and , The initial plan of ReSeLa similar to the EU project XIFI, that is to create a federated cloud that interacts with other ReSeLa OpenStack clouds. Unfortunately, it turned out that the universities of Ukraine and Russia did not have a sufficient capacity of Internet connections to support this on the ENGENSEC level.. To harness this obstacle we decided at an early stage to focus on building local ReSeLa installations. This implies that a future challenge will be to develop support to run a distributed cloud between different universities. Next challenge would be to introduce Green Computing, that is introduce powerup/down computers to save energy/resources. There is support for those developments in next generations of OpenStack. In Proceedings of EDUCON VIII. REFERENCES  John Viega, Gary McGraw, Building Secure Software How to Avoid Security Problems the Right Way, September  Common Vulnerabilities and Exposures, CVE , Published  Hibler, M., Ricci, R., Stoller, L., Duerig, J., Guruprasad, S., Stack, T., Webb, K., and Lepreau, J.: Large-scale Virtualization in the Emulab Network Testbed.  Siaterlis, C., Garcia, A. P., and Gemge. B.: On the Use of EmuLab Testbeds for Scientifically Rigorous Experiments. IEEE Communications, Surveys & Tutorials, Vol. 215, No 2, Second Quarter  Gustavsson, R. and Carlsson, A.: Resilient Smart Grids, Configurable Experiment Platforms. In Proceedings of IEEE First International Scientific Practical Conference PIC S&T 2014, October Kharkiv, Ukraine.  Duravkin, I. and Carlsson, A.; Method of Slow-Attack Detection.. In Proceedings of IEEE First International Scientific Practical Conference PIC S&T 2014, October Kharkiv, Ukraine.