Real-time Operating Systems. VO Embedded Systems Engineering Armin Wasicek 11.12.2012

Real-time Operating Systems VO Embedded Systems Engineering Armin Wasicek 11.12.2012

Overview Introduction OS and RTOS RTOS taxonomy and architecture Application areas Mixed-criticality systems Examples: UAV, Synthetic vision The RTOS Market Implementation Linux as RTOS Conclusion 11.12.2012 Real-time Operating Systems 2

Computer system classification Transformational systems compute output values from input values, then stop. numerical computations, compiler Interactive systems constantly interact with their environment. The system delivers a service to the user. operating systems, databases Reactive systems continuously react to stimuli from the environment. Reaction time is dictated by the environment. signal processors, process controllers

Real-Time Systems In a real-time computer system the correctness of the system behavior depends not only on the logical results of the computations, but also on the physical instant at which these results are produced. Hard Real-Time deadline miss can have catastrophic results Soft Real-Time result has utility after the deadline Utility Utility deadline reaction time reaction time

Determinism A system behaves deterministically if given a set of initial conditions at time t 0 a sequence of inputs always produces the same sequence of outputs at any future instant t. Key difference between reactive and interactive systems Compare e.g. a process controller and a compiler Excurs: Predictability A deterministic system is not necessarily predictable Is our (timing) model a precise characterisation of the system? Can we predict e.g. context switch delay, execution times? Temporal behavior of many non-rt systems in unpredictable

OS and RTOS 11.12.2012 Real-time Operating Systems 6

Operating Systems Operating System (OS): abstracts from hardware provides access to I/O, memory management, sharing of the resources of the computer provides system calls to access low level functions Input devices System calls Registers User Application Hardware Output devices Signals Operating system Interrupts Embedded system is a hardware/software artifact.

Common OS Services Task management Scheduling and Timers Memory Management Memory Protection and Error Handling Hardware Abstraction and I/O interface Inter Process Communication Synchronization (mutual exclusion) Message passing Shared Memory

Real-Time Operating Systems (RTOS) Real-Time (RT) requirements for OS + features for timing constraints guaranteed max. execution time of system calls guaranteed OS response time to external events guaranteed max. execution time of OS functions (ISRs, drivers, context switches,...) Determinism and Predictability Efficiency Fast context switch Minimize intervals during which the interrupts are disabled

Scheduling created dead ready running Scheduling Decision: Which task in ready state can execute? blocked Scheduler uses a scheduling strategy e.g., fixed priorities, dynamic priorities, round-robin (time-slicing), rate monotonic, cooperative How does scheduler get CPU? called by application system timer interrupt 11.12.2012 Real-time Operating Systems 10

Schedulability Analysis In a hard real-time system, we need to a high confidence that no deadline will be missed during operation. Offline Scheduling Timing analysis is much easier without preemption Online Scheduling Static Priority Scheduling (e.g., RMS) Dynamic Priority Scheduling (e.g., EDF) Need to know timing characteristics of runtime system (e.g., OS, uc) and of all tasks (WCET) for real-time scheduling 11.12.2012 Real-time Operating Systems 11

priority Mutual Exclusion Priority Inversion Realized using e.g. semaphores Increases response times Priority inversion due to locking must be avoided in real-time systems! T 1 4. T 2 runs before T 3 1. T 3 starts with resource R 6. T 1 preempts T 3 2. T 1 attempts to lock R T 2 T 3 3. T 3 preempted by T 2 5. T 3 unlocks R 11.12.2012 Real-time Operating Systems 12

Is a scheduler absolutely necessary? No, you can meet deadlines without any RTOS (generate offline schedule, implement it): inefficient (regularly poll infrequent events, reserve max. time for interrupts) long schedule cycle time (least common multiple of all task periods) or unnecessary overhead (shorten task periods) hard to change and maintain (use tools!) 11.12.2012 Real-time Operating Systems 13

RTOS Taxonomy and Architecture 11.12.2012 Real-time Operating Systems 14

RTOS taxonomy (1) Small, fast, proprietary kernels (μc-os II, QNX, WinCE) Highly specialized to specific applications Real-Time extensions (RT-Linux, Xenomai, RT-Posix, RT-MACH, RT-WinNT) Compliant kernels Existing OS is modified such that non-rtos binaries run without modification Dual kernels Thin RT-kernel stay below the native OS Core kernel modification Changes are made in the core of OS The resource kernel approach Kernel is extended to provide support for resource reservation 11.12.2012 Real-time Operating Systems 15

RTOS taxonomy (2) Component based kernels OS components can be selectively included to compose an RTOS Selection depends on the target and application Construction of OS through composition ecos PURE - embedded applications MMLite - dynamic reconfiguration of components 11.12.2012 Real-time Operating Systems 16

RTOS taxonomy (3) Quality-of-Service (QoS) based kernels Used for soft real-time systems Research kernels Developed at university research projects to study research aspects of RTOS MARS SPRING HARTOS time-triggered, distributed admission control, reservation distributed communication 11.12.2012 Real-time Operating Systems 17

Monolithic Kernels All basic system services run in kernel space Provide rich and powerful abstractions of the hardware Amount of context switches and messaging involved are low Runs faster than microkernel Examples: Linux, Windows User space Applications Libraries Kernel space File systems IO/Device management Process management Interprocess comm. Memory management Hardware 11.12.2012 Real-time Operating Systems 18

Microkernels Software User space Server Applications File systems Kernel restricted to a minimum Improves maintainability, predictability, security Hypervisor vs. Microkernel Isolates hard real-time tasks Libraries IPC, Resource Allocation (not scheduling), I/O Example: OKL4 Used in e.g. Mobile Phones for the Android Platform Virtual Environment for other paravirtualized RTOSs IPC Process server Drivers Kernel space Microkernel Hardware 11.12.2012 Real-time Operating Systems 19

Exokernels User space Applications Two basic principles: Exokernel safely exports resources Anyone can manage resources Libraries liboses Kernel space Exokernel Low-level interface to hardware provided by liboses (e.g., for networking, disk access, memory management) Hardware http://pdos.csail.mit.edu/exo/ 11.12.2012 Real-time Operating Systems 20

Partioning RTOS RTOS can partition resources in time and space Spatial Partitioning: Memory protection: one partition cannot corrupt the memory of the software in another partition Temporal Partitioning: Shared resources: each partition enjoys a certain amount of time on the resource, regardless of what other software is doing. Enables the ability to run mixed-criticality applications

Mixed Criticality Systems

Definition: Mixed criticality A mixed-critical system is an integrated suite of hardware, operating system and middleware services and application software that supports the execution of safety-critical, mission-critical, and noncritical software within a single, secure compute platform. Applications at different levels of criticality can interact and coexist on the same computational platform. Mixed criticality systems realize the principle of non-interference 11.12.2012 Real-time Operating Systems 23

Safety Integrity Levels (SIL) SIL provide rigorous requirements concerning Design, Verfication, Certification, Dependability, Security SIL are a qualitative measure of the required protection against software or system failure SILs provide guidance in selecting appropriate techniques and measures for safety related software development. Standards: e.g., IEC 61508 Safety Integrity Levels (SIL) SIL Failures per hour MTTF (years) 4 10-5 to 10-4 10 5 to 10 4 3 10-4 to 10-3 10 4 to 10 3 2 10-3 to 10-2 10 3 to 10 2 1 10-2 to 10-1 10 2 to 10 Verification Technique SIL1 SIL2 SIL3 SIL4 Formal Proof R R HR Probabilistic testing R R HR Static analysis R HR HR HR Dynamic analysis and testing R HR HR HR 11.12.2012 Armin Wasicek Software complexity metrics R R R R 24

Benefits of Mixed-criticality systems Integrated platforms use less physical components Reduction in Cost Weight Power Wiring Common interface for multiple software providers "For size, weight, and power (SWaP) reasons, avionics manufacturers were including more and increasingly complex functions; instead of having a new box on the aircraft every time they added something like that, they wanted to integrate different types of applications onto the same central processing unit (CPU). 11.12.2012 Real-time Operating Systems 25

APPLICATION A APPLICATION B APPLICATION B APPLICATION B APPLICATION C Possible system architecture Mix applications with different levels of criticality on the same computing platform Segregate subsystems in execution compartments Use e.g. a partitioned OS to shield compartments in time and space (PARTITIONED) RTOS COMPUTING PLATFORM 11.12.2012 Real-time Operating Systems 26

Mixed Criticality Challenges Certification of reusable components: Construct a system from components with known levels of certifiability. Develop a certification process and composition framework Certification by Design: Identify and assess the certifiability of a component, even before it is implemented, Formal and analytical methods to define a design process for certifiability Reconfigurable Systems: It is inherently difficult to bind the decision space of a reconfigurable system under a certain fault model Fault Tolerant Systems: Generic approaches to implement fault tolerance are extremely hard It is even harder to define an approach to certify such systems.

Example 1: UAV UAVs are software-centric: Flight control system Ground control system Multiple Criticality levels in flight software: Common CPU(s) due to size, weight, and power (SWAP) constraints Increasing certification requirements would drive separating criticalities Demand for demonstration of higher assurance for more critical functions 11.12.2012 Real-time Operating Systems 28

UAV criticality levels (examples) Level Flight Critical Mission Critical Maintenance Critical Required to prevent Unintended loss of life. Unintended reduction of public safety. Substantial damage to the vehicle or its support structure. Loss of vehicle Unintended event (e.g., crash) into politically sensitive area Unintended loss of the vehicle s function (mission sensors or other effectors). Unintended loss of support structure function (e.g., ability of operator to use on-board UAV sensors). Inefficient use of the vehicle s resources. Inefficient use of ground support structure. 11.12.2012 Real-time Operating Systems 29

Example 2 ARINC 653 An Avionics Standard for Safe, Partitioned Systems ARINC 653 defines a standardized Real-Time Operating System (RTOS) interface for avionics equipment Called: APplication EXecutive (APEX) for space and time partitioning ARINC 653 approach allocates a predefined faction of CPU time and memory of the whole system to each partition Key enabler in the development of Integrated Modular Avionics (IMA). It recognizes the RTOS as key component of an IMA system. Products: PikeOS, INTEGRITY, VxWorks, LynxOS, etc-

http://www.computersociety.it/wpcontent/uploads/2008/08/ieee-ccarinc653_final.pdf Real-time Operating Systems 11.12.2012 31

http://www.computersociety.it/wpcontent/uploads/2008/08/ieee-ccarinc653_final.pdf Real-time Operating Systems 11.12.2012 32

http://www.computersociety.it/wpcontent/uploads/2008/08/ieee-ccarinc653_final.pdf Real-time Operating Systems 11.12.2012 33

Example 3: Synthetic Vision Real-time Operating Systems 11.12.2012 34

The RTOS Market

Cost-Benefit ratio of an RTOS Average number of software developers per project is 25.7 Average cost per engineer is about $150,000 per year Average cost for each month of delay is $321,250 In average 42% of embedded designs are behind schedule by an average of 4.1 months Average expected loss due to design delays per project: ($321,250) * 0.42 * 4.1 = $553,200 Cost of a commercial RTOS ~$1k-20k

Comparison RTOS No OS http://www.rtos.com/pdfs/emfwhitepaper-rtosselection.pdf

RTOS market shares Trend towards commercial RTOS Roll their own" has steadily declined In-house OS projects decline Increase of commercial RTOS is fueled by the rise of 32-bit microprocessors Ratio drops from 55% for 32-bit users to 19% for 8-bit controllers http://www.eetimes.com/discussion/other/4025539/embedded-systems-survey-operating-systems-up-for-grabs 11.12.2012 Real-time Operating Systems 38

Market data for 2004 on tools and services Characteristics Market (2004) Embedded OSs, bundled tools, related services SW development tools, related services Design automation tools, related services Test automation tools, related services Market volume in million Market growth Largest consuming region and its share 712.6 20.9% Americas 52.7% 195.1 1.8% Americas 48.2% 275.6 n.a. Americas n.a. 65.7 19.8% Americas 50.8% Largest consuming sector and its share Consumer Electronics 41.7% n.a. Military / Aerospace, n.a. Military / Aerospace, 27.6% 1,249.0 17.0% (weighted average)

Linux as RTOS 11.12.2012 Real-time Operating Systems 40

Linux as real-time operating system (1) Predictability of real-time task execution Pre-2.6 scheduling algorithm was O(n), so it could delay a even a high priority task s start time. (Since 2.6 O(1) scheduler introduced: time to schedule is both fixed and deterministic regardless of the number of active tasks) Kernel cannot always be preempted. Sometimes it requires exclusive access to resources and internal data structures in order to maintain their consistency Interrupts from hardware may delay a task

Linux as real-time operating system (2) Virtual memory management introduces indeterministic delays (jitter). Real-Time applications may not use the virtual memory Clock granularity of 1ms insufficient for most real-time apps (e.g., a typical control loop task executes with 100 Hz = every 10ms)

The Real-Time Preemption Patch Makes Linux better suited to real-time systems Real-Time Scheduling In the RTPreempt, RT-Posix fixed priority scheduling. There is also an EDF patch for Linux. Critical sections in kernel are preemptable They now use real-time mutexes, including a priority inheritance protocol Interrupt handlers are interruptable kernel threads High-precision timer support Not suitable for safety-critical hard real-time [SCHED_DEADLINE, http://www.evidence.eu.com/content/view/313/390/]

The Dual-kernel Approach Solutions to run hard-real time tasks and Linux: Real Time Linux (RTLinux) Real-Time Application Interface (RTAI) Xenomai 11.12.2012 Real-time Operating Systems 44

RTLinux & RTAI Dual-kernel approach, two schedulers are in operation Supports hard real-time (deterministic) operation through interrupt control between the hardware and the operating system. (Adeos Event Pipeline) Linux runs as its lowest-priority process Real-time applications specifically written for the non- Linux kernel using an associated real-time API Can exchange data with Linux applications (LXRT)

RTLinux & RTAI Tasks are executed inside kernel memory space, which prevents threads to be swapped-out and also the number of TLB misses is reduced. Threads are executed in processor supervisor mode (i.e. ring level 0 in i386 arch), have full access to the underlying hardware. Since the RTOS and the application are linked together in a "single" execution space, system call mechanism is implemented by means of a simple function call (default is software interrupt which produces higher overhead).

Xenomai RTAI spin-off, designed for extensibility portability maintainability Task can switch transparently Primary mode (hard RT) Secondary mode (soft RT) Skins can emulate RTOS APIs 11.12.2012 Real-time Operating Systems 47

Conclusion 11.12.2012 Real-time Operating Systems 48

RTOS Trends Mil-aero systems require more and more functionality often found in traditional desktop operating systems rather than traditional real-time operating systems, including graphical user interfaces (GUIs) and graphics, advanced networking, and disk and SSD storage Robert Day, vice president of marketing at LynuxWorks Inc. Security of network, security of information and data, and access controls and secure logins are now often specified as part of the functionality of the RTOS and its associated stacks. Separation kernels, partitioned operating systems, and secure operating systems are now being mandated for programs. Robert Day, vice president of marketing at LynuxWorks Inc. Real-time Operating Systems 11.12.2012 49

Conclusion RTOS provide guarentees for embedded application developers RTOS can significantly reduce effort for embedded software development RTOS act as means to implement mixed-criticality systems Are readily available as open source, research or commercial products 11.12.2012 Real-time Operating Systems 50

ENDE Danke für die Aufmerksamkeit! 11.12.2012 Real-time Operating Systems 51

Demonstration: PWM signal generation See http://veter-project.blogspot.co.at/2011/09/real-timeenough-about-pwms-and-shaky.html 11.12.2012 Real-time Operating Systems 52