(Advanced Topics in) Operating Systems Winter Term 2008 / 2009 Prof. Dr. André Brinkmann Andre.Brinkmann@uni-paderborn.de Universität Paderborn PC²
Organization Schedules: Lectures: Thursday 9:00 11:00 am Exercises: Thursday 8:15 9:00 am Module Assignment: Module III.3.1 or Module III.3.2 Perhaps together with IRTOS in III.3.6 A. Brinkmann
Overview Overview of chapter 2 2.1 Services offered by an operating system...4 2.2 History of operating systems... 7 2.3 Various kinds of opering systems.22 2.4 Structures of operating systems... 32 A. Brinkmann
Concepts of General Purpose OS User 1 User 1 User 1 Operating System: Services and control Application Application Application Operating System: Management and operation Hardware A. Brinkmann 4
Services to be offered by an OS Three basic classes of tasks: Utilities for application programs (Provide easy access to the computer) Providing a more abstract view of the HW structure (Hide away details which are irrelevant for programmers) Allocation and coordination of available resources to competing, concurrently working users (resolve conflicts due to competing demands) A. Brinkmann 5
Functions of an OS Interrupt handling Dispatching (context switching) Resource management (allocation, releasing, and operation of resources, tools for process synchronization) Program allocation (linkage of partial programs, loading and unloading of programs to/from main memory) File management Organization of secondary memory by means of files Functions to store, modify, and read information Job control (Decisions concerning the sequence of received jobs) Reliability Reaction on failures of HW or SW Providing correctness, robustness and fault tolerance A. Brinkmann 6
2.2 History of Operating Systems Evolution of operating systems closely related to evolution of computer architecture: First generation (1945 1955) Programming in machine code Operating systems and programming languages unknown Second generation (1955 1965) More reliability by use of transistors Distinction of roles: designer, manufacturer, operator, programmer Programs on punched cards Batch processing Manually: operator loads new job upon termination of the old one Automatic: small cheap computer reads jobs, stores them on magnetic tape to be used as input to mainframe A. Brinkmann 7
History of Operating Systems Third Generation (1965 1980) Integrated circuits Existence of two, incompatible architecture classes: Word-oriented architectures for numerical calculations in science and technology Byte-oriented architectures for report generation in commercial applications (bancs, insurances, ) Portability problems IBM System/360: Family of SW-compatible computers, same architectures, unifying word- and byte-oriented computing Consequences: Need of efficient and universal system software Incompatible requirements result in large, complex, and error-prone operating systems A. Brinkmann 8
Upcoming Software Crisis Operating systems are becoming large, complex, error-prone Request for Structured system design Maintainability Reliability Protection and security Modular programming Abstract Data Types and (later) Object-orientation Consequences: Use of higher programming languages for OS implementation Concept of Process to provide Protected context Own protected address space Own protected capabilities A. Brinkmann 9
Introduction of Important Key Technologies Multiprogramming Multiple jobs are kept in MM Jobs waiting for I/O are blocked and other jobs stored in MM are executed HW-mechanisms to protect job partitions in MM Job 1 Job 2 Job 3 OS Memory partition Time Sharing: Introduction of time slices CTSS (Compatible Time Sharing System) Developed by MIT 1962 Commercially introduced after integration of HW protection mechanisms for concurrent jobs (IBM) A. Brinkmann 10
Some More Key Technologies I/O processors allow real parallel computing Spooling (Simultaneous Peripheral Operation On Line) Jobs are copied from punched cards to disks and executed as soon as possible Virtualization of the processor by means of Processes Introduction of virtual memory Use of processes also for internal structuring of operating systems A. Brinkmann 11
Introduction of UNIX UNIX as simple is beautiful version of MULTICS MULTICS (MULT-iplexed Information and Computing System) System developed by Bell Labs, MIT, and General Electric Basic idea: copy electricity distribution (when electricity is needed: plug in) Here: when computing is needed: plug in Consequence: MULTICS was intended to serve all citizens of Boston Bell Labs and GE leave the project, project continued partially by MIT MULTICS developer Ken Thompson (Bell Labs employee) looked for new challenge developed UNICS for PDP-7 mini computer Received support by his entire department at Bell Labs. Developed what became UNIX A. Brinkmann 12
Dennis Ritchie (left) and Ken Thompson at PDP-11 System Source:http://www.psych.usyd.edu.au/pdp-11/real_programmers.html A. Brinkmann 13
Introduction of C-Language Intent at Bell Labs: Porting UNIX to a couple of other machines. Cumbersome when done in Assembler language. Adequate higher programming language needed. In those days used for experimental work: BCPL resp. simpler version, called B. Turned out to be not sufficient. Consequence: Ritchie develops C and together with Thompson reimplements UNIX in C C becomes standard in system programming AT&T decided not to enter computer business UNIX was distributed to universities incl. source code for a nominal fee. UNIX in those days: 8200 lines of C-code + 900 lines of Assembler code. A. Brinkmann 14
UNIX derivates Companies obtained source code licenses to develop own versions of UNIX (e.g. XENIX by start-up Microsoft). Berkely UNIX: Berkeley Software Distribution (BSD) Enhancements concerning virtual memory, paging, file system, reliable signal handling, inclusion of TCP/IP Utilities like vi, csh, compilers for Pascal and Lisp SUN, DEC etc. based their UNIX versions on BSD instead of the official UNIX V by AT&T Application areas: academics, research, defense Standardization project POSIX System calls from intersection of BSD and System V IEEE Standard 1003.1 Related documents standardize Threads, utilities, networking, ANSI/ISO standardized C. A. Brinkmann 15
UNIX and MINIX UNIX versions did again split into OSF (IBM, DEC, HP, ) UI (AT&T, ) See http://www.levenez.com/unix/ Tanenbaum develops MINIX as UNIX for education Micro kernel offers minimal functionality 1600 lines of C-code + 800 lines of assembler code Rapidly growing developers community Design goal restricts MINIX development. A. Brinkmann 16
Linux Linus Torvalds (Univ. Helsinki) creates Linux 0.0.1 in 1991 Monolithic approach, originally only for Intel 386 Version 1.0 in 1994, 165.000 lines of C-code Compatible to UNIX standard Version 2.0 in 1996, 470.000 lines of C-code Freely available (GNU Public License) Evolved to mature, widely used OS A. Brinkmann 17
A high impact Email see also http://www2.educ.umu.se/~bjorn/mhonarc-files/obsolete/ A. Brinkmann 18
History of Operating Systems Forth Generation Development of micro-computers and PCs Mass deployment of operating systems CP/M (Control Program for Microcomputers) Disk-oriented OS for Intel 8080 Dominated micro-computer market for appr. 5 years MS-DOS (Microsoft Disk Operating System) Based on DOS from Seattle Computer Products Delivered by IBM bundled with IBM PC (this marketing strategy caused the break-through) MAC OS: Introduction of GUI and user-friendly interfaces UNIX extended by GUI as well, based on X-Windows (MIT) Evolved in follow-ups, e.g. Motif A. Brinkmann 19
More about Forth Generation Key technologies: network operating systems, distributed operating systems Powerful communications media allow distributed systems Operating systems go beyond limits of computers: from computer communication to real distributed systems ( the network is the computer ) Integration pushes standardization (OSI, TCP/IP, NFS, POSIX, OSF, ) UNIX becomes de facto standard for such systems Additional trends: Introduction of lightweight processes (Threads) Parallel programming included in programming languages Distributed and parallel computing on networks of PCs A. Brinkmann 20
Actual Trends Highly parallel systems Performance explosion and cost reduction of micro processors massively parallel computer clusters Well structured, well adapted system software crucial for fully making use of performance OS extensions concerning parallel execution, load balancing, Multi media (support of pictures, audio/video streams) Embedded systems (customized, efficient, reliable OS, RTOS) Interoperability Distributed Systems on heterogeneous environments Emulation of other OS, multiple OS on one computer A. Brinkmann 21
6.3 Kinds of Operating Systems Mainframe OS Originally used in large computing centers Today: Web-/E-commerce Server, B2b platforms Optimized for large I/O bandwidth, allowing to execute many processes concurrently Various modes of operation: Batch processing, e.g. insurance cases Transaction processing, e.g. money transfers, flight reservations Timesharing for central services Example: IBM OS/390 (follow up of OS/360) A. Brinkmann 22
Server OS One level below mainframes others Serve many users via network, distribute HW and SW resources Usually used in file, mail, web, or printer servers Examples: UNIX, Linux, Windows Market share of server OS in 2001 (source: IDC, Computerzeitung) A. Brinkmann 23
Multiprocessor OS Low-level parallelism gains importance (servers, workstations, PCs) Need for customized server operating systems extended by capabilities for Communication Interoperability Still deficits in supporting the increased HW performance (e.g. scalability of Linux still debatable) A. Brinkmann 24
Network OS Each computer uses own OS, knows locally all users Users have knowledge about existence of the network Users may register at remote computers To copy data To use services A. Brinkmann 25
Distributed OS Underlying HW architecture (network of computers) invisible to the user. User does not know Location of program Location of data storage. User has impression of a uniform computer resource. A. Brinkmann 26
Comparison Distributed/Network/Multiprocessor OS Property Network Distributed Multiprocessor Behaves like virtual CPU? no yes yes Same OS for all nodes? no yes yes No. of copies of the OS n n 1 Implementation of communication shared files message passing shared memory Shared network protocols? yes yes no Single process queue? no no yes A. Brinkmann 27
PC Operating Systems PC operating systems are general purpose OS with some specific requirements: User-friendly interface Simple to install and maintain Support of a variety of peripheral devices Simple development of applications Upwards compatible A. Brinkmann 28
Real-time OS (RTOS) Applied to monitoring systems, fabrication processes,. Primary requirement: Sensing, analysis, computation, and forwarding of data has to be provided within strict time limits Classification into Hard real-time (catastrophic effects if deadlines are missed) Soft real-time (quality reduced if deadlines are missed) Examples: VxWorks, QNX A. Brinkmann 29
OS for Embedded Systems and PDAs In many cases real-time properties Additional optimization towards Small footprint Simple program logics Low energy consumption Used also in devices controlling other devices General purpose OS available for PDAs: PalmOS Windows CE Symbian A. Brinkmann 30
Chip Card OS Increasing importance of SmartCards Hard restrictions concerning memory and computing power In most cases a single service is provided (e.g. payment) JAVA-oriented SmartCards, JVM in ROM Different functions implemented as Applets implicit multiprogramming Need for efficient resource management and protection mechanisms A. Brinkmann 31
2.4 Structures of Operating Systems Design structures for Operating Systems: Monolithical systems Layered architectures Virtual machines Exokernal Microkernal (Client-Server architectures) A. Brinkmann 32
Monolithic OS Application System Interface Operating System Address space Application task OS service A. Brinkmann 33
Monolithic OS Most used approach Large amount of functions Each function provides well defined interface for parameters and results Each function can call each other function All functions are compiled together into a single object code. Functions (services) are invoked by deposing parameters in system interface and causing a system trap. Trap causes switch into system mode, OS identifies request and serves this request. A. Brinkmann 34
Monolithic OS: Service Calls User programs executed in user mode (user address space) System call means switch from user mode to kernel mode After service completion return to user mode User program 1 System call User program 1 User mode Service program Kernel mode Dispatcher table Special register of processors points to start of dispatcher table A. Brinkmann 35
Monolithic OS: Structuring Global structure: Main program: calls the services Services: execute service calls Utilities: support services and provide basic mechanisms like copying data from user program Main program Services Utilities A. Brinkmann 36
Layered OS A. Brinkmann 37
Virtual Machines Basic idea: Model the underlying HW allowing more than one OS to run concurrently on a single computer First system: IBM VM/370, consisting of two components: Monitor of virtual machine: executed on real HW, implements multiprogramming Multiple virtual machines on top of monitor, with exact copies of HW with kernel and user mode I/O and interrupts Additional properties of the real machine System calls are trapped by virtual machine Real execution of system calls takes place on real HW under control of the monitor at HW level Clear distinction of these two layers makes design of both components easier A. Brinkmann 38
VM/370: Basic Principles Virtual machines I/O request OS 1 OS 2 OS 3 System call More recent implementations: Virtual 8086 mode for Pentium processors to execute MS-DOS programs JVM (Java Virtual Machine) Port to many architectures Interprets the output of a Java-compiler Important benefit: running programs can be monitored concerning security. A. Brinkmann 39
Exokernel Combination (MIT) of 8086 virtualization and VM/370 Each user gets a copy of the underlying computer, however only with a (fixed) fraction of the resources Example: 1st virtual machine gets blocks 0-1023 from hard disk 2nd virtual machine gets blocks 1024-2047, On lowest level in kernel mode the Exokernel is running, makes sure that no VM makes use of resources of another one Advantages Saving an intermediate level, as no dynamic mapping tables from resources to VMs necessary Exokernel only needs to know static mapping resource <-> VM Disadvantage: Vast of resources due to static mapping A. Brinkmann 40
Microkernel and Client-Server Model Trend in he last 15 years: Migration from monolithic OS to fine-grain structured ones with floating boundaries to applications. It remains jus a tiny kernel (Microkernel) OS services migrate to user mode Communication between OS services and user applications following client-server model by message passing Run-time model: Totality of the services offered to an application Questions to be answered: What services are part of run-time model? Which set of services is always mandatory, i.e. what is part of the microkernel? How the underlying system architecture looks like? A. Brinkmann 41
Microkernel Applications OS tasks (services) System interface Microkernel Address space Application task OS service A. Brinkmann 42
Microkernel: Examples Mach: developed mid 80s at Carnegie Mellon University. Tru64 (previously DigitalUnix): based on Mach-kernel, maps UNIX system calls to messages to respective services. MacOS X: based on Mach kernel as well. RTOS QNX: Microkernel contains services for message passing, lowlevel communication, HW interrupts. Strict microkernel not always possible, Example WindowsNT Hybrid structure: partly layered approach, each application type (Win32, OS/2, Posix) has own server in user space. Kernel coordinates message passing. A. Brinkmann 43
Client-Server Model: Advantages Result from partitioning of OS in units like file server, process server, terminal server, Efficient implementation of kernel becomes easier Services are processes in user mode without direct access to HW ==> malfunctions may cause individual services to fail, overall system may remain functional Adaptability in case of distributed systems: computer boundaries may be bridged without creating a distinct version for distributed computing. Problems: Some services are implementable only in kernel mode Service calls may become less efficient (more context switches) Proper distinction of mechanisms (kernel) and policies (services in user space) necessary. A. Brinkmann 44
Further Information PC² homepage http://www.upb.de/pc2 A. Brinkmann 45