Grundlagen der Betriebssystemprogrammierung

Grundlagen der Betriebssystemprogrammierung Präsentation A3, A4, A5, A6 21. März 2013 IAIK Grundlagen der Betriebssystemprogrammierung 1 / 73

1 A3 - Function Pointers 2 A4 - C++: The good, the bad and the ugly 3 A5 - Multithreading 4 A6 - Shell Grundlagen der Betriebssystemprogrammierung 2 / 73

Course Overview A10, A11, A12 OS Topics A7, A8, A9 Synchronization, Thread and Process Interaction A4, A5, A6 Multithreading, more C, more C++ A1, A2, A3 Git, Compiler, C, C++ Grundlagen der Betriebssystemprogrammierung 3 / 73

About Function Pointers Facts Pointer store the address of an object in memory. Integer pointers store the address of an integer. Function pointers store the address of a function. A function call includes a jump to an address (and back). This address is stored in a function pointer variable. Grundlagen der Betriebssystemprogrammierung 4 / 73

Function Pointer Syntax Declaration int (* my_main )(int, char **) = & main ; int (* my_main )(int, char **) = main ; void *(* start_ routine ) ( void *) = my_ thread ; The address-of operator & is optional. The parenthesis are important! The type must be correct! (Argument type & number, return type) Grundlagen der Betriebssystemprogrammierung 5 / 73

Function Pointer Syntax Usage int returnvalue = main ( argc, argv ); void * returndata = start_ routine ( input ); Like a statically defined function. Grundlagen der Betriebssystemprogrammierung 6 / 73

A3 - Function Pointers Task Overview Given: a source file with a static array and bubble sort implementation. main calls qsort (form C-library) and bsort to sort static array. void qsort ( void * base, size_ t num, size_ t size, int (* compar )( const void *, const void *)); sortandprintarray takes sort-function and compare-function to sort array. Define (typedef) two function pointer types for the sort and the compare function. Complete the bsort function. Call the appropriate sort function in sortandprintarray. Grundlagen der Betriebssystemprogrammierung 7 / 73

A3 - Function Pointers Task Overview Given: a source file with a static array and bubble sort implementation. main calls qsort (form C-library) and bsort to sort static array. void qsort ( void * base, size_ t num, size_ t size, int (* compar )( const void *, const void *)); sortandprintarray takes sort-function and compare-function to sort array. Define (typedef) two function pointer types for the sort and the compare function. Complete the bsort function. Call the appropriate sort function in sortandprintarray. Grundlagen der Betriebssystemprogrammierung 8 / 73

A3 - Function Pointers Task Overview Given: a source file with a static array and bubble sort implementation. main calls qsort (form C-library) and bsort to sort static array. void qsort ( void * base, size_ t num, size_ t size, int (* compar )( const void *, const void *)); sortandprintarray takes sort-function and compare-function to sort array. Define (typedef) two function pointer types for the sort and the compare function. Complete the bsort function. Call the appropriate sort function in sortandprintarray. Grundlagen der Betriebssystemprogrammierung 9 / 73

First Steps Pull upstream Read the wiki page. Learn about function pointers. Implement all lines marked with TODO. Grundlagen der Betriebssystemprogrammierung 10 / 73

Revisiting C++ Some thoughts about C++ In C++ it s harder to shoot yourself in the foot, but when you do, you blow off your whole leg. - Bjarne Stroustrup Grundlagen der Betriebssystemprogrammierung 11 / 73

Revisiting C++ Outline References and Pointers Templates Operator overloading STL and Containers RAII A4 - Revisiting C++ Grundlagen der Betriebssystemprogrammierung 12 / 73

References and Pointers What is a pointer? We know that already! int n = 41; int * ptr = &n; * ptr ++; // now n is 42 But what is a reference? Simplified: behaves like a pointer, but is syntactically used like a simple copy. int n = 42; int & ptr = n; ptr ++; // now n is 42 Grundlagen der Betriebssystemprogrammierung 13 / 73

References and Pointers What is a pointer? We know that already! int n = 41; int * ptr = &n; * ptr ++; // now n is 42 But what is a reference? Simplified: behaves like a pointer, but is syntactically used like a simple copy. int n = 42; int & ptr = n; ptr ++; // now n is 42 Grundlagen der Betriebssystemprogrammierung 14 / 73

References and Pointers What is a reference? more precisely: an alias for an object In contrast to pointers... no pointer-arithmetic each reference has to be initalised with a valid object no null-pointer for references Grundlagen der Betriebssystemprogrammierung 15 / 73

References and Pointers What is a reference? more precisely: an alias for an object In contrast to pointers... no pointer-arithmetic each reference has to be initalised with a valid object no null-pointer for references Grundlagen der Betriebssystemprogrammierung 16 / 73

References and Pointers Call by reference Changing an object by a function / method bool loadplayers ( std :: vector < Player * >& vec ); performance reasons: just pass a reference instead of copying the whole object. void displayplayers ( const std :: vector < Player * >& vec ) const ; const is important! Grundlagen der Betriebssystemprogrammierung 17 / 73

References and Pointers Call by reference Changing an object by a function / method bool loadplayers ( std :: vector < Player * >& vec ); performance reasons: just pass a reference instead of copying the whole object. void displayplayers ( const std :: vector < Player * >& vec ) const ; const is important! Grundlagen der Betriebssystemprogrammierung 18 / 73

Type Casting Implicit conversion int n = 1024; long m = n; // implicit type cast Explicit conversion dynamic cast static cast reinterpret cast C-style cast Grundlagen der Betriebssystemprogrammierung 19 / 73

Type Casting dynamic cast class Base { virtual void do () {} }; class Derived : public Base { }; Base * base = new Derived (); Derived * d = dynamic_cast < Derived * >( base ); works only with pointers and references to objects requires Run-time type information (RTTI)...... and a polymorphic object as operand used for downcasts (see example) runtime overhead (safe but slow) Grundlagen der Betriebssystemprogrammierung 20 / 73

Type Casting static cast checks done at compile time no runtime checks! be careful with casts between pointers to related classes! static cast performs conversions between... pointers to related classes. enums and integral types. floating point values and integral types. used for downcasts (see example) Grundlagen der Betriebssystemprogrammierung 21 / 73

Type Casting reinterpret cast converts any pointer to any pointer type reinterprets the given bits in a new way no checks done always possible results are system-specific and therefor not portable remember what Bjarne Stroustrup said! however sometimes it is necessary Low level code headers in a binary file data structures in file systems... Grundlagen der Betriebssystemprogrammierung 22 / 73

Type Casting reinterpret cast converts any pointer to any pointer type reinterprets the given bits in a new way no checks done always possible results are system-specific and therefor not portable remember what Bjarne Stroustrup said! however sometimes it is necessary Low level code headers in a binary file data structures in file systems... Grundlagen der Betriebssystemprogrammierung 23 / 73

Type Casting reinterpret cast low-level example struct minix_ super_ block { uint16 s_ ninodes ; uint16 s_ nzones ; uint16 s_ imap_ blocks ; uint16 s_ zmap_ blocks ; uint16 s_ firstdatazone ; uint16 s_ log_ zone_ size ; uint32 s_ max_ size ; uint16 s_ magic ; uint16 s_ state ; uint32 s_ zones ; }; Grundlagen der Betriebssystemprogrammierung 24 / 73

Type Casting reinterpret cast low-level example char buffer [ MINIX_ BLOCK_ SIZE ]; if (! device -> readsector ( sector, buffer, MINIX_BLOCK_SIZE )) { return false ; } minix_ super_ block superblock = * reinterpret_ cast < minix_ super_ block * >( buffer + offset ); Grundlagen der Betriebssystemprogrammierung 25 / 73

Templates What are templates? (Theory) Take a class/function definition Generalize it by allowing parameterization You have a template (class/function) What are templates? (Practice) Parameterization only works on types (and integral constants) Luckily, you can model (almost) anything using types Grundlagen der Betriebssystemprogrammierung 26 / 73

Templates What are templates? (Theory) Take a class/function definition Generalize it by allowing parameterization You have a template (class/function) What are templates? (Practice) Parameterization only works on types (and integral constants) Luckily, you can model (almost) anything using types Grundlagen der Betriebssystemprogrammierung 27 / 73

Templates, example We have: short max ( short a, short b) { return (a>b)?a:b; } That seems useful! Let s template it: template < typename T > T max1 (T a, T b) { return (a>b)?a:b; } That was easy! Templates are fun! Grundlagen der Betriebssystemprogrammierung 28 / 73

Templates, example We have: short max ( short a, short b) { return (a>b)?a:b; } That seems useful! Let s template it: template < typename T > T max1 (T a, T b) { return (a>b)?a:b; } That was easy! Templates are fun! Grundlagen der Betriebssystemprogrammierung 29 / 73

Templates, template parameters Consider again template < typename T > T max1 (T a, T b) { return (a>b)?a:b; } What assumptions do we make about T? Each T must be greater than -comparable. Compiler will reject any type that does not fulfill this Requirements on template parameters are important documentation! Grundlagen der Betriebssystemprogrammierung 30 / 73

Templates, pitfalls Back to our example. One problem: short a = 3; short b = max ( a, 5); // this works short c = max1 ( a, 5); // this does not prog.cpp:11: error: no matching function for call to max1(short int&, int) Template parameters must match exactly, no conversion is done by the compiler. Workarounds: short d = max1 < short >( a, 5); // Force specific instantiation short e = max1 (a, ( short )5); // Cast integer literal to short Grundlagen der Betriebssystemprogrammierung 31 / 73

Templates, pitfalls Back to our example. One problem: short a = 3; short b = max ( a, 5); // this works short c = max1 ( a, 5); // this does not prog.cpp:11: error: no matching function for call to max1(short int&, int) Template parameters must match exactly, no conversion is done by the compiler. Workarounds: short d = max1 < short >( a, 5); // Force specific instantiation short e = max1 (a, ( short )5); // Cast integer literal to short Grundlagen der Betriebssystemprogrammierung 32 / 73

Templates, ugly pitfalls What about classes? template < typename T > class A { public : typedef std :: vector <T > TVec ; }; template < typename T > class B: public A <T > { public : void fill ( const TVec & elements ); }; prog.cpp:11: error: expected, or... before & token prog.cpp:11: error: ISO C++ forbids declaration of TVec with no type Grundlagen der Betriebssystemprogrammierung 33 / 73

Templates, ugly pitfalls Let s try: void fill ( const A<T >:: TVec & elements ); prog.cpp:11: error: expected unqualified-id before & token prog.cpp:11: error: expected, or... before & token Not much better. What does the compiler think this is, if not a type? Grundlagen der Betriebssystemprogrammierung 34 / 73

Templates, typename A<T>::TVec may not name a type for every T template specialization. Consider: template<int> class A { public: static int TVec; }; If we instantiate B<int>, A<T>::TVec is not a type! Workaround: void fill ( const typename A <T >:: TVec & elements ); Assure the compiler This is a typename!. Grundlagen der Betriebssystemprogrammierung 35 / 73

Templates, typename A<T>::TVec may not name a type for every T template specialization. Consider: template<int> class A { public: static int TVec; }; If we instantiate B<int>, A<T>::TVec is not a type! Workaround: void fill ( const typename A <T >:: TVec & elements ); Assure the compiler This is a typename!. Grundlagen der Betriebssystemprogrammierung 36 / 73

Operator overloading All types used for max1 have to be greater than -comparable. What if we want to use our own class? Operator overloading! class Sortable { public : bool operator >( const Sortable & other ); }; Also useful to provide an intuitive interface to your class (e.g. for (mathematical) vectors, matrices,...). Note: Don t overdo it! Grundlagen der Betriebssystemprogrammierung 37 / 73

Standard containers The C++ standard library provides some container class templates: vector<t> list<t> map<k,t> set<k>... Important: Which requirements do T and K have to fulfill? Grundlagen der Betriebssystemprogrammierung 38 / 73

Container requirements Value types have to fulfill the following: Be copy-constructible Implements assigment operator Some operations also require: Be default-constructible Key types have to fulfill the following: Be copy-constructible Implements assigment operator Be less than -comparable (Ordering has to be a strict weak ordering) Some operations also require: Be default-constructible Grundlagen der Betriebssystemprogrammierung 39 / 73

Container requirements Value types have to fulfill the following: Be copy-constructible Implements assigment operator Some operations also require: Be default-constructible Key types have to fulfill the following: Be copy-constructible Implements assigment operator Be less than -comparable (Ordering has to be a strict weak ordering) Some operations also require: Be default-constructible Grundlagen der Betriebssystemprogrammierung 40 / 73

Iterators Containers define iterator types. These behave like pointers into the container void myfunc ( std :: vector <int > vec ) { for ( std :: vector <int >:: iterator it = vec. begin (); it!= vec. end (); ++ it) { std :: cout << (* it) << "\n"; } } Grundlagen der Betriebssystemprogrammierung 41 / 73

Iterators, why? Consider the following task: Provide a function to print an arbitrary number of integer values in a consistent format. What s better? void printvaluesptr ( int * ptr, int n); void printvaluesvec ( std :: vector <int > values ); Grundlagen der Betriebssystemprogrammierung 42 / 73

Iterators, why? Answer: Neither! Use templates! template < typename InputIterator > void printvaluestemplate ( InputIterator it, int n) { for ( int i = 0;i < n; ++i, ++ it) std :: cout << " value : " << (* it) << "\n"; } Pointers and iterators support the same interface (increment and dereferencing)! Grundlagen der Betriebssystemprogrammierung 43 / 73

std::map std::map<k,t> is an associative container mapping arbitrary (comparable) K s to T s. std :: map < std :: string, int > mymap ; int x = mymap [" key1 "]; What will happen? Answer: operator[] always returns a reference. If the entry does not exist, it will be created! if ( mymap. find (" key1 ")!= mymap. end ()) x = mymap [" key1 "]; map<k,t>::find(const K&) returns an iterator it will point one past the end (equal to map<k,t>::end()) in case the key is not found. Grundlagen der Betriebssystemprogrammierung 44 / 73

std::map std::map<k,t> is an associative container mapping arbitrary (comparable) K s to T s. std :: map < std :: string, int > mymap ; int x = mymap [" key1 "]; What will happen? Answer: operator[] always returns a reference. If the entry does not exist, it will be created! if ( mymap. find (" key1 ")!= mymap. end ()) x = mymap [" key1 "]; map<k,t>::find(const K&) returns an iterator it will point one past the end (equal to map<k,t>::end()) in case the key is not found. Grundlagen der Betriebssystemprogrammierung 45 / 73

RAII, basics Resource Acquisition Is Initialization - Acquiring and securely releasing resources. Bit of a misnomer: No one cares about acquisition, releasing resources is more important. Basic idea: Constructor acquires resource (file handle, memory, network connection,...), destructor frees it again. When the object is allocated on the stack (i.e. as a local variable), the destructor will be called automatically when the enclosing scope is left. Grundlagen der Betriebssystemprogrammierung 46 / 73

RAII, example # include < memory > int main () { std :: auto_ptr <int > myptr ( new int ); return 0; // Memory is freed automatically! } Too bad auto_ptr is pretty useless (not copy-able). Grundlagen der Betriebssystemprogrammierung 47 / 73

Better memory management using Shared Pointer Idea: Let s count how many references exist for a resource. When this count reaches zero, the resource can be freed. boost :: shared_ptr <int > myfunc () { boost :: shared_ptr <int > result ( new int ); return result ; // would not work with auto_ ptr } int main () { boost :: shared_ptr <int > myptr = myfunc (); return 0; // again, memory is freed automatically } Grundlagen der Betriebssystemprogrammierung 48 / 73

Better memory management using Shared Pointer shared_ptr maintains a reference counter Assignment operator and copy constructor make reference counter shared between different shared_ptrs to the same object Problem: Circular dependencies (reference count will always stay 1) Other problem : Only available in boost and C++11, but basic functionality is easy to implement Grundlagen der Betriebssystemprogrammierung 49 / 73

Placement new operator new does two things: Allocate memory Call constructor on allocated memory What if we only want to do the second? Placement new: new ( ptr ) MyClass (); Call constructor for MyClass on memory pointed to by ptr Potential uses: Memory pools, memory mapped hardware, shared memory But: There is almost always an alternative Grundlagen der Betriebssystemprogrammierung 50 / 73

Placement new operator new does two things: Allocate memory Call constructor on allocated memory What if we only want to do the second? Placement new: new ( ptr ) MyClass (); Call constructor for MyClass on memory pointed to by ptr Potential uses: Memory pools, memory mapped hardware, shared memory But: There is almost always an alternative Grundlagen der Betriebssystemprogrammierung 51 / 73

A4 - Revisiting C++ Task Overview Given: some C++ files. everything compiles fine but does not run... :( fix the code so that the program prodces the provided reference output. DO NOT change existing method signatures or members! You must NOT create any additional output! Just fix the bugs :) Grundlagen der Betriebssystemprogrammierung 52 / 73

A4 - Revisiting C++ Task Overview Given: some C++ files. everything compiles fine but does not run... :( fix the code so that the program prodces the provided reference output. DO NOT change existing method signatures or members! You must NOT create any additional output! Just fix the bugs :) Grundlagen der Betriebssystemprogrammierung 53 / 73

A4 - Revisiting C++ Task Overview Given: some C++ files. everything compiles fine but does not run... :( fix the code so that the program prodces the provided reference output. DO NOT change existing method signatures or members! You must NOT create any additional output! Just fix the bugs :) Grundlagen der Betriebssystemprogrammierung 54 / 73

First Steps Pull upstream Read the wiki page. Search all bugs and fix them. Rember the pitfalls we just discussed. Some of them are hidden in the code. Grundlagen der Betriebssystemprogrammierung 55 / 73

Threads and Processes The operating system.. can run multiple programs simultaneously. switches between running progams and shares the CPU Timesharing. decides when it switches. saves and restores the process context. provides a virtual CPU for every process, i.e. they don t notice that they share the CPU. Grundlagen der Betriebssystemprogrammierung 56 / 73

Threads and Processes A process.. is an instance of an executing program. has it s own virtual address space, separating it from other processes and the OS. has it s set of state variables (e.g. file descriptors) A thread.. is a sequential thread of execution in a program. belongs to a process (a userthread). is scheduled by the OS, i.e. granted CPU time. has it s own context: program counter, cpu registers,.. Grundlagen der Betriebssystemprogrammierung 57 / 73

Threads and Processes A process.. is an instance of an executing program. has it s own virtual address space, separating it from other processes and the OS. has it s set of state variables (e.g. file descriptors) A thread.. is a sequential thread of execution in a program. belongs to a process (a userthread). is scheduled by the OS, i.e. granted CPU time. has it s own context: program counter, cpu registers,.. Grundlagen der Betriebssystemprogrammierung 58 / 73

Threads and Processes When a program is started.. the OS creates a new process. the new process gets one new thread that starts at the program entry point. Multiple threads.. can be created within a process by using a library (that finally tells the OS). POSIX threads, pthreads. Every thread has it s own execution context. But within a process, all threads share the same address space and other process resources like file descriptors and others. Grundlagen der Betriebssystemprogrammierung 59 / 73

Threads and Processes When a program is started.. the OS creates a new process. the new process gets one new thread that starts at the program entry point. Multiple threads.. can be created within a process by using a library (that finally tells the OS). POSIX threads, pthreads. Every thread has it s own execution context. But within a process, all threads share the same address space and other process resources like file descriptors and others. Grundlagen der Betriebssystemprogrammierung 60 / 73

A5 - Multithreading Task Overview Given: a almost complete textmode game. An example for using multiple threads in a simple application. Thread 1 displays the gfx. Thread 2 waits for keyboard input. To do Pull upstream. Read the wiki. Run the second thread Play the game. Submit. Grundlagen der Betriebssystemprogrammierung 61 / 73

A6 - Shell Task Overview Minimal Shell TODOs in the code: 1 Process creation 2 Pipe commands, <, >, etc. First Steps Pull upstream Try make and execute the shell Grundlagen der Betriebssystemprogrammierung 62 / 73

A6 - Shell Process Creation Windows way: CreateProcess Unix way: fork, exec Fork Creates a new process. Shall be an exact copy... Exec Replaces current process image (Tries to) execute the given binary file Grundlagen der Betriebssystemprogrammierung 63 / 73

A6 - Shell Fork Example # include <unistd.h> // pid_t fork ( void ); int pid = fork (); if ( pid == 0) printf ("I m the child \n"); else printf ("I m the parent," " and my child has pid %u\n", pid ); Grundlagen der Betriebssystemprogrammierung 64 / 73

A6 - Shell Exec Example # include <unistd.h> char * const input_argv [] = {" file. txt ", 0}; // int execv ( const char * path, // char * const argv []) ; execv ("/ bin / vim ", input_argv ); printf (" If program flow gets here " " something went terribly wrong!\ n"); exit ( -1); Grundlagen der Betriebssystemprogrammierung 65 / 73

A6 - Shell Process Creation fork followed by an exec? Worse than the CreateProcess approach? No, because of cow: Forked process shares memory with the old process Memory is copied upon write access Almost nothing copied if followed by an exec! But how do i get output of the new process? Pipes! Grundlagen der Betriebssystemprogrammierung 66 / 73

A6 - Shell File Descriptors A file descriptor has type int You know FILE*? FILE struct contains file descriptor (abstraction) fopen open fread read etc. Grundlagen der Betriebssystemprogrammierung 67 / 73

A6 - Shell Pipes a pair of 2 special file descriptors read end and write end # include <unistd.h> int pipefd [2]; // int pipe ( int pipefd [2]) ; pipe ( pipefd ); write ( pipefd [1], " hello world \n", 12) ; // might block char text [13]; read ( pipefd [0], text, 12) ; // will block ( if no input available ) Grundlagen der Betriebssystemprogrammierung 68 / 73

A6 - Shell How to use pipes instead of STDOUT FILENO, etc.? fork/exec leave file descriptors as they are! But how to change STDOUT FILENO to a pipe write end? dup2! int fd = open (" output. txt ", O_RDWR O_CREAT O_TRUNC ); dup2 (fd, STDOUT_FILENO ); printf (" this will be written to output. txt, but NOT to the terminal!"); Grundlagen der Betriebssystemprogrammierung 69 / 73

A6 - Shell How to wait until a process terminated? # include <sys / types.h> # include <sys / wait.h> int pid = fork (); if ( pid == 0) { // do something exit (0) ; } // pid_t waitpid ( pid_t pid, int * status, // int options ); waitpid ( pid, 0, 0); // look at the man page Grundlagen der Betriebssystemprogrammierung 70 / 73

A6 - Shell # define READ_ STDOUT_ FILENO pipe [0] # define WRITE_ STDOUT_ FILENO pipe [1] int pipe [2]; pipe ( pipe ); int pid = fork (); if ( pid == 0) { dup2 ( WRITE_ STDOUT_ FILENO, STDOUT_ FILENO ); close ( READ_STDOUT_FILENO ); // why? execv ( path_, ( char * const *) args_ ); } waitpid ( child, 0, 0); close ( WRITE_STDOUT_FILENO ); // why? read ( READ_ STDOUT_ FILENO, buf, BUFFER_ SIZE ); close ( READ_STDOUT_FILENO ); Grundlagen der Betriebssystemprogrammierung 71 / 73

A6 - Shell Pipes in the shell./prog1 < file.txt 1 Open file.txt 2 Read from file, write into pipe 3 Replace STDIN FILENO of forked process by this pipe (dup2)./prog1 > file.txt 1 Open file.txt 2 Replace STDOUT FILENO of forked process by a pipe 3 Read from pipe, write into file ls less 1 Run ls with STDOUT FILENO replaced by pipe[1] 2 Run less with STDIN FILENO replaced by pipe[0] etc. Grundlagen der Betriebssystemprogrammierung 72 / 73

Thanks for your Attention! Grundlagen der Betriebssystemprogrammierung 73 / 73