Networks and parallelism

Transcription

1 Networks and parallelism CSE-C3200 Operating systems Autumn 2015 (I), Lecture 9 Vesa Hirvisalo

2 Today Middleware Something between the OS and applications SOAP and DDS as the examples Network virtualization Virtual switches NFV and SDN Parallel processing Shared memory systems SMP systems and locking Parallel programming models POSIX Pthreads 2

3 Middleware 3

4 Introduction Middleware is software between apps and OS Especially for making distributed systems Typically the OS is a network OS Supporting networks in a way sufficient for middleware There are several forms and types Data integration Message oriented middleware Object request brokers Enterprise service bus Often communication-oriented Using generic data formats E.g., XML (Extensible Markup Language) 4

5 An example: SOAP A protocol for exchanging structured information Uses XML and application layer protocols E.g., HTTP or SMTP for doing the transfers For implementing Web services Together with UDDI and WSDL Listing and describing the services Three major characteristics Extensibility (e.g., security) Neutrality (does not depend on details of the transport) Independence (allows for any programming model) Has a layered architecture SOAP messages, patterns, bindings, processing and extension 5

6 Data-centric approach Client-Server Who+Where: Space coupling What: Structural coupling When: Time coupling Server Req/Reply Client Publisher E.g.: Web, CORBA, DCOM Publisher Subscriber Pub/Sub What: Structural coupling Encryption & sandboxing Remember the abstractions Integrity, protection, keys,.. E.g.: DDS, TIBCO RendezVous Subscriber

7 Streams Apache Storm as an example A distributed computation framework Provides a set of general primitives for doing real-time stream processing spout On Storm you run "topologies" (DAGs) Consists of Spouts and Bolts Spout is a source of streams Bolts consume streams do processing and possible emit new streams Edges are named streams directing data from a node to another bolt bolt bolt Have similarity to MapReduce, but data is processed real-time bolt

8 Network virtualization 8

9 Virtual switches Switches are fundamental for network operation Traditionally, physical ports and connections are used Ports are implemented by network interfaces In virtual switches, the ports are virtualized The underlying physical network is hidden For example Open vswitch Can be run within a virtual machine or as the control stack as dedicated switching hardware Linux has support for this Note the difference to middleware Run on top of a hypervisor Implement the basic network 9

10 NFV and SDN (1/2) Network-Function Virtualization (NFV) A network architecture that relies on virtualization Originated from data centers Basically all the network functionality is virtualized Switches, routers, firewalls, balancers, etc. Enables use of COTS HW instead dedicated ones Software Defined Networking (SDN) Actually, pretty much the same thing Originates from the protocol world Control the network using software interfaces Separation of the dataplane and control plane With NFV the control becomes middleware 10

11 NFV and SDN (2/2) 11

12 SR-IOV and MR-IOV Peripheral devices are on busses The bus technology can support virtualization Also for network interfaces (NICs) Connecting network virtualization and I/O virtualization SR-IOV (Single-Root I/O Virtualization) Allows isolation of PCIe resource A single physical PCIe can be shared on a virtual environment Based on mechanisms resembling typical virtual memory support Translation, function, and transfer support (ATC, ARI, ATS) MR-IOV (Multi-Root I/O Virtualization) For complex bus topologies E.g., for blade servers Hardware acceleration Let the hardware do the basic I/O operations 12

13 Parallel processing architectures 13

14 Operating system architectures Architecture means the visible structure Not the implementation or the organization inside The architecture at one level can be the implementation at an other level Some examples (from elsewhere) Layered architectures Database, application logic, user interface Client-server architecture Operating system architecture Monolithic systems Layered systems Virtual machines Client/server a.k.a. microkernels 14

15 Multiple processors and cores There are several ways of organizing parallelism Multiprocessing, several processing units in a system Multiprocessor, tightly coupled units with a shared memory A hardware that allows multiprocessing In multicore, the units are cores CMP, (Single-)Chip Multi-Processor Multiprogramming, multiple concurrent processes Multitasking, multithreading, etc. Today, we concentrate on multiprocessing on a CMP This is about operating systems Based on middleware Multicomputers, e.g., clusters of computers Distributed multiple computer systems, e.g., grids of computers Based on compilers Vector parallelism 15

16 Shared memory Shared memory multiprocessor A single address space visible to all processing elements Uniform Memory Architecture (UMA) The memory is accessed via a shared interconnect Access time differences not processing element specific Non-Uniform Memory Architecture (NUMA) Processing elements have local memory Memory of others accessed via interconnect Memory consistency model Considers writes and reads What properties the memory system guarantees Cache coherency The hardware mechanisms to implement the consistency 16

17 Parallel processing The classical categorization SISD (Single Instruction, Single Data) SIMD (Single Instruction, Multiple Data) MISD (Multiple Instruction, Single Data) MIMD (Multiple Instruction, Multiple Data) The classical categorization is used, but Is obsolete in many ways E.g., is instruction pipeline really a MISD mechanisms? A lot of new names MPMD (Multiple Program, Multiple Data) SWAR (SIMD Within a Register) SIMT (Single Instruction, Multiple Thread) SMT (Simultaneous Multi-Threading) 17

18 Symmetric multiprocessing (SMP) Having a single OS manages all processor cores simultaneously can dynamically schedule any thread/process on any core Advantages provides scalability and parallelism than AMP simpler shared resource management Disadvantages The underlying hardware must support SMP Note SMP refers also to a hardware architecture where multiple processors share a single address space (Symmetric Multi- Processor) 18

19 Asymmetric multiprocessing (AMP) A single OS does not manage across all cores A separate OS, or a separate copy of the same OS pre core Typically, each software process is locked to a single core Advantages Application programming model is similar to uniprocessing Migration of legacy code is easy Developers can own cores Disadvantages Migration of threads/processes (etc.) is hard Note In the past, more about peripheral device management (I/O) Today, mostly about heterogeneous systems 19

20 Systems with accelerators Heterogeneous computer systems Different processing elements Several architectures The typical setup today CPU + (GP)GPU CPU = latency-oriented, GPU = throughput-oriented, data parallel The CPU runs the operating system The GPU can have external access However, now system-wide independent I/O The GPU is handled as peripheral device The OS has a device driver for the GPU Compiler in the processing chain The driver by-passes the kernel in many ways Processing efficiency 20

21 Synchronization in SMP 21

22 SMP problems Shared structures require synchronization between processors Locks cause contention performance degrades when other CPUs must wait Cache thrashing should be avoided Competing over memory is not trivial Interrupts should be balanced between processors Some architectures require explicit synchronization of CPU caches using memory barriers Scheduler must be aware of system configuration schedule tasks on the same CPU (using same data) spread tasks in most efficient way (shared resources) 22

23 SMP locking Short term mutual exclusion Preventing race conditions Access of same data True parallelism on a SMP Short term mutual exclusion with interrupts Critical sections and interrupt handlers collide Disabling interrupts works per processing unit Long term mutual exclusion Slow accesses E.g., accessing file system 23

24 Spinlocks Spinlocks Simple approach to locking uses active waiting and atomic operations Memory is locked during atomic comparison Performance is degraded Cache thrashing and expensive operations (barriers) Barriers keep loads and stores in order RW spinlocks Spinlocks for readers-writers No lock contention for readers Used when writers are rare Higher cost than normal spinlocks when there is contention 24

25 Read-Copy-Update (RCU) Lock-free approach for reader-writer synchronization Based on linked data structured Readers Get reference to some structure Can use it freely until release Writers Copy structure which they want to modify Insert the new structure (needs a mutex) Management Unused structures can be freed Prefers readers No atomic operations or memory barriers for readers Reader data not affected in cache preemption disabling for preemptible kernels 25

26 Locking granularity Whole kernel locking Allow a single processing unit (thread) in the kernel at a time Simple but hampers performance on SMP In Linux, BKL (Big Kernel Lock) A spin lock that could be nested and is blocking-safe Obsolete feature (not the primary mechanisms anymore) Coarse-grained locking Each subsystem protected by an own lock Basically, no cross-subsystem blocking Problems when sharing data between subsystem Fine-grained locking Individual data structures protected by own locks Plenty of locks are needed Hard to understand the whole locking scheme 26

27 SMP scheduling A SMP scheduler does two primary jobs Scheduling runnable jobs on a CPU Scheduling jobs across multiple CPUs in an SMP system With unpredictable load, these translate into Efficient Load Balancing Scalable scheduling Power-aware scheduling Efficiency = performance/power Organization Scheduling domains Measuring and metrics In practice, the operation is based on these It is not trivial to understand a system 27

28 SMP memory management Locality is important for performance Memory regions Full address space rarely needed Merging and remapping regions Exception handling Especially page faults The kernel needs memory management too A slab allocator, managing pieces to avoid fragmentation Manager is complex Booting is not trivial Have a simpler system for booting 28

29 Parallel programming systems 29

30 Parallelism for user programs Threading inside kernel is for system management User-space programs need threading, too Process parallelism Running several programs Thread-level parallelisms More performance for a single program The two basic ways Kernel threads Must be supported by the kernel User-space threads Can be implemented by the OS, too Or a library, etc. 30

31 Pthreads POSIX Portable Operating System Interface a family of standards specified by the IEEE includes a threading API Pthreads defines C language constructs types, functions and constants Contains thread management - creating, joining threads etc. mutexes condition variables synchronization between threads using read/write locks and barriers Compilers build more advance models on threads Some use directly HW thread but most rely on OS threads Note the importance of compiler runtimes 31