Virtual Memory: Systems

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Virtual Memory: Systems"

Transcription

1 Virtual Memory: Systems 5-23 / 8-23: Introduc2on to Computer Systems 7 th Lecture, Mar. 2, 22 Instructors: Anthony Rowe, Seth Goldstein and Gregory Kesden

2 Today Virtual memory ques7ons and answers Simple memory system example Bonus: Case study: Core i7/linux memory system Bonus: Memory mapping 2

3 Virtual memory reminder/review Programmer s view of virtual memory Each process has its own private linear address space Cannot be corrupted by other processes System view of virtual memory Uses memory efficiently by caching virtual memory pages Efficient only because of locality Simplifies memory management and programming Simplifies protec2on by providing a convenient interposi2oning point to check permissions 3

4 Recall: Address Transla7on With a Page Table Page table base register (PTBR) n- Virtual address Virtual page number (VPN) p p- Virtual page offset (VPO) Page table address for process Page table Valid Physical page number (PPN) Valid bit = : page not in memory (page fault) m- Physical page number (PPN) Physical address p p- Physical page offset (PPO) 4

5 Recall: Address Transla7on: Page Hit CPU Chip CPU VA MMU 2 PTEA PTE 3 PA Cache/ Memory 4 Data 5 ) Processor sends virtual address to MMU 2-3) MMU fetches PTE from page table in memory 4) MMU sends physical address to cache/memory 5) Cache/memory sends data word to processor 5

6 Ques7on # Are the PTEs cached like other memory accesses? Yes (and no: see next ques7on) 6

7 Page tables in memory, like other data PTE CPU Chip CPU VA MMU PTEA PA PTEA hit PTEA miss PA miss PTE PTEA PA Memory PA hit Data Data L cache VA: virtual address, PA: physical address, PTE: page table entry, PTEA = PTE address 7

8 Ques7on #2 Isn t it slow to have to go to memory twice every 7me? Yes, it would be so, real MMUs don t 8

9 Speeding up Transla7on with a TLB Page table entries (PTEs) are cached in L like any other memory word PTEs may be evicted by other data references PTE hit s2ll requires a small L delay Solu7on: TranslaAon Lookaside Buffer (TLB) Small, dedicated, super- fast hardware cache of PTEs in MMU Contains complete page table entries for small number of pages 9

10 TLB Hit CPU Chip TLB 2 PTE VPN 3 CPU VA MMU PA 4 Cache/ Memory Data 5 A TLB hit eliminates a memory access

11 TLB Miss CPU Chip TLB 4 2 PTE VPN 3 CPU VA MMU PTEA PA Cache/ Memory 5 Data 6 A TLB miss incurs an addi7onal memory access (the PTE) Fortunately, TLB misses are rare. Why?

12 Ques7on #3 Isn t the page table huge? How can it be stored in RAM? Yes, it would be so, real page tables aren t simple arrays 2

13 Mul7- Level Page Tables Suppose: 4KB (2 2 ) page size, 64- bit address space, 8- byte PTE Level 2 Tables Problem: Would need a 32, TB page table! 2 64 * 2-2 * 2 3 = 2 55 bytes Level Table... Common solu7on: Mul2- level page tables Example: 2- level page table Level table: each PTE points to a page table (always memory resident) Level 2 table: each PTE points to a page (paged in and out like any other data)... 3

14 A Two- Level Page Table Hierarchy Level page table Level 2 page tables Virtual memory VP PTE PTE PTE 2 (null) PTE 3 (null) PTE 4 (null) PTE 5 (null) PTE 6 (null) PTE 7 (null) PTE... PTE 23 PTE... PTE VP 23 VP VP 247 Gap 2K allocated VM pages for code and data 6K unallocated VM pages PTE 8 (K - 9) null PTEs 23 null PTEs 32 bit addresses, 4KB pages, 4- byte PTEs PTE unallocated pages VP unallocated pages allocated VM page for the stack 4

15 Transla7ng with a k- level Page Table n-! VIRTUAL ADDRESS! p-!! VPN! VPN 2!...! VPN k! VPO! Level! page table! Level 2! page table!...!...! Level k! page table! PPN! m-! p-!! PPN! PPO! PHYSICAL ADDRESS! 5

16 Ques7on #4 Shouldn t fork() be really slow, since the child needs a copy of the parent s address space? Yes, it would be so, fork() doesn t really work that way 6

17 Sharing Revisited: Shared Objects Process virtual memory Physical memory Process 2 virtual memory Process maps the shared object. Shared object 7

18 Sharing Revisited: Shared Objects Process virtual memory Physical memory Process 2 virtual memory Process 2 maps the shared object. No7ce how the virtual addresses can be different. Shared object 8

19 Sharing Revisited: Private Copy- on- write (COW) Objects Process virtual memory Physical memory Process 2 virtual memory Two processes mapping a private copy- on- write (COW) object. Private copy-on-write area Area flagged as private copy- on- write PTEs in private areas are flagged as read- only Private copy-on-write object 9

20 Sharing Revisited: Private Copy- on- write (COW) Objects Process virtual memory Physical memory Copy-on-write Private copy-on-write object Process 2 virtual memory Write to private copy-on-write page Instruc7on wri7ng to private page triggers protec7on fault. Handler creates new R/W page. Instruc7on restarts upon handler return. Copying deferred as long as possible! 2

21 The fork Func7on Revisited fork provides private address space for each process To create virtual address for new process Create exact copies of parent page tables Flag each page in both processes (parent and child) as read- only Flag writeable areas in both processes as private COW On return, each process has exact copy of virtual memory Subsequent writes create new physical pages using COW mechanism Perfect approach for common case of fork() followed by exec() Why? 2

22 Today Virtual memory ques7ons and answers Simple memory system example Bonus: Case study: Core i7/linux memory system Bonus: Memory mapping 22

23 Review of Symbols Basic Parameters N = 2 n : Number of addresses in virtual address space M = 2 m : Number of addresses in physical address space P = 2 p : Page size (bytes) Components of the virtual address (VA) VPO: Virtual page offset VPN: Virtual page number TLBI: TLB index TLBT: TLB tag Components of the physical address (PA) PPO: Physical page offset (same as VPO) PPN: Physical page number CO: Byte offset within cache line CI: Cache index CT: Cache tag 23

24 Simple Memory System Example Addressing 4- bit virtual addresses 2- bit physical address Page size = 64 bytes VPN Virtual Page Number VPO Virtual Page Offset PPN Physical Page Number PPO Physical Page Offset 24

25 Simple Memory System Page Table Only show first 6 entries (out of 256) VPN PPN Valid VPN PPN Valid A B 4 C 5 6 D 2D 6 E 7 F D 25

26 Simple Memory System TLB 6 entries 4- way associa7ve TLBT TLBI VPN VPO Set Tag PPN Valid Tag PPN Valid Tag PPN Valid Tag PPN Valid 3 9 D D 2 4 A D A

27 Simple Memory System Cache 6 lines, 4- byte block size Physically addressed Direct mapped CT CI CO PPN PPO Idx Tag Valid B B B2 B3 Idx Tag Valid B B B2 B A D 2 B A 2D 93 5 DA 3B 3 36 B B D 8F 9 C 2 5 D F D D E B D3 7 6 C2 DF 3 F 4 27

28 Address Transla7on Example # Virtual Address: x3d4 TLBT TLBI VPN VPO VPN xf TLBI x3 TLBT x3 TLB Hit? Y Page Fault? N PPN: xd Physical Address CT CI CO PPN PPO CO CI x5 CT xd Hit? Y Byte: x36 28

29 Address Transla7on Example #2 Virtual Address: xcf TLBT TLBI VPN VPO VPN x7 TLBI x3 TLBT x TLB Hit? N Page Fault? Y PPN: TBD Physical Address CT CI CO PPN PPO CO CI CT Hit? Byte: 29

30 Address Transla7on Example #3 Virtual Address: x2 TLBT TLBI VPN VPO VPN x TLBI TLBT x TLB Hit? N Page Fault? N PPN: x28 Physical Address CT CI CO PPN PPO CO CI x8 CT x28 Hit? N Byte: Mem 3

31 Today Virtual memory ques7ons and answers Simple memory system example Bonus: Case study: Core i7/linux memory system Bonus: Memory mapping 3

32 Intel Core i7 Memory System Processor package Core x4 Registers Instruc7on fetch MMU (addr transla7on) L d- cache 32 KB, 8- way L i- cache 32 KB, 8- way L d- TLB 64 entries, 4- way L i- TLB 28 entries, 4- way L2 unified cache 256 KB, 8- way L2 unified TLB 52 entries, 4- way QuickPath interconnect GB/s each To other cores To I/O bridge L3 unified cache 8 MB, 6- way (shared by all cores) DDR3 Memory controller 3 x 64 GB/s 32 GB/s total (shared by all cores) Main memory 32

33 Review of Symbols Basic Parameters N = 2 n : Number of addresses in virtual address space M = 2 m : Number of addresses in physical address space P = 2 p : Page size (bytes) Components of the virtual address (VA) TLBI: TLB index TLBT: TLB tag VPO: Virtual page offset VPN: Virtual page number Components of the physical address (PA) PPO: Physical page offset (same as VPO) PPN: Physical page number CO: Byte offset within cache line CI: Cache index CT: Cache tag 33

34 End- to- end Core i7 Address Transla7on VPN TLB miss CPU 36 2 VPO 32 4 TLBT TLBI Virtual address (VA)... TLB hit 32/64 Result L hit L d- cache (64 sets, 8 lines/set)... L2, L3, and main memory L miss L TLB (6 sets, 4 entries/set) CR3 9 9 VPN VPN2 PTE 9 9 VPN3 VPN4 PTE PTE PTE 4 2 PPN PPO Physical address (PA) CT CI CO Page tables 34

35 Core i7 Level - 3 Page Table Entries 63 XD 62 Unused Page table physical base address Unused G PS A CD WT U/S R/W P= Available for OS (page table loca7on on disk) P= Each entry references a 4K child page table P: Child page table present in physical memory () or not (). R/W: Read- only or read- write access access permission for all reachable pages. U/S: user or supervisor (kernel) mode access permission for all reachable pages. WT: Write- through or write- back cache policy for the child page table. CD: Caching disabled or enabled for the child page table. A: Reference bit (set by MMU on reads and writes, cleared by sohware). PS: Page size either 4 KB or 4 MB (defined for Level PTEs only). G: Global page (don t evict from TLB on task switch) Page table physical base address: 4 most significant bits of physical page table address (forces page tables to be 4KB aligned) 35

36 Core i7 Level 4 Page Table Entries 63 XD 62 Unused Page physical base address Unused G D A CD WT U/S R/W P= Available for OS (page loca7on on disk) P= Each entry references a 4K child page P: Child page is present in memory () or not () R/W: Read- only or read- write access permission for child page U/S: User or supervisor mode access WT: Write- through or write- back cache policy for this page CD: Cache disabled () or enabled () A: Reference bit (set by MMU on reads and writes, cleared by sohware) D: Dirty bit (set by MMU on writes, cleared by sohware) G: Global page (don t evict from TLB on task switch) Page physical base address: 4 most significant bits of physical page address (forces pages to be 4KB aligned) 36

37 Core i7 Page Table Transla7on 9 VPN 9 VPN 2 9 VPN 3 VPN Virtual VPO address CR3 Physical address of L PT 4 / L PT Page global directory L PTE 4 / L2 PT Page upper directory L2 PTE 4 / L3 PT Page middle directory L3 PTE 4 / L4 PT Page table L4 PTE / 2 Offset into physical and virtual page 52 GB region per entry GB region per entry 2 MB region per entry 4 KB region per entry Physical address of page 4 / 4 2 Physical PPO address PPN 37

38 Cute Trick for Speeding Up L Access CT Tag Check Physical address (PA) CT CI CO PPN PPO Virtual address (VA) Address Transla7on VPN VPO 36 2 No Change Observa7on Bits that determine CI iden2cal in virtual and physical address Can index into cache while address transla2on taking place Generally we hit in TLB, so PPN bits (CT bits) available next Virtually indexed, physically tagged Cache carefully sized to make this possible CI L Cache 38

39 Today Virtual memory ques7ons and answers Simple memory system example Bonus: Case study: Core i7/linux memory system Bonus: Memory mapping 39

40 Memory Mapping VM areas ini7alized by associa7ng them with disk objects. Process is known as memory mapping. Area can be backed by (i.e., get its ini7al values from) : Regular file on disk (e.g., an executable object file) Ini2al page bytes come from a sec2on of a file Anonymous file (e.g., nothing) First fault will allocate a physical page full of 's (demand- zero page) Once the page is wrioen to (diraed), it is like any other page Dirty pages are copied back and forth between memory and a special swap file. 4

41 Demand paging Key point: no virtual pages are copied into physical memory un7l they are referenced! Known as demand paging Crucial for 7me and space efficiency 4

42 User- Level Memory Mapping void *mmap(void *start, int len, int prot, int flags, int fd, int offset) Map len bytes star7ng at offset offset of the file specified by file descrip7on fd, preferably at address start start: may be for pick an address prot: PROT_READ, PROT_WRITE,... flags: MAP_ANON, MAP_PRIVATE, MAP_SHARED,... Return a pointer to start of mapped area (may not be start) 42

43 User- Level Memory Mapping void *mmap(void *start, int len, int prot, int flags, int fd, int offset) len bytes len bytes start (or address chosen by kernel) offset (bytes) Disk file specified by file descriptor fd Process virtual memory 43

44 Using mmap to Copy Files Copying without transferring data to user space. #include "csapp.h" /* * mmapcopy - uses mmap to copy * file fd to stdout */ void mmapcopy(int fd, int size) { } /* Ptr to mem-mapped VM area */ char *bufp; bufp = Mmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, ); Write(, bufp, size); return; /* mmapcopy driver */ int main(int argc, char **argv) { struct stat stat; int fd; } /* Check for required cmdline arg */ if (argc!= 2) { printf("usage: %s <filename>\n, argv[]); exit(); } /* Copy the input arg to stdout */ fd = Open(argv[], O_RDONLY, ); Fstat(fd, &stat); mmapcopy(fd, stat.st_size); exit(); 44

45 Virtual Memory of a Linux Process Different for each process IdenAcal for each process Process- specific data structs (ptables, task and mm structs, kernel stack) Physical memory Kernel code and data Kernel virtual memory %esp User stack brk Memory mapped region for shared libraries Run7me heap (malloc) Process virtual memory x848 (32) x4 (64) Unini7alized data (.bss) Ini7alized data (.data) Program text (.text) 45

46 Linux Organizes VM as Collec7on of Areas Process virtual memory vm_area_struct task_struct mm_struct vm_end mm pgd vm_start vm_prot mmap vm_flags Shared libraries vm_next pgd: Page global directory address Points to L page table vm_prot: Read/write permissions for this area vm_flags Pages shared with other processes or private to this process vm_end vm_start vm_prot vm_flags vm_next vm_end vm_start vm_prot vm_flags vm_next Data Text 46

47 Linux Page Fault Handling vm_area_struct Process virtual memory vm_end vm_start vm_prot vm_flags vm_next vm_end vm_start vm_prot vm_flags shared libraries data read 3 read Segmentation fault: accessing a non- exis7ng page Normal page fault vm_next vm_end vm_start vm_prot vm_flags vm_next text 2 write Protec7on excep7on: e.g., viola7ng permission by wri7ng to a read- only page (Linux reports as Segmenta7on fault) 47

48 The execve Func7on Revisited User stack Private, demand-zero To load and run a new program a.out in the current process using execve: libc.so.data.text Memory mapped region for shared libraries Shared, file-backed Free vm_area_struct s and page tables for old areas a.out.data.text Runtime heap (via malloc) Uninitialized data (.bss) Initialized data (.data) Program text (.text) Private, demand-zero Private, demand-zero Private, file-backed Create vm_area_struct s and page tables for new areas Programs and ini2alized data backed by object files..bss and stack backed by anonymous files. Set PC to entry point in.text Linux will fault in code and data pages as needed. 48

Intel P6 Systemprogrammering 2007 Föreläsning 5 P6/Linux Memory System

Intel P6 Systemprogrammering 2007 Föreläsning 5 P6/Linux Memory System Intel P6 Systemprogrammering 07 Föreläsning 5 P6/Linux ory System Topics P6 address translation Linux memory management Linux page fault handling memory mapping Internal Designation for Successor to Pentium

More information

Virtual Memory. How is it possible for each process to have contiguous addresses and so many of them? A System Using Virtual Addressing

Virtual Memory. How is it possible for each process to have contiguous addresses and so many of them? A System Using Virtual Addressing How is it possible for each process to have contiguous addresses and so many of them? Computer Systems Organization (Spring ) CSCI-UA, Section Instructor: Joanna Klukowska Teaching Assistants: Paige Connelly

More information

Carnegie Mellon Virtual Memory of a Linux Process Process-specific data structs (ptables, Different for

Carnegie Mellon Virtual Memory of a Linux Process Process-specific data structs (ptables, Different for Outline CS 6V81-05: System Security and Malicious Code Analysis Understanding the Implementation of Virtual Memory 1 Zhiqiang Lin Department of Computer Science University of Texas at Dallas February 29

More information

Lecture 17: Virtual Memory II. Goals of virtual memory

Lecture 17: Virtual Memory II. Goals of virtual memory Lecture 17: Virtual Memory II Last Lecture: Introduction to virtual memory Today Review and continue virtual memory discussion Lecture 17 1 Goals of virtual memory Make it appear as if each process has:

More information

Lecture 7: Memory Management

Lecture 7: Memory Management Lecture 7: Memory Management CSE 120: Principles of Opera>ng Systems UC San Diego: Summer Session I, 2009 Frank Uyeda Announcements PeerWise ques>ons due tomorrow. Project 2 is due on Friday. Milestone

More information

W4118: segmentation and paging. Instructor: Junfeng Yang

W4118: segmentation and paging. Instructor: Junfeng Yang W4118: segmentation and paging Instructor: Junfeng Yang Outline Memory management goals Segmentation Paging TLB 1 Uni- v.s. multi-programming Simple uniprogramming with a single segment per process Uniprogramming

More information

Virtual Memory and Address Translation

Virtual Memory and Address Translation Virtual Memory and Address Translation Review: the Program and the Process VAS Process text segment is initialized directly from program text section. sections Process data segment(s) are initialized from

More information

Virtual Memory. COMP375 Computer Architecture and Organization

Virtual Memory. COMP375 Computer Architecture and Organization Virtual Memory COMP375 Computer Architecture and Organization You never know when you're making a memory. Rickie Lee Jones Design Project The project is due 1:00pm (start of class) on Monday, October 19,

More information

CS 61C: Great Ideas in Computer Architecture Virtual Memory Cont.

CS 61C: Great Ideas in Computer Architecture Virtual Memory Cont. CS 61C: Great Ideas in Computer Architecture Virtual Memory Cont. Instructors: Vladimir Stojanovic & Nicholas Weaver http://inst.eecs.berkeley.edu/~cs61c/ 1 Bare 5-Stage Pipeline Physical Address PC Inst.

More information

Virtual vs Physical Addresses

Virtual vs Physical Addresses Virtual vs Physical Addresses Physical addresses refer to hardware addresses of physical memory. Virtual addresses refer to the virtual store viewed by the process. virtual addresses might be the same

More information

W4118 Operating Systems. Instructor: Junfeng Yang

W4118 Operating Systems. Instructor: Junfeng Yang W4118 Operating Systems Instructor: Junfeng Yang Outline x86 segmentation and paging hardware Linux address space translation Copy-on-write Linux page replacement algorithm Linux dynamic memory allocation

More information

Memory Management CS 217. Two programs can t control all of memory simultaneously

Memory Management CS 217. Two programs can t control all of memory simultaneously Memory Management CS 217 Memory Management Problem 1: Two programs can t control all of memory simultaneously Problem 2: One program shouldn t be allowed to access/change the memory of another program

More information

Linux/UNIX System Programming. POSIX Shared Memory. Michael Kerrisk, man7.org c 2015. February 2015

Linux/UNIX System Programming. POSIX Shared Memory. Michael Kerrisk, man7.org c 2015. February 2015 Linux/UNIX System Programming POSIX Shared Memory Michael Kerrisk, man7.org c 2015 February 2015 Outline 22 POSIX Shared Memory 22-1 22.1 Overview 22-3 22.2 Creating and opening shared memory objects 22-10

More information

Dynamic Memory Management

Dynamic Memory Management Dynamic Memory Management 1 Goals of this Lecture Help you learn about: The need for dynamic* memory management (DMM) Implementing DMM using the heap section Implementing DMM using virtual memory * During

More information

VM Architecture. Jun-Chiao Wang, 15 Apr. 2002 NCTU CIS Operating System lab. 2002 Linux kernel trace seminar

VM Architecture. Jun-Chiao Wang, 15 Apr. 2002 NCTU CIS Operating System lab. 2002 Linux kernel trace seminar VM Architecture Jun-Chiao Wang, 15 Apr. 2002 NCTU CIS Operating System lab. 2002 Linux kernel trace seminar Outline Introduction What is VM? Segmentation & Paging in Linux Memory Management Page Frame

More information

Measuring Cache Performance

Measuring Cache Performance Measuring Cache Performance Components of CPU time Program execution cycles Includes cache hit time Memory stall cycles Mainly from cache misses With simplifying assumptions: Memory stall cycles = = Memory

More information

Shared Memory Introduction

Shared Memory Introduction 12 Shared Memory Introduction 12.1 Introduction Shared memory is the fastest form of IPC available. Once the memory is mapped into the address space of the processes that are sharing the memory region,

More information

This Unit: Virtual Memory. CIS 501 Computer Architecture. Readings. A Computer System: Hardware

This Unit: Virtual Memory. CIS 501 Computer Architecture. Readings. A Computer System: Hardware This Unit: Virtual CIS 501 Computer Architecture Unit 5: Virtual App App App System software Mem CPU I/O The operating system (OS) A super-application Hardware support for an OS Virtual memory Page tables

More information

Memory Management Outline. Background Swapping Contiguous Memory Allocation Paging Segmentation Segmented Paging

Memory Management Outline. Background Swapping Contiguous Memory Allocation Paging Segmentation Segmented Paging Memory Management Outline Background Swapping Contiguous Memory Allocation Paging Segmentation Segmented Paging 1 Background Memory is a large array of bytes memory and registers are only storage CPU can

More information

6.033 Computer System Engineering

6.033 Computer System Engineering MIT OpenCourseWare http://ocw.mit.edu 6.033 Computer System Engineering Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 6.033 Lecture 3: Naming

More information

Linux kernen. Version 2.4.20, 2.5.59. 11 arch. DIKU Intel, SMP. . NEL, Kernel, 2003.2.5-1 -

Linux kernen. Version 2.4.20, 2.5.59. 11 arch. DIKU Intel, SMP. . NEL, Kernel, 2003.2.5-1 - Linux kernen Version 2.4.20, 2.5.59 11 arch. DIKU Intel, SMP.. NEL, Kernel, 2003.2.5-1 - Linux processer Processer User mode/kernel mode. Kernel threads Processer/tråde. fork(), clone(), shmem etc. Parent

More information

CS5460: Operating Systems

CS5460: Operating Systems CS5460: Operating Systems Lecture 13: Memory Management (Chapter 8) Where are we? Basic OS structure, HW/SW interface, interrupts, scheduling Concurrency Memory management Storage management Other topics

More information

COS 318: Operating Systems. Virtual Memory and Address Translation

COS 318: Operating Systems. Virtual Memory and Address Translation COS 318: Operating Systems Virtual Memory and Address Translation Today s Topics Midterm Results Virtual Memory Virtualization Protection Address Translation Base and bound Segmentation Paging Translation

More information

Cache Mapping. COMP375 Computer Architecture and Organization

Cache Mapping. COMP375 Computer Architecture and Organization Cache Mapping COMP375 Computer Architecture and Organization The only problem in computer architecture that is really hard to overcome is not having enough address bits. Gordon Bell Exam on Wednesday The

More information

(page 0 of the address space) (page 1) (page 2) (page 3) Figure 12.1: A Simple 64-byte Address Space

(page 0 of the address space) (page 1) (page 2) (page 3) Figure 12.1: A Simple 64-byte Address Space 12 Paging: Introduction Remember our goal: to virtualize memory. Segmentation (a generalization of dynamic relocation) helped us do this, but has some problems; in particular, managing free space becomes

More information

Lecture 10: Dynamic Memory Allocation 1: Into the jaws of malloc()

Lecture 10: Dynamic Memory Allocation 1: Into the jaws of malloc() CS61: Systems Programming and Machine Organization Harvard University, Fall 2009 Lecture 10: Dynamic Memory Allocation 1: Into the jaws of malloc() Prof. Matt Welsh October 6, 2009 Topics for today Dynamic

More information

Objectives and Functions

Objectives and Functions Objectives and Functions William Stallings Computer Organization and Architecture 6 th Edition Week 10 Operating System Support Convenience Making the computer easier to use Efficiency Allowing better

More information

Content. Introduction and History. File I/O. The File System. Shell Programming. Standard Unix Files and Configuration. Processes

Content. Introduction and History. File I/O. The File System. Shell Programming. Standard Unix Files and Configuration. Processes Content Introduction and History File I/O The File System Shell Programming Standard Unix Files and Configuration Processes Programs are instruction sets stored on a permanent medium (e.g. harddisc). Processes

More information

361 Computer Architecture Lecture 14: Cache Memory

361 Computer Architecture Lecture 14: Cache Memory 1 361 Computer Architecture Lecture 14 Memory cache.1 The Motivation for s Memory System Processor DRAM Motivation Large memories (DRAM) are slow Small memories (SRAM) are fast Make the average access

More information

Virtual Memory. Chapter 4

Virtual Memory. Chapter 4 Chapter 4 Virtual Memory Linux processes execute in a virtual environment that makes it appear as if each process had the entire address space of the CPU available to itself. This virtual address space

More information

The Quest for Speed - Memory. Cache Memory. A Solution: Memory Hierarchy. Memory Hierarchy

The Quest for Speed - Memory. Cache Memory. A Solution: Memory Hierarchy. Memory Hierarchy The Quest for Speed - Memory Cache Memory CSE 4, Spring 25 Computer Systems http://www.cs.washington.edu/4 If all memory accesses (IF/lw/sw) accessed main memory, programs would run 20 times slower And

More information

Computer Engineering and Systems Group Electrical and Computer Engineering SCMFS: A File System for Storage Class Memory

Computer Engineering and Systems Group Electrical and Computer Engineering SCMFS: A File System for Storage Class Memory SCMFS: A File System for Storage Class Memory Xiaojian Wu, Narasimha Reddy Texas A&M University What is SCM? Storage Class Memory Byte-addressable, like DRAM Non-volatile, persistent storage Example: Phase

More information

Operating Systems. 05. Threads. Paul Krzyzanowski. Rutgers University. Spring 2015

Operating Systems. 05. Threads. Paul Krzyzanowski. Rutgers University. Spring 2015 Operating Systems 05. Threads Paul Krzyzanowski Rutgers University Spring 2015 February 9, 2015 2014-2015 Paul Krzyzanowski 1 Thread of execution Single sequence of instructions Pointed to by the program

More information

Buffer Overflows. Security 2011

Buffer Overflows. Security 2011 Buffer Overflows Security 2011 Memory Organiza;on Topics Kernel organizes memory in pages Typically 4k bytes Processes operate in a Virtual Memory Space Mapped to real 4k pages Could live in RAM or be

More information

Virtualization in Linux KVM + QEMU

Virtualization in Linux KVM + QEMU CS695 Topics in Virtualization and Cloud Computing KVM + QEMU Senthil, Puru, Prateek and Shashank 1 Topics covered KVM and QEMU Architecture VTx support CPU virtualization in KMV Memory virtualization

More information

Main Memory. Memory. Address binding. Memory spaces. All processes need main memory.

Main Memory. Memory. Address binding. Memory spaces. All processes need main memory. Memory If we define memory as a place where data is stored there are many levels of memory: Processor registers Primary (or main) memory RAM Secondary memory slower and more permanent disks Tertiary memory

More information

Betriebssysteme KU Security

Betriebssysteme KU Security Betriebssysteme KU Security IAIK Graz University of Technology 1 1. Drivers 2. Security - The simple stuff 3. Code injection attacks 4. Side-channel attacks 2 1. Drivers 2. Security - The simple stuff

More information

Intel s Virtualization Extensions (VT-x) So you want to build a hypervisor?

Intel s Virtualization Extensions (VT-x) So you want to build a hypervisor? Intel s Virtualization Extensions (VT-x) So you want to build a hypervisor? Mr. Jacob Torrey February 26, 2014 Dartmouth College 153 Brooks Road, Rome, NY 315.336.3306 http://ainfosec.com @JacobTorrey

More information

Chapter 1 Computer System Overview

Chapter 1 Computer System Overview Operating Systems: Internals and Design Principles Chapter 1 Computer System Overview Eighth Edition By William Stallings Operating System Exploits the hardware resources of one or more processors Provides

More information

CS 377: Operating Systems. Outline. A review of what you ve learned, and how it applies to a real operating system. Lecture 25 - Linux Case Study

CS 377: Operating Systems. Outline. A review of what you ve learned, and how it applies to a real operating system. Lecture 25 - Linux Case Study CS 377: Operating Systems Lecture 25 - Linux Case Study Guest Lecturer: Tim Wood Outline Linux History Design Principles System Overview Process Scheduling Memory Management File Systems A review of what

More information

KVM & Memory Management Updates

KVM & Memory Management Updates KVM & Memory Management Updates KVM Forum 2012 Rik van Riel Red Hat, Inc. KVM & Memory Management Updates EPT Accessed & Dirty Bits 1GB hugepages Balloon vs. Transparent Huge Pages Automatic NUMA Placement

More information

The Linux Kernel: Process Management. CS591 (Spring 2001)

The Linux Kernel: Process Management. CS591 (Spring 2001) The Linux Kernel: Process Management Process Descriptors The kernel maintains info about each process in a process descriptor, of type task_struct. See include/linux/sched.h Each process descriptor contains

More information

Virtualization. Explain how today s virtualization movement is actually a reinvention

Virtualization. Explain how today s virtualization movement is actually a reinvention Virtualization Learning Objectives Explain how today s virtualization movement is actually a reinvention of the past. Explain how virtualization works. Discuss the technical challenges to virtualization.

More information

Operating System Structure

Operating System Structure Operating System Structure Lecture 3 Disclaimer: some slides are adopted from the book authors slides with permission Recap Computer architecture CPU, memory, disk, I/O devices Memory hierarchy Architectural

More information

Xen and the Art of. Virtualization. Ian Pratt

Xen and the Art of. Virtualization. Ian Pratt Xen and the Art of Virtualization Ian Pratt Keir Fraser, Steve Hand, Christian Limpach, Dan Magenheimer (HP), Mike Wray (HP), R Neugebauer (Intel), M Williamson (Intel) Computer Laboratory Outline Virtualization

More information

System Calls and Standard I/O

System Calls and Standard I/O System Calls and Standard I/O Professor Jennifer Rexford http://www.cs.princeton.edu/~jrex 1 Goals of Today s Class System calls o How a user process contacts the Operating System o For advanced services

More information

This Unit: Main Memory. CIS 501 Introduction to Computer Architecture. Memory Hierarchy Review. Readings

This Unit: Main Memory. CIS 501 Introduction to Computer Architecture. Memory Hierarchy Review. Readings This Unit: Main Memory CIS 501 Introduction to Computer Architecture Unit 4: Memory Hierarchy II: Main Memory Application OS Compiler Firmware CPU I/O Memory Digital Circuits Gates & Transistors Memory

More information

Chapter 8: Memory Management!

Chapter 8: Memory Management! The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still

More information

Overview. Lecture 1: an introduction to CUDA. Hardware view. Hardware view. hardware view software view CUDA programming

Overview. Lecture 1: an introduction to CUDA. Hardware view. Hardware view. hardware view software view CUDA programming Overview Lecture 1: an introduction to CUDA Mike Giles mike.giles@maths.ox.ac.uk hardware view software view Oxford University Mathematical Institute Oxford e-research Centre Lecture 1 p. 1 Lecture 1 p.

More information

Virtual Memory Paging

Virtual Memory Paging COS 318: Operating Systems Virtual Memory Paging Kai Li Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cos318/) Today s Topics Paging mechanism Page replacement algorithms

More information

Lecture 03 Bits, Bytes and Data Types

Lecture 03 Bits, Bytes and Data Types Lecture 03 Bits, Bytes and Data Types In this lecture Computer Languages Assembly Language The compiler Operating system Data and program instructions Bits, Bytes and Data Types ASCII table Data Types

More information

CS , Spring 2004 Exam 2

CS , Spring 2004 Exam 2 Andrew login ID: Full Name: CS 15-213, Spring 2004 Exam 2 April 8, 2004 Instructions: Make sure that your exam is not missing any sheets, then write your full name and Andrew login ID on the front. Write

More information

Board Notes on Virtual Memory

Board Notes on Virtual Memory Board Notes on Virtual Memory Part A: Why Virtual Memory? - Letʼs user program size exceed the size of the physical address space - Supports protection o Donʼt know which program might share memory at

More information

CSE 30321 Computer Architecture I Fall 2009 Final Exam December 18, 2009

CSE 30321 Computer Architecture I Fall 2009 Final Exam December 18, 2009 CSE 30321 Computer Architecture I Fall 2009 Final Exam December 18, 2009 Test Guidelines: 1. Place your name on EACH page of the test in the space provided. 2. every question in the space provided. If

More information

& Data Processing 2. Exercise 3: Memory Management. Dipl.-Ing. Bogdan Marin. Universität Duisburg-Essen

& Data Processing 2. Exercise 3: Memory Management. Dipl.-Ing. Bogdan Marin. Universität Duisburg-Essen Folie a: Name & Data Processing 2 3: Memory Management Dipl.-Ing. Bogdan Marin Fakultät für Ingenieurwissenschaften Abteilung Elektro-und Informationstechnik -Technische Informatik- Objectives Memory Management

More information

Operating System Overview. Otto J. Anshus

Operating System Overview. Otto J. Anshus Operating System Overview Otto J. Anshus A Typical Computer CPU... CPU Memory Chipset I/O bus ROM Keyboard Network A Typical Computer System CPU. CPU Memory Application(s) Operating System ROM OS Apps

More information

Paging: Introduction. 18.1 A Simple Example And Overview

Paging: Introduction. 18.1 A Simple Example And Overview 18 Paging: Introduction It is sometimes said that the operating system takes one of two approaches when solving most any space-management problem. The first approach is to chop things up into variable-sized

More information

Storing Data: Disks and Files

Storing Data: Disks and Files Storing Data: Disks and Files Chapter 9 Comp 521 Files and Databases Fall 2010 1 Disks and Files DBMS stores information on ( hard ) disks. This has major implications for DBMS design! READ: transfer data

More information

Page 1 of 5. IS 335: Information Technology in Business Lecture Outline Operating Systems

Page 1 of 5. IS 335: Information Technology in Business Lecture Outline Operating Systems Lecture Outline Operating Systems Objectives Describe the functions and layers of an operating system List the resources allocated by the operating system and describe the allocation process Explain how

More information

Linux Driver Devices. Why, When, Which, How?

Linux Driver Devices. Why, When, Which, How? Bertrand Mermet Sylvain Ract Linux Driver Devices. Why, When, Which, How? Since its creation in the early 1990 s Linux has been installed on millions of computers or embedded systems. These systems may

More information

Advanced Mac OS X Rootkits. Dino Dai Zovi Chief Scientist Endgame Systems

Advanced Mac OS X Rootkits. Dino Dai Zovi Chief Scientist Endgame Systems Advanced Mac OS X Rootkits Dino Dai Zovi Chief Scientist Endgame Systems Overview Mac OS X and Mach Why use Mach for rootkits? User mode Mach rootkit techniques Kernel Mach rootkit techniques 2 Why Mach

More information

POSIX. RTOSes Part I. POSIX Versions. POSIX Versions (2)

POSIX. RTOSes Part I. POSIX Versions. POSIX Versions (2) RTOSes Part I Christopher Kenna September 24, 2010 POSIX Portable Operating System for UnIX Application portability at source-code level POSIX Family formally known as IEEE 1003 Originally 17 separate

More information

CSI 402 Lecture 13 (Unix Process Related System Calls) 13 1 / 17

CSI 402 Lecture 13 (Unix Process Related System Calls) 13 1 / 17 CSI 402 Lecture 13 (Unix Process Related System Calls) 13 1 / 17 System Calls for Processes Ref: Process: Chapter 5 of [HGS]. A program in execution. Several processes are executed concurrently by the

More information

Operating Systems. Privileged Instructions

Operating Systems. Privileged Instructions Operating Systems Operating systems manage processes and resources Processes are executing instances of programs may be the same or different programs process 1 code data process 2 code data process 3

More information

Memory management in C: The heap and the stack

Memory management in C: The heap and the stack Memory management in C: The heap and the stack Leo Ferres Department of Computer Science Universidad de Concepción leo@inf.udec.cl October 7, 2010 1 Introduction When a program is loaded into memory, it

More information

COS 318: Operating Systems

COS 318: Operating Systems COS 318: Operating Systems File Performance and Reliability Andy Bavier Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall10/cos318/ Topics File buffer cache

More information

Addendum Intel Architecture Software Developer s Manual

Addendum Intel Architecture Software Developer s Manual Addendum Intel Architecture Software Developer s Manual Volume 3: System Programming Guide Order Number: 243690-001 NOTE: The Intel Architecture Software Developer s Manual consists of the following volumes:

More information

Bringing PowerPC Book E to Linux

Bringing PowerPC Book E to Linux Bringing PowerPC Book E to Linux Challenges in porting Linux to the first PowerPC Book E processor implementation Matthew D. Porter MontaVista Software, Inc. mporter@kernel.crashing.org mporter@mvista.com

More information

ELEC 377. Operating Systems. Week 1 Class 3

ELEC 377. Operating Systems. Week 1 Class 3 Operating Systems Week 1 Class 3 Last Class! Computer System Structure, Controllers! Interrupts & Traps! I/O structure and device queues.! Storage Structure & Caching! Hardware Protection! Dual Mode Operation

More information

A3 Computer Architecture

A3 Computer Architecture A3 Computer Architecture Engineering Science 3rd year A3 Lectures Prof David Murray david.murray@eng.ox.ac.uk www.robots.ox.ac.uk/ dwm/courses/3co Michaelmas 2000 1 / 1 6. Stacks, Subroutines, and Memory

More information

CS161: Operating Systems

CS161: Operating Systems CS161: Operating Systems Matt Welsh mdw@eecs.harvard.edu Lecture 2: OS Structure and System Calls February 6, 2007 1 Lecture Overview Protection Boundaries and Privilege Levels What makes the kernel different

More information

Map- reduce, Hadoop and The communica3on bo5leneck. Yoav Freund UCSD / Computer Science and Engineering

Map- reduce, Hadoop and The communica3on bo5leneck. Yoav Freund UCSD / Computer Science and Engineering Map- reduce, Hadoop and The communica3on bo5leneck Yoav Freund UCSD / Computer Science and Engineering Plan of the talk Why is Hadoop so popular? HDFS Map Reduce Word Count example using Hadoop streaming

More information

Hardware Assisted Virtualization

Hardware Assisted Virtualization Hardware Assisted Virtualization G. Lettieri 21 Oct. 2015 1 Introduction In the hardware-assisted virtualization technique we try to execute the instructions of the target machine directly on the host

More information

Advanced IBM AIX Heap Exploitation. Tim Shelton V.P. Research & Development HAWK Network Defense, Inc. tshelton@hawkdefense.com

Advanced IBM AIX Heap Exploitation. Tim Shelton V.P. Research & Development HAWK Network Defense, Inc. tshelton@hawkdefense.com Advanced IBM AIX Heap Exploitation Tim Shelton V.P. Research & Development HAWK Network Defense, Inc. tshelton@hawkdefense.com Introduction Our society has become dependent on computers and network systems.

More information

Operating Systems and Networks

Operating Systems and Networks recap Operating Systems and Networks How OS manages multiple tasks Virtual memory Brief Linux demo Lecture 04: Introduction to OS-part 3 Behzad Bordbar 47 48 Contents Dual mode API to wrap system calls

More information

The Linux Virtual Filesystem

The Linux Virtual Filesystem Lecture Overview Linux filesystem Linux virtual filesystem (VFS) overview Common file model Superblock, inode, file, dentry Object-oriented Ext2 filesystem Disk data structures Superblock, block group,

More information

Programmation Systèmes Cours 9 Memory Mapping

Programmation Systèmes Cours 9 Memory Mapping Programmation Systèmes Cours 9 Memory Mapping Stefano Zacchiroli zack@pps.jussieu.fr Laboratoire PPS, Université Paris Diderot - Paris 7 24 novembre 2011 URL http://upsilon.cc/zack/teaching/1112/progsyst/

More information

Chapter 10: Virtual Memory. Lesson 03: Page tables and address translation process using page tables

Chapter 10: Virtual Memory. Lesson 03: Page tables and address translation process using page tables Chapter 10: Virtual Memory Lesson 03: Page tables and address translation process using page tables Objective Understand page tables Learn the offset fields in the virtual and physical addresses Understand

More information

Leak Check Version 2.1 for Linux TM

Leak Check Version 2.1 for Linux TM Leak Check Version 2.1 for Linux TM User s Guide Including Leak Analyzer For x86 Servers Document Number DLC20-L-021-1 Copyright 2003-2009 Dynamic Memory Solutions LLC www.dynamic-memory.com Notices Information

More information

Virtual Memory Behavior in Red Hat Linux Advanced Server 2.1

Virtual Memory Behavior in Red Hat Linux Advanced Server 2.1 Virtual Memory Behavior in Red Hat Linux Advanced Server 2.1 Bob Matthews Red Hat, Inc. Kernel Development Team Norm Murray Red Hat, Inc. Client Engineering Team This is an explanation of the virtual memory

More information

Memory Management under Linux: Issues in Linux VM development

Memory Management under Linux: Issues in Linux VM development Memory Management under Linux: Issues in Linux VM development Christoph Lameter, Ph.D. Technical Lead, Linux Kernel Software Silicon Graphics Inc. clameter@sgi.com 2008-03-12 2008 SGI Sunnyvale, California

More information

ò Paper reading assigned for next Thursday ò Lab 2 due next Friday ò What is cooperative multitasking? ò What is preemptive multitasking?

ò Paper reading assigned for next Thursday ò Lab 2 due next Friday ò What is cooperative multitasking? ò What is preemptive multitasking? Housekeeping Paper reading assigned for next Thursday Scheduling Lab 2 due next Friday Don Porter CSE 506 Lecture goals Undergrad review Understand low-level building blocks of a scheduler Understand competing

More information

Software Vulnerabilities

Software Vulnerabilities Software Vulnerabilities -- stack overflow Code based security Code based security discusses typical vulnerabilities made by programmers that can be exploited by miscreants Implementing safe software in

More information

Chapter 6, The Operating System Machine Level

Chapter 6, The Operating System Machine Level Chapter 6, The Operating System Machine Level 6.1 Virtual Memory 6.2 Virtual I/O Instructions 6.3 Virtual Instructions For Parallel Processing 6.4 Example Operating Systems 6.5 Summary Virtual Memory General

More information

OPERATING SYSTEMS MEMORY MANAGEMENT

OPERATING SYSTEMS MEMORY MANAGEMENT OPERATING SYSTEMS MEMORY MANAGEMENT Jerry Breecher 8: Memory Management 1 OPERATING SYSTEM Memory Management What Is In This Chapter? Just as processes share the CPU, they also share physical memory. This

More information

Physical Memory RAM. The kernel keeps track of how it is used

Physical Memory RAM. The kernel keeps track of how it is used Virtual Memory Physical Memory RAM The kernel keeps track of how it is used Physical Memory RAM Kernel and hardware typically with it in pages Usually- 4k on 32-bit architectures 8k on 64-bit architectures

More information

Jorix kernel: real-time scheduling

Jorix kernel: real-time scheduling Jorix kernel: real-time scheduling Joris Huizer Kwie Min Wong May 16, 2007 1 Introduction As a specialized part of the kernel, we implemented two real-time scheduling algorithms: RM (rate monotonic) and

More information

Buffer Management 5. Buffer Management

Buffer Management 5. Buffer Management 5 Buffer Management Copyright 2004, Binnur Kurt A journey of a byte Buffer Management Content 156 A journey of a byte Suppose in our program we wrote: outfile

More information

CSC 2405: Computer Systems II

CSC 2405: Computer Systems II CSC 2405: Computer Systems II Spring 2013 (TR 8:30-9:45 in G86) Mirela Damian http://www.csc.villanova.edu/~mdamian/csc2405/ Introductions Mirela Damian Room 167A in the Mendel Science Building mirela.damian@villanova.edu

More information

Virtual Memory. Virtual Memory. Paging. CSE 380 Computer Operating Systems. Paging (1)

Virtual Memory. Virtual Memory. Paging. CSE 380 Computer Operating Systems. Paging (1) Virtual Memory CSE 380 Computer Operating Systems Instructor: Insup Lee University of Pennsylvania Fall 2003 Lecture Note: Virtual Memory (revised version) 1 Recall: memory allocation with variable partitions

More information

Virtuoso and Database Scalability

Virtuoso and Database Scalability Virtuoso and Database Scalability By Orri Erling Table of Contents Abstract Metrics Results Transaction Throughput Initializing 40 warehouses Serial Read Test Conditions Analysis Working Set Effect of

More information

9. Application/Kernel Interface

9. Application/Kernel Interface 9. Application/Kernel Interface 9. Application/Kernel Interface POSIX Essentials Implementation 164 / 303 9. Application/Kernel Interface Application/Kernel Interface: System Calls Challenge: Interaction

More information

1. Computer System Structure and Components

1. Computer System Structure and Components 1 Computer System Structure and Components Computer System Layers Various Computer Programs OS System Calls (eg, fork, execv, write, etc) KERNEL/Behavior or CPU Device Drivers Device Controllers Devices

More information

W4118: virtual machines

W4118: virtual machines W4118: virtual machines Instructor: Junfeng Yang References: Modern Operating Systems (3 rd edition), Operating Systems Concepts (8 th edition), previous W4118, and OS at MIT, Stanford, and UWisc Virtual

More information

Linux Process Scheduling Policy

Linux Process Scheduling Policy Lecture Overview Introduction to Linux process scheduling Policy versus algorithm Linux overall process scheduling objectives Timesharing Dynamic priority Favor I/O-bound process Linux scheduling algorithm

More information

Computer Architecture

Computer Architecture Computer Architecture Slide Sets WS 2013/2014 Prof. Dr. Uwe Brinkschulte M.Sc. Benjamin Betting Part 11 Memory Management Computer Architecture Part 11 page 1 of 44 Prof. Dr. Uwe Brinkschulte, M.Sc. Benjamin

More information

Memory Management 1. Memory Management. Multitasking without memory management is like having a party in a closet.

Memory Management 1. Memory Management. Multitasking without memory management is like having a party in a closet. Memory Management 1 Memory Management Multitasking without memory management is like having a party in a closet. Charles Petzold. Programming Windows 3.1 Programs expand to fill the memory that holds them.

More information

4.1 PAGING MODES AND CONTROL BITS

4.1 PAGING MODES AND CONTROL BITS CHAPTER 4 PAGING Chapter 3 explains how segmentation converts logical addresses to linear addresses. Paging (or linear-address translation) is the process of translating linear addresses so that they can

More information

Defending Computer Networks Lecture 3: More On Vulnerabili3es. Stuart Staniford Adjunct Professor of Computer Science

Defending Computer Networks Lecture 3: More On Vulnerabili3es. Stuart Staniford Adjunct Professor of Computer Science Defending Computer Networks Lecture 3: More On Vulnerabili3es Stuart Staniford Adjunct Professor of Computer Science Enrollment Logis;cs Send request to cs- course- enroll@cornell.edu Cc me (sgs235@cornell.edu)

More information

IOMMU: A Detailed view

IOMMU: A Detailed view 12/1/14 Security Level: Security Level: IOMMU: A Detailed view Anurup M. Sanil Kumar D. Nov, 2014 HUAWEI TECHNOLOGIES CO., LTD. Contents n IOMMU Introduction n IOMMU for ARM n Use cases n Software Architecture

More information