Lecture 2: Memory Systems

Transcription

1 Lecture 2: Memory Systems Basic components Memory hierarchy Cache memory Virtual Memory Zebo Peng, IDA, LiTH Internal and External Memories CPU Date transfer Control Main Memory Control Data transfer Secondary Memory Zebo Peng, IDA, LiTH 2

2 Main Memory Model A word (8, 6, 32, or 64 bits) Memory Control Unit 3 2 address 0 a bit Read/write control MBR (in CPU) Address selection MAR (in CPU) Zebo Peng, IDA, LiTH 3 Memory Characteristics The most important characteristics of a memory: speed as fast as possible; size as large as possible; cost reasonable price. They are determined by the technology used for implementation. Your personal library Zebo Peng, IDA, LiTH 4 2

3 Memory Access Bottleneck CPU Quantitative measurement of the capacity of the bottleneck is the Memory Bandwidth Memory Zebo Peng, IDA, LiTH 5 Memory Bandwidth Memory bandwidth denotes the amount of data that can be accessed from a memory per second: M-Bandwidth = amount of data per access memory cycle time Ex. MCT = 00 nano second and 4 bytes (a word) per access: M-Bandwidth = 40 mega bytes per second. There are two basic techniques to increase the bandwidth of a given memory: Reduce the memory cycle time Expensive Memory size limitation Divide the memory into several banks, each of which has its own control unit. Zebo Peng, IDA, LiTH 6 3

4 Memory Banks Interleaving placement of program and data Control Unit Control Unit Control Unit Control Unit CPU Zebo Peng, IDA, LiTH 7 Lecture 2: Memory Systems Basic components Memory hierarchy Cache memory Virtual Memory Zebo Peng, IDA, LiTH 8 4

5 Motivation What do we need? A memory to store very large programs and to work at a speed comparable to that of the CPU. The reality is: The larger a memory, the slower it will be; The faster a memory, the greater the cost per bit. A solution: To build a composite memory system which combines a small and fast memory with a large and slow memory, and behaves, most of the time, like a large and fast memory. This two-level principle can be extended to a hierarchy of many levels. Zebo Peng, IDA, LiTH 9 CPU Registers Cache Memory Hierarchy Main Memory Secondary Memory of direct access type Secondary Memory of archive type Zebo Peng, IDA, LiTH 0 5

6 Access time example -0 ns 0-50 ns ns 5-00 ms (for 4KB) s (for 8KB) CPU Registers Cache Main Memory Secondary Memory of direct access type Secondary Memory of archive type Memory Hierarchy Capacity example K 4-256M 40G/unit 50M/tape As one goes down the hierarchy, the following occur: Decreasing cost/bit. Increasing capacity. Increasing access time. Decreasing frequency of access by the CPU. Zebo Peng, IDA, LiTH Lecture 2: Memory Systems Basic components Memory hierarchy Cache memory Virtual Memory Zebo Peng, IDA, LiTH 2 6

7 Mismatch of CPU and MM Speeds Cycle Time (nano second) Speed Gap (ca. one order of magnitude, i.e., 0 times) Zebo Peng, IDA, LiTH 3 Cache Memory CPU addresses Main Memory addresses instructions and data instructions and data Cache addresses instructions and data A cache is a very fast memory which is put between the main memory and the CPU, and used to hold segments of program and data of the main memory. Zebo Peng, IDA, LiTH 4 7

8 Zebo s Cache Memory Model Personal library for a high-speed reader Storage cells Cache Memory controller A computer is a predictable and iterative reader, therefore high cache hit ratio, e.g., 96%, is achievable, even with a relatively small cache. Zebo Peng, IDA, LiTH 5 Cache Memory Features It is transparent to the programmers. It is only a small part of the program/data in the main memory which has its copy in the cache (e.g., 8KB cache with 8MB memory). If the CPU wants to access program/data not in the cache (called a cache miss), the relevant block of the main memory will be copied into the cache. The intermediate-future memory access will usually refer to the same word or words in the neighborhood, and will not have to involve the main memory. This property of program executions is denoted as locality of reference. Zebo Peng, IDA, LiTH 6 8

9 Locality of Reference Temporal locality: If an item is referenced, it will tend to be referenced again soon. Spatial locality: If an item is referenced, items whose addresses are close by will tend to be referenced soon. This access pattern is referred as locality of reference principle, which is an intrinsic features of the von Neumann architecture: Sequential instruction storage. Loops and iterations (e.g., subroutine calls). Sequential data storage (e.g., array). Zebo Peng, IDA, LiTH 7 Layered Memory Performance Average Access Time P hit x T cache_access + ( P hit ) x (T mm_access + T cache_access ) x Block_size + T checking where P hit = the probability of cache hit, cache hit ratio; T cache_access = cache access time; T mm_access = main memory access time; Block_size = number of words in a cache block; and T checking = the time needed to check for cache hit or miss. Ex. A computer has 8MB MM with 00 ns access time, 8KB cache with 0 ns access time, BS=4, and T checking = 2. ns, P hit = 0.97, AAT will be 25 ns. Zebo Peng, IDA, LiTH 8 9

10 Cache Design The size and nature of the copied block must be carefully designed, as well as the algorithm to decide which block to be removed from the cache when it is full: Cache block size (line size). Total cache size. Mapping function. Replacement method. Write policy. Numbers of caches: Single, two-level, or three-level cache. Unified vs. split cache. Zebo Peng, IDA, LiTH 9 Split Data and Instruction Caches? Split caches (Harvard Architectures): + Competition for the cache between instruction processing and execution units is eliminated. + Instruction fetch can proceed in parallel with memory access from the CPU for operands. One may be overloaded while the other is under utilized. Unified caches: + Better balance the load between instruction and data fetches depending on the dynamics of the program execution. + Design and implementation are cheaper. Lower performance. Zebo Peng, IDA, LiTH 20 0

11 Direct Mapping Cache Direct mapping - Each block of the main memory is mapped into a fixed cache slot. 2 2 Storage cells 2 Cache Memory controller Zebo Peng, IDA, LiTH 2 Direct Mapping Cache Example We have a 0,000-word MM and a 00-word cache. 0 memory cells are grouped into a block. Tag Slot Word Memory address = Tag Slot No ,000-Word Memory 00-Word Cache Zebo Peng, IDA, LiTH 22

12 Direct Mapping Pros & Cons Simple to implement and therefore inexpensive. Fixed location for blocks. If a program accesses 2 blocks that map to the same cache slot repeatedly, cache miss rate is very high. 2 2 Storage cells 2 Cache Memory controller Zebo Peng, IDA, LiTH 23 Associative Mapping A main memory block can be loaded into any slot of the cache. To determine if a block is in the cache, a mechanism is needed to simultaneously examine every slot s tag ,000-Word Memory Tag 006, 007 associative memory example Tag (3 ps) Word Cache Zebo Peng, IDA, LiTH 24 2

13 Fully Associative Organization Zebo Peng, IDA, LiTH 25 Set Associative Organization The cache is divided into a number of sets (K). Each set contains a number of slots (W). A given block maps to any slot in a given set. e.g. block i can be in any slot of set j. For example, 2 slots per set (W = 2): 2-way associative mapping. A given block can be in one of 2 slots. Direct mapping: W = (no alternative). Fully associative: K = (W = total number of all slots in the cache, all mappings possible). Zebo Peng, IDA, LiTH 26 3

14 Replacement Algorithms With direct mapping, it is no need. With associative mapping, a replacement algorithm is needed in order to determine which block to replace: First-in-first-out (FIFO). Least-recently used (LRU) - replace the block that has been in the cache longest with not reference to it. Lest-frequently used (LFU) - replace the block that has experienced the fewest references. Random. Use info 54 5:55 Tag Zebo Peng, IDA, LiTH 27 The problem: Write Policy How to keep cache content and main memory content consistent without losing too much performance? Write through: All write operations are passed to main memory: If the addressed location is currently in the cache, the cache is updated so that it is coherent with the main memory. For writes, the processor always slows down to main memory speed. Since the percentage of writes is small (ca. 5%), this scheme doesn t lead to large performance reduction. Zebo Peng, IDA, LiTH 28 4

15 Write through with buffered write: Write Policy (Cont d) The same as write-through, but instead of slowing the processor down by writing directly to main memory, the write address and data are stored in a high-speed write buffer; the write buffer transfers data to main memory while the processor continues its task. Higher speed, but more complex hardware. Write back: Write operations update only the cache memory which is not kept coherent with main memory. When the slot is replaced from the cache, its content has to be copied back to memory. Good performance (usually several writes are performed on a cache block before it is replaced), but more complex hardware is needed. Cache coherence problems are very complex and difficult to solve in multiprocessor systems (to be discussed later)! Zebo Peng, IDA, LiTH 29 Cache Architecture Examples Intel (introduced 989) a single on-chip cache of 8 Kbytes line size: 6 bytes 4-way set associative organization Intel Pentium (introduced 993) two on-chip caches, one for data and one for instructions each cache: 8 Kbytes line size: 32 bytes 2-way set associative organization IBM PowerPC 620 (introduced 995) two on-chip caches, one for data and one for instructions each cache: 32 Kbytes line size: 64 bytes 8-way set associative organization Zebo Peng, IDA, LiTH 30 5

16 Cache Architecture Examples (Cont d) Intel Itanium 2 (introduced 2002) three levels of cache: L L2 L3 Contents Split D and I Unified D + I Unified D + I Size 6 Kbytes each 256 Kbytes 3 Mbytes Line size 64 bytes 28 bytes 28 bytes Associativity 4 way 8 way 2 way Access time cycle 5-7 cycles 4-7 cycles Store policy Write-through Write-back Write-back Zebo Peng, IDA, LiTH 3 Lecture 2: Memory Systems Basic components Memory hierarchy Cache memory Virtual Memory Zebo Peng, IDA, LiTH 32 6

17 Motivation for Virtual Memory The physical main memory (RAM) is very limited in space. It may not be big enough to store all the executing programs at the same time. Some program may need memory larger than the main memory size, but not all the program need to be maintained in the main memory at the same time. Virtual Memory takes advantage of the fact that at any given instant of time, an executing program needs only a fraction of the memory that the whole program occupies. The basic idea: Load only pieces of each executing program which are currently needed. Zebo Peng, IDA, LiTH 33 Paging Divide programs (processes) into equal sized, small blocks, called pages. Divide the primary memory into equal sized, small blocks called page frames. Allocate the required number of page frames to a program. A program does not require continuous page frames! The operating system (OS) is responsible for: Maintaining a list of free frames. Using a page table to keep track of the mapping between pages and page frames. Zebo Peng, IDA, LiTH 34 7

18 Logical and Physical Addresses Implementation of the page-tables: Main memory slow since an extra memory access is needed Separate registers fast but expensive. Cache. Zebo Peng, IDA, LiTH 35 Objective of Virtual Memory To provide the user/programmer with a much bigger memory than the main memory with the help of the operative system. Virtual memory size >> main memory size. Program addresses 0000 MM addresses Secondary memory 5000 Zebo Peng, IDA, LiTH 36 8

19 Page Fault When accessing a VM page which is not in the main memory, a page fault occurs. The page must then be loaded from the secondary memory into the main memory by the OS. Virtual Address Page Number Offset Page Map Page Fault (Interrupt to OS) Pages in MM Zebo Peng, IDA, LiTH 37 Page Replacement When a page fault occurs and all page frames are occupied, one of them must be replaced. If the replaced page has been modified during the time it resides in the main memory, the updated version should be written back to the secondary memory. Our wish is to replace the page which will not be accessed in the future for the longest amount of time. Problem We don t know exactly what will happen in the future. Solution We predict the future by studying the access patterns up till now ( learn from history ). Zebo Peng, IDA, LiTH 38 9

20 Replacement Algorithms FIFO (First In First Out) To replace the one in MM the longest of time. LRU (Least Recently Used) To replace the one that has not be accessed the longest time. LFU (Least Frequently Used) To replace the one that has the smallest number of access during the latest time period. The replacement by random (used for Cache) is not used for VM! Zebo Peng, IDA, LiTH 39 Summary A memory system has to store very large programs and a lot of data and still provide fast access. No single type of memory can provide all the need of a computer system. Usually several different storage mechanisms are organized in a layer hierarchy. Cache is a hardware solution to improve memory access which is transparent to the programmers. Virtual memory provides a much larger address space than the available physical space with the help of the OS (software solution). The layer structure works very well due to the locality of reference principle. Zebo Peng, IDA, LiTH 40 20