Storage Technologies and Memory Hierarchy

Transcription

1 Storage Technologies and Memory Hierarchy CSCI 224 / ECE 317: Computer Architecture Instructor: Prof. Jason Fritts Slides adapted from Bryant & O Hallaron s slides 1

2 Storage Technologies and Memory Hierarchy Storage Technologies and Trends RAM Nonvolatile memory Disk Trends Locality, Cache, and Memory Hierarchy to the Rescue! 2

3 Random-Access Memory (RAM) Basic storage unit is normally a cell (one bit per cell). Static RAM (SRAM) Each cell stores a bit with a four or six-transistor circuit. Retains value indefinitely, as long as it is kept powered. Dynamic RAM (DRAM) Each cell stores bit with a capacitor; one transistor used for access Value must be refreshed every ms Trans. Access Needs per bit time refresh? Cost Applications SRAM 4 or 6 1X No 100x Cache memories DRAM 1 10X Yes 1X Main memories, frame buffers 3

4 Conventional DRAM Organization D x W DRAM: organized as Dsupercellsof size Wbits access desired supercell by first sending row address (RAS), then column address (CAS) 16 x 8 DRAM chip To CPU supercell (2,1) Memory controller RAS = 2 CAS = 1 2 / addr 8 / data Rows Cols supercell (2,1) Internal row buffer 4

5 Memory Modules addr (row = i, col = j) DRAM 7 DRAM 0 : supercell (i,j) 64 MB memory module consisting of eight 8Mx8 DRAMs bits bits bits bits bits bits bits 8-15 bits bit doubleword at main memory address A 0 Memory controller 64-bit doubleword 5

6 Enhanced DRAMs Basic DRAM cell has not changed since its invention in 1966 Commercialized by Intel in 1970 DRAM cores with better interface logic and faster I/O : Synchronous DRAM (SDRAM) Uses a conventional clock signal instead of asynchronous control Double data-rate synchronous DRAM (DDR SDRAM) Double edge clocking sends two bits per cycle per pin Different types distinguished by size of small prefetch buffer: DDR(2 bits), DDR2(4 bits), DDR3(8 bits) DDR RAM standard for most server and desktop systems 6

8 Nonvolatile Memories DRAM and SRAM are volatile memories Data lost when powered off Nonvolatile memories retain value even if powered off Read-only memory (ROM) programmed during production Programmable ROM (PROM) can be programmed once Electrically-Eraseable PROM (EEPROM) electronic erase capability Flash memory EEPROMs with partial (sector) erase capability Wears out after about 100,000 erasings Uses for Nonvolatile Memories Firmware programs stored in a ROM (BIOS, disk controllers, ) Solid state disks (USB sticks, smart phones, tablets, laptops, ) Disk caches 8

9 Traditional Bus Structure Connecting CPU and Memory Saint Louis Carnegie University Mellon A bus is a collection of parallel wires that carry address, data, and control signals bus width in CPU defines architecture width (32-bit, 64-bit, etc.) Buses are typically shared by multiple devices CPU chip Register file ALU System bus Memory bus Bus interface I/O bridge Main memory 9

10 Memory Read Transaction (1) CPU places address A on the memory bus. CPU chip Register file Load operation: movl A, %eax %eax ALU Bus interface I/O bridge A Main memory 0 x A 10

11 Memory Read Transaction (2) Main memory reads A from the memory bus, retrieves word x, and places it on the bus. CPU chip Register file Load operation: movl A, %eax %eax ALU I/O bridge Main memory x 0 Bus interface x A 11

12 Memory Read Transaction (3) CPU read word x from the bus and copies it into register %eax. CPU chip Register file Load operation: movl A, %eax %eax x ALU I/O bridge Main memory 0 Bus interface x A 12

14 What s Inside A Disk Drive? Arm Spindle Platters Actuator SCSI connector Electronics (including a processor and memory!) Image courtesy of Seagate Technology 16

15 Disk Geometry Disks consist of platters, each with two surfaces. Each surface consists of concentric rings called tracks. Each track consists of sectors separated by gaps. Tracks Surface Track k Gaps Spindle Sectors 17

16 Disk Structure - top view of single platter Surface organized into tracks Tracks divided into sectors 18

17 Disk Geometry (Muliple-Platter View) Aligned tracks form a cylinder. Cylinder k Surface 0 Surface 1 Surface 2 Surface 3 Surface 4 Surface 5 Platter 0 Platter 1 Platter 2 Spindle 19

18 Disk Capacity Capacity maximum number of bits that can be stored. Vendors express capacity in units of gigabytes (GB) Capacity is determined by these technology factors: Recording density(bits/in) number of bits per segment of a track Track density (tracks/in) number of tracks per radial segment Areal density (bits/in 2 ) product of recording and track density Modern disks partition tracks into disjoint subsets called recording zones Each track in a zone has the same number of sectors, determined by the circumference of innermost track Each zone has a different number of sectors/track 20

19 Computing Disk Capacity Capacity = (# bytes/sector) x (avg. # sectors/track) x (# tracks/surface) x (# surfaces/platter) x (# platters/disk) Example: 512 bytes/sector 300 sectors/track (on average) 20,000 tracks/surface 2 surfaces/platter 5 platters/disk Capacity = 512 x 300 x x 2 x 5 = 30,720,000,000 = GB 21

20 Disk Operation (Single-Platter View) The disk surface spins at a fixed rotational rate The read/write head is attached to the end of the arm and flies over the disk surface on a thin cushion of air. spindle spindle spindle spindle By moving radially, the arm can position the read/write head over any track. 22

21 Disk Access Head in position above a track 24

22 Disk Access Rotation is counter-clockwise 25

23 Disk Access Read About to read blue sector 26

24 Disk Access Read After BLUE read After reading blue sector 27

25 Disk Access Read After BLUE read Red request scheduled next 28

26 Disk Access Seek After BLUE read Seek for RED Seek to red s track 29

27 Disk Access Rotational Latency After BLUE read Seek for RED Rotational latency Wait for red sector to rotate around 30

28 Disk Access Read After BLUE read Seek for RED Rotational latency After RED read Complete read of red 31

29 Disk Access Service Time Components After BLUE read Seek for RED Rotational latency After RED read Data transfer Seek Rotational latency Data transfer 32

30 Disk Access Time Average time to access some target sector approximated by : T access = T avgseek + T avgrotation + T avgtransfer Seek time (T avg seek ) Time to position heads over cylinder containing target sector. Typical T avgseek is 3 9 ms Rotational latency (T avg rotation ) Time waiting for first bit of target sector to pass under r/whead. T avgrotation = 1/2 x1/rpms x60 sec/1 min Typical T avgrotation = 7200 RPMs Transfer time (T avg transfer ) Time to read the bits in the target sector. T avgtransfer = 1/RPM x1/(avg # sectors/track) x60 secs/1 min. 33

31 Disk Access Time Example Given: Rotational rate = 7,200 RPM Average seek time = 9 ms. Avg# sectors/track = 400. Derived: T avgrotation = 1/2 x(60 secs/7200 RPM) x1000 ms/sec = 4 ms. T avgtransfer = 60/7200 RPM x1/400 secs/track x1000 ms/sec = 0.02 ms T access = 9 ms + 4 ms ms Important points: Access time dominated by seek time and rotational latency. First bit in a sector is the most expensive, the rest are free. SRAM access time is about 4 ns/doubleword, DRAM about 60 ns Disk is about 40,000 times slower than SRAM, and about 2,500 times slower then DRAM 34

32 I/O Bus CPU chip Register file ALU System bus Memory bus Bus interface I/O bridge Main memory USB controller Graphics adapter I/O bus Disk controller Expansion slots for other devices such as network adapters. Mouse Keyboard Monitor Disk 35

33 Reading a Disk Sector (1) CPU chip Register file ALU CPU initiates a disk read by writing a command, logical block number, and destination memory address to a port (address) associated with disk controller. Bus interface Main memory I/O bus USB controller Graphics adapter Disk controller mouse keyboard Monitor Disk 36

34 Reading a Disk Sector (2) CPU chip Register file ALU Disk controller reads the sector and performs a direct memory access (DMA) transfer into main memory. Bus interface Main memory I/O bus USB controller Graphics adapter Disk controller Mouse Keyboard Monitor Disk 37

35 Reading a Disk Sector (3) CPU chip Register file ALU When the DMA transfer completes, the disk controller notifies the CPU with an interrupt (i.e., asserts a special interrupt pin on the CPU) Bus interface Main memory I/O bus USB controller Graphics adapter Disk controller Mouse Keyboard Monitor Disk 38

36 Solid State Disks (SSDs) vs Rotating Disks Advantages No moving parts faster, less power, more rugged Disadvantages Have the potential to wear out Mitigated by wear leveling logic in flash translation layer E.g. Intel X25 guarantees 1 petabyte(10 15 bytes) of random writes before they wear out As of 2014, about 15 times more expensive per byte Applications Smart phones and tablets Found in some laptops, desktops, and servers 41

38 SRAM Storage Trends Metric :1980 $/MB 19,200 2, access (ns) DRAM Metric :1980 $/MB 8, ,000 access (ns) typical size (MB) ,000 8, ,000 Disk Metric :1980 $/MB ,600,000 access (ms) typical size (MB) ,000 20, ,000 1,500,000 1,500,000 43

39 CPU Clock Rates Inflection point in computer history when designers hit the Power Wall :1980 CPU Pentium P-III P-4 Core 2 Core i7 --- Clock rate (MHz) Cycle time (ns) Cores Effective cycle ,000 time (ns) 44

40 The CPU-Memory Gap The gap between DRAM, disk, and CPU speeds. 100,000, ,000,000.0 Disk 1,000, ,000.0 SSD ns 10, , DRAM Disk seek time Flash SSD access time DRAM access time SRAM access time CPU cycle time Effective CPU cycle time CPU Year 45

42 Locality and Cache to the Rescue! The key to bridging this Processor-Memory gap is a fundamental property of computer programs known as Locality 47

43 Locality Principle of Locality: Programs tend to use data and instructions with the same or neighboring addresses as those they have used recently Temporal locality: Recently referenced items are likely to be referenced again in the near future Spatial locality: Items with nearby addresses tend to be referenced close together in time (will discuss locality in more detail later in semester ) 48

44 Caches Cache memory takes advantage of locality Cache: A smaller, faster memory device that acts as a staging area for a subset of the datafrom a larger, slower device that subset of data frequently exhibits good locality Cache memory can be made very fast using SRAM BUT, fast caches are small Making caches larger quickly reduces speed How to get BOTH size and speed? the Memory Hierarchy 49

45 Memory Hierarchy L0: Registers CPU registers hold words retrieved from L1 cache Smaller, faster, more expensive per byte Larger, slower, cheaper per byte L4: L3: L2: L1: L1 cache (SRAM) L2 cache (SRAM) Main memory (DRAM) Local secondary storage (local disks) L1 cache holds cache lines retrieved from L2 cache L2 cache holds cache lines retrieved from main memory Main memory holds disk blocks retrieved from local disks Local disks hold files retrieved from disks on remote network servers L5: Remote secondary storage (tapes, distributed file systems, Web servers) 50

46 Caches Fundamental idea of a memory hierarchy: For each k, the faster, smaller device at level kserves as a cache for the larger, slower device at level k+1. Why do memory hierarchies work? Because of locality, programs tend to access the data at level kmore often than they access the data at level k+1. Thus, the storage at level k+1 can be slower, and thus larger and cheaper per bit. Big Idea: The memory hierarchy creates a large pool of storage that costs as much as the cheap storage near the bottom, but that serves data to programs at the rate of the fast storage near the top. 51

47 Intel Core i7 Cache Hierarchy Processor package Core 0 Regs Core 3 Regs L1 i-cache and d-cache: 32 KB, 8-way, Access: 4 cycles L1 d-cache L1 i-cache L1 d-cache L1 i-cache L2 unified cache: 256 KB, 8-way, Access: 11 cycles L2 unified cache L2 unified cache L3 unified cache: 8 MB, 16-way, Access: cycles L3 unified cache (shared by all cores) Block size: 64 bytes for all caches. Main memory 52

48 Cache Performance Metrics Miss Rate Fraction of memory references not found in cache (misses / accesses) = 1 hit rate Typical numbers (in percentages): 3-10% for L1 can be quite small (e.g., < 1%) for L2, depending on size, etc. Hit Time Time to deliver a line in the cache to the processor includes time to determine whether the line is in the cache Typical numbers: 1-2 clock cycle for L clock cycles for L2 Miss Penalty Additional time required because of a miss typically cycles for main memory (Trend: increasing!) 53

49 Lets think about those numbers Huge difference between a hit and a miss Could be 100x, if just L1 and main memory Would you believe 99% hits is twice as good as 97%? Consider: cache hit time of 1 cycle miss penalty of 100 cycles Average access time: 97% hits: 1 cycle * 100 cycles = 4 cycles 99% hits: 1 cycle * 100 cycles = 2 cycles This is why miss rate is used instead of hit rate 54

50 Summary The speed gap between processor, memory, and mass storage continues to widen. Memory hierarchies based on caching close the gap by exploiting locality. 55