Sistemas Operativos: File System
|
|
- Tyler Ethan Davis
- 7 years ago
- Views:
Transcription
1 Sistemas Operativos: File System Reliability and Performance Pedro F. Souto May 25, 2012
2 Sumário Reliability Performance Virtual File System (VFS) Further Reading
3 Topics Reliability Performance Virtual File System (VFS) Further Reading
4 File System Reliability Users expect data in disk to persist until they explicitly change it Different events contribute to filesystems failing those expectations Disk Failures Disk are fragile electromechanical devices with a relatively short lifetime (about 5 years) Google has reported failure rates of 2% per year Human Errors Many users type faster than they think Windows uses the recycle bin In Unix/Linux one can change rm: alias rm mv -i /tmp/${logname} System Failures caused by power failures or crashes Backups can address the first two problems Disk failures can also be addressed by redundant media such as RAID
5 System Failures and FS Reliability Facts 1. File systems cache data and metadata in main memory Use write back rather than write through 2. Some metadata updates require changing more than one disk sector Problem System failures (that do not damage the media) may Lead to loss of data that has not made it to disk Lead to inconsistency of file system data structures on disk Some sectors are updated but others don t Example File creation: 1. Allocate an inode, and initialize it 2. Allocate a directory entry and make it point to inode If system goes down after writing directory entry to disk but before the inode is written, the file system becomes inconsistent What if the writes to disk are done in inverse order?
6 File System Recovery Upon restart, if the FS was not cleanly shutdown, the OS executes an utility (fsck/scandisk) that: Checks the integrity of the FS Tries to fix the inconsistencies found For example, in the case of the Unix FS, fsck checks, at least: The bitmap of free blocks The inodes and their reference counts by scanning the FS metadata (including directory entries) Also possible that a block be in use and in the free list.
7 Reducing File System Inconsistencies FS Inconsistencies cannot be avoided (in the Unix FS ) Even if the FS uses synchronous writes for metadata update Challenges Asynchrony System failure may happen at any time Recovery Metadata must be updated in the right order to: Allow recovery Avoid full disk scan Performance Synchronous writes hurt performance Goals are to reduce: The metadata update overhead during normal operation The recovery time at startup after system failure Solutions Enforcing order in metadata updates, taking advantage of metadata semantics FS dependent, but usually very hard
8 Reducing File System Inconsistencies with Logs Idea Use logs like transactions in databases. Indeed we want disk metadata updates to be: Atomic i.e. either all of them are performed, or none are Consistent i.e. they must preserve system invariants Isolated i.e. as if metadata updates were executed by a single thread Durable i.e. they should persist until modified by other metadata updates These are known as the ACID properties of transactions Advantage Systematic approach using a very mature tecnhology Variations Pratically all modern FS use logs What is logged? is data also logged? How is (meta)data logged? values vs. operations Log contains all FS data and metadata? log vs. journaled FS Type of log redo (write-ahead) vs. undo log Guarantees fully transactional or only order Some do not ensure isolation
9 Metada-only Write-Ahead (Redo) Log Data structures Log An append only file (on disk) Its tail may be in main memory FS Metadata On disk Cached in main memory Operation Metadata updates are grouped in transactions, sequences of updates that must have ACID properties Update the cached metadata Add entries with the updates at the log tail in main memory Must contain enough information to be able to redo them At the end, add an end of transaction entry to the log An alternative is to use a single log entry per transaction Disk log Log entries must be written to disk before the cached metadata Either, at the end of each transaction Or, when convenient
10 Metadata-only Redo Log: Recovery Idea Reconstruct the cached metadata by scanning the log and applying its entries Problem If log size is large, this may take too long Solution checkpoint the metadata on disk This is a consistent snapshot of the metadata and keep track of the first log entry whose update is not in that checkpoint This also prevents the log from growing too large Log entries for transactions that made it to disk can be freed Recovery becomes a two step process: 1. Read the most recent metadata checkpoint from disk 2. Apply all the entries in the log for transactions that terminated since that checkpoint Why does this work?
11 Metadata-only Redo Log: Assessment Advantages Recovery There is no need to scan and check the entire FS metadata. Need only: Scan the log since the last checkpoint Replay it Must read the metadata that was changed since then Normal operation Log entries are appended at the end of the log Writing to disk may be deferred Minimizes seeks Disadvantages Log requires extra space Metadata updates written to disk more than once Log cleanup adds overhead Optimizing log performance is not trivial What about the data? Programmers can invoke fsync()/fdatasync()
12 Topics Reliability Performance Virtual File System (VFS) Further Reading
13 Performance Problem Disks are too slow Solution Avoid disk access Cache metadata and data in memory Often, data has to be read from disk To ensure data persistency, data has to be written to disk Avoid seeks when disk access is unavoidable Try to put close on disk Data that belongs to the same file Data and metadata for the same file Problem Fine tunning these tecniques is very hard Filesystem and disks are complex File sizes and access patterns vary widely
14 Cache What to cache? Everything that can be frequently reused Data blocks i.e. disk blocks with data a.k.a. buffer cache Inodes of opened files Directory names But not the on-disk blocks/inodes of directories Indirect blocks i.e. disk blocks with pointers to data blocks How to manage the buffer cache? Can use pure LRU Hash table Front (LRU) Rear (MRU)... almost. Ensuring consistency in system failures, may prevent it.
15 Cache Management How large should be the cache? Difficult to say... In systems with VM use integrated buffer management I.e., any frame can be used either for VM pages or for the buffer cache, as needed. For example, Linux: $top [...] Mem: k total, k used, k free, 45560k buffers Swap: k total, k used, k free, k cached [...] free total used free buffers cached Mem: /+ buffers/cache: Swap: buffers is the buffer cache cached appears to be the in memory cache of swap These can be freed, if the system needs more pages, hence the 2nd line in free s output This is useful, because in this system the swap area is smaller than the physical memory, and hibernation...
16 Buffer Cache and Reads/Writes Reads Prefetch, i.e. read block ahead Works with sequential access Usually, disks controllers cache entire tracks in the disks cache Why not free-behind/replace-behind? Discard buffer from cache whent next is requested Writes Synchronous writes write block to disk immediately No data loss Deferred writes write block later May lead to less disk writes If a block is modified several times between writes to disk Temporary files may not even go to disk Allows further performance gains, by disk scheduling Applications may flush the cache by invoking fsync()
17 Performance: Avoiding Seeks (1/2) Access to even small file requires reading at least two blocks The file inode The file data block Performance may be improved by locating a file s metadata close to its data I-nodes are located near the start of the disk Disk is divided into cylinder groups, each with its own i-nodes Cylinder group (a) (b) What about multi-platter disks? Anyway, nowadays disk controllers hide the disk geometry
18 Performance: Avoiding Seeks (2/2) Keep data blocks of the same file sequentially on disk. Use Extents is a set of consecutive blocks on disk Extent sizes range from 128 KiB (Kibi =2 10 ) to several MiB Space in each extent is allocated sequentially For each extent, keeps only the first block number and its length When a new file is created, allocate an extent rather than a single block for it As the file grows use the remaining space in the extent If the extent runs out of space allocate another extent src: Getting to know the Solarois filesystem, Part 1
19 Other Issues (Not covered) Disk Caches Nowadays disks have caches of tens of MiB And some disks write sectors to disk when they deem best, not when the OS tells them to do it SSDs won t be on servers for a while No seeks Access time gap is much shorter than for disk Networked file systems add the network, server and client side caches, consistency issues... The design space is considerably larger
20 Topics Reliability Performance Virtual File System (VFS) Further Reading
21 Virtual File System (VFS) Layer (1/2) Problem How to use diferent FS types on the same OS? ext2/ext3/ext4 and NTFS, for disk FS (V)FAT, on USB pen ISO9660 on CDs/DVDs NFS via the network /proc, for access to kernel structures Solution Add another layer on top of the disk stack src: Anatomy of the Linux virtual file system switch The VFS layer is implemented with main memory data structures only The VFS layer was originally designed by Sun for NFS
22 Virtual File System (VFS) Layer (2/2) Each File System Must provide a uniform interface, i.e. a set of filesystem (e.g. mount()) and file/directory operations Just like character device drivers in the Linux kernel must implement a set of functions defined in struct file_operations The VFS Layer Provides file system independent functionality Validates system call parameters Copies data to and from user-space Manages the directory name caches Maps system calls to the VFS operations that are implemented by the underlying FS In Linux, the buffer cache is in the Block Layer, between the different FS and the device drivers
23 Topics Reliability Performance Virtual File System (VFS) Further Reading
24 Leitura Adicional Sistemas Operativos Subsecção 9.2.3: Estruturas de Suporte à Utilização dos Ficheiros Secção 9.3: Linux Starting at Subsecção (inclusive) Modern Operating Systems, 2nd. Ed. Secções 6.1 e 6.2: Files e Directories Secção 6.3: File System Implementation Subsecções 6.3.6, e 6.3.8
COS 318: Operating Systems
COS 318: Operating Systems File Performance and Reliability Andy Bavier Computer Science Department Princeton University http://www.cs.princeton.edu/courses/archive/fall10/cos318/ Topics File buffer cache
More informationRecovery and the ACID properties CMPUT 391: Implementing Durability Recovery Manager Atomicity Durability
Database Management Systems Winter 2004 CMPUT 391: Implementing Durability Dr. Osmar R. Zaïane University of Alberta Lecture 9 Chapter 25 of Textbook Based on slides by Lewis, Bernstein and Kifer. University
More informationLecture 18: Reliable Storage
CS 422/522 Design & Implementation of Operating Systems Lecture 18: Reliable Storage Zhong Shao Dept. of Computer Science Yale University Acknowledgement: some slides are taken from previous versions of
More informationOutline. Failure Types
Outline Database Management and Tuning Johann Gamper Free University of Bozen-Bolzano Faculty of Computer Science IDSE Unit 11 1 2 Conclusion Acknowledgements: The slides are provided by Nikolaus Augsten
More informationFile-System Implementation
File-System Implementation 11 CHAPTER In this chapter we discuss various methods for storing information on secondary storage. The basic issues are device directory, free space management, and space allocation
More informationThe Linux Virtual Filesystem
Lecture Overview Linux filesystem Linux virtual filesystem (VFS) overview Common file model Superblock, inode, file, dentry Object-oriented Ext2 filesystem Disk data structures Superblock, block group,
More informationStorage and File Systems. Chester Rebeiro IIT Madras
Storage and File Systems Chester Rebeiro IIT Madras 1 Two views of a file system system calls protection rwx attributes Application View Look & Feel File system Hardware view 2 Magnetic Disks Chester Rebeiro
More informationReview: The ACID properties
Recovery Review: The ACID properties A tomicity: All actions in the Xaction happen, or none happen. C onsistency: If each Xaction is consistent, and the DB starts consistent, it ends up consistent. I solation:
More informationLast Class Carnegie Mellon Univ. Dept. of Computer Science 15-415/615 - DB Applications
Last Class Carnegie Mellon Univ. Dept. of Computer Science 15-415/615 - DB Applications C. Faloutsos A. Pavlo Lecture#23: Crash Recovery Part 2 (R&G ch. 18) Write-Ahead Log Checkpoints Logging Schemes
More informationCS3210: Crash consistency. Taesoo Kim
1 CS3210: Crash consistency Taesoo Kim 2 Administrivia Quiz #2. Lab4-5, Ch 3-6 (read "xv6 book") Open laptop/book, no Internet 3:05pm ~ 4:25-30pm (sharp) NOTE Lab6: 10% bonus, a single lab (bump up your
More informationFile System Design and Implementation
Transactions and Reliability Sarah Diesburg Operating Systems CS 3430 Motivation File systems have lots of metadata: Free blocks, directories, file headers, indirect blocks Metadata is heavily cached for
More informationReview. Lecture 21: Reliable, High Performance Storage. Overview. Basic Disk & File System properties CSC 468 / CSC 2204 11/23/2006
S 468 / S 2204 Review Lecture 2: Reliable, High Performance Storage S 469HF Fall 2006 ngela emke rown We ve looked at fault tolerance via server replication ontinue operating with up to f failures Recovery
More informationUVA. Failure and Recovery. Failure and inconsistency. - transaction failures - system failures - media failures. Principle of recovery
Failure and Recovery Failure and inconsistency - transaction failures - system failures - media failures Principle of recovery - redundancy - DB can be protected by ensuring that its correct state can
More informationJournaling the Linux ext2fs Filesystem
Journaling the Linux ext2fs Filesystem Stephen C. Tweedie sct@dcs.ed.ac.uk Abstract This paper describes a work-in-progress to design and implement a transactional metadata journal for the Linux ext2fs
More informationCOSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters
COSC 6374 Parallel I/O (I) I/O basics Fall 2012 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network card 1 Network card
More informationMicrokernels & Database OSs. Recovery Management in QuickSilver. DB folks: Stonebraker81. Very different philosophies
Microkernels & Database OSs Recovery Management in QuickSilver. Haskin88: Roger Haskin, Yoni Malachi, Wayne Sawdon, Gregory Chan, ACM Trans. On Computer Systems, vol 6, no 1, Feb 1988. Stonebraker81 OS/FS
More informationCSE 120 Principles of Operating Systems
CSE 120 Principles of Operating Systems Fall 2004 Lecture 13: FFS, LFS, RAID Geoffrey M. Voelker Overview We ve looked at disks and file systems generically Now we re going to look at some example file
More informationInformation Systems. Computer Science Department ETH Zurich Spring 2012
Information Systems Computer Science Department ETH Zurich Spring 2012 Lecture VI: Transaction Management (Recovery Manager) Recovery Manager ETH Zurich, Spring 2012 Information Systems 3 Failure Recovery
More informationStorage in Database Systems. CMPSCI 445 Fall 2010
Storage in Database Systems CMPSCI 445 Fall 2010 1 Storage Topics Architecture and Overview Disks Buffer management Files of records 2 DBMS Architecture Query Parser Query Rewriter Query Optimizer Query
More informationRecovery Protocols For Flash File Systems
Recovery Protocols For Flash File Systems Ravi Tandon and Gautam Barua Indian Institute of Technology Guwahati, Department of Computer Science and Engineering, Guwahati - 781039, Assam, India {r.tandon}@alumni.iitg.ernet.in
More informationChapter 6, The Operating System Machine Level
Chapter 6, The Operating System Machine Level 6.1 Virtual Memory 6.2 Virtual I/O Instructions 6.3 Virtual Instructions For Parallel Processing 6.4 Example Operating Systems 6.5 Summary Virtual Memory General
More informationNetwork File System (NFS) Pradipta De pradipta.de@sunykorea.ac.kr
Network File System (NFS) Pradipta De pradipta.de@sunykorea.ac.kr Today s Topic Network File System Type of Distributed file system NFS protocol NFS cache consistency issue CSE506: Ext Filesystem 2 NFS
More informationDatabase Concurrency Control and Recovery. Simple database model
Database Concurrency Control and Recovery Pessimistic concurrency control Two-phase locking (2PL) and Strict 2PL Timestamp ordering (TSO) and Strict TSO Optimistic concurrency control (OCC) definition
More informationTransactions and Recovery. Database Systems Lecture 15 Natasha Alechina
Database Systems Lecture 15 Natasha Alechina In This Lecture Transactions Recovery System and Media Failures Concurrency Concurrency problems For more information Connolly and Begg chapter 20 Ullmanand
More informationCSE 544 Principles of Database Management Systems. Magdalena Balazinska Fall 2007 Lecture 5 - DBMS Architecture
CSE 544 Principles of Database Management Systems Magdalena Balazinska Fall 2007 Lecture 5 - DBMS Architecture References Anatomy of a database system. J. Hellerstein and M. Stonebraker. In Red Book (4th
More informationFile Systems Management and Examples
File Systems Management and Examples Today! Efficiency, performance, recovery! Examples Next! Distributed systems Disk space management! Once decided to store a file as sequence of blocks What s the size
More informationThe World According to the OS. Operating System Support for Database Management. Today s talk. What we see. Banking DB Application
The World According to the OS Operating System Support for Database Management App1 App2 App3 notes from Stonebraker s paper that appeared in Computing Practices, 1981 Operating System Anastassia Ailamaki
More informationChapter 11 I/O Management and Disk Scheduling
Operating Systems: Internals and Design Principles, 6/E William Stallings Chapter 11 I/O Management and Disk Scheduling Dave Bremer Otago Polytechnic, NZ 2008, Prentice Hall I/O Devices Roadmap Organization
More informationTwo Parts. Filesystem Interface. Filesystem design. Interface the user sees. Implementing the interface
File Management Two Parts Filesystem Interface Interface the user sees Organization of the files as seen by the user Operations defined on files Properties that can be read/modified Filesystem design Implementing
More informationAdvanced File Systems. CS 140 Nov 4, 2016 Ali Jose Mashtizadeh
Advanced File Systems CS 140 Nov 4, 2016 Ali Jose Mashtizadeh Outline FFS Review and Details Crash Recoverability Soft Updates Journaling Copy-on-Write: LFS/WAFL/ZFS Review: Improvements to UNIX FS Problems
More informationCOSC 6374 Parallel Computation. Parallel I/O (I) I/O basics. Concept of a clusters
COSC 6374 Parallel Computation Parallel I/O (I) I/O basics Spring 2008 Concept of a clusters Processor 1 local disks Compute node message passing network administrative network Memory Processor 2 Network
More informationDatabase Hardware Selection Guidelines
Database Hardware Selection Guidelines BRUCE MOMJIAN Database servers have hardware requirements different from other infrastructure software, specifically unique demands on I/O and memory. This presentation
More informationChapter 13 File and Database Systems
Chapter 13 File and Database Systems Outline 13.1 Introduction 13.2 Data Hierarchy 13.3 Files 13.4 File Systems 13.4.1 Directories 13.4. Metadata 13.4. Mounting 13.5 File Organization 13.6 File Allocation
More informationChapter 13 File and Database Systems
Chapter 13 File and Database Systems Outline 13.1 Introduction 13.2 Data Hierarchy 13.3 Files 13.4 File Systems 13.4.1 Directories 13.4. Metadata 13.4. Mounting 13.5 File Organization 13.6 File Allocation
More informationChapter 3 Operating-System Structures
Contents 1. Introduction 2. Computer-System Structures 3. Operating-System Structures 4. Processes 5. Threads 6. CPU Scheduling 7. Process Synchronization 8. Deadlocks 9. Memory Management 10. Virtual
More informationRyusuke KONISHI NTT Cyberspace Laboratories NTT Corporation
Ryusuke KONISHI NTT Cyberspace Laboratories NTT Corporation NILFS Introduction FileSystem Design Development Status Wished features & Challenges Copyright (C) 2009 NTT Corporation 2 NILFS is the Linux
More informationStoring Data: Disks and Files. Disks and Files. Why Not Store Everything in Main Memory? Chapter 7
Storing : Disks and Files Chapter 7 Yea, from the table of my memory I ll wipe away all trivial fond records. -- Shakespeare, Hamlet base Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Disks and
More informationBookKeeper overview. Table of contents
by Table of contents 1 BookKeeper overview...2 1.1 BookKeeper introduction... 2 1.2 In slightly more detail...2 1.3 Bookkeeper elements and concepts...3 1.4 Bookkeeper initial design... 3 1.5 Bookkeeper
More informationLast Class: OS and Computer Architecture. Last Class: OS and Computer Architecture
Last Class: OS and Computer Architecture System bus Network card CPU, memory, I/O devices, network card, system bus Lecture 3, page 1 Last Class: OS and Computer Architecture OS Service Protection Interrupts
More informationChapter 15: Recovery System
Chapter 15: Recovery System Failure Classification Storage Structure Recovery and Atomicity Log-Based Recovery Shadow Paging Recovery With Concurrent Transactions Buffer Management Failure with Loss of
More information09'Linux Plumbers Conference
09'Linux Plumbers Conference Data de duplication Mingming Cao IBM Linux Technology Center cmm@us.ibm.com 2009 09 25 Current storage challenges Our world is facing data explosion. Data is growing in a amazing
More informationDigital Forensics Lecture 3. Hard Disk Drive (HDD) Media Forensics
Digital Forensics Lecture 3 Hard Disk Drive (HDD) Media Forensics Current, Relevant Topics defendants should not use disk-cleaning utilities to wipe portions of their hard drives before turning them over
More informationSQL Server Transaction Log from A to Z
Media Partners SQL Server Transaction Log from A to Z Paweł Potasiński Product Manager Data Insights pawelpo@microsoft.com http://blogs.technet.com/b/sqlblog_pl/ Why About Transaction Log (Again)? http://zine.net.pl/blogs/sqlgeek/archive/2008/07/25/pl-m-j-log-jest-za-du-y.aspx
More informationNetwork Attached Storage. Jinfeng Yang Oct/19/2015
Network Attached Storage Jinfeng Yang Oct/19/2015 Outline Part A 1. What is the Network Attached Storage (NAS)? 2. What are the applications of NAS? 3. The benefits of NAS. 4. NAS s performance (Reliability
More informationSurvey of Filesystems for Embedded Linux. Presented by Gene Sally CELF
Survey of Filesystems for Embedded Linux Presented by Gene Sally CELF Presentation Filesystems In Summary What is a filesystem Kernel and User space filesystems Picking a root filesystem Filesystem Round-up
More informationTaking Linux File and Storage Systems into the Future. Ric Wheeler Director Kernel File and Storage Team Red Hat, Incorporated
Taking Linux File and Storage Systems into the Future Ric Wheeler Director Kernel File and Storage Team Red Hat, Incorporated 1 Overview Going Bigger Going Faster Support for New Hardware Current Areas
More informationMaximizing Cylinder Locality. Disk Drives and Geometry. Seek and Latency Scheduling. (maximizing cylinder locality) 5/16/2016
File Systems: Performance & Robustness Operating Systems Principles File Systems: Performance & Robustness 11G. File System Performance 11H. File System Robustness 11I. Checksums 11J. Log Structured File
More informationA Deduplication File System & Course Review
A Deduplication File System & Course Review Kai Li 12/13/12 Topics A Deduplication File System Review 12/13/12 2 Traditional Data Center Storage Hierarchy Clients Network Server SAN Storage Remote mirror
More informationIntroduction to Database Management Systems
Database Administration Transaction Processing Why Concurrency Control? Locking Database Recovery Query Optimization DB Administration 1 Transactions Transaction -- A sequence of operations that is regarded
More informationDualFS: A New Journaling File System for Linux
2007 Linux Storage & Filesystem Workshop February 12-13, 13, 2007, San Jose, CA DualFS: A New Journaling File System for Linux Juan Piernas SDM Project Pacific Northwest National
More informationCSE 120 Principles of Operating Systems. Modules, Interfaces, Structure
CSE 120 Principles of Operating Systems Fall 2000 Lecture 3: Operating System Modules, Interfaces, and Structure Geoffrey M. Voelker Modules, Interfaces, Structure We roughly defined an OS as the layer
More informationCrashes and Recovery. Write-ahead logging
Crashes and Recovery Write-ahead logging Announcements Exams back at the end of class Project 2, part 1 grades tags/part1/grades.txt Last time Transactions and distributed transactions The ACID properties
More informationChapter 3: Operating-System Structures. System Components Operating System Services System Calls System Programs System Structure Virtual Machines
Chapter 3: Operating-System Structures System Components Operating System Services System Calls System Programs System Structure Virtual Machines Operating System Concepts 3.1 Common System Components
More informationDistributed File Systems
Distributed File Systems Paul Krzyzanowski Rutgers University October 28, 2012 1 Introduction The classic network file systems we examined, NFS, CIFS, AFS, Coda, were designed as client-server applications.
More information2 nd Semester 2008/2009
Chapter 17: System Departamento de Engenharia Informática Instituto Superior Técnico 2 nd Semester 2008/2009 Slides baseados nos slides oficiais do livro Database System c Silberschatz, Korth and Sudarshan.
More informationOperating Systems CSE 410, Spring 2004. File Management. Stephen Wagner Michigan State University
Operating Systems CSE 410, Spring 2004 File Management Stephen Wagner Michigan State University File Management File management system has traditionally been considered part of the operating system. Applications
More informationOperating Systems. Design and Implementation. Andrew S. Tanenbaum Melanie Rieback Arno Bakker. Vrije Universiteit Amsterdam
Operating Systems Design and Implementation Andrew S. Tanenbaum Melanie Rieback Arno Bakker Vrije Universiteit Amsterdam Operating Systems - Winter 2012 Outline Introduction What is an OS? Concepts Processes
More informationOutline. Operating Systems Design and Implementation. Chap 1 - Overview. What is an OS? 28/10/2014. Introduction
Operating Systems Design and Implementation Andrew S. Tanenbaum Melanie Rieback Arno Bakker Outline Introduction What is an OS? Concepts Processes and Threads Memory Management File Systems Vrije Universiteit
More informationChapter 11 I/O Management and Disk Scheduling
Operatin g Systems: Internals and Design Principle s Chapter 11 I/O Management and Disk Scheduling Seventh Edition By William Stallings Operating Systems: Internals and Design Principles An artifact can
More informationCrash Recovery. Chapter 18. Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke
Crash Recovery Chapter 18 Database Management Systems, 3ed, R. Ramakrishnan and J. Gehrke Review: The ACID properties A tomicity: All actions in the Xact happen, or none happen. C onsistency: If each Xact
More informationChapter 16: Recovery System
Chapter 16: Recovery System Failure Classification Failure Classification Transaction failure : Logical errors: transaction cannot complete due to some internal error condition System errors: the database
More informationStorage Class Memory Support in the Windows Operating System Neal Christiansen Principal Development Lead Microsoft nealch@microsoft.
Storage Class Memory Support in the Windows Operating System Neal Christiansen Principal Development Lead Microsoft nealch@microsoft.com What is Storage Class Memory? Paradigm Shift: A non-volatile storage
More informationLiveBackup. Jagane Sundar jagane@sundar.org
LiveBackup Jagane Sundar jagane@sundar.org LiveBackup A complete Backup Solution Create Full and Incremental Backups of running VMs A System Administrator or Backup Software can use livebackup_client to
More information! Volatile storage: ! Nonvolatile storage:
Chapter 17: Recovery System Failure Classification! Failure Classification! Storage Structure! Recovery and Atomicity! Log-Based Recovery! Shadow Paging! Recovery With Concurrent Transactions! Buffer Management!
More informationFile System & Device Drive. Overview of Mass Storage Structure. Moving head Disk Mechanism. HDD Pictures 11/13/2014. CS341: Operating System
CS341: Operating System Lect 36: 1 st Nov 2014 Dr. A. Sahu Dept of Comp. Sc. & Engg. Indian Institute of Technology Guwahati File System & Device Drive Mass Storage Disk Structure Disk Arm Scheduling RAID
More informationTELE 301 Lecture 7: Linux/Unix file
Overview Last Lecture Scripting This Lecture Linux/Unix file system Next Lecture System installation Sources Installation and Getting Started Guide Linux System Administrators Guide Chapter 6 in Principles
More informationEncrypted File Systems. Don Porter CSE 506
Encrypted File Systems Don Porter CSE 506 Goals Protect confidentiality of data at rest (i.e., on disk) Even if the media is lost or stolen Protecting confidentiality of in-memory data much harder Continue
More informationViolin: A Framework for Extensible Block-level Storage
Violin: A Framework for Extensible Block-level Storage Michail Flouris Dept. of Computer Science, University of Toronto, Canada flouris@cs.toronto.edu Angelos Bilas ICS-FORTH & University of Crete, Greece
More informationIntroduction Disks RAID Tertiary storage. Mass Storage. CMSC 412, University of Maryland. Guest lecturer: David Hovemeyer.
Guest lecturer: David Hovemeyer November 15, 2004 The memory hierarchy Red = Level Access time Capacity Features Registers nanoseconds 100s of bytes fixed Cache nanoseconds 1-2 MB fixed RAM nanoseconds
More informationChapter 10 Case Study 1: LINUX
MODERN OPERATING SYSTEMS Third Edition ANDREW S. TANENBAUM Chapter 10 Case Study 1: LINUX History of UNIX and Linux UNICS PDP-11 UNIX Portable UNIX Berkeley UNIX Standard UNIX MINIX Linux UNIX/Linux Goals
More informationPart III Storage Management. Chapter 11: File System Implementation
Part III Storage Management Chapter 11: File System Implementation 1 Layered File System 2 Overview: 1/4 A file system has on-disk and in-memory information. A disk may contain the following for implementing
More informationFile System Management
Lecture 7: Storage Management File System Management Contents Non volatile memory Tape, HDD, SSD Files & File System Interface Directories & their Organization File System Implementation Disk Space Allocation
More informationFile System Reliability (part 2)
File System Reliability (part 2) Main Points Approaches to reliability Careful sequencing of file system opera@ons Copy- on- write (WAFL, ZFS) Journalling (NTFS, linux ext4) Log structure (flash storage)
More informationOverview. File Management. File System Properties. File Management
File Management Lecture 15b 1 2 File Management File management system consists of system utility programs that run as privileged applications Input to applications is by means of a file Output is saved
More informationLecture 25 Symbian OS
CS 423 Operating Systems Design Lecture 25 Symbian OS Klara Nahrstedt Fall 2011 Based on slides from Andrew S. Tanenbaum textbook and other web-material (see acknowledgements) cs423 Fall 2011 1 Overview
More informationCHAPTER 17: File Management
CHAPTER 17: File Management The Architecture of Computer Hardware, Systems Software & Networking: An Information Technology Approach 4th Edition, Irv Englander John Wiley and Sons 2010 PowerPoint slides
More informationReview from last time. CS 537 Lecture 3 OS Structure. OS structure. What you should learn from this lecture
Review from last time CS 537 Lecture 3 OS Structure What HW structures are used by the OS? What is a system call? Michael Swift Remzi Arpaci-Dussea, Michael Swift 1 Remzi Arpaci-Dussea, Michael Swift 2
More informationLinux Powered Storage:
Linux Powered Storage: Building a Storage Server with Linux Architect & Senior Manager rwheeler@redhat.com June 6, 2012 1 Linux Based Systems are Everywhere Used as the base for commercial appliances Enterprise
More informationRecover EDB and Export Exchange Database to PST 2010
Recover EDB and Export Exchange Database to PST 2010 Overview: The Exchange Store (store.exe) is the main repository of Exchange Server 2010 edition. In this article, the infrastructure of store.exe along
More informationChapter 11: File System Implementation. Operating System Concepts with Java 8 th Edition
Chapter 11: File System Implementation 11.1 Silberschatz, Galvin and Gagne 2009 Chapter 11: File System Implementation File-System Structure File-System Implementation Directory Implementation Allocation
More informationChapter 11: File System Implementation. Operating System Concepts 8 th Edition
Chapter 11: File System Implementation Operating System Concepts 8 th Edition Silberschatz, Galvin and Gagne 2009 Chapter 11: File System Implementation File-System Structure File-System Implementation
More informationSistemas Operativos: Input/Output Disks
Sistemas Operativos: Input/Output Disks Pedro F. Souto (pfs@fe.up.pt) April 28, 2012 Topics Magnetic Disks RAID Solid State Disks Topics Magnetic Disks RAID Solid State Disks Magnetic Disk Construction
More informationStorage and File Structure
Storage and File Structure Chapter 10: Storage and File Structure Overview of Physical Storage Media Magnetic Disks RAID Tertiary Storage Storage Access File Organization Organization of Records in Files
More informationChapter 12 File Management
Operating Systems: Internals and Design Principles, 6/E William Stallings Chapter 12 File Management Dave Bremer Otago Polytechnic, N.Z. 2008, Prentice Hall Roadmap Overview File organisation and Access
More informationChapter 12 File Management. Roadmap
Operating Systems: Internals and Design Principles, 6/E William Stallings Chapter 12 File Management Dave Bremer Otago Polytechnic, N.Z. 2008, Prentice Hall Overview Roadmap File organisation and Access
More informationPhysical Data Organization
Physical Data Organization Database design using logical model of the database - appropriate level for users to focus on - user independence from implementation details Performance - other major factor
More informationLinux Filesystem Comparisons
Linux Filesystem Comparisons Jerry Feldman Boston Linux and Unix Presentation prepared in LibreOffice Impress Boston Linux and Unix 12/17/2014 Background My Background. I've worked as a computer programmer/software
More informationHow do Users and Processes interact with the Operating System? Services for Processes. OS Structure with Services. Services for the OS Itself
How do Users and Processes interact with the Operating System? Users interact indirectly through a collection of system programs that make up the operating system interface. The interface could be: A GUI,
More informationHow To Recover From Failure In A Relational Database System
Chapter 17: Recovery System Database System Concepts See www.db-book.com for conditions on re-use Chapter 17: Recovery System Failure Classification Storage Structure Recovery and Atomicity Log-Based Recovery
More informationThe Classical Architecture. Storage 1 / 36
1 / 36 The Problem Application Data? Filesystem Logical Drive Physical Drive 2 / 36 Requirements There are different classes of requirements: Data Independence application is shielded from physical storage
More informationCS 153 Design of Operating Systems Spring 2015
CS 153 Design of Operating Systems Spring 2015 Lecture 22: File system optimizations Physical Disk Structure Disk components Platters Surfaces Tracks Arm Track Sector Surface Sectors Cylinders Arm Heads
More information<Insert Picture Here> Btrfs Filesystem
Btrfs Filesystem Chris Mason Btrfs Goals General purpose filesystem that scales to very large storage Feature focused, providing features other Linux filesystems cannot Administration
More informationCS 377: Operating Systems. Outline. A review of what you ve learned, and how it applies to a real operating system. Lecture 25 - Linux Case Study
CS 377: Operating Systems Lecture 25 - Linux Case Study Guest Lecturer: Tim Wood Outline Linux History Design Principles System Overview Process Scheduling Memory Management File Systems A review of what
More informationLast Class: OS and Computer Architecture. Last Class: OS and Computer Architecture
Last Class: OS and Computer Architecture System bus Network card CPU, memory, I/O devices, network card, system bus Lecture 3, page 1 Last Class: OS and Computer Architecture OS Service Protection Interrupts
More informationPractical Online Filesystem Checking and Repair
Practical Online Filesystem Checking and Repair Daniel Phillips Samsung Research America (Silicon Valley) d.phillips@partner.samsung.com 1 2013 SAMSUNG Electronics Co. Why we want online checking: Fsck
More informationTransaction Management Overview
Transaction Management Overview Chapter 16 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Transactions Concurrent execution of user programs is essential for good DBMS performance. Because
More informationConfiguring Apache Derby for Performance and Durability Olav Sandstå
Configuring Apache Derby for Performance and Durability Olav Sandstå Database Technology Group Sun Microsystems Trondheim, Norway Overview Background > Transactions, Failure Classes, Derby Architecture
More informationChapter 7. File system data structures. File system layout. Code: Block allocator
DRAFT as of November 19, 2010: Copyright 2009 Cox, Kaashoek, Morris Chapter 7 File system data structures The disk driver and buffer cache (Chapter 6) provide safe, synchronized access to disk blocks.
More informationReconFS: A Reconstructable File System on Flash Storage
ReconFS: A Reconstructable File System on Flash Storage Youyou Lu, Jiwu Shu, and Wei Wang, Tsinghua University https://www.usenix.org/conference/fast14/technical-sessions/presentation/lu This paper is
More informationFile Systems for Flash Memories. Marcela Zuluaga Sebastian Isaza Dante Rodriguez
File Systems for Flash Memories Marcela Zuluaga Sebastian Isaza Dante Rodriguez Outline Introduction to Flash Memories Introduction to File Systems File Systems for Flash Memories YAFFS (Yet Another Flash
More information