File Management Lecture 15b 1 2 File Management File management system consists of system utility programs that run as privileged applications Input to applications is by means of a file Output is saved in a file for long-term storage File System Properties Long-term existence Sharable between processes Structure 3 4
File Operations File Management Systems Create Delete Open Close Read Write The way a user of application may access files Programmer does not need to develop file management software 5 6 Layers of Access to File Data Also known as: File organization/access methods Basic I/O system Physical I/O 7 As is often the case in operating systems, each layer should (in theory) talk to the layer above and the layer below it only 8
Device Drivers Lowest level Communicates directly with peripheral devices Responsible for starting I/O operations on a device Processes the completion of an I/O request Basic File System Physical I/O Deals with exchanging blocks of data Concerned with the placement of blocks Can be concerned with buffering blocks in main memory 9 10 Basic File System Contd This layer determines the temporal ordering of writes to make efficient use of the disk For example At the user level we want to write: A, B, C, D in this order But the Physical I/O layer could by default write out: A, C, B, D in this order Perhaps because A and C are near each other on the disk, and B and D are near each other on the disk 11 Basic I/O Supervisor Responsible for file I/O initiation and termination Control structures are maintained Concerned with selection of the device on which file I/O is to be performed Concerned with scheduling access to optimize performance Again, can be concerned with buffering Part of the operating system 12
Logical I/O Enables users and applications to access records Provides general-purpose record I/O capability Maintains basic data about file Access Method Reflect different file structures Different ways to access and process data See last lecture slide 3 13 14 I/O Buffering Reasons for buffering: if we didn t Processes would have to wait for I/O to complete before proceeding Certain pages would have to remain in main memory during I/O (e.g. not being paged out to save memory) Instead if e.g. block-oriented Input transfers made to buffer Block moved to user space when needed User process can process one block of data while next block is read in 15 16
I/O Buffering 17 18 Blocking and File Space Allocation See Appendix 19 20
Windows File System Key features of NTFS Recoverability Security Large disks and large files Multiple data streams E.g to allow Windows to be used by remote Macintosh systems, in which each file has file data and file information data streams' General indexing facility Also: Caching to increase performance (of course) RAID support etc NTFS Recoverability A Ability to recover from system crashes and disk failures. In the event of such failures, NTFS is able to reconstruct disk volumes and return them to a consistent state. It does this by using a transaction processing model for changes to the file system each significant change is treated as an atomic action that is either entirely performed or not performed at all. Each transaction that was in process at the time of a failure is subsequently backed out or brought to completion. 21 22 NTFS Recoverability A What if NTFS didn t have this recoverability feature? What is an inconsistent state? In the file system/file management system data stored includes: File system data structures as mentioned on slides in this lecture and previous lecture (e.g. pretty much everything that has to be stored, except the file data) The file data itself If we are making a change to a file and we get half way through updating the above data and then stop because something goes wrong: we might be fooled into later on, thinking the change has been fully made 23 NTFS Recoverability A So where does this fit in?? Works in the upper layers of the access to file data diagram E.g. the Basic I/O Supervisor layer (also known as Basic I/O system) As this is a high-level function; maintaining the consistency of the file system 24
NTFS Recoverability In addition: NTFS uses redundant storage for critical file system data, so that failure of a disk sector does not cause the loss of data describing the structure and status of the file system. 25 NTFS Recoverability It is important to note that the recovery procedures used by NTFS are designed to recover file system data, not file contents. Thus, the user should never lose a volume or the directory/file structure of an application because of a crash. However, user data are not guaranteed by the file system. Providing full recoverability, including user data, would make for a much more elaborate and resource-consuming recovery facility. 26 NTFS Recoverability: A The essence of the NTFS recovery capability is logging. Each operation that alters a file system is treated as a transaction. Each suboperation of a transaction that alters important file system data structures is recorded in a log file before being recorded on the disk volume. Using the log, a partially completed transaction at the time of a crash can later be redone or undone when the system recovers. This log is sometime referred to as the journal, and the operations that are logged the journalled operations 27 NTFS Recoverability: A In general terms, these are the steps taken to ensure recoverability, NTFS first calls the log file system to record in the log file in the cache any transactions that will modify the volume structure. NTFS modifies the volume (in the cache). The changes are written to disk 28
Exercise Read up about other file system structures for extreme levels of recoverability, on the web If you have a highly mission-critical application what other kind of relatively unusual file systems might you use? 29 30 File Management Lecture 15b 31