File system interface Pre Final
- 1. 1 CPU-Scheduling (Galvin) Outline FILE CONCEPT o File Attributes (Name, Identifier, Type, Location, Size, Protection, Time & Date, UserID) o File Operations o File Types o File Structure o Internal File Structure ACCESS METHODS o Sequential Access o Direct Access o Other Access Methods DIRECTORY STRUCTURE o Storage Structure o Directory Overview o Single-Level Directory o Two-Level Directory o Tree-Structured Directories o Acyclic-Graph Directories o General Graph Directory FILE-SYSTEMMOUNTING FILE SHARING o Multiple Users o Remote File Systems (The Client-Server Model, Distributed Information Systems, Failure Modes) o Consistency Semantics (UNIX Semantics, Session Semantics, Immutable-Shared-Files Semantics) PROTECTION o Types of Access o Access Control o Other Protection Approaches and Issues Contents FILE CONCEPT File Attributes Different OSes keeptrack ofdifferent file attributes, includingName, Identifier (e.g. inode number), Type (Text, executable, other binary, etc.), Location (E.g., Hard drive), Size, Protection, Time & Date, User ID. Some systems give special significance to names, and particularlyextensions (.exe, .txt, etc.), and some donot. Some extensions maybe of significance to the OS (.exe), andothers onlyto certain applications (.jpg). File Operations The file ADT supports manycommonoperations:Creating a file, Writing a file, Readinga file, Repositioning withina file, Deletinga file, Truncatinga file. Informationabout currentlyopenfiles is storedinan openfile table, containing for example:
- 2. 2 CPU-Scheduling (Galvin) o File pointer - records the current position inthe file, for the next reador write access. o File-open count - How manytimes has the current file beenopened(simultaneouslybydifferent processes)andnot yet closed? When this counter reacheszerothe file canbe removed from the table. o Disk location of the file. o Access rights Some systems provide support for file locking. o A shared lock is for readingonly. o An exclusive lock is for writing as well as reading. o An advisory lock is informationalonly, andnot enforced. (A "KeepOut" sign, whichmaybe ignored.) o A mandatory lock is enforced. (A trulylockeddoor.) UNIXusedadvisory locks, andWindows uses mandatorylocks. File Types Windows (andsome other systems) use specialfile extensions to indicate the type of each file. Macintoshstores a creator attribute for eachfile, according to the program that first createdit with the create() system call. Macintoshstores a creator attribute for eachfile, accordingto the program that first createdit withthe create()systemcall. File Structure Some files containaninternal structure, whichmayor maynot be knownto the OS. For the OS to support particular file formats increasesthe size andcomplexityof the OS. UNIXtreats all files as sequences of bytes, withnofurther considerationof the internal structure. (Withthe exceptionof executable binary programs, whichit must know how to load and findthe first executable statement, etc.) Macintosh files have two forks - a resource fork, and a datafork. The resource forkcontains informationrelatingto the UI, such as icons and button images, and canbe modified independentlyof the data fork, which contains the code or data as appropriate. Internal File Structure Diskfiles are accessed in units of physical blocks, typically512 bytes or some power-of-twomultiple thereof. (Larger physical disks use larger block sizes, to keepthe range ofblock numbers withinthe range of a 32-bit integer.) Internallyfiles are organizedinunits oflogical units, which maybe as small as a single byte, or maybe a larger size corresponding to some data record or structure size. The number of logical units which fit into one physical block determines its packing, and has animpact on the amount of internal fragmentation(wasted space) that occurs. As a general rule, half a physicalblockis wastedfor eachfile, andthe larger the block sizes the more space is lost to internalfragmentation. ACCESS METHODS Sequential Access: A sequentialaccessfile emulates magnetic tape operation, andgenerallysupports a few operations: a)readnext - read a record andadvance the tape to the next position. b)write next - write a record andadvance the tape to the next position. c) rewind d) skipn records - Mayor maynot be supported. N maybe limitedto positive numbers, or maybe limitedto +/- 1. Direct Access: Jump to anyrecord andread that record. Operations supportedinclude: read n - readrecord number n. (Note an argument is now required.) write n - write recordnumber n. (Note anargument is now required.) jumpto recordn - couldbe 0 or the endof file. Query current record - usedto return backto this record later. Sequential access canbe easilyemulated using direct access. The inverse is complicatedandinefficient. Other Access Methods: An indexed access scheme canbe easilybuilt ontop ofa direct access system. Verylarge files mayrequire a multi- tieredindexingscheme, i.e. indexes of indexes. (Lot of cool and relevant content is there in the book for all chapters) DIRECTORY STRUCTURE Storage Structure: A diskcanbe usedinits entiretyfor a file system. Alternativelya physical diskcanbe brokenup into multiple partitions, slices, or mini-disks, each of which becomes a virtual diskand canhave its own filesystem. (or be usedfor raw storage, swapspace, etc.)Or, multiple physicaldisks can be combinedintoone volume, i.e. a larger virtual disk, withits own filesystem spanning the physicaldisks.
- 3. 3 CPU-Scheduling (Galvin) Directory Overview: Directoryoperations to be supported include: a) Search for a file, b)Create a file (addto the directory) C) Delete a file (erase fromthe directory) d) List a directory(possiblyorderedin different ways) e)Rename a file (maychange sorting order) f) Traverse the file system. Single-Level Directory: Simple to implement, but each file must have a unique name. Two-Level Directory: Each user gets their own directoryspace. File names only need to be unique within a givenuser's directory. A master file directoryis used to keep track of each users directory, andmust be maintained when users are added to or removedfrom the system. A separate directoryis generallyneeded for system(executable) files. Systems mayor maynot allowusers to access other directoriesbesides their own If access to other directories is allowed, thenprovision must be made to specifythe directorybeing accessed. Ifaccessis denied, thenspecial consideration must be made for users to run programs locatedin system directories. A searchpath is the list of directories in whichto searchfor executable programs, andcan be set uniquelyfor each user. Tree-Structured Directories: This is an obvious extensionto the two-tiereddirectorystructure. Eachuser / processhas the concept of a current directoryfrom whichall (relative) searches take place. Files maybe accessedusing either absolute pathnames (relative to the root of the tree) or relative pathnames(relative to the current directory.) Directories are storedthe same as anyother file in the system, except there is a bit that identifies them as directories, andtheyhave some specialstructure that the OS understands. Acyclic-Graph Directories: When the same files needto be accessed in more thanone place inthe directorystructure (e.g. because theyare being sharedbymore thanone user / process), it can be useful to provide anacyclic-graph structure. UNIXprovidestwo types of links for implementing the acyclic-graph structure. A hardlink (usuallyjust called a link) involves multiple directoryentries that bothrefer to the same file. Hardlinks are onlyvalid for ordinaryfilesinthe same filesystem. A symbolic link, that involves a special file, containing information about where to find the linkedfile. Symbolic links maybe used to link directories and/or filesin other filesystems, as well as ordinaryfilesinthe current filesystem. Windows onlysupports symbolic links, termedshortcuts. Hard links require a reference count, or linkcount for each file, keeping track of howmanydirectoryentries are currently referring to this file. Whenever one of the referencesis removedthe linkcount is reduced, andwhenit reaches zero, the diskspace canbe reclaimed. General-Graph Directory: If cycles are allowedinthe graphs, thenseveralproblems canarise: Search algorithms cango intoinfinite loops. One solution is to not followlinks in searchalgorithms. (Or not to followsymbolic links, and to onlyallowsymbolic links to refer to directories.) Sub- trees can become disconnected from the rest ofthe tree andstill not have their reference counts reducedto zero. Periodic garbage collection is requiredto detect andresolve this problem. (chkdsk inDOS and fsck in UNIXsearch for these problems, amongothers, eventhough cycles are not supposedto be allowedineither system. Disconnecteddisk blocks that are not markedas free are added back to the file systems with made-upfile names, andcan usuallybe safelydeleted.).ReferFigure11.3
- 4. 4 CPU-Scheduling (Galvin) FILE SYSTEM MOUNTING The basic idea behind mounting file systems is to combine multiple file systems intoone large tree structure. The mount command is given a filesystem to mount anda mount point (directory) on which to attachit. Once a file system is mountedontoa mount point, anyfurther references to that directoryactuallyrefer to the root of the mountedfile system. Any files (or sub-directories)that hadbeenstored inthe mount point directory prior to mounting the newfilesystem are nowhiddenbythe mounted filesystem, and are no longer available. For this reason some systems only allowmountingontoemptydirectories. Filesystems canonlybe mountedbyroot, unless root has previously configured certainfilesystems to be mountable ontocertainpre- determined mount points. (E.g. root mayallow users to mount floppy filesystems to /mnt or somethinglike it.) Anyone canrunthe mount commandto see what filesystems are currentlymounted. Filesystems maybe mountedread-only, or have other restrictions imposed. The traditional Windows OS runs anextendedtwo-tier directorystructure, where the first tier of the structure separatesvolumesbydrive letters, and a tree structure is implemented belowthat level. Macintoshruns a similar system, where eachnew volume that is found is automaticallymountedand added to the desktop whenit is found. More recent Windows systems allow filesystems to be mountedto anydirectoryinthe filesystem, muchlike UNIX. FILE SHARING Multiple Users: On a multi-user system, more informationneeds to be stored for eachfile: The owner (user)whoowns the file, andwhocan control its access. The group ofother user IDs that mayhave some specialaccessto the file. What access rights are afforded to the owner (User), the Group, and to the rest of the world (the universe, a.k.a. Others.) Some systems have more complicatedaccesscontrol, allowing or denying specific accessesto specificallynamedusers or groups. Remote File Systems: The advent ofthe Internet introduces issuesfor accessing files storedonremote computers The original methodwas ftp, allowing individual filesto be transportedacross systems as needed. Ftp can be either account andpassword controlled, or anonymous, not requiring anyuser name or password. Various forms of distributedfile systems allowremote file systems to be mountedontoa local directorystructure, andaccessedusing normal file access commands. (The actualfiles are still transportedacrossth e network as needed, possiblyusing ftpas the underlyingtransport mechanism.)The WWW hasmade it easyonce againto access files onremote systems without mountingtheir filesystems, generallyusing (anonymous)ftp as the underlying file transport mechanism. The Client-Server Model: When one computer system remotelymounts a filesystem that is physicallylocated onanother system, the system which physicallyowns the filesacts as a server, andthe systemwhichmounts them is the client. User IDs and gro upIDs must be consistent across bothsystems for the systemto work properly. (I.e. this is most applicable across multiple computers managed bythe same organization, sharedbya common groupof users.) The same computer canbe both a client anda server. (E.g. cross-linkedfile systems.). The NFS (NetworkFile System) is a classic example of sucha system. Distributed Information Systems: The DomainName System, DNS, provides for a unique naming system acrossall of the Internet. Domain names are maintainedbythe Network Information System, NIS. Microsoft's CommonInternet File System, CIFS, establishes a network login for each user ona networked system withsharedfile access. Older Windows systems useddomains, andnewer systems (XP, 2000), use active directories. User names must match acrossthe network for thissystemto be valid. A newer approachis the Lightweight Directory-Access Protocol, LDAP, which providesa secure single sign-onfor all users to accessallresources ona network. Thisis a secure system which is gaininginpopularity, andwhich has the maintenance advantage of combining authorizationinformationin one central location . Consistency Semantics: Consistency Semantics dealswith the consistencybetweenthe views of sharedfileson a networkedsystem. When one user changes the file, when doother users see the changes? PROTECTION Files must be kept safe for reliability(against accidental damage), and protection (against deliberate malicious access.) The former is usuallymanaged with backup copies. This section discusses the latter. Types of Access: The following low-level operations are oftencontrolled: o Read - View the contents of the file o Write - Change the contents ofthe file. o Execute - Loadthe file onto the CPU and follow the instructions contained therein. o Append - Add to the endof an existing file. o Delete - Remove a file from the system. o List -View the name andother attributes offiles onthe system. Higher-level operations, suchas copy, cangenerallybe performedthrough combinations ofthe above.
- 5. 5 CPU-Scheduling (Galvin) Access Control: One approach is to have complicated Access Control Lists, ACL, which specifyexactlywhat access is allowedor denied for specific users or groups. The AFS usesthis system for distributedaccess. Control is veryfinelyadjustable, but maybe complicated, particularly when the specific users involved are unknown. (AFSallows some wildcards, so for example all users on a certainremote system maybe trusted, or a givenusername maybe trustedwhenaccessing fromanyremote system.) UNIXuses a set of 9 access control bits, in three groups of three. These correspondto R, W, and Xpermissions for eachof the Owner, Group, and Others. (See"manchmod" for full details.) The RWXbits control the following privileges for ordinaryfilesanddirectories: bit Files Directories R Read (view) file contents. Read directory contents. Required to get a listing of the directory. W Write (change) file contents. Change directory contents. Required to create or delete files. X Execute file contents as a program. Access detailed directory information. Required to get a long listing, or to access any specific file in the directory. Note that if a user has X but not R permissions on a directory, they can still access specific files, but only if they already know the name of the file they are trying to access. In addition there are some special bits that canalsobe applied:The set user ID (SUID)bit and/or the set group ID (SGID) bits appliedto executable files temporarily change the identityof whoever runs the program to match that ofthe owner / groupof the executable program. Thisallows users running specific programs to have access to files (whilerunning that program) to which theywouldnormallybe unable to access. Setting of these twobits is usuallyrestricted to root, andmust be done withcaution, as it introduces a potential securityleak. Windows adjusts files access througha simple GUI. Other Protection Approaches and Issues: o Older systems which didnot originallyhave multi-user file access permissions (DOS andolder versions of Mac)must now be retrofittedif theyare to share files ona network. o Access to a file requires access to all the files along its pathas well. Ina cyclic directorystructure, users mayhave different access to the same file accessedthroughdifferent paths. o Sometimesjust the knowledge of the existence ofa file ofa certain name is a security(or privacy) concern. Hence the distinction betweenthe R andXbits onUNIXdirectories. AssortedContent XXX To be cleared I Glossary ReadLater Further Reading S