Lecture 07- The FAT File Systemgoogle.pptx

Editor's Notes

• #2: We're going to talk about the FAT file system. We're going to talk about the history and versions of FAT. We're going to talk about the data structures that make up the FAT file system, things like the root directory and the volume boot record. We're also going to talk about the directory structure within the FAT file system. We're going to talk about file and directory creation and deletion. What happens when we create a file? What happens when we delete it? What occurs when we create a directory? And what happens when we delete that directory? We're also going to talk about recovering data in the FAT file system and some of the limitations of data recovery.
• #3: Let's start out by defining what a file system is. A file system is a method of reading, writing, tracking, and recovering data from a drive. Your file system tracks the location of the data on your drive so it knows where all your files, folders, and applications are. So, when we double click on a file or an application, the file system knows where to go to retrieve that data. It also tracks which clusters are in use, being occupied by a file. And which clusters are available for us to write a file to, or save an application to.
• #4: The history of the FAT file system. When we started out with FAT12, which was 2 to the 12th power of bits, that's how many bits we had to work with. Which gave us 4,096 maximum clusters for the volume. So we can only have 4,096 clusters on a FAT12 volume. Now FAT16 which was 2 to the 16th power of bits to work with. And this gave us 65,536 maximum clusters for the volume. So it's better but it's still not something you're going to see very often today. Considering the size of drives, these file systems really are not going to be applicable.
• #5: The next version of FAT we see in the evolution is FAT32. Now FAT32 you will see today on removable media, and it is possible to format a partition on an internal drive on a Windows system FAT32. FAT32 uses 28 bits not 32, 4 bits are reserved. So it's 2 to the 28th power of bits that we have to work with. Which gives us 268,435,456 maximum clusters that we can have on a FAT32 volume. Now that's quite a bit more than we had with FAT16. The latest version of FAT is ExFAT. An ExFAT does use all 32 bits, so we have 32 bits to work with in ExFAT. Which is 2 to the 32nd power. And you can see the number on the slide of maximum clusters it is quite a bit larger than FAT32. And we are going to have a whole section on ExFAT throughout this path.
• #6: The FAT file system overview, well we have a system area and a data area. Now in FAT12 and 16, the system area which comprised of the volume boot records, FAT1 and 2, and the root directory. And then we had our files and folders saved out in the data area. Now because the root directory was in the system area on FAT12 and 16, the size was limited to one cluster. So we only had one cluster to work with when we were addressing our files. So that did cause some issues. So we went to FAT32. Now in FAT32, the volume boot record is in the system area and so is FAT1 and 2. But the root directory in FAT32 was moved out to the data area to allow the root directory to expand beyond that one cluster limitation, allowing us to write a lot more files, folders, directories to our volumes. We're going to talk about the FAT volume boot record and see the important information that is contained in that record.
• #7: The FAT 32 volume boot record, it's located in the first sector of the volume, which is going to be logical sectors 0. Now, don't confuse this with the first sector of the disk, which is Physical sector 0. This is the first sector of the logical volume. This file is going to contain information about the volume. Some of the information of interest to us in the volume boot record. What we're going to see the OEM ID, which will most likely be MS-DOS5.0. This will be the volume OEM ID for operating systems, Windows 2000 and up. Most likely that's the only OEM ID we're going to see. We're going to be interested in bytes per sector, which is usually going to be 512. We're usually going to have 512 bytes in a sector. We also need to know how many sectors are in a cluster. This is going to be very important when we're navigating the disk. Sectors per cluster varies. It can either be selected by the operating system or the user does have the option to change that if they would like. We're going to see the number of FATs. There should be two, one is for backup purposes. We're going to see hidden sectors. These are sectors proceeding the volume. These are the sectors on the physical drive that are in front of or preceding the volume itself. We're going to see total sectors. Now, total sectors, times sector signs will give us the size of the volume. We need to know how big the volume is that we're looking at.
• #8: This is more information that we're going to look at in the FAT 32 volume boot record. We're going to look at sectors per FAT, how many sectors are in each FAT? We want to know the starting cluster of the root directory because the root directory is going to be a very important part of the FAT file system that we need to look at. That's usually going to be clustered too. We also want to know where the backup boot sector is located. The backup boot sector will usually be in cluster 6. We're going to want to know the volume serial number because most likely you're going to be looking at FAT on removable media and we're going to need the volume serial number to track that drive across volumes. We're also going to be interested in the volume name, note the volume name in the boot record is not going to be the user-created volume name that's going to be in the root directory. It'll usually say no name in the volume boot record. We're going to want to know the file system type. That's also information in the boot record. Now, do not confuse the volume boot record with the master boot record. The master boot record relates to the physical dense and volume boot record relates to the logical volume.
• #9: We can see here in the chart, all the important information we just talked about in the chart shows us the offsets where we can find that information. These offsets are relative to the start of the Data Structure. If we're looking at the volume boot record, the offsets you see, they're relative to the start of a volume boot record, not the start of the physical drive. Just keep that in mind. The start of the Data Structure.
• #10: This is a look at a FAT 32 volume book record. This is what it's going to look like. We'll look at it with our tools. It's located on logical sector 0 and these values are going to be interpreted little-endian. The volume boot record is going to be one sector in size, it's 512 bytes, one sector.
• #11: This is a look at a FAT 32 volume book record. This is what it's going to look like. We'll look at it with our tools. It's located on logical sector 0 and these values are going to be interpreted little-endian. The volume boot record is going to be one sector in size, it's 512 bytes, one sector.
• #12: This is a look at a FAT 32 volume book record. This is what it's going to look like. We'll look at it with our tools. It's located on logical sector 0 and these values are going to be interpreted little-endian. The volume boot record is going to be one sector in size, it's 512 bytes, one sector.
• #13: This is a look at what the root directory looks like when we do it with our tools. We can see we have entries at the top and then we have an entry starting with 00. Now, the system will stop reading here. Your file system will stop when it comes to these 00s. But be aware, it is possible for data to be written in using a hex editor and another tool beyond those 00s, and this data would be hidden from the file system.
• #14: There are three types of root directory entries. We can see a volume name entry and this would be a user created name for the volume. This would always be the top entry in the root directory. We also have what we call short filenames entry. Every file and directory will have one of these short filename entries. What short filename means is it means that the name of the file conforms with that 8.3 naming convention, which means it has up to eight uppercase characters, a dot, and then a three-character file extension. If a file does not conform to this, it will also have what we call long filename entries. There'll be a short filename alias, which will be six characters of the file name, followed by a tilde, and then a three-character extension. But the full filename will be in the long filename entries, and these will be located above the short filename.
• #15: This is a look at a volume directory entry. Remember, we said we had three types and this is the look at a volume name entry. This would be the top entry in the directory if there was a user-created volume name. We can see it begins with the filename, which in our case is FAT 32. It also contains attributes such as read-only in system volume or directory. It has also a modified date and time. Because it is a volume directory entry, it will only have that one timestamp. When we look at a file entry, you will see three timestamps.
• #16: The short filename entry, just to reiterate, is eight uppercase characters and a three-character file extension, and it is referred to as 8.3 or DOS-compliant. The long filename happens when we mix cases where we have a filename that is longer than eight characters or an extension longer than three characters. This is what causes those long filename entries to be created.
• #17: Here's a look at a directory entry set. Down here at the bottom, we can see our short filename entry. Every file and directory will have one of these entries. We see the DOS alias, that six character, all uppercase, a tilde, and then an extension. We also see above the long filename entries. We're going to talk about how we read these long filename entries as we go through this course. But this is what a directory entry set would look like.
• #18: The short filename entry, in this case, we have our filename, SHORT.TXT, and it was named. You can see this is not an alias, there's no tilde D if you look over in the ASCII. This is the name of the file and it would have no long filename entries above it. It's all capital letters, up to eight characters, and then it has that three-character TXT extension. You can see in here, we have the attributes and the attributes would be located at Offset 0B for a length of one byte. That's what we call a packed byte. We break that out to binary, to see which flags were set. In other words, which bits were turned on, and that would tell us which attributes this file had. In our case, it just has the archive bit set. Then we see at Offset 0D, the final time millisecond refinement. This is just a millisecond timestamp that we're going to add to our created date and timestamp. It only refers to the created date. Below that, and at Offset 0E, we see our created date and time. In this case, it's 6/25/2020 at 1:13 PM and that is going to be a length of four bytes. These two bytes at 0E and 0F of the time, and at 01 and 02 would be our date. We have a last access date. Last Access just has a date. You can see that it's 02 and 03 highlighted here in green. This is our last access date. There's no time with that. Next to that we have what we call our first cluster high word. In this case it's 00. The high word is only used if we need more than two bytes to address the cluster. In this case, we did not. Next, we have our modified date and time. Again, the first two bytes are 06 and 07, and these are the time. Then the next two bytes would be the date. You can see those located at offset 108 and 109. Next we have our first cluster little word. Now, since we did not have a high word, this two byte value represents the starting cluster of this file. Then the last four bytes of the entry are the file size in bytes.
• #19: These are the possible values we can have for that attribute byte. Remember, it's a one-byte value, and it's a packed byte. It gets broken out into binary, and we can see which flags are up. It is possible to have more than one attribute. You could have a read-only system file. In which case should have a hex value of 05. When you broke that out, you would see that those two bits were set.
• #20: The reserve byte off at 0C, is what we call a case indicator. Reserve byte, if it is hexadecimal 1,0, that means this is a short final day. It is 8.3 compliant file name. If at the reserve byte we saw, a hexadecimal 00, that would indicate you're looking at a long file name and it was not an 8.3 compliant filename. That's when we can identify a short filename from a long filename.
• #21: File time. FAT, 32 file time in all FAT file times are going to be in local time, whatever time the machine is set to. They're not recorded in Universal Time, UTC. Here we see at offset 0E and 0F are creation time. Down at offset 1,0 and 1,1, we see our creation date. They are read together as a 32 bit value.
• #22: This is how the value would be converted into a date and time. Because if you just converted this two-byte value, you're going to get a number, not a date time. But what happens is, your file system is first going to convert those two bytes for time, until little-endian. We saw in the previous slide, it was AB69. Now this is converted to little-endian, so we have 69AB. These bytes are separated. Then each nibble is separated. The six and the nine, and the A and the B. Then each nibble is individually translated from hex to a binary value. Once we get each binary value, we can then translate the date and time. We are allowed five bits for the seconds. We take the bits and you can see here at the far right of the screen outlined in yellow, these would be our bits per seconds. We would fill them in to the column values, and since we have five, we have column values from 1-16, 1, 2, 4, 8, 16. Each column value increasing by a power of two. We can see in our example, we have 11 seconds. We must multiply this by two, because we don't have enough bits here to account for 60 seconds. When we multiply 11 by 2, we get 22. Now we're going to move to minutes. We have more bytes to work within minutes. We're allowed six bytes. So we have column values from 1-32 and you can see them in the slide outlined in green. We take our binary values that we converted from A and drop those down into the column values. We can see we have 13 minutes. Then we would move on to hours. You can see hours at the left-hand side here, outlined in red. We would have, again, five bytes toward the hour. We have column values from 1-16. We drop down our binary conversion of our decimal number six into the columns and we can see we end up with 13. Our time would be 13:13:22 PM. We could go back and add in our creation time millisecond value.
• #23: The FAT creation date is translated in much the same way as the FAT creation time. It is a little endian value, so we would have to translate that. Original value was D950. We converge it here to little endian and we have 50D9. Again, we break the bytes apart, 50 and D9, and then we split the nibbles, the 5, the 0, the D, and the 9. We convert each nibble into its binary value. When we're translating this, we have five bits for the day. We have column values one through 16. Our nibble was nine, so we would drop down our column value nibbles into the correct column values for nine. But we also have one of the D column values since we're using five bytes. For months, we have four bits to work with, and they are outlined in green. We take the next four bits of our value and drop them down into the proper column values. Our year is outlined in red. We take the remaining bytes that we have and we would drop them down into their proper column values. We have seven bits for the year. We come up with a number of decimal 40. Now, the FAT file system uses an absolute date of January 1st, 1980. So we would have to add 1980 to 40, and we come up with 2020. For the months, we come up with six, which would be the sixth month and for the date we come up with 25. That makes our creation date 25, June 2020.
• #24: That brings us to our last access date. This is translated the same way we do for the creation date. Last accessed only has a date, not a time. You can see in the slide, you have D9 50. You must convert that to little-endian, 50 D9 and that gives us a date of the 25th of June, 2020. We just converted that in the previous slide.
• #25: Now we're going to talk about the FAT high word. The FAT high word values start at 65,536 and they continue up, a power of two for each column value. If you had a high word, you'd be reading it as a string of four bytes. So it'd be a 32-bit value it'd be four bytes long. We would have to account for 32 bits. We'd start with one, with two, doubling all the way as we go out, to get to 32 bits. We need place values for 32 bits. Then we would split our bytes to the low and the high word. We'd first convert the low word. To convert from hex to binary, we're going to convert each nibble at a time. When we convert nibbles, remember back in the path when we went over that, we used the values one through eight for each nibble. Because a nibble only has four bits. Four bits would be 1, 2, 4, 8. Those would be our 4-bit values. We look at the one. We would start with eight. Eight can't go into one, four does not go into one, two does not go into one, one goes into one. We put a one. We'd have a one in the ones column, a zero in the two, a zero in the four, and a zero in the eight. Then we would move to the seven. We do the same thing. We'd have column values of 1, 2, 4, and 8. Because we're converting a nibble which is four bits long. Eight won't go into seven so we put a zero in the eights column. Four will go into seven, so we put a one, and we subtract 4 from 7, which would give us a remainder of 3. Two will go into three one time, and one will go into one time. That is how we would convert the byte. We would do the same thing for 2E. We would convert the E, which in hexadecimal E is 14. We would start with our eights. Eight will go into 14 one time, so we would put a one in the eight. That will give us a remainder of six. Four will go into six one time. It will give us a remainder of two. Two will go into two once. We put a one in that column and then we would have zero. Zero will not go into one so we would put a zero in the ones column. Then we continue with two. We'd start with eight, eight will not go into two so we'd put a zero. Four will not go into two. We'd put a zero. Two will go into two so we would put a one and that would give us a remainder of zero. One will not go into zero so we would put a zero in the ones column. That would bring us to our high word. Our high word value when we split the byte is nine. Then we have all zeros for the rest of our value. We'd start with the nine. Eight will go into nine once. A remainder of one. Four will not go into one, so we put a zero. Two will not go into one so we put a zero in that column. One will go into one, we put a one in that column. Now the rest of our values is zero, so would just put zeros in the remainder of the columns to fill in the 32 bits. Now we go back and add the actual column values for our bits. We would start with 524,288. That's where we have our 1, so we have that number. We'd write that down, we'd go to the next column there where we have a 1, and that would be column 65,536. We would write that number down. Our next column value, we have a 1, we have 8,192, we'd write that number down. The next number where we have a 1 in our column value is 2,048, I'm going to write that down. The next column where we have a 1 in our column values is 1,024, and we will write that down. The next number where we have a 1 in our column values is 512, we write that down. The next number would be 64, 32, 16, and 1. We would write all these numbers down and then add them together. That gives us a total of 601,713. The starting cluster for this file would be cluster 601,713.
• #26: Here we're taking another look at our high word. There's our high word value, 9, and there's our low word value of 71 2E. We convert them to little-Endian, we break them apart, we'd get our values just like we did in the last slide, and we can add those values together, and we come up with our 601,714. Now, your tools are going to do this for you, and you can use the Window's calculator to convert these bytes also. You'd be converting them as a D word, a double word, a 32-bit value.
• #27: The FAT file modification date and time, it is read little-Endian. If you want to manually decode it, you would do it the same as you did for the file creation date and time, just convert it the same way. Again, your tools will do this for you and you'll see that when we do the demo with Active Disk Editor at the end of the module.
• #28: That file size is read little-Endian, it's four bytes, 32-bit value, and it's the file size in bytes, is the size of the file in bytes.
• #29: The long file name directory entry. We're looking at a long file name directory entry. We're looking at a file directory set right here, but we're going to focus on the long file name because we've already talked about the short file name. But you can see the DOS alias in slide, and now we can see the long file name entries.
• #30: The first byte of a long file name entry is called the sequence byte. Now, the right nibble is the sequence order. The right nibble of that byte, you can see we have 1, 2, 3, and 4, that's the order that this file name is read. The left nibble is going to indicate the last entry in the set. You can see we have a 01, 02, a 03, and look, we have a 44. That 4 indicates the last entry in the directory set. It could have been 43 or 42, depending on the length of the file, but the first nibble, the right nibble represents the order, the left nibble indicates the last entry. We've talked about the status byte when we're talking about the long file name. The rest of the long file name pretty much only gives us the name of the file.
• #31: We've talked about the status byte when we're talking about the long file name. The rest of the long file name pretty much only gives us the name of the file. There is a long file name flag at offset 0B, you'll see a value of 0F. When you see that value of 0F at offset 0B, that tells you you're looking at a long file name. We can also see the reserved byte is 00. We saw a few slides back that, that also indicates a long file name. That is how you can tell you're looking at a long file name entry.
• #33: The facts of the file allocation table is the call. What they do is they track cluster allocation which clusters are in use being occupied by a file and which clusters are not in use and available to be written to? File allocation table has an entry for every cluster on the volume.
• #34: File allocation table has an entry for every cluster on the volume. Fat 1 and fat 2 should be the same because Fat 2 is a backup for recovery purposes. If that one becomes corrupted, they are located in the system area on all versions of that. And again, they're used to track cluster allocation status, whether the cluster is being occupied by a file or it's available to be written to Each of these entries which represents a cluster on the drive will give us the location of the next cluster of that file.
• #35: Here is kind of a visual example of what the facts do we have? Our system area and our fats are in the system area now out in the data area we have our files, we have a file here just written. And cluster too, cluster two Points to cluster six where the rest of this file lives. So this file is what we call fragmented. Yeah. In other words it's not right next to each other. Part of it is in cluster too. And then the other part of it is in cluster six. If we look at cluster three, Cluster three points us to cluster four and then we have an end of file marker E O f. This would be called a contiguous file Because cluster three is right next to cluster four. And then looking at our diagram and cluster five, we just have an end of file marker which this tells me that the file occupying Cluster five only needs one cluster so it's not big enough to need more than one cluster. A fat writes the data to the first available cluster. So if we deleted one of these and wrote a new file it could go into this empty space.
• #36: Now we're going to see Type three Types of entries in the fat, You're going to see this marker, 0x ffffff0f that indicates end of file That tells us we're either at the end of a cluster chain or if it's only occupies one cluster will just see that marker in the cluster that represents that file on the draft. You could see 000000. And this indicates that the cluster is not being used, it's available to be written to or we'll see a pointer to the next cluster And down here we can see this is what a fat table is going to look like when we view it with our tools. This F A f f f f Of. This is what we call a media descriptor generally indicates fixed disk but it's what we call a media descriptor. These next four bites Tell us the version of Fat one looking at and since they were all maxed out at ff ff ff We know we're looking at Fat 32. Fat 32 has four byte entries. Fat 12 and 16 Have to bite entries for each cluster but we are going to focus on fat 32 because that's most likely what we'll see today. So four Byte entries. These next four bites represent cluster two because our cluster numbering out on our drives is going to start it too. So this represents cluster too and it's pointing to would have to translate that number but it's pointing to another cost around in the drive This would be the Filed clustered 3's representation and cluster 3 has an end of file marker which means the file that is occupying cluster three only takes up one cluster And we had the four bytes for close to four and we have an end of file marker. Cluster five we have an entire file marker. Now we look at cluster six Cluster six points to cluster seven. If we look at the four bytes that represent cluster seven in our fat table, They pointed to cluster eight 9 10. It just goes you can see this is a long continuous file.
• #37: Yes this is showing us what we have in our fat tables. It's the same fat table we looked at in the previous slide we have our media descriptor for bites, then we have our fat type. Then this The next four bytes represents cluster too. And cluster two points out to cluster 42656. If we translated and these are read Little Endiene again, If we translated 0000 86 Ao would come up with 42,665. So, that's what cluster 2 cluster to points to that cluster. Now if we look at cluster 3, Has file marker, cluster 4 has a file marker 5 And then cluster 6 Points to cluster 7. We look at the entry for cluster seven It points to 8 8 points to 9. And this file just goes on to the very end. We have clustered 20 down here Pointing to cluster 21. So that would be how we would look at our fat table. And that would be how we would take a look at our fat table. Yeah.
• #39: The creation process? What happens when files created on our fat file system? Well a directory entry is written in the parent directory and we looked at our directory entries are short file name and our long final name. Directory entries in our last module, The data is written to the first available clusters. So the data is going to get written out on the drive and entries are made in the fat the file allocation table to show the chain of clusters used by the file. And if it only uses one cluster we'll have that end of file marker like we saw in our previous module.
• #40: What happens when we delete a file, the file system? Well the first character of the directory entry set is changed to a hexa decimal E5. And this indicates the file system that the file is deleted. So it just skips it, the clusters in the fat are zeroed out. What this means is those fat directory entries, those fat table entries that we saw in our previous module would be all zeroed out So we would not be able to look at the fat and see that the cluster was occupied from the fats point of view would be all zeros, and that cluster would be available to be written to. The data that's out on the drive remains unchanged. Nothing happens to the data until another file overwrites it, another file gets assigned. Are allocated to those clusters and overwrites the data. But until that time the data stays out on the drive, it remains unchanged.
• #42: The first step recovering a file and the fat file system is we need to change the first bite in the directory sets from Hexi five to a different character. If we know the first letter of the file, if the file has a long file name you can get the first letter from that. If it only has a short file name entry, we will change that back to another character. Any keyboard character will work. The next thing we need to do is we name to change the values in the fat table to indicate that the clusters are allocated from the short file name entry. We will be able to get the starting cluster and the file size so we'll know which file injury and the fat table. We need to change to indicate that it's allocated. And we'll know if we need to put an end of file marker there or if the file takes up more than one cluster we would need to put a pointer to that next cluster. So we will determine the cluster size from the volume boot record. Then we will calculate the number of clusters needed. Once we find out how many clusters we need for our file we will divide the file size by the cluster sides and round up. That's how we're going to determine how many clusters we need and once we determine that we can reach ain't our entries in the fat table and then we will have successfully recovered our file.
• #43: some issues with fat file recovery. Well if we have fragmented files it's going to be harder to recover them because if they are not contiguous it's going to be very hard to tell where that file was in the fat table. Another issue is if the directory entry was reused or overwritten and we don't have that short file in that directory entry to find our starting cluster and file size. That will make it very difficult to recover the file. And the other issue we come across is if the data has been overridden out on the disk.
• #45: some issues with fat file recovery. Well if we have fragmented files it's going to be harder to recover them because if they are not contiguous it's going to be very hard to tell where that file was in the fat table. Another issue is if the directory entry was reused or overwritten and we don't have that short file in that directory entry to find our starting cluster and file size. That will make it very difficult to recover the file. And the other issue we come across is if the data has been overridden out on the disk.
• #46: some issues with fat file recovery. Well if we have fragmented files it's going to be harder to recover them because if they are not contiguous it's going to be very hard to tell where that file was in the fat table. Another issue is if the directory entry was reused or overwritten and we don't have that short file in that directory entry to find our starting cluster and file size. That will make it very difficult to recover the file. And the other issue we come across is if the data has been overridden out on the disk.
• #47: some issues with fat file recovery. Well if we have fragmented files it's going to be harder to recover them because if they are not contiguous it's going to be very hard to tell where that file was in the fat table. Another issue is if the directory entry was reused or overwritten and we don't have that short file in that directory entry to find our starting cluster and file size. That will make it very difficult to recover the file. And the other issue we come across is if the data has been overridden out on the disk.