Dark Data and Missing Evidence
Download as PPTX, PDF3 likes1,080 views
The document discusses 'dark data', which refers to hidden and potentially harmful data on digital devices that can go undetected by forensic tools, including unformatted disk space and deleted files. It highlights the significance of recovering such data, as it can provide crucial evidence in legal cases and indicate misconduct or intellectual property theft. Forensic Innovations offers tools designed to capture and analyze dark data, emphasizing the need for early data collection to prevent loss during ongoing computer activity.
Dark Data and Missing Evidence
- 1. Dark DataandMissing EvidenceRob ZirnsteinPresidentForensic InnovationsJanuary 13th, 2011
- 2. Darth Vader?No, “Dark Data”, but they bothAre often associated with evilKeep secrets (“Luke, I’m your father”)Are potentially harmful
- 3. Dark Matter?No, “Dark Data”! But they bothGo undetectedAre surrounded by detectable stuffAffect things around them
- 4. What is Dark Data?Dark Data in our digital devicesEveryone creates it (unintentionally)Criminals may hide it (Anti-Forensics)Forensic tools can’t see itBut it is there!Data that we can’t seeOn our hard drivesOn out flash drivesIn our computer files
- 5. Where is Dark Data?DCO & HPAUnformatted Disk SpaceDeleted FilesUnknown FilesBetween FilesInside Common FilesDeleted Data Objects
- 6. Hard Drive LayoutDevice Configuration Overlay (DCO)http://www.forensicswiki.org/wiki/SAFE_Block_XPData Cleaner+ http://www.mp3cdsoftware.com/blancco---data-cleaner--download-16317.htmhttp://www.utica.edu/academic/institutes/ecii/publications/articles/EFE36584-D13F-2962-67BEB146864A2671.pdfHost Protected Area (HPA)http://www.thinkwiki.org/wiki/Hidden_Protected_AreaForensic Duplicatorhttp://www.tableau.com/pdf/en/Tableau_TD1_Product_Brief.pdfHDD Capacity Restore Tool http://hddguru.com/software/2007.07.20-HDD-Capacity-Restore-Tool/Unformatted Disk Space
- 7. Deleted FilesDeleted Files aren’t really gone?Unused Disk Space (in a volume)Disk Caches / Swap FilesWindows Recycle BinAre they hard to recover?Fragmentation is deadlyLarge databases tend to be heavily fragmentedEven DFRWS Researchers find that fragmentation can make some file types impossible to recover (http://www.dfrws.org/2007/challenge/results.shtml)
- 8. Unknown Files (1)500 types of files handled by eDiscovery, Document Management & Computer Forensics Tools50,000+* types of files in the world5,000 types of files typically in use*http://filext.com
- 9. Unknown Files (2) Typical ToolsFI Tools (23 wrong files) (26 Correct Files)
- 10. Between FilesAlternate Data Streams (ADS)Files hiding behind files (on NTFS)RAM SlackPadding between the end of a file and the end of the current sectorTypically zeros, sometimes random contentFile/Cluster/Residual/Drive SlackPadding between sectors used & the end of the current clusterPrevious sector content that should be used in File Carvinghttp://www.forensics-intl.com/def6.html
- 11. Inside Common FilesDeleted ObjectsEx: Adobe PDF & MS Office 2003 (OLE) not removing deleted data (change tracking)Smuggled ObjectsEx: MS Office 2007 (Zip) and MS Wave (RIFF) formats ignore foreign objectsObject / Stream SlackEx: OLE objects have sector size issues, just like with disk sectorsField SlackEx: Image files that don’t use the whole palette, and/or less than 8/16/32/48 bppSteganography
- 12. Smuggled ObjectsSome formats ignore foreign objectsMS Office 2007 (Zip)MS Wave (RIFF)This exampleI added a file to a Word 2007 document.The document opens without any error.
- 13. Deleted Data in Slack Deleted Data that evades Redaction
- 15. Is Dark Data Important?Cases are won or lost based on the ability to find the evidence.The strongest evidence may be hidden accidentally or intentionally.Corporate Digital Assets may be lost, but recoverable.Employee misconduct is tracked by the hidden trail of improper acts.Intellectual Property theft can put a company out of business.Identify in-house criminals by detect-ing smuggled data before it leaves.
- 16. Dark Data Can Be FragileLive Forensics software tools run on the live system.The RAM that they use affects the memory cache files on the hard drive.The running computer deletes, fragments & over writes files on the hard drive constantly.Hard drive activity can destroy Dark Data!Dark Data must be collected first!Before other tools interfere with the data.Image RAMImage Hard Drive (when possible)Analyze Unallocated Disk SpaceAnalyze File Slack SpaceCollect relevant file types
- 17. What Does FI Do?Create Technologies to Capture Dark DataFile InvestigatorFile ExpanderFile HarvesterEquip Law Enforcement with ToolsFI TOOLSFI Object ExplorerFI Data Profiler Portable
- 18. FI TechnologiesFile InvestigatorDiscovers Files Masquerading as Other TypesIdentifies 3,953+ File TypesHigh Accuracy & SpeedFile ExpanderDiscovers Hidden Data within filesData missed by all forensic toolsFile Harvester (Under Development)Recovers deleted/lost files the rest of the industry can’tWill eventually rebuild partial files
Editor's Notes
- #2: This presentation was provided for an ASDFED Indianapolis Chapter meeting.
- #3: How did I get the term “Dark Data”? Not from Darth Vader, but they do have some things in common.
- #4: I copied “Dark Matter”, because it also goes undetected yet still affects things (objects/solar systems) around it.This image was created by observing the gravitational effects on light and objects around the matter. No instrument can actually see the dark matter directly.
- #5: Dark Data is in everything digital that we create, yet we don’t see it.
- #6: Dark Data is hiding in the most unsuspecting places.
- #7: DCO – Used to reduce the disk size to exactly match the size of another hard drive. This makes it easier to clone hard drives.HPA – Used to store vendor utilities on a hard drive, where a user can’t delete them.These areas are difficult to access and add or remove.Unformatted Disk Space is the remaining space that has not been allocated to a disk volume that the user can access.
- #8: Many recovery tools falsely report their recovery success. Many of the successfully recovered files are actually corrupted with other file fragments.
- #9: Most Forensics Tools keep these files in the Exception Bin. Have you ever seen an investigation with an empty Exception Bin? What if the best evidence was hiding in that Exception Bin?!?Ex: Hidden TrueCrypt volume file, that looks like random data.
- #10: The list on the left was produced with Windows, as an extreme example. Although, many eDiscovery tools don’t do much better than this.The list on the right was produced by a tool that specializes in accurately identifying thousands of file types.Notice the 3 Alternate Data Streams identified on the right. They weren’t just detected, but analyzed to catch any hidden file types.
- #11: Many tools combine RAM slack with Drive Slack. This causes confusion when file carving for partial files, because these slacks come from different sources.
- #12: Common files may contain stowaways.Bpp = Bits Per Pixel
- #13: Step 1: Rename the file to be smuggled to ‘document.xml’ (I used a simple text file)Step 2: Rename Word.docx to Word.zipStep 3: Open Word.zip with WinZipStep 4: Add the new smuggled ‘document.xml’ to Word.zip (in the root)Step 5: Rename Word.zip to Word.docx
- #14: This example shows an MS Outlook Form Template that was edited to remove part of a sentence. The deleted content is still there!When the paragraph/object shrank, the Stream Slack inherited the end of the paragraph.Existing Redaction tools use Microsoft libraries that ignore the Stream Slack.
- #15: Smuggled data is broken down into bits and substituted for picture data that doesn’t effect the visible image enough to be noticed.May just change 1 bit per pixel, or fill the Field Slack.The smuggled data may also be encrypted before insertion.