Caring for file formats
3 likes749 views
The document discusses the challenges posed by outdated and overly complex file formats, emphasizing the need for realistic, open libraries to reduce the attack surface. It critiques existing specifications as inadequate and advocates for a more unified approach to file format management, suggesting the preservation and deprecation of older formats to enhance security. The author calls for updated and freely available specifications that reflect the current realities of file formats to better serve the digital preservation community.
![If you [/your program] generate
a picture of any kind,
you might want to export
the result to something
that you can re-use later.
(same for any form of information)](https://image.slidesharecdn.com/caringforfileformatsrev1-160416152028/85/Caring-for-file-formats-12-320.jpg)
![We should reduce the attack surface.
How many unsuspected supported
[sub-]formats and parsers?
https://lcamtuf.blogspot.com/2014/10/psa-dont-run-strings-on-untrusted-files.html](https://image.slidesharecdn.com/caringforfileformatsrev1-160416152028/85/Caring-for-file-formats-21-320.jpg)
![PoC||GTFO 11 is a webserver serving itself, with its own HTML page
extracting its own attachments from its ZIP.
$ruby pocorgtfo11.pdf
Listening for connections on port 8080.
To listen on a different port,
re-run with the desired port as a command-line argument.
A neighbor at 127.0.0.1 is requesting /
A neighbor at 127.0.0.1 is requesting /ajax/feelies.json
A neighbor at 127.0.0.1 is requesting /favicon.png
$unzip -l pocorgtfo11.pdf
Archive: pocorgtfo11.pdf
Length Date Time Name
-------- ---- ---- ----
0 03-16-16 13:37 4am/
25955 03-11-16 15:06 4am/Stickybear Math 2 (4am crack).txt
[...]
3241 03-16-16 13:37 wafflehouse.txt
-------- -------
8177332 23 files](https://image.slidesharecdn.com/caringforfileformatsrev1-160416152028/85/Caring-for-file-formats-45-320.jpg)
Caring for file formats
- 1. Caring for file formats Caring for file formats Ange Albertini Troopers 2016 Ange Albertini Troopers 2016
- 2. TL;DR ● Attack surface with file formats is too big. ● Specs are useless (just a nice ‘guide’), not representing reality. ● We can’t deprecate formats because we can’t preserve and we can’t define how they really work ● We need open good libraries to simplify landscape, and create a corpus to express the reality of file format, which gives us real “documentation”. ● Then we can preserve and deprecate older format, which reduces attack surface. ● From then on, we can focus on making the present more secure. ● We don’t need “new” formats: we need ‘alive’ specs and files corpus. Otherwise specs will always diverge from reality.
- 3. Ange Albertini reverse engineering & visual documentation @angealbertini ange@corkami.com http://www.corkami.comWelcome to my talk!
- 4. I make polyglots (multi-type files), schizophrenics (multi-behavior)...
- 5. I tried to explain file formats with cows… But that didn’t really tell why people should care.
- 6. 1 3DES I really like to play with file formats... AESK AESK JPG JAR (ZIP + CLASS) PDF FLV PNG 2
- 7. I’m a part of PoC||GTFO, for which I’m a file format user and abuser.
- 8. PoC||GTFO: many file formats ● Articles PDFLaTeX PDFBook Inkscape GhostScript Scribus Blender Gimp Fontforge PDFFont Mutool ● Proof of Concept Qpdf Xpdf Ruby Python Bash Truecrypt Wavpack Audacity Baudline Sox Tar Zip MkIsoFS LSnes PngOpt JpegSnoop AdvPNG Nasm Qemu BPGEnc And many custom scripts handling file formats in unconventional ways…
- 9. I'm interested about hardware preservation and digital preservation.
- 10. My interests ● Using file formats ○ graphics, 3d, music… ● Abusing file formats ○ polyglot, schizophrenia, hash collisions… ● Preserving file formats ○ Retro-gaming, digital archeology...
- 11. A miserable little pile of secrets Not just a sequence of binary What is a file format?
- 12. If you [/your program] generate a picture of any kind, you might want to export the result to something that you can re-use later. (same for any form of information)
- 13. A computer dialect to communicate between communities. What is a file format?
- 14. File formats are community connectors. Don’t think so? Try exporting everything as XML ;)
- 15. Most people don’t care about <actor> They only care about <roles> We mostly care about the input/output.
- 16. Example: We don’t care about GIF We mostly care about its characteristics and how easy it is to use. No need to be emotional, and stay in our comfort zone.
- 17. We don’t really care about file formats. We care about their caracteristics. Not groundbreaking, but supported “everywhere”.
- 18. Why should infosec care? Fuzz formats. Blame “bad” devs. Collect CVEs. Boast your ego. 10 PRINT “SOLVED ANYTHING YET?” 20 GOTO 10
- 19. Attack surface ● 1 OS = N supported formats ● For each format: ○ How many parsers? ○ For each parser: ■ Which version, compiler...
- 20. The PGM or PPM formats are the easiest way to convert any data in valid grayscale or RGB pictures. But most people don’t know it’s supported out of the box by many softwares.
- 21. We should reduce the attack surface. How many unsuspected supported [sub-]formats and parsers? https://lcamtuf.blogspot.com/2014/10/psa-dont-run-strings-on-untrusted-files.html
- 22. How many file formats supported by your browser ? By your OS? How many do you really need ? Think “embedded”.
- 23. Capacity is still too cheap: we keep stacking formats/features, which doesn’t solve anything. It’s a problem everywhere. We keep losing ground.
- 25. “Pokemon plays Twitch” 1. Exploit a GameBoy game via input 2. Take over the Super GameBoy 3. Take over the Super Nintendo
- 26. The file itself can perform the exploit (on the hardware or an emulator). The payload displays the article.
- 27. --> PoC||GTFO 10 is a PoC-ception: - a PDF article describing the exploit - a file performing the exploit (to display the article)
- 29. “young celebs” What they were supposed to be doesn’t really matter.
- 30. What file formats were supposed to be doesn't matter anymore, what they are now is all we care. Security cares about current reality, not obsolete theory.
- 31. We can blame bad parsers. What about the file formats? If the map is unclear enough, you’ll get lost anyway. A blurry file format will never lead to a clean parser.
- 32. use a ready-made translator: an import/export library Write your own: read the specs. 2 ways to communicate
- 33. Landscapes
- 34. To exploit hash collisions, I abused JPEG. To abuse JPEG “everywhere”, just abuse LibJPEG.
- 35. JPEG format’s landscape in practice, JPEG is LibJPEG turbo v6 ● de facto standard ○ later versions not used (different API) Even if you create your own JPEG library, you want to have full LibJPEG compatibility. JPEG format is defined by LibJPEG.
- 36. I made extremely custom PDFs for each reader.
- 37. These "extreme" PDFs fail on any other reader.
- 38. PDF’s current landscape PDF: 6 interpretations of the specs ● specs are even more useless
- 39. One good open library: a unified attack surface Fuzz it, pwn everyone ? True, but also fixed for everyone! Is diversity really good? We’re all supposed to use the same file format.
- 40. Diversity is good? Attack surface is worse. Unofficial substandards.
- 41. In any cases... Specs are merely an introduction guide. A free set of examples w/ corner cases. A grammar ?
- 42. PDF’s future PDF/E (engineer): 3d crap PDF/A (archiving): already 8 flavours Specs: ● specs are now commercial ● the main implementation is not open ● no set of free files. And all countries preserve their culture with that format?!?! We’re waiting for a new disaster...
- 43. many file formats are abandoned One specs. then nothing. It’s like knowing about someone only from a baby’s picture.
- 45. PoC||GTFO 11 is a webserver serving itself, with its own HTML page extracting its own attachments from its ZIP. $ruby pocorgtfo11.pdf Listening for connections on port 8080. To listen on a different port, re-run with the desired port as a command-line argument. A neighbor at 127.0.0.1 is requesting / A neighbor at 127.0.0.1 is requesting /ajax/feelies.json A neighbor at 127.0.0.1 is requesting /favicon.png $unzip -l pocorgtfo11.pdf Archive: pocorgtfo11.pdf Length Date Time Name -------- ---- ---- ---- 0 03-16-16 13:37 4am/ 25955 03-11-16 15:06 4am/Stickybear Math 2 (4am crack).txt [...] 3241 03-16-16 13:37 wafflehouse.txt -------- ------- 8177332 23 files
- 46. --> PoC||GTFO 11 is self-aware: a PDF that serves itself (HTTP quine), parses its own ZIP to serve its archived feelies.
- 48. Do you still sleep with a teddy bear?
- 49. Kids really deprecate stuff Our computers still handle always more and more file formats. ⇒ The attack surface just keeps growing.
- 50. Obsolete formats are still omnipresent Formats, sub-formats, features...
- 51. Because it’s unclear if we can go back. We’d be too afraid to deprecate them.
- 52. Yet we deprecate for security. Example for PDF: JPEG-compressed text is not supported anymore (it could bypass security).
- 53. Windows PE format becomes stricter (deprecates packers)
- 54. For example, EPUB 3.1 suddenly killed backward compatibility. http://blog.kbresearch.nl/2016/03/10/the-future-of-epub-a-first-look-at-the-epub-3-1-editors-draft/ Sometimes, it’s not even for security reasons
- 55. We don’t need new file formats. It’s the same problem again if eventually their specs stop reflecting reality.
- 56. Even dictionaries have regular updates, to reflect reality.
- 57. Story time Digipres = PDF worshippers. 150 years of availability? ● Non free specs + closed source software? Here comes the grim reaper: ● Fix your stuff or it will be killed (like Flash) We store our knowledge. What about files born digital? Not infosec, but worrying.
- 58. veraPDF and its test files: a great initiative.
- 59. PE.corkami.com: my own collection of hand-made executables and "documentation" (completely free).
- 60. Some of these failed a lot of software...
- 61. Consequence of my PE page+corpus ● 'corkami-proof' software ● raises the bar for everyone ● become a hub of knowledge ○ "I can't share the sample", but from the knowledge, my own file will be shared ⇒ even useful for the original contact
- 62. Conclusion
- 63. Attack surface Too many (sub)formats Too many parsers (= no good open lib)
- 64. Specs Specs shouldn’t be a religious text ● Worshipped, but outdated and worthless Specs should reflect reality (a law) ● updated, enforced, realistic, freely available A good open lib
- 65. Deprecation Deprecation is a natural cycle, and yet... We are afraid to deprecate because no file format is fully preserved: ● open, up to date specs ● free test coverage
- 66. But it won’t happen... ...until a great disaster ? It ends up on CNN, with a logo & a website :)
- 67. Ack Phil Fabrice Travis Sergey Micah Kurt QKumba Hanno...
- 68. Thank you!
- 69. Caring for file formats corkami.com @angealbertini Hail to the king, baby!