Denser containers with PF cache - Pavel Emelyanov
Download as ODP, PDF0 likes615 views
This document discusses using PFCache to improve storage density for container files by deduplicating identical data. PFCache uses a cache area to store deduplicated file contents, referenced via cache links in container image files. Evaluation showed PFCache improved storage density. Future work includes upstreaming PLoop for containers and pursuing IO deduplication in the Linux kernel for additional benefits.
Denser containers with PF cache - Pavel Emelyanov
- 1. Denser containers with PFCacheDenser containers with PFCache Pavel Emelyanov ContainerCon, Seattle, 2015
- 2. AgendaAgenda • How to store container files • Why shared template matters • What can be deduplicated and what should be • PFCache • Q&A 2
- 3. How to store container filesHow to store container files 3 Filesystem Container processes
- 4. How to store container filesHow to store container files 4 Filesystem Container processes Block device NetworkHost Filesystem Host block device Hardware
- 5. How to store container files (1)How to store container files (1) 5 Filesystem Container processes Block device NetworkHost Filesystem Host block device Hardware Chroot() Union FS
- 6. How to store container files (2)How to store container files (2) 6 Filesystem Container processes Block device NetworkHost Filesystem Host block device Hardware Loop device ZFS ZVol BTRFS subvolume PLoop
- 7. What's PLoopWhat's PLoop • Loop device plus – AIO for better performance – Snapshots – QCOW2-like format for thin provisioning – Thin provisionong itself • Upstreaming work in progress 7
- 8. How to store container files (3)How to store container files (3) 8 Filesystem Container processes Block device NetworkHost Filesystem Host block device Hardware LVM DM-thin
- 9. How to store container files (4)How to store container files (4) 9 Filesystem Container processes Block device NetworkHost Filesystem Host block device Hardware NBD Ceph RBD iSCSI
- 10. How to store container files (5)How to store container files (5) 10 Filesystem Container processes Block device NetworkHost Filesystem Host block device Hardware NFS GFS2 OCFS Ceph
- 11. Containers vs TemplatesContainers vs Templates • Containers ... – are massively cloned from pre-created “templates” – do not have direct access to the underlying (block) storage • Identical data can be effectively deduplicated – Higher density – Lower IO and/or memory consumption 11
- 12. Who can do shared templatesWho can do shared templates 12 Storage OpenVZ Docker LXC Union FSs + + + Btrfs + DM-thin + PLoop + Ceph ZFS +
- 13. What can be de-duplicatedWhat can be de-duplicated 13 Filesystem Container processes Block device Network
- 14. What can be de-duplicatedWhat can be de-duplicated 14 Filesystem Container processes Block device Network Page cache Cached pages
- 15. What can be de-duplicatedWhat can be de-duplicated 15 Filesystem Container processes Block device Network Page cache Cached pages IO flow
- 16. What is deduplicatedWhat is deduplicated 16 Storage Memory IO Union FSs + + Btrfs +/- DM-thin PLoop + + Ceph ZFS
- 17. Additional OpenVZ constraintsAdditional OpenVZ constraints • Containers disks are independent image files – Can be easily copied across nodes – No single (shared) point of failure • Deduplicated data is volatile – “Templates” can be lost (e.g. while migrating) – Too big pool with shared data can be easily shrunk 17
- 18. Virtuozzo IO stackVirtuozzo IO stack 18 Ext4 Container processes PLoop device Image file
- 19. PF-CachePF-Cache 19 Ext4 PLoop device Cache area Cache link (xattr)Ext4 Container processes PLoop device Image file Image file
- 20. Cache and cache link behaviorCache and cache link behavior • Cache area – target file name is sha1 sum of the contents – files are created by user-space daemon – cache size is limited by ploop • Cache link – created automatically upon file creation – dropped when file is opened for writing – Is kept during metadata update (chown/chmod) 20
- 22. Future workFuture work • PLoop is available in OpenVZ & Virtuozzo – Upstream WIP • IO deduplication in the upstream – Issue raied at 2013'th LSFMM – DM-thin/btrfs IO dedup for containers – KSM++ for VM-s 22
Editor's Notes
- #3: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #4: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #5: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #6: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #7: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #8: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #9: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #10: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #11: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #12: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #13: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #14: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #15: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #16: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #17: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #18: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #19: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #20: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #21: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #22: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>
- #23: To install a font: Open Fonts by clicking the Start button , clicking Control Panel, clicking Appearance and Personalization, and then clicking Fonts. Click File, and then click Install New Font. ... In the Add Fonts dialog box, under Drives, click the drive where the font that you want to install is located. http://windows.microsoft.com/en-us/windows-vista/install-or-uninstall-fonts <number>