C++ Programming and the Persistent Memory Developers Kit
1 like804 views
This document provides an overview of Intel's Persistent Memory and its associated programming model, including the Persistent Memory Developers Kit (PMDK). It discusses the benefits of persistent memory, including its byte-addressable nature, high capacity, and low latency, along with the necessary architectural changes required for applications to fully leverage these features. Additionally, it outlines the functionalities, APIs, and transactional support provided by PMDK for developers to effectively work with persistent memory in C++.
![28
Closure Transaction Example
auto pop = pool<root>::open("/path/to/poolfile", "layout string");
transaction::exec_tx(pop, [] {
// do some work...
}, persistent_mtx, persistent_shmtx);
auto aborted = transaction::get_last_tx_error();](https://image.slidesharecdn.com/persistentmemory-programming-using-non-volatile-memory-libraries-and-c-bindings-webinar-180420204918/85/C-Programming-and-the-Persistent-Memory-Developers-Kit-28-320.jpg)
C++ Programming and the Persistent Memory Developers Kit
- 2. 2 Legal Notices and Disclaimers This presentation is for informational purposes only. INTEL MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. Check with your system manufacturer or retailer or learn more at intel.com. This sample source code is released under the Intel Sample Source Code License Agreement. Intel, the Intel logo, Intel Core, Intel Xeon Phi, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. Copyright © 2017, Intel Corporation. All rights reserved. Microsoft, Windows, and the Windows logo are trademarks, or registered trademarks of Microsoft Corporation in the United States and/or other countries.
- 3. 3 Topics • Introduction to Persistent Memory • Introduction to Persistent Memory Developers Kit (PMDK) • Working with PMDK • Building PMDK • Emulating Persistent Memory with DRAM • Persistent Memory Programming with PMDK C++ Bindings • Allocating Simple Data Structure in Persistent Memory • Transactions • Thread Synchronization • Sample ( putting it all together) • Container Enabling • Summary • Q&A
- 4. 4 What is Persistent Memory? • Byte Addressable • Persistent Like Storage • Load /Store Access • No page caching • Cache Coherent • Memory-like Performance Why Does it Matter now? • Adds a new tier between DRAM and Block Storage (SSD/HDD) • Larger Capacity, Higher Endurance, Consistent low latency • Ability to do in-place persistence • No Paging, No Context Switching, No Interrupts, No Kernel Code Running • Ability to do DMA & RDMA Introduction to Persistent Memory What’s the impact on the Applications? • Need Ways to Enable Access to New High Performance Tier • May need to re- architect to unlock the new features and performance
- 5. 5 New Type of Memory • Persistent, Large Capacity & Byte Addressable • 6 TB per two-socket system • Cheaper than DRAM • DIMM Capacities • 128 GB • Demonstrated at SAP Sapphire and Oracle Open World 2017 • Availability • 2018 Intel Persistent Memory
- 6. 6 What does Re-architecting to Byte-Addressable Persistent Memory Mean • Applications Need to • Ensure Data Persistence • Stores are not guaranteed persistence until flushed out of CPU caches • Assure Data Consistency • Maintain Transactional Consistency of Persistent Data structures • Preventing torn updates using Transactions • Use Persistent Memory Aware Allocator • Prevent Persistent Memory Leaks • Detect and Handle Persistent Memory Errors WPQ ADR -or- WPQ Flush (kernel only) Core L1 L1 L2 L3 W P Q MOV DIMM CPUCACHES CLWB + fence -or- CLFLUSHOPT + fence -or- CLFLUSH -or- NT stores + fence -or- WBINVD (kernel only) Minimum Required Power fail protected domain: Memory subsystem Custom Power fail protected domain indicated by ACPI property: CPU Cache Hierarchy
- 7. NVDIMMs 7 Byte Addressable Storage With Memory Mapped Files– Before and After User Space Kernel Space Application NVDIMM NVDIMM DRAM … … load/store access mmap() msync() User Space Kernel Space Application NVDIMM NVDIMM DRAM … … Page cache load/store access Avoid Overhead of Paging/Context Switching into Kernel Page cache Before After pmem- Aware File System mmap()
- 8. 8 Topics • Introduction to Persistent Memory Programming • Persistent Memory Developers Kit (PMDK) • Working with PMDK • Building PMDK • Emulating Persistent Memory with DRAM • Deep Dive with C++ Bindings with Code Samples and Demos • Summary
- 9. 9 The SNIA NVM Programming Model – Exposing Persistent Memory to Applications NVDIMMs User Space Kernel Space Standard File API NVDIMM Driver Application File System ApplicationApplication Standard Raw Device Access Load/Store Management Library Management UI Standard File API pmem-Aware File System MMU Mappings
- 10. Support for volatile memory usage PMDK : A Suite of Open Source of Libraries memkind Low level support for local persistent memory libpmem Low level support for remote access to persistent memory librpmem In Development Link to Open Source : http://pmem.io/pmdk/ NVDIMM User Space Kernel Space Application Load/Store Standard File API pmem-Aware File System MMU Mappings Libraries Interface to create arrays of pmem- resident blocks of same size for atomic updates Interface for persistent memory allocation, transactions and general facilities Interface to create a persistent memory resident log file libpmemblklibpmemlog libpmemobj Link to Intel Developer Zone: https://software.intel.com/en- us/persistent-memory Support Transactions C++ C Java* Python* Low-level support
- 11. 11 Why PMDK • Simplifies/Facilitates Adoption of Persistent Memory Programming with Higher Level Language Support • Built on top of SNIA Programming Model • Builds on DAX capabilities on Linux* and Windows* • Data Access Granularity at Cache Line • Provides API to Ensure Data Persistence Data Consistency Assurance • Transactional Operations • Power Fail Atomicity Allocate/Manage Persistent Memory • Developer Tools For Debugging and Error Handling
- 12. 12 libpmemobj: Core Concepts Data Consistency Assurance • Support for Transactional & Atomic Updates Implicit Persistence during Transactions • Knows When and How to Flush the Data to Persistence Domain Persistent Memory Allocation & Management API • Memory Mapped Region Exposed as Object Pools • Thread-safe • Allocate/Free Persistent Data Structures • Position Independent • Root Object • Known Location in the Persistent Memory Pool • Anchor for Objects in the Pool Multi-thread Support Persistent Memory Pool Application Pool Pool Pool 1 2 n Pool Root Object Memory mapped files Threads text data BSS heap memory mapped stack kernel persistent memory Application libpmem Load/Store memory-mapped pmem libpmemobj transactionsatomic operations allocatorlists locks A P I A P I A P I
- 13. 13 PMDK Binary Packages • Download Pre-built Packages: • https://github.com/pmem/pmdk/releases • Works on 64-bit Linux* or Windows* 2016 • Upstreamed into multiple Linux Distros • For example: Fedora* 24 • dnf install libpmemobj–devel • Installs libpmem, libpmem-devel, libpmemobj, libpmemobj-devel • C++ header files are not part of this install
- 14. 14 • git clone • https://github.com/pmem/pmdk • cd pmdk • make • make install (as root) • Installs man pages and libraries • Install libraries in non-standard locations • make install prefix=/xxx/xxx • Customizing Build Settings • Compile with persistent memory aware Valgrind* • make EXTRA_CFLAGS=-DUSE_VG_PMEMCHECK Building Tech Preview PMDK Code from GitHub*
- 15. 15 Get Started with Persistent Memory Programming without Real Hardware • Persistent memory aware applications can be written/tested • Using SSD or Hard Disk • Without DAX • Any Linux* or Windows* distro that supports Memory Mapped Files • With DAX • EXT4, XFS, NTFS • Emulate Persistent Memory with DRAM • https://software.intel.com/en-us/articles/how-to-emulate-persistent- memory-on-an-intel-architecture-serverStart with Emulation; Validate & Tune on Platforms with Real DIMMs
- 16. Emulating Persistent Memory Using DRAM (Linux*) • Reserve PMEM through Kernel Configuration(kernel >4.2) • Enable BIOS to treat Memory Marked as e820- type 12 as PMEM • Enable Direct Access (DAX support) • Identify Usable regions • Specify Kernel Parameter (Grub file) • memmap=<size of pmem>!<offset in DRAM> • On reboot • pmem region created • Kernel offers this space as PMEM to the PMEM driver • OS sees this DRAM region as PMEM & Creates • /dev/pmem0, /dev/pmem1 .. • Make File system • EXT4 and XFS modified to support PMEM • sudo mkfs.ext4 /dev/pmem0 or sudo mkfs.xfs /dev/pmem0 • Mount it on /dev/pmemx with DAX option • Ex: sudo mount –o dax /dev/pmem0 /mnt/pmem/
- 17. 17 Persistent Memory Programming with C++ Bindings
- 18. 18 Sample Code Download the sample code used in this presentation from GitHub* at https://github.com/pmem/pmdk-examples/tree/master/cpp_queue
- 19. 19 Main Features of C++ Bindings • Support for Transactions • C++ Wrappers for • Scoped Transactions • Both Manual and Automatic • Closure-like Transactions • Support for Automatic Snapshotting of Basic Data Types in a Transaction • p<> Property : Wrapper for basic types • Indicates persistent variable • Persistent Pointer : C++ Wrapper with pointer semantics • Designed to handle complex classes • Support for Allocating/Deallocating functions • Transactions & Atomic API • Does not work with polymorphic types
- 20. 20 Topics • Introduction to Persistent Memory Programming • Introduction to PMDK • Persistent Memory Programming with PMDK C++ Bindings • Deep Dive on C++ Bindings with Code Samples & Demos • Transactions • P<> • Persistent_ptr<> • Synchronization • Summary
- 21. 21 Video: Code Sample Introduction This video provides an introduction to the sample queue application that will be made persistent memory-aware. Click this link: https://software.intel.com/en-us/videos/introduction-to-the- pmem-with-c-bindings-hello-world-code-demonstration.
- 22. Transactions 22
- 23. 23 Intro to Transactions • Undo log based transactions • In case of interruption it is rolled-back or completed upon next pool open • ACID like properties • Can be nested • Locks are held until the end of a transaction
- 24. 24 Manual Transaction Example auto pop = pool<root>::open("/path/to/poolfile", "layout string"); { transaction::manual(pop, persistent_mtx, persistent_shmtx); // do some work... transaction::commit(); } auto aborted = transaction::get_last_tx_error();
- 25. 25 Manual Transactions • Based on the familiar RAII concept • Fairly easy to use • Explicit transaction commit because of std::uncaught_exception • Does not throw en exception on transaction abort • By default aborts to account for third-party exceptions or amnesia • Accepts an arbitrary number of (persistent memory resident) locks
- 26. 26 Automatic Transaction Example auto pop = pool<root>::open("/path/to/poolfile", "layout string"); try { transaction::automatic(pop, persistent_mtx, persistent_shmtx); // do some work... } catch (...) { // do something meaningful } auto aborted = transaction::get_last_tx_error();
- 27. 27 Automatic Transactions • Functionally and semantically almost identical to the manual transaction • No explicit transaction commit • Need C++17 • Relies on std::uncaught_exceptions
- 28. 28 Closure Transaction Example auto pop = pool<root>::open("/path/to/poolfile", "layout string"); transaction::exec_tx(pop, [] { // do some work... }, persistent_mtx, persistent_shmtx); auto aborted = transaction::get_last_tx_error();
- 29. 29 Closure Transactions • Take an std::function object as transaction body • No explicit transaction commit • Available with every C++11 compliant compiler • Throw an exception when the transaction is aborted • Take an arbitrary number of locks • Unfortunately at the very end
- 30. 30 Video: Modify Queue to Use Transactions This video demonstrates how to use transactions to safely modify our queue in a persistent memory context. Click this link: https://software.intel.com/en-us/videos/use-transactions-to- modify-our-persistent-memory-queue.
- 31. Resides in persistent memory
- 32. 32 The p<> Property • AKA the workhorse • Overloads operator= for snapshotting in a transaction • Overloads a bunch of other operators for seamless integration • Arithmetic • Logical • Should be used for fundamental types • No convenient way to access members of aggregate types • No operator. to overload
- 33. 33 Video: The p<> property In this video, we put the p<> template to work in our queue example. Click this link: https://software.intel.com/en-us/videos/use-the-p-template-in- our-queue-example.
- 35. 35 Persistent Smart Pointer • Points to objects within a persistent memory pool • Manages translating persistent addresses to runtime addresses • Is a random access iterator • Has primitives for flushing contents to persistence • std::allocator friendly
- 36. 36 Persistent Smart Pointer • Does not manage object lifetime • Does not automatically add contents to the transaction • But it does add itself to the transaction • Does not point to polymorphic objects • No good way to rebuild runtime state after pool reboot
- 37. 37 Transactional Allocation Functions • Can be used only within transactions • Use transaction logic to enable allocation/delete rollback of persistent state • make_persistent calls appropriate constructor • Syntax similar to std::make_shared • delete_persistent calls the destructor • Not similar to anything found in std
- 38. 38 Video: Using the Persistent Pointer In this video, we use the persistent pointer to safely manage queue data. Click this link: https://software.intel.com/en-us/videos/use-the-persistent- pointer-in-queue-example.
- 40. 40 Persistent Memory Synchronization • Types: • mutex • shared_mutex • timed_mutex • condition_variable • All with an interface similar to their std counterparts • Auto reinitializing • Can be used with transactions
- 41. 41 • Webinar Code Sample: • https://github.com/pmem/pmdk- examples/tree/master/cpp_queue • PMDK Resources: • Intel® Developer Zone: https://software.intel.com/persistent- memory • pmem.io - http://pmem.io/ • Google group: https://groups.google.com/forum/#!foru m/pmem • Source : https://github.com/pmem/pmdk • Additional Resources • https://www.usenix.org/system/files/l ogin/articles/login_summer17_07_ru doff.pdf • An overview of persistent memory programming • http://www.snia.org/PM • SNIA Standards Portfolio - NVM Programming Model (NPM) • https://www.snia.org/tech_activities/st andards/curr_standards/npm Summary