Persistent Memory Programming: The Current State of the Ecosystem
1 like421 views
The document discusses the current state of persistent memory programming within the ecosystem, particularly focusing on the integration of 3D XPoint technology and its accessibility through APIs. It outlines programming models, instruction set architectures, and the exposure of persistent memory to applications, including various libraries and resources available. Additionally, it highlights ongoing developments in operating systems and virtualization support for persistent memory usage.
Persistent Memory Programming: The Current State of the Ecosystem
- 1. DCG Data Center Group Persistent Memory Programming: The Current State of the Ecosystem MSST 2017 Andy Rudoff Non-Volatile Memory Software Architect Intel Corporation
- 2. DCG Data Center Group 2 Persistent Memory
- 3. 3 Optimized System Interconnect Reach full potential of 3D XPoint™ Technology by connecting it as Memory NVM Express* (NVMe) Sources: “Storage as Fast as the rest of the system” 2016 IEEE 8th International Memory Workshop and measurement, Intel® Optane™ SSD measurements and Intel P3700 measurements, and technology projections (4k)
- 4. DCG Data Center Group 4 Java PersistentSortedMap 4 PersistentSortedMap employees = new PersistentSortedMap(); … employees.put(id, data); See “pilot” project at: https://github.com/pmem/pcj No flush calls. Transactional. Java library handles it all.
- 5. DCG Data Center Group 5 How Did We Get from a DIMM to a Java API? Persistent Memory High-Level Language Support 5 PersistentSortedMap employees.put(id, data);
- 6. DCG Data Center Group 6 How Did We Get from a DIMM to a Java API? Persistent Memory High-Level Language Support 6 PersistentSortedMap employees.put(id, data); Programming Models
- 7. DCG Data Center Group 7 “Programming Model” At least four meanings… 1. Interface between HW and SW 2. Instruction Set Architecture (ISA) 3. Exposed to Applications (by the OS) 4. The Programmer Experience
- 8. DCG Data Center Group 8 Programming Model: SW Interface to HW 8 Core L1 L1 L2 L3 Core L1 L1 L2 NVDIMM NVDIMM Memory Subsystem … Memory bus
- 9. DCG Data Center Group 9 Programming Model: SW Interface to HW 9 Core L1 L1 L2 L3 Core L1 L1 L2 NVDIMM NVDIMM Memory Subsystem … Memory bus Result: Persistent Memory hardware accessed like memory (cache coherent). Exposed by ACPI 6.0+ on x86.
- 10. DCG Data Center Group 10 Programming Model: Instruction Set Architecture 10 Core L1 L1 L2 L3 Core L1 L1 L2 NVDIMM NVDIMM Memory Subsystem … persistent domain MOV CLWB
- 11. DCG Data Center Group 11 Programming Model: Instruction Set Architecture 11 Core L1 L1 L2 L3 Core L1 L1 L2 NVDIMM NVDIMM Memory Subsystem … persistent domain MOV CLWB Result: Stores flushed from CPU cache, globally-visible Persistent (on x86)
- 12. DCG Data Center Group 1212 NVDIMM User Space Kernel Space Standard File API NVDIMM Driver Application File System ApplicationApplication Standard Raw Device Access Load/Store Management Library Management UI Standard File API PM-Aware File System MMU Mappings Programming Model: Exposing to Applications
- 13. DCG Data Center Group 1313 NVDIMM User Space Kernel Space Standard File API NVDIMM Driver Application File System ApplicationApplication Standard Raw Device Access Block File Memory Load/Store Management Library Management UI Standard File API Mgmt. PM-Aware File System MMU Mappings SNIA NVM Programming Model (Persistent Memory Portion)
- 14. DCG Data Center Group 1414 NVDIMM User Space Kernel Space Application Load/Store Standard File API PM-Aware File System MMU Mappings No Page Cache Result: Direct Access (DAX) Memory-Mapped Files The Heart of the PM Programming Model
- 15. DCG Data Center Group 1515 NVDIMM Kernel Space Application Load/Store Standard File API PM-Aware File System MMU Mappings Result: Safer, less error-prone, idiomatic in common languages Programming Model: The Programmer Experience Language Runtime Libraries Tools
- 16. DCG Data Center Group 16 NVM Libraries: pmem.io C/C++ on Linux and Windows 16 NVDIMM User Space Kernel Space Application Load/Store Standard File API PM-Aware File System MMU Mappings NVM Libraries • Open Source • http://pmem.io • libpmem • libpmemobj • libpmemblk • libpmemlog • libvmem Transactional
- 17. DCG Data Center Group 17 libpmemobj Application libpmem Load/Store memory-mapped pmem libpmemobj transactionsatomic operations allocatorlists locks API API API
- 18. DCG Data Center Group 18 OS Detection of NVDIMMs ACPI 6.0+ OS Exposes pmem to apps DAX provides SNIA Programming Model Fully supported: • Linux (ext4, XFS) • Windows (NTFS) OS Supports Optimized Flush Specified, but evolving (ask when safe) • Linux: unsafe except Device DAX • (and new file systems like NOVA) • Windows: safe Remote Flush Proposals under discussion (works today with extra round trip) Deep Flush Upcoming Specification Transactions, Allocators Built on above via libraries and languages: • http://pmem.io Much more language support to do Virtualization All VMMs planning to support PM in guest (KVM changes upstream, Xen coming, others too…) 18 State of Ecosystem Today
- 19. DCG Data Center Group 19 NVM Programming Model Resources www.snia.org/PM SNIA Standards Portfolio • NVM Programming Model v1.2a – Draft for public review • NVM Programming Model v1.1- SNIA Technical Position • NVM Programming Model v1.0 - SNIA Technical Position SNIA Technical White Papers • NVM PM Remote Access for High Availability • Persistent Memory Atomics and Transactions SNIA Videos and Presentations • The SNIA NVM Programming Model – Latest Developments and Challenges • Persistent Memory Summit 2017
- 20. DCG Data Center Group 20 NVM Libraries http://pmem.io C/C++ C++ bindings: http://pmem.io/nvml/cpp_obj libpmemobj page: http://pmem.io/nvml/libpmemobj Upstream in some distros already, Windows preview available NVML Source Tree https://github.com/pmem/nvml Persistent Collections for Java (experimental) https://github.com/pmem/pcj Enhanced valgrind for Persistent Memory https://github.com/pmem/valgrind