High Performance Managed Languages
1 like141 views
The document discusses the advantages and considerations of using managed languages for high-frequency trading applications, particularly focusing on performance aspects like garbage collection, runtime optimization, and algorithm design. It highlights the trade-offs between managed and native languages in terms of latency, memory performance, and the importance of efficient data structures. The concluding remarks emphasize the need for profiling and understanding the performance implications of design choices in software development.
![void foo(int[] array, int length)
{
for (int i = 0; i < length; i++)
{
bar(Integer.bitCount(array[i]));
}
}
Loops](https://image.slidesharecdn.com/highperformancemanagedlanguages-170606080622/85/High-Performance-Managed-Languages-49-320.jpg)
![void foo(int[] array, int length)
{
for (int i = 0; i < length; i += 4)
{
bar(Integer.bitCount(array[i]));
bar(Integer.bitCount(array[i + 1]));
bar(Integer.bitCount(array[i + 2]));
bar(Integer.bitCount(array[i + 3]));
}
}
Loops](https://image.slidesharecdn.com/highperformancemanagedlanguages-170606080622/85/High-Performance-Managed-Languages-50-320.jpg)
![Loops
void foo(int[] array, int length)
{
for (int i = 0; i < length; i++)
{
bar(Integer.bitCount(array[i]));
}
}
Intrinsics](https://image.slidesharecdn.com/highperformancemanagedlanguages-170606080622/85/High-Performance-Managed-Languages-51-320.jpg)
![Subtype Polymorphism
void draw(Shape[] shapes)
{
for (int i = 0; i < shapes.length; i++)
{
shapes[i].draw();
}
}](https://image.slidesharecdn.com/highperformancemanagedlanguages-170606080622/85/High-Performance-Managed-Languages-52-320.jpg)
![Subtype Polymorphism
void draw(Shape[] shapes)
{
for (int i = 0; i < shapes.length; i++)
{
shapes[i].draw();
}
}
void bar(Shape shape)
{
bar(shape.isVisible());
}](https://image.slidesharecdn.com/highperformancemanagedlanguages-170606080622/85/High-Performance-Managed-Languages-53-320.jpg)
![Subtype Polymorphism
void draw(Shape[] shapes)
{
for (int i = 0; i < shapes.length; i++)
{
shapes[i].draw();
}
}
void bar(Shape shape)
{
bar(shape.isVisible());
}
Class Hierarchy Analysis
& Inline Caching](https://image.slidesharecdn.com/highperformancemanagedlanguages-170606080622/85/High-Performance-Managed-Languages-54-320.jpg)
High Performance Managed Languages
- 1. High Performance Managed Languages Martin Thompson - @mjpt777
- 2. Really, what is your preferred platform for building HFT applications?
- 4. Why would you build low-latency applications on a GC’ed platform?
- 6. 1. Some Context 2. Runtime Optimisation 3. Garbage Collection 4. Algorithms & Design Agenda
- 7. Some Context
- 8. Let’s be clear A Managed Runtime is not always a good choice…
- 11. Are native languages faster?
- 15. Time to add 2 integers?
- 16. 1 CPU Cycle : < 1ns http://www.agner.org/optimize/instruction_tables.pdf
- 17. Time per operation to sum the values in an array of integers?
- 18. Access Pattern Benchmark Benchmark Score Error Units =========================================== sequential 0.832 ± 0.006 ns/op
- 19. Really??? Less than 1ns per operation?
- 21. What if the access pattern is different?
- 22. Access Pattern Benchmark Benchmark Score Error Units =========================================== sequential 0.832 ± 0.006 ns/op randomPage 2.703 ± 0.025 ns/op dependentRandomPage 7.102 ± 0.326 ns/op randomHeap 19.896 ± 3.110 ns/op dependentRandomHeap 89.516 ± 4.573 ns/op
- 23. Data Dependent Loads aka “Pointer Chasing”!!!
- 24. Performance 101
- 25. 1. Memory is transported in Cachelines Performance 101
- 26. 1. Memory is transported in Cachelines 2. Memory is managed in OS Pages Performance 101
- 27. 1. Memory is transported in Cachelines 2. Memory is managed in OS Pages 3. Memory is pre-fetched on predictable access patterns Performance 101
- 29. 1. Profile guided optimisations Runtime JIT
- 30. 1. Profile guided optimisations 2. Bets can be taken and later revoked Runtime JIT
- 31. Branches void foo() { // code if (condition) { // code } // code }
- 32. Block A Branches void foo() { // code if (condition) { // code } // code }
- 33. Block A Block B Branches void foo() { // code if (condition) { // code } // code }
- 34. Block A Block C Block B Branches void foo() { // code if (condition) { // code } // code }
- 35. Block A Block C Block B Branches void foo() { // code if (condition) { // code } // code } Block A Block C
- 36. Block A Block C Block B Branches void foo() { // code if (condition) { // code } // code } Block A Block C Block B
- 37. Subtle Branches int result = (i > 7) ? a : b;
- 38. Subtle Branches int result = (i > 7) ? a : b; CMOV vs Branch Prediction?
- 39. Method/Function Inlining void foo() { // code bar(); // code }
- 40. Method/Function Inlining void foo() { // code bar(); // code } Block A
- 41. Method/Function Inlining void foo() { // code bar(); // code } Block A bar()
- 42. Method/Function Inlining void foo() { // code bar(); // code } Block A bar()
- 43. Method/Function Inlining void foo() { // code bar(); // code } Block A Block B bar()
- 44. Method/Function Inlining void foo() { // code bar(); // code } Block A Block B bar() Block A
- 45. Method/Function Inlining void foo() { // code bar(); // code } Block A Block B bar() Block A bar()
- 46. Method/Function Inlining void foo() { // code bar(); // code } Block A Block B bar() Block A Block B bar()
- 47. Method/Function Inlining void foo() { // code bar(); // code } i-cache & code bloat?
- 48. “Inlining is THE optimisation.” - Cliff Click Method/Function Inlining
- 49. void foo(int[] array, int length) { for (int i = 0; i < length; i++) { bar(Integer.bitCount(array[i])); } } Loops
- 50. void foo(int[] array, int length) { for (int i = 0; i < length; i += 4) { bar(Integer.bitCount(array[i])); bar(Integer.bitCount(array[i + 1])); bar(Integer.bitCount(array[i + 2])); bar(Integer.bitCount(array[i + 3])); } } Loops
- 51. Loops void foo(int[] array, int length) { for (int i = 0; i < length; i++) { bar(Integer.bitCount(array[i])); } } Intrinsics
- 52. Subtype Polymorphism void draw(Shape[] shapes) { for (int i = 0; i < shapes.length; i++) { shapes[i].draw(); } }
- 53. Subtype Polymorphism void draw(Shape[] shapes) { for (int i = 0; i < shapes.length; i++) { shapes[i].draw(); } } void bar(Shape shape) { bar(shape.isVisible()); }
- 54. Subtype Polymorphism void draw(Shape[] shapes) { for (int i = 0; i < shapes.length; i++) { shapes[i].draw(); } } void bar(Shape shape) { bar(shape.isVisible()); } Class Hierarchy Analysis & Inline Caching
- 55. 1. Profile guided optimisations 2. Bets can be taken and later revoked Runtime JIT
- 57. Generational Garbage Collection “Only the good die young” - Billy Joel
- 58. Eden Survivor 0 Survivor 1 Young/New Generation TLAB TLAB Tenured Virtual Virtual Old Generation Generational Garbage Collection
- 59. Modern Hardware (Intel Sandy Bridge EP) C 1 C n C 1 C nRegisters/Buffers <1ns L1 L1 L1 L1~4 cycles ~1ns L2 L2 L2 L2~12 cycles ~3ns L3 L3 ~40 cycles ~15ns ~75 cycles ~25ns (dirty hit) ~65ns DRAM QPI ~40ns MC MC DRAM DRAM DRAM DRAM DRAM DRAM DRAM ... ... ... ... ... ... QPI QPIPCI-e 3 PCI-e 3 40X IO 40X IO * Assumption: 3GHz Processor
- 60. Broadwell EX – 24 cores & 60MB L3 Cache
- 61. Eden Young/New Generation TLAB TLAB Thread Local Allocation Buffers
- 62. Eden Young/New Generation TLAB TLAB Thread Local Allocation Buffers • Affords locality of reference • Avoid false sharing • Can have NUMA aware allocation
- 63. Eden Survivor 0 Survivor 1 Young/New Generation TLAB TLAB Virtual Object Survival
- 64. Eden Survivor 0 Survivor 1 Young/New Generation TLAB TLAB Virtual Object Survival • Aging Policies • Compacting Copy • NUMA Interleave • Fast Parallel Scavenging • Only the survivors require work
- 65. Eden Survivor 0 Survivor 1 Young/New Generation TLAB TLAB Tenured Virtual Virtual Old Generation Object Promotion
- 66. Eden Survivor 0 Survivor 1 Young/New Generation TLAB TLAB Tenured Virtual Virtual Old Generation Object Promotion • Concurrent Collection • String Deduplication
- 68. Compacting Collections – Depth first copy
- 70. Compacting Collections OS Pages and cache lines?
- 71. EdenE O S O S O E S OO E O E O O O Survivor Old Unused O S E H HumongousH H O O G1 – Concurrent Compaction?
- 72. Azul Zing C4 True Concurrent Compacting Collector
- 73. Where next for GC?
- 75. GC vs Manual Memory Management Not easy to pick clear winner…
- 76. GC vs Manual Memory Management Managed GC • GC Implementation • Card Marking • Read/Write Barriers • Object Headers • Background Overhead on CPU and Memory Not easy to pick clear winner…
- 77. GC vs Manual Memory Management Managed GC • GC Implementation • Card Marking • Read/Write Barriers • Object Headers • Background Overhead on CPU and Memory Not easy to pick clear winner… Native • Malloc Implementation • Arena/pool contention • Bin Wastage • Fragmentation • Debugging Effort • Inter-thread costs
- 79. What is most important to performance?
- 80. Time
- 81. “If I had more time, I would have written a shorter letter.” - Blaise Pascal
- 82. • Avoiding duplicate work • Avoiding cache misses • Avoiding contention • Strength reduction • Amortising expensive operations • Mechanical Sympathy • Choice of algorithms & data structures • API design
- 83. In a large codebase it is really difficult to do everything well
- 84. It also takes some “uncommon” disciplines such as: profiling, telemetry, modelling…
- 85. The story of Aeron https://github.com/real-logic/Aeron
- 86. Aeron is an interesting lesson in “time to performance”
- 87. Lots of others exists such at the C# Roslyn compiler
- 88. Time spent on Mechanical Sympathy vs Debugging Pointers ???
- 89. GC Immutable Data & Concurrency
- 90. In Closing …
- 91. What does the future hold?
- 93. What about footprint, startup, warm up, etc. ???
- 97. Blog: http://mechanical-sympathy.blogspot.com/ Twitter: @mjpt777 “Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius, and a lot of courage, to move in the opposite direction.” - Albert Einstein Questions?