Near Data Computing Architectures for Apache Spark: Challenges and Opportunities with Ahsan Awan
3 likes856 views
The document discusses the opportunities and challenges of near data computing architectures for improving Apache Spark's performance, particularly addressing issues like multi-core scalability and data movement. A proposed project, Night-King, aims to enhance performance using programmable accelerators positioned close to RAM. It highlights that Spark workloads with high I/O wait times can benefit from in-storage processing while also outlining the future potential of hybrid architectures incorporating FPGA and CPU collaboration.
Near Data Computing Architectures for Apache Spark: Challenges and Opportunities with Ahsan Awan
- 1. Ahsan J. Awan (KTH) Near Data Computing Architectures: Opportunities and Challenges for Apache Spark #EUres10 1
- 2. 2#EUres10 About me ? 2 1988 20112010 20132012 2014 2015 2016 2017 B.E. MTS NUST, Pakistan EMECS, TUKL, Germany EMECS, UoS, UK Lecturer, NUST, Pakistan EMJD-DC, KTH/SICS, Sweden PhD Intern, Recore Netherlands EMJD-DC, UPC/BSC, Spain PhD Intern, IBM Research, Japan Born in Pakistan
- 3. 3#EUres10 A Quick Recap from last year ? 3 Problems Identified Solutions Proposed https://spark-summit.org/eu-2016/events/performance-characterization-of-apache-spark-on-scale-up-servers/ Work Time Inflation Poor Multi-core Scalability of data analytics with Spark Thread Level Load Imbalance Wait Time on I/O GC overhead DRAM Bound Latency NUMA Awareness Hyper Threaded Cores No next-line prefetchers Lower DRAM speed Exploiting Near Data Processing Choice of GC algorithm Multiple Small executors
- 4. 4#EUres10 Further Reading ? 4 ● Project Night-King: Improving the single performance for Apache Spark using Near Data Processing Architectures. ● Identifying the potential of Near Data Computing Architectures for Apache Spark in Memory Systems Symposium, 2017. ● Node Architecture Implications for In-Memory Data Analytics in Scale-in Clusters in IEEE/ACM Conference in Big Data Computing, Applications and Technologies, 2016. ● Micro-architectural Characterization of Apache Spark on Batch and Stream Processing Workloads, in IEEE Conference on Big Data and Cloud Computing, 2016. ● How Data Volume Affects Spark Based Data Analytics on a Scale-up Server in 6th Workshop on Big Data Benchmarks, Performance Optimization and Emerging Hardware (BpoE), held in conjunction with VLDB 2015, Hawaii, USA . ● Performance characterization of in-memory data analytics on a modern cloud server, in IEEE Conference on Big Data and Cloud Computing, 2015 (Best Paper Award).
- 5. 5#EUres10 Exploiting NDP/Moving compute closer to data ? 5 Loh et al. A processing in memory taxonomy and a case for studying fixedfunction pim. In Workshop on Near-Data Processing (WoNDP), 2013. 1. Processing in Memory 2. In-Storage Processing Improve the performance by reducing costly data movements back and forth between the CPUs and Memories
- 6. 6#EUres10 Trends of Integrating NVM in the System Architecture ? 6 Chang et al. A limits study of benefits from nanostore-based future data-centric system architectures. In Computing Frontiers 2012
- 7. 7#EUres10 Can Spark workloads benefit from Near data processing ? 7 Host CPU PIM device ISP device Project: NightKing
- 8. 8#EUres10 The case for in-storage processing ? 8 Grep (Gp) K-means (Km)Windowed Word Count (Wwc)
- 9. 9#EUres10 The case for 2D integrated PIM instead of 3D Stacked PIM ? 9 M. Radulovic et al. Another Trip to the Wall: How Much Will Stacked DRAM Benefit HPC?
- 10. 10#EUres10 A refined hypothesis based on workload characterization ? 10 ● Spark workloads, which are not iterative and have high ratio of I/O wait time / CPU time like join, aggregation, filter, word count and sort are ideal candidates for ISP. ● Spark workloads, which have low ratio of I/O wait time / CPU time like stream processing and iterative graph processing workloads are bound by latency of frequent accesses to DRAM and are ideal candidates for 2D integrated PIM. ● Spark workloads, which are iterative and have moderate ratio of I/O wait time / CPU time like K-means, have both I/O bound and memory bound phases and hence will benefit from hybrid 2D integrated PIM and ISP. ● In order to satisfy the varying compute demands of Spark workloads, we envision an NDC architecture with programmable logic based hybrid ISP and 2D integrated PIM.
- 11. 11#EUres10 How to test the refined hypothesis ? 11 ● Simulation Approach ● Very slow for big data applications :( ● Modeling Approach ● Overly estimated numbers :( ● Emulation Approach ● A lot of development :( How about a combination of Modeling and partial Emulation ?
- 12. 12#EUres10 Can existing tightly coupled servers be used as emulators ? 12
- 13. 13#EUres10 Our System Design ? 13
- 14. 14#EUres10 Which programming model ? 14 Iterative MapReduce
- 15. 15#EUres10 Which workloads ? 15 K-means and SGD Mahan et al. TABLA: A unified template-based framework for accelerating statistical machine learning
- 17. 17#EUres10 Advantages of the design ? 17 ● Template based design to support generality. ● No of mappers and reducers can be instantiated based on the FPGA card. ● General Sequencer is a Finite State Machine whose states can be varied to meet the diverse set of workloads ● Mappers and Reducers can be programmed in C/C++ and can be synthesized using High Level Synthesis. ● Support hardware acceleration of Diverse set of workloads
- 18. 18#EUres10 How about using a roof-line model ? 18 Used to estimate Arithmetic Intensities Used to generate roof-line model
- 19. 19#EUres10 Let's show some numbers ? 19 Poor multi-core scalability of Apache Spark
- 20. 20#EUres10 What are the opportunities ? 20 K-means (Km)
- 21. 21#EUres10 What are the challenges ? 21 ● How to design the best hybrid CPU + FPGA ML workloads ? ● How to attain peak performance on CPU side ? ● How to attain peak performance on FPGA side ? ● How to balance load between CPU and FPGA ? ● How hide communication between JVM and FPGA ? ● How to attain peak CAPI bandwidth consumption ? ● How to design the clever ML workload accelerators using HLS tools ?
- 22. 22#EUres10 What High Level Synthesis (Xilinx SDSoC Tool Chain) can do ? 22 20x 10x
- 23. 23#EUres10 Things to remember from this talk ? 23 ● 3D Stacked Memories are the over-kill for Apache Spark Workloads. ● Project Night-King aims at improving the single node performance of Apache Spark using programmable accelerators near DRAM and NVRAM. ● Conservatively, Near-data accelerators augmented Scale-up Servers can improve the performance of Spark MLlib by 5x. ● Never Trust the 20x speed-up claims being made in the industry. Most of the time, the reference points are wrong! ● Xilinx SDSoC Tool Chain needs to support pragmas for map-reduce programming model. ● Performance Characterization and Optimization of In-Memory Data Analytics on a Scale- up Server, PhD thesis, Ahsan Javed Awan (ISBN: 978-91-7729-584-6)
- 24. 24#EUres10 That's all for now ? 24 Email: ajawan@kth.seEmail: ajawan@kth.se Profile:Profile: www.kth.se/profile/ajawan/www.kth.se/profile/ajawan/ https://se.linkedin.com/in/ahsanjavedawanhttps://se.linkedin.com/in/ahsanjavedawan THANK YOU