Make AI ecosystem more interoperable
0 likes877 views
Kazuaki Ishizaki from IBM Research discusses improving interoperability within the AI ecosystem, particularly focusing on the challenges posed by endianness across different systems. He explains the nuances of endian representation and provides strategies for detecting and resolving interoperability issues in software. The presentation emphasizes the importance of continuous testing and code enhancement to accommodate both little and big endian systems for seamless data exchange.
![Does It Have Bad Smell?
▪ Get 32-bit data from int8 datum by reinterpret_cast
15 Make AI ecosystem more interoperable - Kazuaki Ishizaki
uint8_t *i8p = ...
i8p[0] = 4; i8p[1] = 3; i8p[2] = 2; i8p[3] = 3;
int32_t i32 = *reinterpret_cast<int32_t *>(i8p);
printf(“%08x”, i32);](https://image.slidesharecdn.com/dais2021ltishizaki202105-210528094521/85/Make-AI-ecosystem-more-interoperable-15-320.jpg)
![Results are Different on Different Endian Machines
16 Make AI ecosystem more interoperable - Kazuaki Ishizaki
uint8_t *i8p = ...
i8p[0] = 4; i8p[1] = 3; i8p[2] = 2; i8p[3] = 3;
int32_t i32 = *reinterpret_cast<int32_t *>(i8p);
printf(“%08x”, i32);
Little endian Big endian
01020304 04030201](https://image.slidesharecdn.com/dais2021ltishizaki202105-210528094521/85/Make-AI-ecosystem-more-interoperable-16-320.jpg)
![Why Problem Occurs?
▪ Different endian processors interpret the same memory sequence in
different ways
17 Make AI ecosystem more interoperable - Kazuaki Ishizaki
04 03 02 01
04 03 02 01
memory layout
memory layout
04030201
01020304
uint8_t *i8p = ...
i8p[0] = 4; i8p[1] = 3; i8p[2] = 2; i8p[3] = 1;
int32_t i32 = *reinterpret_cast<int32_t *>(i8p);
printf(“%08x”, i32);
Little endian Big endian
i8p i8p](https://image.slidesharecdn.com/dais2021ltishizaki202105-210528094521/85/Make-AI-ecosystem-more-interoperable-17-320.jpg)
![Support Both Endians
▪ Swap data for big endian
18 Make AI ecosystem more interoperable - Kazuaki Ishizaki
uint8_t *i8p = ...;
i8p[0] = 4; i8p[1] = 3; i8p[2] = 2; i8p[3] = 1;
int32_t i32 = reinterpret_cast<int32_t *>(i8p);
#if !defined(__LITTLE_ENDIAN__)
i32 = __builtin_bswap32(i32);
#endif
printf(“%08x”, i32);
Little endian Big endian
01020304 01020304](https://image.slidesharecdn.com/dais2021ltishizaki202105-210528094521/85/Make-AI-ecosystem-more-interoperable-18-320.jpg)
Make AI ecosystem more interoperable
- 1. Make AI ecosystem more interoperable Kazuaki Ishizaki IBM Research - Tokyo
- 2. About Me – Kazuaki Ishizaki ▪ Researcher at IBM Research – Tokyo https://ibm.biz/ishizaki – Compiler optimization, language runtime, and parallel processing ▪ Apache Spark committer from 2018/9 (SQL module) ▪ Work for IBM Java (Open J9, now) from 1996 – Technical lead for Just-in-time compiler for PowerPC ▪ ACM Distinguished Member ▪ SNS – @kiszk – https://www.slideshare.net/ishizaki/ 2 Make AI ecosystem more interoperable - Kazuaki Ishizaki
- 3. Agenda ▪ Motivation ▪ What is an inhibitor of interoperability? – Endianness on each machine ▪ What is endian? ▪ What happens in a program? ▪ How to find and fix issues? ▪ How to keep interoperability in AI ecosystem 3 Make AI ecosystem more interoperable - Kazuaki Ishizaki
- 4. Very Impressive Performance Improvement on x86 ▪ Improve performance of Spark with Python by over 100x 4 Make AI ecosystem more interoperable - Kazuaki Ishizaki Source: https://databricks.com/blog/2017/10/30/introducing-vectorized-udfs-for-pyspark.html Apache Spark uses Apache Arrow A cross-language development platform for in-memory analytics
- 5. I Want to Do This on IBM Z 5 Make AI ecosystem more interoperable - Kazuaki Ishizaki $ bin/pyspark ... >>> df.show()
- 6. Oh!!! 6 Make AI ecosystem more interoperable - Kazuaki Ishizaki $ bin/pyspark ... >>> df.show() ... java.lang.IllegalStateException: Arrow only runs on LittleEndian systems… ... >>>
- 7. Apache Arrow supported only Little Endian 7 Make AI ecosystem more interoperable - Kazuaki Ishizaki $ bin/pyspark ... >>> df.show() ... java.lang.IllegalStateException: Arrow only runs on LittleEndian systems… ... >>>
- 8. One Pager for Current AI Ecosystem ▪ Data can be exchanged among little endian machines (i.e. x86, Arm, PowerLinux, …) 8 Make AI ecosystem more interoperable - Kazuaki Ishizaki PowerLinux Arm Origin: https://www.dremio.com/webinars/apache-arrow-in-theory-practice/
- 9. One Pager for Expected AI Ecosystem ▪ Data can be exchanged among both endian machines (i.e. x86, Arm, s390x, PowerLinux, …) 9 Make AI ecosystem more interoperable - Kazuaki Ishizaki PowerLinux Arm s390x Origin: https://www.dremio.com/webinars/apache-arrow-in-theory-practice/
- 10. What is Endian? ▪ Data layout on a memory – Example of integer 32bit value 10 Make AI ecosystem more interoperable - Kazuaki Ishizaki 0x01020304
- 11. What is Endian? ▪ Data layout on a memory – Example of integer 32bit value 11 Make AI ecosystem more interoperable - Kazuaki Ishizaki 04 memory layout Little endian 03 0x01020304 02 01 10 addr 11 12 13 x86_64, ppc64le, …
- 12. What is Endian? ▪ Data layout on a memory – Example of integer 32bit value 12 Make AI ecosystem more interoperable - Kazuaki Ishizaki 04 memory layout Little endian Big endian 03 0x01020304 02 01 01 02 03 04 memory layout 10 addr 11 12 13 10 addr 11 12 13 x86_64, ppc64le, … s390x
- 13. Why Programs Usually Work Well? ▪ Programs work well without special cares if – No explicit memory access of a subset of data and/or of a super-set of data – A program is closed itself (no data exchange with other machines) 13 Make AI ecosystem more interoperable - Kazuaki Ishizaki int32_t a, b; int16_t d; ... int32_t c = a + b; int16_t d = static_cast<int16_t>(c) + 1; int32_t e = static_cast<int32_t>(d); ...
- 14. How to Find Issues on Different Endians? ▪ Find bad smells in source code 14 Make AI ecosystem more interoperable - Kazuaki Ishizaki
- 15. Does It Have Bad Smell? ▪ Get 32-bit data from int8 datum by reinterpret_cast 15 Make AI ecosystem more interoperable - Kazuaki Ishizaki uint8_t *i8p = ... i8p[0] = 4; i8p[1] = 3; i8p[2] = 2; i8p[3] = 3; int32_t i32 = *reinterpret_cast<int32_t *>(i8p); printf(“%08x”, i32);
- 16. Results are Different on Different Endian Machines 16 Make AI ecosystem more interoperable - Kazuaki Ishizaki uint8_t *i8p = ... i8p[0] = 4; i8p[1] = 3; i8p[2] = 2; i8p[3] = 3; int32_t i32 = *reinterpret_cast<int32_t *>(i8p); printf(“%08x”, i32); Little endian Big endian 01020304 04030201
- 17. Why Problem Occurs? ▪ Different endian processors interpret the same memory sequence in different ways 17 Make AI ecosystem more interoperable - Kazuaki Ishizaki 04 03 02 01 04 03 02 01 memory layout memory layout 04030201 01020304 uint8_t *i8p = ... i8p[0] = 4; i8p[1] = 3; i8p[2] = 2; i8p[3] = 1; int32_t i32 = *reinterpret_cast<int32_t *>(i8p); printf(“%08x”, i32); Little endian Big endian i8p i8p
- 18. Support Both Endians ▪ Swap data for big endian 18 Make AI ecosystem more interoperable - Kazuaki Ishizaki uint8_t *i8p = ...; i8p[0] = 4; i8p[1] = 3; i8p[2] = 2; i8p[3] = 1; int32_t i32 = reinterpret_cast<int32_t *>(i8p); #if !defined(__LITTLE_ENDIAN__) i32 = __builtin_bswap32(i32); #endif printf(“%08x”, i32); Little endian Big endian 01020304 01020304
- 19. Support Both Endians in Java ▪ Swap data for big endian 19 Make AI ecosystem more interoperable - Kazuaki Ishizaki static final boolean LITTLE_ENDIAN = ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN; int i32 = ... // get the value from a buffer if (!LITTLE_ENDIAN) { i32 = Integer.reverseBytes(i32); }
- 20. Potential Bad Smell and Enhancements ▪ Intra-process (i.e. In-memory) – Get data from the different data type 20 Make AI ecosystem more interoperable - Kazuaki Ishizaki 04 03 02 01 04 03 02 01 memory layout memory layout 01020304 01020304 Swap
- 21. Potential Bad Smell and Enhancements ▪ Intra-process (i.e. In-memory) – Get data from memory in different data type ▪ Inter-process (i.e. host – client) – Exchange data with other machines 21 Make AI ecosystem more interoperable - Kazuaki Ishizaki 01 02 03 04 04 03 02 01 04 03 02 01 memory layout memory layout memory layout 04 03 02 01 memory layout 01020304 01020304 Swap 01020304 Swap
- 22. Can We Find All Issues? ▪ Find bad smells in source code 22 Make AI ecosystem more interoperable - Kazuaki Ishizaki New code is coming everyday
- 23. Automatically Detect Issues ▪ Continuously run test cases on machines with different endians 23 Make AI ecosystem more interoperable - Kazuaki Ishizaki Run test cases
- 24. Automatically Detect Issues ▪ Continuously run test cases on machines with different endians ▪ Enhance code to support both endians if we find issues 24 Make AI ecosystem more interoperable - Kazuaki Ishizaki Run test cases Enhance code
- 25. CI Tools and Instances Help OSS Community ▪ TravisCI ▪ Jenkins ▪ Virtual machine instance 25 Make AI ecosystem more interoperable - Kazuaki Ishizaki Enhance code
- 26. Free Resources of Big Endian for OSS Community ▪ TravisCI – https://docs.travis-ci.com/user/multi-cpu-architectures/ ▪ Jenkins – https://osuosl.org/services/ibm-z/ ▪ Virtual machine instance – https://developer.ibm.com/components/ibm-linuxone/gettingstarted/ 26 Make AI ecosystem more interoperable - Kazuaki Ishizaki
- 27. Apache Arrow Supports Both Endians ▪ Intra-process (from Apache Arrow 3.0) – C and Java bindings ▪ Inter-process (from Apache Arrow 4.0) – C bindings 27 Make AI ecosystem more interoperable - Kazuaki Ishizaki CI on big ending is running for every PR update
- 28. Takeaway ▪ We know different endians on machines – Little endian and big endian ▪ When do we take care of endians? – Get a sub-set or super-set of data in memory – Exchange data with other machines ▪ How to find potential issues and support both endians? – Find bad smell – Automatically run test cases ▪ How to keep interoperability in AI ecosystem? – Easy and free to run CI on different types of machines 28 Make AI ecosystem more interoperable - Kazuaki Ishizaki Visit https://www.slideshare.net/ishizaki if you are interested in this slide