HCQC : HPC Compiler Quality Checker
1 like2,839 views
The document introduces the HPC Compiler Quality Checker (hcqc), a tool developed by Masaki Arai to evaluate and improve the quality of compiler optimizations in high-performance computing applications. It outlines the metrics and configuration needed for quality evaluation, presents various comparisons between different compilers, and discusses future work for enhancements including support for new features and architectures. The hcqc aims to facilitate better performance assessments through detailed analysis of compiler behavior on specific kernels.
![Subject of Quality Check
• Configuration file defines the subject of quality check.
• Main items:
Compiler
Compiler version
Optimization flags
{
“DISTRIBUTION" : "OpenSUSE Tumbleweed",
"ARCH" : “aarch64",
"CPU" : "AMD Opteron A1100 Cortex A57",
"LANGUAGE" : "C",
"COMPILER" : “GCC",
"COMMAND" : "/usr/bin/gcc",
"VERSION" : “7.1.1",
"OPT_FLAGS" : ["-O2"],
"ASM_FLAGS" : ["-S“, “-fverbose-asm”],
“FLAG_DB” : [[“?DEBUG_FLAG", “-g”],
[“?C99_STANDARD", “-std=c99”]]
}
3
Example of configuration file](https://image.slidesharecdn.com/arm-hpc-ws-arai-171213201740/85/HCQC-HPC-Compiler-Quality-Checker-3-320.jpg)
![Example of Comparison(2)
• mVMC-mini-calculateNewPfMTwo_child(op)
– GCC generates codes with better addressing modes.
8
CFG DEPTH lsl
calculateNewPfMTwo_child cond .LBB0_3 0 0
0 0
.LBB0_2 cond .LBB0_2 1 1
.LBB0_3 cond .LBB0_11 0 0
0 0
.LBB0_5 cond .LBB0_5 1 0
cond .LBB0_12 0 0
0 1
.LBB0_8 1 0
.LBB0_9 cond .LBB0_9 2 0
cond .LBB0_8 1 0
goto .LBB0_13 0 0
.LBB0_11 0 0
.LBB0_12 0 0
.LBB0_13 end 0 0
*SUMMARY* - 2
CFG DEPTH lsl
calculateNewPfMTwo_child cond .L2 0
0
.L3 cond .L3 1
.L2 cond .L8 0
0
.L5 cond .L5 1
0
.L7 1
.L6 cond .L6 2
cond .L7 1
0
.L4 end 0
.L8 goto .L4 0
*SUMMARY* -
LLVM
GCC
.LBB0_2:
ldr w1, [x5, x16, lsl #2]
sdiv w2, w16, w13
lsl x3, x16, #3
add x16, x16, #1
madd w1, w17, w2, w1
sbfiz x1, x1, #3, #32
ldr x2, [x18, x1]
cmp x11, x16
str x2, [x10, x3]
ldr x1, [x0, x1]
str x1, [x9, x3]
b.ne .LBB0_2 // x3 is dead
LLVM](https://image.slidesharecdn.com/arm-hpc-ws-arai-171213201740/85/HCQC-HPC-Compiler-Quality-Checker-8-320.jpg)
![Example of Comparison(3)
• HimenoBMT-static(height)
Clang/LLVM is not good for register usage.
9
CFG DEPTH # height
.LBB1_8 cond .LBB1_8 4 71 25
CFG DEPTH # height
.L19 cond .L19 4 62 17
LLVM
GCC
71/25 = 2.84
62/17 = 3.647
.LBB1_8:
add x17, x4, x14
mov v6.16b, v17.16b
ldr s17, [x17, #8844]
add x16, x3, x14
mov v7.16b, v16.16b
ldr s16, [x16, #8844]
add x17, x25, x14
ldr s21, [x17, x7]
fmul s0, s17, s0
ldr s17, [x17, x21]
add x16, x5, x14
mov v5.16b, v18.16b
ldr s18, [x16, #8844]
……
LLVM
The metric `height’ is not committed on GitHub yet.](https://image.slidesharecdn.com/arm-hpc-ws-arai-171213201740/85/HCQC-HPC-Compiler-Quality-Checker-9-320.jpg)
HCQC : HPC Compiler Quality Checker
- 1. HCQC - HPC compiler quality checker Masaki Arai masaki.arai@linaro.org LEG HPC-SIG arai.masaki@jp.fujitsu.com FUJITSU LABORATORIES LTD. 1
- 2. Background and Purpose • The quality of the kernel part is important in HPC applications(number crunching on supercomputers). • Make it easy to check the quality of compiler optimizations and acquire data to improve them HCQC：HPC compiler quality checker 2
- 3. Subject of Quality Check • Configuration file defines the subject of quality check. • Main items: Compiler Compiler version Optimization flags { “DISTRIBUTION" : "OpenSUSE Tumbleweed", "ARCH" : “aarch64", "CPU" : "AMD Opteron A1100 Cortex A57", "LANGUAGE" : "C", "COMPILER" : “GCC", "COMMAND" : "/usr/bin/gcc", "VERSION" : “7.1.1", "OPT_FLAGS" : ["-O2"], "ASM_FLAGS" : ["-S“, “-fverbose-asm”], “FLAG_DB” : [[“?DEBUG_FLAG", “-g”], [“?C99_STANDARD", “-std=c99”]] } 3 Example of configuration file
- 4. Metrics for Quality Evaluation • HCQC has the following metrics: op : # of mnemonics in an assembly code kind : The kind of mnemonics in an assembly code(memory, branch, other) regalloc : The quality of register allocation(# of spill in/out instructions) height : The height of instruction dependence graph ilp :Instruction level parallelism by instruction scheduler vectorize : Vectrization/SIMDization situation swpl : # of initiation interval by software pipelining 4 These data are basically static data at compile time.
- 5. Investigation Result 5 ARCH : aarch64 CPU : AMD Opteron A1100 Cortex A57 LANGUAGE : C COMPILER : ClangLLVM COMMAND : /usr/bin/clang VERSION :4.0.1 OPT_FLAGS : -O2 TEST_PROGRAM: sample KERNEL_FUNCTION_NAME : kernel DATE: 2017/11/07 ilp swpl memorybranch other spill in spill out IPC II kind mem arith other BB0 cond LBB0_11 0 3 1 3 0 0 0.5 0 0 0 BB1 0 0 0 4 0 0 0.5 0 0 0 LBB0_2 1 3 0 5 0 1 0.7 0 0 0 LBB0_3 cond LBB0_5 2 2 1 1 2 0 0.9 SLP 0 1 0 BB4 LBB0_7 LBB0_9 2 1 2 5 0 0 0.9 SLP 0 1 1 LBB0_5 cond LBB0_9 2 3 1 5 0 0 1.3 SLP 0 1 1 LBB0_7 2 3 0 5 0 3 1.7 SLP 0 1 2 LBB0_8 cond LBB0_8 3 7 1 5 0 0 2.5 5 LOOP,SLP 2 2 2 LBB0_9 cond LBB0_3 2 2 1 3 0 2 1.3 SLP 0 1 1 BB10 cond LBB0_2 1 1 1 2 0 0 0.8 0 0 0 LBB0_11 0 0 0 1 0 0 0.2 0 0 0 *SUMMARY* 25 8 39 2 6 2 7 7 vectorizekind regalloc CFG DEPTH
- 6. Quality Evaluation by Comparison • One investigation result has little meaning. • Typical comparison examples: GCC vs. LLVM(on AArch64) LLVM 4.0.0 vs. LLVM 5.0.0(on AArch64) LLVM with –O2 vs. LLVM with –O3(on AArch64) LLVM on AArch64 vs. LLVM on x86_64 Missing optimizations on AArch64 LLVM on AArch64 vs. ICC on x86_84 Optimization hints for SVE from AVX codes 6
- 7. Example of Comparison(1) • HimenoBMT-dynamic(regalloc) GCC is better than Clang/LLVM. 7 CFG DEPTH spill in spill out CFG DEPTH spill in spill out jacobi cond .L18 0 0 7 jacobi cond .LBB0_32 0 0 12 0 0 5 0 0 6 .L3 cond .L16 1 1 0 .LBB0_2 cond .LBB0_31 1 1 0 1 3 2 1 0 2 .L8 cond .L28 2 1 0 .LBB0_4 cond .LBB0_11 2 0 1 2 4 6 2 0 1 .L11 cond .L7 3 0 1 .LBB0_6 cond .LBB0_10 4 0 0 3 10 0 3 13 7 .L4 cond .L4 4 0 0 .LBB0_8 cond .LBB0_8 4 2 0 .L7 cond .L11 3 2 0 cond .LBB0_6 3 3 0 .L5 cond .L8 2 5 1 goto .LBB0_11 2 0 0 1 5 0 .LBB0_10 cond .LBB0_6 3 0 0 .L9 cond .L13 2 0 0 .LBB0_11 cond .LBB0_4 2 3 2 2 0 0 cond .LBB0_30 1 1 0 .L17 cond .L15 3 0 0 1 3 0 3 0 0 .LBB0_14 cond .LBB0_29 2 0 0 .L12 cond .L12 4 0 0 goto .LBB0_19 2 0 0 .L15 cond .L17 3 0 0 .LBB0_16 3 0 0 .L13 cond .L9 2 0 0 .LBB0_17 cond .LBB0_17 4 0 0 .L16 cond .L3 1 2 1 cond .LBB0_28 3 0 0 end 0 0 0 goto .LBB0_26 3 0 0 .L28 goto .L5 2 1 2 .LBB0_19 cond .LBB0_28 3 0 0 .L18 end 0 0 0 cond .LBB0_22 3 1 0 *SUMMARY* - 34 25 goto .LBB0_26 3 0 0 .LBB0_22 cond .LBB0_25 3 0 0 cond .LBB0_16 3 0 0 cond .LBB0_16 3 0 0 .LBB0_25 3 0 0 .LBB0_26 3 0 0 .LBB0_27 cond .LBB0_27 4 0 0 .LBB0_28 cond .LBB0_19 3 0 0 .LBB0_29 cond .LBB0_14 2 1 0 goto .LBB0_31 1 0 0 .LBB0_30 1 1 0 .LBB0_31 cond .LBB0_2 1 0 0 goto .LBB0_33 0 0 0 .LBB0_32 0 0 0 .LBB0_33 end 0 8 0 *SUMMARY* - 37 31 LLVMGCC LLVMGCC
- 8. Example of Comparison(2) • mVMC-mini-calculateNewPfMTwo_child(op) – GCC generates codes with better addressing modes. 8 CFG DEPTH lsl calculateNewPfMTwo_child cond .LBB0_3 0 0 0 0 .LBB0_2 cond .LBB0_2 1 1 .LBB0_3 cond .LBB0_11 0 0 0 0 .LBB0_5 cond .LBB0_5 1 0 cond .LBB0_12 0 0 0 1 .LBB0_8 1 0 .LBB0_9 cond .LBB0_9 2 0 cond .LBB0_8 1 0 goto .LBB0_13 0 0 .LBB0_11 0 0 .LBB0_12 0 0 .LBB0_13 end 0 0 *SUMMARY* - 2 CFG DEPTH lsl calculateNewPfMTwo_child cond .L2 0 0 .L3 cond .L3 1 .L2 cond .L8 0 0 .L5 cond .L5 1 0 .L7 1 .L6 cond .L6 2 cond .L7 1 0 .L4 end 0 .L8 goto .L4 0 *SUMMARY* - LLVM GCC .LBB0_2: ldr w1, [x5, x16, lsl #2] sdiv w2, w16, w13 lsl x3, x16, #3 add x16, x16, #1 madd w1, w17, w2, w1 sbfiz x1, x1, #3, #32 ldr x2, [x18, x1] cmp x11, x16 str x2, [x10, x3] ldr x1, [x0, x1] str x1, [x9, x3] b.ne .LBB0_2 // x3 is dead LLVM
- 9. Example of Comparison(3) • HimenoBMT-static(height) Clang/LLVM is not good for register usage. 9 CFG DEPTH # height .LBB1_8 cond .LBB1_8 4 71 25 CFG DEPTH # height .L19 cond .L19 4 62 17 LLVM GCC 71/25 = 2.84 62/17 = 3.647 .LBB1_8: add x17, x4, x14 mov v6.16b, v17.16b ldr s17, [x17, #8844] add x16, x3, x14 mov v7.16b, v16.16b ldr s16, [x16, #8844] add x17, x25, x14 ldr s21, [x17, x7] fmul s0, s17, s0 ldr s17, [x17, x21] add x16, x5, x14 mov v5.16b, v18.16b ldr s18, [x16, #8844] …… LLVM The metric `height’ is not committed on GitHub yet.
- 10. Workflow of HCQC ① Compile one test program ② Run the executable file and verify its result by comparing output and answer data ③ Generate the assembly code file ④ Make the control flow graph of the kernel part from the assembly code ⑤ Get result data using metric programs ⑥ Make the report file from data 10 % hcqc config test metric+ % hcqc-report config test metric+
- 11. Workflow of HCQC 11 COMMAND:/usr/bin/clang OPT_FLAGS:-O2 Configuration FileTest Program Executable File out.data resut.data in.data Assembly Code File cfg.py CFG+DEPTH hcqc-report diff check DISTRIBUTION : OpenSUSE Tumbleweed ARCH : aarch64 CPU : AMD Opteron A1100 Cortex A57 LANGUAGE : C COMPILER : ClangLLVM COMMAND : /usr/bin/clang VERSION :4.0.1 OPT_FLAGS : -O2 TEST_PROGRAM: sample KERNEL=FUNCTION=NAME : kernel DATE: 2017/11/07 ilp swpl mem branch other spill in spill out IPC II kind mem arith other BB0 cond LBB0_11 0 3 1 3 0 0 0.5 0 0 0 BB1 0 0 0 4 0 0 0.5 0 0 0 LBB0_2 1 3 0 5 0 1 0.7 0 0 0 LBB0_3 cond LBB0_5 2 2 1 1 2 0 0.9 SLP 0 1 0 BB4 LBB0_7 LBB0_9 2 1 2 5 0 0 0.9 SLP 0 1 1 LBB0_5 cond LBB0_9 2 3 1 5 0 0 1.3 SLP 0 1 1 LBB0_7 2 3 0 5 0 3 1.7 SLP 0 1 2 LBB0_8 cond LBB0_8 3 7 1 5 0 0 2.5 5 LOOP,SLP 2 2 2 LBB0_9 cond LBB0_3 2 2 1 3 0 2 1.3 SLP 0 1 1 BB10 cond LBB0_2 1 1 1 2 0 0 0.8 0 0 0 LBB0_11 0 0 0 1 0 0 0.2 0 0 0 *SUMMARY* 25 8 39 2 6 2 7 7 vectorizekind regalloc CFG DEPTH Report file(csv) Result Data Result Data Result Data Metric Program Metric Program Metric Program Test Program Info File ① ② ③ ④ ⑤ ⑥ JSON format file Debug Information?
- 12. Test Programs for HCQC • Generate from programs that were problematic in Fujitsu's production compilers in the past • Extract kernel parts and modify them to use under HCQC – Extract hot spots – If it is Fortran program, then convert them to C language(for comparison between GCC and Clang/LLVM) – Prepare the data to run and check those kernel parts 12
- 13. Test Programs for HCQC • All original benchmarks are publically available. • I/O data for HCQC is being prepared. – Some data file sizes are very large. – For different architectures or different optimization levels, error tolerance is required. 13 benchark name kernel name HimenoBMT-dynamic jacobi HimenoBMT-static jacobi hpcg-3.0 ComputeSYMGS_ref ccs-qcd bicgstab_hmc ccs-qcd clover ffb-mini CALAX3 ffb-mini FLD3X2 ffb-mini GRAD3X ffvc-mini poi_residual ffvc-mini psor2sma_core mVMC-mini calculateNewPfMTwo_child mVMC-mini updateMAllTwo_child mVMC-mini updateMAll_child ngsa-mini bwt_match_exact_alt ngsa-mini bwt_match_gap nicam-dc-mini vi_path2
- 14. Future Work • Add supports for SVE(if available in GCC or LLVM) • Implement metric programs: – vectorization(vectorize) – software pipelining(swpl) – instruction level parallelism(ilp) • Add features for comparing with x86_64(SVE vs. AVX) • Add tools for automatic and intelligent comparison 14
- 15. 15 URL https://github.com/Linaro/hcqc Thank you very much! Any comments or suggestions are welcome.