How to test OpenGL drivers using Free Software (FOSDEM 2015)
0 likes737 views
The document discusses OpenGL, an API for rendering 2D and 3D graphics, detailing its history, management by the Khronos Group, and the existence of potential implementation incompatibilities. It describes various OpenGL conformance test suites, including official and unofficial options such as Glean, DEQP, and Piglit, along with instructions for their use and installation. The document concludes with the importance of these test suites in ensuring API conformance and the licensing of the work under a Creative Commons license.
![dEQP
● Clone repository:
$ git clone https://android.googlesource.com/platform/external/deqp/
● [Optional] Add to CMakeLists.txt:
set(PNG_INCLUDE_PATH "/usr/include/libpng")
set(PNG_LIBRARY "/usr/lib/x86_64-linux-gnu/libpng.so")
set(CMAKE_BUILD_TYPE DEBUG)
set(CMAKE_C_FLAGS "-O0 -ggdb")
set(CMAKE_C_FLAGS_DEBUG "-O0 -ggdb")
set(CMAKE_C_FLAGS_RELEASE "-O0 -ggdb")
set(CMAKE_CXX_FLAGS "-O0 -ggdb")
set(CMAKE_CXX_FLAGS_DEBUG "-O0 -ggdb")
set(CMAKE_CXX_FLAGS_RELEASE "-O0 -ggdb")](https://image.slidesharecdn.com/howtotestopengldriversusingfreesoftware-150202091502-conversion-gate01/85/How-to-test-OpenGL-drivers-using-Free-Software-FOSDEM-2015-14-320.jpg)
![Piglit
● CLI output
[...]
spec/glsl-1.50/compiler/interface-block-name-uses-gl-
prefix.vert: pass
spec/EXT_framebuffer_object/fbo-clear-formats/GL_ALPHA16 (fbo
incomplete): skip
spec/ARB_copy_image/arb_copy_image-targets
GL_TEXTURE_CUBE_MAP_ARRAY 32 32 18 GL_TEXTURE_2D_ARRAY 32 16 15 11
12 5 5 1 2 14 15 9: pass
spec/glsl-1.30/execution/built-in-functions/fs-op-bitxor-uvec3-
uint: pass
spec/ARB_depth_texture/depthstencil-render-miplevels 146 d=z16:
pass
spec/glsl-1.10/execution/variable-indexing/fs-varying-mat2-col-
row-rd: pass](https://image.slidesharecdn.com/howtotestopengldriversusingfreesoftware-150202091502-conversion-gate01/85/How-to-test-OpenGL-drivers-using-Free-Software-FOSDEM-2015-27-320.jpg)
![Piglit
● How to run standalone tests
$ bin/texsubimage pbo -auto -fbo
Using test set: Core formats
texsubimage failed
target: GL_TEXTURE_2D
internal format: GL_COMPRESSED_RGB_S3TC_DXT1_EXT
region: 8, 8 48 x 28
[...]
PIGLIT: {"result": "fail" }](https://image.slidesharecdn.com/howtotestopengldriversusingfreesoftware-150202091502-conversion-gate01/85/How-to-test-OpenGL-drivers-using-Free-Software-FOSDEM-2015-31-320.jpg)
How to test OpenGL drivers using Free Software (FOSDEM 2015)
- 1. glGenVertexArrays(1, &vao); glBindVertexArray(vao); /* Draw and record */ glBeginTransformFeedback(GL_POINTS); glDrawArrays(GL_POINTS, 0, 1); for (i = 0; i < STREAMS; i++) { glEndQueryIndexed(GL_PRIMITIVES_GENERATED, i); glEndQueryIndexed(GL_TRANSFORM_FEEDBACK_ PRIMITIVES_WRITTEN, i); } glEndTransformFeedback(); glDeleteVertexArrays(1, &vao); if (!piglit_check_gl_error(GL_NO_ERROR)) piglit_report_result(PIGLIT_FAIL); pass = probe_buffers(xfb, queries, primitive_n); PIGLIT_FAIL); Samuel Iglesias Gonsálvez siglesias@igalia.com How to test OpenGL drivers with Free Software
- 2. Introduction
- 3. OpenGL ● OpenGL is an API for rendering 2D and 3D graphics. ● Silicon Graphics Inc. (SGI) started developing OpenGL in 1991 and released it in January 1992 ● Now managed by the non-profit technology consortium Khronos Group.
- 4. OpenGL ● It is widely use in different form factor devices (PC, tablets, mobile phones, embedded devices...) for running applications that require 2D/3D graphics rendering (games, scientific applications, etc)
- 5. OpenGL ● However, OpenGL is only an API specification... ● Implementors are free to provide their own libraries and drivers. ● Possible incompatibilities between implementations and OpenGL. ● Solution: OpenGL conformance test suites
- 6. OpenGL conformance test suites
- 7. OpenGL conformance test suites ● There is an official OpenGL conformance test suite from Khronos group: https://www.khronos.org/opengl/adopters/ ● It is for free (as in beer) for software developers but it is not Free Software (as in speech). https://www.khronos.org/files/adopters_agreement.pdf
- 8. OpenGL conformance test suites ● There are unofficial Free Software OpenGL conformance test suites: ● Glean ● DrawElements Quality Program (dEQP) ● Piglit
- 9. Glean
- 10. Glean ● Created in 1999 by Allen Akin. ● MIT license. ● Tested on GNU/Linux and Microsoft Windows. ● There hasn't been much activity in Glean for a few years now. ● Nowadays, it's part of Piglit http://glean.sourceforge.net/
- 11. dEQP
- 12. dEQP ● Draw Elements Quality Program (dEQP) ● Developed by Finnish company called drawElements. – Bought by Google in 2014 ● Google open-sourced dEQP in the Android Lollipop release. ● Apache License 2.0
- 13. dEQP ● Supports OpenGL ES 2.0 and better, OpenCL 1.X, and EGL 1.4. ● Supported OSes include Android, GNU/Linux, Windows, iOS. ● Recently, piglit added support for running dEQP GLES3 tests.
- 14. dEQP ● Clone repository: $ git clone https://android.googlesource.com/platform/external/deqp/ ● [Optional] Add to CMakeLists.txt: set(PNG_INCLUDE_PATH "/usr/include/libpng") set(PNG_LIBRARY "/usr/lib/x86_64-linux-gnu/libpng.so") set(CMAKE_BUILD_TYPE DEBUG) set(CMAKE_C_FLAGS "-O0 -ggdb") set(CMAKE_C_FLAGS_DEBUG "-O0 -ggdb") set(CMAKE_C_FLAGS_RELEASE "-O0 -ggdb") set(CMAKE_CXX_FLAGS "-O0 -ggdb") set(CMAKE_CXX_FLAGS_DEBUG "-O0 -ggdb") set(CMAKE_CXX_FLAGS_RELEASE "-O0 -ggdb")
- 15. dEQP ● Build: $ cmake . $ make ● Note: it doesn't provide an "install" target.
- 16. dEQP ● Run the tests with execserver ● Create list of tests to run $ cd modules/gles3 $ ./deqp-gles3 --deqp-runmode=xml-caselist ● Launch server $ execserver/execserver ● Launch executor (in other terminal) $ ./executor/executor -b deqp-gles3 -wd modules/gles3/ -cd modules/gles3/ -t dEQP-GLES3.functional.* -o ./gles3- functional.xml --cmdline="--deqp-log-images=disable"
- 17. dEQP
- 18. dEQP ● Run the tests without execserver: $ cd ./modules/gles3 $ ./deqp-gles3 -n dEQP-GLES3.functional.* ● Results are saved in TestResults.qpa file
- 19. dEQP
- 20. dEQP ● How to convert Base64 string to PNG images: ● http://www.base64-image.net/ Result Reference Error mask
- 21. dEQP ● It can convert XML files to CSV $ executor/testlog-to-csv ● No HTML output
- 22. Piglit
- 23. Piglit ● Created by Nicolai Hähnle in 2007 ● Widely used in Mesa for testing and development ● MIT license ● Include tests for OpenGL, OpenGL ES, OpenCL. http://piglit.freedesktop.org
- 24. Piglit ● Dependencies ● Python 2.7.x ● Python mako module ● numpy ● cmake ● GL, glu and glut libraries and development packages (i.e. headers) ● X11 libraries and development packages (i.e. headers) ● Waffle: http://www.waffle-gl.org
- 25. Piglit ● Build: $ git clone git://anongit.freedesktop.org/git/piglit $ cmake . $ make ● Check installation: $ ./piglit run tests/sanity.tests results/sanity.results ● Your first complete piglit run: $ ./piglit run tests/all results/all-run
- 26. Piglit ● Process results file: ● HTML output $ ./piglit summary html summary/summary-all results/all-run ● CLI output $ ./piglit summary console results/all-run ● CSV output $ ./piglit summary csv results/all-run
- 27. Piglit ● CLI output [...] spec/glsl-1.50/compiler/interface-block-name-uses-gl- prefix.vert: pass spec/EXT_framebuffer_object/fbo-clear-formats/GL_ALPHA16 (fbo incomplete): skip spec/ARB_copy_image/arb_copy_image-targets GL_TEXTURE_CUBE_MAP_ARRAY 32 32 18 GL_TEXTURE_2D_ARRAY 32 16 15 11 12 5 5 1 2 14 15 9: pass spec/glsl-1.30/execution/built-in-functions/fs-op-bitxor-uvec3- uint: pass spec/ARB_depth_texture/depthstencil-render-miplevels 146 d=z16: pass spec/glsl-1.10/execution/variable-indexing/fs-varying-mat2-col- row-rd: pass
- 28. Piglit summary: pass: 25085 fail: 262 crash: 5 skip: 9746 timeout: 0 warn: 13 dmesg-warn: 0 dmesg-fail: 0 total: 35111
- 29. Piglit
- 30. Piglit
- 31. Piglit ● How to run standalone tests $ bin/texsubimage pbo -auto -fbo Using test set: Core formats texsubimage failed target: GL_TEXTURE_2D internal format: GL_COMPRESSED_RGB_S3TC_DXT1_EXT region: 8, 8 48 x 28 [...] PIGLIT: {"result": "fail" }
- 32. Piglit ● Tailor your piglit run ● Add Dmesg information: --dmesg ● Verbose: -l verbose ● Run all concurrent: --all-concurrent ● Disable concurrent test runs: --no- concurrency
- 33. Piglit ● Tailor your piglit run ● Execute by name (*texture*) $ ./piglit run tests/all -t texture results/all-texture ● Remove tests by name (*color*) $ ./piglit run tests/all -t texture -x color results/all- texture ● More information: $ ./piglit run -h
- 34. Piglit ● Test profile – List of tests to be executed by piglit in a given run. – Usually imports other test profile and edits the list of test (add/remove). – Written in python.
- 35. Piglit
- 36. Piglit ● DEQP GLES 3 support $ export PIGLIT_DEQP_GLES3_EXE=~/deqp/modules/gles3/deqp- gles3 $ ./piglit run tests/deqp_gles3.py -t dEQP- GLES3.functional.* results/gles3-functional
- 37. Conclusions
- 38. Conclusions ● OpenGL is an API ● Implementations can be incompatible ● Need OpenGL conformance test suites ● Khronos provides an official one but it is not Free Software (as in speech). ● There are unofficial Free Software alternatives: – Glean, dEQP, piglit
- 39. Questions?
- 40. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.