Fuzzing Janus @ IPTComm 2019
- 1. Fuzzing Janus for Fun and Profit A. Amirante, T. Castaldi, L. Miniero, S.P. Romano, P. Saviano, A. Toppi IPTComm 2019 October 15th 2019, Chicago, IL, USA
- 2. A few words about me Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Main author of Janus® Contacts and info • lorenzo@meetecho.com • https://twitter.com/elminiero • https://www.slideshare.net/LorenzoMiniero
- 3. Vulnerabilities in RTC communications • Project Zero is a team of security analysts employed by Google • https://googleprojectzero.blogspot.com/ • Recently focused on videoconferencing applications • Focus on end-to-end, and RTP testing • Malicious endpoint generating randomized input • Built new tools required for the task • Targeted many applications, and found dangerous bugs • Apple FaceTime • WhatsApp • WebRTC Philipp Hancke’s wakeup call https://webrtchacks.com/lets-get-better-at-fuzzing-in-2019-heres-how/
- 4. Vulnerabilities in RTC communications • Project Zero is a team of security analysts employed by Google • https://googleprojectzero.blogspot.com/ • Recently focused on videoconferencing applications • Focus on end-to-end, and RTP testing • Malicious endpoint generating randomized input • Built new tools required for the task • Targeted many applications, and found dangerous bugs • Apple FaceTime • WhatsApp • WebRTC Philipp Hancke’s wakeup call https://webrtchacks.com/lets-get-better-at-fuzzing-in-2019-heres-how/
- 5. Vulnerabilities in RTC communications • Project Zero is a team of security analysts employed by Google • https://googleprojectzero.blogspot.com/ • Recently focused on videoconferencing applications • Focus on end-to-end, and RTP testing • Malicious endpoint generating randomized input • Built new tools required for the task • Targeted many applications, and found dangerous bugs • Apple FaceTime • WhatsApp • WebRTC Philipp Hancke’s wakeup call https://webrtchacks.com/lets-get-better-at-fuzzing-in-2019-heres-how/
- 6. Vulnerabilities in RTC communications • Project Zero is a team of security analysts employed by Google • https://googleprojectzero.blogspot.com/ • Recently focused on videoconferencing applications • Focus on end-to-end, and RTP testing • Malicious endpoint generating randomized input • Built new tools required for the task • Targeted many applications, and found dangerous bugs • Apple FaceTime • WhatsApp • WebRTC Philipp Hancke’s wakeup call https://webrtchacks.com/lets-get-better-at-fuzzing-in-2019-heres-how/
- 7. What can cause trouble in WebRTC? • WebRTC is signalling agnostic, so typically not that • You can use SIP, XMPP, some JSON flavour, etc. • A lot of media-related protocols to worry about, though • STUN/TURN (NAT traversal) • DTLS/DTLS-SRTP (secure exchange of keys and data) • RTP/RTCP (or actually, SRTP/SRTCP), including RTP extensions • SCTP (data channels) • ... and codec specific payloads • Identifying keyframes (VP8, VP9, H.264) • Simulcast & SVC (inspecting payloads)
- 8. What can cause trouble in WebRTC? • WebRTC is signalling agnostic, so typically not that • You can use SIP, XMPP, some JSON flavour, etc. • A lot of media-related protocols to worry about, though • STUN/TURN (NAT traversal) • DTLS/DTLS-SRTP (secure exchange of keys and data) • RTP/RTCP (or actually, SRTP/SRTCP), including RTP extensions • SCTP (data channels) • ... and codec specific payloads • Identifying keyframes (VP8, VP9, H.264) • Simulcast & SVC (inspecting payloads)
- 9. What can cause trouble in WebRTC? • WebRTC is signalling agnostic, so typically not that • You can use SIP, XMPP, some JSON flavour, etc. • A lot of media-related protocols to worry about, though • STUN/TURN (NAT traversal) • DTLS/DTLS-SRTP (secure exchange of keys and data) • RTP/RTCP (or actually, SRTP/SRTCP), including RTP extensions • SCTP (data channels) • ... and codec specific payloads • Identifying keyframes (VP8, VP9, H.264) • Simulcast & SVC (inspecting payloads)
- 10. Why can fuzz testing help? • Automated software testing technique • Unexpected or invalid data submitted to a program • Input pattern modified according to a defined strategy (e.g., for coverage) • Typical workflow 1 Engine generates input based on existing dataset (“Corpus”) 2 Input mutated slightly over time 3 Input data passed to target function and monitored (e.g., via sanitizers) 4 Coverage of new lines updates stats and Corpus (new pattern) 5 Repeat until it crashes • Repeatability can be ensured using the same seeds or previous dumps
- 11. Why can fuzz testing help? • Automated software testing technique • Unexpected or invalid data submitted to a program • Input pattern modified according to a defined strategy (e.g., for coverage) • Typical workflow 1 Engine generates input based on existing dataset (“Corpus”) 2 Input mutated slightly over time 3 Input data passed to target function and monitored (e.g., via sanitizers) 4 Coverage of new lines updates stats and Corpus (new pattern) 5 Repeat until it crashes • Repeatability can be ensured using the same seeds or previous dumps
- 12. Why can fuzz testing help? • Automated software testing technique • Unexpected or invalid data submitted to a program • Input pattern modified according to a defined strategy (e.g., for coverage) • Typical workflow 1 Engine generates input based on existing dataset (“Corpus”) 2 Input mutated slightly over time 3 Input data passed to target function and monitored (e.g., via sanitizers) 4 Coverage of new lines updates stats and Corpus (new pattern) 5 Repeat until it crashes • Repeatability can be ensured using the same seeds or previous dumps
- 13. Introducing the Janus WebRTC server Janus General purpose, open source WebRTC server • https://github.com/meetecho/janus-gateway • Demos and documentation: https://janus.conf.meetecho.com • Community: https://groups.google.com/forum/#!forum/meetecho-janus
- 14. Modular architecture • The core only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
- 15. Modular architecture • The core only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
- 16. Modular architecture • The core only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
- 17. Modular architecture • The core only implements the WebRTC stack • JSEP/SDP, ICE, DTLS-SRTP, Data Channels, Simulcast, VP9-SVC, ... • Plugins expose Janus API over different “transports” • Currently HTTP / WebSockets / RabbitMQ / Unix Sockets / MQTT / Nanomsg • “Application” logic implemented in plugins too • Users attach to plugins via the Janus core • The core handles the WebRTC stuff • Plugins route/manipulate the media/data • Plugins can be combined on client side as “bricks” • Video SFU, Audio MCU, SIP gatewaying, broadcasting, etc.
- 18. Choosing the fuzzing targets • Many protocols via dependencies are fuzzed already • ICE/STUN/TURN (libnice) • DTLS/DTLS-SRTP (OpenSSL/LibreSSL/BoringSSL) • SRTP/SRTCP (libsrtp) • SCTP (usrsctplib) • Some other dependencies MAY need fuzzing (but not in Janus?) • Transports (HTTP, WebSockets, RabbitMQ, etc.) • JSON support (Jansson) • Custom code DEFINITELY needs fuzzing • RTCP parsing (e.g., compound packets) • RTP processing (e.g., RTP extensions, codec specific payloads) • SDP parsing and processing
- 19. Choosing the fuzzing targets • Many protocols via dependencies are fuzzed already • ICE/STUN/TURN (libnice) • DTLS/DTLS-SRTP (OpenSSL/LibreSSL/BoringSSL) • SRTP/SRTCP (libsrtp) • SCTP (usrsctplib) • Some other dependencies MAY need fuzzing (but not in Janus?) • Transports (HTTP, WebSockets, RabbitMQ, etc.) • JSON support (Jansson) • Custom code DEFINITELY needs fuzzing • RTCP parsing (e.g., compound packets) • RTP processing (e.g., RTP extensions, codec specific payloads) • SDP parsing and processing
- 20. Choosing the fuzzing targets • Many protocols via dependencies are fuzzed already • ICE/STUN/TURN (libnice) • DTLS/DTLS-SRTP (OpenSSL/LibreSSL/BoringSSL) • SRTP/SRTCP (libsrtp) • SCTP (usrsctplib) • Some other dependencies MAY need fuzzing (but not in Janus?) • Transports (HTTP, WebSockets, RabbitMQ, etc.) • JSON support (Jansson) • Custom code DEFINITELY needs fuzzing • RTCP parsing (e.g., compound packets) • RTP processing (e.g., RTP extensions, codec specific payloads) • SDP parsing and processing
- 21. A quick intro to libFuzzer • Popular coverage-guided fuzzing engine, part of the LLVM project • https://llvm.org/docs/LibFuzzer.html • Used by several well known applications • glibc, OpenSSL/LibreSSL/BoringSSL, SQLite, FFmpeg and many more • A few key characteristics • Needs sources to be compiled with Clang • Works in-process (linked with the library/application under test) • Feeds inputs to the target via a fuzzing entrypoint (target function) • Execution of the target function is monitored with sanitizers tools (e.g., libasan)
- 22. A quick intro to libFuzzer • Popular coverage-guided fuzzing engine, part of the LLVM project • https://llvm.org/docs/LibFuzzer.html • Used by several well known applications • glibc, OpenSSL/LibreSSL/BoringSSL, SQLite, FFmpeg and many more • A few key characteristics • Needs sources to be compiled with Clang • Works in-process (linked with the library/application under test) • Feeds inputs to the target via a fuzzing entrypoint (target function) • Execution of the target function is monitored with sanitizers tools (e.g., libasan)
- 23. A quick intro to libFuzzer • Popular coverage-guided fuzzing engine, part of the LLVM project • https://llvm.org/docs/LibFuzzer.html • Used by several well known applications • glibc, OpenSSL/LibreSSL/BoringSSL, SQLite, FFmpeg and many more • A few key characteristics • Needs sources to be compiled with Clang • Works in-process (linked with the library/application under test) • Feeds inputs to the target via a fuzzing entrypoint (target function) • Execution of the target function is monitored with sanitizers tools (e.g., libasan)
- 25. libFuzzer in (simplified) practice 1 Implement the method to receive and process the input data // my_fuzzer.c int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { ProcessData(Data, Size); return 0; } 2 Compile with Clang and the right flags > clang -g -O1 -fsanitize=fuzzer,address,undefined my_fuzzer.c 3 Launch passing the Corpus folder as the argument > ./my_fuzzer CORPUS_DIR 4 In case of crashes, pass the dumped input (e.g., via gdb, or to test regressions) > gdb --args ./my_fuzzer crash-file-dump
- 26. libFuzzer in (simplified) practice 1 Implement the method to receive and process the input data // my_fuzzer.c int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { ProcessData(Data, Size); return 0; } 2 Compile with Clang and the right flags > clang -g -O1 -fsanitize=fuzzer,address,undefined my_fuzzer.c 3 Launch passing the Corpus folder as the argument > ./my_fuzzer CORPUS_DIR 4 In case of crashes, pass the dumped input (e.g., via gdb, or to test regressions) > gdb --args ./my_fuzzer crash-file-dump
- 27. libFuzzer in (simplified) practice 1 Implement the method to receive and process the input data // my_fuzzer.c int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { ProcessData(Data, Size); return 0; } 2 Compile with Clang and the right flags > clang -g -O1 -fsanitize=fuzzer,address,undefined my_fuzzer.c 3 Launch passing the Corpus folder as the argument > ./my_fuzzer CORPUS_DIR 4 In case of crashes, pass the dumped input (e.g., via gdb, or to test regressions) > gdb --args ./my_fuzzer crash-file-dump
- 28. libFuzzer in (simplified) practice 1 Implement the method to receive and process the input data // my_fuzzer.c int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { ProcessData(Data, Size); return 0; } 2 Compile with Clang and the right flags > clang -g -O1 -fsanitize=fuzzer,address,undefined my_fuzzer.c 3 Launch passing the Corpus folder as the argument > ./my_fuzzer CORPUS_DIR 4 In case of crashes, pass the dumped input (e.g., via gdb, or to test regressions) > gdb --args ./my_fuzzer crash-file-dump
- 29. Integrating libFuzzer in Janus • First step was Clang support (Janus normally built with gcc) • Streamlined compilation flags in the process • Got useful warnings that led to some fixes too • Next step was choosing what to fuzz • Decided to start with RTCP • Compound packets + length values + overflows = “fun”... • Then worked on the libFuzzer workflow 1 Fuzzing target with critical RTCP-related functions 2 Helper script to build the fuzzer 3 Helper script to run the fuzzer Original pull request (now merged, with RTP and SDP fuzzing as well) https://github.com/meetecho/janus-gateway/pull/1492
- 30. Integrating libFuzzer in Janus • First step was Clang support (Janus normally built with gcc) • Streamlined compilation flags in the process • Got useful warnings that led to some fixes too • Next step was choosing what to fuzz • Decided to start with RTCP • Compound packets + length values + overflows = “fun”... • Then worked on the libFuzzer workflow 1 Fuzzing target with critical RTCP-related functions 2 Helper script to build the fuzzer 3 Helper script to run the fuzzer Original pull request (now merged, with RTP and SDP fuzzing as well) https://github.com/meetecho/janus-gateway/pull/1492
- 31. Integrating libFuzzer in Janus • First step was Clang support (Janus normally built with gcc) • Streamlined compilation flags in the process • Got useful warnings that led to some fixes too • Next step was choosing what to fuzz • Decided to start with RTCP • Compound packets + length values + overflows = “fun”... • Then worked on the libFuzzer workflow 1 Fuzzing target with critical RTCP-related functions 2 Helper script to build the fuzzer 3 Helper script to run the fuzzer Original pull request (now merged, with RTP and SDP fuzzing as well) https://github.com/meetecho/janus-gateway/pull/1492
- 32. Integrating libFuzzer in Janus • First step was Clang support (Janus normally built with gcc) • Streamlined compilation flags in the process • Got useful warnings that led to some fixes too • Next step was choosing what to fuzz • Decided to start with RTCP • Compound packets + length values + overflows = “fun”... • Then worked on the libFuzzer workflow 1 Fuzzing target with critical RTCP-related functions 2 Helper script to build the fuzzer 3 Helper script to run the fuzzer Original pull request (now merged, with RTP and SDP fuzzing as well) https://github.com/meetecho/janus-gateway/pull/1492
- 33. Integrating libFuzzer in Janus // fuzz-rtcp.c #include "janus/rtcp.h" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { if (size < 8 || size > 1472) return 0; if (!janus_is_rtcp(data, size)) return 0; /* Initialize an empty RTCP context */ janus_rtcp_context ctx; janus_rtcp_parse(ctx, (char *)data, size); GSList *list = janus_rtcp_get_nacks((char *)data, size); ... if (list) g_slist_free(list); return 0; }
- 34. Presenting the code coverage
- 35. Corpora files: a shared effort https://github.com/RTC-Cartel/webrtc-fuzzer-corpora
- 36. Scalable distributed fuzzing via OSS-Fuzz https://github.com/google/oss-fuzz/pull/2241 (Janus addition)
- 37. Scalable distributed fuzzing via OSS-Fuzz https://github.com/google/oss-fuzz/pull/2241 (Janus addition)
- 38. A detailed tutorial on how to setup all this https://webrtchacks.com/fuzzing-janus/
- 39. What’s next? • So far, we only fuzzed RTP, RTCP and in part SDP in the core • SDP fuzzing should be improved (maybe with structure-aware fuzzing?) • What about plugins and their custom interactions? • Definitely expand the corpora • The shared RTC-Cartel repo should help with that • Exchanging crash causes with other projects will make both more robust • libFuzzer is not the only option here • Some popular alternatives are AFL, Radamsa, Gasoline, etc. • KITE and its “weaponised” browsers can be very helpful as an orthogonal testing tool
- 40. What’s next? • So far, we only fuzzed RTP, RTCP and in part SDP in the core • SDP fuzzing should be improved (maybe with structure-aware fuzzing?) • What about plugins and their custom interactions? • Definitely expand the corpora • The shared RTC-Cartel repo should help with that • Exchanging crash causes with other projects will make both more robust • libFuzzer is not the only option here • Some popular alternatives are AFL, Radamsa, Gasoline, etc. • KITE and its “weaponised” browsers can be very helpful as an orthogonal testing tool
- 41. What’s next? • So far, we only fuzzed RTP, RTCP and in part SDP in the core • SDP fuzzing should be improved (maybe with structure-aware fuzzing?) • What about plugins and their custom interactions? • Definitely expand the corpora • The shared RTC-Cartel repo should help with that • Exchanging crash causes with other projects will make both more robust • libFuzzer is not the only option here • Some popular alternatives are AFL, Radamsa, Gasoline, etc. • KITE and its “weaponised” browsers can be very helpful as an orthogonal testing tool
- 42. Thanks! Questions? Comments? Get in touch! • https://twitter.com/elminiero • https://twitter.com/meetecho • https://www.meetecho.com