JamRTC @ Wonder WebRTC unConference
- 1. JamRTC Jam sessions with Janus! Lorenzo Miniero @elminiero Wonder WebRTC unConference May 18th 2021
- 2. Who am I? Lorenzo Miniero • Ph.D @ UniNA • Chairman @ Meetecho • Main author of Janus Contacts and info • lorenzo@meetecho.com • https://twitter.com/elminiero • https://www.slideshare.net/LorenzoMiniero • https://soundcloud.com/lminiero
- 3. “Hi, I’m Troy McClure. You may remember me from...” Janus General purpose, open source WebRTC server • https://github.com/meetecho/janus-gateway • Demos and documentation: https://janus.conf.meetecho.com • Community: https://groups.google.com/forum/#!forum/meetecho-janus
- 4. A middle age crisis! https://soundcloud.com/lminiero
- 5. An amazing music ecosystem on Linux
- 6. “Can WebRTC help musicians?” https://fosdem.org/2021/schedule/event/webrtc_musicians/
- 7. Native approach for Jam sessions? • A few, non-WebRTC, native solutions exist already • e.g., Jamulus and NINJAM (both open source) • Thinking WebRTC, only apparently a traditional use case • Yes, we can see it as a “conference” of sorts... • ... but we’re not really talking, and latency is much more important • Browsers are not a good option, here • Pipeline may be good for voice, but latency too high for live music • Unfortunately, on Linux they don’t support Jack, only Pulseaudio • Hard to capture anything else than a microphone • Besides, as we said they’ll mess with the source audio anyway
- 8. Native approach for Jam sessions? • A few, non-WebRTC, native solutions exist already • e.g., Jamulus and NINJAM (both open source) • Thinking WebRTC, only apparently a traditional use case • Yes, we can see it as a “conference” of sorts... • ... but we’re not really talking, and latency is much more important • Browsers are not a good option, here • Pipeline may be good for voice, but latency too high for live music • Unfortunately, on Linux they don’t support Jack, only Pulseaudio • Hard to capture anything else than a microphone • Besides, as we said they’ll mess with the source audio anyway
- 9. Native approach for Jam sessions? • A few, non-WebRTC, native solutions exist already • e.g., Jamulus and NINJAM (both open source) • Thinking WebRTC, only apparently a traditional use case • Yes, we can see it as a “conference” of sorts... • ... but we’re not really talking, and latency is much more important • Browsers are not a good option, here • Pipeline may be good for voice, but latency too high for live music • Unfortunately, on Linux they don’t support Jack, only Pulseaudio • Hard to capture anything else than a microphone • Besides, as we said they’ll mess with the source audio anyway
- 10. JamRTC – Jam sessions with Janus! https://github.com/lminiero/jamrtc
- 11. JamRTC – Jam sessions with Janus! • Simple (and very ugly) GTK+ 3 application • Static UI done with Glade • Janus used as an SFU • Signalling implemented via libwebsockets • VideoRoom display property “abused” for correlation • GStreamer 1.x pipelines for WebRTC PeerConnections • Audio/video PeerConnections for “chat” • Audio-only PeerConnections for instruments • Focus on Jack to minimize latency, so Linux-only at the moment • GStreamer pipelines easy to manipulate, though • Any equivalent technology on Windows/MacOS?
- 12. JamRTC – Jam sessions with Janus! • Simple (and very ugly) GTK+ 3 application • Static UI done with Glade • Janus used as an SFU • Signalling implemented via libwebsockets • VideoRoom display property “abused” for correlation • GStreamer 1.x pipelines for WebRTC PeerConnections • Audio/video PeerConnections for “chat” • Audio-only PeerConnections for instruments • Focus on Jack to minimize latency, so Linux-only at the moment • GStreamer pipelines easy to manipulate, though • Any equivalent technology on Windows/MacOS?
- 13. JamRTC – Jam sessions with Janus! • Simple (and very ugly) GTK+ 3 application • Static UI done with Glade • Janus used as an SFU • Signalling implemented via libwebsockets • VideoRoom display property “abused” for correlation • GStreamer 1.x pipelines for WebRTC PeerConnections • Audio/video PeerConnections for “chat” • Audio-only PeerConnections for instruments • Focus on Jack to minimize latency, so Linux-only at the moment • GStreamer pipelines easy to manipulate, though • Any equivalent technology on Windows/MacOS?
- 14. JamRTC – Jam sessions with Janus! • Simple (and very ugly) GTK+ 3 application • Static UI done with Glade • Janus used as an SFU • Signalling implemented via libwebsockets • VideoRoom display property “abused” for correlation • GStreamer 1.x pipelines for WebRTC PeerConnections • Audio/video PeerConnections for “chat” • Audio-only PeerConnections for instruments • Focus on Jack to minimize latency, so Linux-only at the moment • GStreamer pipelines easy to manipulate, though • Any equivalent technology on Windows/MacOS?
- 15. The ugliest UI you’ll see today...
- 16. The ugliest UI you’ll see today...
- 17. Integrating with local Jack setup
- 18. Integrating with local Jack setup
- 19. Integrating with local Jack setup
- 20. Integrating with local Jack setup
- 21. What’s next?
- 22. Still far from done... • Latency still too high • Only tested by myself with two laptops connected via WiFi, though • Would love some feedback on actual sessions! • Not clear how to evaluate actual latency • We need to minimize the “local” work (little we can do about the network) • Can jack_delay or jack_iodelay help? • A few hacks to try and mitigate that • Latency in rtpbin forced to 0 (bad idea?) • Anything else? • Would it be pointless anyway, without server-side synchronization?
- 23. Still far from done... • Latency still too high • Only tested by myself with two laptops connected via WiFi, though • Would love some feedback on actual sessions! • Not clear how to evaluate actual latency • We need to minimize the “local” work (little we can do about the network) • Can jack_delay or jack_iodelay help? • A few hacks to try and mitigate that • Latency in rtpbin forced to 0 (bad idea?) • Anything else? • Would it be pointless anyway, without server-side synchronization?
- 24. Still far from done... • Latency still too high • Only tested by myself with two laptops connected via WiFi, though • Would love some feedback on actual sessions! • Not clear how to evaluate actual latency • We need to minimize the “local” work (little we can do about the network) • Can jack_delay or jack_iodelay help? • A few hacks to try and mitigate that • Latency in rtpbin forced to 0 (bad idea?) • Anything else? • Would it be pointless anyway, without server-side synchronization?
- 25. Thanks! Questions? Comments? Get in touch! • https://twitter.com/elminiero • https://twitter.com/meetecho • https://www.meetecho.com