Jitsi Videobridge, Octopodes, and Kotlin

Jitsi Videobridge, Octopodes, and
Kotlin
JITSI.ORG | @JITSINEWS | 8X8
BORIS GROZEV

https://meet.jit.si

Jitsi Videobridge: Selective Forwarding Unit (SFU)

Server-to-server
C
B
A
D
S2
S1

END OF “WHY”
Also: big conferneces?

RTP
(Video) RTP
(Video)
XMPP/Jingle (Signaling)
Colibri

RTP
(Video) RTP
(Video)
Colibri
OCTO

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R| M | S | res | Conference ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Endpoint ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

src: https://en.wikipedia.org/wiki/Recurrent_laryngeal_nerve

• SIP Communicator (2003-)
• Softphone for PC, mac, linux
• SIP, XMPP
• Multi-party calls (c. 2009)
• Transcoding/relaying in the client
• Cross protocol
• Offload the media to a server (2011)
• Jitsi Videobridge
src: https://en.wikipedia.org/wiki/Recurrent_laryngeal_nerve

• SIP Communicator (2003-)
• Softphone for PC, mac, linux
• SIP, XMPP
• Multi-party calls (c. 2009)
• Transcoding/relaying in the client
• Cross protocol
• Offload the media to a server (2011)
• Jitsi Videobridge
• Support for WebRTC (late 2012)
• ICE & DTLS
• Simulcast
• Bandwidth estimation & adaptivity
• Data Channels (SCTP and WS)
• Octo src: https://en.wikipedia.org/wiki/Recurrent_laryngeal_nerve

src: https://rationalwiki.org/wiki/File:GiraffaRecurrEn.svg

SRTP
HandleDTLS,initiateSRTP
Decryption
Encryption
DebugTransformEngine
Optional.Savepacketstoa.pcapfile.
TransportCCEngine
Transporrt-widecongestioncontrol.
Readincomingseq.num.,
sendRTCPfeedback
Addtransport-cc
sequencenumberext
AbsSendTimeEngine
RTPAbsoluteSendTimeHeaderextension
No-op
Addtheabs-sendtime
headerextension
RemoteBitrateEstimator
ImplementGoogleCongestionControl
theoverusedetectorUpdate
onreceivedRTPpackets
No-op.
RtxTransformer
HandleretransmissionwiththeRTXformat.
APItoretransmitpackets.
RTCP:handleincomingNACK
RTP:de-RTX
No-op.
PaddingTermination
Discardspadding-onlypackets
RTP:discardpadding-only
No-op.
MediaStreamTrackReceiver
Simulcast-relatedlogic.
FindandupdatetheFrameDesc
No-op.
DiscardTransformEngine
Timestampandsequencenumberrewriting.
DiscardingpacketsmarkedwiththeDISCARD
flag.
???
???
CachingTransformer
CacheRTPpackets.
Savetoincomingcache.
Savetooutgoingcache.
RetransmissionRequester
Monitorincomingstreamsandrequest
missingRTPpacketswithRTCPNACK.
SendNACKsformissingRTPpackets.
No-op.
StatisticsEngine
UpdateMediaStreamStatswithinfogathered
fromreceivedpackets.ForRTP,count
numberofpacketsandbytes.ForRTCP,
parsethecontentsandfireevents.
ParseincomingREMB.TODO:transport-cc?
PayloadTypeTransformEngine
Re-writeRTPPayloadTypenumberstowhat
thereceiverexpects.
No-op.
RewriteRTPPT.
RTCPReceiverFeedbackTermination
DropRTCPRRand
REMB.TODO:drop
transport-cc?
DropRTCPRRand
REMB.TODO:drop
transport-cc?
MediaStream
-TransformEngineChainchain
RTPConnectorInputStreamRTPConnectorInputStream
DatagramSocket
RTPTranslator
ExternalTransformer
Anothertransformerchain,implementedin
jitsi-videobridge.
jitsi-videobridge
(RTPChannelTransformEngine)
LipSyncHack
No-op.
Insertsframes,Aaaah!
rewritesstuff...
BitrateController
Bitratecontrolandsimulcastlogic?
No-op.
TheTransportCCEngineand
DTLSTransformEngineinstancesmightbe
sharedbetweenmorethanoneMediaStream
iftheyusethesametransport(e.g.an
audioandvideostreamforthesame
endpointinajitsi-videobridge
conference.
WemayneedtomaketheRemoteBitrateEstimator
sharedbetweenMediaStreamswhichhavethesame
transport,andfeeditinformationreceivedinRTCP
transport-ccpackets,insteadofactualRTPpackets.
WedropReceiverReportsandREMBs,becausewe
performtermination--thesepacketsareinterpreted
bridge-to-endpointonly,andnotend-to-end.Wedrop
themhere,becauseonthereceivesidethisisafterwe
havehandledthem.Whensending,weinsertthemlater
inthechain(viaMediaStream#injectPacket,see
RTCPReceiverFeedbackTermination.javalines~140).
Weneedtodecidewhattodofortransport-cc.
Thisiscodewhichdeterminedwhattosendtoa
particularreceiver(whichsenders,atwhatquality).It
doesnotdobandwidthestimationandIdon'tthinkwe
willneedtomodifyitfortransport-cc.
• Hard to maintain
• Hard to improve performance

Major surgery
• Remove deprecated code
• Recording, Transcoding, Non-ICE, non-bundle, non-rtcpmux
• Replace libjitsi
• Re-design the Conference and Endpoint abstractions
• Re-design the threading model
• Keep the COLIBRI interface
• Shim layer

jitsi-videobridge
(40k LoC)
libjitsi
(230k LoC)
jitsi-videobridge
(34k LoC)
jitsi-media-transform
(12k LoC)
jitsi-rtp
(6k LoC)

Jitsi-rtp
• Generic library for RTP and RTCP
• Designed to be efficient
• 6000 lines of Kotlin
• https://github.com/jitsi/jitsi-rtp

jitsi-media-transform
• RTP processing pipeline
• Transceiver
• Node tree
• Useful Node
• 12000 lines of Kotlin
• https://github.com/jitsi/jitsi-media-transform

Demux
Parse RTP SRTP decrypt
RTP
Read audio
level
Stats Demux
Video
Parse VP8 NACK
RTC
P
Parse RTCPSRTCP decrypt
RTCP
Termination
Audio
Discard silence

OLD NEW
NEW + POOL NEW + POOL + CMS

Current status
• The libraries are released
• Builds cleanly (try it out)
• Automated tests pass
• Merges cleanly with master
• The last remaining blockers
• BWE stability
• Debian packaging

Thank you!
Any questions?
JITSI.ORG | @JITSINEWS | 8X8
BORIS GROZEV

Jitsi Videobridge, Octopodes, and Kotlin

More Related Content

What's hot (20)

Similar to Jitsi Videobridge, Octopodes, and Kotlin (20)

Recently uploaded (20)

Jitsi Videobridge, Octopodes, and Kotlin