feat(federation): video conferences in Matrix-federated rooms over MatrixRTC (POC)
Proposed changes (including videos or screenshots)
Proof of concept. LiveKit video conferences in Matrix-federated rooms, as one MatrixRTC session shared by
every homeserver in the room. Rocket.Chat ā Rocket.Chat first, on the wire formats Element Call, matrix-js-sdk,
Synapse and lk-jwt-service use, so other Matrix servers and clients can join later.
Design: docs/features/video-conference-federation/README.md.
Wire reference with sources: protocol.md.
Media: one SFU per homeserver. Each server's members publish on that server's own LiveKit and watch the other
servers' members on theirs, with subscribe-only tokens. Nothing relays media between servers. The client opens one
extra, subscribe-only LiveKit connection per other SFU in the call. It tells members from watchers by LiveKit
identity, using the roster's federation entries.
Signalling: memberships in the Matrix room. The server announces its users' call memberships and reads the
other servers'. Element Call is mid-migration, so Federation_Service_MatrixRTC_Mode picks the profile this server
announces in. All three profiles are always read:
| Profile | Membership | Transport | Identity |
|---|---|---|---|
compatibility (default) | state org.matrix.msc3401.call.member | livekit_service_url | @user:server:DEVICE |
matrix_2_0 | sticky org.matrix.msc4143.rtc.member (MSC4354) | livekit_service_url | H([user, device, member]) |
matrix_2_0_msc4195 | sticky org.matrix.msc4143.rtc.member | url | H([user, member]) |
Tokens. This server serves:
- an OpenID provider:
/_matrix/federation/v1/openid/userinfo; - an lk-jwt-service-compatible endpoint,
/matrixrtc/livekit-jwt/{get_token,sfu/get}, thelivekit_service_urlit
announces; - MSC4195's federation
get_token.
It also fetches tokens for its users from other servers' SFUs through video-conference.federatedToken.
Conference lifecycle.
- A conference mirrors the session while any membership, local or remote, counts.
- Remote members are on the roster as federated users.
- Rings and declines cross servers as
rtc.notification/rtc.decline. - The call message is not sent to Matrix, and the chat of a federated call is the room.
Depends on RocketChat/homeserver#415 (federation SDK: any-event notification, arbitrary state events, MSC4354
sticky events). Until it is published, this branch uses a vendored build of it (0.8.1-matrixrtc.2).
Commits
The history is a stack, so the parts can be extracted:
chorevendored SDK build (to be replaced by a published version), and the local two-server environment;refactor(ui-livekit)participant lookup through one function (no behaviour change; can go todevelopalone);featcontract (types, membership model, REST shapes, LiveKit derivations), then the token side, the client,
settings, wire formats, outbound/inbound signalling and conference lifecycle, ringing, sticky profiles;docsthe feature doc as built.
Issue(s)
PROD-462: Matrix Federation, video conferencing.
Steps to test or reproduce
A local two-server environment is in ee/packages/federation-matrix/docker-compose/matrixrtc/ (see its README):
rc1 and rc2 built from this branch, one LiveKit each, Traefik with TLS.
./init-env.sh && ./build-image.sh && ./up.sh && ./seed.sh # seed: alice@rc1 ā carol@rc2 federated DM
Then start a call in that DM from either side, with Federation_Service_MatrixRTC_Enabled on. Both servers need
an enterprise license with the federation and video-conference-native modules.
Further comments
Status. Unit and service tests, typecheck and lint pass:
| Suite | Tests |
|---|---|
| federation-matrix | 339 |
| ui-livekit | 89 |
| apps/meteor mocha | 3086 |
| apps/meteor client jest | 172 (conference, room actions, user card) |
It has not been exercised end to end yet: the local environment's licenses had expired. A two-server run in
the three profiles, with media checked in two browsers, is the next step.
Before leaving draft:
- replace the vendored SDK with a published version (RocketChat/homeserver#415);
- end-to-end validation;
- changeset;
- extract the refactor.
Known limitations:
- Unencrypted Matrix rooms only. Rocket.Chat users have no Matrix devices to receive Olm-encrypted call keys.
- Single instance. OpenID tokens and the per-session queues that prevent duplicate memberships and conferences
live in process memory. - Identity drift. The
matrix_2_0identity is the one deployed Element Call expects today; its next release
changes it. - Late joins with Synapse. A server joining a room mid-call sees Synapse members only when they refresh (up to an
hour), because Synapse does not yet push existing sticky events to newly joined servers. - Data-channel features are Rocket.Chat-only. Hands, reactions and mute requests are not read by Element Call.
š¤ Generated with Claude Code