Skip to content

Commit b0aa20b

Browse files
toger5toger5
committed
lowercase filename
Signed-off-by: Timo K <[email protected]>
1 parent 9cbe448 commit b0aa20b

File tree

1 file changed

+150
-0
lines changed

1 file changed

+150
-0
lines changed

‎proposals/4143-matrixRTC.md

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# MSC4143: MatrixRTC
2+
3+
This MSC defines the modules with which the matrix real time system is build with.
4+
5+
The MatrixRTC specification is separated into different modules.
6+
7+
- The MatrixRTC room state that defines the state of the real time application.\
8+
It is the source of truth for:
9+
- Who is part of a session
10+
- Who is connected via what technology/backend
11+
- Metadata per device used by other participants to decide whether the streams
12+
from this source are of interest / need to be subscribed.
13+
- The RTC backend.
14+
- It defines how to connect the participating peers.
15+
- Livekit is the standard for this as of writing.
16+
- Defines how to connect to a server/other peers, how to update the connection,
17+
how to subscribe to different streams...
18+
- Another planned backend is a full mesh implementation based on MSC3401.
19+
- The RTCSession types (application) have their own per application spec.
20+
- Calls can be done with an application of type `m.call` see (TODO: link call msc)
21+
- The application defines all the details of the RTC experience:
22+
- How to interpret the metadata of the member events.
23+
- What streams to connect to.
24+
- What data in which format to sent over the RTC channels.
25+
26+
This MSC will focus on the matrix room state which can be seen as the most high
27+
level signalling of a call:
28+
29+
## Proposal
30+
31+
Each RTC session is made out of a collection of `m.rtc.member` events.
32+
Each `m.rtc.member` event defines the application type: `application`
33+
and a `call_id`. And is stored in a state event of type `m.rtc.member`.
34+
The first element of the state key is the `userId` and the second the `deviceId`.
35+
(see [this proposal for state keys](https://github.com/matrix-org/matrix-spec-proposals/pull/3757#issuecomment-2099010555)
36+
for context about second/first state key.)
37+
38+
### The MatrixRTC room state
39+
40+
Everything required for working MatrixRTC
41+
(current session, sessions history, join/leave events, ...) only
42+
require one event type.
43+
44+
A complete `m.rtc.member` state event looks like this:
45+
46+
```json
47+
// event type: m.rtc.member
48+
// event key: ["@user:matrix.domain", "DEVICEID"]
49+
{
50+
"m.application": "m.my_session_type",
51+
"m.call_id": "",
52+
"focus_active": {...FOCUS_A},
53+
"foci_preferred": [
54+
{...FOCUS_1},
55+
{...FOCUS_2}
56+
]
57+
}
58+
```
59+
60+
giving us the information, that user: `@user:matrix.domain` with device `DEVICEID`
61+
is part of an RTCSession of type `m.call` in the scope/sub-session `""` (empty
62+
string as call id) connected over `FOCUS_A`. This is all information that is needed
63+
for another room member to detect the running session and join it.
64+
65+
There is **no event** to represent a session. This event would include shared
66+
information where it is not trivial to decide who has authority over it.
67+
Instead the session is a computed value based on `m.rtc.member` events.
68+
The list of events with the same `m.application` and `m.call_id` represent one session.
69+
This array allows to compute fields like participant count, start time ...
70+
71+
Sending an empty `m.rtc.member` event represents a leave action.
72+
Sending a well formatted `m.rtc.member` represents a join action.
73+
74+
Based on the value of `m.application`, the event might include additional parameters
75+
required to provide additional session parameters.
76+
77+
> A thirdRoom like experience could include the information of an approximate position
78+
on the map, so that clients can omit connecting to participants that are not in their
79+
area of interest.
80+
81+
#### Historic sessions
82+
83+
Since there is no singe entry for a historic session (because of the owner ship discussion),
84+
historic sessions need to be computed and most likely cached on the client.
85+
86+
Each state event can either mark a join or leave:
87+
88+
- join: `prev_state.m.application != current_state.m.application` &&
89+
`prev_state.m.call_id != current_state.m.call_id` &&
90+
`current_state.m.application != undefined`
91+
(where an empty `m.rtc.member` event would imply `state.m.application == undefined`)
92+
- leave: `prev_state.m.application != current_state.m.application` &&
93+
`prev_state.m.call_id != current_state.m.call_id` &&
94+
`current_state.m.application == undefined`
95+
96+
Based on this one can find user sessions. (The range between a join and a leave
97+
event) of specific times.
98+
The collection of all overlapping user sessions with the same `call_id` and
99+
`application` define one MatrixRTC history event.
100+
101+
### The RTC backend
102+
103+
`foci_active` and `foci_preferred` are used to communicate
104+
105+
- how a user is connected to the session (`foci_active`)
106+
- what connection method this user knows about would like to connect with.
107+
108+
The only enforced parameter of a `foci_preferred` or `foci_active` is `type`.
109+
Based on the focus type a different amount of parameters might be needed to,
110+
communicate how to connect to other users.
111+
`foci_preferred` and `foci_active` can have different parameters so that it is,
112+
possible to use a combination of the two to figure our that everyone is connected
113+
with each other.
114+
115+
Only users with the same type can connect in one session. If a frontend does
116+
not support the used type they cannot connect.
117+
118+
Each focus type will get its own MSC in which the detailed procedure to get from
119+
the foci information to working webRTC connections to the streams of all the
120+
participants is explained.
121+
122+
- [`livekit`](www.example.com) TODO: create `livekit` focus MSC and add link here.
123+
- [`full_mesh`](https://github.com/matrix-org/matrix-spec-proposals/pull/3401)
124+
TODO: create `full-mesh` focus MSC based on[MSC3401](https://github.com/matrix-org/matrix-spec-proposals/pull/3401)
125+
and add link here.
126+
127+
### The RTCSession types (application)
128+
129+
Each session type might have its own specification in how the different streams
130+
are interpreted and even what focus type to use. This makes this proposal extremely
131+
flexible. A Jitsi conference could be added by introducing a new `m.application`
132+
and a new focus type and would be MatrixRTC compatible. It would not be compatible
133+
with applications that do not use the Jitsi focus but clients would know that there
134+
is an ongoing session of unknown type and unknown focus and could display/represent
135+
this in the user interface.
136+
137+
To make it easy for clients to support different RTC session types, the recommended
138+
approach is to provide a matrix widget for each session type, so that client developers
139+
can use the widget as the first implementation if they want to support this RTC
140+
session type.
141+
142+
Each application should get its own MSC in which the all the additional
143+
fields are explained and how the communication with the possible foci is
144+
defined:
145+
146+
- [`m.call`](www.example.com) TODO: create `m.call` MSC and add link here.
147+
148+
## Potential issues
149+
## Alternatives
150+
## Security considerations

0 commit comments

Comments
 (0)