Everything goes over one TCP connection per person, client → room server.
Integers are little-endian. str = u16 length + UTF-8 bytes (the sender
truncates at 4000 bytes).
Field names in the tables below are the identifiers used in the code, which is written in Portuguese.
Each person can stream two things at the same time: the screen and the camera. A stream is identified by a 32-bit stream:
stream = person_id | (type << 31) type: 0 = screen, 1 = camera
The person id never uses the top bit, so the screen's stream is the id itself.
1. Handshake (in the clear)
Both sides send, at the same time, 54 bytes:
| field | size | |
|---|---|---|
"TLNH" |
4 | identifies the protocol |
| version | u16 | today 7 (6 is still accepted, see below) |
| X25519 public key | 32 | ephemeral, new for every connection |
| nonce | 16 | random |
If the versions differ, the server still sends its hello (so the client can
report "different version") and closes. A server on 7 answers a client on 6 in
version 6 (apps before 3.8), unless started with --so-protocolo-novo; an app
on 7 that gets a hello 6 back (a server before 3.8, which closes) connects
again speaking 6.
Invite and several paths
Full form: telinha://addresses[/channel][!fingerprint][#password] — all three
optional (see "Server identity" and "Channels").
The invite can carry several addresses for the same room, separated by commas,
with the password after the first #:
telinha://192.168.0.10:7777,203.0.113.7:7777,[2804:14c::1]:7777,bore.pub:41234#password
A channel can come after the addresses, separated by /:
telinha://203.0.113.7:7777/games#password joins straight into #games.
Each address is host:port; IPv6 goes in brackets (without brackets it is
ambiguous and the address is ignored). Up to 8 addresses; the order is the
preference. The host builds the invite with the local network IP, the port
opened on the router (UPnP), the machine's global IPv6 addresses, the typed
tunnel (ngrok etc.) and the public link last. Whoever joined passes on the paths
they used.
Whoever joins runs a race (connect.zig): one attempt per address, attempt i
starting i × 250 ms after the first (the first ones in the list, the direct
ones, go ahead). Each one connects, sends the hello above and waits for the
other side's (up to 8 s). The first one that answers a telinha hello with the
same version wins: the others give up, and the ones that finish later are
closed. A closed port, a silent path and another service on the port (no
"TLNH") fall behind by themselves. The password is only checked afterwards, on
the connection that won (next section). If nobody wins (all failed or 20 s went
by), the error shown follows this order: different version → "not a telinha
room" → connection refused → address not found (only with one address) → did not
connect.
Key derivation
secret = X25519(my_private, their_public)
password = SHA-256("telinha-senha-v1" || room_password)
prk = HKDF-Extract(salt = client_nonce || server_nonce, ikm = secret || password)
k_c2s = HKDF-Expand(prk, "telinha c2s", 32)
k_s2c = HKDF-Expand(prk, "telinha s2c", 32)
Wrong password → different keys → the client's first message doesn't decrypt.
The server then sends u32 0xFFFFFFFF in the clear (= "wrong password") and
closes. Until the encrypted hello arrives, every server read has a 10 s
deadline.
Server identity
The server has a fixed Ed25519 key (a 32-byte seed in a file: --chave).
Version 7: right after the handshake, before reading anything, it sends in
the clear u32 0xFFFFFFFE, u8 has_identity and, if 1, its public key (32)
and a signature (64) over
transcript = SHA-256("telinha-identidade-v2" || u16 version || client_public || client_nonce || server_public || server_nonce)
Nothing encrypted with the password-derived key leaves the server before the
client's hello decrypts. (In version 6 the identity went encrypted, as the
identity message below: anyone who connected once got a ciphertext to test
passwords against offline.) Repeated wrong passwords from one IP wait 2, 4, 8 …
up to 60 s before the next try.
Version 6: the same, as the first encrypted message, identity (31), with
the transcript SHA-256("telinha-identidade-v1" || client_public || client_nonce
|| server_public || server_nonce).
Someone in the middle has a different handshake with each side, so the real
server's signature is useless to them. The fingerprint is base32 (lowercase,
a-z2-7) of the first 15 bytes of SHA-256(public key): 24 characters, carried in
the invite after !. The app checks it before sending hello (so name and
admin token never reach an impostor): with a fingerprint from the invite it must
match; without one, the first fingerprint seen at that address is remembered
(servidores.txt, like SSH) and required afterwards — if it changes, the app
refuses and offers "Trust the new identity". A server without a key sends no
identity (an old app just ignores the message). With the wrong password the
identity doesn't decrypt, which the app reports as a wrong password.
2. Messages (encrypted)
u32 size | ChaCha20-Poly1305( u8 type | payload ) | tag (16 bytes)
size counts the ciphertext + tag. 12-byte nonce = the direction's u64 counter
+ 4 zeros (starts at 0, never repeats on a connection). Maximum 32 MiB.
| type | name | direction | payload |
|---|---|---|---|
| 1 | hello | C→S | str nome, str token_admin, u64 chave_do_cliente, str versão_do_app, u8 codecs (what it decodes, bits 1<<codec), u8 recursos (feat_*), str canal (channel to join; empty or missing = the default), str token_bot (bots only; see feat_bot), str senha_do_canal (if the channel has a password; the app sends the last one it used there, to get back to the channel when it reconnects) |
| 2 | welcome | S→C | u32 meu_id, str nome_da_sala, u8 sou_admin, u64 relógio_µs, u64 instância, u8 rota, str versão_do_app (the host's), u8 udp (1 = offers UDP), u32 token_udp (if 1), u8 recursos_do_servidor (srv_*: 1 = private messages and quoted replies, 2 = files, 4 = cards, 8 = deleting channels, 16 = channel OPs, 32 = bot panel, 64 = collects logs; an older server doesn't send it = 0) |
| 3 | denied | S→C | str motivo, u8 código (the server closes right after) |
| 4 | members | S→C | u8 sala (1 = locked), u16 n, n × member (below), n × str versão_do_app (same order), n × u8 tipo (1 = bot, 2 = OP of my text channel, 4 = OP of my call's channel, 8 = created my text channel, 16 = created my call's channel; an older server doesn't send it) |
| 5 | chat | C→S | str texto, image (optional, below), u32 resposta (id of the message replied to; 0 = none), u32 pra (private to this member; 0 = the channel), u32 n + n bytes of a card (0 = none) |
| 5 | chat | S→C | u32 id, u32 de (0 = system), str nome, str texto, u64 hora (room clock, µs), u8 flags (1 = history), image, u8 n, n × (u8 reação, u32 de), u32 resposta (if ≠ 0: u32 de, str nome, str trecho), u32 pra (if ≠ 0: str nome), u32 arquivo (if ≠ 0: str nome, u64 tamanho), u32 n + n bytes of a card; to bots only, on live messages: str canal, str call of whoever sent it (where they are now; call empty = not in one) |
| 6 | stream_start | C→S | u8 tipo, u16 w, u16 h, u8 fps, u8 flags (1 = with audio; screen only) |
| 7 | stream_stop | C→S | u8 tipo |
| 8 | frame | C→S | frame (below) |
| 8 | frame | S→C | u32 stream, frame |
| 9 | watch | C→S | u32 stream, u8 ligado, u8 opções (1 = clips: wants an IDR now and then) |
| 10 | keyreq | S→C | u8 tipo (send an IDR for that stream) |
| 10 | keyreq | C→S | u32 stream, u8 motivo (0 = I lost the reference, 1 = recording starting, 2 = my decoder is behind but caught up by dropping droppable frames: no IDR, just thin me, 3 = I threw the decoder queue away because it couldn't keep up: like 0, and with no droppable frames the server tells the streamer (viewer_slow byte 3); an older server treats 2 and 3 as 0) |
| 11 | ping | C→S | u64 t |
| 12 | pong | S→C | u64 t (echo), u64 relógio_µs |
| 13 | kick | C→S | u32 alvo (the same as admin with action 1; still accepted) |
| 14 | audio | C→S | audio (below) |
| 14 | audio | S→C | u32 stream (always the screen's), audio |
| 15 | ack | S→C | u8 tipo, u32 seq (the server received that frame of yours) |
| 15 | ack | C→S | u32 stream, u32 seq (I received that frame of stream) |
| 16 | viewer_slow | S→C | u8 tipo, u8 piso (the viewer can't keep up: lower the rate, down to piso% of the target) Optional third byte: 1 = someone can't decode a stream that has no droppable frames in time (comes with floor 100, so an older app ignores it); a streamer with automatic fps drops to 30 for 5 minutes. |
| 17 | voice | C→S | u32 seq, Opus packet (voice: 48 kHz mono, 20 ms) |
| 17 | voice | S→C | u32 de, u32 seq, Opus packet |
| 18 | voice_state | C→S | u8 flags (1 = microphone on, 2 = deafened) |
| 19 | admin | C→S | u8 ação, u32 alvo, u8 valor (admin only, see §3) |
| 20 | notice | S→C | u8 tipo, str texto, u8 extra: something an admin did to you |
| 21 | pointer | C→S | u32 stream, u16 x, u16 y, u8 flags |
| 21 | pointer | S→C | u32 de, u32 stream, u16 x, u16 y, u8 flags |
| 22 | react | C→S | u32 id_da_mensagem, u8 reação (0–5), u8 ligado |
| 22 | react | S→C | u32 id_da_mensagem, u32 de, u8 reação, u8 ligado |
| 23 | codec | S→C | u8 tipo (screen/camera), u8 codecs: what everyone watching that stream decodes; u8 recursos: what everyone watching understands |
| 24 | draw | C→S / S→C | drawing on top of the stream (see proto.zig) |
| 25 | cursor | C→S | u8 tipo, u16 x, u16 y (0xFFFF = hidden): the streamer's cursor, outside the video |
| 25 | cursor | S→C | u32 stream, u16 x, u16 y |
| 26 | view_size | C→S | u32 stream, u16 altura at which that stream appears (0xFFFF = wants everything: fullscreen, recording) |
| 26 | view_size | S→C | u8 tipo, u16 altura: the largest among those watching (0xFFFF = no limit) |
| 27 | log_chunk | C→S | u32 id, u8 what (0 = debug log, 1 = crash report), u32 total, u32 offset, bytes: a debug log for the room owner, in pieces of up to 64 KB (8 MB max, one per minute) |
| 27 | log_chunk | S→C | u32 from, str name + the same fields: relayed to everyone who joined with the admin token |
| 28 | log_ack | S→C | u32 id, u8 result (0 = delivered, 1 = only saved on the server, 2 = nobody to receive it, 3 = refused), u8 how many owners got it |
| 29 | channel | C→S | str canal: switch to that channel (created if it doesn't exist and the server allows it) |
| 29 | channel | S→C | str atual (the channel you are in), u8 n, n × (str nome, u16 pessoas, u8 flags: 1 = someone live, 2 = fixed) |
| 30 | stats | C→S | u8 ligado: start/stop receiving server statistics (only whoever joined with the admin token) |
| 30 | stats | S→C | server statistics, every 2 s while on (below) |
| 31 | identity | S→C | [32] public key, [64] signature of this handshake (see "Server identity"): the first message, before hello |
| 32 | chan_admin | C→S | u8 action, str channel, str value, u32 target (admins; table below) |
| 33 | call | C→S | str channel (empty = leave the call), str password: join that channel's call |
| 34 | file_put | C→S | u32 request, str name, u64 size, u32 to (private; 0 = the channel), u32 reply: I want to send a file |
| 35 | file_ready | S→C | u32 request, u32 file (0 = refused), str reason |
| 36 | file_data | C→S / S→C | u32 file, u64 position, bytes (up to 64 KB): uploading / downloading |
| 37 | file_ack | S→C | u32 file, u64 received (how far it got) |
| 38 | file_get | C→S | u32 file, u64 position (multiple of 64), u32 size (up to 64 KB) |
| 39 | file_cancel | C→S | u32 file: give up uploading |
| 40 | web_hello | C→S | browser only: str password + the same fields as hello |
| 41 | web_link | S→C | str address: the https link to watch in a browser (empty = none) |
| 42 | chat_edit | C→S / S→C | u32 id, str texto, u32 n + n bytes of a card (0 = none): the author changes their own message (the image stays) |
| 43 | embed_action | C→S | u32 id, u8 botão: someone clicked a button on a card |
| 43 | embed_action | S→C | u32 id, u8 botão, u32 de, str nome and, if the card's author is a bot, str canal, str call of whoever clicked; goes only to the card's author |
| 44 | bans | S→C | u16 n, n × (u32 id, str nome, u32 há_quantos_s): who is banned; only to admins, after welcome (if anyone is) and whenever it changes (empty to someone who lost admin). admin action 9 with alvo = id unbans (0 = everyone) |
| 45 | bot_panel | C→S | bots only: u32 n, card (n = 0 removes the panel); see "Bot panel" |
| 45 | bot_panel | S→C | u32 bot id, u32 n, card (n = 0: no panel anymore) |
| 46 | panel_action | C→S | u32 bot id, u8 botão: click on a panel button; reaches the bot as an embed_action with id 0 |
| 47 | log_request | C→S | u32 id, u8 what: 0 = whoever has the crown asks for id's log; 1 = I refuse id's request; see "Log collection" |
| 47 | log_request | S→C | u32 who asked, str name: send your log (or refuse) |
versão_do_app ("3.0.0") only serves to warn whoever is out of date (the
interface shows "so-and-so is on telinha X: there is a new version"); empty = not
informed. It is also a field at the end, read with an empty default.
Some fields at the end are read with a default value (0, or "screen") if
missing: chave_do_cliente, flags of stream_start, tipo of stream_stop,
opções of watch, motivo of keyreq and valor of admin on the server;
código of denied, tipo of keyreq and extra of notice on the client. It's
only tolerance: a different version is already refused in the handshake.
hello: chave_do_cliente is a random number drawn once and kept in the app's
settings (never 0); it is used for banning. The right token_admin = room owner.
welcome: sou_admin = 1 for the owner. instância is drawn when the server
starts (if it changed, the server restarted and the chat ids started over).
rota = how you got in:
| rota | |
|---|---|
| 0 | unknown |
| 1 | on this machine (loopback, or connected to its own IP) |
| 2 | local network: 10/8, 172.16/12, 192.168/16, 169.254/16, 100.64/10 (CGNAT, Tailscale), fe80::/10, fc00::/7 |
| 3 | internet, IPv4 (port opened on the router, dedicated server) |
| 4 | internet, IPv6 |
| 5 | public link (connection delivered by the embedded tunnel) |
| 6 | watching in a browser (web_hello) |
denied: código 0 = generic, 1 = kicked, 2 = banned, 3 = the owner closed
the room, 4 = the public link you came in through was turned off, 5 = room
locked, 6 = room full, 7 = the room doesn't accept bots (or the bot token doesn't
match). With any of them the client doesn't try to reconnect.
Image in the chat: u8 tipo — 0 = none; 1 = u16 w, u16 h, u32 tamanho,
bytes; 2 = it was an image, but the server no longer keeps it (history). The
bytes are one H.264 frame (IDR): the sender reads the file/clipboard on its
own PC, shrinks it to at most 1920 px and encodes it with openh264; the receiver
decodes it with the same video decoder and checks that the size matches. That way
no PNG/JPEG reader is exposed to what comes from the network. Limits: side
between 16 and 1920 px, up to 1.5 MB, one image every 1.5 s per person (a burst
is ignored); outside that the message is invalid and the server drops the
connection. History keeps up to 24 MB of images; the oldest become type 2.
Reactions: 0 like, 1 love, 2 haha, 3 fire, 4 wow, 5 sad. One per person and per type on each message (turning it on again changes nothing); only on messages that are still in the history, and not on system notices. In the history, the chat message already comes with the reactions; after that they arrive one by one.
Member: u32 id, str nome, u8 flags, u8 rota (bit 0x80 = owner; the
low 7 = route, table above), u16 w, u16 h, u8 fps (screen), u16 w,
u16 h, u8 fps (camera), u16 k, k × u32 (streams that person watches, at
most 64). A stream that is off goes with size and fps 0. Flags: 1 = streaming
the screen, 2 = admin, 4 = screen with audio, 8 = microphone on, 16 = deafened,
32 = muted by an admin, 64 = camera on, 128 = the admin took their pen. In
rota, besides 0x80 (owner) and the route in the low 5 bits: 0x40 = in my
text channel, 0x20 = in my call (each person gets their own list: everyone in
their chat plus everyone in their call, and the bots of every channel with
neither bit, so a bot can be called from afar; old apps ignore anyone in
neither).
Frame (21-byte header):
| field | size | |
|---|---|---|
| seq | u32 | per stream; goes back to 0 when it restarts |
| flags | u8 | 1 = IDR, 2 = skippable, bits 2–3 = codec (0 H.264, 1 HEVC, 2 AV1), 0x10 = cursor outside the video, 0x80 = camera |
| hora_da_captura | u64 | room clock, µs |
| w, h | u16, u16 | |
| cursor_x, cursor_y | u16, u16 | the streamer's cursor tip, relative to the frame (0..65534); 0xFFFF = no cursor |
and the bitstream until the end: Annex B for H.264 and HEVC, AV1 OBUs in the
"low overhead" format (each with its size, a temporal unit per frame, starting
with the temporal delimiter) — see flags. Every IDR carries SPS/PPS (and VPS for
HEVC as well); every AV1 key frame carries the sequence header. The server figures out the stream
from the camera bit. "Skippable" = nobody uses that frame as reference (all
slices with nal_ref_idc = 0; the streamer encodes with two temporal layers, one
frame yes, one no): the server may stop sending it to whoever can't keep up
without breaking the decoding of the following ones. Whoever doesn't know the bit
ignores it. A frame resent with the screen still carries the resend time (it is
the current state of the screen).
Codec: everyone decodes H.264 (openh264 in software, if needed); HEVC and
AV1 only those with a GPU decoder (today NVDEC) or a browser that has them
(WebCodecs). In hello each one says what it decodes; the server sends the
streamer, on every change in who is watching, the intersection (codec), and
the streamer picks the best its encoder can do — AV1 (experimental, opt-in,
NVENC on RTX 40 and newer), then HEVC, otherwise H.264. Someone with only H.264 joined: the stream goes back
to H.264 right away (new encoder, starting on an IDR); those already watching
switch decoders when they see the codec change in the flags.
Features (hello and codec, one byte at the end; an old app or server
doesn't send it and it counts as 0, everything as before):
feat_cursor(1): viewers draw the arrow. If everyone watching understands it, the streamer stops painting the cursor into the video, marks the frames with the0x10flag and sends the position in thecursormessage (up to ~120 per second; the server cuts above 130). Moving the mouse over a still screen generates no frame. Someone who doesn't understand joined: the cursor goes back into the video right away. Recording your own screen too (the recording carries the cursor).feat_bot(16): whoever connected is a bot (the SDK insdk/python). It's marked in the list (members, the kind byte at the end) and the app shows a BOT tag; it never becomes admin (the server ignores itstoken_adminand refuses to promote it); the server can refuse them all (--sem-bots) or require thetoken_bot(--token-bots), answeringdenied7. It's an honest label, not a lock: whoever doesn't label itself joins as a person, with the same powers.feat_embed(32): draws cards and their buttons and understandschat_edit. Informational: the card rides at the end ofchat, which older apps don't read (they show the text), and they ignorechat_edit.feat_dm(8): understands private messages. The server refuses a private message to someone who doesn't (telling the sender withnotice10): an older app would show it as if it were for everyone.feat_view_size(2): viewers say at what height they show each stream (view_size, in steps: 360, 480, 540, 720, 900, 1080, 1440, 2160; it goes up instantly, and only down after 3 s smaller). The server sends the streamer the largest among those watching (whoever doesn't understand counts as "everything"; whoever started watching but hasn't said a size yet doesn't count), and it uses that as the maximum height — together with the chosen one and automatic quality, the smallest wins. The streamer goes up immediately and only goes down after 10 s of a smaller request (the first size after nobody was watching applies immediately).
Audio: u32 seq, u64 hora_da_captura, Opus packet until the end (48 kHz,
stereo, 10 ms per packet, ~96 kbit/s). Only the screen has audio.
Pointer ("look here"): x, y relative to the frame (0..65534), 0xFFFF =
hides it; flags 1 = visible, 2 = click (ring calling attention).
Draw flags: 1 = new stroke, 2 = clear my strokes on that stream, bits 2–3 =
width (0 normal, 1 thin, 2 thick, 3 highlighter), bits 4–7 = palette color (0 =
the person's own color; proto.draw_colors). An old app ignores the style bits.
admin — actions:
| action | value | |
|---|---|---|
| 1 | kick | — |
| 2 | ban (kicks and doesn't let back in while the room exists) | — |
| 3 | mute the microphone | 1 = mutes, 0 = releases |
| 4 | stop a stream | type (0 = screen, 1 = camera) |
| 5 | give/remove admin (owner only) | 1 = gives, 0 = removes |
| 6 | lock the room (target ignored) | 1 = locks, 0 = unlocks |
| 7 | pen | 1 = gives it back, 0 = takes it away (their drawings disappear; remembered by key) |
| 8 | clear the room's chat: every channel's history (private messages too) and the room's files (target ignored) | — |
chan_admin — actions: 1 = pin (value "1"/"0": a pinned channel stays empty
forever and across restarts), 2 = lock (only admins and OPs get in), 3 =
password (value; empty removes it), 4 = topic (value, up to 120 bytes), 5 =
move target to the channel (chat and call), 6 = invisible (value "1"/"0":
left out of the list of whoever isn't in it; you join by knowing the name), 7 =
remove target from the channel and don't let them back (by key; an OP loses
the OP), 8 = delete the channel (whoever was in its chat goes to the default
one, and whoever was in its call leaves it; never the default; a pinned one,
admins only), 9 = OP (value "1" gives it to target, who must be in the
channel's chat or call; "0" removes it).
Channel OPs. Whoever creates a channel is its OP, and an OP can make
anyone in the channel an OP (up to 16 per channel besides the creator; the
server remembers app keys and saves them in the state file). Room admins can
do everything, in any channel. An OP: password, lock, invisible, topic,
removing people, giving OP and moving people. For an OP, moving only works on
people in their channel (the chat, the call, or both, depending on where they
are an OP) and to a channel that already exists and that the person could
join on their own (no password or lock), or where they're also an OP; an
admin moves anyone anywhere, even into a locked channel (and creates it if it
doesn't exist). An OP doesn't move or remove admins or the creator; another
OP, only the creator does. Only the creator (and admins) delete the channel
and take OP away from others; the creator never loses OP. The default channel
has no OPs. The password applies to everyone but room admins (OPs and the
creator too, and another window of the same app); the lock lets admins and OPs
in; an invisible channel shows up for whoever is in it, its OPs and admins.
The channel list marks for each person chan_mine = 16 (I'm an OP),
chan_creator = 64 (I created it) and chan_hidden = 32 (invisible). A
channel someone created stays in the list when it empties (and in the state
file across restarts) for 3 days without anyone; a pinned one, forever.
notice — types: 1 = microphone muted, 2 = microphone released, 3 = stream
stopped (extra = its type), 4 = became admin, 5 = is no longer admin, 7/8 = pen
taken away/given back, 9 = an admin moved you to another channel, 10 = the
private message didn't go (the person left or their app can't receive it), 11 =
the requested file isn't in the room anymore, 12 = an admin cleared the room's
chat (the app erases what it shows; an older app just shows the text), 13 =
the person you asked for a log refused (the text is just their name). The
texto already comes ready ("Ana muted your microphone."). 6 = couldn't switch
channel (invalid name, switching too fast, the server doesn't allow new
channels or has too many).
stats (S→C): u32 uptime_s, u32 cpu (the server process, in thousandths
of one core), u8 cores, u32 rss_kb, u32 out_Bps, u32 in_Bps (relayed and
received bytes per second since the previous one), u16 people, u16 on_udp,
u16 max_people, u8 channels, then counters since the previous one — u16
resyncs (a viewer fell behind and restarted from an IDR), u16 thinned
(started getting half the frames), u16 keyreqs, u16 slow_hints — u32
window_ms, u8 flags (1 = the room runs inside the app, 2 = locked, 4 = new
channels allowed), u16 n and n × (u32 id, str name, str channel, u8
route, u8 flags (1 = lagging, 2 = thinned, 4 = streaming the screen, 8 =
camera, 16 = owner), u8 udp, u16 udp_rtt_ms, u32 to_them_Bps, u32
from_them_Bps, u32 queued_bytes, u16 watching). Up to 64 people. The app
turns these into the "Server" window: CPU, memory, bandwidth, who is lagging and
what the bottleneck probably is.
2b. Video, sound and voice over UDP
If the server has the UDP port (the same as TCP) and the connection didn't come
through the public link, welcome offers UDP with a token. The client sends
hello over UDP to the same address and port (every 250 ms, then every 3 s)
until it receives hello_ok; from then on frames, audio and voice go over
UDP in both directions — everything else (and whatever doesn't fit) stays on
TCP. With no reply, or 5–6 s with nothing arriving, it goes back to TCP by
itself; the hello keeps trying.
Datagram:
| field | size | |
|---|---|---|
| mark | u8 | 0xA7 |
| token | u32 | which connection it is (the server finds the connection by it) |
| number | u64 | per direction, never repeats; it is the nonce |
| ciphertext | rest | ChaCha20-Poly1305 of u8 type + body; the header goes in as associated data |
Key for each direction = SHA-256("telinha udp v1" ‖ TCP key in that direction). A packet that doesn't decrypt, repeats, or is more than 128 numbers behind the highest already seen is ignored. The server starts replying to the address the last decrypted packet came from (NAT that changes port).
| type | body |
|---|---|
| 1 hello / 2 hello_ok | — |
| 3 ping / 4 pong | u64 t (echo): keeps the NAT open, measures the RTT |
| 5 data | u32 canal, u32 id, u16 pedaço, u16 total, bytes (up to 1150) |
| 6 nack | u32 canal, u32 id, u16 n, n × u16 pedaço (n = 0xFFFF = the whole message) |
A message (u8 type + payload, same as TCP) goes in 1150-byte pieces.
Channel: frames go in order per stream (channel = stream; from client to
server, 0x80000000 screen / 0x80000001 camera), with ids 1, 2, 3... per
channel; sound and voice use channel 0xFFFFFFFF, without order (each has its
own sequence and buffer). The receiver reassembles; a missing piece (out of order
for more than ~RTT/4) becomes a NACK, repeated every ~1.25 × RTT; a message of
which nothing arrived is discovered by the next one (a hole in the ids) and
requested whole. The sender keeps what it sent for 1.5 s to resend. It gives up
on a message when the pieces stop arriving for 4 × RTT (150–400 ms) or after
1 s: the following frames are delivered and the decoder asks for an IDR
(keyreq) if needed. Sending is paced (a burst of up to 128 KB, then at most 4×
the average rate), so a big IDR doesn't overflow the router's buffer.
3. Server rules
Entry
- Name without leading/trailing spaces, up to 32 bytes; empty becomes
anônimo. Repeated names get a suffix:Ana (2). - Whoever isn't the owner is refused (
denied) if banned (2), if the room is locked (5) or full (6; 32 people by default). The owner goes over all of that. A ban holds by the app's key until an admin unbans it, and by IP for 24 h (never by name: that caught anyone else with the same name, forever); the server keeps the newest 256. - Right after
welcome, whoever joins receives the last 50 chat messages (with the history flag: no sound or notification); then everyone receives "So-and-so joined the room" and the newmembers.
Streams
- Frames only go to whoever sent
watchfor the stream. You can only watch someone who exists and not yourself, up to 2 × the people limit in streams.watchturned on again for the same stream only changes the options. - Whoever starts watching (or when the streamer changes resolution with a new
stream_start) only receives from an IDR on; the server sendskeyreqto the streamer (at most one every 500 ms per stream, except for whoever just started watching). - A frame of a stream that isn't on (the admin stopped it and old frames are still arriving) is discarded.
keyreqwith reason "recording" only asks for the IDR (respecting the 500 ms), without touching that viewer's delay control.- Clips: viewers with the clips option keep the last 30 s, and a clip can only start on an IDR. Every 250 ms the server checks: if that stream's last IDR is older than 20 s and none was requested in that time, it asks for one.
ack,viewer_slow,voiceandpointerjump the send queue (they don't wait behind frames), keeping their order relative to each other.- Voice goes to everyone in the room, except the speaker and those deafened; the voice of whoever was muted by an admin is thrown away. It is only discarded if the recipient's queue passes 6 MiB.
- Audio goes to whoever watches the streamer's screen, without waiting for an IDR; it is only discarded if the viewer's queue passes 6 MiB.
- Pointer: only accepted from whoever streams that stream or watches it, at most 40 per second per person (the rest is thrown away); it goes to the stream's owner and to everyone watching, except the sender.
Chat
Text without leading/trailing spaces; empty is ignored. Every message — including
the system ones (de = 0: joined, left, started streaming, turned the camera on,
kicked, locked...) — gets a growing id (starts at 1 for each server instance)
and the time on the room clock, and enters a ring with the last 50. The client
ignores a history message with an id it has already seen and converts the time
to the local clock.
Reply: the quoted snippet (up to 140 bytes, no line breaks) comes from the server, from the replier's channel history — nobody can forge a quote. A message too old (out of the history) goes without a quote.
Private (pra ≠ 0): goes only to the sender and to pra, wherever they are
(any channel); it isn't stored in the channel history and has no reactions. The
server keeps the last 64 (without images) only to check quotes in a reply inside
the private conversation; in public, quoting a private message carries no snippet.
Cards
A card (embed) rides at the end of chat and chat_edit, up to 8 KB, and
the server stores it as it came (it only checks the size and the version
byte; whoever draws validates the rest). Little-endian, str = u16 +
UTF-8:
u8 version (1)
u32 color 0xRRGGBB (0 = the theme accent)
u8 flags: 1 = the image is the cover, 2 = has progress, 4 = large image,
8 = the panel of the author's stream (the app docks it under the
stream when it's alone or featured on the stage)
str kicker, str title, str url (http/https), str description (Markdown), str footer
[flags & 2] u32 position_ms, u32 total_ms (0 = live), u8 paused
u8 n_fields (≤ 12) × (str name, str value, u8 inline)
u8 n_buttons (≤ 5) × (u8 style: 0 neutral, 1 primary, 2 danger, 3 success,
u8 icon (0–22), str label, u8 flags: 1 = disabled)
The app moves the progress bar by itself from the moment the message arrives
(or was edited) until paused. Icons: 0 none, 1 play, 2 pause, 3 next,
4 previous, 5 stop, 6 shuffle, 7 repeat, 8 volume up, 9 volume down, 10 list,
11 heart, 12 music, 13 radio, 14 search, 15 plus, 16 check, 17 close, 18 link,
19 join, 20 trash, 21 star, 22 dice.
Editing (chat_edit): only the author, on a message still in the history
of the channel they're in (or a private one of theirs), at most once every
200 ms (faster is ignored). It goes to whoever is in that channel (or to the
two in the private conversation).
Clicks (embed_action): on a card visible to whoever clicked, not their
own, at most once every 120 ms per person. Only the author gets it; nobody
else sees the click. A bot learns where whoever clicked is, and where whoever
wrote to it is (in chat, also in private): that's how /msg Bot !join
works from any channel.
Bot panel (bot_panel, srv_* 32): a fixed card of the bot, outside the
chat, shown to everyone in the room in any channel (in the app: clicking the
bot in the list opens its controls; the title shows under its name). The bot
sends bot_panel with the card (same format as the end of chat; n = 0
removes it), on average once every 200 ms with bursts of up to 4 (beyond that
is ignored; the SDK coalesces).
The server keeps the latest and relays it to whoever understands cards (not to
bots or the browser): right away, and to whoever joins later, right after the
member list. When the bot leaves, everyone gets an empty panel. Clicks:
panel_action (u32 bot id, u8 botão), with the same limit as
embed_action; it reaches the bot as an embed_action for message 0, with
who clicked and where they are.
Log collection (log_request, srv_* 64): only on a server with
--coletar-logs. The app says so on join and records the debug log while in the
room (whoever turned off "Send my log when the room asks" is only told). Whoever
has the crown sends log_request (0, id); the person gets log_request (who
asked, name) and sends the log the usual way (log_chunk, to whoever has the
crown and to the server's --logs folder) or answers log_request (1, who
asked), and the asker gets notice 13. Bots and the browser never get asked. An
app in such a room also sends its last crash report on its own.
Browser
The dedicated server with --web serves a page (on 127.0.0.1, behind HTTPS:
browsers only decode video on a secure page) and a WebSocket at /ws. Each
message travels in a binary frame, the same as what goes encrypted over TCP
(u8 tipo + payload), without encrypting again: HTTPS does that. The first one
is web_hello: the password is checked directly (constant-time comparison) and
the rest is a normal hello — except nobody becomes admin from a browser, and
there's no UDP.
Whoever joins this way is a regular member with route 6: they get the same
already-encoded frames (nothing is re-encoded; the streamer spends nothing
extra), confirm each one with ack (delay control works the same), report the
size they show it at (view_size) and draw the cursor (feat_cursor: the
streamer stays on the direct GPU path). From a browser the server only accepts
watch, keyreq, ack, ping, view_size, chat, react, channel and
call — it watches and chats, it doesn't stream. The viewer's IP comes from the
proxy header (CF-Connecting-IP, X-Forwarded-For), for bans.
web_link is sent to people joining from the app when the server knows the
public address (the Cloudflare quick tunnel it starts itself, or a fixed one); it
changes when the tunnel drops and comes back. The app builds the link with the
password and the channel after the # (that part never leaves the browser in
requests).
Files
The sender uploads once to the server; everyone downloads when they click, at
their own pace. file_put reserves the slot (up to 256 MB per file, 3
uploading per person); file_data goes up in order, in 32 KB pieces, each one
getting a file_ack. Once complete it becomes a chat message in the channel (or
a private one) carrying the arquivo; its text is "📎 name (size)", for older
apps. Downloaders ask for pieces with file_get; only people in the channel the
file was posted to (or the two in a private conversation) can download. The id
is random.
Both directions run in the background (bgflow.zig, like LEDBAT): the window
of unconfirmed bytes grows while the confirmation delay stays near the lowest
seen and halves when it rises more than 40 ms above it (a queue building up,
almost always someone's video competing for the same connection). So the file
uses the spare bandwidth and doesn't disturb streams, voice or chat.
On the server, files live on disk (--arquivos, see DEPLOY), encrypted with
ChaCha20 and a key that only exists in its memory; they expire 2 h after
completing, 2 GB at most in total (the oldest goes first), an upload stalled for
2 min is dropped, and someone leaving mid-upload takes the file with them.
Channels
- A server has one or more channels (like IRC). Each person is in one text
channel (its chat, history of 50) and in at most one call (voice and
streams) — not necessarily the same channel (
feat_calls). You can onlywatch(and point/draw at) someone in the same call. Bans, admin mutes, the room lock and the people limit apply to the whole server. - Joining the room puts you in the call of the channel you joined.
channelchanges only the chat;callchanges the call (your streams follow: the app sendsstream_startagain). An app withoutfeat_callsmoves both together. - A channel can have a password (
channel/callcarry it; without it the server answersnotice6 withextra= 1 and the app asks), be locked (only admins) and have a topic. A pinned channel — with its password, lock and topic — plus bans, admin mutes and pen removals are kept in the state file (--estado) and come back when the server restarts. channel(S→C), after the list: n ×u16in the call, n ×strtopic,strmy current call (empty = none); flags 4 = password, 8 = locked.- The first channel is the default (
geral, or the first of--canais). Joining (hello) or switching (channel) to a name that doesn't exist creates it, unless the server was started with--sem-canais-novos; at most 32 channels. A created channel disappears (with its chat) when it empties; fixed ones stay. - Names: no
#, lowercase, spaces become-, only letters (accented ones too), digits,-,_and., up to 24 bytes. - Switching: whoever was watching you in the old channel stops receiving; you stop
watching whoever stayed there. The server sends
channel(with the new current one) and then the new channel's history, and posts "So-and-so went to #x" / "So-and-so joined #x" in each chat. Your streams follow you: the app sendsstream_startagain in the new channel. At most one switch every 800 ms. channel(S→C) also goes to everyone whenever the list changes (people, someone going live, a channel created or closed). An old app ignores it and stays in the default channel; a new app on an old server never gets it and shows no channels.
Admin
- Only admins; a request from someone who isn't is silently ignored.
- Lock/unlock: any admin. It goes to everyone's chat and
memberscomes out with the room state. - The rest needs a target that exists, that isn't you or the owner. A promoted admin (not the owner) can't touch another admin. Giving/removing admin is the owner's alone.
- Kick/ban: everyone's chat finds out; the target receives
denied(1 or 2, "So-and-so kicked you out of the room.") and leaves. The ban keeps thechave_do_cliente, the name without the(n)suffix (compared case-insensitively) and the IP — the latter doesn't apply to whoever came through the public link (everyone there has the tunnel's IP) nor to whoever is on the same machine. Hitting any of the three already blocks entry. - Mute, stop stream and give/remove admin send a
noticeto the target and a newmembersto the room; stopping a stream works as if the target had sentstream_stop.
Leaving
- Whoever leaves: everyone stops receiving their two streams, receives
"So-and-so left the room" and the new
members. - The server never drops someone without the reason arriving: it sends
denied, waits for their queue to empty (up to 600 ms), closes only the sending side of the socket and waits for the client to close (up to 600 ms) before dropping it for good. - Room closing: everyone receives
denied3 ("The owner closed the room."). - Public link turned off: whoever came in through it receives
denied4 and leaves; the rest carries on.
4. End-to-end delay control
Tunnels (ngrok, bore...) have huge buffers: the sender's TCP never feels the congestion, and looking only at your own queue doesn't help. That is why everything is measured with acknowledgements — separately for each stream (someone's screen and camera have independent controls):
- The server sends
ack tipo seqto the streamer as soon as it receives each frame. The streamer keeps the time it queued each frame and computesaccumulated delay = age of the oldest unacknowledged frame − reference RTT(the reference RTT is the lowest ack delay in 10 s windows, not counting IDRs — their delay includes the transfer — and capped at 400 ms). - Each viewer sends
ack stream seqas soon as it receives a frame (before decoding). The server does the same math per viewer and per stream.
Streamer (ratectl.zig), before each frame:
- delay > 100 ms: rate −7% (at most every 500 ms), without skipping a frame;
- delay > 250 ms: skip the frame; if it persists for 300 ms (or comes back in
less than 1 s), rate −25%. Delay explained by a recent IDR (slack = the time it
took, decreasing over 4 s) only causes a frame skip, doesn't lower the rate;
- delay < 60 ms and 2 s without trouble: climb 12% up to 90% of the "ceiling" (the
rate at which there was trouble last time); above that, after 15 s without
trouble, only +2%/s. After 30 s without trouble the ceiling is forgotten.
- viewer_slow tipo piso: −20% (at most every 2 s), not below piso% of the
target; the rate doesn't climb for 12 s.
Server, on every frame, for every viewer:
- delay > 500 ms (+ IDR slack) or more than 3 MiB in the queue: discard the frames
that haven't gone out yet, wait for the ones already out to be acknowledged (the
IDR can't stay behind them) and restart from an IDR. Whoever lags again in less
than 10 s waits longer for the next IDR (0.5 → 1 → 2 → 4 → 8 s), so the room
doesn't fill up with IDRs;
- delayed > 150 ms and not draining (didn't fall 50 ms in 1 s): viewer_slow to
the streamer before reaching the restart point;
- the viewer_slow floor is 10% if more than half the viewers are struggling (a
common bottleneck, e.g. the upload of whoever hosts the room, or there is only
one viewer) and 50% if it's the minority — whoever can't even with half keeps
restarting from an IDR now and then, without ruining the others' quality;
- thinning (if the stream has skippable frames): whoever lags as the minority
starts receiving only the reference frames — half the frames, ~half the
bandwidth —, and the skippable ones still in their queue are discarded right
away. It stays like that for 10 s (double, up to 1 min, if needed again in less
than 30 s); when it goes back to receiving everything and lags in the next 3 s,
it thins again right away. A thinned viewer only counts as "struggling" for
viewer_slow if it can't keep up even then (lagging 3 s after thinning, or
restarting from an IDR). Whoever asks for an IDR because the decoder couldn't
keep up is always thinned (half the work for the decoder).
Viewer: acknowledges right away; if the decoder falls more than 150 ms behind,
it first skips to an IDR already in the queue, or drops the droppable frames and
sends keyreq stream with reason 2 (no IDR: the server only thins it), giving the
decoder 100 ms; only when neither helps does it throw the queue away and send
reason 0 (the server applies the same growing wait). Frames that already have a newer one waiting are
decoded but don't go to the screen.
The IDR request for a viewer waiting normally goes out when the streamer's next frame arrives; with their screen still (nothing arriving for 1 s and no request in the last 2 s), the server asks by itself — checks every 250 ms —, otherwise the viewer would be stuck on the old picture.
On Linux the sockets use TCP_NOTSENT_LOWAT = 32 KB: the kernel holds little
not-yet-sent data and the rest waits in the app's queue, where it can be
discarded.
5. Clock and latency
The server has a clock (µs since it started). The client estimates the offset with
the pings (offset = server_clock + RTT/2 − now, weighting measurements with a
lower RTT). The streamer marks each frame with the capture time on that clock;
viewers compute delay = now_in_room − hora_da_captura when the frame goes to the
GPU. Chat message times also come from that clock.
