9e6d8dd4e3
sendmsg/recvmsg with SCM_RIGHTS. Item 1 of docs/plan/foundations.md and
mechanism 1 of docs/plan/hybrid.md, which are the same primitive seen from two
directions: Wayland's wl_shm_create_pool passes a file descriptor with no
integer-shaped alternative, and handing a driver process the device it is
allowed to touch is the same act. Both plans stopped here.
Linux's structures byte for byte -- msghdr, cmsghdr, the CMSG_ macros -- because
the programs that need this are not ours. libwayland and toybox both call
recvmsg with Linux's headers and would not notice a friendlier door.
The delivery rule is the whole design. A batch of descriptors is pinned to the
stream position of the first byte of the message that carried it, and that has
two halves: a reader arriving at that position is handed them, and a reader that
has not reached it is stopped SHORT of it. Leave out the second half and a
program asking for 4096 bytes swallows three messages and is handed one
message's descriptors, with nothing in the result saying which. socktest checks
exactly that: two one-byte messages, an eight-byte read, one byte back.
The kernel holds a reference to every descriptor in flight, so a sender may
close its own copy the moment sendmsg returns. Giving that reference back is the
part that leaks if it is not written down, and there are three ways a message
ends:
- received with room installed in the receiver's table
- received without room, or by
plain read(2) closed; MSG_CTRUNC for the first case
- never received, socket closed closed by sockclose
The middle row looks like a choice and is not: holding kernel references for a
receiver that has proved it is not looking is a leak with extra steps.
No garbage collector. A socket descriptor sent over that same socket and never
received holds a reference to the socket that holds the reference to it, and
neither close reaches zero. Linux carries several hundred lines for this.
Nothing on either plan passes a socket over a socket -- Wayland passes memory,
the driver work passes devices -- so it is a bounded leak that is documented
rather than a silent one.
Fourteen checks in cmd/socktest, and every one moves data through the descriptor
rather than inspecting it. The descriptor passed is always a pipe end, because a
pipe announces its own lifetime: a read of a pipe whose writers are all gone
returns 0, which turns "was the kernel's reference released" -- otherwise
invisible from userspace -- into a value a test can compare.
Two bugs fell out, neither in the new code:
- sockpair() never set the socket's domain, so every socketpair() socket
claimed domain 0. Invisible while the data paths asked only whether a socket
was AF_INET or AF_NETLINK, since zero answers "neither" correctly.
SCM_RIGHTS asks "is this AF_UNIX", and zero is wrong for that one.
- pipe_poll() reports POLLHUP but not POLLIN at end of file, so a non-blocking
read of a pipe whose writers had all gone returned EAGAIN forever instead of
0 -- a program that polls and reads spins waiting for an end that already
happened. sockreadable() always had the equivalent condition; the pipe path
did not. Found because the test helper for "is this descriptor closed" is a
non-blocking read of a pipe.
projects/schcom/README.md said SCM_RIGHTS was done for a long time while nothing
in the kernel implemented it, and an entire GUI plan was drawn on that line.
It is true now, which is not the same as it having been right, and the README
says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
157 lines
7 KiB
Markdown
157 lines
7 KiB
Markdown
# Foundations
|
||
|
||
The kernel work that [wayland.md](wayland.md) and [hybrid.md](hybrid.md) both
|
||
stop at. **This is the critical path**, and it is the reason those two are one
|
||
plan rather than two.
|
||
|
||
The insight is small and it decides the schedule: the substrate for passing a
|
||
buffer to a compositor and the substrate for handing a device to a driver process
|
||
are the same substrate. Descriptor passing and shared memory, built once.
|
||
|
||
---
|
||
|
||
## What is actually in the tree
|
||
|
||
Measured against the source, not against a README. Each of these was checked
|
||
before anything below was planned around it:
|
||
|
||
| | |
|
||
|---|---|
|
||
| ~~`sendmsg`/`recvmsg`, and therefore **`SCM_RIGHTS`**~~ | **Built.** See item 1 below. |
|
||
| Writable `MAP_SHARED` of a file | **Refused** — `kernel/sys/sys_mem.c:131` returns `-ENODEV`. |
|
||
| `memfd_create` | Does not exist. |
|
||
| `AF_UNIX` names | In a table in `kernel/net/unix.c`, not in the filesystem. |
|
||
| `setsockopt` | `return 0;` unconditionally — `kernel/sys/sys_net.c:436`. |
|
||
| `exec` | `ET_EXEC` only. No `PT_INTERP`, no `ET_DYN`. |
|
||
|
||
> **`projects/schcom/README.md` said `SCM_RIGHTS` was done while nothing in the
|
||
> kernel implemented it** — the most load-bearing wrong line in the tree, with an
|
||
> entire GUI plan drawn on top of it. The line is now true, which is not the same
|
||
> as it having been right, and the README says so.
|
||
|
||
The `MAP_SHARED` refusal is worth defending rather than fixing carelessly. The
|
||
code returns `-ENODEV` *honestly*: the contract of a writable shared mapping is
|
||
that stores land in the file and other mappers see them, and there is no page
|
||
cache that could make that true. Quietly handing back a private copy would be the
|
||
same lie in a new place.
|
||
|
||
---
|
||
|
||
## The five items
|
||
|
||
In dependency order. The first three are what Wayland cannot start without.
|
||
Item 1 is built; items 2 and 3 are what a buffer still cannot cross on.
|
||
|
||
### 1. `sendmsg` / `recvmsg` with `SCM_RIGHTS` — **done**
|
||
|
||
Not optional and not substitutable. `wl_shm_create_pool` passes a **file
|
||
descriptor**; there is no path in the Wayland protocol that passes an integer
|
||
instead.
|
||
|
||
The `handle_kind` field in `projects/schcom/proto.h` was good foresight and it
|
||
cannot help here — *our* protocol is the flexible one, and Wayland's is not.
|
||
|
||
Built as described: `struct msghdr`/`cmsghdr` in Linux's layout, ancillary data
|
||
queued alongside the byte stream in `kernel/net/unix.c`, descriptor installation
|
||
into the receiver's file table, and the three ways a message can end — received,
|
||
received without room, never received — each releasing the kernel's reference.
|
||
Fourteen checks in `cmd/socktest`, every one of them moving data through the
|
||
descriptor rather than inspecting it. Documented in
|
||
[../architecture/networking.md](../architecture/networking.md#descriptor-passing).
|
||
|
||
The delivery rule is the part worth knowing before building on it: **descriptors
|
||
are pinned to the stream position of the first byte of the message that carried
|
||
them**, and a read that has not reached that position is cut short so it stops
|
||
before it. That is what keeps "which message did these belong to" answerable.
|
||
|
||
**This was also item 1 of the hybrid work**, where it is how a driver process is
|
||
handed the device it is allowed to touch.
|
||
|
||
Two bugs fell out of building it, neither in the new code:
|
||
|
||
- `sockpair()` never set the socket's domain, so **every `socketpair()` socket
|
||
claimed domain 0**. Invisible for as long as the data paths asked only whether
|
||
a socket was `AF_INET` or `AF_NETLINK` — zero answers "neither" correctly.
|
||
`SCM_RIGHTS` asks "is this `AF_UNIX`", and zero is wrong for that one.
|
||
- `pipe_poll()` reports `POLLHUP` but not `POLLIN` for a pipe at end of file, so
|
||
a **non-blocking read of a pipe whose writers have all gone returned `EAGAIN`
|
||
forever** instead of 0. A program that polls and reads would spin waiting for
|
||
an end that had already happened. `sockreadable()` always had the equivalent
|
||
condition; the pipe path did not.
|
||
|
||
### 2. A shareable memory object
|
||
|
||
`memfd_create`, plus `mmap(MAP_SHARED)` on it genuinely working between two
|
||
processes.
|
||
|
||
Narrower than a general page cache, and sufficient. An anonymous shared object
|
||
needs no file behind it, which is exactly why `memfd` exists on Linux at all —
|
||
and it means this can be built on the shared-memory machinery already in
|
||
`kernel/sys/sys_mem.c` (`shmget`/`shmat`/`shmdt` work, and `struct vma` already
|
||
carries `shmid` and a `no_free` flag for pages that are not ours to release).
|
||
|
||
The general answer — a page cache, and writable `MAP_SHARED` of a real file —
|
||
stays owed. It is a larger project and nothing on this plan needs it.
|
||
|
||
### 3. `AF_UNIX` sockets in the filesystem namespace
|
||
|
||
Clients find a Wayland compositor by `bind()`ing `$XDG_RUNTIME_DIR/wayland-0` as
|
||
a real path, and libwayland `flock()`s the lock file beside it. A name table can
|
||
express neither.
|
||
|
||
`kernel/net/unix.c`'s own header comment already predicted this:
|
||
|
||
> *"What is lost is `ls -l /tmp/schcom` showing an `s`, and `unlink()` of a socket
|
||
> path failing with ENOENT. Both are worth fixing when the VFS next gets opened
|
||
> up; neither blocks anything."*
|
||
|
||
They block something now. This is that moment, and it is a VFS change rather than
|
||
a socket change: a socket inode type threaded through `iget`, `dirlookup` and
|
||
`stat`, plus `flock`.
|
||
|
||
### 4. `setsockopt` that does not lie
|
||
|
||
At minimum `SO_PASSCRED` and `SO_SNDBUF`. Returning 0 for an option you ignored
|
||
is worse than `ENOPROTOOPT`, because the caller adjusts its behaviour on the
|
||
strength of the lie.
|
||
|
||
Small, and grouped here because it is in the same file as item 3 and will be
|
||
found by the same testing.
|
||
|
||
### 5. Dynamic linking
|
||
|
||
`PT_INTERP` and `ET_DYN` in `kernel/fs/exec.c`.
|
||
|
||
**Deferrable, and deferred.** Static binaries are enough to reach a working
|
||
desktop with software rendering. This becomes necessary when Mesa does — EGL
|
||
means `dlopen` means a loader — and that is a separate decision made later.
|
||
|
||
---
|
||
|
||
## Verification
|
||
|
||
Each item gets a test before the thing that needs it is built, because a
|
||
substrate that is subtly wrong fails inside the consumer and looks like the
|
||
consumer's bug.
|
||
|
||
- **Item 1:** two processes, one passes an open descriptor, the other reads
|
||
through it. Plus the leak case: a sender that exits with descriptors in flight.
|
||
- **Item 2:** two processes map the same `memfd`, one writes, the other sees it.
|
||
- **Item 3:** `bind()` a socket, then `stat()` and `ls -l` the path and see an
|
||
`s`, then `unlink()` it.
|
||
- **Items 1–3 together:** a third test that does all of it at once, because that
|
||
combination is what Wayland actually performs and the three passing separately
|
||
does not prove it.
|
||
|
||
These belong in `bootcheck` as a new switch, following the `-Sock` model.
|
||
|
||
---
|
||
|
||
## What this does not deliver
|
||
|
||
- **No page cache**, so writable `MAP_SHARED` of a real file stays refused.
|
||
- **No `dlopen`** until item 5 is done, which means no Mesa and no third-party
|
||
shared libraries.
|
||
- **`SO_*` coverage is the two options named**, not the full set.
|
||
- **No credential passing** (`SCM_CREDENTIALS`) — only descriptors. Nothing on
|
||
this plan needs it, and a compositor that wants to know who connected will.
|