sockets: a descriptor can cross between processes
sendmsg/recvmsg with SCM_RIGHTS. Item 1 of docs/plan/foundations.md and
mechanism 1 of docs/plan/hybrid.md, which are the same primitive seen from two
directions: Wayland's wl_shm_create_pool passes a file descriptor with no
integer-shaped alternative, and handing a driver process the device it is
allowed to touch is the same act. Both plans stopped here.
Linux's structures byte for byte -- msghdr, cmsghdr, the CMSG_ macros -- because
the programs that need this are not ours. libwayland and toybox both call
recvmsg with Linux's headers and would not notice a friendlier door.
The delivery rule is the whole design. A batch of descriptors is pinned to the
stream position of the first byte of the message that carried it, and that has
two halves: a reader arriving at that position is handed them, and a reader that
has not reached it is stopped SHORT of it. Leave out the second half and a
program asking for 4096 bytes swallows three messages and is handed one
message's descriptors, with nothing in the result saying which. socktest checks
exactly that: two one-byte messages, an eight-byte read, one byte back.
The kernel holds a reference to every descriptor in flight, so a sender may
close its own copy the moment sendmsg returns. Giving that reference back is the
part that leaks if it is not written down, and there are three ways a message
ends:
- received with room installed in the receiver's table
- received without room, or by
plain read(2) closed; MSG_CTRUNC for the first case
- never received, socket closed closed by sockclose
The middle row looks like a choice and is not: holding kernel references for a
receiver that has proved it is not looking is a leak with extra steps.
No garbage collector. A socket descriptor sent over that same socket and never
received holds a reference to the socket that holds the reference to it, and
neither close reaches zero. Linux carries several hundred lines for this.
Nothing on either plan passes a socket over a socket -- Wayland passes memory,
the driver work passes devices -- so it is a bounded leak that is documented
rather than a silent one.
Fourteen checks in cmd/socktest, and every one moves data through the descriptor
rather than inspecting it. The descriptor passed is always a pipe end, because a
pipe announces its own lifetime: a read of a pipe whose writers are all gone
returns 0, which turns "was the kernel's reference released" -- otherwise
invisible from userspace -- into a value a test can compare.
Two bugs fell out, neither in the new code:
- sockpair() never set the socket's domain, so every socketpair() socket
claimed domain 0. Invisible while the data paths asked only whether a socket
was AF_INET or AF_NETLINK, since zero answers "neither" correctly.
SCM_RIGHTS asks "is this AF_UNIX", and zero is wrong for that one.
- pipe_poll() reports POLLHUP but not POLLIN at end of file, so a non-blocking
read of a pipe whose writers had all gone returned EAGAIN forever instead of
0 -- a program that polls and reads spins waiting for an end that already
happened. sockreadable() always had the equivalent condition; the pipe path
did not. Found because the test helper for "is this descriptor closed" is a
non-blocking read of a pipe.
projects/schcom/README.md said SCM_RIGHTS was done for a long time while nothing
in the kernel implemented it, and an entire GUI plan was drawn on that line.
It is true now, which is not the same as it having been right, and the README
says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
parent
ea68ef4b54
commit
9e6d8dd4e3
15 changed files with 1133 additions and 42 deletions
|
|
@ -195,8 +195,6 @@ allocation over 8 MB mapped straight into a neighbouring live mapping.
|
|||
- **No demand paging and no page cache.** `mmap` of a file *reads the whole
|
||||
thing in* at map time. Writable `MAP_SHARED` of a file is therefore refused
|
||||
outright rather than faked — see [../plan/foundations.md](../plan/foundations.md).
|
||||
- **No `sendmsg`/`recvmsg`, so no `SCM_RIGHTS`.** File descriptors cannot be
|
||||
passed between processes.
|
||||
- **No dynamic linking.** `exec` handles `ET_EXEC` only: no `PT_INTERP`, no
|
||||
`ET_DYN`, no shared libraries, no `dlopen`.
|
||||
- **No threads.** `clone` exists but does not share an address space;
|
||||
|
|
|
|||
|
|
@ -176,6 +176,53 @@ server by a name agreed in advance — and what is lost is `ls -l` showing an `s
|
|||
and `unlink()` of a socket path working. Both want a VFS change rather than a
|
||||
socket change.
|
||||
|
||||
### Descriptor passing
|
||||
|
||||
`sendmsg`/`recvmsg` with `SCM_RIGHTS`, which is the one thing a socket can carry
|
||||
that a pipe cannot: a reference to an object the kernel already knows, rather
|
||||
than a copy of its contents. Wayland's `wl_shm_create_pool` passes a descriptor
|
||||
and has no integer-shaped alternative, and
|
||||
[hybrid.md](../plan/hybrid.md)'s mechanism 1 is how a driver process is handed
|
||||
the device it is allowed to touch. Both wanted the same primitive.
|
||||
|
||||
**A batch of descriptors is pinned to the stream position of the first byte of
|
||||
the message that carried it.** That single rule is the whole delivery model, and
|
||||
it has two halves that are easy to get half-right:
|
||||
|
||||
- a reader that arrives at that position is handed the descriptors, and
|
||||
- a reader that has not reached it yet is **stopped short of it**.
|
||||
|
||||
Without the second half, a program that asked for 4096 bytes would swallow three
|
||||
messages in one call and be handed one message's descriptors, with nothing in
|
||||
the result saying which message they belonged to. `socktest` checks exactly
|
||||
this: two messages, each one byte and one descriptor, read with an eight-byte
|
||||
buffer, and the first read must return one byte.
|
||||
|
||||
The kernel holds a reference on every descriptor in flight, so a sender may
|
||||
close its own copy the moment `sendmsg` returns. That reference is also the
|
||||
thing that has to be given back, and there are three ways a message ends:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| received by `recvmsg` with room | installed in the receiver's table |
|
||||
| received by `recvmsg` without room, or by plain `read(2)` | **closed**, and `MSG_CTRUNC` set for the first case |
|
||||
| never received — the socket is closed | **closed** by `sockclose` |
|
||||
|
||||
The second row is the one that looks like a choice and is not. Holding kernel
|
||||
references for a receiver that has already proved it is not looking is a leak
|
||||
with extra steps, and it is what Linux does for the same reason.
|
||||
|
||||
Descriptors need at least one byte to ride on; `sendmsg` refuses a message of
|
||||
pure ancillary data with `EINVAL` rather than accepting it and pinning the
|
||||
descriptors to a position that never arrives.
|
||||
|
||||
**There is no garbage collector.** An AF_UNIX socket descriptor, sent over that
|
||||
same socket and never received, holds a reference to the socket that holds the
|
||||
reference to it, and neither close can reach zero. Linux carries several hundred
|
||||
lines to break these cycles. Nothing on either plan passes a socket over a
|
||||
socket — Wayland passes memory, the driver work passes devices — so this is a
|
||||
known and bounded leak (thirty-two sockets) rather than a silent one.
|
||||
|
||||
### AF_INET
|
||||
|
||||
`kernel/net/inet.c`, on lwIP's **raw** API rather than lwIP's own sockets layer,
|
||||
|
|
@ -296,11 +343,18 @@ be fixed before `recvfrom` could report a source address at all.
|
|||
|
||||
## What it does not do
|
||||
|
||||
- **`ping` is a first-party program**, `cmd/ping`, not toybox's. toybox's reads
|
||||
replies with `recvmsg(2)` and asks for the TTL through an `IP_RECVTTL` control
|
||||
message, and neither exists — `recvmsg` is item 1 of
|
||||
[../plan/foundations.md](../plan/foundations.md), where it is wanted for
|
||||
`SCM_RIGHTS` anyway. Ours has no `-I`, no flood mode and no TTL display.
|
||||
- **`ping` is a first-party program**, `cmd/ping`, not toybox's. `recvmsg` now
|
||||
exists, so half of what stopped toybox's is gone; what remains is
|
||||
`IP_RECVTTL`, an `IPPROTO_IP` control message that would have to be generated
|
||||
on the receive path rather than merely carried. Ours has no `-I`, no flood
|
||||
mode and no TTL display.
|
||||
- **`SCM_RIGHTS` is the only control message.** `SCM_CREDENTIALS` is refused
|
||||
with `EINVAL` rather than ignored: a caller told its credentials were passed
|
||||
would have the receiver read a uid out of uninitialised memory and act on it.
|
||||
- **A message may carry eight descriptors, and a socket eight such messages.**
|
||||
Past that `sendmsg` answers `ENOBUFS`. The cap exists because the kernel holds
|
||||
a reference to each one until it is received; a sender eight unreceived
|
||||
messages ahead of its receiver is not rescued by a ninth slot.
|
||||
- **No resolver reachable from userland.** lwIP's DNS client is compiled but
|
||||
nothing exposes it, so `ping` takes a dotted quad. `localhost` is special-cased
|
||||
in the program rather than resolved.
|
||||
|
|
|
|||
|
|
@ -113,8 +113,9 @@ field by field, each through its own translation, or use `copyin`/`copyout`.
|
|||
|
||||
## What it does not do
|
||||
|
||||
- **No `sendmsg`/`recvmsg`**, and therefore no ancillary data and no
|
||||
`SCM_RIGHTS`.
|
||||
- **`sendmsg`/`recvmsg` carry `SCM_RIGHTS` and nothing else.**
|
||||
`SCM_CREDENTIALS` is refused rather than ignored. See
|
||||
[networking.md](networking.md#descriptor-passing).
|
||||
- **No `memfd_create`.**
|
||||
- **No `getsockname` or `getpeername`**, and `accept()` does not report an
|
||||
AF_INET peer's address.
|
||||
|
|
|
|||
|
|
@ -17,16 +17,17 @@ before anything below was planned around it:
|
|||
|
||||
| | |
|
||||
|---|---|
|
||||
| `sendmsg`/`recvmsg`, and therefore **`SCM_RIGHTS`** | **Do not exist.** Not in `kernel/sys/table.inc`, not anywhere in `kernel/`. |
|
||||
| ~~`sendmsg`/`recvmsg`, and therefore **`SCM_RIGHTS`**~~ | **Built.** See item 1 below. |
|
||||
| Writable `MAP_SHARED` of a file | **Refused** — `kernel/sys/sys_mem.c:131` returns `-ENODEV`. |
|
||||
| `memfd_create` | Does not exist. |
|
||||
| `AF_UNIX` names | In a table in `kernel/net/unix.c`, not in the filesystem. |
|
||||
| `setsockopt` | `return 0;` unconditionally — `kernel/sys/sys_net.c:436`. |
|
||||
| `exec` | `ET_EXEC` only. No `PT_INTERP`, no `ET_DYN`. |
|
||||
|
||||
> **`projects/schcom/README.md` says `SCM_RIGHTS` is done. That line is wrong**,
|
||||
> and it is the most load-bearing wrong line in the tree — an entire GUI plan was
|
||||
> drawn on top of it. It is corrected as part of this work.
|
||||
> **`projects/schcom/README.md` said `SCM_RIGHTS` was done while nothing in the
|
||||
> kernel implemented it** — the most load-bearing wrong line in the tree, with an
|
||||
> entire GUI plan drawn on top of it. The line is now true, which is not the same
|
||||
> as it having been right, and the README says so.
|
||||
|
||||
The `MAP_SHARED` refusal is worth defending rather than fixing carelessly. The
|
||||
code returns `-ENODEV` *honestly*: the contract of a writable shared mapping is
|
||||
|
|
@ -39,8 +40,9 @@ same lie in a new place.
|
|||
## The five items
|
||||
|
||||
In dependency order. The first three are what Wayland cannot start without.
|
||||
Item 1 is built; items 2 and 3 are what a buffer still cannot cross on.
|
||||
|
||||
### 1. `sendmsg` / `recvmsg` with `SCM_RIGHTS`
|
||||
### 1. `sendmsg` / `recvmsg` with `SCM_RIGHTS` — **done**
|
||||
|
||||
Not optional and not substitutable. `wl_shm_create_pool` passes a **file
|
||||
descriptor**; there is no path in the Wayland protocol that passes an integer
|
||||
|
|
@ -49,14 +51,34 @@ instead.
|
|||
The `handle_kind` field in `projects/schcom/proto.h` was good foresight and it
|
||||
cannot help here — *our* protocol is the flexible one, and Wayland's is not.
|
||||
|
||||
Needs: `struct msghdr`/`cmsghdr` handling, ancillary data queued alongside the
|
||||
byte stream in `kernel/net/unix.c`, descriptor installation into the receiver's
|
||||
file table, and correct behaviour when a message carrying descriptors is never
|
||||
received (they must be closed, not leaked).
|
||||
Built as described: `struct msghdr`/`cmsghdr` in Linux's layout, ancillary data
|
||||
queued alongside the byte stream in `kernel/net/unix.c`, descriptor installation
|
||||
into the receiver's file table, and the three ways a message can end — received,
|
||||
received without room, never received — each releasing the kernel's reference.
|
||||
Fourteen checks in `cmd/socktest`, every one of them moving data through the
|
||||
descriptor rather than inspecting it. Documented in
|
||||
[../architecture/networking.md](../architecture/networking.md#descriptor-passing).
|
||||
|
||||
**This is also item 1 of the hybrid work**, where it is how a driver process is
|
||||
The delivery rule is the part worth knowing before building on it: **descriptors
|
||||
are pinned to the stream position of the first byte of the message that carried
|
||||
them**, and a read that has not reached that position is cut short so it stops
|
||||
before it. That is what keeps "which message did these belong to" answerable.
|
||||
|
||||
**This was also item 1 of the hybrid work**, where it is how a driver process is
|
||||
handed the device it is allowed to touch.
|
||||
|
||||
Two bugs fell out of building it, neither in the new code:
|
||||
|
||||
- `sockpair()` never set the socket's domain, so **every `socketpair()` socket
|
||||
claimed domain 0**. Invisible for as long as the data paths asked only whether
|
||||
a socket was `AF_INET` or `AF_NETLINK` — zero answers "neither" correctly.
|
||||
`SCM_RIGHTS` asks "is this `AF_UNIX`", and zero is wrong for that one.
|
||||
- `pipe_poll()` reports `POLLHUP` but not `POLLIN` for a pipe at end of file, so
|
||||
a **non-blocking read of a pipe whose writers have all gone returned `EAGAIN`
|
||||
forever** instead of 0. A program that polls and reads would spin waiting for
|
||||
an end that had already happened. `sockreadable()` always had the equivalent
|
||||
condition; the pipe path did not.
|
||||
|
||||
### 2. A shareable memory object
|
||||
|
||||
`memfd_create`, plus `mmap(MAP_SHARED)` on it genuinely working between two
|
||||
|
|
|
|||
|
|
@ -157,7 +157,8 @@ re-fires for ever, drowning the e1000 that shares it. The fix is one line,
|
|||
is that the storm detector built for history/kernel.md section 13 named the
|
||||
line, the rate and the neighbour without anyone having to look.
|
||||
|
||||
**The control channel (mechanism 1) is what remains.**
|
||||
**All four mechanisms now exist.** What remains on this plan is the second
|
||||
eviction and then the foreign driver host.
|
||||
|
||||
### One thing the interface is deliberately not
|
||||
|
||||
|
|
@ -175,11 +176,20 @@ version that survives a licensing conversation.
|
|||
|
||||
Four, and only the fourth is large.
|
||||
|
||||
### 1. A control channel
|
||||
### 1. A control channel — **built**
|
||||
|
||||
`AF_UNIX` plus `SCM_RIGHTS` — item 1 of [foundations.md](foundations.md), which
|
||||
the Wayland work builds first. A descriptor is how a driver process is handed the
|
||||
device it is allowed to touch.
|
||||
the Wayland work also needed first. A descriptor is how a driver process is
|
||||
handed the device it is allowed to touch.
|
||||
|
||||
Built as `sendmsg`/`recvmsg` in Linux's shape, because the programs that will
|
||||
use it are not ours: libwayland and toybox both call `recvmsg` with Linux's
|
||||
structures and would not notice a friendlier door. Descriptors are pinned to the
|
||||
stream position of the first byte of their message, and the kernel holds a
|
||||
reference to each until it is received — or closes it when the socket is dropped
|
||||
with the message still queued, which is the case that leaks if nobody writes it
|
||||
down. See
|
||||
[../architecture/networking.md](../architecture/networking.md#descriptor-passing).
|
||||
|
||||
**Do not build Mach ports.** The port model is worth its cost when there is a
|
||||
naming and capability story to hang on it; there is not one yet, and inventing it
|
||||
|
|
|
|||
Loading…
Reference in a new issue