sockets: a descriptor can cross between processes
sendmsg/recvmsg with SCM_RIGHTS. Item 1 of docs/plan/foundations.md and
mechanism 1 of docs/plan/hybrid.md, which are the same primitive seen from two
directions: Wayland's wl_shm_create_pool passes a file descriptor with no
integer-shaped alternative, and handing a driver process the device it is
allowed to touch is the same act. Both plans stopped here.
Linux's structures byte for byte -- msghdr, cmsghdr, the CMSG_ macros -- because
the programs that need this are not ours. libwayland and toybox both call
recvmsg with Linux's headers and would not notice a friendlier door.
The delivery rule is the whole design. A batch of descriptors is pinned to the
stream position of the first byte of the message that carried it, and that has
two halves: a reader arriving at that position is handed them, and a reader that
has not reached it is stopped SHORT of it. Leave out the second half and a
program asking for 4096 bytes swallows three messages and is handed one
message's descriptors, with nothing in the result saying which. socktest checks
exactly that: two one-byte messages, an eight-byte read, one byte back.
The kernel holds a reference to every descriptor in flight, so a sender may
close its own copy the moment sendmsg returns. Giving that reference back is the
part that leaks if it is not written down, and there are three ways a message
ends:
- received with room installed in the receiver's table
- received without room, or by
plain read(2) closed; MSG_CTRUNC for the first case
- never received, socket closed closed by sockclose
The middle row looks like a choice and is not: holding kernel references for a
receiver that has proved it is not looking is a leak with extra steps.
No garbage collector. A socket descriptor sent over that same socket and never
received holds a reference to the socket that holds the reference to it, and
neither close reaches zero. Linux carries several hundred lines for this.
Nothing on either plan passes a socket over a socket -- Wayland passes memory,
the driver work passes devices -- so it is a bounded leak that is documented
rather than a silent one.
Fourteen checks in cmd/socktest, and every one moves data through the descriptor
rather than inspecting it. The descriptor passed is always a pipe end, because a
pipe announces its own lifetime: a read of a pipe whose writers are all gone
returns 0, which turns "was the kernel's reference released" -- otherwise
invisible from userspace -- into a value a test can compare.
Two bugs fell out, neither in the new code:
- sockpair() never set the socket's domain, so every socketpair() socket
claimed domain 0. Invisible while the data paths asked only whether a socket
was AF_INET or AF_NETLINK, since zero answers "neither" correctly.
SCM_RIGHTS asks "is this AF_UNIX", and zero is wrong for that one.
- pipe_poll() reports POLLHUP but not POLLIN at end of file, so a non-blocking
read of a pipe whose writers had all gone returned EAGAIN forever instead of
0 -- a program that polls and reads spins waiting for an end that already
happened. sockreadable() always had the equivalent condition; the pipe path
did not. Found because the test helper for "is this descriptor closed" is a
non-blocking read of a pipe.
projects/schcom/README.md said SCM_RIGHTS was done for a long time while nothing
in the kernel implemented it, and an entire GUI plan was drawn on that line.
It is true now, which is not the same as it having been right, and the README
says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz
parent
ea68ef4b54
commit
9e6d8dd4e3
15 changed files with 1133 additions and 42 deletions
113
include/socket.h
113
include/socket.h
|
|
@ -55,6 +55,90 @@
|
|||
|
||||
#define TCP_NODELAY 1
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Ancillary data -- sendmsg/recvmsg, and SCM_RIGHTS
|
||||
//
|
||||
// A descriptor is the only thing that can be passed between two processes that
|
||||
// is not a number: it names an object the kernel already knows, and handing it
|
||||
// over is the difference between "here is a buffer" and "here is a *copy* of a
|
||||
// buffer". Wayland's wl_shm_create_pool passes one and has no integer-shaped
|
||||
// alternative; docs/plan/hybrid.md's mechanism 1 is this and nothing else.
|
||||
//
|
||||
// Every structure below is Linux's, byte for byte, because a musl-linked
|
||||
// program builds them with Linux's headers and we do not get a vote. The
|
||||
// offsets that matter on x86-64 are, for msghdr:
|
||||
//
|
||||
// 0 msg_name 8 msg_namelen(+4 pad) 16 msg_iov 24 msg_iovlen
|
||||
// 32 msg_control 40 msg_controllen 48 msg_flags
|
||||
//
|
||||
// -- 56 bytes total, which is what musl's int-typed msg_iovlen plus a pad word
|
||||
// adds up to as well.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#define MSG_OOB 0x0001
|
||||
#define MSG_PEEK 0x0002
|
||||
#define MSG_DONTROUTE 0x0004
|
||||
#define MSG_CTRUNC 0x0008 // some ancillary data was dropped
|
||||
#define MSG_TRUNC 0x0020
|
||||
#define MSG_DONTWAIT 0x0040
|
||||
#define MSG_EOR 0x0080
|
||||
#define MSG_WAITALL 0x0100
|
||||
#define MSG_NOSIGNAL 0x4000
|
||||
#define MSG_CMSG_CLOEXEC 0x40000000
|
||||
|
||||
// The only control message that exists here. SCM_CREDENTIALS is deliberately
|
||||
// absent rather than stubbed: a caller told it succeeded would read a uid out
|
||||
// of uninitialised memory and act on it.
|
||||
#define SCM_RIGHTS 1
|
||||
|
||||
// The most descriptors one message may carry. A cap has to exist -- the kernel
|
||||
// holds a reference to every one of them until it is received -- and saying so
|
||||
// here means a sender can size its control buffer against the same number the
|
||||
// kernel will enforce.
|
||||
#define SCM_MAXFD 8
|
||||
|
||||
struct iovec {
|
||||
void *iov_base;
|
||||
uint64 iov_len;
|
||||
};
|
||||
|
||||
struct msghdr {
|
||||
void *msg_name; // struct sockaddr *, or 0
|
||||
uint32 msg_namelen;
|
||||
uint32 __pad1;
|
||||
struct iovec *msg_iov;
|
||||
uint64 msg_iovlen;
|
||||
void *msg_control; // struct cmsghdr *, or 0
|
||||
uint64 msg_controllen;
|
||||
int msg_flags; // out on recvmsg: MSG_CTRUNC
|
||||
};
|
||||
|
||||
struct cmsghdr {
|
||||
uint64 cmsg_len; // CMSG_LEN(payload), NOT including padding
|
||||
int cmsg_level; // SOL_SOCKET
|
||||
int cmsg_type; // SCM_RIGHTS
|
||||
};
|
||||
|
||||
// cmsg_len counts the header plus the payload; CMSG_SPACE additionally counts
|
||||
// the padding to the next header. Conflating the two is the classic way to
|
||||
// build a control buffer the kernel then rejects, which is why both exist.
|
||||
#define CMSG_ALIGN(n) (((uint64)(n) + sizeof(uint64) - 1) & ~(uint64)(sizeof(uint64) - 1))
|
||||
#define CMSG_LEN(n) (CMSG_ALIGN(sizeof(struct cmsghdr)) + (uint64)(n))
|
||||
#define CMSG_SPACE(n) (CMSG_ALIGN(sizeof(struct cmsghdr)) + CMSG_ALIGN(n))
|
||||
#define CMSG_DATA(c) ((unsigned char *)((struct cmsghdr *)(c) + 1))
|
||||
|
||||
#define CMSG_FIRSTHDR(m) \
|
||||
((m)->msg_control && (m)->msg_controllen >= sizeof(struct cmsghdr) \
|
||||
? (struct cmsghdr *)(m)->msg_control \
|
||||
: (struct cmsghdr *)0)
|
||||
|
||||
#define CMSG_NXTHDR(m, c) \
|
||||
((c)->cmsg_len < sizeof(struct cmsghdr) || \
|
||||
(uint64)((char *)(c) + CMSG_ALIGN((c)->cmsg_len) + sizeof(struct cmsghdr) \
|
||||
- (char *)(m)->msg_control) > (m)->msg_controllen \
|
||||
? (struct cmsghdr *)0 \
|
||||
: (struct cmsghdr *)((char *)(c) + CMSG_ALIGN((c)->cmsg_len)))
|
||||
|
||||
#define UNIX_PATH_MAX 108
|
||||
|
||||
#define SHUT_RD 0
|
||||
|
|
@ -164,6 +248,29 @@ struct rtattr {
|
|||
#define SOCKBUF 4096 // receive buffer per connected socket
|
||||
#define NBACKLOG 8
|
||||
|
||||
// Descriptors in flight.
|
||||
//
|
||||
// A batch is pinned to the stream position of the first byte of the message
|
||||
// that carried it, and that is the whole delivery rule: a reader that arrives
|
||||
// at that position gets the descriptors, and a reader that has not reached it
|
||||
// yet is stopped *short* of it rather than allowed to read past. Without the
|
||||
// second half a program that read 4096 bytes in one call would swallow three
|
||||
// messages and be handed one message's descriptors, with no way to tell which.
|
||||
//
|
||||
// Eight batches in flight is not a guess: a sender that has eight unreceived
|
||||
// messages carrying descriptors and keeps going is not going to be rescued by
|
||||
// a ninth slot, and ENOBUFS says so where a larger array would only move the
|
||||
// same failure further away.
|
||||
#define NSCM 8 // messages carrying descriptors, in flight
|
||||
|
||||
struct file;
|
||||
|
||||
struct scm_fds {
|
||||
uint pos; // stream position of the message's first byte
|
||||
int n;
|
||||
struct file *f[SCM_MAXFD]; // references held by the kernel, not by anyone
|
||||
};
|
||||
|
||||
enum sockstate {
|
||||
SS_FREE = 0,
|
||||
SS_UNBOUND,
|
||||
|
|
@ -209,6 +316,12 @@ struct socket {
|
|||
int o_keepalive;
|
||||
int o_broadcast;
|
||||
int o_nodelay;
|
||||
|
||||
// Descriptors sent to this socket and not yet received. They live on the
|
||||
// *receiving* side, next to the bytes they were sent with, because that is
|
||||
// the only side that can say when they have been delivered.
|
||||
struct scm_fds scm[NSCM];
|
||||
int nscm;
|
||||
};
|
||||
|
||||
#endif
|
||||
|
|
|
|||
Loading…
Reference in a new issue