SchOS/docs/architecture/networking.md

# Networking

ARP, IPv4, ICMP, UDP, TCP and DHCP, over a driver-agnostic interface class,
reached through the same socket syscalls as Unix domain sockets — plus DNS
(resolved in the process, by musl), HTTPS (`wget`, Alpine's TLS libraries) and a
small netlink responder for `getifaddrs`.

Verified end to end rather than by inspection: `bootcheck -Net` starts a
listener and a UDP responder on the host, tells the guest to talk to them, and
asserts on **both** ends — one side alone cannot tell a working stack from one
that completes a handshake and drops every byte.

---

## The four layers

```
        socket() bind() connect() send() recv()        syscalls, one set
    ┌────────────────┴───────────────────┐
    │  AF_UNIX              AF_INET      │   socknew() dispatches on domain
    │  kernel/net/unix.c    kernel/net/inet.c
    └────────────────┬───────────────────┘
                     │  lwIP raw API
    ┌────────────────┴───────────────────┐
    │  lwIP 2.2.0, NO_SYS=1              │   ports/lwip, compiled into the kernel
    │  port layer: kernel/net/lwip/      │   ~400 lines: a lock, a clock, a netif
    └────────────────┬───────────────────┘
                     │  struct netdev_ops
    ┌────────────────┴───────────────────┐
    │  e1000  rtl8139  r8169  — or a    │   in the kernel
    │  Linux driver in a process, which  │   (LDLS, via /dev/netdev)
    │  is a netdev like any other        │
    └────────────────────────────────────┘
```

## Intel e1000, and the PCIe generation it used to refuse

`drivers/net/e1000/` matched the 8254x parts and deliberately refused the PCIe
8257x, with a note saying they "differ in NVM access and initialisation, and
claiming a device this driver has not been shown to initialise is worse than
not claiming it".

That was right, and the second half of it is answerable: QEMU emulates the
**82574L**, so the part can be shown to initialise rather than assumed to. What
actually differs turned out to be **one register**. `EERD` moved its address
field from bits 8-15 to bits 2-15 and its done bit from 4 to 1; the descriptor
rings, `RCTL`, `TCTL` and the interrupt cause register are identical. So the
8257x ids are matched now, with a per-instance flag for the NVM layout — not a
second copy of four hundred lines of ring code.

Using the wrong pair of constants does not fail loudly, which is why it is
named rather than papered over: it reads the wrong word or waits for a bit that
will never be set, and the MAC comes out as zeroes.

Still refused: the ICH/PCH parts (82562, 82577-82579, I217-I219). Those put the
PHY behind a MAC-PHY interconnect and need a genuinely different bring-up, and
nothing here can test one.


## Realtek RTL8139 — `drivers/net/rtl8139/`

A separate driver from the RTL8111 next door, despite the neighbouring name,
because it is a different chip. The 8139 has **no descriptor rings**: the
receiver is one circular buffer the chip writes packets into back to back, each
behind a four-byte header, and the transmitter is four fixed slots used
round-robin. Sharing a driver with the 8169 would put two programming models
behind one set of function pointers, which is how a mode flag ends up threaded
through every path.

Three details that are the whole driver:

- **`CAPR` runs sixteen bytes behind.** The register that tells the chip how
  far the driver has read is offset by 16 — a quirk of the part, not an
  off-by-one, and the buffer is allocated with that padding in mind.
- **`WRAP` means the allocation is bigger than the buffer.** With it set the
  chip runs a packet past the end rather than splitting it, so the 8 KB buffer
  is allocated as 8 KB + 16 + one maximum frame. Getting that wrong is a chip
  writing past the allocation at line rate.
- **A length of `0xFFF0` means "still writing"**, not a length. Copying on it
  gives a partial packet *and* loses the alignment for everything after.

It is also the one Realtek part QEMU emulates, so unlike the 8169 every line of
it has run: DHCP completes and `ping` is 4 of 4.

## Realtek RTL8169/8168/8111 — `drivers/net/r8169/`

The NIC in a very large number of laptops and desktop boards. Every
RTL8111/8168 variant answers to **10EC:8168**, which is why one match entry
covers two decades of silicon — and why the driver prints the chip's XID on
probe, since the PCI device id tells you almost nothing about which part
arrived.

### Native first, then Linux's own

The first plan was to run Linux's `r8169.ko` under [ldls.md](ldls.md), and
measuring it changed that: the module has **183 undefined symbols, 130 of them not
implemented**, and 35 of those 130 are **phylib** — `phy_connect_direct`,
`phy_start`, `mdiobus_read`, `genphy_soft_reset`. phylib is a subsystem with a link
state machine, not a set of leaf calls, and LDLS's usual answer for an
unimplemented symbol — a stub that reports itself and returns zero — is precisely
wrong for it: `phy_start` returning zero means the link never comes up and nothing
says why. So a **native driver** came first: it reaches the PHY through the
chip's own MII window (`PHYAR`, one register that is a whole MII bus) and skips
the subsystem, in the shape of the e1000 beside it.

It has since been done the other way as well. LDLS grew a driver core, devres,
firmware loading and phylib's tables ([ldls.md](ldls.md), *A set of modules*),
and the ASUS TUF A15's RTL8168H is now driven by **Linux's r8169, unmodified, in
a process**: `make ldls-r8169` stages it with libphy, mdio-bus and the PHY driver,
and `etc/schinit/ldls-r8169.job` starts it at boot. The native driver is **out of
the registry** — it would claim the device first — and stays in the tree, with
Linux's 8168H setup, as a fallback.

On that laptop the interface is `ldls0`, and it appears *after* boot, which is why
`net_attach_late()` starts the stack when the first interface arrives instead of
turning it away.

### The Chromebox has one of these

The Asus Chromebox CN62 "Guado" carries an RTL8111 rather than the Intel part its
spec sheet suggests, so the family match (`10EC:8168`) covers it as well as the
laptop — which is the argument for matching a family in one line rather than a
device id per board.

### It has not been shown to work on hardware

**Stated plainly because it cannot be tested here.** QEMU emulates `rtl8139` and
nothing else from Realtek, and the 8139 is a different programming model — a
circular receive buffer rather than descriptors — so it is deliberately *not* in
the match table. Claiming it would break a NIC that works.

What has been verified: both drivers build, their presence changes nothing on a
machine without the device, and Linux's r8169 under LDLS loads, links and
initialises in QEMU, registering and finding no device. On the laptop it reached
probe and read the PHY (after a bug in LDLS that made every MDIO read return
`0x02460246` was fixed); the native 8168H bring-up — the PLL, the receive gate, the
PHY through the OCP window — has not run on a part.

What has not: traffic on real hardware, from either.

The native probe says everything a first bug report would need, with `cprintf`
rather than `kinfo` so it reaches the screen on a machine with no serial port:

```
r8169: xid 5c0, 1c:1b:b5:xx:xx:xx, irq 11, link up (bmsr 796d)
```

`xid` is the hardware version out of the top of `TxConfig`; `bmsr` is MII
register 1, where bit 2 is link and bit 5 is autonegotiation complete. Those
four facts separate "the chip is not there", "the chip is there and the PHY
sees nothing", and "everything is up and the rings are wrong".

The kernel turns **ASPM off** on Realtek NIC links at boot, as Linux's driver
does through `pci_disable_link_state` — which a process under LDLS cannot do.

### Two constants that were wrong first

Recorded because both would have worked, quietly:

- `RxConfig` had `0x7700` for unlimited burst and threshold. `MXDMA` is bits
  8-10 and `RXFTH` is bits 13-15, so that is one bit short of a full burst and
  a half-full FIFO threshold rather than none — slower, and invisible.
- `CPCR` bit 3 was being set under a comment calling it "PCI multiple
  read/write". Bit 3 is PCIDAC, 64-bit DMA addressing; bit 4 is the one the
  comment described. Neither is wanted, since the rings come from the DMA arena
  below 4 GB, so the write is gone rather than corrected.


## The netdev class

A driver publishes a [`struct netdev_ops`](../../include/netdev.h) and is then
finished with the network stack forever: it never mentions lwIP, and the stack
never mentions it.

```c
struct netdev_ops {
  int (*transmit)(struct device *, const void *frame, int len);
  int (*receive) (struct device *, void *out, int cap);
  const unsigned char *(*mac)(struct device *);
  int (*link_up) (struct device *);        // optional
};
```

`transmit` returning -1 is **congestion, not failure** — the ring is full and
the caller is expected to retry. `receive` never blocks; it returns 0 when
nothing is waiting, and the caller decides how to wait.

This class was added late, and the reason it had to be is instructive: the stack
had reached straight into the driver. `net.c` called `e1000_present()`, the lwIP
glue was named `netif_e1000.c`, and the e1000 shipped with `.ops = 0` and a
comment saying so. A second NIC driver would have meant editing the protocol
stack to add itself. lwIP now gets one netif per bound device, `/proc/netif`
lists them all, and the e1000 drives up to four cards rather than refusing the
second — verified with two attached, each with its own MAC and its own DHCP
lease.

### `netdev_selftest()`

`echo n > /proc/sysrq` sends a hand-built ARP request and waits for the reply.

It proves the whole driver path in one exchange — descriptor formats, the DD
handshake, tail bookkeeping, DMA addresses, the receive ring hand-back, and the
MAC filter, because an ARP reply is addressed to this station and does not
arrive if the filter is wrong.

It was written *before* there was any protocol stack, deliberately: a stack on
an unproven driver fails in ways indistinguishable from its own bugs. It belongs
to the class rather than to the e1000, because it is the check every new NIC
driver will want first.

## lwIP

lwIP 2.2.0, fetched by `ports/lwip` and **compiled into the kernel** — it is the
one port that installs nothing into the sysroot, because it is kernel code.

`NO_SYS=1`, so lwIP has no threads, mailboxes or semaphores of its own. The port
layer is `kernel/net/lwip/`: an architecture header, a lock, a clock, an
Ethernet netif, and a set of shim headers standing in for the libc lwIP expects.

### Why lwIP and not a BSD stack

The licence was not the deciding factor — FreeBSD's `sys/netinet` is BSD-3 too.
**It is that FreeBSD's stack is not a library.** `tcp_input.c` alone is 122 KB
and reaches into mbufs, `epoch(9)`, netisr, VNET, `uma(9)`, sockbufs and inpcb
hash tables. Taking it means porting FreeBSD's kernel infrastructure first and
finding the TCP somewhere underneath — the same conclusion already reached about
FreeBSD's USB stack.

NetBSD was ruled out on licence: its `tcp_input.c` and `tcp_output.c` carry the
WIDE Project's 4-clause advertising clause, which Berkeley's rescission does not
reach. See [../licensing.md](../licensing.md) §3.

lwIP was written for exactly the case SchOS is in: a small operating system with
a raw Ethernet driver and no infrastructure to speak of.

**The reason not to write one, kept because it is the reason to resist later
too:** it is not the packet formats that are hard. It is congestion control,
retransmission, window management and twenty years of corner cases that only
appear against a real peer.

## The network thread

`kernel/net/net.c`. One kernel thread, started by `netinit()`, feeds lwIP
received frames and runs its timers.

**A thread rather than the interrupt handler, and the reason is the lock:** a
call into lwIP can reach the driver, the driver can sleep, and an interrupt
handler cannot. So the frame is always *pulled*, by this thread, through
`.receive`.

The interrupt's only job is to end the thread's wait early. `netdev_rx_signal()`
is what a driver calls from its handler; it delivers no frame and reads no ring.
Without it a frame that arrived just after a poll sat there until the next clock
tick — up to 10 ms, and the largest fixed latency in the system.

**The wait stays on the timer tick rather than moving to a channel of its own**,
and that is the design rather than an accident. lwIP's timers must still run when
the link is silent, so the clock has to remain a floor under the wait, and
`sleep()` cannot wait on two things at once. Joining the tick's existing wakeup
costs nothing and leaves one wait with one predicate.

The flag lives under `tickslock` for the same reason it does not get a lock of
its own: `sleep()` releases `tickslock` atomically, so a driver setting the flag
and waking under that lock cannot lose the wakeup by arriving between the
predicate test and the sleep. A private lock would not be held across the sleep,
and that window would be back.

`/proc/netif` reports `wake_rx` and `wake_tick` so this is observable rather
than assumed — if `wake_rx` stays at zero while frames arrive, the interrupt is
not reaching the stack and the thread is running on the clock exactly as it used
to. Measured across three pings: `rx_frames` and `wake_rx` both went 2 → 6.

`netinit()` runs after `sinit()` because `kthread_create` needs a usable process
table, and it returns immediately: everything happens in that thread, which
cannot run until the scheduler is entered.

**The thread follows each device's link into lwIP.** The link used to be read once
at attach, 0.02 s into boot; real autonegotiation takes seconds, so on hardware
the interface stayed down for good. It also starts the stack when an interface
appears after boot (a driver in a process, a USB NIC), and attaches every device
there is as its first act; `net_eth_attach()` refuses one it already has.

DHCP obtains an address at boot. `/proc/netif` reports each interface's address,
netmask, gateway, MAC and frame counts, taken at the lwIP seam.

The lease announcement goes to the **serial line only**, through `klog()` rather
than `cprintf()`. A lease arrives seconds *after* the shell prompt does, so on
the display it lands in the middle of whatever the user is doing — which is the
same rule that makes `cons_serial_only` start set: the display is the product,
the serial line is the diagnostics. Someone who wants to know their address
reads `/proc/netif`.

> It is `/proc/netif` and not `/proc/net` deliberately: on Linux `/proc/net` is a
> directory of many files, and a single file by that name would be a lie about a
> well-known path.

## Sockets

One set of syscalls, at their real Linux numbers. `socknew()` dispatches on the
domain, and **no syscall changes shape** between the two families.

### AF_UNIX

`kernel/net/unix.c`. `SOCK_STREAM`, `socketpair`, `shutdown`.

A connected pair is **two buffers, not one**: writing appends to the peer's
buffer and reading drains your own, which is what makes a socket bidirectional
where a pipe is not, without needing two of anything.

**One lock covers the whole table.** A pipe gets away with a lock per pipe
because every operation touches one object; a socket operation touches two, and
two-object locking needs an ordering rule every path obeys or it deadlocks.
Worse, the wakeup races are subtle — a reader testing `peer_closed` under its own
lock while the closer sets it under the peer's has no interlock at all.

**A bound socket is a file.** `bind()` creates a `T_SOCK` node at the path, and
`connect()` resolves the path and looks for the socket bound to **that inode**.

Matching on the inode rather than on the path *string* is the part that
matters. The name used to live only in this table, so two spellings of one path
missed each other, a relative path never matched an absolute one, and there was
nothing to `stat()` or `unlink()`. An inode is the thing a path resolves *to* —
so `connect()` can walk the path however it likes and still arrive at the same
socket, which is precisely what a string comparison could never do.

```
# ls -l /tmp
srwxr-xr-x 1 root root 0 2026-09-22 08:04 socktest
```

The node is a **marker, not the binding**. It holds no data, and the live
association is in this table. So a socket file left behind by a crash is what
Linux leaves behind: a dead file that `connect()` refuses until something binds
to it again. Conversely, `unlink()`ing a live socket's path does not disturb
anyone already connected — the socket simply stops being findable.

`EADDRINUSE` comes from the *directory*, not from a scan of this table, and
that is deliberate: a stale socket file has to refuse a rebind for the same
reason a running server's does, or two servers both believe they own it. The
consequence is the one every real server already handles — `unlink()` before
`bind()`, which `cmd/socktest` now does too.

The creation lives in `sys_bind()` rather than here, because that is the layer
with both: `kernel/net/unix.c` knows about sockets and has no business calling
`create()`.

`flock()` and `fcntl` record locks exist (`kernel/fs/lock.c`), so the lock files
next to a socket work the way they do on Linux. Names in
the **abstract namespace** (a `sun_path` starting with a NUL) name no file; they
work, and `getsockname` reports them unnamed. `SO_PASSCRED` is honoured, and
`SO_PEERCRED` answers (tmux asks).

### Descriptor passing

`sendmsg`/`recvmsg` with `SCM_RIGHTS`, which is the one thing a socket can carry
that a pipe cannot: a reference to an object the kernel already knows, rather
than a copy of its contents. Wayland's `wl_shm_create_pool` passes a descriptor
and has no integer-shaped alternative, and the same mechanism is how a driver
process is handed the device it is allowed to touch ([devices.md](devices.md)).
Both wanted the same primitive.

**A batch of descriptors is pinned to the stream position of the first byte of
the message that carried it.** That single rule is the whole delivery model, and
it has two halves that are easy to get half-right:

- a reader that arrives at that position is handed the descriptors, and
- a reader that has not reached it yet is **stopped short of it**.

Without the second half, a program that asked for 4096 bytes would swallow three
messages in one call and be handed one message's descriptors, with nothing in
the result saying which message they belonged to. `socktest` checks exactly
this: two messages, each one byte and one descriptor, read with an eight-byte
buffer, and the first read must return one byte.

The kernel holds a reference on every descriptor in flight, so a sender may
close its own copy the moment `sendmsg` returns. That reference is also the
thing that has to be given back, and there are three ways a message ends:

| | |
|---|---|
| received by `recvmsg` with room | installed in the receiver's table |
| received by `recvmsg` without room, or by plain `read(2)` | **closed**, and `MSG_CTRUNC` set for the first case |
| never received — the socket is closed | **closed** by `sockclose` |

The second row is the one that looks like a choice and is not. Holding kernel
references for a receiver that has already proved it is not looking is a leak
with extra steps, and it is what Linux does for the same reason.

Descriptors need at least one byte to ride on; `sendmsg` refuses a message of
pure ancillary data with `EINVAL` rather than accepting it and pinning the
descriptors to a position that never arrives.

**There is no garbage collector.** An AF_UNIX socket descriptor, sent over that
same socket and never received, holds a reference to the socket that holds the
reference to it, and neither close can reach zero. Linux carries several hundred
lines to break these cycles. Nothing passes a socket over a
socket — display servers pass memory, the driver work passes devices — so this is
a known and bounded leak (thirty-two sockets) rather than a silent one.

### AF_INET

`kernel/net/inet.c`, on lwIP's **raw** API rather than lwIP's own sockets layer,
which would need `NO_SYS=0` and a second socket abstraction alongside this
kernel's own.

**Everything is protected by the lwIP lock and nothing else**, and that is not a
shortcut: lwIP's callbacks run in the network thread with that lock held and are
the only writers to a socket's receive buffer. A syscall reading it takes the
same lock. There is no second lock because there is no second writer.

#### How a blocking socket waits

Worth reading before touching, because the obvious implementation does not work.
A blocking call cannot `sleep(&socket, &lwip_lock)`: `sleep()` wants a spinlock
and the lwIP lock is a *sleeplock* — it has to be, because a call into lwIP can
reach the driver. And a waiter cannot hold that lock while sleeping, or the
network thread could never deliver the thing being waited for.

So the wait is on a **counter** with a spinlock of its own:

```
  reader:  lwip_lock; check condition; s = net_seq(); lwip_unlock; net_wait(s)
  waker:   (inside a callback, lwIP lock held) net_wake()
```

No wakeup can be lost. `s` is read while the reader still holds the lwIP lock,
so anything changing the condition must do so afterwards and therefore bumps the
counter after the read; `net_wait` re-checks under its own spinlock and does not
sleep.

One counter and one channel for **all** inet sockets rather than one each. At
this scale waking sixteen sockets costs a handful of instructions in the ones
with nothing to do, and a per-socket channel would need careful teardown in
`close()` against callbacks that can still arrive.

Before this, a blocked socket woke on every timer tick and looked — costing a
wakeup per socket per 10 ms and answering up to 10 ms late even when the data
was already there. `tcp_sent` had to be registered for it as well, or a write
blocked on a full send buffer slept with nobody left to wake it.

#### ICMP, and ping sockets

`socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP)` — Linux's *ping socket*, which gives
an unprivileged process the one thing raw sockets were used for. The caller
reads and writes the ICMP message alone; the kernel owns the IP header.

Built on lwIP's raw API: `raw_new(IP_PROTO_ICMP)`, with the receive callback
stripping the variable-length IP header before queueing the message on the same
datagram queue UDP uses.

**Only echo replies are consumed**, and that restriction is load-bearing.
`raw_input` runs *before* lwIP's own `icmp_input`, so a callback returning
"eaten" for every ICMP message would swallow the echo **requests** too and this
machine would silently stop answering pings from outside. Returning 0 leaves the
packet for lwIP, which is what should happen to every ICMP message nobody asked
for.

There is no per-socket echo-id filtering. Linux rewrites the id and matches on
it; here a reply goes to every open ICMP socket, which is invisible with one
ping running and means a second one may see a sequence number it never sent.

### AF_NETLINK, and why it exists

`getifaddrs(3)` is how every program asks *what interfaces does this machine
have, and what addresses do they carry* — and on Linux there is exactly one
answer: a `NETLINK_ROUTE` socket. musl implements the libc call over it, so what
looks like a syscall is really a small protocol conversation, and without this
side of it every caller fails identically.

`kernel/net/netlink.c` is a **responder**, not a general socket family. There is
no kernel-to-userspace event stream, no multicast groups, nothing subscribes to
link state. A request arrives, the whole answer is generated into the socket's
buffer, and reads drain it.

That shape is deliberate. `getifaddrs` sends a dump request and reads until
`NLMSG_DONE` with `MSG_DONTWAIT`, so generating the answer at send time means a
read never blocks and the non-blocking flag is honoured for free.

Two message types are served — `RTM_GETLINK` returns one `RTM_NEWLINK` per
interface (name, flags, MAC), `RTM_GETADDR` returns one `RTM_NEWADDR` per
address (address, prefix length, label). Anything else gets a bare `NLMSG_DONE`.

**The data is not netlink's.** Interfaces come from the netdev class and
addresses from lwIP's netifs, exactly as `/proc/netif` reads them. Nothing in
the file knows what an e1000 is.

Interface indices matter more than they look: `getifaddrs` matches address
records to link records *by index*, so the two dumps have to agree. Loopback is
1 and the netdevs follow from 2.

What it produces, read back through musl:

```
lo     flags=0x49    fam=-1
eth0   flags=0x1043  fam=17                    (AF_PACKET, the MAC)
lo     flags=0x49    fam=2  addr=127.0.0.1  mask=255.0.0.0
eth0   flags=0x1043  fam=2  addr=10.0.2.15  mask=255.255.255.0
```

### Loopback

`127.0.0.1` is a real destination: `LWIP_HAVE_LOOPIF` creates the interface and
`lwip_init()` brings it up. `LWIP_NETIF_LOOPBACK` is the half that is easy to
miss — without it, a packet addressed to an interface's *own* address goes out
of the wire and never comes back.

With `NO_SYS=1` there is no thread to drain the loopback queue, so the network
thread calls `netif_poll_all()` alongside `sys_check_timeouts()`.

### Datagrams

UDP has a **datagram queue**, not a byte ring. It shared TCP's ring at first, and
two datagrams arriving together came back as one — which is the bug that had to
be fixed before `recvfrom` could report a source address at all.

---

## Names

For a long time nothing on this system could turn a name into an address, and
the shape of that failure is worth keeping: it did not look like one missing
feature. `wget` did not work. `telnet` did not work. Anything taking a hostname
did not work, each in its own way, and each looked like a separate broken
program — which is a good argument for a userland with fewer applets, and a
wrong one.

**There is still no DNS client in SchOS, and there does not need to be.** musl
contains a complete resolver: `getaddrinfo` reads `/etc/resolv.conf`, sends a
query over an ordinary UDP socket and parses the reply. Every one of those
pieces already worked. The single missing fact was *which nameserver* — and
that arrives in a DHCP reply the kernel handled and never told anyone about.

So the mechanism is one file:

```
# cat /proc/resolv.conf
nameserver 10.0.2.3
```

`net_resolv()` renders **resolv.conf's exact syntax**, so that putting it where
the resolver looks is a copy and not a translation. It is empty until DHCP
answers, which is the honest report: a file saying nothing is right, and one
naming a server that does not exist is not.

The policy is `cmd/resolvconf`, run once at boot by a `schinit` job. It waits
for a nameserver, writes `/etc/resolv.conf` and exits — and **will not
overwrite an existing one** without `-f`, because a machine with a hand-written
`resolv.conf` has been configured on purpose.

That division is the usual one here: the kernel knows a fact a process cannot
find out for itself and publishes it; whether that fact should become
`/etc/resolv.conf` is not the kernel's business.

```
# wget -O - http://example.com/ | head -1
<!doctype html><html lang="en"><head><title>Example Domain</title>
```

## Serving

`cmd/httpd` is the other direction, and it existed for the same reason this
section does: a machine that could fetch a page but not serve one is a strange
shape for something with a TCP stack and a filesystem.

GET and HEAD, static files out of a directory, `index.html` for a directory and
a generated listing when there is none. A child per connection, so one client
that stops reading cannot stop the server. No CGI, no keep-alive, no TLS, no
configuration file — it is a *file server*, and each of those is a decision
somebody should make on purpose.

**The path check is the only security-relevant code in it**, and it refuses
rather than normalises. The tempting version resolves `..`, collapses slashes
and then checks the result — and it has to agree exactly with the kernel's
resolver, and where it does not, the difference is a file outside the root. So
no component may be `..` at all. Percent-decoding happens first, because
`%2e%2e` is `..` and a check that runs before decoding does not see it.

```
$ curl -s -o /dev/null -w '%{http_code}\n' --path-as-is host/%2e%2e/etc/passwd
403
```

### A listening socket is a different object

Worth its own note, because it cost a kernel panic and the shape of it is not
obvious from the API.

`tcp_listen()` does not put a pcb into a listening state. It **frees the
connection pcb and returns a new, smaller one** — a `struct tcp_pcb_listen`,
which has no `recv`, `sent`, `err` or `poll` field at all. The two are
different types behind one `struct tcp_pcb *`, and the only callback a
listening one has is `accept`.

`inet_close()` did not know that, and cleared the callbacks the same way for
both. On a listening socket that is a write past the end of the allocation;
lwIP asserts rather than allowing it, and an assert here is a panic:

```
lwip assert: invalid socket state for recv callback at tcp.c:2024
  #1 tcp_recv   #2 inet_close   #3 sockclose
  #4 fileclose  #5 exit         #6 syscall
```

Which is **a user process taking the machine down by pressing Ctrl-C** — the
close path runs from `exit()`, so any program that listened and then died did
it. Nothing had, because until `cmd/httpd` nothing in the tree listened on TCP
and exited; `schdbgd` would have done it too, a few days later.

The lesson is not about lwIP. It is that `listening` was already tracked one
field away in `struct inet_sock` and the close path did not consult it — the
information was there and the code did not ask.

## What it does not do

- **`ping` is a first-party program**, `cmd/ping`, not toybox's or busybox's. Ours
  has no `-I`, no flood mode and no TTL display (`IP_RECVTTL` is an `IPPROTO_IP`
  control message that would have to be generated on the receive path), and takes a
  dotted quad or `localhost` — it does not link musl's resolver. Busybox's `ping`,
  from the Alpine root, works over a `SOCK_RAW` ICMP socket.
- **Only `SCM_RIGHTS` and `SCM_CREDENTIALS` travel in a control message.** Anything
  else is refused rather than ignored.
- **A message may carry eight descriptors, and a socket eight such messages.**
  Past that `sendmsg` answers `ENOBUFS`. The cap exists because the kernel holds
  a reference to each one until it is received; a sender eight unreceived
  messages ahead of its receiver is not rescued by a ninth slot.
- **Nothing re-runs `resolvconf`.** The nameserver is written once at boot
  (`etc/schinit/resolvconf.job`). A lease that changes the DNS server, or a
  `/dev/pci` handover that brings up a new interface, leaves a stale
  `/etc/resolv.conf` until somebody runs it again.
- **lwIP's own DNS client is unused.** Resolution happens entirely in the process
  (musl); nothing in the kernel can resolve a name.
- **`httpd` serves HTTP/1.0** with `Connection: close`, one request per connection,
  and has no timeouts: a client that connects and says nothing ties up a child
  process until it goes away, and `SO_RCVTIMEO` does not exist here.
- **A queued-but-never-accepted connection is closed without clearing its
  callbacks.** `inet_close()` clears the argument and closes, which is enough
  because the pcb is freed, but it is a narrower guarantee than the main path
  makes and relies on `tcp_close()` succeeding.
- **No routing table**, and no `/proc/net/route`. With two interfaces up, the first
  is the default route and the second is reachable only by an address on its own
  subnet. That is a rule rather than an accident, and it is not routing.
- **Netlink is a responder**: `RTM_GETLINK` and `RTM_GETADDR` dumps (what
  `getifaddrs` needs), `MSG_PEEK|MSG_TRUNC` sizing (GLib's network monitor measures
  each datagram that way and spun at 100% when it answered 0), and nothing else. No
  `RTM_GETROUTE`, no event stream, no multicast groups; a program that expects to
  *subscribe* to link changes gets a socket that answers dumps and never speaks
  again.
- **`setsockopt` applies what it understands** (`SO_REUSEADDR`, `SO_KEEPALIVE`,
  `SO_BROADCAST`, `TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` report the
  real size; `SO_PASSCRED` on Unix sockets) and refuses the rest with `ENOPROTOOPT`.
  Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`.
- **`SOCK_DGRAM` over AF_UNIX does not exist** — stream only.
- **No IPv6**, no firewall or packet filter of any kind, and a TCP stack that has
  been used for downloads at tens of megabits and no more than that.
- **Three drivers in the kernel, one Linux driver outside it**, and the class is
  proven with two e1000 cards and an LDLS handover, not on a wide range of
  hardware. The Realtek parts have never carried a packet on real silicon from this
  tree.