docs: four directories, and a rule for when a thing leaves the roadmap
docs/ was ten files where a plan, a description, a guide and a bug report all
sat next to each other. It is now architecture/ (what SchOS is), plan/ (what it
will be), guides/ (how to work on it) and history/ (why it is like this), with
an index that states the one rule holding it together: when something is
finished it moves from plan/ to architecture/ and is rewritten as a description
of the thing rather than a record of building it.
A ticked checkbox says the work happened. It does not say what the work
*produced*, and six months later the checkbox is all that is left. The network
stack proved the point: finished, verified end to end, and for a month the only
description of it anywhere was a checkbox four levels of indentation deep.
So roadmap.md went from 534 lines to 181 -- nothing deleted, the done work
graduated. Seven subsystems that were finished and undescribed now have
descriptions: boot, kernel, syscalls, filesystem, networking, graphics, usb,
init. Every one ends with "What it does not do", which is the section that pays;
a document listing only strengths is marketing, and the reader who needs it most
is the one about to trip over a limitation nobody wrote down.
schfs.md still opened with "phases 1 through 3 1/2 are done, phase 4 is next"
long after all five had landed. It is a description now, with the phase record
and the hard-won bugs moved to history/.
Three corrections found on the way, all load-bearing:
* projects/schcom/README.md claimed SCM_RIGHTS was done. There is no
sendmsg/recvmsg in the kernel at all, and an entire GUI plan rested on it.
* The root README said "No TCP/IP" months after the stack landed.
* history/kernel.md sec.7 was marked "in progress" on a problem that process
groups solved.
Also adds plan/1.0.md, which draws a finish line so the open list stops being
infinite, and marks schwm/ and schde/ superseded: in Wayland the compositor is
the window manager, so that split dissolves rather than being abandoned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz
parent
817d666068
commit
57aff40228
59 changed files with 3981 additions and 1124 deletions
260
docs/architecture/networking.md
Normal file
260
docs/architecture/networking.md
Normal file
|
|
@ -0,0 +1,260 @@
|
|||
# Networking
|
||||
|
||||
ARP, IPv4, ICMP, UDP, TCP and DHCP, over a driver-agnostic interface class,
|
||||
reached through the same socket syscalls as Unix domain sockets.
|
||||
|
||||
Verified end to end rather than by inspection: `bootcheck -Net` starts a
|
||||
listener and a UDP responder on the host, tells the guest to talk to them, and
|
||||
asserts on **both** ends — one side alone cannot tell a working stack from one
|
||||
that completes a handshake and drops every byte.
|
||||
|
||||
---
|
||||
|
||||
## The four layers
|
||||
|
||||
```
|
||||
socket() bind() connect() send() recv() syscalls, one set
|
||||
┌────────────────┴───────────────────┐
|
||||
│ AF_UNIX AF_INET │ socknew() dispatches on domain
|
||||
│ kernel/net/unix.c kernel/net/inet.c
|
||||
└────────────────┬───────────────────┘
|
||||
│ lwIP raw API
|
||||
┌────────────────┴───────────────────┐
|
||||
│ lwIP 2.2.0, NO_SYS=1 │ ports/lwip, compiled into the kernel
|
||||
│ port layer: kernel/net/lwip/ │ ~400 lines: a lock, a clock, a netif
|
||||
└────────────────┬───────────────────┘
|
||||
│ struct netdev_ops
|
||||
┌────────────────┴───────────────────┐
|
||||
│ e1000, and any future NIC driver │
|
||||
└────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## The netdev class
|
||||
|
||||
A driver publishes a [`struct netdev_ops`](../../include/netdev.h) and is then
|
||||
finished with the network stack forever: it never mentions lwIP, and the stack
|
||||
never mentions it.
|
||||
|
||||
```c
|
||||
struct netdev_ops {
|
||||
int (*transmit)(struct device *, const void *frame, int len);
|
||||
int (*receive) (struct device *, void *out, int cap);
|
||||
const unsigned char *(*mac)(struct device *);
|
||||
int (*link_up) (struct device *); // optional
|
||||
};
|
||||
```
|
||||
|
||||
`transmit` returning -1 is **congestion, not failure** — the ring is full and
|
||||
the caller is expected to retry. `receive` never blocks; it returns 0 when
|
||||
nothing is waiting, and the caller decides how to wait.
|
||||
|
||||
This class was added late, and the reason it had to be is instructive: the stack
|
||||
had reached straight into the driver. `net.c` called `e1000_present()`, the lwIP
|
||||
glue was named `netif_e1000.c`, and the e1000 shipped with `.ops = 0` and a
|
||||
comment saying so. A second NIC driver would have meant editing the protocol
|
||||
stack to add itself. lwIP now gets one netif per bound device, `/proc/netif`
|
||||
lists them all, and the e1000 drives up to four cards rather than refusing the
|
||||
second — verified with two attached, each with its own MAC and its own DHCP
|
||||
lease.
|
||||
|
||||
### `netdev_selftest()`
|
||||
|
||||
`echo n > /proc/sysrq` sends a hand-built ARP request and waits for the reply.
|
||||
|
||||
It proves the whole driver path in one exchange — descriptor formats, the DD
|
||||
handshake, tail bookkeeping, DMA addresses, the receive ring hand-back, and the
|
||||
MAC filter, because an ARP reply is addressed to this station and does not
|
||||
arrive if the filter is wrong.
|
||||
|
||||
It was written *before* there was any protocol stack, deliberately: a stack on
|
||||
an unproven driver fails in ways indistinguishable from its own bugs. It belongs
|
||||
to the class rather than to the e1000, because it is the check every new NIC
|
||||
driver will want first.
|
||||
|
||||
## lwIP
|
||||
|
||||
lwIP 2.2.0, fetched by `ports/lwip` and **compiled into the kernel** — it is the
|
||||
one port that installs nothing into the sysroot, because it is kernel code.
|
||||
|
||||
`NO_SYS=1`, so lwIP has no threads, mailboxes or semaphores of its own. The port
|
||||
layer is `kernel/net/lwip/`: an architecture header, a lock, a clock, an
|
||||
Ethernet netif, and a set of shim headers standing in for the libc lwIP expects.
|
||||
|
||||
### Why lwIP and not a BSD stack
|
||||
|
||||
The licence was not the deciding factor — FreeBSD's `sys/netinet` is BSD-3 too.
|
||||
**It is that FreeBSD's stack is not a library.** `tcp_input.c` alone is 122 KB
|
||||
and reaches into mbufs, `epoch(9)`, netisr, VNET, `uma(9)`, sockbufs and inpcb
|
||||
hash tables. Taking it means porting FreeBSD's kernel infrastructure first and
|
||||
finding the TCP somewhere underneath — the same conclusion already reached about
|
||||
FreeBSD's USB stack.
|
||||
|
||||
NetBSD was ruled out on licence: its `tcp_input.c` and `tcp_output.c` carry the
|
||||
WIDE Project's 4-clause advertising clause, which Berkeley's rescission does not
|
||||
reach. See [../licensing.md](../licensing.md) §3.
|
||||
|
||||
lwIP was written for exactly the case SchOS is in: a small operating system with
|
||||
a raw Ethernet driver and no infrastructure to speak of.
|
||||
|
||||
**The reason not to write one, kept because it is the reason to resist later
|
||||
too:** it is not the packet formats that are hard. It is congestion control,
|
||||
retransmission, window management and twenty years of corner cases that only
|
||||
appear against a real peer.
|
||||
|
||||
## The network thread
|
||||
|
||||
`kernel/net/net.c`. One kernel thread, started by `netinit()`, feeds lwIP
|
||||
received frames and runs its timers.
|
||||
|
||||
**A thread rather than the interrupt handler, and the reason is the lock:** a
|
||||
call into lwIP can reach the driver, the driver can sleep, and an interrupt
|
||||
handler cannot. So the interrupt keeps doing what it always did — clear the
|
||||
cause register — and this thread polls.
|
||||
|
||||
`netinit()` runs after `sinit()` because `kthread_create` needs a usable process
|
||||
table, and it returns immediately: everything happens in that thread, which
|
||||
cannot run until the scheduler is entered.
|
||||
|
||||
DHCP obtains an address at boot. `/proc/netif` reports each interface's address,
|
||||
netmask, gateway, MAC and frame counts, taken at the lwIP seam.
|
||||
|
||||
> It is `/proc/netif` and not `/proc/net` deliberately: on Linux `/proc/net` is a
|
||||
> directory of many files, and a single file by that name would be a lie about a
|
||||
> well-known path.
|
||||
|
||||
## Sockets
|
||||
|
||||
One set of syscalls, at their real Linux numbers. `socknew()` dispatches on the
|
||||
domain, and **no syscall changes shape** between the two families.
|
||||
|
||||
### AF_UNIX
|
||||
|
||||
`kernel/net/unix.c`. `SOCK_STREAM`, `socketpair`, `shutdown`.
|
||||
|
||||
A connected pair is **two buffers, not one**: writing appends to the peer's
|
||||
buffer and reading drains your own, which is what makes a socket bidirectional
|
||||
where a pipe is not, without needing two of anything.
|
||||
|
||||
**One lock covers the whole table.** A pipe gets away with a lock per pipe
|
||||
because every operation touches one object; a socket operation touches two, and
|
||||
two-object locking needs an ordering rule every path obeys or it deadlocks.
|
||||
Worse, the wakeup races are subtle — a reader testing `peer_closed` under its own
|
||||
lock while the closer sets it under the peer's has no interlock at all.
|
||||
|
||||
**Names live in a table here, not in the filesystem.** Binding creates no
|
||||
directory entry. The table gives the behaviour that matters — a client finds a
|
||||
server by a name agreed in advance — and what is lost is `ls -l` showing an `s`,
|
||||
and `unlink()` of a socket path working. Both want a VFS change rather than a
|
||||
socket change.
|
||||
|
||||
### AF_INET
|
||||
|
||||
`kernel/net/inet.c`, on lwIP's **raw** API rather than lwIP's own sockets layer,
|
||||
which would need `NO_SYS=0` and a second socket abstraction alongside this
|
||||
kernel's own.
|
||||
|
||||
**Everything is protected by the lwIP lock and nothing else**, and that is not a
|
||||
shortcut: lwIP's callbacks run in the network thread with that lock held and are
|
||||
the only writers to a socket's receive buffer. A syscall reading it takes the
|
||||
same lock. There is no second lock because there is no second writer.
|
||||
|
||||
#### How a blocking socket waits
|
||||
|
||||
Worth reading before touching, because the obvious implementation does not work.
|
||||
A blocking call cannot `sleep(&socket, &lwip_lock)`: `sleep()` wants a spinlock
|
||||
and the lwIP lock is a *sleeplock* — it has to be, because a call into lwIP can
|
||||
reach the driver. And a waiter cannot hold that lock while sleeping, or the
|
||||
network thread could never deliver the thing being waited for.
|
||||
|
||||
So the wait is on a **counter** with a spinlock of its own:
|
||||
|
||||
```
|
||||
reader: lwip_lock; check condition; s = net_seq(); lwip_unlock; net_wait(s)
|
||||
waker: (inside a callback, lwIP lock held) net_wake()
|
||||
```
|
||||
|
||||
No wakeup can be lost. `s` is read while the reader still holds the lwIP lock,
|
||||
so anything changing the condition must do so afterwards and therefore bumps the
|
||||
counter after the read; `net_wait` re-checks under its own spinlock and does not
|
||||
sleep.
|
||||
|
||||
One counter and one channel for **all** inet sockets rather than one each. At
|
||||
this scale waking sixteen sockets costs a handful of instructions in the ones
|
||||
with nothing to do, and a per-socket channel would need careful teardown in
|
||||
`close()` against callbacks that can still arrive.
|
||||
|
||||
Before this, a blocked socket woke on every timer tick and looked — costing a
|
||||
wakeup per socket per 10 ms and answering up to 10 ms late even when the data
|
||||
was already there. `tcp_sent` had to be registered for it as well, or a write
|
||||
blocked on a full send buffer slept with nobody left to wake it.
|
||||
|
||||
#### ICMP, and ping sockets
|
||||
|
||||
`socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP)` — Linux's *ping socket*, which gives
|
||||
an unprivileged process the one thing raw sockets were used for. The caller
|
||||
reads and writes the ICMP message alone; the kernel owns the IP header.
|
||||
|
||||
Built on lwIP's raw API: `raw_new(IP_PROTO_ICMP)`, with the receive callback
|
||||
stripping the variable-length IP header before queueing the message on the same
|
||||
datagram queue UDP uses.
|
||||
|
||||
**Only echo replies are consumed**, and that restriction is load-bearing.
|
||||
`raw_input` runs *before* lwIP's own `icmp_input`, so a callback returning
|
||||
"eaten" for every ICMP message would swallow the echo **requests** too and this
|
||||
machine would silently stop answering pings from outside. Returning 0 leaves the
|
||||
packet for lwIP, which is what should happen to every ICMP message nobody asked
|
||||
for.
|
||||
|
||||
There is no per-socket echo-id filtering. Linux rewrites the id and matches on
|
||||
it; here a reply goes to every open ICMP socket, which is invisible with one
|
||||
ping running and means a second one may see a sequence number it never sent.
|
||||
|
||||
### Loopback
|
||||
|
||||
`127.0.0.1` is a real destination: `LWIP_HAVE_LOOPIF` creates the interface and
|
||||
`lwip_init()` brings it up. `LWIP_NETIF_LOOPBACK` is the half that is easy to
|
||||
miss — without it, a packet addressed to an interface's *own* address goes out
|
||||
of the wire and never comes back.
|
||||
|
||||
With `NO_SYS=1` there is no thread to drain the loopback queue, so the network
|
||||
thread calls `netif_poll_all()` alongside `sys_check_timeouts()`.
|
||||
|
||||
### Datagrams
|
||||
|
||||
UDP has a **datagram queue**, not a byte ring. It shared TCP's ring at first, and
|
||||
two datagrams arriving together came back as one — which is the bug that had to
|
||||
be fixed before `recvfrom` could report a source address at all.
|
||||
|
||||
---
|
||||
|
||||
## What it does not do
|
||||
|
||||
- **`ping` is a first-party program**, `cmd/ping`, not toybox's. toybox's reads
|
||||
replies with `recvmsg(2)` and asks for the TTL through an `IP_RECVTTL` control
|
||||
message, and neither exists — `recvmsg` is item 1 of
|
||||
[../plan/foundations.md](../plan/foundations.md), where it is wanted for
|
||||
`SCM_RIGHTS` anyway. Ours has no `-I`, no flood mode and no TTL display.
|
||||
- **No resolver reachable from userland.** lwIP's DNS client is compiled but
|
||||
nothing exposes it, so `ping` takes a dotted quad. `localhost` is special-cased
|
||||
in the program rather than resolved.
|
||||
- **Receive is polled.** The network thread looks at the NIC every timer tick
|
||||
when the link is idle, so an arriving frame waits up to 10 ms. Waking the
|
||||
thread from the e1000 interrupt is the real fix, and was deliberately not the
|
||||
first version: a receive path woken by an interrupt is much harder to debug
|
||||
than one that is asked.
|
||||
- **No routing table.** With two interfaces up, the first is the default route
|
||||
and the second is reachable only by an address on its own subnet. That is a
|
||||
rule rather than an accident, and it is not routing.
|
||||
- **No `getsockname`/`getpeername`**, and `accept()` does not report the peer's
|
||||
address for an AF_INET socket. A server that wants to log who connected cannot.
|
||||
- **`setsockopt` covers six options**, and refuses the rest with `ENOPROTOOPT`
|
||||
rather than pretending: `SO_REUSEADDR`, `SO_KEEPALIVE`, `SO_BROADCAST` and
|
||||
`TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` are accepted and
|
||||
report the real size on read. Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`.
|
||||
- **`SOCK_DGRAM` over AF_UNIX does not exist** — stream only.
|
||||
- **AF_UNIX names are not filesystem paths**, so nothing can `stat()` or
|
||||
`unlink()` a socket.
|
||||
- **One NIC driver exists.** The class is proven with two e1000 cards, not with
|
||||
two different drivers.
|
||||
- **No IPv6**, no DNS resolver in the kernel (lwIP's is compiled but unused by
|
||||
anything), and no firewall or packet filter of any kind.
|
||||
Loading…
Reference in a new issue