userland: 268 applets, an editor, names that resolve, and a web server
Four complaints, and only one of them was the thing it looked like. **"toybox is too small."** Half true, and the fix cost no licensing at all: 236 of toybox's applets are what defconfig builds, and the rest live in toys/pending/ -- awk, bc, diff, more, expr, tr, hexdump, stty among them. Same 0BSD tarball, switched off by a config default. 26 of them are on now, 236 -> 268. Not everything: strace wants ptrace, syslogd wants BSD's facilitynames[], git wants OpenSSL, and those three break the link. The toybox build also works again on this machine. Its scripts start #!/bin/bash, which does not exist on NixOS, so the binary in the tree was one nobody could reproduce; the shebangs are rewritten to /usr/bin/env bash on the way in. Pending applets are enabled by editing .config directly, because `make oldconfig` is the obvious way and undoes it -- every pending symbol is `default n`. **BusyBox was asked for and is not here.** It is GPL-2.0-only, and `make ramroot` links the root image *into the kernel binary*, so it would put GPLv2 inside an Apache-2.0 executable -- which is the exact case ports/Makefile's NOTPORTS filter and docs/licensing.md §4 exist for. nano is the same answer one version later. Both recorded in licensing.md with what was done instead, since "no" without an alternative is how a rule gets quietly broken later. **An editor that is not vi.** kilo, BSD-2-Clause, one 1300-line file, no dependencies beyond termios and TIOCGWINSZ, both of which zsh already needed. Installed as /bin/edit as well as /bin/kilo, because somebody with a file to fix will type `edit`. **"wget doesn't work."** Not a userland problem, and BusyBox's wget would have failed identically: nothing on this system could turn a name into an address, so every program taking a hostname failed in its own way and each looked like a separate bug. There is still no DNS client in SchOS and there does not need to be -- musl has a complete resolver and it works over ordinary UDP sockets. The one missing fact was which nameserver, which the kernel has had all along from DHCP and never published. /proc/resolv.conf publishes it in resolv.conf's exact syntax; cmd/resolvconf decides whether it becomes /etc/resolv.conf, and will not overwrite a hand-written one without -f. # wget -O - http://example.com/ | head -1 <!doctype html><html lang="en"><head><title>Example Domain</title> **Something to host with.** cmd/httpd: GET and HEAD, static files, index.html for a directory and a listing otherwise, a child per connection. No CGI, no keep-alive, no TLS, no config file. The path check is the only security-relevant code in it and it refuses rather than normalises -- no component may be `..`, and percent-decoding happens first, because %2e%2e is `..` and a check before decoding does not see it. Both forms answer 403. And the disk is 100 MB, from 32. It is sparse, so the image is 21 MB on the host. RAMDISK_BLOCKS went 6000 -> 8000 with it: the ramdisk was at 86% and the next thing added would have broken `make ramroot`, which nobody would have noticed until they used it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
parent
bc39b73969
commit
b08432b2c3
17 changed files with 994 additions and 21 deletions
|
|
@ -29,7 +29,7 @@ each with its own screen, line discipline and foreground process group. See
|
|||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ toybox mksh zsh musl first-party cmd/ │ userland
|
||||
│ toybox kilo mksh zsh musl first-party cmd/ │ userland
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ vgpud — a driver that is an ordinary process │
|
||||
├─────────────────────────────────────────────────┤
|
||||
|
|
|
|||
|
|
@ -341,6 +341,71 @@ be fixed before `recvfrom` could report a source address at all.
|
|||
|
||||
---
|
||||
|
||||
## Names
|
||||
|
||||
For a long time nothing on this system could turn a name into an address, and
|
||||
the shape of that failure is worth keeping: it did not look like one missing
|
||||
feature. `wget` did not work. `telnet` did not work. Anything taking a hostname
|
||||
did not work, each in its own way, and each looked like a separate broken
|
||||
program — which is a good argument for a userland with fewer applets, and a
|
||||
wrong one.
|
||||
|
||||
**There is still no DNS client in SchOS, and there does not need to be.** musl
|
||||
contains a complete resolver: `getaddrinfo` reads `/etc/resolv.conf`, sends a
|
||||
query over an ordinary UDP socket and parses the reply. Every one of those
|
||||
pieces already worked. The single missing fact was *which nameserver* — and
|
||||
that arrives in a DHCP reply the kernel handled and never told anyone about.
|
||||
|
||||
So the mechanism is one file:
|
||||
|
||||
```
|
||||
# cat /proc/resolv.conf
|
||||
nameserver 10.0.2.3
|
||||
```
|
||||
|
||||
`net_resolv()` renders **resolv.conf's exact syntax**, so that putting it where
|
||||
the resolver looks is a copy and not a translation. It is empty until DHCP
|
||||
answers, which is the honest report: a file saying nothing is right, and one
|
||||
naming a server that does not exist is not.
|
||||
|
||||
The policy is `cmd/resolvconf`, run once at boot by a `schinit` job. It waits
|
||||
for a nameserver, writes `/etc/resolv.conf` and exits — and **will not
|
||||
overwrite an existing one** without `-f`, because a machine with a hand-written
|
||||
`resolv.conf` has been configured on purpose.
|
||||
|
||||
That division is the usual one here: the kernel knows a fact a process cannot
|
||||
find out for itself and publishes it; whether that fact should become
|
||||
`/etc/resolv.conf` is not the kernel's business.
|
||||
|
||||
```
|
||||
# wget -O - http://example.com/ | head -1
|
||||
<!doctype html><html lang="en"><head><title>Example Domain</title>
|
||||
```
|
||||
|
||||
## Serving
|
||||
|
||||
`cmd/httpd` is the other direction, and it existed for the same reason this
|
||||
section does: a machine that could fetch a page but not serve one is a strange
|
||||
shape for something with a TCP stack and a filesystem.
|
||||
|
||||
GET and HEAD, static files out of a directory, `index.html` for a directory and
|
||||
a generated listing when there is none. A child per connection, so one client
|
||||
that stops reading cannot stop the server. No CGI, no keep-alive, no TLS, no
|
||||
configuration file — it is a *file server*, and each of those is a decision
|
||||
somebody should make on purpose.
|
||||
|
||||
**The path check is the only security-relevant code in it**, and it refuses
|
||||
rather than normalises. The tempting version resolves `..`, collapses slashes
|
||||
and then checks the result — and it has to agree exactly with the kernel's
|
||||
resolver, and where it does not, the difference is a file outside the root. So
|
||||
no component may be `..` at all. Percent-decoding happens first, because
|
||||
`%2e%2e` is `..` and a check that runs before decoding does not see it.
|
||||
|
||||
```
|
||||
$ curl -s -o /dev/null -w '%{http_code}\n' --path-as-is host/%2e%2e/etc/passwd
|
||||
403
|
||||
```
|
||||
|
||||
## What it does not do
|
||||
|
||||
- **`ping` is a first-party program**, `cmd/ping`, not toybox's. `recvmsg` now
|
||||
|
|
@ -355,9 +420,24 @@ be fixed before `recvfrom` could report a source address at all.
|
|||
Past that `sendmsg` answers `ENOBUFS`. The cap exists because the kernel holds
|
||||
a reference to each one until it is received; a sender eight unreceived
|
||||
messages ahead of its receiver is not rescued by a ninth slot.
|
||||
- **No resolver reachable from userland.** lwIP's DNS client is compiled but
|
||||
nothing exposes it, so `ping` takes a dotted quad. `localhost` is special-cased
|
||||
in the program rather than resolved.
|
||||
- **`ping` still takes a dotted quad**, even though names now work. It is a
|
||||
first-party program that does not link musl's resolver, and `localhost` is
|
||||
special-cased in it rather than resolved. Every *ported* program resolves
|
||||
names properly; ours is the one that does not.
|
||||
- **Nothing re-runs `resolvconf`.** The nameserver is written once at boot. A
|
||||
lease that changes the DNS server, or a `/dev/pci` handover that brings up a
|
||||
new interface, leaves a stale `/etc/resolv.conf` until somebody runs it again.
|
||||
- **lwIP's own DNS client is still unused.** It is compiled in and only
|
||||
`dns_getserver()` is ever called; resolution happens entirely in the process.
|
||||
Nothing in the kernel can resolve a name, which matters the day something in
|
||||
the kernel wants to.
|
||||
- **`httpd` does not serve HTTP/1.1.** Every response is HTTP/1.0 with
|
||||
`Connection: close`, because the directory listing is generated straight into
|
||||
the socket and so has no `Content-Length` to give. One request per connection
|
||||
is the cost.
|
||||
- **`httpd` has no timeouts.** A client that connects and says nothing ties up
|
||||
a child process until it goes away; `SO_RCVTIMEO` does not exist here (see
|
||||
below), so there is nothing to set.
|
||||
- **No routing table**, and no `/proc/net/route` to report one. With two
|
||||
interfaces up, the first is the default route and the second is reachable only
|
||||
by an address on its own subnet. That is a rule rather than an accident, and it
|
||||
|
|
|
|||
|
|
@ -212,6 +212,43 @@ with SchOS. Keeping the distinction sharp is what stops it eroding:
|
|||
rebuild never needs the network; not part of any image, and not all of it is
|
||||
redistributable.
|
||||
|
||||
### The userland this rules out, and what was done instead
|
||||
|
||||
The rule costs something real and it is worth naming the bill rather than only
|
||||
the principle.
|
||||
|
||||
**BusyBox** is the obvious userland for a system this size — about four hundred
|
||||
applets, including a shell, an editor, `less` and an `httpd`. It is
|
||||
**GPL-2.0-only**, so §2 applies, and here the consequence is sharper than
|
||||
"cannot be linked": `make ramroot` links the root filesystem image *into the
|
||||
kernel binary*, so a BusyBox in `sysroot/` would put GPLv2 code inside an
|
||||
Apache-2.0 executable. Excluding it from the ramdisk avoids that and leaves the
|
||||
ramroot build with no userland at all, which is not a trade worth making.
|
||||
|
||||
**nano** is the same answer for the same reason, one version later: GPL-3.0,
|
||||
which is incompatible with Apache-2.0 in the other direction.
|
||||
|
||||
What was done instead, when the complaint "the userland is too thin" came up:
|
||||
|
||||
| wanted | answer |
|
||||
|---|---|
|
||||
| `awk`, `bc`, `diff`, `more`, `expr`, `tr`, … | already in toybox, in `toys/pending/`, switched off by `defconfig`. Turning 26 of them on took the shipped set from 236 applets to 268, at no licensing cost — it is the same 0BSD tarball. |
|
||||
| an editor that is not `vi` | [kilo](https://github.com/antirez/kilo), BSD-2-Clause, one file, no dependencies. Installed as `/bin/edit` as well as `/bin/kilo`. |
|
||||
| `wget` that works | not a userland problem at all. See below. |
|
||||
| something to host with | `cmd/httpd`, written here, Apache-2.0. |
|
||||
|
||||
The `wget` one is worth recording because it was diagnosed wrongly at first,
|
||||
and switching userland would not have fixed it: **SchOS had no DNS reachable
|
||||
from a process**, so every program taking a hostname failed and each looked
|
||||
like its own bug. BusyBox's `wget` would have failed identically. musl already
|
||||
contains a complete resolver; the only missing fact was which nameserver, which
|
||||
the kernel had all along from DHCP and never published. See
|
||||
[architecture/networking.md](architecture/networking.md).
|
||||
|
||||
The general shape: **a permissive answer existed for every one of these**, and
|
||||
three of the four were already in the tree. That will not always be true, and
|
||||
when it is not the answer is still no.
|
||||
|
||||
The mechanisms that keep this from eroding, since good intentions have already
|
||||
failed once here:
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue