d3c37f82b6
The site and several documents still described the system as it was before virtual terminals, the new schinit, the hybrid work and descriptor passing. docs/architecture/init.md described the wrong program. It documented a single schinit that parses manifests and supervises jobs, which has not been true since PID 1 was split: PID 1 forks a supervisor, spawns jobs on its behalf and reaps; the job table lives in /proc/schinit; a heartbeat runs both ways; and proc_reincarnate() re-execs PID 1 in place rather than panicking. Rewritten around what exists, keeping the reasoning that is still load-bearing -- the five failures of the sibling-init scheme, the escalation ladder, and the three bugs in the reincarnation path that each had to happen before they were obvious. docs/plan/init.md is deleted, which is the tree's own rule for finished work and had not been applied. Fourteen source comments referenced it by part number, so they are repointed at the architecture doc by section name rather than left dangling -- a deleted document nobody notices is worse than a stale one. docs/architecture/devices.md gained the section it has been missing since /dev/pci existed: what a driver needs from a kernel, why mmap is checked against the caller's claims, why DMA is a call and not a mapping, and why the write back after an interrupt is not optional. Also that a released device does not come back by itself, which is true and was nowhere. docs/architecture/networking.md gained a descriptor-passing section, and docs/architecture/README.md's stack diagram gained the two layers and the third seam it was missing. "Two seams hold the whole thing together" has not been true since /dev/pci. docs/plan/README.md said the hybrid work was deliberately last, on the argument that an interface with no consumer is wrong in ways nobody can see. The argument was right; the conclusion was not, and the document now says why it changed -- the alternative to a driver compatibility layer is writing a driver for every device that exists, and the original objection is answered rather than ignored because SchOS's own drivers are the consumers. The site: a Hybrid section with the eviction transcript, cards for virtual terminals, schinit and drivers outside the kernel, and the numbers remeasured rather than estimated -- 46,961 lines, 207 syscalls. The boot log in app.js is from a boot taken today, and the lines are trimmed to what the console can show without clipping, each still a verbatim prefix of real output. Two small rendering fixes found by looking at the pages rather than assuming: .console pre's min-height is now scoped to the hero, so a static transcript sizes to its content instead of reserving eight lines it does not have; and the docs reader's contents panel strips inline marks from headings, so a heading written `proc_reincarnate()` no longer carries its backticks into the sidebar. Every intra-doc link checked; none broken. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
157 lines
7.6 KiB
Markdown
157 lines
7.6 KiB
Markdown
# Testing
|
|
|
|
> **Booting to a prompt proves nothing.** A filesystem can mount and still fail
|
|
> the first real command, and that has happened here.
|
|
|
|
So the harnesses drive the system and assert on what comes back, rather than
|
|
reporting that it started.
|
|
|
|
---
|
|
|
|
## The three levels
|
|
|
|
| | What it runs | What it proves |
|
|
|---|---|---|
|
|
| `make schfs-check` | Host-native unit suites | The filesystem's logic, in about a second |
|
|
| `tools/windows/bootcheck.ps1` | A real boot, driven over the serial console | That the system answers correctly |
|
|
| `make crashqr`, `make schcom-test` | A boot, then pixel assertions from the host | That the *screen* is right |
|
|
|
|
The levels exist because each catches what the one below cannot. A unit test
|
|
cannot tell you the kernel fails to boot; a boot test cannot tell you the screen
|
|
is blank.
|
|
|
|
---
|
|
|
|
## `make schfs-check` — the fast one
|
|
|
|
Run this first and run it often. It is host-native, so a test takes a second
|
|
instead of a boot.
|
|
|
|
```sh
|
|
make schfs-check # B-tree, object and image suites
|
|
make schfs-test # the B-tree alone
|
|
make schfs-objtest # the object layer alone
|
|
make schfs-image-test # build an image with mkschfs, check it with fsck
|
|
```
|
|
|
|
This works because **the kernel and the host tools compile the same SchFS
|
|
source**. `mkschfs` cannot disagree with the kernel about the on-disk format,
|
|
because there is only one implementation — so format bugs are findable on the
|
|
host, where the loop is fast.
|
|
|
|
## `bootcheck` — the boot suite
|
|
|
|
```sh
|
|
tools/windows/bootcheck.ps1 # the default run
|
|
tools/windows/bootcheck.ps1 -Tty -Ps -Net # switches compose
|
|
```
|
|
|
|
It boots SchOS under QEMU, types commands at the serial console, and greps the
|
|
transcript for the answers they should produce.
|
|
|
|
**It is PowerShell because `pwsh` runs on every supported host** — Linux, macOS
|
|
and the BSDs as well as Windows. `brew install powershell`, or your
|
|
distribution's package. It finds QEMU on `PATH`.
|
|
|
|
### The switches, and why each exists
|
|
|
|
Each of these turns on something the default run cannot reach. That is the point:
|
|
a subsystem nothing exercises looks identical whether it works or not.
|
|
|
|
| Switch | Why it is not in the default run |
|
|
|---|---|
|
|
| `-Tty` | Pipes, `isatty`, `SIGPIPE`, Ctrl-C. **None can be checked from a screenshot**, and all were once broken in ways that looked like broken *commands*. |
|
|
| `-Usb` | Attaches an xHCI controller with a keyboard, mouse and mass-storage device. Nothing on the default machine has a USB port, so without it a broken driver looks exactly like a working one. |
|
|
| `-Sock` | Unix sockets. A socket layer that accepted every call and dropped every byte would look identical from outside. `socktest` has one process write and another read — including the `SCM_RIGHTS` checks, where the descriptor passed is always a pipe end, because a pipe announces its own lifetime and that is what makes "was the kernel's reference released" a value rather than a belief. |
|
|
| `-Evdev` | Raw input. Needs the QEMU monitor, because a key has to be *pressed* — characters typed at the serial port never touch the keyboard controller. |
|
|
| `-Ps` | Process enumeration. `/proc`'s whole reason for existing is one directory per process, and nothing else in the suite touches it. |
|
|
| `-Net` | TCP/IP. A listener runs **on the host** and the guest connects out. |
|
|
| `-Iso` | Boots `schos.iso` from an emulated optical drive with **no hard disk attached**, which is the only way to prove the ISO is self-contained. Needs `make iso` first. |
|
|
| `-Fb` | Framebuffer checks. |
|
|
|
|
`-Net` deserves its own note, because it is the shape every integration test
|
|
should copy: **both ends are asserted.** The guest reports what it received and
|
|
the listener reports what arrived. A stack that completed a handshake and then
|
|
dropped every byte would pass any check made from one side alone.
|
|
|
|
Under QEMU's user-mode networking the host is `10.0.2.2` seen from inside, and
|
|
SLIRP connects to the host's loopback — so the listener binds there.
|
|
|
|
## Pixel assertions
|
|
|
|
```sh
|
|
make crashshot && make crashqr # crash the kernel, decode the QR out of the framebuffer
|
|
make schcom-test # boot, run the compositor and a client, check the pixels
|
|
```
|
|
|
|
`tools/windows/splashshot.ps1` counts colours. **That exists because a log-only
|
|
harness once passed on a blank screen.**
|
|
|
|
`schcom-test` asserts **from the host** — the compositor's own report that it
|
|
composited something proves nothing. It runs two overlapping clients and checks
|
|
that the newer one wins in the overlap, because a compositor that cannot order
|
|
two surfaces cannot host a window manager.
|
|
|
|
Other capture tools, all in `tools/windows/`: `dispshot`, `fbshot`,
|
|
`splashshot`, `textshot`, `ttyshot`.
|
|
|
|
---
|
|
|
|
## Two lessons worth keeping
|
|
|
|
**Read the transcript, not the verdict.** A change to the input grab once passed
|
|
every assertion green while not having been applied at all — a scripted string
|
|
replacement had silently failed to match, and the `grep` that should have caught
|
|
it returned three hits (a comment, a `#define`, and the *release* call) that were
|
|
read as confirmation without counting what they were. What caught it was four
|
|
bare `#` prompts in the transcript, which is precisely the symptom the change was
|
|
meant to remove.
|
|
|
|
**A test that stops finding bugs has a standing cost.** A ported 1993 shooter
|
|
served as the full-stack acceptance test for a while and was a genuinely harsh
|
|
one, because it was not written against our bugs — it found three real defects
|
|
(`O_NONBLOCK` being a documented no-op, evdev replaying its backlog, and no
|
|
`EVIOCGRAB`). It was removed once it stopped finding anything, because carrying
|
|
it meant 33 MB of non-redistributable data staged into `sysroot/`, which is
|
|
linked into the kernel binary for a `ramroot` build.
|
|
|
|
> **The shape of that test is worth copying even though the program is gone:**
|
|
> something large that we did not write, driven to a real pixel-level assertion
|
|
> rather than a log line.
|
|
|
|
---
|
|
|
|
## The suite is green
|
|
|
|
`make schfs-check` passes with no sanitizer output at all. It was not, until
|
|
recently, and both failures are worth knowing about because neither was in the
|
|
filesystem's logic:
|
|
|
|
**A leak with no API to fix it.** The test leaked 518 pages because the
|
|
allocation map had no release path *at all* — which meant the kernel leaked the
|
|
same map on every unmount. **A suite that leaks cannot detect a leak**: the
|
|
filesystem's own allocations would have arrived as noise inside the harness's.
|
|
`schfs_objtest` now asserts that every page it handed out came back.
|
|
|
|
**31 misaligned reads.** Records are read by casting a pointer into a tree leaf,
|
|
where they sit at arbitrary offsets. That works on x86 and is undefined in C, so
|
|
it had never shown as a bug and would have arrived as apparent filesystem
|
|
corruption at some future optimisation level.
|
|
|
|
The shared lesson is the one this directory keeps relearning: **a sanitizer that
|
|
is reliably noisy is a sanitizer nobody reads.** Both were dismissible as "known
|
|
and harmless" for as long as they were there, and one of them was a real kernel
|
|
leak.
|
|
|
|
## What is not tested
|
|
|
|
- **No automated test runs on real hardware.** Everything above is QEMU.
|
|
- **SMP is exercised but not stressed** — boots on 1, 2 and 4 processors are
|
|
checked; nothing hunts for races.
|
|
- **No fuzzing anywhere**, and no `fsx`/`xfstests` against SchFS. Getting SchFS
|
|
under those is the main argument for the FUSE implementation in
|
|
[../plan/roadmap.md](../plan/roadmap.md).
|
|
- **The SchFS round-trip test needs symlinks and hard links**, so it skips on a
|
|
Git Bash host and only runs on Unix.
|
|
- **`make ports` does not work on Windows.** musl, toybox and mksh must be built
|
|
once on a Unix host and staged.
|