# Testing > **Booting to a prompt proves nothing.** A filesystem can mount and still fail > the first real command, and that has happened here. So the harnesses drive the system and assert on what comes back, rather than reporting that it started. --- ## The three levels | | What it runs | What it proves | |---|---|---| | `make schfs-check` | Host-native unit suites | The filesystem's logic, in about a second | | `tools/windows/bootcheck.ps1` | A real boot, driven over the serial console | That the system answers correctly | | `make crashqr`, `make schcom-test` | A boot, then pixel assertions from the host | That the *screen* is right | The levels exist because each catches what the one below cannot. A unit test cannot tell you the kernel fails to boot; a boot test cannot tell you the screen is blank. --- ## `make schfs-check` — the fast one Run this first and run it often. It is host-native, so a test takes a second instead of a boot. ```sh make schfs-check # B-tree, object and image suites make schfs-test # the B-tree alone make schfs-objtest # the object layer alone make schfs-image-test # build an image with mkschfs, check it with fsck ``` This works because **the kernel and the host tools compile the same SchFS source**. `mkschfs` cannot disagree with the kernel about the on-disk format, because there is only one implementation — so format bugs are findable on the host, where the loop is fast. ## `bootcheck` — the boot suite ```sh tools/windows/bootcheck.ps1 # the default run tools/windows/bootcheck.ps1 -Tty -Ps -Net # switches compose ``` It boots SchOS under QEMU, types commands at the serial console, and greps the transcript for the answers they should produce. **It is PowerShell because `pwsh` runs on every supported host** — Linux, macOS and the BSDs as well as Windows. `brew install powershell`, or your distribution's package. It finds QEMU on `PATH`. ### The switches, and why each exists Each of these turns on something the default run cannot reach. That is the point: a subsystem nothing exercises looks identical whether it works or not. | Switch | Why it is not in the default run | |---|---| | `-Tty` | Pipes, `isatty`, `SIGPIPE`, Ctrl-C. **None can be checked from a screenshot**, and all were once broken in ways that looked like broken *commands*. | | `-Usb` | Attaches an xHCI controller with a keyboard, mouse and mass-storage device. Nothing on the default machine has a USB port, so without it a broken driver looks exactly like a working one. | | `-Sock` | Unix sockets. A socket layer that accepted every call and dropped every byte would look identical from outside. `socktest` has one process write and another read. | | `-Evdev` | Raw input. Needs the QEMU monitor, because a key has to be *pressed* — characters typed at the serial port never touch the keyboard controller. | | `-Ps` | Process enumeration. `/proc`'s whole reason for existing is one directory per process, and nothing else in the suite touches it. | | `-Net` | TCP/IP. A listener runs **on the host** and the guest connects out. | | `-Iso` | Boots `schos.iso` from an emulated optical drive with **no hard disk attached**, which is the only way to prove the ISO is self-contained. Needs `make iso` first. | | `-Fb` | Framebuffer checks. | `-Net` deserves its own note, because it is the shape every integration test should copy: **both ends are asserted.** The guest reports what it received and the listener reports what arrived. A stack that completed a handshake and then dropped every byte would pass any check made from one side alone. Under QEMU's user-mode networking the host is `10.0.2.2` seen from inside, and SLIRP connects to the host's loopback — so the listener binds there. ## Pixel assertions ```sh make crashshot && make crashqr # crash the kernel, decode the QR out of the framebuffer make schcom-test # boot, run the compositor and a client, check the pixels ``` `tools/windows/splashshot.ps1` counts colours. **That exists because a log-only harness once passed on a blank screen.** `schcom-test` asserts **from the host** — the compositor's own report that it composited something proves nothing. It runs two overlapping clients and checks that the newer one wins in the overlap, because a compositor that cannot order two surfaces cannot host a window manager. Other capture tools, all in `tools/windows/`: `dispshot`, `fbshot`, `splashshot`, `textshot`, `ttyshot`. --- ## Two lessons worth keeping **Read the transcript, not the verdict.** A change to the input grab once passed every assertion green while not having been applied at all — a scripted string replacement had silently failed to match, and the `grep` that should have caught it returned three hits (a comment, a `#define`, and the *release* call) that were read as confirmation without counting what they were. What caught it was four bare `#` prompts in the transcript, which is precisely the symptom the change was meant to remove. **A test that stops finding bugs has a standing cost.** A ported 1993 shooter served as the full-stack acceptance test for a while and was a genuinely harsh one, because it was not written against our bugs — it found three real defects (`O_NONBLOCK` being a documented no-op, evdev replaying its backlog, and no `EVIOCGRAB`). It was removed once it stopped finding anything, because carrying it meant 33 MB of non-redistributable data staged into `sysroot/`, which is linked into the kernel binary for a `ramroot` build. > **The shape of that test is worth copying even though the program is gone:** > something large that we did not write, driven to a real pixel-level assertion > rather than a log line. --- ## What is not tested - **No automated test runs on real hardware.** Everything above is QEMU. - **SMP is exercised but not stressed** — boots on 1, 2 and 4 processors are checked; nothing hunts for races. - **No fuzzing anywhere**, and no `fsx`/`xfstests` against SchFS. Getting SchFS under those is the main argument for the FUSE implementation in [../plan/roadmap.md](../plan/roadmap.md). - **The SchFS round-trip test needs symlinks and hard links**, so it skips on a Git Bash host and only runs on Unix. - **`make ports` does not work on Windows.** musl, toybox and mksh must be built once on a Unix host and staged.