docs: four directories, and a rule for when a thing leaves the roadmap
docs/ was ten files where a plan, a description, a guide and a bug report all
sat next to each other. It is now architecture/ (what SchOS is), plan/ (what it
will be), guides/ (how to work on it) and history/ (why it is like this), with
an index that states the one rule holding it together: when something is
finished it moves from plan/ to architecture/ and is rewritten as a description
of the thing rather than a record of building it.
A ticked checkbox says the work happened. It does not say what the work
*produced*, and six months later the checkbox is all that is left. The network
stack proved the point: finished, verified end to end, and for a month the only
description of it anywhere was a checkbox four levels of indentation deep.
So roadmap.md went from 534 lines to 181 -- nothing deleted, the done work
graduated. Seven subsystems that were finished and undescribed now have
descriptions: boot, kernel, syscalls, filesystem, networking, graphics, usb,
init. Every one ends with "What it does not do", which is the section that pays;
a document listing only strengths is marketing, and the reader who needs it most
is the one about to trip over a limitation nobody wrote down.
schfs.md still opened with "phases 1 through 3 1/2 are done, phase 4 is next"
long after all five had landed. It is a description now, with the phase record
and the hard-won bugs moved to history/.
Three corrections found on the way, all load-bearing:
* projects/schcom/README.md claimed SCM_RIGHTS was done. There is no
sendmsg/recvmsg in the kernel at all, and an entire GUI plan rested on it.
* The root README said "No TCP/IP" months after the stack landed.
* history/kernel.md sec.7 was marked "in progress" on a problem that process
groups solved.
Also adds plan/1.0.md, which draws a finish line so the open list stops being
infinite, and marks schwm/ and schde/ superseded: in Wayland the compositor is
the window manager, so that split dissolves rather than being abandoned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
parent
817d666068
commit
57aff40228
59 changed files with 3981 additions and 1124 deletions
112
docs/guides/adding-a-driver.md
Normal file
112
docs/guides/adding-a-driver.md
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
# Adding a driver
|
||||
|
||||
The device model itself is described in
|
||||
[../architecture/devices.md](../architecture/devices.md). This is the procedure.
|
||||
|
||||
---
|
||||
|
||||
## Where to get one from
|
||||
|
||||
**The best source is the specification, not another kernel.** xHCI, AHCI,
|
||||
virtio, NVMe and the Intel datasheets are public documents with no licence on
|
||||
what you write from them. `kernel/drivers/bus/xhci/xhci.c` came from the spec,
|
||||
and so did `net/e1000/` and `gpu/virtio_gpu/`.
|
||||
|
||||
**BSD: yes. Linux: no.** This is a licence incompatibility rather than a
|
||||
preference, and it decides the whole strategy — see
|
||||
[../licensing.md](../licensing.md). BSD-2 and BSD-3 code can be used directly
|
||||
with a `NOTICE` entry; GPL-2.0-only code can never be, at any distance. Reading
|
||||
Linux to understand *hardware* is fine and normal; translating a Linux driver
|
||||
function by function produces a derivative work whatever the identifiers say.
|
||||
|
||||
One caveat about "port it from BSD" as a plan, learned by looking: FreeBSD's
|
||||
`sys/dev/usb` is roughly 100k lines welded to newbus, devclass, `bus_dma`,
|
||||
callouts, taskqueues and FreeBSD's locking. Porting it means porting FreeBSD's
|
||||
device framework first and finding the USB stack somewhere underneath.
|
||||
|
||||
> **BSD is far more valuable as the reference you check yourself against than as
|
||||
> the code you paste.** Read it when the spec is ambiguous — which it is, often —
|
||||
> and write your own.
|
||||
|
||||
Where a *table* of constants is genuinely faster to transcribe than to retype
|
||||
from a PDF — device IDs, register offsets — taking it from a BSD tree is fine
|
||||
after the licence check, and goes in `NOTICE`. That is data; the logic stays
|
||||
ours.
|
||||
|
||||
---
|
||||
|
||||
## The procedure
|
||||
|
||||
1. Write `kernel/drivers/<class>/<name>/<name>.c` exporting a
|
||||
`struct pci_driver` or `struct usb_driver`. One directory per driver, so a
|
||||
driver that grows a register header, a second source file or a firmware blob
|
||||
has somewhere to put it:
|
||||
|
||||
kernel/drivers/
|
||||
registry.c
|
||||
gpu/vga/ gpu/virtio_gpu/
|
||||
net/e1000/
|
||||
bus/xhci/
|
||||
usb/core/ usb/hid/ usb/storage/ usb/hub/
|
||||
|
||||
2. Declare it in `kernel/drivers/registry.c` and add it to the array.
|
||||
3. Add the object to `KERN_DRIVERS` in the `Makefile`.
|
||||
4. Fill in the **class vtable** for what it is, and point `.ops` at it. That is
|
||||
the whole of what the rest of the kernel sees:
|
||||
|
||||
| class | vtable | consumer |
|
||||
|---|---|---|
|
||||
| `DEVCLASS_DISPLAY` | `struct display_ops` ([display.h](../../include/display.h)) | the compositor |
|
||||
| `DEVCLASS_NET` | `struct netdev_ops` ([netdev.h](../../include/netdev.h)) | the TCP/IP stack |
|
||||
|
||||
A display driver that owns a linear framebuffer also publishes it with
|
||||
`gfx_attach_scanout()` ([gfx.h](../../include/gfx.h)). That is what the text
|
||||
console and the crash screen draw on, and it is separate from the display
|
||||
class on purpose: the class is about *frames* — acquire, present, vblank —
|
||||
and the crash screen has to work when nothing else in the system does.
|
||||
|
||||
A driver that fills these in never mentions its consumer, and its consumer
|
||||
never mentions it. Getting this wrong has a characteristic shape: the e1000
|
||||
shipped with `.ops = 0` and the network stack called `e1000_transmit()`
|
||||
directly, so `kernel/net/` was full of one driver's name and the second NIC
|
||||
driver would have had to edit the protocol stack to add itself.
|
||||
|
||||
Private headers go **next to the driver**, not in `include/`. The build adds
|
||||
the driver's own directory to the include path, so `e1000.h` — a register map
|
||||
that exactly one file has any use for — lives in `kernel/drivers/net/e1000/`.
|
||||
`include/` is for headers the rest of the kernel needs.
|
||||
|
||||
Registration in the device model is automatic — `pci_bind()` and
|
||||
`usb_bind_all()` publish whatever they claim, so a driver cannot forget to. Set
|
||||
`devclass` on a USB driver, or the device lands in `other`.
|
||||
|
||||
A driver that can drive **more than one card at once** keeps its instances in
|
||||
an array and implements `.attached`, which the bus calls once the `struct
|
||||
device` exists; hang the instance on `dev->priv` there and read it back in
|
||||
every op. It cannot be done inside `probe()`, because the device is published
|
||||
afterwards. Skipping this is how a driver ends up refusing the second card, as
|
||||
the e1000 did until it was asked to drive two.
|
||||
|
||||
The best source for a new driver is the **specification**, not another kernel:
|
||||
xHCI, AHCI, virtio, NVMe and the Intel datasheets are public documents with no
|
||||
licence on what you write from them. `kernel/drivers/bus/xhci/xhci.c` came from the spec, and so did
|
||||
`net/e1000/` and `gpu/virtio_gpu/`. Where a *table* of constants is genuinely
|
||||
faster to transcribe than to retype from a PDF -- device IDs, register offsets --
|
||||
taking it from a BSD tree is fine after the licence check, and goes in `NOTICE`.
|
||||
That is data; the logic stays ours.
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] One directory under `kernel/drivers/<class>/<name>/`
|
||||
- [ ] Exports a `struct pci_driver` or `struct usb_driver`
|
||||
- [ ] Declared in `kernel/drivers/registry.c` **and** added to its array
|
||||
- [ ] Object added to `KERN_DRIVERS` in the `Makefile`
|
||||
- [ ] Class vtable filled in and `.ops` pointed at it
|
||||
- [ ] `.attached` implemented if it can drive more than one card
|
||||
- [ ] `devclass` set, if it is a USB driver
|
||||
- [ ] Private headers next to the driver, not in `include/`
|
||||
- [ ] A `NOTICE` entry, if anything was copied — **before** the commit, not after
|
||||
- [ ] A selftest reachable from `/proc/sysrq`, if the class has one
|
||||
(`netdev_selftest` is the model)
|
||||
125
docs/guides/building.md
Normal file
125
docs/guides/building.md
Normal file
|
|
@ -0,0 +1,125 @@
|
|||
<!-- SPDX-License-Identifier: Apache-2.0 -->
|
||||
# Building SchOS
|
||||
|
||||
Supported hosts: **Linux, macOS, FreeBSD/OpenBSD/NetBSD, and Windows** (under
|
||||
Git Bash or MSYS2). The build is developed from both Windows and Linux, so a
|
||||
change that only works on one of them is a bug.
|
||||
|
||||
```sh
|
||||
make # kernel + UEFI loader + root filesystem
|
||||
make qemu # build and boot it
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Need | Why | Windows | Linux | macOS | BSD |
|
||||
|---|---|---|---|---|---|
|
||||
| `clang`, `ld.lld`, `llvm-objcopy`, `llvm-objdump` | cross-compiles the kernel, the 16/32-bit boot code, and musl userland | LLVM installer | `clang lld llvm` | `brew install llvm` | `pkg install llvm` |
|
||||
| `make` | — | Git Bash | `make` | Xcode CLT | `gmake` |
|
||||
| `perl` | generates the interrupt vector table | Git Bash | usually present | present | `pkg install perl5` |
|
||||
| `python3` | boot logo, crash-screen QR | python.org (`python`) | `python3` | `brew install python` | `pkg install python3` |
|
||||
| `qemu-system-x86_64` | running and testing | `scoop install qemu` | `qemu-system-x86` | `brew install qemu` | `pkg install qemu` |
|
||||
| `curl`, `tar` | fetching ports | Git Bash | present | present | present |
|
||||
| `xorriso` **or** `mkisofs` | `make iso` only | `scoop install cdrtools` | `xorriso` | `brew install xorriso` | `pkg install xorriso` |
|
||||
|
||||
A host C compiler (`cc`) is also needed, for the tools that build the disk
|
||||
images. On Windows that must be **mingw-w64's gcc**, not clang: clang defaults
|
||||
to an MSVC target there, and `tools/mkfs.c` needs `<unistd.h>` and `<dirent.h>`,
|
||||
which the Windows SDK does not have.
|
||||
|
||||
## Why clang rather than a cross-gcc
|
||||
|
||||
clang is a cross-compiler by construction, so one ordinary LLVM install targets
|
||||
everything this tree needs — bare-metal ELF, code linked below one megabyte, and
|
||||
Linux/musl userland — from any host. A cross binutils+gcc would be a day of
|
||||
setup per platform that then rots, and on macOS it means brewing
|
||||
`x86_64-elf-gcc` before anything builds at all.
|
||||
|
||||
The Makefile selects clang+lld whenever both are on `PATH`. To force the GNU
|
||||
toolchain instead:
|
||||
|
||||
```sh
|
||||
make USE_GNU=1 # needs a cross toolchain that emits x86_64 ELF
|
||||
make USE_GNU=1 TOOLPREFIX=x86_64-elf-
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
```sh
|
||||
tools/windows/bootcheck.ps1 # boots, drives the shell, asserts on output
|
||||
tools/windows/bootcheck.ps1 -Tty -Ps # plus terminal behaviour and process listing
|
||||
make schfs-check # SchFS unit, object and image suites
|
||||
```
|
||||
|
||||
`bootcheck` is PowerShell, and PowerShell 7 (`pwsh`) runs on Linux, macOS and
|
||||
the BSDs as well as Windows — `brew install powershell`, or your distribution's
|
||||
`powershell` package. It finds QEMU on `PATH`.
|
||||
|
||||
Booting to a prompt proves nothing on its own: a filesystem can mount and still
|
||||
fail the first real command, which has happened here. `bootcheck` types commands
|
||||
at the serial console and greps the transcript for the answers they should
|
||||
produce.
|
||||
|
||||
## Ports
|
||||
|
||||
`musl`, `toybox` and `mksh` are built from source into `sysroot/`. **This does
|
||||
not work on a Windows host** — `make` prints `ports: skipped` and uses whatever
|
||||
is already staged in `sysroot/`. Building them needs a Linux host (or any other
|
||||
Unix); do it once and the staged result serves the Windows builds.
|
||||
|
||||
## Where the tools live
|
||||
|
||||
`tools/` is grouped by what runs the file, so it is obvious at a glance which
|
||||
tools a given host can use:
|
||||
|
||||
```
|
||||
tools/
|
||||
windows/ .ps1 PowerShell (pwsh runs these on Linux and macOS too)
|
||||
posix/ .sh shell
|
||||
python/ .py
|
||||
*.c host programs -- compiled by the build, run everywhere
|
||||
*.pl perl, likewise
|
||||
```
|
||||
|
||||
The `.c` and `.pl` files stay at the top level because they are not
|
||||
host-specific: the build compiles or invokes them on every platform.
|
||||
|
||||
A script that needs the repo root derives it from its own location, so anything
|
||||
moved between these directories has to have that adjusted — the PowerShell tools
|
||||
use `Split-Path (Split-Path $PSScriptRoot -Parent) -Parent`, two levels up.
|
||||
|
||||
## Known host differences, already handled
|
||||
|
||||
Listed because they are the shape of the next such bug, not because anything is
|
||||
outstanding:
|
||||
|
||||
- `<sys/sysmacros.h>` is glibc/musl only. macOS and the BSDs take `major()` and
|
||||
`minor()` from `<sys/types.h>`.
|
||||
- `sha256sum` exists on Linux and Git Bash; macOS has `shasum -a 256`,
|
||||
FreeBSD/NetBSD `sha256`, OpenBSD `cksum -a sha256`. `tools/posix/sha256check.sh`
|
||||
finds whichever is present.
|
||||
- `nproc` is coreutils, so absent on macOS and BSD, where it is
|
||||
`sysctl -n hw.ncpu`.
|
||||
- In the PowerShell tools, path literals must use forward slashes:
|
||||
`Join-Path $root 'build\x'` produces a *filename containing a backslash* on a
|
||||
Unix host.
|
||||
|
||||
## Fedora, in one command
|
||||
|
||||
```sh
|
||||
tools/posix/setup-fedora.sh --check # what is missing
|
||||
tools/posix/setup-fedora.sh # install it
|
||||
```
|
||||
|
||||
Moving an existing checkout to a Linux machine: see
|
||||
[../history/moving-to-linux.md](../history/moving-to-linux.md).
|
||||
|
||||
## Working on two machines
|
||||
|
||||
Claude Code's project memory is kept in the repo and moved with:
|
||||
|
||||
```sh
|
||||
tools/posix/memory-sync.sh status
|
||||
tools/posix/memory-sync.sh export # after a session that learned something; then commit
|
||||
tools/posix/memory-sync.sh import # after cloning or pulling elsewhere
|
||||
```
|
||||
177
docs/guides/layout.md
Normal file
177
docs/guides/layout.md
Normal file
|
|
@ -0,0 +1,177 @@
|
|||
# Repository layout
|
||||
|
||||
Where everything is, and how to add to it.
|
||||
|
||||
One rule holds the whole thing together:
|
||||
|
||||
> **A directory contains either hand-written source or generated output. Never both.**
|
||||
|
||||
Everything generated lives under `build/`. Delete `build/` and you lose nothing
|
||||
that can't be rebuilt (except upstream tarballs, which live in `dist/` and are
|
||||
deliberately kept out of `clean`).
|
||||
|
||||
```
|
||||
schos/
|
||||
├─ boot/ Stage 1 MBR + stage 2 loader (splash, F8 menu, kernel load)
|
||||
├─ kernel/ The kernel, by subsystem: arch/ core/ dev/ fs/ proc/ sys/
|
||||
├─ include/ Headers shared by kernel, userland, and tools/mkfs
|
||||
├─ lib/ First-party userland runtime: _start glue, syscall stubs,
|
||||
│ the initcode blob embedded in the kernel, user.ld
|
||||
├─ cmd/ First-party programs — one directory per program:
|
||||
│ cmd/<name>/main.c → sysroot/bin/<name>
|
||||
├─ ports/ Third-party software — one directory per port.
|
||||
│ Contains ONLY a Makefile and optional patches/.
|
||||
├─ projects/ Things that run ON SchOS rather than things that ARE it:
|
||||
│ the compositor, window manager, desktop, and installer.
|
||||
│ Not part of the kernel build. See projects/README.md.
|
||||
├─ tools/ Host-side helpers, SOURCE ONLY: mkfs.c, the SchFS test
|
||||
│ suites, sign.pl, vectors.pl, and the PowerShell harnesses.
|
||||
│ Their binaries build into build/tools/ -- this directory
|
||||
│ used to hold both, which is the one rule this layout has.
|
||||
│ tools/windows/loc.ps1 counts how much of the tree is first-party.
|
||||
├─ sysroot/ Staging for the disk image. etc/ is hand-maintained;
|
||||
│ bin/ lib/ sbin/ are populated by `make sysroot-install`.
|
||||
├─ dist/ Upstream tarball cache. Survives `make clean`.
|
||||
├─ Refrence/ A FreeBSD checkout (Refrence/freebsd-src), kept as reference
|
||||
│ material and a parts bin. Never built, never in an image.
|
||||
│ Read it when a specification is ambiguous; copy from it only
|
||||
│ after the per-file licence check in docs/licensing.md.
|
||||
│ A full clone is 2.1 GB and a previous NetBSD one was deleted
|
||||
│ for that reason. Only sys/dev, sys/net and sys/netinet have
|
||||
│ ever been read, so a sparse checkout is ~100 MB:
|
||||
│ git clone --filter=blob:none --sparse <url> Refrence/freebsd-src
|
||||
│ git -C Refrence/freebsd-src sparse-checkout set │ sys/dev sys/net sys/netinet
|
||||
├─ src-trunk/ A NetBSD checkout, when present -- reference material and a
|
||||
│ parts bin. NOT built, NOT installed, must never reach an
|
||||
│ image. Currently absent; it was deleted for disk space and
|
||||
│ can be re-fetched sparsely. Copying from it is governed by
|
||||
│ the per-file licence check in docs/licensing.md §3 --
|
||||
│ NetBSD is not uniformly licensed.
|
||||
├─ docs/ architecture/ what it is plan/ what's next
|
||||
│ guides/ how to work history/ why it's like this
|
||||
└─ build/ ALL generated output (see below)
|
||||
```
|
||||
|
||||
## build/
|
||||
|
||||
```
|
||||
build/
|
||||
├─ boot/ uefi/ loader, common/ UI
|
||||
├─ kernel/ object files, the linked kernel, kernel.asm
|
||||
├─ lib/ libc glue objects, initcode blob
|
||||
├─ cmd/ linked first-party programs
|
||||
├─ fs.img the filesystem image
|
||||
└─ ports/<name>/
|
||||
├─ src/ extracted upstream tarball
|
||||
├─ obj/ out-of-tree build directory
|
||||
└─ prefix/ install prefix (musl's headers and libs land here)
|
||||
```
|
||||
|
||||
## Adding your own program
|
||||
|
||||
```sh
|
||||
mkdir cmd/hello
|
||||
$EDITOR cmd/hello/main.c # include "types.h", "user.h", ...
|
||||
```
|
||||
|
||||
Then add `hello` to `CMDS` in the root `Makefile`. It gets linked against
|
||||
`lib/` and installed to `sysroot/bin/hello` by `make sysroot-install`.
|
||||
|
||||
## Adding a third-party port
|
||||
|
||||
```sh
|
||||
mkdir ports/foo
|
||||
$EDITOR ports/foo/Makefile # start with `include ../mk/port.mk`
|
||||
```
|
||||
|
||||
`port.mk` derives everything from the directory name — it gives you `$(SRCDIR)`,
|
||||
`$(BUILDDIR)`, `$(PREFIX)`, `$(DISTDIR)`, `$(SYSROOT)`, and `$(MUSL)` (the
|
||||
musl install prefix that every port links against). Follow the pattern in
|
||||
`ports/mksh/Makefile`: fetch tarball to `$(DISTDIR)`, extract to `$(SRCDIR)`,
|
||||
build into `$(BUILDDIR)`, install into `$(SYSROOT)`. `clean` removes
|
||||
`$(PORTDIR)` and nothing else.
|
||||
|
||||
`ports/Makefile` picks up the new directory automatically. If it links against
|
||||
musl, add `foo: musl` there so ordering is right.
|
||||
|
||||
Local modifications to upstream go in `ports/foo/patches/*.patch` and are
|
||||
applied after extraction — never by editing files under `build/`, which are
|
||||
wiped on every clean.
|
||||
|
||||
## Ramdisk root
|
||||
|
||||
```sh
|
||||
make # normal: root on the IDE disk (build/fs.img as hdb)
|
||||
make ramroot # root from a SCHFS image linked into the kernel
|
||||
```
|
||||
|
||||
`make ramroot` builds `build/fs-ram.img` (default 6000 blocks, override with
|
||||
`RAMDISK_BLOCKS=`) and links it into the kernel, so the system boots to a shell
|
||||
with **no data disk attached at all**. That is what makes destructive work on
|
||||
the real disk survivable — reformatting it, or bringing up a new on-disk format
|
||||
— and it is the first piece of the installer.
|
||||
|
||||
Use `make ramroot`, not `make RAMROOT=1`: the flag changes `CFLAGS`, which make
|
||||
cannot see, so objects compiled for one setting would be silently reused for
|
||||
the other and `ROOTDEV` would point at the wrong device. The target drops the
|
||||
compiled objects first. Switching back is the same — plain `make` after a
|
||||
`ramroot` build reuses stale objects, so run `rm -rf build/kernel build/lib
|
||||
build/cmd` first (or just `make clean`).
|
||||
|
||||
The image costs boot time: the loader reads the kernel through the firmware's
|
||||
file protocol, so a 6 MB ramroot kernel is visibly slower to start than the
|
||||
350 KB default.
|
||||
|
||||
**What may go in it is a licensing question, not only a size one.** The image is
|
||||
linked into the kernel binary, so anything in `sysroot/` when it is built becomes
|
||||
part of a distributable artifact. A file staged there under a licence that
|
||||
cannot be combined with Apache-2.0 does not merely ride along — it makes the
|
||||
kernel itself undistributable. `RAMDISK_EXCLUDE` in the Makefile is the list of
|
||||
paths kept out, and enforcing it in the build rather than relying on someone
|
||||
remembering to unstage something first is deliberate. It has been needed once
|
||||
already: a 33 MB test payload landed in `sysroot/` and both overflowed the image
|
||||
and would have gone into the kernel binary.
|
||||
|
||||
Three knobs, all in the Makefile, none of them `FSSIZE`:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `RAMDISK_BLOCKS` | image size, 6000 (× 1 KiB `BSIZE`) |
|
||||
| `RAMDISK_NINODES` | inode table size, 2000 |
|
||||
| `RAMDISK_EXCLUDE` | sysroot-relative paths to omit, with their subtrees |
|
||||
|
||||
`RAMDISK_NINODES` is worth knowing about because the inode table is
|
||||
*preallocated*: mkfs's default of 12000 inodes cost 1501 of the 6000 blocks — a
|
||||
quarter of the image — to describe about 260 files. At 2000 it costs 251, and
|
||||
metadata overall fell from 1596 blocks to 345.
|
||||
|
||||
If `mkfs` reports the image is full, read its message: it now prints the
|
||||
metadata breakdown and names the right knob for whichever image overflowed. It
|
||||
used to say only "increase FSSIZE in param.h", which is a constant that has
|
||||
nothing to do with the ramdisk root and sends you to the wrong file.
|
||||
|
||||
## Two things that will bite you
|
||||
|
||||
The second is about a bootloader that no longer exists. It is kept because the
|
||||
*shape* of the bug outlived the code.
|
||||
|
||||
**`ld -r -b binary` derives symbol names from the input file's path.** The
|
||||
kernel embeds two blobs this way (`initcode`, `entryother`), and the root
|
||||
Makefile has `--redefine-sym` lines mapping e.g.
|
||||
`_binary_build_lib_initcode_start` → `_binary_initcode_start`. Move either
|
||||
blob's build location and those symbol names change; update the Makefile to
|
||||
match or the kernel fails to link.
|
||||
|
||||
**`STAGE2_SECTORS` is gone, with the BIOS loader it sized.** It is recorded here
|
||||
because the shape of the bug outlived it: one number that four files each
|
||||
believed independently — `boot/bios/bootasm.S`, `stage2main.c`'s `readseg()`,
|
||||
the `dd ... seek=` offset, and twice inside `mkiso.sh`. Raising it from 24 to 48
|
||||
for the boot logo left the `dd seek=25` offset behind, so the kernel was written
|
||||
directly over stage 2's own tail, and nothing said so.
|
||||
|
||||
The fix was to derive all five from one Makefile variable. The lesson worth
|
||||
keeping is the other half: `tools/padstage2.pl` *failed the build* when stage 2
|
||||
overflowed its budget rather than truncating it, which is why the overflow to
|
||||
25440 bytes against a 24576 limit was found at all. A layout constant that
|
||||
several artifacts must agree on wants a check that stops the build, not a
|
||||
comment asking people to remember.
|
||||
135
docs/guides/testing.md
Normal file
135
docs/guides/testing.md
Normal file
|
|
@ -0,0 +1,135 @@
|
|||
# Testing
|
||||
|
||||
> **Booting to a prompt proves nothing.** A filesystem can mount and still fail
|
||||
> the first real command, and that has happened here.
|
||||
|
||||
So the harnesses drive the system and assert on what comes back, rather than
|
||||
reporting that it started.
|
||||
|
||||
---
|
||||
|
||||
## The three levels
|
||||
|
||||
| | What it runs | What it proves |
|
||||
|---|---|---|
|
||||
| `make schfs-check` | Host-native unit suites | The filesystem's logic, in about a second |
|
||||
| `tools/windows/bootcheck.ps1` | A real boot, driven over the serial console | That the system answers correctly |
|
||||
| `make crashqr`, `make schcom-test` | A boot, then pixel assertions from the host | That the *screen* is right |
|
||||
|
||||
The levels exist because each catches what the one below cannot. A unit test
|
||||
cannot tell you the kernel fails to boot; a boot test cannot tell you the screen
|
||||
is blank.
|
||||
|
||||
---
|
||||
|
||||
## `make schfs-check` — the fast one
|
||||
|
||||
Run this first and run it often. It is host-native, so a test takes a second
|
||||
instead of a boot.
|
||||
|
||||
```sh
|
||||
make schfs-check # B-tree, object and image suites
|
||||
make schfs-test # the B-tree alone
|
||||
make schfs-objtest # the object layer alone
|
||||
make schfs-image-test # build an image with mkschfs, check it with fsck
|
||||
```
|
||||
|
||||
This works because **the kernel and the host tools compile the same SchFS
|
||||
source**. `mkschfs` cannot disagree with the kernel about the on-disk format,
|
||||
because there is only one implementation — so format bugs are findable on the
|
||||
host, where the loop is fast.
|
||||
|
||||
## `bootcheck` — the boot suite
|
||||
|
||||
```sh
|
||||
tools/windows/bootcheck.ps1 # the default run
|
||||
tools/windows/bootcheck.ps1 -Tty -Ps -Net # switches compose
|
||||
```
|
||||
|
||||
It boots SchOS under QEMU, types commands at the serial console, and greps the
|
||||
transcript for the answers they should produce.
|
||||
|
||||
**It is PowerShell because `pwsh` runs on every supported host** — Linux, macOS
|
||||
and the BSDs as well as Windows. `brew install powershell`, or your
|
||||
distribution's package. It finds QEMU on `PATH`.
|
||||
|
||||
### The switches, and why each exists
|
||||
|
||||
Each of these turns on something the default run cannot reach. That is the point:
|
||||
a subsystem nothing exercises looks identical whether it works or not.
|
||||
|
||||
| Switch | Why it is not in the default run |
|
||||
|---|---|
|
||||
| `-Tty` | Pipes, `isatty`, `SIGPIPE`, Ctrl-C. **None can be checked from a screenshot**, and all were once broken in ways that looked like broken *commands*. |
|
||||
| `-Usb` | Attaches an xHCI controller with a keyboard, mouse and mass-storage device. Nothing on the default machine has a USB port, so without it a broken driver looks exactly like a working one. |
|
||||
| `-Sock` | Unix sockets. A socket layer that accepted every call and dropped every byte would look identical from outside. `socktest` has one process write and another read. |
|
||||
| `-Evdev` | Raw input. Needs the QEMU monitor, because a key has to be *pressed* — characters typed at the serial port never touch the keyboard controller. |
|
||||
| `-Ps` | Process enumeration. `/proc`'s whole reason for existing is one directory per process, and nothing else in the suite touches it. |
|
||||
| `-Net` | TCP/IP. A listener runs **on the host** and the guest connects out. |
|
||||
| `-Iso` | Boots `schos.iso` from an emulated optical drive with **no hard disk attached**, which is the only way to prove the ISO is self-contained. Needs `make iso` first. |
|
||||
| `-Fb` | Framebuffer checks. |
|
||||
|
||||
`-Net` deserves its own note, because it is the shape every integration test
|
||||
should copy: **both ends are asserted.** The guest reports what it received and
|
||||
the listener reports what arrived. A stack that completed a handshake and then
|
||||
dropped every byte would pass any check made from one side alone.
|
||||
|
||||
Under QEMU's user-mode networking the host is `10.0.2.2` seen from inside, and
|
||||
SLIRP connects to the host's loopback — so the listener binds there.
|
||||
|
||||
## Pixel assertions
|
||||
|
||||
```sh
|
||||
make crashshot && make crashqr # crash the kernel, decode the QR out of the framebuffer
|
||||
make schcom-test # boot, run the compositor and a client, check the pixels
|
||||
```
|
||||
|
||||
`tools/windows/splashshot.ps1` counts colours. **That exists because a log-only
|
||||
harness once passed on a blank screen.**
|
||||
|
||||
`schcom-test` asserts **from the host** — the compositor's own report that it
|
||||
composited something proves nothing. It runs two overlapping clients and checks
|
||||
that the newer one wins in the overlap, because a compositor that cannot order
|
||||
two surfaces cannot host a window manager.
|
||||
|
||||
Other capture tools, all in `tools/windows/`: `dispshot`, `fbshot`,
|
||||
`splashshot`, `textshot`, `ttyshot`.
|
||||
|
||||
---
|
||||
|
||||
## Two lessons worth keeping
|
||||
|
||||
**Read the transcript, not the verdict.** A change to the input grab once passed
|
||||
every assertion green while not having been applied at all — a scripted string
|
||||
replacement had silently failed to match, and the `grep` that should have caught
|
||||
it returned three hits (a comment, a `#define`, and the *release* call) that were
|
||||
read as confirmation without counting what they were. What caught it was four
|
||||
bare `#` prompts in the transcript, which is precisely the symptom the change was
|
||||
meant to remove.
|
||||
|
||||
**A test that stops finding bugs has a standing cost.** A ported 1993 shooter
|
||||
served as the full-stack acceptance test for a while and was a genuinely harsh
|
||||
one, because it was not written against our bugs — it found three real defects
|
||||
(`O_NONBLOCK` being a documented no-op, evdev replaying its backlog, and no
|
||||
`EVIOCGRAB`). It was removed once it stopped finding anything, because carrying
|
||||
it meant 33 MB of non-redistributable data staged into `sysroot/`, which is
|
||||
linked into the kernel binary for a `ramroot` build.
|
||||
|
||||
> **The shape of that test is worth copying even though the program is gone:**
|
||||
> something large that we did not write, driven to a real pixel-level assertion
|
||||
> rather than a log line.
|
||||
|
||||
---
|
||||
|
||||
## What is not tested
|
||||
|
||||
- **No automated test runs on real hardware.** Everything above is QEMU.
|
||||
- **SMP is exercised but not stressed** — boots on 1, 2 and 4 processors are
|
||||
checked; nothing hunts for races.
|
||||
- **No fuzzing anywhere**, and no `fsx`/`xfstests` against SchFS. Getting SchFS
|
||||
under those is the main argument for the FUSE implementation in
|
||||
[../plan/roadmap.md](../plan/roadmap.md).
|
||||
- **The SchFS round-trip test needs symlinks and hard links**, so it skips on a
|
||||
Git Bash host and only runs on Unix.
|
||||
- **`make ports` does not work on Windows.** musl, toybox and mksh must be built
|
||||
once on a Unix host and staged.
|
||||
Loading…
Reference in a new issue