From 57aff40228e673b11d7e50aa0f9ae01b4c401699 Mon Sep 17 00:00:00 2001 From: Nexoniarz Date: Sat, 12 Sep 2026 17:13:35 +0200 Subject: [PATCH] docs: four directories, and a rule for when a thing leaves the roadmap docs/ was ten files where a plan, a description, a guide and a bug report all sat next to each other. It is now architecture/ (what SchOS is), plan/ (what it will be), guides/ (how to work on it) and history/ (why it is like this), with an index that states the one rule holding it together: when something is finished it moves from plan/ to architecture/ and is rewritten as a description of the thing rather than a record of building it. A ticked checkbox says the work happened. It does not say what the work *produced*, and six months later the checkbox is all that is left. The network stack proved the point: finished, verified end to end, and for a month the only description of it anywhere was a checkbox four levels of indentation deep. So roadmap.md went from 534 lines to 181 -- nothing deleted, the done work graduated. Seven subsystems that were finished and undescribed now have descriptions: boot, kernel, syscalls, filesystem, networking, graphics, usb, init. Every one ends with "What it does not do", which is the section that pays; a document listing only strengths is marketing, and the reader who needs it most is the one about to trip over a limitation nobody wrote down. schfs.md still opened with "phases 1 through 3 1/2 are done, phase 4 is next" long after all five had landed. It is a description now, with the phase record and the hard-won bugs moved to history/. Three corrections found on the way, all load-bearing: * projects/schcom/README.md claimed SCM_RIGHTS was done. There is no sendmsg/recvmsg in the kernel at all, and an entire GUI plan rested on it. * The root README said "No TCP/IP" months after the stack landed. * history/kernel.md sec.7 was marked "in progress" on a problem that process groups solved. Also adds plan/1.0.md, which draws a finish line so the open list stops being infinite, and marks schwm/ and schde/ superseded: in Wayland the compositor is the window manager, so that split dissolves rather than being abandoned. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2 --- .claude/memory/schos-linux-machine.md | 2 +- .claude/memory/schunix-boot-splash-logo.md | 2 +- .claude/memory/schunix-build-environment.md | 2 +- .claude/memory/schunix-device-model.md | 2 +- .../memory/schunix-doom-acceptance-test.md | 4 +- .../memory/schunix-e1000-and-driver-layout.md | 2 +- .claude/memory/schunix-layout-migration.md | 2 +- .claude/memory/schunix-licensing.md | 2 +- .../memory/schunix-network-stack-choice.md | 4 +- .claude/memory/schunix-qr-encoder.md | 2 +- .../memory/schunix-roadmap-and-projects.md | 2 +- .claude/memory/schunix-schapi-direction.md | 2 +- .claude/memory/schunix-smp-and-memory-bugs.md | 2 +- .claude/memory/schunix-usb-ids.md | 2 +- .claude/memory/schunix-usb-xhci.md | 2 +- README.md | 30 +- docs/README.md | 129 +++++ docs/architecture/README.md | 102 ++++ docs/architecture/boot.md | 215 +++++++ docs/architecture/devices.md | 104 ++++ docs/architecture/filesystem.md | 170 ++++++ docs/architecture/graphics.md | 233 ++++++++ docs/architecture/init.md | 155 +++++ docs/architecture/kernel.md | 211 +++++++ docs/architecture/networking.md | 260 +++++++++ docs/architecture/schfs.md | 478 ++++------------ docs/architecture/syscalls.md | 131 +++++ docs/architecture/usb.md | 116 ++++ docs/drivers.md | 161 ------ docs/guides/adding-a-driver.md | 112 ++++ docs/{ => guides}/building.md | 2 +- docs/{ => guides}/layout.md | 19 +- docs/guides/testing.md | 135 +++++ docs/history/README.md | 54 ++ .../kernel-history.md => history/kernel.md} | 52 +- docs/{ => history}/moving-to-linux.md | 27 +- docs/history/schfs.md | 370 ++++++++++++ docs/licensing.md | 8 +- docs/plan/1.0.md | 267 +++++++++ docs/plan/README.md | 105 ++++ docs/plan/foundations.md | 135 +++++ docs/plan/hybrid.md | 165 ++++++ docs/plan/init.md | 178 ++++++ docs/plan/roadmap.md | 179 ++++++ docs/{architecture => plan}/schapi.md | 2 +- docs/plan/wayland.md | 167 ++++++ docs/roadmap.md | 534 ------------------ include/device.h | 2 +- include/qr.h | 2 +- kernel/arch/x86_64/mp.c | 2 +- kernel/arch/x86_64/trap.c | 2 +- kernel/drivers/bus/xhci/xhci.c | 2 +- kernel/drivers/gpu/vga/vga_fb.c | 2 +- kernel/drivers/net/e1000/e1000.h | 2 +- kernel/proc/proc.c | 4 +- projects/README.md | 2 +- projects/schcom/README.md | 21 +- projects/schde/README.md | 9 +- projects/schwm/README.md | 7 + 59 files changed, 3977 insertions(+), 1120 deletions(-) create mode 100644 docs/README.md create mode 100644 docs/architecture/README.md create mode 100644 docs/architecture/boot.md create mode 100644 docs/architecture/devices.md create mode 100644 docs/architecture/filesystem.md create mode 100644 docs/architecture/graphics.md create mode 100644 docs/architecture/init.md create mode 100644 docs/architecture/kernel.md create mode 100644 docs/architecture/networking.md create mode 100644 docs/architecture/syscalls.md create mode 100644 docs/architecture/usb.md delete mode 100644 docs/drivers.md create mode 100644 docs/guides/adding-a-driver.md rename docs/{ => guides}/building.md (98%) rename docs/{ => guides}/layout.md (91%) create mode 100644 docs/guides/testing.md create mode 100644 docs/history/README.md rename docs/{architecture/kernel-history.md => history/kernel.md} (92%) rename docs/{ => history}/moving-to-linux.md (79%) create mode 100644 docs/history/schfs.md create mode 100644 docs/plan/1.0.md create mode 100644 docs/plan/README.md create mode 100644 docs/plan/foundations.md create mode 100644 docs/plan/hybrid.md create mode 100644 docs/plan/init.md create mode 100644 docs/plan/roadmap.md rename docs/{architecture => plan}/schapi.md (99%) create mode 100644 docs/plan/wayland.md delete mode 100644 docs/roadmap.md diff --git a/.claude/memory/schos-linux-machine.md b/.claude/memory/schos-linux-machine.md index ce6e023..f534f78 100644 --- a/.claude/memory/schos-linux-machine.md +++ b/.claude/memory/schos-linux-machine.md @@ -19,7 +19,7 @@ Fedora 44, Ryzen 5 7000-series, RTX 4050, 16 GB DDR5. The GPU is irrelevant to the build — QEMU runs headless for tests. `tools/posix/setup-fedora.sh` installs the toolchain (`--check` to only report). -`docs/moving-to-linux.md` is the procedure. Memory moves with +`docs/history/moving-to-linux.md` is the procedure. Memory moves with [[schos-memory-sync]]: `import` after arriving, `export` and commit after a session that learned something. diff --git a/.claude/memory/schunix-boot-splash-logo.md b/.claude/memory/schunix-boot-splash-logo.md index 33ab778..8f7248a 100644 --- a/.claude/memory/schunix-boot-splash-logo.md +++ b/.claude/memory/schunix-boot-splash-logo.md @@ -32,7 +32,7 @@ stays in [[schunix-console-tty]]'s `include/theme.h`; the loader composites it i for the one rule. 4 bits rather than 8 because the header is included by the *bootloader*, where size is a boot constraint. -**`STAGE2_SECTORS` was encoded in five places, not the three `docs/layout.md` +**`STAGE2_SECTORS` was encoded in five places, not the three `docs/guides/layout.md` claimed.** Raising it 24 → 48 to fit the 8 KiB mask proved it: the `dd seek=25` offset stayed behind and the kernel got written straight over stage2's tail. The five were `boot/bootasm.S`, `boot/stage2main.c`, the `dd seek=`, and *twice* diff --git a/.claude/memory/schunix-build-environment.md b/.claude/memory/schunix-build-environment.md index 7203591..766fbe0 100644 --- a/.claude/memory/schunix-build-environment.md +++ b/.claude/memory/schunix-build-environment.md @@ -32,7 +32,7 @@ the Makefile says so out loud rather than skipping silently. Everything is a no-op on Linux, so a Linux/VM build still works identically. **Five latent bugs the move exposed**, all real, none Windows-specific — see -`docs/architecture/kernel-history.md` §8 for the two hardest: +`docs/history/kernel.md` §8 for the two hardest: 1. `kernel.ld` named no `.ltext`/`.ldata`/`.lbss` — with `-mcmodel=large`, LLVM and GCC 13+ put nearly everything there, and orphan placement put them past diff --git a/.claude/memory/schunix-device-model.md b/.claude/memory/schunix-device-model.md index e58fe63..c6160b0 100644 --- a/.claude/memory/schunix-device-model.md +++ b/.claude/memory/schunix-device-model.md @@ -9,7 +9,7 @@ metadata: --- Built 2026-08-08. `include/device.h` + `kernel/dev/device.c`, readable at -`/proc/devices`. Written up in `docs/drivers.md`. Supersedes the two separate +`/proc/devices`. Written up in `docs/architecture/devices.md`. Supersedes the two separate arrays in `kernel/drivers/registry.c` as the answer to "what is attached". **Bus is provenance, class is capability** — keep them separate. `bus` is diff --git a/.claude/memory/schunix-doom-acceptance-test.md b/.claude/memory/schunix-doom-acceptance-test.md index f03fd05..cb574b1 100644 --- a/.claude/memory/schunix-doom-acceptance-test.md +++ b/.claude/memory/schunix-doom-acceptance-test.md @@ -42,5 +42,5 @@ undistributable, not just the image. See [[schunix-ramdisk-root-sizing]] and faulting `rip` to get a function name, then use the fault offset to identify the exact field. "Null pointer plus 24" alone is nearly useless. -Both lessons are written up in `docs/roadmap.md` Phase 7 and -`docs/architecture/kernel-history.md` §10 and §12, with the program genericised. +Both lessons are written up in `docs/plan/roadmap.md` Phase 7 and +`docs/history/kernel.md` §10 and §12, with the program genericised. diff --git a/.claude/memory/schunix-e1000-and-driver-layout.md b/.claude/memory/schunix-e1000-and-driver-layout.md index ebfa508..f89075d 100644 --- a/.claude/memory/schunix-e1000-and-driver-layout.md +++ b/.claude/memory/schunix-e1000-and-driver-layout.md @@ -43,7 +43,7 @@ machine with only virtio-gpu boots blind until the kernel is up. `Refrence/freebsd-src` (spelled that way on disk) is a full FreeBSD checkout, re-cloned 2026-08-08 after the NetBSD one was deleted for space. Never built, -never in an image; recorded in `docs/layout.md` and `docs/licensing.md` §4. +never in an image; recorded in `docs/guides/layout.md` and `docs/licensing.md` §4. **It is 2.1 GB** — measured, and the same reason the NetBSD one was deleted. Only `sys/dev`, `sys/net` and `sys/netinet` have ever been read, so a sparse diff --git a/.claude/memory/schunix-layout-migration.md b/.claude/memory/schunix-layout-migration.md index 4828678..97a3333 100644 --- a/.claude/memory/schunix-layout-migration.md +++ b/.claude/memory/schunix-layout-migration.md @@ -11,7 +11,7 @@ metadata: Restructured Schunix's tree on 2026-08-01. `bt/` became `ports/` (third-party only, Makefile + patches, nothing else), first-party userland split into `lib/` (runtime glue) and `cmd//main.c` (programs), tarballs to `dist/`, and all -generated output to `build/`. Documented in `docs/layout.md`. Verified: full +generated output to `build/`. Documented in `docs/guides/layout.md`. Verified: full build passes and the image boots to `init: ready (pid 1)`. **Two traps that bit during this and will bite again:** diff --git a/.claude/memory/schunix-licensing.md b/.claude/memory/schunix-licensing.md index 38b8b69..65579d0 100644 --- a/.claude/memory/schunix-licensing.md +++ b/.claude/memory/schunix-licensing.md @@ -44,7 +44,7 @@ add to `NOTICE`, done. Reading Linux to understand hardware is fine; translating a driver function-by-function produces a derivative work whatever the identifiers say. Best source of all is the public specification — xHCI, virtio, AHCI, NVMe, e1000 — which carries no licence on what you write from -it. Recorded in `docs/roadmap.md` §1. See [[schunix-roadmap-and-projects]]. +it. Recorded in `docs/plan/roadmap.md` §1. See [[schunix-roadmap-and-projects]]. ## fs.c is third-party, and was unattributed until 2026-08-09 diff --git a/.claude/memory/schunix-network-stack-choice.md b/.claude/memory/schunix-network-stack-choice.md index 98469ea..e0dec2f 100644 --- a/.claude/memory/schunix-network-stack-choice.md +++ b/.claude/memory/schunix-network-stack-choice.md @@ -28,7 +28,7 @@ appear against a real peer. The NetBSD clause belongs to **WIDE**, not Berkeley, so the 1999 rescission does not reach it. Checked FreeBSD's file directly from `raw.githubusercontent.com`; zero matches for "All advertising materials". Checked lwIP's `COPYING`; likewise -zero. This **reverses** the older assumption in `docs/roadmap.md` that NetBSD +zero. This **reverses** the older assumption in `docs/plan/roadmap.md` that NetBSD was the intended starting point. ## The porting evidence, which matters more @@ -37,7 +37,7 @@ FreeBSD's `sys/netinet` is not a library. `tcp_input.c` alone is 122 KB and reaches into mbufs, `epoch(9)`, netisr, VNET, `uma(9)`, sockbufs, inpcb hash tables and FreeBSD's locking. Taking it means porting FreeBSD's kernel infrastructure first and finding the TCP underneath — the same conclusion -`docs/roadmap.md` §1 already reached about `sys/dev/usb`. +`docs/plan/roadmap.md` §1 already reached about `sys/dev/usb`. **lwIP was written for the situation Schunix is actually in:** a small OS with a raw Ethernet driver and no infrastructure. Port layer is a handful of files, no diff --git a/.claude/memory/schunix-qr-encoder.md b/.claude/memory/schunix-qr-encoder.md index 49d78e7..990b207 100644 --- a/.claude/memory/schunix-qr-encoder.md +++ b/.claude/memory/schunix-qr-encoder.md @@ -9,7 +9,7 @@ metadata: --- Built 2026-08-08. `kernel/core/qr.c` + `include/qr.h`, the Phase 8 crash channel -(see [[schunix-crash-screen]] and `docs/roadmap.md`). +(see [[schunix-crash-screen]] and `docs/plan/roadmap.md`). **Wired into the crash screen the same day.** `panic.c` builds a compact payload and paints a white card on the right of PSOGM. Verified end-to-end with diff --git a/.claude/memory/schunix-roadmap-and-projects.md b/.claude/memory/schunix-roadmap-and-projects.md index 0ec32ca..2002164 100644 --- a/.claude/memory/schunix-roadmap-and-projects.md +++ b/.claude/memory/schunix-roadmap-and-projects.md @@ -8,7 +8,7 @@ metadata: modified: 2026-08-09T20:20:11.342Z --- -**Set out 2026-08-04.** Recorded in `docs/roadmap.md`; the naming and the +**Set out 2026-08-04.** Recorded in `docs/plan/roadmap.md`; the naming and the split are Nexoniarz's. Order asked for: USB → GPU → OTF/TTF → network + sockets → GUI → installer. diff --git a/.claude/memory/schunix-schapi-direction.md b/.claude/memory/schunix-schapi-direction.md index 175317a..797f54c 100644 --- a/.claude/memory/schunix-schapi-direction.md +++ b/.claude/memory/schunix-schapi-direction.md @@ -12,7 +12,7 @@ metadata: attached". It is aimed at being a **system**: NeXTSTEP/macOS-shaped layering, with POSIX/Unix kept complete but demoted from identity to *personality*, and a native higher-level API called **SchAPI** above it. Written up in -`docs/architecture/schapi.md`. +`docs/plan/schapi.md`. The architecture is borrowed from macOS; the compatibility policy is borrowed from **Windows** (nothing is ever removed after 1.0). Those are two different diff --git a/.claude/memory/schunix-smp-and-memory-bugs.md b/.claude/memory/schunix-smp-and-memory-bugs.md index 1cae046..9b683e8 100644 --- a/.claude/memory/schunix-smp-and-memory-bugs.md +++ b/.claude/memory/schunix-smp-and-memory-bugs.md @@ -8,7 +8,7 @@ metadata: modified: 2026-08-08T19:21:20.756Z --- -Both of Nexoniarz's 2026-08-08 reports are **fixed**. `docs/roadmap.md` §3b has +Both of Nexoniarz's 2026-08-08 reports are **fixed**. `docs/plan/roadmap.md` §3b has the full write-up. ## SMP works diff --git a/.claude/memory/schunix-usb-ids.md b/.claude/memory/schunix-usb-ids.md index 23aa366..ef14293 100644 --- a/.claude/memory/schunix-usb-ids.md +++ b/.claude/memory/schunix-usb-ids.md @@ -24,7 +24,7 @@ carries the attribution and says which option was taken. `make usb-ids`. Fetching per build would break offline builds, make the same source produce different images on different days, and put a third-party web server on the critical path of `make` — and `dist/` exists precisely so a -rebuild never needs the network (stated in `docs/layout.md`). The file is +rebuild never needs the network (stated in `docs/guides/layout.md`). The file is committed and carries its own version and date, so what shipped is always knowable. The update target validates the download (`grep -q "^# Version:"`) before replacing the cached copy, so a truncated file or an error page cannot diff --git a/.claude/memory/schunix-usb-xhci.md b/.claude/memory/schunix-usb-xhci.md index e77820f..fca5f8f 100644 --- a/.claude/memory/schunix-usb-xhci.md +++ b/.claude/memory/schunix-usb-xhci.md @@ -115,4 +115,4 @@ drains the event ring by hand — that is what separates "no event" from "event, no interrupt". `echo s > /proc/sysrq` reads block 1 off the USB disk through the buffer cache. -**Still not done:** hubs. See `docs/roadmap.md` phase 1. +**Still not done:** hubs. See `docs/plan/roadmap.md` phase 1. diff --git a/README.md b/README.md index 730e5ae..3954901 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ **A 64-bit operating system, written from the metal up.** -SMP · copy-on-write filesystem · PCI, USB and virtio drivers · TrueType text · Linux syscall ABI +SMP · copy-on-write filesystem · TCP/IP · PCI, USB and virtio drivers · TrueType text · Linux syscall ABI @@ -57,6 +57,10 @@ generation counter advancing as writes commit. displays, **xHCI** USB with HID, hub and mass-storage class drivers, IDE, PS/2, serial. `/dev/fb0` speaks the Linux fbdev interface. +**Networking** — ARP, IPv4, ICMP, UDP, TCP and DHCP, on lwIP behind a +driver-agnostic interface class. Verified end to end from the host, both +directions, rather than by inspection. + **Userland** — musl, toybox and mksh, built from source. `ls`, `cat`, `ps`, `top`, `wc`, `cp`, `mount` and the rest behave the way you expect, because they are the real programs. @@ -80,9 +84,9 @@ make iso # a bootable ISO for VMware or real hardware Builds on **Linux, macOS, FreeBSD/OpenBSD/NetBSD and Windows** (Git Bash). The toolchain is clang + lld everywhere — it is a cross-compiler by construction, so one LLVM install targets the bare-metal kernel, the 16-bit boot code and musl -userland from any host. See **[docs/building.md](docs/building.md)** for +userland from any host. See **[docs/guides/building.md](docs/guides/building.md)** for prerequisites per platform, and -**[docs/moving-to-linux.md](docs/moving-to-linux.md)** for moving a checkout +**[docs/history/moving-to-linux.md](docs/history/moving-to-linux.md)** for moving a checkout between machines. ## Testing @@ -109,9 +113,14 @@ boot/ uefi/ (PE loader) common/ (shared loader UI) lib/ cmd/ first-party userland runtime and programs ports/ third-party software built from source tools/ windows/ (.ps1) posix/ (.sh) python/ (.py) and .c/.pl -docs/ architecture, roadmap, licensing, building +docs/ architecture/ (what it is) plan/ (what's next) + guides/ (how to work on it) history/ (why it's like this) ``` +**[docs/README.md](docs/README.md)** is the index. If you are reading the tree +for the first time, [docs/architecture/README.md](docs/architecture/README.md) +is the system layer by layer. + One rule worth knowing: **the kernel and the host tools compile the same SchFS source**. `mkschfs` cannot disagree with the kernel about the on-disk format, because there is only one implementation — found and fixed on the host, where a @@ -122,9 +131,16 @@ Another: **syscalls are declared in exactly one place**, ## Not there yet -- No TCP/IP. The decision is made — port lwIP rather than write one — but it is - not done. The e1000 driver works; nothing speaks to it yet. -- No compositor, window manager or desktop. That is the next large piece. +- **No desktop.** Schcom composites and routes input; the plan is now to put it + on Wayland and delete the homegrown protocol. See + [docs/plan/wayland.md](docs/plan/wayland.md). +- **No descriptor passing.** No `sendmsg`/`recvmsg`, so no `SCM_RIGHTS` — which + is what the GUI work stops at. [docs/plan/foundations.md](docs/plan/foundations.md). +- **No dynamic linking.** `exec` handles static `ET_EXEC` only. +- **No page cache**, so writable `MAP_SHARED` of a file is refused rather than + faked. +- **A USB disk cannot be booted from** — the root device is compiled in rather + than selectable. - SchFS has no Linux driver yet; the plan is on-disk format version first, then FUSE, then a kernel module. - CFF/OpenType outlines and TrueType hinting are deliberately out of scope. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..7a5de3f --- /dev/null +++ b/docs/README.md @@ -0,0 +1,129 @@ +# SchOS documentation + +Four kinds of document, kept in four directories, because they answer four +different questions and go stale at four different rates. + +| | | | +|---|---|---| +| **[architecture/](architecture/)** | *What SchOS is* | Present tense. Describes what is built and working. | +| **[plan/](plan/)** | *What SchOS will be* | Future tense. Everything here is unbuilt. | +| **[guides/](guides/)** | *How to work on it* | Imperative. Build it, test it, add to it. | +| **[history/](history/)** | *Why it is the way it is* | Past tense. Bugs whose cause is not visible from the fix. | + +Plus one rule that belongs to no category and applies to everything: +**[licensing.md](licensing.md)** — read it before copying anyone else's code in. + +--- + +## The rule that keeps this from rotting + +> **When something is finished, it moves from `plan/` to `architecture/` and is +> rewritten as a description of the thing rather than a record of building it.** + +A completed item is not a ticked checkbox. A ticked checkbox says the work +happened; it does not say what the work *produced*, and six months later the +checkbox is the only thing left and it explains nothing. + +So finishing a piece of work has three steps, and the third is not optional: + +1. The code works and is tested. +2. The plan entry is deleted. +3. A description exists in `architecture/` written for somebody who was not + there — what it does, what shape it has, what it costs, and what it still + does not do. + +This is not bookkeeping. The tree has already proved it necessary: the network +stack was finished, verified end to end, and for a month afterwards the only +description of it anywhere was a checkbox in a roadmap, four levels of +indentation deep, written in the past tense of somebody arguing for the work +rather than explaining the result. + +**The "what it does not do" section is the one that pays.** Every description in +`architecture/` ends with one. A document that only lists a subsystem's +strengths is marketing, and the reader who most needs the document is the one +about to trip over a limitation nobody wrote down. + +--- + +## Start here + +**New to the tree** → [architecture/README.md](architecture/README.md), the +system layer by layer, then [guides/layout.md](guides/layout.md) for where the +files are. + +**Trying to build it** → [guides/building.md](guides/building.md). + +**Wondering what happens next** → [plan/1.0.md](plan/1.0.md), which is the finish line, then [plan/README.md](plan/README.md) for the order. + +**Something is broken and it feels familiar** → [history/](history/). There is a +good chance it is in there. + +--- + +## architecture/ — what SchOS is + +| | | +|---|---| +| [README](architecture/README.md) | The system, layer by layer. Read first. | +| [boot.md](architecture/boot.md) | Firmware to the first userspace process. | +| [kernel.md](architecture/kernel.md) | SchIFT: memory, processes, scheduling, traps, signals, SMP. | +| [syscalls.md](architecture/syscalls.md) | The call table, and the Linux personality that lets real software run. | +| [filesystem.md](architecture/filesystem.md) | The VFS, the buffer cache, procfs, mounts. | +| [schfs.md](architecture/schfs.md) | SchFS: the copy-on-write root filesystem. | +| [devices.md](architecture/devices.md) | The device model, driver binding, the class vtables. | +| [usb.md](architecture/usb.md) | xHCI, the USB core, and the class drivers. | +| [graphics.md](architecture/graphics.md) | Framebuffer, text rendering, console, and the crash screen. | +| [networking.md](architecture/networking.md) | Drivers, lwIP, sockets — Unix and INET. | +| [init.md](architecture/init.md) | `schinit`, PID 1, and the job manifests. | + +## plan/ — what SchOS will be + +| | | +|---|---| +| [README](plan/README.md) | The three current decisions and the order they happen in. | +| [1.0.md](plan/1.0.md) | **The finish line.** What 1.0 contains, and what is explicitly deferred past it. | +| [foundations.md](plan/foundations.md) | The kernel work everything else is blocked on. | +| [init.md](plan/init.md) | PID 1 becomes a role rather than a process. | +| [wayland.md](plan/wayland.md) | The GUI stack, and what gets deleted to make room. | +| [hybrid.md](plan/hybrid.md) | Moving drivers out of the kernel. | +| [schapi.md](plan/schapi.md) | The long-term API layering, and the compatibility promise. | +| [roadmap.md](plan/roadmap.md) | Everything else still open. | + +## guides/ — how to work on it + +| | | +|---|---| +| [building.md](guides/building.md) | Prerequisites and the build, per host. | +| [testing.md](guides/testing.md) | The harnesses, and what each one actually proves. | +| [layout.md](guides/layout.md) | Repository map, and how to add a program or a port. | +| [adding-a-driver.md](guides/adding-a-driver.md) | Writing one, binding it, publishing its class vtable. | + +## history/ — why it is the way it is + +| | | +|---|---| +| [README](history/README.md) | What belongs in here and what does not. | +| [kernel.md](history/kernel.md) | Bugs whose cause was nowhere near their symptom. | +| [moving-to-linux.md](history/moving-to-linux.md) | The Windows-to-Fedora move, kept for the parts that recur. | + +--- + +## Writing one of these + +Four conventions, all of them observable in the existing files: + +**State the cost.** Every design choice gave something up. A document that does +not say what is a document that will be argued with later by somebody who only +sees the downside. + +**Prefer the measured number to the remembered one.** "500 of 3565 files carry +the advertising clause" is worth more than "some do", and it was worth the +minute it took to count. + +**Write down the bug that shaped it.** Half the non-obvious code in this tree is +non-obvious because something failed once. If a design is strange, the strangeness +is load-bearing and the reader deserves to know why before changing it. + +**No dates in headings.** A date in a heading turns a description into a +changelog, and the description is the thing with a long life. Dates belong in +`history/`, where they mean something. diff --git a/docs/architecture/README.md b/docs/architecture/README.md new file mode 100644 index 0000000..2d63ded --- /dev/null +++ b/docs/architecture/README.md @@ -0,0 +1,102 @@ +# The system, layer by layer + +What SchOS is today. Everything in this directory is built and running; what is +merely intended lives in [../plan/](../plan/). + +--- + +## The names, and what each one covers + +`uname -s` reports `SchIFT`, the way every Unix reports its kernel rather than +its distribution. + +| Layer | Name | What it is | +|---|---|---| +| System | **SchOS** | The distribution — kernel, userland, tooling | +| Kernel | **SchIFT** | Multi-core x86-64 kernel with a modular VFS | +| Filesystem | **SchFS** | Transactional, copy-on-write, the root filesystem | +| API | **SchAPI** | The native programming interface — *designed, not built* | +| Compositor | **Schcom** | Owns the display; v0 runs | + +--- + +## The stack, bottom to top + +``` + ┌─────────────────────────────────────────────────┐ + │ toybox mksh musl first-party cmd/ │ userland + ├─────────────────────────────────────────────────┤ + │ schinit — PID 1, job manifests, supervision │ + ├─────────────────────────────────────────────────┤ + │ syscall table — Linux numbering, one .inc file │ ← the ABI seam + ╞═════════════════════════════════════════════════╡ + │ VFS │ sockets │ signals │ mmap │ poll │ procfs │ + ├──────┴─────────┴─────────┴──────┴──────┴────────┤ + │ SchFS │ lwIP │ text/gfx │ buffer cache │ + ├─────────┴────────┴────────────┴─────────────────┤ + │ device model — class vtables, bus binding │ + ├─────────────────────────────────────────────────┤ + │ drivers: xHCI e1000 virtio-gpu VBE IDE PS/2│ + ├─────────────────────────────────────────────────┤ + │ scheduler │ paging │ traps │ IRQ │ SMP │ DMA │ SchIFT core + └─────────────────────────────────────────────────┘ + ▲ + UEFI loader (boot/uefi) +``` + +The two seams that matter are the syscall table and the device model. Above the +first, software is ordinary and portable — musl and toybox are unmodified. +Below the second, a driver never names its consumer and a consumer never names a +driver. + +--- + +## Where it came from + +SchOS began as [xv6](https://github.com/mit-pdos/xv6-public) and is now mostly +something else. What survives from xv6 is the *shape* of the process table, the +sleep/wakeup discipline, and the buffer cache. What has been replaced: the +scheduler and trap path (rewritten for x86-64 and for more than one processor), +the memory manager, the entire filesystem, the syscall layer, and every driver. + +That inheritance is worth knowing when reading the code, because the parts that +still look like xv6 follow xv6's conventions and the parts that do not, do not. + +--- + +## The documents + +Read in this order if you are new: + +| | | +|---|---| +| [boot.md](boot.md) | Firmware to the first userspace process. The shortest path through the whole system. | +| [kernel.md](kernel.md) | Memory, processes, scheduling, traps, signals, SMP. | +| [syscalls.md](syscalls.md) | The one table everything is generated from, and how a musl binary runs unmodified. | +| [filesystem.md](filesystem.md) | The VFS, the buffer cache, mounts, procfs. | +| [schfs.md](schfs.md) | The copy-on-write root filesystem. | +| [devices.md](devices.md) | Buses, classes, binding, and the vtables consumers talk to. | +| [usb.md](usb.md) | xHCI, the USB core, and the three class drivers. | +| [graphics.md](graphics.md) | Framebuffer, text rendering, the console, and PSOGM. | +| [networking.md](networking.md) | NIC drivers, lwIP, and both socket families. | +| [init.md](init.md) | `schinit` and the job manifests. | + +--- + +## Three rules that hold across all of it + +**The kernel and the host tools compile the same SchFS source.** `mkschfs` +cannot disagree with the kernel about the on-disk format, because there is only +one implementation. Format bugs are found on the host, where a test takes a +second instead of a boot. + +**Syscalls are declared in exactly one place.** `kernel/sys/table.inc`. +The constants, the dispatch table and the Linux translation map are all +generated from it, so a syscall cannot be half-registered. + +**A list you can read beats a list assembled by side effects.** Drivers are +written out by hand in `kernel/drivers/registry.c` rather than collected by a +linker section, and platform devices are registered in one function rather than +inside each driver's init. The tradeoff is deliberate: self-registration is +nicer to write, and it means the list of what is in your kernel is not written +down anywhere you can read. diff --git a/docs/architecture/boot.md b/docs/architecture/boot.md new file mode 100644 index 0000000..e65dc0c --- /dev/null +++ b/docs/architecture/boot.md @@ -0,0 +1,215 @@ +# Boot + +Firmware to the first shell prompt. This is the shortest path through the whole +system, which makes it the best thing to read first. + +The loader's side of the hand-off is specified in +[`boot/README.md`](../../boot/README.md) — that file is the contract, and this +one is the narrative. + +--- + +## 1. Firmware + +**UEFI only.** There was a BIOS path and it was removed: everything since +roughly 2012 boots UEFI, and CSM — the firmware-side BIOS emulation that let +legacy loaders run — has been going away since 2020. The cost of keeping it was +not the code but the tax on everything it touched, and the asymmetry is total. A +BIOS loader lives in 16-bit real mode and calls the firmware through software +interrupts; UEFI hands a loader long mode with paging already on and a table of +function pointers. + +The loader is `boot/uefi/`, a 64-bit PE application built by the same clang +install as everything else (`--target=x86_64-unknown-windows`). It imports +neither gnu-efi nor EDK II — the handful of structures it uses were written from +the specification, which is both a smaller dependency and no licence question. + +## 2. The loader + +Six steps, in this order, and the order is load-bearing: + +1. Find the kernel on the volume this image was loaded from — `\schos\kernel`, + or `\kernel`. +2. Load its ELF `PT_LOAD` segments to their physical addresses. +3. Record the framebuffer geometry from the Graphics Output Protocol. +4. Translate `GetMemoryMap()` into the 24-byte E820-shaped entries + `kernel/arch/x86_64/e820.c` already parses — not a second format for the + kernel to learn. +5. `ExitBootServices`. +6. Build the page tables the kernel's 64-bit entry expects, and jump. + +**Nothing allocates after step 5 and nothing calls a firmware service after it, +because there are none.** Everything the jump needs is computed before. + +### The hand-off block + +One page of low memory at `0x30000`, defined once in +[`include/schboot.h`](../../include/schboot.h) and included by both the loader +and `entry.S` so the two cannot disagree. + +| Offset | What | +|---|---| +| `+0x000` | VBE-shaped mode info: LFB address, stride, width, height, depth | +| `+0x100` | Verbose-boot flag | +| `+0x101` | Boot medium (`1` = CD; `0` = not stated) | +| `+0x200` | E820 entry count, then up to 40 × 24-byte entries | +| `+0x600` | ACPI RSDP physical address, or 0 | + +Two of these exist because of specific failures worth knowing about. + +**The RSDP is passed explicitly** because the kernel's own search scans the EBDA +and `0xE0000`–`0x100000`, which is a BIOS convention. UEFI publishes the RSDP in +its configuration table and is under no obligation to leave a copy anywhere; +OVMF does not. Without it the kernel finds no MADT, so no IOAPIC, so no +interrupt routing — and the machine boots to a prompt on polled serial output +and then ignores every keystroke, which looks like a broken console rather than +a missing table. + +**The E820 cap is 40 entries, not 32.** Real UEFI firmware produces far more +descriptors than QEMU's, and the space was already there. + +### The entry point + +The kernel publishes a `sch64_header` — magic `SCH64BOO`, 8-byte aligned within +the first 16 KB, followed by a 64-bit physical entry address — found by +scanning, exactly as a multiboot loader scans for its own magic. + +It exists because the firmware has *already* put the machine in long mode. +Dropping back to protected mode to re-enter through the 32-bit multiboot path +is a mode switch whose failure mode is a triple fault with no console. The +32-bit entry is still present in `entry.S` and now has no caller. + +## 3. `kmain()` + +`kernel/core/main.c`. The ordering here is not arbitrary and several steps are +where they are because of a bug: + +``` + e820_init() memory map first — PHYSTOP is a variable, not a constant + kinit1() the allocator, given everything below the DMA arena + dma_init() the arena, carved off the TOP of physical memory + kvmalloc() the kernel page tables + acpi_init() CPU enumeration from the MADT; mpinit() only as fallback + lapicinit() tvinit() seginit() ioapicinit() + irq_init() before any driver that registers a handler + inputinit() before the keyboard and mouse start producing events + consoleinit() uartinit() mouseinit() + platform_register_devices(fb) the fixed furniture of a PC, plus the framebuffer + pci_init() enumerate, and bind drivers + pinit() binit() fileinit() sockinit() + time_init() after the timer runs, before userspace + ideinit() ramdiskinit() + startothers() the other processors + kinit2() sinit() the rest of memory, then PID 1 + netinit() after sinit — it starts a kernel thread + mpmain() enter the scheduler; never returns +``` + +**The DMA arena is taken before `kalloc` gets the rest.** A device needs +physically contiguous memory and `kalloc`'s free list makes no such promise — +two pages from it are adjacent only by chance. Taking it first is the only way +to be sure the region exists. + +**ACPI is asked before the legacy MP table**, and getting this backwards is +invisible under QEMU. Modern firmware publishes a complete MADT *and* an MP +table containing one processor, so asking the older interface first means never +seeing more than one CPU. VMware published the real count, which is why the +resulting triple fault only ever appeared there. + +**`main()` names no display driver.** Whichever driver owns the framebuffer the +loader left running publishes it with `gfx_attach_scanout()`, and the console +and crash screen draw on that. See [graphics.md](graphics.md). + +### Memory sizing + +`PHYSTOP` is a variable read from the firmware's map, not a constant. It used to +be `0xE000000` — 224 MB — inherited from xv6, and it was not a limit on what the +kernel used but an **assertion that exactly that much RAM existed**. Less and the +allocator walked off the end of memory; more and it was invisible. + +Three details survive that fix: + +- It takes the end of the usable region **containing the kernel**, not the + highest usable address anywhere. A memory map has reserved holes, and freeing + a single range from the kernel's end to a global maximum puts firmware tables + on the free list — failing later, elsewhere, looking like anything but a + memory map problem. +- The usable regions are *also* kept as a list, and `kinit1()` frees each one. + One range ending at a top address is what a BIOS map looks like and what UEFI + never does: OVMF drops an ACPI NVS region at `0x800000`, so the run containing + the kernel ends at 8 MB and everything above the hole was invisible. +- The ceiling is 3 GB, because `kvmalloc()` maps 0–4 GB as four 1 GB pages and + marks the top one uncached for PCI BARs. The floor is 24 MB, and below it the + kernel prints why and stops rather than triple faulting in silence. + +## 4. SMP + +`startothers()` wakes each application processor through the local APIC, pointing +it at a trampoline copied to `0x7000`. Each AP runs `mpenter()` → `mpmain()`, +reports `cpu%d: starting`, and enters the scheduler. + +This was broken from the project's beginning and is fixed. Five distinct bugs, +each hidden behind the one before it — the details are in +[../history/kernel.md](../history/kernel.md), and the one worth carrying +forward is in [kernel.md](kernel.md#per-cpu-state): **`__thread` is wrong for +per-CPU kernel data.** + +APs are counted on the boot processor *after* each one reports in, so the count +cannot include a processor that started and died. + +## 5. The first process + +`sinit()` builds PID 1 by hand: a page directory, `initcode` mapped at 0, 32 +pages of stack, a trapframe pointing at user mode, and session and process group +both set to 1. + +It does **not** mount the root filesystem, and cannot: mounting reads blocks, +`bread()` sleeps, and sleeping needs a process context. So the mount happens in +`forkret()`, the first time any process is scheduled — guarded by a `static int +first`. + +That is also where PID 1's working directory is re-pointed at the real root, +and the reason is worth keeping: + +> `sinit()` sets `cwd = namei("/")` before any filesystem is mounted. The old +> v1 root survived that because `iget(ROOTDEV, ROOTINO)` is two compile-time +> constants and touches no disk. SchFS has no constant to offer — the root +> object id comes off the uberblock — so that early `namei("/")` builds an inode +> from an unmounted volume. Nothing complains, because absolute paths never +> consult cwd: `exec("/bin/init")` works fine. It is the first *relative* lookup +> that fails, and in init that is `open("console")`. No console means no fd +> 0/1/2, so every shell it spawns exits immediately and init respawns it +> forever. The symptom is a silent boot that reads like a hang and is not one. + +`initcode` then execs `/bin/init`, and [init.md](init.md) takes over. + +## 6. What the screen shows + +**The display is not the kernel log.** `cons_serial_only` starts *set*: every +boot message goes out the UART and nothing goes to the framebuffer. The loader's +splash therefore survives the whole of boot rather than being wiped a few +hundred milliseconds in, and `console_takeover()` clears the flag on the first +`exec()`, when userspace genuinely starts. + +Nothing is lost — the serial line is where `bootcheck` reads from and where you +want the messages when something is wrong — and the F8 verbose boot never sets +the flag, so "show me everything" still works. + +--- + +## What it does not do + +- **No BIOS path.** A machine without UEFI does not boot SchOS. +- **The root device is compiled in.** Booting from a USB disk needs the root + device to be selectable, which it is not; the USB mass-storage driver works + and a superblock reads back through `bread()`, but nothing can mount it. +- **`make ramroot` is the only way to boot with no data disk.** It links a + filesystem image into the kernel binary, which costs boot time and carries a + licensing constraint — see [../licensing.md](../licensing.md) §4. +- **The 32-bit multiboot entry is dead code.** It is left in place deliberately: + deleting kernel entry code is its own piece of work with its own way of + failing silently. +- **`make qemu-uefi` under OVMF reports less memory than the machine has** in + some configurations. The loader's map is correct; the fix went in with the + region list above, and any remaining case is a fragmentation pattern not yet + seen. diff --git a/docs/architecture/devices.md b/docs/architecture/devices.md new file mode 100644 index 0000000..d5804b1 --- /dev/null +++ b/docs/architecture/devices.md @@ -0,0 +1,104 @@ +# The device model + +How hardware gets found, claimed and used, and what the rest of the kernel sees +of it. + +Writing a driver is [../guides/adding-a-driver.md](../guides/adding-a-driver.md). +Moving drivers out of the kernel entirely is [../plan/hybrid.md](../plan/hybrid.md). + +--- + +## One list of everything attached + +There is one list of everything attached, whatever bus found it: +`include/device.h` and `kernel/dev/device.c`, readable at `/proc/devices`. + +``` +name bus class driver device +kbd0 platform input ps2kbd PS/2 keyboard +mouse0 platform input ps2mouse PS/2 mouse +uart0 platform other uart8250 16550 serial port +ide0 platform storage ide ATA disk controller +vga0 pci display vga display controller +``` + +Before this there were two registries in `kernel/drivers/registry.c` — an array +of PCI drivers and an array of USB drivers — with nothing that knew about both. +That is sufficient for *binding* and useless for the three things wanted next. + +**Enumeration.** "What is in this machine, and what claimed it" had no answer +covering a USB disk and a PCI display at once. A device manager is exactly that +question. + +**Classes.** A compositor should ask for *the display* and not care whether that +is the VBE framebuffer or virtio-gpu. Without a class every consumer hardcodes a +driver, and adding the second driver means editing the consumers — which is how +a framebuffer address ended up hardcoded in three places here once already. + +**A boundary for drivers that are not compiled in.** They have to attach +somewhere that is not an array fixed at link time. + +Two vocabularies are kept deliberately separate: a device's **bus** is +provenance (`platform`, `pci`, `usb`), its **class** is capability +(`display`, `input`, `storage`, `net`, `bus`). A USB disk and a SATA disk are +both `storage` despite sharing no numbering at all. Where a driver claims a +device, the driver's opinion wins over the descriptor's: `struct usb_driver` has +a `devclass` field separate from the USB interface class, because the descriptor +says what the device *claims* to be and the driver knows what it *made* of it. + +It is a flat list with a bus tag, **not** a device tree. Nothing yet needs to +walk a hierarchy, and a tree nobody traverses is just a harder list. Hot-unplug +would need more; the `in_use` flag is the hook. + +Platform devices are registered in one place +(`platform_register_devices()` in `registry.c`) rather than inside each driver's +init, for the same reason the driver arrays are written out by hand: a list you +can read beats a list assembled by side effects. + +--- + +## Linux drivers + +**Decided 2026-08-08.** Linux drivers are not shipped with SchOS and never +will be. The user may install one on their own machine, through a device +manager or equivalent, and SchOS will load it. + +The licensing reasoning matters and is worth stating exactly, because it is +easy to get wrong in both directions: + +- **Distribution is what triggers the GPL.** Shipping a GPL-2.0-only driver in + an Apache-2.0 image is what cannot be done — see [../licensing.md](../licensing.md) + §2. A user obtaining a driver themselves and loading it on their own machine + is their act, not ours, and is not a distribution by us. +- **Implementing a loader is not copying.** Writing support for a module format + and a driver ABI from published documentation produces our own work. It is a + clean-room implementation, and it stays one only if it is written from + documentation rather than transcribed from Linux source. Reading Linux to + understand *hardware* is fine; translating its code is not, whatever the + identifiers say. + +So the rule for this subsystem is the same as everywhere else in the tree: the +loader is ours and Apache-2.0, what it loads is the user's business, and nothing +GPL ever enters an image we build. `RAMDISK_EXCLUDE` and `NOTPORTS` are the +existing mechanisms for keeping that true mechanically rather than by intention. + +--- + + +--- + +## What it does not do + +- **It is a flat list, not a device tree.** There is no parent/child nesting, + only a bus tag. Nothing yet needs to walk a hierarchy, and a tree nobody + traverses is just a harder list. +- **No hot-plug.** Devices are registered at boot and never removed; the + `in_use` flag is the hook a hot-unplug path would need, and nothing sets it + for that purpose. +- **`NDEVICE` is 32**, compile-time, like every other table in the kernel. +- **Drivers are fixed at link time.** There is no module loader, and nothing can + be installed after the build — the class vtable is the attachment point a + loader would use, and it is the only part that exists. +- **Only two class vtables are defined**: `display_ops` and `netdev_ops`. + Storage and input devices appear in the list with `ops = 0` and are reached + through their own subsystems instead. diff --git a/docs/architecture/filesystem.md b/docs/architecture/filesystem.md new file mode 100644 index 0000000..9ba5eea --- /dev/null +++ b/docs/architecture/filesystem.md @@ -0,0 +1,170 @@ +# The VFS + +The layer between `open(2)` and a filesystem. SchFS is described separately in +[schfs.md](schfs.md); this is the machinery it plugs into, plus the buffer cache, +mounts, and procfs. + +--- + +## `struct fs_ops` + +A filesystem is a vtable, chosen by `iget()` from the device number and never +null once an inode is held. + +```c +struct fs_ops { + const char *name; + + void (*iload) (struct inode *); // fill type/size/nlink from disk + void (*istore) (struct inode *); // write them back + void (*ifree) (struct inode *); // last link gone, release it + + int (*read) (struct inode *, int user_dst, uint64 dst, uint64 off, uint n); + int (*write) (struct inode *, int user_src, uint64 src, uint64 off, uint n); + void (*truncate)(struct inode *, uint64 newsize); + + struct inode *(*dirlookup)(struct inode *dp, char *name, uint *poff); + int (*dirlink) (struct inode *dp, char *name, uint64 inum); + int (*dirunlink)(struct inode *dp, char *name, uint off); + int (*dirempty) (struct inode *dp); + + struct inode *(*ialloc)(uint dev, short type); + int (*getdents)(struct inode *dp, char *out, int cap, uint64 *off); +}; +``` + +Two implementations exist: `schfs_ops` (`kernel/fs/schfs/vfs.c`) and `procfs_ops` +(`kernel/fs/procfs.c`). + +`poff` is deliberately **an opaque cookie** rather than a byte offset — +meaningful only to the same filesystem's `dirunlink`. A directory does not have +to be an array of fixed-size records for this interface to work. + +`ialloc` creates an object **with no name**. The caller sets `nlink` and calls +`dirlink`; until it does, nothing can reach the object. If the caller dies in +between, the result is an unreferenced record that is harmless and, in SchFS, +never reaches the disk anyway — nothing is durable until the transaction commits. + +### The one place the models disagree + +The VFS inherited xv6's shape: allocate an inode, fill it in, link a name to it. +SchFS has no free-standing inode — an object exists because a record with its id +exists. + +`nlink` is the sharper disagreement. **The VFS adjusts it itself** and then calls +`iupdate`, so SchFS's own link accounting is deliberately bypassed in the VFS +path: `schfs_dir_add`/`schfs_dir_del` are used rather than `schfs_link`/ +`schfs_unlink`, and `iput`'s `ifree` hook releases the object once the VFS has +decided there are no names left. + +> Two pieces of code counting links independently is precisely how a filesystem +> loses a file that is still referenced. + +`nlink` is an `int`, not a `short`. SchFS stores a `uint32` and the narrower type +truncated it, so a file past 32767 names came back with a *negative* count — and +`iput` frees an object whose count reaches zero. Signed on purpose: `nlink--` on +a count that is somehow already zero should land somewhere a check can see it, +not wrap to four billion. + +## Path resolution + +`namex()` walks a path one component at a time. It was refactored into +`namex_from(struct inode *start, ...)`, parameterising what used to be a +hardcoded `idup(proc->cwd)`. + +That refactor is what made the `*at()` family work. Every one of them — `openat`, +`fstatat`, `unlinkat`, `renameat`, `linkat`, `symlinkat`, `mkdirat`, `mknodat`, +`readlinkat`, `faccessat` — previously handled only two cases: `AT_FDCWD`, or an +absolute path with `dirfd` ignored. A **real** dirfd with a **relative** path is +the standard, efficient way any directory-tree walker avoids rebuilding full +path strings, and it was either rejected outright or — worse, in `openat` — +silently resolved against `proc->cwd`, which can open the wrong file entirely. + +`sys_fs.c` now has `at_dir(dirfd)`, `at_namei()` and `at_nameiparent()`, and +every `*at()` call uses them. + +## The buffer cache + +`kernel/dev/bio.c`, xv6's design unchanged: a fixed pool of buffers, an LRU +list, one sleeplock per buffer. `bread()` returns a locked buffer; `bwrite()` +flushes it; `brelse()` releases. + +**Sleeping is the point.** A buffer lock is held across a disk read, which is +why every path that touches a filesystem must be able to sleep — and why the +root mount happens in `forkret()` rather than in `main()`. + +## Mounts + +`mount()` and `umount2()` at their Linux numbers, with a mount table consulted +during path resolution. `/proc` is mounted by init: + +```c +mount("proc", "/proc", "proc", 0, 0); +``` + +## procfs + +`kernel/fs/procfs.c`. A real filesystem behind the same vtable, not a set of +character devices. + +**The reason it had to become one is `ps`.** /proc used to be `mknod`'d nodes — +init ran `mknod("/proc/uptime", 6, 2)` and friends, so every file was a real +inode on the root filesystem with a major/minor behind it. That is fine for a +fixed list and cannot work at all for the thing /proc is mostly for: **one +directory per process**. You cannot `mknod` a directory for a process that does +not exist yet and will be gone in a minute. `ps` listed `/proc`, got no numbers, +and reported nothing. It was never a `ps` problem. + +Nothing is stored. Directory listings are generated from the process table when +asked; file contents when read. + +Three things worth knowing: + +- **The split is deliberate.** `kernel/fs/procfs.c` owns the *namespace* — what + exists. `kernel/dev/procfs.c` keeps the *generators* — what the global files + say. They change for different reasons, and `proc_gen_global()` is the only + thing that crosses between them. +- **procfs never walks the process table.** `proc_snapshot()` copies what is + needed out from under `ptable.lock`, so a process that exits between the + readdir that listed it and the open that reads it produces a stale line rather + than a use-after-free. +- **Sizes are reported as zero**, exactly as Linux does. A synthetic file has no + length until it is generated, and generating every one on every `stat()` to + fill in a number almost nothing reads would be absurd. + +### What is there + +Per-process directories, plus the global files: `meminfo`, `uptime`, `cpuinfo`, +`version`, `stat`, `loadavg`, `mounts`, `filesystems`, `schfs`, `pci`, +`interrupts`, `devices`, `netif`, and `sysrq`. + +**/proc is read-only except `sysrq`**, which is the one deliberate write target, +and the code says so in one place. Marking every procfs inode read-only once +broke `echo s > /proc/sysrq` and took out three USB checks within a minute — +they drive the disk through sysrq. + +`sysrq` keys: `c` crash on purpose, `u` / `s` storage checks, `d` display +selftest, `t` text specimen, `n` the netdev ARP selftest. + +--- + +## What it does not do + +- **No page cache.** `mmap` of a file reads the whole thing in at map time, and + writable `MAP_SHARED` of a file is refused rather than faked. This is the + single largest gap in the layer. +- **No symlinks in procfs**, which is why `/proc//exe` is a regular file + and `ps -o comm` prints `[init]`. +- **No `AF_UNIX` socket inodes.** Socket names live in a table in + `kernel/net/unix.c`, so `bind()` creates nothing that can be `stat()`ed or + `unlink()`ed. +- **No unified inode cache across filesystems** beyond the single `itable`. +- **A known, un-root-caused hang:** heavy `ls -lR` use across many commands in + one long shell session can eventually make a *later*, unrelated directory + hang; the identical command succeeds on a fresh boot. It is not inode or file + table exhaustion — both fail cleanly and neither fired. It smells like a lock + or refcount leak in the `*at()` paths or in toybox's own fd handling. The next + step is instrumenting `itable`/`ftable` occupancy across a long session to see + what is climbing. +- **`sys_delete`/`sys_show`/`sys_showstat`** are orphaned entries from the old + v1 filesystem's recycle bin. Nothing reaches them and they die with `fs.c`. diff --git a/docs/architecture/graphics.md b/docs/architecture/graphics.md new file mode 100644 index 0000000..8901f68 --- /dev/null +++ b/docs/architecture/graphics.md @@ -0,0 +1,233 @@ +# Graphics, text and the console + +Four things share this area and they are deliberately separate: the kernel's own +drawing surface, the compositor's display interface, the text-rendering layer, +and the crash screen. + +--- + +## The two display interfaces, and why there are two + +This distinction causes confusion every time somebody meets it, so it is the +first thing here. + +| | [`gfx.h`](../../include/gfx.h) | [`display.h`](../../include/display.h) | +|---|---|---| +| **What** | The kernel's own drawing surface | The compositor's interface | +| **About** | Pixels and text | *Frames* | +| **Operations** | `draw_pixel`, `fill_rect`, `putc`, `text_at` | `acquire`, `present`, `wait_vblank` | +| **Buffering** | None. Direct writes to the scanout. | Double-buffered | +| **Users** | The console and PSOGM | Schcom | +| **Must work when** | **Nothing else in the system does** | Userspace is healthy | + +That last row is the reason they are not one interface. The console and the +crash screen cannot depend on a back buffer being allocatable, on a driver +having a working present path, or on anything that could be the thing that just +broke. + +### `gfx` — the scanout + +A driver that owns a linear framebuffer publishes it with +`gfx_attach_scanout()`, and everything above draws through `gfx_*` calls. + +This used to be `kernel/dev/vga.c`, and every consumer called it by that name: +`panic.c` drew the crash screen with `vga_fill_rect`, the console printed with +`vga_putc`, `/dev/fb0` handed out `vga_fb_phys()`. None of that has anything to +do with a VGA — it is bitmap drawing over a linear framebuffer, and the only +VGA-specific part was where the framebuffer came from. A second display driver +is now a second caller of one function rather than an edit to the crash screen. + +`main()` names no display driver. Whichever one owns the framebuffer the loader +left running says so before any bus is enumerated, because the console draws +long before PCI is walked. + +### `display` — the class + +`struct display_ops`, found with `device_find_class(DEVCLASS_DISPLAY)`. + +`acquire()` allocates the back buffer **on first call, never at boot** — it is +around two megabytes at 800×600, and a machine whose compositor never starts +should not pay that. A driver that cannot double-buffer returns 0, and the +caller draws directly and accepts tearing rather than not drawing at all. +`wait_vblank` is optional for the same reason. + +`echo d > /proc/sysrq` drives the whole path and draws a test pattern. + +## Drivers + +| | | +|---|---| +| **VBE framebuffer** (`gpu/vga/`) | Whatever the loader set up. Always present; this is what the splash and the console are drawn on. | +| **virtio-gpu** (`gpu/virtio_gpu/`) | 2D, written from the OASIS specification. Full path: `GET_DISPLAY_INFO`, `RESOURCE_CREATE_2D`, `RESOURCE_ATTACH_BACKING`, `SET_SCANOUT`, `TRANSFER_TO_HOST_2D`, `RESOURCE_FLUSH`. | + +virtio-gpu is offered the bus **before** the VBE driver, so on a machine with +both, the device with real mode setting and an explicit flush wins the display +class. `vga_pci` still claims the stdvga card underneath, which is what the boot +splash is drawn on — so `-vga std` must stay on the QEMU command line even when +adding `-device virtio-gpu-pci`, because the loader's splash goes through VBE +and there is no virtio path that early. + +`/dev/fb0` speaks the Linux fbdev interface: geometry through the fbdev ioctls, +and `mmap` of the real framebuffer at its real physical address. + +> `mmap` of `/dev/fb0` is also **the kernel's only chance to notice that the +> screen no longer belongs to the text console** — after that the drawing +> happens with no further syscalls. `fb_mapped()` is called there, and +> `fbclose()` puts it back. + +## Text + +[`include/text.h`](../../include/text.h) + `kernel/dev/text.c`. + +**The font was never the problem — the character-cell grid was.** The console +draws text by indexing `font_data[256][16]` and stepping the cursor eight +pixels. A grid cannot do variable-width glyphs, cannot position text anywhere +except on a multiple of eight, cannot anti-alias, and cannot represent a +codepoint above 255. Swapping the bitmap for a nicer bitmap changes none of that. + +So the text layer fixes three things: + +- **Glyphs have metrics** — size, bearing, advance. That *is* variable-width + text; everything else follows from it. +- **It draws into a `struct surface`, not into the screen.** A compositor draws + off-screen and presents; rendering that writes straight to the VGA aperture is + unusable to it. +- **The font is behind `struct font_ops`.** The bitmap font is one + implementation and a rasteriser is another, arriving without any caller + changing — glyphs carry 8-bit coverage, so an anti-aliased outline renderer + already has somewhere to put its output. + +### The TrueType rasteriser + +`kernel/dev/ttf.c`: `glyf` outlines, composite glyphs, `cmap` format 4, +anti-aliased scanline fill, with a host harness at `tools/ttftest.c`. A +direct-mapped glyph cache over ASCII in a 48 KB pool. + +**The typefaces**, both licence-checked at source: + +| Font | Where | Licence | +|---|---|---| +| **JetBrains Mono** | terminal and TTY | SIL OFL 1.1 | +| **Roboto** | desktop and compositor | Apache-2.0 | + +JetBrains Mono declares **no Reserved Font Name**, which matters more than it +sounds: pre-rasterising a font into bitmaps at build time is a modification, and +an RFN would forbid keeping the name on the result. + +## The console + +`kernel/core/console.c`. A cursor, scrolling, and an ANSI escape parser +including SGR colours and the erase operations. + +**`cons_serial_only` starts set.** The display is the product; the serial line +is the diagnostics. A machine that boots by printing `pci: 6 devices` and +`vga: display controller 1234:1111, aperture fd000000` to the screen the user is +looking at is showing them a developer's log, and that single fact is most of +what makes an OS look like a hobby project. Everything still goes out the UART. +`console_takeover()` clears the flag on the first `exec()`. + +The console still uses the 8×16 cell grid. Porting it onto the text layer and +deleting the grid is open work. + +### Input + +Keys reach both `/dev/input/eventN` and the tty line discipline, which is +correct for two consumers of one keyboard and completely wrong for a full-screen +program — the shell behind it quietly accumulates every keystroke and runs the +lot at the next Enter. + +`EVIOCGRAB` is the fix and it is a primitive rather than a patch. Three details +matter more than the feature: + +- **The grab is released by the `close` hook, not by the program.** A program + that grabs the keyboard and then crashes would otherwise leave the console + permanently deaf, with no way to type the command that fixes it. +- **Modifier state is still tracked while grabbed**, or releasing the grab hands + the line discipline a stuck Shift. +- `/dev/input/eventN` **drops its backlog on open** and only queues while + something has it open. Without that, opening the device replayed the + keystrokes of the command line that had launched the program. + +## PSOGM — the crash screen + +`kernel/core/panic.c`. The Purple Screen of Guru Meditation. Built around three +decisions: + +**It does not clear the screen.** Whatever was displayed stays, dimmed and +pushed back, with the report floating over it. A crash screen that wipes the +display tells you the machine is gone; one that leaves your work visible tells +you it stopped — which is both truer and much less alarming. + +**It is not frightening.** No red, no capitals, no "FATAL". A calm sentence, a +face that is obviously a cartoon, then the facts. Whoever is reading this is +already having a bad time. + +**It says considerably more than it needs to.** The screen is the only artefact +of a panic — no core dump, no log to go back to — so everything cheap to collect +is collected: stop code, fault address, which CPU, which process, how far into +the boot, where the kernel is loaded, and a call chain. Anything left out is +gone. All of it goes to the serial line in the same order, which is what +`bootcheck` reads and what you get when the crash happens before there is a +framebuffer at all. + +Once painted, `panicked` is set and `consputc` spins rather than writing, so +nothing can scribble over the report. + +### The QR code + +`kernel/core/qr.c`. **A panic report is worth nothing if it cannot leave a +machine whose network stack is dead**, and a phone camera is an out-of-band +channel needing no driver on our side. + +It encodes the **report itself**, not a URL — it reaches nothing. Deliberately +small in three ways: byte mode only, EC level L only (the screen is a clean, +evenly lit, high-contrast source; the damage QR's stronger levels protect +against does not apply), and versions 1–6 only, stopping short of version 7's +extra version-information blocks and version 10's wider count field. + +134 bytes is a large crash report. If a payload ever needs more, the honest fix +is to shorten the report. + +It draws unconditionally rather than on a keypress, which is better: it costs +nothing, and requiring input would mean the one case where input is least +trustworthy is also the case where the report is hardest to retrieve. Verified +by `make crashshot && make crashqr`, which decodes the symbol out of the +captured framebuffer. + +## One palette + +[`include/theme.h`](../../include/theme.h). There were three unrelated designs +and the seams showed: the splash was deep purple with gold text deliberately +matching the *panic* screen, the console was cream with near-black text, and so +a healthy boot was dressed as a crash and then abruptly became an amber +terminal. + +> **The rule: normal states are calm and dark; only failure is loud.** + +Boot, console and shell share `SCH_BG`/`SCH_FG` — near-black with a slight blue +cast rather than pure `#000`, because pure black on an LCD reads as "the machine +is off". PSOGM keeps the full-bleed purple entirely to itself, which is what +makes it mean something. + +--- + +## What it does not do + +- **The console still has its cell grid.** The text layer exists and the console + does not use it; the rasteriser exists and no font is loaded from disk. +- **No mode setting.** Resolution is whatever the loader asked for. There is no + `/dev/dri`-style device and no way to change it at runtime. +- **No double buffering in practice.** `display_ops.acquire` is specified and + the compositor needs it; virtio-gpu binds but exposes no modeset API, and + `wait_vblank` is unimplemented everywhere. +- **No hardware acceleration of any kind**, and none is planned before 2D blit. +- **No CFF/Type 2 charstrings**, so OTF fonts do not render — that is a separate + `font_ops` provider and a bytecode interpreter, not an extension of the `glyf` + path. TrueType hinting is deliberately out of scope. +- **No text layout library.** Advance widths exist per glyph; kerning, shaping + and line breaking do not. +- **The glyph cache covers ASCII only**, direct-mapped. A codepoint outside it + is rasterised every time. +- **PSOGM renders and stops.** The interactive crash mode described in + [../plan/roadmap.md](../plan/roadmap.md) — walk the stack, read another + process's memory, dump page tables — is designed and not built. diff --git a/docs/architecture/init.md b/docs/architecture/init.md new file mode 100644 index 0000000..20cdc9f --- /dev/null +++ b/docs/architecture/init.md @@ -0,0 +1,155 @@ +# `schinit` — PID 1 + +Shaped after launchd rather than sysvinit. The reason is not the configuration +format. + +--- + +## Why launchd + +**Demand launching.** The supervisor holds the listening socket or watches the +path, and starts a job when something actually arrives. + +That inverts the dependency graph. A job stops needing to declare what must be +running before it, because the thing it depends on answers on first use — and +runlevels stop being a concept at all. + +SchOS has the two pieces that makes possible: `AF_UNIX` sockets and a real +`/proc`. **This milestone is the supervisor; demand launching is the next one.** +Socket sections are parsed and remembered but not yet held. + +## Constraints on the program itself + +`cmd/schinit/main.c` links against `lib/`, which is raw syscall stubs. **There is +no libc here** — no `malloc`, no `printf`, no `strlen`. Everything is static +storage and hand-rolled, which is also why the limits are constants rather than +growable: + +| | | +|---|---| +| `MAXJOBS` | 32 | +| `MAXARGS` | 16 | +| manifest size | 4 KB, read into one static buffer | + +## Bootstrap + +Before any job runs, PID 1 creates the device nodes and mounts that a job needs +in order to have somewhere to send its output. **This cannot itself be expressed +as a job** without a bootstrapping problem, which is why it is a function and not +a manifest. + +Console first, as fd 0/1/2, so every message after it has somewhere to go. Then +the `/dev` nodes — minor numbers under major 1 follow Linux's mem-major layout +where it costs nothing (`null=2`, `zero=5`, `full=7`, `random=8`, `urandom=9`) — +then `/dev/input/event0` and `event1`, the raw disks, and `/proc`. + +Two details that look like carelessness and are not: + +**The nodes are created unconditionally, including for absent hardware.** Opening +`/dev/hdb` with no drive present fails with `ENXIO`, which is a better answer +than a missing path. It matters more than it looks: with no node, +`echo x > /dev/hdb` silently creates an ordinary *file* of that name and appears +to succeed. + +**The `mknod`'d `/proc` files still happen and are then covered by the mount.** +A failure to mount degrades to the old behaviour rather than losing `/proc` +entirely. + +## Job manifests + +Every `*.job` file in `/etc/schinit` is loaded at boot, found with `getdents64` +rather than a hardcoded list — so dropping a file in the directory is all it +takes to add a job. + +Sectioned `key = value`, `#` starts a comment, keys are case-insensitive +(a manifest may say `KeepAlive`, `keepalive` or `KEEPALIVE`; keys are for humans +and this is not a language). + +| Key | | +|---|---| +| `Label` | A reverse-DNS name, as launchd uses. Identity, not a path. | +| `Program` | The binary to execute | +| `Args` | argv, space separated. argv[0] is conventionally the basename, so `Args = sh` for `/bin/sh` | +| `RunAtLoad` | Start it when schinit loads | +| `KeepAlive` | Restart it when it exits | +| `Disabled` | Load it but do not start it | +| `ThrottleInterval` | Seconds between respawns (default 1) | +| `ControllingTerminal` | Claim the console | +| `StandardIn` / `StandardOut` / `StandardError` | Paths to open as fd 0, 1, 2 | +| `[Socket ]` → `Path` | Parsed, recorded, not yet honoured | + +**Unknown keys and unknown sections are ignored rather than fatal.** A manifest +written for a newer schinit must not stop this one booting the machine. + +## Starting a job + +Each job gets **its own session** (`setsid`), so it cannot reach back and signal +the rest of the system through a shared controlling terminal. + +That is exactly what makes `ControllingTerminal` necessary, and the interaction +is the least obvious thing in the program: + +> A fresh session has no controlling terminal, and the console only delivers +> input to the foreground process group it is tracking. Without +> `ControllingTerminal = yes`, the shell comes up, echoes every keystroke through +> the line discipline, and executes none of them — which looks like a hung shell +> and is really a terminal that is not listening to it. + +It is **off by default**. A daemon has no business seizing the console, and +launchd's model is that the console is something a job asks for. + +`exec.c` enables the Linux ABI automatically for musl binaries, so a job must +not call `enable_linux_abi()` itself — it would renumber the syscalls the child +still has to make. + +## Supervision + +One loop around `wait()`. + +**It reaps whatever exited, including processes reparented here when their own +parent died.** Those are not jobs, `job_by_pid` returns 0 for them, and that is +the whole handling they need. + +A `KeepAlive` job is restarted, subject to `ThrottleInterval` — a job that exits +immediately and forever would otherwise spin the machine. launchd defaults to +10 seconds; 1 is enough here, where the common case is a shell someone just +exited on purpose and expects back. + +When `wait()` returns -1 there is nothing left to wait for, which should not +happen while any job is alive. It sleeps rather than spinning, so a mistake +there costs an idle machine and not a hot CPU. + +## The fallback + +With no manifests found, schinit says so and starts `/bin/sh` on the console +anyway. + +> An unbootable-looking machine because a manifest was missing would be a poor +> trade for purity. + +## PID 1 cannot be killed + +Enforced in the kernel, not here. `init_ignores()` discards any signal whose +disposition is the default action when the target is PID 1, **including +`SIGKILL`** — see [kernel.md](kernel.md#signals) for why Ctrl-C is the input that +finds that window. A handler init did install is still delivered. + +--- + +## What it does not do + +- **Socket activation is not implemented.** The manifest key is parsed and the + path remembered; nothing holds the socket, so demand launching — the entire + reason for copying launchd — does not happen yet. +- **No `schinitctl`.** Jobs cannot be started, stopped, reloaded or inspected at + runtime. The only interface is rebooting with a different manifest. +- **The job table is invisible.** Nothing reports what is loaded, what is + running, or how many times something has respawned. +- **No dependency ordering of any kind**, deliberately — demand launching is + meant to make it unnecessary, and until that exists there is nothing in its + place either. +- **No log capture.** `StandardOut` is a path, so a job's output goes wherever + the manifest says and nothing aggregates it. +- **A crash in schinit is fatal to the machine**, because `exit()` panics on + initproc. Fixing that is [../plan/init.md](../plan/init.md). +- **32 jobs, 16 arguments, 4 KB manifests**, all compile-time. diff --git a/docs/architecture/kernel.md b/docs/architecture/kernel.md new file mode 100644 index 0000000..85fc5f0 --- /dev/null +++ b/docs/architecture/kernel.md @@ -0,0 +1,211 @@ +# SchIFT — the kernel + +Long mode, 4-level paging, preemptive multitasking, and more than one +processor. What follows is the parts of the kernel that are not a filesystem, a +driver or a protocol stack: memory, processes, scheduling, traps and signals. + +--- + +## Address space + +One page table per process; the kernel is mapped into all of them at +`KERNBASE = 0xFFFF800000000000`, so a trap does not switch `CR3`. + +`kvmalloc()` maps physical 0–4 GB as four 1 GB pages, with the top one marked +uncached — that is where PCI BARs and the framebuffer live. This is why 3 GB is +the most memory the kernel can treat as RAM, and it is a deliberate simplicity: +a direct map with a fixed offset makes `P2V`/`V2P` arithmetic rather than a +lookup. + +`kalloc()` hands out one 4 KB page at a time from a free list. **There is no +slab or pool allocator**, and the consequence is visible throughout the tree: +every subsystem that needs many small objects has invented its own fixed-size +table — `NPROC` processes, `NSOCK` sockets, `NDEVICE` devices, `NSHM` segments. +That is why the limits are compile-time constants rather than a design decision +about each one. + +The free list carries a canary, checked at the top of both `kalloc()` and +`kfree()`, so a wild write into it panics at the point it is touched rather than +handing back a bad pointer that fails somewhere else entirely. + +### DMA + +`kernel/core/dma.c` owns an arena carved off the top of physical memory before +`kalloc` is given the rest. Devices need physically contiguous memory; `kalloc` +cannot promise it, so the region has to be reserved before there is any +competition for it. + +## Per-CPU state + +The per-CPU block's base is in **both** `FS_BASE` and `GS_BASE`, set by +`seginit()`: + +``` + +0 self-pointer required by the x86-64 TLS ABI (%fs:0) + +8 syscall scratch trapasm.S parks %rsp here across the stack switch + +16 struct cpu * + +24 struct proc * +``` + +**These offsets are shared with assembly and must not be reordered.** Offset 8 +is not free: `syscall_entry` uses `%fs:(8)`, and because FS and GS base point at +the same address, that is the same memory as `%gs:8`. + +Reached through `mycpu()` and `myproc()`, which compile to a single +`movq %gs:16` / `%gs:24` with no cached base. + +> **`__thread` is wrong for per-CPU kernel data**, and this is the single most +> expensive lesson in the tree. `__thread` means per *thread*, and the compiler +> is entitled to read the thread pointer once and keep it in a register across +> calls. But `%fs` here is per *CPU*, and a kernel thread changes CPU at every +> `sched()`. So the assumption is false for exactly the variables that most need +> to be right, and the failure is a null dereference deep in the scheduler on any +> machine with more than one core. `include/proc.h` carries the disassembly that +> proves it. + +## Processes + +`struct proc` in [`include/proc.h`](../../include/proc.h). A fixed table of +`NPROC` entries, allocated by linear scan. + +**The field order at the top of the struct is fixed by assembly.** `trapasm.S` +hard-codes `fs_base` at offset `0x18`, and there is a `_Static_assert` in +`proc.c` that fails the build if it moves. Anything new goes at the end. + +Each process carries: page directory and size, kernel stack, open file table, +cwd inode *and* cwd path string, signal handlers and masks, VMA table for +`mmap`, POSIX identity (pgid, sid, and all six of real/effective/saved uid and +gid), and the Linux-ABI syscall map. + +### Slot reuse + +The table recycles slots, and a recycled slot that inherits the previous +occupant's state is a whole class of bug that has bitten here twice. +`allocproc()` therefore explicitly clears the signal fields, the exit status, +the kill signal, and the identity fields. The failure mode when it did not: +`deliver_signals()` jumping to a totally unrelated earlier process's handler +address on this process's very first syscall return. + +### Kernel threads + +`kthread_create(fn, name)` makes a process that runs `fn` in kernel mode and +never returns to userland. It gets a page directory with only the kernel mapped, +no files, no cwd, no parent — it is not in the process tree, cannot be waited +for, and must never exit. + +The point is that it is a *process* and not a timer callback: it shares the +scheduler's ordinary rotation, so it can hold sleeplocks and block on the disk +exactly as a syscall would. The network thread is the current user. + +## Scheduling + +Round-robin over the process table, one scheduler per CPU, each with its own +context. `swtch()` saves callee-saved registers and switches stacks; `sched()` +returns to the scheduler and asserts four invariants on the way — holding +`ptable.lock`, `ncli == 1`, not `RUNNING`, interrupts off. + +When a full pass finds nothing runnable the CPU halts rather than spinning. + +**Locking is a single `ptable.lock` over the whole table.** Not per-process: a +scheduling operation touches two processes (the one leaving and the one +arriving) and two-object locking needs an ordering rule that every path obeys. +The same reasoning produced a single lock over the socket table. + +### sleep / wakeup + +xv6's discipline, unchanged because it is correct: `sleep(chan, lk)` atomically +releases `lk` and blocks; `wakeup(chan)` makes every sleeper on that channel +runnable. The atomicity is what matters — a sleeper that tests its condition +under one lock while the waker sets it under another has no interlock and sleeps +through the wakeup meant for it. + +## Traps and syscalls + +Interrupts and exceptions enter through `vectors.S` → `trapasm.S` → `trap()`. +Syscalls enter through `SYSCALL`/`SYSRET` and `syscall_entry`, which is a +separate and faster path. + +Every entry from user mode checks a **canary at the bottom of the kernel stack +page**. A deep or nested trap that overflows the 4 KB stack clobbers that word +first, so the failure is reported as `kstack overflow` at the point it happened +rather than as arbitrary corruption later. + +One configuration detail with an expensive history: `MSR_STAR`'s `SYSRET` +selector field must hold the **compat 32-bit** code selector, because `SYSRETQ` +computes the returned `cs`/`ss` as `STAR[63:48]+16`/`+8`. Putting the real +64-bit user selector there makes every `sysretq` load an invalid pair — which +QEMU's TCG does not validate, so it stayed hidden until a hardware interrupt +forced a strictly-validated `IRETQ`. + +## Signals + +Real delivery, not stubs. `sys_rt_sigaction()` records the handler; +`deliver_signals()` runs in the tail of `syscall()`, just before returning to +user mode, and if a pending unblocked signal has a handler it rewrites the +trapframe to enter the handler instead of resuming — stashing the original in +`proc->saved_sigtf`. `sys_rt_sigreturn()` puts it back. + +The return address is **not** `sa_restorer`. `SA_RESTORER` is not defined in +this musl build, so `sigaction()` never fills one in, and pushing 0 as a return +address crashes with `rip = 0` the moment the handler returns. Instead a 7-byte +trampoline (`mov eax,15; syscall`) is injected onto the user's own stack below +`rsp`, clear of the SysV red zone — the standard technique when the C library +supplies no restorer. + +Signals without a handler fall back to a default action: termination for the +fatal set, dropped otherwise. + +### PID 1 does not die from a signal it never asked to handle + +`init_ignores()` discards any signal whose disposition is the default action +when the target is PID 1 — **including `SIGKILL`**. This is what POSIX requires +and what Linux does in `sig_task_ignored()`, and without it the system has a +single keystroke that kills it: Ctrl-C signals the foreground process group, +init is pgid 1, and any moment where the terminal's foreground group is still 1 +lands SIGINT on init. It has no handler, so `exit()` runs, and `exit()` panics +outright on initproc. + +Spamming Ctrl-C at a frozen program is exactly the input that finds that window. + +## Process groups and sessions + +`setsid`, `setpgid`, `getpgid`, `getsid`, and signal delivery to a whole group. +Job control is defined entirely in terms of these — a shell puts each pipeline +in its own process group and signals the group, never individual pids — and +mksh's job control could not make progress until they existed. + +## What userspace gets + +`fork`, `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and real pid selection, +pipes, signals, `mmap` (anonymous and file-backed), POSIX shared memory, Unix +domain sockets, `poll`/`select`/`epoll`, `eventfd`, `timerfd`, `signalfd`. + +`mmap` uses a bump allocator (`proc->mmap_next`, starting at `0x40000000`) so +every mapping gets its own disjoint, exactly-sized range. It previously assigned +a fixed 8 MB window per VMA slot regardless of the requested length, so a single +allocation over 8 MB mapped straight into a neighbouring live mapping. + +--- + +## What it does not do + +- **No slab allocator.** One page at a time, and every subsystem's fixed-size + table is a consequence. This is the single change that would most improve the + rest of the kernel. +- **No demand paging and no page cache.** `mmap` of a file *reads the whole + thing in* at map time. Writable `MAP_SHARED` of a file is therefore refused + outright rather than faked — see [../plan/foundations.md](../plan/foundations.md). +- **No `sendmsg`/`recvmsg`, so no `SCM_RIGHTS`.** File descriptors cannot be + passed between processes. +- **No dynamic linking.** `exec` handles `ET_EXEC` only: no `PT_INTERP`, no + `ET_DYN`, no shared libraries, no `dlopen`. +- **No threads.** `clone` exists but does not share an address space; + `gettid` returns the pid. +- **Permissions are tracked but not enforced.** uid/gid are maintained honestly + so `getuid()`/`setuid()` stop lying, and nothing consults them on access. +- **No swap**, and no memory pressure handling of any kind. `kalloc` failing is + reported up as `-ENOMEM` and that is the whole policy. +- **Nested signal delivery is not supported.** `in_signal` guards against + re-entering a handler while one is running. +- **Scheduling is round-robin with no priorities**, no fairness accounting, and + no CPU affinity despite `sched_setaffinity` existing. diff --git a/docs/architecture/networking.md b/docs/architecture/networking.md new file mode 100644 index 0000000..9a1dc12 --- /dev/null +++ b/docs/architecture/networking.md @@ -0,0 +1,260 @@ +# Networking + +ARP, IPv4, ICMP, UDP, TCP and DHCP, over a driver-agnostic interface class, +reached through the same socket syscalls as Unix domain sockets. + +Verified end to end rather than by inspection: `bootcheck -Net` starts a +listener and a UDP responder on the host, tells the guest to talk to them, and +asserts on **both** ends — one side alone cannot tell a working stack from one +that completes a handshake and drops every byte. + +--- + +## The four layers + +``` + socket() bind() connect() send() recv() syscalls, one set + ┌────────────────┴───────────────────┐ + │ AF_UNIX AF_INET │ socknew() dispatches on domain + │ kernel/net/unix.c kernel/net/inet.c + └────────────────┬───────────────────┘ + │ lwIP raw API + ┌────────────────┴───────────────────┐ + │ lwIP 2.2.0, NO_SYS=1 │ ports/lwip, compiled into the kernel + │ port layer: kernel/net/lwip/ │ ~400 lines: a lock, a clock, a netif + └────────────────┬───────────────────┘ + │ struct netdev_ops + ┌────────────────┴───────────────────┐ + │ e1000, and any future NIC driver │ + └────────────────────────────────────┘ +``` + +## The netdev class + +A driver publishes a [`struct netdev_ops`](../../include/netdev.h) and is then +finished with the network stack forever: it never mentions lwIP, and the stack +never mentions it. + +```c +struct netdev_ops { + int (*transmit)(struct device *, const void *frame, int len); + int (*receive) (struct device *, void *out, int cap); + const unsigned char *(*mac)(struct device *); + int (*link_up) (struct device *); // optional +}; +``` + +`transmit` returning -1 is **congestion, not failure** — the ring is full and +the caller is expected to retry. `receive` never blocks; it returns 0 when +nothing is waiting, and the caller decides how to wait. + +This class was added late, and the reason it had to be is instructive: the stack +had reached straight into the driver. `net.c` called `e1000_present()`, the lwIP +glue was named `netif_e1000.c`, and the e1000 shipped with `.ops = 0` and a +comment saying so. A second NIC driver would have meant editing the protocol +stack to add itself. lwIP now gets one netif per bound device, `/proc/netif` +lists them all, and the e1000 drives up to four cards rather than refusing the +second — verified with two attached, each with its own MAC and its own DHCP +lease. + +### `netdev_selftest()` + +`echo n > /proc/sysrq` sends a hand-built ARP request and waits for the reply. + +It proves the whole driver path in one exchange — descriptor formats, the DD +handshake, tail bookkeeping, DMA addresses, the receive ring hand-back, and the +MAC filter, because an ARP reply is addressed to this station and does not +arrive if the filter is wrong. + +It was written *before* there was any protocol stack, deliberately: a stack on +an unproven driver fails in ways indistinguishable from its own bugs. It belongs +to the class rather than to the e1000, because it is the check every new NIC +driver will want first. + +## lwIP + +lwIP 2.2.0, fetched by `ports/lwip` and **compiled into the kernel** — it is the +one port that installs nothing into the sysroot, because it is kernel code. + +`NO_SYS=1`, so lwIP has no threads, mailboxes or semaphores of its own. The port +layer is `kernel/net/lwip/`: an architecture header, a lock, a clock, an +Ethernet netif, and a set of shim headers standing in for the libc lwIP expects. + +### Why lwIP and not a BSD stack + +The licence was not the deciding factor — FreeBSD's `sys/netinet` is BSD-3 too. +**It is that FreeBSD's stack is not a library.** `tcp_input.c` alone is 122 KB +and reaches into mbufs, `epoch(9)`, netisr, VNET, `uma(9)`, sockbufs and inpcb +hash tables. Taking it means porting FreeBSD's kernel infrastructure first and +finding the TCP somewhere underneath — the same conclusion already reached about +FreeBSD's USB stack. + +NetBSD was ruled out on licence: its `tcp_input.c` and `tcp_output.c` carry the +WIDE Project's 4-clause advertising clause, which Berkeley's rescission does not +reach. See [../licensing.md](../licensing.md) §3. + +lwIP was written for exactly the case SchOS is in: a small operating system with +a raw Ethernet driver and no infrastructure to speak of. + +**The reason not to write one, kept because it is the reason to resist later +too:** it is not the packet formats that are hard. It is congestion control, +retransmission, window management and twenty years of corner cases that only +appear against a real peer. + +## The network thread + +`kernel/net/net.c`. One kernel thread, started by `netinit()`, feeds lwIP +received frames and runs its timers. + +**A thread rather than the interrupt handler, and the reason is the lock:** a +call into lwIP can reach the driver, the driver can sleep, and an interrupt +handler cannot. So the interrupt keeps doing what it always did — clear the +cause register — and this thread polls. + +`netinit()` runs after `sinit()` because `kthread_create` needs a usable process +table, and it returns immediately: everything happens in that thread, which +cannot run until the scheduler is entered. + +DHCP obtains an address at boot. `/proc/netif` reports each interface's address, +netmask, gateway, MAC and frame counts, taken at the lwIP seam. + +> It is `/proc/netif` and not `/proc/net` deliberately: on Linux `/proc/net` is a +> directory of many files, and a single file by that name would be a lie about a +> well-known path. + +## Sockets + +One set of syscalls, at their real Linux numbers. `socknew()` dispatches on the +domain, and **no syscall changes shape** between the two families. + +### AF_UNIX + +`kernel/net/unix.c`. `SOCK_STREAM`, `socketpair`, `shutdown`. + +A connected pair is **two buffers, not one**: writing appends to the peer's +buffer and reading drains your own, which is what makes a socket bidirectional +where a pipe is not, without needing two of anything. + +**One lock covers the whole table.** A pipe gets away with a lock per pipe +because every operation touches one object; a socket operation touches two, and +two-object locking needs an ordering rule every path obeys or it deadlocks. +Worse, the wakeup races are subtle — a reader testing `peer_closed` under its own +lock while the closer sets it under the peer's has no interlock at all. + +**Names live in a table here, not in the filesystem.** Binding creates no +directory entry. The table gives the behaviour that matters — a client finds a +server by a name agreed in advance — and what is lost is `ls -l` showing an `s`, +and `unlink()` of a socket path working. Both want a VFS change rather than a +socket change. + +### AF_INET + +`kernel/net/inet.c`, on lwIP's **raw** API rather than lwIP's own sockets layer, +which would need `NO_SYS=0` and a second socket abstraction alongside this +kernel's own. + +**Everything is protected by the lwIP lock and nothing else**, and that is not a +shortcut: lwIP's callbacks run in the network thread with that lock held and are +the only writers to a socket's receive buffer. A syscall reading it takes the +same lock. There is no second lock because there is no second writer. + +#### How a blocking socket waits + +Worth reading before touching, because the obvious implementation does not work. +A blocking call cannot `sleep(&socket, &lwip_lock)`: `sleep()` wants a spinlock +and the lwIP lock is a *sleeplock* — it has to be, because a call into lwIP can +reach the driver. And a waiter cannot hold that lock while sleeping, or the +network thread could never deliver the thing being waited for. + +So the wait is on a **counter** with a spinlock of its own: + +``` + reader: lwip_lock; check condition; s = net_seq(); lwip_unlock; net_wait(s) + waker: (inside a callback, lwIP lock held) net_wake() +``` + +No wakeup can be lost. `s` is read while the reader still holds the lwIP lock, +so anything changing the condition must do so afterwards and therefore bumps the +counter after the read; `net_wait` re-checks under its own spinlock and does not +sleep. + +One counter and one channel for **all** inet sockets rather than one each. At +this scale waking sixteen sockets costs a handful of instructions in the ones +with nothing to do, and a per-socket channel would need careful teardown in +`close()` against callbacks that can still arrive. + +Before this, a blocked socket woke on every timer tick and looked — costing a +wakeup per socket per 10 ms and answering up to 10 ms late even when the data +was already there. `tcp_sent` had to be registered for it as well, or a write +blocked on a full send buffer slept with nobody left to wake it. + +#### ICMP, and ping sockets + +`socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP)` — Linux's *ping socket*, which gives +an unprivileged process the one thing raw sockets were used for. The caller +reads and writes the ICMP message alone; the kernel owns the IP header. + +Built on lwIP's raw API: `raw_new(IP_PROTO_ICMP)`, with the receive callback +stripping the variable-length IP header before queueing the message on the same +datagram queue UDP uses. + +**Only echo replies are consumed**, and that restriction is load-bearing. +`raw_input` runs *before* lwIP's own `icmp_input`, so a callback returning +"eaten" for every ICMP message would swallow the echo **requests** too and this +machine would silently stop answering pings from outside. Returning 0 leaves the +packet for lwIP, which is what should happen to every ICMP message nobody asked +for. + +There is no per-socket echo-id filtering. Linux rewrites the id and matches on +it; here a reply goes to every open ICMP socket, which is invisible with one +ping running and means a second one may see a sequence number it never sent. + +### Loopback + +`127.0.0.1` is a real destination: `LWIP_HAVE_LOOPIF` creates the interface and +`lwip_init()` brings it up. `LWIP_NETIF_LOOPBACK` is the half that is easy to +miss — without it, a packet addressed to an interface's *own* address goes out +of the wire and never comes back. + +With `NO_SYS=1` there is no thread to drain the loopback queue, so the network +thread calls `netif_poll_all()` alongside `sys_check_timeouts()`. + +### Datagrams + +UDP has a **datagram queue**, not a byte ring. It shared TCP's ring at first, and +two datagrams arriving together came back as one — which is the bug that had to +be fixed before `recvfrom` could report a source address at all. + +--- + +## What it does not do + +- **`ping` is a first-party program**, `cmd/ping`, not toybox's. toybox's reads + replies with `recvmsg(2)` and asks for the TTL through an `IP_RECVTTL` control + message, and neither exists — `recvmsg` is item 1 of + [../plan/foundations.md](../plan/foundations.md), where it is wanted for + `SCM_RIGHTS` anyway. Ours has no `-I`, no flood mode and no TTL display. +- **No resolver reachable from userland.** lwIP's DNS client is compiled but + nothing exposes it, so `ping` takes a dotted quad. `localhost` is special-cased + in the program rather than resolved. +- **Receive is polled.** The network thread looks at the NIC every timer tick + when the link is idle, so an arriving frame waits up to 10 ms. Waking the + thread from the e1000 interrupt is the real fix, and was deliberately not the + first version: a receive path woken by an interrupt is much harder to debug + than one that is asked. +- **No routing table.** With two interfaces up, the first is the default route + and the second is reachable only by an address on its own subnet. That is a + rule rather than an accident, and it is not routing. +- **No `getsockname`/`getpeername`**, and `accept()` does not report the peer's + address for an AF_INET socket. A server that wants to log who connected cannot. +- **`setsockopt` covers six options**, and refuses the rest with `ENOPROTOOPT` + rather than pretending: `SO_REUSEADDR`, `SO_KEEPALIVE`, `SO_BROADCAST` and + `TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` are accepted and + report the real size on read. Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`. +- **`SOCK_DGRAM` over AF_UNIX does not exist** — stream only. +- **AF_UNIX names are not filesystem paths**, so nothing can `stat()` or + `unlink()` a socket. +- **One NIC driver exists.** The class is proven with two e1000 cards, not with + two different drivers. +- **No IPv6**, no DNS resolver in the kernel (lwIP's is compiled but unused by + anything), and no firewall or packet filter of any kind. diff --git a/docs/architecture/schfs.md b/docs/architecture/schfs.md index a40ed3b..db312b5 100644 --- a/docs/architecture/schfs.md +++ b/docs/architecture/schfs.md @@ -1,21 +1,31 @@ -# SchFS — design +# SchFS -**Status: phases 1 through 3½ are done, and SchFS is the root filesystem.** -`kernel/proc/proc.c` mounts it before pid 1 has a working directory and panics -if it is not there. The copy-on-write B-tree, the volume layer, the object -layer, the write path, snapshots, an image writer and a checker all exist, and -several volumes can be mounted at once. Phase 4 (extent refcounts) is next. -See §8 for exactly what is built and how to run its tests. +The root filesystem. A copy-on-write B-tree with an uberblock ring, 64-bit +object ids, checksummed metadata and extents. -Supersedes the on-disk format described in `include/fs.h`. Keeps the name SchFS; -the on-disk magic changes so v1 images are detected and rejected rather than -silently misread. +**It is the root.** `forkret()` mounts it before PID 1 has a working directory +and panics if it is not there. All five design phases are complete: the B-tree, +read-only mount, the write path and transaction groups, the commit barrier and +uberblock fallback, extent refcounts and reflink, and attributes with an index +and a query. -The brief was "the best aspects of BeFS, ZFS and btrfs." Those three disagree -with each other in places, so this document says explicitly what is taken from -each and what is deliberately left behind. +The phases themselves, and the bugs they produced, are in +[../history/schfs.md](../history/schfs.md). The VFS it plugs into is +[filesystem.md](filesystem.md). -## 1. What each one is actually good at +--- + +## The argument in one line + +A crash leaves either the old tree or the new one, so **there is nothing to +repair** — and free space is derived from the tree rather than stored beside it, +so **there is no second structure to disagree with the first**. + +`/proc/schfs` shows the generation counter advancing as writes commit. + +--- + +## What was taken, from where **ZFS — never be inconsistent.** Copy-on-write: no block that is part of the live tree is ever overwritten. An update writes new blocks, then atomically @@ -41,20 +51,20 @@ also the right *scale* model: a small team, a readable codebase, and a book (Giampaolo, *Practical File System Design*, free online) that describes it end to end. -## 2. What is deliberately not taken +## What was deliberately not taken - **No ZFS storage pools / RAID-Z / vdevs.** Multi-disk redundancy is a large subsystem that solves a problem SchOS does not have yet. - **No btrfs-style rebalancing or online format conversion.** Enormous complexity, and it is the historical source of btrfs's worst bugs. -- **No actual SQL.** No parser, no query planner, no relational engine. See §6 — +- **No actual SQL.** No parser, no query planner, no relational engine. See *Attributes, indexes and the query* below — the useful part of "database filesystem" is the *index*, not the language. WinFS spent 2003-2006 and a large team on the other interpretation and shipped nothing. - **No deduplication.** ZFS's dedup is famous for needing more RAM than anyone expects. Extent refcounts give reflink/snapshot benefits without it. -## 3. On-disk shape +## On-disk shape ``` +--------------------+ block 0 boot block (untouched by the FS) @@ -98,278 +108,7 @@ checksum fields in `dinode` and `superblock` — same algorithm, better placemen Block size moves 1024 → 4096 to match modern media and make extents worthwhile. -## 4. What already exists and gets reused - -Not starting from zero: - -- **`struct schfs_extent`** is already defined in `include/fs.h` and unused. - The extent concept is already sketched; v2 gives it refcounts and 64-bit - fields. -- **CRC32c checksum fields** already exist in `dinode` and `superblock`, and - mkfs already computes them. Same algorithm, relocated into parent pointers. -- **`kernel/fs/log.c`** is a working transaction log with `begin_op`/`end_op` - call sites throughout the FS. Under CoW the log is not needed for consistency, - but `begin_op`/`end_op` are exactly the right places to bracket a transaction - group, so the *call sites* survive even though the implementation is replaced. -- **The buffer cache** (`bio.c`) is format-agnostic and needs no change beyond - the block-size constant. - -## 5. Phasing - -Each phase leaves a bootable system. Do not start phase 2 before phase 1 is -solid — the B-tree is the load-bearing piece and everything else assumes it. - -**Phase 1 — the B-tree, in userland first.** Write the CoW B-tree as a plain C -library with a block-device abstraction, and test it on a *host* file with a -fuzz/torture harness (random insert/delete/lookup, simulated crash at every -write point, verify the tree always mounts). This is the one component where a -subtle bug is unfixable later, and testing it in the kernel is far harder. It is -also portable straight into `tools/mkfs`. - -**Phase 2 — read-only mount. DONE.** New magic, uberblock scan, tree walk, inode -and dirent key types. `tools/mkschfs` writes images; the kernel only reads them. -The system boots from a ramdisk root (see `docs/layout.md` and the ramdisk work) -so a broken disk cannot stop you from booting. - -Went further than planned in two places, both cheaply: data blocks carry -checksums too (a `CSUM` key type, as btrfs does — without it "end-to-end -checksums" would cover only metadata, which is the part least likely to rot), -and extents landed early because reserving a file's blocks in one go was -simpler than not doing it. - -Not yet done: SchFS files are not reachable through `open(2)`. The mount is -observable through `/proc/schfs` and through the kernel API in `schfs_int.h`, -but wiring it into `namei`/`readi` means the VFS work that phase 3 needs anyway. - -**Phase 3 — writes, transaction groups. DONE.** Every `schfs_k*` call that -changes anything is one transaction group, committed by writing a new uberblock -before it returns. The full write API — create, link, unlink, rename, write at -an arbitrary offset, truncate — lives in `objw.c` and is exercised by -`tools/schfs_objtest.c`. - -Two things went differently from the plan above, both for reasons worth -recording. - -*Free space is not tracked as tree keys.* Allocating from a free map that lives -in the tree mutates the tree, which allocates, which changes the map; btrfs -resolves that with a fixpoint loop and ZFS with log-structured space maps. SchFS -sidesteps it: the committed tree already says exactly which blocks are live, so -the map is **derived** by walking the tree once at mount and held only in RAM. -It cannot disagree with the filesystem, because it is computed from it, and a -crash raises no question at all. The cost is mount time proportional to metadata -size, and a ceiling on volume size set by how much bitmap fits in memory. If -mount time ever hurts, the map can be cached on disk purely as a hint, -validated against the generation and discarded on a mismatch — an addition, not -a format change. See `kernel/fs/schfs/space.c`. - -*Data blocks needed a generation stamp too.* The B-tree already rewrites its own -nodes in place within a group, since a block that no committed root can reach is -nobody's fallback. Data blocks had no such marker, so every hundred-byte write -copied a whole 4 KB block and split an extent record around it. `schfs_extent` -gained a 64-bit `gen`, and the same rule now applies to file data. - -**Phase 3½ — reachable through `open(2)`. DONE.** SchFS is a mounted filesystem: - -``` -mkdir /mnt -mount -t schfs /dev/hdb /mnt -ls /mnt ; echo hi > /mnt/tmp/w ; cat /mnt/tmp/w ; mkdir /mnt/tmp/d -``` - -The VFS grew a real dispatch layer rather than a set of `if (dev == SCHFS_DEV)` -checks. `struct fs_ops` (include/file.h) is one table per filesystem, chosen by -`iget()` from the device number and hung off the inode; v1's implementations -keep their bodies and are simply named `v1_*`. Adding a third filesystem is a -new table and nothing else. `kernel/fs/schfs/vfs.c` is SchFS's, and it is pure -translation — object ids in, inodes out — because all the filesystem logic -already lives below it. - -Three things had to change on the way, none of them SchFS's fault: - -- `struct inode.inum` and `.size` were 32-bit, capping objects at 2³² and files - at 4 GiB on a filesystem that promises far more. Both are `uint64` now, as are - `readi`/`writei` offsets and `struct file.off`. -- `struct dirent` carries a 16-bit inum and a 14-byte name, so it cannot - represent a SchFS entry at all. `getdents64` is now an fs_op: each filesystem - emits the record stream itself, and `f->off` is whatever position it needs to - resume. v1 walks its hash buckets; SchFS walks a key range and synthesises - "." and "..", which it does not store. -- Mounting is a substitution made during path resolution (`mount_add` / - `mount_cross` in `fs.c`). Nothing on either disk records it. The reverse - direction matters as much as the forward one: `..` at a mounted root has to - leave the mount, and the filesystem underneath has no idea it is mounted - anywhere. - -Three long-standing bugs surfaced the moment real commands ran against it, all -in the shared VFS rather than in SchFS: - -- `include/fcntl.h` used xv6's open flags while every program in the userland - sends Linux's. `O_APPEND` (0x400) landed on `O_TRUNC`, so `cmd >> file` - *truncated* it; `O_TRUNC` (0x200) landed on `O_CREATE`, so `cmd > file` left - whatever was past the new end in place. Nothing about either pointed at a - header. -- `sys_openat` silently dropped `O_TRUNC` and `O_APPEND` while `sys_open` - honoured both, so the same redirection behaved differently depending on which - entry point libc happened to call. -- `rmdir` had no Linux syscall number, so every Linux-ABI program got ENOSYS. - -**Phase 3¾ — durability, and the cost of it. DONE.** Three things that the -earlier phases had assumed rather than arranged. - -*The commit barrier.* Copy-on-write promises that a crash lands on either the -old tree or the new one, but only if the blocks a root points at reach the -medium *before* the block naming that root. Nothing enforced that: `bwrite` -order is not platter order on any drive with a write cache. `struct schfs_bdev` -gained a `bflush` hook, `schfs_vol_commit` calls it either side of the uberblock -write, and the kernel implements it as "drain this device's dirty buffers, then -ATA FLUSH CACHE" (`ideflush`, and SCSI SYNCHRONIZE CACHE for a USB stick). - -*Falling back through the ring.* Mount took the highest-generation uberblock -that validated, checked the tree it named, and gave up if that failed — with up -to 127 older, entirely intact roots sitting on the disk. It now walks the ring -newest-first and adopts the first root it can actually read, counting what it -skipped into `vol->recovered` so the rollback is reported rather than silent. - -*Deferred writes, and brackets.* The B-tree seals and writes a node on every -record it inserts, so a leaf collecting one file's data checksums went to the -disk once per 4 KB of that file. The kernel's block hook marks buffers dirty -instead (`bdirty`), and the commit barrier is what makes that safe. On its own -that changes little, because a commit *contains* a flush and every `schfs_k*` -call was one commit — so `schfs_ktx_open`/`schfs_ktx_close` bracket a run of -calls into one group, and `schfs_vfs_write` wraps a whole `write(2)` in one. - -Measured over 1 MB of file data, in block writes reaching the device: - -| | before | after | -|---|---|---| -| committing every 4 KB | 1947 | 1410 | -| one commit for the megabyte | 1798 | **267** | - -267 device writes for 256 blocks of data. Note that the two changes are worth -almost nothing separately — deferral cannot help while every commit flushes, -and batching cannot help while every write goes straight to the disk — which is -why the first attempt at this measured an 8% difference and concluded batching -was not the problem. - -**Phase 3⅞ — where the time actually went. DONE.** The filesystem was correct -and durable and the system still felt slow: `ls /bin` took the better part of a -second. Two causes, neither of them the disk. - -Measured first, through `/proc/fsstat` (counters added for exactly this). The -decisive number was **398,999 block reads against 445 actual disk misses** — -99.9% of all block reads were re-reading a block already in the buffer cache. So -the disk was never the problem, and every plan that started with "cache more" or -"read ahead" would have been aimed at 1% of the cost. - -*The cursor re-read its own buffer.* `cur_load` called `node_read` -unconditionally, and `schfs_cursor_val` calls `cur_load` on **every access to -the current record** — deliberately, so that a callback disturbing the shared -block buffer cannot corrupt a directory walk. But `node_read` copies 4096 bytes -out of the buffer cache and then runs CRC32c over all 4096. Listing a 243-entry -directory paid both once per entry. The cursor now remembers which block its -buffer holds and skips the reload, cross-checked against the block header (so a -second cursor sharing the page cannot fool it) and against a global epoch that -every mutation bumps (so a block rewritten in place cannot be served stale). - -*The checksum was bit-at-a-time.* `schfs_crc32c` did eight shifts per byte — -32,768 iterations to checksum one block — to stay usable before any -initialisation. It builds a 1 KB table on first use now, which keeps that -property and is eight times faster. This only became the top cost *after* the -cursor fix, which is the usual shape: removing the largest item promotes -whatever was beneath it. - -| cycles, one session | before | after | -|---|---|---| -| `getdents` | 26.8 G | **0.14 G** (196×) | -| `read` | 26.4 G | **5.5 G** (4.8×) | -| `exec` | 18.7 G | **4.0 G** (4.7×) | -| block reads | 398,999 | **127,713** | - -`ls /bin` went from ~400 ms to ~20 ms. Verified by reading an 802 KB binary -through the whole path and checking its md5 against the host's copy. - -There was also a third of the problem that was not the kernel at all: `make -qemu` ran QEMU without `-enable-kvm`, so every instruction was emulated. The run -targets probe `/dev/kvm` now. Measure against the KVM number or you are -measuring TCG. - -**Phase 4 — extents with refcounts. DONE.** `cp` that costs nothing. - -A reference count per run of blocks, in its own tree (`SCHFS_KEY_EXTREF` under -`SCHFS_EXTREF_OBJECTID`) rather than in the file's own extent record — because a -count kept per file cannot be decremented by the *other* file sharing the run. -`schfs_file_reflink` copies one extent record per run and bumps the count; both -files stay ordinary and writable, and the first write to either copies only the -run it touches. Reachable from userland as `FICLONE`, Linux's ioctl, so -`cp --reflink` works unmodified; `cmd/reflink` is a two-line caller of it. - -Measured on the running system: cloning a 411 KB file moved `df` by 24 KB. - -Three things had to be right, and each was wrong first: - -- **The count is "how many extent records point into this run", not "how many - files".** Splitting a shared extent around a written block leaves a head *and* - a tail, which is one more reference than before. Missing that leaked four - blocks per split. -- **A file's extent may point into the middle of a run.** Records are keyed by - the block a run was *allocated* at, so every lookup is a range query - (`extref_find`), not an exact one. An exact lookup finds nothing for a split - tail, "nothing" means unshared, and the tail is then freed while another file - still reaches it. -- **The space scan has to walk the reference records too.** A file truncated - inside a shared run describes fewer blocks than exist; the rest would be - derived as free while still referenced, and the allocator would hand out a - block that a later release frees a second time. - -None of those is visible until every holder of a run is deleted, which is why -the test that matters is the least glamorous one: after deleting every file that -shared a run, free space must be back to exactly what it was. - -**Phase 5 — attribute index and query. DONE.** The reason the design was one -B-tree with many key types from the start: the "database filesystem" feature -turned out to be `kernel/fs/schfs/attr.c` and nothing else. - -An attribute is a named value on an object, stored under `(ino, ATTR, -hash-of-name)` and packed exactly as directory entries are — same collision -argument, same shape. An attribute may be *indexed*, and an indexed attribute -gets a second set of records keyed by value: `(hash-of-name, INDEX, -hash-of-value)` holding the object ids that carry it. So "every object whose -`tag` is `holiday`" is one tree descent instead of a walk of every object. - -There is no query language, no parser and no planner, and §6 above explains why: -the useful half of "query the filesystem" is the index, and WinFS spent three -years and a large team on the other interpretation. - -``` -schattr index tag # declare the index -schattr set /tmp/a tag holiday -schattr find tag holiday # one descent, whatever the volume holds -``` - -Four decisions worth keeping: - -- **Which attributes are indexed is cached in the volume**, read once at mount. - Every `setxattr` has to ask, and the answer is almost always no; a tree - descent to say so would make every attribute write pay for a feature it is - not using. -- **The value is stored in the index entry, not just its hash.** Two values can - hash alike, and a query that returned the wrong file would be far worse than a - slow one. -- **An index covers values written after it is declared.** There is no - reindexing pass. Saying so is better than a half-truth that makes a query - answer confidently and wrongly. -- **A query on an unindexed attribute is refused, not answered by scanning.** A - query that silently degrades into a full walk teaches you to use it where it - will not scale. - -Reachable as `setxattr`/`getxattr`/`removexattr` at Linux's syscall numbers, so -anything that already knows how to attach metadata needs no changes, plus two -native calls (`schfs_index`, `schfs_query`) that have no Linux equivalent -because asking a filesystem a question does not. - -**Phase 5 — attribute index and query.** See below. - -## 6. The "SQL" part, honestly +## Attributes, indexes, and the query What you want from "works like SQL" is: *ask a question about a lot of files and get an answer fast, without walking every directory*. That is an **index**, not @@ -382,28 +121,14 @@ a query language. BeFS's design: index ranges. In SchFS this is just another key type — `(idx, INDEX, key)` — in the tree -that already exists. That is the whole reason to build phases 1-4 the btrfs way: +that already exists. That is the whole reason the tree was built the btrfs way: the "database filesystem" feature becomes a few hundred lines on top of machinery already written and debugged, rather than a separate engine. A concrete `query()` syscall and a `find`-like tool come after that, and are small by comparison. -## 7. Honest cost - -Phases 1-3 are roughly 3,000-4,000 lines and are the bulk of the work. Phase 1 -alone (B-tree + torture harness) is worth doing carefully and unhurriedly; every -later phase compounds on it. - -This is the largest single piece of work in SchOS. It is also entirely -ordinary filesystem engineering with excellent published references -(Giampaolo for BeFS; the btrfs design papers; the ZFS uberblock/checksum-in- -parent scheme) — there is no research risk here, only volume. - -**Do this before the installer.** The installer's entire job is to create and -populate this format. Building it against v1 means building it twice. - -## 7½. Limits +## Limits Three different things get called "the limit", and they are worth keeping apart: what the **on-disk format** can express, what **this implementation** can @@ -464,7 +189,75 @@ filesystem operation now, so SchFS emits its own records with 64-bit inode numbers and names up to 255 bytes, and the inode carries 64-bit numbers and sizes throughout. -## 8. What exists now +--- + +## Format version, and what a bump means + +Two mechanisms, because one is not enough. `SCHFS_VERSION` changes only when +the **frame** changes — the uberblock, the node header, the key, the things any +reader must understand before it can parse anything at all. Everything else is +a **feature flag**, in one of three classes: + +| Class | An unknown bit means | +|---|---| +| `compat` | Ignore it. Mount read-write. | +| `ro_compat` | Mount **read-only**. A writer would maintain a structure this driver does not know about, and writing would leave it stale. | +| `incompat` | **Refuse.** The volume cannot be parsed correctly without it. | + +Choosing the class is the whole design decision, and the test is one question: +*what happens if a driver that does not know this feature writes to the volume?* +Nothing → `compat`. Something on disk goes stale → `ro_compat`. Something +becomes unreadable → `incompat`. + +The flags occupy what was `reserved0`, so every field after them keeps its +offset and existing images — which have zeros there — read identically. +`schfs_vol_commit` memsets the uberblock before filling it in, which is why +volumes written before the field existed already say "no features". **That trick +is available exactly once**; the next addition costs a version bump. + +Four `_Static_assert`s pin the uberblock to one block and pin `version`, +the feature words and the field after them to fixed offsets, because those are +on-disk facts rather than C conveniences. + +### Refusing is not falling back + +The uberblock ring exists so that a damaged newest uberblock can be stepped +over — that is crash recovery, and the older root is a consistent filesystem. +A newest uberblock that is *sound but too new* is the opposite case, and +treating the two alike is how a mount quietly loses data: + +> Falling back mounts a **stale** filesystem. Every commit made since the volume +> was upgraded is discarded, the mount reports success, and nobody finds out. + +So `uber_check()` answers with a state rather than a yes/no — +`BAD` / `OK` / `RO` / `TOO_NEW` — and the scan stops on `TOO_NEW` instead of +continuing to the next candidate. The order of its checks is the order of the +questions: is it ours (magic), is it intact (crc), can we read it (version, +features), does it make sense (root in range). Compatibility is asked before +sanity deliberately — the fields a sanity check reads are only meaningfully +placed if the format is one we know. + +Demonstrated rather than asserted, on a forged image whose two uberblocks are +`(version 1, generation 1, root 129)` and `(version 2, generation 2, root 130)`: +the old code mounted root 129 and exited 0, discarding the live filesystem; the +current code refuses and says why. + +--- + +## One source, two compilers + +Everything except `kmount.c` is compiled **both** into the kernel and into the +host tools. + +> The writer and the reader cannot drift apart, because there is one +> implementation of the format and nothing to disagree with. + +That is why `mkschfs` cannot produce an image the kernel misreads, and why +format bugs are found on the host where a test takes a second instead of a boot. + +--- + +## The source ``` include/schfs.h on-disk format + compile-time layout assertions @@ -490,7 +283,7 @@ Everything except `kmount.c` is compiled *both* into the kernel and into the host tools, so the writer and the reader cannot drift apart: there is one implementation of the format and nothing to disagree with. -### Running the tests +### The test suites ``` make schfs-check all three suites @@ -526,7 +319,7 @@ cutting the power one write later each time, and after every one the volume must mount, check out structurally, and still hold the data committed before it. `tools/posix/mutate.sh` injects six deliberate bugs and fails if any of them goes -undetected. It is how the suites are kept honest; see the note in §9. +undetected. It is how the suites are kept honest; see [../history/schfs.md](../history/schfs.md). ### Verifying on real hardware @@ -539,53 +332,26 @@ make qemu-schfs then, at the shell: cat /proc/schfs `.\run.ps1 -SchFS` does the same on Windows, and `tools\bootcheck.ps1 -SchFS` does it unattended and asserts on the transcript. -## 9. Things learned the hard way +--- -Recorded because each one cost real time and none is obvious from the code. +## What it does not do -- **The root's object id must be reserved.** `next_objectid` started at - `SCHFS_FIRST_OBJECTID`, which *is* the root's id, so the first directory - created silently became the root. It then contained itself, and every tree - walk recursed until the stack ran out. `schfs_fsck --check` now detects - directory loops explicitly rather than crashing. -- **Cursor buffers alias.** A range callback gets a pointer into the cursor's - block buffer, and almost every readdir caller immediately looks the entry up, - which overwrites it. Directory iteration copies each entry out before calling - back. Extracting a tree with directories reparented under their own children - is what this looks like when it goes wrong. -- **A 4096-byte block does not fit on a 4096-byte kernel stack.** The cursor - exists precisely to avoid one block buffer per tree level, but - `schfs_vol_open` still kept a whole uberblock as a local and blew the stack on - the first boot. The kstack canary caught it cleanly. -- **Extents only merge if you allocate the run first.** Writing an extent record - allocates blocks for its own copy-on-write path, so allocating data one block - at a time guarantees the blocks are never adjacent and every block needs its - own extent. `schfs_file_reserve` claims the whole run up front; an 822 KB - binary is one extent instead of 201. -- **Freed blocks are reusable during a build.** Copy-on-write forbids reusing a - released block because the previously committed tree still points at it — but - a build has committed nothing. Recycling them cut the test image from 3596 - blocks to 374. -- **Reproducibility dies from atime.** mkschfs reads every source file, which - updates its atime, so copying atime into the image made two builds of the same - tree differ. All three timestamps come from mtime. -- **An optional hook must still be initialised.** `struct schfs_bdev` gained - `bflush`, and every host tool built one on the stack field by field. The tools - that did not set the new field handed `schfs_vol_commit` a garbage pointer to - test and call, which segfaulted at the first commit. Each of those helpers - memsets the struct first now — the fix is not "remember to set it". -- **Deferring writes makes an all-dirty buffer cache normal.** `bget` panicked - with "no buffers" when it could find nothing clean to recycle, which was - correct while a dirty buffer meant "the log has not committed". It now writes - the least recently used victim out and takes it; a transaction touching more - than `NBUF` blocks reaches that state long before it commits. -- **The two performance fixes are worthless apart.** Deferring tree writes buys - nothing while every commit flushes, and batching commits buys nothing while - every write goes straight to the disk. Measuring either alone says "this is - not where the time goes", and both measurements are correct. -- **`fsid_t` is eight bytes.** `struct statfs` declared `f_fsid` as `int64[2]`, - which is one slot too long, so everything after it landed in the wrong place: - userland read `f_frsize` out of `f_namelen` and got 255. df then computed in - units of 255 bytes and called a 32 MB root filesystem 1992 KB. It survived - because every field in that struct used to be an invented constant, so no - number in it had ever been checked against reality. +- **No snapshots.** The B-tree is copy-on-write, which is most of the + machinery — nothing takes or names a snapshot. This is the missing feature the + existing design already pays for. +- **No journal replay** beyond what copy-on-write gives for free. +- **No sparse files.** A hole is not representable; writing at a large offset + allocates everything before it. +- **No online `fsck`.** `schfs_fsck` is a host tool and the volume must not be + mounted. +- **No upgrade path.** The version policy below refuses a volume it is too old + for; nothing converts one format to another, because there is only one format. + The first bump is when that has to be written. +- **No Linux driver.** FUSE first, then possibly a kernel module, and not before + the format is pinned. +- **No deduplication**, deliberately — ZFS's is famous for needing more RAM than + anyone expects, and extent refcounts give the reflink and snapshot benefits + without it. +- **No multi-disk anything** — no pools, no RAID-Z, no vdevs. +- **No rebalancing or online format conversion**, deliberately: enormous + complexity, and historically the source of btrfs's worst bugs. diff --git a/docs/architecture/syscalls.md b/docs/architecture/syscalls.md new file mode 100644 index 0000000..acbc180 --- /dev/null +++ b/docs/architecture/syscalls.md @@ -0,0 +1,131 @@ +# Syscalls, and the Linux personality + +Two things happen at this boundary. SchOS has its own syscall numbering, and it +also answers Linux's — which is why musl, toybox and mksh run unmodified. + +--- + +## One table, everything generated + +**Syscalls are declared in exactly one place:** +[`kernel/sys/table.inc`](../../kernel/sys/table.inc). One line each. + +```c +SYSCALL(nr, linux_nr, name) // implemented as sys_() +SYSCALL_ALIAS(linux_nr, name) // a second Linux number for the same handler +``` + +Three things are generated from it by macro expansion, and none is +hand-maintained: + +| Generated | Where | +|---|---| +| The `SYS_*` constants | `include/syscall.h` | +| The dispatch table | `kernel/sys/syscall.c` | +| The Linux translation map | `kernel/sys/linux_abi.c` | + +Adding a syscall is therefore two edits: a line here, and a body written with +`SYSCALL_DEFINE()` in one of the `kernel/sys/sys_*.c` files. + +**This replaced a two-place scheme and the reason is worth keeping.** The Linux +number used to live in a second hand-written table. Adding a syscall meant three +edits across two files, and forgetting the second one left the call *unreachable +from every musl-linked program* while looking perfectly registered in the first. +A syscall can no longer be half-registered. + +## How a Linux binary runs + +The translation is a flat array indexed by Linux syscall number, whose value is +the native `SYS_*` number — or 0, meaning not implemented, which dispatches to +`-ENOSYS`. + +It is enabled **per process**, and selected automatically by `exec()`: + +```c +// musl binaries have entry >= 0x100000; SchOS-native ones are linked at 0x4000 +if (elf.entry >= LINUX_ABI_ENTRY_THRESHOLD) + myproc()->syscall_map = linux_syscall_map; +``` + +A link address is a crude discriminator and it is honest about being one. It +works because the two classes of binary in this system are linked by two +different linker scripts at two very different addresses. + +There is also `enable_linux_abi()` as an explicit call, which is why +`cmd/schinit/main.c` carries a warning not to use it: doing so mid-process would +renumber the syscalls the caller still has to make. + +### Native vs. Linux numbering + +Some calls have no Linux equivalent and are marked `NO_LINUX`: `sleep`, +`uptime`, `reboot`, `shutdown`, `getmouse`, `vga_sync`, `schfs_index`, +`schfs_query`. These are reachable only from first-party programs linked +against `lib/`. + +They all land on the same scratch index in the translation array and overwrite +each other there, which is harmless — no real Linux syscall uses that number, so +the slot is never dispatched. + +### Where the two disagree on a name + +`shutdown` is the one collision. SchOS has a native `shutdown()` that powers the +machine off, and Linux number 48 is the socket call. Two syscalls cannot share a +`sys_` function name, so Linux 48 maps to `sockshutdown`. + +## What is implemented + +Roughly 190 native numbers covering: + +| Area | | +|---|---| +| Process | fork, exec, exit, wait/waitpid/wait4/waitid, clone, kill, sessions, process groups | +| Identity | the full real/effective/saved set for both uid and gid, groups, umask | +| Memory | mmap, munmap, mprotect, brk/sbrk, madvise, System V shared memory | +| Filesystem | the ordinary set, the whole `*at()` family, statfs, xattr, mount/umount2 | +| I/O | read/write, readv/writev, pread/pwrite, sendfile, poll/select/ppoll/pselect6, epoll, ioctl, fcntl | +| Sockets | socket, bind, listen, accept, connect, sendto, recvfrom, socketpair, setsockopt, getsockopt | +| Signals | rt_sigaction, rt_sigreturn, rt_sigsuspend, rt_sigpending, rt_sigtimedwait, sigaltstack | +| Time | gettimeofday, clock_gettime, clock_getres, nanosleep, clock_nanosleep, alarm, timers | +| Notification | eventfd2, timerfd_create/settime/gettime, signalfd4 | + +### Argument fetching + +`fetchint`, `fetchaddr`, `fetchstr` and `argptr` validate a user pointer through +the page tables and nothing else. They previously rejected any address +`>= proc->sz`, on the assumption that every valid user pointer lives in the +heap — which is wrong for stack addresses and for every `mmap`'d buffer, i.e. +for most pointers real programs pass. + +`uva2ka()` translates a user virtual address and **returns the in-page offset**, +which it did not originally do. Every caller that read or wrote through a +non-page-aligned pointer — `TIOCGPGRP`, `TCGETS`, `getcwd`, signal masks, +`times()`, `sysinfo()` — was touching the wrong address whenever the pointer was +not page-aligned, i.e. almost always. + +**`uva2ka()` guarantees only the page containing that address.** A structure +copied in bulk through one translation runs off the end of the mapped page when +the pointer lands near a boundary — and `P2V` is a flat offset, so "the next +page" in kernel space has nothing to do with the user's next page. That bug +corrupted unrelated kernel memory and crashed every freshly `exec`'d shell. Copy +field by field, each through its own translation, or use `copyin`/`copyout`. + +--- + +## What it does not do + +- **No `sendmsg`/`recvmsg`**, and therefore no ancillary data and no + `SCM_RIGHTS`. +- **No `memfd_create`.** +- **No `getsockname` or `getpeername`**, and `accept()` does not report an + AF_INET peer's address. +- **`setsockopt` handles six options** and answers `ENOPROTOOPT` for everything + else. `SO_RCVTIMEO`/`SO_SNDTIMEO` are among the refused. +- **`clone` does not share an address space**, so it is `fork` with extra steps. +- **`sched_setaffinity` is accepted and ignored.** +- **`getrusage` does not distinguish `RUSAGE_CHILDREN`** from `RUSAGE_SELF`, and + fills in only the calling process's own tick count. +- **The ABI is chosen by link address.** It works, and it is a heuristic rather + than a fact about the binary. +- **`/proc//exe` is a regular file, not a symlink**, so `readlink()` on it + fails and `ps -o comm` prints `[init]` rather than `init`. Default `ps` and + `top` output is unaffected. diff --git a/docs/architecture/usb.md b/docs/architecture/usb.md new file mode 100644 index 0000000..a56d101 --- /dev/null +++ b/docs/architecture/usb.md @@ -0,0 +1,116 @@ +# USB + +An xHCI host controller driver, a device-independent core, and three class +drivers. Written from the specification. + +--- + +## Why not ported from BSD + +"Take the USB stack from BSD" is the obvious plan and it does not survive contact +with the code. + +FreeBSD's `sys/dev/usb` is roughly 100k lines welded to newbus, devclass, +`bus_dma`, callouts, taskqueues and FreeBSD's locking primitives, none of which +exist here. **Porting it means porting FreeBSD's device framework first and then +discovering the USB stack underneath.** NetBSD's is more portable and the same +argument applies with a smaller multiplier. + +The specification is public and the controller is not complicated. + +> BSD is much more valuable as the reference you check yourself against than as +> the code you paste. + +Licensing points the same way: the BSD stacks are BSD-2/3-clause and compatible, +Linux's drivers are GPL-2.0-only and are not — see +[../licensing.md](../licensing.md). + +## What the spec gives you for free + +Everything in [`include/usb.h`](../../include/usb.h) comes from USB 2.0 +chapter 9, which every device implements regardless of speed, class or vendor. + +**That is what makes enumeration possible at all:** before you know anything +about a device you can still ask it what it is, because the question is +standardised. + +## The controller + +`kernel/drivers/bus/xhci/`. The full bring-up path: + +``` + reset → DCBAA → command ring → event ring → scratchpad + → port enumeration → Enable Slot → Address Device → control transfers + → configuration and interface descriptors → SET_CONFIGURATION + → interrupt and bulk endpoints +``` + +Rings are producer rings: the driver writes TRBs and the controller reads them, +with a Link TRB as the last entry pointing back at the start — which is how a +fixed-size buffer becomes an endless queue. 64 TRBs per ring, 16 device slots. + +### Enumeration polls; data uses interrupts + +**And that is not laziness.** `pci_bind_all()` runs from `kmain()` with +interrupts disabled and no scheduler, so a driver that waited for an interrupt +during probe would wait forever. + +Interrupts are registered afterwards, for when a device is actually delivering +data. + +## Class drivers + +| | | +|---|---| +| **HID** (`usb/hid/`) | Boot protocol keyboard and mouse, driven from the IRQ handler, feeding the same input paths as PS/2. | +| **Mass storage** (`usb/storage/`) | Bulk-only transport and SCSI, presenting a block device. A superblock reads back through `bread()`. | +| **Hub** (`usb/hub/`) | Route strings, hub slot contexts, per-port reset and power. | + +Class drivers are offered a device by the core once its interface descriptor has +been read, from the list in `kernel/drivers/registry.c`. Whichever claims it +publishes a `struct device` in the bus-agnostic list, **with the class the driver +chose rather than the one the descriptor claimed** — the descriptor says what the +device says it is, and the driver knows what it made of it. + +### What proves the hub works + +A mouse behind a hub enumerating a tier deep. Hubs are the part of USB where +route strings and slot contexts stop being theoretical, and a hub driver that +handles only directly-attached devices looks identical to a working one until +something is plugged into it. + +### The USB HID grab detail + +The HID path updates its `prev[]` key state **before returning** even when the +device is grabbed — otherwise the next report looks like every held key had just +been pressed again. See [graphics.md](graphics.md#input) for the grab itself. + +## Testing + +`bootcheck -Usb` attaches an xHCI controller with a keyboard, a mouse and a +mass-storage device. + +**Nothing on the default machine has a USB port**, so without that switch the +driver is never asked to do anything and a broken one looks exactly like a +working one. + +`echo u > /proc/sysrq` and `echo s > /proc/sysrq` drive the storage path. + +--- + +## What it does not do + +- **A USB disk cannot be mounted or booted from.** The block device works and + `bread()` reads a superblock off it; the root device is compiled in rather than + selectable, so nothing can be told to use it. This is the largest single gap + and it is item 1 of [../plan/roadmap.md](../plan/roadmap.md). +- **Boot protocol only** for HID. No report descriptor parsing, so anything that + is not a plain keyboard or mouse is not understood. +- **No isochronous transfers**, so no audio and no cameras. +- **No USB 3 SuperSpeed-specific features** beyond what enumeration needs — the + USB2 and USB3 descriptor paths both work, the rest of the difference is unused. +- **16 device slots**, compile-time. +- **No hot-unplug.** A device removed is not noticed; the device model has no + removal path either. +- **One controller.** A second xHCI card would bind and then contend for state + the driver keeps per-driver rather than per-instance. diff --git a/docs/drivers.md b/docs/drivers.md deleted file mode 100644 index 8567d81..0000000 --- a/docs/drivers.md +++ /dev/null @@ -1,161 +0,0 @@ -# Drivers - -How hardware gets found, claimed and used, and where drivers that are not part -of the build will attach. - ---- - -## 1. The device model - -There is one list of everything attached, whatever bus found it: -`include/device.h` and `kernel/dev/device.c`, readable at `/proc/devices`. - -``` -name bus class driver device -kbd0 platform input ps2kbd PS/2 keyboard -mouse0 platform input ps2mouse PS/2 mouse -uart0 platform other uart8250 16550 serial port -ide0 platform storage ide ATA disk controller -vga0 pci display vga display controller -``` - -Before this there were two registries in `kernel/drivers/registry.c` — an array -of PCI drivers and an array of USB drivers — with nothing that knew about both. -That is sufficient for *binding* and useless for the three things wanted next. - -**Enumeration.** "What is in this machine, and what claimed it" had no answer -covering a USB disk and a PCI display at once. A device manager is exactly that -question. - -**Classes.** A compositor should ask for *the display* and not care whether that -is the VBE framebuffer or virtio-gpu. Without a class every consumer hardcodes a -driver, and adding the second driver means editing the consumers — which is how -a framebuffer address ended up hardcoded in three places here once already. - -**A boundary for drivers that are not compiled in.** They have to attach -somewhere that is not an array fixed at link time. - -Two vocabularies are kept deliberately separate: a device's **bus** is -provenance (`platform`, `pci`, `usb`), its **class** is capability -(`display`, `input`, `storage`, `net`, `bus`). A USB disk and a SATA disk are -both `storage` despite sharing no numbering at all. Where a driver claims a -device, the driver's opinion wins over the descriptor's: `struct usb_driver` has -a `devclass` field separate from the USB interface class, because the descriptor -says what the device *claims* to be and the driver knows what it *made* of it. - -It is a flat list with a bus tag, **not** a device tree. Nothing yet needs to -walk a hierarchy, and a tree nobody traverses is just a harder list. Hot-unplug -would need more; the `in_use` flag is the hook. - -Platform devices are registered in one place -(`platform_register_devices()` in `registry.c`) rather than inside each driver's -init, for the same reason the driver arrays are written out by hand: a list you -can read beats a list assembled by side effects. - ---- - -## 2. Where this is going: a hybrid model - -The intended shape is the NeXTSTEP/macOS one rather than Linux's — a device is -an object with a class and an ops vtable, and consumers talk to the class. - -Two things often get bundled together as "loadable drivers" and they are -separable: - -| | | -|---|---| -| **Loadable modules** | A stable driver ABI, drivers as files loaded at runtime (`.sys`, `.kext`). A linker/loader job on top of the model above. | -| **Drivers out of kernel** | Drivers as user-space processes (macOS DriverKit, which is why kexts are being retired). An IPC and memory-mapping job; the `AF_UNIX` and shared-memory work it needs already exists. | - -The second is more in the spirit of the SchAPI direction and is slower and -larger. Neither is built yet. What exists is the attachment point they both -need. - ---- - -## 3. Linux drivers - -**Decided 2026-08-08.** Linux drivers are not shipped with SchOS and never -will be. The user may install one on their own machine, through a device -manager or equivalent, and SchOS will load it. - -The licensing reasoning matters and is worth stating exactly, because it is -easy to get wrong in both directions: - -- **Distribution is what triggers the GPL.** Shipping a GPL-2.0-only driver in - an Apache-2.0 image is what cannot be done — see [licensing.md](licensing.md) - §2. A user obtaining a driver themselves and loading it on their own machine - is their act, not ours, and is not a distribution by us. -- **Implementing a loader is not copying.** Writing support for a module format - and a driver ABI from published documentation produces our own work. It is a - clean-room implementation, and it stays one only if it is written from - documentation rather than transcribed from Linux source. Reading Linux to - understand *hardware* is fine; translating its code is not, whatever the - identifiers say. - -So the rule for this subsystem is the same as everywhere else in the tree: the -loader is ours and Apache-2.0, what it loads is the user's business, and nothing -GPL ever enters an image we build. `RAMDISK_EXCLUDE` and `NOTPORTS` are the -existing mechanisms for keeping that true mechanically rather than by intention. - ---- - -## 4. Adding a driver today - -1. Write `kernel/drivers///.c` exporting a - `struct pci_driver` or `struct usb_driver`. One directory per driver, so a - driver that grows a register header, a second source file or a firmware blob - has somewhere to put it: - - kernel/drivers/ - registry.c - gpu/vga/ gpu/virtio_gpu/ - net/e1000/ - bus/xhci/ - usb/core/ usb/hid/ usb/storage/ usb/hub/ - -2. Declare it in `kernel/drivers/registry.c` and add it to the array. -3. Add the object to `KERN_DRIVERS` in the `Makefile`. -4. Fill in the **class vtable** for what it is, and point `.ops` at it. That is - the whole of what the rest of the kernel sees: - - | class | vtable | consumer | - |---|---|---| - | `DEVCLASS_DISPLAY` | `struct display_ops` ([display.h](../include/display.h)) | the compositor | - | `DEVCLASS_NET` | `struct netdev_ops` ([netdev.h](../include/netdev.h)) | the TCP/IP stack | - - A display driver that owns a linear framebuffer also publishes it with - `gfx_attach_scanout()` ([gfx.h](../include/gfx.h)). That is what the text - console and the crash screen draw on, and it is separate from the display - class on purpose: the class is about *frames* — acquire, present, vblank — - and the crash screen has to work when nothing else in the system does. - - A driver that fills these in never mentions its consumer, and its consumer - never mentions it. Getting this wrong has a characteristic shape: the e1000 - shipped with `.ops = 0` and the network stack called `e1000_transmit()` - directly, so `kernel/net/` was full of one driver's name and the second NIC - driver would have had to edit the protocol stack to add itself. - -Private headers go **next to the driver**, not in `include/`. The build adds -the driver's own directory to the include path, so `e1000.h` — a register map -that exactly one file has any use for — lives in `kernel/drivers/net/e1000/`. -`include/` is for headers the rest of the kernel needs. - -Registration in the device model is automatic — `pci_bind()` and -`usb_bind_all()` publish whatever they claim, so a driver cannot forget to. Set -`devclass` on a USB driver, or the device lands in `other`. - -A driver that can drive **more than one card at once** keeps its instances in -an array and implements `.attached`, which the bus calls once the `struct -device` exists; hang the instance on `dev->priv` there and read it back in -every op. It cannot be done inside `probe()`, because the device is published -afterwards. Skipping this is how a driver ends up refusing the second card, as -the e1000 did until it was asked to drive two. - -The best source for a new driver is the **specification**, not another kernel: -xHCI, AHCI, virtio, NVMe and the Intel datasheets are public documents with no -licence on what you write from them. `kernel/drivers/bus/xhci/xhci.c` came from the spec, and so did -`net/e1000/` and `gpu/virtio_gpu/`. Where a *table* of constants is genuinely -faster to transcribe than to retype from a PDF -- device IDs, register offsets -- -taking it from a BSD tree is fine after the licence check, and goes in `NOTICE`. -That is data; the logic stays ours. diff --git a/docs/guides/adding-a-driver.md b/docs/guides/adding-a-driver.md new file mode 100644 index 0000000..a345c7a --- /dev/null +++ b/docs/guides/adding-a-driver.md @@ -0,0 +1,112 @@ +# Adding a driver + +The device model itself is described in +[../architecture/devices.md](../architecture/devices.md). This is the procedure. + +--- + +## Where to get one from + +**The best source is the specification, not another kernel.** xHCI, AHCI, +virtio, NVMe and the Intel datasheets are public documents with no licence on +what you write from them. `kernel/drivers/bus/xhci/xhci.c` came from the spec, +and so did `net/e1000/` and `gpu/virtio_gpu/`. + +**BSD: yes. Linux: no.** This is a licence incompatibility rather than a +preference, and it decides the whole strategy — see +[../licensing.md](../licensing.md). BSD-2 and BSD-3 code can be used directly +with a `NOTICE` entry; GPL-2.0-only code can never be, at any distance. Reading +Linux to understand *hardware* is fine and normal; translating a Linux driver +function by function produces a derivative work whatever the identifiers say. + +One caveat about "port it from BSD" as a plan, learned by looking: FreeBSD's +`sys/dev/usb` is roughly 100k lines welded to newbus, devclass, `bus_dma`, +callouts, taskqueues and FreeBSD's locking. Porting it means porting FreeBSD's +device framework first and finding the USB stack somewhere underneath. + +> **BSD is far more valuable as the reference you check yourself against than as +> the code you paste.** Read it when the spec is ambiguous — which it is, often — +> and write your own. + +Where a *table* of constants is genuinely faster to transcribe than to retype +from a PDF — device IDs, register offsets — taking it from a BSD tree is fine +after the licence check, and goes in `NOTICE`. That is data; the logic stays +ours. + +--- + +## The procedure + +1. Write `kernel/drivers///.c` exporting a + `struct pci_driver` or `struct usb_driver`. One directory per driver, so a + driver that grows a register header, a second source file or a firmware blob + has somewhere to put it: + + kernel/drivers/ + registry.c + gpu/vga/ gpu/virtio_gpu/ + net/e1000/ + bus/xhci/ + usb/core/ usb/hid/ usb/storage/ usb/hub/ + +2. Declare it in `kernel/drivers/registry.c` and add it to the array. +3. Add the object to `KERN_DRIVERS` in the `Makefile`. +4. Fill in the **class vtable** for what it is, and point `.ops` at it. That is + the whole of what the rest of the kernel sees: + + | class | vtable | consumer | + |---|---|---| + | `DEVCLASS_DISPLAY` | `struct display_ops` ([display.h](../../include/display.h)) | the compositor | + | `DEVCLASS_NET` | `struct netdev_ops` ([netdev.h](../../include/netdev.h)) | the TCP/IP stack | + + A display driver that owns a linear framebuffer also publishes it with + `gfx_attach_scanout()` ([gfx.h](../../include/gfx.h)). That is what the text + console and the crash screen draw on, and it is separate from the display + class on purpose: the class is about *frames* — acquire, present, vblank — + and the crash screen has to work when nothing else in the system does. + + A driver that fills these in never mentions its consumer, and its consumer + never mentions it. Getting this wrong has a characteristic shape: the e1000 + shipped with `.ops = 0` and the network stack called `e1000_transmit()` + directly, so `kernel/net/` was full of one driver's name and the second NIC + driver would have had to edit the protocol stack to add itself. + +Private headers go **next to the driver**, not in `include/`. The build adds +the driver's own directory to the include path, so `e1000.h` — a register map +that exactly one file has any use for — lives in `kernel/drivers/net/e1000/`. +`include/` is for headers the rest of the kernel needs. + +Registration in the device model is automatic — `pci_bind()` and +`usb_bind_all()` publish whatever they claim, so a driver cannot forget to. Set +`devclass` on a USB driver, or the device lands in `other`. + +A driver that can drive **more than one card at once** keeps its instances in +an array and implements `.attached`, which the bus calls once the `struct +device` exists; hang the instance on `dev->priv` there and read it back in +every op. It cannot be done inside `probe()`, because the device is published +afterwards. Skipping this is how a driver ends up refusing the second card, as +the e1000 did until it was asked to drive two. + +The best source for a new driver is the **specification**, not another kernel: +xHCI, AHCI, virtio, NVMe and the Intel datasheets are public documents with no +licence on what you write from them. `kernel/drivers/bus/xhci/xhci.c` came from the spec, and so did +`net/e1000/` and `gpu/virtio_gpu/`. Where a *table* of constants is genuinely +faster to transcribe than to retype from a PDF -- device IDs, register offsets -- +taking it from a BSD tree is fine after the licence check, and goes in `NOTICE`. +That is data; the logic stays ours. + +--- + +## Checklist + +- [ ] One directory under `kernel/drivers///` +- [ ] Exports a `struct pci_driver` or `struct usb_driver` +- [ ] Declared in `kernel/drivers/registry.c` **and** added to its array +- [ ] Object added to `KERN_DRIVERS` in the `Makefile` +- [ ] Class vtable filled in and `.ops` pointed at it +- [ ] `.attached` implemented if it can drive more than one card +- [ ] `devclass` set, if it is a USB driver +- [ ] Private headers next to the driver, not in `include/` +- [ ] A `NOTICE` entry, if anything was copied — **before** the commit, not after +- [ ] A selftest reachable from `/proc/sysrq`, if the class has one + (`netdev_selftest` is the model) diff --git a/docs/building.md b/docs/guides/building.md similarity index 98% rename from docs/building.md rename to docs/guides/building.md index c8c0696..a76eb6c 100644 --- a/docs/building.md +++ b/docs/guides/building.md @@ -112,7 +112,7 @@ tools/posix/setup-fedora.sh # install it ``` Moving an existing checkout to a Linux machine: see -[moving-to-linux.md](moving-to-linux.md). +[../history/moving-to-linux.md](../history/moving-to-linux.md). ## Working on two machines diff --git a/docs/layout.md b/docs/guides/layout.md similarity index 91% rename from docs/layout.md rename to docs/guides/layout.md index c74c4b3..a8dfbcc 100644 --- a/docs/layout.md +++ b/docs/guides/layout.md @@ -1,5 +1,7 @@ # Repository layout +Where everything is, and how to add to it. + One rule holds the whole thing together: > **A directory contains either hand-written source or generated output. Never both.** @@ -39,12 +41,14 @@ schos/ │ ever been read, so a sparse checkout is ~100 MB: │ git clone --filter=blob:none --sparse Refrence/freebsd-src │ git -C Refrence/freebsd-src sparse-checkout set │ sys/dev sys/net sys/netinet -├─ src-trunk/ A NetBSD checkout, kept as reference material and a parts -│ bin. NOT built, NOT installed, and it must never reach an -│ image. Copying from it is governed by the per-file licence -│ check in docs/architecture/schapi.md §5 — NetBSD is not -│ uniformly licensed. -├─ docs/ Design docs +├─ src-trunk/ A NetBSD checkout, when present -- reference material and a +│ parts bin. NOT built, NOT installed, must never reach an +│ image. Currently absent; it was deleted for disk space and +│ can be re-fetched sparsely. Copying from it is governed by +│ the per-file licence check in docs/licensing.md §3 -- +│ NetBSD is not uniformly licensed. +├─ docs/ architecture/ what it is plan/ what's next +│ guides/ how to work history/ why it's like this └─ build/ ALL generated output (see below) ``` @@ -148,6 +152,9 @@ nothing to do with the ramdisk root and sends you to the wrong file. ## Two things that will bite you +The second is about a bootloader that no longer exists. It is kept because the +*shape* of the bug outlived the code. + **`ld -r -b binary` derives symbol names from the input file's path.** The kernel embeds two blobs this way (`initcode`, `entryother`), and the root Makefile has `--redefine-sym` lines mapping e.g. diff --git a/docs/guides/testing.md b/docs/guides/testing.md new file mode 100644 index 0000000..11e3c4a --- /dev/null +++ b/docs/guides/testing.md @@ -0,0 +1,135 @@ +# Testing + +> **Booting to a prompt proves nothing.** A filesystem can mount and still fail +> the first real command, and that has happened here. + +So the harnesses drive the system and assert on what comes back, rather than +reporting that it started. + +--- + +## The three levels + +| | What it runs | What it proves | +|---|---|---| +| `make schfs-check` | Host-native unit suites | The filesystem's logic, in about a second | +| `tools/windows/bootcheck.ps1` | A real boot, driven over the serial console | That the system answers correctly | +| `make crashqr`, `make schcom-test` | A boot, then pixel assertions from the host | That the *screen* is right | + +The levels exist because each catches what the one below cannot. A unit test +cannot tell you the kernel fails to boot; a boot test cannot tell you the screen +is blank. + +--- + +## `make schfs-check` — the fast one + +Run this first and run it often. It is host-native, so a test takes a second +instead of a boot. + +```sh +make schfs-check # B-tree, object and image suites +make schfs-test # the B-tree alone +make schfs-objtest # the object layer alone +make schfs-image-test # build an image with mkschfs, check it with fsck +``` + +This works because **the kernel and the host tools compile the same SchFS +source**. `mkschfs` cannot disagree with the kernel about the on-disk format, +because there is only one implementation — so format bugs are findable on the +host, where the loop is fast. + +## `bootcheck` — the boot suite + +```sh +tools/windows/bootcheck.ps1 # the default run +tools/windows/bootcheck.ps1 -Tty -Ps -Net # switches compose +``` + +It boots SchOS under QEMU, types commands at the serial console, and greps the +transcript for the answers they should produce. + +**It is PowerShell because `pwsh` runs on every supported host** — Linux, macOS +and the BSDs as well as Windows. `brew install powershell`, or your +distribution's package. It finds QEMU on `PATH`. + +### The switches, and why each exists + +Each of these turns on something the default run cannot reach. That is the point: +a subsystem nothing exercises looks identical whether it works or not. + +| Switch | Why it is not in the default run | +|---|---| +| `-Tty` | Pipes, `isatty`, `SIGPIPE`, Ctrl-C. **None can be checked from a screenshot**, and all were once broken in ways that looked like broken *commands*. | +| `-Usb` | Attaches an xHCI controller with a keyboard, mouse and mass-storage device. Nothing on the default machine has a USB port, so without it a broken driver looks exactly like a working one. | +| `-Sock` | Unix sockets. A socket layer that accepted every call and dropped every byte would look identical from outside. `socktest` has one process write and another read. | +| `-Evdev` | Raw input. Needs the QEMU monitor, because a key has to be *pressed* — characters typed at the serial port never touch the keyboard controller. | +| `-Ps` | Process enumeration. `/proc`'s whole reason for existing is one directory per process, and nothing else in the suite touches it. | +| `-Net` | TCP/IP. A listener runs **on the host** and the guest connects out. | +| `-Iso` | Boots `schos.iso` from an emulated optical drive with **no hard disk attached**, which is the only way to prove the ISO is self-contained. Needs `make iso` first. | +| `-Fb` | Framebuffer checks. | + +`-Net` deserves its own note, because it is the shape every integration test +should copy: **both ends are asserted.** The guest reports what it received and +the listener reports what arrived. A stack that completed a handshake and then +dropped every byte would pass any check made from one side alone. + +Under QEMU's user-mode networking the host is `10.0.2.2` seen from inside, and +SLIRP connects to the host's loopback — so the listener binds there. + +## Pixel assertions + +```sh +make crashshot && make crashqr # crash the kernel, decode the QR out of the framebuffer +make schcom-test # boot, run the compositor and a client, check the pixels +``` + +`tools/windows/splashshot.ps1` counts colours. **That exists because a log-only +harness once passed on a blank screen.** + +`schcom-test` asserts **from the host** — the compositor's own report that it +composited something proves nothing. It runs two overlapping clients and checks +that the newer one wins in the overlap, because a compositor that cannot order +two surfaces cannot host a window manager. + +Other capture tools, all in `tools/windows/`: `dispshot`, `fbshot`, +`splashshot`, `textshot`, `ttyshot`. + +--- + +## Two lessons worth keeping + +**Read the transcript, not the verdict.** A change to the input grab once passed +every assertion green while not having been applied at all — a scripted string +replacement had silently failed to match, and the `grep` that should have caught +it returned three hits (a comment, a `#define`, and the *release* call) that were +read as confirmation without counting what they were. What caught it was four +bare `#` prompts in the transcript, which is precisely the symptom the change was +meant to remove. + +**A test that stops finding bugs has a standing cost.** A ported 1993 shooter +served as the full-stack acceptance test for a while and was a genuinely harsh +one, because it was not written against our bugs — it found three real defects +(`O_NONBLOCK` being a documented no-op, evdev replaying its backlog, and no +`EVIOCGRAB`). It was removed once it stopped finding anything, because carrying +it meant 33 MB of non-redistributable data staged into `sysroot/`, which is +linked into the kernel binary for a `ramroot` build. + +> **The shape of that test is worth copying even though the program is gone:** +> something large that we did not write, driven to a real pixel-level assertion +> rather than a log line. + +--- + +## What is not tested + +- **No automated test runs on real hardware.** Everything above is QEMU. +- **SMP is exercised but not stressed** — boots on 1, 2 and 4 processors are + checked; nothing hunts for races. +- **No fuzzing anywhere**, and no `fsx`/`xfstests` against SchFS. Getting SchFS + under those is the main argument for the FUSE implementation in + [../plan/roadmap.md](../plan/roadmap.md). +- **The SchFS round-trip test needs symlinks and hard links**, so it skips on a + Git Bash host and only runs on Unix. +- **`make ports` does not work on Windows.** musl, toybox and mksh must be built + once on a Unix host and staged. diff --git a/docs/history/README.md b/docs/history/README.md new file mode 100644 index 0000000..e20176b --- /dev/null +++ b/docs/history/README.md @@ -0,0 +1,54 @@ +# History + +**Why SchOS is the way it is.** Bugs whose cause was nowhere near their symptom, +and the few write-ups that have nowhere better to live. + +| | | +|---|---| +| [kernel.md](kernel.md) | The expensive ones. Boot, SMP, signals, ABI assumptions, `/proc`, input. | +| [schfs.md](schfs.md) | How the filesystem was built, and what each phase cost. | +| [moving-to-linux.md](moving-to-linux.md) | The Windows-to-Fedora move. | + +--- + +## What belongs in here + +> **A bug whose cause is not visible from the code that fixed it.** + +That is the whole test. If reading the fix explains the problem, the fix is the +documentation and nothing needs writing down. If the fix looks arbitrary — a +constant moved by eight, a field reordered, a check inverted — then the +strangeness is load-bearing and the next person to touch it deserves to know +why before they change it back. + +Every entry here is a thing that looked like a different problem than it was. + +## What does not belong + +- **A changelog.** `git log` is the changelog. +- **A bug that was found and fixed in an hour.** Cost is the filter. +- **How something works.** That is [../architecture/](../architecture/), written + in the present tense for somebody who does not care what it used to be. +- **Anything still open.** An unfinished investigation goes in + [../plan/](../plan/) or in the code as a comment. This directory is past tense. + +## How to write one + +**Symptom first, then diagnosis, then the fix, then the transferable part.** + +The transferable part is the reason the entry exists. `__thread` being wrong for +per-CPU data is worth more than the specific null dereference it caused; +"a layout constant that several artifacts must agree on wants a check that stops +the build, not a comment asking people to remember" is worth more than the +bootloader offset that taught it. + +**Say how it was found**, when the technique was the hard part. A hardware +watchpoint through QEMU's gdb stub named an instruction on the first run after +three rounds of `cprintf` had narrowed it to nothing — that is reusable, and the +bug is not. + +**Do not clean up the embarrassing part.** The near-miss where a change passed +every assertion green *without having been applied at all* is one of the most +useful entries in this directory, and it only works because it admits that a +`grep` returning three hits was read as confirmation without counting what they +were. diff --git a/docs/architecture/kernel-history.md b/docs/history/kernel.md similarity index 92% rename from docs/architecture/kernel-history.md rename to docs/history/kernel.md index 1290f53..3d5c06a 100644 --- a/docs/architecture/kernel-history.md +++ b/docs/history/kernel.md @@ -1,18 +1,36 @@ -# Kernel history — what broke, and what the fix actually was +# Kernel history -Not a design document. This is the record of bugs that were expensive to find -and whose causes are not visible from the code that fixed them, plus the few -subsystem write-ups (bootloader, boot speed, naming) that have nowhere better -to live. +Bugs that were expensive to find and whose causes are not visible from the code +that fixed them, plus a few subsystem write-ups that have nowhere better to live. -It is worth keeping because every entry here is a thing that looked like a -different problem than it was. +**Every entry here is a thing that looked like a different problem than it was.** +What each subsystem *is* today is [../architecture/](../architecture/); this is +only how it got that way. + +| | | +|---|---| +| [1](#1-fixed-boot-didnt-reliably-reach-a-working-shell) | Boot did not reliably reach a working shell — ten bugs behind one symptom | +| [2](#2-naming-standardization) · [3](#3-coreutils) | Naming, and where the userland comes from. Not bugs. | +| [4](#4-two-stage-bootloader--psogm-splash) | The two-stage BIOS bootloader. **Since removed** — kept for the bugs. | +| [5](#5-at-syscalls-ignoredrejected-real-dirfds) | `*at()` ignored real dirfds, and `ls -l` hung | +| [6](#6-boot-speed) | Boot speed, in the loader | +| [7](#7-at-syscalls-real-signal-delivery-and-mksh-job-control) | Real signal delivery, and mksh's job control | +| [8](#8-two-abi-assumptions-that-only-gcc-was-keeping-true) | Two ABI assumptions only GCC was keeping true | +| [9](#9-the-boot-looked-like-a-hobby-os-and-two-things-were-why) | The boot looked like a hobby OS | +| [10](#10-a-full-screen-program-typed-the-whole-session-into-the-shell) | A full-screen program typed the session into the shell | +| [11](#11-ps-and-top-and-why-they-never-worked) | `ps` and `top`, and why they never worked | +| [12](#12-a-test-program-crashed-at-random-and-had-been-typing-its-own-launch-command) | A program crashed at random, typing its own launch command | + +The SMP bring-up bugs are not here — they are summarised in +[../architecture/boot.md](../architecture/boot.md#4-smp), because the lesson +(`__thread` is wrong for per-CPU data) is load-bearing on the current code +rather than merely historical. > **Where the old design plan went.** Sections 1-6 of this file used to hold a > "Segregated Kernel" design: a Ring 0 Alpha / Ring 0 Beta split with driver > modules isolated by CR3 switching, from an `IDEA.txt` that no longer exists. > None of it was ever implemented, and it is superseded by -> [schapi.md](schapi.md) — a macOS-shaped system answers driver isolation with +> [../plan/schapi.md](../plan/schapi.md) — a macOS-shaped system answers driver isolation with > userspace drivers and a personality boundary, not a bespoke in-kernel ring > split. It also referenced a Zig toolchain that is now clang/LLVM. Removed > rather than left to be mistaken for a plan. @@ -135,6 +153,12 @@ shared inode, but needed no mkfs changes). ## 4. Two-stage bootloader + PSOGM splash +> **This describes the BIOS loader, which no longer exists** — see +> [`boot/README.md`](../../boot/README.md) for why it was removed and what +> replaced it. The section is kept because four of the bugs in it are about +> things that are still true: extended scancodes, register hand-off across a +> mode switch, and layout constants that several artifacts must agree on. + `boot/` used to be a single BIOS boot sector doing everything (VBE mode set, A20, GDT, protected-mode switch, multiboot kernel scan/load) — silent, and packed to within 4 bytes of the hard 510-byte MBR limit, so there was @@ -307,10 +331,18 @@ Two cheap, real wins in `boot/bios/stage2main.c`, no functional changes: or not the menu is wanted, so it's the most direct lever available without giving up the feature entirely. -## 7. `*at()` syscalls, real signal delivery, and mksh job control (in progress) +## 7. `*at()` syscalls, real signal delivery, and mksh job control + +> **Resolved since this was written.** Process groups and sessions are real +> (`pgid`/`sid` in `struct proc`, `setpgid`/`setsid`/`getpgid`/`getsid`, and +> signal delivery to a whole group), which was the first of the three suspects +> listed at the end. mksh's job control works. The investigation is kept in full +> because the signal-delivery mechanism described below is still exactly how the +> kernel does it, and because the reasoning about *why SIGCHLD was left enabled +> while broken* is the transferable part. `ls -l`/`ls -la` hanging outright turned out to be the same root cause -described in section 11 above (dirfd-relative path resolution) — fixed +described in §5 above (dirfd-relative path resolution) — fixed and verified extensively (plain `ls`, `ls -l`, `ls -la`, `ls -lR`, piped output, all confirmed working repeatedly). diff --git a/docs/moving-to-linux.md b/docs/history/moving-to-linux.md similarity index 79% rename from docs/moving-to-linux.md rename to docs/history/moving-to-linux.md index fd97554..3d18ce2 100644 --- a/docs/moving-to-linux.md +++ b/docs/history/moving-to-linux.md @@ -1,8 +1,23 @@ # Moving the project to a Linux machine -Written for the move to `/home/nexoniarz/Documents/SchOS` on Fedora 44. Nothing -here is Fedora-specific except the package names. +**This move happened.** The tree lives on Fedora now, and the checkout is at +`/home/nexoniarz/Dokumenty/Projects/SchOS` rather than the path written below. + +It is kept in `history/` rather than `guides/` because most of it is a +one-time procedure that has been performed — but three parts recur and are +worth knowing before the next machine: + +- **what is and is not in the repo** (§1), which decides what a `git clone` + gives you versus an `rsync` +- **the memory key is derived from the absolute path** (§2), so it differs per + machine and `memory-sync.sh` exists for exactly that +- **the two things that behave differently on Linux** (§4) — `make ports` works, + and the SchFS round-trip test actually runs, because it needs symlinks and + hard links that Git Bash cannot create + +Nothing here is Fedora-specific except the package names. For building on any +host, see [../guides/building.md](../guides/building.md). ## 1. Copy the repo @@ -12,8 +27,8 @@ Copy the whole directory, `.git` included. Everything that matters is committed ```sh # whatever transport suits: a USB stick, rsync, scp -rsync -a /path/to/SchOS/ /home/nexoniarz/Documents/SchOS/ -cd /home/nexoniarz/Documents/SchOS +rsync -a /path/to/SchOS/ /home/nexoniarz/Dokumenty/Projects/SchOS/ +cd /home/nexoniarz/Dokumenty/Projects/SchOS git log --oneline | head -3 # should show the same commits git status # should be clean ``` @@ -44,7 +59,7 @@ absolute path, so it differs per machine: | | | |---|---| | `D:\DEV\SchOS` | `D--DEV-SchOS` | -| `/home/nexoniarz/Documents/SchOS` | `-home-nexoniarz-Documents-SchOS` | +| `/home/nexoniarz/Dokumenty/Projects/SchOS` | `-home-nexoniarz-Dokumenty-Projects-SchOS` | One command puts it where this machine looks: @@ -71,7 +86,7 @@ tools/posix/setup-fedora.sh # install it ``` That covers clang, lld, llvm, make, gcc, perl, python3, qemu-system-x86, -edk2-ovmf, xorriso and mtools. See [building.md](building.md) for what each is +edk2-ovmf, xorriso and mtools. See [../guides/building.md](../guides/building.md) for what each is for and for the other supported hosts. **PowerShell is the one thing dnf cannot supply.** `tools/windows/bootcheck.ps1` diff --git a/docs/history/schfs.md b/docs/history/schfs.md new file mode 100644 index 0000000..f92410e --- /dev/null +++ b/docs/history/schfs.md @@ -0,0 +1,370 @@ +# SchFS — how it was built + +The filesystem as it is now is [../architecture/schfs.md](../architecture/schfs.md). +This is the record of building it: the phase plan, what each phase actually +produced, and the bugs that cost real time. + +**All five phases are complete.** The plan is kept because the reasoning in it +is still the reasoning behind the format, and because the ordering argument — +*do not start phase 2 before phase 1 is solid, the B-tree is the load-bearing +piece* — is the transferable part. + +--- + +## The phases, as they were planned and as they went + +Each phase leaves a bootable system. Do not start phase 2 before phase 1 is +solid — the B-tree is the load-bearing piece and everything else assumes it. + +**Phase 1 — the B-tree, in userland first.** Write the CoW B-tree as a plain C +library with a block-device abstraction, and test it on a *host* file with a +fuzz/torture harness (random insert/delete/lookup, simulated crash at every +write point, verify the tree always mounts). This is the one component where a +subtle bug is unfixable later, and testing it in the kernel is far harder. It is +also portable straight into `tools/mkfs`. + +**Phase 2 — read-only mount. DONE.** New magic, uberblock scan, tree walk, inode +and dirent key types. `tools/mkschfs` writes images; the kernel only reads them. +The system boots from a ramdisk root (see [../guides/layout.md](../guides/layout.md) and the ramdisk work) +so a broken disk cannot stop you from booting. + +Went further than planned in two places, both cheaply: data blocks carry +checksums too (a `CSUM` key type, as btrfs does — without it "end-to-end +checksums" would cover only metadata, which is the part least likely to rot), +and extents landed early because reserving a file's blocks in one go was +simpler than not doing it. + +Not yet done: SchFS files are not reachable through `open(2)`. The mount is +observable through `/proc/schfs` and through the kernel API in `schfs_int.h`, +but wiring it into `namei`/`readi` means the VFS work that phase 3 needs anyway. + +**Phase 3 — writes, transaction groups. DONE.** Every `schfs_k*` call that +changes anything is one transaction group, committed by writing a new uberblock +before it returns. The full write API — create, link, unlink, rename, write at +an arbitrary offset, truncate — lives in `objw.c` and is exercised by +`tools/schfs_objtest.c`. + +Two things went differently from the plan above, both for reasons worth +recording. + +*Free space is not tracked as tree keys.* Allocating from a free map that lives +in the tree mutates the tree, which allocates, which changes the map; btrfs +resolves that with a fixpoint loop and ZFS with log-structured space maps. SchFS +sidesteps it: the committed tree already says exactly which blocks are live, so +the map is **derived** by walking the tree once at mount and held only in RAM. +It cannot disagree with the filesystem, because it is computed from it, and a +crash raises no question at all. The cost is mount time proportional to metadata +size, and a ceiling on volume size set by how much bitmap fits in memory. If +mount time ever hurts, the map can be cached on disk purely as a hint, +validated against the generation and discarded on a mismatch — an addition, not +a format change. See `kernel/fs/schfs/space.c`. + +*Data blocks needed a generation stamp too.* The B-tree already rewrites its own +nodes in place within a group, since a block that no committed root can reach is +nobody's fallback. Data blocks had no such marker, so every hundred-byte write +copied a whole 4 KB block and split an extent record around it. `schfs_extent` +gained a 64-bit `gen`, and the same rule now applies to file data. + +**Phase 3½ — reachable through `open(2)`. DONE.** SchFS is a mounted filesystem: + +``` +mkdir /mnt +mount -t schfs /dev/hdb /mnt +ls /mnt ; echo hi > /mnt/tmp/w ; cat /mnt/tmp/w ; mkdir /mnt/tmp/d +``` + +The VFS grew a real dispatch layer rather than a set of `if (dev == SCHFS_DEV)` +checks. `struct fs_ops` (include/file.h) is one table per filesystem, chosen by +`iget()` from the device number and hung off the inode; v1's implementations +keep their bodies and are simply named `v1_*`. Adding a third filesystem is a +new table and nothing else. `kernel/fs/schfs/vfs.c` is SchFS's, and it is pure +translation — object ids in, inodes out — because all the filesystem logic +already lives below it. + +Three things had to change on the way, none of them SchFS's fault: + +- `struct inode.inum` and `.size` were 32-bit, capping objects at 2³² and files + at 4 GiB on a filesystem that promises far more. Both are `uint64` now, as are + `readi`/`writei` offsets and `struct file.off`. +- `struct dirent` carries a 16-bit inum and a 14-byte name, so it cannot + represent a SchFS entry at all. `getdents64` is now an fs_op: each filesystem + emits the record stream itself, and `f->off` is whatever position it needs to + resume. v1 walks its hash buckets; SchFS walks a key range and synthesises + "." and "..", which it does not store. +- Mounting is a substitution made during path resolution (`mount_add` / + `mount_cross` in `fs.c`). Nothing on either disk records it. The reverse + direction matters as much as the forward one: `..` at a mounted root has to + leave the mount, and the filesystem underneath has no idea it is mounted + anywhere. + +Three long-standing bugs surfaced the moment real commands ran against it, all +in the shared VFS rather than in SchFS: + +- `include/fcntl.h` used xv6's open flags while every program in the userland + sends Linux's. `O_APPEND` (0x400) landed on `O_TRUNC`, so `cmd >> file` + *truncated* it; `O_TRUNC` (0x200) landed on `O_CREATE`, so `cmd > file` left + whatever was past the new end in place. Nothing about either pointed at a + header. +- `sys_openat` silently dropped `O_TRUNC` and `O_APPEND` while `sys_open` + honoured both, so the same redirection behaved differently depending on which + entry point libc happened to call. +- `rmdir` had no Linux syscall number, so every Linux-ABI program got ENOSYS. + +**Phase 3¾ — durability, and the cost of it. DONE.** Three things that the +earlier phases had assumed rather than arranged. + +*The commit barrier.* Copy-on-write promises that a crash lands on either the +old tree or the new one, but only if the blocks a root points at reach the +medium *before* the block naming that root. Nothing enforced that: `bwrite` +order is not platter order on any drive with a write cache. `struct schfs_bdev` +gained a `bflush` hook, `schfs_vol_commit` calls it either side of the uberblock +write, and the kernel implements it as "drain this device's dirty buffers, then +ATA FLUSH CACHE" (`ideflush`, and SCSI SYNCHRONIZE CACHE for a USB stick). + +*Falling back through the ring.* Mount took the highest-generation uberblock +that validated, checked the tree it named, and gave up if that failed — with up +to 127 older, entirely intact roots sitting on the disk. It now walks the ring +newest-first and adopts the first root it can actually read, counting what it +skipped into `vol->recovered` so the rollback is reported rather than silent. + +*Deferred writes, and brackets.* The B-tree seals and writes a node on every +record it inserts, so a leaf collecting one file's data checksums went to the +disk once per 4 KB of that file. The kernel's block hook marks buffers dirty +instead (`bdirty`), and the commit barrier is what makes that safe. On its own +that changes little, because a commit *contains* a flush and every `schfs_k*` +call was one commit — so `schfs_ktx_open`/`schfs_ktx_close` bracket a run of +calls into one group, and `schfs_vfs_write` wraps a whole `write(2)` in one. + +Measured over 1 MB of file data, in block writes reaching the device: + +| | before | after | +|---|---|---| +| committing every 4 KB | 1947 | 1410 | +| one commit for the megabyte | 1798 | **267** | + +267 device writes for 256 blocks of data. Note that the two changes are worth +almost nothing separately — deferral cannot help while every commit flushes, +and batching cannot help while every write goes straight to the disk — which is +why the first attempt at this measured an 8% difference and concluded batching +was not the problem. + +**Phase 3⅞ — where the time actually went. DONE.** The filesystem was correct +and durable and the system still felt slow: `ls /bin` took the better part of a +second. Two causes, neither of them the disk. + +Measured first, through `/proc/fsstat` (counters added for exactly this). The +decisive number was **398,999 block reads against 445 actual disk misses** — +99.9% of all block reads were re-reading a block already in the buffer cache. So +the disk was never the problem, and every plan that started with "cache more" or +"read ahead" would have been aimed at 1% of the cost. + +*The cursor re-read its own buffer.* `cur_load` called `node_read` +unconditionally, and `schfs_cursor_val` calls `cur_load` on **every access to +the current record** — deliberately, so that a callback disturbing the shared +block buffer cannot corrupt a directory walk. But `node_read` copies 4096 bytes +out of the buffer cache and then runs CRC32c over all 4096. Listing a 243-entry +directory paid both once per entry. The cursor now remembers which block its +buffer holds and skips the reload, cross-checked against the block header (so a +second cursor sharing the page cannot fool it) and against a global epoch that +every mutation bumps (so a block rewritten in place cannot be served stale). + +*The checksum was bit-at-a-time.* `schfs_crc32c` did eight shifts per byte — +32,768 iterations to checksum one block — to stay usable before any +initialisation. It builds a 1 KB table on first use now, which keeps that +property and is eight times faster. This only became the top cost *after* the +cursor fix, which is the usual shape: removing the largest item promotes +whatever was beneath it. + +| cycles, one session | before | after | +|---|---|---| +| `getdents` | 26.8 G | **0.14 G** (196×) | +| `read` | 26.4 G | **5.5 G** (4.8×) | +| `exec` | 18.7 G | **4.0 G** (4.7×) | +| block reads | 398,999 | **127,713** | + +`ls /bin` went from ~400 ms to ~20 ms. Verified by reading an 802 KB binary +through the whole path and checking its md5 against the host's copy. + +There was also a third of the problem that was not the kernel at all: `make +qemu` ran QEMU without `-enable-kvm`, so every instruction was emulated. The run +targets probe `/dev/kvm` now. Measure against the KVM number or you are +measuring TCG. + +**Phase 4 — extents with refcounts. DONE.** `cp` that costs nothing. + +A reference count per run of blocks, in its own tree (`SCHFS_KEY_EXTREF` under +`SCHFS_EXTREF_OBJECTID`) rather than in the file's own extent record — because a +count kept per file cannot be decremented by the *other* file sharing the run. +`schfs_file_reflink` copies one extent record per run and bumps the count; both +files stay ordinary and writable, and the first write to either copies only the +run it touches. Reachable from userland as `FICLONE`, Linux's ioctl, so +`cp --reflink` works unmodified; `cmd/reflink` is a two-line caller of it. + +Measured on the running system: cloning a 411 KB file moved `df` by 24 KB. + +Three things had to be right, and each was wrong first: + +- **The count is "how many extent records point into this run", not "how many + files".** Splitting a shared extent around a written block leaves a head *and* + a tail, which is one more reference than before. Missing that leaked four + blocks per split. +- **A file's extent may point into the middle of a run.** Records are keyed by + the block a run was *allocated* at, so every lookup is a range query + (`extref_find`), not an exact one. An exact lookup finds nothing for a split + tail, "nothing" means unshared, and the tail is then freed while another file + still reaches it. +- **The space scan has to walk the reference records too.** A file truncated + inside a shared run describes fewer blocks than exist; the rest would be + derived as free while still referenced, and the allocator would hand out a + block that a later release frees a second time. + +None of those is visible until every holder of a run is deleted, which is why +the test that matters is the least glamorous one: after deleting every file that +shared a run, free space must be back to exactly what it was. + +**Phase 5 — attribute index and query. DONE.** The reason the design was one +B-tree with many key types from the start: the "database filesystem" feature +turned out to be `kernel/fs/schfs/attr.c` and nothing else. + +An attribute is a named value on an object, stored under `(ino, ATTR, +hash-of-name)` and packed exactly as directory entries are — same collision +argument, same shape. An attribute may be *indexed*, and an indexed attribute +gets a second set of records keyed by value: `(hash-of-name, INDEX, +hash-of-value)` holding the object ids that carry it. So "every object whose +`tag` is `holiday`" is one tree descent instead of a walk of every object. + +There is no query language, no parser and no planner, and §6 above explains why: +the useful half of "query the filesystem" is the index, and WinFS spent three +years and a large team on the other interpretation. + +``` +schattr index tag # declare the index +schattr set /tmp/a tag holiday +schattr find tag holiday # one descent, whatever the volume holds +``` + +Four decisions worth keeping: + +- **Which attributes are indexed is cached in the volume**, read once at mount. + Every `setxattr` has to ask, and the answer is almost always no; a tree + descent to say so would make every attribute write pay for a feature it is + not using. +- **The value is stored in the index entry, not just its hash.** Two values can + hash alike, and a query that returned the wrong file would be far worse than a + slow one. +- **An index covers values written after it is declared.** There is no + reindexing pass. Saying so is better than a half-truth that makes a query + answer confidently and wrongly. +- **A query on an unindexed attribute is refused, not answered by scanning.** A + query that silently degrades into a full walk teaches you to use it where it + will not scale. + +Reachable as `setxattr`/`getxattr`/`removexattr` at Linux's syscall numbers, so +anything that already knows how to attach metadata needs no changes, plus two +native calls (`schfs_index`, `schfs_query`) that have no Linux equivalent +because asking a filesystem a question does not. + +**Phase 5 — attribute index and query.** See below. + + +--- + +## What was reused from v1 + +Not starting from zero: + +- **`struct schfs_extent`** is already defined in `include/fs.h` and unused. + The extent concept is already sketched; v2 gives it refcounts and 64-bit + fields. +- **CRC32c checksum fields** already exist in `dinode` and `superblock`, and + mkfs already computes them. Same algorithm, relocated into parent pointers. +- **`kernel/fs/log.c`** is a working transaction log with `begin_op`/`end_op` + call sites throughout the FS. Under CoW the log is not needed for consistency, + but `begin_op`/`end_op` are exactly the right places to bracket a transaction + group, so the *call sites* survive even though the implementation is replaced. +- **The buffer cache** (`bio.c`) is format-agnostic and needs no change beyond + the block-size constant. + + +--- + +## What it was estimated to cost + +Phases 1-3 are roughly 3,000-4,000 lines and are the bulk of the work. Phase 1 +alone (B-tree + torture harness) is worth doing carefully and unhurriedly; every +later phase compounds on it. + +This is the largest single piece of work in SchOS. It is also entirely +ordinary filesystem engineering with excellent published references +(Giampaolo for BeFS; the btrfs design papers; the ZFS uberblock/checksum-in- +parent scheme) — there is no research risk here, only volume. + +**Do this before the installer.** The installer's entire job is to create and +populate this format. Building it against v1 means building it twice. + + +--- + +## Things learned the hard way + +Recorded because each one cost real time and none is obvious from the code. + +- **The root's object id must be reserved.** `next_objectid` started at + `SCHFS_FIRST_OBJECTID`, which *is* the root's id, so the first directory + created silently became the root. It then contained itself, and every tree + walk recursed until the stack ran out. `schfs_fsck --check` now detects + directory loops explicitly rather than crashing. +- **Cursor buffers alias.** A range callback gets a pointer into the cursor's + block buffer, and almost every readdir caller immediately looks the entry up, + which overwrites it. Directory iteration copies each entry out before calling + back. Extracting a tree with directories reparented under their own children + is what this looks like when it goes wrong. +- **A 4096-byte block does not fit on a 4096-byte kernel stack.** The cursor + exists precisely to avoid one block buffer per tree level, but + `schfs_vol_open` still kept a whole uberblock as a local and blew the stack on + the first boot. The kstack canary caught it cleanly. +- **Extents only merge if you allocate the run first.** Writing an extent record + allocates blocks for its own copy-on-write path, so allocating data one block + at a time guarantees the blocks are never adjacent and every block needs its + own extent. `schfs_file_reserve` claims the whole run up front; an 822 KB + binary is one extent instead of 201. +- **Freed blocks are reusable during a build.** Copy-on-write forbids reusing a + released block because the previously committed tree still points at it — but + a build has committed nothing. Recycling them cut the test image from 3596 + blocks to 374. +- **Reproducibility dies from atime.** mkschfs reads every source file, which + updates its atime, so copying atime into the image made two builds of the same + tree differ. All three timestamps come from mtime. +- **An optional hook must still be initialised.** `struct schfs_bdev` gained + `bflush`, and every host tool built one on the stack field by field. The tools + that did not set the new field handed `schfs_vol_commit` a garbage pointer to + test and call, which segfaulted at the first commit. Each of those helpers + memsets the struct first now — the fix is not "remember to set it". +- **Deferring writes makes an all-dirty buffer cache normal.** `bget` panicked + with "no buffers" when it could find nothing clean to recycle, which was + correct while a dirty buffer meant "the log has not committed". It now writes + the least recently used victim out and takes it; a transaction touching more + than `NBUF` blocks reaches that state long before it commits. +- **The two performance fixes are worthless apart.** Deferring tree writes buys + nothing while every commit flushes, and batching commits buys nothing while + every write goes straight to the disk. Measuring either alone says "this is + not where the time goes", and both measurements are correct. +- **`fsid_t` is eight bytes.** `struct statfs` declared `f_fsid` as `int64[2]`, + which is one slot too long, so everything after it landed in the wrong place: + userland read `f_frsize` out of `f_namelen` and got 255. df then computed in + units of 255 bytes and called a 32 MB root filesystem 1992 KB. It survived + because every field in that struct used to be an invented constant, so no + number in it had ever been checked against reality. + +--- + +## The one about the tests themselves + +`tools/posix/mutate.sh` injects six deliberate bugs and fails if any goes +undetected. + +It exists because a suite that passes is not evidence until something has +checked that it can fail. The most valuable single assertion in the whole SchFS +suite is also the least glamorous: **after deleting everything, free space must +be back to exactly what it was on an empty volume.** Every other test still +passes on a filesystem that leaks a block on every delete. diff --git a/docs/licensing.md b/docs/licensing.md index ae1a107..d7c08f7 100644 --- a/docs/licensing.md +++ b/docs/licensing.md @@ -68,7 +68,7 @@ specification rather than ported. ## 3. NetBSD in particular `src-trunk/` is a NetBSD checkout kept as reference material and a parts bin -(see [architecture/schapi.md](architecture/schapi.md) §5). NetBSD is **not +(see [plan/schapi.md](plan/schapi.md) §5). NetBSD is **not uniformly licensed**, and the difference matters per file: | Template | Status | @@ -142,7 +142,7 @@ the FreeBSD Foundation and Juniper Networks -- all BSD-2 or BSD-3. So if BSD networking code is taken, it must come from **FreeBSD, not NetBSD**, and the specific reason is TCP. This reverses the earlier assumption in -`docs/roadmap.md` that NetBSD was the intended starting point; NetBSD remains +`docs/plan/roadmap.md` that NetBSD was the intended starting point; NetBSD remains the better-written reference to *read*, but FreeBSD is the one that can be copied from. @@ -159,7 +159,7 @@ FreeBSD's `sys/netinet` is not a library -- `tcp_input.c` alone is 122 KB and reaches into mbufs, `epoch(9)`, netisr, VNET, `uma(9)`, sockbufs, inpcb hash tables and FreeBSD's locking. Taking it means porting FreeBSD's kernel infrastructure first and finding the TCP somewhere underneath, which is the same -conclusion `docs/roadmap.md` §1 already reached about `sys/dev/usb`. +conclusion `docs/guides/adding-a-driver.md` already reached about `sys/dev/usb`. lwIP was written for the case SchOS is in: a small operating system with a raw Ethernet driver and no infrastructure to speak of. Its port layer is a handful @@ -225,7 +225,7 @@ has since been removed. It is worth recording *why* it was safe: it was never in `PORTS`, and its 33 MB of data was excluded from the ramdisk. It is equally worth recording that the safety depended on two separate opt-outs staying correct, which is a lot of standing risk for a test that had stopped finding -bugs. See `docs/roadmap.md` Phase 7. +bugs. See `docs/guides/testing.md`. --- diff --git a/docs/plan/1.0.md b/docs/plan/1.0.md new file mode 100644 index 0000000..999639f --- /dev/null +++ b/docs/plan/1.0.md @@ -0,0 +1,267 @@ +# SchOS 1.0 + +A finish line, so that the list of open work stops being infinite. + +--- + +## Why this document exists + +[schapi.md](schapi.md) §3 says the stability freeze starts at 1.0 and covers +whatever is marked `stable` at that moment. **1.0 was never defined**, which +means there has been no finish line — and with no finish line every gap is +equally mandatory, the list only grows, and the project feels larger every week +regardless of how much gets done. + +That feeling is structural, not a judgement about progress. This document fixes +the structure by drawing a line and putting most of the open work on the far +side of it. + +> **Anything not named in this document is not part of 1.0.** It is not +> cancelled, not forgotten, and not up for reconsideration until 1.0 ships. + +--- + +## What 1.0 is + +> **A machine one person can sit down at and use for an afternoon without +> falling off an edge.** + +Concretely — boots on real hardware, roots on SchFS, runs a shell and the real +toybox userland, is on the network, and puts a windowed terminal on the screen. + +It is deliberately a **small complete system** rather than a rich incomplete +one. SchOS 1.0 is not Linux, does not run third-party applications, and has no +package manager. It is a system that works, end to end, with nothing in it that +lies about what it did. + +### The acceptance test + +One unattended run, asserted from the host, the way everything else here is +tested. When this passes, it is 1.0: + +``` +boot on real hardware from a USB stick + → SchFS root mounts, generation counter advances + → schinit starts the compositor + → a terminal window appears and is asserted by pixel + → in it: ls, cat, a file written, a network fetch + → reboot + → the file is still there and the checksum matches +``` + +Nothing in that sentence is a feature list. Every clause is something that is +currently either broken, missing, or unproven. + +--- + +## The four gates + +In order. **Do not start a gate before the one before it closes**, and closing +means a description exists in [../architecture/](../architecture/) — see +[the rule](../README.md#the-rule-that-keeps-this-from-rotting). + +### Gate 0 — Fix what is already wrong + +**This comes first, before any new capability.** The desktop work is the +temptation and it is the wrong thing to do next: a compositor built on a kernel +with an unexplained hang and a socket layer that reports success for things it +did not do will fail in ways indistinguishable from its own bugs. That is the +same argument that put `netdev_selftest()` before the TCP/IP stack, and it was +right then. + +**Things that are actively wrong:** + +- [x] ~~**The `ls -lR` hang.**~~ **Not reproducible.** Instrumented and hunted; + see below. Left here rather than deleted because the absence of a result + is the result. +- [x] ~~**`setsockopt` returns 0 without setting anything.**~~ Six options now + genuinely reach lwIP's pcb and everything else is `ENOPROTOOPT`. Round-trip + checks in `cmd/socktest`, which a permissive stub cannot pass: it has to + read back a value it was set *and* refuse an unknown option. +- [x] ~~**`accept4` silently drops its flags.**~~ It is its own table entry now + rather than an alias for `accept` — an alias would have had to read a + fourth argument that `accept` never passes, inventing a `SOCK_NONBLOCK` + out of whatever the caller last left in that register. +- [x] ~~**Timestamps on everything created after boot read 1970.**~~ Both + `know()` (SchFS) and `now_sec()` (v1) returned `uptime()` under comments + saying the kernel had no wall clock. It has had one since `time_init()` + started reading the RTC; the comments outlived the condition. Two errors, + not one — `uptime()` counts 10 ms ticks and they were used as seconds, so + the clocks also ran 100× fast. +- [ ] **`getrusage` does not distinguish `RUSAGE_CHILDREN`** from `RUSAGE_SELF`. +- [ ] **`/proc//exe` is a regular file, not a symlink**, so `readlink()` + fails and `ps -o comm` prints `[init]`. Needs symlink support in procfs. +- [ ] **OVMF memory reporting**, if the region-list fix did not fully close it. + +**Work already done and not connected:** + +These are the galling ones — the replacement exists, is tested, and is unused. + +- [ ] **Port the console onto the text layer and delete the cell grid.** + `text.c` and `ttf.c` both exist and work. The console still indexes + `font_data[256][16]`. +- [ ] **Load a font from disk** and point the console at it. +- [ ] **Interrupt-driven receive.** The network thread polls the NIC every timer + tick, so a frame waits up to 10 ms. Waking it from the e1000 interrupt was + deliberately not the first version; it is time. + +**Structural, because everything after this compounds on it:** + +- [ ] **A slab or pool allocator.** `kalloc` hands out one page at a time, and + every subsystem written since has invented its own fixed-size table to + work around it — `NPROC`, `NSOCK`, `NDEVICE`, `NSHM`, `MAXJOBS`. This is + the single change that most improves everything else in the kernel, and + it gets more expensive with every table added. +- [x] ~~**SchFS on-disk format version policy.**~~ Done. `SCHFS_VERSION` for + frame changes, plus `compat`/`ro_compat`/`incompat` feature flags in what + was `reserved0` — so existing images keep every field offset and read + unchanged. `uber_check()` now distinguishes *damaged* (step over it; that + is what the ring is for) from *too new* (stop), because falling back to an + older uberblock mounts a stale filesystem and reports success. Proven on a + forged image: the old code mounted the stale root and exited 0. + Still owed: an actual upgrade step, which is only writable once there are + two formats. See [../architecture/schfs.md](../architecture/schfs.md). + +**Found while doing the above, and fixed:** + +- [x] ~~Every `AF_INET` socket failed with `ENFILE` until DHCP bound.~~ + `inet_new()` required an *address* before it would hand out a socket, and + reported the refusal as "file table overflow". Creating a socket is not + the same as having somewhere to send; a bind or connect with no route + fails on its own with an error that says so. `net_is_ready()` (lwIP up) is + now distinct from `net_is_up()` (DHCP bound). +- [x] ~~`socktest` asserted "AF_INET is refused, not faked".~~ True when written, + false since AF_INET landed — and **green the whole time**, because the + suite runs before DHCP binds and the `ENFILE` above made the refusal look + real. A test asserting a failure has to say *which* failure. + +**The identity item:** + +- [ ] **Boot from a USB disk.** The xHCI stack, bulk-only transport, SCSI and + the block device all work; a superblock reads back through `bread()`. What + is missing is that the root device is compiled in rather than selectable. + Without this the acceptance test above cannot run at all. + +#### The `ls -lR` hang, and why it is closed + +Hunted with the instrumentation it asked for, under a workload heavier than the +one that originally triggered it — ten iterations of `ls -lR /bin`, interleaved +with writes and `cd`, 33 execs and 13378 tree descents. Measured across it: + +``` +inode_peak 9 / 200 unchanged +file_peak 9 / 512 unchanged +buf_pinned_peak 1 / 256 unchanged +schfs0 locked 0 tx_owner_pid 0 tx_depth 0 +``` + +Nothing climbed, nothing wedged, and the repaired `holdingsleep` assertion never +fired. **That eliminates this file's own stated hypothesis** — "a lock or +reference-count leak somewhere in the `*at()` paths" — and it is very likely the +bug was fixed in passing by the `*at()` dirfd rewrite, the getdents cookie +rewrite, or SchFS becoming the root, all of which landed after it was reported. + +It is closed rather than chased further, and that is a deliberate application of +the two-things-in-flight rule. The instrumentation is permanent: if it ever +happens again, `/proc/fsstat` names the table, or shows `locked 1` with +`tx_owner_pid 0` and the volume lock is orphaned. That is a much better position +than the one it was reported from. + +Two real defects were found on the way and fixed, neither proven to be the +cause: `holdingsleep()` never compared the owning pid, which disarmed all seven +`panic()` assertions built on it; and SchFS's `tx_mine()` compared a bare +`struct proc *`, which a recycled ptable slot makes ambiguous. + +### Gate 1 — The foundations + +[foundations.md](foundations.md), items 1–4. Descriptor passing, a shareable +memory object, `AF_UNIX` in the filesystem namespace. + +Nothing above needs these; everything below stops without them. + +### Gate 2 — Init as a role + +[init.md](init.md), milestones I0–I3. + +Before the desktop rather than after, because a compositor being brought up for +the first time crashes constantly, and something that restarts it and says why is +worth far more during that work than at the end of it. + +### Gate 3 — The desktop + +[wayland.md](wayland.md), W1–W3, and the terminal. + +**W4 and W5 are not in 1.0.** Multi-window focus policy and a panel are +post-1.0. One terminal window that works is the bar. + +--- + +## Not in 1.0 + +The point of this list is that it is *written down*, so it can stop occupying +attention. Nothing here is cancelled. + +| | Why it waits | +|---|---| +| **The hybrid kernel** ([hybrid.md](hybrid.md)) | An architecture improvement, not a completeness requirement. It is also the work that most wants a demanding consumer first, and 1.0's compositor is exactly that consumer. | +| **Dynamic linking, Mesa, SDL3** | Everything at 1.0 is statically linked. This is the single biggest post-1.0 unlock and it is a whole project. | +| **A page cache** | `memfd` covers the desktop's needs. General writable `MAP_SHARED` of a file is a real project with no 1.0 consumer. | +| **`.sip` and the installer** | 1.0 installs by writing a USB stick. That is how most systems start and it is honest. | +| **SchFS snapshots, FUSE, Linux module, sparse files, online fsck** | The format version (Gate 0) is the prerequisite for all of them, and it is the only part on a clock. | +| **The interactive crash screen** | PSOGM renders, prints a full report and emits a QR code. That is enough. | +| **CFF/OTF fonts, kerning, text layout** | TrueType and fixed advances are enough for a terminal. | +| **Routing, `getsockname`/`getpeername`, IPv6** | One interface, one default route, works. | +| **Multiple outputs, damage tracking, subsurfaces, xkbcommon** | One screen, full repaints, one keymap. | +| **Audio** | Does not exist in any form and is not required to use a machine. | +| **Panel, notifications, settings, file manager** | A terminal is a desktop. | +| **SchAPI itself** | Deliberately. It should be designed *from* the compositor's experience, not before it. | + +--- + +## The rule for this list + +**Adding to Gate 0–3 requires deleting something else from it.** + +Not a productivity trick — a forcing function. The reason this project feels +overwhelming is that ~20 fronts have been open simultaneously, and no amount of +discipline makes that workable. The fix is structural. + +And one working limit, worth holding to literally: + +> **Two things in flight. Never three.** + +One in Gate 0 and one being written up. Everything else is closed — in the +document, and out of your head. + +--- + +## How you will know it is going well + +Not by velocity. By this: + +**Every closed item leaves a description behind.** When the `ls -lR` hang is +fixed, `history/kernel.md` gains an entry saying what it actually was. When the +console moves onto the text layer, `architecture/graphics.md` loses a line from +its *What it does not do* section. + +Those "does not do" sections shrinking is the real progress bar, and it is the +only one that does not lie. + +--- + +## A note worth keeping + +This document was written because the work started to feel overwhelming, and +that is worth recording rather than tidying away — because the feeling was +about the *shape* of the work and not its quality, and the two are easy to +confuse when you are inside it. + +The evidence at the time: SMP working on real hardware after five bugs stacked +behind each other; a copy-on-write filesystem whose crash test cuts the power one +write later, 119 times, and demands a clean mount after each; a TCP/IP stack +asserted from both ends; an xHCI driver written from the specification; and a +documentation tree that records why things are strange so they do not get +un-fixed by accident. + +None of that is the output of a project in trouble. It was a project with too +many tabs open. diff --git a/docs/plan/README.md b/docs/plan/README.md new file mode 100644 index 0000000..91dfd99 --- /dev/null +++ b/docs/plan/README.md @@ -0,0 +1,105 @@ +# What happens next + +Everything in this directory is **unbuilt**. When a piece of it is finished it +leaves — deleted from here, rewritten as a description in +[../architecture/](../architecture/). See [the rule](../README.md#the-rule-that-keeps-this-from-rotting). + +> ## Read [1.0.md](1.0.md) first. +> +> It draws a finish line and puts most of the work below on the far side of it. +> **Anything not named there is not being worked on**, and the documents in this +> directory describe more than 1.0 contains. + +--- + +## The three current decisions + +Taken together, because they turn out to need the same foundation. + +| | | +|---|---| +| **[wayland.md](wayland.md)** | The GUI stack becomes Wayland. SchWM and SchDE are deleted; Schcom survives and changes its wire format. | +| **[hybrid.md](hybrid.md)** | The kernel becomes hybrid — drivers move out into userspace processes, starting with the GPU. | +| **[init.md](init.md)** | PID 1 stops being a process and becomes a role the kernel guarantees is filled. | + +And the one they all wait on: + +| | | +|---|---| +| **[foundations.md](foundations.md)** | Descriptor passing, shared memory, sockets in the filesystem. The critical path. | + +And the one that says which parts of the above are actually being worked on: + +| | | +|---|---| +| **[1.0.md](1.0.md)** | The finish line. Four gates, and everything else explicitly deferred. | + +Two longer-horizon documents sit behind these: [schapi.md](schapi.md), the API +layering and the compatibility promise, and [roadmap.md](roadmap.md), which holds +everything still open that is not one of the three above. + +--- + +## The order + +Set by [1.0.md](1.0.md), which is the authority. In short: + +### 0. Fix what is already wrong — **first** + +Not a document of its own; it is Gate 0 of [1.0.md](1.0.md), assembled from the +*What it does not do* sections in [../architecture/](../architecture/). + +**Before any new capability**, and the reason is the one that put +`netdev_selftest()` before the TCP/IP stack: a compositor built on a kernel with +an unexplained hang and a socket layer that reports success for things it did not +do will fail in ways indistinguishable from its own bugs. + +### 1. `foundations.md` — the critical path + +Both the Wayland work and the hybrid work stop here. Descriptor passing and +shared memory are the substrate for handing a buffer to a compositor *and* for +handing a device to a driver process. **Doing it once for both is the entire +reason these are one plan rather than three.** + +### 2. `init.md` — before the desktop, not after + +A compositor being brought up for the first time crashes constantly, and +something that restarts it and says why is worth far more during that work than +at the end of it. + +### 3. `wayland.md` — with the GPU driver still in the kernel + +W1–W3 and a terminal. W4 and W5 are post-1.0. + +### 4. `hybrid.md` — **after 1.0** + +Deliberately last: **designing a userspace driver interface with no driver +consuming it produces an interface that is wrong in ways nobody can see.** The +compositor is the demanding consumer that makes the interface judgeable, so it +has to exist first. + +--- + +## In one sentence + +> Fix what is broken, build the descriptor-passing and shared-memory substrate, +> make init survivable, put a terminal on the screen — and only then use the +> compositor as the pressure that shapes the userspace driver interface. + +--- + +## What this plan does not address + +Stated here so it is not mistaken for covered: + +- **No IOMMU**, so the hybrid work isolates against bugs and not against a + hostile driver. +- **No page cache**, so `foundations.md` delivers a `memfd` and not a general + answer to shared file mappings. The general answer is still owed. +- **No dynamic linking**, so no Mesa, no accelerated clients, and no third party + shipping a `.so` for anything. +- **A new compatibility surface** between PID 1 and PID 2, created by a plan + whose own [schapi.md](schapi.md) §3 spends a section warning about those. +- **The three open networking items** — `getsockname`, routing, and + interrupt-driven receive — are untouched by any of this and still open. They + are in [roadmap.md](roadmap.md). diff --git a/docs/plan/foundations.md b/docs/plan/foundations.md new file mode 100644 index 0000000..7e4f165 --- /dev/null +++ b/docs/plan/foundations.md @@ -0,0 +1,135 @@ +# Foundations + +The kernel work that [wayland.md](wayland.md) and [hybrid.md](hybrid.md) both +stop at. **This is the critical path**, and it is the reason those two are one +plan rather than two. + +The insight is small and it decides the schedule: the substrate for passing a +buffer to a compositor and the substrate for handing a device to a driver process +are the same substrate. Descriptor passing and shared memory, built once. + +--- + +## What is actually in the tree + +Measured against the source, not against a README. Each of these was checked +before anything below was planned around it: + +| | | +|---|---| +| `sendmsg`/`recvmsg`, and therefore **`SCM_RIGHTS`** | **Do not exist.** Not in `kernel/sys/table.inc`, not anywhere in `kernel/`. | +| Writable `MAP_SHARED` of a file | **Refused** — `kernel/sys/sys_mem.c:131` returns `-ENODEV`. | +| `memfd_create` | Does not exist. | +| `AF_UNIX` names | In a table in `kernel/net/unix.c`, not in the filesystem. | +| `setsockopt` | `return 0;` unconditionally — `kernel/sys/sys_net.c:436`. | +| `exec` | `ET_EXEC` only. No `PT_INTERP`, no `ET_DYN`. | + +> **`projects/schcom/README.md` says `SCM_RIGHTS` is done. That line is wrong**, +> and it is the most load-bearing wrong line in the tree — an entire GUI plan was +> drawn on top of it. It is corrected as part of this work. + +The `MAP_SHARED` refusal is worth defending rather than fixing carelessly. The +code returns `-ENODEV` *honestly*: the contract of a writable shared mapping is +that stores land in the file and other mappers see them, and there is no page +cache that could make that true. Quietly handing back a private copy would be the +same lie in a new place. + +--- + +## The five items + +In dependency order. The first three are what Wayland cannot start without. + +### 1. `sendmsg` / `recvmsg` with `SCM_RIGHTS` + +Not optional and not substitutable. `wl_shm_create_pool` passes a **file +descriptor**; there is no path in the Wayland protocol that passes an integer +instead. + +The `handle_kind` field in `projects/schcom/proto.h` was good foresight and it +cannot help here — *our* protocol is the flexible one, and Wayland's is not. + +Needs: `struct msghdr`/`cmsghdr` handling, ancillary data queued alongside the +byte stream in `kernel/net/unix.c`, descriptor installation into the receiver's +file table, and correct behaviour when a message carrying descriptors is never +received (they must be closed, not leaked). + +**This is also item 1 of the hybrid work**, where it is how a driver process is +handed the device it is allowed to touch. + +### 2. A shareable memory object + +`memfd_create`, plus `mmap(MAP_SHARED)` on it genuinely working between two +processes. + +Narrower than a general page cache, and sufficient. An anonymous shared object +needs no file behind it, which is exactly why `memfd` exists on Linux at all — +and it means this can be built on the shared-memory machinery already in +`kernel/sys/sys_mem.c` (`shmget`/`shmat`/`shmdt` work, and `struct vma` already +carries `shmid` and a `no_free` flag for pages that are not ours to release). + +The general answer — a page cache, and writable `MAP_SHARED` of a real file — +stays owed. It is a larger project and nothing on this plan needs it. + +### 3. `AF_UNIX` sockets in the filesystem namespace + +Clients find a Wayland compositor by `bind()`ing `$XDG_RUNTIME_DIR/wayland-0` as +a real path, and libwayland `flock()`s the lock file beside it. A name table can +express neither. + +`kernel/net/unix.c`'s own header comment already predicted this: + +> *"What is lost is `ls -l /tmp/schcom` showing an `s`, and `unlink()` of a socket +> path failing with ENOENT. Both are worth fixing when the VFS next gets opened +> up; neither blocks anything."* + +They block something now. This is that moment, and it is a VFS change rather than +a socket change: a socket inode type threaded through `iget`, `dirlookup` and +`stat`, plus `flock`. + +### 4. `setsockopt` that does not lie + +At minimum `SO_PASSCRED` and `SO_SNDBUF`. Returning 0 for an option you ignored +is worse than `ENOPROTOOPT`, because the caller adjusts its behaviour on the +strength of the lie. + +Small, and grouped here because it is in the same file as item 3 and will be +found by the same testing. + +### 5. Dynamic linking + +`PT_INTERP` and `ET_DYN` in `kernel/fs/exec.c`. + +**Deferrable, and deferred.** Static binaries are enough to reach a working +desktop with software rendering. This becomes necessary when Mesa does — EGL +means `dlopen` means a loader — and that is a separate decision made later. + +--- + +## Verification + +Each item gets a test before the thing that needs it is built, because a +substrate that is subtly wrong fails inside the consumer and looks like the +consumer's bug. + +- **Item 1:** two processes, one passes an open descriptor, the other reads + through it. Plus the leak case: a sender that exits with descriptors in flight. +- **Item 2:** two processes map the same `memfd`, one writes, the other sees it. +- **Item 3:** `bind()` a socket, then `stat()` and `ls -l` the path and see an + `s`, then `unlink()` it. +- **Items 1–3 together:** a third test that does all of it at once, because that + combination is what Wayland actually performs and the three passing separately + does not prove it. + +These belong in `bootcheck` as a new switch, following the `-Sock` model. + +--- + +## What this does not deliver + +- **No page cache**, so writable `MAP_SHARED` of a real file stays refused. +- **No `dlopen`** until item 5 is done, which means no Mesa and no third-party + shared libraries. +- **`SO_*` coverage is the two options named**, not the full set. +- **No credential passing** (`SCM_CREDENTIALS`) — only descriptors. Nothing on + this plan needs it, and a compositor that wants to know who connected will. diff --git a/docs/plan/hybrid.md b/docs/plan/hybrid.md new file mode 100644 index 0000000..33c474b --- /dev/null +++ b/docs/plan/hybrid.md @@ -0,0 +1,165 @@ +# The kernel becomes hybrid + +"Hybrid" is used to mean everything from "a monolith with a nice device model" to +"a microkernel that gave up". **Drawing the line precisely is most of the work**, +so that is what this document is. + +The device model it builds on is +[../architecture/devices.md](../architecture/devices.md). It is last on the +plan, and the reason is in [README.md](README.md#the-order-and-why-it-is-not-the-order-they-are-listed-in). + +--- + +## The line + +### Stays in the kernel, permanently + +- scheduling, address spaces, the trap and syscall path +- IPC and shared memory — the mechanism everything else is built out of +- the VFS and the **root** filesystem +- IRQ demultiplexing +- the console and PSOGM + +**That last one is not a size compromise, it is an invariant.** The crash screen +must work when everything else in the system is dead, which means it cannot +depend on anything that could *be* dead. A crash screen served by a userspace +process is a crash screen that is absent in exactly the case it exists for. + +### May leave, and should + +- GPU drivers +- USB, and every USB class driver under it +- non-root filesystems +- NICs after the first +- input transformation — layouts, gestures, remapping + +### The rule that separates the columns + +> **Anything needed to reach a shell stays in. Anything discovered at runtime, +> replaceable, or plausibly written by someone else goes out.** + +That is NT's compromise and macOS's, arrived at from the same pressure, and it is +the one that survives contact with a real machine. + +--- + +## The mechanisms + +Four, and only the fourth is large. + +### 1. A control channel + +`AF_UNIX` plus `SCM_RIGHTS` — item 1 of [foundations.md](foundations.md), which +the Wayland work builds first. A descriptor is how a driver process is handed the +device it is allowed to touch. + +**Do not build Mach ports.** The port model is worth its cost when there is a +naming and capability story to hang on it; there is not one yet, and inventing it +before the first userspace driver exists is designing against an imaginary user. + +### 2. MMIO to userspace + +Map a PCI BAR into a process. + +`sys_mmap` already maps the framebuffer's real physical range at its real +address — the path exists. This is that path generalised, behind a check that the +caller was granted that device. + +### 3. Interrupts to userspace + +**The handler stays in the kernel and does one thing: write to an `eventfd`.** +The driver process reads. + +`eventfd` exists already, and that is why it is the right primitive. A signal +would be delivered to a process that cannot safely be interrupted mid-transaction. + +### 4. DMA to userspace + +Physically contiguous memory whose physical address the process is told. + +`kernel/core/dma.c` already owns an arena reserved for exactly this, before +`kalloc` gets anything. It needs a door, a per-device budget, and a way to hand +the region back. + +--- + +## The honest limitation + +> **There is no IOMMU, so a userspace driver is isolated against its own bugs and +> not against malice.** + +A process that can program a device's DMA engine can write to any physical +address, whatever its page tables say. + +That is not a reason to stop — isolation against bugs is most of the value, and a +crashed driver that restarts instead of panicking the machine is the entire +point. But it must be written down, because *"userspace driver"* reads as +*"sandboxed"* and here it does not mean that yet. VT-d/AMD-Vi is its own project +and comes after. + +--- + +## Order, and one thing not to do + +### Do not build loadable kernel modules first + +It is the obvious first step and it is a generation behind. It means designing a +**stable in-kernel driver ABI**, which is the most expensive promise in this +entire plan, for a mechanism Apple is actively retiring in favour of the one +above. + +`../architecture/devices.md` already separates the two ideas. They are separable, +and the second one is the one to build. + +### First eviction: the GPU driver + +Four reasons it is the right one and not just the interesting one: + +- **The boundary already exists and is already correct** — `struct display_ops`. +- **It has exactly one consumer**, the compositor, so the interface has a real + user to be judged by. +- **There is a working fallback underneath.** The VBE framebuffer stays + in-kernel, so a GPU driver that fails to start degrades to a working screen + rather than to no screen. +- It directly serves the work that will actually be happening. + +### Second: USB + +`xhci.c` is 1364 lines, self-contained, already spoken to through a bus +abstraction, and it is where third-party drivers will show up first. This is what +DriverKit was built for and the resemblance is not accidental. + +### The rule that keeps the boot path honest + +> **The kernel keeps a minimal driver for everything on the path to a shell, and +> userspace owns the good one.** + +Break this and you get the circular boot failure where the compositor cannot +start because the display driver is a process that needs a compositor. + +--- + +## Milestones + +| | | +|---|---| +| **H0** | Mechanisms 2, 3 and 4, each with a test that exercises it from a userspace program against a device nothing else is using. | +| **H1** | A userspace virtio-gpu driver, publishing `display_ops` across the channel. Schcom talks to it and does not know the difference. | +| **H2** | Kill the driver process on purpose. The screen falls back to VBE, the driver restarts, the compositor reattaches. **This is the milestone that proves the whole idea**, and it is the one that cannot be skipped. | +| **H3** | A modeset API, which is the thing an out-of-kernel GPU driver makes affordable and an in-kernel one never got. | +| **H4** | USB, same shape. | + +--- + +## What it will still not do + +- **No isolation against a hostile driver**, per the IOMMU note above. +- **No stable driver ABI**, deliberately. Out-of-tree drivers remain a + recompile-with-the-tree affair until there is a reason to promise otherwise. +- **The root filesystem stays in the kernel**, so SchFS cannot be restarted and a + bug in it is still fatal. Moving it out needs an initramfs story that does not + exist. +- **The first NIC stays in the kernel**, because the network thread and lwIP are + in the kernel and moving them is a separate project with no current benefit. +- **No driver sandboxing beyond address-space separation** — no seccomp-shaped + syscall restriction, no resource limits per driver. diff --git a/docs/plan/init.md b/docs/plan/init.md new file mode 100644 index 0000000..af546d1 --- /dev/null +++ b/docs/plan/init.md @@ -0,0 +1,178 @@ +# PID 1 becomes a role + +What `schinit` is today is [../architecture/init.md](../architecture/init.md). +This is what it becomes, and it is the one item on this plan that deliberately +breaks a Unix rule. + +--- + +## The law being changed + +Unix's rule is *PID 1 is special and must never die*. Every init since 1975 has +honoured it by being careful. + +> **SchOS's rule instead: PID 1 is a role. The kernel guarantees the role is +> filled. The role's state lives outside whichever process currently holds it.** + +That is a real change and it is worth making, because "must never die" is not a +mechanism — it is a hope, and the thing it hopes about is the one program on the +machine with no supervisor. + +--- + +## The proposal this replaces + +The original sketch was: init 1 starts init 2; if init 1 is killed, init 2 starts +init 3 and revives init 1. + +The instinct is right — init needs something watching it — and one half of the +idea survives below. The specific arrangement has five problems, and four of them +are structural rather than fixable. They are recorded because each one is a trap +that any redundant-supervisor design falls into: + +**It defends against the attack that already fails.** Userspace cannot kill PID 1 +on this kernel. `init_ignores()` (`kernel/proc/proc.c:620`) drops `SIGKILL` to +`initproc`. The threat is already handled. + +**Init 2 is the same binary as init 1.** If init 1 died of its own bug — a null +dereference parsing a manifest, an exec of a truncated binary — init 2 reaches +the same line and dies the same way, and *faster*, because it just watched the +input that caused it. + +**The chain has no bound.** Init 2 makes init 3, which by symmetry watches init 2 +and makes init 4. Without a cap it is a fork bomb whose author is the supervisor; +with a cap, the cap is the real design and the chain is decoration. + +**Two supervisors is split brain.** Both hold the job table. Both see a job's pid +exit. Both restart it. Now the console shell exists twice, both reading +`/dev/console`, and every keystroke goes to whichever woke first. This is the +standard failure of redundant supervisors, and it is worse than no redundancy +because it corrupts the state it was added to protect. + +**It only detects death.** The common way an init breaks is not dying — it is +**hanging**: deadlocked, spinning, or blocked forever on a disk that stopped +answering. `wait()` never returns for a process that is merely stuck, so init 2 +watches a wedged init 1 forever and reports everything normal. The scheme is +armed against the rare failure and blind to the likely one. + +--- + +## What gets built instead + +Four parts. The first three are userspace; the fourth is the kernel change. + +### Part 1 — split by role, not by copy + +**`schinit` (PID 1)** becomes small enough to be obviously correct and then stops +changing: reap children, hold the job table in memory, supervise exactly one +child. No manifest parsing. No filesystem walking after boot. No allocation. Its +entire job is to be too boring to fail. + +**`schinitd` (PID 2)** gets everything else — manifests, socket activation, +respawn policy, backoff, and the eventual `schinitctl`. It is where the bugs will +be, and it is restartable. + +The detail that makes this work, and that the two-copy scheme cannot have: + +> **Jobs are forked by PID 1, not by `schinitd`.** `schinitd` decides and asks; +> PID 1 acts. So every job on the machine is a child of PID 1 at all times, and +> `schinitd` crashing orphans nothing, reparents nothing and restarts nothing. It +> comes back, reads the state, and finds the world exactly as it left it. + +This keeps the good half of the original idea — two processes, one watching the +other — with the roles made *different*, so the second is not a second copy of +the first one's bug. + +### Part 2 — state outside the process + +**`/proc/schinit`**: the serialised job table, written by PID 1 whenever it +changes, readable by anyone. A resurrected supervisor reads it back and knows +what is running without having started it. + +procfs rather than a file on disk, for four reasons: it survives a read-only +root, it survives a full disk, it costs no I/O, and it is already the pattern +this kernel uses for exactly this kind of question. + +The side benefit closes an item the architecture doc lists as missing — the job +table is currently invisible. `cat /proc/schinit` is the observation. + +### Part 3 — a heartbeat, in both directions + +The failure the original scheme cannot see. + +PID 1 and `schinitd` hold a `socketpair`; each must be heard from within a +deadline. Silence from `schinitd` means PID 1 kills and restarts it. Silence from +PID 1 is the case part 4 exists for. + +**A hang is now as detectable as a death**, which is the whole point. + +### Part 4 — `proc_reincarnate()` + +The kernel change, and the actual law change. + +Today `exit()` on `initproc` calls `panic("init exiting")` +(`kernel/proc/proc.c:277`). Instead: tear down the address space, keep the +`struct proc`, keep **pid 1**, and re-exec `/sbin/schinit` in place with a +generation counter passed in. + +No init 2, no init 3, no reparenting problem — **because pid 1 never stops being +pid 1**. Every orphan the kernel already reparented to `initproc` is still +correctly parented, `initproc` is still the pointer it always was, the children +survive, and the state is in `/proc/schinit`. + +This is strictly better than resurrection-by-sibling for every case, and it is +the smaller change. + +#### The escalation ladder + +What makes it safe to have at all: + +| Generation | What happens | +|---|---| +| 1 | Normal boot. | +| 2–3 | Re-exec in place. Log it, keep the jobs, carry on. | +| 4 | Re-exec, but ignore `/etc/schinit` and start only a console shell. **A manifest that kills init must not kill it forever.** | +| 5 | Stop. `panic()` with stop code `INIT_RESURRECTION_FAILED`. | + +The last row is the honest one. A machine that cannot hold PID 1 up five times is +not a machine that should keep trying, and PSOGM already renders a stop code, a +call chain and a QR code of the whole report — which is a far better end than a +silent loop. The crash screen is the correct place for this to arrive. + +--- + +## Milestones + +| | | +|---|---| +| **I0** | Split the program. PID 1 supervises one child; `schinitd` does everything else. Jobs still forked by PID 1. | +| **I1** | `/proc/schinit`, written and read back. Kill `schinitd` by hand and watch it return without disturbing a single job. | +| **I2** | The heartbeat both ways, with a `sysrq` key that wedges `schinitd` on purpose so the timeout path is exercised rather than assumed. | +| **I3** | `proc_reincarnate()` and the ladder, with a `sysrq` key that kills PID 1 on purpose. | + +--- + +## What it costs + +**PID 1 and PID 2 must agree on a protocol**, which is a compatibility surface +where there was none — created by a plan whose own [schapi.md](schapi.md) §3 +warns about exactly that. Keep it tiny, version it in its first message, and keep +it entirely internal. It is not SchAPI and nothing else may speak it. + +**`proc_reincarnate()` is a genuinely dangerous function.** It runs in the +context of a process that is dying, and it must not take a lock that process may +already hold. It belongs in `proc.c` next to `exit()`, written with the care of +the panic path, and tested by a `sysrq` key rather than by reasoning. + +--- + +## What it will still not do + +- **It does not make init correct**, only survivable. A `schinitd` that starts + the wrong jobs starts them again after every restart. +- **It does not protect the jobs from each other.** A job that fills the disk or + exhausts the process table still takes the machine down, and the supervisor + surviving does not help. +- **A hang in PID 1 itself is only caught by `schinitd` noticing**, and if both + are wedged simultaneously nothing catches it. There is no hardware watchdog. +- **Five generations is a guess**, not a measured number. diff --git a/docs/plan/roadmap.md b/docs/plan/roadmap.md new file mode 100644 index 0000000..39f5e2d --- /dev/null +++ b/docs/plan/roadmap.md @@ -0,0 +1,179 @@ +# Roadmap — everything else still open + +The three current decisions have their own documents; see +[README.md](README.md). This file holds what is left: work that is scoped, +agreed and unstarted, with nothing in it that is finished. + +**Finished work is not here.** It moved to +[../architecture/](../architecture/) and was rewritten as a description of what +it produced — see [the rule](../README.md#the-rule-that-keeps-this-from-rotting). +What used to be phases 1 through 4 of this file, several hundred lines of ticked +boxes, is now `usb`, `networking.md`, `graphics.md` and `devices.md` over there. + +--- + +## 1. Storage and the boot path + +**Make a USB disk mountable and bootable.** The xHCI stack, the bulk-only +transport, the SCSI layer and the block device all work — a superblock reads back +through `bread()`. What is missing is that **the root device is compiled in +rather than selectable**, so nothing can be told to mount it. + +This is also the first piece of the installer, and it is the single change that +would most enlarge what SchOS can boot from. + +## 2. Graphics + +- **A `/dev/dri`-style device with mode setting**, so resolution stops being + whatever the loader asked for. Worth doing *with* + [hybrid.md](hybrid.md)'s GPU eviction rather than before it — moving the driver + out is what makes a modeset API affordable to design properly. +- **Double buffering and vblank.** `display_ops` specifies both and no driver + implements either. Schcom needs both to not tear. +- **Port the console onto the text layer and delete the cell grid.** The layer + exists, the rasteriser exists, and the console still indexes `font_data[256][16]`. +- **Load a font from disk** and point the console at it. +- **CFF/Type 2 charstrings**, so OTF fonts work. A separate `font_ops` provider + and a bytecode interpreter, *not* an extension of the `glyf` path. +- **Text layout** — advance widths, kerning — as a library in `lib/`, so the + terminal and the file manager are not each solving it. + +The console keeps its fixed grid until the first item above; proportional text +belongs to the GUI. + +## 3. Networking + +Three things the stack owed on the day it landed, all still owed: + +- **`accept()` does not report the peer's address**, and there is no + `getsockname`/`getpeername`. Nothing has needed them yet; a server that wants + to log who connected will. +- **No routing table.** With two interfaces up, the first is the default route + and the second is reachable only by an address on its own subnet. That is a + rule rather than an accident, and it is not routing. +- **Receive is polled.** The network thread looks at the NIC every timer tick + when the link is idle, so an arriving frame waits up to 10 ms. Waking that + thread from the e1000 interrupt is the real fix, and was deliberately not the + first version: a receive path woken by an interrupt is much harder to debug + than one that is asked. + +`setsockopt` returning 0 for everything is in +[foundations.md](foundations.md) instead, because the Wayland work needs it. + +## 4. The interactive crash screen + +PSOGM renders and stops. The idea is *a small OS inside the crash*, and the +framing decides the design: + +> At panic time the kernel is the only thing running, which is exactly why it can +> offer what a live system cannot: walk the stack, read another process's memory, +> dump page tables, inspect the process table — with no scheduler and no locking +> to respect, because nothing else can race you. + +What makes it a *system* rather than a print statement is that it keeps the base +machine's drivers. Concretely, the ones that work with interrupts off: the +framebuffer (`gfx_draw_pixel` is a plain memory write), the PS/2 keyboard +(pollable via ports 0x60/0x64), and the serial port. **Those three are a complete +console.** + +Everything else — IDE, USB, the timer, anything that sleeps or takes a +sleeplock — is off limits, because the lock it needs may be precisely the one +held by the CPU that just died. + +So the shape is a **polled event loop over preallocated state**: no `kalloc`, no +`sleep`/`wakeup`, no `acquire` on anything a live path might hold, one static +command buffer, and that driver set. **That restriction is not a limitation to +work around later; it is the invariant that makes the mode trustworthy when +everything else is not.** Anything wanting more should be answered by rebooting +into a recovery environment. + +A useful precedent to read but not copy: DDB, the BSD in-kernel debugger, is +exactly this, and its hard-won rule is the same one — *a debugger that can +deadlock is worse than no debugger.* + +## 5. The installer and `.sip` + +Scoped in [`projects/schinst/README.md`](../../projects/schinst/README.md), +which holds the detail. In short: + +- [ ] `.sip` format specification, **written down before any code** +- [ ] `tools/mksip` — capture a directory tree into a package +- [ ] The apply path +- [ ] An installation environment that boots from the ISO, partitions, applies + the `.sip`, and installs the loader + +`.sip` is modelled on WIM, and the thing worth copying from WIM is that it is +**file-based, not block-based**: one file holds a whole filesystem, applies to a +partition of any size, and several images can share their common files. + +**Most of this already exists.** `tools/mkschfs.c` and `tools/mkfs.c` both do +content dedup with an `(st_dev, st_ino)` fast path, and SchFS's B-tree is content +addressed. `mksip` should share that code rather than reimplement it — that is +the reason the dedup was factored the way it was. + +Open questions, all wanting a decision before code: + +- **Compression.** Something with a small decompressor is worth more than + something with a good ratio, because the decompressor runs in the installation + environment. +- **Multiple images per `.sip`** — WIM's real trick, install/recovery/boot + sharing one blob store. Worth the complexity now, or not? +- **Can the kernel mount a `.sip` read-only directly?** It is close to a + filesystem already, and if it can, the installation environment's root can + simply *be* the `.sip`. + +The GUI for this is a Wayland client; see [wayland.md](wayland.md). + +## 6. SchFS beyond SchOS + +In the order they should happen. + +**~~An on-disk format version~~, and an upgrade path.** The version policy is +done — see [../architecture/schfs.md](../architecture/schfs.md). What remains is +the **upgrade step**, and it cannot be written until there are two formats to +convert between; the first bump is when that becomes real work rather than +speculation. + +The format is now pinned, which is what the other two items were waiting on. + +**A Linux implementation, FUSE first.** Three reasons in order of weight: it can +be tested with `fsx` and `xfstests`, which are far harsher than anything written +alongside the code; it carries no licence entanglement, since libfuse is LGPL and +used across a process boundary; and it can reuse the read/write logic already in +`tools/mkschfs` and `tools/schfs_fsck` rather than being written twice and having +the two disagree. + +A kernel module is a different project, comes only after FUSE passes `xfstests`, +and carries the licence problem [../licensing.md](../licensing.md) describes. + +**The missing features**, listed honestly before adding any: no journal replay +across an unclean shutdown beyond what copy-on-write gives for free, no +snapshots, no sparse-file support, and no online `fsck`. **Snapshots are the one +where the existing design already pays for most of the cost** — the B-tree is +copy-on-write, which is most of the machinery; nothing takes or names a snapshot. + +## 7. The kernel's own gaps + +Not phases, and each one improves everything above it: + +- **A slab or pool allocator.** `kalloc` hands out one page at a time, and every + subsystem written since has invented its own fixed-size table to work around + that. This is the single change that would most improve the rest of the kernel. +- **A page cache**, which is what a general writable `MAP_SHARED` needs. + [foundations.md](foundations.md) delivers a `memfd` instead, which is narrower + and sufficient for the GUI. +- **Symlinks in procfs**, so `/proc//exe` can be one and `ps -o comm` stops + printing `[init]`. Belongs with the rest of the VFS work rather than on its own. +- **The `ls -lR` hang.** Heavy directory listing across many commands in one long + shell session eventually makes a *later*, unrelated directory hang; the same + command succeeds on a fresh boot. Not inode or file table exhaustion — both + fail cleanly and neither fired. Next step: instrument `itable`/`ftable` + occupancy across a long session and watch what climbs. + +--- + +## Kept out of scope on purpose + +- Anything that needs pushing or publishing. This work stays local. +- TrueType hinting. +- XWayland. diff --git a/docs/architecture/schapi.md b/docs/plan/schapi.md similarity index 99% rename from docs/architecture/schapi.md rename to docs/plan/schapi.md index 6fc13c4..7ebb3ef 100644 --- a/docs/architecture/schapi.md +++ b/docs/plan/schapi.md @@ -76,7 +76,7 @@ Rules that follow, and they are not negotiable if the promise is to hold: `libschos` still presents the same surface. This *reconciles* with the existing decision to adopt Linux syscall numbering -(`docs/roadmap.md`, and the reason musl/toybox/mksh work at all). Linux +(`roadmap.md`, and the reason musl/toybox/mksh work at all). Linux numbering does not go away — it becomes the **Linux personality's** entry path, which is exactly what it already is in practice. It is a compatibility surface that happens to be entered via `syscall`, and it is frozen because *Linux* diff --git a/docs/plan/wayland.md b/docs/plan/wayland.md new file mode 100644 index 0000000..f6d0460 --- /dev/null +++ b/docs/plan/wayland.md @@ -0,0 +1,167 @@ +# The GUI becomes Wayland + +Schcom survives and changes its wire format. SchWM and SchDE are deleted. What +follows is why that is not a retreat, and what it costs. + +Blocked on [foundations.md](foundations.md), items 1–4. + +--- + +## What is deleted + +`projects/schwm/` and `projects/schde/`. Both are README-only; no code is lost. + +### SchWM is not abandoned — Wayland dissolves it + +In X11 the window manager is a separate privileged client because X11 puts +compositing in the server and policy nowhere. **Wayland's central decision is +that the compositor *is* the window manager**: there is no protocol by which a +second process places another's windows, and every attempt to add one has been a +private extension. + +So the argument in `projects/schwm/README.md` — + +> *"Compositing is mechanical and must never stall: it runs every frame and a +> mistake there is a frozen screen. Window management is policy, full of special +> cases and personal taste, and it changes constantly."* + +— is a real and correct observation **about X11**, which Wayland answers a +different way. It stays worth honouring, but as a **module boundary inside +Schcom**, not a process boundary: policy in its own file behind its own +interface. Do not keep it in its own process, because the protocol to talk to it +does not exist. + +### SchDE is re-sited rather than deleted + +A panel, a launcher and a wallpaper are ordinary Wayland clients using +`wlr-layer-shell` — MIT, an XML file, no dependency. They need no directory of +their own until one of them is written. + +### `projects/schinst/` stays + +Its GUI was the part tied to SchWM, and that part becomes a Wayland client. +Everything else in it — the `.sip` format, `mksip`, the apply path, the +installation environment — is untouched by any of this, and remains the +best-specified unstarted work in the tree. + +--- + +## What survives + +**Schcom stays, and keeps its name.** It already owns the display, allocates and +owns client buffers, implements surface/attach/commit, routes input from +`/dev/input/eventN`, composites multiple clients bottom-to-top, and cleans up +after clients that die. + +Its own README says the protocol is deliberately Wayland-shaped — surfaces rather +than windows, buffers attached and made visible only on commit, the compositor +telling clients when to draw. That was the right call and this is where it is +cashed in: **what changes is the wire format and the buffer handle, not the +architecture.** + +--- + +## What has to exist first + +Items 1–4 of [foundations.md](foundations.md), and the first three are absolute — +libwayland does not start without them. + +The sharpest is descriptor passing. `wl_shm_create_pool` passes a **file +descriptor**, and there is no path in the protocol that passes an integer +instead. `struct schcom_surface`'s `handle_kind` field was written precisely so +that switching would be a new value rather than a protocol break, and it cannot +help: *our* protocol is the flexible one. + +--- + +## Port libwayland. Do not port Weston. + +**libwayland is the small piece**: a wire protocol, an object map, and an event +loop, needing `libffi` and — for the build-time scanner, on the host only — +`libexpat`. A real port, but a bounded one. + +**Weston is the trap.** It arrives welded to `libinput`, `udev`, `libdrm`, +`logind`-shaped seat management and a GL renderer, and porting it means porting +Linux's device management first and finding the compositor somewhere underneath. + +This is precisely the lesson already recorded twice in this tree — about +FreeBSD's `sys/dev/usb` (100k lines welded to newbus) and about FreeBSD's +`sys/netinet` (not a library; reaches into mbufs, epoch, netisr, VNET and uma). + +> The tree has learned this twice. It should not pay to learn it a third time. + +So **Schcom links `libwayland-server` and remains ours.** Input comes from +`/dev/input/eventN`, which already works, rather than from libinput. Output goes +through `display_ops`, which already exists. + +### Software rendering first, and probably for a long time + +`pixman` for composition, `wl_shm` for buffers. Mesa means EGL means `dlopen` +means a dynamic loader, and that is a large kernel project that does not have to +be on this path. Static binaries are enough to reach a working desktop; keep it +that way until something concrete demands otherwise. + +### One sharp edge + +`wl_keyboard` hands the client **a keymap as an XKB text blob over a file +descriptor**. A compositor cannot avoid xkbcommon by having simpler opinions +about keyboards — the protocol requires the blob. + +The cheap route is one precompiled keymap string shipped in the binary, with +xkbcommon and `xkeyboard-config` deferred until more than one layout is wanted. + +--- + +## Licensing + +Checked before anything is fetched — the standing rule is that the licence goes +in `NOTICE` **before** the file is committed, not after. + +| | | +|---|---| +| wayland, wayland-protocols | MIT | +| `wlr-layer-shell` | MIT | +| pixman | MIT | +| libffi | MIT | +| libxkbcommon | MIT | +| xkeyboard-config | MIT | + +All Apache-2.0 compatible, all needing only a `NOTICE` entry. **Nothing here +carries the GPL problem that governs drivers**, which is a pleasant change and +worth noticing: the Wayland ecosystem is permissively licensed almost throughout, +and that is a genuine argument for it beyond any technical one. + +--- + +## Milestones + +| | | +|---|---| +| **W0** | [foundations.md](foundations.md) items 1–4, with the combined test that passes a descriptor *and* shares a `memfd` *and* binds a socket at a real path. | +| **W1** | libwayland builds and installs into the sysroot. `wayland-scanner` runs on the host. | +| **W2** | Schcom serves `wl_compositor`, `wl_shm`, `wl_surface`, `xdg_shell`. A client draws one window. | +| **W3** | `wl_seat`: keyboard and pointer, from the evdev path that exists. | +| **W4** | Two clients, focus, stacking, move and resize. **This is where SchWM's policy lives, as a file.** | +| **W5** | `wlr-layer-shell`, and a panel. This is SchDE's first pixel. | + +Every milestone from W2 on is asserted **from the host, by pixel**, the way +`schcom-test` already does. The compositor's own report that it composited +something proves nothing. + +--- + +## What it will still not do + +- **No hardware acceleration.** Software rendering only, until dynamic linking + and Mesa, neither of which is on this plan. +- **No XWayland**, so no X11 clients, ever, unless someone ports it. +- **No mode setting.** Resolution stays whatever the loader asked for — see + [hybrid.md](hybrid.md), where the GPU driver's eviction is also where a modeset + API would arrive. +- **No damage tracking beyond "repaint everything"**, no multiple outputs, no + subsurfaces. Schcom's README already lists these; they are additions rather + than rewrites, which is the reason for committing to the surface/commit model + now. +- **One keyboard layout**, until xkbcommon is ported. +- **No clipboard, no drag and drop, no screen capture protocol**, none of which + W0–W5 touches. diff --git a/docs/roadmap.md b/docs/roadmap.md deleted file mode 100644 index 16ae784..0000000 --- a/docs/roadmap.md +++ /dev/null @@ -1,534 +0,0 @@ -# Roadmap - -Written 2026-08-04, from the plan: USB, GPU, fonts, networking and sockets, -then a compositor (Schcom), a window manager (SchWM) and a desktop (SchDE), -plus an installer and a `.sip` package format. - -This file records two things that are easy to lose: the *order* the work has -to happen in, and the decisions that constrain it. - -**Superseded in direction, not in order.** SchOS is now aimed at being a -*system* rather than a Unix-like kernel with a userland attached: a NeXTSTEP/ -macOS-shaped layering, with POSIX kept complete but demoted from identity to -personality, and a native API (**SchAPI**) above it. See -[architecture/schapi.md](architecture/schapi.md). That decision changes what -the phases below are building towards; it does not reorder them, and it does -not start until the tree builds and boots natively again. - -That assumption about §1 has since been revised. NetBSD's TCP/IP stack was the -intended starting point; the per-file licence check found that NetBSD's -`tcp_input.c` and `tcp_output.c` carry the WIDE Project's 4-clause advertising -clause, which Berkeley's rescission does not reach. FreeBSD's equivalent is -clean, and lwIP is cleaner still and a far smaller port. See Phase 2 below and -docs/licensing.md §3. - ---- - -## 1. Where drivers can come from - -**BSD: yes. Linux: no.** This is not a preference, it is a licence -incompatibility, and it decides the whole driver strategy. - -The reasoning — Apache-2.0's patent terms counting as "further restrictions" -under GPLv2 §6, which is why the two cannot be combined and redistributed — is -in [licensing.md](licensing.md), along with the per-licence table and the -NetBSD-specific rules. Read it before copying anything. The short version: -BSD-2 and BSD-3 code can be used directly with a `NOTICE` entry; GPL-2.0-only -code can never be, at any distance. - -There is a third source that is better than either: **the specifications**. -xHCI, AHCI, virtio, NVMe, the Intel e1000 and i915 datasheets are all public -documents with no licence attached to what you write from them. That is where -`kernel/drivers/xhci.c` came from. - -One caveat about "port it from BSD" as a plan, learned by looking: FreeBSD's -`sys/dev/usb` is roughly 100k lines welded to newbus, devclass, `bus_dma`, -callouts, taskqueues and FreeBSD's locking. Porting it means porting -FreeBSD's device framework first and finding the USB stack somewhere -underneath. NetBSD's is more portable and the same argument applies with a -smaller multiplier. **BSD is far more valuable as the reference you check -yourself against than as the code you paste.** Read it when the spec is -ambiguous — which it is, often — and write your own. - -Reading Linux for *understanding* is fine and normal. Translating a Linux -driver function-by-function is not; the result is a derivative work whatever -the variable names say. - ---- - -## 2. Ordering, and why it is not the order it was asked in - -The requested order was USB → GPU → fonts → network → GUI → installer. That -is a reasonable order by *interest*. By dependency it wants one change: - -**Shared driver infrastructure comes before all of it.** USB, any GPU beyond -the VBE framebuffer, and any NIC all need the same three things, none of -which existed: - -- a way to register an interrupt handler at runtime (`kernel/dev/irq.c`) -- physically contiguous memory for rings and descriptors - (`kernel/core/dma.c`) -- PCI capability walking and MSI (`kernel/dev/pci.c`) - -That work is done. It was the first step of USB, not a detour from it. - -The other honest note: **USB is the item that does least for the GUI.** Under -QEMU there is already a PS/2 keyboard and mouse, so USB buys real hardware -support — which matters, but not for getting Schcom on screen. Sockets and -raw input events are what actually block the compositor. USB is being done -first because it was asked for first; the ordering below reflects that with -the dependencies fixed. - ---- - -## 3. The plan - -### Phase 1 — USB *(done, bar the boot path)* - -- [x] Shared infrastructure: IRQ registration, DMA arena, PCI capabilities -- [x] xHCI: reset, DCBAA, command ring, event ring, scratchpad -- [x] Port enumeration, Enable Slot, Address Device, control transfers -- [x] Device descriptors read from real devices (USB2 and USB3 paths) -- [x] Configuration and interface descriptor parsing, `SET_CONFIGURATION` -- [x] Interrupt endpoints, driven from the IRQ handler -- [x] HID boot protocol: keyboard and mouse into the existing input paths -- [x] Bulk endpoints, bulk-only transport, SCSI, a block device at - `USBDISK_DEV` — a superblock reads back through `bread()` -- [x] Hubs — route strings, hub slot contexts, per-port reset and power. A - mouse behind a hub enumerates a tier deep, which is what proves it -- [ ] Making a USB disk mountable and bootable (needs the root device to be - selectable rather than compiled in) - -### Phase 2 — Sockets, raw input, and networking - -Do sockets **before** the network stack. **Unix domain sockets are what Schcom -needs** — clients and compositor have to pass messages and share buffers, and -a pipe cannot carry an shm handle. Unix sockets are also far simpler than -TCP/IP, so they are the right thing to build the socket layer's shape around. - -- [x] `socket`/`bind`/`listen`/`accept`/`connect`/`send`/`recv` syscalls, at - their real Linux numbers — they were `NO_LINUX` stubs, so a musl program - calling `socket()` never even reached the stub -- [x] `AF_UNIX` SOCK_STREAM, `socketpair`, `shutdown`; names live in a table - rather than the filesystem (see the comment at the top of - `kernel/net/unix.c` for what that costs) -- [x] Raw input events — `/dev/input/event0` and `event1`, Linux evdev layout, - fed by the PS/2 keyboard and mouse and by USB HID. The console keeps its - own path; both consumers see every key -- [ ] `SCM_RIGHTS` for passing descriptors. Possibly unnecessary: an shm id is - an integer and travels fine over a stream socket, so this waits until - something actually needs to pass an open file -- [x] e1000 driver — `kernel/drivers/net/e1000/e1000.c`, from Intel's public manual - 317453. MAC from EEPROM, transmit and receive rings, link detection. - Run with `-netdev user,id=n0 -device e1000,netdev=n0`. - **Verified to carry real traffic**: `echo n > /proc/sysrq` sends a - hand-built ARP request and receives the gateway's reply, which exercises - the descriptor formats, the DD handshake, the tail bookkeeping, the DMA - addresses and the MAC filter in one exchange. That check came before any - protocol stack on purpose -- a stack on an unproven driver fails in ways - indistinguishable from its own bugs -- [x] ARP, IPv4, ICMP, UDP, then TCP -- **done 2026-08-15, lwIP 2.2.0.** - Fetched by `ports/lwip` and compiled into the kernel; the port layer is - `kernel/net/lwip/` (about 400 lines: a lock, a clock, and the e1000 - netif). NO_SYS=1, so lwIP has no threads of its own and one kernel thread - (`kernel/net/net.c`, started by `netinit`) polls the NIC and runs its - timers. DHCP obtains an address at boot. - DHCP obtains an address at boot and `/proc/netif` reports it, along with - the MAC, the gateway and frame counts taken at the lwIP seam. - **Verified end to end**, by `bootcheck -Net` rather than by hand: the - guest opens a TCP connection out, the listener on the host receives - `HELLO-FROM-SCHOS`, and the guest receives the host's reply -- so the - handshake, both data directions and the close all work, not just the - driver. `kthread_create` was added for this and is generally useful. - The reasoning behind not writing it, kept because it is the reason to - resist doing so later as well: it is not the packet format that is hard, - it is congestion control, retransmission, window management and twenty - years of accumulated corner cases that only appear against a real peer. - lwIP was preferred over a BSD stack on structure rather than licence -- - FreeBSD's `sys/netinet` is BSD-3 too, but it is not a library: it reaches - into mbufs, epoch(9), netisr, VNET and uma(9), so taking it means porting - FreeBSD's kernel infrastructure first. NetBSD was ruled out on licence: - its `tcp_input.c` carries the WIDE Project's 4-clause advertising clause. - See docs/licensing.md §3. -- [x] `AF_INET` on top of the same socket layer -- done, exactly as planned: - `socknew` dispatches on the domain, `kernel/net/inet.c` implements the - AF_INET side on lwIP's raw API, and no syscall changed shape. TCP and UDP - both work. -- [x] The two things AF_INET owed on the day it landed, both closed 2026-08-15. - `recvfrom` reports the source address -- which meant giving UDP a - datagram queue first, since the payloads shared TCP's byte ring and two - datagrams arriving together came back as one -- and `sendto` takes a - destination. Blocking sockets sleep on a counter bumped by the lwIP - callbacks instead of waking on every timer tick; `tcp_sent` had to be - registered for it, or a write blocked on a full send buffer would sleep - with nobody left to wake it. -- [x] `bootcheck -Net`, so none of the above is verified by hand any more. A - listener and a UDP responder run on the host, the guest is told to talk - to them, and both ends are asserted -- one side alone cannot tell a - working stack from one that completes a handshake and drops every byte. - 45 [ok] with it on. -- [x] The **netdev class**, 2026-08-15. The stack reached straight into the - e1000: `net.c` called `e1000_present()`, the lwIP glue was called - `netif_e1000.c`, and a second NIC driver would have meant editing the - protocol stack to add itself. A driver now publishes a - `struct netdev_ops` (include/netdev.h) and nothing above it learns its - name; lwIP gets one netif per bound device, `/proc/netif` lists them all, - and the e1000 drives up to four cards rather than refusing the second. - Verified with two NICs attached: `eth0` and `eth1`, each with its own MAC - and its own DHCP lease. -- [ ] `accept()` does not report the peer's address, and there is no - `getsockname`/`getpeername`. Nothing has needed them yet; a server that - wants to log who connected will. -- [ ] No routing table. With two interfaces up, the first is the default route - and the second is reachable only by an address on its own subnet. That is - a rule rather than an accident, but it is not routing. -- [ ] Receive is still polled. The network thread looks at the NIC every timer - tick when the link is idle, so an arriving frame waits up to 10 ms before - anything sees it. Waking that thread from the e1000 interrupt is the real - fix and was deliberately not the first version: a receive path woken by an - interrupt is much harder to debug than one that is asked. - -### Phase 3 — GPU - -"Basic GPU driver" has a right answer and a wrong one. The wrong one is -starting with Intel or AMD hardware: modern GPU drivers are enormous and most -of the size is mode setting, power management and a command submission model -that only pays off once there is something to submit. - -- [x] **virtio-gpu**, 2D. `kernel/drivers/virtio_gpu.c`, written from the OASIS - specification. Full path: GET_DISPLAY_INFO, RESOURCE_CREATE_2D, - RESOURCE_ATTACH_BACKING, SET_SCANOUT, TRANSFER_TO_HOST_2D, - RESOURCE_FLUSH. Bound as the display class ahead of the VBE framebuffer. - Run with `-vga std -device virtio-gpu-pci`; keep `-vga std`, because the - bootloader's splash goes through VBE and there is no virtio path that - early -- [ ] A `/dev/dri`-style device with mode setting, so resolution stops being - whatever the bootloader asked VBE for -- [ ] Double buffering and vblank — Schcom needs both to not tear -- [ ] Only then: real hardware, and only 2D/blit acceleration - -### Phase 4 — Fonts - -This is a bigger job than it sounds and it should be sized honestly. A TTF/OTF -rasteriser needs: `cmap`/`glyf`/`loca`/`head`/`hmtx` table parsing, quadratic -béziers for TrueType outlines and *cubic* ones for OpenType/CFF, a scanline -rasteriser with anti-aliasing, and a glyph cache. - -The bigger structural problem is that the console assumes a fixed 8×16 cell -grid throughout — `vga_putc`, the cursor, the ANSI erase operations. Variable -width text is not a font change, it is a text-rendering layer. - -- [x] **The text layer** — `include/text.h` + `kernel/dev/text.c`. Glyph - metrics, a `struct surface` target so a compositor can draw off-screen, - 8-bit coverage with real blending, UTF-8, and `font_ops` so a rasteriser - plugs in as a second provider. This was the actual blocker: the font was - never the problem, the character-cell grid was. -- [ ] Port the console onto it and delete the cell grid -- [x] A TrueType rasteriser — `kernel/dev/ttf.c`, glyf outlines, composite - glyphs, cmap format 4, anti-aliased scanline fill. Host harness at - `tools/ttftest.c` -- [x] A glyph cache — direct-mapped over ASCII into a 48 KB pool -- [ ] Load a font from disk and point the console at it, deleting the cell grid -- [ ] CFF/Type 2 charstrings, so OTF fonts work too (a separate `font_ops` - provider; a bytecode interpreter, not an extension of the glyf path) -- [ ] A rasteriser. `stb_truetype.h` is public domain and therefore usable - as-is if wanted; writing one is also reasonable and more interesting -- [ ] **The typefaces, decided 2026-08-09.** Both verified, not assumed: - **Roboto** for the desktop, window manager and compositor -- - **Apache-2.0**, the same licence as SchOS, so it needs nothing beyond a - NOTICE entry. - **JetBrains Mono** for the terminal and TTY -- **SIL OFL 1.1**, and - notably with *no* Reserved Font Name declared, so it may be modified and - still called JetBrains Mono. That matters more than it sounds: - pre-rasterising a font into bitmaps at build time is a modification, and - an RFN would have forbidden keeping the name on the result. - OFL's one real constraint is that the font may not be sold on its own and - must stay under OFL; bundling it in an Apache-2.0 system is exactly the - case it is written for. -- [ ] Text layout (advance widths, kerning) as a library in `lib/` -- [ ] The console keeps its fixed grid; proportional text belongs to the GUI - -**Font licensing: public domain or very permissive only.** No SIL OFL if it -can be avoided, no "free for personal use", no CC-BY-SA. The constraint is -the same one that governs drivers (§1): a font file is redistributed inside -every SchOS image, so its licence attaches to the image. - -Safe choices, in rough order of usefulness: - -- **CC0 / public domain** — best case, nothing to comply with. -- **MIT / BSD / Apache-2.0 fonts** — fine, needs a `NOTICE` entry. -- **SIL OFL** — usable but has a real string attached: a modified font must - be renamed, and OFL fonts cannot be sold on their own. Workable, but read - it before committing one. - -Whatever is chosen, the licence goes in `NOTICE` **before** the file is -committed, not after. - -### Phase 5 — The GUI, as separate projects - -See `projects/README.md`. Schcom, SchWM and SchDE live outside the kernel -tree on purpose. - -- [ ] **Schcom** — the compositor. Owns the framebuffer, composites client - buffers, routes input. Needs: unix sockets, shm (done), raw input - events, and a GPU driver with double buffering. -- [ ] **SchWM** — window management on top of Schcom's protocol -- [ ] **SchDE** — panel, launcher, settings - -Raw input is the missing kernel piece nobody thinks of until it bites: -keyboard events currently only arrive through the tty line discipline, cooked -and line-buffered and delivered to one foreground process. A compositor needs -key-down/key-up with modifiers and no line discipline at all — an evdev-style -`/dev/input`. - -### Phase 6 — Installer and `.sip` - -`.sip` (SchOS Installation Package), the stated model being WIM in Windows -PE: a *file-based* image rather than a block image, so one file can hold a -whole filesystem, be applied to a partition of any size, and hold several -images that share their common files. - -The pieces for this mostly exist already. `tools/mkschfs.c` does content -dedup with an `(st_dev, st_ino)` fast path; SchFS's B-tree is content -addressed. A sensible `.sip` is: a header, a file index (path, metadata, -content hash), and a blob store keyed by hash, with compression per blob. - -- [ ] `.sip` format specification, written down before any code -- [ ] `tools/mksip` — capture a directory tree into a `.sip` -- [ ] Kernel or userspace apply path -- [ ] An installation environment that boots from the ISO, partitions, - applies the `.sip`, and installs the bootloader - ---- - -### Phase 7 — The full-stack acceptance test *(served its purpose; removed)* - -For a while this phase was a ported copy of a well-known 1993 shooter, kept as -an acceptance test rather than a feature. The reasoning was sound and is worth -keeping even though the test is gone: a large program that somebody else wrote, -which exercises `/dev/fb0`, `/dev/input/event0`, `mmap`, the ELF loader, -`gettimeofday` and a fixed-rate tic loop *simultaneously* and notices when any -of them lie, is a much harsher test than anything written in-house — because it -was not written against our bugs. - -It found real ones, all of which are fixed and covered by narrower tests now: - -- `fcntl(F_SETFL, O_NONBLOCK)` was a documented no-op, so the first read blocked - forever and looked exactly like a hung renderer. -- `/dev/input/eventN` replayed its backlog to each new opener, handing a program - the keystrokes that had launched it as input. -- Without `EVIOCGRAB`, a full-screen program and the shell behind it both read - the same keystrokes. - -It was removed once it stopped finding anything: the port was ~100k lines of -third-party source under licences that can never ship in an Apache-2.0 image, -and carrying it had a standing cost — 33 MB of non-redistributable data staged -into `sysroot/`, which is linked into the kernel binary for a ramdisk-root -build. That combination is a licensing accident waiting to happen, for a test -that was passing every time. - -**What replaced it:** `tools/windows/bootcheck.ps1` with its `-Sock`, `-Evdev` and `-Ps` -modes, plus `tools/windows/splashshot.ps1` for the boot path. These are narrower and do -not prove the whole stack composes, which is the honest cost of the removal. - -**The lesson worth keeping** is the shape of the test, not the program: the next -full-stack test should be something large that we did not write, driven to a -real pixel-level assertion rather than a log line. `tools/windows/splashshot.ps1`'s -colour counting exists because a log-only harness once passed on a blank screen. - -### Phase 8 — The system layer: `schinit`, firmware, and a crash channel - -Three ideas parked here rather than started, because each one is a project and -the phases above are the agreed order. Recorded so they are not re-derived from -scratch later. - -**`schinit`, a launchd-shaped init.** The point of copying launchd rather than -sysvinit is not the config format, it is *demand launching*: launchd holds the -listening socket or watches the path, and starts the job when something actually -arrives. That inverts the dependency graph — jobs stop needing to declare what -must start before them, because the thing they depend on answers on first use. -SchOS already has the two pieces that makes possible: `AF_UNIX` sockets -(`kernel/net/unix.c`) and a real `/proc`. What is missing is a supervisor that -can hold a socket on a job's behalf and hand it over on exec, plus a job -description format. Prerequisite for this being worth anything: `ps` and `top` -must be able to enumerate processes, or the supervisor is unobservable. - -**A crash screen you can work in, and a QR code channel out of it.** PSOGM -(`kernel/core/panic.c`) currently renders and stops. The idea is two staged -things, and the staging matters: - -- *An interactive mode — "a small OS inside the crash".* This is the right - framing and it is worth stating precisely, because it decides the design. At - panic time the kernel is the only thing running, which is exactly why it can - offer what a live system cannot: walk the stack, read another process's - memory, dump page tables, inspect the process table — with no scheduler and no - locking to respect, because nothing else can race you. - - What makes it a *system* rather than a print statement is that it keeps the - base machine's drivers. Concretely, the ones that work with interrupts off: - the VGA/framebuffer (`vga_draw_pixel` is a plain memory write), the PS/2 - keyboard (pollable via port 0x60/0x64), and the serial port. Those three are - a complete console. Everything else — IDE, USB, the timer, anything that - sleeps or takes a sleeplock — is off limits, because the lock it needs may be - precisely the one held by the CPU that just died. - - So the shape is a **polled event loop over preallocated state**: no `kalloc`, - no `sleep`/`wakeup`, no `acquire` on anything a live path might hold, one - static command buffer, and a driver set restricted to the three above. That - restriction is not a limitation to work around later; it is the invariant that - makes the mode trustworthy when everything else is not. Anything that wants - more should be answered by rebooting into a recovery environment instead. - - A useful precedent to read but not copy: DDB (NetBSD/FreeBSD's in-kernel - debugger) is exactly this, and its hard-won rule is the same one — a debugger - that can deadlock is worse than no debugger. -- *A QR code.* **Done.** `kernel/core/qr.c` (byte mode, EC level L, versions - 1–6, no allocation) and a card on the right of the crash screen. A panic - report is worth nothing if it cannot leave a machine whose network stack is - dead, and a phone camera is an out-of-band channel needing no driver on our - side. Deliberately *not* a URL: it encodes the report itself and reaches - nothing. - - It ended up unconditional rather than behind Enter, which is better — it - costs nothing to always draw, and requiring a keypress would mean the one - case where input is least trustworthy is also the case where the report is - hardest to retrieve. Verified end-to-end by `make crashshot && make crashqr`, - which decodes the symbol out of the captured framebuffer. - -**UEFI.** Not a feature so much as the point at which the current bootloader -stops being extendable: it is 16-bit real mode, INT 10h VBE, and PIO ATA, none -of which exist on a UEFI-only machine. The realistic shape is a *second* -loader — an EFI application that gets the framebuffer from GOP, the memory map -from `GetMemoryMap()`, loads the kernel off a FAT ESP, and enters at the same -place `boot/bios/stage2main.c` does — rather than a port of the existing one. Worth -noting that the splash work just done survives that transition: it is a mask, a -palette and a layout, none of which care how the pixels were found. What does -not survive is every assumption in `stage2main.c` about where things live in low -memory (see `docs/layout.md`). - -### SchFS: all five phases are done (2026-08-15) - -Phases 1-3 were finished earlier. Phase 3¾ (the commit barrier, uberblock -fallback, deferred writes), phase 4 (extent refcounts and reflink) and phase 5 -(attributes, the index and the query) all landed on 2026-08-15. See -docs/architecture/schfs.md, which is the detailed record. - -### SchFS beyond SchOS - -Three ideas parked 2026-08-08, in the order they should happen. - -**An on-disk format version, and an upgrade path.** This one is not optional and -gets harder every day it is postponed: the moment an image exists that someone -cares about, changing the layout means being able to read the old one. The -superblock already carries a version field and a CRC32c; what is missing is a -*policy* — a documented rule for what a version bump means, a refusal to mount -something newer than the driver understands (rather than misreading it), and an -explicit upgrade step. Cheap now, expensive later, and it blocks the other two: -a second implementation is only meaningful against a format that is pinned. - -**A Linux implementation.** The realistic first form is **FUSE**, not a kernel -module — it needs no GPL entanglement (`libfuse` is LGPL and used across a -process boundary), it can reuse the read/write logic already in `tools/mkschfs` -and `tools/schfs_fsck` rather than being written twice, and it turns "does our -filesystem work" into something testable with `fsx` and `xfstests`, which is a -far harsher examination than anything in this tree. A Linux *kernel* module is a -different project and carries the licence problem that -[licensing.md](licensing.md) describes. - -**The missing features.** Worth listing honestly before adding any: no journal -replay across an unclean shutdown beyond what CoW gives for free, no snapshots -(despite the B-tree being copy-on-write, which is most of the work), no extended -attributes, no sparse-file support, and no online `fsck`. Snapshots are the one -where the existing design already pays for most of the cost. - ---- - -## 3b. Known broken - -Two things that are wrong *now*, both reported 2026-08-08 and both diagnosed -rather than merely observed. Neither blocks the GUI work, which is why they are -here and not in a phase. - -### ~~SMP is broken~~ *(fixed 2026-08-08)* - -**SMP works.** One, two and four processors all boot; every CPU reports in with -`cpu%d: starting`, and `/proc/cpuinfo` lists them all. `smp_start_aps` in -`kernel/core/main.c` is 1. - -Five distinct bugs, each hidden behind the one before it, and all invisible -under QEMU because it publishes only one processor in the legacy MP table. -VMware published the real count, which is why the triple fault only ever -appeared there. - -| | | -|---|---| -| CPU enumeration | `mpinit()` read the 1994 MP table. `kernel/arch/x86_64/acpi.c` now reads the ACPI MADT. | -| triple fault | `entry64high` had **no AP branch** — every AP took the BSP's stack and ran `kmain()` twice. Now tests `EBX == 1`. | -| `panic("acquire")` | `seginit()` called `kalloc()` before `%fs` existed; kalloc locks, locking reads `cpu`. BSP now preallocates each AP's page. | -| `panic("sched locks")` | `kernel_fs_base` was **one global** read by `trapasm.S` on every entry, so all CPUs shared one per-CPU block. Now `%gs:0`. | -| null `proc` in `sleep()` | `cpu`/`proc` were `__thread` — per *thread*, so the compiler cached the thread pointer across `sched()`, but `%fs` is per *CPU*. Now `mycpu()`/`myproc()`. | -| `#GP` in `syscall_entry` | The new per-CPU slots landed on `%fs:(8)`, the syscall scratch word — the same memory as `%gs:8`. Moved to +16 and +24. | - -The `__thread` one is the lesson worth keeping: **`__thread` is wrong for -per-CPU kernel data.** It means per thread, and the compiler is entitled to -cache the thread pointer across calls on that basis. A kernel thread changes CPU -at every `sched()`, so the assumption is false for exactly the variables that -most need to be right. `include/proc.h` carries the disassembly that proves it. - -Per-CPU block layout, shared with assembly and not to be reordered: - - +0 self-pointer x86-64 TLS ABI (%fs:0) - +8 syscall scratch trapasm.S parks %rsp here - +16 struct cpu * - +24 struct proc * - -### ~~The kernel demands 224 MB of RAM~~ *(fixed 2026-08-08)* - -`kernel/arch/x86_64/e820.c` reads the firmware's memory map and sets `phystop` -from it. **The floor went from 224 MB to 32 MB**, and the kernel now sees all -the memory a machine actually has: 512 MB in gives `MemTotal: 524160 kB`, where -before it reported 224 MB whatever was installed. - -Three details worth keeping: - -- It takes the end of the usable region **containing the kernel**, not the - highest usable address anywhere. A memory map has reserved holes in it, and - `freerange()` walks straight from the end of the kernel image to `phystop` - handing out every page between — so the global maximum would put firmware - tables on the free list, failing later and elsewhere and looking like anything - but a memory map problem. -- The ceiling is **3 GB**, because `kvmalloc()` maps 0–4 GB as four 1 GB pages - and marks the top one uncached for PCI BARs. -- Below `PHYSTOP_MIN` it prints why and stops, rather than triple faulting in - silence. At 24 MB: `e820: only 23 MB usable; SchOS needs at least 24 MB`. - -The original text is kept below because the reasoning still applies to the SMP -half of §3b, which is the same shape of bug: a constant nobody revisited. - -### The kernel demanded 224 MB of RAM - -**Symptom as reported:** the OS needs ~256 MB, which is absurd next to Windows 98 -or an early Linux. - -**Diagnosis:** it is one constant. `PHYSTOP` in `include/memlayout.h` is -`0xE000000` — 224 MB — inherited unchanged from xv6, and `main()` hands -`P2V(PHYSTOP)` straight to `kinit1()`/`dma_init()`. The kernel therefore -*assumes* that much physical memory exists and walks off the end of RAM if it -does not. Measured 2026-08-08: boots at 224M and above, triple faults at 192M -and below. Nothing is actually being *used* — the requirement is entirely -fictional. - -**The fix is already half-built.** `boot/bios/stage2.S` collects a full E820 memory -map in real mode and leaves it at `VBE_INFO_ADDR + 0x200` (count) and `+ 0x208` -(24-byte entries); `boot/bios/stage2main.c` already parses it to print detected RAM. -**The kernel never reads it.** So: have `main()` take the top of the highest -usable E820 region instead of the constant, clamp it to what the direct map -covers, and keep `PHYSTOP` only as a fallback for the case where the firmware -returned nothing. That should drop the floor to something in the tens of -megabytes and is a contained change. - ---- - -## 4. Kept out of scope on purpose - -- Anything that needs pushing or publishing. This work stays local. diff --git a/include/device.h b/include/device.h index 8827dfe..2f765ac 100644 --- a/include/device.h +++ b/include/device.h @@ -18,7 +18,7 @@ // places, as it once was here. // // *A boundary for drivers that are not compiled in.* Drivers the user -// installs (see docs/drivers.md) have to attach somewhere that is not "an +// installs (see docs/architecture/devices.md) have to attach somewhere that is not "an // array in registry.c", because that array is fixed at link time. // // The shape is deliberately the one NeXTSTEP/macOS uses rather than Linux's: diff --git a/include/qr.h b/include/qr.h index 1c7b762..a261855 100644 --- a/include/qr.h +++ b/include/qr.h @@ -6,7 +6,7 @@ // A panic report is worth nothing if it cannot leave the machine, and at panic // time there is no network, no filesystem write that can be trusted, and often // no second machine on the desk. A phone camera is a working out-of-band -// channel that needs no drivers on our side at all. See docs/roadmap.md Phase 8. +// channel that needs no drivers on our side at all. See docs/architecture/graphics.md. // // Deliberately small, in three ways: // diff --git a/kernel/arch/x86_64/mp.c b/kernel/arch/x86_64/mp.c index 9e68624..fccf02d 100644 --- a/kernel/arch/x86_64/mp.c +++ b/kernel/arch/x86_64/mp.c @@ -146,7 +146,7 @@ mpinit(void) // interrupt assignments. So ncpu is 1 on QEMU no matter what, and SMP is // silently never exercised there. // - // See docs/roadmap.md "Known broken" -- this is half of the SMP bug, and the + // See docs/architecture/boot.md -- this is half of the SMP bug, and the // reason the other half (AP startup triple-faulting) cannot be reproduced // under QEMU until the MADT is parsed. cprintf("Seems we are SMP, ncpu = %d\n",ncpu); diff --git a/kernel/arch/x86_64/trap.c b/kernel/arch/x86_64/trap.c index e79397e..84a1dd3 100644 --- a/kernel/arch/x86_64/trap.c +++ b/kernel/arch/x86_64/trap.c @@ -133,7 +133,7 @@ trap(struct trapframe *tf) // General-purpose register dump. This used to be followed by ~25 lines // that walked musl's mallocng group/meta structures out of rax/rdi and // dumped the user stack -- scaffolding for the boot-crash investigation - // recorded in docs/architecture/kernel-history.md section 1. That + // recorded in docs/history/kernel.md section 1. That // investigation is closed and the underlying kernel bugs are fixed, so // the allocator-specific part is gone; the registers stay, since they are // useful for any fault. diff --git a/kernel/drivers/bus/xhci/xhci.c b/kernel/drivers/bus/xhci/xhci.c index c54f6bd..afad6e3 100644 --- a/kernel/drivers/bus/xhci/xhci.c +++ b/kernel/drivers/bus/xhci/xhci.c @@ -12,7 +12,7 @@ // // (Licensing points the same way. The BSD stacks are BSD-2/3-clause, which is // compatible with Apache-2.0 -- Linux's drivers are GPL-2.0-only, which is -// not. See docs/roadmap.md.) +// not. See docs/guides/adding-a-driver.md.) // // Enumeration polls the event ring instead of using interrupts. That is not // laziness: pci_bind_all() runs from kmain() with interrupts disabled and no diff --git a/kernel/drivers/gpu/vga/vga_fb.c b/kernel/drivers/gpu/vga/vga_fb.c index fe36660..1c61703 100644 --- a/kernel/drivers/gpu/vga/vga_fb.c +++ b/kernel/drivers/gpu/vga/vga_fb.c @@ -136,7 +136,7 @@ vga_disp_acquire(struct device *d, uint32 *stride_out) // At 800x600x32 this is 1.9 MB of a 4 MB arena meant for device rings, which // is tolerable only because nothing else large lives there yet. The real fix // is a contiguous page allocator, or reserving this at boot from the memory - // map -- which is work that belongs with the E820 change in docs/roadmap.md + // map -- which is work that belongs with the E820 change in docs/architecture/boot.md // §3b, since that is what stops the kernel pretending 224 MB exists. // // Allocated on first use rather than at boot on purpose: a machine that never diff --git a/kernel/drivers/net/e1000/e1000.h b/kernel/drivers/net/e1000/e1000.h index 12f63ee..21b38bb 100644 --- a/kernel/drivers/net/e1000/e1000.h +++ b/kernel/drivers/net/e1000/e1000.h @@ -5,7 +5,7 @@ // From Intel's public "PCI/PCI-X Family of Gigabit Ethernet Controllers // Software Developer's Manual" (317453). A public document with no licence on // what you write from it, which is why this is written rather than ported: see -// docs/licensing.md and docs/roadmap.md §1. +// docs/licensing.md and docs/guides/adding-a-driver.md. // // QEMU's `-device e1000` is an 82540EM and implements enough of the manual to // be worth targeting; it is also what the default `-net nic` gives you. diff --git a/kernel/proc/proc.c b/kernel/proc/proc.c index 26197c9..96678c0 100644 --- a/kernel/proc/proc.c +++ b/kernel/proc/proc.c @@ -298,8 +298,8 @@ exit(int status) // this, sigsuspend() can never return for the right reason and the // parent hangs unconditionally). Signal delivery itself is verified // working (handler invocation + sigreturn round-trip is clean); mksh's - // own job-control bookkeeping has further gaps (see docs/architecture/ - // kernel-history.md) that this alone doesn't fully resolve, but + // own job-control bookkeeping had further gaps (see docs/history/kernel.md + // §7) that this alone did not resolve, but // disabling it guarantees no progress, whereas leaving it enabled at // least allows the cases that don't hit those gaps to work. if (myproc()->parent) diff --git a/projects/README.md b/projects/README.md index d37f614..0f2d1b5 100644 --- a/projects/README.md +++ b/projects/README.md @@ -49,7 +49,7 @@ the other way, and nothing here is a dependency of the kernel build. ## Status All four are scoped and none is started. What each is waiting on is written -in its own README, and the ordering is in `docs/roadmap.md`. +in its own README, and the ordering is in `docs/plan/roadmap.md`. The short version: Schcom cannot start until the kernel has **unix domain sockets** (for the client protocol and for passing buffer handles) and **raw diff --git a/projects/schcom/README.md b/projects/schcom/README.md index 82a3a60..1f558f0 100644 --- a/projects/schcom/README.md +++ b/projects/schcom/README.md @@ -54,7 +54,7 @@ make schcom-test # boots it and checks the pixels | Framebuffer geometry via the fbdev ioctls | done | | Unix domain sockets | done | | Raw input events — key down/up with modifiers | done | -| `SCM_RIGHTS` | done, and **not yet used** — see below | +| `SCM_RIGHTS` | **not implemented** — there is no `sendmsg`/`recvmsg` at all | | Mode setting and double buffering | partial — virtio-gpu binds, no modeset API yet | ## Design notes @@ -75,15 +75,24 @@ make schcom-test # boots it and checks the pixels ## On Wayland +> **Decided: Schcom moves onto Wayland**, replacing the protocol below while +> keeping the architecture. See [`docs/plan/wayland.md`](../../docs/plan/wayland.md). + + The protocol is deliberately Wayland-shaped — surfaces rather than windows, buffers attached and made visible only on commit, and the compositor telling clients when to draw. Wayland is **MIT-licensed**, so it can be linked against -without any conflict with Apache-2.0 (see `docs/architecture/licensing.md`), and +without any conflict with Apache-2.0 (see `docs/licensing.md`), and supporting it later is intended. The one place v0 knowingly diverges: a buffer handle is a System V shared memory id passed as an integer, where Wayland passes a file descriptor over -`SCM_RIGHTS`. The kernel supports `SCM_RIGHTS` already; an integer simply needs -no ancillary-data handling on either side. `struct schcom_surface` carries a -`handle_kind` field precisely so that switching is a new value rather than a -protocol break. +`SCM_RIGHTS`. `struct schcom_surface` carries a `handle_kind` field precisely so +that switching would be a new value rather than a protocol break. + +**That field cannot save us, and this README used to claim `SCM_RIGHTS` was +done.** It is not — there is no `sendmsg`/`recvmsg` in the kernel at all. And +Wayland's `wl_shm_create_pool` passes a descriptor with no alternative form, so +*our* protocol being flexible is not the flexibility that is needed. Adding +descriptor passing is item 1 of +[`docs/plan/foundations.md`](../../docs/plan/foundations.md). diff --git a/projects/schde/README.md b/projects/schde/README.md index 2f27bf2..1345551 100644 --- a/projects/schde/README.md +++ b/projects/schde/README.md @@ -1,5 +1,12 @@ # SchDE — the SchOS Desktop Environment +> **Superseded as a directory, not as an idea.** A panel, a launcher and a +> settings app are ordinary Wayland clients using `wlr-layer-shell`. This +> directory is to be deleted; they need no home of their own until one of them +> is written. +> +> See [`docs/plan/wayland.md`](../../docs/plan/wayland.md). + The parts you actually look at: a panel, a launcher, settings, a file manager, a terminal. @@ -16,7 +23,7 @@ designing a panel before there is a compositor to put it on. ## What it will need that does not exist yet - **Text rendering.** Everything here is text, and the console's fixed 8×16 - CP437 grid is not usable for it. See phase 4 of `docs/roadmap.md` — a real + CP437 grid is not usable for it. See `docs/plan/wayland.md` — a real glyph cache and rasteriser, and layout as a library in `lib/` so the terminal and the file manager are not each solving it. - **A widget toolkit**, however small. Even a panel needs buttons, labels and diff --git a/projects/schwm/README.md b/projects/schwm/README.md index 6973df4..26d33be 100644 --- a/projects/schwm/README.md +++ b/projects/schwm/README.md @@ -1,5 +1,12 @@ # SchWM — the SchOS Window Manager +> **Superseded.** In Wayland the compositor *is* the window manager — there is +> no protocol by which a second process places another's windows. This directory +> is to be deleted, and what it argues for survives as a module boundary inside +> Schcom rather than a process boundary. +> +> See [`docs/plan/wayland.md`](../../docs/plan/wayland.md). + Policy. Where windows go, which one has focus, what happens when you drag a title bar, how alt-tab behaves, what a window looks like at its edges.