Initial commit: Schunix as it stands before the SchOS rename
An x86-64 operating system built from xv6, now largely rewritten: SMP, a copy-on-write root filesystem (SchFS), PCI/USB/virtio drivers, a TTF rasteriser, and a Linux-numbered syscall ABI. No history before this point -- the tree was never under version control. This commit exists so the rename that follows has something to undo to.
This commit is contained in:
Nexoniarz
commit
d0f7acf9eb
275 changed files with 51417 additions and 0 deletions
666
docs/architecture/kernel-history.md
Normal file
666
docs/architecture/kernel-history.md
Normal file
|
|
@ -0,0 +1,666 @@
|
|||
# Kernel history — what broke, and what the fix actually was
|
||||
|
||||
Not a design document. This is the record of bugs that were expensive to find
|
||||
and whose causes are not visible from the code that fixed them, plus the few
|
||||
subsystem write-ups (bootloader, boot speed, naming) that have nowhere better
|
||||
to live.
|
||||
|
||||
It is worth keeping because every entry here is a thing that looked like a
|
||||
different problem than it was.
|
||||
|
||||
> **Where the old design plan went.** Sections 1-6 of this file used to hold a
|
||||
> "Segregated Kernel" design: a Ring 0 Alpha / Ring 0 Beta split with driver
|
||||
> modules isolated by CR3 switching, from an `IDEA.txt` that no longer exists.
|
||||
> None of it was ever implemented, and it is superseded by
|
||||
> [schapi.md](schapi.md) — a macOS-shaped system answers driver isolation with
|
||||
> userspace drivers and a personality boundary, not a bespoke in-kernel ring
|
||||
> split. It also referenced a Zig toolchain that is now clang/LLVM. Removed
|
||||
> rather than left to be mistaken for a plan.
|
||||
|
||||
---
|
||||
|
||||
## 1. Fixed: boot didn't reliably reach a working shell
|
||||
|
||||
Investigated 2026-07-05, before phase 1 started (predates this design, not
|
||||
caused by it). Originally manifested as `ksh` (mksh) crashing on every
|
||||
launch, apparently inside musl's mallocng allocator (`get_meta`,
|
||||
`src/malloc/mallocng/meta.h`, deliberately executing `hlt` on detecting
|
||||
"corrupted" heap metadata — a `trap 13` GPF at a fixed rip). It wasn't a
|
||||
musl bug. Six real kernel bugs were found and fixed across three
|
||||
investigation passes, and mksh boots and runs cleanly with all of them in
|
||||
place — the calloc assertion was a symptom of real memory/pointer-handling
|
||||
bugs downstream in the kernel, not anything musl or mallocng did wrong.
|
||||
|
||||
Fixed:
|
||||
- `copyuvm()` (`kernel/arch/x86_64/vm.c`, used by `fork()`) never copied
|
||||
`mmap()`-backed `proc->vma[]` regions into the child — only `[0, sz)`.
|
||||
Fixed via new `copy_vma_range()`, called from `fork()` (`kernel/proc/proc.c`).
|
||||
- `kmem.freelist` (`kernel/core/kalloc.c`) could get corrupted by a wild
|
||||
write elsewhere and would silently hand back a bad pointer, panicking far
|
||||
from the actual cause. Added a canary check (`check_run()`) at the top of
|
||||
both `kalloc()`/`kfree()` so corruption panics immediately at the point
|
||||
it's touched, with a clear message.
|
||||
- No TLB invalidation existed anywhere except the full `lcr3` reload on
|
||||
context switch — `mappages()`/`vm_unmap_range()` never called `invlpg`.
|
||||
Fixed by adding `invlpg()` (`include/x86.h`) to both functions.
|
||||
- **`MSR_STAR` was misconfigured** (`syscallinit()`, `kernel/arch/x86_64/vm.c`):
|
||||
`SYSRETQ` computes the returned `cs`/`ss` as `STAR[63:48]+16`/`+8`, which
|
||||
requires the *compat* 32-bit code selector (`SEG_UCODE32`) in that field —
|
||||
the kernel was putting the real 64-bit `USER_CS` there instead. Every
|
||||
`sysretq` silently loaded an invalid `cs` (`0x3b`, pointing at the TSS
|
||||
descriptor)/`ss` pair; QEMU's TCG doesn't validate segment descriptors on
|
||||
`SYSRET`, so this went unnoticed until a hardware interrupt forced a
|
||||
strictly-validated `IRETQ` with that bogus `cs` still loaded, which
|
||||
correctly `#GP`'d.
|
||||
- `sys_mmap`'s non-`MAP_FIXED` path (`kernel/sys/sys_mem.c`) assigned a fixed
|
||||
8MB virtual-address window per `vma[]` slot regardless of the requested
|
||||
length. A single allocation over 8MB (e.g. `realloc()` growth past that
|
||||
size) mapped straight into a neighboring live mapping and hit
|
||||
`mappages()`'s `panic("remap")`. Replaced with a real bump allocator
|
||||
(`proc->mmap_next`, `include/proc.h`), so every mapping gets its own
|
||||
disjoint, exactly-sized range regardless of size.
|
||||
- **`uva2ka()`** (`kernel/arch/x86_64/vm.c`) returned only the page-aligned
|
||||
base address of a user pointer's page, silently dropping the in-page byte
|
||||
offset. Direct callers (`sys_ioctl`'s `TIOCGPGRP`/`TCGETS`, `sys_getcwd`,
|
||||
signal masks, `times()`, `sysinfo()`, ...) read/wrote near the wrong
|
||||
address whenever the pointer wasn't page-aligned — i.e. almost always.
|
||||
Fixed at the source so every caller benefits.
|
||||
- **`fetchint`/`fetchaddr`/`fetchstr`/`argptr`** (`kernel/sys/syscall.c`)
|
||||
rejected any address `>= proc->sz`, wrongly assuming every valid user
|
||||
pointer lives in the heap. Stack addresses and `mmap()`'d buffers (used
|
||||
constantly by real programs, e.g. path-resolution buffers) live above
|
||||
`proc->sz` and were being rejected outright. Fixed to rely solely on
|
||||
`uva2ka()`'s page-table validation.
|
||||
|
||||
Bringing in toybox (see "Coreutils" below) surfaced four more, all fixed:
|
||||
- Linux syscall 58 (`vfork`) wasn't in the syscall-ABI translation table at
|
||||
all (`kernel/sys/linux_abi.c`) — every external command spawn from a
|
||||
program that prefers `vfork()` over `fork()` failed outright. Mapped to
|
||||
the existing `fork()` (a fully correct, if slightly slower, substitute).
|
||||
- `sys_stat`/`fstat`/`fstatat` (`kernel/sys/sys_fs.c`) filled in Schunix's
|
||||
own native `struct stat` layout regardless of the caller — musl/toybox
|
||||
parse a fixed Linux ABI byte layout instead, so `st_mode` was garbage and
|
||||
tools like `ls` couldn't tell files from directories, or executables from
|
||||
non-executables. Added `struct linux_stat` + `stati_linux()`, gated on
|
||||
Linux-ABI mode (`proc->syscall_map`).
|
||||
- `sys_open`/`sys_openat` required `omode == O_RDONLY` *exactly* to open a
|
||||
directory, but `opendir()`-style calls legitimately OR in `O_DIRECTORY`/
|
||||
`O_CLOEXEC` alongside it. Fixed to check just the access-mode bits
|
||||
(`O_ACCMODE`, added to `include/fcntl.h`).
|
||||
- `sys_getdents64` read one `struct dirent` (16 bytes) at a time — a full
|
||||
`bmap()`+`bread()` per slot. Directories here are hash-indexed (see
|
||||
`tree_search()`/`tree_expand()` below), so a directory with a few dozen
|
||||
real entries can have a size in the hundreds of KB, nearly all empty
|
||||
slots — this made listing such a directory look hung (it wasn't a
|
||||
deadlock, just thousands of redundant single-slot disk reads). Rewrote it
|
||||
to read a full block at a time and resolve each entry's type via
|
||||
`iget(inum)` (exported from `fs.c`, it was `static`) instead of
|
||||
re-searching the whole directory by name with `dirlookup()` for every
|
||||
entry. That rewrite's first version introduced a **new** self-deadlock:
|
||||
the "." entry's inode number equals the directory's own, and `iget()`
|
||||
correctly hands back the *same, already-locked* `struct inode*` — calling
|
||||
`ilock()` on it again sleeps forever on a lock the process already holds.
|
||||
Fixed by special-casing `xde->inum == ip->inum` (we already know it's a
|
||||
directory, no need to re-lock at all).
|
||||
|
||||
A synthetic stress test (interleaved malloc/free, realloc growth crossing
|
||||
`MMAP_THRESHOLD`, calloc, large allocations) runs to completion with zero
|
||||
crashes, and mksh now boots straight to an interactive prompt, runs external
|
||||
programs, pipes, and `ls`/`mkdir`/`grep` without issue.
|
||||
|
||||
## 2. Naming standardization
|
||||
|
||||
Schunix standardizes on exactly one name per role, regardless of the
|
||||
underlying implementation: the shell is always installed and exec'd as
|
||||
`/bin/sh` (currently mksh, built from `ports/mksh/`), and PID 1 is always
|
||||
`/bin/init`. `cmd/init/main.c`'s exec loop only ever tries `/bin/sh` — no
|
||||
ksh/mksh/dash/zsh/other-name fallbacks. dash was tried first
|
||||
(BSD-licensed, minimal) while the boot crash above was still open, then
|
||||
replaced with mksh once the underlying kernel bugs were fixed and dash's
|
||||
lack of interactive features (history, completion, line editing) made it
|
||||
feel unusably bare for day-to-day use; the dash port's source tree was left in
|
||||
place rather than deleted, in case it's useful again (e.g. as a `/bin/sh`
|
||||
POSIX-strict target distinct from an interactive login shell).
|
||||
|
||||
## 3. Coreutils
|
||||
|
||||
`ports/toybox/` (0BSD-licensed, https://landley.net/toybox/) provides `ls`,
|
||||
`cat`, `cp`, `mv`, `rm`, `mkdir`, `grep`, `sed`, `find`, `chmod`, `date`,
|
||||
`sort`, and more — see the `APPLETS` list in `ports/toybox/Makefile`. It builds
|
||||
as a single multicall binary; `tools/mkfs.c` doesn't understand symlinks or
|
||||
hardlink dedup, so `ports/toybox/Makefile`'s `install` target copies the built
|
||||
binary once per applet name into `sysroot/bin/` rather than symlinking
|
||||
(costs disk space per applet — currently ~44MB total — instead of one
|
||||
shared inode, but needed no mkfs changes).
|
||||
|
||||
## 4. Two-stage bootloader + PSOGM splash
|
||||
|
||||
`boot/` used to be a single BIOS boot sector doing everything (VBE mode
|
||||
set, A20, GDT, protected-mode switch, multiboot kernel scan/load) — silent,
|
||||
and packed to within 4 bytes of the hard 510-byte MBR limit, so there was
|
||||
no room to add so much as a print statement.
|
||||
|
||||
Split into two stages, the standard way real bootloaders solve this:
|
||||
|
||||
- **Stage 1** (`boot/bootasm.S`, still the literal MBR sector, still
|
||||
≤510 bytes) does exactly one thing: raw IDE PIO (LBA28) reads
|
||||
`STAGE2_SECTORS` (16) sectors starting at disk sector 1 into memory at
|
||||
`0x7E00`, then jumps there. Nothing else fits, nothing else is needed.
|
||||
(Caught a real bug writing this: `outb $0x1F7,%al`-style immediates
|
||||
silently truncate to 8 bits — ports 0x1F2-0x1F7 are all > 0xFF and need
|
||||
the `%dx` register form, or you're hitting the wrong I/O port entirely.)
|
||||
- **Stage 2** (`boot/stage2.S` + `boot/stage2main.c`, loaded at `0x7E00`,
|
||||
occupies disk sectors 1-16) does everything stage 1 used to skip straight
|
||||
to, plus a boot banner (BIOS teletype, real mode, before the VBE mode
|
||||
switch — note this only reaches the video device, not serial, so it
|
||||
won't show up in a `-serial file:...`-captured log), a splash screen, a
|
||||
real-time loading bar, and an F8 boot menu (all described below). Not
|
||||
size-constrained the same way as stage 1 — `tools/padstage2.pl` zero-pads
|
||||
it out to a fixed `STAGE2_SECTORS*512` (8192) bytes so the kernel's
|
||||
on-disk starting sector stays predictable regardless of stage2's actual
|
||||
compiled size, and errors out (like `tools/sign.pl` does for stage 1) if
|
||||
it ever grows past that budget instead of silently corrupting the layout.
|
||||
|
||||
The kernel now starts at disk sector 17 instead of 1 (`schunix.img`'s `dd
|
||||
... seek=17` in the root `Makefile`, and the equivalent `+(1+STAGE2_SECTORS)`
|
||||
in `stage2main.c`'s `readseg()` — all three places that encode
|
||||
`STAGE2_SECTORS` have to agree, or the kernel won't be found on disk at
|
||||
all). `boot/bootmain.c` no longer exists; its logic was split between
|
||||
`stage2.S` (real-mode/protected-mode setup) and `stage2main.c` (splash +
|
||||
the unchanged multiboot-header-scan-and-load logic).
|
||||
|
||||
The VBE mode info block itself lives at `0x30000`, not the classic
|
||||
single-stage bootloader's `0x9000` — stage2's own code now runs from
|
||||
`0x7E00` up to nearly `0x9000` once the boot-menu/progress-bar logic is
|
||||
included, and a block at `0x9000` gets partially overwritten by stage2's
|
||||
own tail end (found this the hard way: the kernel's `LFB:` cprintf started
|
||||
reporting a different wrong value on every boot, because it was reading
|
||||
whatever stage2 code/data now happened to occupy that address).
|
||||
`kernel/dev/vga.c`'s `vga_init()` has to agree on `0x30000` too, since it
|
||||
independently re-reads mode fields from the same block.
|
||||
|
||||
Splash/progress bar/boot menu rendering is a small standalone pixel
|
||||
renderer in `stage2main.c` (not the kernel's `vga.c` — paging isn't set up
|
||||
yet, so the LFB physical address from the VBE block is used as a plain
|
||||
pointer) reusing the kernel's own font data (`include/font.h`) directly.
|
||||
It fills the framebuffer with PSOGM's purple (matching
|
||||
`kernel/core/console.c`'s panic screen) and draws "SCHUNIX" centered, then:
|
||||
|
||||
- **Real-time loading bar**: `readsect()` (in `stage2main.c`, not the
|
||||
kernel) calls `progress_tick()` after every 512-byte disk read, updating
|
||||
an actual pixel-filled bar against the real total byte count — first a
|
||||
small one while scanning for the multiboot header, then a second,
|
||||
properly-sized one once the header's found and the true kernel size is
|
||||
known, labeled "Loading kernel...". Not simulated/timed — it reflects
|
||||
real bytes read off disk.
|
||||
- **F8 boot menu**: `wait_for_f8()` busy-polls the keyboard controller
|
||||
(`inb(0x64)`/`inb(0x60)`, raw scancodes — real-mode BIOS keyboard calls
|
||||
aren't available once we're in protected mode) for about a second after
|
||||
the splash appears. F8 (scancode `0x42`) drops into `boot_menu()`: three
|
||||
options (boot normal, boot verbose, reboot to firmware setup) navigated
|
||||
with up/down arrows and Enter, plus a "detected drives" panel from
|
||||
`ata_present()` — a standard floating-bus presence check (select the
|
||||
drive, write a pattern to two registers, read it back) on the primary
|
||||
IDE channel's master/slave (the only channel this project ever actually
|
||||
puts drives on; QEMU's IDE emulation answers the same test
|
||||
"successfully" on the unpopulated secondary channel too, so that channel
|
||||
isn't shown — it'd just be confidently wrong). "Reboot to firmware
|
||||
setup" pulses the keyboard controller's reset line
|
||||
(`outb(0x64, 0xFE)`, same technique as `kernel/sys/sys_proc.c`'s
|
||||
`sys_reboot()`); a real bootloader can't jump into BIOS/UEFI setup
|
||||
directly, so this is the closest honest equivalent — it gives the user
|
||||
a fresh POST during which their firmware's own setup-menu key works.
|
||||
- **Verbose boot**: picking "Boot Schunix (verbose)" writes a flag byte to
|
||||
`BOOT_VERBOSE_FLAG_ADDR` (`VBE_INFO_ADDR + 0x100`, i.e. just past the VBE
|
||||
block) before jumping to the kernel. `kernel/core/main.c`'s `kmain()`
|
||||
reads it into a global `boot_verbose` as its very first action and, if
|
||||
set, prints a line after every subsystem init call
|
||||
(`uartearlyinit`, `kinit1`, ..., `sinit`) — real per-step boot logging,
|
||||
not a canned message.
|
||||
|
||||
Two bugs worth remembering if this code gets touched again:
|
||||
|
||||
- **Keyboard extended-scancode handling**: `0xE0` (the prefix byte for
|
||||
arrow keys and friends) also has bit 7 set, the same bit used to detect
|
||||
key-*release* ("break") codes. A naive "skip anything with bit 7 set,
|
||||
then check for 0xE0" loop skips the 0xE0 byte itself before ever testing
|
||||
for it, so extended keys silently never register. The 0xE0 check has to
|
||||
come before the break-code check, not after.
|
||||
- **The kernel entry call is not a real C function call.** Both the
|
||||
original single-stage `bootmain()` and the new `stage2main()` do
|
||||
`((void(*)(uint32))entry)(fbaddr)` — normal cdecl, which pushes `fbaddr`
|
||||
on the stack. But `kernel/arch/x86_64/entry.S`'s `mboot_entry` reads its
|
||||
framebuffer argument out of **EAX** (a multiboot-style raw-register
|
||||
handoff, then `mov %eax,%ebp` / `mov %ebp,%ebx` / `mov %rbx,%rdi` carries
|
||||
it through the 32→64-bit transition into `kmain()`'s first argument).
|
||||
The cdecl call never puts anything in EAX — it "worked" for years purely
|
||||
because simple bootmain.c code happened, as a side effect of its own
|
||||
codegen, to leave the right value sitting in EAX right up until the
|
||||
`call` instruction. Adding the splash/menu/progress-bar code changed
|
||||
that codegen and broke it silently: the kernel booted fine and reported
|
||||
a plausible-looking but wrong `LFB:` address (garbage left over in EAX
|
||||
from unrelated prior code, different on every boot). Fixed by pinning
|
||||
the argument down explicitly with `asm volatile("jmp *%0" : : "r"(entry),
|
||||
"a"(fbaddr))` instead of hoping the compiler's register allocation
|
||||
cooperates. If this code moves around again, re-verify with a screendump
|
||||
taken *immediately* before the jump compared against the kernel's own
|
||||
`LFB:` log line — they silently diverging is exactly this bug.
|
||||
|
||||
## 5. `*at()` syscalls ignored/rejected real dirfds
|
||||
|
||||
`ls -l`/`ls -lR` (and, by the same code path, `find`/`du`/anything using
|
||||
toybox's `lib/dirtree.c`) would hang on `ls -l` and either hang or silently
|
||||
misbehave on recursive listings. Root cause: `kernel/sys/sys_fs.c`'s
|
||||
`*at()` syscalls (`openat`, `fstatat`, `unlinkat`, `renameat`, `linkat`,
|
||||
`symlinkat`, `mkdirat`, `mknodat`, `readlinkat`, `faccessat`) only handled
|
||||
two cases for their `dirfd` argument — `AT_FDCWD` (resolve relative to
|
||||
`proc->cwd`) or an absolute path (resolve from root, ignoring `dirfd`
|
||||
entirely). A **real** dirfd with a **relative** path — e.g.
|
||||
`fstatat(open("/bin", O_RDONLY), "ls", &st, ...)`, the standard, efficient
|
||||
way any directory-tree walker avoids rebuilding full path strings for every
|
||||
entry — was either rejected outright (`fstatat`, `unlinkat`, ...) or
|
||||
silently ignored in favor of `proc->cwd` (`openat`, which never checked at
|
||||
all — worse than rejecting, since it can silently resolve the *wrong*
|
||||
file if cwd happens to contain a same-named entry).
|
||||
|
||||
Fixed by adding a real dirfd-relative path resolver: `kernel/fs/fs.c`'s
|
||||
`namex()` was refactored into `namex_from(struct inode *start, ...)`,
|
||||
parameterizing what used to be a hardcoded `idup(proc->cwd)`, with two new
|
||||
exported entry points, `nameifrom()`/`nameiparentfrom()`. `sys_fs.c` gained
|
||||
`at_dir(dirfd)` (resolves `AT_FDCWD` or a real fd to a starting inode) and
|
||||
`at_namei()`/`at_nameiparent()` built on top of it, plus a `createat()`
|
||||
(parameterizing `create()`'s hardcoded `nameiparent()` call the same way).
|
||||
Every `*at()` syscall now uses these instead of its own inline
|
||||
absolute-path/`AT_FDCWD`-only check.
|
||||
|
||||
**Known follow-on issue, not yet root-caused**: heavy `ls`/`ls -lR` use
|
||||
across many commands in one long-running shell session can eventually make
|
||||
a *later*, unrelated directory hang (reproduced with `ls -lR /var` after
|
||||
~10 prior commands in the same session; the identical command succeeds
|
||||
immediately on a fresh boot). Not an inode-table or file-table exhaustion
|
||||
— both `iget()` and `filealloc()` fail cleanly (panic / return 0
|
||||
respectively) when full, and neither fired. Smells like a lock or
|
||||
reference-count leak somewhere in the *at() paths above, or in toybox's
|
||||
own fd handling, that only manifests after enough accumulated opens/closes.
|
||||
Next step: instrument `itable`/`ftable` occupancy counts and watch them
|
||||
across a long `ls`-heavy session to see what's actually climbing.
|
||||
|
||||
## 6. Boot speed
|
||||
|
||||
Two cheap, real wins in `boot/stage2main.c`, no functional changes:
|
||||
|
||||
- `progress_tick()` (called once per 512-byte disk sector) used to trigger
|
||||
a full progress-bar redraw — border plus fill, a few thousand pixel
|
||||
writes — on *every* sector. For the ~330KB kernel image that's ~650 full
|
||||
redraws (the multiboot-header-scan phase adds another ~32) for a bar
|
||||
that only has 101 visually distinct states (0-100%). Split into
|
||||
`progress_border()` (drawn once, in `progress_start()`, since the border
|
||||
never changes) and `progress_fill()`, and `progress_tick()` now skips the
|
||||
redraw entirely unless the percentage actually changed. This was the
|
||||
single biggest chunk of the bootloader's own wall-clock time.
|
||||
- `fill_rect()` (the full-screen splash fill, 480,000 pixels at 800x600,
|
||||
run at least twice per boot) used to call `draw_pixel()` per pixel, which
|
||||
recomputes `g_fb + y*stride + x*4` from scratch every time. Now computes
|
||||
each row's base pointer once and indexes into it.
|
||||
- The F8 boot-menu wait was shortened from ~1s (`5000000` busy-loop
|
||||
iterations) to ~0.6s (`3000000`) — flat overhead on every boot whether
|
||||
or not the menu is wanted, so it's the most direct lever available
|
||||
without giving up the feature entirely.
|
||||
|
||||
## 7. `*at()` syscalls, real signal delivery, and mksh job control (in progress)
|
||||
|
||||
`ls -l`/`ls -la` hanging outright turned out to be the same root cause
|
||||
described in section 11 above (dirfd-relative path resolution) — fixed
|
||||
and verified extensively (plain `ls`, `ls -l`, `ls -la`, `ls -lR`, piped
|
||||
output, all confirmed working repeatedly).
|
||||
|
||||
A *second*, deeper issue was found investigating a follow-up report
|
||||
(`cd` with no `HOME` set, then `cd var`, then `ls` hanging): mksh's
|
||||
interactive job control (`ports/mksh (mksh R59c source): jobs.c`) blocks in
|
||||
`sigsuspend()` waiting for `SIGCHLD`, expecting its own handler
|
||||
(`j_sigchld()`) to run, reap the child via `waitpid(-1, &status, WNOHANG)`,
|
||||
and update its internal job-state tracking. This kernel had **no real
|
||||
signal delivery at all** — `sys_rt_sigaction()` was a no-op stub, so no
|
||||
handler was ever invoked for anything, and `sigsuspend()` just slept out
|
||||
one tick and returned `-EINTR` unconditionally. mksh's wait loop had no
|
||||
way to ever learn the child had exited, hanging forever.
|
||||
|
||||
Real signal delivery was implemented from scratch:
|
||||
- `struct proc` (`include/proc.h`) gained `sig_handler[32]`/
|
||||
`sig_restorer[32]` (populated by `sys_rt_sigaction()`,
|
||||
`kernel/sys/sys_sig.c`), `saved_sigtf` (stashed pre-signal trapframe),
|
||||
and `in_signal` (re-entrancy guard). `allocproc()` zeroes these for a
|
||||
freshly claimed ptable slot (they weren't before — a reused slot
|
||||
silently inherited a previous, unrelated process's handler addresses,
|
||||
which crashed the *next* process to occupy that slot the moment
|
||||
anything tried to delivver a signal to it). `fork()` copies handler/
|
||||
restorer/mask from parent to child (POSIX semantics) but not pending
|
||||
signals.
|
||||
- `kernel/proc/proc.c`'s `exit()` sets `SIGCHLD` pending on the parent.
|
||||
- `kernel/sys/syscall.c`'s `syscall()` calls `deliver_signals(tf)` in its
|
||||
tail, right before returning to user mode: if there's a pending,
|
||||
unblocked signal with a registered handler, it redirects the trapframe
|
||||
to jump to the handler (`rdi`=signal number per SysV ABI, `rip`=handler)
|
||||
instead of resuming the interrupted code, having first stashed the
|
||||
original trapframe in `proc->saved_sigtf`.
|
||||
- The return address is **not** `sig_restorer[sig]` — checked, `SA_RESTORER`
|
||||
is not defined anywhere in this musl build, so `sigaction()` never fills
|
||||
in a restorer; it's always 0, and pushing 0 as a return address crashes
|
||||
with `rip=0` the instant the handler function returns. Instead,
|
||||
`deliver_signals()` injects a tiny 7-byte trampoline (`mov eax,15;
|
||||
syscall` — `rt_sigreturn`) directly onto the user's own stack, below the
|
||||
current `rsp` (clear of the SysV red zone), and points the return
|
||||
address at that — the standard technique for when the C library doesn't
|
||||
supply its own restorer.
|
||||
- `sys_rt_sigreturn()` restores `*proc->tf` from `proc->saved_sigtf`,
|
||||
clearing `in_signal`.
|
||||
|
||||
**A real bug found and fixed along the way**: the first version of
|
||||
`sys_rt_sigaction()` translated the user's `struct sigaction` pointer
|
||||
*once* via `uva2ka()` and then did a bulk `memset`+copy across the whole
|
||||
~152-byte structure. `uva2ka()` only guarantees the *page containing that
|
||||
address* is mapped, not everything up to some further offset — when the
|
||||
user's pointer landed close enough to a page boundary, the write ran past
|
||||
the mapped page into whatever physical page happened to be next in kernel
|
||||
address space (P2V is a flat offset, so "next page" there has nothing to
|
||||
do with the user's actual next page), corrupting unrelated kernel memory.
|
||||
This crashed every freshly `exec`'d `/bin/sh` almost immediately, since
|
||||
mksh calls `sigaction()` during its own startup. Fixed by only touching
|
||||
the two individual 8-byte fields actually needed (`sa_handler` at offset
|
||||
0, `sa_restorer` at offset 144), each through its own `uva2ka()` call.
|
||||
|
||||
`sys_waitpid()`/`sys_wait4()` (`kernel/sys/sys_proc.c`) were also rewritten
|
||||
on top of a new `waitpid(pid, status, options)` in `kernel/proc/proc.c`
|
||||
(refactored from the old any-child-only, always-blocking `wait()`, which
|
||||
is now just `waitpid(-1, 0, 0)`): a specific `pid` (not just "any child"),
|
||||
`WNOHANG` (return 0 immediately instead of blocking), and — importantly —
|
||||
a genuinely optional status pointer. The old code ran a NULL status
|
||||
pointer through `argptr()`, which rejects NULL/unmapped addresses outright
|
||||
(page 0 is deliberately unmapped) — but `waitpid(pid, NULL, WNOHANG)` is
|
||||
completely standard, and is exactly what mksh's job control does.
|
||||
|
||||
**Status: signal delivery mechanically works** — verified directly (a
|
||||
debug build logged the handler being invoked with the correct trapframe
|
||||
and `sys_rt_sigreturn()` restoring cleanly, no crash, repeatable). **mksh's
|
||||
job control still doesn't reliably progress afterward**, including for
|
||||
plain non-piped foreground commands, not just the original piped-into-head
|
||||
case. Root cause not yet isolated. Leading suspects, in order of
|
||||
likelihood:
|
||||
- `sys_setpgid()`/`sys_getpgid()` are no-op stubs (`kernel/sys/sys_proc.c`)
|
||||
— `j_sigchld()` (`ports/mksh (mksh R59c source): jobs.c:1344-1348`) has an early-out
|
||||
keyed on `j->ppid == procpid`, which may never resolve the way mksh
|
||||
expects without real process-group tracking.
|
||||
- `getrusage(RUSAGE_CHILDREN, ...)` — called at the top of `j_sigchld()` —
|
||||
isn't verified to behave sanely under this kernel's implementation
|
||||
(`kernel/sys/sys_proc.c`'s `sys_getrusage()` only fills in `ru_utime`
|
||||
from the *calling* process's own tick count, not accumulated child
|
||||
time; `RUSAGE_CHILDREN` isn't distinguished from `RUSAGE_SELF` at all).
|
||||
- Possibly something in how `j_sigchld()` matches a reaped `pid` against
|
||||
its own `job_list` (`ports/mksh (mksh R59c source): jobs.c:1372-1385`) silently fails
|
||||
to find the job for reasons not yet traced.
|
||||
|
||||
This was left here deliberately rather than continuing to iterate blind:
|
||||
SIGCHLD delivery is left **enabled** (`kernel/proc/proc.c`'s `exit()`)
|
||||
since disabling it guarantees zero chance of mksh's wait loop ever exiting,
|
||||
whereas leaving it on at least gives the cases that don't hit the
|
||||
still-unknown remaining gap a chance to work. Next session should start
|
||||
by instrumenting `j_sigchld()`'s own control flow (or strace-equivalent
|
||||
logging in the kernel around `sys_setpgid`/`sys_getrusage`/`sys_waitpid`
|
||||
calls specifically while reproducing `cd; cd var; ls` or even simpler
|
||||
`ls -l /bin` followed by a second command) rather than re-deriving the
|
||||
signal-delivery mechanism, which is now confirmed solid.
|
||||
|
||||
---
|
||||
|
||||
## 8. Two ABI assumptions that only GCC was keeping true
|
||||
|
||||
Found when the build moved from GCC-in-WSL to clang-on-Windows. Neither is a
|
||||
clang bug; both are places where the code relied on GCC doing something it
|
||||
never promised, and both had been latent for years.
|
||||
|
||||
### `%fs:0` was being used as scratch — it is the TCB self-pointer
|
||||
|
||||
`syscall_entry` (`kernel/arch/x86_64/trapasm.S`) stashed `%rax` at `%fs:(0)`
|
||||
while it fetched `proc` to find the kernel stack. The comment called it a
|
||||
"per-CPU scratch slot". It is not: the x86-64 TLS ABI reserves the word *at*
|
||||
the thread pointer to hold the thread pointer, and a compiler may read it to
|
||||
materialise the TP before applying a variable's offset.
|
||||
|
||||
GCC compiles `__thread` access to a direct `%fs:-8` segment reference and never
|
||||
reads that word, so the clobber was invisible. clang at `-O0` emits
|
||||
`movq %fs:0, %rax; addq $-8, %rax; (%rax)` — reading back the syscall number
|
||||
this code had just stored. The first process's first syscall is
|
||||
`exec` (7), so `proc` resolved to address `7 - 8 = -1` and the kernel
|
||||
page-faulted recursively until the stack was gone.
|
||||
|
||||
Fixed by moving the scratch to `%fs:(8)`. Positive offsets are safe: the TLS
|
||||
block lives at negative offsets and `seginit` places the thread pointer 2048
|
||||
bytes into a 4096-byte page.
|
||||
|
||||
**How it was found** is the part worth copying next time. Three rounds of
|
||||
`cprintf` narrowed it to "the word is right at `mpmain`, wrong by the first
|
||||
trap" and no further — the instrumentation perturbed the timing enough to move
|
||||
the symptom. A hardware watchpoint through QEMU's gdb stub named the
|
||||
instruction on the first run:
|
||||
|
||||
```
|
||||
qemu ... -S -gdb tcp::26000
|
||||
gdb -batch -x tcp.gdb # hbreak mpmain; watch *(unsigned long*)kernel_fs_base
|
||||
```
|
||||
|
||||
`seginit` also now writes the self-pointer, which it never did. That is ABI
|
||||
-correct regardless and costs one store per CPU.
|
||||
|
||||
### `_start` read incoming registers with `register ... asm("rdi")`
|
||||
|
||||
`lib/glue.c` captured `argc`/`argv` from `%rdi`/`%rsi` using GCC local register
|
||||
variables. Both compilers only guarantee that binding for inline-asm operands;
|
||||
using it to pick up an incoming argument is undefined. clang ignored it and
|
||||
read two uninitialised stack slots:
|
||||
|
||||
```asm
|
||||
movl -0x4(%rbp), %eax # should have been %edi
|
||||
movq -0x10(%rbp), %rax # should have been %rsi
|
||||
```
|
||||
|
||||
Every program still ran. `init`, `shmtest` and `socktest` take no arguments and
|
||||
passed their entire suites; the only symptom in the whole system was
|
||||
`inputtest -n 6` ignoring its limit. Moved to `lib/start.S`, where reading
|
||||
incoming registers is expressible and correct — `exec` passes argc/argv in the
|
||||
SysV argument registers already, so `_start` calls `main` without touching
|
||||
them.
|
||||
|
||||
---
|
||||
|
||||
## 9. The boot looked like a hobby OS, and two things were why
|
||||
|
||||
Not a bug — a design problem, fixed by looking at a screenshot instead of the
|
||||
code.
|
||||
|
||||
**The display was the kernel log.** Boot printed `pci: 6 devices` and
|
||||
`vga: display controller 1234:1111, aperture fd000000 (16 MB)` to the screen
|
||||
the user is looking at. That single fact is most of what separates a hobby
|
||||
project from a product; every shipping system puts that behind a flag.
|
||||
|
||||
`cons_serial_only` already existed for panic, so the fix was to start it *set*.
|
||||
Everything still goes out the UART — which is where `tools/bootcheck.ps1` reads
|
||||
it from, so all five suites kept passing untouched — and `console_takeover()`
|
||||
clears it on the first `exec()`, the moment userspace genuinely starts.
|
||||
`vga_init` no longer clears the screen, so the splash survives the whole of
|
||||
boot rather than being wiped a few hundred milliseconds in.
|
||||
|
||||
**There were three unrelated designs.** The splash was deep purple with gold
|
||||
text, deliberately matching the *panic* screen. The console was cream with
|
||||
near-black text. So a healthy boot was dressed as a crash, then abruptly
|
||||
became an amber terminal. `include/theme.h` is now the one palette, and the
|
||||
rule is: **normal states are calm and dark; only failure is loud.** PSOGM keeps
|
||||
the purple entirely to itself, which is what makes it mean something.
|
||||
|
||||
The splash copy went from "Schunix Boot Manager / version 1.0 / Memory: 1023 MB
|
||||
/ Press F8 for boot menu..." to the product name, a rule in the accent colour,
|
||||
and the one hint that is actionable — moved to the bottom, where a hint
|
||||
belongs. No shipping OS names its bootloader on the splash.
|
||||
|
||||
### Three latent bugs it flushed out
|
||||
|
||||
Retheming meant `stage2main.c` included `types.h` properly for the first time,
|
||||
and the 32-bit build stopped compiling:
|
||||
|
||||
- `_Static_assert(sizeof(uint64) == 8)` fired. `uint64` was `unsigned long`,
|
||||
which is **4 bytes** in the `-m32` loader — the file even carried a local
|
||||
`e820_u64` workaround with a comment saying so. Now conditioned on
|
||||
`__SIZEOF_LONG__` rather than on the platform, which covers LP64, LLP64 *and*
|
||||
ILP32 correctly.
|
||||
- `x86.h`'s `stosq` and `xchg` use `rep stosq` and `xchgq`, which do not exist
|
||||
in 32-bit mode. They only ever compiled because `uint64` was silently
|
||||
narrowed. Now behind `#ifdef __x86_64__`.
|
||||
- The progress bar computed `(uint64)done * 100 / total`, a 64-bit divide that
|
||||
needs libgcc's `__udivdi3` — which a freestanding loader does not link. Now
|
||||
scaled to KiB and done in 32 bits.
|
||||
|
||||
All three had been sitting in a bootloader that worked fine, waiting for
|
||||
someone to make `uint64` mean what it says.
|
||||
|
||||
---
|
||||
|
||||
## 10. A full-screen program typed the whole session into the shell
|
||||
|
||||
Keys reached `/dev/input/event0` **and** the tty line discipline at the same
|
||||
time. That is correct for two consumers of one keyboard and completely wrong
|
||||
for a full-screen program: while it had the display, the shell behind it was
|
||||
quietly accumulating every keystroke, and the first Enter afterwards ran
|
||||
whatever had piled up. Arrow keys arrived as CP437 glyphs on the console for
|
||||
good measure.
|
||||
|
||||
The fix is the standard one and it is a *primitive*, not a patch: an exclusive
|
||||
grab, `EVIOCGRAB`, with Linux's ioctl number. While a device is grabbed its
|
||||
events still flow to `/dev/input/eventN` but both keyboard drivers withhold the
|
||||
decoded character from the console.
|
||||
|
||||
Three details that matter more than the feature:
|
||||
|
||||
- **The grab is released by the devsw `close` hook**, not by the program. A
|
||||
program that grabs the keyboard and then crashes would otherwise leave the
|
||||
console permanently deaf, with no way to type the command that fixes it.
|
||||
- **Modifier state is still tracked while grabbed** (`kbdgetc` keeps updating
|
||||
shift/caps and only skips the character). Otherwise releasing the grab would
|
||||
hand the line discipline a stuck Shift.
|
||||
- **The USB path updates `prev[]` before returning**, or the next report would
|
||||
look like every held key had just been pressed again.
|
||||
|
||||
It is also exactly what Schcom will need, which is why it lives in the input
|
||||
layer rather than in any one client.
|
||||
|
||||
### The near-miss worth remembering
|
||||
|
||||
The first attempt appeared to work: the harness passed, every assertion green.
|
||||
It had not worked at all — the ioctl call was never inserted into the client
|
||||
source, because a scripted string replacement silently failed to match. The
|
||||
`grep` that should have caught it printed three hits (a comment, a `#define`,
|
||||
and the *release* call) and was read as confirmation without counting what
|
||||
they were.
|
||||
|
||||
What actually caught it was reading the transcript rather than the verdict:
|
||||
four bare `#` prompts after quitting, which is precisely the symptom the change
|
||||
was supposed to remove. `input_grab()` now announces itself, so "did the grab
|
||||
happen" is never again a thing anyone has to infer.
|
||||
|
||||
---
|
||||
|
||||
## 11. `ps` and `top`, and why they never worked
|
||||
|
||||
/proc was a set of character devices: init ran `mknod("/proc/uptime", 6, 2)` and
|
||||
friends, so every file was a real inode on the root filesystem with a
|
||||
major/minor behind it. That is fine for a fixed list and cannot work at all for
|
||||
the thing /proc is mostly for -- one directory per process. You cannot mknod a
|
||||
directory for a process that does not exist yet and will be gone in a minute.
|
||||
`ps` calls `dirtree_flagread("/proc")`, got a listing with no numbers in it, and
|
||||
reported nothing. It was never a `ps` problem.
|
||||
|
||||
So /proc is now a real filesystem behind the same `struct fs_ops` vtable SchFS
|
||||
uses (`kernel/fs/procfs.c`, `PROCFS_DEV`), mounted by init with
|
||||
`mount("proc", "/proc", "proc", ...)`. Nothing is stored: directory listings
|
||||
are generated from the process table when asked, file contents when read.
|
||||
|
||||
Three things worth keeping in mind:
|
||||
|
||||
- **The split is deliberate.** `kernel/fs/procfs.c` owns the *namespace* (what
|
||||
exists); `kernel/dev/procfs.c` keeps the *generators* (what the global files
|
||||
say). They change for different reasons, so `proc_gen_global()` is the only
|
||||
thing that crosses between them.
|
||||
- **procfs never touches ptable.** `proc_snapshot()` copies what is needed out
|
||||
from under the lock, so a process that exits between the readdir that listed
|
||||
it and the open that reads it produces a stale line rather than a
|
||||
use-after-free.
|
||||
- **Sizes are reported as zero**, exactly as Linux does. A synthetic file has
|
||||
no length until it is generated, and generating every one on every `stat()`
|
||||
to fill in a number almost nothing reads would be absurd.
|
||||
|
||||
### What the suite caught
|
||||
|
||||
Marking every procfs inode read-only broke `echo s > /proc/sysrq`, and three
|
||||
USB checks failed on it within a minute of the change -- they drive the disk
|
||||
through sysrq. /proc is read-only *except* sysrq, which is the one deliberate
|
||||
write target, and the code now says so in one place.
|
||||
|
||||
### Still not right
|
||||
|
||||
`ps -o comm` prints `[init]` rather than `init`. That field comes from
|
||||
`/proc/<pid>/exe`, which Linux makes a **symlink** and toybox `readlink()`s;
|
||||
ours is a regular file, so the readlink fails and toybox brackets the name as
|
||||
it would for a kernel thread. Default `ps` and `top` output are unaffected.
|
||||
Fixing it needs symlink support in procfs, which belongs with the rest of the
|
||||
VFS work rather than as a one-off.
|
||||
|
||||
---
|
||||
|
||||
## 12. A test program crashed at random, and had been typing its own launch command
|
||||
|
||||
Symptom: a full-screen framebuffer program, launched with arguments, worked
|
||||
four times and then died with
|
||||
|
||||
pid 9 <prog>: trap 14 err 4 on cpu 0 rip 0x2269be addr 0x18--kill proc
|
||||
|
||||
`err 4` is a user-mode read of a non-present page and `addr 0x18` is a null
|
||||
pointer at offset 24. `llvm-nm` put `rip` inside the routine that opens the
|
||||
program's overlay view, which dereferences a pointer the program only fills in
|
||||
once a level is loaded -- at offset 0x18 on x86-64, because the three 64-bit
|
||||
words of a base struct sit in front of it. So the program had been asked to open
|
||||
that view before there was anything to show.
|
||||
|
||||
**The technique is the transferable part:** the fault address alone said
|
||||
"null pointer plus 24", which is nearly useless; `llvm-nm` on the faulting `rip`
|
||||
turned it into a function name, and the offset then identified the exact field.
|
||||
Reach for that before adding printfs to a program you did not write.
|
||||
|
||||
The cause was ours, and it was visible in the log the whole time: every run
|
||||
printed a burst of garbage bytes *before* the grab was taken, and the amount
|
||||
differed each run.
|
||||
|
||||
**`/dev/input/eventN` had no notion of being opened.** Events were queued from
|
||||
boot onwards whether or not anything was listening, into one 256-entry ring
|
||||
with one read cursor. When the program opened the device it inherited the entire
|
||||
backlog -- including the keystrokes of the command line that had launched it,
|
||||
which the shell had already consumed -- and replayed them as input. Whether that
|
||||
crashed depended on which keys happened to be sitting there, which is exactly
|
||||
the "sometimes" in the report.
|
||||
|
||||
Fixed by giving `struct devsw` an `open` hook to match `close`, and having
|
||||
`/dev/input`:
|
||||
|
||||
- drop the backlog on open (`r = w`) -- a reader starts from *now*;
|
||||
- only queue events while `opens > 0`, which is what Linux does: an evdev
|
||||
device with no clients does not accumulate.
|
||||
|
||||
Linux gets both for free by giving every open its own client queue. One shared
|
||||
ring plus "start from now" is the same answer for a single reader, which is the
|
||||
only case that exists today; per-client queues become necessary the moment
|
||||
Schcom and a client both want the keyboard.
|
||||
|
||||
### Also fixed
|
||||
|
||||
`cd` into a file reported "Operation not permitted". It was right to fail, but
|
||||
`sys_chdir` returned a bare `-1`, so the shell printed whatever errno happened
|
||||
to be lying around. It returns `-ENOTDIR` now, and `-ENOENT`/`-EFAULT` on the
|
||||
other two failure paths -- "Not a directory" sends you to the right place,
|
||||
"Operation not permitted" sends you looking at permissions.
|
||||
239
docs/architecture/schapi.md
Normal file
239
docs/architecture/schapi.md
Normal file
|
|
@ -0,0 +1,239 @@
|
|||
# SchAPI, and Schunix as a system
|
||||
|
||||
Status: **direction agreed, not yet implemented.** Nothing in this document
|
||||
changes what is being built this week; it changes what everything is being
|
||||
built *towards*. See §7 for what actually happens now.
|
||||
|
||||
---
|
||||
|
||||
## 1. The decision
|
||||
|
||||
Schunix stops being "a Unix-like kernel with a userland bolted on" and becomes
|
||||
a **system**: a kernel, a native API, and the services and applications that
|
||||
sit on it, designed together.
|
||||
|
||||
The model is NeXTSTEP/macOS, not Linux. Specifically:
|
||||
|
||||
- A kernel that provides mechanism, with **more than one personality** on top.
|
||||
- **POSIX/Unix is a personality, not the identity.** It stays, completely and
|
||||
seriously, because it is how you get software. It is not the thing programs
|
||||
are expected to be written against.
|
||||
- **SchAPI** is the native interface: higher-level, object-shaped, designed for
|
||||
the GUI and services rather than retrofitted onto 1970s process semantics.
|
||||
|
||||
Two systems are being borrowed from, and they are *different* systems. It is
|
||||
worth being precise about which, because they disagree:
|
||||
|
||||
| Borrowed | From | Why |
|
||||
|---|---|---|
|
||||
| Layered architecture, personalities, native API over a Unix substrate | macOS / NeXTSTEP | It is the only mainstream design that got a modern API onto a Unix base without abandoning either |
|
||||
| Permanent backward compatibility | **Windows** | macOS is emphatically *not* this — Apple dropped 32-bit apps, Carbon, kexts and OpenGL, each time on a deadline |
|
||||
|
||||
That mix is coherent, but it is a mix. Taking the architecture from Apple and
|
||||
the compatibility policy from Microsoft is a deliberate choice, and the rest of
|
||||
this document is mostly about making that choice survivable — because
|
||||
compatibility-forever is the single most expensive promise an OS can make, and
|
||||
Windows pays for it every day.
|
||||
|
||||
---
|
||||
|
||||
## 2. The one mechanism that makes the promise affordable
|
||||
|
||||
**The stable interface is a library, not the syscall table.**
|
||||
|
||||
This is the whole trick, and it is what lets macOS keep decade-old binaries
|
||||
running while freely renumbering its syscalls. On macOS, nothing may issue a
|
||||
syscall directly; you link `libSystem`. Apple therefore owes compatibility only
|
||||
at the library boundary, and the kernel stays editable forever.
|
||||
|
||||
Linux made the opposite choice — the syscall ABI *is* the contract, "we do not
|
||||
break userspace" — and consequently every syscall mistake ever made is still
|
||||
there, with a `2` or a `64` suffixed to the name of its replacement.
|
||||
|
||||
So, for Schunix:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ applications, SchWM, SchDE │
|
||||
├──────────────────────────────────────────────┤
|
||||
│ SchAPI │ POSIX / Linux personality│ ← THE STABLE BOUNDARY
|
||||
├──────────────────┴───────────────────────────┤
|
||||
│ libschunix — the only thing that may syscall│
|
||||
├──────────────────────────────────────────────┤
|
||||
│ syscall table — PRIVATE, unstable, renumberable
|
||||
├──────────────────────────────────────────────┤
|
||||
│ kernel │
|
||||
└──────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Rules that follow, and they are not negotiable if the promise is to hold:
|
||||
|
||||
1. **A program that issues `syscall` directly is unsupported.** It may break at
|
||||
any release. Only `libschunix` may.
|
||||
2. `libschunix` exports versioned symbols. An entry point's behaviour never
|
||||
changes; a changed behaviour is a new version of the symbol.
|
||||
3. The kernel may renumber, merge, split or delete syscalls at will, provided
|
||||
`libschunix` still presents the same surface.
|
||||
|
||||
This *reconciles* with the existing decision to adopt Linux syscall numbering
|
||||
(`docs/roadmap.md`, and the reason musl/toybox/mksh work at all). Linux
|
||||
numbering does not go away — it becomes the **Linux personality's** entry path,
|
||||
which is exactly what it already is in practice. It is a compatibility surface
|
||||
that happens to be entered via `syscall`, and it is frozen because *Linux*
|
||||
froze it, not because we did.
|
||||
|
||||
---
|
||||
|
||||
## 3. Stability tiers
|
||||
|
||||
"Nothing can ever be removed" is only affordable if you are precise about what
|
||||
"anything" covers. Windows' real problem was never the documented API — it was
|
||||
that undocumented behaviour became load-bearing, and then it was too late.
|
||||
|
||||
Every public interface carries exactly one tier:
|
||||
|
||||
| Tier | Promise | May change |
|
||||
|---|---|---|
|
||||
| `stable` | Forever, from the release that introduced it | Never. Additions only. |
|
||||
| `provisional` | Until the next release | Freely |
|
||||
| `internal` | None | Freely, without notice |
|
||||
|
||||
**Everything in Schunix today is `provisional`.** The freeze starts at 1.0 and
|
||||
covers only what is marked `stable` at that moment. This is not a way of
|
||||
dodging the commitment — it is the only way to make it, because a promise made
|
||||
now would permanently enshrine, among other things, a `fcntl(F_SETFL)` that
|
||||
was a documented no-op until last week and a `struct file` whose `O_NONBLOCK`
|
||||
handling had never been exercised.
|
||||
|
||||
Nothing becomes `stable` by being old. It becomes `stable` by someone deciding
|
||||
it is right and writing that down.
|
||||
|
||||
---
|
||||
|
||||
## 4. `compat/` — what it actually means
|
||||
|
||||
The instinct is right: old interfaces should not litter the current tree. But
|
||||
moving *source* into a folder does not reduce the *runtime* cost, and the
|
||||
runtime cost is the one that hurts.
|
||||
|
||||
The rule that keeps compatibility cheap:
|
||||
|
||||
> **A compatibility entry point must be implementable as a translation into the
|
||||
> current API. If it cannot be, that is a defect in the current API — fix the
|
||||
> current API before freezing it.**
|
||||
|
||||
So `compat/` holds shims, not parallel implementations. A shim:
|
||||
|
||||
- lives in `compat/<release-it-was-current-in>/`,
|
||||
- is written in terms of today's API and nothing else,
|
||||
- is never called by new code,
|
||||
- carries its own tests, which are the only thing that proves the promise is
|
||||
still being kept.
|
||||
|
||||
When a shim *cannot* be expressed as a translation, that is the alarm. It means
|
||||
the new API lost information the old one carried, and it is far cheaper to
|
||||
notice that before 1.0 than to maintain two real implementations forever. That
|
||||
is precisely the trap Win32 fell into.
|
||||
|
||||
---
|
||||
|
||||
## 5. Taking from the BSDs
|
||||
|
||||
`src-trunk/` is a NetBSD checkout, and a FreeBSD one is being added alongside
|
||||
it. Both are reference material and parts bins; neither is built and neither
|
||||
may ever end up in an image.
|
||||
|
||||
**Two trees is not redundancy.** They are worth having together for three
|
||||
reasons that a single tree cannot give:
|
||||
|
||||
- Where an RFC is ambiguous — and TCP's are, constantly — two independent
|
||||
implementations agreeing is far stronger evidence than one implementation
|
||||
asserting.
|
||||
- Where one is licence-encumbered, the other may not be. NetBSD's
|
||||
`tcp_input.c` carries a WIDE Project advertising clause that Berkeley's
|
||||
rescission does not cover; FreeBSD's equivalent may well be 2-clause. See
|
||||
[../licensing.md](../licensing.md) §3.
|
||||
- They have opposite biases, and knowing which you are reading matters:
|
||||
|
||||
| | FreeBSD | NetBSD |
|
||||
|---|---|---|
|
||||
| Stack quality | The better stack in absolute terms — faster, more complete, more actively developed | Solid, conservative, less featureful |
|
||||
| Extractability | Welded to FreeBSD: `epoch(9)`, `uma`, `netisr`, VIMAGE/vnet, `if_t`, a rich locking vocabulary | Portability *is* the project's identity; assumes far less of its host kernel |
|
||||
| Use it as | The reference you check yourself against | The code you actually lift |
|
||||
|
||||
That table is the same lesson §1 of the roadmap already learned the hard way
|
||||
with FreeBSD's `sys/dev/usb`: 100k lines welded to newbus, where porting the
|
||||
USB stack means porting the device framework first. Networking is the same
|
||||
shape. Read FreeBSD, lift NetBSD, write Schunix.
|
||||
|
||||
### Licensing — read this before copying anything
|
||||
|
||||
NetBSD is not uniformly licensed: 2-clause, UC 3-clause and a minority of
|
||||
4-clause-with-advertising files all appear, and only the first two are
|
||||
comfortable. The per-template rules, the measured counts, and the one-line
|
||||
check to run before copying a file are in [../licensing.md](../licensing.md)
|
||||
§3. Do not copy from `src-trunk/` without reading it.
|
||||
|
||||
### What is actually worth taking
|
||||
|
||||
Ranked by value, not by size:
|
||||
|
||||
1. **`sys/net`, `sys/netinet` — the TCP/IP stack.** The largest single win
|
||||
available. Phase 2 of the roadmap wants ARP/IPv4/ICMP/UDP/TCP, and this is a
|
||||
mature, clean, readable, 2-clause-heavy implementation of exactly that.
|
||||
2. **`sys/kern/subr_pool.c` — the pool/slab allocator.** Schunix has `kalloc`
|
||||
and a page at a time. Every subsystem written since has invented its own
|
||||
fixed-size table to work around that.
|
||||
3. **`autoconf` — the driver framework** (`config(1)`, `cfdriver`/`cfattach`,
|
||||
`device_t`). This is the grown-up version of what `kernel/drivers/registry.c`
|
||||
and `kernel/dev/irq.c` are hand-rolling. Adopt the *model*; do not try to
|
||||
port the code wholesale.
|
||||
4. `sys/kern/subr_*` generally — rbtree, workqueue, condvar, kmem.
|
||||
|
||||
Low value: `lib/libc` (musl already fills that role), `sys/dev/usb` (we have a
|
||||
working xHCI stack written from the spec).
|
||||
|
||||
**Take designs before code.** Every file copied is a file to be understood,
|
||||
relicensed-in-notice, and maintained forever under the §3 promise.
|
||||
|
||||
---
|
||||
|
||||
## 6. What SchAPI is, concretely
|
||||
|
||||
Not yet designed. What is decided about it:
|
||||
|
||||
- It is the native interface; POSIX is available beneath it but not the idiom.
|
||||
- It is reached through `libschunix`, never by direct syscall.
|
||||
- It is designed *with* Schcom in mind, because a compositor and its clients
|
||||
are the first real users and their needs (IPC, shared buffers, event streams,
|
||||
object lifetimes) are what will shape it.
|
||||
- The IPC primitive comes first. Everything in a NeXT-shaped system rests on
|
||||
it, and unix domain sockets — already built and working — are the substrate,
|
||||
not the interface.
|
||||
|
||||
What is explicitly *not* decided: whether it is object-oriented in the Objective-C
|
||||
sense, what language it is expressed in, and whether it takes Mach's port model
|
||||
or something smaller. Those need their own document, written when Schcom has
|
||||
told us what it actually needs.
|
||||
|
||||
---
|
||||
|
||||
## 7. What changes now
|
||||
|
||||
Almost nothing, on purpose. The ordering does not change:
|
||||
|
||||
- **The native Windows build must boot first.** The clang-built kernel
|
||||
currently reaches `sinit` and dies on the first trap. A direction is worth
|
||||
nothing while the tree does not run.
|
||||
- Then the roadmap continues as written: network, GPU, fonts, Schcom.
|
||||
|
||||
What this document changes *today* is only this:
|
||||
|
||||
1. `src-trunk/` is reference material, excluded from every build and image.
|
||||
2. New interfaces are marked `provisional` from now on, so that §3 has
|
||||
something to work with at 1.0.
|
||||
3. When the network stack is written, NetBSD's is the starting point rather
|
||||
than a blank file — with the licence check in §5 applied per file.
|
||||
4. `libschunix` is created when there is a second personality to justify it.
|
||||
Until then, the *rule* stands even though the library does not exist yet:
|
||||
no new first-party program issues raw syscalls.
|
||||
403
docs/architecture/schfs.md
Normal file
403
docs/architecture/schfs.md
Normal file
|
|
@ -0,0 +1,403 @@
|
|||
# SchFS — design
|
||||
|
||||
**Status: phases 1 and 2 are done.** The copy-on-write B-tree, the volume layer,
|
||||
the object layer, a working image writer and a checker all exist, and the kernel
|
||||
mounts a SchFS volume read-only at boot. Phase 3 (writes and transaction groups)
|
||||
is next. See §8 for exactly what is built and how to run its tests.
|
||||
|
||||
Supersedes the on-disk format described in `include/fs.h`. Keeps the name SchFS;
|
||||
the on-disk magic changes so v1 images are detected and rejected rather than
|
||||
silently misread.
|
||||
|
||||
The brief was "the best aspects of BeFS, ZFS and btrfs." Those three disagree
|
||||
with each other in places, so this document says explicitly what is taken from
|
||||
each and what is deliberately left behind.
|
||||
|
||||
## 1. What each one is actually good at
|
||||
|
||||
**ZFS — never be inconsistent.** Copy-on-write: no block that is part of the
|
||||
live tree is ever overwritten. An update writes new blocks, then atomically
|
||||
flips a single root pointer. A crash therefore lands on either the old tree or
|
||||
the new one — never a half-written mix. Consequence: **no fsck**, ever. Paired
|
||||
with end-to-end checksums where a block's checksum is stored in its *parent*
|
||||
pointer, not in the block itself, which makes the whole tree self-validating and
|
||||
catches misdirected writes and bit rot that a self-stored checksum cannot.
|
||||
|
||||
**btrfs — one data structure, reused everywhere.** btrfs has essentially one
|
||||
engine: a copy-on-write B-tree. Inodes, directory entries, extent maps, free
|
||||
space, and snapshots are all just different key types in that same tree. For a
|
||||
small team this is the single most important idea in the whole design: you write
|
||||
and debug *one* tricky data structure, then get five subsystems from it. Also
|
||||
takes extents with **reference counts**, which is what makes snapshots and
|
||||
reflink copies fall out almost for free.
|
||||
|
||||
**BeFS — the part that feels like a database, at a size one person can build.**
|
||||
BeFS is a 64-bit journaled filesystem whose distinguishing feature is B+tree
|
||||
**indexes over file attributes**, with a query interface on top. That is the
|
||||
"SQL-like" capability, and unlike WinFS it actually shipped. Critically, BeFS is
|
||||
also the right *scale* model: a small team, a readable codebase, and a book
|
||||
(Giampaolo, *Practical File System Design*, free online) that describes it
|
||||
end to end.
|
||||
|
||||
## 2. What is deliberately not taken
|
||||
|
||||
- **No ZFS storage pools / RAID-Z / vdevs.** Multi-disk redundancy is a large
|
||||
subsystem that solves a problem Schunix does not have yet.
|
||||
- **No btrfs-style rebalancing or online format conversion.** Enormous
|
||||
complexity, and it is the historical source of btrfs's worst bugs.
|
||||
- **No actual SQL.** No parser, no query planner, no relational engine. See §6 —
|
||||
the useful part of "database filesystem" is the *index*, not the language.
|
||||
WinFS spent 2003-2006 and a large team on the other interpretation and
|
||||
shipped nothing.
|
||||
- **No deduplication.** ZFS's dedup is famous for needing more RAM than anyone
|
||||
expects. Extent refcounts give reflink/snapshot benefits without it.
|
||||
|
||||
## 3. On-disk shape
|
||||
|
||||
```
|
||||
+--------------------+ block 0 boot block (untouched by the FS)
|
||||
+--------------------+ blocks 1..N uberblock array (128 slots, rotated)
|
||||
+--------------------+
|
||||
| CoW B-tree | one tree, many key types:
|
||||
| | (ino, INODE, 0) -> inode record
|
||||
| | (ino, DIRENT, hash) -> directory entry
|
||||
| | (ino, EXTENT, offset) -> extent mapping
|
||||
| | (ino, ATTR, nameid) -> extended attribute
|
||||
| | (idx, INDEX, key) -> attribute index entry
|
||||
+--------------------+
|
||||
| free space | extent-based, tracked as tree keys, not a bitmap
|
||||
+--------------------+
|
||||
```
|
||||
|
||||
**Uberblock array (from ZFS).** 128 slots written round-robin, each carrying a
|
||||
transaction number and a checksum. Mount scans all slots and takes the
|
||||
highest-numbered one that validates. This is what makes crash recovery
|
||||
"pick the newest good root" instead of "replay a journal."
|
||||
|
||||
**Everything is one B-tree (from btrfs).** Keyed by
|
||||
`(objectid, type, offset)`, 24 bytes. Adding a feature usually means adding a
|
||||
key type, not a new on-disk structure.
|
||||
|
||||
**Checksums live in the parent (from ZFS).** Each internal node stores, for
|
||||
every child pointer, that child's CRC32c. A block that is read is verified
|
||||
against the pointer that led to it. v1 already chose CRC32c and already has
|
||||
checksum fields in `dinode` and `superblock` — same algorithm, better placement.
|
||||
|
||||
### Widths — every v1 limit that has to die
|
||||
|
||||
| v1 | limit | v2 |
|
||||
|---|---|---|
|
||||
| `dirent.inum` is `ushort` | 65,535 files, total | `uint64` object ids |
|
||||
| `dinode.size` is `uint` | 4 GB max file | `uint64` |
|
||||
| `uint` block numbers, BSIZE 1024 | 4 TB volume | `uint64` blocks, 4096-byte |
|
||||
| `DIRSIZ` 14 | 14-char filenames | variable, up to 255 |
|
||||
| fixed hash table per dir | every dir ≥113 KB, capped at 119 KB | B+tree, pay per entry |
|
||||
| `addrs[NDIRECT+3]` | block-at-a-time | extents with refcounts |
|
||||
|
||||
Block size moves 1024 → 4096 to match modern media and make extents worthwhile.
|
||||
|
||||
## 4. What already exists and gets reused
|
||||
|
||||
Not starting from zero:
|
||||
|
||||
- **`struct schfs_extent`** is already defined in `include/fs.h` and unused.
|
||||
The extent concept is already sketched; v2 gives it refcounts and 64-bit
|
||||
fields.
|
||||
- **CRC32c checksum fields** already exist in `dinode` and `superblock`, and
|
||||
mkfs already computes them. Same algorithm, relocated into parent pointers.
|
||||
- **`kernel/fs/log.c`** is a working transaction log with `begin_op`/`end_op`
|
||||
call sites throughout the FS. Under CoW the log is not needed for consistency,
|
||||
but `begin_op`/`end_op` are exactly the right places to bracket a transaction
|
||||
group, so the *call sites* survive even though the implementation is replaced.
|
||||
- **The buffer cache** (`bio.c`) is format-agnostic and needs no change beyond
|
||||
the block-size constant.
|
||||
|
||||
## 5. Phasing
|
||||
|
||||
Each phase leaves a bootable system. Do not start phase 2 before phase 1 is
|
||||
solid — the B-tree is the load-bearing piece and everything else assumes it.
|
||||
|
||||
**Phase 1 — the B-tree, in userland first.** Write the CoW B-tree as a plain C
|
||||
library with a block-device abstraction, and test it on a *host* file with a
|
||||
fuzz/torture harness (random insert/delete/lookup, simulated crash at every
|
||||
write point, verify the tree always mounts). This is the one component where a
|
||||
subtle bug is unfixable later, and testing it in the kernel is far harder. It is
|
||||
also portable straight into `tools/mkfs`.
|
||||
|
||||
**Phase 2 — read-only mount. DONE.** New magic, uberblock scan, tree walk, inode
|
||||
and dirent key types. `tools/mkschfs` writes images; the kernel only reads them.
|
||||
The system boots from a ramdisk root (see `docs/layout.md` and the ramdisk work)
|
||||
so a broken disk cannot stop you from booting.
|
||||
|
||||
Went further than planned in two places, both cheaply: data blocks carry
|
||||
checksums too (a `CSUM` key type, as btrfs does — without it "end-to-end
|
||||
checksums" would cover only metadata, which is the part least likely to rot),
|
||||
and extents landed early because reserving a file's blocks in one go was
|
||||
simpler than not doing it.
|
||||
|
||||
Not yet done: SchFS files are not reachable through `open(2)`. The mount is
|
||||
observable through `/proc/schfs` and through the kernel API in `schfs_int.h`,
|
||||
but wiring it into `namei`/`readi` means the VFS work that phase 3 needs anyway.
|
||||
|
||||
**Phase 3 — writes, transaction groups. DONE.** Every `schfs_k*` call that
|
||||
changes anything is one transaction group, committed by writing a new uberblock
|
||||
before it returns. The full write API — create, link, unlink, rename, write at
|
||||
an arbitrary offset, truncate — lives in `objw.c` and is exercised by
|
||||
`tools/schfs_objtest.c`.
|
||||
|
||||
Two things went differently from the plan above, both for reasons worth
|
||||
recording.
|
||||
|
||||
*Free space is not tracked as tree keys.* Allocating from a free map that lives
|
||||
in the tree mutates the tree, which allocates, which changes the map; btrfs
|
||||
resolves that with a fixpoint loop and ZFS with log-structured space maps. SchFS
|
||||
sidesteps it: the committed tree already says exactly which blocks are live, so
|
||||
the map is **derived** by walking the tree once at mount and held only in RAM.
|
||||
It cannot disagree with the filesystem, because it is computed from it, and a
|
||||
crash raises no question at all. The cost is mount time proportional to metadata
|
||||
size, and a ceiling on volume size set by how much bitmap fits in memory. If
|
||||
mount time ever hurts, the map can be cached on disk purely as a hint,
|
||||
validated against the generation and discarded on a mismatch — an addition, not
|
||||
a format change. See `kernel/fs/schfs/space.c`.
|
||||
|
||||
*Data blocks needed a generation stamp too.* The B-tree already rewrites its own
|
||||
nodes in place within a group, since a block that no committed root can reach is
|
||||
nobody's fallback. Data blocks had no such marker, so every hundred-byte write
|
||||
copied a whole 4 KB block and split an extent record around it. `schfs_extent`
|
||||
gained a 64-bit `gen`, and the same rule now applies to file data.
|
||||
|
||||
**Phase 3½ — reachable through `open(2)`. DONE.** SchFS is a mounted filesystem:
|
||||
|
||||
```
|
||||
mkdir /mnt
|
||||
mount -t schfs /dev/hdb /mnt
|
||||
ls /mnt ; echo hi > /mnt/tmp/w ; cat /mnt/tmp/w ; mkdir /mnt/tmp/d
|
||||
```
|
||||
|
||||
The VFS grew a real dispatch layer rather than a set of `if (dev == SCHFS_DEV)`
|
||||
checks. `struct fs_ops` (include/file.h) is one table per filesystem, chosen by
|
||||
`iget()` from the device number and hung off the inode; v1's implementations
|
||||
keep their bodies and are simply named `v1_*`. Adding a third filesystem is a
|
||||
new table and nothing else. `kernel/fs/schfs/vfs.c` is SchFS's, and it is pure
|
||||
translation — object ids in, inodes out — because all the filesystem logic
|
||||
already lives below it.
|
||||
|
||||
Three things had to change on the way, none of them SchFS's fault:
|
||||
|
||||
- `struct inode.inum` and `.size` were 32-bit, capping objects at 2³² and files
|
||||
at 4 GiB on a filesystem that promises far more. Both are `uint64` now, as are
|
||||
`readi`/`writei` offsets and `struct file.off`.
|
||||
- `struct dirent` carries a 16-bit inum and a 14-byte name, so it cannot
|
||||
represent a SchFS entry at all. `getdents64` is now an fs_op: each filesystem
|
||||
emits the record stream itself, and `f->off` is whatever position it needs to
|
||||
resume. v1 walks its hash buckets; SchFS walks a key range and synthesises
|
||||
"." and "..", which it does not store.
|
||||
- Mounting is a substitution made during path resolution (`mount_add` /
|
||||
`mount_cross` in `fs.c`). Nothing on either disk records it. The reverse
|
||||
direction matters as much as the forward one: `..` at a mounted root has to
|
||||
leave the mount, and the filesystem underneath has no idea it is mounted
|
||||
anywhere.
|
||||
|
||||
Three long-standing bugs surfaced the moment real commands ran against it, all
|
||||
in the shared VFS rather than in SchFS:
|
||||
|
||||
- `include/fcntl.h` used xv6's open flags while every program in the userland
|
||||
sends Linux's. `O_APPEND` (0x400) landed on `O_TRUNC`, so `cmd >> file`
|
||||
*truncated* it; `O_TRUNC` (0x200) landed on `O_CREATE`, so `cmd > file` left
|
||||
whatever was past the new end in place. Nothing about either pointed at a
|
||||
header.
|
||||
- `sys_openat` silently dropped `O_TRUNC` and `O_APPEND` while `sys_open`
|
||||
honoured both, so the same redirection behaved differently depending on which
|
||||
entry point libc happened to call.
|
||||
- `rmdir` had no Linux syscall number, so every Linux-ABI program got ENOSYS.
|
||||
|
||||
**Phase 4 — extents with refcounts.** Enables reflink copies and snapshots
|
||||
nearly for free.
|
||||
|
||||
**Phase 5 — attribute index and query.** See below.
|
||||
|
||||
## 6. The "SQL" part, honestly
|
||||
|
||||
What you want from "works like SQL" is: *ask a question about a lot of files and
|
||||
get an answer fast, without walking every directory*. That is an **index**, not
|
||||
a query language. BeFS's design:
|
||||
|
||||
- Files carry typed attributes (`string`, `int`, `time`, …).
|
||||
- An attribute can be *indexed*: a dedicated B+tree keyed by attribute value,
|
||||
mapping value → object ids.
|
||||
- A query is a predicate over indexed attributes, evaluated by intersecting
|
||||
index ranges.
|
||||
|
||||
In SchFS this is just another key type — `(idx, INDEX, key)` — in the tree
|
||||
that already exists. That is the whole reason to build phases 1-4 the btrfs way:
|
||||
the "database filesystem" feature becomes a few hundred lines on top of
|
||||
machinery already written and debugged, rather than a separate engine.
|
||||
|
||||
A concrete `query()` syscall and a `find`-like tool come after that, and are
|
||||
small by comparison.
|
||||
|
||||
## 7. Honest cost
|
||||
|
||||
Phases 1-3 are roughly 3,000-4,000 lines and are the bulk of the work. Phase 1
|
||||
alone (B-tree + torture harness) is worth doing carefully and unhurriedly; every
|
||||
later phase compounds on it.
|
||||
|
||||
This is the largest single piece of work in Schunix. It is also entirely
|
||||
ordinary filesystem engineering with excellent published references
|
||||
(Giampaolo for BeFS; the btrfs design papers; the ZFS uberblock/checksum-in-
|
||||
parent scheme) — there is no research risk here, only volume.
|
||||
|
||||
**Do this before the installer.** The installer's entire job is to create and
|
||||
populate this format. Building it against v1 means building it twice.
|
||||
|
||||
## 7½. Limits
|
||||
|
||||
Three different things get called "the limit", and they are worth keeping apart:
|
||||
what the **on-disk format** can express, what **this implementation** can
|
||||
currently mount, and what the **rest of Schunix** can carry. The smallest one is
|
||||
what you actually hit.
|
||||
|
||||
### The format
|
||||
|
||||
| | limit | where it comes from |
|
||||
|---|---|---|
|
||||
| Volume | **64 ZiB** (2⁶⁴ × 4 KiB blocks) | `uint64` block numbers |
|
||||
| File | **16 EiB** (2⁶⁴ bytes) | `schfs_inode.size` |
|
||||
| Files per volume | **1.8 × 10¹⁹** | `uint64` object ids |
|
||||
| Hard links per file | **4.3 × 10⁹** | `uint32 nlink` |
|
||||
| Filename | **255 bytes** | `SCHFS_NAME_MAX` |
|
||||
| Entries per directory | no fixed limit | one tree key per name hash |
|
||||
| Extent | **64 ZiB** in one record | `uint64 nblocks` |
|
||||
| Tree depth | **8 levels** | `SCHFS_MAX_LEVEL` |
|
||||
| Block size | **4096 bytes**, fixed | `SCHFS_BLKSIZE`; a mismatch is refused at mount |
|
||||
| Transaction groups | **1.8 × 10¹⁹** | `uint64` generation, never reused |
|
||||
|
||||
Directory *depth* is unbounded on disk; path resolution is what limits it.
|
||||
|
||||
### This implementation
|
||||
|
||||
| | limit | where it comes from |
|
||||
|---|---|---|
|
||||
| Volume, mounted | **64 GiB** | the in-RAM allocation bitmap: `SCHFS_SPACE_PAGES` (512) pages × 32768 blocks |
|
||||
| Volume, via the buffer cache | **4 TiB** | `bio.c` addresses blocks with a `uint`, and one SchFS block spans four of them |
|
||||
| RAM per mount | **8 KiB per GiB** of volume | two bitmaps, one bit per block each |
|
||||
| Inline value in a leaf | **1016 bytes** | `SCHFS_MAX_INLINE` = a quarter of a node |
|
||||
| Colliding names in one directory | ~**60**, sharing a hash | the packed list for one hash must fit in one inline value |
|
||||
| Directory entries at 8-byte names | ~**42** per key, unlimited keys | as above |
|
||||
| Records per leaf / children per node | **127 / 84** | 4 KiB minus a 32-byte header |
|
||||
| Mount time | one pass over the metadata | the allocation map is derived, not stored |
|
||||
|
||||
The 64 GiB mount ceiling is the one that bites first, and it is one constant:
|
||||
raising `SCHFS_SPACE_PAGES` to 4096 makes it 512 GiB at 64 KiB of pointers per
|
||||
mount. It is set where it is because the page-pointer vector lives inside
|
||||
`struct schfs_vol`, which the kernel holds as static storage.
|
||||
|
||||
### The rest of Schunix
|
||||
|
||||
These are VFS and driver limits rather than SchFS ones:
|
||||
|
||||
- `MAXPATH` caps a path at **128 bytes**.
|
||||
- `NMOUNT` allows **8** mounts.
|
||||
- The IDE driver is 28-bit LBA: **128 GiB** per disk.
|
||||
- A single `read`/`write` call is capped at **4 GiB** (`n` is a `uint`); the
|
||||
file itself is not.
|
||||
|
||||
The two that used to be here — a 16-bit inode number in `struct dirent` and a
|
||||
32-bit `inum`/`size` in `struct inode` — are gone. `getdents64` is a per-
|
||||
filesystem operation now, so SchFS emits its own records with 64-bit inode
|
||||
numbers and names up to 255 bytes, and the inode carries 64-bit numbers and
|
||||
sizes throughout.
|
||||
|
||||
## 8. What exists now
|
||||
|
||||
```
|
||||
include/schfs.h on-disk format + compile-time layout assertions
|
||||
kernel/fs/schfs/btree.{c,h} the copy-on-write B-tree and its cursor
|
||||
kernel/fs/schfs/schfs_int.h volume + object layer interface
|
||||
kernel/fs/schfs/space.c allocation bitmap, derived from the tree at mount
|
||||
kernel/fs/schfs/vol.c uberblock ring: open, format, commit
|
||||
kernel/fs/schfs/obj.c read side: inodes, dirents, extents, path lookup
|
||||
kernel/fs/schfs/objw.c write side: create, link, unlink, rename, write
|
||||
kernel/fs/schfs/kmount.c kernel: block device, mount, transaction bracket
|
||||
kernel/fs/schfs/vfs.c the fs_ops table: SchFS behind open/read/write
|
||||
tools/mkschfs.c build an image from a directory tree
|
||||
tools/schfs_fsck.c verify, list, extract
|
||||
tools/schfs_test.c B-tree torture harness
|
||||
tools/schfs_objtest.c object layer: files, directories, space accounting
|
||||
tools/schfs_roundtrip.sh end-to-end filesystem test
|
||||
tools/mutate.sh mutation-tests the suites themselves
|
||||
tools/bootcheck.ps1 boots under QEMU and drives the shell
|
||||
```
|
||||
|
||||
Everything except `kmount.c` is compiled *both* into the kernel and into the
|
||||
host tools, so the writer and the reader cannot drift apart: there is one
|
||||
implementation of the format and nothing to disagree with.
|
||||
|
||||
### Running the tests
|
||||
|
||||
```
|
||||
make schfs-check all three suites
|
||||
make schfs-test B-tree only
|
||||
make schfs-objtest object layer only
|
||||
make schfs-image-test round-trip only
|
||||
```
|
||||
|
||||
`make schfs-check` after touching anything under `kernel/fs/schfs`.
|
||||
|
||||
The round-trip's oracle is the host's own filesystem: an image is built from a
|
||||
directory tree, extracted with a separate program, and `diff -r`'d against the
|
||||
original. It also asserts hard-link counts (which `diff` cannot see), checks
|
||||
that rebuilding produces byte-identical output, and runs the whole thing a
|
||||
second time with the name hash cut to 6 bits so the packed-dirent collision path
|
||||
— which a 64-bit hash would never reach — is actually exercised.
|
||||
|
||||
`schfs_objtest` runs against a memory-backed volume, so a full
|
||||
create/write/delete/remount cycle takes milliseconds. Its most important check
|
||||
is the least glamorous: after deleting everything, free space must be back to
|
||||
exactly what it was on an empty volume. Every other test in the suite still
|
||||
passes on a filesystem that leaks a block on every delete.
|
||||
|
||||
`tools/mutate.sh` injects six deliberate bugs and fails if any of them goes
|
||||
undetected. It is how the suites are kept honest; see the note in §9.
|
||||
|
||||
### Verifying on real hardware
|
||||
|
||||
```
|
||||
make ramroot root from RAM, so device 1 is free
|
||||
make build/schfs.img
|
||||
make qemu-schfs then, at the shell: cat /proc/schfs
|
||||
```
|
||||
|
||||
`.\run.ps1 -SchFS` does the same on Windows, and `tools\bootcheck.ps1 -SchFS`
|
||||
does it unattended and asserts on the transcript.
|
||||
|
||||
## 9. Things learned the hard way
|
||||
|
||||
Recorded because each one cost real time and none is obvious from the code.
|
||||
|
||||
- **The root's object id must be reserved.** `next_objectid` started at
|
||||
`SCHFS_FIRST_OBJECTID`, which *is* the root's id, so the first directory
|
||||
created silently became the root. It then contained itself, and every tree
|
||||
walk recursed until the stack ran out. `schfs_fsck --check` now detects
|
||||
directory loops explicitly rather than crashing.
|
||||
- **Cursor buffers alias.** A range callback gets a pointer into the cursor's
|
||||
block buffer, and almost every readdir caller immediately looks the entry up,
|
||||
which overwrites it. Directory iteration copies each entry out before calling
|
||||
back. Extracting a tree with directories reparented under their own children
|
||||
is what this looks like when it goes wrong.
|
||||
- **A 4096-byte block does not fit on a 4096-byte kernel stack.** The cursor
|
||||
exists precisely to avoid one block buffer per tree level, but
|
||||
`schfs_vol_open` still kept a whole uberblock as a local and blew the stack on
|
||||
the first boot. The kstack canary caught it cleanly.
|
||||
- **Extents only merge if you allocate the run first.** Writing an extent record
|
||||
allocates blocks for its own copy-on-write path, so allocating data one block
|
||||
at a time guarantees the blocks are never adjacent and every block needs its
|
||||
own extent. `schfs_file_reserve` claims the whole run up front; an 822 KB
|
||||
binary is one extent instead of 201.
|
||||
- **Freed blocks are reusable during a build.** Copy-on-write forbids reusing a
|
||||
released block because the previously committed tree still points at it — but
|
||||
a build has committed nothing. Recycling them cut the test image from 3596
|
||||
blocks to 374.
|
||||
- **Reproducibility dies from atime.** mkschfs reads every source file, which
|
||||
updates its atime, so copying atime into the image made two builds of the same
|
||||
tree differ. All three timestamps come from mtime.
|
||||
Loading…
Reference in a new issue