nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

docs: four directories, and a rule for when a thing leaves the roadmap

Browse source 
docs/ was ten files where a plan, a description, a guide and a bug report all
sat next to each other. It is now architecture/ (what SchOS is), plan/ (what it
will be), guides/ (how to work on it) and history/ (why it is like this), with
an index that states the one rule holding it together: when something is
finished it moves from plan/ to architecture/ and is rewritten as a description
of the thing rather than a record of building it.

A ticked checkbox says the work happened. It does not say what the work
*produced*, and six months later the checkbox is all that is left. The network
stack proved the point: finished, verified end to end, and for a month the only
description of it anywhere was a checkbox four levels of indentation deep.

So roadmap.md went from 534 lines to 181 -- nothing deleted, the done work
graduated. Seven subsystems that were finished and undescribed now have
descriptions: boot, kernel, syscalls, filesystem, networking, graphics, usb,
init. Every one ends with "What it does not do", which is the section that pays;
a document listing only strengths is marketing, and the reader who needs it most
is the one about to trip over a limitation nobody wrote down.

schfs.md still opened with "phases 1 through 3 1/2 are done, phase 4 is next"
long after all five had landed. It is a description now, with the phase record
and the hard-won bugs moved to history/.

Three corrections found on the way, all load-bearing:

  * projects/schcom/README.md claimed SCM_RIGHTS was done. There is no
    sendmsg/recvmsg in the kernel at all, and an entire GUI plan rested on it.
  * The root README said "No TCP/IP" months after the stack landed.
  * history/kernel.md sec.7 was marked "in progress" on a problem that process
    groups solved.

Also adds plan/1.0.md, which draws a finish line so the open list stops being
infinite, and marks schwm/ and schde/ superseded: in Wayland the compositor is
the window manager, so that split dissolves rather than being abandoned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz 2026-09-12 17:13:35 +02:00
commit 57aff40228
59 changed files with 3981 additions and 1124 deletions
Download patch file Download diff file Expand all files Collapse all files

102
docs/architecture/README.md Normal file
View file

@ -0,0 +1,102 @@
# The system, layer by layer
What SchOS is today. Everything in this directory is built and running; what is
merely intended lives in [../plan/](../plan/).
---
## The names, and what each one covers
`uname -s` reports `SchIFT`, the way every Unix reports its kernel rather than
its distribution.
| Layer | Name | What it is |
|---|---|---|
| System | **SchOS** | The distribution — kernel, userland, tooling |
| Kernel | **SchIFT** | Multi-core x86-64 kernel with a modular VFS |
| Filesystem | **SchFS** | Transactional, copy-on-write, the root filesystem |
| API | **SchAPI** | The native programming interface — *designed, not built* |
| Compositor | **Schcom** | Owns the display; v0 runs |
---
## The stack, bottom to top
```
┌─────────────────────────────────────────────────┐
│ toybox mksh musl first-party cmd/ │ userland
├─────────────────────────────────────────────────┤
│ schinit — PID 1, job manifests, supervision │
├─────────────────────────────────────────────────┤
│ syscall table — Linux numbering, one .inc file │ ← the ABI seam
╞═════════════════════════════════════════════════╡
│ VFS │ sockets │ signals │ mmap │ poll │ procfs │
├──────┴─────────┴─────────┴──────┴──────┴────────┤
│ SchFS │ lwIP │ text/gfx │ buffer cache │
├─────────┴────────┴────────────┴─────────────────┤
│ device model — class vtables, bus binding │
├─────────────────────────────────────────────────┤
│ drivers: xHCI e1000 virtio-gpu VBE IDE PS/2│
├─────────────────────────────────────────────────┤
│ scheduler │ paging │ traps │ IRQ │ SMP │ DMA │ SchIFT core
└─────────────────────────────────────────────────┘
▲
UEFI loader (boot/uefi)
```
The two seams that matter are the syscall table and the device model. Above the
first, software is ordinary and portable — musl and toybox are unmodified.
Below the second, a driver never names its consumer and a consumer never names a
driver.
---
## Where it came from
SchOS began as [xv6](https://github.com/mit-pdos/xv6-public) and is now mostly
something else. What survives from xv6 is the *shape* of the process table, the
sleep/wakeup discipline, and the buffer cache. What has been replaced: the
scheduler and trap path (rewritten for x86-64 and for more than one processor),
the memory manager, the entire filesystem, the syscall layer, and every driver.
That inheritance is worth knowing when reading the code, because the parts that
still look like xv6 follow xv6's conventions and the parts that do not, do not.
---
## The documents
Read in this order if you are new:
| | |
|---|---|
| [boot.md](boot.md) | Firmware to the first userspace process. The shortest path through the whole system. |
| [kernel.md](kernel.md) | Memory, processes, scheduling, traps, signals, SMP. |
| [syscalls.md](syscalls.md) | The one table everything is generated from, and how a musl binary runs unmodified. |
| [filesystem.md](filesystem.md) | The VFS, the buffer cache, mounts, procfs. |
| [schfs.md](schfs.md) | The copy-on-write root filesystem. |
| [devices.md](devices.md) | Buses, classes, binding, and the vtables consumers talk to. |
| [usb.md](usb.md) | xHCI, the USB core, and the three class drivers. |
| [graphics.md](graphics.md) | Framebuffer, text rendering, the console, and PSOGM. |
| [networking.md](networking.md) | NIC drivers, lwIP, and both socket families. |
| [init.md](init.md) | `schinit` and the job manifests. |
---
## Three rules that hold across all of it
**The kernel and the host tools compile the same SchFS source.** `mkschfs`
cannot disagree with the kernel about the on-disk format, because there is only
one implementation. Format bugs are found on the host, where a test takes a
second instead of a boot.
**Syscalls are declared in exactly one place.** `kernel/sys/table.inc`.
The constants, the dispatch table and the Linux translation map are all
generated from it, so a syscall cannot be half-registered.
**A list you can read beats a list assembled by side effects.** Drivers are
written out by hand in `kernel/drivers/registry.c` rather than collected by a
linker section, and platform devices are registered in one function rather than
inside each driver's init. The tradeoff is deliberate: self-registration is
nicer to write, and it means the list of what is in your kernel is not written
down anywhere you can read.

215
docs/architecture/boot.md Normal file
View file

@ -0,0 +1,215 @@
# Boot
Firmware to the first shell prompt. This is the shortest path through the whole
system, which makes it the best thing to read first.
The loader's side of the hand-off is specified in
[`boot/README.md`](../../boot/README.md) — that file is the contract, and this
one is the narrative.
---
## 1. Firmware
**UEFI only.** There was a BIOS path and it was removed: everything since
roughly 2012 boots UEFI, and CSM — the firmware-side BIOS emulation that let
legacy loaders run — has been going away since 2020. The cost of keeping it was
not the code but the tax on everything it touched, and the asymmetry is total. A
BIOS loader lives in 16-bit real mode and calls the firmware through software
interrupts; UEFI hands a loader long mode with paging already on and a table of
function pointers.
The loader is `boot/uefi/`, a 64-bit PE application built by the same clang
install as everything else (`--target=x86_64-unknown-windows`). It imports
neither gnu-efi nor EDK II — the handful of structures it uses were written from
the specification, which is both a smaller dependency and no licence question.
## 2. The loader
Six steps, in this order, and the order is load-bearing:
1. Find the kernel on the volume this image was loaded from — `\schos\kernel`,
or `\kernel`.
2. Load its ELF `PT_LOAD` segments to their physical addresses.
3. Record the framebuffer geometry from the Graphics Output Protocol.
4. Translate `GetMemoryMap()` into the 24-byte E820-shaped entries
`kernel/arch/x86_64/e820.c` already parses — not a second format for the
kernel to learn.
5. `ExitBootServices`.
6. Build the page tables the kernel's 64-bit entry expects, and jump.
**Nothing allocates after step 5 and nothing calls a firmware service after it,
because there are none.** Everything the jump needs is computed before.
### The hand-off block
One page of low memory at `0x30000`, defined once in
[`include/schboot.h`](../../include/schboot.h) and included by both the loader
and `entry.S` so the two cannot disagree.
| Offset | What |
|---|---|
| `+0x000` | VBE-shaped mode info: LFB address, stride, width, height, depth |
| `+0x100` | Verbose-boot flag |
| `+0x101` | Boot medium (`1` = CD; `0` = not stated) |
| `+0x200` | E820 entry count, then up to 40 × 24-byte entries |
| `+0x600` | ACPI RSDP physical address, or 0 |
Two of these exist because of specific failures worth knowing about.
**The RSDP is passed explicitly** because the kernel's own search scans the EBDA
and `0xE0000`–`0x100000`, which is a BIOS convention. UEFI publishes the RSDP in
its configuration table and is under no obligation to leave a copy anywhere;
OVMF does not. Without it the kernel finds no MADT, so no IOAPIC, so no
interrupt routing — and the machine boots to a prompt on polled serial output
and then ignores every keystroke, which looks like a broken console rather than
a missing table.
**The E820 cap is 40 entries, not 32.** Real UEFI firmware produces far more
descriptors than QEMU's, and the space was already there.
### The entry point
The kernel publishes a `sch64_header` — magic `SCH64BOO`, 8-byte aligned within
the first 16 KB, followed by a 64-bit physical entry address — found by
scanning, exactly as a multiboot loader scans for its own magic.
It exists because the firmware has *already* put the machine in long mode.
Dropping back to protected mode to re-enter through the 32-bit multiboot path
is a mode switch whose failure mode is a triple fault with no console. The
32-bit entry is still present in `entry.S` and now has no caller.
## 3. `kmain()`
`kernel/core/main.c`. The ordering here is not arbitrary and several steps are
where they are because of a bug:
```
e820_init() memory map first — PHYSTOP is a variable, not a constant
kinit1() the allocator, given everything below the DMA arena
dma_init() the arena, carved off the TOP of physical memory
kvmalloc() the kernel page tables
acpi_init() CPU enumeration from the MADT; mpinit() only as fallback
lapicinit() tvinit() seginit() ioapicinit()
irq_init() before any driver that registers a handler
inputinit() before the keyboard and mouse start producing events
consoleinit() uartinit() mouseinit()
platform_register_devices(fb) the fixed furniture of a PC, plus the framebuffer
pci_init() enumerate, and bind drivers
pinit() binit() fileinit() sockinit()
time_init() after the timer runs, before userspace
ideinit() ramdiskinit()
startothers() the other processors
kinit2() sinit() the rest of memory, then PID 1
netinit() after sinit — it starts a kernel thread
mpmain() enter the scheduler; never returns
```
**The DMA arena is taken before `kalloc` gets the rest.** A device needs
physically contiguous memory and `kalloc`'s free list makes no such promise —
two pages from it are adjacent only by chance. Taking it first is the only way
to be sure the region exists.
**ACPI is asked before the legacy MP table**, and getting this backwards is
invisible under QEMU. Modern firmware publishes a complete MADT *and* an MP
table containing one processor, so asking the older interface first means never
seeing more than one CPU. VMware published the real count, which is why the
resulting triple fault only ever appeared there.
**`main()` names no display driver.** Whichever driver owns the framebuffer the
loader left running publishes it with `gfx_attach_scanout()`, and the console
and crash screen draw on that. See [graphics.md](graphics.md).
### Memory sizing
`PHYSTOP` is a variable read from the firmware's map, not a constant. It used to
be `0xE000000` — 224 MB — inherited from xv6, and it was not a limit on what the
kernel used but an **assertion that exactly that much RAM existed**. Less and the
allocator walked off the end of memory; more and it was invisible.
Three details survive that fix:
- It takes the end of the usable region **containing the kernel**, not the
highest usable address anywhere. A memory map has reserved holes, and freeing
a single range from the kernel's end to a global maximum puts firmware tables
on the free list — failing later, elsewhere, looking like anything but a
memory map problem.
- The usable regions are *also* kept as a list, and `kinit1()` frees each one.
One range ending at a top address is what a BIOS map looks like and what UEFI
never does: OVMF drops an ACPI NVS region at `0x800000`, so the run containing
the kernel ends at 8 MB and everything above the hole was invisible.
- The ceiling is 3 GB, because `kvmalloc()` maps 0–4 GB as four 1 GB pages and
marks the top one uncached for PCI BARs. The floor is 24 MB, and below it the
kernel prints why and stops rather than triple faulting in silence.
## 4. SMP
`startothers()` wakes each application processor through the local APIC, pointing
it at a trampoline copied to `0x7000`. Each AP runs `mpenter()` → `mpmain()`,
reports `cpu%d: starting`, and enters the scheduler.
This was broken from the project's beginning and is fixed. Five distinct bugs,
each hidden behind the one before it — the details are in
[../history/kernel.md](../history/kernel.md), and the one worth carrying
forward is in [kernel.md](kernel.md#per-cpu-state): **`__thread` is wrong for
per-CPU kernel data.**
APs are counted on the boot processor *after* each one reports in, so the count
cannot include a processor that started and died.
## 5. The first process
`sinit()` builds PID 1 by hand: a page directory, `initcode` mapped at 0, 32
pages of stack, a trapframe pointing at user mode, and session and process group
both set to 1.
It does **not** mount the root filesystem, and cannot: mounting reads blocks,
`bread()` sleeps, and sleeping needs a process context. So the mount happens in
`forkret()`, the first time any process is scheduled — guarded by a `static int
first`.
That is also where PID 1's working directory is re-pointed at the real root,
and the reason is worth keeping:
> `sinit()` sets `cwd = namei("/")` before any filesystem is mounted. The old
> v1 root survived that because `iget(ROOTDEV, ROOTINO)` is two compile-time
> constants and touches no disk. SchFS has no constant to offer — the root
> object id comes off the uberblock — so that early `namei("/")` builds an inode
> from an unmounted volume. Nothing complains, because absolute paths never
> consult cwd: `exec("/bin/init")` works fine. It is the first *relative* lookup
> that fails, and in init that is `open("console")`. No console means no fd
> 0/1/2, so every shell it spawns exits immediately and init respawns it
> forever. The symptom is a silent boot that reads like a hang and is not one.
`initcode` then execs `/bin/init`, and [init.md](init.md) takes over.
## 6. What the screen shows
**The display is not the kernel log.** `cons_serial_only` starts *set*: every
boot message goes out the UART and nothing goes to the framebuffer. The loader's
splash therefore survives the whole of boot rather than being wiped a few
hundred milliseconds in, and `console_takeover()` clears the flag on the first
`exec()`, when userspace genuinely starts.
Nothing is lost — the serial line is where `bootcheck` reads from and where you
want the messages when something is wrong — and the F8 verbose boot never sets
the flag, so "show me everything" still works.
---
## What it does not do
- **No BIOS path.** A machine without UEFI does not boot SchOS.
- **The root device is compiled in.** Booting from a USB disk needs the root
device to be selectable, which it is not; the USB mass-storage driver works
and a superblock reads back through `bread()`, but nothing can mount it.
- **`make ramroot` is the only way to boot with no data disk.** It links a
filesystem image into the kernel binary, which costs boot time and carries a
licensing constraint — see [../licensing.md](../licensing.md) §4.
- **The 32-bit multiboot entry is dead code.** It is left in place deliberately:
deleting kernel entry code is its own piece of work with its own way of
failing silently.
- **`make qemu-uefi` under OVMF reports less memory than the machine has** in
some configurations. The loader's map is correct; the fix went in with the
region list above, and any remaining case is a fragmentation pattern not yet
seen.

104
docs/architecture/devices.md Normal file
View file

@ -0,0 +1,104 @@
# The device model
How hardware gets found, claimed and used, and what the rest of the kernel sees
of it.
Writing a driver is [../guides/adding-a-driver.md](../guides/adding-a-driver.md).
Moving drivers out of the kernel entirely is [../plan/hybrid.md](../plan/hybrid.md).
---
## One list of everything attached
There is one list of everything attached, whatever bus found it:
`include/device.h` and `kernel/dev/device.c`, readable at `/proc/devices`.
```
name bus class driver device
kbd0 platform input ps2kbd PS/2 keyboard
mouse0 platform input ps2mouse PS/2 mouse
uart0 platform other uart8250 16550 serial port
ide0 platform storage ide ATA disk controller
vga0 pci display vga display controller
```
Before this there were two registries in `kernel/drivers/registry.c` — an array
of PCI drivers and an array of USB drivers — with nothing that knew about both.
That is sufficient for *binding* and useless for the three things wanted next.
**Enumeration.** "What is in this machine, and what claimed it" had no answer
covering a USB disk and a PCI display at once. A device manager is exactly that
question.
**Classes.** A compositor should ask for *the display* and not care whether that
is the VBE framebuffer or virtio-gpu. Without a class every consumer hardcodes a
driver, and adding the second driver means editing the consumers — which is how
a framebuffer address ended up hardcoded in three places here once already.
**A boundary for drivers that are not compiled in.** They have to attach
somewhere that is not an array fixed at link time.
Two vocabularies are kept deliberately separate: a device's **bus** is
provenance (`platform`, `pci`, `usb`), its **class** is capability
(`display`, `input`, `storage`, `net`, `bus`). A USB disk and a SATA disk are
both `storage` despite sharing no numbering at all. Where a driver claims a
device, the driver's opinion wins over the descriptor's: `struct usb_driver` has
a `devclass` field separate from the USB interface class, because the descriptor
says what the device *claims* to be and the driver knows what it *made* of it.
It is a flat list with a bus tag, **not** a device tree. Nothing yet needs to
walk a hierarchy, and a tree nobody traverses is just a harder list. Hot-unplug
would need more; the `in_use` flag is the hook.
Platform devices are registered in one place
(`platform_register_devices()` in `registry.c`) rather than inside each driver's
init, for the same reason the driver arrays are written out by hand: a list you
can read beats a list assembled by side effects.
---
## Linux drivers
**Decided 2026-08-08.** Linux drivers are not shipped with SchOS and never
will be. The user may install one on their own machine, through a device
manager or equivalent, and SchOS will load it.
The licensing reasoning matters and is worth stating exactly, because it is
easy to get wrong in both directions:
- **Distribution is what triggers the GPL.** Shipping a GPL-2.0-only driver in
an Apache-2.0 image is what cannot be done — see [../licensing.md](../licensing.md)
§2. A user obtaining a driver themselves and loading it on their own machine
is their act, not ours, and is not a distribution by us.
- **Implementing a loader is not copying.** Writing support for a module format
and a driver ABI from published documentation produces our own work. It is a
clean-room implementation, and it stays one only if it is written from
documentation rather than transcribed from Linux source. Reading Linux to
understand *hardware* is fine; translating its code is not, whatever the
identifiers say.
So the rule for this subsystem is the same as everywhere else in the tree: the
loader is ours and Apache-2.0, what it loads is the user's business, and nothing
GPL ever enters an image we build. `RAMDISK_EXCLUDE` and `NOTPORTS` are the
existing mechanisms for keeping that true mechanically rather than by intention.
---
---
## What it does not do
- **It is a flat list, not a device tree.** There is no parent/child nesting,
only a bus tag. Nothing yet needs to walk a hierarchy, and a tree nobody
traverses is just a harder list.
- **No hot-plug.** Devices are registered at boot and never removed; the
`in_use` flag is the hook a hot-unplug path would need, and nothing sets it
for that purpose.
- **`NDEVICE` is 32**, compile-time, like every other table in the kernel.
- **Drivers are fixed at link time.** There is no module loader, and nothing can
be installed after the build — the class vtable is the attachment point a
loader would use, and it is the only part that exists.
- **Only two class vtables are defined**: `display_ops` and `netdev_ops`.
Storage and input devices appear in the list with `ops = 0` and are reached
through their own subsystems instead.

170
docs/architecture/filesystem.md Normal file
View file

@ -0,0 +1,170 @@
# The VFS
The layer between `open(2)` and a filesystem. SchFS is described separately in
[schfs.md](schfs.md); this is the machinery it plugs into, plus the buffer cache,
mounts, and procfs.
---
## `struct fs_ops`
A filesystem is a vtable, chosen by `iget()` from the device number and never
null once an inode is held.
```c
struct fs_ops {
const char *name;
void (*iload) (struct inode *); // fill type/size/nlink from disk
void (*istore) (struct inode *); // write them back
void (*ifree) (struct inode *); // last link gone, release it
int (*read) (struct inode *, int user_dst, uint64 dst, uint64 off, uint n);
int (*write) (struct inode *, int user_src, uint64 src, uint64 off, uint n);
void (*truncate)(struct inode *, uint64 newsize);
struct inode *(*dirlookup)(struct inode *dp, char *name, uint *poff);
int (*dirlink) (struct inode *dp, char *name, uint64 inum);
int (*dirunlink)(struct inode *dp, char *name, uint off);
int (*dirempty) (struct inode *dp);
struct inode *(*ialloc)(uint dev, short type);
int (*getdents)(struct inode *dp, char *out, int cap, uint64 *off);
};
```
Two implementations exist: `schfs_ops` (`kernel/fs/schfs/vfs.c`) and `procfs_ops`
(`kernel/fs/procfs.c`).
`poff` is deliberately **an opaque cookie** rather than a byte offset —
meaningful only to the same filesystem's `dirunlink`. A directory does not have
to be an array of fixed-size records for this interface to work.
`ialloc` creates an object **with no name**. The caller sets `nlink` and calls
`dirlink`; until it does, nothing can reach the object. If the caller dies in
between, the result is an unreferenced record that is harmless and, in SchFS,
never reaches the disk anyway — nothing is durable until the transaction commits.
### The one place the models disagree
The VFS inherited xv6's shape: allocate an inode, fill it in, link a name to it.
SchFS has no free-standing inode — an object exists because a record with its id
exists.
`nlink` is the sharper disagreement. **The VFS adjusts it itself** and then calls
`iupdate`, so SchFS's own link accounting is deliberately bypassed in the VFS
path: `schfs_dir_add`/`schfs_dir_del` are used rather than `schfs_link`/
`schfs_unlink`, and `iput`'s `ifree` hook releases the object once the VFS has
decided there are no names left.
> Two pieces of code counting links independently is precisely how a filesystem
> loses a file that is still referenced.
`nlink` is an `int`, not a `short`. SchFS stores a `uint32` and the narrower type
truncated it, so a file past 32767 names came back with a *negative* count — and
`iput` frees an object whose count reaches zero. Signed on purpose: `nlink--` on
a count that is somehow already zero should land somewhere a check can see it,
not wrap to four billion.
## Path resolution
`namex()` walks a path one component at a time. It was refactored into
`namex_from(struct inode *start, ...)`, parameterising what used to be a
hardcoded `idup(proc->cwd)`.
That refactor is what made the `*at()` family work. Every one of them — `openat`,
`fstatat`, `unlinkat`, `renameat`, `linkat`, `symlinkat`, `mkdirat`, `mknodat`,
`readlinkat`, `faccessat` — previously handled only two cases: `AT_FDCWD`, or an
absolute path with `dirfd` ignored. A **real** dirfd with a **relative** path is
the standard, efficient way any directory-tree walker avoids rebuilding full
path strings, and it was either rejected outright or — worse, in `openat` —
silently resolved against `proc->cwd`, which can open the wrong file entirely.
`sys_fs.c` now has `at_dir(dirfd)`, `at_namei()` and `at_nameiparent()`, and
every `*at()` call uses them.
## The buffer cache
`kernel/dev/bio.c`, xv6's design unchanged: a fixed pool of buffers, an LRU
list, one sleeplock per buffer. `bread()` returns a locked buffer; `bwrite()`
flushes it; `brelse()` releases.
**Sleeping is the point.** A buffer lock is held across a disk read, which is
why every path that touches a filesystem must be able to sleep — and why the
root mount happens in `forkret()` rather than in `main()`.
## Mounts
`mount()` and `umount2()` at their Linux numbers, with a mount table consulted
during path resolution. `/proc` is mounted by init:
```c
mount("proc", "/proc", "proc", 0, 0);
```
## procfs
`kernel/fs/procfs.c`. A real filesystem behind the same vtable, not a set of
character devices.
**The reason it had to become one is `ps`.** /proc used to be `mknod`'d nodes —
init ran `mknod("/proc/uptime", 6, 2)` and friends, so every file was a real
inode on the root filesystem with a major/minor behind it. That is fine for a
fixed list and cannot work at all for the thing /proc is mostly for: **one
directory per process**. You cannot `mknod` a directory for a process that does
not exist yet and will be gone in a minute. `ps` listed `/proc`, got no numbers,
and reported nothing. It was never a `ps` problem.
Nothing is stored. Directory listings are generated from the process table when
asked; file contents when read.
Three things worth knowing:
- **The split is deliberate.** `kernel/fs/procfs.c` owns the *namespace* — what
exists. `kernel/dev/procfs.c` keeps the *generators* — what the global files
say. They change for different reasons, and `proc_gen_global()` is the only
thing that crosses between them.
- **procfs never walks the process table.** `proc_snapshot()` copies what is
needed out from under `ptable.lock`, so a process that exits between the
readdir that listed it and the open that reads it produces a stale line rather
than a use-after-free.
- **Sizes are reported as zero**, exactly as Linux does. A synthetic file has no
length until it is generated, and generating every one on every `stat()` to
fill in a number almost nothing reads would be absurd.
### What is there
Per-process directories, plus the global files: `meminfo`, `uptime`, `cpuinfo`,
`version`, `stat`, `loadavg`, `mounts`, `filesystems`, `schfs`, `pci`,
`interrupts`, `devices`, `netif`, and `sysrq`.
**/proc is read-only except `sysrq`**, which is the one deliberate write target,
and the code says so in one place. Marking every procfs inode read-only once
broke `echo s > /proc/sysrq` and took out three USB checks within a minute —
they drive the disk through sysrq.
`sysrq` keys: `c` crash on purpose, `u` / `s` storage checks, `d` display
selftest, `t` text specimen, `n` the netdev ARP selftest.
---
## What it does not do
- **No page cache.** `mmap` of a file reads the whole thing in at map time, and
writable `MAP_SHARED` of a file is refused rather than faked. This is the
single largest gap in the layer.
- **No symlinks in procfs**, which is why `/proc/<pid>/exe` is a regular file
and `ps -o comm` prints `[init]`.
- **No `AF_UNIX` socket inodes.** Socket names live in a table in
`kernel/net/unix.c`, so `bind()` creates nothing that can be `stat()`ed or
`unlink()`ed.
- **No unified inode cache across filesystems** beyond the single `itable`.
- **A known, un-root-caused hang:** heavy `ls -lR` use across many commands in
one long shell session can eventually make a *later*, unrelated directory
hang; the identical command succeeds on a fresh boot. It is not inode or file
table exhaustion — both fail cleanly and neither fired. It smells like a lock
or refcount leak in the `*at()` paths or in toybox's own fd handling. The next
step is instrumenting `itable`/`ftable` occupancy across a long session to see
what is climbing.
- **`sys_delete`/`sys_show`/`sys_showstat`** are orphaned entries from the old
v1 filesystem's recycle bin. Nothing reaches them and they die with `fs.c`.

233
docs/architecture/graphics.md Normal file
View file

@ -0,0 +1,233 @@
# Graphics, text and the console
Four things share this area and they are deliberately separate: the kernel's own
drawing surface, the compositor's display interface, the text-rendering layer,
and the crash screen.
---
## The two display interfaces, and why there are two
This distinction causes confusion every time somebody meets it, so it is the
first thing here.
| | [`gfx.h`](../../include/gfx.h) | [`display.h`](../../include/display.h) |
|---|---|---|
| **What** | The kernel's own drawing surface | The compositor's interface |
| **About** | Pixels and text | *Frames* |
| **Operations** | `draw_pixel`, `fill_rect`, `putc`, `text_at` | `acquire`, `present`, `wait_vblank` |
| **Buffering** | None. Direct writes to the scanout. | Double-buffered |
| **Users** | The console and PSOGM | Schcom |
| **Must work when** | **Nothing else in the system does** | Userspace is healthy |
That last row is the reason they are not one interface. The console and the
crash screen cannot depend on a back buffer being allocatable, on a driver
having a working present path, or on anything that could be the thing that just
broke.
### `gfx` — the scanout
A driver that owns a linear framebuffer publishes it with
`gfx_attach_scanout()`, and everything above draws through `gfx_*` calls.
This used to be `kernel/dev/vga.c`, and every consumer called it by that name:
`panic.c` drew the crash screen with `vga_fill_rect`, the console printed with
`vga_putc`, `/dev/fb0` handed out `vga_fb_phys()`. None of that has anything to
do with a VGA — it is bitmap drawing over a linear framebuffer, and the only
VGA-specific part was where the framebuffer came from. A second display driver
is now a second caller of one function rather than an edit to the crash screen.
`main()` names no display driver. Whichever one owns the framebuffer the loader
left running says so before any bus is enumerated, because the console draws
long before PCI is walked.
### `display` — the class
`struct display_ops`, found with `device_find_class(DEVCLASS_DISPLAY)`.
`acquire()` allocates the back buffer **on first call, never at boot** — it is
around two megabytes at 800×600, and a machine whose compositor never starts
should not pay that. A driver that cannot double-buffer returns 0, and the
caller draws directly and accepts tearing rather than not drawing at all.
`wait_vblank` is optional for the same reason.
`echo d > /proc/sysrq` drives the whole path and draws a test pattern.
## Drivers
| | |
|---|---|
| **VBE framebuffer** (`gpu/vga/`) | Whatever the loader set up. Always present; this is what the splash and the console are drawn on. |
| **virtio-gpu** (`gpu/virtio_gpu/`) | 2D, written from the OASIS specification. Full path: `GET_DISPLAY_INFO`, `RESOURCE_CREATE_2D`, `RESOURCE_ATTACH_BACKING`, `SET_SCANOUT`, `TRANSFER_TO_HOST_2D`, `RESOURCE_FLUSH`. |
virtio-gpu is offered the bus **before** the VBE driver, so on a machine with
both, the device with real mode setting and an explicit flush wins the display
class. `vga_pci` still claims the stdvga card underneath, which is what the boot
splash is drawn on — so `-vga std` must stay on the QEMU command line even when
adding `-device virtio-gpu-pci`, because the loader's splash goes through VBE
and there is no virtio path that early.
`/dev/fb0` speaks the Linux fbdev interface: geometry through the fbdev ioctls,
and `mmap` of the real framebuffer at its real physical address.
> `mmap` of `/dev/fb0` is also **the kernel's only chance to notice that the
> screen no longer belongs to the text console** — after that the drawing
> happens with no further syscalls. `fb_mapped()` is called there, and
> `fbclose()` puts it back.
## Text
[`include/text.h`](../../include/text.h) + `kernel/dev/text.c`.
**The font was never the problem — the character-cell grid was.** The console
draws text by indexing `font_data[256][16]` and stepping the cursor eight
pixels. A grid cannot do variable-width glyphs, cannot position text anywhere
except on a multiple of eight, cannot anti-alias, and cannot represent a
codepoint above 255. Swapping the bitmap for a nicer bitmap changes none of that.
So the text layer fixes three things:
- **Glyphs have metrics** — size, bearing, advance. That *is* variable-width
text; everything else follows from it.
- **It draws into a `struct surface`, not into the screen.** A compositor draws
off-screen and presents; rendering that writes straight to the VGA aperture is
unusable to it.
- **The font is behind `struct font_ops`.** The bitmap font is one
implementation and a rasteriser is another, arriving without any caller
changing — glyphs carry 8-bit coverage, so an anti-aliased outline renderer
already has somewhere to put its output.
### The TrueType rasteriser
`kernel/dev/ttf.c`: `glyf` outlines, composite glyphs, `cmap` format 4,
anti-aliased scanline fill, with a host harness at `tools/ttftest.c`. A
direct-mapped glyph cache over ASCII in a 48 KB pool.
**The typefaces**, both licence-checked at source:
| Font | Where | Licence |
|---|---|---|
| **JetBrains Mono** | terminal and TTY | SIL OFL 1.1 |
| **Roboto** | desktop and compositor | Apache-2.0 |
JetBrains Mono declares **no Reserved Font Name**, which matters more than it
sounds: pre-rasterising a font into bitmaps at build time is a modification, and
an RFN would forbid keeping the name on the result.
## The console
`kernel/core/console.c`. A cursor, scrolling, and an ANSI escape parser
including SGR colours and the erase operations.
**`cons_serial_only` starts set.** The display is the product; the serial line
is the diagnostics. A machine that boots by printing `pci: 6 devices` and
`vga: display controller 1234:1111, aperture fd000000` to the screen the user is
looking at is showing them a developer's log, and that single fact is most of
what makes an OS look like a hobby project. Everything still goes out the UART.
`console_takeover()` clears the flag on the first `exec()`.
The console still uses the 8×16 cell grid. Porting it onto the text layer and
deleting the grid is open work.
### Input
Keys reach both `/dev/input/eventN` and the tty line discipline, which is
correct for two consumers of one keyboard and completely wrong for a full-screen
program — the shell behind it quietly accumulates every keystroke and runs the
lot at the next Enter.
`EVIOCGRAB` is the fix and it is a primitive rather than a patch. Three details
matter more than the feature:
- **The grab is released by the `close` hook, not by the program.** A program
that grabs the keyboard and then crashes would otherwise leave the console
permanently deaf, with no way to type the command that fixes it.
- **Modifier state is still tracked while grabbed**, or releasing the grab hands
the line discipline a stuck Shift.
- `/dev/input/eventN` **drops its backlog on open** and only queues while
something has it open. Without that, opening the device replayed the
keystrokes of the command line that had launched the program.
## PSOGM — the crash screen
`kernel/core/panic.c`. The Purple Screen of Guru Meditation. Built around three
decisions:
**It does not clear the screen.** Whatever was displayed stays, dimmed and
pushed back, with the report floating over it. A crash screen that wipes the
display tells you the machine is gone; one that leaves your work visible tells
you it stopped — which is both truer and much less alarming.
**It is not frightening.** No red, no capitals, no "FATAL". A calm sentence, a
face that is obviously a cartoon, then the facts. Whoever is reading this is
already having a bad time.
**It says considerably more than it needs to.** The screen is the only artefact
of a panic — no core dump, no log to go back to — so everything cheap to collect
is collected: stop code, fault address, which CPU, which process, how far into
the boot, where the kernel is loaded, and a call chain. Anything left out is
gone. All of it goes to the serial line in the same order, which is what
`bootcheck` reads and what you get when the crash happens before there is a
framebuffer at all.
Once painted, `panicked` is set and `consputc` spins rather than writing, so
nothing can scribble over the report.
### The QR code
`kernel/core/qr.c`. **A panic report is worth nothing if it cannot leave a
machine whose network stack is dead**, and a phone camera is an out-of-band
channel needing no driver on our side.
It encodes the **report itself**, not a URL — it reaches nothing. Deliberately
small in three ways: byte mode only, EC level L only (the screen is a clean,
evenly lit, high-contrast source; the damage QR's stronger levels protect
against does not apply), and versions 1–6 only, stopping short of version 7's
extra version-information blocks and version 10's wider count field.
134 bytes is a large crash report. If a payload ever needs more, the honest fix
is to shorten the report.
It draws unconditionally rather than on a keypress, which is better: it costs
nothing, and requiring input would mean the one case where input is least
trustworthy is also the case where the report is hardest to retrieve. Verified
by `make crashshot && make crashqr`, which decodes the symbol out of the
captured framebuffer.
## One palette
[`include/theme.h`](../../include/theme.h). There were three unrelated designs
and the seams showed: the splash was deep purple with gold text deliberately
matching the *panic* screen, the console was cream with near-black text, and so
a healthy boot was dressed as a crash and then abruptly became an amber
terminal.
> **The rule: normal states are calm and dark; only failure is loud.**
Boot, console and shell share `SCH_BG`/`SCH_FG` — near-black with a slight blue
cast rather than pure `#000`, because pure black on an LCD reads as "the machine
is off". PSOGM keeps the full-bleed purple entirely to itself, which is what
makes it mean something.
---
## What it does not do
- **The console still has its cell grid.** The text layer exists and the console
does not use it; the rasteriser exists and no font is loaded from disk.
- **No mode setting.** Resolution is whatever the loader asked for. There is no
`/dev/dri`-style device and no way to change it at runtime.
- **No double buffering in practice.** `display_ops.acquire` is specified and
the compositor needs it; virtio-gpu binds but exposes no modeset API, and
`wait_vblank` is unimplemented everywhere.
- **No hardware acceleration of any kind**, and none is planned before 2D blit.
- **No CFF/Type 2 charstrings**, so OTF fonts do not render — that is a separate
`font_ops` provider and a bytecode interpreter, not an extension of the `glyf`
path. TrueType hinting is deliberately out of scope.
- **No text layout library.** Advance widths exist per glyph; kerning, shaping
and line breaking do not.
- **The glyph cache covers ASCII only**, direct-mapped. A codepoint outside it
is rasterised every time.
- **PSOGM renders and stops.** The interactive crash mode described in
[../plan/roadmap.md](../plan/roadmap.md) — walk the stack, read another
process's memory, dump page tables — is designed and not built.

155
docs/architecture/init.md Normal file
View file

@ -0,0 +1,155 @@
# `schinit` — PID 1
Shaped after launchd rather than sysvinit. The reason is not the configuration
format.
---
## Why launchd
**Demand launching.** The supervisor holds the listening socket or watches the
path, and starts a job when something actually arrives.
That inverts the dependency graph. A job stops needing to declare what must be
running before it, because the thing it depends on answers on first use — and
runlevels stop being a concept at all.
SchOS has the two pieces that makes possible: `AF_UNIX` sockets and a real
`/proc`. **This milestone is the supervisor; demand launching is the next one.**
Socket sections are parsed and remembered but not yet held.
## Constraints on the program itself
`cmd/schinit/main.c` links against `lib/`, which is raw syscall stubs. **There is
no libc here** — no `malloc`, no `printf`, no `strlen`. Everything is static
storage and hand-rolled, which is also why the limits are constants rather than
growable:
| | |
|---|---|
| `MAXJOBS` | 32 |
| `MAXARGS` | 16 |
| manifest size | 4 KB, read into one static buffer |
## Bootstrap
Before any job runs, PID 1 creates the device nodes and mounts that a job needs
in order to have somewhere to send its output. **This cannot itself be expressed
as a job** without a bootstrapping problem, which is why it is a function and not
a manifest.
Console first, as fd 0/1/2, so every message after it has somewhere to go. Then
the `/dev` nodes — minor numbers under major 1 follow Linux's mem-major layout
where it costs nothing (`null=2`, `zero=5`, `full=7`, `random=8`, `urandom=9`) —
then `/dev/input/event0` and `event1`, the raw disks, and `/proc`.
Two details that look like carelessness and are not:
**The nodes are created unconditionally, including for absent hardware.** Opening
`/dev/hdb` with no drive present fails with `ENXIO`, which is a better answer
than a missing path. It matters more than it looks: with no node,
`echo x > /dev/hdb` silently creates an ordinary *file* of that name and appears
to succeed.
**The `mknod`'d `/proc` files still happen and are then covered by the mount.**
A failure to mount degrades to the old behaviour rather than losing `/proc`
entirely.
## Job manifests
Every `*.job` file in `/etc/schinit` is loaded at boot, found with `getdents64`
rather than a hardcoded list — so dropping a file in the directory is all it
takes to add a job.
Sectioned `key = value`, `#` starts a comment, keys are case-insensitive
(a manifest may say `KeepAlive`, `keepalive` or `KEEPALIVE`; keys are for humans
and this is not a language).
| Key | |
|---|---|
| `Label` | A reverse-DNS name, as launchd uses. Identity, not a path. |
| `Program` | The binary to execute |
| `Args` | argv, space separated. argv[0] is conventionally the basename, so `Args = sh` for `/bin/sh` |
| `RunAtLoad` | Start it when schinit loads |
| `KeepAlive` | Restart it when it exits |
| `Disabled` | Load it but do not start it |
| `ThrottleInterval` | Seconds between respawns (default 1) |
| `ControllingTerminal` | Claim the console |
| `StandardIn` / `StandardOut` / `StandardError` | Paths to open as fd 0, 1, 2 |
| `[Socket <name>]` → `Path` | Parsed, recorded, not yet honoured |
**Unknown keys and unknown sections are ignored rather than fatal.** A manifest
written for a newer schinit must not stop this one booting the machine.
## Starting a job
Each job gets **its own session** (`setsid`), so it cannot reach back and signal
the rest of the system through a shared controlling terminal.
That is exactly what makes `ControllingTerminal` necessary, and the interaction
is the least obvious thing in the program:
> A fresh session has no controlling terminal, and the console only delivers
> input to the foreground process group it is tracking. Without
> `ControllingTerminal = yes`, the shell comes up, echoes every keystroke through
> the line discipline, and executes none of them — which looks like a hung shell
> and is really a terminal that is not listening to it.
It is **off by default**. A daemon has no business seizing the console, and
launchd's model is that the console is something a job asks for.
`exec.c` enables the Linux ABI automatically for musl binaries, so a job must
not call `enable_linux_abi()` itself — it would renumber the syscalls the child
still has to make.
## Supervision
One loop around `wait()`.
**It reaps whatever exited, including processes reparented here when their own
parent died.** Those are not jobs, `job_by_pid` returns 0 for them, and that is
the whole handling they need.
A `KeepAlive` job is restarted, subject to `ThrottleInterval` — a job that exits
immediately and forever would otherwise spin the machine. launchd defaults to
10 seconds; 1 is enough here, where the common case is a shell someone just
exited on purpose and expects back.
When `wait()` returns -1 there is nothing left to wait for, which should not
happen while any job is alive. It sleeps rather than spinning, so a mistake
there costs an idle machine and not a hot CPU.
## The fallback
With no manifests found, schinit says so and starts `/bin/sh` on the console
anyway.
> An unbootable-looking machine because a manifest was missing would be a poor
> trade for purity.
## PID 1 cannot be killed
Enforced in the kernel, not here. `init_ignores()` discards any signal whose
disposition is the default action when the target is PID 1, **including
`SIGKILL`** — see [kernel.md](kernel.md#signals) for why Ctrl-C is the input that
finds that window. A handler init did install is still delivered.
---
## What it does not do
- **Socket activation is not implemented.** The manifest key is parsed and the
path remembered; nothing holds the socket, so demand launching — the entire
reason for copying launchd — does not happen yet.
- **No `schinitctl`.** Jobs cannot be started, stopped, reloaded or inspected at
runtime. The only interface is rebooting with a different manifest.
- **The job table is invisible.** Nothing reports what is loaded, what is
running, or how many times something has respawned.
- **No dependency ordering of any kind**, deliberately — demand launching is
meant to make it unnecessary, and until that exists there is nothing in its
place either.
- **No log capture.** `StandardOut` is a path, so a job's output goes wherever
the manifest says and nothing aggregates it.
- **A crash in schinit is fatal to the machine**, because `exit()` panics on
initproc. Fixing that is [../plan/init.md](../plan/init.md).
- **32 jobs, 16 arguments, 4 KB manifests**, all compile-time.

666
docs/architecture/kernel-history.md
View file

@ -1,666 +0,0 @@
# Kernel history — what broke, and what the fix actually was
Not a design document. This is the record of bugs that were expensive to find
and whose causes are not visible from the code that fixed them, plus the few
subsystem write-ups (bootloader, boot speed, naming) that have nowhere better
to live.
It is worth keeping because every entry here is a thing that looked like a
different problem than it was.
> **Where the old design plan went.** Sections 1-6 of this file used to hold a
> "Segregated Kernel" design: a Ring 0 Alpha / Ring 0 Beta split with driver
> modules isolated by CR3 switching, from an `IDEA.txt` that no longer exists.
> None of it was ever implemented, and it is superseded by
> [schapi.md](schapi.md) — a macOS-shaped system answers driver isolation with
> userspace drivers and a personality boundary, not a bespoke in-kernel ring
> split. It also referenced a Zig toolchain that is now clang/LLVM. Removed
> rather than left to be mistaken for a plan.
---
## 1. Fixed: boot didn't reliably reach a working shell
Investigated 2026-07-05, before phase 1 started (predates this design, not
caused by it). Originally manifested as `ksh` (mksh) crashing on every
launch, apparently inside musl's mallocng allocator (`get_meta`,
`src/malloc/mallocng/meta.h`, deliberately executing `hlt` on detecting
"corrupted" heap metadata — a `trap 13` GPF at a fixed rip). It wasn't a
musl bug. Six real kernel bugs were found and fixed across three
investigation passes, and mksh boots and runs cleanly with all of them in
place — the calloc assertion was a symptom of real memory/pointer-handling
bugs downstream in the kernel, not anything musl or mallocng did wrong.
Fixed:
- `copyuvm()` (`kernel/arch/x86_64/vm.c`, used by `fork()`) never copied
`mmap()`-backed `proc->vma[]` regions into the child — only `[0, sz)`.
Fixed via new `copy_vma_range()`, called from `fork()` (`kernel/proc/proc.c`).
- `kmem.freelist` (`kernel/core/kalloc.c`) could get corrupted by a wild
write elsewhere and would silently hand back a bad pointer, panicking far
from the actual cause. Added a canary check (`check_run()`) at the top of
both `kalloc()`/`kfree()` so corruption panics immediately at the point
it's touched, with a clear message.
- No TLB invalidation existed anywhere except the full `lcr3` reload on
context switch — `mappages()`/`vm_unmap_range()` never called `invlpg`.
Fixed by adding `invlpg()` (`include/x86.h`) to both functions.
- **`MSR_STAR` was misconfigured** (`syscallinit()`, `kernel/arch/x86_64/vm.c`):
`SYSRETQ` computes the returned `cs`/`ss` as `STAR[63:48]+16`/`+8`, which
requires the *compat* 32-bit code selector (`SEG_UCODE32`) in that field —
the kernel was putting the real 64-bit `USER_CS` there instead. Every
`sysretq` silently loaded an invalid `cs` (`0x3b`, pointing at the TSS
descriptor)/`ss` pair; QEMU's TCG doesn't validate segment descriptors on
`SYSRET`, so this went unnoticed until a hardware interrupt forced a
strictly-validated `IRETQ` with that bogus `cs` still loaded, which
correctly `#GP`'d.
- `sys_mmap`'s non-`MAP_FIXED` path (`kernel/sys/sys_mem.c`) assigned a fixed
8MB virtual-address window per `vma[]` slot regardless of the requested
length. A single allocation over 8MB (e.g. `realloc()` growth past that
size) mapped straight into a neighboring live mapping and hit
`mappages()`'s `panic("remap")`. Replaced with a real bump allocator
(`proc->mmap_next`, `include/proc.h`), so every mapping gets its own
disjoint, exactly-sized range regardless of size.
- **`uva2ka()`** (`kernel/arch/x86_64/vm.c`) returned only the page-aligned
base address of a user pointer's page, silently dropping the in-page byte
offset. Direct callers (`sys_ioctl`'s `TIOCGPGRP`/`TCGETS`, `sys_getcwd`,
signal masks, `times()`, `sysinfo()`, ...) read/wrote near the wrong
address whenever the pointer wasn't page-aligned — i.e. almost always.
Fixed at the source so every caller benefits.
- **`fetchint`/`fetchaddr`/`fetchstr`/`argptr`** (`kernel/sys/syscall.c`)
rejected any address `>= proc->sz`, wrongly assuming every valid user
pointer lives in the heap. Stack addresses and `mmap()`'d buffers (used
constantly by real programs, e.g. path-resolution buffers) live above
`proc->sz` and were being rejected outright. Fixed to rely solely on
`uva2ka()`'s page-table validation.
Bringing in toybox (see "Coreutils" below) surfaced four more, all fixed:
- Linux syscall 58 (`vfork`) wasn't in the syscall-ABI translation table at
all (`kernel/sys/linux_abi.c`) — every external command spawn from a
program that prefers `vfork()` over `fork()` failed outright. Mapped to
the existing `fork()` (a fully correct, if slightly slower, substitute).
- `sys_stat`/`fstat`/`fstatat` (`kernel/sys/sys_fs.c`) filled in SchOS's
own native `struct stat` layout regardless of the caller — musl/toybox
parse a fixed Linux ABI byte layout instead, so `st_mode` was garbage and
tools like `ls` couldn't tell files from directories, or executables from
non-executables. Added `struct linux_stat` + `stati_linux()`, gated on
Linux-ABI mode (`proc->syscall_map`).
- `sys_open`/`sys_openat` required `omode == O_RDONLY` *exactly* to open a
directory, but `opendir()`-style calls legitimately OR in `O_DIRECTORY`/
`O_CLOEXEC` alongside it. Fixed to check just the access-mode bits
(`O_ACCMODE`, added to `include/fcntl.h`).
- `sys_getdents64` read one `struct dirent` (16 bytes) at a time — a full
`bmap()`+`bread()` per slot. Directories here are hash-indexed (see
`tree_search()`/`tree_expand()` below), so a directory with a few dozen
real entries can have a size in the hundreds of KB, nearly all empty
slots — this made listing such a directory look hung (it wasn't a
deadlock, just thousands of redundant single-slot disk reads). Rewrote it
to read a full block at a time and resolve each entry's type via
`iget(inum)` (exported from `fs.c`, it was `static`) instead of
re-searching the whole directory by name with `dirlookup()` for every
entry. That rewrite's first version introduced a **new** self-deadlock:
the "." entry's inode number equals the directory's own, and `iget()`
correctly hands back the *same, already-locked* `struct inode*` — calling
`ilock()` on it again sleeps forever on a lock the process already holds.
Fixed by special-casing `xde->inum == ip->inum` (we already know it's a
directory, no need to re-lock at all).
A synthetic stress test (interleaved malloc/free, realloc growth crossing
`MMAP_THRESHOLD`, calloc, large allocations) runs to completion with zero
crashes, and mksh now boots straight to an interactive prompt, runs external
programs, pipes, and `ls`/`mkdir`/`grep` without issue.
## 2. Naming standardization
SchOS standardizes on exactly one name per role, regardless of the
underlying implementation: the shell is always installed and exec'd as
`/bin/sh` (currently mksh, built from `ports/mksh/`), and PID 1 is always
`/bin/init`. `cmd/init/main.c`'s exec loop only ever tries `/bin/sh` — no
ksh/mksh/dash/zsh/other-name fallbacks. dash was tried first
(BSD-licensed, minimal) while the boot crash above was still open, then
replaced with mksh once the underlying kernel bugs were fixed and dash's
lack of interactive features (history, completion, line editing) made it
feel unusably bare for day-to-day use; the dash port's source tree was left in
place rather than deleted, in case it's useful again (e.g. as a `/bin/sh`
POSIX-strict target distinct from an interactive login shell).
## 3. Coreutils
`ports/toybox/` (0BSD-licensed, https://landley.net/toybox/) provides `ls`,
`cat`, `cp`, `mv`, `rm`, `mkdir`, `grep`, `sed`, `find`, `chmod`, `date`,
`sort`, and more — see the `APPLETS` list in `ports/toybox/Makefile`. It builds
as a single multicall binary; `tools/mkfs.c` doesn't understand symlinks or
hardlink dedup, so `ports/toybox/Makefile`'s `install` target copies the built
binary once per applet name into `sysroot/bin/` rather than symlinking
(costs disk space per applet — currently ~44MB total — instead of one
shared inode, but needed no mkfs changes).
## 4. Two-stage bootloader + PSOGM splash
`boot/` used to be a single BIOS boot sector doing everything (VBE mode
set, A20, GDT, protected-mode switch, multiboot kernel scan/load) — silent,
and packed to within 4 bytes of the hard 510-byte MBR limit, so there was
no room to add so much as a print statement.
Split into two stages, the standard way real bootloaders solve this:
- **Stage 1** (`boot/bios/bootasm.S`, still the literal MBR sector, still
≤510 bytes) does exactly one thing: raw IDE PIO (LBA28) reads
`STAGE2_SECTORS` (16) sectors starting at disk sector 1 into memory at
`0x7E00`, then jumps there. Nothing else fits, nothing else is needed.
(Caught a real bug writing this: `outb $0x1F7,%al`-style immediates
silently truncate to 8 bits — ports 0x1F2-0x1F7 are all > 0xFF and need
the `%dx` register form, or you're hitting the wrong I/O port entirely.)
- **Stage 2** (`boot/bios/stage2.S` + `boot/bios/stage2main.c`, loaded at `0x7E00`,
occupies disk sectors 1-16) does everything stage 1 used to skip straight
to, plus a boot banner (BIOS teletype, real mode, before the VBE mode
switch — note this only reaches the video device, not serial, so it
won't show up in a `-serial file:...`-captured log), a splash screen, a
real-time loading bar, and an F8 boot menu (all described below). Not
size-constrained the same way as stage 1 — `tools/padstage2.pl` zero-pads
it out to a fixed `STAGE2_SECTORS*512` (8192) bytes so the kernel's
on-disk starting sector stays predictable regardless of stage2's actual
compiled size, and errors out (like `tools/sign.pl` does for stage 1) if
it ever grows past that budget instead of silently corrupting the layout.
The kernel now starts at disk sector 17 instead of 1 (`schos.img`'s `dd
... seek=17` in the root `Makefile`, and the equivalent `+(1+STAGE2_SECTORS)`
in `stage2main.c`'s `readseg()` — all three places that encode
`STAGE2_SECTORS` have to agree, or the kernel won't be found on disk at
all). `boot/bootmain.c` no longer exists; its logic was split between
`stage2.S` (real-mode/protected-mode setup) and `stage2main.c` (splash +
the unchanged multiboot-header-scan-and-load logic).
The VBE mode info block itself lives at `0x30000`, not the classic
single-stage bootloader's `0x9000` — stage2's own code now runs from
`0x7E00` up to nearly `0x9000` once the boot-menu/progress-bar logic is
included, and a block at `0x9000` gets partially overwritten by stage2's
own tail end (found this the hard way: the kernel's `LFB:` cprintf started
reporting a different wrong value on every boot, because it was reading
whatever stage2 code/data now happened to occupy that address).
`kernel/dev/vga.c`'s `vga_init()` has to agree on `0x30000` too, since it
independently re-reads mode fields from the same block.
Splash/progress bar/boot menu rendering is a small standalone pixel
renderer in `stage2main.c` (not the kernel's `vga.c` — paging isn't set up
yet, so the LFB physical address from the VBE block is used as a plain
pointer) reusing the kernel's own font data (`include/font.h`) directly.
It fills the framebuffer with PSOGM's purple (matching
`kernel/core/console.c`'s panic screen) and draws "SCHOS" centered, then:
- **Real-time loading bar**: `readsect()` (in `stage2main.c`, not the
kernel) calls `progress_tick()` after every 512-byte disk read, updating
an actual pixel-filled bar against the real total byte count — first a
small one while scanning for the multiboot header, then a second,
properly-sized one once the header's found and the true kernel size is
known, labeled "Loading kernel...". Not simulated/timed — it reflects
real bytes read off disk.
- **F8 boot menu**: `wait_for_f8()` busy-polls the keyboard controller
(`inb(0x64)`/`inb(0x60)`, raw scancodes — real-mode BIOS keyboard calls
aren't available once we're in protected mode) for about a second after
the splash appears. F8 (scancode `0x42`) drops into `boot_menu()`: three
options (boot normal, boot verbose, reboot to firmware setup) navigated
with up/down arrows and Enter, plus a "detected drives" panel from
`ata_present()` — a standard floating-bus presence check (select the
drive, write a pattern to two registers, read it back) on the primary
IDE channel's master/slave (the only channel this project ever actually
puts drives on; QEMU's IDE emulation answers the same test
"successfully" on the unpopulated secondary channel too, so that channel
isn't shown — it'd just be confidently wrong). "Reboot to firmware
setup" pulses the keyboard controller's reset line
(`outb(0x64, 0xFE)`, same technique as `kernel/sys/sys_proc.c`'s
`sys_reboot()`); a real bootloader can't jump into BIOS/UEFI setup
directly, so this is the closest honest equivalent — it gives the user
a fresh POST during which their firmware's own setup-menu key works.
- **Verbose boot**: picking "Boot SchOS (verbose)" writes a flag byte to
`BOOT_VERBOSE_FLAG_ADDR` (`VBE_INFO_ADDR + 0x100`, i.e. just past the VBE
block) before jumping to the kernel. `kernel/core/main.c`'s `kmain()`
reads it into a global `boot_verbose` as its very first action and, if
set, prints a line after every subsystem init call
(`uartearlyinit`, `kinit1`, ..., `sinit`) — real per-step boot logging,
not a canned message.
Two bugs worth remembering if this code gets touched again:
- **Keyboard extended-scancode handling**: `0xE0` (the prefix byte for
arrow keys and friends) also has bit 7 set, the same bit used to detect
key-*release* ("break") codes. A naive "skip anything with bit 7 set,
then check for 0xE0" loop skips the 0xE0 byte itself before ever testing
for it, so extended keys silently never register. The 0xE0 check has to
come before the break-code check, not after.
- **The kernel entry call is not a real C function call.** Both the
original single-stage `bootmain()` and the new `stage2main()` do
`((void(*)(uint32))entry)(fbaddr)` — normal cdecl, which pushes `fbaddr`
on the stack. But `kernel/arch/x86_64/entry.S`'s `mboot_entry` reads its
framebuffer argument out of **EAX** (a multiboot-style raw-register
handoff, then `mov %eax,%ebp` / `mov %ebp,%ebx` / `mov %rbx,%rdi` carries
it through the 32→64-bit transition into `kmain()`'s first argument).
The cdecl call never puts anything in EAX — it "worked" for years purely
because simple bootmain.c code happened, as a side effect of its own
codegen, to leave the right value sitting in EAX right up until the
`call` instruction. Adding the splash/menu/progress-bar code changed
that codegen and broke it silently: the kernel booted fine and reported
a plausible-looking but wrong `LFB:` address (garbage left over in EAX
from unrelated prior code, different on every boot). Fixed by pinning
the argument down explicitly with `asm volatile("jmp *%0" : : "r"(entry),
"a"(fbaddr))` instead of hoping the compiler's register allocation
cooperates. If this code moves around again, re-verify with a screendump
taken *immediately* before the jump compared against the kernel's own
`LFB:` log line — they silently diverging is exactly this bug.
## 5. `*at()` syscalls ignored/rejected real dirfds
`ls -l`/`ls -lR` (and, by the same code path, `find`/`du`/anything using
toybox's `lib/dirtree.c`) would hang on `ls -l` and either hang or silently
misbehave on recursive listings. Root cause: `kernel/sys/sys_fs.c`'s
`*at()` syscalls (`openat`, `fstatat`, `unlinkat`, `renameat`, `linkat`,
`symlinkat`, `mkdirat`, `mknodat`, `readlinkat`, `faccessat`) only handled
two cases for their `dirfd` argument — `AT_FDCWD` (resolve relative to
`proc->cwd`) or an absolute path (resolve from root, ignoring `dirfd`
entirely). A **real** dirfd with a **relative** path — e.g.
`fstatat(open("/bin", O_RDONLY), "ls", &st, ...)`, the standard, efficient
way any directory-tree walker avoids rebuilding full path strings for every
entry — was either rejected outright (`fstatat`, `unlinkat`, ...) or
silently ignored in favor of `proc->cwd` (`openat`, which never checked at
all — worse than rejecting, since it can silently resolve the *wrong*
file if cwd happens to contain a same-named entry).
Fixed by adding a real dirfd-relative path resolver: `kernel/fs/fs.c`'s
`namex()` was refactored into `namex_from(struct inode *start, ...)`,
parameterizing what used to be a hardcoded `idup(proc->cwd)`, with two new
exported entry points, `nameifrom()`/`nameiparentfrom()`. `sys_fs.c` gained
`at_dir(dirfd)` (resolves `AT_FDCWD` or a real fd to a starting inode) and
`at_namei()`/`at_nameiparent()` built on top of it, plus a `createat()`
(parameterizing `create()`'s hardcoded `nameiparent()` call the same way).
Every `*at()` syscall now uses these instead of its own inline
absolute-path/`AT_FDCWD`-only check.
**Known follow-on issue, not yet root-caused**: heavy `ls`/`ls -lR` use
across many commands in one long-running shell session can eventually make
a *later*, unrelated directory hang (reproduced with `ls -lR /var` after
~10 prior commands in the same session; the identical command succeeds
immediately on a fresh boot). Not an inode-table or file-table exhaustion
— both `iget()` and `filealloc()` fail cleanly (panic / return 0
respectively) when full, and neither fired. Smells like a lock or
reference-count leak somewhere in the *at() paths above, or in toybox's
own fd handling, that only manifests after enough accumulated opens/closes.
Next step: instrument `itable`/`ftable` occupancy counts and watch them
across a long `ls`-heavy session to see what's actually climbing.
## 6. Boot speed
Two cheap, real wins in `boot/bios/stage2main.c`, no functional changes:
- `progress_tick()` (called once per 512-byte disk sector) used to trigger
a full progress-bar redraw — border plus fill, a few thousand pixel
writes — on *every* sector. For the ~330KB kernel image that's ~650 full
redraws (the multiboot-header-scan phase adds another ~32) for a bar
that only has 101 visually distinct states (0-100%). Split into
`progress_border()` (drawn once, in `progress_start()`, since the border
never changes) and `progress_fill()`, and `progress_tick()` now skips the
redraw entirely unless the percentage actually changed. This was the
single biggest chunk of the bootloader's own wall-clock time.
- `fill_rect()` (the full-screen splash fill, 480,000 pixels at 800x600,
run at least twice per boot) used to call `draw_pixel()` per pixel, which
recomputes `g_fb + y*stride + x*4` from scratch every time. Now computes
each row's base pointer once and indexes into it.
- The F8 boot-menu wait was shortened from ~1s (`5000000` busy-loop
iterations) to ~0.6s (`3000000`) — flat overhead on every boot whether
or not the menu is wanted, so it's the most direct lever available
without giving up the feature entirely.
## 7. `*at()` syscalls, real signal delivery, and mksh job control (in progress)
`ls -l`/`ls -la` hanging outright turned out to be the same root cause
described in section 11 above (dirfd-relative path resolution) — fixed
and verified extensively (plain `ls`, `ls -l`, `ls -la`, `ls -lR`, piped
output, all confirmed working repeatedly).
A *second*, deeper issue was found investigating a follow-up report
(`cd` with no `HOME` set, then `cd var`, then `ls` hanging): mksh's
interactive job control (`ports/mksh (mksh R59c source): jobs.c`) blocks in
`sigsuspend()` waiting for `SIGCHLD`, expecting its own handler
(`j_sigchld()`) to run, reap the child via `waitpid(-1, &status, WNOHANG)`,
and update its internal job-state tracking. This kernel had **no real
signal delivery at all** — `sys_rt_sigaction()` was a no-op stub, so no
handler was ever invoked for anything, and `sigsuspend()` just slept out
one tick and returned `-EINTR` unconditionally. mksh's wait loop had no
way to ever learn the child had exited, hanging forever.
Real signal delivery was implemented from scratch:
- `struct proc` (`include/proc.h`) gained `sig_handler[32]`/
`sig_restorer[32]` (populated by `sys_rt_sigaction()`,
`kernel/sys/sys_sig.c`), `saved_sigtf` (stashed pre-signal trapframe),
and `in_signal` (re-entrancy guard). `allocproc()` zeroes these for a
freshly claimed ptable slot (they weren't before — a reused slot
silently inherited a previous, unrelated process's handler addresses,
which crashed the *next* process to occupy that slot the moment
anything tried to delivver a signal to it). `fork()` copies handler/
restorer/mask from parent to child (POSIX semantics) but not pending
signals.
- `kernel/proc/proc.c`'s `exit()` sets `SIGCHLD` pending on the parent.
- `kernel/sys/syscall.c`'s `syscall()` calls `deliver_signals(tf)` in its
tail, right before returning to user mode: if there's a pending,
unblocked signal with a registered handler, it redirects the trapframe
to jump to the handler (`rdi`=signal number per SysV ABI, `rip`=handler)
instead of resuming the interrupted code, having first stashed the
original trapframe in `proc->saved_sigtf`.
- The return address is **not** `sig_restorer[sig]` — checked, `SA_RESTORER`
is not defined anywhere in this musl build, so `sigaction()` never fills
in a restorer; it's always 0, and pushing 0 as a return address crashes
with `rip=0` the instant the handler function returns. Instead,
`deliver_signals()` injects a tiny 7-byte trampoline (`mov eax,15;
syscall` — `rt_sigreturn`) directly onto the user's own stack, below the
current `rsp` (clear of the SysV red zone), and points the return
address at that — the standard technique for when the C library doesn't
supply its own restorer.
- `sys_rt_sigreturn()` restores `*proc->tf` from `proc->saved_sigtf`,
clearing `in_signal`.
**A real bug found and fixed along the way**: the first version of
`sys_rt_sigaction()` translated the user's `struct sigaction` pointer
*once* via `uva2ka()` and then did a bulk `memset`+copy across the whole
~152-byte structure. `uva2ka()` only guarantees the *page containing that
address* is mapped, not everything up to some further offset — when the
user's pointer landed close enough to a page boundary, the write ran past
the mapped page into whatever physical page happened to be next in kernel
address space (P2V is a flat offset, so "next page" there has nothing to
do with the user's actual next page), corrupting unrelated kernel memory.
This crashed every freshly `exec`'d `/bin/sh` almost immediately, since
mksh calls `sigaction()` during its own startup. Fixed by only touching
the two individual 8-byte fields actually needed (`sa_handler` at offset
0, `sa_restorer` at offset 144), each through its own `uva2ka()` call.
`sys_waitpid()`/`sys_wait4()` (`kernel/sys/sys_proc.c`) were also rewritten
on top of a new `waitpid(pid, status, options)` in `kernel/proc/proc.c`
(refactored from the old any-child-only, always-blocking `wait()`, which
is now just `waitpid(-1, 0, 0)`): a specific `pid` (not just "any child"),
`WNOHANG` (return 0 immediately instead of blocking), and — importantly —
a genuinely optional status pointer. The old code ran a NULL status
pointer through `argptr()`, which rejects NULL/unmapped addresses outright
(page 0 is deliberately unmapped) — but `waitpid(pid, NULL, WNOHANG)` is
completely standard, and is exactly what mksh's job control does.
**Status: signal delivery mechanically works** — verified directly (a
debug build logged the handler being invoked with the correct trapframe
and `sys_rt_sigreturn()` restoring cleanly, no crash, repeatable). **mksh's
job control still doesn't reliably progress afterward**, including for
plain non-piped foreground commands, not just the original piped-into-head
case. Root cause not yet isolated. Leading suspects, in order of
likelihood:
- `sys_setpgid()`/`sys_getpgid()` are no-op stubs (`kernel/sys/sys_proc.c`)
— `j_sigchld()` (`ports/mksh (mksh R59c source): jobs.c:1344-1348`) has an early-out
keyed on `j->ppid == procpid`, which may never resolve the way mksh
expects without real process-group tracking.
- `getrusage(RUSAGE_CHILDREN, ...)` — called at the top of `j_sigchld()` —
isn't verified to behave sanely under this kernel's implementation
(`kernel/sys/sys_proc.c`'s `sys_getrusage()` only fills in `ru_utime`
from the *calling* process's own tick count, not accumulated child
time; `RUSAGE_CHILDREN` isn't distinguished from `RUSAGE_SELF` at all).
- Possibly something in how `j_sigchld()` matches a reaped `pid` against
its own `job_list` (`ports/mksh (mksh R59c source): jobs.c:1372-1385`) silently fails
to find the job for reasons not yet traced.
This was left here deliberately rather than continuing to iterate blind:
SIGCHLD delivery is left **enabled** (`kernel/proc/proc.c`'s `exit()`)
since disabling it guarantees zero chance of mksh's wait loop ever exiting,
whereas leaving it on at least gives the cases that don't hit the
still-unknown remaining gap a chance to work. Next session should start
by instrumenting `j_sigchld()`'s own control flow (or strace-equivalent
logging in the kernel around `sys_setpgid`/`sys_getrusage`/`sys_waitpid`
calls specifically while reproducing `cd; cd var; ls` or even simpler
`ls -l /bin` followed by a second command) rather than re-deriving the
signal-delivery mechanism, which is now confirmed solid.
---
## 8. Two ABI assumptions that only GCC was keeping true
Found when the build moved from GCC-in-WSL to clang-on-Windows. Neither is a
clang bug; both are places where the code relied on GCC doing something it
never promised, and both had been latent for years.
### `%fs:0` was being used as scratch — it is the TCB self-pointer
`syscall_entry` (`kernel/arch/x86_64/trapasm.S`) stashed `%rax` at `%fs:(0)`
while it fetched `proc` to find the kernel stack. The comment called it a
"per-CPU scratch slot". It is not: the x86-64 TLS ABI reserves the word *at*
the thread pointer to hold the thread pointer, and a compiler may read it to
materialise the TP before applying a variable's offset.
GCC compiles `__thread` access to a direct `%fs:-8` segment reference and never
reads that word, so the clobber was invisible. clang at `-O0` emits
`movq %fs:0, %rax; addq $-8, %rax; (%rax)` — reading back the syscall number
this code had just stored. The first process's first syscall is
`exec` (7), so `proc` resolved to address `7 - 8 = -1` and the kernel
page-faulted recursively until the stack was gone.
Fixed by moving the scratch to `%fs:(8)`. Positive offsets are safe: the TLS
block lives at negative offsets and `seginit` places the thread pointer 2048
bytes into a 4096-byte page.
**How it was found** is the part worth copying next time. Three rounds of
`cprintf` narrowed it to "the word is right at `mpmain`, wrong by the first
trap" and no further — the instrumentation perturbed the timing enough to move
the symptom. A hardware watchpoint through QEMU's gdb stub named the
instruction on the first run:
```
qemu ... -S -gdb tcp::26000
gdb -batch -x tcp.gdb # hbreak mpmain; watch *(unsigned long*)kernel_fs_base
```
`seginit` also now writes the self-pointer, which it never did. That is ABI
-correct regardless and costs one store per CPU.
### `_start` read incoming registers with `register ... asm("rdi")`
`lib/glue.c` captured `argc`/`argv` from `%rdi`/`%rsi` using GCC local register
variables. Both compilers only guarantee that binding for inline-asm operands;
using it to pick up an incoming argument is undefined. clang ignored it and
read two uninitialised stack slots:
```asm
movl -0x4(%rbp), %eax # should have been %edi
movq -0x10(%rbp), %rax # should have been %rsi
```
Every program still ran. `init`, `shmtest` and `socktest` take no arguments and
passed their entire suites; the only symptom in the whole system was
`inputtest -n 6` ignoring its limit. Moved to `lib/start.S`, where reading
incoming registers is expressible and correct — `exec` passes argc/argv in the
SysV argument registers already, so `_start` calls `main` without touching
them.
---
## 9. The boot looked like a hobby OS, and two things were why
Not a bug — a design problem, fixed by looking at a screenshot instead of the
code.
**The display was the kernel log.** Boot printed `pci: 6 devices` and
`vga: display controller 1234:1111, aperture fd000000 (16 MB)` to the screen
the user is looking at. That single fact is most of what separates a hobby
project from a product; every shipping system puts that behind a flag.
`cons_serial_only` already existed for panic, so the fix was to start it *set*.
Everything still goes out the UART — which is where `tools/windows/bootcheck.ps1` reads
it from, so all five suites kept passing untouched — and `console_takeover()`
clears it on the first `exec()`, the moment userspace genuinely starts.
`vga_init` no longer clears the screen, so the splash survives the whole of
boot rather than being wiped a few hundred milliseconds in.
**There were three unrelated designs.** The splash was deep purple with gold
text, deliberately matching the *panic* screen. The console was cream with
near-black text. So a healthy boot was dressed as a crash, then abruptly
became an amber terminal. `include/theme.h` is now the one palette, and the
rule is: **normal states are calm and dark; only failure is loud.** PSOGM keeps
the purple entirely to itself, which is what makes it mean something.
The splash copy went from "SchOS Boot Manager / version 1.0 / Memory: 1023 MB
/ Press F8 for boot menu..." to the product name, a rule in the accent colour,
and the one hint that is actionable — moved to the bottom, where a hint
belongs. No shipping OS names its bootloader on the splash.
### Three latent bugs it flushed out
Retheming meant `stage2main.c` included `types.h` properly for the first time,
and the 32-bit build stopped compiling:
- `_Static_assert(sizeof(uint64) == 8)` fired. `uint64` was `unsigned long`,
which is **4 bytes** in the `-m32` loader — the file even carried a local
`e820_u64` workaround with a comment saying so. Now conditioned on
`__SIZEOF_LONG__` rather than on the platform, which covers LP64, LLP64 *and*
ILP32 correctly.
- `x86.h`'s `stosq` and `xchg` use `rep stosq` and `xchgq`, which do not exist
in 32-bit mode. They only ever compiled because `uint64` was silently
narrowed. Now behind `#ifdef __x86_64__`.
- The progress bar computed `(uint64)done * 100 / total`, a 64-bit divide that
needs libgcc's `__udivdi3` — which a freestanding loader does not link. Now
scaled to KiB and done in 32 bits.
All three had been sitting in a bootloader that worked fine, waiting for
someone to make `uint64` mean what it says.
---
## 10. A full-screen program typed the whole session into the shell
Keys reached `/dev/input/event0` **and** the tty line discipline at the same
time. That is correct for two consumers of one keyboard and completely wrong
for a full-screen program: while it had the display, the shell behind it was
quietly accumulating every keystroke, and the first Enter afterwards ran
whatever had piled up. Arrow keys arrived as CP437 glyphs on the console for
good measure.
The fix is the standard one and it is a *primitive*, not a patch: an exclusive
grab, `EVIOCGRAB`, with Linux's ioctl number. While a device is grabbed its
events still flow to `/dev/input/eventN` but both keyboard drivers withhold the
decoded character from the console.
Three details that matter more than the feature:
- **The grab is released by the devsw `close` hook**, not by the program. A
program that grabs the keyboard and then crashes would otherwise leave the
console permanently deaf, with no way to type the command that fixes it.
- **Modifier state is still tracked while grabbed** (`kbdgetc` keeps updating
shift/caps and only skips the character). Otherwise releasing the grab would
hand the line discipline a stuck Shift.
- **The USB path updates `prev[]` before returning**, or the next report would
look like every held key had just been pressed again.
It is also exactly what Schcom will need, which is why it lives in the input
layer rather than in any one client.
### The near-miss worth remembering
The first attempt appeared to work: the harness passed, every assertion green.
It had not worked at all — the ioctl call was never inserted into the client
source, because a scripted string replacement silently failed to match. The
`grep` that should have caught it printed three hits (a comment, a `#define`,
and the *release* call) and was read as confirmation without counting what
they were.
What actually caught it was reading the transcript rather than the verdict:
four bare `#` prompts after quitting, which is precisely the symptom the change
was supposed to remove. `input_grab()` now announces itself, so "did the grab
happen" is never again a thing anyone has to infer.
---
## 11. `ps` and `top`, and why they never worked
/proc was a set of character devices: init ran `mknod("/proc/uptime", 6, 2)` and
friends, so every file was a real inode on the root filesystem with a
major/minor behind it. That is fine for a fixed list and cannot work at all for
the thing /proc is mostly for -- one directory per process. You cannot mknod a
directory for a process that does not exist yet and will be gone in a minute.
`ps` calls `dirtree_flagread("/proc")`, got a listing with no numbers in it, and
reported nothing. It was never a `ps` problem.
So /proc is now a real filesystem behind the same `struct fs_ops` vtable SchFS
uses (`kernel/fs/procfs.c`, `PROCFS_DEV`), mounted by init with
`mount("proc", "/proc", "proc", ...)`. Nothing is stored: directory listings
are generated from the process table when asked, file contents when read.
Three things worth keeping in mind:
- **The split is deliberate.** `kernel/fs/procfs.c` owns the *namespace* (what
exists); `kernel/dev/procfs.c` keeps the *generators* (what the global files
say). They change for different reasons, so `proc_gen_global()` is the only
thing that crosses between them.
- **procfs never touches ptable.** `proc_snapshot()` copies what is needed out
from under the lock, so a process that exits between the readdir that listed
it and the open that reads it produces a stale line rather than a
use-after-free.
- **Sizes are reported as zero**, exactly as Linux does. A synthetic file has
no length until it is generated, and generating every one on every `stat()`
to fill in a number almost nothing reads would be absurd.
### What the suite caught
Marking every procfs inode read-only broke `echo s > /proc/sysrq`, and three
USB checks failed on it within a minute of the change -- they drive the disk
through sysrq. /proc is read-only *except* sysrq, which is the one deliberate
write target, and the code now says so in one place.
### Still not right
`ps -o comm` prints `[init]` rather than `init`. That field comes from
`/proc/<pid>/exe`, which Linux makes a **symlink** and toybox `readlink()`s;
ours is a regular file, so the readlink fails and toybox brackets the name as
it would for a kernel thread. Default `ps` and `top` output are unaffected.
Fixing it needs symlink support in procfs, which belongs with the rest of the
VFS work rather than as a one-off.
---
## 12. A test program crashed at random, and had been typing its own launch command
Symptom: a full-screen framebuffer program, launched with arguments, worked
four times and then died with
pid 9 <prog>: trap 14 err 4 on cpu 0 rip 0x2269be addr 0x18--kill proc
`err 4` is a user-mode read of a non-present page and `addr 0x18` is a null
pointer at offset 24. `llvm-nm` put `rip` inside the routine that opens the
program's overlay view, which dereferences a pointer the program only fills in
once a level is loaded -- at offset 0x18 on x86-64, because the three 64-bit
words of a base struct sit in front of it. So the program had been asked to open
that view before there was anything to show.
**The technique is the transferable part:** the fault address alone said
"null pointer plus 24", which is nearly useless; `llvm-nm` on the faulting `rip`
turned it into a function name, and the offset then identified the exact field.
Reach for that before adding printfs to a program you did not write.
The cause was ours, and it was visible in the log the whole time: every run
printed a burst of garbage bytes *before* the grab was taken, and the amount
differed each run.
**`/dev/input/eventN` had no notion of being opened.** Events were queued from
boot onwards whether or not anything was listening, into one 256-entry ring
with one read cursor. When the program opened the device it inherited the entire
backlog -- including the keystrokes of the command line that had launched it,
which the shell had already consumed -- and replayed them as input. Whether that
crashed depended on which keys happened to be sitting there, which is exactly
the "sometimes" in the report.
Fixed by giving `struct devsw` an `open` hook to match `close`, and having
`/dev/input`:
- drop the backlog on open (`r = w`) -- a reader starts from *now*;
- only queue events while `opens > 0`, which is what Linux does: an evdev
device with no clients does not accumulate.
Linux gets both for free by giving every open its own client queue. One shared
ring plus "start from now" is the same answer for a single reader, which is the
only case that exists today; per-client queues become necessary the moment
Schcom and a client both want the keyboard.
### Also fixed
`cd` into a file reported "Operation not permitted". It was right to fail, but
`sys_chdir` returned a bare `-1`, so the shell printed whatever errno happened
to be lying around. It returns `-ENOTDIR` now, and `-ENOENT`/`-EFAULT` on the
other two failure paths -- "Not a directory" sends you to the right place,
"Operation not permitted" sends you looking at permissions.

211
docs/architecture/kernel.md Normal file
View file

@ -0,0 +1,211 @@
# SchIFT — the kernel
Long mode, 4-level paging, preemptive multitasking, and more than one
processor. What follows is the parts of the kernel that are not a filesystem, a
driver or a protocol stack: memory, processes, scheduling, traps and signals.
---
## Address space
One page table per process; the kernel is mapped into all of them at
`KERNBASE = 0xFFFF800000000000`, so a trap does not switch `CR3`.
`kvmalloc()` maps physical 0–4 GB as four 1 GB pages, with the top one marked
uncached — that is where PCI BARs and the framebuffer live. This is why 3 GB is
the most memory the kernel can treat as RAM, and it is a deliberate simplicity:
a direct map with a fixed offset makes `P2V`/`V2P` arithmetic rather than a
lookup.
`kalloc()` hands out one 4 KB page at a time from a free list. **There is no
slab or pool allocator**, and the consequence is visible throughout the tree:
every subsystem that needs many small objects has invented its own fixed-size
table — `NPROC` processes, `NSOCK` sockets, `NDEVICE` devices, `NSHM` segments.
That is why the limits are compile-time constants rather than a design decision
about each one.
The free list carries a canary, checked at the top of both `kalloc()` and
`kfree()`, so a wild write into it panics at the point it is touched rather than
handing back a bad pointer that fails somewhere else entirely.
### DMA
`kernel/core/dma.c` owns an arena carved off the top of physical memory before
`kalloc` is given the rest. Devices need physically contiguous memory; `kalloc`
cannot promise it, so the region has to be reserved before there is any
competition for it.
## Per-CPU state
The per-CPU block's base is in **both** `FS_BASE` and `GS_BASE`, set by
`seginit()`:
```
+0 self-pointer required by the x86-64 TLS ABI (%fs:0)
+8 syscall scratch trapasm.S parks %rsp here across the stack switch
+16 struct cpu *
+24 struct proc *
```
**These offsets are shared with assembly and must not be reordered.** Offset 8
is not free: `syscall_entry` uses `%fs:(8)`, and because FS and GS base point at
the same address, that is the same memory as `%gs:8`.
Reached through `mycpu()` and `myproc()`, which compile to a single
`movq %gs:16` / `%gs:24` with no cached base.
> **`__thread` is wrong for per-CPU kernel data**, and this is the single most
> expensive lesson in the tree. `__thread` means per *thread*, and the compiler
> is entitled to read the thread pointer once and keep it in a register across
> calls. But `%fs` here is per *CPU*, and a kernel thread changes CPU at every
> `sched()`. So the assumption is false for exactly the variables that most need
> to be right, and the failure is a null dereference deep in the scheduler on any
> machine with more than one core. `include/proc.h` carries the disassembly that
> proves it.
## Processes
`struct proc` in [`include/proc.h`](../../include/proc.h). A fixed table of
`NPROC` entries, allocated by linear scan.
**The field order at the top of the struct is fixed by assembly.** `trapasm.S`
hard-codes `fs_base` at offset `0x18`, and there is a `_Static_assert` in
`proc.c` that fails the build if it moves. Anything new goes at the end.
Each process carries: page directory and size, kernel stack, open file table,
cwd inode *and* cwd path string, signal handlers and masks, VMA table for
`mmap`, POSIX identity (pgid, sid, and all six of real/effective/saved uid and
gid), and the Linux-ABI syscall map.
### Slot reuse
The table recycles slots, and a recycled slot that inherits the previous
occupant's state is a whole class of bug that has bitten here twice.
`allocproc()` therefore explicitly clears the signal fields, the exit status,
the kill signal, and the identity fields. The failure mode when it did not:
`deliver_signals()` jumping to a totally unrelated earlier process's handler
address on this process's very first syscall return.
### Kernel threads
`kthread_create(fn, name)` makes a process that runs `fn` in kernel mode and
never returns to userland. It gets a page directory with only the kernel mapped,
no files, no cwd, no parent — it is not in the process tree, cannot be waited
for, and must never exit.
The point is that it is a *process* and not a timer callback: it shares the
scheduler's ordinary rotation, so it can hold sleeplocks and block on the disk
exactly as a syscall would. The network thread is the current user.
## Scheduling
Round-robin over the process table, one scheduler per CPU, each with its own
context. `swtch()` saves callee-saved registers and switches stacks; `sched()`
returns to the scheduler and asserts four invariants on the way — holding
`ptable.lock`, `ncli == 1`, not `RUNNING`, interrupts off.
When a full pass finds nothing runnable the CPU halts rather than spinning.
**Locking is a single `ptable.lock` over the whole table.** Not per-process: a
scheduling operation touches two processes (the one leaving and the one
arriving) and two-object locking needs an ordering rule that every path obeys.
The same reasoning produced a single lock over the socket table.
### sleep / wakeup
xv6's discipline, unchanged because it is correct: `sleep(chan, lk)` atomically
releases `lk` and blocks; `wakeup(chan)` makes every sleeper on that channel
runnable. The atomicity is what matters — a sleeper that tests its condition
under one lock while the waker sets it under another has no interlock and sleeps
through the wakeup meant for it.
## Traps and syscalls
Interrupts and exceptions enter through `vectors.S` → `trapasm.S` → `trap()`.
Syscalls enter through `SYSCALL`/`SYSRET` and `syscall_entry`, which is a
separate and faster path.
Every entry from user mode checks a **canary at the bottom of the kernel stack
page**. A deep or nested trap that overflows the 4 KB stack clobbers that word
first, so the failure is reported as `kstack overflow` at the point it happened
rather than as arbitrary corruption later.
One configuration detail with an expensive history: `MSR_STAR`'s `SYSRET`
selector field must hold the **compat 32-bit** code selector, because `SYSRETQ`
computes the returned `cs`/`ss` as `STAR[63:48]+16`/`+8`. Putting the real
64-bit user selector there makes every `sysretq` load an invalid pair — which
QEMU's TCG does not validate, so it stayed hidden until a hardware interrupt
forced a strictly-validated `IRETQ`.
## Signals
Real delivery, not stubs. `sys_rt_sigaction()` records the handler;
`deliver_signals()` runs in the tail of `syscall()`, just before returning to
user mode, and if a pending unblocked signal has a handler it rewrites the
trapframe to enter the handler instead of resuming — stashing the original in
`proc->saved_sigtf`. `sys_rt_sigreturn()` puts it back.
The return address is **not** `sa_restorer`. `SA_RESTORER` is not defined in
this musl build, so `sigaction()` never fills one in, and pushing 0 as a return
address crashes with `rip = 0` the moment the handler returns. Instead a 7-byte
trampoline (`mov eax,15; syscall`) is injected onto the user's own stack below
`rsp`, clear of the SysV red zone — the standard technique when the C library
supplies no restorer.
Signals without a handler fall back to a default action: termination for the
fatal set, dropped otherwise.
### PID 1 does not die from a signal it never asked to handle
`init_ignores()` discards any signal whose disposition is the default action
when the target is PID 1 — **including `SIGKILL`**. This is what POSIX requires
and what Linux does in `sig_task_ignored()`, and without it the system has a
single keystroke that kills it: Ctrl-C signals the foreground process group,
init is pgid 1, and any moment where the terminal's foreground group is still 1
lands SIGINT on init. It has no handler, so `exit()` runs, and `exit()` panics
outright on initproc.
Spamming Ctrl-C at a frozen program is exactly the input that finds that window.
## Process groups and sessions
`setsid`, `setpgid`, `getpgid`, `getsid`, and signal delivery to a whole group.
Job control is defined entirely in terms of these — a shell puts each pipeline
in its own process group and signals the group, never individual pids — and
mksh's job control could not make progress until they existed.
## What userspace gets
`fork`, `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and real pid selection,
pipes, signals, `mmap` (anonymous and file-backed), POSIX shared memory, Unix
domain sockets, `poll`/`select`/`epoll`, `eventfd`, `timerfd`, `signalfd`.
`mmap` uses a bump allocator (`proc->mmap_next`, starting at `0x40000000`) so
every mapping gets its own disjoint, exactly-sized range. It previously assigned
a fixed 8 MB window per VMA slot regardless of the requested length, so a single
allocation over 8 MB mapped straight into a neighbouring live mapping.
---
## What it does not do
- **No slab allocator.** One page at a time, and every subsystem's fixed-size
table is a consequence. This is the single change that would most improve the
rest of the kernel.
- **No demand paging and no page cache.** `mmap` of a file *reads the whole
thing in* at map time. Writable `MAP_SHARED` of a file is therefore refused
outright rather than faked — see [../plan/foundations.md](../plan/foundations.md).
- **No `sendmsg`/`recvmsg`, so no `SCM_RIGHTS`.** File descriptors cannot be
passed between processes.
- **No dynamic linking.** `exec` handles `ET_EXEC` only: no `PT_INTERP`, no
`ET_DYN`, no shared libraries, no `dlopen`.
- **No threads.** `clone` exists but does not share an address space;
`gettid` returns the pid.
- **Permissions are tracked but not enforced.** uid/gid are maintained honestly
so `getuid()`/`setuid()` stop lying, and nothing consults them on access.
- **No swap**, and no memory pressure handling of any kind. `kalloc` failing is
reported up as `-ENOMEM` and that is the whole policy.
- **Nested signal delivery is not supported.** `in_signal` guards against
re-entering a handler while one is running.
- **Scheduling is round-robin with no priorities**, no fairness accounting, and
no CPU affinity despite `sched_setaffinity` existing.

260
docs/architecture/networking.md Normal file
View file

@ -0,0 +1,260 @@
# Networking
ARP, IPv4, ICMP, UDP, TCP and DHCP, over a driver-agnostic interface class,
reached through the same socket syscalls as Unix domain sockets.
Verified end to end rather than by inspection: `bootcheck -Net` starts a
listener and a UDP responder on the host, tells the guest to talk to them, and
asserts on **both** ends — one side alone cannot tell a working stack from one
that completes a handshake and drops every byte.
---
## The four layers
```
socket() bind() connect() send() recv() syscalls, one set
┌────────────────┴───────────────────┐
│ AF_UNIX AF_INET │ socknew() dispatches on domain
│ kernel/net/unix.c kernel/net/inet.c
└────────────────┬───────────────────┘
│ lwIP raw API
┌────────────────┴───────────────────┐
│ lwIP 2.2.0, NO_SYS=1 │ ports/lwip, compiled into the kernel
│ port layer: kernel/net/lwip/ │ ~400 lines: a lock, a clock, a netif
└────────────────┬───────────────────┘
│ struct netdev_ops
┌────────────────┴───────────────────┐
│ e1000, and any future NIC driver │
└────────────────────────────────────┘
```
## The netdev class
A driver publishes a [`struct netdev_ops`](../../include/netdev.h) and is then
finished with the network stack forever: it never mentions lwIP, and the stack
never mentions it.
```c
struct netdev_ops {
int (*transmit)(struct device *, const void *frame, int len);
int (*receive) (struct device *, void *out, int cap);
const unsigned char *(*mac)(struct device *);
int (*link_up) (struct device *); // optional
};
```
`transmit` returning -1 is **congestion, not failure** — the ring is full and
the caller is expected to retry. `receive` never blocks; it returns 0 when
nothing is waiting, and the caller decides how to wait.
This class was added late, and the reason it had to be is instructive: the stack
had reached straight into the driver. `net.c` called `e1000_present()`, the lwIP
glue was named `netif_e1000.c`, and the e1000 shipped with `.ops = 0` and a
comment saying so. A second NIC driver would have meant editing the protocol
stack to add itself. lwIP now gets one netif per bound device, `/proc/netif`
lists them all, and the e1000 drives up to four cards rather than refusing the
second — verified with two attached, each with its own MAC and its own DHCP
lease.
### `netdev_selftest()`
`echo n > /proc/sysrq` sends a hand-built ARP request and waits for the reply.
It proves the whole driver path in one exchange — descriptor formats, the DD
handshake, tail bookkeeping, DMA addresses, the receive ring hand-back, and the
MAC filter, because an ARP reply is addressed to this station and does not
arrive if the filter is wrong.
It was written *before* there was any protocol stack, deliberately: a stack on
an unproven driver fails in ways indistinguishable from its own bugs. It belongs
to the class rather than to the e1000, because it is the check every new NIC
driver will want first.
## lwIP
lwIP 2.2.0, fetched by `ports/lwip` and **compiled into the kernel** — it is the
one port that installs nothing into the sysroot, because it is kernel code.
`NO_SYS=1`, so lwIP has no threads, mailboxes or semaphores of its own. The port
layer is `kernel/net/lwip/`: an architecture header, a lock, a clock, an
Ethernet netif, and a set of shim headers standing in for the libc lwIP expects.
### Why lwIP and not a BSD stack
The licence was not the deciding factor — FreeBSD's `sys/netinet` is BSD-3 too.
**It is that FreeBSD's stack is not a library.** `tcp_input.c` alone is 122 KB
and reaches into mbufs, `epoch(9)`, netisr, VNET, `uma(9)`, sockbufs and inpcb
hash tables. Taking it means porting FreeBSD's kernel infrastructure first and
finding the TCP somewhere underneath — the same conclusion already reached about
FreeBSD's USB stack.
NetBSD was ruled out on licence: its `tcp_input.c` and `tcp_output.c` carry the
WIDE Project's 4-clause advertising clause, which Berkeley's rescission does not
reach. See [../licensing.md](../licensing.md) §3.
lwIP was written for exactly the case SchOS is in: a small operating system with
a raw Ethernet driver and no infrastructure to speak of.
**The reason not to write one, kept because it is the reason to resist later
too:** it is not the packet formats that are hard. It is congestion control,
retransmission, window management and twenty years of corner cases that only
appear against a real peer.
## The network thread
`kernel/net/net.c`. One kernel thread, started by `netinit()`, feeds lwIP
received frames and runs its timers.
**A thread rather than the interrupt handler, and the reason is the lock:** a
call into lwIP can reach the driver, the driver can sleep, and an interrupt
handler cannot. So the interrupt keeps doing what it always did — clear the
cause register — and this thread polls.
`netinit()` runs after `sinit()` because `kthread_create` needs a usable process
table, and it returns immediately: everything happens in that thread, which
cannot run until the scheduler is entered.
DHCP obtains an address at boot. `/proc/netif` reports each interface's address,
netmask, gateway, MAC and frame counts, taken at the lwIP seam.
> It is `/proc/netif` and not `/proc/net` deliberately: on Linux `/proc/net` is a
> directory of many files, and a single file by that name would be a lie about a
> well-known path.
## Sockets
One set of syscalls, at their real Linux numbers. `socknew()` dispatches on the
domain, and **no syscall changes shape** between the two families.
### AF_UNIX
`kernel/net/unix.c`. `SOCK_STREAM`, `socketpair`, `shutdown`.
A connected pair is **two buffers, not one**: writing appends to the peer's
buffer and reading drains your own, which is what makes a socket bidirectional
where a pipe is not, without needing two of anything.
**One lock covers the whole table.** A pipe gets away with a lock per pipe
because every operation touches one object; a socket operation touches two, and
two-object locking needs an ordering rule every path obeys or it deadlocks.
Worse, the wakeup races are subtle — a reader testing `peer_closed` under its own
lock while the closer sets it under the peer's has no interlock at all.
**Names live in a table here, not in the filesystem.** Binding creates no
directory entry. The table gives the behaviour that matters — a client finds a
server by a name agreed in advance — and what is lost is `ls -l` showing an `s`,
and `unlink()` of a socket path working. Both want a VFS change rather than a
socket change.
### AF_INET
`kernel/net/inet.c`, on lwIP's **raw** API rather than lwIP's own sockets layer,
which would need `NO_SYS=0` and a second socket abstraction alongside this
kernel's own.
**Everything is protected by the lwIP lock and nothing else**, and that is not a
shortcut: lwIP's callbacks run in the network thread with that lock held and are
the only writers to a socket's receive buffer. A syscall reading it takes the
same lock. There is no second lock because there is no second writer.
#### How a blocking socket waits
Worth reading before touching, because the obvious implementation does not work.
A blocking call cannot `sleep(&socket, &lwip_lock)`: `sleep()` wants a spinlock
and the lwIP lock is a *sleeplock* — it has to be, because a call into lwIP can
reach the driver. And a waiter cannot hold that lock while sleeping, or the
network thread could never deliver the thing being waited for.
So the wait is on a **counter** with a spinlock of its own:
```
reader: lwip_lock; check condition; s = net_seq(); lwip_unlock; net_wait(s)
waker: (inside a callback, lwIP lock held) net_wake()
```
No wakeup can be lost. `s` is read while the reader still holds the lwIP lock,
so anything changing the condition must do so afterwards and therefore bumps the
counter after the read; `net_wait` re-checks under its own spinlock and does not
sleep.
One counter and one channel for **all** inet sockets rather than one each. At
this scale waking sixteen sockets costs a handful of instructions in the ones
with nothing to do, and a per-socket channel would need careful teardown in
`close()` against callbacks that can still arrive.
Before this, a blocked socket woke on every timer tick and looked — costing a
wakeup per socket per 10 ms and answering up to 10 ms late even when the data
was already there. `tcp_sent` had to be registered for it as well, or a write
blocked on a full send buffer slept with nobody left to wake it.
#### ICMP, and ping sockets
`socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP)` — Linux's *ping socket*, which gives
an unprivileged process the one thing raw sockets were used for. The caller
reads and writes the ICMP message alone; the kernel owns the IP header.
Built on lwIP's raw API: `raw_new(IP_PROTO_ICMP)`, with the receive callback
stripping the variable-length IP header before queueing the message on the same
datagram queue UDP uses.
**Only echo replies are consumed**, and that restriction is load-bearing.
`raw_input` runs *before* lwIP's own `icmp_input`, so a callback returning
"eaten" for every ICMP message would swallow the echo **requests** too and this
machine would silently stop answering pings from outside. Returning 0 leaves the
packet for lwIP, which is what should happen to every ICMP message nobody asked
for.
There is no per-socket echo-id filtering. Linux rewrites the id and matches on
it; here a reply goes to every open ICMP socket, which is invisible with one
ping running and means a second one may see a sequence number it never sent.
### Loopback
`127.0.0.1` is a real destination: `LWIP_HAVE_LOOPIF` creates the interface and
`lwip_init()` brings it up. `LWIP_NETIF_LOOPBACK` is the half that is easy to
miss — without it, a packet addressed to an interface's *own* address goes out
of the wire and never comes back.
With `NO_SYS=1` there is no thread to drain the loopback queue, so the network
thread calls `netif_poll_all()` alongside `sys_check_timeouts()`.
### Datagrams
UDP has a **datagram queue**, not a byte ring. It shared TCP's ring at first, and
two datagrams arriving together came back as one — which is the bug that had to
be fixed before `recvfrom` could report a source address at all.
---
## What it does not do
- **`ping` is a first-party program**, `cmd/ping`, not toybox's. toybox's reads
replies with `recvmsg(2)` and asks for the TTL through an `IP_RECVTTL` control
message, and neither exists — `recvmsg` is item 1 of
[../plan/foundations.md](../plan/foundations.md), where it is wanted for
`SCM_RIGHTS` anyway. Ours has no `-I`, no flood mode and no TTL display.
- **No resolver reachable from userland.** lwIP's DNS client is compiled but
nothing exposes it, so `ping` takes a dotted quad. `localhost` is special-cased
in the program rather than resolved.
- **Receive is polled.** The network thread looks at the NIC every timer tick
when the link is idle, so an arriving frame waits up to 10 ms. Waking the
thread from the e1000 interrupt is the real fix, and was deliberately not the
first version: a receive path woken by an interrupt is much harder to debug
than one that is asked.
- **No routing table.** With two interfaces up, the first is the default route
and the second is reachable only by an address on its own subnet. That is a
rule rather than an accident, and it is not routing.
- **No `getsockname`/`getpeername`**, and `accept()` does not report the peer's
address for an AF_INET socket. A server that wants to log who connected cannot.
- **`setsockopt` covers six options**, and refuses the rest with `ENOPROTOOPT`
rather than pretending: `SO_REUSEADDR`, `SO_KEEPALIVE`, `SO_BROADCAST` and
`TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` are accepted and
report the real size on read. Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`.
- **`SOCK_DGRAM` over AF_UNIX does not exist** — stream only.
- **AF_UNIX names are not filesystem paths**, so nothing can `stat()` or
`unlink()` a socket.
- **One NIC driver exists.** The class is proven with two e1000 cards, not with
two different drivers.
- **No IPv6**, no DNS resolver in the kernel (lwIP's is compiled but unused by
anything), and no firewall or packet filter of any kind.

239
docs/architecture/schapi.md
View file

@ -1,239 +0,0 @@
# SchAPI, and SchOS as a system
Status: **direction agreed, not yet implemented.** Nothing in this document
changes what is being built this week; it changes what everything is being
built *towards*. See §7 for what actually happens now.
---
## 1. The decision
SchOS stops being "a Unix-like kernel with a userland bolted on" and becomes
a **system**: a kernel, a native API, and the services and applications that
sit on it, designed together.
The model is NeXTSTEP/macOS, not Linux. Specifically:
- A kernel that provides mechanism, with **more than one personality** on top.
- **POSIX/Unix is a personality, not the identity.** It stays, completely and
seriously, because it is how you get software. It is not the thing programs
are expected to be written against.
- **SchAPI** is the native interface: higher-level, object-shaped, designed for
the GUI and services rather than retrofitted onto 1970s process semantics.
Two systems are being borrowed from, and they are *different* systems. It is
worth being precise about which, because they disagree:
| Borrowed | From | Why |
|---|---|---|
| Layered architecture, personalities, native API over a Unix substrate | macOS / NeXTSTEP | It is the only mainstream design that got a modern API onto a Unix base without abandoning either |
| Permanent backward compatibility | **Windows** | macOS is emphatically *not* this — Apple dropped 32-bit apps, Carbon, kexts and OpenGL, each time on a deadline |
That mix is coherent, but it is a mix. Taking the architecture from Apple and
the compatibility policy from Microsoft is a deliberate choice, and the rest of
this document is mostly about making that choice survivable — because
compatibility-forever is the single most expensive promise an OS can make, and
Windows pays for it every day.
---
## 2. The one mechanism that makes the promise affordable
**The stable interface is a library, not the syscall table.**
This is the whole trick, and it is what lets macOS keep decade-old binaries
running while freely renumbering its syscalls. On macOS, nothing may issue a
syscall directly; you link `libSystem`. Apple therefore owes compatibility only
at the library boundary, and the kernel stays editable forever.
Linux made the opposite choice — the syscall ABI *is* the contract, "we do not
break userspace" — and consequently every syscall mistake ever made is still
there, with a `2` or a `64` suffixed to the name of its replacement.
So, for SchOS:
```
┌──────────────────────────────────────────────┐
│ applications, SchWM, SchDE │
├──────────────────────────────────────────────┤
│ SchAPI │ POSIX / Linux personality│ ← THE STABLE BOUNDARY
├──────────────────┴───────────────────────────┤
│ libschos — the only thing that may syscall│
├──────────────────────────────────────────────┤
│ syscall table — PRIVATE, unstable, renumberable
├──────────────────────────────────────────────┤
│ kernel │
└──────────────────────────────────────────────┘
```
Rules that follow, and they are not negotiable if the promise is to hold:
1. **A program that issues `syscall` directly is unsupported.** It may break at
any release. Only `libschos` may.
2. `libschos` exports versioned symbols. An entry point's behaviour never
changes; a changed behaviour is a new version of the symbol.
3. The kernel may renumber, merge, split or delete syscalls at will, provided
`libschos` still presents the same surface.
This *reconciles* with the existing decision to adopt Linux syscall numbering
(`docs/roadmap.md`, and the reason musl/toybox/mksh work at all). Linux
numbering does not go away — it becomes the **Linux personality's** entry path,
which is exactly what it already is in practice. It is a compatibility surface
that happens to be entered via `syscall`, and it is frozen because *Linux*
froze it, not because we did.
---
## 3. Stability tiers
"Nothing can ever be removed" is only affordable if you are precise about what
"anything" covers. Windows' real problem was never the documented API — it was
that undocumented behaviour became load-bearing, and then it was too late.
Every public interface carries exactly one tier:
| Tier | Promise | May change |
|---|---|---|
| `stable` | Forever, from the release that introduced it | Never. Additions only. |
| `provisional` | Until the next release | Freely |
| `internal` | None | Freely, without notice |
**Everything in SchOS today is `provisional`.** The freeze starts at 1.0 and
covers only what is marked `stable` at that moment. This is not a way of
dodging the commitment — it is the only way to make it, because a promise made
now would permanently enshrine, among other things, a `fcntl(F_SETFL)` that
was a documented no-op until last week and a `struct file` whose `O_NONBLOCK`
handling had never been exercised.
Nothing becomes `stable` by being old. It becomes `stable` by someone deciding
it is right and writing that down.
---
## 4. `compat/` — what it actually means
The instinct is right: old interfaces should not litter the current tree. But
moving *source* into a folder does not reduce the *runtime* cost, and the
runtime cost is the one that hurts.
The rule that keeps compatibility cheap:
> **A compatibility entry point must be implementable as a translation into the
> current API. If it cannot be, that is a defect in the current API — fix the
> current API before freezing it.**
So `compat/` holds shims, not parallel implementations. A shim:
- lives in `compat/<release-it-was-current-in>/`,
- is written in terms of today's API and nothing else,
- is never called by new code,
- carries its own tests, which are the only thing that proves the promise is
still being kept.
When a shim *cannot* be expressed as a translation, that is the alarm. It means
the new API lost information the old one carried, and it is far cheaper to
notice that before 1.0 than to maintain two real implementations forever. That
is precisely the trap Win32 fell into.
---
## 5. Taking from the BSDs
`src-trunk/` is a NetBSD checkout, and a FreeBSD one is being added alongside
it. Both are reference material and parts bins; neither is built and neither
may ever end up in an image.
**Two trees is not redundancy.** They are worth having together for three
reasons that a single tree cannot give:
- Where an RFC is ambiguous — and TCP's are, constantly — two independent
implementations agreeing is far stronger evidence than one implementation
asserting.
- Where one is licence-encumbered, the other may not be. NetBSD's
`tcp_input.c` carries a WIDE Project advertising clause that Berkeley's
rescission does not cover; FreeBSD's equivalent may well be 2-clause. See
[../licensing.md](../licensing.md) §3.
- They have opposite biases, and knowing which you are reading matters:
| | FreeBSD | NetBSD |
|---|---|---|
| Stack quality | The better stack in absolute terms — faster, more complete, more actively developed | Solid, conservative, less featureful |
| Extractability | Welded to FreeBSD: `epoch(9)`, `uma`, `netisr`, VIMAGE/vnet, `if_t`, a rich locking vocabulary | Portability *is* the project's identity; assumes far less of its host kernel |
| Use it as | The reference you check yourself against | The code you actually lift |
That table is the same lesson §1 of the roadmap already learned the hard way
with FreeBSD's `sys/dev/usb`: 100k lines welded to newbus, where porting the
USB stack means porting the device framework first. Networking is the same
shape. Read FreeBSD, lift NetBSD, write SchOS.
### Licensing — read this before copying anything
NetBSD is not uniformly licensed: 2-clause, UC 3-clause and a minority of
4-clause-with-advertising files all appear, and only the first two are
comfortable. The per-template rules, the measured counts, and the one-line
check to run before copying a file are in [../licensing.md](../licensing.md)
§3. Do not copy from `src-trunk/` without reading it.
### What is actually worth taking
Ranked by value, not by size:
1. **`sys/net`, `sys/netinet` — the TCP/IP stack.** The largest single win
available. Phase 2 of the roadmap wants ARP/IPv4/ICMP/UDP/TCP, and this is a
mature, clean, readable, 2-clause-heavy implementation of exactly that.
2. **`sys/kern/subr_pool.c` — the pool/slab allocator.** SchOS has `kalloc`
and a page at a time. Every subsystem written since has invented its own
fixed-size table to work around that.
3. **`autoconf` — the driver framework** (`config(1)`, `cfdriver`/`cfattach`,
`device_t`). This is the grown-up version of what `kernel/drivers/registry.c`
and `kernel/dev/irq.c` are hand-rolling. Adopt the *model*; do not try to
port the code wholesale.
4. `sys/kern/subr_*` generally — rbtree, workqueue, condvar, kmem.
Low value: `lib/libc` (musl already fills that role), `sys/dev/usb` (we have a
working xHCI stack written from the spec).
**Take designs before code.** Every file copied is a file to be understood,
relicensed-in-notice, and maintained forever under the §3 promise.
---
## 6. What SchAPI is, concretely
Not yet designed. What is decided about it:
- It is the native interface; POSIX is available beneath it but not the idiom.
- It is reached through `libschos`, never by direct syscall.
- It is designed *with* Schcom in mind, because a compositor and its clients
are the first real users and their needs (IPC, shared buffers, event streams,
object lifetimes) are what will shape it.
- The IPC primitive comes first. Everything in a NeXT-shaped system rests on
it, and unix domain sockets — already built and working — are the substrate,
not the interface.
What is explicitly *not* decided: whether it is object-oriented in the Objective-C
sense, what language it is expressed in, and whether it takes Mach's port model
or something smaller. Those need their own document, written when Schcom has
told us what it actually needs.
---
## 7. What changes now
Almost nothing, on purpose. The ordering does not change:
- **The native Windows build must boot first.** The clang-built kernel
currently reaches `sinit` and dies on the first trap. A direction is worth
nothing while the tree does not run.
- Then the roadmap continues as written: network, GPU, fonts, Schcom.
What this document changes *today* is only this:
1. `src-trunk/` is reference material, excluded from every build and image.
2. New interfaces are marked `provisional` from now on, so that §3 has
something to work with at 1.0.
3. When the network stack is written, NetBSD's is the starting point rather
than a blank file — with the licence check in §5 applied per file.
4. `libschos` is created when there is a second personality to justify it.
Until then, the *rule* stands even though the library does not exist yet:
no new first-party program issues raw syscalls.

478
docs/architecture/schfs.md
View file

@ -1,21 +1,31 @@
# SchFS — design
# SchFS
**Status: phases 1 through 3½ are done, and SchFS is the root filesystem.**
`kernel/proc/proc.c` mounts it before pid 1 has a working directory and panics
if it is not there. The copy-on-write B-tree, the volume layer, the object
layer, the write path, snapshots, an image writer and a checker all exist, and
several volumes can be mounted at once. Phase 4 (extent refcounts) is next.
See §8 for exactly what is built and how to run its tests.
The root filesystem. A copy-on-write B-tree with an uberblock ring, 64-bit
object ids, checksummed metadata and extents.
Supersedes the on-disk format described in `include/fs.h`. Keeps the name SchFS;
the on-disk magic changes so v1 images are detected and rejected rather than
silently misread.
**It is the root.** `forkret()` mounts it before PID 1 has a working directory
and panics if it is not there. All five design phases are complete: the B-tree,
read-only mount, the write path and transaction groups, the commit barrier and
uberblock fallback, extent refcounts and reflink, and attributes with an index
and a query.
The brief was "the best aspects of BeFS, ZFS and btrfs." Those three disagree
with each other in places, so this document says explicitly what is taken from
each and what is deliberately left behind.
The phases themselves, and the bugs they produced, are in
[../history/schfs.md](../history/schfs.md). The VFS it plugs into is
[filesystem.md](filesystem.md).
## 1. What each one is actually good at
---
## The argument in one line
A crash leaves either the old tree or the new one, so **there is nothing to
repair** — and free space is derived from the tree rather than stored beside it,
so **there is no second structure to disagree with the first**.
`/proc/schfs` shows the generation counter advancing as writes commit.
---
## What was taken, from where
**ZFS — never be inconsistent.** Copy-on-write: no block that is part of the
live tree is ever overwritten. An update writes new blocks, then atomically
@ -41,20 +51,20 @@ also the right *scale* model: a small team, a readable codebase, and a book
(Giampaolo, *Practical File System Design*, free online) that describes it
end to end.
## 2. What is deliberately not taken
## What was deliberately not taken
- **No ZFS storage pools / RAID-Z / vdevs.** Multi-disk redundancy is a large
subsystem that solves a problem SchOS does not have yet.
- **No btrfs-style rebalancing or online format conversion.** Enormous
complexity, and it is the historical source of btrfs's worst bugs.
- **No actual SQL.** No parser, no query planner, no relational engine. See §6 —
- **No actual SQL.** No parser, no query planner, no relational engine. See *Attributes, indexes and the query* below —
the useful part of "database filesystem" is the *index*, not the language.
WinFS spent 2003-2006 and a large team on the other interpretation and
shipped nothing.
- **No deduplication.** ZFS's dedup is famous for needing more RAM than anyone
expects. Extent refcounts give reflink/snapshot benefits without it.
## 3. On-disk shape
## On-disk shape
```
+--------------------+ block 0 boot block (untouched by the FS)
@ -98,278 +108,7 @@ checksum fields in `dinode` and `superblock` — same algorithm, better placemen
Block size moves 1024 → 4096 to match modern media and make extents worthwhile.
## 4. What already exists and gets reused
Not starting from zero:
- **`struct schfs_extent`** is already defined in `include/fs.h` and unused.
The extent concept is already sketched; v2 gives it refcounts and 64-bit
fields.
- **CRC32c checksum fields** already exist in `dinode` and `superblock`, and
mkfs already computes them. Same algorithm, relocated into parent pointers.
- **`kernel/fs/log.c`** is a working transaction log with `begin_op`/`end_op`
call sites throughout the FS. Under CoW the log is not needed for consistency,
but `begin_op`/`end_op` are exactly the right places to bracket a transaction
group, so the *call sites* survive even though the implementation is replaced.
- **The buffer cache** (`bio.c`) is format-agnostic and needs no change beyond
the block-size constant.
## 5. Phasing
Each phase leaves a bootable system. Do not start phase 2 before phase 1 is
solid — the B-tree is the load-bearing piece and everything else assumes it.
**Phase 1 — the B-tree, in userland first.** Write the CoW B-tree as a plain C
library with a block-device abstraction, and test it on a *host* file with a
fuzz/torture harness (random insert/delete/lookup, simulated crash at every
write point, verify the tree always mounts). This is the one component where a
subtle bug is unfixable later, and testing it in the kernel is far harder. It is
also portable straight into `tools/mkfs`.
**Phase 2 — read-only mount. DONE.** New magic, uberblock scan, tree walk, inode
and dirent key types. `tools/mkschfs` writes images; the kernel only reads them.
The system boots from a ramdisk root (see `docs/layout.md` and the ramdisk work)
so a broken disk cannot stop you from booting.
Went further than planned in two places, both cheaply: data blocks carry
checksums too (a `CSUM` key type, as btrfs does — without it "end-to-end
checksums" would cover only metadata, which is the part least likely to rot),
and extents landed early because reserving a file's blocks in one go was
simpler than not doing it.
Not yet done: SchFS files are not reachable through `open(2)`. The mount is
observable through `/proc/schfs` and through the kernel API in `schfs_int.h`,
but wiring it into `namei`/`readi` means the VFS work that phase 3 needs anyway.
**Phase 3 — writes, transaction groups. DONE.** Every `schfs_k*` call that
changes anything is one transaction group, committed by writing a new uberblock
before it returns. The full write API — create, link, unlink, rename, write at
an arbitrary offset, truncate — lives in `objw.c` and is exercised by
`tools/schfs_objtest.c`.
Two things went differently from the plan above, both for reasons worth
recording.
*Free space is not tracked as tree keys.* Allocating from a free map that lives
in the tree mutates the tree, which allocates, which changes the map; btrfs
resolves that with a fixpoint loop and ZFS with log-structured space maps. SchFS
sidesteps it: the committed tree already says exactly which blocks are live, so
the map is **derived** by walking the tree once at mount and held only in RAM.
It cannot disagree with the filesystem, because it is computed from it, and a
crash raises no question at all. The cost is mount time proportional to metadata
size, and a ceiling on volume size set by how much bitmap fits in memory. If
mount time ever hurts, the map can be cached on disk purely as a hint,
validated against the generation and discarded on a mismatch — an addition, not
a format change. See `kernel/fs/schfs/space.c`.
*Data blocks needed a generation stamp too.* The B-tree already rewrites its own
nodes in place within a group, since a block that no committed root can reach is
nobody's fallback. Data blocks had no such marker, so every hundred-byte write
copied a whole 4 KB block and split an extent record around it. `schfs_extent`
gained a 64-bit `gen`, and the same rule now applies to file data.
**Phase 3½ — reachable through `open(2)`. DONE.** SchFS is a mounted filesystem:
```
mkdir /mnt
mount -t schfs /dev/hdb /mnt
ls /mnt ; echo hi > /mnt/tmp/w ; cat /mnt/tmp/w ; mkdir /mnt/tmp/d
```
The VFS grew a real dispatch layer rather than a set of `if (dev == SCHFS_DEV)`
checks. `struct fs_ops` (include/file.h) is one table per filesystem, chosen by
`iget()` from the device number and hung off the inode; v1's implementations
keep their bodies and are simply named `v1_*`. Adding a third filesystem is a
new table and nothing else. `kernel/fs/schfs/vfs.c` is SchFS's, and it is pure
translation — object ids in, inodes out — because all the filesystem logic
already lives below it.
Three things had to change on the way, none of them SchFS's fault:
- `struct inode.inum` and `.size` were 32-bit, capping objects at 2³² and files
at 4 GiB on a filesystem that promises far more. Both are `uint64` now, as are
`readi`/`writei` offsets and `struct file.off`.
- `struct dirent` carries a 16-bit inum and a 14-byte name, so it cannot
represent a SchFS entry at all. `getdents64` is now an fs_op: each filesystem
emits the record stream itself, and `f->off` is whatever position it needs to
resume. v1 walks its hash buckets; SchFS walks a key range and synthesises
"." and "..", which it does not store.
- Mounting is a substitution made during path resolution (`mount_add` /
`mount_cross` in `fs.c`). Nothing on either disk records it. The reverse
direction matters as much as the forward one: `..` at a mounted root has to
leave the mount, and the filesystem underneath has no idea it is mounted
anywhere.
Three long-standing bugs surfaced the moment real commands ran against it, all
in the shared VFS rather than in SchFS:
- `include/fcntl.h` used xv6's open flags while every program in the userland
sends Linux's. `O_APPEND` (0x400) landed on `O_TRUNC`, so `cmd >> file`
*truncated* it; `O_TRUNC` (0x200) landed on `O_CREATE`, so `cmd > file` left
whatever was past the new end in place. Nothing about either pointed at a
header.
- `sys_openat` silently dropped `O_TRUNC` and `O_APPEND` while `sys_open`
honoured both, so the same redirection behaved differently depending on which
entry point libc happened to call.
- `rmdir` had no Linux syscall number, so every Linux-ABI program got ENOSYS.
**Phase 3¾ — durability, and the cost of it. DONE.** Three things that the
earlier phases had assumed rather than arranged.
*The commit barrier.* Copy-on-write promises that a crash lands on either the
old tree or the new one, but only if the blocks a root points at reach the
medium *before* the block naming that root. Nothing enforced that: `bwrite`
order is not platter order on any drive with a write cache. `struct schfs_bdev`
gained a `bflush` hook, `schfs_vol_commit` calls it either side of the uberblock
write, and the kernel implements it as "drain this device's dirty buffers, then
ATA FLUSH CACHE" (`ideflush`, and SCSI SYNCHRONIZE CACHE for a USB stick).
*Falling back through the ring.* Mount took the highest-generation uberblock
that validated, checked the tree it named, and gave up if that failed — with up
to 127 older, entirely intact roots sitting on the disk. It now walks the ring
newest-first and adopts the first root it can actually read, counting what it
skipped into `vol->recovered` so the rollback is reported rather than silent.
*Deferred writes, and brackets.* The B-tree seals and writes a node on every
record it inserts, so a leaf collecting one file's data checksums went to the
disk once per 4 KB of that file. The kernel's block hook marks buffers dirty
instead (`bdirty`), and the commit barrier is what makes that safe. On its own
that changes little, because a commit *contains* a flush and every `schfs_k*`
call was one commit — so `schfs_ktx_open`/`schfs_ktx_close` bracket a run of
calls into one group, and `schfs_vfs_write` wraps a whole `write(2)` in one.
Measured over 1 MB of file data, in block writes reaching the device:
| | before | after |
|---|---|---|
| committing every 4 KB | 1947 | 1410 |
| one commit for the megabyte | 1798 | **267** |
267 device writes for 256 blocks of data. Note that the two changes are worth
almost nothing separately — deferral cannot help while every commit flushes,
and batching cannot help while every write goes straight to the disk — which is
why the first attempt at this measured an 8% difference and concluded batching
was not the problem.
**Phase 3⅞ — where the time actually went. DONE.** The filesystem was correct
and durable and the system still felt slow: `ls /bin` took the better part of a
second. Two causes, neither of them the disk.
Measured first, through `/proc/fsstat` (counters added for exactly this). The
decisive number was **398,999 block reads against 445 actual disk misses** —
99.9% of all block reads were re-reading a block already in the buffer cache. So
the disk was never the problem, and every plan that started with "cache more" or
"read ahead" would have been aimed at 1% of the cost.
*The cursor re-read its own buffer.* `cur_load` called `node_read`
unconditionally, and `schfs_cursor_val` calls `cur_load` on **every access to
the current record** — deliberately, so that a callback disturbing the shared
block buffer cannot corrupt a directory walk. But `node_read` copies 4096 bytes
out of the buffer cache and then runs CRC32c over all 4096. Listing a 243-entry
directory paid both once per entry. The cursor now remembers which block its
buffer holds and skips the reload, cross-checked against the block header (so a
second cursor sharing the page cannot fool it) and against a global epoch that
every mutation bumps (so a block rewritten in place cannot be served stale).
*The checksum was bit-at-a-time.* `schfs_crc32c` did eight shifts per byte —
32,768 iterations to checksum one block — to stay usable before any
initialisation. It builds a 1 KB table on first use now, which keeps that
property and is eight times faster. This only became the top cost *after* the
cursor fix, which is the usual shape: removing the largest item promotes
whatever was beneath it.
| cycles, one session | before | after |
|---|---|---|
| `getdents` | 26.8 G | **0.14 G** (196×) |
| `read` | 26.4 G | **5.5 G** (4.8×) |
| `exec` | 18.7 G | **4.0 G** (4.7×) |
| block reads | 398,999 | **127,713** |
`ls /bin` went from ~400 ms to ~20 ms. Verified by reading an 802 KB binary
through the whole path and checking its md5 against the host's copy.
There was also a third of the problem that was not the kernel at all: `make
qemu` ran QEMU without `-enable-kvm`, so every instruction was emulated. The run
targets probe `/dev/kvm` now. Measure against the KVM number or you are
measuring TCG.
**Phase 4 — extents with refcounts. DONE.** `cp` that costs nothing.
A reference count per run of blocks, in its own tree (`SCHFS_KEY_EXTREF` under
`SCHFS_EXTREF_OBJECTID`) rather than in the file's own extent record — because a
count kept per file cannot be decremented by the *other* file sharing the run.
`schfs_file_reflink` copies one extent record per run and bumps the count; both
files stay ordinary and writable, and the first write to either copies only the
run it touches. Reachable from userland as `FICLONE`, Linux's ioctl, so
`cp --reflink` works unmodified; `cmd/reflink` is a two-line caller of it.
Measured on the running system: cloning a 411 KB file moved `df` by 24 KB.
Three things had to be right, and each was wrong first:
- **The count is "how many extent records point into this run", not "how many
files".** Splitting a shared extent around a written block leaves a head *and*
a tail, which is one more reference than before. Missing that leaked four
blocks per split.
- **A file's extent may point into the middle of a run.** Records are keyed by
the block a run was *allocated* at, so every lookup is a range query
(`extref_find`), not an exact one. An exact lookup finds nothing for a split
tail, "nothing" means unshared, and the tail is then freed while another file
still reaches it.
- **The space scan has to walk the reference records too.** A file truncated
inside a shared run describes fewer blocks than exist; the rest would be
derived as free while still referenced, and the allocator would hand out a
block that a later release frees a second time.
None of those is visible until every holder of a run is deleted, which is why
the test that matters is the least glamorous one: after deleting every file that
shared a run, free space must be back to exactly what it was.
**Phase 5 — attribute index and query. DONE.** The reason the design was one
B-tree with many key types from the start: the "database filesystem" feature
turned out to be `kernel/fs/schfs/attr.c` and nothing else.
An attribute is a named value on an object, stored under `(ino, ATTR,
hash-of-name)` and packed exactly as directory entries are — same collision
argument, same shape. An attribute may be *indexed*, and an indexed attribute
gets a second set of records keyed by value: `(hash-of-name, INDEX,
hash-of-value)` holding the object ids that carry it. So "every object whose
`tag` is `holiday`" is one tree descent instead of a walk of every object.
There is no query language, no parser and no planner, and §6 above explains why:
the useful half of "query the filesystem" is the index, and WinFS spent three
years and a large team on the other interpretation.
```
schattr index tag # declare the index
schattr set /tmp/a tag holiday
schattr find tag holiday # one descent, whatever the volume holds
```
Four decisions worth keeping:
- **Which attributes are indexed is cached in the volume**, read once at mount.
Every `setxattr` has to ask, and the answer is almost always no; a tree
descent to say so would make every attribute write pay for a feature it is
not using.
- **The value is stored in the index entry, not just its hash.** Two values can
hash alike, and a query that returned the wrong file would be far worse than a
slow one.
- **An index covers values written after it is declared.** There is no
reindexing pass. Saying so is better than a half-truth that makes a query
answer confidently and wrongly.
- **A query on an unindexed attribute is refused, not answered by scanning.** A
query that silently degrades into a full walk teaches you to use it where it
will not scale.
Reachable as `setxattr`/`getxattr`/`removexattr` at Linux's syscall numbers, so
anything that already knows how to attach metadata needs no changes, plus two
native calls (`schfs_index`, `schfs_query`) that have no Linux equivalent
because asking a filesystem a question does not.
**Phase 5 — attribute index and query.** See below.
## 6. The "SQL" part, honestly
## Attributes, indexes, and the query
What you want from "works like SQL" is: *ask a question about a lot of files and
get an answer fast, without walking every directory*. That is an **index**, not
@ -382,28 +121,14 @@ a query language. BeFS's design:
index ranges.
In SchFS this is just another key type — `(idx, INDEX, key)` — in the tree
that already exists. That is the whole reason to build phases 1-4 the btrfs way:
that already exists. That is the whole reason the tree was built the btrfs way:
the "database filesystem" feature becomes a few hundred lines on top of
machinery already written and debugged, rather than a separate engine.
A concrete `query()` syscall and a `find`-like tool come after that, and are
small by comparison.
## 7. Honest cost
Phases 1-3 are roughly 3,000-4,000 lines and are the bulk of the work. Phase 1
alone (B-tree + torture harness) is worth doing carefully and unhurriedly; every
later phase compounds on it.
This is the largest single piece of work in SchOS. It is also entirely
ordinary filesystem engineering with excellent published references
(Giampaolo for BeFS; the btrfs design papers; the ZFS uberblock/checksum-in-
parent scheme) — there is no research risk here, only volume.
**Do this before the installer.** The installer's entire job is to create and
populate this format. Building it against v1 means building it twice.
## 7½. Limits
## Limits
Three different things get called "the limit", and they are worth keeping apart:
what the **on-disk format** can express, what **this implementation** can
@ -464,7 +189,75 @@ filesystem operation now, so SchFS emits its own records with 64-bit inode
numbers and names up to 255 bytes, and the inode carries 64-bit numbers and
sizes throughout.
## 8. What exists now
---
## Format version, and what a bump means
Two mechanisms, because one is not enough. `SCHFS_VERSION` changes only when
the **frame** changes — the uberblock, the node header, the key, the things any
reader must understand before it can parse anything at all. Everything else is
a **feature flag**, in one of three classes:
| Class | An unknown bit means |
|---|---|
| `compat` | Ignore it. Mount read-write. |
| `ro_compat` | Mount **read-only**. A writer would maintain a structure this driver does not know about, and writing would leave it stale. |
| `incompat` | **Refuse.** The volume cannot be parsed correctly without it. |
Choosing the class is the whole design decision, and the test is one question:
*what happens if a driver that does not know this feature writes to the volume?*
Nothing → `compat`. Something on disk goes stale → `ro_compat`. Something
becomes unreadable → `incompat`.
The flags occupy what was `reserved0`, so every field after them keeps its
offset and existing images — which have zeros there — read identically.
`schfs_vol_commit` memsets the uberblock before filling it in, which is why
volumes written before the field existed already say "no features". **That trick
is available exactly once**; the next addition costs a version bump.
Four `_Static_assert`s pin the uberblock to one block and pin `version`,
the feature words and the field after them to fixed offsets, because those are
on-disk facts rather than C conveniences.
### Refusing is not falling back
The uberblock ring exists so that a damaged newest uberblock can be stepped
over — that is crash recovery, and the older root is a consistent filesystem.
A newest uberblock that is *sound but too new* is the opposite case, and
treating the two alike is how a mount quietly loses data:
> Falling back mounts a **stale** filesystem. Every commit made since the volume
> was upgraded is discarded, the mount reports success, and nobody finds out.
So `uber_check()` answers with a state rather than a yes/no —
`BAD` / `OK` / `RO` / `TOO_NEW` — and the scan stops on `TOO_NEW` instead of
continuing to the next candidate. The order of its checks is the order of the
questions: is it ours (magic), is it intact (crc), can we read it (version,
features), does it make sense (root in range). Compatibility is asked before
sanity deliberately — the fields a sanity check reads are only meaningfully
placed if the format is one we know.
Demonstrated rather than asserted, on a forged image whose two uberblocks are
`(version 1, generation 1, root 129)` and `(version 2, generation 2, root 130)`:
the old code mounted root 129 and exited 0, discarding the live filesystem; the
current code refuses and says why.
---
## One source, two compilers
Everything except `kmount.c` is compiled **both** into the kernel and into the
host tools.
> The writer and the reader cannot drift apart, because there is one
> implementation of the format and nothing to disagree with.
That is why `mkschfs` cannot produce an image the kernel misreads, and why
format bugs are found on the host where a test takes a second instead of a boot.
---
## The source
```
include/schfs.h on-disk format + compile-time layout assertions
@ -490,7 +283,7 @@ Everything except `kmount.c` is compiled *both* into the kernel and into the
host tools, so the writer and the reader cannot drift apart: there is one
implementation of the format and nothing to disagree with.
### Running the tests
### The test suites
```
make schfs-check all three suites
@ -526,7 +319,7 @@ cutting the power one write later each time, and after every one the volume must
mount, check out structurally, and still hold the data committed before it.
`tools/posix/mutate.sh` injects six deliberate bugs and fails if any of them goes
undetected. It is how the suites are kept honest; see the note in §9.
undetected. It is how the suites are kept honest; see [../history/schfs.md](../history/schfs.md).
### Verifying on real hardware
@ -539,53 +332,26 @@ make qemu-schfs then, at the shell: cat /proc/schfs
`.\run.ps1 -SchFS` does the same on Windows, and `tools\bootcheck.ps1 -SchFS`
does it unattended and asserts on the transcript.
## 9. Things learned the hard way
---
Recorded because each one cost real time and none is obvious from the code.
## What it does not do
- **The root's object id must be reserved.** `next_objectid` started at
`SCHFS_FIRST_OBJECTID`, which *is* the root's id, so the first directory
created silently became the root. It then contained itself, and every tree
walk recursed until the stack ran out. `schfs_fsck --check` now detects
directory loops explicitly rather than crashing.
- **Cursor buffers alias.** A range callback gets a pointer into the cursor's
block buffer, and almost every readdir caller immediately looks the entry up,
which overwrites it. Directory iteration copies each entry out before calling
back. Extracting a tree with directories reparented under their own children
is what this looks like when it goes wrong.
- **A 4096-byte block does not fit on a 4096-byte kernel stack.** The cursor
exists precisely to avoid one block buffer per tree level, but
`schfs_vol_open` still kept a whole uberblock as a local and blew the stack on
the first boot. The kstack canary caught it cleanly.
- **Extents only merge if you allocate the run first.** Writing an extent record
allocates blocks for its own copy-on-write path, so allocating data one block
at a time guarantees the blocks are never adjacent and every block needs its
own extent. `schfs_file_reserve` claims the whole run up front; an 822 KB
binary is one extent instead of 201.
- **Freed blocks are reusable during a build.** Copy-on-write forbids reusing a
released block because the previously committed tree still points at it — but
a build has committed nothing. Recycling them cut the test image from 3596
blocks to 374.
- **Reproducibility dies from atime.** mkschfs reads every source file, which
updates its atime, so copying atime into the image made two builds of the same
tree differ. All three timestamps come from mtime.
- **An optional hook must still be initialised.** `struct schfs_bdev` gained
`bflush`, and every host tool built one on the stack field by field. The tools
that did not set the new field handed `schfs_vol_commit` a garbage pointer to
test and call, which segfaulted at the first commit. Each of those helpers
memsets the struct first now — the fix is not "remember to set it".
- **Deferring writes makes an all-dirty buffer cache normal.** `bget` panicked
with "no buffers" when it could find nothing clean to recycle, which was
correct while a dirty buffer meant "the log has not committed". It now writes
the least recently used victim out and takes it; a transaction touching more
than `NBUF` blocks reaches that state long before it commits.
- **The two performance fixes are worthless apart.** Deferring tree writes buys
nothing while every commit flushes, and batching commits buys nothing while
every write goes straight to the disk. Measuring either alone says "this is
not where the time goes", and both measurements are correct.
- **`fsid_t` is eight bytes.** `struct statfs` declared `f_fsid` as `int64[2]`,
which is one slot too long, so everything after it landed in the wrong place:
userland read `f_frsize` out of `f_namelen` and got 255. df then computed in
units of 255 bytes and called a 32 MB root filesystem 1992 KB. It survived
because every field in that struct used to be an invented constant, so no
number in it had ever been checked against reality.
- **No snapshots.** The B-tree is copy-on-write, which is most of the
machinery — nothing takes or names a snapshot. This is the missing feature the
existing design already pays for.
- **No journal replay** beyond what copy-on-write gives for free.
- **No sparse files.** A hole is not representable; writing at a large offset
allocates everything before it.
- **No online `fsck`.** `schfs_fsck` is a host tool and the volume must not be
mounted.
- **No upgrade path.** The version policy below refuses a volume it is too old
for; nothing converts one format to another, because there is only one format.
The first bump is when that has to be written.
- **No Linux driver.** FUSE first, then possibly a kernel module, and not before
the format is pinned.
- **No deduplication**, deliberately — ZFS's is famous for needing more RAM than
anyone expects, and extent refcounts give the reflink and snapshot benefits
without it.
- **No multi-disk anything** — no pools, no RAID-Z, no vdevs.
- **No rebalancing or online format conversion**, deliberately: enormous
complexity, and historically the source of btrfs's worst bugs.

131
docs/architecture/syscalls.md Normal file
View file

@ -0,0 +1,131 @@
# Syscalls, and the Linux personality
Two things happen at this boundary. SchOS has its own syscall numbering, and it
also answers Linux's — which is why musl, toybox and mksh run unmodified.
---
## One table, everything generated
**Syscalls are declared in exactly one place:**
[`kernel/sys/table.inc`](../../kernel/sys/table.inc). One line each.
```c
SYSCALL(nr, linux_nr, name) // implemented as sys_<name>()
SYSCALL_ALIAS(linux_nr, name) // a second Linux number for the same handler
```
Three things are generated from it by macro expansion, and none is
hand-maintained:
| Generated | Where |
|---|---|
| The `SYS_*` constants | `include/syscall.h` |
| The dispatch table | `kernel/sys/syscall.c` |
| The Linux translation map | `kernel/sys/linux_abi.c` |
Adding a syscall is therefore two edits: a line here, and a body written with
`SYSCALL_DEFINE(<name>)` in one of the `kernel/sys/sys_*.c` files.
**This replaced a two-place scheme and the reason is worth keeping.** The Linux
number used to live in a second hand-written table. Adding a syscall meant three
edits across two files, and forgetting the second one left the call *unreachable
from every musl-linked program* while looking perfectly registered in the first.
A syscall can no longer be half-registered.
## How a Linux binary runs
The translation is a flat array indexed by Linux syscall number, whose value is
the native `SYS_*` number — or 0, meaning not implemented, which dispatches to
`-ENOSYS`.
It is enabled **per process**, and selected automatically by `exec()`:
```c
// musl binaries have entry >= 0x100000; SchOS-native ones are linked at 0x4000
if (elf.entry >= LINUX_ABI_ENTRY_THRESHOLD)
myproc()->syscall_map = linux_syscall_map;
```
A link address is a crude discriminator and it is honest about being one. It
works because the two classes of binary in this system are linked by two
different linker scripts at two very different addresses.
There is also `enable_linux_abi()` as an explicit call, which is why
`cmd/schinit/main.c` carries a warning not to use it: doing so mid-process would
renumber the syscalls the caller still has to make.
### Native vs. Linux numbering
Some calls have no Linux equivalent and are marked `NO_LINUX`: `sleep`,
`uptime`, `reboot`, `shutdown`, `getmouse`, `vga_sync`, `schfs_index`,
`schfs_query`. These are reachable only from first-party programs linked
against `lib/`.
They all land on the same scratch index in the translation array and overwrite
each other there, which is harmless — no real Linux syscall uses that number, so
the slot is never dispatched.
### Where the two disagree on a name
`shutdown` is the one collision. SchOS has a native `shutdown()` that powers the
machine off, and Linux number 48 is the socket call. Two syscalls cannot share a
`sys_` function name, so Linux 48 maps to `sockshutdown`.
## What is implemented
Roughly 190 native numbers covering:
| Area | |
|---|---|
| Process | fork, exec, exit, wait/waitpid/wait4/waitid, clone, kill, sessions, process groups |
| Identity | the full real/effective/saved set for both uid and gid, groups, umask |
| Memory | mmap, munmap, mprotect, brk/sbrk, madvise, System V shared memory |
| Filesystem | the ordinary set, the whole `*at()` family, statfs, xattr, mount/umount2 |
| I/O | read/write, readv/writev, pread/pwrite, sendfile, poll/select/ppoll/pselect6, epoll, ioctl, fcntl |
| Sockets | socket, bind, listen, accept, connect, sendto, recvfrom, socketpair, setsockopt, getsockopt |
| Signals | rt_sigaction, rt_sigreturn, rt_sigsuspend, rt_sigpending, rt_sigtimedwait, sigaltstack |
| Time | gettimeofday, clock_gettime, clock_getres, nanosleep, clock_nanosleep, alarm, timers |
| Notification | eventfd2, timerfd_create/settime/gettime, signalfd4 |
### Argument fetching
`fetchint`, `fetchaddr`, `fetchstr` and `argptr` validate a user pointer through
the page tables and nothing else. They previously rejected any address
`>= proc->sz`, on the assumption that every valid user pointer lives in the
heap — which is wrong for stack addresses and for every `mmap`'d buffer, i.e.
for most pointers real programs pass.
`uva2ka()` translates a user virtual address and **returns the in-page offset**,
which it did not originally do. Every caller that read or wrote through a
non-page-aligned pointer — `TIOCGPGRP`, `TCGETS`, `getcwd`, signal masks,
`times()`, `sysinfo()` — was touching the wrong address whenever the pointer was
not page-aligned, i.e. almost always.
**`uva2ka()` guarantees only the page containing that address.** A structure
copied in bulk through one translation runs off the end of the mapped page when
the pointer lands near a boundary — and `P2V` is a flat offset, so "the next
page" in kernel space has nothing to do with the user's next page. That bug
corrupted unrelated kernel memory and crashed every freshly `exec`'d shell. Copy
field by field, each through its own translation, or use `copyin`/`copyout`.
---
## What it does not do
- **No `sendmsg`/`recvmsg`**, and therefore no ancillary data and no
`SCM_RIGHTS`.
- **No `memfd_create`.**
- **No `getsockname` or `getpeername`**, and `accept()` does not report an
AF_INET peer's address.
- **`setsockopt` handles six options** and answers `ENOPROTOOPT` for everything
else. `SO_RCVTIMEO`/`SO_SNDTIMEO` are among the refused.
- **`clone` does not share an address space**, so it is `fork` with extra steps.
- **`sched_setaffinity` is accepted and ignored.**
- **`getrusage` does not distinguish `RUSAGE_CHILDREN`** from `RUSAGE_SELF`, and
fills in only the calling process's own tick count.
- **The ABI is chosen by link address.** It works, and it is a heuristic rather
than a fact about the binary.
- **`/proc/<pid>/exe` is a regular file, not a symlink**, so `readlink()` on it
fails and `ps -o comm` prints `[init]` rather than `init`. Default `ps` and
`top` output is unaffected.

116
docs/architecture/usb.md Normal file
View file

@ -0,0 +1,116 @@
# USB
An xHCI host controller driver, a device-independent core, and three class
drivers. Written from the specification.
---
## Why not ported from BSD
"Take the USB stack from BSD" is the obvious plan and it does not survive contact
with the code.
FreeBSD's `sys/dev/usb` is roughly 100k lines welded to newbus, devclass,
`bus_dma`, callouts, taskqueues and FreeBSD's locking primitives, none of which
exist here. **Porting it means porting FreeBSD's device framework first and then
discovering the USB stack underneath.** NetBSD's is more portable and the same
argument applies with a smaller multiplier.
The specification is public and the controller is not complicated.
> BSD is much more valuable as the reference you check yourself against than as
> the code you paste.
Licensing points the same way: the BSD stacks are BSD-2/3-clause and compatible,
Linux's drivers are GPL-2.0-only and are not — see
[../licensing.md](../licensing.md).
## What the spec gives you for free
Everything in [`include/usb.h`](../../include/usb.h) comes from USB 2.0
chapter 9, which every device implements regardless of speed, class or vendor.
**That is what makes enumeration possible at all:** before you know anything
about a device you can still ask it what it is, because the question is
standardised.
## The controller
`kernel/drivers/bus/xhci/`. The full bring-up path:
```
reset → DCBAA → command ring → event ring → scratchpad
→ port enumeration → Enable Slot → Address Device → control transfers
→ configuration and interface descriptors → SET_CONFIGURATION
→ interrupt and bulk endpoints
```
Rings are producer rings: the driver writes TRBs and the controller reads them,
with a Link TRB as the last entry pointing back at the start — which is how a
fixed-size buffer becomes an endless queue. 64 TRBs per ring, 16 device slots.
### Enumeration polls; data uses interrupts
**And that is not laziness.** `pci_bind_all()` runs from `kmain()` with
interrupts disabled and no scheduler, so a driver that waited for an interrupt
during probe would wait forever.
Interrupts are registered afterwards, for when a device is actually delivering
data.
## Class drivers
| | |
|---|---|
| **HID** (`usb/hid/`) | Boot protocol keyboard and mouse, driven from the IRQ handler, feeding the same input paths as PS/2. |
| **Mass storage** (`usb/storage/`) | Bulk-only transport and SCSI, presenting a block device. A superblock reads back through `bread()`. |
| **Hub** (`usb/hub/`) | Route strings, hub slot contexts, per-port reset and power. |
Class drivers are offered a device by the core once its interface descriptor has
been read, from the list in `kernel/drivers/registry.c`. Whichever claims it
publishes a `struct device` in the bus-agnostic list, **with the class the driver
chose rather than the one the descriptor claimed** — the descriptor says what the
device says it is, and the driver knows what it made of it.
### What proves the hub works
A mouse behind a hub enumerating a tier deep. Hubs are the part of USB where
route strings and slot contexts stop being theoretical, and a hub driver that
handles only directly-attached devices looks identical to a working one until
something is plugged into it.
### The USB HID grab detail
The HID path updates its `prev[]` key state **before returning** even when the
device is grabbed — otherwise the next report looks like every held key had just
been pressed again. See [graphics.md](graphics.md#input) for the grab itself.
## Testing
`bootcheck -Usb` attaches an xHCI controller with a keyboard, a mouse and a
mass-storage device.
**Nothing on the default machine has a USB port**, so without that switch the
driver is never asked to do anything and a broken one looks exactly like a
working one.
`echo u > /proc/sysrq` and `echo s > /proc/sysrq` drive the storage path.
---
## What it does not do
- **A USB disk cannot be mounted or booted from.** The block device works and
`bread()` reads a superblock off it; the root device is compiled in rather than
selectable, so nothing can be told to use it. This is the largest single gap
and it is item 1 of [../plan/roadmap.md](../plan/roadmap.md).
- **Boot protocol only** for HID. No report descriptor parsing, so anything that
is not a plain keyboard or mouse is not understood.
- **No isochronous transfers**, so no audio and no cameras.
- **No USB 3 SuperSpeed-specific features** beyond what enumeration needs — the
USB2 and USB3 descriptor paths both work, the rest of the difference is unused.
- **16 device slots**, compile-time.
- **No hot-unplug.** A device removed is not noticed; the device model has no
removal path either.
- **One controller.** A second xHCI card would bind and then contend for state
the driver keeps per-driver rather than per-instance.