docs: four directories, and a rule for when a thing leaves the roadmap

docs/ was ten files where a plan, a description, a guide and a bug report all
sat next to each other. It is now architecture/ (what SchOS is), plan/ (what it
will be), guides/ (how to work on it) and history/ (why it is like this), with
an index that states the one rule holding it together: when something is
finished it moves from plan/ to architecture/ and is rewritten as a description
of the thing rather than a record of building it.

A ticked checkbox says the work happened. It does not say what the work
*produced*, and six months later the checkbox is all that is left. The network
stack proved the point: finished, verified end to end, and for a month the only
description of it anywhere was a checkbox four levels of indentation deep.

So roadmap.md went from 534 lines to 181 -- nothing deleted, the done work
graduated. Seven subsystems that were finished and undescribed now have
descriptions: boot, kernel, syscalls, filesystem, networking, graphics, usb,
init. Every one ends with "What it does not do", which is the section that pays;
a document listing only strengths is marketing, and the reader who needs it most
is the one about to trip over a limitation nobody wrote down.

schfs.md still opened with "phases 1 through 3 1/2 are done, phase 4 is next"
long after all five had landed. It is a description now, with the phase record
and the hard-won bugs moved to history/.

Three corrections found on the way, all load-bearing:

  * projects/schcom/README.md claimed SCM_RIGHTS was done. There is no
    sendmsg/recvmsg in the kernel at all, and an entire GUI plan rested on it.
  * The root README said "No TCP/IP" months after the stack landed.
  * history/kernel.md sec.7 was marked "in progress" on a problem that process
    groups solved.

Also adds plan/1.0.md, which draws a finish line so the open list stops being
infinite, and marks schwm/ and schde/ superseded: in Wayland the compositor is
the window manager, so that split dissolves rather than being abandoned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz 2026-09-12 17:13:35 +02:00
commit 57aff40228
59 changed files with 3981 additions and 1124 deletions

267
docs/plan/1.0.md Normal file
View file

@ -0,0 +1,267 @@
# SchOS 1.0
A finish line, so that the list of open work stops being infinite.
---
## Why this document exists
[schapi.md](schapi.md) §3 says the stability freeze starts at 1.0 and covers
whatever is marked `stable` at that moment. **1.0 was never defined**, which
means there has been no finish line — and with no finish line every gap is
equally mandatory, the list only grows, and the project feels larger every week
regardless of how much gets done.
That feeling is structural, not a judgement about progress. This document fixes
the structure by drawing a line and putting most of the open work on the far
side of it.
> **Anything not named in this document is not part of 1.0.** It is not
> cancelled, not forgotten, and not up for reconsideration until 1.0 ships.
---
## What 1.0 is
> **A machine one person can sit down at and use for an afternoon without
> falling off an edge.**
Concretely — boots on real hardware, roots on SchFS, runs a shell and the real
toybox userland, is on the network, and puts a windowed terminal on the screen.
It is deliberately a **small complete system** rather than a rich incomplete
one. SchOS 1.0 is not Linux, does not run third-party applications, and has no
package manager. It is a system that works, end to end, with nothing in it that
lies about what it did.
### The acceptance test
One unattended run, asserted from the host, the way everything else here is
tested. When this passes, it is 1.0:
```
boot on real hardware from a USB stick
→ SchFS root mounts, generation counter advances
→ schinit starts the compositor
→ a terminal window appears and is asserted by pixel
→ in it: ls, cat, a file written, a network fetch
→ reboot
→ the file is still there and the checksum matches
```
Nothing in that sentence is a feature list. Every clause is something that is
currently either broken, missing, or unproven.
---
## The four gates
In order. **Do not start a gate before the one before it closes**, and closing
means a description exists in [../architecture/](../architecture/) — see
[the rule](../README.md#the-rule-that-keeps-this-from-rotting).
### Gate 0 — Fix what is already wrong
**This comes first, before any new capability.** The desktop work is the
temptation and it is the wrong thing to do next: a compositor built on a kernel
with an unexplained hang and a socket layer that reports success for things it
did not do will fail in ways indistinguishable from its own bugs. That is the
same argument that put `netdev_selftest()` before the TCP/IP stack, and it was
right then.
**Things that are actively wrong:**
- [x] ~~**The `ls -lR` hang.**~~ **Not reproducible.** Instrumented and hunted;
see below. Left here rather than deleted because the absence of a result
is the result.
- [x] ~~**`setsockopt` returns 0 without setting anything.**~~ Six options now
genuinely reach lwIP's pcb and everything else is `ENOPROTOOPT`. Round-trip
checks in `cmd/socktest`, which a permissive stub cannot pass: it has to
read back a value it was set *and* refuse an unknown option.
- [x] ~~**`accept4` silently drops its flags.**~~ It is its own table entry now
rather than an alias for `accept` — an alias would have had to read a
fourth argument that `accept` never passes, inventing a `SOCK_NONBLOCK`
out of whatever the caller last left in that register.
- [x] ~~**Timestamps on everything created after boot read 1970.**~~ Both
`know()` (SchFS) and `now_sec()` (v1) returned `uptime()` under comments
saying the kernel had no wall clock. It has had one since `time_init()`
started reading the RTC; the comments outlived the condition. Two errors,
not one — `uptime()` counts 10 ms ticks and they were used as seconds, so
the clocks also ran 100× fast.
- [ ] **`getrusage` does not distinguish `RUSAGE_CHILDREN`** from `RUSAGE_SELF`.
- [ ] **`/proc/<pid>/exe` is a regular file, not a symlink**, so `readlink()`
fails and `ps -o comm` prints `[init]`. Needs symlink support in procfs.
- [ ] **OVMF memory reporting**, if the region-list fix did not fully close it.
**Work already done and not connected:**
These are the galling ones — the replacement exists, is tested, and is unused.
- [ ] **Port the console onto the text layer and delete the cell grid.**
`text.c` and `ttf.c` both exist and work. The console still indexes
`font_data[256][16]`.
- [ ] **Load a font from disk** and point the console at it.
- [ ] **Interrupt-driven receive.** The network thread polls the NIC every timer
tick, so a frame waits up to 10 ms. Waking it from the e1000 interrupt was
deliberately not the first version; it is time.
**Structural, because everything after this compounds on it:**
- [ ] **A slab or pool allocator.** `kalloc` hands out one page at a time, and
every subsystem written since has invented its own fixed-size table to
work around it — `NPROC`, `NSOCK`, `NDEVICE`, `NSHM`, `MAXJOBS`. This is
the single change that most improves everything else in the kernel, and
it gets more expensive with every table added.
- [x] ~~**SchFS on-disk format version policy.**~~ Done. `SCHFS_VERSION` for
frame changes, plus `compat`/`ro_compat`/`incompat` feature flags in what
was `reserved0` — so existing images keep every field offset and read
unchanged. `uber_check()` now distinguishes *damaged* (step over it; that
is what the ring is for) from *too new* (stop), because falling back to an
older uberblock mounts a stale filesystem and reports success. Proven on a
forged image: the old code mounted the stale root and exited 0.
Still owed: an actual upgrade step, which is only writable once there are
two formats. See [../architecture/schfs.md](../architecture/schfs.md).
**Found while doing the above, and fixed:**
- [x] ~~Every `AF_INET` socket failed with `ENFILE` until DHCP bound.~~
`inet_new()` required an *address* before it would hand out a socket, and
reported the refusal as "file table overflow". Creating a socket is not
the same as having somewhere to send; a bind or connect with no route
fails on its own with an error that says so. `net_is_ready()` (lwIP up) is
now distinct from `net_is_up()` (DHCP bound).
- [x] ~~`socktest` asserted "AF_INET is refused, not faked".~~ True when written,
false since AF_INET landed — and **green the whole time**, because the
suite runs before DHCP binds and the `ENFILE` above made the refusal look
real. A test asserting a failure has to say *which* failure.
**The identity item:**
- [ ] **Boot from a USB disk.** The xHCI stack, bulk-only transport, SCSI and
the block device all work; a superblock reads back through `bread()`. What
is missing is that the root device is compiled in rather than selectable.
Without this the acceptance test above cannot run at all.
#### The `ls -lR` hang, and why it is closed
Hunted with the instrumentation it asked for, under a workload heavier than the
one that originally triggered it — ten iterations of `ls -lR /bin`, interleaved
with writes and `cd`, 33 execs and 13378 tree descents. Measured across it:
```
inode_peak 9 / 200 unchanged
file_peak 9 / 512 unchanged
buf_pinned_peak 1 / 256 unchanged
schfs0 locked 0 tx_owner_pid 0 tx_depth 0
```
Nothing climbed, nothing wedged, and the repaired `holdingsleep` assertion never
fired. **That eliminates this file's own stated hypothesis** — "a lock or
reference-count leak somewhere in the `*at()` paths" — and it is very likely the
bug was fixed in passing by the `*at()` dirfd rewrite, the getdents cookie
rewrite, or SchFS becoming the root, all of which landed after it was reported.
It is closed rather than chased further, and that is a deliberate application of
the two-things-in-flight rule. The instrumentation is permanent: if it ever
happens again, `/proc/fsstat` names the table, or shows `locked 1` with
`tx_owner_pid 0` and the volume lock is orphaned. That is a much better position
than the one it was reported from.
Two real defects were found on the way and fixed, neither proven to be the
cause: `holdingsleep()` never compared the owning pid, which disarmed all seven
`panic()` assertions built on it; and SchFS's `tx_mine()` compared a bare
`struct proc *`, which a recycled ptable slot makes ambiguous.
### Gate 1 — The foundations
[foundations.md](foundations.md), items 1–4. Descriptor passing, a shareable
memory object, `AF_UNIX` in the filesystem namespace.
Nothing above needs these; everything below stops without them.
### Gate 2 — Init as a role
[init.md](init.md), milestones I0–I3.
Before the desktop rather than after, because a compositor being brought up for
the first time crashes constantly, and something that restarts it and says why is
worth far more during that work than at the end of it.
### Gate 3 — The desktop
[wayland.md](wayland.md), W1–W3, and the terminal.
**W4 and W5 are not in 1.0.** Multi-window focus policy and a panel are
post-1.0. One terminal window that works is the bar.
---
## Not in 1.0
The point of this list is that it is *written down*, so it can stop occupying
attention. Nothing here is cancelled.
| | Why it waits |
|---|---|
| **The hybrid kernel** ([hybrid.md](hybrid.md)) | An architecture improvement, not a completeness requirement. It is also the work that most wants a demanding consumer first, and 1.0's compositor is exactly that consumer. |
| **Dynamic linking, Mesa, SDL3** | Everything at 1.0 is statically linked. This is the single biggest post-1.0 unlock and it is a whole project. |
| **A page cache** | `memfd` covers the desktop's needs. General writable `MAP_SHARED` of a file is a real project with no 1.0 consumer. |
| **`.sip` and the installer** | 1.0 installs by writing a USB stick. That is how most systems start and it is honest. |
| **SchFS snapshots, FUSE, Linux module, sparse files, online fsck** | The format version (Gate 0) is the prerequisite for all of them, and it is the only part on a clock. |
| **The interactive crash screen** | PSOGM renders, prints a full report and emits a QR code. That is enough. |
| **CFF/OTF fonts, kerning, text layout** | TrueType and fixed advances are enough for a terminal. |
| **Routing, `getsockname`/`getpeername`, IPv6** | One interface, one default route, works. |
| **Multiple outputs, damage tracking, subsurfaces, xkbcommon** | One screen, full repaints, one keymap. |
| **Audio** | Does not exist in any form and is not required to use a machine. |
| **Panel, notifications, settings, file manager** | A terminal is a desktop. |
| **SchAPI itself** | Deliberately. It should be designed *from* the compositor's experience, not before it. |
---
## The rule for this list
**Adding to Gate 0–3 requires deleting something else from it.**
Not a productivity trick — a forcing function. The reason this project feels
overwhelming is that ~20 fronts have been open simultaneously, and no amount of
discipline makes that workable. The fix is structural.
And one working limit, worth holding to literally:
> **Two things in flight. Never three.**
One in Gate 0 and one being written up. Everything else is closed — in the
document, and out of your head.
---
## How you will know it is going well
Not by velocity. By this:
**Every closed item leaves a description behind.** When the `ls -lR` hang is
fixed, `history/kernel.md` gains an entry saying what it actually was. When the
console moves onto the text layer, `architecture/graphics.md` loses a line from
its *What it does not do* section.
Those "does not do" sections shrinking is the real progress bar, and it is the
only one that does not lie.
---
## A note worth keeping
This document was written because the work started to feel overwhelming, and
that is worth recording rather than tidying away — because the feeling was
about the *shape* of the work and not its quality, and the two are easy to
confuse when you are inside it.
The evidence at the time: SMP working on real hardware after five bugs stacked
behind each other; a copy-on-write filesystem whose crash test cuts the power one
write later, 119 times, and demands a clean mount after each; a TCP/IP stack
asserted from both ends; an xHCI driver written from the specification; and a
documentation tree that records why things are strange so they do not get
un-fixed by accident.
None of that is the output of a project in trouble. It was a project with too
many tabs open.

105
docs/plan/README.md Normal file
View file

@ -0,0 +1,105 @@
# What happens next
Everything in this directory is **unbuilt**. When a piece of it is finished it
leaves — deleted from here, rewritten as a description in
[../architecture/](../architecture/). See [the rule](../README.md#the-rule-that-keeps-this-from-rotting).
> ## Read [1.0.md](1.0.md) first.
>
> It draws a finish line and puts most of the work below on the far side of it.
> **Anything not named there is not being worked on**, and the documents in this
> directory describe more than 1.0 contains.
---
## The three current decisions
Taken together, because they turn out to need the same foundation.
| | |
|---|---|
| **[wayland.md](wayland.md)** | The GUI stack becomes Wayland. SchWM and SchDE are deleted; Schcom survives and changes its wire format. |
| **[hybrid.md](hybrid.md)** | The kernel becomes hybrid — drivers move out into userspace processes, starting with the GPU. |
| **[init.md](init.md)** | PID 1 stops being a process and becomes a role the kernel guarantees is filled. |
And the one they all wait on:
| | |
|---|---|
| **[foundations.md](foundations.md)** | Descriptor passing, shared memory, sockets in the filesystem. The critical path. |
And the one that says which parts of the above are actually being worked on:
| | |
|---|---|
| **[1.0.md](1.0.md)** | The finish line. Four gates, and everything else explicitly deferred. |
Two longer-horizon documents sit behind these: [schapi.md](schapi.md), the API
layering and the compatibility promise, and [roadmap.md](roadmap.md), which holds
everything still open that is not one of the three above.
---
## The order
Set by [1.0.md](1.0.md), which is the authority. In short:
### 0. Fix what is already wrong — **first**
Not a document of its own; it is Gate 0 of [1.0.md](1.0.md), assembled from the
*What it does not do* sections in [../architecture/](../architecture/).
**Before any new capability**, and the reason is the one that put
`netdev_selftest()` before the TCP/IP stack: a compositor built on a kernel with
an unexplained hang and a socket layer that reports success for things it did not
do will fail in ways indistinguishable from its own bugs.
### 1. `foundations.md` — the critical path
Both the Wayland work and the hybrid work stop here. Descriptor passing and
shared memory are the substrate for handing a buffer to a compositor *and* for
handing a device to a driver process. **Doing it once for both is the entire
reason these are one plan rather than three.**
### 2. `init.md` — before the desktop, not after
A compositor being brought up for the first time crashes constantly, and
something that restarts it and says why is worth far more during that work than
at the end of it.
### 3. `wayland.md` — with the GPU driver still in the kernel
W1–W3 and a terminal. W4 and W5 are post-1.0.
### 4. `hybrid.md` — **after 1.0**
Deliberately last: **designing a userspace driver interface with no driver
consuming it produces an interface that is wrong in ways nobody can see.** The
compositor is the demanding consumer that makes the interface judgeable, so it
has to exist first.
---
## In one sentence
> Fix what is broken, build the descriptor-passing and shared-memory substrate,
> make init survivable, put a terminal on the screen — and only then use the
> compositor as the pressure that shapes the userspace driver interface.
---
## What this plan does not address
Stated here so it is not mistaken for covered:
- **No IOMMU**, so the hybrid work isolates against bugs and not against a
hostile driver.
- **No page cache**, so `foundations.md` delivers a `memfd` and not a general
answer to shared file mappings. The general answer is still owed.
- **No dynamic linking**, so no Mesa, no accelerated clients, and no third party
shipping a `.so` for anything.
- **A new compatibility surface** between PID 1 and PID 2, created by a plan
whose own [schapi.md](schapi.md) §3 spends a section warning about those.
- **The three open networking items** — `getsockname`, routing, and
interrupt-driven receive — are untouched by any of this and still open. They
are in [roadmap.md](roadmap.md).

135
docs/plan/foundations.md Normal file
View file

@ -0,0 +1,135 @@
# Foundations
The kernel work that [wayland.md](wayland.md) and [hybrid.md](hybrid.md) both
stop at. **This is the critical path**, and it is the reason those two are one
plan rather than two.
The insight is small and it decides the schedule: the substrate for passing a
buffer to a compositor and the substrate for handing a device to a driver process
are the same substrate. Descriptor passing and shared memory, built once.
---
## What is actually in the tree
Measured against the source, not against a README. Each of these was checked
before anything below was planned around it:
| | |
|---|---|
| `sendmsg`/`recvmsg`, and therefore **`SCM_RIGHTS`** | **Do not exist.** Not in `kernel/sys/table.inc`, not anywhere in `kernel/`. |
| Writable `MAP_SHARED` of a file | **Refused** — `kernel/sys/sys_mem.c:131` returns `-ENODEV`. |
| `memfd_create` | Does not exist. |
| `AF_UNIX` names | In a table in `kernel/net/unix.c`, not in the filesystem. |
| `setsockopt` | `return 0;` unconditionally — `kernel/sys/sys_net.c:436`. |
| `exec` | `ET_EXEC` only. No `PT_INTERP`, no `ET_DYN`. |
> **`projects/schcom/README.md` says `SCM_RIGHTS` is done. That line is wrong**,
> and it is the most load-bearing wrong line in the tree — an entire GUI plan was
> drawn on top of it. It is corrected as part of this work.
The `MAP_SHARED` refusal is worth defending rather than fixing carelessly. The
code returns `-ENODEV` *honestly*: the contract of a writable shared mapping is
that stores land in the file and other mappers see them, and there is no page
cache that could make that true. Quietly handing back a private copy would be the
same lie in a new place.
---
## The five items
In dependency order. The first three are what Wayland cannot start without.
### 1. `sendmsg` / `recvmsg` with `SCM_RIGHTS`
Not optional and not substitutable. `wl_shm_create_pool` passes a **file
descriptor**; there is no path in the Wayland protocol that passes an integer
instead.
The `handle_kind` field in `projects/schcom/proto.h` was good foresight and it
cannot help here — *our* protocol is the flexible one, and Wayland's is not.
Needs: `struct msghdr`/`cmsghdr` handling, ancillary data queued alongside the
byte stream in `kernel/net/unix.c`, descriptor installation into the receiver's
file table, and correct behaviour when a message carrying descriptors is never
received (they must be closed, not leaked).
**This is also item 1 of the hybrid work**, where it is how a driver process is
handed the device it is allowed to touch.
### 2. A shareable memory object
`memfd_create`, plus `mmap(MAP_SHARED)` on it genuinely working between two
processes.
Narrower than a general page cache, and sufficient. An anonymous shared object
needs no file behind it, which is exactly why `memfd` exists on Linux at all —
and it means this can be built on the shared-memory machinery already in
`kernel/sys/sys_mem.c` (`shmget`/`shmat`/`shmdt` work, and `struct vma` already
carries `shmid` and a `no_free` flag for pages that are not ours to release).
The general answer — a page cache, and writable `MAP_SHARED` of a real file —
stays owed. It is a larger project and nothing on this plan needs it.
### 3. `AF_UNIX` sockets in the filesystem namespace
Clients find a Wayland compositor by `bind()`ing `$XDG_RUNTIME_DIR/wayland-0` as
a real path, and libwayland `flock()`s the lock file beside it. A name table can
express neither.
`kernel/net/unix.c`'s own header comment already predicted this:
> *"What is lost is `ls -l /tmp/schcom` showing an `s`, and `unlink()` of a socket
> path failing with ENOENT. Both are worth fixing when the VFS next gets opened
> up; neither blocks anything."*
They block something now. This is that moment, and it is a VFS change rather than
a socket change: a socket inode type threaded through `iget`, `dirlookup` and
`stat`, plus `flock`.
### 4. `setsockopt` that does not lie
At minimum `SO_PASSCRED` and `SO_SNDBUF`. Returning 0 for an option you ignored
is worse than `ENOPROTOOPT`, because the caller adjusts its behaviour on the
strength of the lie.
Small, and grouped here because it is in the same file as item 3 and will be
found by the same testing.
### 5. Dynamic linking
`PT_INTERP` and `ET_DYN` in `kernel/fs/exec.c`.
**Deferrable, and deferred.** Static binaries are enough to reach a working
desktop with software rendering. This becomes necessary when Mesa does — EGL
means `dlopen` means a loader — and that is a separate decision made later.
---
## Verification
Each item gets a test before the thing that needs it is built, because a
substrate that is subtly wrong fails inside the consumer and looks like the
consumer's bug.
- **Item 1:** two processes, one passes an open descriptor, the other reads
through it. Plus the leak case: a sender that exits with descriptors in flight.
- **Item 2:** two processes map the same `memfd`, one writes, the other sees it.
- **Item 3:** `bind()` a socket, then `stat()` and `ls -l` the path and see an
`s`, then `unlink()` it.
- **Items 1–3 together:** a third test that does all of it at once, because that
combination is what Wayland actually performs and the three passing separately
does not prove it.
These belong in `bootcheck` as a new switch, following the `-Sock` model.
---
## What this does not deliver
- **No page cache**, so writable `MAP_SHARED` of a real file stays refused.
- **No `dlopen`** until item 5 is done, which means no Mesa and no third-party
shared libraries.
- **`SO_*` coverage is the two options named**, not the full set.
- **No credential passing** (`SCM_CREDENTIALS`) — only descriptors. Nothing on
this plan needs it, and a compositor that wants to know who connected will.

165
docs/plan/hybrid.md Normal file
View file

@ -0,0 +1,165 @@
# The kernel becomes hybrid
"Hybrid" is used to mean everything from "a monolith with a nice device model" to
"a microkernel that gave up". **Drawing the line precisely is most of the work**,
so that is what this document is.
The device model it builds on is
[../architecture/devices.md](../architecture/devices.md). It is last on the
plan, and the reason is in [README.md](README.md#the-order-and-why-it-is-not-the-order-they-are-listed-in).
---
## The line
### Stays in the kernel, permanently
- scheduling, address spaces, the trap and syscall path
- IPC and shared memory — the mechanism everything else is built out of
- the VFS and the **root** filesystem
- IRQ demultiplexing
- the console and PSOGM
**That last one is not a size compromise, it is an invariant.** The crash screen
must work when everything else in the system is dead, which means it cannot
depend on anything that could *be* dead. A crash screen served by a userspace
process is a crash screen that is absent in exactly the case it exists for.
### May leave, and should
- GPU drivers
- USB, and every USB class driver under it
- non-root filesystems
- NICs after the first
- input transformation — layouts, gestures, remapping
### The rule that separates the columns
> **Anything needed to reach a shell stays in. Anything discovered at runtime,
> replaceable, or plausibly written by someone else goes out.**
That is NT's compromise and macOS's, arrived at from the same pressure, and it is
the one that survives contact with a real machine.
---
## The mechanisms
Four, and only the fourth is large.
### 1. A control channel
`AF_UNIX` plus `SCM_RIGHTS` — item 1 of [foundations.md](foundations.md), which
the Wayland work builds first. A descriptor is how a driver process is handed the
device it is allowed to touch.
**Do not build Mach ports.** The port model is worth its cost when there is a
naming and capability story to hang on it; there is not one yet, and inventing it
before the first userspace driver exists is designing against an imaginary user.
### 2. MMIO to userspace
Map a PCI BAR into a process.
`sys_mmap` already maps the framebuffer's real physical range at its real
address — the path exists. This is that path generalised, behind a check that the
caller was granted that device.
### 3. Interrupts to userspace
**The handler stays in the kernel and does one thing: write to an `eventfd`.**
The driver process reads.
`eventfd` exists already, and that is why it is the right primitive. A signal
would be delivered to a process that cannot safely be interrupted mid-transaction.
### 4. DMA to userspace
Physically contiguous memory whose physical address the process is told.
`kernel/core/dma.c` already owns an arena reserved for exactly this, before
`kalloc` gets anything. It needs a door, a per-device budget, and a way to hand
the region back.
---
## The honest limitation
> **There is no IOMMU, so a userspace driver is isolated against its own bugs and
> not against malice.**
A process that can program a device's DMA engine can write to any physical
address, whatever its page tables say.
That is not a reason to stop — isolation against bugs is most of the value, and a
crashed driver that restarts instead of panicking the machine is the entire
point. But it must be written down, because *"userspace driver"* reads as
*"sandboxed"* and here it does not mean that yet. VT-d/AMD-Vi is its own project
and comes after.
---
## Order, and one thing not to do
### Do not build loadable kernel modules first
It is the obvious first step and it is a generation behind. It means designing a
**stable in-kernel driver ABI**, which is the most expensive promise in this
entire plan, for a mechanism Apple is actively retiring in favour of the one
above.
`../architecture/devices.md` already separates the two ideas. They are separable,
and the second one is the one to build.
### First eviction: the GPU driver
Four reasons it is the right one and not just the interesting one:
- **The boundary already exists and is already correct** — `struct display_ops`.
- **It has exactly one consumer**, the compositor, so the interface has a real
user to be judged by.
- **There is a working fallback underneath.** The VBE framebuffer stays
in-kernel, so a GPU driver that fails to start degrades to a working screen
rather than to no screen.
- It directly serves the work that will actually be happening.
### Second: USB
`xhci.c` is 1364 lines, self-contained, already spoken to through a bus
abstraction, and it is where third-party drivers will show up first. This is what
DriverKit was built for and the resemblance is not accidental.
### The rule that keeps the boot path honest
> **The kernel keeps a minimal driver for everything on the path to a shell, and
> userspace owns the good one.**
Break this and you get the circular boot failure where the compositor cannot
start because the display driver is a process that needs a compositor.
---
## Milestones
| | |
|---|---|
| **H0** | Mechanisms 2, 3 and 4, each with a test that exercises it from a userspace program against a device nothing else is using. |
| **H1** | A userspace virtio-gpu driver, publishing `display_ops` across the channel. Schcom talks to it and does not know the difference. |
| **H2** | Kill the driver process on purpose. The screen falls back to VBE, the driver restarts, the compositor reattaches. **This is the milestone that proves the whole idea**, and it is the one that cannot be skipped. |
| **H3** | A modeset API, which is the thing an out-of-kernel GPU driver makes affordable and an in-kernel one never got. |
| **H4** | USB, same shape. |
---
## What it will still not do
- **No isolation against a hostile driver**, per the IOMMU note above.
- **No stable driver ABI**, deliberately. Out-of-tree drivers remain a
recompile-with-the-tree affair until there is a reason to promise otherwise.
- **The root filesystem stays in the kernel**, so SchFS cannot be restarted and a
bug in it is still fatal. Moving it out needs an initramfs story that does not
exist.
- **The first NIC stays in the kernel**, because the network thread and lwIP are
in the kernel and moving them is a separate project with no current benefit.
- **No driver sandboxing beyond address-space separation** — no seccomp-shaped
syscall restriction, no resource limits per driver.

178
docs/plan/init.md Normal file
View file

@ -0,0 +1,178 @@
# PID 1 becomes a role
What `schinit` is today is [../architecture/init.md](../architecture/init.md).
This is what it becomes, and it is the one item on this plan that deliberately
breaks a Unix rule.
---
## The law being changed
Unix's rule is *PID 1 is special and must never die*. Every init since 1975 has
honoured it by being careful.
> **SchOS's rule instead: PID 1 is a role. The kernel guarantees the role is
> filled. The role's state lives outside whichever process currently holds it.**
That is a real change and it is worth making, because "must never die" is not a
mechanism — it is a hope, and the thing it hopes about is the one program on the
machine with no supervisor.
---
## The proposal this replaces
The original sketch was: init 1 starts init 2; if init 1 is killed, init 2 starts
init 3 and revives init 1.
The instinct is right — init needs something watching it — and one half of the
idea survives below. The specific arrangement has five problems, and four of them
are structural rather than fixable. They are recorded because each one is a trap
that any redundant-supervisor design falls into:
**It defends against the attack that already fails.** Userspace cannot kill PID 1
on this kernel. `init_ignores()` (`kernel/proc/proc.c:620`) drops `SIGKILL` to
`initproc`. The threat is already handled.
**Init 2 is the same binary as init 1.** If init 1 died of its own bug — a null
dereference parsing a manifest, an exec of a truncated binary — init 2 reaches
the same line and dies the same way, and *faster*, because it just watched the
input that caused it.
**The chain has no bound.** Init 2 makes init 3, which by symmetry watches init 2
and makes init 4. Without a cap it is a fork bomb whose author is the supervisor;
with a cap, the cap is the real design and the chain is decoration.
**Two supervisors is split brain.** Both hold the job table. Both see a job's pid
exit. Both restart it. Now the console shell exists twice, both reading
`/dev/console`, and every keystroke goes to whichever woke first. This is the
standard failure of redundant supervisors, and it is worse than no redundancy
because it corrupts the state it was added to protect.
**It only detects death.** The common way an init breaks is not dying — it is
**hanging**: deadlocked, spinning, or blocked forever on a disk that stopped
answering. `wait()` never returns for a process that is merely stuck, so init 2
watches a wedged init 1 forever and reports everything normal. The scheme is
armed against the rare failure and blind to the likely one.
---
## What gets built instead
Four parts. The first three are userspace; the fourth is the kernel change.
### Part 1 — split by role, not by copy
**`schinit` (PID 1)** becomes small enough to be obviously correct and then stops
changing: reap children, hold the job table in memory, supervise exactly one
child. No manifest parsing. No filesystem walking after boot. No allocation. Its
entire job is to be too boring to fail.
**`schinitd` (PID 2)** gets everything else — manifests, socket activation,
respawn policy, backoff, and the eventual `schinitctl`. It is where the bugs will
be, and it is restartable.
The detail that makes this work, and that the two-copy scheme cannot have:
> **Jobs are forked by PID 1, not by `schinitd`.** `schinitd` decides and asks;
> PID 1 acts. So every job on the machine is a child of PID 1 at all times, and
> `schinitd` crashing orphans nothing, reparents nothing and restarts nothing. It
> comes back, reads the state, and finds the world exactly as it left it.
This keeps the good half of the original idea — two processes, one watching the
other — with the roles made *different*, so the second is not a second copy of
the first one's bug.
### Part 2 — state outside the process
**`/proc/schinit`**: the serialised job table, written by PID 1 whenever it
changes, readable by anyone. A resurrected supervisor reads it back and knows
what is running without having started it.
procfs rather than a file on disk, for four reasons: it survives a read-only
root, it survives a full disk, it costs no I/O, and it is already the pattern
this kernel uses for exactly this kind of question.
The side benefit closes an item the architecture doc lists as missing — the job
table is currently invisible. `cat /proc/schinit` is the observation.
### Part 3 — a heartbeat, in both directions
The failure the original scheme cannot see.
PID 1 and `schinitd` hold a `socketpair`; each must be heard from within a
deadline. Silence from `schinitd` means PID 1 kills and restarts it. Silence from
PID 1 is the case part 4 exists for.
**A hang is now as detectable as a death**, which is the whole point.
### Part 4 — `proc_reincarnate()`
The kernel change, and the actual law change.
Today `exit()` on `initproc` calls `panic("init exiting")`
(`kernel/proc/proc.c:277`). Instead: tear down the address space, keep the
`struct proc`, keep **pid 1**, and re-exec `/sbin/schinit` in place with a
generation counter passed in.
No init 2, no init 3, no reparenting problem — **because pid 1 never stops being
pid 1**. Every orphan the kernel already reparented to `initproc` is still
correctly parented, `initproc` is still the pointer it always was, the children
survive, and the state is in `/proc/schinit`.
This is strictly better than resurrection-by-sibling for every case, and it is
the smaller change.
#### The escalation ladder
What makes it safe to have at all:
| Generation | What happens |
|---|---|
| 1 | Normal boot. |
| 2–3 | Re-exec in place. Log it, keep the jobs, carry on. |
| 4 | Re-exec, but ignore `/etc/schinit` and start only a console shell. **A manifest that kills init must not kill it forever.** |
| 5 | Stop. `panic()` with stop code `INIT_RESURRECTION_FAILED`. |
The last row is the honest one. A machine that cannot hold PID 1 up five times is
not a machine that should keep trying, and PSOGM already renders a stop code, a
call chain and a QR code of the whole report — which is a far better end than a
silent loop. The crash screen is the correct place for this to arrive.
---
## Milestones
| | |
|---|---|
| **I0** | Split the program. PID 1 supervises one child; `schinitd` does everything else. Jobs still forked by PID 1. |
| **I1** | `/proc/schinit`, written and read back. Kill `schinitd` by hand and watch it return without disturbing a single job. |
| **I2** | The heartbeat both ways, with a `sysrq` key that wedges `schinitd` on purpose so the timeout path is exercised rather than assumed. |
| **I3** | `proc_reincarnate()` and the ladder, with a `sysrq` key that kills PID 1 on purpose. |
---
## What it costs
**PID 1 and PID 2 must agree on a protocol**, which is a compatibility surface
where there was none — created by a plan whose own [schapi.md](schapi.md) §3
warns about exactly that. Keep it tiny, version it in its first message, and keep
it entirely internal. It is not SchAPI and nothing else may speak it.
**`proc_reincarnate()` is a genuinely dangerous function.** It runs in the
context of a process that is dying, and it must not take a lock that process may
already hold. It belongs in `proc.c` next to `exit()`, written with the care of
the panic path, and tested by a `sysrq` key rather than by reasoning.
---
## What it will still not do
- **It does not make init correct**, only survivable. A `schinitd` that starts
the wrong jobs starts them again after every restart.
- **It does not protect the jobs from each other.** A job that fills the disk or
exhausts the process table still takes the machine down, and the supervisor
surviving does not help.
- **A hang in PID 1 itself is only caught by `schinitd` noticing**, and if both
are wedged simultaneously nothing catches it. There is no hardware watchdog.
- **Five generations is a guess**, not a measured number.

179
docs/plan/roadmap.md Normal file
View file

@ -0,0 +1,179 @@
# Roadmap — everything else still open
The three current decisions have their own documents; see
[README.md](README.md). This file holds what is left: work that is scoped,
agreed and unstarted, with nothing in it that is finished.
**Finished work is not here.** It moved to
[../architecture/](../architecture/) and was rewritten as a description of what
it produced — see [the rule](../README.md#the-rule-that-keeps-this-from-rotting).
What used to be phases 1 through 4 of this file, several hundred lines of ticked
boxes, is now `usb`, `networking.md`, `graphics.md` and `devices.md` over there.
---
## 1. Storage and the boot path
**Make a USB disk mountable and bootable.** The xHCI stack, the bulk-only
transport, the SCSI layer and the block device all work — a superblock reads back
through `bread()`. What is missing is that **the root device is compiled in
rather than selectable**, so nothing can be told to mount it.
This is also the first piece of the installer, and it is the single change that
would most enlarge what SchOS can boot from.
## 2. Graphics
- **A `/dev/dri`-style device with mode setting**, so resolution stops being
whatever the loader asked for. Worth doing *with*
[hybrid.md](hybrid.md)'s GPU eviction rather than before it — moving the driver
out is what makes a modeset API affordable to design properly.
- **Double buffering and vblank.** `display_ops` specifies both and no driver
implements either. Schcom needs both to not tear.
- **Port the console onto the text layer and delete the cell grid.** The layer
exists, the rasteriser exists, and the console still indexes `font_data[256][16]`.
- **Load a font from disk** and point the console at it.
- **CFF/Type 2 charstrings**, so OTF fonts work. A separate `font_ops` provider
and a bytecode interpreter, *not* an extension of the `glyf` path.
- **Text layout** — advance widths, kerning — as a library in `lib/`, so the
terminal and the file manager are not each solving it.
The console keeps its fixed grid until the first item above; proportional text
belongs to the GUI.
## 3. Networking
Three things the stack owed on the day it landed, all still owed:
- **`accept()` does not report the peer's address**, and there is no
`getsockname`/`getpeername`. Nothing has needed them yet; a server that wants
to log who connected will.
- **No routing table.** With two interfaces up, the first is the default route
and the second is reachable only by an address on its own subnet. That is a
rule rather than an accident, and it is not routing.
- **Receive is polled.** The network thread looks at the NIC every timer tick
when the link is idle, so an arriving frame waits up to 10 ms. Waking that
thread from the e1000 interrupt is the real fix, and was deliberately not the
first version: a receive path woken by an interrupt is much harder to debug
than one that is asked.
`setsockopt` returning 0 for everything is in
[foundations.md](foundations.md) instead, because the Wayland work needs it.
## 4. The interactive crash screen
PSOGM renders and stops. The idea is *a small OS inside the crash*, and the
framing decides the design:
> At panic time the kernel is the only thing running, which is exactly why it can
> offer what a live system cannot: walk the stack, read another process's memory,
> dump page tables, inspect the process table — with no scheduler and no locking
> to respect, because nothing else can race you.
What makes it a *system* rather than a print statement is that it keeps the base
machine's drivers. Concretely, the ones that work with interrupts off: the
framebuffer (`gfx_draw_pixel` is a plain memory write), the PS/2 keyboard
(pollable via ports 0x60/0x64), and the serial port. **Those three are a complete
console.**
Everything else — IDE, USB, the timer, anything that sleeps or takes a
sleeplock — is off limits, because the lock it needs may be precisely the one
held by the CPU that just died.
So the shape is a **polled event loop over preallocated state**: no `kalloc`, no
`sleep`/`wakeup`, no `acquire` on anything a live path might hold, one static
command buffer, and that driver set. **That restriction is not a limitation to
work around later; it is the invariant that makes the mode trustworthy when
everything else is not.** Anything wanting more should be answered by rebooting
into a recovery environment.
A useful precedent to read but not copy: DDB, the BSD in-kernel debugger, is
exactly this, and its hard-won rule is the same one — *a debugger that can
deadlock is worse than no debugger.*
## 5. The installer and `.sip`
Scoped in [`projects/schinst/README.md`](../../projects/schinst/README.md),
which holds the detail. In short:
- [ ] `.sip` format specification, **written down before any code**
- [ ] `tools/mksip` — capture a directory tree into a package
- [ ] The apply path
- [ ] An installation environment that boots from the ISO, partitions, applies
the `.sip`, and installs the loader
`.sip` is modelled on WIM, and the thing worth copying from WIM is that it is
**file-based, not block-based**: one file holds a whole filesystem, applies to a
partition of any size, and several images can share their common files.
**Most of this already exists.** `tools/mkschfs.c` and `tools/mkfs.c` both do
content dedup with an `(st_dev, st_ino)` fast path, and SchFS's B-tree is content
addressed. `mksip` should share that code rather than reimplement it — that is
the reason the dedup was factored the way it was.
Open questions, all wanting a decision before code:
- **Compression.** Something with a small decompressor is worth more than
something with a good ratio, because the decompressor runs in the installation
environment.
- **Multiple images per `.sip`** — WIM's real trick, install/recovery/boot
sharing one blob store. Worth the complexity now, or not?
- **Can the kernel mount a `.sip` read-only directly?** It is close to a
filesystem already, and if it can, the installation environment's root can
simply *be* the `.sip`.
The GUI for this is a Wayland client; see [wayland.md](wayland.md).
## 6. SchFS beyond SchOS
In the order they should happen.
**~~An on-disk format version~~, and an upgrade path.** The version policy is
done — see [../architecture/schfs.md](../architecture/schfs.md). What remains is
the **upgrade step**, and it cannot be written until there are two formats to
convert between; the first bump is when that becomes real work rather than
speculation.
The format is now pinned, which is what the other two items were waiting on.
**A Linux implementation, FUSE first.** Three reasons in order of weight: it can
be tested with `fsx` and `xfstests`, which are far harsher than anything written
alongside the code; it carries no licence entanglement, since libfuse is LGPL and
used across a process boundary; and it can reuse the read/write logic already in
`tools/mkschfs` and `tools/schfs_fsck` rather than being written twice and having
the two disagree.
A kernel module is a different project, comes only after FUSE passes `xfstests`,
and carries the licence problem [../licensing.md](../licensing.md) describes.
**The missing features**, listed honestly before adding any: no journal replay
across an unclean shutdown beyond what copy-on-write gives for free, no
snapshots, no sparse-file support, and no online `fsck`. **Snapshots are the one
where the existing design already pays for most of the cost** — the B-tree is
copy-on-write, which is most of the machinery; nothing takes or names a snapshot.
## 7. The kernel's own gaps
Not phases, and each one improves everything above it:
- **A slab or pool allocator.** `kalloc` hands out one page at a time, and every
subsystem written since has invented its own fixed-size table to work around
that. This is the single change that would most improve the rest of the kernel.
- **A page cache**, which is what a general writable `MAP_SHARED` needs.
[foundations.md](foundations.md) delivers a `memfd` instead, which is narrower
and sufficient for the GUI.
- **Symlinks in procfs**, so `/proc/<pid>/exe` can be one and `ps -o comm` stops
printing `[init]`. Belongs with the rest of the VFS work rather than on its own.
- **The `ls -lR` hang.** Heavy directory listing across many commands in one long
shell session eventually makes a *later*, unrelated directory hang; the same
command succeeds on a fresh boot. Not inode or file table exhaustion — both
fail cleanly and neither fired. Next step: instrument `itable`/`ftable`
occupancy across a long session and watch what climbs.
---
## Kept out of scope on purpose
- Anything that needs pushing or publishing. This work stays local.
- TrueType hinting.
- XWayland.

239
docs/plan/schapi.md Normal file
View file

@ -0,0 +1,239 @@
# SchAPI, and SchOS as a system
Status: **direction agreed, not yet implemented.** Nothing in this document
changes what is being built this week; it changes what everything is being
built *towards*. See §7 for what actually happens now.
---
## 1. The decision
SchOS stops being "a Unix-like kernel with a userland bolted on" and becomes
a **system**: a kernel, a native API, and the services and applications that
sit on it, designed together.
The model is NeXTSTEP/macOS, not Linux. Specifically:
- A kernel that provides mechanism, with **more than one personality** on top.
- **POSIX/Unix is a personality, not the identity.** It stays, completely and
seriously, because it is how you get software. It is not the thing programs
are expected to be written against.
- **SchAPI** is the native interface: higher-level, object-shaped, designed for
the GUI and services rather than retrofitted onto 1970s process semantics.
Two systems are being borrowed from, and they are *different* systems. It is
worth being precise about which, because they disagree:
| Borrowed | From | Why |
|---|---|---|
| Layered architecture, personalities, native API over a Unix substrate | macOS / NeXTSTEP | It is the only mainstream design that got a modern API onto a Unix base without abandoning either |
| Permanent backward compatibility | **Windows** | macOS is emphatically *not* this — Apple dropped 32-bit apps, Carbon, kexts and OpenGL, each time on a deadline |
That mix is coherent, but it is a mix. Taking the architecture from Apple and
the compatibility policy from Microsoft is a deliberate choice, and the rest of
this document is mostly about making that choice survivable — because
compatibility-forever is the single most expensive promise an OS can make, and
Windows pays for it every day.
---
## 2. The one mechanism that makes the promise affordable
**The stable interface is a library, not the syscall table.**
This is the whole trick, and it is what lets macOS keep decade-old binaries
running while freely renumbering its syscalls. On macOS, nothing may issue a
syscall directly; you link `libSystem`. Apple therefore owes compatibility only
at the library boundary, and the kernel stays editable forever.
Linux made the opposite choice — the syscall ABI *is* the contract, "we do not
break userspace" — and consequently every syscall mistake ever made is still
there, with a `2` or a `64` suffixed to the name of its replacement.
So, for SchOS:
```
┌──────────────────────────────────────────────┐
│ applications, SchWM, SchDE │
├──────────────────────────────────────────────┤
│ SchAPI │ POSIX / Linux personality│ ← THE STABLE BOUNDARY
├──────────────────┴───────────────────────────┤
│ libschos — the only thing that may syscall│
├──────────────────────────────────────────────┤
│ syscall table — PRIVATE, unstable, renumberable
├──────────────────────────────────────────────┤
│ kernel │
└──────────────────────────────────────────────┘
```
Rules that follow, and they are not negotiable if the promise is to hold:
1. **A program that issues `syscall` directly is unsupported.** It may break at
any release. Only `libschos` may.
2. `libschos` exports versioned symbols. An entry point's behaviour never
changes; a changed behaviour is a new version of the symbol.
3. The kernel may renumber, merge, split or delete syscalls at will, provided
`libschos` still presents the same surface.
This *reconciles* with the existing decision to adopt Linux syscall numbering
(`roadmap.md`, and the reason musl/toybox/mksh work at all). Linux
numbering does not go away — it becomes the **Linux personality's** entry path,
which is exactly what it already is in practice. It is a compatibility surface
that happens to be entered via `syscall`, and it is frozen because *Linux*
froze it, not because we did.
---
## 3. Stability tiers
"Nothing can ever be removed" is only affordable if you are precise about what
"anything" covers. Windows' real problem was never the documented API — it was
that undocumented behaviour became load-bearing, and then it was too late.
Every public interface carries exactly one tier:
| Tier | Promise | May change |
|---|---|---|
| `stable` | Forever, from the release that introduced it | Never. Additions only. |
| `provisional` | Until the next release | Freely |
| `internal` | None | Freely, without notice |
**Everything in SchOS today is `provisional`.** The freeze starts at 1.0 and
covers only what is marked `stable` at that moment. This is not a way of
dodging the commitment — it is the only way to make it, because a promise made
now would permanently enshrine, among other things, a `fcntl(F_SETFL)` that
was a documented no-op until last week and a `struct file` whose `O_NONBLOCK`
handling had never been exercised.
Nothing becomes `stable` by being old. It becomes `stable` by someone deciding
it is right and writing that down.
---
## 4. `compat/` — what it actually means
The instinct is right: old interfaces should not litter the current tree. But
moving *source* into a folder does not reduce the *runtime* cost, and the
runtime cost is the one that hurts.
The rule that keeps compatibility cheap:
> **A compatibility entry point must be implementable as a translation into the
> current API. If it cannot be, that is a defect in the current API — fix the
> current API before freezing it.**
So `compat/` holds shims, not parallel implementations. A shim:
- lives in `compat/<release-it-was-current-in>/`,
- is written in terms of today's API and nothing else,
- is never called by new code,
- carries its own tests, which are the only thing that proves the promise is
still being kept.
When a shim *cannot* be expressed as a translation, that is the alarm. It means
the new API lost information the old one carried, and it is far cheaper to
notice that before 1.0 than to maintain two real implementations forever. That
is precisely the trap Win32 fell into.
---
## 5. Taking from the BSDs
`src-trunk/` is a NetBSD checkout, and a FreeBSD one is being added alongside
it. Both are reference material and parts bins; neither is built and neither
may ever end up in an image.
**Two trees is not redundancy.** They are worth having together for three
reasons that a single tree cannot give:
- Where an RFC is ambiguous — and TCP's are, constantly — two independent
implementations agreeing is far stronger evidence than one implementation
asserting.
- Where one is licence-encumbered, the other may not be. NetBSD's
`tcp_input.c` carries a WIDE Project advertising clause that Berkeley's
rescission does not cover; FreeBSD's equivalent may well be 2-clause. See
[../licensing.md](../licensing.md) §3.
- They have opposite biases, and knowing which you are reading matters:
| | FreeBSD | NetBSD |
|---|---|---|
| Stack quality | The better stack in absolute terms — faster, more complete, more actively developed | Solid, conservative, less featureful |
| Extractability | Welded to FreeBSD: `epoch(9)`, `uma`, `netisr`, VIMAGE/vnet, `if_t`, a rich locking vocabulary | Portability *is* the project's identity; assumes far less of its host kernel |
| Use it as | The reference you check yourself against | The code you actually lift |
That table is the same lesson §1 of the roadmap already learned the hard way
with FreeBSD's `sys/dev/usb`: 100k lines welded to newbus, where porting the
USB stack means porting the device framework first. Networking is the same
shape. Read FreeBSD, lift NetBSD, write SchOS.
### Licensing — read this before copying anything
NetBSD is not uniformly licensed: 2-clause, UC 3-clause and a minority of
4-clause-with-advertising files all appear, and only the first two are
comfortable. The per-template rules, the measured counts, and the one-line
check to run before copying a file are in [../licensing.md](../licensing.md)
§3. Do not copy from `src-trunk/` without reading it.
### What is actually worth taking
Ranked by value, not by size:
1. **`sys/net`, `sys/netinet` — the TCP/IP stack.** The largest single win
available. Phase 2 of the roadmap wants ARP/IPv4/ICMP/UDP/TCP, and this is a
mature, clean, readable, 2-clause-heavy implementation of exactly that.
2. **`sys/kern/subr_pool.c` — the pool/slab allocator.** SchOS has `kalloc`
and a page at a time. Every subsystem written since has invented its own
fixed-size table to work around that.
3. **`autoconf` — the driver framework** (`config(1)`, `cfdriver`/`cfattach`,
`device_t`). This is the grown-up version of what `kernel/drivers/registry.c`
and `kernel/dev/irq.c` are hand-rolling. Adopt the *model*; do not try to
port the code wholesale.
4. `sys/kern/subr_*` generally — rbtree, workqueue, condvar, kmem.
Low value: `lib/libc` (musl already fills that role), `sys/dev/usb` (we have a
working xHCI stack written from the spec).
**Take designs before code.** Every file copied is a file to be understood,
relicensed-in-notice, and maintained forever under the §3 promise.
---
## 6. What SchAPI is, concretely
Not yet designed. What is decided about it:
- It is the native interface; POSIX is available beneath it but not the idiom.
- It is reached through `libschos`, never by direct syscall.
- It is designed *with* Schcom in mind, because a compositor and its clients
are the first real users and their needs (IPC, shared buffers, event streams,
object lifetimes) are what will shape it.
- The IPC primitive comes first. Everything in a NeXT-shaped system rests on
it, and unix domain sockets — already built and working — are the substrate,
not the interface.
What is explicitly *not* decided: whether it is object-oriented in the Objective-C
sense, what language it is expressed in, and whether it takes Mach's port model
or something smaller. Those need their own document, written when Schcom has
told us what it actually needs.
---
## 7. What changes now
Almost nothing, on purpose. The ordering does not change:
- **The native Windows build must boot first.** The clang-built kernel
currently reaches `sinit` and dies on the first trap. A direction is worth
nothing while the tree does not run.
- Then the roadmap continues as written: network, GPU, fonts, Schcom.
What this document changes *today* is only this:
1. `src-trunk/` is reference material, excluded from every build and image.
2. New interfaces are marked `provisional` from now on, so that §3 has
something to work with at 1.0.
3. When the network stack is written, NetBSD's is the starting point rather
than a blank file — with the licence check in §5 applied per file.
4. `libschos` is created when there is a second personality to justify it.
Until then, the *rule* stands even though the library does not exist yet:
no new first-party program issues raw syscalls.

167
docs/plan/wayland.md Normal file
View file

@ -0,0 +1,167 @@
# The GUI becomes Wayland
Schcom survives and changes its wire format. SchWM and SchDE are deleted. What
follows is why that is not a retreat, and what it costs.
Blocked on [foundations.md](foundations.md), items 1–4.
---
## What is deleted
`projects/schwm/` and `projects/schde/`. Both are README-only; no code is lost.
### SchWM is not abandoned — Wayland dissolves it
In X11 the window manager is a separate privileged client because X11 puts
compositing in the server and policy nowhere. **Wayland's central decision is
that the compositor *is* the window manager**: there is no protocol by which a
second process places another's windows, and every attempt to add one has been a
private extension.
So the argument in `projects/schwm/README.md` —
> *"Compositing is mechanical and must never stall: it runs every frame and a
> mistake there is a frozen screen. Window management is policy, full of special
> cases and personal taste, and it changes constantly."*
— is a real and correct observation **about X11**, which Wayland answers a
different way. It stays worth honouring, but as a **module boundary inside
Schcom**, not a process boundary: policy in its own file behind its own
interface. Do not keep it in its own process, because the protocol to talk to it
does not exist.
### SchDE is re-sited rather than deleted
A panel, a launcher and a wallpaper are ordinary Wayland clients using
`wlr-layer-shell` — MIT, an XML file, no dependency. They need no directory of
their own until one of them is written.
### `projects/schinst/` stays
Its GUI was the part tied to SchWM, and that part becomes a Wayland client.
Everything else in it — the `.sip` format, `mksip`, the apply path, the
installation environment — is untouched by any of this, and remains the
best-specified unstarted work in the tree.
---
## What survives
**Schcom stays, and keeps its name.** It already owns the display, allocates and
owns client buffers, implements surface/attach/commit, routes input from
`/dev/input/eventN`, composites multiple clients bottom-to-top, and cleans up
after clients that die.
Its own README says the protocol is deliberately Wayland-shaped — surfaces rather
than windows, buffers attached and made visible only on commit, the compositor
telling clients when to draw. That was the right call and this is where it is
cashed in: **what changes is the wire format and the buffer handle, not the
architecture.**
---
## What has to exist first
Items 1–4 of [foundations.md](foundations.md), and the first three are absolute —
libwayland does not start without them.
The sharpest is descriptor passing. `wl_shm_create_pool` passes a **file
descriptor**, and there is no path in the protocol that passes an integer
instead. `struct schcom_surface`'s `handle_kind` field was written precisely so
that switching would be a new value rather than a protocol break, and it cannot
help: *our* protocol is the flexible one.
---
## Port libwayland. Do not port Weston.
**libwayland is the small piece**: a wire protocol, an object map, and an event
loop, needing `libffi` and — for the build-time scanner, on the host only —
`libexpat`. A real port, but a bounded one.
**Weston is the trap.** It arrives welded to `libinput`, `udev`, `libdrm`,
`logind`-shaped seat management and a GL renderer, and porting it means porting
Linux's device management first and finding the compositor somewhere underneath.
This is precisely the lesson already recorded twice in this tree — about
FreeBSD's `sys/dev/usb` (100k lines welded to newbus) and about FreeBSD's
`sys/netinet` (not a library; reaches into mbufs, epoch, netisr, VNET and uma).
> The tree has learned this twice. It should not pay to learn it a third time.
So **Schcom links `libwayland-server` and remains ours.** Input comes from
`/dev/input/eventN`, which already works, rather than from libinput. Output goes
through `display_ops`, which already exists.
### Software rendering first, and probably for a long time
`pixman` for composition, `wl_shm` for buffers. Mesa means EGL means `dlopen`
means a dynamic loader, and that is a large kernel project that does not have to
be on this path. Static binaries are enough to reach a working desktop; keep it
that way until something concrete demands otherwise.
### One sharp edge
`wl_keyboard` hands the client **a keymap as an XKB text blob over a file
descriptor**. A compositor cannot avoid xkbcommon by having simpler opinions
about keyboards — the protocol requires the blob.
The cheap route is one precompiled keymap string shipped in the binary, with
xkbcommon and `xkeyboard-config` deferred until more than one layout is wanted.
---
## Licensing
Checked before anything is fetched — the standing rule is that the licence goes
in `NOTICE` **before** the file is committed, not after.
| | |
|---|---|
| wayland, wayland-protocols | MIT |
| `wlr-layer-shell` | MIT |
| pixman | MIT |
| libffi | MIT |
| libxkbcommon | MIT |
| xkeyboard-config | MIT |
All Apache-2.0 compatible, all needing only a `NOTICE` entry. **Nothing here
carries the GPL problem that governs drivers**, which is a pleasant change and
worth noticing: the Wayland ecosystem is permissively licensed almost throughout,
and that is a genuine argument for it beyond any technical one.
---
## Milestones
| | |
|---|---|
| **W0** | [foundations.md](foundations.md) items 1–4, with the combined test that passes a descriptor *and* shares a `memfd` *and* binds a socket at a real path. |
| **W1** | libwayland builds and installs into the sysroot. `wayland-scanner` runs on the host. |
| **W2** | Schcom serves `wl_compositor`, `wl_shm`, `wl_surface`, `xdg_shell`. A client draws one window. |
| **W3** | `wl_seat`: keyboard and pointer, from the evdev path that exists. |
| **W4** | Two clients, focus, stacking, move and resize. **This is where SchWM's policy lives, as a file.** |
| **W5** | `wlr-layer-shell`, and a panel. This is SchDE's first pixel. |
Every milestone from W2 on is asserted **from the host, by pixel**, the way
`schcom-test` already does. The compositor's own report that it composited
something proves nothing.
---
## What it will still not do
- **No hardware acceleration.** Software rendering only, until dynamic linking
and Mesa, neither of which is on this plan.
- **No XWayland**, so no X11 clients, ever, unless someone ports it.
- **No mode setting.** Resolution stays whatever the loader asked for — see
[hybrid.md](hybrid.md), where the GPU driver's eviction is also where a modeset
API would arrive.
- **No damage tracking beyond "repaint everything"**, no multiple outputs, no
subsurfaces. Schcom's README already lists these; they are additions rather
than rewrites, which is the reason for committing to the surface/commit model
now.
- **One keyboard layout**, until xkbcommon is ported.
- **No clipboard, no drag and drop, no screen capture protocol**, none of which
W0–W5 touches.