57aff40228
docs/ was ten files where a plan, a description, a guide and a bug report all
sat next to each other. It is now architecture/ (what SchOS is), plan/ (what it
will be), guides/ (how to work on it) and history/ (why it is like this), with
an index that states the one rule holding it together: when something is
finished it moves from plan/ to architecture/ and is rewritten as a description
of the thing rather than a record of building it.
A ticked checkbox says the work happened. It does not say what the work
*produced*, and six months later the checkbox is all that is left. The network
stack proved the point: finished, verified end to end, and for a month the only
description of it anywhere was a checkbox four levels of indentation deep.
So roadmap.md went from 534 lines to 181 -- nothing deleted, the done work
graduated. Seven subsystems that were finished and undescribed now have
descriptions: boot, kernel, syscalls, filesystem, networking, graphics, usb,
init. Every one ends with "What it does not do", which is the section that pays;
a document listing only strengths is marketing, and the reader who needs it most
is the one about to trip over a limitation nobody wrote down.
schfs.md still opened with "phases 1 through 3 1/2 are done, phase 4 is next"
long after all five had landed. It is a description now, with the phase record
and the hard-won bugs moved to history/.
Three corrections found on the way, all load-bearing:
* projects/schcom/README.md claimed SCM_RIGHTS was done. There is no
sendmsg/recvmsg in the kernel at all, and an entire GUI plan rested on it.
* The root README said "No TCP/IP" months after the stack landed.
* history/kernel.md sec.7 was marked "in progress" on a problem that process
groups solved.
Also adds plan/1.0.md, which draws a finish line so the open list stops being
infinite, and marks schwm/ and schde/ superseded: in Wayland the compositor is
the window manager, so that split dissolves rather than being abandoned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
165 lines
6.3 KiB
Markdown
165 lines
6.3 KiB
Markdown
# The kernel becomes hybrid
|
|
|
|
"Hybrid" is used to mean everything from "a monolith with a nice device model" to
|
|
"a microkernel that gave up". **Drawing the line precisely is most of the work**,
|
|
so that is what this document is.
|
|
|
|
The device model it builds on is
|
|
[../architecture/devices.md](../architecture/devices.md). It is last on the
|
|
plan, and the reason is in [README.md](README.md#the-order-and-why-it-is-not-the-order-they-are-listed-in).
|
|
|
|
---
|
|
|
|
## The line
|
|
|
|
### Stays in the kernel, permanently
|
|
|
|
- scheduling, address spaces, the trap and syscall path
|
|
- IPC and shared memory — the mechanism everything else is built out of
|
|
- the VFS and the **root** filesystem
|
|
- IRQ demultiplexing
|
|
- the console and PSOGM
|
|
|
|
**That last one is not a size compromise, it is an invariant.** The crash screen
|
|
must work when everything else in the system is dead, which means it cannot
|
|
depend on anything that could *be* dead. A crash screen served by a userspace
|
|
process is a crash screen that is absent in exactly the case it exists for.
|
|
|
|
### May leave, and should
|
|
|
|
- GPU drivers
|
|
- USB, and every USB class driver under it
|
|
- non-root filesystems
|
|
- NICs after the first
|
|
- input transformation — layouts, gestures, remapping
|
|
|
|
### The rule that separates the columns
|
|
|
|
> **Anything needed to reach a shell stays in. Anything discovered at runtime,
|
|
> replaceable, or plausibly written by someone else goes out.**
|
|
|
|
That is NT's compromise and macOS's, arrived at from the same pressure, and it is
|
|
the one that survives contact with a real machine.
|
|
|
|
---
|
|
|
|
## The mechanisms
|
|
|
|
Four, and only the fourth is large.
|
|
|
|
### 1. A control channel
|
|
|
|
`AF_UNIX` plus `SCM_RIGHTS` — item 1 of [foundations.md](foundations.md), which
|
|
the Wayland work builds first. A descriptor is how a driver process is handed the
|
|
device it is allowed to touch.
|
|
|
|
**Do not build Mach ports.** The port model is worth its cost when there is a
|
|
naming and capability story to hang on it; there is not one yet, and inventing it
|
|
before the first userspace driver exists is designing against an imaginary user.
|
|
|
|
### 2. MMIO to userspace
|
|
|
|
Map a PCI BAR into a process.
|
|
|
|
`sys_mmap` already maps the framebuffer's real physical range at its real
|
|
address — the path exists. This is that path generalised, behind a check that the
|
|
caller was granted that device.
|
|
|
|
### 3. Interrupts to userspace
|
|
|
|
**The handler stays in the kernel and does one thing: write to an `eventfd`.**
|
|
The driver process reads.
|
|
|
|
`eventfd` exists already, and that is why it is the right primitive. A signal
|
|
would be delivered to a process that cannot safely be interrupted mid-transaction.
|
|
|
|
### 4. DMA to userspace
|
|
|
|
Physically contiguous memory whose physical address the process is told.
|
|
|
|
`kernel/core/dma.c` already owns an arena reserved for exactly this, before
|
|
`kalloc` gets anything. It needs a door, a per-device budget, and a way to hand
|
|
the region back.
|
|
|
|
---
|
|
|
|
## The honest limitation
|
|
|
|
> **There is no IOMMU, so a userspace driver is isolated against its own bugs and
|
|
> not against malice.**
|
|
|
|
A process that can program a device's DMA engine can write to any physical
|
|
address, whatever its page tables say.
|
|
|
|
That is not a reason to stop — isolation against bugs is most of the value, and a
|
|
crashed driver that restarts instead of panicking the machine is the entire
|
|
point. But it must be written down, because *"userspace driver"* reads as
|
|
*"sandboxed"* and here it does not mean that yet. VT-d/AMD-Vi is its own project
|
|
and comes after.
|
|
|
|
---
|
|
|
|
## Order, and one thing not to do
|
|
|
|
### Do not build loadable kernel modules first
|
|
|
|
It is the obvious first step and it is a generation behind. It means designing a
|
|
**stable in-kernel driver ABI**, which is the most expensive promise in this
|
|
entire plan, for a mechanism Apple is actively retiring in favour of the one
|
|
above.
|
|
|
|
`../architecture/devices.md` already separates the two ideas. They are separable,
|
|
and the second one is the one to build.
|
|
|
|
### First eviction: the GPU driver
|
|
|
|
Four reasons it is the right one and not just the interesting one:
|
|
|
|
- **The boundary already exists and is already correct** — `struct display_ops`.
|
|
- **It has exactly one consumer**, the compositor, so the interface has a real
|
|
user to be judged by.
|
|
- **There is a working fallback underneath.** The VBE framebuffer stays
|
|
in-kernel, so a GPU driver that fails to start degrades to a working screen
|
|
rather than to no screen.
|
|
- It directly serves the work that will actually be happening.
|
|
|
|
### Second: USB
|
|
|
|
`xhci.c` is 1364 lines, self-contained, already spoken to through a bus
|
|
abstraction, and it is where third-party drivers will show up first. This is what
|
|
DriverKit was built for and the resemblance is not accidental.
|
|
|
|
### The rule that keeps the boot path honest
|
|
|
|
> **The kernel keeps a minimal driver for everything on the path to a shell, and
|
|
> userspace owns the good one.**
|
|
|
|
Break this and you get the circular boot failure where the compositor cannot
|
|
start because the display driver is a process that needs a compositor.
|
|
|
|
---
|
|
|
|
## Milestones
|
|
|
|
| | |
|
|
|---|---|
|
|
| **H0** | Mechanisms 2, 3 and 4, each with a test that exercises it from a userspace program against a device nothing else is using. |
|
|
| **H1** | A userspace virtio-gpu driver, publishing `display_ops` across the channel. Schcom talks to it and does not know the difference. |
|
|
| **H2** | Kill the driver process on purpose. The screen falls back to VBE, the driver restarts, the compositor reattaches. **This is the milestone that proves the whole idea**, and it is the one that cannot be skipped. |
|
|
| **H3** | A modeset API, which is the thing an out-of-kernel GPU driver makes affordable and an in-kernel one never got. |
|
|
| **H4** | USB, same shape. |
|
|
|
|
---
|
|
|
|
## What it will still not do
|
|
|
|
- **No isolation against a hostile driver**, per the IOMMU note above.
|
|
- **No stable driver ABI**, deliberately. Out-of-tree drivers remain a
|
|
recompile-with-the-tree affair until there is a reason to promise otherwise.
|
|
- **The root filesystem stays in the kernel**, so SchFS cannot be restarted and a
|
|
bug in it is still fatal. Moving it out needs an initramfs story that does not
|
|
exist.
|
|
- **The first NIC stays in the kernel**, because the network thread and lwIP are
|
|
in the kernel and moving them is a separate project with no current benefit.
|
|
- **No driver sandboxing beyond address-space separation** — no seccomp-shaped
|
|
syscall restriction, no resource limits per driver.
|