72074a960e
docs/plan/hybrid.md mechanism 3. A process asks for a device's interrupts with
PCIIOC_IRQ and then reads them, which is how Linux's uio does it and for the
same reason: a descriptor composes with poll, and a driver that stops reading
falls behind rather than losing events. read() returns a count rather than a
flag, so a driver that was busy sees it missed three and acts once.
Proven against an e1000e by asserting a cause through its ICS register and
reading the interrupt back out in userspace:
interrupts routed here (line 10)
interrupt delivered, count = 1
ICR = 0x80000004
0x80000004 is the cause the test set plus INT_ASSERTED -- the device's own
register, read by userspace as its acknowledgement.
**The masking is the design, not a detail.** A PCI line is level-triggered and
stays asserted until the cause is cleared at the device, which is a different
register on every chip and so is something only the driver can do. The kernel
masks the line on arrival and userspace's write() unmasks it. Signalling EOI on
a still-asserted line re-enters the handler immediately and for ever; this
kernel did exactly that by accident at 72,000 interrupts a second, and what it
looked like from outside was a clock running seven times slow (§13).
Which decides what happens when a driver dies mid-interrupt: the line is left
**masked**. The instinct is to unmask -- do not leave the device dead -- and it
is wrong, because unmasking an asserted line with no handler is a crashed
driver taking the machine down with it. Masked is safe and not permanent: the
next PCIIOC_IRQ arms it. Both halves verified by crashing a driver on purpose
and then starting another one.
A line already carrying kernel handlers is refused outright. Masking is
per-line, so handing out a shared one would silence whichever kernel driver is
also there, with nothing logged. MSI is the real answer and pci_msi_enable() is
already in the tree waiting for a caller.
Also fixed, found by running the crash twice: DMA buffers were never returned
when a process exited. The arena is 4 MB and contiguous, so a driver that
restarts a few times exhausted it -- "out of DMA memory, 3584 free" -- which is
a poor property for an interface whose whole promise is that a driver can crash
and come back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
238 lines
9.3 KiB
Markdown
238 lines
9.3 KiB
Markdown
# The kernel becomes hybrid
|
|
|
|
"Hybrid" is used to mean everything from "a monolith with a nice device model" to
|
|
"a microkernel that gave up". **Drawing the line precisely is most of the work**,
|
|
so that is what this document is.
|
|
|
|
The device model it builds on is
|
|
[../architecture/devices.md](../architecture/devices.md). It is last on the
|
|
plan, and the reason is in [README.md](README.md#the-order-and-why-it-is-not-the-order-they-are-listed-in).
|
|
|
|
---
|
|
|
|
## The line
|
|
|
|
### Stays in the kernel, permanently
|
|
|
|
- scheduling, address spaces, the trap and syscall path
|
|
- IPC and shared memory — the mechanism everything else is built out of
|
|
- the VFS and the **root** filesystem
|
|
- IRQ demultiplexing
|
|
- the console and PSOGM
|
|
|
|
**That last one is not a size compromise, it is an invariant.** The crash screen
|
|
must work when everything else in the system is dead, which means it cannot
|
|
depend on anything that could *be* dead. A crash screen served by a userspace
|
|
process is a crash screen that is absent in exactly the case it exists for.
|
|
|
|
### May leave, and should
|
|
|
|
- GPU drivers
|
|
- USB, and every USB class driver under it
|
|
- non-root filesystems
|
|
- NICs after the first
|
|
- input transformation — layouts, gestures, remapping
|
|
|
|
### The rule that separates the columns
|
|
|
|
> **Anything needed to reach a shell stays in. Anything discovered at runtime,
|
|
> replaceable, or plausibly written by someone else goes out.**
|
|
|
|
That is NT's compromise and macOS's, arrived at from the same pressure, and it is
|
|
the one that survives contact with a real machine.
|
|
|
|
---
|
|
|
|
## Built so far
|
|
|
|
**Mechanisms 2 and 4 exist.** `/dev/pci` (include/pciuser.h, kernel/dev/pciuser.c)
|
|
enumerates the bus, hands a device to a process exclusively, maps a BAR into it
|
|
and allocates memory a device can be pointed at. `pcitest` proves it end to end:
|
|
|
|
```
|
|
# bdf vendor:device class driver
|
|
6 00:04.0 8086:100e network e1000
|
|
7 00:07.0 8086:10d3 network -
|
|
|
|
device 7: 8086:10d3 (network)
|
|
claimed
|
|
BAR0 at 0x0000000081060000, 128 KB
|
|
regs[0..3] = 0x00140261 0x00000000 0x00080283 0x00000000
|
|
MMIO reaches the device
|
|
DMA 8 KB at virtual 0x40001000, physical 0x3f8eb000
|
|
```
|
|
|
|
Those register values are the e1000e's CTRL and STATUS, with full-duplex and
|
|
link-up set -- which is how you tell a working mapping from one that reads
|
|
0xFFFFFFFF off an unmapped address. Claiming a device the kernel drives is
|
|
refused, and everything a process held is released when it exits, because a
|
|
driver that crashed must not keep its hardware away from its replacement.
|
|
|
|
**Mechanism 3 exists too.** A process asks for a device's interrupts and then
|
|
reads them, uio-style:
|
|
|
|
```
|
|
interrupts routed here (line 10)
|
|
interrupt delivered, count = 1
|
|
ICR = 0x80000004
|
|
```
|
|
|
|
`0x80000004` is the e1000e's cause register with the bit the test asserted and
|
|
INT_ASSERTED set, read back by userspace as its acknowledgement.
|
|
|
|
The line is **masked by the kernel on arrival and unmasked by userspace's
|
|
write()**, and that is the entire design rather than a detail. A PCI interrupt
|
|
line is level-triggered: it stays asserted until the cause is cleared *at the
|
|
device*, which is a different register on every chip and therefore something
|
|
only the driver can do. Signalling EOI on a line that is still asserted
|
|
re-enters the handler immediately and for ever -- 72,000 times a second, when
|
|
this kernel last did it by accident (history/kernel.md §13).
|
|
|
|
So a driver that dies mid-interrupt leaves the line **masked, not unmasked**.
|
|
The instinct is the other way round -- do not leave the device dead -- and it
|
|
is wrong: unmasking an asserted line with no handler attached is a crashed
|
|
driver taking the machine down with it. Verified by killing one on purpose:
|
|
|
|
```
|
|
interrupt asserted; exiting WITHOUT acking (simulated crash)
|
|
timer: 3431 ticks taken, 3436 expected
|
|
```
|
|
|
|
Five ticks lost in seven seconds, which is the ordinary background. The next
|
|
driver to claim the device re-arms the line, so masked is safe without being
|
|
permanent -- also verified.
|
|
|
|
**The control channel (mechanism 1) is what remains.**
|
|
|
|
### One thing the interface is deliberately not
|
|
|
|
It is about **PCI devices**, not about any driver model, and its first users
|
|
will be SchOS's own drivers. That ordering is the design: an interface whose
|
|
first consumers are native drivers is a general mechanism that something
|
|
foreign can use later, rather than a hole shaped like something foreign. The
|
|
difference matters technically -- a Linux-shaped interface would be wrong for
|
|
the evictions this plan actually calls for -- and it happens to be the only
|
|
version that survives a licensing conversation.
|
|
|
|
---
|
|
|
|
## The mechanisms
|
|
|
|
Four, and only the fourth is large.
|
|
|
|
### 1. A control channel
|
|
|
|
`AF_UNIX` plus `SCM_RIGHTS` — item 1 of [foundations.md](foundations.md), which
|
|
the Wayland work builds first. A descriptor is how a driver process is handed the
|
|
device it is allowed to touch.
|
|
|
|
**Do not build Mach ports.** The port model is worth its cost when there is a
|
|
naming and capability story to hang on it; there is not one yet, and inventing it
|
|
before the first userspace driver exists is designing against an imaginary user.
|
|
|
|
### 2. MMIO to userspace
|
|
|
|
Map a PCI BAR into a process.
|
|
|
|
`sys_mmap` already maps the framebuffer's real physical range at its real
|
|
address — the path exists. This is that path generalised, behind a check that the
|
|
caller was granted that device.
|
|
|
|
### 3. Interrupts to userspace
|
|
|
|
**The handler stays in the kernel and does one thing: write to an `eventfd`.**
|
|
The driver process reads.
|
|
|
|
`eventfd` exists already, and that is why it is the right primitive. A signal
|
|
would be delivered to a process that cannot safely be interrupted mid-transaction.
|
|
|
|
### 4. DMA to userspace
|
|
|
|
Physically contiguous memory whose physical address the process is told.
|
|
|
|
`kernel/core/dma.c` already owns an arena reserved for exactly this, before
|
|
`kalloc` gets anything. It needs a door, a per-device budget, and a way to hand
|
|
the region back.
|
|
|
|
---
|
|
|
|
## The honest limitation
|
|
|
|
> **There is no IOMMU, so a userspace driver is isolated against its own bugs and
|
|
> not against malice.**
|
|
|
|
A process that can program a device's DMA engine can write to any physical
|
|
address, whatever its page tables say.
|
|
|
|
That is not a reason to stop — isolation against bugs is most of the value, and a
|
|
crashed driver that restarts instead of panicking the machine is the entire
|
|
point. But it must be written down, because *"userspace driver"* reads as
|
|
*"sandboxed"* and here it does not mean that yet. VT-d/AMD-Vi is its own project
|
|
and comes after.
|
|
|
|
---
|
|
|
|
## Order, and one thing not to do
|
|
|
|
### Do not build loadable kernel modules first
|
|
|
|
It is the obvious first step and it is a generation behind. It means designing a
|
|
**stable in-kernel driver ABI**, which is the most expensive promise in this
|
|
entire plan, for a mechanism Apple is actively retiring in favour of the one
|
|
above.
|
|
|
|
`../architecture/devices.md` already separates the two ideas. They are separable,
|
|
and the second one is the one to build.
|
|
|
|
### First eviction: the GPU driver
|
|
|
|
Four reasons it is the right one and not just the interesting one:
|
|
|
|
- **The boundary already exists and is already correct** — `struct display_ops`.
|
|
- **It has exactly one consumer**, the compositor, so the interface has a real
|
|
user to be judged by.
|
|
- **There is a working fallback underneath.** The VBE framebuffer stays
|
|
in-kernel, so a GPU driver that fails to start degrades to a working screen
|
|
rather than to no screen.
|
|
- It directly serves the work that will actually be happening.
|
|
|
|
### Second: USB
|
|
|
|
`xhci.c` is 1364 lines, self-contained, already spoken to through a bus
|
|
abstraction, and it is where third-party drivers will show up first. This is what
|
|
DriverKit was built for and the resemblance is not accidental.
|
|
|
|
### The rule that keeps the boot path honest
|
|
|
|
> **The kernel keeps a minimal driver for everything on the path to a shell, and
|
|
> userspace owns the good one.**
|
|
|
|
Break this and you get the circular boot failure where the compositor cannot
|
|
start because the display driver is a process that needs a compositor.
|
|
|
|
---
|
|
|
|
## Milestones
|
|
|
|
| | |
|
|
|---|---|
|
|
| **H0** | Mechanisms 2, 3 and 4, each with a test that exercises it from a userspace program against a device nothing else is using. |
|
|
| **H1** | A userspace virtio-gpu driver, publishing `display_ops` across the channel. Schcom talks to it and does not know the difference. |
|
|
| **H2** | Kill the driver process on purpose. The screen falls back to VBE, the driver restarts, the compositor reattaches. **This is the milestone that proves the whole idea**, and it is the one that cannot be skipped. |
|
|
| **H3** | A modeset API, which is the thing an out-of-kernel GPU driver makes affordable and an in-kernel one never got. |
|
|
| **H4** | USB, same shape. |
|
|
|
|
---
|
|
|
|
## What it will still not do
|
|
|
|
- **No isolation against a hostile driver**, per the IOMMU note above.
|
|
- **No stable driver ABI**, deliberately. Out-of-tree drivers remain a
|
|
recompile-with-the-tree affair until there is a reason to promise otherwise.
|
|
- **The root filesystem stays in the kernel**, so SchFS cannot be restarted and a
|
|
bug in it is still fatal. Moving it out needs an initramfs story that does not
|
|
exist.
|
|
- **The first NIC stays in the kernel**, because the network thread and lwIP are
|
|
in the kernel and moving them is a separate project with no current benefit.
|
|
- **No driver sandboxing beyond address-space separation** — no seccomp-shaped
|
|
syscall restriction, no resource limits per driver.
|