docs/plan/hybrid.md mechanism 3. A process asks for a device's interrupts with
PCIIOC_IRQ and then reads them, which is how Linux's uio does it and for the
same reason: a descriptor composes with poll, and a driver that stops reading
falls behind rather than losing events. read() returns a count rather than a
flag, so a driver that was busy sees it missed three and acts once.
Proven against an e1000e by asserting a cause through its ICS register and
reading the interrupt back out in userspace:
interrupts routed here (line 10)
interrupt delivered, count = 1
ICR = 0x80000004
0x80000004 is the cause the test set plus INT_ASSERTED -- the device's own
register, read by userspace as its acknowledgement.
**The masking is the design, not a detail.** A PCI line is level-triggered and
stays asserted until the cause is cleared at the device, which is a different
register on every chip and so is something only the driver can do. The kernel
masks the line on arrival and userspace's write() unmasks it. Signalling EOI on
a still-asserted line re-enters the handler immediately and for ever; this
kernel did exactly that by accident at 72,000 interrupts a second, and what it
looked like from outside was a clock running seven times slow (§13).
Which decides what happens when a driver dies mid-interrupt: the line is left
**masked**. The instinct is to unmask -- do not leave the device dead -- and it
is wrong, because unmasking an asserted line with no handler is a crashed
driver taking the machine down with it. Masked is safe and not permanent: the
next PCIIOC_IRQ arms it. Both halves verified by crashing a driver on purpose
and then starting another one.
A line already carrying kernel handlers is refused outright. Masking is
per-line, so handing out a shared one would silence whichever kernel driver is
also there, with nothing logged. MSI is the real answer and pci_msi_enable() is
already in the tree waiting for a caller.
Also fixed, found by running the crash twice: DMA buffers were never returned
when a process exited. The arena is 4 MB and contiguous, so a driver that
restarts a few times exhausted it -- "out of DMA memory, 3584 free" -- which is
a poor property for an interface whose whole promise is that a driver can crash
and come back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
9.3 KiB
The kernel becomes hybrid
"Hybrid" is used to mean everything from "a monolith with a nice device model" to "a microkernel that gave up". Drawing the line precisely is most of the work, so that is what this document is.
The device model it builds on is ../architecture/devices.md. It is last on the plan, and the reason is in README.md.
The line
Stays in the kernel, permanently
- scheduling, address spaces, the trap and syscall path
- IPC and shared memory — the mechanism everything else is built out of
- the VFS and the root filesystem
- IRQ demultiplexing
- the console and PSOGM
That last one is not a size compromise, it is an invariant. The crash screen must work when everything else in the system is dead, which means it cannot depend on anything that could be dead. A crash screen served by a userspace process is a crash screen that is absent in exactly the case it exists for.
May leave, and should
- GPU drivers
- USB, and every USB class driver under it
- non-root filesystems
- NICs after the first
- input transformation — layouts, gestures, remapping
The rule that separates the columns
Anything needed to reach a shell stays in. Anything discovered at runtime, replaceable, or plausibly written by someone else goes out.
That is NT's compromise and macOS's, arrived at from the same pressure, and it is the one that survives contact with a real machine.
Built so far
Mechanisms 2 and 4 exist. /dev/pci (include/pciuser.h, kernel/dev/pciuser.c)
enumerates the bus, hands a device to a process exclusively, maps a BAR into it
and allocates memory a device can be pointed at. pcitest proves it end to end:
# bdf vendor:device class driver
6 00:04.0 8086:100e network e1000
7 00:07.0 8086:10d3 network -
device 7: 8086:10d3 (network)
claimed
BAR0 at 0x0000000081060000, 128 KB
regs[0..3] = 0x00140261 0x00000000 0x00080283 0x00000000
MMIO reaches the device
DMA 8 KB at virtual 0x40001000, physical 0x3f8eb000
Those register values are the e1000e's CTRL and STATUS, with full-duplex and link-up set -- which is how you tell a working mapping from one that reads 0xFFFFFFFF off an unmapped address. Claiming a device the kernel drives is refused, and everything a process held is released when it exits, because a driver that crashed must not keep its hardware away from its replacement.
Mechanism 3 exists too. A process asks for a device's interrupts and then reads them, uio-style:
interrupts routed here (line 10)
interrupt delivered, count = 1
ICR = 0x80000004
0x80000004 is the e1000e's cause register with the bit the test asserted and
INT_ASSERTED set, read back by userspace as its acknowledgement.
The line is masked by the kernel on arrival and unmasked by userspace's write(), and that is the entire design rather than a detail. A PCI interrupt line is level-triggered: it stays asserted until the cause is cleared at the device, which is a different register on every chip and therefore something only the driver can do. Signalling EOI on a line that is still asserted re-enters the handler immediately and for ever -- 72,000 times a second, when this kernel last did it by accident (history/kernel.md §13).
So a driver that dies mid-interrupt leaves the line masked, not unmasked. The instinct is the other way round -- do not leave the device dead -- and it is wrong: unmasking an asserted line with no handler attached is a crashed driver taking the machine down with it. Verified by killing one on purpose:
interrupt asserted; exiting WITHOUT acking (simulated crash)
timer: 3431 ticks taken, 3436 expected
Five ticks lost in seven seconds, which is the ordinary background. The next driver to claim the device re-arms the line, so masked is safe without being permanent -- also verified.
The control channel (mechanism 1) is what remains.
One thing the interface is deliberately not
It is about PCI devices, not about any driver model, and its first users will be SchOS's own drivers. That ordering is the design: an interface whose first consumers are native drivers is a general mechanism that something foreign can use later, rather than a hole shaped like something foreign. The difference matters technically -- a Linux-shaped interface would be wrong for the evictions this plan actually calls for -- and it happens to be the only version that survives a licensing conversation.
The mechanisms
Four, and only the fourth is large.
1. A control channel
AF_UNIX plus SCM_RIGHTS — item 1 of foundations.md, which
the Wayland work builds first. A descriptor is how a driver process is handed the
device it is allowed to touch.
Do not build Mach ports. The port model is worth its cost when there is a naming and capability story to hang on it; there is not one yet, and inventing it before the first userspace driver exists is designing against an imaginary user.
2. MMIO to userspace
Map a PCI BAR into a process.
sys_mmap already maps the framebuffer's real physical range at its real
address — the path exists. This is that path generalised, behind a check that the
caller was granted that device.
3. Interrupts to userspace
The handler stays in the kernel and does one thing: write to an eventfd.
The driver process reads.
eventfd exists already, and that is why it is the right primitive. A signal
would be delivered to a process that cannot safely be interrupted mid-transaction.
4. DMA to userspace
Physically contiguous memory whose physical address the process is told.
kernel/core/dma.c already owns an arena reserved for exactly this, before
kalloc gets anything. It needs a door, a per-device budget, and a way to hand
the region back.
The honest limitation
There is no IOMMU, so a userspace driver is isolated against its own bugs and not against malice.
A process that can program a device's DMA engine can write to any physical address, whatever its page tables say.
That is not a reason to stop — isolation against bugs is most of the value, and a crashed driver that restarts instead of panicking the machine is the entire point. But it must be written down, because "userspace driver" reads as "sandboxed" and here it does not mean that yet. VT-d/AMD-Vi is its own project and comes after.
Order, and one thing not to do
Do not build loadable kernel modules first
It is the obvious first step and it is a generation behind. It means designing a stable in-kernel driver ABI, which is the most expensive promise in this entire plan, for a mechanism Apple is actively retiring in favour of the one above.
../architecture/devices.md already separates the two ideas. They are separable,
and the second one is the one to build.
First eviction: the GPU driver
Four reasons it is the right one and not just the interesting one:
- The boundary already exists and is already correct —
struct display_ops. - It has exactly one consumer, the compositor, so the interface has a real user to be judged by.
- There is a working fallback underneath. The VBE framebuffer stays in-kernel, so a GPU driver that fails to start degrades to a working screen rather than to no screen.
- It directly serves the work that will actually be happening.
Second: USB
xhci.c is 1364 lines, self-contained, already spoken to through a bus
abstraction, and it is where third-party drivers will show up first. This is what
DriverKit was built for and the resemblance is not accidental.
The rule that keeps the boot path honest
The kernel keeps a minimal driver for everything on the path to a shell, and userspace owns the good one.
Break this and you get the circular boot failure where the compositor cannot start because the display driver is a process that needs a compositor.
Milestones
| H0 | Mechanisms 2, 3 and 4, each with a test that exercises it from a userspace program against a device nothing else is using. |
| H1 | A userspace virtio-gpu driver, publishing display_ops across the channel. Schcom talks to it and does not know the difference. |
| H2 | Kill the driver process on purpose. The screen falls back to VBE, the driver restarts, the compositor reattaches. This is the milestone that proves the whole idea, and it is the one that cannot be skipped. |
| H3 | A modeset API, which is the thing an out-of-kernel GPU driver makes affordable and an in-kernel one never got. |
| H4 | USB, same shape. |
What it will still not do
- No isolation against a hostile driver, per the IOMMU note above.
- No stable driver ABI, deliberately. Out-of-tree drivers remain a recompile-with-the-tree affair until there is a reason to promise otherwise.
- The root filesystem stays in the kernel, so SchFS cannot be restarted and a bug in it is still fatal. Moving it out needs an initramfs story that does not exist.
- The first NIC stays in the kernel, because the network thread and lwIP are in the kernel and moving them is a separate project with no current benefit.
- No driver sandboxing beyond address-space separation — no seccomp-shaped syscall restriction, no resource limits per driver.