SchOS/docs/architecture/devices.md
Nexoniarz 071499a378 ldls: a second driver, and everything it found
e1000 -- Linux's, unmodified -- probes, opens, gets an address by DHCP and
carries ping traffic in a SchOS process. The kernel was driving that NIC and
gave it up, which makes it the first real handover as well as the second
driver.

None of what it needed was a missing feature. Every one was a call that
succeeded, returned zero and did nothing:

  _find_next_bit          "bit 0 is set"          -- for_each_cpu never ended
  netif_carrier_off       carrier already up      -- TCTL.EN never set
  __x86_indirect_thunk_r8 the jump discarded      -- 255 rx buffers, none made
  dma_map_page_attrs      bus address 0           -- NIC aimed at physical 0

so the rule from memcpy is now general: a stub returning zero is unsafe
wherever zero is a meaningful answer.

The loader grew ko_fixup_percpu(). The stack guard turned out to be one
per-CPU variable among several; `current` cannot be defused because the driver
dereferences it. Taking the %gs off instead leaves the same opcode reading the
same width from storage the host supplies -- a per-CPU variable in a
single-threaded host is just a variable. Eight bytes become seven and a nop,
and the displacement is not always last, so there is now just enough of an x86
decoder in ko.c to know how long the instruction is.

ldls/kpi/thunk.c is fifteen lines of assembly and the highest-value file here:
retpolines are control flow, and a stub cannot stand in for a jump.

The kernel side:

  - ioperm.c, the TSS I/O permission bitmap. A driver's outl is an instruction,
    not a call, and e1000's reset path picks it deliberately. IOPL=3 would hand
    over the PIC and the disk with it, so it is per-port. Writing it found that
    the I/O map base had been written at offset 64 rather than 102, leaving the
    CPU checking ports 0-831 against the TSS's own RSP0.
  - claim masks the device's IRQ line: 450,000 a second, otherwise.
  - net_detach(), the missing half of net_attach_late(). An interface left in
    the stack keeps its route and sends every frame to a driver that refuses
    it, which looks exactly like a userspace driver that does not work.
  - e1000_release() gives up the interface, not just the hardware.

8139too still passes 4/4; ldls-check still passes 27; four CPUs, no panic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
2026-09-14 15:20:07 +02:00

257 lines
13 KiB
Markdown

# The device model
How hardware gets found, claimed and used, and what the rest of the kernel sees
of it.
Writing a driver is [../guides/adding-a-driver.md](../guides/adding-a-driver.md).
Moving drivers out of the kernel entirely is [../plan/hybrid.md](../plan/hybrid.md).
---
## One list of everything attached
There is one list of everything attached, whatever bus found it:
`include/device.h` and `kernel/dev/device.c`, readable at `/proc/devices`.
```
name bus class driver device
kbd0 platform input ps2kbd PS/2 keyboard
mouse0 platform input ps2mouse PS/2 mouse
uart0 platform other uart8250 16550 serial port
ide0 platform storage ide ATA disk controller
vga0 pci display vga display controller
```
Before this there were two registries in `kernel/drivers/registry.c` — an array
of PCI drivers and an array of USB drivers — with nothing that knew about both.
That is sufficient for *binding* and useless for the three things wanted next.
**Enumeration.** "What is in this machine, and what claimed it" had no answer
covering a USB disk and a PCI display at once. A device manager is exactly that
question.
**Classes.** A compositor should ask for *the display* and not care whether that
is the VBE framebuffer or virtio-gpu. Without a class every consumer hardcodes a
driver, and adding the second driver means editing the consumers — which is how
a framebuffer address ended up hardcoded in three places here once already.
**A boundary for drivers that are not compiled in.** They have to attach
somewhere that is not an array fixed at link time.
Two vocabularies are kept deliberately separate: a device's **bus** is
provenance (`platform`, `pci`, `usb`), its **class** is capability
(`display`, `input`, `storage`, `net`, `bus`). A USB disk and a SATA disk are
both `storage` despite sharing no numbering at all. Where a driver claims a
device, the driver's opinion wins over the descriptor's: `struct usb_driver` has
a `devclass` field separate from the USB interface class, because the descriptor
says what the device *claims* to be and the driver knows what it *made* of it.
It is a flat list with a bus tag, **not** a device tree. Nothing yet needs to
walk a hierarchy, and a tree nobody traverses is just a harder list. Hot-unplug
would need more; the `in_use` flag is the hook.
Platform devices are registered in one place
(`platform_register_devices()` in `registry.c`) rather than inside each driver's
init, for the same reason the driver arrays are written out by hand: a list you
can read beats a list assembled by side effects.
---
## Drivers that are not in the kernel
`/dev/pci` (major 9) hands a PCI device to a process. It is deliberately about
*PCI devices* and not about any driver model, and its first users are SchOS's
own drivers, evicted one at a time — which is the ordering that keeps it a
general mechanism rather than a hole shaped like something foreign.
What a driver needs from a kernel, and all it needs:
| | |
|---|---|
| find its device, and read what the bus says | `PCIIOC_INFO`, `PCIIOC_CFG` |
| be given it, exclusively | `PCIIOC_CLAIM` |
| reach its registers | `mmap`, where the offset is a BAR's physical address |
| memory it can hand to the device | `PCIIOC_DMA` |
| be told when the device wants attention | `PCIIOC_IRQ`, then `read()` |
| use its ports, if it has an I/O BAR | nothing — `CLAIM` grants them |
Four of those are worth expanding on, because each encodes a decision:
**`mmap` is checked against the caller's claims.** An offset that is not inside
a BAR of a device this process holds is refused. That is the whole difference
between this and `/dev/mem` with extra steps.
The check is against the *pages* a BAR lies in rather than the BAR itself,
because a page is the only granularity an MMU has. A BAR smaller than a page
therefore hands the process the whole page it sits in — forced by the hardware
rather than chosen. Comparing against the BAR's own size instead meant a
sub-page BAR could not be mapped at all, which nothing noticed until an
rtl8139 with 256 bytes of registers turned up; see
[ldls.md](ldls.md), "The bug a foreign driver found in its
first minute".
**DMA is a call, not a mapping.** The process is told the *physical* address,
which is precisely what it cannot work out for itself, and the memory comes from
an arena reserved at boot before the general allocator gets anything.
**Interrupts arrive by `read()`**, the way Linux's uio does it and for the same
reason: a descriptor composes with `poll`, and a driver that stops reading falls
behind rather than losing events. The kernel masks the line when the interrupt
arrives and leaves it masked until the driver `write()`s back. **That write is
not optional.** A PCI interrupt is level-triggered — it stays asserted until the
cause is cleared *at the device*, which is a different register on every chip and
therefore something only the driver can do. Signal EOI on a line that is still
asserted and it re-fires immediately, for ever. That is not hypothetical: it
happened here at 72,000 interrupts a second, and what it looked like from
userspace was a clock running seven times slow. See §13 of
[../history/kernel.md](../history/kernel.md).
**Ports come with the claim, and only that device's.** There is no ioctl for
this because there is no decision left to make: a process holding a device with
an I/O BAR gets exactly the ports in that BAR, through the TSS I/O permission
bitmap, and loses them when it releases or exits. Granting it is not optional —
a driver's `outl` is an instruction compiled into it, not a call anything can
intercept — and granting it *coarsely* is not acceptable, because `IOPL=3` is
one flag that hands over the interrupt controller and the disk along with the
NIC. See [../../include/ioperm.h](../../include/ioperm.h).
### Handing a device over
`struct pci_driver` has a `release` hook. A driver that does not define one keeps
its device — always safe, and for anything on the path to a shell also correct.
A driver that does define one has to give back everything, not just the device:
the first eviction found this the hard way, because releasing the GPU while
keeping its 4 MB framebuffer meant the incoming driver was told there was no DMA
memory left.
`cmd/vgpud` is the virtio-gpu driver as an ordinary process. It does the whole
virtio 1.0 bring-up from outside — reset, feature negotiation, a virtqueue in DMA
memory, `SET_SCANOUT`, transfer and flush — and while it owns the GPU the kernel
console falls back to the VBE framebuffer.
**No syscall on that path knows it is talking about a GPU.** The kernel hands
over a PCI device; the process decides what it is.
Two things a release has to do beyond quiescing the hardware, both found by
handing over a NIC the kernel was actively using:
- **Mask the line.** The kernel's handler is gone and the incoming driver is
about to enable the device, so for a while nothing will acknowledge the
cause. Left unmasked that is an interrupt storm — 450,000 a second, measured.
- **Give up the class device too.** A NIC's `release` calls `net_detach()`, the
missing half of `net_attach_late()`. An interface left in the stack keeps its
address and the route to its subnet, so the stack goes on choosing it and
every frame reaches a `transmit` that now correctly refuses. Nothing reports
an error. See [ldls.md](ldls.md).
It also **changes the display mode**, on a control socket, with `schmode` as the
client — and that is worth a sentence here because it is the thing the device
model makes possible rather than the thing it does. A modeset is five commands
to the device. In the kernel those five commands are also a reason to be
nervous, because a wrong one takes the machine; in a process the worst case is
that `schinit` starts another one. See
[../plan/hybrid.md](../plan/hybrid.md), "A modeset, from a process".
### Taking it back
A driver that dies is the case this whole arrangement exists for, so release is a
symmetric operation rather than a courtesy. Three things happen that are not
obvious from "the process exited":
**Bus mastering is cleared before the DMA is freed.** A device whose driver has
just died is still writing into memory the kernel is about to hand to somebody
else. The kernel cannot quiesce a device it does not understand — that is a
different register on every chip — but it can clear `PCI_CMD_BUS_MASTER`, which
stops the writes whatever the device is, and it is the only halt available to a
kernel that knows nothing but the header. So `PCIIOC_CLAIM` sets the bit and
release clears it, in that order relative to freeing the memory. `pcitest` prints
the bit in a `bm` column, with a separate mark for "could not read it", because a
failed read rendered as "off" is worse than no column at all.
**DMA memory a driver mapped is not `freevm()`'s to release.** `freevm()` frees
every mapped page below `PHYSTOP` and the DMA arena is the top 16 MB of exactly
that range, so a DMA block left for it went onto the page allocator's free list
while `pciuser_release_all()` handed the same pages back to the arena — two
allocators, one set of pages. `PCIIOC_DMA` records a VMA with `no_free` set, the
same way shared memory does, and `proc_release_shared()` unmaps the range before
the page tables are torn down. The same rule covers every future case: **anything
mapped into a process that the process does not own the pages of needs a VMA, or
exit will free it twice.**
**A device that is the only display will not be released.** `vgpu_release()` asks
`vga_fb_restore()` whether there is a framebuffer to put the console back on, and
declines if the answer is no. The rule that the kernel keeps a minimal driver for
everything on the path to a shell is only worth writing down if something
enforces it.
Coming back is a job, not a kernel feature. `etc/schinit/vgpud.job` carries
`KeepAlive = yes`, so killing the driver is the entire test: `schinitd` starts
another one, it claims the device, and the pid of whichever process drew the
frame is legible in the frame itself. See
[../plan/hybrid.md](../plan/hybrid.md), "The driver came back".
### What this is not
**It is not a security boundary.** There is no IOMMU, so a process holding a DMA
buffer can point the device at any physical address. This isolates a driver
against its own *bugs* and not against its own *intent* — a crashed driver that
restarts instead of taking the machine down is the entire benefit, and it is
worth having on those terms. [../plan/hybrid.md](../plan/hybrid.md) says so at
length.
**One device per process.** A driver drives one thing; if you need two, run two.
---
## Linux drivers
**Decided 2026-08-08.** Linux drivers are not shipped with SchOS and never
will be. The user may install one on their own machine, through a device
manager or equivalent, and SchOS will load it.
The licensing reasoning matters and is worth stating exactly, because it is
easy to get wrong in both directions:
- **Distribution is what triggers the GPL.** Shipping a GPL-2.0-only driver in
an Apache-2.0 image is what cannot be done — see [../licensing.md](../licensing.md)
§2. A user obtaining a driver themselves and loading it on their own machine
is their act, not ours, and is not a distribution by us.
- **Implementing a loader is not copying.** Writing support for a module format
and a driver ABI from published documentation produces our own work. It is a
clean-room implementation, and it stays one only if it is written from
documentation rather than transcribed from Linux source. Reading Linux to
understand *hardware* is fine; translating its code is not, whatever the
identifiers say.
So the rule for this subsystem is the same as everywhere else in the tree: the
loader is ours and Apache-2.0, what it loads is the user's business, and nothing
GPL ever enters an image we build. `RAMDISK_EXCLUDE` and `NOTPORTS` are the
existing mechanisms for keeping that true mechanically rather than by intention.
---
---
## What it does not do
- **It is a flat list, not a device tree.** There is no parent/child nesting,
only a bus tag. Nothing yet needs to walk a hierarchy, and a tree nobody
traverses is just a harder list.
- **No hot-plug.** Devices are registered at boot and never removed; the
`in_use` flag is the hook a hot-unplug path would need, and nothing sets it
for that purpose.
- **`NDEVICE` is 32**, compile-time, like every other table in the kernel.
- **In-kernel drivers are fixed at link time.** There is no module loader, and
nothing can be added to the kernel after the build. A driver *outside* the
kernel is an ordinary program and can be started whenever — but it reaches its
device through `/dev/pci` and publishes nothing to the class vtables, so the
console cannot be handed to one yet.
- **The kernel does not rebind its own driver.** `PCIIOC_RELEASE`, or the process
exiting, leaves the interrupt line masked and the in-kernel driver unbound —
the console stays wherever the release put it. A *supervisor* can start a
replacement driver, and `schinitd` does; the kernel itself watches for
nothing.
- **Only two class vtables are defined**: `display_ops` and `netdev_ops`.
Storage and input devices appear in the list with `ops = 0` and are reached
through their own subsystems instead.