ldls: L4 was free after all, and the plan closes
L4 was listed as "free -- H2 built all of this" and never actually tested,
which is a claim rather than a result. Tested now, and it holds:
ldlsd running ping 2/2; holds index 7 and DMA slots 3 and 4
killed interface DOWN, device released, bus mastering
cleared, DMA reclaimed -- back to vgpud's 3 blocks
ping fails honestly: sendto: -5
a second ldlsd ping 3/3, and the arena hands back the same addresses
pciuser_release_all() and netuser_release_all() sit beside each other in
exit() for the same reason, and that they apply unchanged to a foreign driver
is the whole point of having built them against our own.
With that, every milestone on ldls.md is done, so it leaves docs/plan/ for
docs/architecture/ -- the rule docs/plan/README.md states and the one that
keeps the two directories meaning different things. The new document describes
what is, not what was planned: the five rules, the four pieces, what it costs
(polling, no stack protector, locks that are no-ops), that it can be killed,
and a "what it does not do" that leads with one driver, one kernel version,
tried once.
Sixteen source files pointed at docs/plan/ldls.md and would have been dangling
the moment it moved. All repointed, and the milestone references -- which the
architecture doc deliberately does not have -- rewritten to name sections
instead, so a reader lands somewhere real.
The register row closes with it. hybrid.md freed a slot, ldls.md took it, and
it is now free again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz
parent
888c48fd96
commit
f38eb8a7fd
22 changed files with 278 additions and 929 deletions
232
docs/architecture/ldls.md
Normal file
232
docs/architecture/ldls.md
Normal file
|
|
@ -0,0 +1,232 @@
|
|||
# LDLS — running somebody else's drivers
|
||||
|
||||
SchOS cannot write a driver for every device that exists. No project can. The
|
||||
only route to the hardware support an established system has is to run Linux's
|
||||
drivers, and LDLS is what does that.
|
||||
|
||||
It works. `8139too` — an unmodified GPL kernel module — loads into a SchOS
|
||||
process, drives QEMU's rtl8139 through `/dev/pci`, and carries packets:
|
||||
|
||||
```
|
||||
# ldlsd /lib/modules/8139too.ko /lib/modules/btf &
|
||||
kpi: the driver read a MAC off the hardware: 52:54:00:ab:cd:ef
|
||||
ndo_open returned 0
|
||||
carrying frames. the interface is ldls0.
|
||||
|
||||
# ping 10.9.0.2 4
|
||||
64 bytes from 10.9.0.2: icmp_seq=1 time=30 ms
|
||||
...
|
||||
4 packets transmitted, 4 received, 0% packet loss
|
||||
```
|
||||
|
||||
`10.9.0.0/24` is reachable only through `ldls0`. The address on it came from
|
||||
DHCP, over the same driver.
|
||||
|
||||
---
|
||||
|
||||
## The rules it is built on
|
||||
|
||||
Five, and everything else is an implementation of them.
|
||||
|
||||
### 1. Nothing GPL ever enters an image we build
|
||||
|
||||
The driver is **the user's**: obtained by them, loaded on their machine. SchOS
|
||||
ships the layer and never the thing it loads. `.gitignore` refuses `*.ko`,
|
||||
`make ldls-stage` is the one command that puts a module into a local image and
|
||||
is deliberately not part of `all`, and `make ldls-unstage` takes it out again.
|
||||
|
||||
[../licensing.md](../licensing.md) §6 is the whole argument. The short form is
|
||||
that GPL obligations attach to *distribution*, we distribute Apache-2.0 code
|
||||
containing no GPL, and the combination happens in a user process on the user's
|
||||
machine — so no combined work is distributed by anyone.
|
||||
|
||||
### 2. It runs in a process, never in the kernel
|
||||
|
||||
A module linked into our kernel is the strongest derivative-work argument that
|
||||
exists. A module combined with our shim inside a user process is a different
|
||||
thing at arm's length — the position FreeBSD has shipped from for a decade.
|
||||
|
||||
And practically: this is foreign code. A driver we did not write, cannot fully
|
||||
audit and did not choose the quality of should not be able to take the machine
|
||||
down.
|
||||
|
||||
### 3. Facts, not expression
|
||||
|
||||
A symbol's signature and a struct's field offsets are facts a compiled driver
|
||||
depends on. Comments, macro bodies and implementations are expression and are
|
||||
never carried over. Every file in `ldls/kpi/` records where its shape came
|
||||
from.
|
||||
|
||||
Mostly the question does not arise, because of rule 3½:
|
||||
|
||||
### 3½. The layouts are not in our source at all
|
||||
|
||||
A kernel built with `CONFIG_DEBUG_INFO_BTF` publishes
|
||||
`/sys/kernel/btf/vmlinux`: every struct layout and every function prototype, as
|
||||
generated type data describing the kernel that is installed. `ldls/btf.c` reads
|
||||
it at load time, on the user's machine, from the user's own kernel.
|
||||
|
||||
```
|
||||
$ ldls-types /sys/kernel/btf/vmlinux --need
|
||||
pci_driver probe offset 16 size 8
|
||||
pci_dev resource offset 1000 size 1088
|
||||
resource start offset 0 size 8
|
||||
all present
|
||||
```
|
||||
|
||||
Nothing there is ours to have transcribed, and it is right for the kernel
|
||||
actually present rather than one pinned by guessing.
|
||||
|
||||
### 4. One kernel version at a time
|
||||
|
||||
Linux states in writing that it has no stable in-kernel ABI. BTF mostly
|
||||
dissolves this, because it describes the kernel that is there — but a kernel
|
||||
without BTF needs hand-written layouts for a named version, and that is when
|
||||
this rule bites.
|
||||
|
||||
### 5. The unresolved-symbol list is the specification
|
||||
|
||||
Do not guess the API surface. Load a real module and let the loader say what is
|
||||
missing. Every symbol LDLS does not implement still *resolves*, to a stub that
|
||||
reports itself and returns zero, so the driver carries on and the next thing it
|
||||
wants becomes visible. The work queue keeps writing itself.
|
||||
|
||||
---
|
||||
|
||||
## The four pieces
|
||||
|
||||
### The loader — `ldls/ko.c`
|
||||
|
||||
A `.ko` is an ELF64 relocatable object. Index the sections, place the
|
||||
allocatable ones, resolve the undefined symbols, apply the relocations. Written
|
||||
from the ELF-64 gABI and the x86-64 psABI, both published specifications, so
|
||||
there is no licence question in it at all.
|
||||
|
||||
Measured over a stock 6.18.46 tree: **2,440 modules, every one lays out and
|
||||
relocates to completion**, five relocation types, nothing unhandled. That sweep
|
||||
is also what retired the fixed section table — `nvidia-modeset` has 7,069
|
||||
sections against a median of 50 — so the caller supplies it and there is no
|
||||
limit to run into.
|
||||
|
||||
It is **freestanding**: no libc, no allocation, no I/O. The same source builds
|
||||
as a host tool, where a format failure costs a second, and as the SchOS-side
|
||||
host. That is the arrangement `mkschfs` and the kernel already share.
|
||||
|
||||
#### Two things it has to fix in the object
|
||||
|
||||
**The `%gs` stack guard.** Modules are built `-mstack-protector-guard-reg=gs`,
|
||||
so the canary is fetched through the GS segment — an ABI between the kernel and
|
||||
its own per-CPU area that a host which is not that kernel cannot provide.
|
||||
`ko_defuse_gs()` rewrites those references to `xor %reg, %reg`. There are two
|
||||
forms: the prologue *loads* the guard with `mov`, the epilogue *compares* it
|
||||
with `sub` before a `jne`. `xor` is right for both — it zeroes the canary and
|
||||
sets ZF, so the branch falls through as a passing check does. It checks every
|
||||
site before patching any and refuses the whole operation on anything it does
|
||||
not recognise.
|
||||
|
||||
**Where it is loaded.** `-mcmodel=kernel` means `R_X86_64_32S` relocations, so
|
||||
the object has to live where a signed 32-bit absolute address reaches. A
|
||||
process's mmap area starts at `0x40000000`, which is inside that window; on the
|
||||
host harness `MAP_32BIT` reaches it from the other end.
|
||||
|
||||
### The KPI — `ldls/kpi/`
|
||||
|
||||
The Linux symbols, implemented. Register access to a mapped BAR; the PCI calls
|
||||
to `/dev/pci`; allocation to `mmap` and `PCIIOC_DMA`; `register_netdev` to
|
||||
`/dev/netdev`. Organised under Linux's own header names so a reader who knows
|
||||
where something lives in Linux knows where it is here.
|
||||
|
||||
`kpi_io_ok()` bounds every `ioread`/`iowrite` against the mapping `pci_iomap`
|
||||
returned. A stray access would otherwise be a page fault with no context, and
|
||||
with no IOMMU a hosted driver's wild writes have to be caught early or not at
|
||||
all.
|
||||
|
||||
### The back end — `/dev/pci`
|
||||
|
||||
Already built, for `vgpud`, before any of this existed.
|
||||
[hybrid.md](../plan/hybrid.md) insisted the interface be about *PCI devices*
|
||||
rather than about a driver model, on the argument that an interface whose first
|
||||
consumers are native drivers is a general mechanism rather than a hole shaped
|
||||
like something foreign. A Linux driver's `pci_iomap`, `dma_alloc_coherent` and
|
||||
`request_irq` land on it unaltered, which is that argument being collected.
|
||||
|
||||
### The class bridge — `/dev/netdev`
|
||||
|
||||
The one piece the kernel had to grow. `netdev_ops` is four entries —
|
||||
`transmit`, `receive`, `mac`, `link_up` — so a process claims the device,
|
||||
supplies a MAC, and `read()`/`write()` carry the frames. `/dev/pci` with the
|
||||
arrows reversed: that hands a device *to* a process, this lets a process *be*
|
||||
one. See [devices.md](devices.md).
|
||||
|
||||
lwIP never learns that the thing behind the vtable is not a driver.
|
||||
|
||||
---
|
||||
|
||||
## What it costs
|
||||
|
||||
**It polls.** A Linux driver expects its interrupt handler to be called from an
|
||||
interrupt. `/dev/pci` does deliver them — by blocking `read()`, which needs a
|
||||
thread `ldlsd` does not have — so the driver's NAPI poll is called from a loop
|
||||
instead. The same work, in a context LDLS can provide, at the cost of latency
|
||||
and a spinning process.
|
||||
|
||||
**The stack protector is gone**, per `ko_defuse_gs()` above. Foreign code, no
|
||||
IOMMU, one hardening measure fewer.
|
||||
|
||||
**A lock is a no-op.** In one process with no preemption inside the driver
|
||||
there is nobody to race with. That is correct right up until a driver has an
|
||||
interrupt thread, and it is the deepest unfixed assumption in the layer.
|
||||
|
||||
---
|
||||
|
||||
## It can be killed
|
||||
|
||||
The whole H2 machinery applies to a foreign driver unchanged, which was the
|
||||
point of having built it against our own:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `ldlsd` running | `ping` works; it holds the device and two DMA blocks |
|
||||
| killed | interface **down**, device released, bus mastering cleared, **DMA reclaimed** |
|
||||
| `ping` | fails honestly — `sendto: -5` |
|
||||
| a second `ldlsd` | `ping` works again, and the arena hands back the same addresses |
|
||||
|
||||
`pciuser_release_all()` and `netuser_release_all()` sit beside each other in
|
||||
`exit()` for the same reason: a process holding a device is one that may crash.
|
||||
|
||||
---
|
||||
|
||||
## What it does not do
|
||||
|
||||
- **One driver, one kernel version, tried once.** `8139too` on 6.18.46 against
|
||||
QEMU's rtl8139. Nothing here has been tried on a second driver, a second
|
||||
kernel, or real silicon, and until it has, "LDLS runs Linux drivers" is a
|
||||
claim rather than a description.
|
||||
- **No interrupts**, per the polling note above.
|
||||
- **It is not a security boundary, and here that is worse than usual.** There
|
||||
is no IOMMU, so a hosted driver can point its device at any physical address.
|
||||
For SchOS's own drivers that isolates bugs and not intent; for *foreign*
|
||||
drivers the distinction between bug and intent is one we are in no position
|
||||
to make. This is the strongest argument for an IOMMU the project has
|
||||
produced.
|
||||
- **It relies on SchOS having no NX.** Every mapping is executable because
|
||||
`EFER.NXE` is never set. If that is ever fixed — and it should be — `ldlsd`
|
||||
needs a way to ask for executable memory, and `mmap` does not offer one.
|
||||
- **Nothing is ever freed.** A hosted driver's allocations last as long as the
|
||||
process, which is a leak in the strict sense and the right trade for a
|
||||
program that will be killed to restart it.
|
||||
- **BTF has to come from somewhere.** SchOS has no `/sys/kernel/btf/vmlinux`,
|
||||
so the user brings it with the module, from the kernel the module was built
|
||||
for. A kernel built without `CONFIG_DEBUG_INFO_BTF` has neither.
|
||||
- **No config-space writes.** `/dev/pci` has none, deliberately — a write is
|
||||
how a BAR gets moved out from under the kernel — so a driver that needs one
|
||||
is refused loudly rather than quietly succeeded.
|
||||
- **No module-to-module dependencies, no firmware loading, no sysfs, no
|
||||
unload.** `request_firmware` fails, `module_exit` is never called, and the
|
||||
way to stop a driver is to kill the process.
|
||||
- **Some symbols are still logging stubs.** Deliberate, per rule 5 — but a
|
||||
driver reaching one gets zero and carries on, and `memcpy` proved how quiet
|
||||
that can be: it was a stub while the receive path was being brought up, so
|
||||
every frame arrived as the right *length* of zeroes and nothing reported an
|
||||
error. **A stub that returns zero is not safe for a function whose whole job
|
||||
is a side effect.**
|
||||
Loading…
Reference in a new issue