SchOS/docs/architecture/ldls.md

LDLS — running somebody else's drivers

SchOS cannot write a driver for every device that exists. No project can. The only route to the hardware support an established system has is to run Linux's drivers, and LDLS is what does that.

It works. 8139too — an unmodified GPL kernel module — loads into a SchOS process, drives QEMU's rtl8139 through /dev/pci, and carries packets:

# ldlsd /lib/modules/8139too.ko /lib/modules/btf &
  kpi: the driver read a MAC off the hardware: 52:54:00:ab:cd:ef
  ndo_open returned 0
  carrying frames. the interface is ldls0.

# ping 10.9.0.2 4
64 bytes from 10.9.0.2: icmp_seq=1 time=30 ms
...
4 packets transmitted, 4 received, 0% packet loss

10.9.0.0/24 is reachable only through ldls0. The address on it came from DHCP, over the same driver.

---

The rules it is built on

Five, and everything else is an implementation of them.

1. Nothing GPL ever enters an image we build

The driver is the user's: obtained by them, loaded on their machine. SchOS ships the layer and never the thing it loads. .gitignore refuses *.ko, make ldls-stage is the one command that puts a module into a local image and is deliberately not part of all, and make ldls-unstage takes it out again.

../licensing.md §6 is the whole argument. The short form is that GPL obligations attach to distribution, we distribute Apache-2.0 code containing no GPL, and the combination happens in a user process on the user's machine — so no combined work is distributed by anyone.

2. It runs in a process, never in the kernel

A module linked into our kernel is the strongest derivative-work argument that exists. A module combined with our shim inside a user process is a different thing at arm's length — the position FreeBSD has shipped from for a decade.

And practically: this is foreign code. A driver we did not write, cannot fully audit and did not choose the quality of should not be able to take the machine down.

3. Facts, not expression

A symbol's signature and a struct's field offsets are facts a compiled driver depends on. Comments, macro bodies and implementations are expression and are never carried over. Every file in ldls/kpi/ records where its shape came from.

Mostly the question does not arise, because of rule 3½:

3½. The layouts are not in our source at all

A kernel built with CONFIG_DEBUG_INFO_BTF publishes /sys/kernel/btf/vmlinux: every struct layout and every function prototype, as generated type data describing the kernel that is installed. ldls/btf.c reads it at load time, on the user's machine, from the user's own kernel.

$ ldls-types /sys/kernel/btf/vmlinux --need
  pci_driver   probe      offset    16  size 8
  pci_dev      resource   offset  1000  size 1088
  resource     start      offset     0  size 8
  all present

Nothing there is ours to have transcribed, and it is right for the kernel actually present rather than one pinned by guessing.

4. One kernel version at a time

Linux states in writing that it has no stable in-kernel ABI. BTF mostly dissolves this, because it describes the kernel that is there — but a kernel without BTF needs hand-written layouts for a named version, and that is when this rule bites.

5. The unresolved-symbol list is the specification

Do not guess the API surface. Load a real module and let the loader say what is missing. Every symbol LDLS does not implement still resolves, to a stub that reports itself and returns zero, so the driver carries on and the next thing it wants becomes visible. The work queue keeps writing itself.

---

The four pieces

The loader — ldls/ko.c

A .ko is an ELF64 relocatable object. Index the sections, place the allocatable ones, resolve the undefined symbols, apply the relocations. Written from the ELF-64 gABI and the x86-64 psABI, both published specifications, so there is no licence question in it at all.

Measured over a stock 6.18.46 tree: 2,440 modules, every one lays out and relocates to completion, five relocation types, nothing unhandled. That sweep is also what retired the fixed section table — nvidia-modeset has 7,069 sections against a median of 50 — so the caller supplies it and there is no limit to run into.

It is freestanding: no libc, no allocation, no I/O. The same source builds as a host tool, where a format failure costs a second, and as the SchOS-side host. That is the arrangement mkschfs and the kernel already share.

Two things it has to fix in the object

The %gs stack guard. Modules are built -mstack-protector-guard-reg=gs, so the canary is fetched through the GS segment — an ABI between the kernel and its own per-CPU area that a host which is not that kernel cannot provide. ko_defuse_gs() rewrites those references to xor %reg, %reg. There are two forms: the prologue loads the guard with mov, the epilogue compares it with sub before a jne. xor is right for both — it zeroes the canary and sets ZF, so the branch falls through as a passing check does. It checks every site before patching any and refuses the whole operation on anything it does not recognise.

Where it is loaded. -mcmodel=kernel means R_X86_64_32S relocations, so the object has to live where a signed 32-bit absolute address reaches. A process's mmap area starts at 0x40000000, which is inside that window; on the host harness MAP_32BIT reaches it from the other end.

The KPI — ldls/kpi/

The Linux symbols, implemented. Register access to a mapped BAR; the PCI calls to /dev/pci; allocation to mmap and PCIIOC_DMA; register_netdev to /dev/netdev. Organised under Linux's own header names so a reader who knows where something lives in Linux knows where it is here.

kpi_io_ok() bounds every ioread/iowrite against the mapping pci_iomap returned. A stray access would otherwise be a page fault with no context, and with no IOMMU a hosted driver's wild writes have to be caught early or not at all.

The back end — /dev/pci

Already built, for vgpud, before any of this existed. hybrid.md insisted the interface be about PCI devices rather than about a driver model, on the argument that an interface whose first consumers are native drivers is a general mechanism rather than a hole shaped like something foreign. A Linux driver's pci_iomap, dma_alloc_coherent and request_irq land on it unaltered, which is that argument being collected.

The class bridge — /dev/netdev

The one piece the kernel had to grow. netdev_ops is four entries — transmit, receive, mac, link_up — so a process claims the device, supplies a MAC, and read()/write() carry the frames. /dev/pci with the arrows reversed: that hands a device to a process, this lets a process be one. See devices.md.

lwIP never learns that the thing behind the vtable is not a driver.

---

What it costs

It polls. A Linux driver expects its interrupt handler to be called from an interrupt. /dev/pci does deliver them — by blocking read(), which needs a thread ldlsd does not have — so the driver's NAPI poll is called from a loop instead. The same work, in a context LDLS can provide, at the cost of latency and a spinning process.

The stack protector is gone, per ko_defuse_gs() above. Foreign code, no IOMMU, one hardening measure fewer.

A lock is a no-op. In one process with no preemption inside the driver there is nobody to race with. That is correct right up until a driver has an interrupt thread, and it is the deepest unfixed assumption in the layer.

---

It can be killed

The whole H2 machinery applies to a foreign driver unchanged, which was the point of having built it against our own:

ldlsd running	ping works; it holds the device and two DMA blocks
killed	interface down, device released, bus mastering cleared, DMA reclaimed
ping	fails honestly — sendto: -5
a second ldlsd	ping works again, and the arena hands back the same addresses

pciuser_release_all() and netuser_release_all() sit beside each other in exit() for the same reason: a process holding a device is one that may crash.

---

What it does not do

• One driver, one kernel version, tried once. 8139too on 6.18.46 against QEMU's rtl8139. Nothing here has been tried on a second driver, a second kernel, or real silicon, and until it has, "LDLS runs Linux drivers" is a claim rather than a description.
• No interrupts, per the polling note above.
• It is not a security boundary, and here that is worse than usual. There is no IOMMU, so a hosted driver can point its device at any physical address. For SchOS's own drivers that isolates bugs and not intent; for foreign drivers the distinction between bug and intent is one we are in no position to make. This is the strongest argument for an IOMMU the project has produced.
• It relies on SchOS having no NX. Every mapping is executable because EFER.NXE is never set. If that is ever fixed — and it should be — ldlsd needs a way to ask for executable memory, and mmap does not offer one.
• Nothing is ever freed. A hosted driver's allocations last as long as the process, which is a leak in the strict sense and the right trade for a program that will be killed to restart it.
• BTF has to come from somewhere. SchOS has no /sys/kernel/btf/vmlinux, so the user brings it with the module, from the kernel the module was built for. A kernel built without CONFIG_DEBUG_INFO_BTF has neither.
• No config-space writes. /dev/pci has none, deliberately — a write is how a BAR gets moved out from under the kernel — so a driver that needs one is refused loudly rather than quietly succeeded.
• No module-to-module dependencies, no firmware loading, no sysfs, no unload. request_firmware fails, module_exit is never called, and the way to stop a driver is to kill the process.
• Some symbols are still logging stubs. Deliberate, per rule 5 — but a driver reaching one gets zero and carries on, and memcpy proved how quiet that can be: it was a stub while the receive path was being brought up, so every frame arrived as the right length of zeroes and nothing reported an error. A stub that returns zero is not safe for a function whose whole job is a side effect.