SchOS/docs/plan/ldls.md
Nexoniarz 42f98130c4 ldls: a Linux driver, driving real hardware
docs/plan/ldls.md milestone L3c. ldls/kpi/ implements 41 of 8139too's 94
symbols rather than logging them: register access to a mapped BAR, the PCI
calls to /dev/pci, allocation to mmap and PCIIOC_DMA, register_netdev to
/dev/netdev.

    # ldlsd /lib/modules/8139too.ko /lib/modules/btf
      found   10ec:8139 at 00:05.0
      bar 1    0x0000000081060000  256 mem
      calling probe, with 41 of 94 symbols implemented
      kpi: mapped BAR 1 -- phys 0x0000000081060000, 256 bytes
      kpi: the driver read a MAC off the hardware: 52:54:00🆎cd:ef
      netuser: ldls0 is a process now, 52:54:0🆎cd:ef
      probe returned 0 after 0 stub call(s)

52:54:00🆎cd:ef is what the machine was configured with. Nothing invented
it: the driver bit-banged the card's EEPROM, serially, through a BAR mapped by
/dev/pci, and handed back what it found. That it matches means the loader, the
relocations, the BTF offsets, the BAR mapping and the register accessors are
all correct at once, which is why it was the thing to aim at. Zero stub calls:
every symbol probe reaches is implemented.

Two bugs, one of them the kernel's.

Mine first: pci_iomap checked a failed mmap for -1 only, while SchOS returns
-errno. It returned -13 as a pointer, the driver took that as its register
base, and the first write landed at 0x4e -- thirteen below where it meant to
be.

Underneath it, the real one. sys_mmap rounds a length up to a page, the only
granularity an MMU has, and pciuser_mmio_ok() compared that against the BAR's
own size. rtl8139's registers are 256 bytes, the request became 4096, and the
answer was -EACCES for a device the process legitimately held. A BAR smaller
than a page could never be mapped at all, and nothing had noticed because the
only consumer until now was virtio-gpu, whose BARs are large. The check is now
against the pages the BAR lies in, and the widening is named where it happens:
a sub-page BAR hands over the whole page, forced by the hardware rather than
chosen.

That is exactly what hybrid.md said a foreign driver would be good for --
"what it asks for is what a foreign driver would ask for too" -- arriving as a
concrete bug rather than a prediction.

kpi_io_ok() now bounds every ioread/iowrite against the mapping pci_iomap
returned. A stray access would otherwise be a page fault with no context, and
with no IOMMU a hosted driver's wild writes have to be caught early or not at
all. It is also what turned the fault above into two lines naming the offsets.

Not done, and said so in the output rather than only in the doc: no frame has
moved. ndo_open is never called and there is no transmit or receive path, so
ldlsd releases the interface instead of leaving it up -- a link that reports
itself up with nothing draining its queue is a black hole, and the stack would
send DHCP into it for ever. Down is the true answer until packets move. That
is L3d, now listed separately.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
2026-09-14 09:59:14 +02:00

36 KiB

LDLS — the Linux Driver Layer for SchOS

SchOS cannot write a driver for every device that exists. No project can. The only route to the hardware support an established system has is to run someone else's drivers, and in practice "someone else's drivers" means Linux's.

LDLS itself is Apache-2.0, like everything else in this tree, and stays that way because it never contains a line of Linux. What the user combines it with on their own machine is a separate question, and ../licensing.md §6 is where that one is answered.

hybrid.md built the mechanisms for this and said so in as many words — MMIO, DMA, interrupts and config space to a process, with SchOS's own drivers as the first consumers so the interface would be general rather than a hole shaped like something foreign. vgpud proved it. LDLS is what those mechanisms were built for.


The rules

These are the spine. Everything below is an implementation of them, and anything that violates one is wrong however well it works.

1. Nothing GPL ever enters an image we build

The driver is the user's, obtained by them, loaded on their machine. SchOS ships the layer and never the thing it loads. This is not a new position — it was decided on 2026-08-08 and is written up in ../architecture/devices.md and ../licensing.md §2:

Apache-2.0 code and GPL-2.0-only code cannot be combined into one distributable work. It is not a matter of taste or of how carefully the files are kept apart in the tree.

RAMDISK_EXCLUDE and NOTPORTS are the existing mechanisms that keep this true mechanically rather than by intention, and LDLS must stay inside them.

2. It runs in a process, never in the kernel

Two independent reasons, and either alone would decide it.

Legally, a module linked into our kernel is the strongest derivative-work argument that exists — it is the case the FSF has always made. A module combined with our shim inside a user process, on the user's machine, at their direction is a different thing at arm's length, and it is the position FreeBSD has shipped from for about a decade: a BSD-licensed linuxkpi in base, GPL drm-kmod installed by the user from ports.

Practically, this is foreign code. A driver we did not write, cannot fully audit and did not choose the quality of, running with the machine's memory in reach, is exactly the code that should not be able to take the machine down. Crash isolation is not a nicety here; it is the reason.

3. Facts, not expression

The shim has to present Linux's internal API, and that API is expressed in GPL headers. The line LDLS holds:

A symbol exists, and takes these arguments fact — reproduce it
A struct has these fields in this order at these offsets fact — a compiled driver indexes them, so there is no other way to express it. And mostly it need not be in our source at all; see BTF below
The comment above it expression — never
The macro body, the inline function body, the implementation expression — never

Every KPI file records where its shape came from. Not decoration: it is the difference between a clean-room reimplementation and a transcription, and the only time anyone will ask is the time it matters.

This is the same discipline ../licensing.md §5 already describes — "the xHCI, USB, PCI, fbdev and ELF specifications were all read and implemented directly, which is why those drivers have no licence entanglement at all" — applied to a case where the specification is thinner.

Not legal advice, and it should be checked by someone whose job that is before anything ships. The reasoning rests on interfaces being outside copyright (EU Software Directive Art. 1(2)) and on Google v. Oracle (2021), and neither is a blanket permission.

The layouts do not have to be ours to write

The uncomfortable row above is the struct layouts, and it turns out to be largely avoidable. Kernels built with CONFIG_DEBUG_INFO_BTF publish /sys/kernel/btf/vmlinux, which carries the layouts and the function prototypes as generated binary type data about the kernel that is installed:

struct pci_dev {
        struct list_head           bus_list;      /*     0    16 */
        struct pci_bus *           bus;           /*    16     8 */

[97814] FUNC 'pci_iomap' type_id=97813 linkage=static
[97815] FUNC_PROTO '(anon)' ret_type_id=108 vlen=4

So LDLS reads them at load time, on the user's machine, from the user's own kernel. Our source contains neither, and there is nothing to have transcribed. It also mostly dissolves rule 4 below, because BTF describes the kernel that is actually there rather than one we pinned by guessing.

BTF gives shape, not behaviour. What napi_complete_done does is still ours to implement from documentation, and that is the whole of the real work. A kernel without BTF needs hand-written layouts for a named version, which is why this section stays rather than being replaced.

4. One kernel version at a time, pinned and named

(Mostly superseded by BTF, above — but it still governs any kernel that does not carry it.)

Linux states in writing that it has no stable in-kernel ABI — Documentation/process/stable-api-nonsense.rst exists to say so. Believing them is cheaper than being surprised by them.

So LDLS targets one version, says which, and is re-pinned deliberately. FreeBSD's LinuxKPI does exactly this per drm-kmod release. A layer that claims to support "Linux drivers" without a version is claiming something nobody has ever delivered.

5. The unresolved-symbol list is the specification

Do not guess the API surface. Load a real module and let the loader say what is missing. That list is the work queue, it is exact, and it is available before a single line of shim is written.


What it is made of

Four pieces, of which three are ordinary and one is the project.

The loader — ours, clean, and small

A .ko is a relocatable ELF64 object. Loading one is: lay out the allocatable sections, apply the relocations, resolve the undefined symbols against a table, and find the entry point. SchOS already parses ELF in kernel/fs/exec.c, for a different object type but with the same plumbing.

There is no licence question here at all — ELF is a published specification and so is the relocation set.

The KPI — the shim, and the actual work

Our Apache-2.0 reimplementation of the Linux symbols a given driver needs, organised under Linux's own header names so it is navigable: kpi/slab.h, kpi/pci.h, kpi/netdevice.h. Rule 3 governs every file.

The back end — already built

/dev/pci: PCIIOC_INFO, PCIIOC_CLAIM, PCIIOC_CFG, mmap of a BAR, PCIIOC_DMA, PCIIOC_IRQ. A Linux driver's pci_iomap, dma_alloc_coherent and request_irq land on these, which is the whole reason they were specified in terms of PCI devices rather than in terms of drivers.

The class bridge — the one new kernel interface

This is the piece that does not exist, and it is the interesting one.

A hosted driver produces packets in a process. SchOS's network stack consumes them through struct netdev_ops in the kernel — transmit, receive, mac, link_up, four entries and nothing else. So what is missing is a netdev whose backend is a process: the kernel publishes the vtable, the four calls are forwarded over a descriptor, and lwIP never learns that the driver is not in the kernel.

It is the exact analogue of what display_ops was for the GPU, and netdev.h's own opening comment already anticipates the shape — "the stack should ask for the network interfaces and get whatever is bound … exactly as a compositor asks for the display and does not care which driver answers."

One device, one process, claimed and released, on the /dev/pci discipline.


Milestones

L0 The loader. Parse a real .ko, lay it out, relocate it, and print every unresolved symbol. Nothing executes. The output is the measured API surface. done — ldls/ko.c, make ldls-check
L1 Stubs. Every symbol resolves, to a function that logs its name. init_module runs, which yields the call order — the second thing you need and the other thing you cannot guess. done — ldls/stub.c, ldls-run
L2 The hardware. probe() receives a working struct pci_dev, maps its BARs, allocates coherent DMA and registers an interrupt, all through /dev/pci. The driver reaches its device. done, except /dev/pci — see below
L3a The class bridge. /dev/netdev: a process claims it and becomes an interface. ping over a process. done — kernel/dev/netuser.c, cmd/netecho
L3b The SchOS-side host: ldls-run's loader and BTF work, as a program on SchOS. done — cmd/ldlsd; /dev/pci is L3c's half
L3c The fifteen symbols, implemented rather than stubbed, against a real rtl8139. done — ldls/kpi/; the driver reads the real MAC
L3d The packet path: ndo_open, ndo_start_xmit, receive.
L3 A packet. The two joined: ping over a Linux driver. This is the milestone that proves the whole idea and the one that cannot be skipped.
L4 Kill it. The host process dies, the kernel clears bus mastering and reclaims the DMA, a new host starts and the interface comes back. free — H2 built all of this

L4 is listed for completeness rather than as work. pciuser_stop(), pciuser_release_all() and KeepAlive already do it, and that they apply unchanged to a foreign driver is the point of having built them against our own.

The first target: 8139too, not r8169

The obvious choice is a Realtek r8169, because that silicon is everywhere. It is the wrong first choice for one decisive reason: QEMU cannot emulate it, so L0 through L3 would have no way to be tested, and a driver layer developed against hardware you cannot run is a layer nobody can debug.

QEMU emulates rtl8139. 8139too is small, self-contained, needs no firmware blob, and uses the thin mii helpers rather than dragging in phylib — which r8169 does, and which is a substantial surface on its own.

So: 8139too to build the layer, r8169 to prove it was general. The second is the interesting one and it is much cheaper once the first works.

This is the /dev/pci lesson applied backwards. There, a native driver was the first consumer so the interface would not be shaped like something foreign. Here the interface is fixed by Linux, and what is chosen instead is the smallest real consumer that can be tested — because the risk is no longer designing the wrong interface, it is building a hundred stubs nobody has ever run.


L0, and what it measured

ldls/ko.c is the loader: index the sections, place the allocatable ones, resolve the undefined symbols against a table, apply the relocations. It is freestanding — no libc, no allocation, no I/O — so the same source builds as a host tool today and as the SchOS-side host later, which is the arrangement mkschfs and the kernel already share.

$ ldls-inspect 8139too.ko
  119288 bytes, 55 sections, 363 symbols
  module    8139too
  license   GPL
  describes RealTek RTL-8139 Fast Ethernet driver
  loads to  33988 bytes

  relocations: 1980
    R_X86_64_64      363
    R_X86_64_PC32    991
    R_X86_64_PLT32   428
    R_X86_64_32S     169
    R_X86_64_PC64     29

  undefined symbols: 94
    ...
  stops at: unresolved: __fentry__

The licence is printed first, before anything is loaded, because it is the one property that decides whether the object may be on this machine at all and it is a plain string in .modinfo. Rule 1 is easier to keep when the answer is on screen.

What the measurement says

94 symbols is the API surface of 8139too on Linux 6.18.46. Not an estimate — the list, exactly, and it is L1's work queue. It is also smaller than it looks, because a good fraction of it is not API at all: __fentry__, __stack_chk_fail, __ref_stack_chk_guard, __x86_return_thunk and __fortify_panic are compiler and instrumentation artefacts that stub out in a line each.

Five relocation types, and a corpus run says that is the whole set: over 2,440 real modules from a stock 6.18.46 tree, every one lays out and relocates to completion with nothing unhandled. That includes nvidia-modeset at 7,069 sections and 74,305 relocations, which is what retired the loader's fixed section table — the caller supplies it now, so there is no limit to run into rather than a generous one.

Relocating against stubs is a separate question from listing what is missing, and it is the one that says the loader is finished: with nothing resolved, the first unknown name stops everything and no relocation type past it is ever reached.

Nothing foreign is in this tree

make ldls-check runs the loader against ldls/test/refmod.c — ours, compiled with -c, written to contain one of each thing the loader has to get right: undefined functions and undefined data, .text with calls, .data holding a pointer into .rodata, a .bss that must arrive zeroed, and a .modinfo shaped the way a module's is. The modules above were read from the host's own /nix/store and never copied here; .gitignore refuses *.ko outright so that cannot happen by accident.


L1, and the driver's own voice

Every undefined symbol resolves to a trampoline — twenty generated bytes that load the symbol's index and jump to a shared thunk, which saves the argument registers and reports the call. init_module is then simply called.

$ ldls-run 8139too.ko
  module  8139too
  license GPL
  wants   94 symbols
  loaded  33988 bytes at 0x416f8000, 94 stubs at 0x40377000

  calling init_module at 0x416fc020

    1  _printk(0x416fb6c0, 0, 0)
      | 8139too: 8139too Fast Ethernet driver 0.9.28
    2  __pci_register_driver(0x416fb400, 0x416fff80, 0x416fbdbb)

  init_module returned 0 after 2 stub call(s)

That second line is the driver's own banner, printed through a _printk that is ours, from a string in the module's own .rodata that a relocation put there. There is no stronger evidence that a load worked than the loaded thing speaking.

The stubs return zero rather than stopping, which is a deliberate departure from what this milestone originally said. Stopping at the first call tells you one name; carrying on tells you the sequence, and the sequence is the thing that cannot be read off a symbol table. A driver that then dereferences the zero faults, and that is a result too — it says exactly how far it got before it needed the kernel to have really done something.

The constraint nobody would guess

The first run failed on a relocation, not on anything to do with stubs:

ldls-run: relocate: a signed 32-bit relocation does not fit

A module is compiled -mcmodel=kernel, so the compiler emits R_X86_64_32S — a signed 32-bit absolute address — for anything it believes lives in the top 2 GB. Load the object at an ordinary mmap address around 0x7f… and those relocations have nowhere to go. Linux avoids this by giving modules a region within 2 GB of the kernel text; the harness uses MAP_32BIT to reach the same place from the other end.

On SchOS this costs nothing — a process's mmap area starts at 0x40000000 and is already inside the window — but it is a real constraint on where the LDLS host may put a module, and it would have been found much later and much more confusingly without a loader that checks its own arithmetic.

What running 500 of them says

init_module ran and returned 449
faulted on a stub's zero 22
no init_module at all 29
loader errors 0

The 22 are not failures of the loader; they are drivers that needed an allocation to come back non-null. The 29 without an entry point are mostly library modules that exist to be depended on.

And what it says about the work queue

Counting calls is the wrong metric — one module walking an rbtree called rb_first sixty-four times and told us nothing. Counting how many modules need a symbol is the useful one, and over 110 modules it comes to this:

symbol modules
__kmem_cache_create_args 25
crypto_register_template 11
_printk 10
crypto_register_alg 10
__register_nls 9
crypto_register_shash 4
af_alg_register_type 4
register_filesystem, misc_register, … 2 each

Forty-four distinct symbols would carry every one of those 110 modules through init_module. That is a small enough number to be a plan rather than an estimate, and it is not a number anybody could have guessed — the slab allocator at the top, and a long tail of registration calls that mostly want to record a pointer and return zero.

It says nothing about probe(), which is where the real surface is and which the kernel, not the module, is supposed to call. Reaching that is L2.


L2, and being the PCI core

In Linux, __pci_register_driver ends at the PCI core, which matches the driver's id_table against the bus and calls probe(). There is no PCI core here, so LDLS keeps the pointer that call was given and plays the part itself.

To do that it has to know where pci_driver.probe is — and it asks the kernel rather than being told by us. ldls/btf.c reads /sys/kernel/btf/vmlinux: 122,279 types, every struct layout and every function prototype, as generated data describing the kernel that is installed.

$ ldls-types /sys/kernel/btf/vmlinux --need
  pci_driver             probe                   offset    16  size 8
  pci_driver             id_table                offset     8  size 8
  pci_dev                vendor                  offset    60  size 2
  pci_dev                resource                offset  1000  size 1088
  resource               start                   offset     0  size 8
  ...
  all present

Cross-checked against pahole on the same file, field for field. No offset in this is ours, which is rule 3's uncomfortable row disappearing rather than being argued about.

What the driver then did

  pci_driver at 0x41662400, from BTF: probe +16, id_table +8
  driver name "8139too"

  devices it claims:
    10ec:8139  class 000000/000000
    ... and 16 more

  calling probe(pci_dev 10ec:8139, ...)

    1  alloc_etherdev_mqs(0x368, 0x1, 0x1)
    2  pci_enable_device(...)
    3  pci_request_regions(...)
    4  pci_set_master(...)
    5  pci_iomap(pdev, 1, 0)
    6..544   ioread8 / iowrite8   -- 0x88, 0x8a, 0x8c, 0x8e, over and over
  545  dev_addr_mod(...)
  547  netif_napi_add_weight_locked(...)
  549  timer_init_key(...)
  550  register_netdev(...)
  551  netdev_info(...)
      | %s at 0x%p, %pM, IRQ %d

  probe returned 0 after 552 stub call(s)

10ec:8139 is the real RTL8139 id. The driver mapped BAR 1, which is the one we marked IORESOURCE_MEM. And those 538 alternating byte accesses with values stepping 0x88, 0x8a, 0x8c, 0x8e are serial EEPROM clocking — the driver bit-banging the chip to read its MAC address, then handing the result to dev_addr_mod. It registered a NAPI instance, started a watchdog timer, registered the interface, printed its "found it" line, and returned 0.

That is a complete driver bring-up, on shape alone.

Fifteen symbols

The whole of that probe touches 15 distinct symbols, and 538 of the 552 calls are two of them:

ioread8 / iowrite8 / ioread32 540 calls, 3 symbols
PCI setup pci_enable_device, pci_request_regions, pci_set_master, pci_iomap
net alloc_etherdev_mqs, dev_addr_mod, netif_napi_add_weight_locked, register_netdev, netdev_info
other mutex_lock, mutex_unlock, timer_init_key

L1 said 44 symbols carry 110 modules through init. L2 says 15 carry this one through probe. Neither is a number anyone would have guessed, and both came out of running the thing rather than reading about it.

What is not done

probe() receives a pci_dev whose shape is right and whose contents are invented. The BARs are numbers this harness made up; pci_iomap returns a zeroed page, not a mapping; nothing has touched hardware and nothing could, because this runs on the build machine.

The half of L2 that says "all through /dev/pci" belongs to the SchOS side, where those calls have somewhere real to land. What is settled is the part that had to come first: that probe can be reached at all, with a struct the driver accepts, built from offsets nobody wrote down.


L3a: a network, in a process

L3 is one sentence in the milestone table and three pieces of work. This is the first, and it is the one the rest plugs into: /dev/netdev, an interface whose backend is a process.

The kernel side is netdev_ops and nothing else — transmit, receive, mac, link_up, the same four entries e1000 publishes. A process claims the device, supplies a MAC, and from then on read() gives it frames the stack wants sent and write() hands it frames that arrived. It is /dev/pci's shape with the arrows reversed: that hands a device to a process, this lets a process be one.

cmd/netecho is the proof, and it is deliberately not a driver — it has no hardware and claims none. It answers every frame itself: DHCP so the interface gets an address at all, ARP so the stack can find it, ICMP so it can be pinged.

# netecho &
netuser: ldls0 is a process now, 2:4c:44:4c:53:1
net: ldls0 attached as interface 1, asking for an address
netecho: this process is now an interface. I am 10.9.0.1 and will offer 10.9.0.2
netecho: answered DHCP, offering an address

# cat /proc/netif
interface   ldls0
driver      netuser
address     10.9.0.2
gateway     10.9.0.1

# ping 10.9.0.1 3
netecho: answered an ARP
netecho: answered a ping
64 bytes from 10.9.0.1: icmp_seq=1 time=30 ms
64 bytes from 10.9.0.1: icmp_seq=2 time=10 ms
64 bytes from 10.9.0.1: icmp_seq=3 time=10 ms
3 packets transmitted, 3 received, 0% packet loss

The interface got its address by DHCP from a process, and then answered ping from one. lwIP never learns that the thing on the other side of netdev_ops is not a driver — which is the entire property the bridge exists to have.

It dies the way it should

The same test, continued:

# kill 8
# schdbg netuser
  interface ldls0  DOWN, nobody holds it
# ping 10.9.0.1 1
ping: sendto: -5
0 packets transmitted, 0 received

# netecho &
# ping 10.9.0.1 2
2 packets transmitted, 2 received, 0% packet loss

link_up() answers false the moment nobody holds the descriptor, so the stack refuses to send rather than swallowing packets into a driver that is not there — and when a new process claims it, the same interface comes back, keeping its address. That is H2's property applied to a network interface, and it came free: netuser_release_all() sits beside pciuser_release_all() in exit() for exactly the same reason.

One new thing the kernel needed

netinit() starts DHCP on the devices that exist when it runs. An interface created later — which is how an LDLS-hosted driver arrives — missed that entirely and would have come up with no address and no way to get one. So net_attach_late() exists, in kernel/net/net.c because that is the only file allowed to include lwIP's headers.


L3b: the same loader, on SchOS

ldls/ko.c, ldls/btf.c and ldls/stub.c were written freestanding so they would build unchanged on both sides. cmd/ldlsd is the collection of that promise: the same three files, linked against lib/ instead of a libc, in an ordinary SchOS process.

# ldlsd /lib/modules/8139too.ko /lib/modules/btf
/lib/modules/8139too.ko
  module  8139too
  license GPL
  wants   94 symbols
  defused 6 %gs stack-guard reference(s)
  loaded  33988 bytes at 0x000000004001f000

  calling init_module
  1  _printk(...)
      | 8139too: 8139too Fast Ethernet driver 0.9.28
  2  __pci_register_driver(...)
  init_module returned 0 after 2 stub call(s)

  btf     122279 types
  layout  pci_driver.probe +16  pci_dev 2680 bytes
  driver  "8139too"
  devices it claims:
    10ec:8139
    ... and 18 more

  calling probe
  1  alloc_etherdev_mqs(0x368, 0x1)
  probe returned -12 after 1 stub call(s)

Byte for byte the behaviour of the host harness, which is what the shared source was for. Two predictions from earlier milestones came good without any work: a process's mmap area starts at 0x40000000, so the module lands inside the window R_X86_64_32S needs, and the kernel never sets EFER.NXE, so every mapping is executable.

That second one is a gap, not a feature. A process being able to execute its own heap is normally a thing an operating system prevents, and LDLS is quietly relying on its absence. It belongs on Gate 0's list.

The one thing that did not just work

The first run on SchOS faulted before probe's first call:

pid 8 ldlsd: trap 14 err 5 on cpu 0 rip 0x4002138e addr 0xffff80007de5fc60

err 5 is present, read, user mode, and the address is a kernel one. The rip resolved to .text+0x206e, which is:

mov %gs:0x0(%rip),%rax      # rtl8139_init_one+0x26

Linux builds modules with -mstack-protector-guard-reg=gs, so the stack canary is fetched through the GS segment. That is an ABI between the kernel and its own per-CPU area, and a host that is not that kernel cannot provide it — on SchOS the GS base belongs to the kernel, so a user-mode read of it faults on a kernel address.

ko_defuse_gs() rewrites those references to xor %reg, %reg. There are two forms and the second is easy to miss: the prologue loads the guard with mov (8b), and the epilogue compares it with sub (2b) followed by a jne to __stack_chk_fail. xor is correct for both — it makes the saved canary zero, and it sets ZF so the epilogue's branch falls through exactly as a passing check does. Six sites in 8139too.

It checks every site before patching any, and refuses the whole operation on anything it does not recognise. A loader that half-understands an instruction should not be editing it — and that refusal is what found the sub form, by declining to guess.

Getting the files there

make ldls-stage KO=… BTF=… copies a module and its kernel's BTF into the local sysroot. It is deliberately not part of all, .gitignore refuses *.ko, and make ldls-unstage removes them again. An image built between those two commands has somebody else's GPL code in it and must not be redistributed — which is exactly what ../licensing.md §6 says about the combination, and why it is a thing you type on purpose.


L3c: a Linux driver, driving real hardware

ldls/kpi/ is the shim: 41 of 8139too's 94 symbols, implemented rather than logged. Register access goes to a mapped BAR; pci_enable_device, pci_iomap, pci_request_regions and pci_set_master go to /dev/pci; allocation goes to mmap and PCIIOC_DMA; register_netdev goes to /dev/netdev. Every file records its provenance, per rule 3.

# ldlsd /lib/modules/8139too.ko /lib/modules/btf
  found   10ec:8139 at 00:05.0
  bar 0    0x000000000000c000  256 io
  bar 1    0x0000000081060000  256 mem
  netdev  2688 bytes

  calling probe, with 41 of 94 symbols implemented

  kpi: net_device 2688 bytes + 872 private
  kpi: mapped BAR 1 -- phys 0x0000000081060000, 256 bytes
  kpi: the driver read a MAC off the hardware: 52:54:00:ab:cd:ef
  net: ldls0 attached as interface 1, asking for an address
  netuser: ldls0 is a process now, 52:54:0:ab:cd:ef
  kpi: registered as an interface, with the hardware's own MAC

  probe returned 0 after 0 stub call(s)

52:54:00:ab:cd:ef is what the machine was configured with. Nothing here invented it: the driver bit-banged the card's EEPROM, serially, through a BAR mapped by /dev/pci, and handed back the address it found. That it matches means the loader, the relocations, the BTF offsets, the BAR mapping and the register accessors are all right at once — which is why it was chosen as the thing to aim at.

Zero stub calls. Every symbol that probe reaches is implemented.

The bug a foreign driver found in its first minute

The first run against real hardware wrote to address 0x4e and faulted. 0x4e is 13 below where the driver meant to write, and 13 is EACCES: pci_iomap had returned -13 as a pointer, because the check for a failed mmap looked only for -1 while SchOS returns -errno.

Underneath that was the real one. sys_mmap rounds a length up to a page — the only granularity an MMU has — and pciuser_mmio_ok() compared that against the BAR's own size. rtl8139's registers are 256 bytes, the request became 4096, and the answer was -EACCES for a device the process legitimately held.

A BAR smaller than a page could never be mapped at all, and nothing had noticed because the only consumer until now was virtio-gpu, whose BARs are large. The check is now against the pages the BAR lies in. The widening is real and named in the code: a sub-page BAR hands the process the whole page it sits in, which is forced by the hardware rather than chosen.

That is precisely what hybrid.md said a foreign driver would be good for — "what it asks for is what a foreign driver would ask for too" — arriving as a concrete bug rather than as a prediction.

Register access is checked

kpi_io_ok() bounds every ioread/iowrite against the mapping pci_iomap returned, reports the first few that miss, and returns without touching memory. Two reasons: a stray access would otherwise be a page fault with no context, and there is no IOMMU — a hosted driver's wild writes are exactly the class of thing that has to be caught early or not at all.

It is also what turned the bug above from a crash into two lines naming the offsets, which is how the -13 was spotted.

What is not done

probe() succeeds, so the driver is up and the interface exists with the hardware's own MAC. No frame has moved. ndo_open is never called, there is no ndo_start_xmit, and nothing services the receive side — so ldlsd releases the interface rather than leaving it up:

  the driver is up on real hardware and has an interface.
  carrying frames is the half of L3 that is not built, so the
  interface is released rather than left up and silent.

A link that reports itself up with nothing draining its queue is a black hole: the stack would send DHCP into it for ever and every layer above would believe the network was merely quiet. Down is the true answer until packets move, and that is L3d.


What it will not do

  • It is not a security boundary, and here that is worse than usual. There is no IOMMU, so a hosted driver can point its device at any physical address. For SchOS's own drivers that isolates bugs and not intent, which hybrid.md says at length. For foreign drivers the distinction between bug and intent is one we are in no position to make at all. This is the single strongest argument for an IOMMU that this project has produced, and it should be recorded as such rather than waved past.
  • It implements no licence check, and that is not circumvention. EXPORT_SYMBOL_GPL is Linux marking Linux's own symbols, and the check it drives protects those. Every symbol LDLS provides is ours, reimplemented from facts under Apache-2.0, so there is no gate being bypassed because there is no Linux symbol being exposed. LDLS reads the declared licence and prints it before loading anything, because that is the property that decides whether an object belongs on a machine at all. See ../licensing.md §6.
  • No firmware loading. request_firmware is a stub that fails. Many drivers need a blob, most blobs are redistributable only under their own terms, and that is a second licensing problem which is not this one.
  • No module-to-module dependencies. One object, no depmod, no symbol exports between hosted modules.
  • No sysfs, procfs or debugfs. Drivers that only report through them will work; drivers that are configured through them will not.
  • No unload. module_exit is never called; the way to stop a driver is to kill the process, which H2 made a supported operation rather than a failure.
  • No sleeping-context fidelity. Linux drivers assume a kernel's rules about what may sleep, what runs in interrupt context and what spin_lock_irqsave costs. A process has different rules, and where they differ the shim will be wrong in ways that show up as rare races rather than as failures to build. This is the deepest technical risk in the whole plan.
  • Running init is not working. L0 proves 2,440 modules lay out and relocate; L1 proves 449 of 500 run their init to completion. Both say the mechanism is right and neither says a driver would function: every symbol is a stub that logs and returns zero, so what has been demonstrated is that drivers ask for things in a readable order, not that anything answers them. The distance from here to a packet is the rest of this plan.
  • The stubs cannot tell code from data. Every undefined symbol in a module is NOTYPE in the ELF symbol table, so jiffies and pci_iomap are indistinguishable and both get a trampoline. A driver that reads jiffies reads trampoline bytes. BTF distinguishes them — VAR against FUNC — and that is where the fix belongs rather than in a guess.
  • ldls-run is a development harness, not the deliverable. It runs on the build machine so that loader failures cost a second instead of a boot. The thing that ships is the SchOS-side host.
  • The stack protector is gone from a hosted driver. ko_defuse_gs() removes it because the ABI it depends on cannot be supplied here. That is a hardening measure lost, on foreign code, in a host with no IOMMU — worth saying plainly rather than filing as a detail.
  • LDLS relies on SchOS having no NX. Every mapping is executable because EFER.NXE is never set. If that is ever fixed — and it should be — ldlsd needs a way to ask for executable memory, and nothing in mmap currently offers one.
  • BTF has to come from somewhere. On Linux it is /sys/kernel/btf/vmlinux. SchOS has no such file and never will — so the user brings the BTF along with the module, from the kernel the module was built for. Both are theirs, neither is shipped, and a kernel built without CONFIG_DEBUG_INFO_BTF has neither.
  • L3a proves the bridge, not the driver. ping works over a process; nothing in that test loads a module, and netecho is a fake network rather than a NIC. That is the point of doing it first — if ping had not worked here the bridge would be wrong, and now a failure with a real driver can only be the driver's side.
  • One interface, one process. /dev/netdev is a single device and a second claimant gets -EBUSY. Two hosted NICs would need it to be indexed like /dev/input/eventN, which is a change of shape rather than of size.
  • The interface is never removed. Releasing it leaves the device registered with the link down, because nothing in lwIP supports taking an interface out of its list. From the stack's point of view that is the same thing; from /proc/netif's it is a ldls0 that stays listed after its process is gone.
  • No frame has moved. L3c ends at a successful probe(). The packet path — ndo_open, ndo_start_xmit, receive, and the skb handling all three need — is L3d and is the larger half.
  • 41 of 94 symbols. The rest are still logging stubs, which is deliberate: an unimplemented symbol that reports itself keeps the work queue writing itself, exactly as it did at L1.
  • A lock is a no-op. In one process with no preemption inside the driver there is nobody to race with, which is correct right up until the driver has an interrupt thread. That is the sleeping-context assumption named above and it is the deepest unfixed thing in the layer.
  • It will not support "Linux drivers". It will support one driver, on one kernel version, that has been tried. Anything beyond that is a claim, and the list of what has actually been run belongs in the architecture document when there is one.