docs/plan/ldls.md milestone L3c. ldls/kpi/ implements 41 of 8139too's 94
symbols rather than logging them: register access to a mapped BAR, the PCI
calls to /dev/pci, allocation to mmap and PCIIOC_DMA, register_netdev to
/dev/netdev.
# ldlsd /lib/modules/8139too.ko /lib/modules/btf
found 10ec:8139 at 00:05.0
bar 1 0x0000000081060000 256 mem
calling probe, with 41 of 94 symbols implemented
kpi: mapped BAR 1 -- phys 0x0000000081060000, 256 bytes
kpi: the driver read a MAC off the hardware: 52:54:00🆎cd:ef
netuser: ldls0 is a process now, 52:54:0🆎cd:ef
probe returned 0 after 0 stub call(s)
52:54:00🆎cd:ef is what the machine was configured with. Nothing invented
it: the driver bit-banged the card's EEPROM, serially, through a BAR mapped by
/dev/pci, and handed back what it found. That it matches means the loader, the
relocations, the BTF offsets, the BAR mapping and the register accessors are
all correct at once, which is why it was the thing to aim at. Zero stub calls:
every symbol probe reaches is implemented.
Two bugs, one of them the kernel's.
Mine first: pci_iomap checked a failed mmap for -1 only, while SchOS returns
-errno. It returned -13 as a pointer, the driver took that as its register
base, and the first write landed at 0x4e -- thirteen below where it meant to
be.
Underneath it, the real one. sys_mmap rounds a length up to a page, the only
granularity an MMU has, and pciuser_mmio_ok() compared that against the BAR's
own size. rtl8139's registers are 256 bytes, the request became 4096, and the
answer was -EACCES for a device the process legitimately held. A BAR smaller
than a page could never be mapped at all, and nothing had noticed because the
only consumer until now was virtio-gpu, whose BARs are large. The check is now
against the pages the BAR lies in, and the widening is named where it happens:
a sub-page BAR hands over the whole page, forced by the hardware rather than
chosen.
That is exactly what hybrid.md said a foreign driver would be good for --
"what it asks for is what a foreign driver would ask for too" -- arriving as a
concrete bug rather than a prediction.
kpi_io_ok() now bounds every ioread/iowrite against the mapping pci_iomap
returned. A stray access would otherwise be a page fault with no context, and
with no IOMMU a hosted driver's wild writes have to be caught early or not at
all. It is also what turned the fault above into two lines naming the offsets.
Not done, and said so in the output rather than only in the doc: no frame has
moved. ndo_open is never called and there is no transmit or receive path, so
ldlsd releases the interface instead of leaving it up -- a link that reports
itself up with nothing draining its queue is a black hole, and the stack would
send DHCP into it for ever. Down is the true answer until packets move. That
is L3d, now listed separately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L3b. ldls/ko.c, btf.c and stub.c were written
freestanding so they would build unchanged on both sides; cmd/ldlsd collects
that promise -- the same three files, linked against lib/ instead of a libc,
in an ordinary SchOS process.
# ldlsd /lib/modules/8139too.ko /lib/modules/btf
module 8139too license GPL wants 94 symbols
defused 6 %gs stack-guard reference(s)
calling init_module
1 _printk(...)
| 8139too: 8139too Fast Ethernet driver 0.9.28
2 __pci_register_driver(...)
btf 122279 types
layout pci_driver.probe +16 pci_dev 2680 bytes
driver "8139too" devices it claims: 10ec:8139 ... and 18 more
calling probe
1 alloc_etherdev_mqs(0x368, 0x1)
probe returned -12 after 1 stub call(s)
Byte for byte the host harness's behaviour, which is what the shared source
was for. Two earlier predictions came good with no work: a process's mmap area
starts at 0x40000000, inside the window R_X86_64_32S needs, and the kernel
never sets EFER.NXE so every mapping is executable. The second is a gap rather
than a feature and is now recorded as one -- LDLS is relying on the absence of
a protection an OS should have.
One thing did not just work. The first run faulted before probe's first call,
err 5 -- present, read, user -- at a kernel address, with the rip resolving to
mov %gs:0x0(%rip),%rax # rtl8139_init_one+0x26
Modules are built -mstack-protector-guard-reg=gs, so the canary comes through
the GS segment, and that is an ABI between the kernel and its own per-CPU area
which a host that is not that kernel cannot provide. ko_defuse_gs() rewrites
those references to xor %reg,%reg.
There are two forms and the second is easy to miss: the prologue loads the
guard with mov (8b) and the epilogue compares it with sub (2b) before a jne to
__stack_chk_fail. xor is right for both -- it zeroes the saved canary and sets
ZF, so the epilogue's branch falls through exactly as a passing check does. Six
sites in 8139too.
It checks every site before patching any and refuses the whole operation on
anything it does not recognise. That refusal is what found the sub form, by
declining to guess -- a loader that half-understands an instruction should not
be editing it.
make ldls-stage/ldls-unstage put a module and its kernel's BTF into the local
sysroot and take them out again. Not part of `all`, and .gitignore refuses
*.ko, so an image only carries somebody else's GPL code between two commands
you typed on purpose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md L3a -- the class bridge, and the first of the three pieces
L3 turned out to be.
/dev/netdev is /dev/pci with the arrows reversed. That hands a device to a
process; this lets a process be one. The kernel side is netdev_ops and nothing
else -- transmit, receive, mac, link_up, the same four entries e1000
publishes -- so a process claims the device, supplies a MAC, and from then on
read() gives it frames the stack wants sent and write() hands it frames that
arrived.
cmd/netecho proves it and is deliberately not a driver: no hardware, none
claimed, every frame answered by the program itself. DHCP so the interface
gets an address at all, ARP so the stack can find it, ICMP so it can be
pinged. All built by hand from RFCs 826, 791, 792 and 2131.
# netecho &
netuser: ldls0 is a process now, 2:4c:44:4c:53:1
netecho: I am 10.9.0.1 and will offer 10.9.0.2
netecho: answered DHCP, offering an address
# cat /proc/netif
interface ldls0 driver netuser address 10.9.0.2 gateway 10.9.0.1
# ping 10.9.0.1 3
netecho: answered an ARP
netecho: answered a ping
3 packets transmitted, 3 received, 0% packet loss
The interface got its address by DHCP from a process and then answered ping
from one. lwIP never learns that the thing behind netdev_ops is not a driver,
which is the whole property the bridge exists to have.
It dies the way it should, too. kill the process and link_up() answers false
immediately, so the stack refuses to send -- "ping: sendto: -5" -- rather than
swallowing packets into a driver that is not there. Start another and the
*same* interface comes back, keeping its address. That came free:
netuser_release_all() sits beside pciuser_release_all() in exit() for exactly
the same reason, and the reason is that a process holding a device is one that
may crash.
One thing the kernel needed: netinit() starts DHCP on the devices that exist
when it runs, so an interface created later -- which is how an LDLS-hosted
driver arrives -- would have come up with no address and no way to get one.
net_attach_late() fixes that, and lives in kernel/net/net.c because that is
the only file allowed to include lwIP's headers.
Recorded as not done, because it is not: this proves the bridge, not a driver.
Nothing in that test loads a module. That is why it was done first -- if ping
had not worked here the bridge would be wrong, and now a failure with a real
driver can only be the driver's side. L3b is the SchOS-side host, L3c is the
fifteen symbols for real, and L3 is the two joined.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L2, less the half that needs /dev/pci.
In Linux, __pci_register_driver ends at the PCI core, which matches a
driver's id_table against the bus and calls probe(). There is no PCI core
here, so LDLS keeps the pointer that call was given and plays the part.
To do that it must know where pci_driver.probe is, and it asks the kernel
rather than being told by us. ldls/btf.c reads /sys/kernel/btf/vmlinux --
122,279 types, every struct layout and every function prototype, as generated
data describing the kernel that is installed. Implemented from
Documentation/bpf/btf.rst, which is a published specification.
$ ldls-types /sys/kernel/btf/vmlinux --need
pci_driver probe offset 16 size 8
pci_dev resource offset 1000 size 1088
resource start offset 0 size 8
all present
Cross-checked against pahole on the same file, field for field. No offset in
this is ours, which is rule 3's uncomfortable row disappearing rather than
being argued about.
And then the driver:
driver name "8139too"
devices it claims: 10ec:8139 ... and 16 more
calling probe(pci_dev 10ec:8139, ...)
1 alloc_etherdev_mqs(0x368, 1, 1)
2 pci_enable_device / 3 pci_request_regions / 4 pci_set_master
5 pci_iomap(pdev, 1, 0)
6..544 ioread8 / iowrite8 -- 0x88, 0x8a, 0x8c, 0x8e, over and over
545 dev_addr_mod / 547 netif_napi_add_weight_locked
549 timer_init_key / 550 register_netdev / 551 netdev_info
probe returned 0 after 552 stub call(s)
10ec:8139 is the real RTL8139 id. It mapped BAR 1, the one marked
IORESOURCE_MEM. Those 538 alternating byte accesses stepping 0x88/0x8a/0x8c/
0x8e are serial EEPROM clocking -- the driver bit-banging the chip for its MAC
address and handing the result to dev_addr_mod. Then NAPI, a watchdog timer,
register_netdev, its "found it" line, and zero.
Fifteen distinct symbols carry that entire probe, and three of them are 540 of
the 552 calls. L1 said 44 symbols carry 110 modules through init; L2 says 15
carry this one through probe. Neither is a number anyone would have guessed
and both came from running the thing.
Recorded as not done, because it is not: probe receives a pci_dev whose shape
is right and whose contents are invented. The BARs are made up, pci_iomap
returns a zeroed page, and 8139too read a MAC address of zeroes and was
perfectly happy about it. The "all through /dev/pci" half of L2 belongs to the
SchOS side. Also recorded: BTF has to come from somewhere, and SchOS has no
/sys/kernel/btf/vmlinux -- the user brings it with the module, from the kernel
the module was built for.
make ldls-check is 27 checks, four of them L2's, and it says so out loud when
it skips them on a machine with no BTF rather than passing silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Counting stub calls turned out to be the wrong metric -- one module walking an
rbtree called rb_first sixty-four times and told us nothing. Counting how many
*modules* need a symbol is the useful one:
__kmem_cache_create_args 25
crypto_register_template 11
_printk 10
crypto_register_alg 10
__register_nls 9
...
Forty-four distinct symbols carry all 110 modules through init_module. Small
enough to be a plan rather than an estimate, and not a number anyone could
have guessed: the slab allocator at the top and a long tail of registration
calls that mostly want to record a pointer and return zero.
It says nothing about probe(), which is where the real surface is and which
the kernel rather than the module is supposed to call. That is L2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L1. Every undefined symbol resolves to a
trampoline -- twenty generated bytes that load the symbol's index and jump to
a shared thunk, which saves the argument registers and reports the call. Then
init_module is simply called.
$ ldls-run 8139too.ko
module 8139too
license GPL
wants 94 symbols
calling init_module at 0x416fc020
1 _printk(0x416fb6c0, 0, 0)
| 8139too: 8139too Fast Ethernet driver 0.9.28
2 __pci_register_driver(0x416fb400, 0x416fff80, 0x416fbdbb)
init_module returned 0 after 2 stub call(s)
That second line is the driver's own banner, printed through a _printk that
is ours, from a string in the module's own .rodata that a relocation put
there. There is no stronger evidence that a load worked than the loaded thing
speaking.
Stubs return zero rather than stopping, which departs from what the milestone
originally said and does so deliberately: stopping at the first call tells you
one name, carrying on tells you the sequence, and the sequence is the thing a
symbol table cannot give you. A driver that then dereferences the zero faults,
and that is a result too -- it says how far it got before it needed the kernel
to have really done something.
The first run failed on neither of those. It failed on a relocation:
ldls-run: relocate: a signed 32-bit relocation does not fit
A module is compiled -mcmodel=kernel, so the compiler emits R_X86_64_32S for
anything it believes lives in the top 2 GB, and an ordinary mmap at 0x7f...
has nowhere to put it. Linux gives modules a region within 2 GB of the kernel
text; the harness uses MAP_32BIT to reach the same window from the other end.
On SchOS this costs nothing -- a process's mmap area starts at 0x40000000 and
is already inside it -- but it is a real constraint on where the host may load
a module, and a loader that did not check its own arithmetic would have found
it much later and much more confusingly.
Swept over 500 stock modules: 449 ran init to completion, 22 faulted on a
stub's zero, 29 have no init_module, and zero hit a loader error.
Recorded honestly in "what it will not do": running init is not working. Every
symbol is a stub, so what has been shown is that drivers ask for things in a
readable order, not that anything answers. Also recorded: every undefined
symbol in a module is NOTYPE, so the stubs cannot tell jiffies from
pci_iomap and both get a trampoline -- BTF distinguishes VAR from FUNC and
that is where the fix belongs rather than in a guess.
make ldls-check is 23 checks now, five of them L1's, all against
ldls/test/refmod.c which is ours and now has an init_module of its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Answers the question directly in docs/licensing.md §6, because "LDLS runs
Linux drivers" reads like it contradicts §2 and does not.
LDLS is Apache-2.0. Every file under ldls/ is ours; the loader came from the
ELF-64 gABI and the x86-64 psABI. The licence of LDLS and the licence status
of what happens at run time are two different questions and only the first is
about us:
ldls/ Apache-2.0 distributed by us
a .ko GPL-2.0 distributed by the user's distro
the two in one process -- distributed by nobody
GPLv2 §0 says where its obligations attach -- "the act of running the Program
is not restricted" -- so no combined work is distributed by anyone and no
obligation is triggered. Same position FreeBSD has shipped from for a decade.
Stated honestly in the same section: the *user* cannot redistribute the
combination either, which is why RAMDISK_EXCLUDE exists and why .gitignore
refuses *.ko.
EXPORT_SYMBOL_GPL was left open in the plan; it is now answered and the answer
is cleaner than expected. That marking gates *Linux's own* symbols. Every
symbol LDLS provides is ours, reimplemented from facts, so there is no gate
being bypassed because no Linux symbol is being exposed. LDLS implements no
licence check -- it reads the declared licence and prints it before loading
anything.
And the sharpest edge in rule 3 mostly goes away. The uncomfortable row was
struct layouts, which a compiled driver indexes by offset, so the shim must
reproduce them exactly -- and a field-for-field match looks like transcription
even when it is not. It does not have to be in our source: kernels built with
CONFIG_DEBUG_INFO_BTF publish /sys/kernel/btf/vmlinux, and that carries both
the layouts and the function prototypes as generated type data about the
kernel the user already has. Verified on this machine: struct pci_dev field by
field, and FUNC 'pci_iomap' with its FUNC_PROTO. LDLS reads them at load time
on the user's machine; our source contains neither.
That also mostly dissolves rule 4, since BTF describes the kernel that is
actually installed rather than one pinned by guessing. What BTF does not give
is behaviour -- what napi_complete_done *does* is still ours to write from
documentation, and that is the whole of the real work.
Also adds the SPDX header ldls/ko.h was missing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L0. ldls/ko.c indexes the sections of an ELF64
relocatable object, places the allocatable ones, resolves undefined symbols
against a caller's table and applies the relocations. Nothing is executed.
It is freestanding -- no libc, no allocation, no I/O -- so the same source
builds as a host tool now and as the SchOS-side host later. That is the
arrangement mkschfs and the kernel already share, and for the same reason:
the failures at this layer are format failures, and one found on the host
costs a second instead of a boot.
$ ldls-inspect 8139too.ko
module 8139too
license GPL
loads to 33988 bytes
relocations: 1980
R_X86_64_64 363, PC32 991, PLT32 428, 32S 169, PC64 29
undefined symbols: 94
stops at: unresolved: __fentry__
The licence is printed before anything is loaded, because it is the one
property that decides whether the object may be on the machine at all and it
is a plain string in .modinfo. Rule 1 is easier to keep when the answer is on
screen.
94 symbols is the measured API surface of 8139too on 6.18.46 -- the list,
exactly, rather than an estimate -- and it is L1's work queue. It is smaller
than it looks: __fentry__, __stack_chk_fail, __ref_stack_chk_guard,
__x86_return_thunk and __fortify_panic are compiler and instrumentation
artefacts that stub out in a line each.
Swept over every module in a stock 6.18.46 tree: 2,440 objects, all of them
laid out and relocated to completion, none with a relocation type the loader
does not handle. That sweep is also what retired the fixed section table --
nvidia-modeset has 7,069 sections and 74,305 relocations, against a median of
50 -- so the caller supplies the table and there is no limit to run into
rather than a generous one that fails later.
Loading is not running, and the doc says so: not one symbol is implemented,
and the distance between "relocates" and "transmits a packet" is the rest of
the plan.
No .ko is in this tree and .gitignore now refuses one outright. make
ldls-check runs 18 checks against ldls/test/refmod.c, which is ours, written
to contain one of each thing the loader has to get right.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md. SchOS cannot write a driver for every device that exists
and no project can, so the only route to the hardware support an established
system has is to run Linux's. hybrid.md built the mechanisms for this and said
so; vgpud proved them. This is what they were for.
Five rules, and everything else is an implementation of them:
1. Nothing GPL ever enters an image we build. The driver is the user's,
obtained by them, loaded on their machine.
2. It runs in a process, never in the kernel. Legally a module linked into
our kernel is the strongest derivative-work argument that exists, and
practically this is foreign code that must not be able to take the
machine. FreeBSD has shipped from this position for a decade.
3. Facts, not expression. A symbol's signature and a struct's offsets are
facts a compiled driver depends on; comments, macro bodies and inline
bodies are expression and are never carried over. Provenance recorded per
file. Flagged in the doc as wanting real legal advice before shipping.
4. One kernel version at a time, pinned and named, because Linux says in
writing that it has no stable in-kernel ABI.
5. The unresolved-symbol list is the specification. Load a real module and
let the loader say what is missing rather than guessing the surface.
Four pieces: an ELF relocatable loader (ours, clean, no licence question), the
KPI shim (the actual work), /dev/pci as the back end (built), and one new
kernel interface -- a netdev whose backend is a process, which is the exact
analogue of display_ops and is four vtable entries wide.
First target is 8139too rather than r8169, and the reason is decisive: QEMU
emulates rtl8139 and does not emulate r8169, so the obvious choice cannot be
tested at all. 8139too is also smaller, needs no firmware, and uses the thin
mii helpers instead of dragging in phylib. r8169 is what proves it was general,
afterwards and much more cheaply.
The "what it will not do" section carries the honest list, including the two
that are not yet answered: no EXPORT_SYMBOL_GPL story, and no IOMMU -- which
for foreign drivers is a materially worse problem than for our own, and is the
strongest argument for one this project has produced.
hybrid.md is closed and leaves the register; ldls.md takes the slot, which is
the freeze rule working rather than being bent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Nexoniarz