ldls: a Linux driver, driving real hardware
docs/plan/ldls.md milestone L3c. ldls/kpi/ implements 41 of 8139too's 94
symbols rather than logging them: register access to a mapped BAR, the PCI
calls to /dev/pci, allocation to mmap and PCIIOC_DMA, register_netdev to
/dev/netdev.
# ldlsd /lib/modules/8139too.ko /lib/modules/btf
found 10ec:8139 at 00:05.0
bar 1 0x0000000081060000 256 mem
calling probe, with 41 of 94 symbols implemented
kpi: mapped BAR 1 -- phys 0x0000000081060000, 256 bytes
kpi: the driver read a MAC off the hardware: 52:54:00🆎cd:ef
netuser: ldls0 is a process now, 52:54:0🆎cd:ef
probe returned 0 after 0 stub call(s)
52:54:00🆎cd:ef is what the machine was configured with. Nothing invented
it: the driver bit-banged the card's EEPROM, serially, through a BAR mapped by
/dev/pci, and handed back what it found. That it matches means the loader, the
relocations, the BTF offsets, the BAR mapping and the register accessors are
all correct at once, which is why it was the thing to aim at. Zero stub calls:
every symbol probe reaches is implemented.
Two bugs, one of them the kernel's.
Mine first: pci_iomap checked a failed mmap for -1 only, while SchOS returns
-errno. It returned -13 as a pointer, the driver took that as its register
base, and the first write landed at 0x4e -- thirteen below where it meant to
be.
Underneath it, the real one. sys_mmap rounds a length up to a page, the only
granularity an MMU has, and pciuser_mmio_ok() compared that against the BAR's
own size. rtl8139's registers are 256 bytes, the request became 4096, and the
answer was -EACCES for a device the process legitimately held. A BAR smaller
than a page could never be mapped at all, and nothing had noticed because the
only consumer until now was virtio-gpu, whose BARs are large. The check is now
against the pages the BAR lies in, and the widening is named where it happens:
a sub-page BAR hands over the whole page, forced by the hardware rather than
chosen.
That is exactly what hybrid.md said a foreign driver would be good for --
"what it asks for is what a foreign driver would ask for too" -- arriving as a
concrete bug rather than a prediction.
kpi_io_ok() now bounds every ioread/iowrite against the mapping pci_iomap
returned. A stray access would otherwise be a page fault with no context, and
with no IOMMU a hosted driver's wild writes have to be caught early or not at
all. It is also what turned the fault above into two lines naming the offsets.
Not done, and said so in the output rather than only in the doc: no frame has
moved. ndo_open is never called and there is no transmit or receive path, so
ldlsd releases the interface instead of leaving it up -- a link that reports
itself up with nothing draining its queue is a black hole, and the stack would
send DHCP into it for ever. Down is the true answer until packets move. That
is L3d, now listed separately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz
parent
3fd1cc5279
commit
42f98130c4
11 changed files with 1071 additions and 44 deletions
|
|
@ -80,6 +80,15 @@ Three of those are worth expanding on, because each encodes a decision:
|
|||
a BAR of a device this process holds is refused. That is the whole difference
|
||||
between this and `/dev/mem` with extra steps.
|
||||
|
||||
The check is against the *pages* a BAR lies in rather than the BAR itself,
|
||||
because a page is the only granularity an MMU has. A BAR smaller than a page
|
||||
therefore hands the process the whole page it sits in — forced by the hardware
|
||||
rather than chosen. Comparing against the BAR's own size instead meant a
|
||||
sub-page BAR could not be mapped at all, which nothing noticed until an
|
||||
rtl8139 with 256 bytes of registers turned up; see
|
||||
[../plan/ldls.md](../plan/ldls.md), "The bug a foreign driver found in its
|
||||
first minute".
|
||||
|
||||
**DMA is a call, not a mapping.** The process is told the *physical* address,
|
||||
which is precisely what it cannot work out for itself, and the memory comes from
|
||||
an arena reserved at boot before the general allocator gets anything.
|
||||
|
|
|
|||
|
|
@ -95,7 +95,7 @@ The galling ones. The work exists, is tested, and nothing uses it.
|
|||
|---|---|
|
||||
| **`foundations.md`** | Item 1 built. Items 2 (a shareable memory object), 3 (`AF_UNIX` in the filesystem namespace), 4 and 5 open. **Wayland stops at item 2.** |
|
||||
| ~~**`hybrid.md`**~~ | **Closed.** All four mechanisms built; H1, H2 and H3 done. H4 — the USB eviction — is *withdrawn* rather than outstanding: USB is the bus peripherals arrive on, not a peripheral, and an eviction that has to leave an identical driver behind has moved nothing. See [hybrid.md](hybrid.md), "Decided: USB stays in the kernel". |
|
||||
| **`ldls.md`** | L0, L1, L2 and L3a done — the loader handles 2,440 stock modules; 449 of 500 run `init_module`; `8139too`'s `probe()` runs to completion; and `/dev/netdev` lets a process be an interface, with `ping` working over one. L3b done too — `ldlsd` runs the same loader on SchOS and gets a real driver's `probe()` to return. L3c (the fifteen symbols, against `/dev/pci`) open. No Linux driver has yet touched hardware. |
|
||||
| **`ldls.md`** | L0, L1, L2 and L3a done — the loader handles 2,440 stock modules; 449 of 500 run `init_module`; `8139too`'s `probe()` runs to completion; and `/dev/netdev` lets a process be an interface, with `ping` working over one. L3b done too — `ldlsd` runs the same loader on SchOS and gets a real driver's `probe()` to return. L3c done — a Linux driver now drives real hardware through `/dev/pci` and reads the card's own MAC off its EEPROM. L3d, the packet path, is what remains. |
|
||||
| **Schcom v0** | 715 lines that composite, over a wire format that is superseded. Frozen: it runs, nothing more is added, and it is kept only because it is the one thing in the tree that has ever put a client's pixels on screen. |
|
||||
| **`debugger.md`** | D0 through D4 built and described in [../architecture/debugging.md](../architecture/debugging.md). Only the network transport is open — a userspace daemon bridging TCP to `/dev/dbg`, which is deliberately last. |
|
||||
|
||||
|
|
|
|||
|
|
@ -183,7 +183,8 @@ One device, one process, claimed and released, on the `/dev/pci` discipline.
|
|||
| **L2** | **The hardware.** `probe()` receives a working `struct pci_dev`, maps its BARs, allocates coherent DMA and registers an interrupt, all through `/dev/pci`. The driver reaches its device. | **done**, except `/dev/pci` — see below |
|
||||
| **L3a** | **The class bridge.** `/dev/netdev`: a process claims it and becomes an interface. `ping` over a *process*. | **done** — `kernel/dev/netuser.c`, `cmd/netecho` |
|
||||
| **L3b** | The SchOS-side host: `ldls-run`'s loader and BTF work, as a program on SchOS. | **done** — `cmd/ldlsd`; `/dev/pci` is L3c's half |
|
||||
| **L3c** | The fifteen symbols, implemented rather than stubbed, against a real rtl8139. | |
|
||||
| **L3c** | The fifteen symbols, implemented rather than stubbed, against a real rtl8139. | **done** — `ldls/kpi/`; the driver reads the real MAC |
|
||||
| **L3d** | The packet path: `ndo_open`, `ndo_start_xmit`, receive. | |
|
||||
| **L3** | **A packet.** The two joined: **`ping` over a Linux driver.** This is the milestone that proves the whole idea and the one that cannot be skipped. | |
|
||||
| **L4** | **Kill it.** The host process dies, the kernel clears bus mastering and reclaims the DMA, a new host starts and the interface comes back. | free — H2 built all of this |
|
||||
|
||||
|
|
@ -635,6 +636,95 @@ says about the combination, and why it is a thing you type on purpose.
|
|||
|
||||
---
|
||||
|
||||
## L3c: a Linux driver, driving real hardware
|
||||
|
||||
`ldls/kpi/` is the shim: 41 of `8139too`'s 94 symbols, implemented rather than
|
||||
logged. Register access goes to a mapped BAR; `pci_enable_device`,
|
||||
`pci_iomap`, `pci_request_regions` and `pci_set_master` go to `/dev/pci`;
|
||||
allocation goes to `mmap` and `PCIIOC_DMA`; `register_netdev` goes to
|
||||
`/dev/netdev`. Every file records its provenance, per rule 3.
|
||||
|
||||
```
|
||||
# ldlsd /lib/modules/8139too.ko /lib/modules/btf
|
||||
found 10ec:8139 at 00:05.0
|
||||
bar 0 0x000000000000c000 256 io
|
||||
bar 1 0x0000000081060000 256 mem
|
||||
netdev 2688 bytes
|
||||
|
||||
calling probe, with 41 of 94 symbols implemented
|
||||
|
||||
kpi: net_device 2688 bytes + 872 private
|
||||
kpi: mapped BAR 1 -- phys 0x0000000081060000, 256 bytes
|
||||
kpi: the driver read a MAC off the hardware: 52:54:00:ab:cd:ef
|
||||
net: ldls0 attached as interface 1, asking for an address
|
||||
netuser: ldls0 is a process now, 52:54:0:ab:cd:ef
|
||||
kpi: registered as an interface, with the hardware's own MAC
|
||||
|
||||
probe returned 0 after 0 stub call(s)
|
||||
```
|
||||
|
||||
**`52:54:00:ab:cd:ef` is what the machine was configured with.** Nothing here
|
||||
invented it: the driver bit-banged the card's EEPROM, serially, through a BAR
|
||||
mapped by `/dev/pci`, and handed back the address it found. That it matches
|
||||
means the loader, the relocations, the BTF offsets, the BAR mapping and the
|
||||
register accessors are all right at once — which is why it was chosen as the
|
||||
thing to aim at.
|
||||
|
||||
**Zero stub calls.** Every symbol that probe reaches is implemented.
|
||||
|
||||
### The bug a foreign driver found in its first minute
|
||||
|
||||
The first run against real hardware wrote to address `0x4e` and faulted.
|
||||
`0x4e` is 13 below where the driver meant to write, and 13 is `EACCES`:
|
||||
`pci_iomap` had returned `-13` as a pointer, because the check for a failed
|
||||
`mmap` looked only for `-1` while SchOS returns `-errno`.
|
||||
|
||||
Underneath that was the real one. `sys_mmap` rounds a length up to a page —
|
||||
the only granularity an MMU has — and `pciuser_mmio_ok()` compared that against
|
||||
the BAR's own size. rtl8139's registers are **256 bytes**, the request became
|
||||
4096, and the answer was `-EACCES` for a device the process legitimately held.
|
||||
|
||||
**A BAR smaller than a page could never be mapped at all**, and nothing had
|
||||
noticed because the only consumer until now was virtio-gpu, whose BARs are
|
||||
large. The check is now against the pages the BAR lies in. The widening is
|
||||
real and named in the code: a sub-page BAR hands the process the whole page it
|
||||
sits in, which is forced by the hardware rather than chosen.
|
||||
|
||||
That is precisely what [hybrid.md](hybrid.md) said a foreign driver would be
|
||||
good for — *"what it asks for is what a foreign driver would ask for too"* —
|
||||
arriving as a concrete bug rather than as a prediction.
|
||||
|
||||
### Register access is checked
|
||||
|
||||
`kpi_io_ok()` bounds every `ioread`/`iowrite` against the mapping `pci_iomap`
|
||||
returned, reports the first few that miss, and returns without touching
|
||||
memory. Two reasons: a stray access would otherwise be a page fault with no
|
||||
context, and **there is no IOMMU** — a hosted driver's wild writes are exactly
|
||||
the class of thing that has to be caught early or not at all.
|
||||
|
||||
It is also what turned the bug above from a crash into two lines naming the
|
||||
offsets, which is how the `-13` was spotted.
|
||||
|
||||
### What is not done
|
||||
|
||||
`probe()` succeeds, so the driver is up and the interface exists with the
|
||||
hardware's own MAC. **No frame has moved.** `ndo_open` is never called, there
|
||||
is no `ndo_start_xmit`, and nothing services the receive side — so `ldlsd`
|
||||
releases the interface rather than leaving it up:
|
||||
|
||||
```
|
||||
the driver is up on real hardware and has an interface.
|
||||
carrying frames is the half of L3 that is not built, so the
|
||||
interface is released rather than left up and silent.
|
||||
```
|
||||
|
||||
A link that reports itself up with nothing draining its queue is a black hole:
|
||||
the stack would send DHCP into it for ever and every layer above would believe
|
||||
the network was merely quiet. Down is the true answer until packets move, and
|
||||
that is L3d.
|
||||
|
||||
---
|
||||
|
||||
## What it will not do
|
||||
|
||||
- **It is not a security boundary, and here that is worse than usual.** There
|
||||
|
|
@ -705,10 +795,16 @@ says about the combination, and why it is a thing you type on purpose.
|
|||
with the link down, because nothing in lwIP supports taking an interface out
|
||||
of its list. From the stack's point of view that is the same thing; from
|
||||
`/proc/netif`'s it is a `ldls0` that stays listed after its process is gone.
|
||||
- **`probe()` is reached with invented hardware.** The BARs are numbers the
|
||||
harness made up and `pci_iomap` returns a zeroed page. A driver that reads a
|
||||
status register gets whatever was there, which is why 8139too read a MAC
|
||||
address of zeroes and was perfectly happy about it.
|
||||
- **No frame has moved.** L3c ends at a successful `probe()`. The packet path
|
||||
— `ndo_open`, `ndo_start_xmit`, receive, and the skb handling all three need
|
||||
— is L3d and is the larger half.
|
||||
- **41 of 94 symbols.** The rest are still logging stubs, which is deliberate:
|
||||
an unimplemented symbol that reports itself keeps the work queue writing
|
||||
itself, exactly as it did at L1.
|
||||
- **A lock is a no-op.** In one process with no preemption inside the driver
|
||||
there is nobody to race with, which is correct right up until the driver has
|
||||
an interrupt thread. That is the sleeping-context assumption named above and
|
||||
it is the deepest unfixed thing in the layer.
|
||||
- **It will not support "Linux drivers".** It will support *one driver, on one
|
||||
kernel version, that has been tried*. Anything beyond that is a claim, and
|
||||
the list of what has actually been run belongs in the architecture document
|
||||
|
|
|
|||
Loading…
Reference in a new issue