docs: architecture describes what is built now
kernel (threads, demand paging, TLB shootdowns, dynamic linking, RAM direct map), syscalls, filesystem (/proc, /sys, /dev), terminals (12, VT handshake, ptys), networking (r8169 through LDLS), LDLS (module sets; Linux code may ship), devices (what stays in the kernel; hybrid.md's lasting content), boot (the loader's placement of the kernel), graphics (fbdev, GMMOS, QR 1-40), audio (ALSA over OSS), debugging, usb. New: packages.md (the apk root) and desktop.md (Xorg on the framebuffer). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EVRhxi9zpdW2uDUN4FVcao
This commit is contained in:
parent
655c7d8d01
commit
2556a955b3
15 changed files with 921 additions and 341 deletions
|
|
@ -6,7 +6,11 @@ drivers, and LDLS is what does that.
|
|||
|
||||
It works, and on more than one driver. `8139too` and `e1000` — unmodified GPL
|
||||
kernel modules — load into a SchOS process, drive their hardware through
|
||||
`/dev/pci`, and carry packets:
|
||||
`/dev/pci`, and carry packets (both under QEMU). A third, Linux's `r8169`, is
|
||||
the NIC driver on the ASUS TUF A15 laptop, and it is the first that needs a
|
||||
*set* of modules and not one; it is described under *A set of modules*. It has
|
||||
registered its interface and read its PHY on that laptop; whether it carries
|
||||
traffic there has not been confirmed yet.
|
||||
|
||||
```
|
||||
# ldlsd /lib/modules/8139too.ko /lib/modules/btf &
|
||||
|
|
@ -46,18 +50,20 @@ Every one was a call that succeeded, returned zero, and did nothing.
|
|||
|
||||
Five, and everything else is an implementation of them.
|
||||
|
||||
### 1. Nothing GPL ever enters an image we build
|
||||
### 1. Linux code may ship; so far the tree holds none
|
||||
|
||||
The driver is **the user's**: obtained by them, loaded on their machine. SchOS
|
||||
ships the layer and never the thing it loads. `.gitignore` refuses `*.ko`,
|
||||
`make ldls-stage` is the one command that puts a module into a local image and
|
||||
is deliberately not part of `all`, and `make ldls-unstage` takes it out again.
|
||||
This rule used to say nothing GPL would ever enter an image we build. It was a policy,
|
||||
never a licence constraint — SchOS is GPL-2.0-or-later, and GPL-2.0-only code combines
|
||||
with it — and the owner has overridden it: **Linux code and firmware blobs may be
|
||||
shipped.** What survives of the rule is mechanical, not legal. The repository carries no
|
||||
`*.ko` (`.gitignore` refuses them), and `make ldls-stage KO=…` / `make ldls-r8169` stage
|
||||
modules into a *local* image from the Linux kernel on the build machine, because a module,
|
||||
its dependencies and the BTF that describes them have to come from the same Linux build
|
||||
and that kernel is the easiest place to get a matching set.
|
||||
|
||||
[../licensing.md](../licensing.md) §6 is the whole argument. The short form is
|
||||
that the combination happens in a user process on the user's machine, so no
|
||||
combined work is distributed by anyone. Since SchOS moved to GPL-2.0-or-later
|
||||
that is no longer the only way it *could* be done — it is the way it is done,
|
||||
because a module we never ship is a module we never have to answer for.
|
||||
Shipping one is therefore a decision about the release, not about the code: an ISO that
|
||||
contains `r8169.ko` is a distribution of GPL-2.0-only code and carries the usual
|
||||
obligation — its source goes with it — see [../licensing.md](../licensing.md) §6.
|
||||
|
||||
### 2. It runs in a process, never in the kernel
|
||||
|
||||
|
|
@ -186,6 +192,55 @@ the object has to live where a signed 32-bit absolute address reaches. A
|
|||
process's mmap area starts at `0x40000000`, which is inside that window; on the
|
||||
host harness `MAP_32BIT` reaches it from the other end.
|
||||
|
||||
### A set of modules — what r8169 needed
|
||||
|
||||
Linux's `r8169` is not one module. It needs `mdio-bus`, `libphy`, `mdio_devres`,
|
||||
the Realtek PHY driver, the NIC itself, the RTL PHY firmware, and the BTF of the
|
||||
kernel they came from. `make ldls-r8169` stages all of that from the running
|
||||
kernel; `etc/schinit/ldls-r8169.job` starts `ldlsd` on it at boot, with its output
|
||||
in `/var/log/ldls-r8169.log`. The job has `KeepAlive = no`: a driver that crashes
|
||||
on real hardware would otherwise be restarted into the same crash every few seconds.
|
||||
|
||||
What that added to `ldlsd`:
|
||||
|
||||
- **A comma-separated module set**, loaded in dependency order, each linking
|
||||
against the ones before it, with the BTF read before any `init_module` runs.
|
||||
A library module (`libphy`) just exports symbols to the next; the PCI `probe`
|
||||
runs once, at the end, for whichever module registered a `pci_driver`.
|
||||
- **A driver core**: bus match, driver probe, so the PHY driver binds to the PHY
|
||||
the way it does in Linux; **devres**; a kernel `printf`; real `udelay`/`msleep`/
|
||||
`ktime`; and **1000 Hz `jiffies`**, because the module's `HZ` is compiled in and
|
||||
phylib re-arms its link poll `HZ` jiffies out.
|
||||
- **Firmware** read from `/lib/firmware`.
|
||||
- **phylib's data**, the `link_mode_params` table and its bitmaps.
|
||||
- **The symbols whose stub-zero was a lie**: list validity, `try_module_get`,
|
||||
`dev_err_probe`, `eth_platform_get_mac_address`, `napi_consume_skb` (as a stub it
|
||||
leaked one buffer per transmitted frame until the DMA heap ran out), and the
|
||||
rest of the table at the end of this document.
|
||||
- `pdev->bus` is a real `pci_bus`; `net_device.priv_len` is set from BTF (the
|
||||
driver's `memset` of its ring bookkeeping is bounds-checked against it, and
|
||||
`__fortify_panic` fired on a length of zero); and `cmovcc` from per-CPU memory is
|
||||
rewritten like the other `%gs` forms.
|
||||
- **`__preempt_count` starts at `PREEMPT_ENABLED`.** mdio-bus keeps the register
|
||||
it has just read in `r11` across `preempt_enable()`, which on x86 is
|
||||
`decl %gs:__preempt_count; je`, to a call that preserves every register. With the
|
||||
count at zero the call was made, into a logging stub whose syscall left `RFLAGS`
|
||||
(0x246) in `r11` — so every MDIO read returned `0x02460246`. `preempt_schedule` is
|
||||
now a bare `ret`.
|
||||
- `__fortify_panic` names the module and function (`r8169: rtl_open.cold+0x…`) so
|
||||
a photograph of the screen is enough, and exits instead of spinning on a core and
|
||||
holding `/dev/netdev`.
|
||||
|
||||
The kernel did two things for it. It turns **ASPM off on Realtek NIC links** at
|
||||
boot, as `r8169` does through `pci_disable_link_state`, which a process cannot;
|
||||
and a network interface that **appears after boot starts the stack** — with the
|
||||
native driver out of the registry the laptop boots with no interface at all, and
|
||||
`ldls0` arrives later. `ldlsd` says which refusal it got when attaching: `EBUSY` is
|
||||
another process, anything else is the kernel.
|
||||
|
||||
The native `drivers/net/r8169/` stays in the tree, with Linux's 8168H setup, as
|
||||
the fallback; it is out of the registry.
|
||||
|
||||
### The KPI — `ldls/kpi/`
|
||||
|
||||
The Linux symbols, implemented. Register access to a mapped BAR; the PCI calls
|
||||
|
|
@ -232,8 +287,8 @@ and setting both to zero makes the whole thing reversible with no lookup table.
|
|||
|
||||
### The back end — `/dev/pci`
|
||||
|
||||
Already built, for `vgpud`, before any of this existed.
|
||||
[hybrid.md](../plan/hybrid.md) insisted the interface be about *PCI devices*
|
||||
Already built, for `vgpud`, before any of this existed. The design insisted the
|
||||
interface be about *PCI devices*
|
||||
rather than about a driver model, on the argument that an interface whose first
|
||||
consumers are native drivers is a general mechanism rather than a hole shaped
|
||||
like something foreign. A Linux driver's `pci_iomap`, `dma_alloc_coherent` and
|
||||
|
|
@ -241,10 +296,9 @@ like something foreign. A Linux driver's `pci_iomap`, `dma_alloc_coherent` and
|
|||
|
||||
### Giving a device up — `release`
|
||||
|
||||
A driver in the kernel now has a `release` hook, and `/dev/pci` calls it before
|
||||
handing its device to a process. `docs/plan/hybrid.md` had said the honest
|
||||
answer to "may I have the one you are using" is *no* until a driver could be
|
||||
told to let go; e1000 is the first that can.
|
||||
A driver in the kernel has a `release` hook, and `/dev/pci` calls it before
|
||||
handing its device to a process. The honest answer to "may I have the one you are
|
||||
using" was *no* until a driver could be told to let go; e1000 is the first that can.
|
||||
|
||||
Letting go is three things, and the third was missed at first:
|
||||
|
||||
|
|
@ -336,17 +390,17 @@ point of having built it against our own:
|
|||
|
||||
## What it does not do
|
||||
|
||||
- **Two drivers, one kernel version, one machine.** `8139too` and `e1000` on
|
||||
6.18.46 against QEMU. The second driver was worth more than the first: it
|
||||
found the per-CPU problem, the retpoline problem, the port-I/O problem, the
|
||||
interrupt storm and a seven-year-old hole in the TSS. Nothing here has been
|
||||
tried on a second kernel or on real silicon, and "LDLS runs Linux drivers" is
|
||||
still a claim about a category from a sample of two.
|
||||
- **Three drivers, one kernel version.** `8139too` and `e1000` on 6.18.46
|
||||
against QEMU, and `r8169` on that version against QEMU (where it finds no
|
||||
device) and on one laptop (where it gets as far as the PHY). The second driver
|
||||
was worth more than the first: it found the per-CPU problem, the retpoline
|
||||
problem, the port-I/O problem, the interrupt storm and a seven-year-old hole in
|
||||
the TSS. Nothing here has been tried on a second kernel, and "LDLS runs Linux
|
||||
drivers" is still a claim about a category from a sample of three.
|
||||
- **No interrupts**, per the polling note above.
|
||||
- **`jiffies` is not `HZ`.** It comes from a 100 Hz `uptime()`, which is a
|
||||
plausible `HZ` and not necessarily the one the module was built for. A driver
|
||||
measuring a timeout against it gets a longer timeout, which is the safe
|
||||
direction and is still wrong.
|
||||
- **`jiffies` assumes `HZ` is 1000**, which is what the kernels LDLS loads from
|
||||
are built with (`ldls_load.hz` can say otherwise). It comes from the
|
||||
monotonic clock in milliseconds.
|
||||
- **One work item at a time, on the main loop.** A driver whose deferred work
|
||||
blocks stops everything, including the transmit path it is probably waiting
|
||||
for.
|
||||
|
|
@ -363,14 +417,21 @@ point of having built it against our own:
|
|||
process, which is a leak in the strict sense and the right trade for a
|
||||
program that will be killed to restart it.
|
||||
- **BTF has to come from somewhere.** SchOS has no `/sys/kernel/btf/vmlinux`,
|
||||
so the user brings it with the module, from the kernel the module was built
|
||||
for. A kernel built without `CONFIG_DEBUG_INFO_BTF` has neither.
|
||||
so `make ldls-stage` / `ldls-r8169` copy it from the Linux kernel the module
|
||||
came from. A kernel built without `CONFIG_DEBUG_INFO_BTF` has neither.
|
||||
- **There is address-space groundwork nothing uses.** `kernel/arch/x86_64/vm.c`
|
||||
maps the kernel image a second time at `LDLS_KALIAS`, in the top 2 GB where a
|
||||
`-mcmodel=kernel` module can reach, with a `vmemmap` and a module area beside it
|
||||
— at every boot — and `include/ldls/ldls.h` describes a `/dev/ldls` control
|
||||
channel to load modules into the kernel. That was the start of an in-kernel
|
||||
loader, contradicting rule 2. Nothing calls either, and it should be finished
|
||||
as a decision or deleted.
|
||||
- **No config-space writes.** `/dev/pci` has none, deliberately — a write is
|
||||
how a BAR gets moved out from under the kernel — so a driver that needs one
|
||||
is refused loudly rather than quietly succeeded.
|
||||
- **No module-to-module dependencies, no firmware loading, no sysfs, no
|
||||
unload.** `request_firmware` fails, `module_exit` is never called, and the
|
||||
way to stop a driver is to kill the process.
|
||||
- **No sysfs and no unload.** Modules depend on each other and firmware is read
|
||||
from `/lib/firmware`, but `module_exit` is never called, and the way to stop a
|
||||
driver is to kill the process.
|
||||
- **Some symbols are still logging stubs.** Deliberate, per rule 5 — and the
|
||||
single most expensive thing in this layer. **A stub that returns zero is not
|
||||
safe when zero is a meaningful answer**, and the list of ways that bit is
|
||||
|
|
|
|||
Loading…
Reference in a new issue