diff --git a/docs/architecture/audio.md b/docs/architecture/audio.md index 5f49d12..27f11fc 100644 --- a/docs/architecture/audio.md +++ b/docs/architecture/audio.md @@ -40,7 +40,7 @@ until the speaker has caught up*. **It is one stream.** Mixing several programs into one device is a real feature with a real design behind it, and building it before anything had played a single sound would be exactly the mistake [../plan/1.0.md](../plan/1.0.md) -keeps a register for: work that exists and is connected to nothing. The second +keeps a register against: work that exists and is connected to nothing. The second program that wants to play at the same time is what should cause it to be written. @@ -84,6 +84,25 @@ Both register files are **I/O ports**, not MMIO — which is why nothing in the driver maps anything. `bar[n].is_io` is what says so, and a driver that `ioremap`ed these would be mapping a port number as though it were an address. +### Through ALSA, for programs that expect it + +SDL 3 has no OSS backend and SchOS has no ALSA *kernel* side, so sound from SDL games +and anything else that speaks ALSA goes **SDL → alsa-lib → alsa-plugins' OSS plugin → +`/dev/dsp`**. `alsa-lib` and `alsa-plugins` are in the Alpine `world`, and +`/etc/asound.conf` (from `schos-base`) makes the default device a `plug` onto the OSS +device at 16-bit stereo 48 kHz, which is what every SchOS audio driver plays. + +Running that path found two bugs underneath it: + +- **AC'97:** resetting the PCM-out box also clears the descriptor-list address and nothing + set it again, so every stream after the first stop played a "list" at physical address 0 + — silence — and ALSA resets before each stream. If the engine halts at the end of the + queue, the run bit is set again. +- **`/dev/dsp` was missing the ioctls the plugin's ring-and-pointer model uses:** + `SETFRAGMENT`, `GETFMTS`, `GETCAPS`, `SETTRIGGER`, `GETOPTR`, `GETODELAY`, and `poll()`, + writable once a whole fragment fits (ALSA spun at 76% CPU otherwise). `speaker-test` now + plays in real time at 0% CPU, verified in a recording of QEMU's AC'97. + ## How it is tested QEMU's `wav` audio backend writes what the guest plays to a file, so the test diff --git a/docs/architecture/boot.md b/docs/architecture/boot.md index 965e0e3..e66f2c8 100644 --- a/docs/architecture/boot.md +++ b/docs/architecture/boot.md @@ -131,6 +131,42 @@ when the early console is set up. So a low framebuffer is attached exactly as before, and a high one is attached a few dozen instructions later, immediately after `kvmalloc()` — still thousands of lines before anything else happens. +### Where the kernel goes, and how it gets there + +The kernel is linked at physical **16 MB** (`KERNPHYS` in `memlayout.h`, repeated in +`kernel.ld`, which must agree) and is about 34 MB with the ramdisk inside it. From the +old 1 MB it ran across 8 MB, where OVMF keeps ACPI NVS memory that outlives the firmware, +so it was overwritten on every boot, after `ExitBootServices`. Firmware leaves its +leftovers low; above 16 MB is where a machine is most reliably empty. (The 15 MB below +the kernel is not given to the page allocator yet.) + +**Every free page of that range is claimed first.** A segment lying wholly on claimed +pages is read straight home; one that is not is read into a buffer elsewhere and +**copied home after `ExitBootServices`**, when boot-services memory is ours. An HP +all-in-one from 2012 used to freeze at the start of the loading bar because the loader +said "could not reserve" and read the kernel there anyway, over the firmware's own memory +with the firmware still running on it. Memory under the kernel that outlives the firmware +(ACPI NVS, runtime services) is named, and so is the loader or its stack sitting in the +range. Segments are read a megabyte at a time, so the bar moves through the ramdisk and +shows where a machine stopped. + +The copy and the jump themselves run from **one page below the kernel's lowest address**, +allocated as loader code before the firmware exits: a short position-independent routine, +the copy list and a stack, switching to the loader's page tables and its own stack before +it copies anything. Doing the copy from code the firmware had placed wherever it liked put +some of it inside the kernel's range on that machine: a triple fault and a reset. The +routine is `sysv_abi`, because UEFI code is the Microsoft convention and the first try loaded +the framebuffer address into `rsp`. + +**No 1 GB pages, no problem.** Intel's desktop CPUs before Haswell have none, and there the +PS bit in a PDPT entry is reserved: the first access after the switch is a page fault that +cannot be handled either. Both the loader and the kernel check CPUID `0x80000001` EDX bit 26; +without it the low 4 GB is four page directories of 2 MB pages (placed below the kernel, with +the staged segments kept below 4 GB), and the kernel gives each gigabyte of its direct map +and of `ioremap()` a page directory of its own. Neither KVM nor QEMU enforces the reserved +bit, so the fault could not be reproduced here; with the fallback forced on, the kernel +boots at 2 and 8 GB and both CPUs start. + ### The loader jumped out from under itself A machine with a lot of RAM froze at "Starting the kernel" — the splash still @@ -610,7 +646,9 @@ in the one process that runs first. ## What it does not do -- **No BIOS path.** A machine without UEFI does not boot SchOS. +- **No BIOS path.** A machine without UEFI does not boot SchOS. The ISO carries a real EFI + system partition (MBR type `0xEF`, FAT16 by cluster count, the type old firmware reads best), + so a stick written with `dd` or `write.sh` boots on strict firmware as well as on lenient. - **The quiet boot hides a slow one.** With nothing printing between the loader and the greeting, a boot that stalls for ten seconds looks the same as one that does not. F8 is the answer and it has to be pressed *before* the @@ -621,9 +659,11 @@ in the one process that runs first. with its partitions, but nothing mounts it. See [usb.md](usb.md). - **NVMe and AHCI poll for completion**, one command at a time per disk. That is enough to boot and run from them, not to be fast. -- **`make ramroot` is the only way to boot with no data disk.** It links a - filesystem image into the kernel binary, which costs boot time and carries a - licensing constraint — see [../licensing.md](../licensing.md) §4. +- **The live ISO is a ramdisk root.** `make iso` builds the root as a SchFS image, + compresses it ([scff.md](scff.md)) and links it into the kernel; nothing is written to + the machine's disks, and everything written to the root is gone at reboot. There is + no installer, and no kernel that finds a root on a disk by SchFS's GPT type as its + *default*. `make ramroot` is the same without compression. - **The 32-bit multiboot entry is dead code.** It is left in place deliberately: deleting kernel entry code is its own piece of work with its own way of failing silently. diff --git a/docs/architecture/debugging.md b/docs/architecture/debugging.md index 809af56..108c45c 100644 --- a/docs/architecture/debugging.md +++ b/docs/architecture/debugging.md @@ -179,6 +179,10 @@ part of the boot is blank for the part that is hardest to debug any other way. --- +A panic is not the end of the log. GMMOS ([graphics.md](graphics.md)) keeps running +under the report, can show this log on screen or as QR codes a phone can read, and +has a small terminal for looking around a machine whose kernel is dead. + ## Asking a question `/dev/dbg` is a conversation: write a line, read the answer. @@ -493,10 +497,16 @@ something that reports nonsense. - **One conversation at a time.** The answer buffer is global, so two programs asking at once get one answer each in some order — never half of the other's, but not two independent sessions either. -- **No userspace symbolization.** The kernel names kernel symbols. A fault in a - process still reports a bare address, and resolving it is a host-side job. -- **The stack walk stops where the frame pointer stops.** `getcallerpcs()` walks - `rbp` chains, so a trap frame in the middle of a chain ends the walk. +- **No userspace symbolization.** The kernel names kernel symbols. A process + killed by a fault is told which *file and offset* it died in (from + `/proc//maps`' records), which is enough to resolve with the file's own + symbols on the host; a bare address is all there is for anonymous memory. +- **The stack walk stops where the frame pointer stops, and it will not follow a + bad one.** `getstackpcs()` walks `rbp` chains, and each link must go up the same + stack by less than `KSTACKSIZE`: a chain can end at the syscall entry with a *user* + `rbp` that a program uses as an ordinary register (`-1024` in one game), and + dereferencing that under the syscall tracer once panicked the kernel inside + `acquire()`. A trap frame in the middle of a chain ends the walk. - **`sysrq g` costs you log history.** Its output goes through `cprintf`, so sixteen kilobytes of views push the same amount out of a 128 KB ring. - **No tracing.** No static tracepoints, no event ring, no counters. "What diff --git a/docs/architecture/desktop.md b/docs/architecture/desktop.md new file mode 100644 index 0000000..2594c95 --- /dev/null +++ b/docs/architecture/desktop.md @@ -0,0 +1,74 @@ +# The desktop: Xorg on the framebuffer + +The desktop is X11, on the framebuffer: Alpine's `xorg-server` with the `fbdev` video +driver and the `evdev` input driver, Window Maker as the window manager, and `xterm`. +`startx` runs `wmaker`. Nothing here is written by SchOS except the kernel interfaces it +stands on and four small files in `tools/posix/mkaroot.sh`. + +## Why X and not Wayland + +Every Wayland compositor worth running — labwc, and everything built on wlroots — needs +DRM/KMS, libinput, udev and libseat. DRM/KMS is the largest of them by far: a kernel graphics +subsystem, and in practice a great deal taken from Linux. Xorg runs on a plain framebuffer +through `fbdev` and reads input through `evdev`, both interfaces SchOS already had. That is +the shorter road to a desktop by a long way, and it brings the window managers with it. +DRM/KMS is not planned; libinput, udev and libseat are worth having for their own sake and +may come later. + +## What it stands on + +| | | +|---|---| +| **fbdev** | `/dev/fb0` speaks Linux's fbdev ([graphics.md](graphics.md#devfb0-and-how-xorg-draws)) | +| **evdev** | `/dev/input/event0` (every keyboard) and `event1` (every mouse and touchpad) with Linux's queries, LED writes and key/button/relative-axis bits | +| **VTs** | `/dev/tty0`, `VT_SETMODE(VT_PROCESS)`, `KD_GRAPHICS`, the release/acquire handshake ([terminals.md](terminals.md)) | +| **dlopen** | Xorg loads its drivers as modules, so `exec` does dynamic linking ([kernel.md](kernel.md)) | +| **sysfs** | `/sys/bus/pci/devices/*/{boot_vga,enable,graphics/fb0}`, which libpciaccess reads | +| **ptys** | `xterm` runs its shell on a pseudo-terminal | +| **Unix sockets** | the X11 socket, abstract names, `SO_PEERCRED`, `flock` | +| **headers** | `` in the build sysroot, as `ports/linux-headers` | + +`cmd/xready` checks every one of the kernel calls Xorg makes before it draws anything — the +evdev queries, the fbdev calls, the console calls, and the `/sys` files — one at a time, on a +terminal of its own, and puts everything back. The calls were written against what the +programs are documented to do; each step past it found one that was not, which is what running +the real program is for. + +## The four files + +- **`/root/.xserverrc`** starts `X` on the VT `startx` was started from (startx works that + out only when `uname -s` says Linux, and SchOS's says SchIFT) with **`-dumbSched`**. Xorg + normally reads input on a thread of its own; when SchOS had no threads the server crashed + the first time a client disconnected, in `pthread_join` on a thread that was never made, and + `-dumbSched` is Xorg's own switch for input on the main loop. Threads exist now; the switch + is still on, and nobody has run the server without it since. +- **`/root/.xinitrc`** is `exec wmaker`. Without it xinit falls back to twm and an xterm, which + are not in the image. +- **`/etc/X11/xorg.conf.d/10-schos.conf`** names the input devices. Alpine's Xorg asks udev for + them, SchOS has no udev, and X came up with no keyboard and no mouse: the pointer did not + move and nothing could end the session. SchOS has exactly two input nodes whatever the + machine, so a fixed list is the whole answer (`AutoAddDevices` off, Ctrl+Alt+Backspace + enabled). +- **`/etc/asound.conf`** is for sound, not X ([audio.md](audio.md)). + +## Using it + +Twelve virtual terminals, switched with Ctrl+Alt+F1–F12 — on the console and under X +alike, because X owns the keyboard while it runs and switches only with that chord. `startx` on +a terminal runs the desktop there; Ctrl+Alt+Backspace ends the session and gives the terminal +back. Window Maker, `xterm`, `xeyes`, `xclock`, `xcalc`, DejaVu fonts; SDL 2 games +(`lbreakouthd`) and NetSurf, a GTK 3 browser, run in it. + +Touchpads that sit on an I2C bus (most laptops) are driven by `kernel/acpi/i2c_hid.c`, found +through ACPI, and reach X as a mouse: pointer, click and tap. + +## What it does not do + +- **No acceleration.** Software rendering to the framebuffer: no DRM, no glamor, no DRI, no + OpenGL beyond what software gives. GPU drivers are a plan ([../plan/gpu.md](../plan/gpu.md)). +- **No udev.** Input devices are named in a config file, not found; `/sys` has no `devices` + tree and no uevent socket. +- **Touchpads are mice**: no two-finger scroll, no gestures, no palm rejection. +- **No resolution change from inside X.** The mode is the console's; `schmode` changes the + `vgpud` head only, and the loader's menu is where a real panel's mode is chosen. +- **Not a security boundary.** X runs as root on a system that enforces no file permissions. diff --git a/docs/architecture/devices.md b/docs/architecture/devices.md index 63ad2fa..7926450 100644 --- a/docs/architecture/devices.md +++ b/docs/architecture/devices.md @@ -4,7 +4,50 @@ How hardware gets found, claimed and used, and what the rest of the kernel sees of it. Writing a driver is [../guides/adding-a-driver.md](../guides/adding-a-driver.md). -Moving drivers out of the kernel entirely is [../plan/hybrid.md](../plan/hybrid.md). +Drivers that live in a process rather than in the kernel are described below +(*Drivers that are not in the kernel*), and the foreign ones in [ldls.md](ldls.md). + +--- + +## What stays in the kernel, and what may leave + +"Hybrid" is used to mean everything from "a monolith with a nice device model" to +"a microkernel that gave up", so the line is worth drawing exactly. + +**Stays in, permanently:** scheduling, address spaces, the trap and syscall path; +IPC and shared memory; the VFS and the **root** filesystem; IRQ demultiplexing; and +the console and PSOGM. That last one is not a size compromise but an invariant: the +crash screen must work when everything else is dead, so it cannot depend on anything +that could *be* dead. A crash screen served by a userspace process is absent in +exactly the case it exists for. + +**May leave, and should:** GPU drivers, non-root filesystems, NICs after the first, +and anything written by someone else. + +> **Anything needed to reach a shell stays in. Anything discovered at runtime, +> replaceable, or plausibly written by someone else goes out.** +> +> **The kernel keeps a minimal driver for everything on the path to a shell, and +> userspace owns the good one.** Break that and the compositor cannot start because +> the display driver is a process that needs a compositor. + +### USB stays in the kernel, and why + +It was on the list to leave, and the premise was wrong. USB is not a peripheral; it +is the bus peripherals arrive on, structurally the same kind of thing as PCI, and +nobody has proposed evicting PCI. The keyboard and the root disk that arrive over it +are on the path to a shell, so the stack that finds them is part of that path. Even +with a minimal in-kernel fallback written, the good driver and the minimal one would +be the same driver, and an eviction that leaves an identical copy behind has moved +nothing. A second, quieter difference: `/dev/pci` hands over one PCI *function*, one +owner and nothing shared, while a USB device sits behind a controller other devices +also sit behind — "the xHCI controller" includes the keyboard. What the eviction was +for, third-party drivers living outside the kernel, survives and belongs to +[ldls.md](ldls.md). + +The root filesystem stays in the kernel for the same reason (SchFS cannot be +restarted, and a bug in it is fatal), and so does the first NIC, because the network +thread and lwIP are in the kernel. --- @@ -184,8 +227,47 @@ client — and that is worth a sentence here because it is the thing the device model makes possible rather than the thing it does. A modeset is five commands to the device. In the kernel those five commands are also a reason to be nervous, because a wrong one takes the machine; in a process the worst case is -that `schinit` starts another one. See -[../plan/hybrid.md](../plan/hybrid.md), "A modeset, from a process". +that `schinit` starts another one. That is why the in-kernel driver never grew +one — not difficulty, exposure. + +``` +# schmode +1280x800 current +1280x800 maximum, framebuffer 4000 KB +# schmode 1024 768 +1024x768 ok +``` + +The scanout really resizes, and nothing in the kernel changed for it: `schmode` +writes a line to a socket and `vgpud` sends five commands to the device. The +framebuffer is allocated **once, at the largest mode**, and every smaller mode +reuses it, because `PCIIOC_DMA` has no free — a driver's blocks come back when it +exits and not before — so a modeset that allocated would spend the arena and the +failure would look like a GPU bug rather than like counting. A larger mode is +refused, and says why. The control socket speaks lines of text and has no buffer +submission, damage or vblank, which a compositor would need and nothing has asked +for. + +### A panic has to reach the screen + +Handing a device away costs something that was not on the list. The kernel panics, +paints the report into the console framebuffer, and halts — and the display it gave +to a process keeps showing whatever that process last put there. On a machine with +two heads, one says the kernel is dead and the other shows a shell prompt. There is +no scheduling after a panic, so the process holding the device is never going to run +again, and the kernel let go of the device deliberately. + +So the driver says in advance where it is scanning out, with `PCIIOC_PANICFB` +(`include/dev/pciuser.h`), and the kernel: (1) paints the report into that memory; +(2) asks the *kernel* driver the device was taken from to put it on screen — +`panic_present` in `struct pci_driver`; (3) which resets the device and rebuilds one +queue from **static** memory — no allocation, no locks, no sleeping, because a +corrupted heap is one of the ways a machine gets here. The registration is checked +like the promise it is: the range must lie inside a block that process got from +`PCIIOC_DMA` on that device, or it would be a way to nominate any physical address +to be overwritten at panic. It costs 20 KB of static memory in the virtio-gpu driver, +paid whether or not the machine ever panics; allocating it on demand would be +cheaper and would not work. ### Taking it back @@ -222,8 +304,9 @@ enforces it. Coming back is a job, not a kernel feature. `etc/schinit/vgpud.job` carries `KeepAlive = yes`, so killing the driver is the entire test: `schinitd` starts another one, it claims the device, and the pid of whichever process drew the -frame is legible in the frame itself. See -[../plan/hybrid.md](../plan/hybrid.md), "The driver came back". +frame is legible in the frame itself: read off the scanout across four generations of +the driver it said 4, 8, 10, 12, and the DMA arena handed back the same addresses +every time. ### What this is not @@ -231,8 +314,7 @@ frame is legible in the frame itself. See buffer can point the device at any physical address. This isolates a driver against its own *bugs* and not against its own *intent* — a crashed driver that restarts instead of taking the machine down is the entire benefit, and it is -worth having on those terms. [../plan/hybrid.md](../plan/hybrid.md) says so at -length. +worth having on those terms. VT-d/AMD-Vi is its own project. **One device per process.** A driver drives one thing; if you need two, run two. @@ -240,29 +322,24 @@ length. ## Linux drivers -**Decided 2026-08-08.** Linux drivers are not shipped with SchOS and never -will be. The user may install one on their own machine, through a device -manager or equivalent, and SchOS will load it. +[ldls.md](ldls.md) is how SchOS uses them: a Linux `.ko` runs in a process and +reaches the hardware through `/dev/pci`. The licensing position, stated exactly +because it is easy to get wrong in both directions: -The licensing reasoning matters and is worth stating exactly, because it is -easy to get wrong in both directions: +- **Distribution is what triggers the GPL.** SchOS is GPL-2.0-or-later, so shipping a + GPL-2.0-only driver in an image is *lawful* — see [../licensing.md](../licensing.md) §6 — + and the owner has decided it is allowed (code and firmware blobs). The repository does + not carry one all the same: `.gitignore` refuses `*.ko`, and the modules are staged into + a *local* image from the build machine's own Linux kernel (`make ldls-stage`, + `make ldls-r8169`), where the matching BTF is. +- **Implementing a loader is not copying.** Writing support for a module format and + a driver ABI from published documentation produces our own work; the layouts come + from the kernel's BTF at load time on the user's machine, so the source holds none + of Linux's. -- **Distribution is what triggers the GPL.** SchOS is GPL-2.0-or-later, so - shipping a GPL-2.0-only driver in an image is now *possible* — see - [../licensing.md](../licensing.md) §6 — but LDLS still does not, by policy. A - user obtaining a driver themselves and loading it on their own machine is - their act, not ours, and is not a distribution by us. -- **Implementing a loader is not copying.** Writing support for a module format - and a driver ABI from published documentation produces our own work. It is a - clean-room implementation, and it stays one only if it is written from - documentation rather than transcribed from Linux source. Reading Linux to - understand *hardware* is fine; translating its code is not, whatever the - identifiers say. - -So the rule for this subsystem is the same as everywhere else in the tree: the -loader is ours and GPL-2.0-or-later, what it loads is the user's business, and -no third-party module enters an image we build. `RAMDISK_EXCLUDE` and `NOTPORTS` are the -existing mechanisms for keeping that true mechanically rather than by intention. +The loader is ours and GPL-2.0-or-later; what it loads is not in the tree. +`RAMDISK_EXCLUDE` and `NOTPORTS` are the mechanisms that keep that true mechanically +rather than by intention. --- @@ -371,12 +448,13 @@ the console's minors to the console and the rest to their own functions. But - **No hot-plug.** Devices are registered at boot and never removed; the `in_use` flag is the hook a hot-unplug path would need, and nothing sets it for that purpose. -- **`NDEVICE` is 32**, compile-time, like every other table in the kernel. +- **`NDEVICE` is 128**, compile-time, like every other table in the kernel. - **In-kernel drivers are fixed at link time.** There is no module loader, and nothing can be added to the kernel after the build. A driver *outside* the - kernel is an ordinary program and can be started whenever — but it reaches its - device through `/dev/pci` and publishes nothing to the class vtables, so the - console cannot be handed to one yet. + kernel is an ordinary program and can be started whenever — it reaches its + device through `/dev/pci`, and a NIC driver can publish itself to the network + class through `/dev/netdev` (LDLS does); a display driver publishes nothing to + `display_ops`, so the console cannot be handed to one. - **The kernel does not rebind its own driver.** `PCIIOC_RELEASE`, or the process exiting, leaves the interrupt line masked and the in-kernel driver unbound — the console stays wherever the release put it. A *supervisor* can start a diff --git a/docs/architecture/filesystem.md b/docs/architecture/filesystem.md index 4ebf020..bde8768 100644 --- a/docs/architecture/filesystem.md +++ b/docs/architecture/filesystem.md @@ -199,11 +199,14 @@ provides — reads and writes never reach it, because `fileread`/`filewrite` see It is read-only: `dirlink` and `ialloc` answer `EROFS` rather than pretending. A node exists because the kernel has the device, or it does not exist. -### What is there +### What is there in `/dev` -`console`, `null`, `keyboard`, `zero`, `full`, `random`, `urandom`, `tty`, -`stdin`, `stdout`, `stderr`, `fb0`, `mouse`, `hda`, `hdb`, and `input/event0` -and `input/event1` in one subdirectory. +`console`, `null`, `zero`, `full`, `random`, `urandom`, `tty`, `tty0` and +`tty1`..`tty12`, `stdin`, `stdout`, `stderr`, `keyboard`, `mouse`, `fb0`, `dsp`, +`ptmx` and a `pts/` directory of pseudo-terminal slaves, `hda`, `hdb`, +`input/event0`.., and the SchOS-specific `pci` (a device's registers, DMA and +interrupts, for a driver in a process), `netdev` (a network interface for a +process), `dbg` and `dbgro` (the kernel debugger's views). Every disk and partition the block layer registered is there too, under the name it was given (`sata0`, `sata0p2`, `nvme0p1`, `usb0p1`): major 8, minor its @@ -218,11 +221,18 @@ nothing behind it fails with `ENXIO`, which is a better answer than a missing path — and with no node at all, `echo x > /dev/hdb` silently creates an ordinary file of that name and appears to work. -### What is there +### What is there in `/proc` -Per-process directories, plus the global files: `meminfo`, `uptime`, `cpuinfo`, -`version`, `stat`, `loadavg`, `mounts`, `filesystems`, `schfs`, `pci`, -`interrupts`, `devices`, `netif`, and `sysrq`. +`/proc/self`, and one directory per process holding `stat`, `status`, `cmdline`, +`statm`, `comm`, `schedstat`, `maps`, `exe` (a symlink), `fd/` (a symlink per open +descriptor) and `task/` (one directory per thread). `maps` lists the file +mappings, and a process killed by a fault is told which file and offset it died +in. + +The global files: `meminfo`, `uptime`, `cpuinfo`, `version`, `stat`, `loadavg`, +`mounts`, `filesystems`, `partitions`, `schfs`, `pci`, `interrupts`, `devices`, +`netif`, `resolv.conf`, `slab`, `fsstat`, `kmsg`, `strace`, `bootmode`, +`bootdebug`, `quiet`, `verbose`, `schinit` and `sysrq`. **/proc is read-only except `sysrq`**, which is the one deliberate write target, and the code says so in one place. Marking every procfs inode read-only once @@ -249,6 +259,8 @@ mounted by init with `mount("sysfs", "/sys", "sysfs", 0, 0)`. | `/sys/firmware/memmap/N/` | `start end type`, the firmware's map as the loader handed it over | | `/sys/firmware/efi/` | present when UEFI booted the kernel; empty | | `/sys/class/dmi/id/` | `sys_vendor product_name product_version board_name bios_version`, and the rest Linux has, except serials and the UUID | +| `/sys/class/power_supply/` | `BAT0` (`capacity`, `status`, `energy_*`/`charge_*`, `voltage_now`, `cycle_count`, ...) and `AC` (`online`), from ACPI — the battery and the adapter are read through the laptop's embedded controller | +| `/sys/bus/pci/devices/…/` | also `boot_vga`, `enable` and `graphics/fb0`, which libpciaccess (and so Xorg) reads | **Linux's layout because everything that reads hardware reads it from there.** toybox's `lspci` and `lsusb` walk `/sys/bus/*/devices` and parse each `uevent` @@ -265,7 +277,8 @@ tables, `config`) report their real length, since tools copy them by it. What is missing, compared with Linux: `/sys/devices` and its symlinks (the `bus/*/devices` entries are directories rather than links into it), -`/sys/class` beyond `dmi`, `/sys/block`, and anything writable. PCI `config` is +`/sys/class` beyond `dmi` and `power_supply`, `/sys/block`, a uevent socket (so +no udev), and anything writable. PCI `config` is 256 bytes, because config space is read through port `0xCF8`, which reaches no further. @@ -376,8 +389,6 @@ through. - **No page cache.** `mmap` of a file reads the whole thing in at map time, and writable `MAP_SHARED` of a file is refused rather than faked. This is the single largest gap in the layer. -- **No symlinks in procfs**, which is why `/proc//exe` is a regular file - and `ps -o comm` prints `[init]`. - **`rename()` does not move directories**, and will not replace one. `EISDIR` either way. - **No named pipes.** `mkfifo()` fails with `EPERM`, Linux's answer for a @@ -385,19 +396,13 @@ through. device, because `mknod()` read a Linux program's `(mode, dev)` as the native `(major, minor)`: `ls -l` showed `crw-rw-rw- 438, 0` where a `p` belongs. The zsh port was configured with `zsh_cv_sys_fifo=yes` on the strength of that. -- **No `/dev/fd`, and no `/proc/self`.** A process cannot reach its own open - descriptors by path. `<(cmd)` needs one or the other — zsh now knows it has - neither and says so. -- **No `AF_UNIX` socket inodes.** Socket names live in a table in - `kernel/net/unix.c`, so `bind()` creates nothing that can be `stat()`ed or - `unlink()`ed. +- **No `/sys/devices` and no uevents**, so nothing can be told that a device + appeared; Xorg is given its input devices by a config file instead. - **No unified inode cache across filesystems** beyond the single `itable`. -- **A known, un-root-caused hang:** heavy `ls -lR` use across many commands in - one long shell session can eventually make a *later*, unrelated directory - hang; the identical command succeeds on a fresh boot. It is not inode or file - table exhaustion — both fail cleanly and neither fired. It smells like a lock - or refcount leak in the `*at()` paths or in toybox's own fd handling. The next - step is instrumenting `itable`/`ftable` occupancy across a long session to see - what is climbing. - **`sys_delete`/`sys_show`/`sys_showstat`** are orphaned entries from the old v1 filesystem's recycle bin. Nothing reaches them and they die with `fs.c`. +- **One long-standing hang was never found.** Heavy `ls -lR` use across many + commands in one long shell session once made a *later*, unrelated directory + hang; the identical command succeeded on a fresh boot. It was instrumented and + hunted and could not be reproduced, which is recorded in + [../history/kernel.md](../history/kernel.md) rather than forgotten. diff --git a/docs/architecture/graphics.md b/docs/architecture/graphics.md index 58b0396..4db9f44 100644 --- a/docs/architecture/graphics.md +++ b/docs/architecture/graphics.md @@ -1,8 +1,8 @@ # Graphics, text and the console -Four things share this area and they are deliberately separate: the kernel's own -drawing surface, the compositor's display interface, the text-rendering layer, -and the crash screen. +Five things share this area and they are deliberately separate: the kernel's own +drawing surface, the `display` class a display server could use, the text-rendering +layer, the crash screen — and `/dev/fb0`, which is how Xorg reaches the screen. --- @@ -13,11 +13,11 @@ first thing here. | | [`gfx.h`](../../include/gfx.h) | [`display.h`](../../include/display.h) | |---|---|---| -| **What** | The kernel's own drawing surface | The compositor's interface | +| **What** | The kernel's own drawing surface | A frame interface for a display server | | **About** | Pixels and text | *Frames* | | **Operations** | `draw_pixel`, `fill_rect`, `putc`, `text_at` | `acquire`, `present`, `wait_vblank` | | **Buffering** | None. Direct writes to the scanout. | Double-buffered | -| **Users** | The console and PSOGM | Schcom | +| **Users** | The console, PSOGM, GMMOS | None today — Xorg draws through `/dev/fb0` | | **Must work when** | **Nothing else in the system does** | Userspace is healthy | That last row is the reason they are not one interface. The console and the @@ -67,7 +67,7 @@ and `vgpud` mirroring it see what is on the screen. `struct display_ops`, found with `device_find_class(DEVCLASS_DISPLAY)`. `acquire()` allocates the back buffer **on first call, never at boot** — it is -around two megabytes at 800×600, and a machine whose compositor never starts +around two megabytes at 800×600, and a machine whose display server never starts should not pay that. A driver that cannot double-buffer returns 0, and the caller draws directly and accepts tearing rather than not drawing at all. `wait_vblank` is optional for the same reason. @@ -133,7 +133,7 @@ A machine with both a stdvga and a virtio-gpu has two displays, and `vgpud` — the out-of-kernel driver, `cmd/vgpud/main.c` — owns the second one. For a long time it drew a test card there: colour bands, gradients, a checkerboard, and its own pid in binary along the bottom. That was scaffolding for -[../plan/hybrid.md](../plan/hybrid.md)'s H2 and H3, where a screenshot had to +the out-of-kernel driver work ([devices.md](devices.md)), where a screenshot had to answer *did a userspace process put this here* and *did the driver come back*. Both of those are closed, so the test card became the Shift mark. Which was @@ -223,9 +223,8 @@ shows the top-left corner, which is where a terminal puts everything worth seeing. Scaling is the alternative and it is the wrong one: text resampled to a non-integer ratio is text you cannot read. -**This is not the endpoint.** A compositor (Gate 3) will want this head for -itself, and when there is one, the mirror goes away. Until then the honest -thing for a working display to show is the system running on it. +A display server that wants the head for itself can have it; until one does, the +honest thing for a working display to show is the system running on it. ### The F8 resolution is the head's resolution @@ -263,6 +262,23 @@ not apply: two `vgpud`s cannot coexist, because the first holds the PCI claim and the second is refused the device long before it reaches `bind()`. The claim is the mutual exclusion; the socket node never was. +## /dev/fb0, and how Xorg draws + +Xorg runs on the framebuffer through its `fbdev` driver, so `/dev/fb0` speaks Linux's +fbdev: `FBIOGET_FSCREENINFO`, `FBIOGET_VSCREENINFO`, `FBIOPUT_VSCREENINFO` (accepted +when it asks for the mode already in force), `FBIOPAN_DISPLAY`, `FBIOBLANK`, +`FBIOPUTCMAP`, and `mmap` of the physical framebuffer. `FBIOGET_VSCREENINFO` reports +nominal timings, as Linux's `efifb` does, instead of zeros — a zero pixel clock made +fbdev compute a 0 Hz RandR mode. Pixel layout comes from the same table the kernel +draws with, so a colour a program builds is the colour the kernel would. + +Once a program maps the framebuffer the kernel has no way to know when it draws, so +`fb_mapped()` marks the screen as no longer the text console's and `fbclose()` puts it +back. A terminal switched to `KD_GRAPHICS` is not repainted ([terminals.md](terminals.md)), +and the `VT_PROCESS` handshake tells the server before it loses the screen. `cmd/xready` +checks every one of these calls before Xorg is started; the picture itself is the proof +beyond that. + ## Text [`include/gfx/text.h`](../../include/text.h) + `kernel/dev/text.c`. @@ -537,20 +553,59 @@ channel that works when the machine will not come back at all. machine whose network stack is dead**, and a phone camera is an out-of-band channel needing no driver on our side. -It encodes the **report itself**, not a URL — it reaches nothing. Deliberately -small in three ways: byte mode only, EC level L only (the screen is a clean, -evenly lit, high-contrast source; the damage QR's stronger levels protect -against does not apply), and versions 1–6 only, stopping short of version 7's -extra version-information blocks and version 10's wider count field. +It encodes the **report itself**, not a URL — it reaches nothing. Byte mode only +and EC level L only (the screen is a clean, evenly lit, high-contrast source; the +damage QR's stronger levels protect against does not apply), but **every version, +1 to 40**: version 40 at level L holds 2,953 bytes, which is what lets GMMOS put the +kernel log on screen as a series of codes. Checked against the `segno` library module +for module at every version. -134 bytes is a large crash report. If a payload ever needs more, the honest fix -is to shorten the report. +It draws on the report unconditionally rather than on a keypress, which is better: +it costs nothing, and requiring input would mean the one case where input is least +trustworthy is also the case where the report is hardest to retrieve. Verified by +`make crashshot && make crashqr`, which decodes the symbol out of the captured +framebuffer. -It draws unconditionally rather than on a keypress, which is better: it costs -nothing, and requiring input would mean the one case where input is least -trustworthy is also the case where the report is hardest to retrieve. Verified -by `make crashshot && make crashqr`, which decodes the symbol out of the -captured framebuffer. +### Every processor stops + +A panic used to stop only the processor it happened on; the rest carried on with a +dead kernel, drew over the report, and kept the fans loud until somebody held the power +button. The panicking processor now sends every other one an **NMI**, which arrives even +with interrupts off, and `trap()` halts a processor that takes one while `panic_stop` is +set — first thing, before the canary check, because that stack may be what broke. NMI has +its own per-CPU stack (IST 1): for a few instructions at the top of `syscall_entry` the +live stack is the user's, at ring 0, and a fault pushing the frame there would be a double +fault on the same stack and a reset instead of a report. A second processor that panics +while one is reporting halts. + +## GMMOS — the Guru Meditation Micro OS + +The report used to be the last thing the kernel did: a picture, blinking keyboard lights, +and a machine that needed its power button. GMMOS keeps the report and goes on living +under it (`kernel/gmmos/`, about 2,000 lines). The real kernel stays dead — nothing of it is +called again that takes a lock, sleeps, allocates or waits for an interrupt. GMMOS has its +own: the framebuffer, a clock read from the TSC, and keyboards it polls itself through +**micro drivers**, cut-down copies of the real ones that do only what a keyboard needs. + +| Key | | +|---|---| +| (nothing) | a status line counts down **three minutes** to power-off; any key cancels it | +| **Space** | the last 300 lines of this boot's log as **QR codes**, a page per symbol, numbered so a phone can take them in order; `+` / `-` trade module size for capacity | +| **L** | the log on screen, scrolled with the arrows | +| **T** | a terminal: `why`, `log`, `grep`, `ps`, `cpus`, `pci`, `mem` (page tables walked first, so a bad address cannot fault), `uptime`, `restart`, `poweroff` | +| **R R** / **P P** | restart / power off — twice, so a stray key does neither | + +The micro drivers are **PS/2**, **I2C-HID** (the laptop's own keyboard, over a DesignWare +controller — the live driver hands over what it found at boot, because AML cannot run on a +dead kernel) and **xHCI** (USB keyboards: every controller is halted, reset and enumerated +from nothing, root ports and USB 2 hubs up to two deep, a boot-protocol keyboard, no +interrupts, a static 256 KB pool). GMMOS probes all of them and polls every one that answers. + +Every buffer is static. A panic inside GMMOS, or a second one in the report, is reported +and then only blinks; GMMOS does not re-enter itself. Checked under QEMU: every screen, two +QR pages read off a screenshot with `zbar` and decoded to the log text exactly, restart +reboots, and left alone it powers off at three minutes. What has been seen on real hardware +is the report and the keyboards the laptops have; the xHCI path is the least exercised. ## One palette @@ -571,16 +626,20 @@ makes it mean something. ## What it does not do -- **The console still has its cell grid.** Cells are now a Unicode codepoint - drawn by the TrueType rasteriser rather than a CP437 byte drawn from a bitmap, - and the cell size follows the font's measured advance — but it is still a - fixed grid, and the text layer's proportional path is unused by it. -- **No mode setting.** Resolution is whatever the loader asked for. There is no - `/dev/dri`-style device and no way to change it at runtime. -- **No double buffering in practice.** `display_ops.acquire` is specified and - the compositor needs it; virtio-gpu binds but exposes no modeset API, and +- **The console still has its cell grid.** Cells are a Unicode codepoint drawn by the + TrueType rasteriser, and the cell size follows the font's measured advance (the font + scales to about 45 rows, 16 to 40 px per em, on whatever mode the firmware booted + in) — but it is a fixed grid, and the text layer's proportional path is unused by it. +- **Mode setting is the loader's and `vgpud`'s.** On real hardware resolution is whatever + the loader chose from its menu — which now asks "Keep this resolution?" and falls back + by itself, because AMD's GOP "sets" smaller modes without leaving the panel's native + timing and the screen showed three or four squashed copies side by side. There is no + `/dev/dri`, no KMS, and Xorg runs at the mode the console is in. +- **No double buffering in practice.** `display_ops.acquire` is specified and nothing + uses it; virtio-gpu exposes a modeset API on `vgpud`'s socket and nothing more, and `wait_vblank` is unimplemented everywhere. -- **No hardware acceleration of any kind**, and none is planned before 2D blit. +- **No hardware acceleration of any kind**: Xorg draws in software to the framebuffer + (no glamor, no DRI), and GPU drivers are a plan, not a feature ([../plan/gpu.md](../plan/gpu.md)). - **No CFF/Type 2 charstrings**, so OTF fonts do not render — that is a separate `font_ops` provider and a bytecode interpreter, not an extension of the `glyf` path. TrueType hinting is deliberately out of scope. @@ -593,6 +652,6 @@ makes it mean something. - **No composite point-matching.** Components positioned by matching a point in the base rather than by an offset are placed at the origin. Fonts that use it are rare and none in the tree do. -- **PSOGM renders and stops.** The interactive crash mode described in - [../plan/roadmap.md](../plan/roadmap.md) — walk the stack, read another - process's memory, dump page tables — is designed and not built. +- **GMMOS cannot look at a live process by name.** The "interactive crash mode" is built as far + as a dead machine allows — log, a terminal, `ps`, `cpus`, `pci`, memory by physical walk — but + the scheduler is gone, so there is no running a debugger against a process. diff --git a/docs/architecture/kernel.md b/docs/architecture/kernel.md index 28c722e..80ced5b 100644 --- a/docs/architecture/kernel.md +++ b/docs/architecture/kernel.md @@ -11,18 +11,27 @@ driver or a protocol stack: memory, processes, scheduling, traps and signals. One page table per process; the kernel is mapped into all of them at `KERNBASE = 0xFFFF800000000000`, so a trap does not switch `CR3`. -`kvmalloc()` maps physical 0–4 GB as four 1 GB pages, with the top one marked -uncached — that is where PCI BARs and the framebuffer live. This is why 3 GB is -the most memory the kernel can treat as RAM, and it is a deliberate simplicity: -a direct map with a fixed offset makes `P2V`/`V2P` arithmetic rather than a -lookup. +`kvmalloc()` maps as many 1 GB pages of physical memory as the machine has RAM +(up to `PHYSTOP_MAX`, 256 GB), always at least the low four. The 3–4 GB slot stays +uncached because that is where the PCI hole is; everything above 4 GB is RAM, so +those are cached, and a device BAR up there still gets its own uncached window +from `ioremap()`, which takes the entries the direct map did not. On a CPU +without 1 GB pages (Intel's desktop parts before Haswell), each gigabyte is a +page directory of 2 MB pages instead — the same reach, one extra page of table +per gigabyte. A direct map with a fixed offset makes `P2V`/`V2P` arithmetic +rather than a lookup. -`kalloc()` hands out one 4 KB page at a time from a free list. **There is no -slab or pool allocator**, and the consequence is visible throughout the tree: -every subsystem that needs many small objects has invented its own fixed-size -table — `NPROC` processes, `NSOCK` sockets, `NDEVICE` devices, `NSHM` segments. -That is why the limits are compile-time constants rather than a design decision -about each one. +How much RAM there is comes from the firmware's E820/UEFI map, as a *list* of +usable regions, not one run ending at a top address (OVMF drops an ACPI NVS +region at 8 MB, which used to hide everything above it). The kernel itself is +linked at 16 MB, clear of what firmware keeps low. `kalloc` holds never-used +memory as ranges, so a large machine costs nothing until its memory is used. + +`kalloc()` hands out one 4 KB page at a time from a free list. Objects smaller +than a page come from the slab caches below; objects larger than one do not have +a home yet, so the biggest tables (`NPROC` processes, `NSOCK` sockets, `NDEVICE` +devices, `NSHM` segments) are still fixed-size arrays, which is why their limits +are compile-time constants rather than a design decision about each one. The free list carries a canary, checked at the top of both `kalloc()` and `kfree()`, so a wild write into it panics at the point it is touched rather than @@ -150,8 +159,8 @@ pages, reference counted, mapped through a VMA with `no_free` set so that `memfd_create(2)` is the same machinery reached by **descriptor** instead of by key. That difference is the whole point and the reason it exists on Linux at all: two processes share memory by passing the fd over a socket, so neither has -to agree on a path or race to invent a key. It is what a Wayland buffer crosses -on, and it is why descriptor passing (`SCM_RIGHTS`) had to be built first. +to agree on a path or race to invent a key. It is how a display server and its client +share a buffer, and it is why descriptor passing (`SCM_RIGHTS`) had to be built first. | | | |---|---| @@ -171,10 +180,46 @@ version of something this kernel cannot do: mapper. Here a freed page would simply stay mapped to memory somebody else now owns. -Writable `MAP_SHARED` of a **real file** is still `ENODEV`. That needs a page -cache, it is a separate project, and nothing in 1.0 asks for it — an anonymous -object has no such contract to break, which is exactly why `memfd` is the -narrow answer. +Writable `MAP_SHARED` of a **real file** is still refused. That needs a page +cache, it is a separate project, and nothing that runs today asks for it — an +anonymous object has no such contract to break, which is exactly why `memfd` is +the narrow answer. + +### Demand paging + +Anonymous memory can be recorded without being backed: the page appears, +zeroed, the first time it is touched — by the program, through `trap()`, or by +the kernel on its behalf, through `uva2ka()`. `mmap` does this for mappings of +1 MB or more, for `MAP_NORESERVE`, and for `PROT_NONE`. The reason is WebKit, +which reserves 4 GB of address space at start for its JIT and heaps and touches +a sliver of it; backing all of it at `mmap` time failed with `ENOMEM` and the +browser aborted. Go, Java and Firefox reserve the same way. + +`vm_fault()` takes a spinlock and nothing that sleeps, so `uva2ka()` may call it +from anywhere; the lock keeps two threads of one process from both mapping the +same page. `fork` and `munmap` skip the holes, so a page nobody touched is one +they never see, and a child that touches it reads zeroes — what it would have +read in the parent. + +File-backed mappings are **not** lazy: `mmap` of a file still reads the range in +at map time. + +### TLB shootdowns + +Unmapping from an address space that threads on other processors are running in +interrupts those processors and waits until each has reloaded `CR3` +(`tlb_shootdown()` in `vm.c`; the vector is one of the MSI lines, which `trap()` +already dispatches with an EOI). Without it, when one thread unmapped a range and +`mmap` handed the same addresses straight back, a thread on another CPU went on +reading and writing the *old* pages through its TLB: stores vanished and loads +were stale. musl's allocator noticed first — a group's metadata no longer +pointing back at it — and Epiphany, forty threads on four processors, died at a +different address on every run. + +Only one shootdown runs at a time, and it is taken without disabling interrupts: +two processors each waiting with interrupts off for the other to answer would +wait for ever. On a single processor, or if no vector could be allocated, it +does nothing (and says so at boot). ### What the fixed tables actually cost @@ -184,31 +229,69 @@ partly true: | | size | table | | |---|---|---|---| -| `struct proc` | 11744 | `NPROC` 128 | larger than a page; 1 KB of it is FPU state | -| `struct socket` | 4936 | `NSOCK` 128 | larger than a page | +| `struct proc` | 1984 | `NPROC` 128 | 254 KB in all; the address space, descriptors and signal handlers live in their own objects now, shared by threads | +| `struct mm` | 8232 | one per process, 128 | 1 MB in all, almost all of it the 256-entry VMA table | +| `struct socket` | about 4900 | `NSOCK` 128 | larger than a page | | `struct device` | 120 | `NDEVICE` 128 | 15 KB, never hit | | `struct buf` | 200 | `NBUF` 256 to start | grows from a slab, data from `kalloc`, up to RAM/32 | -| `struct inode` | 296 | none | grows from a slab; `NINODE` 200 is only how many are made up front | -| `struct file` | 64 | none | grows from a slab; `NFILE` 512 is only the static pool | +| `struct inode` | about 460 | none | grows from a slab; `NINODE` 200 is only how many are made up front | +| `struct file` | about 88 | none | grows from a slab; `NFILE` 512 is only the static pool | -The two largest cannot go in a page-backed slab at all. Freeing them needs a -contiguous multi-page allocator, which is a separate thing and is not written. -The buffer header shrank to 200 bytes when its 4 KB of data moved out to a page -of its own, which is what let the cache grow at all. +(Sizes from `nm -S` on a build of the tree, not from a header comment.) + +`struct proc` shrank by a factor of six when threads arrived, because the page +tables, VMAs, descriptor table and signal dispositions moved out of it into +objects a thread group shares. `struct socket` is still too large for a +page-backed slab; freeing it needs a contiguous multi-page allocator, which is a +separate thing and is not written. The buffer header shrank to 200 bytes when its +4 KB of data moved out to a page of its own, which is what let the cache grow at +all. ## Processes -`struct proc` in [`include/core/proc.h`](../../include/proc.h). A fixed table of -`NPROC` entries, allocated by linear scan. +`struct proc` in [`include/core/proc.h`](../../include/core/proc.h). A fixed table of +`NPROC` entries, allocated by linear scan. A thread is a `struct proc` too. **The field order at the top of the struct is fixed by assembly.** `trapasm.S` hard-codes `fs_base` at offset `0x18`, and there is a `_Static_assert` in `proc.c` that fails the build if it moves. Anything new goes at the end. -Each process carries: page directory and size, kernel stack, open file table, -cwd inode *and* cwd path string, signal handlers and masks, VMA table for -`mmap`, POSIX identity (pgid, sid, and all six of real/effective/saved uid and -gid), and the Linux-ABI syscall map. +Each process carries: a kernel stack, cwd inode *and* cwd path string, signal +masks, POSIX identity (pgid, sid, and all six of real/effective/saved uid and +gid), the Linux-ABI syscall map, and its x87/SSE registers (saved on a switch; +the kernel itself stays off them). Three things it *points to* rather than +owns, because a thread group shares them: + +| | | +|---|---| +| `struct mm` | the address space: page tables, heap top, the VMA table, which file each span came from, and a lock so one thread at a time changes the mappings | +| `struct files` | the descriptor table and its `FD_CLOEXEC` flags | +| `struct sighand` | signal dispositions — what `sigaction()` sets | + +Each is reference counted. `clone()` shares what its flags say and `fork()` +copies all three. + +### Threads + +`clone()` takes `pthread_create`'s flags (`CLONE_VM`, `FILES`, `SIGHAND`, +`THREAD`, `SETTLS`, `PARENT_SETTID`, `CHILD_CLEARTID`) and `posix_spawn`'s +(`VM`, `VFORK`, a stack). What follows from that: + +- **futex** — `WAIT`, `WAIT_BITSET` with timeouts, `WAKE`, `REQUEUE`, + `CMP_REQUEUE`, keyed by the word's *kernel* address so processes that map the + same page meet. `set_tid_address` keeps the word that exit zeroes and wakes, + which is what `pthread_join` waits on. +- **Exit.** `exit` ends the calling thread; `exit_group`, a fatal signal, or the + leader exiting ends them all. `reapd` frees exited threads, and the leader is + reaped by its parent once the rest are gone. `getpid` is the group, `gettid` + the thread. +- **Signals.** Stops and resumes reach every thread; other process-directed + signals go to the leader. +- **`exec`** kills the other threads and unshares the dispositions. +- **`/proc`** lists processes, `task/` lists threads; `comm`, `Threads:` and + `Tgid` are there. + +The first program to need all of this was GTK's, then WebKit's. ### Slot reuse @@ -232,12 +315,15 @@ exactly as a syscall would. The network thread is the current user. ## Scheduling -Round-robin over the process table, one scheduler per CPU, each with its own -context. `swtch()` saves callee-saved registers and switches stacks; `sched()` -returns to the scheduler and asserts four invariants on the way — holding -`ptable.lock`, `ncli == 1`, not `RUNNING`, interrupts off. +Fair share by virtual runtime, one scheduler per CPU, each with its own context +— the rule, the guards and `nice` are in [scheduler.md](scheduler.md). +`swtch()` saves callee-saved registers and switches stacks; `sched()` returns to +the scheduler and asserts four invariants on the way — holding `ptable.lock`, +`ncli == 1`, not `RUNNING`, interrupts off. -When a full pass finds nothing runnable the CPU halts rather than spinning. +When nothing is runnable the CPU goes idle rather than spinning: into ACPI's +`_CST` C2 state (CC6 on Zen), by port read or `MWAIT`, where the APIC timer keeps +running through it, and otherwise a plain `hlt`. **Locking is a single `ptable.lock` over the whole table.** Not per-process: a scheduling operation touches two processes (the one leaving and the one @@ -309,9 +395,23 @@ mksh's job control could not make progress until they existed. ## What userspace gets -`fork`, `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and real pid selection, -pipes, signals, `mmap` (anonymous and file-backed), POSIX shared memory, Unix -domain sockets, `poll`/`select`/`epoll`, `eventfd`, `timerfd`, `signalfd`. +`fork`, `clone` (threads), `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and +real pid selection, pipes, signals, `mmap` (anonymous and file-backed), POSIX +shared memory, `memfd`, Unix domain sockets, `poll`/`select`/`epoll`, `eventfd`, +`timerfd`, `signalfd`, `inotify`, `futex`. + +### Dynamic linking + +`exec` handles `ET_EXEC` and `ET_DYN`. A dynamically linked program names an +interpreter in `PT_INTERP` — on musl, `/lib/ld-musl-x86_64.so.1`, which is libc +under another name — and the kernel's whole part is to load that as well and +start it instead of the program. The interpreter finds the program through the +auxiliary vector (`AT_PHDR`, `AT_ENTRY`), maps libc and whatever else it needs +with `mmap()`, relocates, and jumps to `AT_ENTRY` itself. A position-independent +program goes where a fixed one would (`0x400000`); the interpreter goes at the +start of the `mmap` area (`0x40000000`) and `mmap` carries on above it, so the +two never meet. This is what lets Alpine's binaries run unmodified, and +`dlopen` works, so Xorg loads its drivers as modules. `mmap` uses a bump allocator (`proc->mmap_next`, starting at `0x40000000`) so every mapping gets its own disjoint, exactly-sized range. It previously assigned @@ -329,24 +429,23 @@ allocation over 8 MB mapped straight into a neighbouring live mapping. loop reports 97%, `sleep 2` reports 0% — but kernel time needs TSC deltas closed and reopened around every `sched()`, which is scheduler accounting rather than sampling, and is not written. - -- **No slab allocator.** One page at a time, and every subsystem's fixed-size - table is a consequence. This is the single change that would most improve the - rest of the kernel. -- **No demand paging and no page cache.** `mmap` of a file *reads the whole - thing in* at map time. Writable `MAP_SHARED` of a file is therefore refused - outright rather than faked — see [../plan/foundations.md](../plan/foundations.md). -- **No dynamic linking.** `exec` handles `ET_EXEC` only: no `PT_INTERP`, no - `ET_DYN`, no shared libraries, no `dlopen`. -- **No threads.** `clone` exists but does not share an address space; - `gettid` returns the pid. +- **`struct socket` is too big for a slab**, and every subsystem's fixed-size + table (`NPROC`, `NSOCK`, `NDEVICE`, `NSHM`) is still a compile-time constant. + Slabs exist for the objects that fit in a page. +- **No page cache.** `mmap` of a file *reads the range in* at map time, and + writable `MAP_SHARED` of a file is refused outright rather than faked. Demand + paging covers anonymous memory only. - **Permissions are tracked but not enforced.** uid/gid are maintained honestly - so `getuid()`/`setuid()` stop lying, and nothing consults them on access. + so `getuid()`/`setuid()` stop lying, and file access does not consult them; + `setpriority` is the one place the uids are checked. - **No swap**, and no memory pressure handling of any kind. `kalloc` failing is reported up as `-ENOMEM` and that is the whole policy. - **Nested signal delivery is not supported.** `in_signal` guards against re-entering a handler while one is running. -- **Scheduling is fair-share on one global run queue.** Priorities and - fairness accounting exist now ([scheduler.md](scheduler.md)); per-CPU run - queues, work stealing and CPU affinity do not, despite `sched_setaffinity` - existing. +- **One global run queue.** Fair share and `nice` exist; per-CPU run queues, work + stealing and CPU affinity do not, and `sched_setaffinity` is accepted and + ignored. +- **No IOMMU**, so DMA-capable devices (and the driver processes that program + them) are trusted. +- **256 GB is the most RAM** the kernel uses; the rest is ignored, with a line + saying so. diff --git a/docs/architecture/ldls.md b/docs/architecture/ldls.md index 6be30a0..1d34830 100644 --- a/docs/architecture/ldls.md +++ b/docs/architecture/ldls.md @@ -6,7 +6,11 @@ drivers, and LDLS is what does that. It works, and on more than one driver. `8139too` and `e1000` — unmodified GPL kernel modules — load into a SchOS process, drive their hardware through -`/dev/pci`, and carry packets: +`/dev/pci`, and carry packets (both under QEMU). A third, Linux's `r8169`, is +the NIC driver on the ASUS TUF A15 laptop, and it is the first that needs a +*set* of modules and not one; it is described under *A set of modules*. It has +registered its interface and read its PHY on that laptop; whether it carries +traffic there has not been confirmed yet. ``` # ldlsd /lib/modules/8139too.ko /lib/modules/btf & @@ -46,18 +50,20 @@ Every one was a call that succeeded, returned zero, and did nothing. Five, and everything else is an implementation of them. -### 1. Nothing GPL ever enters an image we build +### 1. Linux code may ship; so far the tree holds none -The driver is **the user's**: obtained by them, loaded on their machine. SchOS -ships the layer and never the thing it loads. `.gitignore` refuses `*.ko`, -`make ldls-stage` is the one command that puts a module into a local image and -is deliberately not part of `all`, and `make ldls-unstage` takes it out again. +This rule used to say nothing GPL would ever enter an image we build. It was a policy, +never a licence constraint — SchOS is GPL-2.0-or-later, and GPL-2.0-only code combines +with it — and the owner has overridden it: **Linux code and firmware blobs may be +shipped.** What survives of the rule is mechanical, not legal. The repository carries no +`*.ko` (`.gitignore` refuses them), and `make ldls-stage KO=…` / `make ldls-r8169` stage +modules into a *local* image from the Linux kernel on the build machine, because a module, +its dependencies and the BTF that describes them have to come from the same Linux build +and that kernel is the easiest place to get a matching set. -[../licensing.md](../licensing.md) §6 is the whole argument. The short form is -that the combination happens in a user process on the user's machine, so no -combined work is distributed by anyone. Since SchOS moved to GPL-2.0-or-later -that is no longer the only way it *could* be done — it is the way it is done, -because a module we never ship is a module we never have to answer for. +Shipping one is therefore a decision about the release, not about the code: an ISO that +contains `r8169.ko` is a distribution of GPL-2.0-only code and carries the usual +obligation — its source goes with it — see [../licensing.md](../licensing.md) §6. ### 2. It runs in a process, never in the kernel @@ -186,6 +192,55 @@ the object has to live where a signed 32-bit absolute address reaches. A process's mmap area starts at `0x40000000`, which is inside that window; on the host harness `MAP_32BIT` reaches it from the other end. +### A set of modules — what r8169 needed + +Linux's `r8169` is not one module. It needs `mdio-bus`, `libphy`, `mdio_devres`, +the Realtek PHY driver, the NIC itself, the RTL PHY firmware, and the BTF of the +kernel they came from. `make ldls-r8169` stages all of that from the running +kernel; `etc/schinit/ldls-r8169.job` starts `ldlsd` on it at boot, with its output +in `/var/log/ldls-r8169.log`. The job has `KeepAlive = no`: a driver that crashes +on real hardware would otherwise be restarted into the same crash every few seconds. + +What that added to `ldlsd`: + +- **A comma-separated module set**, loaded in dependency order, each linking + against the ones before it, with the BTF read before any `init_module` runs. + A library module (`libphy`) just exports symbols to the next; the PCI `probe` + runs once, at the end, for whichever module registered a `pci_driver`. +- **A driver core**: bus match, driver probe, so the PHY driver binds to the PHY + the way it does in Linux; **devres**; a kernel `printf`; real `udelay`/`msleep`/ + `ktime`; and **1000 Hz `jiffies`**, because the module's `HZ` is compiled in and + phylib re-arms its link poll `HZ` jiffies out. +- **Firmware** read from `/lib/firmware`. +- **phylib's data**, the `link_mode_params` table and its bitmaps. +- **The symbols whose stub-zero was a lie**: list validity, `try_module_get`, + `dev_err_probe`, `eth_platform_get_mac_address`, `napi_consume_skb` (as a stub it + leaked one buffer per transmitted frame until the DMA heap ran out), and the + rest of the table at the end of this document. +- `pdev->bus` is a real `pci_bus`; `net_device.priv_len` is set from BTF (the + driver's `memset` of its ring bookkeeping is bounds-checked against it, and + `__fortify_panic` fired on a length of zero); and `cmovcc` from per-CPU memory is + rewritten like the other `%gs` forms. +- **`__preempt_count` starts at `PREEMPT_ENABLED`.** mdio-bus keeps the register + it has just read in `r11` across `preempt_enable()`, which on x86 is + `decl %gs:__preempt_count; je`, to a call that preserves every register. With the + count at zero the call was made, into a logging stub whose syscall left `RFLAGS` + (0x246) in `r11` — so every MDIO read returned `0x02460246`. `preempt_schedule` is + now a bare `ret`. +- `__fortify_panic` names the module and function (`r8169: rtl_open.cold+0x…`) so + a photograph of the screen is enough, and exits instead of spinning on a core and + holding `/dev/netdev`. + +The kernel did two things for it. It turns **ASPM off on Realtek NIC links** at +boot, as `r8169` does through `pci_disable_link_state`, which a process cannot; +and a network interface that **appears after boot starts the stack** — with the +native driver out of the registry the laptop boots with no interface at all, and +`ldls0` arrives later. `ldlsd` says which refusal it got when attaching: `EBUSY` is +another process, anything else is the kernel. + +The native `drivers/net/r8169/` stays in the tree, with Linux's 8168H setup, as +the fallback; it is out of the registry. + ### The KPI — `ldls/kpi/` The Linux symbols, implemented. Register access to a mapped BAR; the PCI calls @@ -232,8 +287,8 @@ and setting both to zero makes the whole thing reversible with no lookup table. ### The back end — `/dev/pci` -Already built, for `vgpud`, before any of this existed. -[hybrid.md](../plan/hybrid.md) insisted the interface be about *PCI devices* +Already built, for `vgpud`, before any of this existed. The design insisted the +interface be about *PCI devices* rather than about a driver model, on the argument that an interface whose first consumers are native drivers is a general mechanism rather than a hole shaped like something foreign. A Linux driver's `pci_iomap`, `dma_alloc_coherent` and @@ -241,10 +296,9 @@ like something foreign. A Linux driver's `pci_iomap`, `dma_alloc_coherent` and ### Giving a device up — `release` -A driver in the kernel now has a `release` hook, and `/dev/pci` calls it before -handing its device to a process. `docs/plan/hybrid.md` had said the honest -answer to "may I have the one you are using" is *no* until a driver could be -told to let go; e1000 is the first that can. +A driver in the kernel has a `release` hook, and `/dev/pci` calls it before +handing its device to a process. The honest answer to "may I have the one you are +using" was *no* until a driver could be told to let go; e1000 is the first that can. Letting go is three things, and the third was missed at first: @@ -336,17 +390,17 @@ point of having built it against our own: ## What it does not do -- **Two drivers, one kernel version, one machine.** `8139too` and `e1000` on - 6.18.46 against QEMU. The second driver was worth more than the first: it - found the per-CPU problem, the retpoline problem, the port-I/O problem, the - interrupt storm and a seven-year-old hole in the TSS. Nothing here has been - tried on a second kernel or on real silicon, and "LDLS runs Linux drivers" is - still a claim about a category from a sample of two. +- **Three drivers, one kernel version.** `8139too` and `e1000` on 6.18.46 + against QEMU, and `r8169` on that version against QEMU (where it finds no + device) and on one laptop (where it gets as far as the PHY). The second driver + was worth more than the first: it found the per-CPU problem, the retpoline + problem, the port-I/O problem, the interrupt storm and a seven-year-old hole in + the TSS. Nothing here has been tried on a second kernel, and "LDLS runs Linux + drivers" is still a claim about a category from a sample of three. - **No interrupts**, per the polling note above. -- **`jiffies` is not `HZ`.** It comes from a 100 Hz `uptime()`, which is a - plausible `HZ` and not necessarily the one the module was built for. A driver - measuring a timeout against it gets a longer timeout, which is the safe - direction and is still wrong. +- **`jiffies` assumes `HZ` is 1000**, which is what the kernels LDLS loads from + are built with (`ldls_load.hz` can say otherwise). It comes from the + monotonic clock in milliseconds. - **One work item at a time, on the main loop.** A driver whose deferred work blocks stops everything, including the transmit path it is probably waiting for. @@ -363,14 +417,21 @@ point of having built it against our own: process, which is a leak in the strict sense and the right trade for a program that will be killed to restart it. - **BTF has to come from somewhere.** SchOS has no `/sys/kernel/btf/vmlinux`, - so the user brings it with the module, from the kernel the module was built - for. A kernel built without `CONFIG_DEBUG_INFO_BTF` has neither. + so `make ldls-stage` / `ldls-r8169` copy it from the Linux kernel the module + came from. A kernel built without `CONFIG_DEBUG_INFO_BTF` has neither. +- **There is address-space groundwork nothing uses.** `kernel/arch/x86_64/vm.c` + maps the kernel image a second time at `LDLS_KALIAS`, in the top 2 GB where a + `-mcmodel=kernel` module can reach, with a `vmemmap` and a module area beside it + — at every boot — and `include/ldls/ldls.h` describes a `/dev/ldls` control + channel to load modules into the kernel. That was the start of an in-kernel + loader, contradicting rule 2. Nothing calls either, and it should be finished + as a decision or deleted. - **No config-space writes.** `/dev/pci` has none, deliberately — a write is how a BAR gets moved out from under the kernel — so a driver that needs one is refused loudly rather than quietly succeeded. -- **No module-to-module dependencies, no firmware loading, no sysfs, no - unload.** `request_firmware` fails, `module_exit` is never called, and the - way to stop a driver is to kill the process. +- **No sysfs and no unload.** Modules depend on each other and firmware is read + from `/lib/firmware`, but `module_exit` is never called, and the way to stop a + driver is to kill the process. - **Some symbols are still logging stubs.** Deliberate, per rule 5 — and the single most expensive thing in this layer. **A stub that returns zero is not safe when zero is a meaningful answer**, and the list of ways that bit is diff --git a/docs/architecture/networking.md b/docs/architecture/networking.md index 84de7ba..8ed53b3 100644 --- a/docs/architecture/networking.md +++ b/docs/architecture/networking.md @@ -1,7 +1,9 @@ # Networking ARP, IPv4, ICMP, UDP, TCP and DHCP, over a driver-agnostic interface class, -reached through the same socket syscalls as Unix domain sockets. +reached through the same socket syscalls as Unix domain sockets — plus DNS +(resolved in the process, by musl), HTTPS (`wget`, Alpine's TLS libraries) and a +small netlink responder for `getifaddrs`. Verified end to end rather than by inspection: `bootcheck -Net` starts a listener and a UDP responder on the host, tells the guest to talk to them, and @@ -25,7 +27,9 @@ that completes a handshake and drops every byte. └────────────────┬───────────────────┘ │ struct netdev_ops ┌────────────────┴───────────────────┐ - │ e1000, and any future NIC driver │ + │ e1000 rtl8139 r8169 — or a │ in the kernel + │ Linux driver in a process, which │ (LDLS, via /dev/netdev) + │ is a netdev like any other │ └────────────────────────────────────┘ ``` @@ -86,50 +90,55 @@ covers two decades of silicon — and why the driver prints the chip's XID on probe, since the PCI device id tells you almost nothing about which part arrived. -### Why it is native, when the plan said LDLS +### Native first, then Linux's own -The first plan was to run Linux's `r8169.ko` under [ldls.md](ldls.md). Measuring -it is what changed the answer: the module has **183 undefined symbols, 130 of -them not implemented**, and 35 of those 130 are **phylib** — -`phy_connect_direct`, `phy_start`, `mdiobus_read`, `genphy_soft_reset`. +The first plan was to run Linux's `r8169.ko` under [ldls.md](ldls.md), and +measuring it changed that: the module has **183 undefined symbols, 130 of them not +implemented**, and 35 of those 130 are **phylib** — `phy_connect_direct`, +`phy_start`, `mdiobus_read`, `genphy_soft_reset`. phylib is a subsystem with a link +state machine, not a set of leaf calls, and LDLS's usual answer for an +unimplemented symbol — a stub that reports itself and returns zero — is precisely +wrong for it: `phy_start` returning zero means the link never comes up and nothing +says why. So a **native driver** came first: it reaches the PHY through the +chip's own MII window (`PHYAR`, one register that is a whole MII bus) and skips +the subsystem, in the shape of the e1000 beside it. -phylib is a subsystem with a link state machine, not a set of leaf calls, and -LDLS's usual answer for an unimplemented symbol — a stub that reports itself -and returns zero — is precisely wrong for it. `phy_start` returning zero means -the link never comes up and there is nothing on screen to say why. The 8139too -that ldls.md uses as its model has no phylib dependency, which is why that one -was tractable. +It has since been done the other way as well. LDLS grew a driver core, devres, +firmware loading and phylib's tables ([ldls.md](ldls.md), *A set of modules*), +and the ASUS TUF A15's RTL8168H is now driven by **Linux's r8169, unmodified, in +a process**: `make ldls-r8169` stages it with libphy, mdio-bus and the PHY driver, +and `etc/schinit/ldls-r8169.job` starts it at boot. The native driver is **out of +the registry** — it would claim the device first — and stays in the tree, with +Linux's 8168H setup, as a fallback. -A native driver reaches the PHY through the chip's own MII window (`PHYAR`, -one register that is a whole MII bus) and skips the subsystem entirely. It is -less work than the port, it carries no GPL entanglement, it needs no firmware -blob, and it is not coupled to one Linux version. - -The register layout is a *fact about the hardware*, taken the way -[ldls.md](ldls.md) §3 says facts may be taken. The expression is this tree's: -a descriptor-ring driver in the same shape as the e1000 beside it. +On that laptop the interface is `ldls0`, and it appears *after* boot, which is why +`net_attach_late()` starts the stack when the first interface arrives instead of +turning it away. ### The Chromebox has one of these -The Asus Chromebox CN62 "Guado" turns out to carry an RTL8111 rather than the -Intel part its spec sheet suggests, so this driver covers it as well as the -laptop that prompted it — which is the argument for the family match in one -line rather than a device id per board. +The Asus Chromebox CN62 "Guado" carries an RTL8111 rather than the Intel part its +spec sheet suggests, so the family match (`10EC:8168`) covers it as well as the +laptop — which is the argument for matching a family in one line rather than a +device id per board. -### It has not run on hardware +### It has not been shown to work on hardware -**Stated plainly because it cannot be tested here.** QEMU emulates `rtl8139` -and nothing else from Realtek, and the 8139 is a different programming model — -a circular receive buffer rather than descriptors — so it is deliberately *not* -in the match table. Claiming it would break a NIC that works. +**Stated plainly because it cannot be tested here.** QEMU emulates `rtl8139` and +nothing else from Realtek, and the 8139 is a different programming model — a +circular receive buffer rather than descriptors — so it is deliberately *not* in +the match table. Claiming it would break a NIC that works. -What has been verified: the driver builds, its presence changes nothing on a -machine without the device, and with an `rtl8139` on the bus it correctly -leaves it unclaimed (`10ec:8139 network controller [-]`). +What has been verified: both drivers build, their presence changes nothing on a +machine without the device, and Linux's r8169 under LDLS loads, links and +initialises in QEMU, registering and finding no device. On the laptop it reached +probe and read the PHY (after a bug in LDLS that made every MDIO read return +`0x02460246` was fixed); the native 8168H bring-up — the PLL, the receive gate, the +PHY through the OCP window — has not run on a part. -What has not: every register write, on a real part. +What has not: traffic on real hardware, from either. -So the probe says everything a first bug report would need, with `cprintf` +The native probe says everything a first bug report would need, with `cprintf` rather than `kinfo` so it reaches the screen on a machine with no serial port: ``` @@ -141,6 +150,9 @@ register 1, where bit 2 is link and bit 5 is autonegotiation complete. Those four facts separate "the chip is not there", "the chip is there and the PHY sees nothing", and "everything is up and the rings are wrong". +The kernel turns **ASPM off** on Realtek NIC links at boot, as Linux's driver +does through `pci_disable_link_state` — which a process under LDLS cannot do. + ### Two constants that were wrong first Recorded because both would have worked, quietly: @@ -262,6 +274,12 @@ to. Measured across three pings: `rx_frames` and `wake_rx` both went 2 → 6. table, and it returns immediately: everything happens in that thread, which cannot run until the scheduler is entered. +**The thread follows each device's link into lwIP.** The link used to be read once +at attach, 0.02 s into boot; real autonegotiation takes seconds, so on hardware +the interface stayed down for good. It also starts the stack when an interface +appears after boot (a driver in a process, a USB NIC), and attaches every device +there is as its first act; `net_eth_attach()` refuses one it already has. + DHCP obtains an address at boot. `/proc/netif` reports each interface's address, netmask, gateway, MAC and frame counts, taken at the lwIP seam. @@ -326,19 +344,20 @@ The creation lives in `sys_bind()` rather than here, because that is the layer with both: `kernel/net/unix.c` knows about sockets and has no business calling `create()`. -**`flock()` is still missing.** libwayland locks `wayland-0.lock` beside the -socket so that two compositors cannot claim one display. The lock file can be -created and opened now that paths are real; the advisory lock itself is a -separate mechanism with no other caller yet. +`flock()` and `fcntl` record locks exist (`kernel/fs/lock.c`), so the lock files +next to a socket work the way they do on Linux. Names in +the **abstract namespace** (a `sun_path` starting with a NUL) name no file; they +work, and `getsockname` reports them unnamed. `SO_PASSCRED` is honoured, and +`SO_PEERCRED` answers (tmux asks). ### Descriptor passing `sendmsg`/`recvmsg` with `SCM_RIGHTS`, which is the one thing a socket can carry that a pipe cannot: a reference to an object the kernel already knows, rather than a copy of its contents. Wayland's `wl_shm_create_pool` passes a descriptor -and has no integer-shaped alternative, and -[hybrid.md](../plan/hybrid.md)'s mechanism 1 is how a driver process is handed -the device it is allowed to touch. Both wanted the same primitive. +and has no integer-shaped alternative, and the same mechanism is how a driver +process is handed the device it is allowed to touch ([devices.md](devices.md)). +Both wanted the same primitive. **A batch of descriptors is pinned to the stream position of the first byte of the message that carried it.** That single rule is the whole delivery model, and @@ -374,9 +393,9 @@ descriptors to a position that never arrives. **There is no garbage collector.** An AF_UNIX socket descriptor, sent over that same socket and never received, holds a reference to the socket that holds the reference to it, and neither close can reach zero. Linux carries several hundred -lines to break these cycles. Nothing on either plan passes a socket over a -socket — Wayland passes memory, the driver work passes devices — so this is a -known and bounded leak (thirty-two sockets) rather than a silent one. +lines to break these cycles. Nothing passes a socket over a +socket — display servers pass memory, the driver work passes devices — so this is +a known and bounded leak (thirty-two sockets) rather than a silent one. ### AF_INET @@ -593,59 +612,47 @@ information was there and the code did not ask. ## What it does not do -- **`ping` is a first-party program**, `cmd/ping`, not toybox's. `recvmsg` now - exists, so half of what stopped toybox's is gone; what remains is - `IP_RECVTTL`, an `IPPROTO_IP` control message that would have to be generated - on the receive path rather than merely carried. Ours has no `-I`, no flood - mode and no TTL display. -- **`SCM_RIGHTS` is the only control message.** `SCM_CREDENTIALS` is refused - with `EINVAL` rather than ignored: a caller told its credentials were passed - would have the receiver read a uid out of uninitialised memory and act on it. +- **`ping` is a first-party program**, `cmd/ping`, not toybox's or busybox's. Ours + has no `-I`, no flood mode and no TTL display (`IP_RECVTTL` is an `IPPROTO_IP` + control message that would have to be generated on the receive path), and takes a + dotted quad or `localhost` — it does not link musl's resolver. Busybox's `ping`, + from the Alpine root, works over a `SOCK_RAW` ICMP socket. +- **Only `SCM_RIGHTS` and `SCM_CREDENTIALS` travel in a control message.** Anything + else is refused rather than ignored. - **A message may carry eight descriptors, and a socket eight such messages.** Past that `sendmsg` answers `ENOBUFS`. The cap exists because the kernel holds a reference to each one until it is received; a sender eight unreceived messages ahead of its receiver is not rescued by a ninth slot. -- **`ping` still takes a dotted quad**, even though names now work. It is a - first-party program that does not link musl's resolver, and `localhost` is - special-cased in it rather than resolved. Every *ported* program resolves - names properly; ours is the one that does not. -- **Nothing re-runs `resolvconf`.** The nameserver is written once at boot. A - lease that changes the DNS server, or a `/dev/pci` handover that brings up a - new interface, leaves a stale `/etc/resolv.conf` until somebody runs it again. -- **lwIP's own DNS client is still unused.** It is compiled in and only - `dns_getserver()` is ever called; resolution happens entirely in the process. - Nothing in the kernel can resolve a name, which matters the day something in - the kernel wants to. -- **`httpd` does not serve HTTP/1.1.** Every response is HTTP/1.0 with - `Connection: close`, because the directory listing is generated straight into - the socket and so has no `Content-Length` to give. One request per connection - is the cost. +- **Nothing re-runs `resolvconf`.** The nameserver is written once at boot + (`etc/schinit/resolvconf.job`). A lease that changes the DNS server, or a + `/dev/pci` handover that brings up a new interface, leaves a stale + `/etc/resolv.conf` until somebody runs it again. +- **lwIP's own DNS client is unused.** Resolution happens entirely in the process + (musl); nothing in the kernel can resolve a name. +- **`httpd` serves HTTP/1.0** with `Connection: close`, one request per connection, + and has no timeouts: a client that connects and says nothing ties up a child + process until it goes away, and `SO_RCVTIMEO` does not exist here. - **A queued-but-never-accepted connection is closed without clearing its callbacks.** `inet_close()` clears the argument and closes, which is enough - because the pcb is freed — but it is a narrower guarantee than the one the - main path makes, and it relies on `tcp_close()` succeeding. -- **`httpd` has no timeouts.** A client that connects and says nothing ties up - a child process until it goes away; `SO_RCVTIMEO` does not exist here (see - below), so there is nothing to set. -- **No routing table**, and no `/proc/net/route` to report one. With two - interfaces up, the first is the default route and the second is reachable only - by an address on its own subnet. That is a rule rather than an accident, and it - is not routing. The missing *file* is the more immediately visible half: it is - how a program asks which interface carries the default route, and without it - anything that filters on "the default one" sees nothing. -- **Netlink serves two message types.** No `RTM_GETROUTE`, no `RTM_NEWROUTE`, no - event stream, no multicast groups. A program that expects to *subscribe* to - link changes gets a socket that answers dumps and never speaks again. -- **No `getsockname`/`getpeername`**, and `accept()` does not report the peer's - address for an AF_INET socket. A server that wants to log who connected cannot. -- **`setsockopt` covers six options**, and refuses the rest with `ENOPROTOOPT` - rather than pretending: `SO_REUSEADDR`, `SO_KEEPALIVE`, `SO_BROADCAST` and - `TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` are accepted and - report the real size on read. Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`. + because the pcb is freed, but it is a narrower guarantee than the main path + makes and relies on `tcp_close()` succeeding. +- **No routing table**, and no `/proc/net/route`. With two interfaces up, the first + is the default route and the second is reachable only by an address on its own + subnet. That is a rule rather than an accident, and it is not routing. +- **Netlink is a responder**: `RTM_GETLINK` and `RTM_GETADDR` dumps (what + `getifaddrs` needs), `MSG_PEEK|MSG_TRUNC` sizing (GLib's network monitor measures + each datagram that way and spun at 100% when it answered 0), and nothing else. No + `RTM_GETROUTE`, no event stream, no multicast groups; a program that expects to + *subscribe* to link changes gets a socket that answers dumps and never speaks + again. +- **`setsockopt` applies what it understands** (`SO_REUSEADDR`, `SO_KEEPALIVE`, + `SO_BROADCAST`, `TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` report the + real size; `SO_PASSCRED` on Unix sockets) and refuses the rest with `ENOPROTOOPT`. + Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`. - **`SOCK_DGRAM` over AF_UNIX does not exist** — stream only. -- **AF_UNIX names are not filesystem paths**, so nothing can `stat()` or - `unlink()` a socket. -- **One NIC driver exists.** The class is proven with two e1000 cards, not with - two different drivers. -- **No IPv6**, no DNS resolver in the kernel (lwIP's is compiled but unused by - anything), and no firewall or packet filter of any kind. +- **No IPv6**, no firewall or packet filter of any kind, and a TCP stack that has + been used for downloads at tens of megabits and no more than that. +- **Three drivers in the kernel, one Linux driver outside it**, and the class is + proven with two e1000 cards and an LDLS handover, not on a wide range of + hardware. The Realtek parts have never carried a packet on real silicon from this + tree. diff --git a/docs/architecture/packages.md b/docs/architecture/packages.md new file mode 100644 index 0000000..69f52aa --- /dev/null +++ b/docs/architecture/packages.md @@ -0,0 +1,102 @@ +# Packages: an apk root + +Alpine is musl with a different init, and SchOS is musl with a different kernel. +Alpine's binaries run on SchOS unmodified — its dynamic linker, its busybox, and +`apk` itself, fetching over TLS through SchOS's network stack. So instead of +porting programs one at a time, SchOS installs them. + +## The model + +The root filesystem is an apk-managed root: + +| Part | Where it comes from | +|---|---| +| musl, busybox, apk-tools, alpine-keys, CA certificates | Alpine's minirootfs (`ports/alpine`) | +| what `ports/alpine/world` lists (zsh, Xorg, Window Maker, htop, NetSurf …) | Alpine's packages, pinned in `world.lock` | +| schinit, its jobs, SchOS's tools, toybox, mksh, configuration | **schos-base**, an apk package the build makes from `sysroot/` | + +`apk add` installs anything from Alpine's main and community repositories. +`apk upgrade` updates Alpine's packages, and schos-base from the SchOS repository +at `/var/lib/schos/repo`. + +PID 1 is still schinit. Alpine's OpenRC is not installed, and packages that ship +OpenRC services install them where nothing reads them. + +## Who owns what + +Where Alpine and SchOS both have a program, Alpine's wins, and SchOS keeps only +what is its own (`tools/posix/mkaroot.sh` says so in its header): + +- `/bin/sh`, `cat`, `sed`, `find` and the rest are busybox. Alpine's package scripts + are written against busybox. +- zsh is Alpine's. SchOS keeps its `zshrc` and `zshenv` (`replaces=zsh`). +- `/etc/os-release` and `/etc/motd` are SchOS's (`replaces=`). +- `passwd`, `group` and `profile` are Alpine's. schos-base's post-install sets root's + shell to zsh, and SchOS's profile is `/etc/profile.d/schos.sh`. +- `pci.ids` and `usb.ids` are Alpine's (`hwdata-pci`, `hwdata-usb`): Xorg depends on + `hwdata-pci`, and two owners of one file is an error apk will not install past. +- mksh is `/bin/mksh`. schinit's recovery shell looks for it first because it is static: + a broken musl cannot take recovery down with it. + +## Building + + make qemu boots build/schfs.img, the root as a 1 GB SchFS disk + make iso the same root as the live system's compressed ramdisk + make alpine-lock re-resolve world against Alpine (needs the network) + +It is the only root; `sysroot/` is what schos-base is packaged from. Only +`alpine-lock` asks Alpine what is current. A normal build installs exactly what +`world.lock` names — files in `dist/alpine/`, fetched once and checked against the +lock's sha256, and against Alpine's signatures by apk — and never asks what is newer. +`world.lock` resolves the handful of names in `world` into a few hundred files, every +dependency included, so every image after the first is reproducible. + +schos-base and its repository index are signed with a key made on the first build +(`build/apk-keys/`, never committed). The root trusts it beside Alpine's keys. The +install runs as an ordinary user with `--no-scripts`, because apk's scripts want a +chroot; `mkaroot.sh` does by hand the one change schos-base's post-install would +have made. + +## What it took from the kernel + +All of it found by running apk, and none of it specific to apk: + +- **TCP receive window:** it reopened before the reader drained, so lwIP refused data + and the sender waited on retransmission timers (5 KB/s). It reopens on read now, with a + 32-segment window; downloads run at tens of megabits. +- **`mmap` address reuse:** freed ranges were never reused, so musl's allocator used up the + 256 mapping records with 780 MB of RAM free. +- **`unlinkat` / `renameat`** returned a bare `-1`, read as `EPERM`, where apk expects + `ENOENT`; `renameat` could not replace its target. +- **`#!` scripts in `execve`**, and `ENOEXEC`: apk runs triggers directly. +- **`chroot`**, and mounting devtmpfs and proc more than once. +- Later, as bigger programs arrived: dynamic linking, threads and `futex`, demand paging, + pseudo-terminals, `inotify`, `flock`, Unix-socket credentials. Each is in + [kernel.md](kernel.md) or [syscalls.md](syscalls.md) because a package asked for it. + +## The live system + +The live ISO runs the apk root from RAM. The ramdisk is the compressed image +([scff.md](scff.md)); at boot the volume grows to half of the free memory (4 GB of root on +an 8 GB machine), backed by pages as they are written. Xorg and its dependencies are in the +image already; anything else installs into that space in about a minute, over the network, +and **is gone at reboot**. + +## What it does not do + +- **No installer.** Nothing is written to the machine's disks, so packages do not survive + a reboot. Keeping them needs an installer that writes the root to a SATA or NVMe disk, + and a kernel that finds its root there by the SchFS GPT type rather than as IDE disk 1. +- **Not every Alpine package runs.** The ones in `world` were chosen *because* they run: + terminal programs that draw and poll (htop, btop, tmux, mc), X clients, SDL 2 games, + NetSurf on GTK 3. Others hit whatever the kernel does not do yet; Epiphany (WebKit) is + one — its web-process child dies at start-up, and it has not been run to ground. + A package is one `apk add` away, and finding what it needs is how most of the kernel's + Linux surface got written. +- **Fortified libc is strict, and Alpine's packages are not always correct.** `lbreakout2` + hands `vsnprintf` a size larger than its buffer and traps on Alpine too; `world` carries + `lbreakouthd` instead. +- **apk's scripts and triggers do not run at build time** (`--no-scripts`), so a package + whose script matters has to have it done by hand in `mkaroot.sh`. +- **OpenRC services do nothing.** Anything that expects an init script to start it has to + be given a schinit job ([init.md](init.md)). diff --git a/docs/architecture/syscalls.md b/docs/architecture/syscalls.md index 13fa504..427efd9 100644 --- a/docs/architecture/syscalls.md +++ b/docs/architecture/syscalls.md @@ -74,19 +74,19 @@ machine off, and Linux number 48 is the socket call. Two syscalls cannot share a ## What is implemented -Roughly 190 native numbers covering: +About 215 native numbers (193 of them with a Linux number) covering: | Area | | |---|---| -| Process | fork, exec, exit, wait/waitpid/wait4/waitid, clone, kill, sessions, process groups | +| Process | fork, exec, exit, exit_group, wait/waitpid/wait4/waitid, clone (processes and threads), futex, kill, tgkill, sessions, process groups | | Identity | the full real/effective/saved set for both uid and gid, groups, umask | -| Memory | mmap, munmap, mprotect, brk/sbrk, madvise, System V shared memory | +| Memory | mmap, munmap, mremap, mprotect, brk/sbrk, madvise, System V shared memory, memfd | | Filesystem | the ordinary set, the whole `*at()` family, statfs, xattr, mount/umount2 | | I/O | read/write, readv/writev, pread/pwrite, sendfile, poll/select/ppoll/pselect6, epoll (create, create1, ctl, wait, pwait), ioctl, fcntl | -| Sockets | socket, bind, listen, accept, connect, sendto, recvfrom, socketpair, setsockopt, getsockopt | +| Sockets | socket, bind, listen, accept/accept4, connect, sendto/recvfrom, sendmsg/recvmsg, socketpair, getsockname/getpeername, setsockopt/getsockopt, over AF_UNIX, AF_INET and AF_NETLINK | | Signals | rt_sigaction, rt_sigreturn, rt_sigsuspend, rt_sigpending, rt_sigtimedwait, sigaltstack | | Time | gettimeofday, clock_gettime, clock_getres, nanosleep, clock_nanosleep, alarm, timers | -| Notification | eventfd, eventfd2, timerfd_create/settime/gettime, signalfd, signalfd4 -- see `kernel/sys/sys_event.c` | +| Notification | eventfd, timerfd, signalfd, inotify -- see `kernel/sys/sys_event.c` | ### Argument fetching @@ -113,20 +113,19 @@ field by field, each through its own translation, or use `copyin`/`copyout`. ## What it does not do -- **`sendmsg`/`recvmsg` carry `SCM_RIGHTS` and nothing else.** - `SCM_CREDENTIALS` is refused rather than ignored. See - [networking.md](networking.md#descriptor-passing). -- **No `memfd_create`.** -- **No `getsockname` or `getpeername`**, and `accept()` does not report an - AF_INET peer's address. -- **`setsockopt` handles six options** and answers `ENOPROTOOPT` for everything - else. `SO_RCVTIMEO`/`SO_SNDTIMEO` are among the refused. -- **`clone` does not share an address space**, so it is `fork` with extra steps. +- **`sendmsg`/`recvmsg` carry `SCM_RIGHTS` and `SCM_CREDENTIALS`**, and nothing + else. Credentials are delivered only to a socket that asked with `SO_PASSCRED` + (D-Bus and WebKit do); other control messages are refused rather than ignored. + See [networking.md](networking.md#descriptor-passing). +- **`setsockopt` applies the options it understands** — they reach lwIP's pcb or + the Unix socket — and answers `ENOPROTOOPT` for everything else, never a + success that did nothing. The options that take a `struct` (`SO_RCVTIMEO`, + `SO_SNDTIMEO`) are among the refused. - **`sched_setaffinity` is accepted and ignored.** -- **`getrusage` does not distinguish `RUSAGE_CHILDREN`** from `RUSAGE_SELF`, and - fills in only the calling process's own tick count. -- **The ABI is chosen by link address.** It works, and it is a heuristic rather - than a fact about the binary. -- **`/proc//exe` is a regular file, not a symlink**, so `readlink()` on it - fails and `ps -o comm` prints `[init]` rather than `init`. Default `ps` and - `top` output is unaffected. +- **The ABI is chosen by link address.** Anything linked at `0x100000` or above + — musl's programs, and every dynamically linked one, whose interpreter is + placed at `0x40000000` — gets the Linux map; a first-party program linked at + `0x4000` gets the native one. It works, and it is a heuristic rather than a + fact about the binary. +- **`ptrace` and `seccomp`** are not implemented; a program that needs them + gets `ENOSYS`. diff --git a/docs/architecture/terminals.md b/docs/architecture/terminals.md index 5beb717..8489a8f 100644 --- a/docs/architecture/terminals.md +++ b/docs/architecture/terminals.md @@ -95,7 +95,9 @@ nothing to choose between. It records `proc->ctty` now, which is what |---|---|---| | `/dev/console` | 1 | the **active** terminal, so kernel messages follow a switch | | `/dev/tty` | 10 | the caller's **controlling** terminal, via `proc->ctty` | -| `/dev/tty1`..`tty6` | 11-16 | that terminal, whether or not it is on screen | +| `/dev/tty0` | | the active terminal, which is what Xorg and `openvt` ask for | +| `/dev/tty1`..`tty12` | 11-22 | that terminal, whether or not it is on screen | +| `/dev/ptmx`, `/dev/pts/N` | | pseudo-terminals, below | `/dev/tty` was an alias for `/dev/console` with the same major and minor. They are different questions and now have different answers. @@ -190,13 +192,13 @@ every candidate. |---|---| | **The host takes it, and the guest never sees it** | | | `Alt+F4`, `Alt+F2` | window manager | -| `Ctrl+Alt+F1..F6` | display manager — and it matches **loosely**, so adding Shift does not get past it | +| `Ctrl+Alt+F1..F12` | display manager — and it matches **loosely**, so adding Shift does not get past it | | `Ctrl+Alt+`digit | QEMU's own console tabs | | **Applications want it, and lose it permanently if the kernel eats it** | | | `Alt+`anything | Meta. zsh binds Meta-digit as `digit-argument`; so does every readline editor | | `Ctrl+`letter | control characters | | `Ctrl+Shift+`digit | types a character — `Ctrl+Shift+2` is `^@` | -| `F1..F6` alone | help, and whatever a full-screen program wants | +| `F1..F12` alone | help, and whatever a full-screen program wants | | **The keyboard layout wants it** | | | `AltGr+`letter | Polish diacritics — ą ć ę ł ń ó ś ź ż — and on some setups AltGr arrives as Ctrl+Alt | @@ -207,7 +209,7 @@ chvt # report the active terminal and the current chord chvt -c ctrl+alt+F # Ctrl+Alt+F1..F12 (the default) chvt -c altgr+F # AltGr+F1..F12 (the default until 2026-10) chvt -c ctrl+shift+F # Ctrl+Shift+F1..F12 -chvt -c alt+1 # Alt+1..6 +chvt -c alt+1 # Alt+1..9 chvt -c off # no chord; chvt N is then the only way ``` @@ -368,34 +370,56 @@ and erase characters — were added at the same time, because `TERM=linux` declares them and a terminal that drops them silently is the same class of bug waiting for a different editor. +## Handing a terminal to a program + +A display server takes a terminal with `VT_SETMODE(VT_PROCESS)` and `KDSETMODE(KD_GRAPHICS)`, +and from then on switching away from it is a **request**: the kernel sends the owner +`relsig` and waits for `VT_RELDISP`, so the server stops drawing and only then says +yes; switching back sends `acqsig` and the server redraws. A terminal in graphics mode +is not repainted by the kernel. If the owner dies, the terminal goes back to the kernel +whole. This is how Xorg runs on a terminal of its own, with Ctrl+Alt+F-key switching +between it and the text consoles and Ctrl+Alt+Backspace ending the session. + +## Pseudo-terminals + +`/dev/ptmx` makes a pair on every open and hands back the master; the slave is +`/dev/pts/N`, which the program on the other side gets as its stdin, stdout and +controlling terminal. Between them is a line discipline, the console's in miniature: +what the master writes is the slave's input (canonical editing, echo, `^C` as `SIGINT` +to the foreground group) and what the slave writes is read back by the master with NL +turned into CR NL. xterm, tmux, `script` and `sshd` all need exactly this. The pairs +share one lock; `kernel/dev/pty.c`. + +## The text cursor + +A steady inverted block — the cell with its colours swapped relative to what it would +show, so a cursor on reverse-video text is the plain text and every cursor on screen +is visible against its cell. `ESC[?25l` hides it, `ESC[?25h` shows it. It is drawn into +the grid's repaint and does not blink. + ## What it does not do - **No scrollback.** The grid is exactly the visible screen. Shift+PageUp does nothing because there is nothing above the top row to show. -- **No reflow on resize.** The geometry is read once at `vt_init()`. Nothing - resizes a framebuffer console today; a compositor will, and that is when this - has to grow. -- **No `VT_RELDISP` handshake.** `VT_PROCESS` stops the kernel switching, but - the kernel does not signal the owning process and wait for an answer. A - display server that needs to release the GPU *before* the screen changes - hands will need that, and `struct vt_mode` already has the fields. -- **No terminal past 6.** Linux allows 63. Six is what the chords reach and - what the grids are sized for. +- **No reflow on resize.** The geometry is read once at `vt_init()`; the console's + font is sized to the screen at that point, and a resolution change from `schmode` + repaints it, but text is not re-wrapped. +- **No terminal past 12.** Linux allows 63. Twelve is one per function key. - **Only 240x75 of any screen.** Larger framebuffers work; the grid is clamped and the remainder stays background. Raising `VT_MAXCOLS`/`VT_MAXROWS` is the fix, and costs BSS. - **No UTF-8 the font lacks.** The console draws through the TrueType - rasteriser now, so the limit is the font's `cmap` rather than a 256-entry - table — Latin, Latin Extended, Greek, Cyrillic and the common punctuation and - maths symbols all render. Anything genuinely absent still draws as `?`, which - is deliberate: drawing a *different* character would be worse. + rasteriser, so the limit is the font's `cmap` — Latin, Latin Extended, Greek, + Cyrillic and the common punctuation and maths symbols all render. Anything + genuinely absent still draws as `?`, which is deliberate: drawing a + *different* character would be worse. - **No shaping, and no wide characters.** One codepoint is one cell of one advance. Combining marks land on top of their own cell rather than the preceding one, and CJK draws at half the width it wants. Both need a width table and a cell that can span two columns. - **No dead keys**, so French and Spanish layouts are absent rather than - broken. See above. + broken. - **No per-terminal layout.** One keyboard, one layout. Switching to tty3 does not move anybody's fingers. -- **Only three layouts.** Adding one is a table in `keymap.c` and a line in the +- **Only three layouts** (US, Polish programmers', German). Adding one is a table in `keymap.c` and a line in the catalogue; the mechanism is not the limit, the typing is. diff --git a/docs/architecture/usb.md b/docs/architecture/usb.md index 4e2fb8a..d3bae1a 100644 --- a/docs/architecture/usb.md +++ b/docs/architecture/usb.md @@ -21,9 +21,10 @@ The specification is public and the controller is not complicated. > BSD is much more valuable as the reference you check yourself against than as > the code you paste. -Licensing points the same way: the BSD stacks are BSD-2/3-clause and compatible, -Linux's drivers are GPL-2.0-only and are not — see -[../licensing.md](../licensing.md). +Licensing does not decide it: the BSD stacks are BSD-2/3-clause and compatible, and +since SchOS became GPL-2.0-or-later Linux's GPL-2.0-only drivers are compatible too +— see [../licensing.md](../licensing.md). The argument against porting is the +framework underneath, not the licence. ## What the spec gives you for free @@ -129,7 +130,8 @@ working one. filesystem's next read, by the rule in `blkdev_rw()`: a filesystem that went on with missing blocks would be guessing. - **HID is keyboards and pointers.** Consumer-control interfaces (volume keys), - game pads and the rest are declined. Keyboard LEDs are not driven. + game pads and the rest are declined. Caps, Num and Scroll Lock light their LEDs + on USB and PS/2 keyboards, one state shared between keyboards. - **USB 3 hubs** are not driven: the hub driver speaks the USB 2 hub protocol. - **No isochronous transfers**, so no audio and no cameras. - **No USB 3 SuperSpeed-specific features** beyond what enumeration needs — the diff --git a/docs/licensing.md b/docs/licensing.md index 0a3fff9..899c8ef 100644 --- a/docs/licensing.md +++ b/docs/licensing.md @@ -271,10 +271,12 @@ copied file is a file to be understood, attributed and maintained forever. [architecture/ldls.md](architecture/ldls.md) runs Linux drivers. This section was written when SchOS was Apache-2.0 and the boundary below was a legal -necessity. Under GPL-2.0-or-later it is a **project policy** instead: shipping -a GPL-2.0-only `.ko` with SchOS would now be lawful, provided the source goes -with it. It is still not done, because a module we never ship is a module we -never have to vouch for, track upstream, or provide source for. +necessity. Under GPL-2.0-or-later it was a **project policy** instead, and the owner +has since set the policy aside: Linux code and firmware blobs **may be shipped** with +SchOS, provided the source goes with them (GPLv2 §3). The repository still carries no +`.ko` — they are staged from the build machine's Linux kernel (`make ldls-stage`, +`make ldls-r8169`) — so shipping one is a decision about a release, and the release has +to carry the corresponding source. ### LDLS is ours @@ -298,10 +300,9 @@ GPLv2 §0 is explicit about where its obligations attach: > by this License; they are outside its scope. **The act of running the Program > is not restricted.** -So: we ship LDLS and no modules. The user obtains a module from wherever they -get their software and runs it on their own machine. `RAMDISK_EXCLUDE` and the -`*.ko` rule in `.gitignore` keep that boundary mechanical rather than -intentional. +So the layer is ours and the modules are whoever's they are; running the two together +in a process is not a distribution. `RAMDISK_EXCLUDE` and the `*.ko` rule in `.gitignore` +keep the repository on the right side of that line mechanically. ### `EXPORT_SYMBOL_GPL` does not apply @@ -322,7 +323,7 @@ show. ### BTF: the layouts are not in our source at all -§3 of the plan has one uncomfortable row — struct layouts. A compiled driver +Rule 3 of [architecture/ldls.md](architecture/ldls.md) has one uncomfortable row — struct layouts. A compiled driver indexes fields by offset, so the shim has to reproduce them exactly, and a field-for-field match *looks* like transcription even when it is not.