Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cb3d28e3d6 | ||
|
|
2556a955b3 |
64 changed files with 1606 additions and 2462 deletions
8
Makefile
8
Makefile
|
|
@ -545,7 +545,7 @@ $(shell mkdir -p $(BUILD); \
|
|||
|
||||
# -----------------------------------------------------------------------
|
||||
# The root filesystem is an Alpine root that apk manages, with SchOS installed
|
||||
# into it as the package schos-base -- see docs/plan/alpine.md and "The
|
||||
# into it as the package schos-base -- see docs/architecture/packages.md and "The
|
||||
# apk-managed root" below. Both images are made from it: schfs.img, which
|
||||
# `make qemu` boots, and the ramdisk inside the kernel, which `make iso`
|
||||
# boots. sysroot/ is what schos-base is packaged from, not a root of its own.
|
||||
|
|
@ -1321,7 +1321,7 @@ crashqr:
|
|||
$(PYTHON) tools/python/crashqr.py
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# LDLS -- the Linux Driver Layer. docs/plan/ldls.md.
|
||||
# LDLS -- the Linux Driver Layer. docs/architecture/ldls.md.
|
||||
#
|
||||
# ldls/ko.c is freestanding and builds on both sides: here as part of a host
|
||||
# tool, and later as part of the SchOS-side host. Same discipline as SchFS,
|
||||
|
|
@ -1346,7 +1346,7 @@ $(LDLS_RUN): ldls/host/run.c ldls/ko.c ldls/stub.c ldls/btf.c \
|
|||
$(HOSTCC) $(HOSTWARN) -O1 -fno-strict-aliasing -o $@ \
|
||||
ldls/host/run.c ldls/ko.c ldls/stub.c ldls/btf.c
|
||||
|
||||
# The test object, compiled and never linked. Ours, because docs/plan/ldls.md
|
||||
# The test object, compiled and never linked. Ours, because docs/architecture/ldls.md
|
||||
# rule 1 means no .ko is committed here -- see the header of refmod.c.
|
||||
#
|
||||
# -fno-pic matters: a PIC object routes its data through the GOT, and a GOT is
|
||||
|
|
@ -1369,7 +1369,7 @@ ldls: $(LDLS_INSPECT) $(LDLS_RUN) $(LDLS_TYPES)
|
|||
#
|
||||
# make ldls-stage KO=/path/to/8139too.ko BTF=/sys/kernel/btf/vmlinux
|
||||
#
|
||||
# This is the model docs/plan/ldls.md describes, carried out on the machine
|
||||
# This is the model docs/architecture/ldls.md describes, carried out on the machine
|
||||
# that is doing the testing: the user obtains a module and loads it on their
|
||||
# own machine. Deliberately **not** a dependency of `all`, so an ordinary build
|
||||
# never picks it up, and `.gitignore` refuses `*.ko` so it cannot be committed
|
||||
|
|
|
|||
193
README.md
193
README.md
|
|
@ -4,48 +4,50 @@
|
|||
|
||||
# SchOS
|
||||
|
||||
**A 64-bit operating system, written from the metal up.**
|
||||
**A 64-bit operating system, written from the metal up — and it runs Linux's software.**
|
||||
|
||||
SMP kernel · copy-on-write filesystem · Linux syscall ABI · Alpine packages · X11 on real hardware
|
||||
SMP kernel · copy-on-write filesystem · Linux syscall ABI · Alpine packages · Xorg · Linux's own drivers
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
SchOS began as [xv6](https://github.com/mit-pdos/xv6-public) and is now mostly
|
||||
something else. The scheduler, the memory manager and the trap path have been
|
||||
rewritten for x86-64 and for more than one processor; the filesystem has been
|
||||
replaced outright; the syscall table speaks Linux numbering, so real Linux
|
||||
software runs on it unmodified — statically or dynamically linked, straight
|
||||
out of Alpine's package repositories.
|
||||
SchOS is a hobby operating system built by one person. It began as
|
||||
[xv6](https://github.com/mit-pdos/xv6-public) and almost nothing of that is left:
|
||||
the scheduler, the memory manager, the trap path and every driver have been
|
||||
rewritten for x86-64 and for more than one processor, and the filesystem is a
|
||||
different one. The syscall table speaks Linux's numbering, so **real Linux
|
||||
programs run on it unmodified** — statically or dynamically linked, straight out
|
||||
of Alpine's package repositories — and a desktop (Xorg and Window Maker) and a
|
||||
web browser (NetSurf) come up on the framebuffer.
|
||||
|
||||
It is a hobby system built by one person, and it is honest about that: what
|
||||
works below has been *run*, not inferred, and the gaps are listed as plainly as
|
||||
the features.
|
||||
It is honest about being a hobby system: what is listed below has been *run*,
|
||||
not inferred, and [the gaps](#not-there-yet) are listed as plainly as the
|
||||
features. It boots from a USB stick on three different real machines.
|
||||
|
||||
## In numbers
|
||||
|
||||
<!-- stats:begin -->
|
||||
| | |
|
||||
|---|---|
|
||||
| **First-party code** | 61,367 lines, plus 21,128 lines of comments (26%) in 272 files |
|
||||
| **Kernel** (`kernel/`, `include/`) | 45,207 lines of code, 16,111 of comments |
|
||||
| **Syscalls** | 208, of which 184 on Linux numbering |
|
||||
| **First-party code** | 72,766 lines, plus 23,668 lines of comments (25%) in 307 files |
|
||||
| **Kernel** (`kernel/`, `include/`) | 51,893 lines of code, 17,213 of comments |
|
||||
| **Syscalls** | 215, of which 193 on Linux numbering |
|
||||
| **Drivers** | 15 in the kernel |
|
||||
| **First-party programs** | 29 in `cmd/` |
|
||||
| **Alpine packages in the root** | 88, from 11 asked for in `ports/alpine/world` |
|
||||
| **History** | 253 commits since 2026-08-09 (76 days) |
|
||||
| **Alpine packages in the root** | 179, from 29 asked for in `ports/alpine/world` |
|
||||
| **History** | 289 commits since 2026-08-09 (78 days) |
|
||||
|
||||
<details>
|
||||
<summary>Lines by language</summary>
|
||||
|
||||
| Language | Code | Comments |
|
||||
|---|--:|--:|
|
||||
| C | 56,580 | 19,427 |
|
||||
| C | 67,615 | 21,904 |
|
||||
| Assembly | 1,750 | 336 |
|
||||
| PowerShell | 1,362 | 638 |
|
||||
| Python | 759 | 147 |
|
||||
| Shell | 716 | 469 |
|
||||
| Python | 943 | 150 |
|
||||
| Shell | 896 | 529 |
|
||||
| Perl | 129 | 45 |
|
||||
| Linker script | 71 | 66 |
|
||||
|
||||
|
|
@ -55,78 +57,74 @@ the features.
|
|||
<code>ports/</code> and generated tables are not counted.</sub>
|
||||
<!-- stats:end -->
|
||||
|
||||
## The pieces
|
||||
## What it is made of
|
||||
|
||||
| Layer | Name | What it is |
|
||||
|---|---|---|
|
||||
| System | **SchOS** | The distribution — kernel, userland, tooling |
|
||||
| Kernel | **SchIFT** | Multi-core x86-64 kernel with a modular VFS |
|
||||
| Kernel | **SchIFT** | Multi-core x86-64 kernel with a modular VFS and Linux's syscall ABI |
|
||||
| Filesystem | **SchFS** | Transactional, copy-on-write, the root filesystem |
|
||||
| Image format | **SCFF** | The SchOS Compressed File Format the live root ships in |
|
||||
| API | **SchAPI** | System programming interface — the stable surface |
|
||||
| Compositor | **SchCom** | Display compositor *(planned)* |
|
||||
| Window manager | **SchWM** | *(planned)* |
|
||||
| Desktop | **SchDE** | *(planned)* |
|
||||
| Packages | **apk** | Alpine's package manager manages the root; SchOS itself is the package `schos-base` |
|
||||
| Desktop | **Xorg + Window Maker** | Alpine's, on the framebuffer |
|
||||
| Crash screen | **PSOGM / GMMOS** | The Purple Screen of Guru Meditation, and the micro OS that runs under it |
|
||||
| API | **SchAPI** | The long-term native interface — *designed, not built* |
|
||||
|
||||
`uname -s` reports `SchIFT`, the way every Unix reports its kernel rather than
|
||||
its distribution.
|
||||
|
||||
## What works
|
||||
|
||||
**Kernel** — Long mode, 4-level paging, preemptive multitasking. **SMP** up to
|
||||
64 processors, enumerated from the ACPI MADT. ACPI through
|
||||
[uACPI](https://github.com/uACPI/uACPI): interrupt routing from `_PRT`, the
|
||||
power button, clean power-off and reset. Memory sized from the E820/UEFI map.
|
||||
**Kernel** — Long mode, 4-level paging, preemptive multitasking, a fair-share
|
||||
scheduler with `nice`. **SMP** up to 64 processors, enumerated from the ACPI
|
||||
MADT. **Threads** (`clone`, `futex`, TLB shootdowns), **demand paging** for the
|
||||
huge reservations browsers make, and dynamic linking. ACPI through
|
||||
[uACPI](https://github.com/uACPI/uACPI): interrupt routing, the power button,
|
||||
clean power-off and reset, batteries and AC through a laptop's embedded
|
||||
controller, and deep idle states. Memory sized from the firmware's map, up to 256 GB.
|
||||
|
||||
**Filesystem** — **SchFS** is the root: a copy-on-write B-tree with a
|
||||
uberblock ring, so a crash leaves either the old tree or the new one and there
|
||||
is nothing to repair. The live image ships as **[SCFF](docs/architecture/scff.md)**:
|
||||
the 358 MB root compresses to 102 MB with zstd and is unpacked a chunk at a
|
||||
time, the first time something reads it. `/proc` has `/proc/<pid>/fd`,
|
||||
`/proc/self` and the rest that `ps`, `top` and Xorg read.
|
||||
**Filesystem** — **SchFS** is the root: a copy-on-write B-tree with an uberblock
|
||||
ring, so a crash leaves either the old tree or the new one and there is nothing
|
||||
to repair. The live image ships as **[SCFF](docs/architecture/scff.md)**: the
|
||||
root compresses with zstd to about a third of its size and is unpacked a chunk at
|
||||
a time, the first time something reads it. `/proc`, `/sys` and `/dev` are
|
||||
synthesised, in Linux's layouts, because that is what the programs read.
|
||||
|
||||
**Processes** — fork, exec, wait, pipes, signals, sessions and job control,
|
||||
`mmap` (anonymous and file-backed), POSIX shared memory, `memfd`, Unix domain
|
||||
sockets with descriptor passing (`SCM_RIGHTS`), `poll`, `epoll`, `eventfd`,
|
||||
`timerfd`, `signalfd`. **Dynamic linking**: `exec` loads `ld-musl`, so
|
||||
binaries come from Alpine as they are.
|
||||
**Linux software** — `fork`, `clone`, `exec`, pipes, signals, sessions and job
|
||||
control, `mmap`, POSIX and `memfd` shared memory, Unix sockets with descriptor and
|
||||
credential passing, `poll`, `epoll`, `eventfd`, `timerfd`, `signalfd`, `inotify`,
|
||||
pseudo-terminals. The root is managed by Alpine's **apk**: `ports/alpine/world`
|
||||
says what is installed, `world.lock` pins it, and `apk add` fetches more over HTTPS.
|
||||
|
||||
**Packages** — the root is managed by Alpine's **apk**. `ports/alpine/world`
|
||||
says what is installed; the build resolves it once into `world.lock` and every
|
||||
image after that is reproducible. SchOS itself is one more package in it.
|
||||
**Desktop** — **Xorg with Window Maker**, `xterm`, the classic X clients, SDL games
|
||||
and the NetSurf browser, on the framebuffer (`/dev/fb0`, Linux fbdev) with input
|
||||
as Linux evdev. Twelve virtual terminals, switched with Ctrl+Alt+F1–F12, with X on
|
||||
a terminal of its own. Sound through ALSA (over OSS and an AC'97 driver).
|
||||
|
||||
**Graphics** — **Xorg with Window Maker**, preinstalled, on the framebuffer
|
||||
(`/dev/fb0`, Linux fbdev). Six virtual terminals, switched with
|
||||
Ctrl+Alt+F1–F6 or Alt+F1–F6, with X on a terminal of its own; Ctrl+Alt+Backspace
|
||||
ends the session. Input as Linux evdev on `/dev/input/event*`.
|
||||
|
||||
**Devices** — PCI with a bus-agnostic device model at `/proc/devices`.
|
||||
**Drivers** — written from the specifications, and **Linux's own, unmodified,
|
||||
through [LDLS](docs/architecture/ldls.md)**: a `.ko` runs in an ordinary process
|
||||
and reaches the hardware through `/dev/pci`, so a crashed driver costs an interface,
|
||||
not the machine. The laptop's Realtek NIC is driven that way.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Storage | AHCI (SATA), NVMe, IDE, USB mass storage |
|
||||
| Network | Intel e1000, Realtek r8169 and RTL8139 |
|
||||
| Network | Intel e1000, Realtek RTL8139 and r8169 (the latter also as Linux's driver in a process) |
|
||||
| USB | xHCI, hubs, HID keyboards and mice |
|
||||
| Input | PS/2, I2C-HID laptop touchpads (DesignWare I2C, found through ACPI) |
|
||||
| Input | PS/2, I2C-HID laptop touchpads and keyboards (DesignWare I2C, found through ACPI) |
|
||||
| Display | UEFI GOP framebuffer, virtio-gpu, Bochs/stdvga |
|
||||
| Audio | AC'97 |
|
||||
|
||||
**Networking** — ARP, IPv4, ICMP, UDP, TCP and DHCP on lwIP. DNS, `wget`
|
||||
over **HTTPS**, `ping`, and an `httpd` serving a directory back.
|
||||
**Networking** — ARP, IPv4, ICMP, UDP, TCP and DHCP on lwIP; DNS, `wget` and `curl`
|
||||
over **HTTPS**, `ping`, `httpd`, netlink for `getifaddrs`.
|
||||
|
||||
**Userland** — musl, toybox, mksh and zsh, plus whatever Alpine has: `ls`,
|
||||
`ps`, `top`, `awk`, `diff`, `xterm`, `fastfetch` behave the way you expect,
|
||||
because they are the real programs.
|
||||
**When it dies** — PSOGM, the Purple Screen of Guru Meditation: stop code, fault
|
||||
address, call chain, and the log as QR codes. Under it runs **GMMOS**, a micro OS with
|
||||
its own keyboard drivers, so a dead machine can still show its log, take a few
|
||||
commands, and restart or power off.
|
||||
|
||||
**Text** — a TrueType rasteriser (`glyf` outlines, composite glyphs, cmap
|
||||
format 4). JetBrains Mono for the terminal, Roboto for the desktop to come.
|
||||
|
||||
**When it dies** — PSOGM, the Purple Screen of Guru Meditation: stop code,
|
||||
fault address, call chain, and a QR code containing a text copy of the whole
|
||||
report.
|
||||
|
||||
**On real hardware** — booted on an AMD B550 desktop (Ryzen 5 4600G), an ASUS
|
||||
TUF A15 laptop and an Ivy Bridge HP all-in-one, from a USB stick.
|
||||
**On real hardware** — booted from a USB stick on an AMD B550 desktop (Ryzen 5
|
||||
4600G), an ASUS TUF A15 laptop and an Ivy Bridge HP all-in-one.
|
||||
|
||||
## Quick start
|
||||
|
||||
|
|
@ -134,19 +132,19 @@ TUF A15 laptop and an Ivy Bridge HP all-in-one, from a USB stick.
|
|||
make # kernel + UEFI loader + root filesystem
|
||||
make qemu # build it and boot it
|
||||
make iso # a bootable ISO for VMware or real hardware
|
||||
./write.sh sdX # write that ISO to a USB stick (Linux; it checks what it is overwriting)
|
||||
```
|
||||
|
||||
The ISO is a hybrid image: write it to a USB stick with
|
||||
`dd if=schos.iso of=/dev/sdX bs=4M` (or Rufus/Etcher in DD mode) and boot it
|
||||
on any UEFI machine.
|
||||
The ISO is a hybrid image: it boots on any UEFI machine from a stick written with
|
||||
`write.sh`, `dd if=schos.iso of=/dev/sdX bs=4M`, or Rufus/Etcher in DD mode. **It runs
|
||||
from RAM and touches none of the machine's disks.** Log in on the console, and type
|
||||
`startx` for the desktop.
|
||||
|
||||
Builds on **Linux, macOS, FreeBSD/OpenBSD/NetBSD and Windows** (Git Bash). The
|
||||
toolchain is clang + lld everywhere — it is a cross-compiler by construction, so
|
||||
one LLVM install targets the bare-metal kernel, the 16-bit boot code and musl
|
||||
userland from any host. See **[docs/guides/building.md](docs/guides/building.md)** for
|
||||
prerequisites per platform, and
|
||||
**[docs/history/moving-to-linux.md](docs/history/moving-to-linux.md)** for moving a checkout
|
||||
between machines.
|
||||
userland from any host; `nix-shell` supplies all of it. See
|
||||
**[docs/guides/building.md](docs/guides/building.md)** for prerequisites per platform.
|
||||
|
||||
## Testing
|
||||
|
||||
|
|
@ -160,28 +158,35 @@ tools/windows/bootcheck.ps1 -Tty -Ps # terminal behaviour, process enumeration
|
|||
make schfs-check # SchFS B-tree, object and image suites
|
||||
```
|
||||
|
||||
`bootcheck` is PowerShell because `pwsh` runs on every supported host.
|
||||
|
||||
The numbers at the top are measured, not typed: `python3 tools/python/stats.py`
|
||||
rewrites them here and on the website.
|
||||
`bootcheck` is PowerShell because `pwsh` runs on every supported host. Several
|
||||
programs in `cmd/` (`socktest`, `memfdtest`, `xready`, …) exist to be run *on* the
|
||||
system and report which of their own checks failed. See
|
||||
**[docs/guides/testing.md](docs/guides/testing.md)** for what each proves. The numbers at
|
||||
the top are measured, not typed: `python3 tools/python/stats.py` rewrites them here
|
||||
and on the website.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
kernel/ arch/ core/ dev/ drivers/ fs/ net/ proc/ sys/
|
||||
kernel/ arch/ core/ dev/ drivers/ fs/ net/ proc/ sys/ acpi/ gmmos/ scff/
|
||||
fs/schfs/ the filesystem -- same source the host tools compile
|
||||
ldls/ the Linux driver layer: the .ko loader, BTF reader, Linux symbols
|
||||
include/ kernel and userland headers, by subsystem (arch/ core/ fs/ dev/ ...)
|
||||
boot/ uefi/ (PE loader) common/ (shared loader UI)
|
||||
lib/ cmd/ first-party userland runtime and programs
|
||||
ports/ third-party software built from source
|
||||
etc/ what the image starts with: schinit's job manifests, motd, ...
|
||||
ports/ third-party software built from source; ports/alpine is the apk root
|
||||
tools/ windows/ (.ps1) posix/ (.sh) python/ (.py) and .c/.pl
|
||||
projects/ things that run ON SchOS (an installer, SchFS for Linux: both frozen)
|
||||
docs/ architecture/ (what it is) plan/ (what's next)
|
||||
guides/ (how to work on it) history/ (why it's like this)
|
||||
Website/ (the site, and the docs reader)
|
||||
```
|
||||
|
||||
**[docs/README.md](docs/README.md)** is the index. If you are reading the tree
|
||||
for the first time, [docs/architecture/README.md](docs/architecture/README.md)
|
||||
is the system layer by layer.
|
||||
is the system layer by layer, and [docs/plan/1.0.md](docs/plan/1.0.md) says
|
||||
where it stands against its finish line.
|
||||
|
||||
One rule worth knowing: **the kernel and the host tools compile the same SchFS
|
||||
source**. `mkschfs` cannot disagree with the kernel about the on-disk format,
|
||||
|
|
@ -193,19 +198,24 @@ Another: **syscalls are declared in exactly one place**,
|
|||
|
||||
## Not there yet
|
||||
|
||||
- **No threads.** `clone` makes processes, not threads (`CLONE_VM` is
|
||||
refused), so multi-threaded programs run on one thread or not at all. Xorg
|
||||
runs with `-dumbSched` for this reason.
|
||||
- **No installer, and nothing persists.** The live image runs from RAM and
|
||||
writes nothing to the machine's disks, so what you install or save is gone at
|
||||
reboot. This is the one thing between the system and its 1.0 definition.
|
||||
- **No GPU acceleration.** Xorg draws in software; the plan is Linux's `amdgpu`
|
||||
through LDLS ([docs/plan/gpu.md](docs/plan/gpu.md)). There is no IOMMU either,
|
||||
so a driver in a process is isolated against its bugs, not against malice.
|
||||
- **No udev.** Xorg is told its input devices by a config file instead of
|
||||
finding them; `/sys` and the device-event socket are what is missing.
|
||||
finding them; `/sys` has no device tree and no uevent socket.
|
||||
- **WebKit browsers do not start.** NetSurf does; Epiphany's web process dies at
|
||||
start-up, and it has not been run to ground.
|
||||
- **USB 2.0 controllers (EHCI)** have no driver. On older machines the ports
|
||||
wired only to EHCI do nothing; xHCI ports work.
|
||||
- **Touchpads run in mouse mode**: pointer, click and tap, but no two-finger
|
||||
scroll yet.
|
||||
- **No page cache**, so writable `MAP_SHARED` of a file is refused rather than
|
||||
faked. SysV shared memory is missing too.
|
||||
- **No installer.** SchOS runs live from the USB stick; nothing is written to
|
||||
the machine's disks.
|
||||
faked, and `mprotect` is accepted and ignored (nothing has NX).
|
||||
- **No Wi-Fi, no HDA audio**, and the Realtek NIC on the laptop has reached the
|
||||
point of finding its PHY; whether it carries traffic there is not confirmed.
|
||||
- SchFS has no Linux driver yet; the plan is on-disk format version first, then
|
||||
FUSE, then a kernel module.
|
||||
- CFF/OpenType outlines and TrueType hinting are deliberately out of scope.
|
||||
|
|
@ -215,14 +225,15 @@ Another: **syscalls are declared in exactly one place**,
|
|||
**GPL-2.0-or-later**, copyright Nexoniarz. See [LICENSE](LICENSE).
|
||||
|
||||
The kernel derives from xv6, which is MIT — see [LICENSE.MIT](LICENSE.MIT).
|
||||
Third-party code redistributed in a built image (musl, toybox, mksh, zsh, kilo,
|
||||
the fonts) keeps its own licence; [NOTICE](NOTICE) carries the attributions.
|
||||
See [docs/licensing.md](docs/licensing.md).
|
||||
Third-party code redistributed in a built image (musl, toybox, mksh, zstd, the
|
||||
fonts, and Alpine's packages, each under its own licence) keeps that licence;
|
||||
[NOTICE](NOTICE) carries the attributions. See [docs/licensing.md](docs/licensing.md).
|
||||
|
||||
Drivers may be adapted from BSD sources and, since the move to the GPL, from
|
||||
Linux — GPL-2.0-only code combines with GPL-2.0-or-later, and the result ships
|
||||
under GPLv2. Best of all is still the public specification — xHCI, virtio,
|
||||
AHCI, e1000 — which carries no licence on what you write from it.
|
||||
under GPLv2; Linux's own drivers can also be run unmodified through LDLS. Best of
|
||||
all is still the public specification — xHCI, virtio, AHCI, e1000 — which carries
|
||||
no licence on what you write from it.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
//
|
||||
// memfdtest -- two processes map the same memfd, one writes, the other sees it.
|
||||
//
|
||||
// That sentence is docs/plan/foundations.md's own acceptance criterion for
|
||||
// That sentence is docs/architecture/kernel.md's own acceptance criterion for
|
||||
// item 2, and it is the check a permissive implementation cannot pass: an
|
||||
// mmap that quietly returns private zeroed pages satisfies every return code
|
||||
// here and fails the one assertion that matters, because the reader is a
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
//
|
||||
// pcitest -- prove that a driver could live out here.
|
||||
//
|
||||
// docs/plan/hybrid.md mechanisms 2 and 4, exercised end to end from a process:
|
||||
// docs/architecture/devices.md mechanisms 2 and 4, exercised end to end from a process:
|
||||
// enumerate the bus, claim a device, map its registers, read them, and get
|
||||
// memory the device could be pointed at.
|
||||
//
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
// worth recording because it is not "ours is better". toybox's ping reads its
|
||||
// replies with recvmsg(2) and asks for the TTL through an IP_RECVTTL control
|
||||
// message, and SchOS has neither: no recvmsg, no ancillary data. Both are owed
|
||||
// -- recvmsg is item 1 of docs/plan/foundations.md, where it is wanted for
|
||||
// -- recvmsg is item 1 of docs/architecture/kernel.md, where it is wanted for
|
||||
// SCM_RIGHTS -- and neither should be rushed to make one applet run.
|
||||
//
|
||||
// So this uses sendto/recvfrom, which the kernel does have, and proves the ICMP
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
//
|
||||
// schmode -- change the display mode.
|
||||
//
|
||||
// docs/plan/hybrid.md milestone H3. The interesting thing about this program
|
||||
// docs/architecture/devices.md milestone H3. The interesting thing about this program
|
||||
// is how little it is: it opens a socket to `vgpud` and writes a line. No
|
||||
// syscall it makes has heard of a GPU, no kernel code runs that knows what a
|
||||
// scanout is, and if the driver on the other end gets the sequence wrong the
|
||||
|
|
|
|||
|
|
@ -447,7 +447,7 @@ strneq_(const char *a, const char *b, int n)
|
|||
// ---------------------------------------------------------------------------
|
||||
// 8. SCM_RIGHTS: a descriptor crossing between processes.
|
||||
//
|
||||
// docs/plan/foundations.md item 1, docs/plan/hybrid.md mechanism 1. The thing
|
||||
// docs/architecture/kernel.md item 1, docs/architecture/devices.md mechanism 1. The thing
|
||||
// being proved is not that recvmsg returns a number -- it is that the number
|
||||
// names the *same open file* the other process had, which is why every check
|
||||
// below moves data through the descriptor rather than inspecting it.
|
||||
|
|
@ -763,7 +763,7 @@ test_fdpass_nodata(void)
|
|||
// ---------------------------------------------------------------------------
|
||||
// 15. A bound socket is a file.
|
||||
//
|
||||
// docs/plan/foundations.md item 3, in its own words: bind() a socket, then
|
||||
// docs/architecture/kernel.md item 3, in its own words: bind() a socket, then
|
||||
// stat() and `ls -l` the path and see an `s`, then unlink() it.
|
||||
//
|
||||
// The reason it matters is not tidiness. A Wayland client finds its compositor
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
//
|
||||
// vgpud -- the virtio-gpu driver, outside the kernel.
|
||||
//
|
||||
// docs/plan/hybrid.md's first eviction. The same device the kernel drives in
|
||||
// docs/architecture/devices.md's first eviction. The same device the kernel drives in
|
||||
// kernel/drivers/gpu/virtio_gpu/, driven instead by a process that asks the
|
||||
// kernel for four things and nothing else:
|
||||
//
|
||||
|
|
@ -648,7 +648,7 @@ present(void)
|
|||
//
|
||||
// This used to be a test card: three colour bands, four gradients, a
|
||||
// checkerboard, and this instance's pid in binary along the bottom edge. That
|
||||
// was scaffolding for hybrid.md's H2 and H3 -- a screenshot had to answer "did
|
||||
// was scaffolding for the out-of-kernel driver work -- a screenshot had to answer "did
|
||||
// a *userspace process* put this here", and "did the driver come back" needed
|
||||
// a second run to look different from a first that never stopped. Both
|
||||
// milestones are closed, and what was left was a garish test pattern on a
|
||||
|
|
@ -866,7 +866,7 @@ register_panic_fb(void)
|
|||
|
||||
// --- the modeset --------------------------------------------------------------
|
||||
//
|
||||
// docs/plan/hybrid.md milestone H3: *"a modeset API, which is the thing an
|
||||
// docs/architecture/devices.md milestone H3: *"a modeset API, which is the thing an
|
||||
// out-of-kernel GPU driver makes affordable and an in-kernel one never got."*
|
||||
//
|
||||
// The claim is worth testing rather than repeating, so here is the whole of
|
||||
|
|
@ -944,7 +944,7 @@ set_mode(int w, int h)
|
|||
//
|
||||
// The name is a *name*, not a path. AF_UNIX sockets in this kernel live in a
|
||||
// table keyed by string and never touch the filesystem -- that is item 3 of
|
||||
// docs/plan/foundations.md and it is not built. Nothing here depends on which
|
||||
// docs/architecture/kernel.md and it is not built. Nothing here depends on which
|
||||
// it is, and the day it becomes a real path this keeps working.
|
||||
#define VGPUD_SOCKET "/run/vgpud"
|
||||
|
||||
|
|
|
|||
|
|
@ -72,29 +72,36 @@ good chance it is in there.
|
|||
| | |
|
||||
|---|---|
|
||||
| [README](architecture/README.md) | The system, layer by layer. Read first. |
|
||||
| [boot.md](architecture/boot.md) | Firmware to the first userspace process. |
|
||||
| [kernel.md](architecture/kernel.md) | SchIFT: memory, processes, scheduling, traps, signals, SMP. |
|
||||
| [boot.md](architecture/boot.md) | Firmware to the first userspace process, and where the loader puts the kernel. |
|
||||
| [kernel.md](architecture/kernel.md) | SchIFT: memory, processes and threads, traps, signals, SMP. |
|
||||
| [scheduler.md](architecture/scheduler.md) | Fair share by virtual runtime, slices, and `nice`. |
|
||||
| [syscalls.md](architecture/syscalls.md) | The call table, and the Linux personality that lets real software run. |
|
||||
| [filesystem.md](architecture/filesystem.md) | The VFS, the buffer cache, procfs, mounts. |
|
||||
| [filesystem.md](architecture/filesystem.md) | The VFS, the buffer cache, procfs, sysfs, devtmpfs, mounts. |
|
||||
| [schfs.md](architecture/schfs.md) | SchFS: the copy-on-write root filesystem. |
|
||||
| [devices.md](architecture/devices.md) | The device model, driver binding, the class vtables. |
|
||||
| [scff.md](architecture/scff.md) | SCFF: the compressed image the live root ships as. |
|
||||
| [fat.md](architecture/fat.md) | FAT12/16/32 with long names: USB sticks, SD cards, the ESP. |
|
||||
| [devices.md](architecture/devices.md) | The device model, driver binding, the class vtables, and drivers that live in a process. |
|
||||
| [ldls.md](architecture/ldls.md) | Running Linux's drivers: a `.ko` in a process, reaching hardware through `/dev/pci`. |
|
||||
| [usb.md](architecture/usb.md) | xHCI, the USB core, and the class drivers. |
|
||||
| [graphics.md](architecture/graphics.md) | Framebuffer, text rendering, console, and the crash screen. |
|
||||
| [terminals.md](architecture/terminals.md) | The six virtual terminals, and how one is switched to. |
|
||||
| [networking.md](architecture/networking.md) | Drivers, lwIP, sockets — Unix and INET. |
|
||||
| [graphics.md](architecture/graphics.md) | Framebuffer, `/dev/fb0`, text rendering, the console, PSOGM and GMMOS. |
|
||||
| [terminals.md](architecture/terminals.md) | The twelve virtual terminals, ptys, and handing one to a display server. |
|
||||
| [audio.md](architecture/audio.md) | `/dev/dsp`, AudioEngine, AC'97, and ALSA over OSS. |
|
||||
| [networking.md](architecture/networking.md) | NIC drivers, lwIP, sockets — Unix, INET and netlink. |
|
||||
| [init.md](architecture/init.md) | `schinit`, PID 1 as a role, and the job manifests. |
|
||||
| [packages.md](architecture/packages.md) | The apk root: Alpine's packages, and SchOS as one of them. |
|
||||
| [desktop.md](architecture/desktop.md) | Xorg on the framebuffer, Window Maker, and what they stand on. |
|
||||
| [debugging.md](architecture/debugging.md) | Symbols, a log that outlives the kernel, and asking the kernel a question. |
|
||||
|
||||
## plan/ — what SchOS will be
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| [README](plan/README.md) | The three current decisions and the order they happen in. |
|
||||
| [1.0.md](plan/1.0.md) | **The finish line, and the freeze.** What 1.0 contains, what is deferred, and the register of every half-built thing in the tree. |
|
||||
| [foundations.md](plan/foundations.md) | The kernel work everything else is blocked on. |
|
||||
| [wayland.md](plan/wayland.md) | The GUI stack, and what gets deleted to make room. |
|
||||
| [hybrid.md](plan/hybrid.md) | Moving drivers out of the kernel. |
|
||||
| [README](plan/README.md) | What is here, what used to be, and the order. |
|
||||
| [1.0.md](plan/1.0.md) | **The finish line, and the freeze.** What 1.0 contains, where it stands, and the register of every half-built thing in the tree. |
|
||||
| [roadmap.md](plan/roadmap.md) | Everything else scoped and still open. |
|
||||
| [gpu.md](plan/gpu.md) | Linux's `amdgpu` through LDLS, and `/dev/dri` served by a process. |
|
||||
| [finch.md](plan/finch.md) | The author's language for SchOS's own programs. |
|
||||
| [schapi.md](plan/schapi.md) | The long-term API layering, and the compatibility promise. |
|
||||
| [roadmap.md](plan/roadmap.md) | Everything else still open. |
|
||||
|
||||
## guides/ — how to work on it
|
||||
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ is a token. A hex value in that file means the system is missing something, and
|
|||
the answer is to add it to Niveus rather than to work around it here.
|
||||
|
||||
What that replaced was a 21 KB `style.css` that transcribed
|
||||
[`include/gfx/theme.h`](../../include/theme.h) by hand — palette, type scale,
|
||||
[`include/gfx/theme.h`](../../include/gfx/theme.h) by hand — palette, type scale,
|
||||
shadows, the lot. The intent was right and the mechanism was a copy: two files
|
||||
that had to be kept in step by remembering to. Now the whole identity is one
|
||||
declaration, in the `<head>` of each page:
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@
|
|||
["architecture/syscalls.md", "Syscalls"],
|
||||
["architecture/filesystem.md", "The VFS"],
|
||||
["architecture/schfs.md", "SchFS"],
|
||||
["architecture/scff.md", "SCFF"],
|
||||
["architecture/fat.md", "FAT"],
|
||||
["architecture/terminals.md", "Virtual terminals"],
|
||||
["architecture/devices.md", "Device model"],
|
||||
|
|
@ -38,6 +39,8 @@
|
|||
["architecture/audio.md", "Audio"],
|
||||
["architecture/networking.md", "Networking"],
|
||||
["architecture/init.md", "init"],
|
||||
["architecture/packages.md", "Packages (apk)"],
|
||||
["architecture/desktop.md", "The desktop (Xorg)"],
|
||||
["architecture/ldls.md", "Linux drivers (LDLS)"],
|
||||
["architecture/debugging.md", "Debugging"]
|
||||
]},
|
||||
|
|
@ -51,9 +54,8 @@
|
|||
{ group: "Plan", items: [
|
||||
["plan/README.md", "What plan/ is for"],
|
||||
["plan/roadmap.md", "Roadmap"],
|
||||
["plan/foundations.md", "Foundations"],
|
||||
["plan/wayland.md", "Wayland"],
|
||||
["plan/hybrid.md", "Hybrid kernel"],
|
||||
["plan/gpu.md", "The GPU"],
|
||||
["plan/finch.md", "Finch"],
|
||||
["plan/schapi.md", "SchAPI"]
|
||||
]},
|
||||
{ group: "History", items: [
|
||||
|
|
|
|||
|
|
@ -66,33 +66,34 @@
|
|||
<p class="nv-prose">
|
||||
SchOS ships as one bootable ISO. It boots on UEFI, runs entirely from
|
||||
RAM, and touches nothing on the machine unless you tell it to — there is
|
||||
no installer yet, and that is the point of trying it this way.
|
||||
no installer yet, and that is the point of trying it this way. It comes up with
|
||||
Xorg, Window Maker and Alpine's package manager.
|
||||
</p>
|
||||
|
||||
<div class="nv-cluster">
|
||||
<a class="nv-button nv-button--primary nv-button--lg" href="releases/schos.iso" download>Download the ISO</a>
|
||||
<a class="nv-button nv-button--primary nv-button--lg" href="https://pandor.pl/git/nexoniarz/SchOS/releases">Download the ISO</a>
|
||||
<a class="nv-button nv-button--lg" href="#build">Build it yourself</a>
|
||||
</div>
|
||||
|
||||
<p class="nv-caption">
|
||||
<code class="nv-code">schos.iso</code> · about 62 MB · UEFI x86-64 ·
|
||||
nothing is written to your disks
|
||||
<code class="nv-code">schos.iso</code> · a few hundred MB · UEFI x86-64 ·
|
||||
nothing is written to your disks · only the newest release keeps its ISO
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<!-- The support table. Deliberately not a compatibility list: one Chromebox
|
||||
is the entire bare-metal record, and a longer table would be fiction. -->
|
||||
<!-- The support table. Deliberately not a compatibility list: three machines
|
||||
are the entire bare-metal record, and a longer table would be fiction. -->
|
||||
<article class="nv-card nv-card--elevated">
|
||||
<div class="nv-card__header">
|
||||
<div>
|
||||
<p class="nv-overline nv-text-tertiary">Before you do</p>
|
||||
<h2 class="nv-card__title">This has booted on exactly one real machine</h2>
|
||||
<h2 class="nv-card__title">This has booted on three real machines</h2>
|
||||
</div>
|
||||
</div>
|
||||
<div class="nv-card__body nv-text-secondary">
|
||||
Everything below QEMU is honest reporting rather than a compatibility
|
||||
list. SchOS drives real hardware, and it has been run on a very small
|
||||
number of it.
|
||||
number of it — from a USB stick, each time.
|
||||
</div>
|
||||
<div class="nv-card__body">
|
||||
<div class="nv-table-frame">
|
||||
|
|
@ -103,8 +104,16 @@
|
|||
<td class="nv-table__actions"><span class="nv-badge nv-badge--success">boots</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Asus Chromebox <span class="nv-text-tertiary">Guado CN62</span></td>
|
||||
<td class="nv-table__actions"><span class="nv-badge nv-badge--warning">under test</span></td>
|
||||
<td>AMD B550 desktop <span class="nv-text-tertiary">Ryzen 5 4600G</span></td>
|
||||
<td class="nv-table__actions"><span class="nv-badge nv-badge--success">boots</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ASUS TUF A15 laptop <span class="nv-text-tertiary">Ryzen, RTL8168H</span></td>
|
||||
<td class="nv-table__actions"><span class="nv-badge nv-badge--warning">boots, network under test</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>HP all-in-one <span class="nv-text-tertiary">Ivy Bridge, 2012</span></td>
|
||||
<td class="nv-table__actions"><span class="nv-badge nv-badge--success">boots</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Anything else</td>
|
||||
|
|
@ -231,11 +240,10 @@
|
|||
<span class="nv-overline nv-text-tertiary">Step three</span>
|
||||
<h3 class="nv-steps__title">A shell</h3>
|
||||
<p class="nv-steps__text">
|
||||
<code class="nv-code">schinit</code> starts one on the console and a
|
||||
second on <code class="nv-code">tty2</code>.
|
||||
<code class="nv-code">mksh</code> is <code class="nv-code">/bin/sh</code>,
|
||||
with <code class="nv-code">zsh</code> beside it and the real toybox
|
||||
userland behind both.
|
||||
<code class="nv-code">schinit</code> starts one on the console and one on
|
||||
each of the next five terminals. <code class="nv-code">zsh</code> is the login shell,
|
||||
<code class="nv-code">/bin/sh</code> is busybox's, and <code class="nv-code">mksh</code> is
|
||||
the static one recovery uses. Type <code class="nv-code">startx</code> for the desktop.
|
||||
</p>
|
||||
</div>
|
||||
</li>
|
||||
|
|
|
|||
|
|
@ -398,13 +398,13 @@
|
|||
|
||||
<article class="nv-card">
|
||||
<div class="nv-card__header">
|
||||
<h3 class="nv-card__title"><span class="nv-tile nv-tile--sm" style="--nv-tile-tint: var(--nv-color-warning)"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><rect x="3" y="4" width="13" height="10" rx="1.5"/><rect x="8" y="10" width="13" height="10" rx="1.5"/></svg></span>Schcom</h3>
|
||||
<span class="nv-badge nv-badge--warning">in progress</span>
|
||||
<h3 class="nv-card__title"><span class="nv-tile nv-tile--sm" ><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><rect x="3" y="4" width="13" height="10" rx="1.5"/><rect x="8" y="10" width="13" height="10" rx="1.5"/></svg></span>The desktop</h3>
|
||||
<span class="nv-badge nv-badge--success">runs</span>
|
||||
</div>
|
||||
<div class="nv-card__body nv-text-secondary">
|
||||
The compositor that will own the display. v0 runs, and descriptor
|
||||
passing — the one thing a Wayland client cannot do without — now
|
||||
exists. What a buffer still needs is a shareable memory object.
|
||||
Xorg on the framebuffer with Window Maker, <code class="nv-code">xterm</code> and a few X clients;
|
||||
NetSurf for the web. No acceleration yet — the GPU is a plan, and the plan is Linux's own
|
||||
<code class="nv-code">amdgpu</code> through LDLS.
|
||||
</div>
|
||||
</article>
|
||||
|
||||
|
|
@ -415,7 +415,7 @@
|
|||
</div>
|
||||
<div class="nv-card__body nv-text-secondary">
|
||||
The native programming interface — specified, not built, and deliberately
|
||||
so. It should be designed <i>from</i> the compositor's experience rather
|
||||
so. It should be designed <i>from</i> the desktop's experience rather
|
||||
than before it.
|
||||
</div>
|
||||
</article>
|
||||
|
|
@ -549,19 +549,19 @@
|
|||
<dl class="nv-stat-group">
|
||||
<div class="nv-stat">
|
||||
<dt class="nv-stat__label">lines of code in <code class="nv-code">kernel/</code> and <code class="nv-code">include/</code></dt>
|
||||
<dd class="nv-stat__value">45,207</dd>
|
||||
<dd class="nv-stat__value">51,893</dd>
|
||||
</div>
|
||||
<div class="nv-stat">
|
||||
<dt class="nv-stat__label">lines of comments alongside them</dt>
|
||||
<dd class="nv-stat__value">16,111</dd>
|
||||
<dd class="nv-stat__value">17,213</dd>
|
||||
</div>
|
||||
<div class="nv-stat">
|
||||
<dt class="nv-stat__label">first-party lines in all</dt>
|
||||
<dd class="nv-stat__value">61,367</dd>
|
||||
<dd class="nv-stat__value">72,766</dd>
|
||||
</div>
|
||||
<div class="nv-stat">
|
||||
<dt class="nv-stat__label">syscalls, Linux numbering</dt>
|
||||
<dd class="nv-stat__value">184</dd>
|
||||
<dd class="nv-stat__value">193</dd>
|
||||
</div>
|
||||
<div class="nv-stat">
|
||||
<dt class="nv-stat__label">drivers in the kernel</dt>
|
||||
|
|
@ -569,15 +569,15 @@
|
|||
</div>
|
||||
<div class="nv-stat">
|
||||
<dt class="nv-stat__label">Alpine packages in the live root</dt>
|
||||
<dd class="nv-stat__value">88</dd>
|
||||
<dd class="nv-stat__value">179</dd>
|
||||
</div>
|
||||
<div class="nv-stat">
|
||||
<dt class="nv-stat__label">commits</dt>
|
||||
<dd class="nv-stat__value">253</dd>
|
||||
<dd class="nv-stat__value">289</dd>
|
||||
</div>
|
||||
<div class="nv-stat">
|
||||
<dt class="nv-stat__label">days since the first one</dt>
|
||||
<dd class="nv-stat__value">76</dd>
|
||||
<dd class="nv-stat__value">78</dd>
|
||||
</div>
|
||||
<div class="nv-stat">
|
||||
<dt class="nv-stat__label">patches carried against fastfetch</dt>
|
||||
|
|
|
|||
|
|
@ -16,11 +16,10 @@ its distribution.
|
|||
| Kernel | **SchIFT** | Multi-core x86-64 kernel with a modular VFS |
|
||||
| Filesystem | **SchFS** | Transactional, copy-on-write, the root filesystem |
|
||||
| API | **SchAPI** | The native programming interface — *designed, not built* |
|
||||
| Compositor | **Schcom** | Owns the display; v0 runs |
|
||||
| Packages | **apk** | Alpine's package manager manages the root; SchOS itself is the package `schos-base` |
|
||||
|
||||
Twelve virtual terminals sit under all of it -- `/dev/tty1`..`tty12`, Ctrl+Alt+F1..F12
|
||||
or `chvt N` --
|
||||
each with its own screen, line discipline and foreground process group. See
|
||||
or `chvt N` -- each with its own screen, line discipline and foreground process group. See
|
||||
[terminals.md](terminals.md).
|
||||
|
||||
---
|
||||
|
|
@ -29,9 +28,11 @@ each with its own screen, line discipline and foreground process group. See
|
|||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ toybox kilo mksh zsh musl first-party cmd/ │ userland
|
||||
│ Xorg + Window Maker · Alpine's packages (apk) │ desktop, userland
|
||||
│ toybox mksh musl first-party cmd/ │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ vgpud — a driver that is an ordinary process │
|
||||
│ vgpud, ldlsd — drivers that are ordinary │
|
||||
│ processes (and Linux's r8169 inside ldlsd) │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ schinit — PID 1, job manifests, supervision │
|
||||
├─────────────────────────────────────────────────┤
|
||||
|
|
@ -45,7 +46,8 @@ each with its own screen, line discipline and foreground process group. See
|
|||
├─────────────────────────────────────────────────┤
|
||||
│ /dev/pci — MMIO, DMA and IRQs to a process │ ← the eviction seam
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ drivers: xHCI e1000 virtio-gpu VBE IDE PS/2│
|
||||
│ drivers: xHCI AHCI NVMe e1000 8139 r8169 AC97 │
|
||||
│ virtio-gpu VBE IDE PS/2 I2C-HID │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ scheduler │ paging │ traps │ IRQ │ SMP │ DMA │ SchIFT core
|
||||
└─────────────────────────────────────────────────┘
|
||||
|
|
@ -59,7 +61,8 @@ never names its consumer and a consumer never names a driver. And at
|
|||
**`/dev/pci`**, a driver stops having to be in the kernel at all: a process is
|
||||
handed a device's registers, DMA memory and interrupts, and the in-kernel driver
|
||||
that held it releases it cleanly. `vgpud` is the virtio-gpu driver living out
|
||||
there today. See [../plan/hybrid.md](../plan/hybrid.md).
|
||||
there today, and `ldlsd` runs Linux's own drivers the same way. See
|
||||
[devices.md](devices.md) and [ldls.md](ldls.md).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -91,9 +94,12 @@ Read in this order if you are new:
|
|||
| [fat.md](fat.md) | FAT12/16/32 with long names: USB sticks, SD cards, the ESP. |
|
||||
| [devices.md](devices.md) | Buses, classes, binding, and the vtables consumers talk to. |
|
||||
| [usb.md](usb.md) | xHCI, the USB core, and the three class drivers. |
|
||||
| [graphics.md](graphics.md) | Framebuffer, text rendering, the console, and PSOGM. |
|
||||
| [terminals.md](terminals.md) | The six virtual terminals, and the chord that switches them. |
|
||||
| [networking.md](networking.md) | NIC drivers, lwIP, and both socket families. |
|
||||
| [graphics.md](graphics.md) | Framebuffer, `/dev/fb0`, text rendering, the console, PSOGM and GMMOS. |
|
||||
| [desktop.md](desktop.md) | Xorg on the framebuffer, Window Maker, and what they stand on. |
|
||||
| [packages.md](packages.md) | The apk root: Alpine's packages, and SchOS as one of them. |
|
||||
| [scff.md](scff.md) | The compressed image the live root ships as. |
|
||||
| [terminals.md](terminals.md) | The twelve virtual terminals, ptys, and handing one to a display server. |
|
||||
| [networking.md](networking.md) | NIC drivers, lwIP, and the socket families. |
|
||||
| [audio.md](audio.md) | `/dev/dsp`, AudioEngine, and the AC'97 driver under it. |
|
||||
| [init.md](init.md) | `schinit`, PID 1 as a role, and the job manifests. |
|
||||
| [debugging.md](debugging.md) | Symbols, a log that outlives the kernel, and asking the kernel a question. |
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ until the speaker has caught up*.
|
|||
**It is one stream.** Mixing several programs into one device is a real feature
|
||||
with a real design behind it, and building it before anything had played a
|
||||
single sound would be exactly the mistake [../plan/1.0.md](../plan/1.0.md)
|
||||
keeps a register for: work that exists and is connected to nothing. The second
|
||||
keeps a register against: work that exists and is connected to nothing. The second
|
||||
program that wants to play at the same time is what should cause it to be
|
||||
written.
|
||||
|
||||
|
|
@ -84,6 +84,25 @@ Both register files are **I/O ports**, not MMIO — which is why nothing in the
|
|||
driver maps anything. `bar[n].is_io` is what says so, and a driver that
|
||||
`ioremap`ed these would be mapping a port number as though it were an address.
|
||||
|
||||
### Through ALSA, for programs that expect it
|
||||
|
||||
SDL 3 has no OSS backend and SchOS has no ALSA *kernel* side, so sound from SDL games
|
||||
and anything else that speaks ALSA goes **SDL → alsa-lib → alsa-plugins' OSS plugin →
|
||||
`/dev/dsp`**. `alsa-lib` and `alsa-plugins` are in the Alpine `world`, and
|
||||
`/etc/asound.conf` (from `schos-base`) makes the default device a `plug` onto the OSS
|
||||
device at 16-bit stereo 48 kHz, which is what every SchOS audio driver plays.
|
||||
|
||||
Running that path found two bugs underneath it:
|
||||
|
||||
- **AC'97:** resetting the PCM-out box also clears the descriptor-list address and nothing
|
||||
set it again, so every stream after the first stop played a "list" at physical address 0
|
||||
— silence — and ALSA resets before each stream. If the engine halts at the end of the
|
||||
queue, the run bit is set again.
|
||||
- **`/dev/dsp` was missing the ioctls the plugin's ring-and-pointer model uses:**
|
||||
`SETFRAGMENT`, `GETFMTS`, `GETCAPS`, `SETTRIGGER`, `GETOPTR`, `GETODELAY`, and `poll()`,
|
||||
writable once a whole fragment fits (ALSA spun at 76% CPU otherwise). `speaker-test` now
|
||||
plays in real time at 0% CPU, verified in a recording of QEMU's AC'97.
|
||||
|
||||
## How it is tested
|
||||
|
||||
QEMU's `wav` audio backend writes what the guest plays to a file, so the test
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ because there are none.** Everything the jump needs is computed before.
|
|||
### The hand-off block
|
||||
|
||||
One page of low memory at `0x30000`, defined once in
|
||||
[`include/arch/schboot.h`](../../include/schboot.h) and included by both the loader
|
||||
[`include/arch/schboot.h`](../../include/arch/schboot.h) and included by both the loader
|
||||
and `entry.S` so the two cannot disagree.
|
||||
|
||||
| Offset | What |
|
||||
|
|
@ -131,6 +131,42 @@ when the early console is set up. So a low framebuffer is attached exactly as
|
|||
before, and a high one is attached a few dozen instructions later, immediately
|
||||
after `kvmalloc()` — still thousands of lines before anything else happens.
|
||||
|
||||
### Where the kernel goes, and how it gets there
|
||||
|
||||
The kernel is linked at physical **16 MB** (`KERNPHYS` in `memlayout.h`, repeated in
|
||||
`kernel.ld`, which must agree) and is about 34 MB with the ramdisk inside it. From the
|
||||
old 1 MB it ran across 8 MB, where OVMF keeps ACPI NVS memory that outlives the firmware,
|
||||
so it was overwritten on every boot, after `ExitBootServices`. Firmware leaves its
|
||||
leftovers low; above 16 MB is where a machine is most reliably empty. (The 15 MB below
|
||||
the kernel is not given to the page allocator yet.)
|
||||
|
||||
**Every free page of that range is claimed first.** A segment lying wholly on claimed
|
||||
pages is read straight home; one that is not is read into a buffer elsewhere and
|
||||
**copied home after `ExitBootServices`**, when boot-services memory is ours. An HP
|
||||
all-in-one from 2012 used to freeze at the start of the loading bar because the loader
|
||||
said "could not reserve" and read the kernel there anyway, over the firmware's own memory
|
||||
with the firmware still running on it. Memory under the kernel that outlives the firmware
|
||||
(ACPI NVS, runtime services) is named, and so is the loader or its stack sitting in the
|
||||
range. Segments are read a megabyte at a time, so the bar moves through the ramdisk and
|
||||
shows where a machine stopped.
|
||||
|
||||
The copy and the jump themselves run from **one page below the kernel's lowest address**,
|
||||
allocated as loader code before the firmware exits: a short position-independent routine,
|
||||
the copy list and a stack, switching to the loader's page tables and its own stack before
|
||||
it copies anything. Doing the copy from code the firmware had placed wherever it liked put
|
||||
some of it inside the kernel's range on that machine: a triple fault and a reset. The
|
||||
routine is `sysv_abi`, because UEFI code is the Microsoft convention and the first try loaded
|
||||
the framebuffer address into `rsp`.
|
||||
|
||||
**No 1 GB pages, no problem.** Intel's desktop CPUs before Haswell have none, and there the
|
||||
PS bit in a PDPT entry is reserved: the first access after the switch is a page fault that
|
||||
cannot be handled either. Both the loader and the kernel check CPUID `0x80000001` EDX bit 26;
|
||||
without it the low 4 GB is four page directories of 2 MB pages (placed below the kernel, with
|
||||
the staged segments kept below 4 GB), and the kernel gives each gigabyte of its direct map
|
||||
and of `ioremap()` a page directory of its own. Neither KVM nor QEMU enforces the reserved
|
||||
bit, so the fault could not be reproduced here; with the fallback forced on, the kernel
|
||||
boots at 2 and 8 GB and both CPUs start.
|
||||
|
||||
### The loader jumped out from under itself
|
||||
|
||||
A machine with a lot of RAM froze at "Starting the kernel" — the splash still
|
||||
|
|
@ -610,7 +646,9 @@ in the one process that runs first.
|
|||
|
||||
## What it does not do
|
||||
|
||||
- **No BIOS path.** A machine without UEFI does not boot SchOS.
|
||||
- **No BIOS path.** A machine without UEFI does not boot SchOS. The ISO carries a real EFI
|
||||
system partition (MBR type `0xEF`, FAT16 by cluster count, the type old firmware reads best),
|
||||
so a stick written with `dd` or `write.sh` boots on strict firmware as well as on lenient.
|
||||
- **The quiet boot hides a slow one.** With nothing printing between the
|
||||
loader and the greeting, a boot that stalls for ten seconds looks the same as
|
||||
one that does not. F8 is the answer and it has to be pressed *before* the
|
||||
|
|
@ -621,9 +659,11 @@ in the one process that runs first.
|
|||
with its partitions, but nothing mounts it. See [usb.md](usb.md).
|
||||
- **NVMe and AHCI poll for completion**, one command at a time per disk. That
|
||||
is enough to boot and run from them, not to be fast.
|
||||
- **`make ramroot` is the only way to boot with no data disk.** It links a
|
||||
filesystem image into the kernel binary, which costs boot time and carries a
|
||||
licensing constraint — see [../licensing.md](../licensing.md) §4.
|
||||
- **The live ISO is a ramdisk root.** `make iso` builds the root as a SchFS image,
|
||||
compresses it ([scff.md](scff.md)) and links it into the kernel; nothing is written to
|
||||
the machine's disks, and everything written to the root is gone at reboot. There is
|
||||
no installer, and no kernel that finds a root on a disk by SchFS's GPT type as its
|
||||
*default*. `make ramroot` is the same without compression.
|
||||
- **The 32-bit multiboot entry is dead code.** It is left in place deliberately:
|
||||
deleting kernel entry code is its own piece of work with its own way of
|
||||
failing silently.
|
||||
|
|
|
|||
|
|
@ -179,6 +179,10 @@ part of the boot is blank for the part that is hardest to debug any other way.
|
|||
|
||||
---
|
||||
|
||||
A panic is not the end of the log. GMMOS ([graphics.md](graphics.md)) keeps running
|
||||
under the report, can show this log on screen or as QR codes a phone can read, and
|
||||
has a small terminal for looking around a machine whose kernel is dead.
|
||||
|
||||
## Asking a question
|
||||
|
||||
`/dev/dbg` is a conversation: write a line, read the answer.
|
||||
|
|
@ -493,10 +497,16 @@ something that reports nonsense.
|
|||
- **One conversation at a time.** The answer buffer is global, so two programs
|
||||
asking at once get one answer each in some order — never half of the other's,
|
||||
but not two independent sessions either.
|
||||
- **No userspace symbolization.** The kernel names kernel symbols. A fault in a
|
||||
process still reports a bare address, and resolving it is a host-side job.
|
||||
- **The stack walk stops where the frame pointer stops.** `getcallerpcs()` walks
|
||||
`rbp` chains, so a trap frame in the middle of a chain ends the walk.
|
||||
- **No userspace symbolization.** The kernel names kernel symbols. A process
|
||||
killed by a fault is told which *file and offset* it died in (from
|
||||
`/proc/<pid>/maps`' records), which is enough to resolve with the file's own
|
||||
symbols on the host; a bare address is all there is for anonymous memory.
|
||||
- **The stack walk stops where the frame pointer stops, and it will not follow a
|
||||
bad one.** `getstackpcs()` walks `rbp` chains, and each link must go up the same
|
||||
stack by less than `KSTACKSIZE`: a chain can end at the syscall entry with a *user*
|
||||
`rbp` that a program uses as an ordinary register (`-1024` in one game), and
|
||||
dereferencing that under the syscall tracer once panicked the kernel inside
|
||||
`acquire()`. A trap frame in the middle of a chain ends the walk.
|
||||
- **`sysrq g` costs you log history.** Its output goes through `cprintf`, so
|
||||
sixteen kilobytes of views push the same amount out of a 128 KB ring.
|
||||
- **No tracing.** No static tracepoints, no event ring, no counters. "What
|
||||
|
|
|
|||
74
docs/architecture/desktop.md
Normal file
74
docs/architecture/desktop.md
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
# The desktop: Xorg on the framebuffer
|
||||
|
||||
The desktop is X11, on the framebuffer: Alpine's `xorg-server` with the `fbdev` video
|
||||
driver and the `evdev` input driver, Window Maker as the window manager, and `xterm`.
|
||||
`startx` runs `wmaker`. Nothing here is written by SchOS except the kernel interfaces it
|
||||
stands on and four small files in `tools/posix/mkaroot.sh`.
|
||||
|
||||
## Why X and not Wayland
|
||||
|
||||
Every Wayland compositor worth running — labwc, and everything built on wlroots — needs
|
||||
DRM/KMS, libinput, udev and libseat. DRM/KMS is the largest of them by far: a kernel graphics
|
||||
subsystem, and in practice a great deal taken from Linux. Xorg runs on a plain framebuffer
|
||||
through `fbdev` and reads input through `evdev`, both interfaces SchOS already had. That is
|
||||
the shorter road to a desktop by a long way, and it brings the window managers with it.
|
||||
DRM/KMS is not planned; libinput, udev and libseat are worth having for their own sake and
|
||||
may come later.
|
||||
|
||||
## What it stands on
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **fbdev** | `/dev/fb0` speaks Linux's fbdev ([graphics.md](graphics.md#devfb0-and-how-xorg-draws)) |
|
||||
| **evdev** | `/dev/input/event0` (every keyboard) and `event1` (every mouse and touchpad) with Linux's queries, LED writes and key/button/relative-axis bits |
|
||||
| **VTs** | `/dev/tty0`, `VT_SETMODE(VT_PROCESS)`, `KD_GRAPHICS`, the release/acquire handshake ([terminals.md](terminals.md)) |
|
||||
| **dlopen** | Xorg loads its drivers as modules, so `exec` does dynamic linking ([kernel.md](kernel.md)) |
|
||||
| **sysfs** | `/sys/bus/pci/devices/*/{boot_vga,enable,graphics/fb0}`, which libpciaccess reads |
|
||||
| **ptys** | `xterm` runs its shell on a pseudo-terminal |
|
||||
| **Unix sockets** | the X11 socket, abstract names, `SO_PEERCRED`, `flock` |
|
||||
| **headers** | `<linux/*.h>` in the build sysroot, as `ports/linux-headers` |
|
||||
|
||||
`cmd/xready` checks every one of the kernel calls Xorg makes before it draws anything — the
|
||||
evdev queries, the fbdev calls, the console calls, and the `/sys` files — one at a time, on a
|
||||
terminal of its own, and puts everything back. The calls were written against what the
|
||||
programs are documented to do; each step past it found one that was not, which is what running
|
||||
the real program is for.
|
||||
|
||||
## The four files
|
||||
|
||||
- **`/root/.xserverrc`** starts `X` on the VT `startx` was started from (startx works that
|
||||
out only when `uname -s` says Linux, and SchOS's says SchIFT) with **`-dumbSched`**. Xorg
|
||||
normally reads input on a thread of its own; when SchOS had no threads the server crashed
|
||||
the first time a client disconnected, in `pthread_join` on a thread that was never made, and
|
||||
`-dumbSched` is Xorg's own switch for input on the main loop. Threads exist now; the switch
|
||||
is still on, and nobody has run the server without it since.
|
||||
- **`/root/.xinitrc`** is `exec wmaker`. Without it xinit falls back to twm and an xterm, which
|
||||
are not in the image.
|
||||
- **`/etc/X11/xorg.conf.d/10-schos.conf`** names the input devices. Alpine's Xorg asks udev for
|
||||
them, SchOS has no udev, and X came up with no keyboard and no mouse: the pointer did not
|
||||
move and nothing could end the session. SchOS has exactly two input nodes whatever the
|
||||
machine, so a fixed list is the whole answer (`AutoAddDevices` off, Ctrl+Alt+Backspace
|
||||
enabled).
|
||||
- **`/etc/asound.conf`** is for sound, not X ([audio.md](audio.md)).
|
||||
|
||||
## Using it
|
||||
|
||||
Twelve virtual terminals, switched with Ctrl+Alt+F1–F12 — on the console and under X
|
||||
alike, because X owns the keyboard while it runs and switches only with that chord. `startx` on
|
||||
a terminal runs the desktop there; Ctrl+Alt+Backspace ends the session and gives the terminal
|
||||
back. Window Maker, `xterm`, `xeyes`, `xclock`, `xcalc`, DejaVu fonts; SDL 2 games
|
||||
(`lbreakouthd`) and NetSurf, a GTK 3 browser, run in it.
|
||||
|
||||
Touchpads that sit on an I2C bus (most laptops) are driven by `kernel/acpi/i2c_hid.c`, found
|
||||
through ACPI, and reach X as a mouse: pointer, click and tap.
|
||||
|
||||
## What it does not do
|
||||
|
||||
- **No acceleration.** Software rendering to the framebuffer: no DRM, no glamor, no DRI, no
|
||||
OpenGL beyond what software gives. GPU drivers are a plan ([../plan/gpu.md](../plan/gpu.md)).
|
||||
- **No udev.** Input devices are named in a config file, not found; `/sys` has no `devices`
|
||||
tree and no uevent socket.
|
||||
- **Touchpads are mice**: no two-finger scroll, no gestures, no palm rejection.
|
||||
- **No resolution change from inside X.** The mode is the console's; `schmode` changes the
|
||||
`vgpud` head only, and the loader's menu is where a real panel's mode is chosen.
|
||||
- **Not a security boundary.** X runs as root on a system that enforces no file permissions.
|
||||
|
|
@ -4,7 +4,50 @@ How hardware gets found, claimed and used, and what the rest of the kernel sees
|
|||
of it.
|
||||
|
||||
Writing a driver is [../guides/adding-a-driver.md](../guides/adding-a-driver.md).
|
||||
Moving drivers out of the kernel entirely is [../plan/hybrid.md](../plan/hybrid.md).
|
||||
Drivers that live in a process rather than in the kernel are described below
|
||||
(*Drivers that are not in the kernel*), and the foreign ones in [ldls.md](ldls.md).
|
||||
|
||||
---
|
||||
|
||||
## What stays in the kernel, and what may leave
|
||||
|
||||
"Hybrid" is used to mean everything from "a monolith with a nice device model" to
|
||||
"a microkernel that gave up", so the line is worth drawing exactly.
|
||||
|
||||
**Stays in, permanently:** scheduling, address spaces, the trap and syscall path;
|
||||
IPC and shared memory; the VFS and the **root** filesystem; IRQ demultiplexing; and
|
||||
the console and PSOGM. That last one is not a size compromise but an invariant: the
|
||||
crash screen must work when everything else is dead, so it cannot depend on anything
|
||||
that could *be* dead. A crash screen served by a userspace process is absent in
|
||||
exactly the case it exists for.
|
||||
|
||||
**May leave, and should:** GPU drivers, non-root filesystems, NICs after the first,
|
||||
and anything written by someone else.
|
||||
|
||||
> **Anything needed to reach a shell stays in. Anything discovered at runtime,
|
||||
> replaceable, or plausibly written by someone else goes out.**
|
||||
>
|
||||
> **The kernel keeps a minimal driver for everything on the path to a shell, and
|
||||
> userspace owns the good one.** Break that and the compositor cannot start because
|
||||
> the display driver is a process that needs a compositor.
|
||||
|
||||
### USB stays in the kernel, and why
|
||||
|
||||
It was on the list to leave, and the premise was wrong. USB is not a peripheral; it
|
||||
is the bus peripherals arrive on, structurally the same kind of thing as PCI, and
|
||||
nobody has proposed evicting PCI. The keyboard and the root disk that arrive over it
|
||||
are on the path to a shell, so the stack that finds them is part of that path. Even
|
||||
with a minimal in-kernel fallback written, the good driver and the minimal one would
|
||||
be the same driver, and an eviction that leaves an identical copy behind has moved
|
||||
nothing. A second, quieter difference: `/dev/pci` hands over one PCI *function*, one
|
||||
owner and nothing shared, while a USB device sits behind a controller other devices
|
||||
also sit behind — "the xHCI controller" includes the keyboard. What the eviction was
|
||||
for, third-party drivers living outside the kernel, survives and belongs to
|
||||
[ldls.md](ldls.md).
|
||||
|
||||
The root filesystem stays in the kernel for the same reason (SchFS cannot be
|
||||
restarted, and a bug in it is fatal), and so does the first NIC, because the network
|
||||
thread and lwIP are in the kernel.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -148,7 +191,7 @@ bitmap, and loses them when it releases or exits. Granting it is not optional
|
|||
a driver's `outl` is an instruction compiled into it, not a call anything can
|
||||
intercept — and granting it *coarsely* is not acceptable, because `IOPL=3` is
|
||||
one flag that hands over the interrupt controller and the disk along with the
|
||||
NIC. See [../../include/ioperm.h](../../include/ioperm.h).
|
||||
NIC. See [../../include/arch/ioperm.h](../../include/arch/ioperm.h).
|
||||
|
||||
### Handing a device over
|
||||
|
||||
|
|
@ -184,8 +227,47 @@ client — and that is worth a sentence here because it is the thing the device
|
|||
model makes possible rather than the thing it does. A modeset is five commands
|
||||
to the device. In the kernel those five commands are also a reason to be
|
||||
nervous, because a wrong one takes the machine; in a process the worst case is
|
||||
that `schinit` starts another one. See
|
||||
[../plan/hybrid.md](../plan/hybrid.md), "A modeset, from a process".
|
||||
that `schinit` starts another one. That is why the in-kernel driver never grew
|
||||
one — not difficulty, exposure.
|
||||
|
||||
```
|
||||
# schmode
|
||||
1280x800 current
|
||||
1280x800 maximum, framebuffer 4000 KB
|
||||
# schmode 1024 768
|
||||
1024x768 ok
|
||||
```
|
||||
|
||||
The scanout really resizes, and nothing in the kernel changed for it: `schmode`
|
||||
writes a line to a socket and `vgpud` sends five commands to the device. The
|
||||
framebuffer is allocated **once, at the largest mode**, and every smaller mode
|
||||
reuses it, because `PCIIOC_DMA` has no free — a driver's blocks come back when it
|
||||
exits and not before — so a modeset that allocated would spend the arena and the
|
||||
failure would look like a GPU bug rather than like counting. A larger mode is
|
||||
refused, and says why. The control socket speaks lines of text and has no buffer
|
||||
submission, damage or vblank, which a compositor would need and nothing has asked
|
||||
for.
|
||||
|
||||
### A panic has to reach the screen
|
||||
|
||||
Handing a device away costs something that was not on the list. The kernel panics,
|
||||
paints the report into the console framebuffer, and halts — and the display it gave
|
||||
to a process keeps showing whatever that process last put there. On a machine with
|
||||
two heads, one says the kernel is dead and the other shows a shell prompt. There is
|
||||
no scheduling after a panic, so the process holding the device is never going to run
|
||||
again, and the kernel let go of the device deliberately.
|
||||
|
||||
So the driver says in advance where it is scanning out, with `PCIIOC_PANICFB`
|
||||
(`include/dev/pciuser.h`), and the kernel: (1) paints the report into that memory;
|
||||
(2) asks the *kernel* driver the device was taken from to put it on screen —
|
||||
`panic_present` in `struct pci_driver`; (3) which resets the device and rebuilds one
|
||||
queue from **static** memory — no allocation, no locks, no sleeping, because a
|
||||
corrupted heap is one of the ways a machine gets here. The registration is checked
|
||||
like the promise it is: the range must lie inside a block that process got from
|
||||
`PCIIOC_DMA` on that device, or it would be a way to nominate any physical address
|
||||
to be overwritten at panic. It costs 20 KB of static memory in the virtio-gpu driver,
|
||||
paid whether or not the machine ever panics; allocating it on demand would be
|
||||
cheaper and would not work.
|
||||
|
||||
### Taking it back
|
||||
|
||||
|
|
@ -222,8 +304,9 @@ enforces it.
|
|||
Coming back is a job, not a kernel feature. `etc/schinit/vgpud.job` carries
|
||||
`KeepAlive = yes`, so killing the driver is the entire test: `schinitd` starts
|
||||
another one, it claims the device, and the pid of whichever process drew the
|
||||
frame is legible in the frame itself. See
|
||||
[../plan/hybrid.md](../plan/hybrid.md), "The driver came back".
|
||||
frame is legible in the frame itself: read off the scanout across four generations of
|
||||
the driver it said 4, 8, 10, 12, and the DMA arena handed back the same addresses
|
||||
every time.
|
||||
|
||||
### What this is not
|
||||
|
||||
|
|
@ -231,8 +314,7 @@ frame is legible in the frame itself. See
|
|||
buffer can point the device at any physical address. This isolates a driver
|
||||
against its own *bugs* and not against its own *intent* — a crashed driver that
|
||||
restarts instead of taking the machine down is the entire benefit, and it is
|
||||
worth having on those terms. [../plan/hybrid.md](../plan/hybrid.md) says so at
|
||||
length.
|
||||
worth having on those terms. VT-d/AMD-Vi is its own project.
|
||||
|
||||
**One device per process.** A driver drives one thing; if you need two, run two.
|
||||
|
||||
|
|
@ -240,29 +322,24 @@ length.
|
|||
|
||||
## Linux drivers
|
||||
|
||||
**Decided 2026-08-08.** Linux drivers are not shipped with SchOS and never
|
||||
will be. The user may install one on their own machine, through a device
|
||||
manager or equivalent, and SchOS will load it.
|
||||
[ldls.md](ldls.md) is how SchOS uses them: a Linux `.ko` runs in a process and
|
||||
reaches the hardware through `/dev/pci`. The licensing position, stated exactly
|
||||
because it is easy to get wrong in both directions:
|
||||
|
||||
The licensing reasoning matters and is worth stating exactly, because it is
|
||||
easy to get wrong in both directions:
|
||||
- **Distribution is what triggers the GPL.** SchOS is GPL-2.0-or-later, so shipping a
|
||||
GPL-2.0-only driver in an image is *lawful* — see [../licensing.md](../licensing.md) §6 —
|
||||
and the owner has decided it is allowed (code and firmware blobs). The repository does
|
||||
not carry one all the same: `.gitignore` refuses `*.ko`, and the modules are staged into
|
||||
a *local* image from the build machine's own Linux kernel (`make ldls-stage`,
|
||||
`make ldls-r8169`), where the matching BTF is.
|
||||
- **Implementing a loader is not copying.** Writing support for a module format and
|
||||
a driver ABI from published documentation produces our own work; the layouts come
|
||||
from the kernel's BTF at load time on the user's machine, so the source holds none
|
||||
of Linux's.
|
||||
|
||||
- **Distribution is what triggers the GPL.** SchOS is GPL-2.0-or-later, so
|
||||
shipping a GPL-2.0-only driver in an image is now *possible* — see
|
||||
[../licensing.md](../licensing.md) §6 — but LDLS still does not, by policy. A
|
||||
user obtaining a driver themselves and loading it on their own machine is
|
||||
their act, not ours, and is not a distribution by us.
|
||||
- **Implementing a loader is not copying.** Writing support for a module format
|
||||
and a driver ABI from published documentation produces our own work. It is a
|
||||
clean-room implementation, and it stays one only if it is written from
|
||||
documentation rather than transcribed from Linux source. Reading Linux to
|
||||
understand *hardware* is fine; translating its code is not, whatever the
|
||||
identifiers say.
|
||||
|
||||
So the rule for this subsystem is the same as everywhere else in the tree: the
|
||||
loader is ours and GPL-2.0-or-later, what it loads is the user's business, and
|
||||
no third-party module enters an image we build. `RAMDISK_EXCLUDE` and `NOTPORTS` are the
|
||||
existing mechanisms for keeping that true mechanically rather than by intention.
|
||||
The loader is ours and GPL-2.0-or-later; what it loads is not in the tree.
|
||||
`RAMDISK_EXCLUDE` and `NOTPORTS` are the mechanisms that keep that true mechanically
|
||||
rather than by intention.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -371,12 +448,13 @@ the console's minors to the console and the rest to their own functions. But
|
|||
- **No hot-plug.** Devices are registered at boot and never removed; the
|
||||
`in_use` flag is the hook a hot-unplug path would need, and nothing sets it
|
||||
for that purpose.
|
||||
- **`NDEVICE` is 32**, compile-time, like every other table in the kernel.
|
||||
- **`NDEVICE` is 128**, compile-time, like every other table in the kernel.
|
||||
- **In-kernel drivers are fixed at link time.** There is no module loader, and
|
||||
nothing can be added to the kernel after the build. A driver *outside* the
|
||||
kernel is an ordinary program and can be started whenever — but it reaches its
|
||||
device through `/dev/pci` and publishes nothing to the class vtables, so the
|
||||
console cannot be handed to one yet.
|
||||
kernel is an ordinary program and can be started whenever — it reaches its
|
||||
device through `/dev/pci`, and a NIC driver can publish itself to the network
|
||||
class through `/dev/netdev` (LDLS does); a display driver publishes nothing to
|
||||
`display_ops`, so the console cannot be handed to one.
|
||||
- **The kernel does not rebind its own driver.** `PCIIOC_RELEASE`, or the process
|
||||
exiting, leaves the interrupt line masked and the in-kernel driver unbound —
|
||||
the console stays wherever the release put it. A *supervisor* can start a
|
||||
|
|
|
|||
|
|
@ -199,11 +199,14 @@ provides — reads and writes never reach it, because `fileread`/`filewrite` see
|
|||
It is read-only: `dirlink` and `ialloc` answer `EROFS` rather than pretending.
|
||||
A node exists because the kernel has the device, or it does not exist.
|
||||
|
||||
### What is there
|
||||
### What is there in `/dev`
|
||||
|
||||
`console`, `null`, `keyboard`, `zero`, `full`, `random`, `urandom`, `tty`,
|
||||
`stdin`, `stdout`, `stderr`, `fb0`, `mouse`, `hda`, `hdb`, and `input/event0`
|
||||
and `input/event1` in one subdirectory.
|
||||
`console`, `null`, `zero`, `full`, `random`, `urandom`, `tty`, `tty0` and
|
||||
`tty1`..`tty12`, `stdin`, `stdout`, `stderr`, `keyboard`, `mouse`, `fb0`, `dsp`,
|
||||
`ptmx` and a `pts/` directory of pseudo-terminal slaves, `hda`, `hdb`,
|
||||
`input/event0`.., and the SchOS-specific `pci` (a device's registers, DMA and
|
||||
interrupts, for a driver in a process), `netdev` (a network interface for a
|
||||
process), `dbg` and `dbgro` (the kernel debugger's views).
|
||||
|
||||
Every disk and partition the block layer registered is there too, under the
|
||||
name it was given (`sata0`, `sata0p2`, `nvme0p1`, `usb0p1`): major 8, minor its
|
||||
|
|
@ -218,11 +221,18 @@ nothing behind it fails with `ENXIO`, which is a better answer than a missing
|
|||
path — and with no node at all, `echo x > /dev/hdb` silently creates an ordinary
|
||||
file of that name and appears to work.
|
||||
|
||||
### What is there
|
||||
### What is there in `/proc`
|
||||
|
||||
Per-process directories, plus the global files: `meminfo`, `uptime`, `cpuinfo`,
|
||||
`version`, `stat`, `loadavg`, `mounts`, `filesystems`, `schfs`, `pci`,
|
||||
`interrupts`, `devices`, `netif`, and `sysrq`.
|
||||
`/proc/self`, and one directory per process holding `stat`, `status`, `cmdline`,
|
||||
`statm`, `comm`, `schedstat`, `maps`, `exe` (a symlink), `fd/` (a symlink per open
|
||||
descriptor) and `task/` (one directory per thread). `maps` lists the file
|
||||
mappings, and a process killed by a fault is told which file and offset it died
|
||||
in.
|
||||
|
||||
The global files: `meminfo`, `uptime`, `cpuinfo`, `version`, `stat`, `loadavg`,
|
||||
`mounts`, `filesystems`, `partitions`, `schfs`, `pci`, `interrupts`, `devices`,
|
||||
`netif`, `resolv.conf`, `slab`, `fsstat`, `kmsg`, `strace`, `bootmode`,
|
||||
`bootdebug`, `quiet`, `verbose`, `schinit` and `sysrq`.
|
||||
|
||||
**/proc is read-only except `sysrq`**, which is the one deliberate write target,
|
||||
and the code says so in one place. Marking every procfs inode read-only once
|
||||
|
|
@ -249,6 +259,8 @@ mounted by init with `mount("sysfs", "/sys", "sysfs", 0, 0)`.
|
|||
| `/sys/firmware/memmap/N/` | `start end type`, the firmware's map as the loader handed it over |
|
||||
| `/sys/firmware/efi/` | present when UEFI booted the kernel; empty |
|
||||
| `/sys/class/dmi/id/` | `sys_vendor product_name product_version board_name bios_version`, and the rest Linux has, except serials and the UUID |
|
||||
| `/sys/class/power_supply/` | `BAT0` (`capacity`, `status`, `energy_*`/`charge_*`, `voltage_now`, `cycle_count`, ...) and `AC` (`online`), from ACPI — the battery and the adapter are read through the laptop's embedded controller |
|
||||
| `/sys/bus/pci/devices/…/` | also `boot_vga`, `enable` and `graphics/fb0`, which libpciaccess (and so Xorg) reads |
|
||||
|
||||
**Linux's layout because everything that reads hardware reads it from there.**
|
||||
toybox's `lspci` and `lsusb` walk `/sys/bus/*/devices` and parse each `uevent`
|
||||
|
|
@ -265,7 +277,8 @@ tables, `config`) report their real length, since tools copy them by it.
|
|||
|
||||
What is missing, compared with Linux: `/sys/devices` and its symlinks (the
|
||||
`bus/*/devices` entries are directories rather than links into it),
|
||||
`/sys/class` beyond `dmi`, `/sys/block`, and anything writable. PCI `config` is
|
||||
`/sys/class` beyond `dmi` and `power_supply`, `/sys/block`, a uevent socket (so
|
||||
no udev), and anything writable. PCI `config` is
|
||||
256 bytes, because config space is read through port `0xCF8`, which reaches
|
||||
no further.
|
||||
|
||||
|
|
@ -376,8 +389,6 @@ through.
|
|||
- **No page cache.** `mmap` of a file reads the whole thing in at map time, and
|
||||
writable `MAP_SHARED` of a file is refused rather than faked. This is the
|
||||
single largest gap in the layer.
|
||||
- **No symlinks in procfs**, which is why `/proc/<pid>/exe` is a regular file
|
||||
and `ps -o comm` prints `[init]`.
|
||||
- **`rename()` does not move directories**, and will not replace one. `EISDIR`
|
||||
either way.
|
||||
- **No named pipes.** `mkfifo()` fails with `EPERM`, Linux's answer for a
|
||||
|
|
@ -385,19 +396,13 @@ through.
|
|||
device, because `mknod()` read a Linux program's `(mode, dev)` as the native
|
||||
`(major, minor)`: `ls -l` showed `crw-rw-rw- 438, 0` where a `p` belongs. The
|
||||
zsh port was configured with `zsh_cv_sys_fifo=yes` on the strength of that.
|
||||
- **No `/dev/fd`, and no `/proc/self`.** A process cannot reach its own open
|
||||
descriptors by path. `<(cmd)` needs one or the other — zsh now knows it has
|
||||
neither and says so.
|
||||
- **No `AF_UNIX` socket inodes.** Socket names live in a table in
|
||||
`kernel/net/unix.c`, so `bind()` creates nothing that can be `stat()`ed or
|
||||
`unlink()`ed.
|
||||
- **No `/sys/devices` and no uevents**, so nothing can be told that a device
|
||||
appeared; Xorg is given its input devices by a config file instead.
|
||||
- **No unified inode cache across filesystems** beyond the single `itable`.
|
||||
- **A known, un-root-caused hang:** heavy `ls -lR` use across many commands in
|
||||
one long shell session can eventually make a *later*, unrelated directory
|
||||
hang; the identical command succeeds on a fresh boot. It is not inode or file
|
||||
table exhaustion — both fail cleanly and neither fired. It smells like a lock
|
||||
or refcount leak in the `*at()` paths or in toybox's own fd handling. The next
|
||||
step is instrumenting `itable`/`ftable` occupancy across a long session to see
|
||||
what is climbing.
|
||||
- **`sys_delete`/`sys_show`/`sys_showstat`** are orphaned entries from the old
|
||||
v1 filesystem's recycle bin. Nothing reaches them and they die with `fs.c`.
|
||||
- **One long-standing hang was never found.** Heavy `ls -lR` use across many
|
||||
commands in one long shell session once made a *later*, unrelated directory
|
||||
hang; the identical command succeeded on a fresh boot. It was instrumented and
|
||||
hunted and could not be reproduced, which is recorded in
|
||||
[../history/kernel.md](../history/kernel.md) rather than forgotten.
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
# Graphics, text and the console
|
||||
|
||||
Four things share this area and they are deliberately separate: the kernel's own
|
||||
drawing surface, the compositor's display interface, the text-rendering layer,
|
||||
and the crash screen.
|
||||
Five things share this area and they are deliberately separate: the kernel's own
|
||||
drawing surface, the `display` class a display server could use, the text-rendering
|
||||
layer, the crash screen — and `/dev/fb0`, which is how Xorg reaches the screen.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -11,13 +11,13 @@ and the crash screen.
|
|||
This distinction causes confusion every time somebody meets it, so it is the
|
||||
first thing here.
|
||||
|
||||
| | [`gfx.h`](../../include/gfx.h) | [`display.h`](../../include/display.h) |
|
||||
| | [`gfx.h`](../../include/gfx/gfx.h) | [`display.h`](../../include/gfx/display.h) |
|
||||
|---|---|---|
|
||||
| **What** | The kernel's own drawing surface | The compositor's interface |
|
||||
| **What** | The kernel's own drawing surface | A frame interface for a display server |
|
||||
| **About** | Pixels and text | *Frames* |
|
||||
| **Operations** | `draw_pixel`, `fill_rect`, `putc`, `text_at` | `acquire`, `present`, `wait_vblank` |
|
||||
| **Buffering** | None. Direct writes to the scanout. | Double-buffered |
|
||||
| **Users** | The console and PSOGM | Schcom |
|
||||
| **Users** | The console, PSOGM, GMMOS | None today — Xorg draws through `/dev/fb0` |
|
||||
| **Must work when** | **Nothing else in the system does** | Userspace is healthy |
|
||||
|
||||
That last row is the reason they are not one interface. The console and the
|
||||
|
|
@ -67,7 +67,7 @@ and `vgpud` mirroring it see what is on the screen.
|
|||
`struct display_ops`, found with `device_find_class(DEVCLASS_DISPLAY)`.
|
||||
|
||||
`acquire()` allocates the back buffer **on first call, never at boot** — it is
|
||||
around two megabytes at 800×600, and a machine whose compositor never starts
|
||||
around two megabytes at 800×600, and a machine whose display server never starts
|
||||
should not pay that. A driver that cannot double-buffer returns 0, and the
|
||||
caller draws directly and accepts tearing rather than not drawing at all.
|
||||
`wait_vblank` is optional for the same reason.
|
||||
|
|
@ -133,7 +133,7 @@ A machine with both a stdvga and a virtio-gpu has two displays, and `vgpud` —
|
|||
the out-of-kernel driver, `cmd/vgpud/main.c` — owns the second one. For a long
|
||||
time it drew a test card there: colour bands, gradients, a checkerboard, and
|
||||
its own pid in binary along the bottom. That was scaffolding for
|
||||
[../plan/hybrid.md](../plan/hybrid.md)'s H2 and H3, where a screenshot had to
|
||||
the out-of-kernel driver work ([devices.md](devices.md)), where a screenshot had to
|
||||
answer *did a userspace process put this here* and *did the driver come back*.
|
||||
|
||||
Both of those are closed, so the test card became the Shift mark. Which was
|
||||
|
|
@ -223,9 +223,8 @@ shows the top-left corner, which is where a terminal puts everything worth
|
|||
seeing. Scaling is the alternative and it is the wrong one: text resampled to a
|
||||
non-integer ratio is text you cannot read.
|
||||
|
||||
**This is not the endpoint.** A compositor (Gate 3) will want this head for
|
||||
itself, and when there is one, the mirror goes away. Until then the honest
|
||||
thing for a working display to show is the system running on it.
|
||||
A display server that wants the head for itself can have it; until one does, the
|
||||
honest thing for a working display to show is the system running on it.
|
||||
|
||||
### The F8 resolution is the head's resolution
|
||||
|
||||
|
|
@ -263,9 +262,26 @@ not apply: two `vgpud`s cannot coexist, because the first holds the PCI claim
|
|||
and the second is refused the device long before it reaches `bind()`. The claim
|
||||
is the mutual exclusion; the socket node never was.
|
||||
|
||||
## /dev/fb0, and how Xorg draws
|
||||
|
||||
Xorg runs on the framebuffer through its `fbdev` driver, so `/dev/fb0` speaks Linux's
|
||||
fbdev: `FBIOGET_FSCREENINFO`, `FBIOGET_VSCREENINFO`, `FBIOPUT_VSCREENINFO` (accepted
|
||||
when it asks for the mode already in force), `FBIOPAN_DISPLAY`, `FBIOBLANK`,
|
||||
`FBIOPUTCMAP`, and `mmap` of the physical framebuffer. `FBIOGET_VSCREENINFO` reports
|
||||
nominal timings, as Linux's `efifb` does, instead of zeros — a zero pixel clock made
|
||||
fbdev compute a 0 Hz RandR mode. Pixel layout comes from the same table the kernel
|
||||
draws with, so a colour a program builds is the colour the kernel would.
|
||||
|
||||
Once a program maps the framebuffer the kernel has no way to know when it draws, so
|
||||
`fb_mapped()` marks the screen as no longer the text console's and `fbclose()` puts it
|
||||
back. A terminal switched to `KD_GRAPHICS` is not repainted ([terminals.md](terminals.md)),
|
||||
and the `VT_PROCESS` handshake tells the server before it loses the screen. `cmd/xready`
|
||||
checks every one of these calls before Xorg is started; the picture itself is the proof
|
||||
beyond that.
|
||||
|
||||
## Text
|
||||
|
||||
[`include/gfx/text.h`](../../include/text.h) + `kernel/dev/text.c`.
|
||||
[`include/gfx/text.h`](../../include/gfx/text.h) + `kernel/dev/text.c`.
|
||||
|
||||
**The font was never the problem — the character-cell grid was.** The console
|
||||
draws text by indexing `font_data[256][16]` and stepping the cursor eight
|
||||
|
|
@ -537,24 +553,63 @@ channel that works when the machine will not come back at all.
|
|||
machine whose network stack is dead**, and a phone camera is an out-of-band
|
||||
channel needing no driver on our side.
|
||||
|
||||
It encodes the **report itself**, not a URL — it reaches nothing. Deliberately
|
||||
small in three ways: byte mode only, EC level L only (the screen is a clean,
|
||||
evenly lit, high-contrast source; the damage QR's stronger levels protect
|
||||
against does not apply), and versions 1–6 only, stopping short of version 7's
|
||||
extra version-information blocks and version 10's wider count field.
|
||||
It encodes the **report itself**, not a URL — it reaches nothing. Byte mode only
|
||||
and EC level L only (the screen is a clean, evenly lit, high-contrast source; the
|
||||
damage QR's stronger levels protect against does not apply), but **every version,
|
||||
1 to 40**: version 40 at level L holds 2,953 bytes, which is what lets GMMOS put the
|
||||
kernel log on screen as a series of codes. Checked against the `segno` library module
|
||||
for module at every version.
|
||||
|
||||
134 bytes is a large crash report. If a payload ever needs more, the honest fix
|
||||
is to shorten the report.
|
||||
It draws on the report unconditionally rather than on a keypress, which is better:
|
||||
it costs nothing, and requiring input would mean the one case where input is least
|
||||
trustworthy is also the case where the report is hardest to retrieve. Verified by
|
||||
`make crashshot && make crashqr`, which decodes the symbol out of the captured
|
||||
framebuffer.
|
||||
|
||||
It draws unconditionally rather than on a keypress, which is better: it costs
|
||||
nothing, and requiring input would mean the one case where input is least
|
||||
trustworthy is also the case where the report is hardest to retrieve. Verified
|
||||
by `make crashshot && make crashqr`, which decodes the symbol out of the
|
||||
captured framebuffer.
|
||||
### Every processor stops
|
||||
|
||||
A panic used to stop only the processor it happened on; the rest carried on with a
|
||||
dead kernel, drew over the report, and kept the fans loud until somebody held the power
|
||||
button. The panicking processor now sends every other one an **NMI**, which arrives even
|
||||
with interrupts off, and `trap()` halts a processor that takes one while `panic_stop` is
|
||||
set — first thing, before the canary check, because that stack may be what broke. NMI has
|
||||
its own per-CPU stack (IST 1): for a few instructions at the top of `syscall_entry` the
|
||||
live stack is the user's, at ring 0, and a fault pushing the frame there would be a double
|
||||
fault on the same stack and a reset instead of a report. A second processor that panics
|
||||
while one is reporting halts.
|
||||
|
||||
## GMMOS — the Guru Meditation Micro OS
|
||||
|
||||
The report used to be the last thing the kernel did: a picture, blinking keyboard lights,
|
||||
and a machine that needed its power button. GMMOS keeps the report and goes on living
|
||||
under it (`kernel/gmmos/`, about 2,000 lines). The real kernel stays dead — nothing of it is
|
||||
called again that takes a lock, sleeps, allocates or waits for an interrupt. GMMOS has its
|
||||
own: the framebuffer, a clock read from the TSC, and keyboards it polls itself through
|
||||
**micro drivers**, cut-down copies of the real ones that do only what a keyboard needs.
|
||||
|
||||
| Key | |
|
||||
|---|---|
|
||||
| (nothing) | a status line counts down **three minutes** to power-off; any key cancels it |
|
||||
| **Space** | the last 300 lines of this boot's log as **QR codes**, a page per symbol, numbered so a phone can take them in order; `+` / `-` trade module size for capacity |
|
||||
| **L** | the log on screen, scrolled with the arrows |
|
||||
| **T** | a terminal: `why`, `log`, `grep`, `ps`, `cpus`, `pci`, `mem` (page tables walked first, so a bad address cannot fault), `uptime`, `restart`, `poweroff` |
|
||||
| **R R** / **P P** | restart / power off — twice, so a stray key does neither |
|
||||
|
||||
The micro drivers are **PS/2**, **I2C-HID** (the laptop's own keyboard, over a DesignWare
|
||||
controller — the live driver hands over what it found at boot, because AML cannot run on a
|
||||
dead kernel) and **xHCI** (USB keyboards: every controller is halted, reset and enumerated
|
||||
from nothing, root ports and USB 2 hubs up to two deep, a boot-protocol keyboard, no
|
||||
interrupts, a static 256 KB pool). GMMOS probes all of them and polls every one that answers.
|
||||
|
||||
Every buffer is static. A panic inside GMMOS, or a second one in the report, is reported
|
||||
and then only blinks; GMMOS does not re-enter itself. Checked under QEMU: every screen, two
|
||||
QR pages read off a screenshot with `zbar` and decoded to the log text exactly, restart
|
||||
reboots, and left alone it powers off at three minutes. What has been seen on real hardware
|
||||
is the report and the keyboards the laptops have; the xHCI path is the least exercised.
|
||||
|
||||
## One palette
|
||||
|
||||
[`include/gfx/theme.h`](../../include/theme.h). There were three unrelated designs
|
||||
[`include/gfx/theme.h`](../../include/gfx/theme.h). There were three unrelated designs
|
||||
and the seams showed: the splash was deep purple with gold text deliberately
|
||||
matching the *panic* screen, the console was cream with near-black text, and so
|
||||
a healthy boot was dressed as a crash and then abruptly became an amber
|
||||
|
|
@ -571,16 +626,20 @@ makes it mean something.
|
|||
|
||||
## What it does not do
|
||||
|
||||
- **The console still has its cell grid.** Cells are now a Unicode codepoint
|
||||
drawn by the TrueType rasteriser rather than a CP437 byte drawn from a bitmap,
|
||||
and the cell size follows the font's measured advance — but it is still a
|
||||
fixed grid, and the text layer's proportional path is unused by it.
|
||||
- **No mode setting.** Resolution is whatever the loader asked for. There is no
|
||||
`/dev/dri`-style device and no way to change it at runtime.
|
||||
- **No double buffering in practice.** `display_ops.acquire` is specified and
|
||||
the compositor needs it; virtio-gpu binds but exposes no modeset API, and
|
||||
- **The console still has its cell grid.** Cells are a Unicode codepoint drawn by the
|
||||
TrueType rasteriser, and the cell size follows the font's measured advance (the font
|
||||
scales to about 45 rows, 16 to 40 px per em, on whatever mode the firmware booted
|
||||
in) — but it is a fixed grid, and the text layer's proportional path is unused by it.
|
||||
- **Mode setting is the loader's and `vgpud`'s.** On real hardware resolution is whatever
|
||||
the loader chose from its menu — which now asks "Keep this resolution?" and falls back
|
||||
by itself, because AMD's GOP "sets" smaller modes without leaving the panel's native
|
||||
timing and the screen showed three or four squashed copies side by side. There is no
|
||||
`/dev/dri`, no KMS, and Xorg runs at the mode the console is in.
|
||||
- **No double buffering in practice.** `display_ops.acquire` is specified and nothing
|
||||
uses it; virtio-gpu exposes a modeset API on `vgpud`'s socket and nothing more, and
|
||||
`wait_vblank` is unimplemented everywhere.
|
||||
- **No hardware acceleration of any kind**, and none is planned before 2D blit.
|
||||
- **No hardware acceleration of any kind**: Xorg draws in software to the framebuffer
|
||||
(no glamor, no DRI), and GPU drivers are a plan, not a feature ([../plan/gpu.md](../plan/gpu.md)).
|
||||
- **No CFF/Type 2 charstrings**, so OTF fonts do not render — that is a separate
|
||||
`font_ops` provider and a bytecode interpreter, not an extension of the `glyf`
|
||||
path. TrueType hinting is deliberately out of scope.
|
||||
|
|
@ -593,6 +652,6 @@ makes it mean something.
|
|||
- **No composite point-matching.** Components positioned by matching a point in
|
||||
the base rather than by an offset are placed at the origin. Fonts that use it
|
||||
are rare and none in the tree do.
|
||||
- **PSOGM renders and stops.** The interactive crash mode described in
|
||||
[../plan/roadmap.md](../plan/roadmap.md) — walk the stack, read another
|
||||
process's memory, dump page tables — is designed and not built.
|
||||
- **GMMOS cannot look at a live process by name.** The "interactive crash mode" is built as far
|
||||
as a dead machine allows — log, a terminal, `ps`, `cpus`, `pci`, memory by physical walk — but
|
||||
the scheduler is gone, so there is no running a debugger against a process.
|
||||
|
|
|
|||
|
|
@ -11,18 +11,27 @@ driver or a protocol stack: memory, processes, scheduling, traps and signals.
|
|||
One page table per process; the kernel is mapped into all of them at
|
||||
`KERNBASE = 0xFFFF800000000000`, so a trap does not switch `CR3`.
|
||||
|
||||
`kvmalloc()` maps physical 0–4 GB as four 1 GB pages, with the top one marked
|
||||
uncached — that is where PCI BARs and the framebuffer live. This is why 3 GB is
|
||||
the most memory the kernel can treat as RAM, and it is a deliberate simplicity:
|
||||
a direct map with a fixed offset makes `P2V`/`V2P` arithmetic rather than a
|
||||
lookup.
|
||||
`kvmalloc()` maps as many 1 GB pages of physical memory as the machine has RAM
|
||||
(up to `PHYSTOP_MAX`, 256 GB), always at least the low four. The 3–4 GB slot stays
|
||||
uncached because that is where the PCI hole is; everything above 4 GB is RAM, so
|
||||
those are cached, and a device BAR up there still gets its own uncached window
|
||||
from `ioremap()`, which takes the entries the direct map did not. On a CPU
|
||||
without 1 GB pages (Intel's desktop parts before Haswell), each gigabyte is a
|
||||
page directory of 2 MB pages instead — the same reach, one extra page of table
|
||||
per gigabyte. A direct map with a fixed offset makes `P2V`/`V2P` arithmetic
|
||||
rather than a lookup.
|
||||
|
||||
`kalloc()` hands out one 4 KB page at a time from a free list. **There is no
|
||||
slab or pool allocator**, and the consequence is visible throughout the tree:
|
||||
every subsystem that needs many small objects has invented its own fixed-size
|
||||
table — `NPROC` processes, `NSOCK` sockets, `NDEVICE` devices, `NSHM` segments.
|
||||
That is why the limits are compile-time constants rather than a design decision
|
||||
about each one.
|
||||
How much RAM there is comes from the firmware's E820/UEFI map, as a *list* of
|
||||
usable regions, not one run ending at a top address (OVMF drops an ACPI NVS
|
||||
region at 8 MB, which used to hide everything above it). The kernel itself is
|
||||
linked at 16 MB, clear of what firmware keeps low. `kalloc` holds never-used
|
||||
memory as ranges, so a large machine costs nothing until its memory is used.
|
||||
|
||||
`kalloc()` hands out one 4 KB page at a time from a free list. Objects smaller
|
||||
than a page come from the slab caches below; objects larger than one do not have
|
||||
a home yet, so the biggest tables (`NPROC` processes, `NSOCK` sockets, `NDEVICE`
|
||||
devices, `NSHM` segments) are still fixed-size arrays, which is why their limits
|
||||
are compile-time constants rather than a design decision about each one.
|
||||
|
||||
The free list carries a canary, checked at the top of both `kalloc()` and
|
||||
`kfree()`, so a wild write into it panics at the point it is touched rather than
|
||||
|
|
@ -150,8 +159,8 @@ pages, reference counted, mapped through a VMA with `no_free` set so that
|
|||
`memfd_create(2)` is the same machinery reached by **descriptor** instead of by
|
||||
key. That difference is the whole point and the reason it exists on Linux at
|
||||
all: two processes share memory by passing the fd over a socket, so neither has
|
||||
to agree on a path or race to invent a key. It is what a Wayland buffer crosses
|
||||
on, and it is why descriptor passing (`SCM_RIGHTS`) had to be built first.
|
||||
to agree on a path or race to invent a key. It is how a display server and its client
|
||||
share a buffer, and it is why descriptor passing (`SCM_RIGHTS`) had to be built first.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
|
|
@ -171,10 +180,46 @@ version of something this kernel cannot do:
|
|||
mapper. Here a freed page would simply stay mapped to memory somebody else
|
||||
now owns.
|
||||
|
||||
Writable `MAP_SHARED` of a **real file** is still `ENODEV`. That needs a page
|
||||
cache, it is a separate project, and nothing in 1.0 asks for it — an anonymous
|
||||
object has no such contract to break, which is exactly why `memfd` is the
|
||||
narrow answer.
|
||||
Writable `MAP_SHARED` of a **real file** is still refused. That needs a page
|
||||
cache, it is a separate project, and nothing that runs today asks for it — an
|
||||
anonymous object has no such contract to break, which is exactly why `memfd` is
|
||||
the narrow answer.
|
||||
|
||||
### Demand paging
|
||||
|
||||
Anonymous memory can be recorded without being backed: the page appears,
|
||||
zeroed, the first time it is touched — by the program, through `trap()`, or by
|
||||
the kernel on its behalf, through `uva2ka()`. `mmap` does this for mappings of
|
||||
1 MB or more, for `MAP_NORESERVE`, and for `PROT_NONE`. The reason is WebKit,
|
||||
which reserves 4 GB of address space at start for its JIT and heaps and touches
|
||||
a sliver of it; backing all of it at `mmap` time failed with `ENOMEM` and the
|
||||
browser aborted. Go, Java and Firefox reserve the same way.
|
||||
|
||||
`vm_fault()` takes a spinlock and nothing that sleeps, so `uva2ka()` may call it
|
||||
from anywhere; the lock keeps two threads of one process from both mapping the
|
||||
same page. `fork` and `munmap` skip the holes, so a page nobody touched is one
|
||||
they never see, and a child that touches it reads zeroes — what it would have
|
||||
read in the parent.
|
||||
|
||||
File-backed mappings are **not** lazy: `mmap` of a file still reads the range in
|
||||
at map time.
|
||||
|
||||
### TLB shootdowns
|
||||
|
||||
Unmapping from an address space that threads on other processors are running in
|
||||
interrupts those processors and waits until each has reloaded `CR3`
|
||||
(`tlb_shootdown()` in `vm.c`; the vector is one of the MSI lines, which `trap()`
|
||||
already dispatches with an EOI). Without it, when one thread unmapped a range and
|
||||
`mmap` handed the same addresses straight back, a thread on another CPU went on
|
||||
reading and writing the *old* pages through its TLB: stores vanished and loads
|
||||
were stale. musl's allocator noticed first — a group's metadata no longer
|
||||
pointing back at it — and Epiphany, forty threads on four processors, died at a
|
||||
different address on every run.
|
||||
|
||||
Only one shootdown runs at a time, and it is taken without disabling interrupts:
|
||||
two processors each waiting with interrupts off for the other to answer would
|
||||
wait for ever. On a single processor, or if no vector could be allocated, it
|
||||
does nothing (and says so at boot).
|
||||
|
||||
### What the fixed tables actually cost
|
||||
|
||||
|
|
@ -184,31 +229,69 @@ partly true:
|
|||
|
||||
| | size | table | |
|
||||
|---|---|---|---|
|
||||
| `struct proc` | 11744 | `NPROC` 128 | larger than a page; 1 KB of it is FPU state |
|
||||
| `struct socket` | 4936 | `NSOCK` 128 | larger than a page |
|
||||
| `struct proc` | 1984 | `NPROC` 128 | 254 KB in all; the address space, descriptors and signal handlers live in their own objects now, shared by threads |
|
||||
| `struct mm` | 8232 | one per process, 128 | 1 MB in all, almost all of it the 256-entry VMA table |
|
||||
| `struct socket` | about 4900 | `NSOCK` 128 | larger than a page |
|
||||
| `struct device` | 120 | `NDEVICE` 128 | 15 KB, never hit |
|
||||
| `struct buf` | 200 | `NBUF` 256 to start | grows from a slab, data from `kalloc`, up to RAM/32 |
|
||||
| `struct inode` | 296 | none | grows from a slab; `NINODE` 200 is only how many are made up front |
|
||||
| `struct file` | 64 | none | grows from a slab; `NFILE` 512 is only the static pool |
|
||||
| `struct inode` | about 460 | none | grows from a slab; `NINODE` 200 is only how many are made up front |
|
||||
| `struct file` | about 88 | none | grows from a slab; `NFILE` 512 is only the static pool |
|
||||
|
||||
The two largest cannot go in a page-backed slab at all. Freeing them needs a
|
||||
contiguous multi-page allocator, which is a separate thing and is not written.
|
||||
The buffer header shrank to 200 bytes when its 4 KB of data moved out to a page
|
||||
of its own, which is what let the cache grow at all.
|
||||
(Sizes from `nm -S` on a build of the tree, not from a header comment.)
|
||||
|
||||
`struct proc` shrank by a factor of six when threads arrived, because the page
|
||||
tables, VMAs, descriptor table and signal dispositions moved out of it into
|
||||
objects a thread group shares. `struct socket` is still too large for a
|
||||
page-backed slab; freeing it needs a contiguous multi-page allocator, which is a
|
||||
separate thing and is not written. The buffer header shrank to 200 bytes when its
|
||||
4 KB of data moved out to a page of its own, which is what let the cache grow at
|
||||
all.
|
||||
|
||||
## Processes
|
||||
|
||||
`struct proc` in [`include/core/proc.h`](../../include/proc.h). A fixed table of
|
||||
`NPROC` entries, allocated by linear scan.
|
||||
`struct proc` in [`include/core/proc.h`](../../include/core/proc.h). A fixed table of
|
||||
`NPROC` entries, allocated by linear scan. A thread is a `struct proc` too.
|
||||
|
||||
**The field order at the top of the struct is fixed by assembly.** `trapasm.S`
|
||||
hard-codes `fs_base` at offset `0x18`, and there is a `_Static_assert` in
|
||||
`proc.c` that fails the build if it moves. Anything new goes at the end.
|
||||
|
||||
Each process carries: page directory and size, kernel stack, open file table,
|
||||
cwd inode *and* cwd path string, signal handlers and masks, VMA table for
|
||||
`mmap`, POSIX identity (pgid, sid, and all six of real/effective/saved uid and
|
||||
gid), and the Linux-ABI syscall map.
|
||||
Each process carries: a kernel stack, cwd inode *and* cwd path string, signal
|
||||
masks, POSIX identity (pgid, sid, and all six of real/effective/saved uid and
|
||||
gid), the Linux-ABI syscall map, and its x87/SSE registers (saved on a switch;
|
||||
the kernel itself stays off them). Three things it *points to* rather than
|
||||
owns, because a thread group shares them:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `struct mm` | the address space: page tables, heap top, the VMA table, which file each span came from, and a lock so one thread at a time changes the mappings |
|
||||
| `struct files` | the descriptor table and its `FD_CLOEXEC` flags |
|
||||
| `struct sighand` | signal dispositions — what `sigaction()` sets |
|
||||
|
||||
Each is reference counted. `clone()` shares what its flags say and `fork()`
|
||||
copies all three.
|
||||
|
||||
### Threads
|
||||
|
||||
`clone()` takes `pthread_create`'s flags (`CLONE_VM`, `FILES`, `SIGHAND`,
|
||||
`THREAD`, `SETTLS`, `PARENT_SETTID`, `CHILD_CLEARTID`) and `posix_spawn`'s
|
||||
(`VM`, `VFORK`, a stack). What follows from that:
|
||||
|
||||
- **futex** — `WAIT`, `WAIT_BITSET` with timeouts, `WAKE`, `REQUEUE`,
|
||||
`CMP_REQUEUE`, keyed by the word's *kernel* address so processes that map the
|
||||
same page meet. `set_tid_address` keeps the word that exit zeroes and wakes,
|
||||
which is what `pthread_join` waits on.
|
||||
- **Exit.** `exit` ends the calling thread; `exit_group`, a fatal signal, or the
|
||||
leader exiting ends them all. `reapd` frees exited threads, and the leader is
|
||||
reaped by its parent once the rest are gone. `getpid` is the group, `gettid`
|
||||
the thread.
|
||||
- **Signals.** Stops and resumes reach every thread; other process-directed
|
||||
signals go to the leader.
|
||||
- **`exec`** kills the other threads and unshares the dispositions.
|
||||
- **`/proc`** lists processes, `task/` lists threads; `comm`, `Threads:` and
|
||||
`Tgid` are there.
|
||||
|
||||
The first program to need all of this was GTK's, then WebKit's.
|
||||
|
||||
### Slot reuse
|
||||
|
||||
|
|
@ -232,12 +315,15 @@ exactly as a syscall would. The network thread is the current user.
|
|||
|
||||
## Scheduling
|
||||
|
||||
Round-robin over the process table, one scheduler per CPU, each with its own
|
||||
context. `swtch()` saves callee-saved registers and switches stacks; `sched()`
|
||||
returns to the scheduler and asserts four invariants on the way — holding
|
||||
`ptable.lock`, `ncli == 1`, not `RUNNING`, interrupts off.
|
||||
Fair share by virtual runtime, one scheduler per CPU, each with its own context
|
||||
— the rule, the guards and `nice` are in [scheduler.md](scheduler.md).
|
||||
`swtch()` saves callee-saved registers and switches stacks; `sched()` returns to
|
||||
the scheduler and asserts four invariants on the way — holding `ptable.lock`,
|
||||
`ncli == 1`, not `RUNNING`, interrupts off.
|
||||
|
||||
When a full pass finds nothing runnable the CPU halts rather than spinning.
|
||||
When nothing is runnable the CPU goes idle rather than spinning: into ACPI's
|
||||
`_CST` C2 state (CC6 on Zen), by port read or `MWAIT`, where the APIC timer keeps
|
||||
running through it, and otherwise a plain `hlt`.
|
||||
|
||||
**Locking is a single `ptable.lock` over the whole table.** Not per-process: a
|
||||
scheduling operation touches two processes (the one leaving and the one
|
||||
|
|
@ -309,9 +395,23 @@ mksh's job control could not make progress until they existed.
|
|||
|
||||
## What userspace gets
|
||||
|
||||
`fork`, `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and real pid selection,
|
||||
pipes, signals, `mmap` (anonymous and file-backed), POSIX shared memory, Unix
|
||||
domain sockets, `poll`/`select`/`epoll`, `eventfd`, `timerfd`, `signalfd`.
|
||||
`fork`, `clone` (threads), `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and
|
||||
real pid selection, pipes, signals, `mmap` (anonymous and file-backed), POSIX
|
||||
shared memory, `memfd`, Unix domain sockets, `poll`/`select`/`epoll`, `eventfd`,
|
||||
`timerfd`, `signalfd`, `inotify`, `futex`.
|
||||
|
||||
### Dynamic linking
|
||||
|
||||
`exec` handles `ET_EXEC` and `ET_DYN`. A dynamically linked program names an
|
||||
interpreter in `PT_INTERP` — on musl, `/lib/ld-musl-x86_64.so.1`, which is libc
|
||||
under another name — and the kernel's whole part is to load that as well and
|
||||
start it instead of the program. The interpreter finds the program through the
|
||||
auxiliary vector (`AT_PHDR`, `AT_ENTRY`), maps libc and whatever else it needs
|
||||
with `mmap()`, relocates, and jumps to `AT_ENTRY` itself. A position-independent
|
||||
program goes where a fixed one would (`0x400000`); the interpreter goes at the
|
||||
start of the `mmap` area (`0x40000000`) and `mmap` carries on above it, so the
|
||||
two never meet. This is what lets Alpine's binaries run unmodified, and
|
||||
`dlopen` works, so Xorg loads its drivers as modules.
|
||||
|
||||
`mmap` uses a bump allocator (`proc->mmap_next`, starting at `0x40000000`) so
|
||||
every mapping gets its own disjoint, exactly-sized range. It previously assigned
|
||||
|
|
@ -329,24 +429,30 @@ allocation over 8 MB mapped straight into a neighbouring live mapping.
|
|||
loop reports 97%, `sleep 2` reports 0% — but kernel time needs TSC deltas
|
||||
closed and reopened around every `sched()`, which is scheduler accounting
|
||||
rather than sampling, and is not written.
|
||||
|
||||
- **No slab allocator.** One page at a time, and every subsystem's fixed-size
|
||||
table is a consequence. This is the single change that would most improve the
|
||||
rest of the kernel.
|
||||
- **No demand paging and no page cache.** `mmap` of a file *reads the whole
|
||||
thing in* at map time. Writable `MAP_SHARED` of a file is therefore refused
|
||||
outright rather than faked — see [../plan/foundations.md](../plan/foundations.md).
|
||||
- **No dynamic linking.** `exec` handles `ET_EXEC` only: no `PT_INTERP`, no
|
||||
`ET_DYN`, no shared libraries, no `dlopen`.
|
||||
- **No threads.** `clone` exists but does not share an address space;
|
||||
`gettid` returns the pid.
|
||||
- **`struct socket` is too big for a slab**, and every subsystem's fixed-size
|
||||
table (`NPROC`, `NSOCK`, `NDEVICE`, `NSHM`) is still a compile-time constant.
|
||||
Slabs exist for the objects that fit in a page.
|
||||
- **No page cache.** `mmap` of a file *reads the range in* at map time, and
|
||||
writable `MAP_SHARED` of a file is refused outright rather than faked. Demand
|
||||
paging covers anonymous memory only.
|
||||
- **Permissions are tracked but not enforced.** uid/gid are maintained honestly
|
||||
so `getuid()`/`setuid()` stop lying, and nothing consults them on access.
|
||||
so `getuid()`/`setuid()` stop lying, and file access does not consult them;
|
||||
`setpriority` is the one place the uids are checked.
|
||||
- **`mprotect` is accepted and not enforced** (`sys_mprotect` returns 0). RELRO is not read-only,
|
||||
a `PROT_NONE` guard does not fault, and nothing has `NX` (`EFER.NXE` is never set), so every
|
||||
mapping is executable.
|
||||
- **`poll`, `select` and `epoll` wait a tick at a time** (10 ms at `HZ=100`) unless something
|
||||
that can make a descriptor ready calls `poll_kick()` — which the pipe, socket and event paths
|
||||
do. busybox's `wget` polls before every read and downloaded at 85 KB/s, a segment per tick,
|
||||
until the network path started kicking.
|
||||
- **No swap**, and no memory pressure handling of any kind. `kalloc` failing is
|
||||
reported up as `-ENOMEM` and that is the whole policy.
|
||||
- **Nested signal delivery is not supported.** `in_signal` guards against
|
||||
re-entering a handler while one is running.
|
||||
- **Scheduling is fair-share on one global run queue.** Priorities and
|
||||
fairness accounting exist now ([scheduler.md](scheduler.md)); per-CPU run
|
||||
queues, work stealing and CPU affinity do not, despite `sched_setaffinity`
|
||||
existing.
|
||||
- **One global run queue.** Fair share and `nice` exist; per-CPU run queues, work
|
||||
stealing and CPU affinity do not, and `sched_setaffinity` is accepted and
|
||||
ignored.
|
||||
- **No IOMMU**, so DMA-capable devices (and the driver processes that program
|
||||
them) are trusted.
|
||||
- **256 GB is the most RAM** the kernel uses; the rest is ignored, with a line
|
||||
saying so.
|
||||
|
|
|
|||
|
|
@ -6,7 +6,11 @@ drivers, and LDLS is what does that.
|
|||
|
||||
It works, and on more than one driver. `8139too` and `e1000` — unmodified GPL
|
||||
kernel modules — load into a SchOS process, drive their hardware through
|
||||
`/dev/pci`, and carry packets:
|
||||
`/dev/pci`, and carry packets (both under QEMU). A third, Linux's `r8169`, is
|
||||
the NIC driver on the ASUS TUF A15 laptop, and it is the first that needs a
|
||||
*set* of modules and not one; it is described under *A set of modules*. It has
|
||||
registered its interface and read its PHY on that laptop; whether it carries
|
||||
traffic there has not been confirmed yet.
|
||||
|
||||
```
|
||||
# ldlsd /lib/modules/8139too.ko /lib/modules/btf &
|
||||
|
|
@ -46,18 +50,20 @@ Every one was a call that succeeded, returned zero, and did nothing.
|
|||
|
||||
Five, and everything else is an implementation of them.
|
||||
|
||||
### 1. Nothing GPL ever enters an image we build
|
||||
### 1. Linux code may ship; so far the tree holds none
|
||||
|
||||
The driver is **the user's**: obtained by them, loaded on their machine. SchOS
|
||||
ships the layer and never the thing it loads. `.gitignore` refuses `*.ko`,
|
||||
`make ldls-stage` is the one command that puts a module into a local image and
|
||||
is deliberately not part of `all`, and `make ldls-unstage` takes it out again.
|
||||
This rule used to say nothing GPL would ever enter an image we build. It was a policy,
|
||||
never a licence constraint — SchOS is GPL-2.0-or-later, and GPL-2.0-only code combines
|
||||
with it — and the owner has overridden it: **Linux code and firmware blobs may be
|
||||
shipped.** What survives of the rule is mechanical, not legal. The repository carries no
|
||||
`*.ko` (`.gitignore` refuses them), and `make ldls-stage KO=…` / `make ldls-r8169` stage
|
||||
modules into a *local* image from the Linux kernel on the build machine, because a module,
|
||||
its dependencies and the BTF that describes them have to come from the same Linux build
|
||||
and that kernel is the easiest place to get a matching set.
|
||||
|
||||
[../licensing.md](../licensing.md) §6 is the whole argument. The short form is
|
||||
that the combination happens in a user process on the user's machine, so no
|
||||
combined work is distributed by anyone. Since SchOS moved to GPL-2.0-or-later
|
||||
that is no longer the only way it *could* be done — it is the way it is done,
|
||||
because a module we never ship is a module we never have to answer for.
|
||||
Shipping one is therefore a decision about the release, not about the code: an ISO that
|
||||
contains `r8169.ko` is a distribution of GPL-2.0-only code and carries the usual
|
||||
obligation — its source goes with it — see [../licensing.md](../licensing.md) §6.
|
||||
|
||||
### 2. It runs in a process, never in the kernel
|
||||
|
||||
|
|
@ -186,6 +192,55 @@ the object has to live where a signed 32-bit absolute address reaches. A
|
|||
process's mmap area starts at `0x40000000`, which is inside that window; on the
|
||||
host harness `MAP_32BIT` reaches it from the other end.
|
||||
|
||||
### A set of modules — what r8169 needed
|
||||
|
||||
Linux's `r8169` is not one module. It needs `mdio-bus`, `libphy`, `mdio_devres`,
|
||||
the Realtek PHY driver, the NIC itself, the RTL PHY firmware, and the BTF of the
|
||||
kernel they came from. `make ldls-r8169` stages all of that from the running
|
||||
kernel; `etc/schinit/ldls-r8169.job` starts `ldlsd` on it at boot, with its output
|
||||
in `/var/log/ldls-r8169.log`. The job has `KeepAlive = no`: a driver that crashes
|
||||
on real hardware would otherwise be restarted into the same crash every few seconds.
|
||||
|
||||
What that added to `ldlsd`:
|
||||
|
||||
- **A comma-separated module set**, loaded in dependency order, each linking
|
||||
against the ones before it, with the BTF read before any `init_module` runs.
|
||||
A library module (`libphy`) just exports symbols to the next; the PCI `probe`
|
||||
runs once, at the end, for whichever module registered a `pci_driver`.
|
||||
- **A driver core**: bus match, driver probe, so the PHY driver binds to the PHY
|
||||
the way it does in Linux; **devres**; a kernel `printf`; real `udelay`/`msleep`/
|
||||
`ktime`; and **1000 Hz `jiffies`**, because the module's `HZ` is compiled in and
|
||||
phylib re-arms its link poll `HZ` jiffies out.
|
||||
- **Firmware** read from `/lib/firmware`.
|
||||
- **phylib's data**, the `link_mode_params` table and its bitmaps.
|
||||
- **The symbols whose stub-zero was a lie**: list validity, `try_module_get`,
|
||||
`dev_err_probe`, `eth_platform_get_mac_address`, `napi_consume_skb` (as a stub it
|
||||
leaked one buffer per transmitted frame until the DMA heap ran out), and the
|
||||
rest of the table at the end of this document.
|
||||
- `pdev->bus` is a real `pci_bus`; `net_device.priv_len` is set from BTF (the
|
||||
driver's `memset` of its ring bookkeeping is bounds-checked against it, and
|
||||
`__fortify_panic` fired on a length of zero); and `cmovcc` from per-CPU memory is
|
||||
rewritten like the other `%gs` forms.
|
||||
- **`__preempt_count` starts at `PREEMPT_ENABLED`.** mdio-bus keeps the register
|
||||
it has just read in `r11` across `preempt_enable()`, which on x86 is
|
||||
`decl %gs:__preempt_count; je`, to a call that preserves every register. With the
|
||||
count at zero the call was made, into a logging stub whose syscall left `RFLAGS`
|
||||
(0x246) in `r11` — so every MDIO read returned `0x02460246`. `preempt_schedule` is
|
||||
now a bare `ret`.
|
||||
- `__fortify_panic` names the module and function (`r8169: rtl_open.cold+0x…`) so
|
||||
a photograph of the screen is enough, and exits instead of spinning on a core and
|
||||
holding `/dev/netdev`.
|
||||
|
||||
The kernel did two things for it. It turns **ASPM off on Realtek NIC links** at
|
||||
boot, as `r8169` does through `pci_disable_link_state`, which a process cannot;
|
||||
and a network interface that **appears after boot starts the stack** — with the
|
||||
native driver out of the registry the laptop boots with no interface at all, and
|
||||
`ldls0` arrives later. `ldlsd` says which refusal it got when attaching: `EBUSY` is
|
||||
another process, anything else is the kernel.
|
||||
|
||||
The native `drivers/net/r8169/` stays in the tree, with Linux's 8168H setup, as
|
||||
the fallback; it is out of the registry.
|
||||
|
||||
### The KPI — `ldls/kpi/`
|
||||
|
||||
The Linux symbols, implemented. Register access to a mapped BAR; the PCI calls
|
||||
|
|
@ -232,8 +287,8 @@ and setting both to zero makes the whole thing reversible with no lookup table.
|
|||
|
||||
### The back end — `/dev/pci`
|
||||
|
||||
Already built, for `vgpud`, before any of this existed.
|
||||
[hybrid.md](../plan/hybrid.md) insisted the interface be about *PCI devices*
|
||||
Already built, for `vgpud`, before any of this existed. The design insisted the
|
||||
interface be about *PCI devices*
|
||||
rather than about a driver model, on the argument that an interface whose first
|
||||
consumers are native drivers is a general mechanism rather than a hole shaped
|
||||
like something foreign. A Linux driver's `pci_iomap`, `dma_alloc_coherent` and
|
||||
|
|
@ -241,10 +296,9 @@ like something foreign. A Linux driver's `pci_iomap`, `dma_alloc_coherent` and
|
|||
|
||||
### Giving a device up — `release`
|
||||
|
||||
A driver in the kernel now has a `release` hook, and `/dev/pci` calls it before
|
||||
handing its device to a process. `docs/plan/hybrid.md` had said the honest
|
||||
answer to "may I have the one you are using" is *no* until a driver could be
|
||||
told to let go; e1000 is the first that can.
|
||||
A driver in the kernel has a `release` hook, and `/dev/pci` calls it before
|
||||
handing its device to a process. The honest answer to "may I have the one you are
|
||||
using" was *no* until a driver could be told to let go; e1000 is the first that can.
|
||||
|
||||
Letting go is three things, and the third was missed at first:
|
||||
|
||||
|
|
@ -276,7 +330,7 @@ one flag and hands over all 65536 ports, including the interrupt controller,
|
|||
the timer and the disk; the TSS I/O permission bitmap is per port. A foreign
|
||||
driver already has DMA with no IOMMU, and the answer to "this is not a security
|
||||
boundary" is not to move it further — so it is the bitmap, at 8 KB per CPU of
|
||||
resident memory. See [../../include/ioperm.h](../../include/ioperm.h).
|
||||
resident memory. See [../../include/arch/ioperm.h](../../include/arch/ioperm.h).
|
||||
|
||||
Writing it found a latent hole. The TSS's I/O map base is at offset 102 and the
|
||||
old code wrote offset 64; the field it meant to set stayed 0, and a base of 0
|
||||
|
|
@ -336,17 +390,17 @@ point of having built it against our own:
|
|||
|
||||
## What it does not do
|
||||
|
||||
- **Two drivers, one kernel version, one machine.** `8139too` and `e1000` on
|
||||
6.18.46 against QEMU. The second driver was worth more than the first: it
|
||||
found the per-CPU problem, the retpoline problem, the port-I/O problem, the
|
||||
interrupt storm and a seven-year-old hole in the TSS. Nothing here has been
|
||||
tried on a second kernel or on real silicon, and "LDLS runs Linux drivers" is
|
||||
still a claim about a category from a sample of two.
|
||||
- **Three drivers, one kernel version.** `8139too` and `e1000` on 6.18.46
|
||||
against QEMU, and `r8169` on that version against QEMU (where it finds no
|
||||
device) and on one laptop (where it gets as far as the PHY). The second driver
|
||||
was worth more than the first: it found the per-CPU problem, the retpoline
|
||||
problem, the port-I/O problem, the interrupt storm and a seven-year-old hole in
|
||||
the TSS. Nothing here has been tried on a second kernel, and "LDLS runs Linux
|
||||
drivers" is still a claim about a category from a sample of three.
|
||||
- **No interrupts**, per the polling note above.
|
||||
- **`jiffies` is not `HZ`.** It comes from a 100 Hz `uptime()`, which is a
|
||||
plausible `HZ` and not necessarily the one the module was built for. A driver
|
||||
measuring a timeout against it gets a longer timeout, which is the safe
|
||||
direction and is still wrong.
|
||||
- **`jiffies` assumes `HZ` is 1000**, which is what the kernels LDLS loads from
|
||||
are built with (`ldls_load.hz` can say otherwise). It comes from the
|
||||
monotonic clock in milliseconds.
|
||||
- **One work item at a time, on the main loop.** A driver whose deferred work
|
||||
blocks stops everything, including the transmit path it is probably waiting
|
||||
for.
|
||||
|
|
@ -363,14 +417,21 @@ point of having built it against our own:
|
|||
process, which is a leak in the strict sense and the right trade for a
|
||||
program that will be killed to restart it.
|
||||
- **BTF has to come from somewhere.** SchOS has no `/sys/kernel/btf/vmlinux`,
|
||||
so the user brings it with the module, from the kernel the module was built
|
||||
for. A kernel built without `CONFIG_DEBUG_INFO_BTF` has neither.
|
||||
so `make ldls-stage` / `ldls-r8169` copy it from the Linux kernel the module
|
||||
came from. A kernel built without `CONFIG_DEBUG_INFO_BTF` has neither.
|
||||
- **There is address-space groundwork nothing uses.** `kernel/arch/x86_64/vm.c`
|
||||
maps the kernel image a second time at `LDLS_KALIAS`, in the top 2 GB where a
|
||||
`-mcmodel=kernel` module can reach, with a `vmemmap` and a module area beside it
|
||||
— at every boot — and `include/ldls/ldls.h` describes a `/dev/ldls` control
|
||||
channel to load modules into the kernel. That was the start of an in-kernel
|
||||
loader, contradicting rule 2. Nothing calls either, and it should be finished
|
||||
as a decision or deleted.
|
||||
- **No config-space writes.** `/dev/pci` has none, deliberately — a write is
|
||||
how a BAR gets moved out from under the kernel — so a driver that needs one
|
||||
is refused loudly rather than quietly succeeded.
|
||||
- **No module-to-module dependencies, no firmware loading, no sysfs, no
|
||||
unload.** `request_firmware` fails, `module_exit` is never called, and the
|
||||
way to stop a driver is to kill the process.
|
||||
- **No sysfs and no unload.** Modules depend on each other and firmware is read
|
||||
from `/lib/firmware`, but `module_exit` is never called, and the way to stop a
|
||||
driver is to kill the process.
|
||||
- **Some symbols are still logging stubs.** Deliberate, per rule 5 — and the
|
||||
single most expensive thing in this layer. **A stub that returns zero is not
|
||||
safe when zero is a meaningful answer**, and the list of ways that bit is
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
# Networking
|
||||
|
||||
ARP, IPv4, ICMP, UDP, TCP and DHCP, over a driver-agnostic interface class,
|
||||
reached through the same socket syscalls as Unix domain sockets.
|
||||
reached through the same socket syscalls as Unix domain sockets — plus DNS
|
||||
(resolved in the process, by musl), HTTPS (`wget`, Alpine's TLS libraries) and a
|
||||
small netlink responder for `getifaddrs`.
|
||||
|
||||
Verified end to end rather than by inspection: `bootcheck -Net` starts a
|
||||
listener and a UDP responder on the host, tells the guest to talk to them, and
|
||||
|
|
@ -25,7 +27,9 @@ that completes a handshake and drops every byte.
|
|||
└────────────────┬───────────────────┘
|
||||
│ struct netdev_ops
|
||||
┌────────────────┴───────────────────┐
|
||||
│ e1000, and any future NIC driver │
|
||||
│ e1000 rtl8139 r8169 — or a │ in the kernel
|
||||
│ Linux driver in a process, which │ (LDLS, via /dev/netdev)
|
||||
│ is a netdev like any other │
|
||||
└────────────────────────────────────┘
|
||||
```
|
||||
|
||||
|
|
@ -86,50 +90,55 @@ covers two decades of silicon — and why the driver prints the chip's XID on
|
|||
probe, since the PCI device id tells you almost nothing about which part
|
||||
arrived.
|
||||
|
||||
### Why it is native, when the plan said LDLS
|
||||
### Native first, then Linux's own
|
||||
|
||||
The first plan was to run Linux's `r8169.ko` under [ldls.md](ldls.md). Measuring
|
||||
it is what changed the answer: the module has **183 undefined symbols, 130 of
|
||||
them not implemented**, and 35 of those 130 are **phylib** —
|
||||
`phy_connect_direct`, `phy_start`, `mdiobus_read`, `genphy_soft_reset`.
|
||||
The first plan was to run Linux's `r8169.ko` under [ldls.md](ldls.md), and
|
||||
measuring it changed that: the module has **183 undefined symbols, 130 of them not
|
||||
implemented**, and 35 of those 130 are **phylib** — `phy_connect_direct`,
|
||||
`phy_start`, `mdiobus_read`, `genphy_soft_reset`. phylib is a subsystem with a link
|
||||
state machine, not a set of leaf calls, and LDLS's usual answer for an
|
||||
unimplemented symbol — a stub that reports itself and returns zero — is precisely
|
||||
wrong for it: `phy_start` returning zero means the link never comes up and nothing
|
||||
says why. So a **native driver** came first: it reaches the PHY through the
|
||||
chip's own MII window (`PHYAR`, one register that is a whole MII bus) and skips
|
||||
the subsystem, in the shape of the e1000 beside it.
|
||||
|
||||
phylib is a subsystem with a link state machine, not a set of leaf calls, and
|
||||
LDLS's usual answer for an unimplemented symbol — a stub that reports itself
|
||||
and returns zero — is precisely wrong for it. `phy_start` returning zero means
|
||||
the link never comes up and there is nothing on screen to say why. The 8139too
|
||||
that ldls.md uses as its model has no phylib dependency, which is why that one
|
||||
was tractable.
|
||||
It has since been done the other way as well. LDLS grew a driver core, devres,
|
||||
firmware loading and phylib's tables ([ldls.md](ldls.md), *A set of modules*),
|
||||
and the ASUS TUF A15's RTL8168H is now driven by **Linux's r8169, unmodified, in
|
||||
a process**: `make ldls-r8169` stages it with libphy, mdio-bus and the PHY driver,
|
||||
and `etc/schinit/ldls-r8169.job` starts it at boot. The native driver is **out of
|
||||
the registry** — it would claim the device first — and stays in the tree, with
|
||||
Linux's 8168H setup, as a fallback.
|
||||
|
||||
A native driver reaches the PHY through the chip's own MII window (`PHYAR`,
|
||||
one register that is a whole MII bus) and skips the subsystem entirely. It is
|
||||
less work than the port, it carries no GPL entanglement, it needs no firmware
|
||||
blob, and it is not coupled to one Linux version.
|
||||
|
||||
The register layout is a *fact about the hardware*, taken the way
|
||||
[ldls.md](ldls.md) §3 says facts may be taken. The expression is this tree's:
|
||||
a descriptor-ring driver in the same shape as the e1000 beside it.
|
||||
On that laptop the interface is `ldls0`, and it appears *after* boot, which is why
|
||||
`net_attach_late()` starts the stack when the first interface arrives instead of
|
||||
turning it away.
|
||||
|
||||
### The Chromebox has one of these
|
||||
|
||||
The Asus Chromebox CN62 "Guado" turns out to carry an RTL8111 rather than the
|
||||
Intel part its spec sheet suggests, so this driver covers it as well as the
|
||||
laptop that prompted it — which is the argument for the family match in one
|
||||
line rather than a device id per board.
|
||||
The Asus Chromebox CN62 "Guado" carries an RTL8111 rather than the Intel part its
|
||||
spec sheet suggests, so the family match (`10EC:8168`) covers it as well as the
|
||||
laptop — which is the argument for matching a family in one line rather than a
|
||||
device id per board.
|
||||
|
||||
### It has not run on hardware
|
||||
### It has not been shown to work on hardware
|
||||
|
||||
**Stated plainly because it cannot be tested here.** QEMU emulates `rtl8139`
|
||||
and nothing else from Realtek, and the 8139 is a different programming model —
|
||||
a circular receive buffer rather than descriptors — so it is deliberately *not*
|
||||
in the match table. Claiming it would break a NIC that works.
|
||||
**Stated plainly because it cannot be tested here.** QEMU emulates `rtl8139` and
|
||||
nothing else from Realtek, and the 8139 is a different programming model — a
|
||||
circular receive buffer rather than descriptors — so it is deliberately *not* in
|
||||
the match table. Claiming it would break a NIC that works.
|
||||
|
||||
What has been verified: the driver builds, its presence changes nothing on a
|
||||
machine without the device, and with an `rtl8139` on the bus it correctly
|
||||
leaves it unclaimed (`10ec:8139 network controller [-]`).
|
||||
What has been verified: both drivers build, their presence changes nothing on a
|
||||
machine without the device, and Linux's r8169 under LDLS loads, links and
|
||||
initialises in QEMU, registering and finding no device. On the laptop it reached
|
||||
probe and read the PHY (after a bug in LDLS that made every MDIO read return
|
||||
`0x02460246` was fixed); the native 8168H bring-up — the PLL, the receive gate, the
|
||||
PHY through the OCP window — has not run on a part.
|
||||
|
||||
What has not: every register write, on a real part.
|
||||
What has not: traffic on real hardware, from either.
|
||||
|
||||
So the probe says everything a first bug report would need, with `cprintf`
|
||||
The native probe says everything a first bug report would need, with `cprintf`
|
||||
rather than `kinfo` so it reaches the screen on a machine with no serial port:
|
||||
|
||||
```
|
||||
|
|
@ -141,6 +150,9 @@ register 1, where bit 2 is link and bit 5 is autonegotiation complete. Those
|
|||
four facts separate "the chip is not there", "the chip is there and the PHY
|
||||
sees nothing", and "everything is up and the rings are wrong".
|
||||
|
||||
The kernel turns **ASPM off** on Realtek NIC links at boot, as Linux's driver
|
||||
does through `pci_disable_link_state` — which a process under LDLS cannot do.
|
||||
|
||||
### Two constants that were wrong first
|
||||
|
||||
Recorded because both would have worked, quietly:
|
||||
|
|
@ -156,7 +168,7 @@ Recorded because both would have worked, quietly:
|
|||
|
||||
## The netdev class
|
||||
|
||||
A driver publishes a [`struct netdev_ops`](../../include/netdev.h) and is then
|
||||
A driver publishes a [`struct netdev_ops`](../../include/dev/netdev.h) and is then
|
||||
finished with the network stack forever: it never mentions lwIP, and the stack
|
||||
never mentions it.
|
||||
|
||||
|
|
@ -262,6 +274,12 @@ to. Measured across three pings: `rx_frames` and `wake_rx` both went 2 → 6.
|
|||
table, and it returns immediately: everything happens in that thread, which
|
||||
cannot run until the scheduler is entered.
|
||||
|
||||
**The thread follows each device's link into lwIP.** The link used to be read once
|
||||
at attach, 0.02 s into boot; real autonegotiation takes seconds, so on hardware
|
||||
the interface stayed down for good. It also starts the stack when an interface
|
||||
appears after boot (a driver in a process, a USB NIC), and attaches every device
|
||||
there is as its first act; `net_eth_attach()` refuses one it already has.
|
||||
|
||||
DHCP obtains an address at boot. `/proc/netif` reports each interface's address,
|
||||
netmask, gateway, MAC and frame counts, taken at the lwIP seam.
|
||||
|
||||
|
|
@ -326,19 +344,20 @@ The creation lives in `sys_bind()` rather than here, because that is the layer
|
|||
with both: `kernel/net/unix.c` knows about sockets and has no business calling
|
||||
`create()`.
|
||||
|
||||
**`flock()` is still missing.** libwayland locks `wayland-0.lock` beside the
|
||||
socket so that two compositors cannot claim one display. The lock file can be
|
||||
created and opened now that paths are real; the advisory lock itself is a
|
||||
separate mechanism with no other caller yet.
|
||||
`flock()` and `fcntl` record locks exist (`kernel/fs/lock.c`), so the lock files
|
||||
next to a socket work the way they do on Linux. Names in
|
||||
the **abstract namespace** (a `sun_path` starting with a NUL) name no file; they
|
||||
work, and `getsockname` reports them unnamed. `SO_PASSCRED` is honoured, and
|
||||
`SO_PEERCRED` answers (tmux asks).
|
||||
|
||||
### Descriptor passing
|
||||
|
||||
`sendmsg`/`recvmsg` with `SCM_RIGHTS`, which is the one thing a socket can carry
|
||||
that a pipe cannot: a reference to an object the kernel already knows, rather
|
||||
than a copy of its contents. Wayland's `wl_shm_create_pool` passes a descriptor
|
||||
and has no integer-shaped alternative, and
|
||||
[hybrid.md](../plan/hybrid.md)'s mechanism 1 is how a driver process is handed
|
||||
the device it is allowed to touch. Both wanted the same primitive.
|
||||
and has no integer-shaped alternative, and the same mechanism is how a driver
|
||||
process is handed the device it is allowed to touch ([devices.md](devices.md)).
|
||||
Both wanted the same primitive.
|
||||
|
||||
**A batch of descriptors is pinned to the stream position of the first byte of
|
||||
the message that carried it.** That single rule is the whole delivery model, and
|
||||
|
|
@ -374,9 +393,9 @@ descriptors to a position that never arrives.
|
|||
**There is no garbage collector.** An AF_UNIX socket descriptor, sent over that
|
||||
same socket and never received, holds a reference to the socket that holds the
|
||||
reference to it, and neither close can reach zero. Linux carries several hundred
|
||||
lines to break these cycles. Nothing on either plan passes a socket over a
|
||||
socket — Wayland passes memory, the driver work passes devices — so this is a
|
||||
known and bounded leak (thirty-two sockets) rather than a silent one.
|
||||
lines to break these cycles. Nothing passes a socket over a
|
||||
socket — display servers pass memory, the driver work passes devices — so this is
|
||||
a known and bounded leak (thirty-two sockets) rather than a silent one.
|
||||
|
||||
### AF_INET
|
||||
|
||||
|
|
@ -593,59 +612,47 @@ information was there and the code did not ask.
|
|||
|
||||
## What it does not do
|
||||
|
||||
- **`ping` is a first-party program**, `cmd/ping`, not toybox's. `recvmsg` now
|
||||
exists, so half of what stopped toybox's is gone; what remains is
|
||||
`IP_RECVTTL`, an `IPPROTO_IP` control message that would have to be generated
|
||||
on the receive path rather than merely carried. Ours has no `-I`, no flood
|
||||
mode and no TTL display.
|
||||
- **`SCM_RIGHTS` is the only control message.** `SCM_CREDENTIALS` is refused
|
||||
with `EINVAL` rather than ignored: a caller told its credentials were passed
|
||||
would have the receiver read a uid out of uninitialised memory and act on it.
|
||||
- **`ping` is a first-party program**, `cmd/ping`, not toybox's or busybox's. Ours
|
||||
has no `-I`, no flood mode and no TTL display (`IP_RECVTTL` is an `IPPROTO_IP`
|
||||
control message that would have to be generated on the receive path), and takes a
|
||||
dotted quad or `localhost` — it does not link musl's resolver. Busybox's `ping`,
|
||||
from the Alpine root, works over a `SOCK_RAW` ICMP socket.
|
||||
- **Only `SCM_RIGHTS` and `SCM_CREDENTIALS` travel in a control message.** Anything
|
||||
else is refused rather than ignored.
|
||||
- **A message may carry eight descriptors, and a socket eight such messages.**
|
||||
Past that `sendmsg` answers `ENOBUFS`. The cap exists because the kernel holds
|
||||
a reference to each one until it is received; a sender eight unreceived
|
||||
messages ahead of its receiver is not rescued by a ninth slot.
|
||||
- **`ping` still takes a dotted quad**, even though names now work. It is a
|
||||
first-party program that does not link musl's resolver, and `localhost` is
|
||||
special-cased in it rather than resolved. Every *ported* program resolves
|
||||
names properly; ours is the one that does not.
|
||||
- **Nothing re-runs `resolvconf`.** The nameserver is written once at boot. A
|
||||
lease that changes the DNS server, or a `/dev/pci` handover that brings up a
|
||||
new interface, leaves a stale `/etc/resolv.conf` until somebody runs it again.
|
||||
- **lwIP's own DNS client is still unused.** It is compiled in and only
|
||||
`dns_getserver()` is ever called; resolution happens entirely in the process.
|
||||
Nothing in the kernel can resolve a name, which matters the day something in
|
||||
the kernel wants to.
|
||||
- **`httpd` does not serve HTTP/1.1.** Every response is HTTP/1.0 with
|
||||
`Connection: close`, because the directory listing is generated straight into
|
||||
the socket and so has no `Content-Length` to give. One request per connection
|
||||
is the cost.
|
||||
- **Nothing re-runs `resolvconf`.** The nameserver is written once at boot
|
||||
(`etc/schinit/resolvconf.job`). A lease that changes the DNS server, or a
|
||||
`/dev/pci` handover that brings up a new interface, leaves a stale
|
||||
`/etc/resolv.conf` until somebody runs it again.
|
||||
- **lwIP's own DNS client is unused.** Resolution happens entirely in the process
|
||||
(musl); nothing in the kernel can resolve a name.
|
||||
- **`httpd` serves HTTP/1.0** with `Connection: close`, one request per connection,
|
||||
and has no timeouts: a client that connects and says nothing ties up a child
|
||||
process until it goes away, and `SO_RCVTIMEO` does not exist here.
|
||||
- **A queued-but-never-accepted connection is closed without clearing its
|
||||
callbacks.** `inet_close()` clears the argument and closes, which is enough
|
||||
because the pcb is freed — but it is a narrower guarantee than the one the
|
||||
main path makes, and it relies on `tcp_close()` succeeding.
|
||||
- **`httpd` has no timeouts.** A client that connects and says nothing ties up
|
||||
a child process until it goes away; `SO_RCVTIMEO` does not exist here (see
|
||||
below), so there is nothing to set.
|
||||
- **No routing table**, and no `/proc/net/route` to report one. With two
|
||||
interfaces up, the first is the default route and the second is reachable only
|
||||
by an address on its own subnet. That is a rule rather than an accident, and it
|
||||
is not routing. The missing *file* is the more immediately visible half: it is
|
||||
how a program asks which interface carries the default route, and without it
|
||||
anything that filters on "the default one" sees nothing.
|
||||
- **Netlink serves two message types.** No `RTM_GETROUTE`, no `RTM_NEWROUTE`, no
|
||||
event stream, no multicast groups. A program that expects to *subscribe* to
|
||||
link changes gets a socket that answers dumps and never speaks again.
|
||||
- **No `getsockname`/`getpeername`**, and `accept()` does not report the peer's
|
||||
address for an AF_INET socket. A server that wants to log who connected cannot.
|
||||
- **`setsockopt` covers six options**, and refuses the rest with `ENOPROTOOPT`
|
||||
rather than pretending: `SO_REUSEADDR`, `SO_KEEPALIVE`, `SO_BROADCAST` and
|
||||
`TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` are accepted and
|
||||
report the real size on read. Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`.
|
||||
because the pcb is freed, but it is a narrower guarantee than the main path
|
||||
makes and relies on `tcp_close()` succeeding.
|
||||
- **No routing table**, and no `/proc/net/route`. With two interfaces up, the first
|
||||
is the default route and the second is reachable only by an address on its own
|
||||
subnet. That is a rule rather than an accident, and it is not routing.
|
||||
- **Netlink is a responder**: `RTM_GETLINK` and `RTM_GETADDR` dumps (what
|
||||
`getifaddrs` needs), `MSG_PEEK|MSG_TRUNC` sizing (GLib's network monitor measures
|
||||
each datagram that way and spun at 100% when it answered 0), and nothing else. No
|
||||
`RTM_GETROUTE`, no event stream, no multicast groups; a program that expects to
|
||||
*subscribe* to link changes gets a socket that answers dumps and never speaks
|
||||
again.
|
||||
- **`setsockopt` applies what it understands** (`SO_REUSEADDR`, `SO_KEEPALIVE`,
|
||||
`SO_BROADCAST`, `TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` report the
|
||||
real size; `SO_PASSCRED` on Unix sockets) and refuses the rest with `ENOPROTOOPT`.
|
||||
Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`.
|
||||
- **`SOCK_DGRAM` over AF_UNIX does not exist** — stream only.
|
||||
- **AF_UNIX names are not filesystem paths**, so nothing can `stat()` or
|
||||
`unlink()` a socket.
|
||||
- **One NIC driver exists.** The class is proven with two e1000 cards, not with
|
||||
two different drivers.
|
||||
- **No IPv6**, no DNS resolver in the kernel (lwIP's is compiled but unused by
|
||||
anything), and no firewall or packet filter of any kind.
|
||||
- **No IPv6**, no firewall or packet filter of any kind, and a TCP stack that has
|
||||
been used for downloads at tens of megabits and no more than that.
|
||||
- **Three drivers in the kernel, one Linux driver outside it**, and the class is
|
||||
proven with two e1000 cards and an LDLS handover, not on a wide range of
|
||||
hardware. The Realtek parts have never carried a packet on real silicon from this
|
||||
tree.
|
||||
|
|
|
|||
102
docs/architecture/packages.md
Normal file
102
docs/architecture/packages.md
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
# Packages: an apk root
|
||||
|
||||
Alpine is musl with a different init, and SchOS is musl with a different kernel.
|
||||
Alpine's binaries run on SchOS unmodified — its dynamic linker, its busybox, and
|
||||
`apk` itself, fetching over TLS through SchOS's network stack. So instead of
|
||||
porting programs one at a time, SchOS installs them.
|
||||
|
||||
## The model
|
||||
|
||||
The root filesystem is an apk-managed root:
|
||||
|
||||
| Part | Where it comes from |
|
||||
|---|---|
|
||||
| musl, busybox, apk-tools, alpine-keys, CA certificates | Alpine's minirootfs (`ports/alpine`) |
|
||||
| what `ports/alpine/world` lists (zsh, Xorg, Window Maker, htop, NetSurf …) | Alpine's packages, pinned in `world.lock` |
|
||||
| schinit, its jobs, SchOS's tools, toybox, mksh, configuration | **schos-base**, an apk package the build makes from `sysroot/` |
|
||||
|
||||
`apk add` installs anything from Alpine's main and community repositories.
|
||||
`apk upgrade` updates Alpine's packages, and schos-base from the SchOS repository
|
||||
at `/var/lib/schos/repo`.
|
||||
|
||||
PID 1 is still schinit. Alpine's OpenRC is not installed, and packages that ship
|
||||
OpenRC services install them where nothing reads them.
|
||||
|
||||
## Who owns what
|
||||
|
||||
Where Alpine and SchOS both have a program, Alpine's wins, and SchOS keeps only
|
||||
what is its own (`tools/posix/mkaroot.sh` says so in its header):
|
||||
|
||||
- `/bin/sh`, `cat`, `sed`, `find` and the rest are busybox. Alpine's package scripts
|
||||
are written against busybox.
|
||||
- zsh is Alpine's. SchOS keeps its `zshrc` and `zshenv` (`replaces=zsh`).
|
||||
- `/etc/os-release` and `/etc/motd` are SchOS's (`replaces=`).
|
||||
- `passwd`, `group` and `profile` are Alpine's. schos-base's post-install sets root's
|
||||
shell to zsh, and SchOS's profile is `/etc/profile.d/schos.sh`.
|
||||
- `pci.ids` and `usb.ids` are Alpine's (`hwdata-pci`, `hwdata-usb`): Xorg depends on
|
||||
`hwdata-pci`, and two owners of one file is an error apk will not install past.
|
||||
- mksh is `/bin/mksh`. schinit's recovery shell looks for it first because it is static:
|
||||
a broken musl cannot take recovery down with it.
|
||||
|
||||
## Building
|
||||
|
||||
make qemu boots build/schfs.img, the root as a 1 GB SchFS disk
|
||||
make iso the same root as the live system's compressed ramdisk
|
||||
make alpine-lock re-resolve world against Alpine (needs the network)
|
||||
|
||||
It is the only root; `sysroot/` is what schos-base is packaged from. Only
|
||||
`alpine-lock` asks Alpine what is current. A normal build installs exactly what
|
||||
`world.lock` names — files in `dist/alpine/`, fetched once and checked against the
|
||||
lock's sha256, and against Alpine's signatures by apk — and never asks what is newer.
|
||||
`world.lock` resolves the handful of names in `world` into a few hundred files, every
|
||||
dependency included, so every image after the first is reproducible.
|
||||
|
||||
schos-base and its repository index are signed with a key made on the first build
|
||||
(`build/apk-keys/`, never committed). The root trusts it beside Alpine's keys. The
|
||||
install runs as an ordinary user with `--no-scripts`, because apk's scripts want a
|
||||
chroot; `mkaroot.sh` does by hand the one change schos-base's post-install would
|
||||
have made.
|
||||
|
||||
## What it took from the kernel
|
||||
|
||||
All of it found by running apk, and none of it specific to apk:
|
||||
|
||||
- **TCP receive window:** it reopened before the reader drained, so lwIP refused data
|
||||
and the sender waited on retransmission timers (5 KB/s). It reopens on read now, with a
|
||||
32-segment window; downloads run at tens of megabits.
|
||||
- **`mmap` address reuse:** freed ranges were never reused, so musl's allocator used up the
|
||||
256 mapping records with 780 MB of RAM free.
|
||||
- **`unlinkat` / `renameat`** returned a bare `-1`, read as `EPERM`, where apk expects
|
||||
`ENOENT`; `renameat` could not replace its target.
|
||||
- **`#!` scripts in `execve`**, and `ENOEXEC`: apk runs triggers directly.
|
||||
- **`chroot`**, and mounting devtmpfs and proc more than once.
|
||||
- Later, as bigger programs arrived: dynamic linking, threads and `futex`, demand paging,
|
||||
pseudo-terminals, `inotify`, `flock`, Unix-socket credentials. Each is in
|
||||
[kernel.md](kernel.md) or [syscalls.md](syscalls.md) because a package asked for it.
|
||||
|
||||
## The live system
|
||||
|
||||
The live ISO runs the apk root from RAM. The ramdisk is the compressed image
|
||||
([scff.md](scff.md)); at boot the volume grows to half of the free memory (4 GB of root on
|
||||
an 8 GB machine), backed by pages as they are written. Xorg and its dependencies are in the
|
||||
image already; anything else installs into that space in about a minute, over the network,
|
||||
and **is gone at reboot**.
|
||||
|
||||
## What it does not do
|
||||
|
||||
- **No installer.** Nothing is written to the machine's disks, so packages do not survive
|
||||
a reboot. Keeping them needs an installer that writes the root to a SATA or NVMe disk,
|
||||
and a kernel that finds its root there by the SchFS GPT type rather than as IDE disk 1.
|
||||
- **Not every Alpine package runs.** The ones in `world` were chosen *because* they run:
|
||||
terminal programs that draw and poll (htop, btop, tmux, mc), X clients, SDL 2 games,
|
||||
NetSurf on GTK 3. Others hit whatever the kernel does not do yet; Epiphany (WebKit) is
|
||||
one — its web-process child dies at start-up, and it has not been run to ground.
|
||||
A package is one `apk add` away, and finding what it needs is how most of the kernel's
|
||||
Linux surface got written.
|
||||
- **Fortified libc is strict, and Alpine's packages are not always correct.** `lbreakout2`
|
||||
hands `vsnprintf` a size larger than its buffer and traps on Alpine too; `world` carries
|
||||
`lbreakouthd` instead.
|
||||
- **apk's scripts and triggers do not run at build time** (`--no-scripts`), so a package
|
||||
whose script matters has to have it done by hand in `mkaroot.sh`.
|
||||
- **OpenRC services do nothing.** Anything that expects an init script to start it has to
|
||||
be given a schinit job ([init.md](init.md)).
|
||||
|
|
@ -74,19 +74,19 @@ machine off, and Linux number 48 is the socket call. Two syscalls cannot share a
|
|||
|
||||
## What is implemented
|
||||
|
||||
Roughly 190 native numbers covering:
|
||||
About 215 native numbers (193 of them with a Linux number) covering:
|
||||
|
||||
| Area | |
|
||||
|---|---|
|
||||
| Process | fork, exec, exit, wait/waitpid/wait4/waitid, clone, kill, sessions, process groups |
|
||||
| Process | fork, exec, exit, exit_group, wait/waitpid/wait4/waitid, clone (processes and threads), futex, kill, tgkill, sessions, process groups |
|
||||
| Identity | the full real/effective/saved set for both uid and gid, groups, umask |
|
||||
| Memory | mmap, munmap, mprotect, brk/sbrk, madvise, System V shared memory |
|
||||
| Memory | mmap, munmap, mremap, mprotect, brk/sbrk, madvise, System V shared memory, memfd |
|
||||
| Filesystem | the ordinary set, the whole `*at()` family, statfs, xattr, mount/umount2 |
|
||||
| I/O | read/write, readv/writev, pread/pwrite, sendfile, poll/select/ppoll/pselect6, epoll (create, create1, ctl, wait, pwait), ioctl, fcntl |
|
||||
| Sockets | socket, bind, listen, accept, connect, sendto, recvfrom, socketpair, setsockopt, getsockopt |
|
||||
| Sockets | socket, bind, listen, accept/accept4, connect, sendto/recvfrom, sendmsg/recvmsg, socketpair, getsockname/getpeername, setsockopt/getsockopt, over AF_UNIX, AF_INET and AF_NETLINK |
|
||||
| Signals | rt_sigaction, rt_sigreturn, rt_sigsuspend, rt_sigpending, rt_sigtimedwait, sigaltstack |
|
||||
| Time | gettimeofday, clock_gettime, clock_getres, nanosleep, clock_nanosleep, alarm, timers |
|
||||
| Notification | eventfd, eventfd2, timerfd_create/settime/gettime, signalfd, signalfd4 -- see `kernel/sys/sys_event.c` |
|
||||
| Notification | eventfd, timerfd, signalfd, inotify -- see `kernel/sys/sys_event.c` |
|
||||
|
||||
### Argument fetching
|
||||
|
||||
|
|
@ -113,20 +113,19 @@ field by field, each through its own translation, or use `copyin`/`copyout`.
|
|||
|
||||
## What it does not do
|
||||
|
||||
- **`sendmsg`/`recvmsg` carry `SCM_RIGHTS` and nothing else.**
|
||||
`SCM_CREDENTIALS` is refused rather than ignored. See
|
||||
[networking.md](networking.md#descriptor-passing).
|
||||
- **No `memfd_create`.**
|
||||
- **No `getsockname` or `getpeername`**, and `accept()` does not report an
|
||||
AF_INET peer's address.
|
||||
- **`setsockopt` handles six options** and answers `ENOPROTOOPT` for everything
|
||||
else. `SO_RCVTIMEO`/`SO_SNDTIMEO` are among the refused.
|
||||
- **`clone` does not share an address space**, so it is `fork` with extra steps.
|
||||
- **`sendmsg`/`recvmsg` carry `SCM_RIGHTS` and `SCM_CREDENTIALS`**, and nothing
|
||||
else. Credentials are delivered only to a socket that asked with `SO_PASSCRED`
|
||||
(D-Bus and WebKit do); other control messages are refused rather than ignored.
|
||||
See [networking.md](networking.md#descriptor-passing).
|
||||
- **`setsockopt` applies the options it understands** — they reach lwIP's pcb or
|
||||
the Unix socket — and answers `ENOPROTOOPT` for everything else, never a
|
||||
success that did nothing. The options that take a `struct` (`SO_RCVTIMEO`,
|
||||
`SO_SNDTIMEO`) are among the refused.
|
||||
- **`sched_setaffinity` is accepted and ignored.**
|
||||
- **`getrusage` does not distinguish `RUSAGE_CHILDREN`** from `RUSAGE_SELF`, and
|
||||
fills in only the calling process's own tick count.
|
||||
- **The ABI is chosen by link address.** It works, and it is a heuristic rather
|
||||
than a fact about the binary.
|
||||
- **`/proc/<pid>/exe` is a regular file, not a symlink**, so `readlink()` on it
|
||||
fails and `ps -o comm` prints `[init]` rather than `init`. Default `ps` and
|
||||
`top` output is unaffected.
|
||||
- **The ABI is chosen by link address.** Anything linked at `0x100000` or above
|
||||
— musl's programs, and every dynamically linked one, whose interpreter is
|
||||
placed at `0x40000000` — gets the Linux map; a first-party program linked at
|
||||
`0x4000` gets the native one. It works, and it is a heuristic rather than a
|
||||
fact about the binary.
|
||||
- **`ptrace` and `seccomp`** are not implemented; a program that needs them
|
||||
gets `ENOSYS`.
|
||||
|
|
|
|||
|
|
@ -95,7 +95,9 @@ nothing to choose between. It records `proc->ctty` now, which is what
|
|||
|---|---|---|
|
||||
| `/dev/console` | 1 | the **active** terminal, so kernel messages follow a switch |
|
||||
| `/dev/tty` | 10 | the caller's **controlling** terminal, via `proc->ctty` |
|
||||
| `/dev/tty1`..`tty6` | 11-16 | that terminal, whether or not it is on screen |
|
||||
| `/dev/tty0` | | the active terminal, which is what Xorg and `openvt` ask for |
|
||||
| `/dev/tty1`..`tty12` | 11-22 | that terminal, whether or not it is on screen |
|
||||
| `/dev/ptmx`, `/dev/pts/N` | | pseudo-terminals, below |
|
||||
|
||||
`/dev/tty` was an alias for `/dev/console` with the same major and minor. They
|
||||
are different questions and now have different answers.
|
||||
|
|
@ -190,13 +192,13 @@ every candidate.
|
|||
|---|---|
|
||||
| **The host takes it, and the guest never sees it** | |
|
||||
| `Alt+F4`, `Alt+F2` | window manager |
|
||||
| `Ctrl+Alt+F1..F6` | display manager — and it matches **loosely**, so adding Shift does not get past it |
|
||||
| `Ctrl+Alt+F1..F12` | display manager — and it matches **loosely**, so adding Shift does not get past it |
|
||||
| `Ctrl+Alt+`digit | QEMU's own console tabs |
|
||||
| **Applications want it, and lose it permanently if the kernel eats it** | |
|
||||
| `Alt+`anything | Meta. zsh binds Meta-digit as `digit-argument`; so does every readline editor |
|
||||
| `Ctrl+`letter | control characters |
|
||||
| `Ctrl+Shift+`digit | types a character — `Ctrl+Shift+2` is `^@` |
|
||||
| `F1..F6` alone | help, and whatever a full-screen program wants |
|
||||
| `F1..F12` alone | help, and whatever a full-screen program wants |
|
||||
| **The keyboard layout wants it** | |
|
||||
| `AltGr+`letter | Polish diacritics — ą ć ę ł ń ó ś ź ż — and on some setups AltGr arrives as Ctrl+Alt |
|
||||
|
||||
|
|
@ -207,7 +209,7 @@ chvt # report the active terminal and the current chord
|
|||
chvt -c ctrl+alt+F # Ctrl+Alt+F1..F12 (the default)
|
||||
chvt -c altgr+F # AltGr+F1..F12 (the default until 2026-10)
|
||||
chvt -c ctrl+shift+F # Ctrl+Shift+F1..F12
|
||||
chvt -c alt+1 # Alt+1..6
|
||||
chvt -c alt+1 # Alt+1..9
|
||||
chvt -c off # no chord; chvt N is then the only way
|
||||
```
|
||||
|
||||
|
|
@ -270,12 +272,10 @@ Three layers, in increasing order of how much a program is claiming:
|
|||
`VT_SETMODE` used to be accepted and ignored — a program could ask for the
|
||||
chord, be told yes, and still not get it. It works now.
|
||||
|
||||
What `VT_PROCESS` does **not** do yet is the release handshake: Linux signals
|
||||
the owning process on every switch request and blocks until it answers
|
||||
`VT_RELDISP`, which is how a display server releases the GPU before the screen
|
||||
changes hands. Nothing here has anything to release, so the kernel simply
|
||||
stops switching rather than negotiating. `struct vt_mode` already carries
|
||||
`relsig`/`acqsig` for when Schcom needs it.
|
||||
`VT_PROCESS` also does the release handshake Linux does: a switch away from a terminal in
|
||||
that mode signals the owning process (`relsig`) and waits for `VT_RELDISP`, which is how a
|
||||
display server stops drawing before the screen changes hands; switching back sends `acqsig`.
|
||||
See *Handing a terminal to a program*, below.
|
||||
|
||||
Input goes to whichever terminal is on screen. That is what makes a terminal a
|
||||
terminal: the keyboard follows the eyes.
|
||||
|
|
@ -368,34 +368,56 @@ and erase characters — were added at the same time, because `TERM=linux`
|
|||
declares them and a terminal that drops them silently is the same class of bug
|
||||
waiting for a different editor.
|
||||
|
||||
## Handing a terminal to a program
|
||||
|
||||
A display server takes a terminal with `VT_SETMODE(VT_PROCESS)` and `KDSETMODE(KD_GRAPHICS)`,
|
||||
and from then on switching away from it is a **request**: the kernel sends the owner
|
||||
`relsig` and waits for `VT_RELDISP`, so the server stops drawing and only then says
|
||||
yes; switching back sends `acqsig` and the server redraws. A terminal in graphics mode
|
||||
is not repainted by the kernel. If the owner dies, the terminal goes back to the kernel
|
||||
whole. This is how Xorg runs on a terminal of its own, with Ctrl+Alt+F-key switching
|
||||
between it and the text consoles and Ctrl+Alt+Backspace ending the session.
|
||||
|
||||
## Pseudo-terminals
|
||||
|
||||
`/dev/ptmx` makes a pair on every open and hands back the master; the slave is
|
||||
`/dev/pts/N`, which the program on the other side gets as its stdin, stdout and
|
||||
controlling terminal. Between them is a line discipline, the console's in miniature:
|
||||
what the master writes is the slave's input (canonical editing, echo, `^C` as `SIGINT`
|
||||
to the foreground group) and what the slave writes is read back by the master with NL
|
||||
turned into CR NL. xterm, tmux, `script` and `sshd` all need exactly this. The pairs
|
||||
share one lock; `kernel/dev/pty.c`.
|
||||
|
||||
## The text cursor
|
||||
|
||||
A steady inverted block — the cell with its colours swapped relative to what it would
|
||||
show, so a cursor on reverse-video text is the plain text and every cursor on screen
|
||||
is visible against its cell. `ESC[?25l` hides it, `ESC[?25h` shows it. It is drawn into
|
||||
the grid's repaint and does not blink.
|
||||
|
||||
## What it does not do
|
||||
|
||||
- **No scrollback.** The grid is exactly the visible screen. Shift+PageUp does
|
||||
nothing because there is nothing above the top row to show.
|
||||
- **No reflow on resize.** The geometry is read once at `vt_init()`. Nothing
|
||||
resizes a framebuffer console today; a compositor will, and that is when this
|
||||
has to grow.
|
||||
- **No `VT_RELDISP` handshake.** `VT_PROCESS` stops the kernel switching, but
|
||||
the kernel does not signal the owning process and wait for an answer. A
|
||||
display server that needs to release the GPU *before* the screen changes
|
||||
hands will need that, and `struct vt_mode` already has the fields.
|
||||
- **No terminal past 6.** Linux allows 63. Six is what the chords reach and
|
||||
what the grids are sized for.
|
||||
- **No reflow on resize.** The geometry is read once at `vt_init()`; the console's
|
||||
font is sized to the screen at that point, and a resolution change from `schmode`
|
||||
repaints it, but text is not re-wrapped.
|
||||
- **No terminal past 12.** Linux allows 63. Twelve is one per function key.
|
||||
- **Only 240x75 of any screen.** Larger framebuffers work; the grid is clamped
|
||||
and the remainder stays background. Raising `VT_MAXCOLS`/`VT_MAXROWS` is the
|
||||
fix, and costs BSS.
|
||||
- **No UTF-8 the font lacks.** The console draws through the TrueType
|
||||
rasteriser now, so the limit is the font's `cmap` rather than a 256-entry
|
||||
table — Latin, Latin Extended, Greek, Cyrillic and the common punctuation and
|
||||
maths symbols all render. Anything genuinely absent still draws as `?`, which
|
||||
is deliberate: drawing a *different* character would be worse.
|
||||
rasteriser, so the limit is the font's `cmap` — Latin, Latin Extended, Greek,
|
||||
Cyrillic and the common punctuation and maths symbols all render. Anything
|
||||
genuinely absent still draws as `?`, which is deliberate: drawing a
|
||||
*different* character would be worse.
|
||||
- **No shaping, and no wide characters.** One codepoint is one cell of one
|
||||
advance. Combining marks land on top of their own cell rather than the
|
||||
preceding one, and CJK draws at half the width it wants. Both need a
|
||||
width table and a cell that can span two columns.
|
||||
- **No dead keys**, so French and Spanish layouts are absent rather than
|
||||
broken. See above.
|
||||
broken.
|
||||
- **No per-terminal layout.** One keyboard, one layout. Switching to tty3 does
|
||||
not move anybody's fingers.
|
||||
- **Only three layouts.** Adding one is a table in `keymap.c` and a line in the
|
||||
- **Only three layouts** (US, Polish programmers', German). Adding one is a table in `keymap.c` and a line in the
|
||||
catalogue; the mechanism is not the limit, the typing is.
|
||||
|
|
|
|||
|
|
@ -21,13 +21,14 @@ The specification is public and the controller is not complicated.
|
|||
> BSD is much more valuable as the reference you check yourself against than as
|
||||
> the code you paste.
|
||||
|
||||
Licensing points the same way: the BSD stacks are BSD-2/3-clause and compatible,
|
||||
Linux's drivers are GPL-2.0-only and are not — see
|
||||
[../licensing.md](../licensing.md).
|
||||
Licensing does not decide it: the BSD stacks are BSD-2/3-clause and compatible, and
|
||||
since SchOS became GPL-2.0-or-later Linux's GPL-2.0-only drivers are compatible too
|
||||
— see [../licensing.md](../licensing.md). The argument against porting is the
|
||||
framework underneath, not the licence.
|
||||
|
||||
## What the spec gives you for free
|
||||
|
||||
Everything in [`include/dev/usb.h`](../../include/usb.h) comes from USB 2.0
|
||||
Everything in [`include/dev/usb.h`](../../include/dev/usb.h) comes from USB 2.0
|
||||
chapter 9, which every device implements regardless of speed, class or vendor.
|
||||
|
||||
**That is what makes enumeration possible at all:** before you know anything
|
||||
|
|
@ -129,7 +130,8 @@ working one.
|
|||
filesystem's next read, by the rule in `blkdev_rw()`: a filesystem that went on
|
||||
with missing blocks would be guessing.
|
||||
- **HID is keyboards and pointers.** Consumer-control interfaces (volume keys),
|
||||
game pads and the rest are declined. Keyboard LEDs are not driven.
|
||||
game pads and the rest are declined. Caps, Num and Scroll Lock light their LEDs
|
||||
on USB and PS/2 keyboards, one state shared between keyboards.
|
||||
- **USB 3 hubs** are not driven: the hub driver speaks the USB 2 hub protocol.
|
||||
- **No isochronous transfers**, so no audio and no cameras.
|
||||
- **No USB 3 SuperSpeed-specific features** beyond what enumeration needs — the
|
||||
|
|
|
|||
|
|
@ -192,7 +192,7 @@ If either is edited, edit both.
|
|||
|---|---|
|
||||
| Loader splash | Shift, from `include/gfx/logo.h`, shaded per row by `metal()` in `boot/common/ui.c` — silver, not accent. |
|
||||
| Kernel console / early boot | Shift, same mask, composited flat in a theme colour by `gfx_draw_logo`. |
|
||||
| A display with no compositor | Shift again, drawn by `vgpud` on `SCH_BG`. It replaced a test card — colour bands, gradients, a checkerboard and the driver's pid in binary — which was scaffolding for [../plan/hybrid.md](../plan/hybrid.md)'s H2 and H3 and outlived them. The mark answers the same question a black screen cannot, *something is driving this device*, and answers it as the system rather than as a test rig. |
|
||||
| A display with no compositor | Shift again, drawn by `vgpud` on `SCH_BG`. It replaced a test card — colour bands, gradients, a checkerboard and the driver's pid in binary — which was scaffolding for [../architecture/devices.md](../architecture/devices.md)'s driver work and outlived them. The mark answers the same question a black screen cannot, *something is driving this device*, and answers it as the system rather than as a test rig. |
|
||||
| PSOGM | Shift, the same mask again, by `gfx_draw_logo()` from `kernel/core/panic.c`. At its native 128 px — there is no resampler in the crash path, and the mark was only ever drawn at one size there anyway. |
|
||||
| Serial crash report | `:(`, as text. The mark cannot be drawn in ASCII, and a mood marker in a log is not a mark. |
|
||||
| SchFS | **Nowhere yet.** `uberblock.svg` has no consumer; it is waiting for `projects/schfs-linux/` to have a README worth heading. |
|
||||
|
|
|
|||
|
|
@ -12,12 +12,20 @@ virtio, NVMe and the Intel datasheets are public documents with no licence on
|
|||
what you write from them. `kernel/drivers/bus/xhci/xhci.c` came from the spec,
|
||||
and so did `net/e1000/` and `gpu/virtio_gpu/`.
|
||||
|
||||
**BSD: yes. Linux: no.** This is a licence incompatibility rather than a
|
||||
preference, and it decides the whole strategy — see
|
||||
[../licensing.md](../licensing.md). BSD-2 and BSD-3 code can be used directly
|
||||
with a `NOTICE` entry; GPL-2.0-only code can never be, at any distance. Reading
|
||||
Linux to understand *hardware* is fine and normal; translating a Linux driver
|
||||
function by function produces a derivative work whatever the identifiers say.
|
||||
**Three sources, in the order to try them:**
|
||||
|
||||
1. **A native driver, written from the specification.** The default, and what every driver in the
|
||||
kernel is.
|
||||
2. **Linux's own driver, unmodified, through LDLS** ([ldls.md](../architecture/ldls.md)), when the
|
||||
device is one Linux supports and SchOS cannot afford to write a driver for — a NIC with a PHY
|
||||
behind it, a GPU, a Wi-Fi card. It runs in a process, so a crash costs the interface and not the
|
||||
machine. This is how the laptop's Realtek NIC is driven.
|
||||
3. **BSD or Linux code taken into the tree.** Licensing no longer forbids either: SchOS is
|
||||
GPL-2.0-or-later, so GPL-2.0-only code combines with it, and BSD-2/3 code can be used directly with
|
||||
a `NOTICE` entry — see [../licensing.md](../licensing.md). The reason to prefer 1 and 2 is
|
||||
maintenance, not law: a driver copied in is a driver that has to be tracked.
|
||||
|
||||
Reading another kernel to understand *hardware* is fine and normal.
|
||||
|
||||
One caveat about "port it from BSD" as a plan, learned by looking: FreeBSD's
|
||||
`sys/dev/usb` is roughly 100k lines welded to newbus, devclass, `bus_dma`,
|
||||
|
|
@ -45,7 +53,9 @@ ours.
|
|||
kernel/drivers/
|
||||
registry.c
|
||||
gpu/vga/ gpu/virtio_gpu/
|
||||
net/e1000/
|
||||
net/e1000/ net/rtl8139/ net/r8169/
|
||||
storage/ahci/ storage/nvme/
|
||||
audio/ac97.c
|
||||
bus/xhci/
|
||||
usb/core/ usb/hid/ usb/storage/ usb/hub/
|
||||
|
||||
|
|
@ -56,11 +66,12 @@ ours.
|
|||
|
||||
| class | vtable | consumer |
|
||||
|---|---|---|
|
||||
| `DEVCLASS_DISPLAY` | `struct display_ops` ([display.h](../../include/display.h)) | the compositor |
|
||||
| `DEVCLASS_NET` | `struct netdev_ops` ([netdev.h](../../include/netdev.h)) | the TCP/IP stack |
|
||||
| `DEVCLASS_DISPLAY` | `struct display_ops` ([display.h](../../include/gfx/display.h)) | none today — Xorg draws through `/dev/fb0` |
|
||||
| `DEVCLASS_NET` | `struct netdev_ops` ([netdev.h](../../include/dev/netdev.h)) | the TCP/IP stack |
|
||||
| `DEVCLASS_AUDIO` | `struct audio_ops` | `/dev/dsp` |
|
||||
|
||||
A display driver that owns a linear framebuffer also publishes it with
|
||||
`gfx_attach_scanout()` ([gfx.h](../../include/gfx.h)). That is what the text
|
||||
`gfx_attach_scanout()` ([gfx.h](../../include/gfx/gfx.h)). That is what the text
|
||||
console and the crash screen draw on, and it is separate from the display
|
||||
class on purpose: the class is about *frames* — acquire, present, vblank —
|
||||
and the crash screen has to work when nothing else in the system does.
|
||||
|
|
|
|||
|
|
@ -8,8 +8,15 @@ change that only works on one of them is a bug.
|
|||
```sh
|
||||
make # kernel + UEFI loader + root filesystem
|
||||
make qemu # build and boot it
|
||||
make iso # a bootable ISO (the live system) for VMware or real hardware
|
||||
./write.sh sdX # write that ISO to a USB stick (Linux; it checks what it is writing over)
|
||||
```
|
||||
|
||||
The first build downloads what it needs (musl, toybox, Alpine's packages -- a few
|
||||
hundred megabytes, kept in `dist/`) and compiles Xorg's dependencies not at all: the
|
||||
packages are Alpine's binaries, pinned in `ports/alpine/world.lock`. See
|
||||
[packages](../architecture/packages.md).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Need | Why | Windows | Linux | macOS | BSD |
|
||||
|
|
@ -17,7 +24,7 @@ make qemu # build and boot it
|
|||
| `clang`, `ld.lld`, `llvm-objcopy`, `llvm-objdump` | cross-compiles the kernel, the 16/32-bit boot code, and musl userland | LLVM installer | `clang lld llvm` | `brew install llvm` | `pkg install llvm` |
|
||||
| `make` | — | Git Bash | `make` | Xcode CLT | `gmake` |
|
||||
| `perl` | generates the interrupt vector table | Git Bash | usually present | present | `pkg install perl5` |
|
||||
| `python3` | boot logo, crash-screen QR | python.org (`python`) | `python3` | `brew install python` | `pkg install python3` |
|
||||
| `python3` | boot logo, crash-screen QR, the numbers in the README | python.org (`python`) | `python3` | `brew install python` | `pkg install python3` |
|
||||
| `qemu-system-x86_64` | running and testing | `scoop install qemu` | `qemu-system-x86` | `brew install qemu` | `pkg install qemu` |
|
||||
| `curl`, `tar` | fetching ports | Git Bash | present | present | present |
|
||||
| `xorriso` **or** `mkisofs` | `make iso` only | `scoop install cdrtools` | `xorriso` | `brew install xorriso` | `pkg install xorriso` |
|
||||
|
|
@ -81,10 +88,11 @@ produce.
|
|||
|
||||
## Ports
|
||||
|
||||
`musl`, `toybox` and `mksh` are built from source into `sysroot/`. **This does
|
||||
not work on a Windows host** — `make` prints `ports: skipped` and uses whatever
|
||||
is already staged in `sysroot/`. Building them needs a Linux host (or any other
|
||||
Unix); do it once and the staged result serves the Windows builds.
|
||||
`musl`, `toybox`, `mksh`, lwIP, uACPI, zstd, ncurses and the rest of `ports/` are built from source
|
||||
into `sysroot/`; Alpine's packages are unpacked into the root from `dist/alpine/`. **This does not work
|
||||
on a Windows host** -- `make` prints `ports: skipped` and uses whatever is already staged in
|
||||
`sysroot/`. Building them needs a Linux host (or any other Unix); do it once and the staged result
|
||||
serves the Windows builds.
|
||||
|
||||
## Where the tools live
|
||||
|
||||
|
|
|
|||
|
|
@ -14,47 +14,46 @@ deliberately kept out of `clean`).
|
|||
schos/
|
||||
├─ boot/ uefi/ the PE/COFF loader (splash, F8 menu, kernel load),
|
||||
│ common/ the UI it shares. UEFI only -- there is no BIOS path.
|
||||
├─ kernel/ The kernel, by subsystem: arch/ core/ dev/ fs/ proc/ sys/
|
||||
├─ kernel/ The kernel, by subsystem: arch/ core/ dev/ drivers/ fs/ net/
|
||||
│ proc/ sys/ acpi/ (uACPI glue, the EC, I2C-HID) gmmos/ (the
|
||||
│ micro OS under the crash screen) scff/ (the decompressor).
|
||||
│ drivers/ is the in-kernel drivers, by class: audio bus gpu
|
||||
│ net storage usb.
|
||||
├─ ldls/ The Linux driver layer: the .ko loader, the BTF reader, the
|
||||
│ Linux symbols reimplemented (kpi/), and host-side test tools.
|
||||
│ The process that runs it is cmd/ldlsd.
|
||||
├─ include/ Headers shared by kernel, userland, and tools/mkfs, by
|
||||
│ subsystem -- see "include/" below
|
||||
├─ lib/ First-party userland runtime: _start glue, syscall stubs,
|
||||
│ the initcode blob embedded in the kernel, user.ld
|
||||
├─ cmd/ First-party programs — one directory per program:
|
||||
│ cmd/<name>/main.c → sysroot/bin/<name>
|
||||
│ schinit and schinitd are PID 1 and its supervisor; vgpud is
|
||||
│ the virtio-gpu driver, which runs out here rather than in
|
||||
│ the kernel; socktest/shmtest/nettest/pcitest are suites.
|
||||
├─ ports/ Third-party software — one directory per port.
|
||||
│ Contains ONLY a Makefile and optional patches/.
|
||||
├─ projects/ Things that run ON SchOS rather than things that ARE it:
|
||||
│ the compositor, window manager, desktop, and installer.
|
||||
│ schinit and schinitd are PID 1 and its supervisor; vgpud and
|
||||
│ ldlsd are drivers that run out here rather than in the
|
||||
│ kernel; socktest/shmtest/nettest/pcitest/xready are suites.
|
||||
├─ etc/ The configuration the image starts with: schinit's job
|
||||
│ manifests (etc/schinit/*.job), motd, os-release, the zsh files.
|
||||
├─ ports/ Third-party software -- one directory per port, containing
|
||||
│ ONLY a Makefile and optional patches/. musl, toybox, mksh,
|
||||
│ lwIP, uACPI, zstd, ncurses, the fonts, fastfetch, kilo; and
|
||||
│ alpine/, which is the apk root (world, world.lock).
|
||||
├─ projects/ Things that run ON SchOS rather than things that ARE it.
|
||||
│ Not part of the kernel build. See projects/README.md.
|
||||
├─ tools/ Host-side helpers, SOURCE ONLY: mkfs.c, the SchFS test
|
||||
│ suites, sign.pl, vectors.pl, and the PowerShell harnesses.
|
||||
│ Their binaries build into build/tools/ -- this directory
|
||||
│ used to hold both, which is the one rule this layout has.
|
||||
│ tools/windows/loc.ps1 counts how much of the tree is first-party.
|
||||
├─ sysroot/ Staging for the disk image. etc/ is hand-maintained;
|
||||
│ bin/ lib/ sbin/ are populated by `make sysroot-install`.
|
||||
├─ dist/ Upstream tarball cache. Survives `make clean`.
|
||||
├─ Refrence/ A FreeBSD checkout (Refrence/freebsd-src), kept as reference
|
||||
│ material and a parts bin. Never built, never in an image.
|
||||
│ Read it when a specification is ambiguous; copy from it only
|
||||
│ after the per-file licence check in docs/licensing.md.
|
||||
│ A full clone is 2.1 GB and a previous NetBSD one was deleted
|
||||
│ for that reason. Only sys/dev, sys/net and sys/netinet have
|
||||
│ ever been read, so a sparse checkout is ~100 MB:
|
||||
│ git clone --filter=blob:none --sparse <url> Refrence/freebsd-src
|
||||
│ git -C Refrence/freebsd-src sparse-checkout set │ sys/dev sys/net sys/netinet
|
||||
├─ src-trunk/ A NetBSD checkout, when present -- reference material and a
|
||||
│ parts bin. NOT built, NOT installed, must never reach an
|
||||
│ image. Currently absent; it was deleted for disk space and
|
||||
│ can be re-fetched sparsely. Copying from it is governed by
|
||||
│ the per-file licence check in docs/licensing.md §3 --
|
||||
│ NetBSD is not uniformly licensed.
|
||||
├─ tools/ Host-side helpers, SOURCE ONLY: mkfs.c, mkschfs.c, mkscff.c,
|
||||
│ the SchFS test suites, vectors.pl, and the harnesses, grouped
|
||||
│ by what runs them (windows/ .ps1, posix/ .sh, python/ .py).
|
||||
│ Their binaries build into build/tools/. tools/windows/loc.ps1
|
||||
│ and tools/python/stats.py count how much of the tree is
|
||||
│ first-party.
|
||||
├─ sysroot/ Staging for the disk image, GENERATED by the build -- edit
|
||||
│ tools/posix/mkaroot.sh and etc/, not this.
|
||||
├─ dist/ Upstream tarball and package cache. Survives `make clean`.
|
||||
├─ shell.nix NixOS development shell: the whole toolchain in one file
|
||||
├─ write.sh Writes schos.iso to a USB stick, safely (Linux).
|
||||
├─ run.ps1 The Windows-side QEMU launcher.
|
||||
├─ docs/ architecture/ what it is plan/ what's next
|
||||
│ guides/ how to work history/ why it's like this
|
||||
│ Website/ the site, and the docs reader that renders this
|
||||
└─ build/ ALL generated output (see below)
|
||||
```
|
||||
|
||||
|
|
@ -92,7 +91,10 @@ build/
|
|||
├─ kernel/ object files, the linked kernel, kernel.asm
|
||||
├─ lib/ libc glue objects, initcode blob
|
||||
├─ cmd/ linked first-party programs
|
||||
├─ fs.img the filesystem image
|
||||
├─ aroot/ the apk root the image is made from
|
||||
├─ schfs.img the root as a disk image (make qemu)
|
||||
├─ fs-ram.img the same root as the live ramdisk, fs-ram.scff compressed
|
||||
├─ iso/ … what make iso assembles
|
||||
└─ ports/<name>/
|
||||
├─ src/ extracted upstream tarball
|
||||
├─ obj/ out-of-tree build directory
|
||||
|
|
@ -130,57 +132,43 @@ Local modifications to upstream go in `ports/foo/patches/*.patch` and are
|
|||
applied after extraction — never by editing files under `build/`, which are
|
||||
wiped on every clean.
|
||||
|
||||
## Ramdisk root
|
||||
## The live root
|
||||
|
||||
```sh
|
||||
make # normal: root on the IDE disk (build/fs.img as hdb)
|
||||
make ramroot # root from a SCHFS image linked into the kernel
|
||||
make # normal: the root is a SchFS disk image, build/schfs.img
|
||||
make iso # the same root as a compressed ramdisk linked into the kernel
|
||||
make ramroot # the ramdisk, uncompressed
|
||||
```
|
||||
|
||||
`make ramroot` builds `build/fs-ram.img` (default 6000 blocks, override with
|
||||
`RAMDISK_BLOCKS=`) and links it into the kernel, so the system boots to a shell
|
||||
with **no data disk attached at all**. That is what makes destructive work on
|
||||
the real disk survivable — reformatting it, or bringing up a new on-disk format
|
||||
— and it is the first piece of the installer.
|
||||
`make iso` builds the root as a SchFS image, compresses it
|
||||
([SCFF](../architecture/scff.md)) and links it into the kernel, so the system boots
|
||||
to a shell with **no data disk attached at all** -- which is what the USB stick is.
|
||||
It is also what makes destructive work on a real disk survivable. Nothing written
|
||||
to a live root survives a reboot.
|
||||
|
||||
Use `make ramroot`, not `make RAMROOT=1`: the flag changes `CFLAGS`, which make
|
||||
cannot see, so objects compiled for one setting would be silently reused for
|
||||
the other and `ROOTDEV` would point at the wrong device. The target drops the
|
||||
compiled objects first. Switching back is the same — plain `make` after a
|
||||
`ramroot` build reuses stale objects, so run `rm -rf build/kernel build/lib
|
||||
build/cmd` first (or just `make clean`).
|
||||
|
||||
The image costs boot time: the loader reads the kernel through the firmware's
|
||||
file protocol, so a 6 MB ramroot kernel is visibly slower to start than the
|
||||
350 KB default.
|
||||
Use the targets, not `make RAMROOT=1`: the flag changes `CFLAGS`, which make cannot
|
||||
see, so objects compiled for one setting would be silently reused for the other and
|
||||
`ROOTDEV` would point at the wrong device. The targets drop the compiled objects
|
||||
first. Going back to the disk root is the same -- run `make clean` or remove
|
||||
`build/kernel build/lib build/cmd` first.
|
||||
|
||||
**What may go in it is a licensing question, not only a size one.** The image is
|
||||
linked into the kernel binary, so anything in `sysroot/` when it is built becomes
|
||||
part of a distributable artifact. A file staged there under a licence that
|
||||
is not ours to redistribute does not merely ride along — it makes the kernel
|
||||
itself undistributable. `RAMDISK_EXCLUDE` in the Makefile is the list of
|
||||
paths kept out, and enforcing it in the build rather than relying on someone
|
||||
remembering to unstage something first is deliberate. It has been needed once
|
||||
already: a 33 MB test payload landed in `sysroot/` and both overflowed the image
|
||||
and would have gone into the kernel binary.
|
||||
part of a distributable artifact. A file staged there under a licence that is not
|
||||
ours to redistribute does not merely ride along -- it makes the kernel itself
|
||||
undistributable. `RAMDISK_EXCLUDE` in the Makefile is the list of paths kept out,
|
||||
and enforcing it in the build rather than relying on someone remembering to unstage
|
||||
something first is deliberate. (Linux modules for LDLS are staged by `make
|
||||
ldls-stage`/`ldls-r8169` only on request, and `.gitignore` refuses `*.ko`.)
|
||||
|
||||
Three knobs, all in the Makefile, none of them `FSSIZE`:
|
||||
The knobs: `RAMDISK_BLOCKS` (image size), `RAMDISK_NINODES` (the inode table is
|
||||
*preallocated*: the default of 12000 once cost a quarter of a small image to
|
||||
describe about 260 files), `RAMDISK_EXCLUDE`. If `mkfs` reports the image is full,
|
||||
read its message: it prints the metadata breakdown and names the right knob.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `RAMDISK_BLOCKS` | image size, 6000 (× 1 KiB `BSIZE`) |
|
||||
| `RAMDISK_NINODES` | inode table size, 2000 |
|
||||
| `RAMDISK_EXCLUDE` | sysroot-relative paths to omit, with their subtrees |
|
||||
|
||||
`RAMDISK_NINODES` is worth knowing about because the inode table is
|
||||
*preallocated*: mkfs's default of 12000 inodes cost 1501 of the 6000 blocks — a
|
||||
quarter of the image — to describe about 260 files. At 2000 it costs 251, and
|
||||
metadata overall fell from 1596 blocks to 345.
|
||||
|
||||
If `mkfs` reports the image is full, read its message: it now prints the
|
||||
metadata breakdown and names the right knob for whichever image overflowed. It
|
||||
used to say only "increase FSSIZE in param.h", which is a constant that has
|
||||
nothing to do with the ramdisk root and sends you to the wrong file.
|
||||
The ISO build is fast on purpose (a few seconds when little changed): it copies with
|
||||
`cp -p` and `install -C` so unchanged files keep their time stamps, and toggling the
|
||||
ramdisk does not wipe the tree.
|
||||
|
||||
## Two things that will bite you
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ reporting that it started.
|
|||
|
||||
| | What it runs | What it proves |
|
||||
|---|---|---|
|
||||
| `make schfs-check` | Host-native unit suites | The filesystem's logic, in about a second |
|
||||
| `make schfs-check`, `make kmem-test`, `ldls-inspect`/`ldls-run` | Host-native unit suites and the Linux-module loader run on the build machine | The filesystem's logic, the slab allocator's bookkeeping, and a `.ko`'s layout, in about a second |
|
||||
| `tools/windows/bootcheck.ps1` | A real boot, driven over the serial console | That the system answers correctly |
|
||||
| `make crashqr`, `make schcom-test` | A boot, then pixel assertions from the host | That the *screen* is right |
|
||||
|
||||
|
|
@ -77,6 +77,22 @@ dropped every byte would pass any check made from one side alone.
|
|||
Under QEMU's user-mode networking the host is `10.0.2.2` seen from inside, and
|
||||
SLIRP connects to the host's loopback — so the listener binds there.
|
||||
|
||||
## Test programs that run on SchOS
|
||||
|
||||
Several first-party programs in `cmd/` exist to be run **on** the system and print which of their checks
|
||||
failed, because only the system can say whether its own interfaces behave. `bootcheck` runs the ones it
|
||||
can; the rest are run by hand or by a script:
|
||||
|
||||
| | Checks |
|
||||
|---|---|
|
||||
| `socktest`, `nettest`, `netecho` | Unix and INET sockets, `SCM_RIGHTS` and credentials, `setsockopt`'s refusals |
|
||||
| `shmtest`, `memfdtest` | System V shared memory; `memfd` mapped by two processes, and one sent over a socket |
|
||||
| `pcitest` | `/dev/pci`: claim, map, DMA, interrupts, release |
|
||||
| `evtest`, `locktest`, `sigtest` | the event descriptors; `flock` and record locks; signals |
|
||||
| `dynhello` | dynamic linking: TLS, the heap, a forked child that still has its libraries |
|
||||
| `xready` | every kernel call Xorg makes, before Xorg is started — the evdev queries, the fbdev ioctls, the console calls, the `/sys` files |
|
||||
| `tools/python/ldls_check.py` | what LDLS needs from a kernel's BTF |
|
||||
|
||||
## Pixel assertions
|
||||
|
||||
```sh
|
||||
|
|
@ -148,10 +164,12 @@ leak.
|
|||
- **No automated test runs on real hardware.** Everything above is QEMU.
|
||||
- **SMP is exercised but not stressed** — boots on 1, 2 and 4 processors are
|
||||
checked; nothing hunts for races.
|
||||
- **Nothing asserts the desktop by pixel.** `xready` checks the calls Xorg makes; whether Window Maker
|
||||
and an `xterm` appear on a real screen is looked at in screenshots, not asserted.
|
||||
- **No fuzzing anywhere**, and no `fsx`/`xfstests` against SchFS. Getting SchFS
|
||||
under those is the main argument for the FUSE implementation in
|
||||
[../plan/roadmap.md](../plan/roadmap.md).
|
||||
- **The SchFS round-trip test needs symlinks and hard links**, so it skips on a
|
||||
Git Bash host and only runs on Unix.
|
||||
- **`make ports` does not work on Windows.** musl, toybox and mksh must be built
|
||||
- **`make ports` does not work on Windows.** musl, toybox, mksh and the rest must be built
|
||||
once on a Unix host and staged.
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ and the few write-ups that have nowhere better to live.
|
|||
|
||||
| | |
|
||||
|---|---|
|
||||
| [kernel.md](kernel.md) | The expensive ones. Boot, SMP, signals, ABI assumptions, `/proc`, input. |
|
||||
| [kernel.md](kernel.md) | The expensive ones. Boot, SMP, signals, ABI assumptions, `/proc`, input, TLBs, a stub that returned a flag word. |
|
||||
| [schfs.md](schfs.md) | How the filesystem was built, and what each phase cost. |
|
||||
| [moving-to-linux.md](moving-to-linux.md) | The Windows-to-Fedora move. |
|
||||
|
||||
|
|
|
|||
|
|
@ -298,17 +298,8 @@ exported entry points, `nameifrom()`/`nameiparentfrom()`. `sys_fs.c` gained
|
|||
Every `*at()` syscall now uses these instead of its own inline
|
||||
absolute-path/`AT_FDCWD`-only check.
|
||||
|
||||
**Known follow-on issue, not yet root-caused**: heavy `ls`/`ls -lR` use
|
||||
across many commands in one long-running shell session can eventually make
|
||||
a *later*, unrelated directory hang (reproduced with `ls -lR /var` after
|
||||
~10 prior commands in the same session; the identical command succeeds
|
||||
immediately on a fresh boot). Not an inode-table or file-table exhaustion
|
||||
— both `iget()` and `filealloc()` fail cleanly (panic / return 0
|
||||
respectively) when full, and neither fired. Smells like a lock or
|
||||
reference-count leak somewhere in the *at() paths above, or in toybox's
|
||||
own fd handling, that only manifests after enough accumulated opens/closes.
|
||||
Next step: instrument `itable`/`ftable` occupancy counts and watch them
|
||||
across a long `ls`-heavy session to see what's actually climbing.
|
||||
**Known follow-on issue** (heavy `ls -lR` in one long shell session eventually hanging a
|
||||
*later* directory): hunted, and not reproducible — see section 24.
|
||||
|
||||
## 6. Boot speed
|
||||
|
||||
|
|
@ -1255,3 +1246,86 @@ is at least as long as the width, and that the aperture holds that many rows.
|
|||
If not, it goes back to the previous mode and says so in the menu. This proves
|
||||
nothing about a mode that passes, but it stops a lying one from reaching the
|
||||
kernel.
|
||||
|
||||
---
|
||||
|
||||
## 24. The `ls -lR` hang that was hunted and not found
|
||||
|
||||
**Symptom.** After about ten `ls`-heavy commands in one long shell session, a later and
|
||||
unrelated `ls -lR /var` would hang; the same command succeeded at once on a fresh boot. It
|
||||
was not inode-table or file-table exhaustion (both fail cleanly), and the first hypothesis
|
||||
was a lock or reference-count leak in the `*at()` paths.
|
||||
|
||||
**What was done.** `/proc/fsstat` was given the occupancy counters the hypothesis asked
|
||||
for (inode, file and buffer peaks, the SchFS volume lock and transaction owner), and the
|
||||
workload was made heavier than the report: ten rounds of `ls -lR /bin` interleaved with
|
||||
writes and `cd` — 33 execs and 13,378 tree descents. Nothing climbed:
|
||||
|
||||
inode_peak 9 / 200 file_peak 9 / 512
|
||||
buf_pinned_peak 1 / 256 schfs0 locked 0 tx_owner_pid 0 tx_depth 0
|
||||
|
||||
**Conclusion.** The hypothesis is eliminated, and the bug is very likely gone — fixed in passing
|
||||
by the `*at()` dirfd rewrite, the `getdents` cookie rewrite, or SchFS becoming the root, all of
|
||||
which landed after it was reported. It was closed rather than chased. The instrumentation
|
||||
stayed: if it happens again, `/proc/fsstat` names the table, or shows `locked 1` with
|
||||
`tx_owner_pid 0`, which is an orphaned volume lock.
|
||||
|
||||
Two real defects turned up on the way and were fixed, neither proven to be the cause:
|
||||
`holdingsleep()` never compared the owning pid, which disarmed all seven `panic()` assertions
|
||||
built on it; and SchFS's `tx_mine()` compared a bare `struct proc *`, which a recycled ptable
|
||||
slot makes ambiguous.
|
||||
|
||||
**The transferable part.** A hang you cannot reproduce still earns its instrumentation. The
|
||||
absence of a result *is* the result, and the right place to leave it is where the next person
|
||||
with the same symptom will look.
|
||||
|
||||
## 25. Epiphany's heap was corrupted by a page nobody had unmapped yet
|
||||
|
||||
**Symptom.** Epiphany (WebKit: forty threads on four processors) died at a different address
|
||||
on every run; musl's allocator was the first to notice, a group's metadata no longer pointing
|
||||
back at its group.
|
||||
|
||||
**Diagnosis.** Nothing was freed twice. One thread unmapped a range, `mmap` handed the same
|
||||
addresses straight back, and a thread on *another* processor went on reading and writing the
|
||||
**old** pages through its TLB: stores vanished, loads were stale. Deferring the free of the
|
||||
unmapped pages had already kept a stale translation from reaching a page someone else owned;
|
||||
it did not keep one address space *coherent*.
|
||||
|
||||
**Fix.** Unmapping from a shared address space interrupts the processors running it and waits
|
||||
until each has reloaded `CR3` (`tlb_shootdown()`, `kernel/arch/x86_64/vm.c`).
|
||||
|
||||
**The transferable part.** Threads turned an invisible assumption — one CPU's TLB is the only
|
||||
one that matters — into the loudest bug in the tree, in the one program that used the most
|
||||
threads. Memory corruption that moves every run is a coherence bug until proven otherwise.
|
||||
|
||||
## 26. Every MDIO read returned `0x02460246`
|
||||
|
||||
**Symptom.** Linux's `r8169` under LDLS, on the laptop, read its PHY's ID as `0x02460246`.
|
||||
|
||||
**Diagnosis.** `0x246` is a value of `RFLAGS`, not a register. `mdio-bus` keeps the word it
|
||||
has just read in `r11` across `preempt_enable()`, which on x86 is `decl %gs:__preempt_count; je`
|
||||
to a call that preserves every register. LDLS had given `__preempt_count` zeroes, so the count
|
||||
reached zero after the decrement and the call was made — into a logging stub whose `syscall`
|
||||
instruction leaves `RFLAGS` in `r11`.
|
||||
|
||||
**Fix.** `__preempt_count` starts at `PREEMPT_ENABLED` (x86 keeps `NEED_RESCHED` inverted), and
|
||||
`preempt_schedule` and its notrace twin are a bare `ret`.
|
||||
|
||||
**The transferable part.** A stub that returns zero is not safe when zero changes which path
|
||||
runs — the same family as the five in [../architecture/ldls.md](../architecture/ldls.md). The
|
||||
tell here was a value that was a plausible *flag word* rather than a plausible *register*.
|
||||
|
||||
## 27. GLib's network monitor spun at 100%
|
||||
|
||||
**Symptom.** Epiphany's main thread consumed a full core in `poll()` returning at once.
|
||||
|
||||
**Diagnosis.** GLib sizes each netlink datagram with `recvmsg(MSG_PEEK|MSG_TRUNC)` and no buffer.
|
||||
The kernel answered 0, GLib read 0 bytes, the datagram stayed queued, and `poll()` said readable
|
||||
again, for ever. A drained netlink socket also reported hangup, because `sock_hup` took "no peer"
|
||||
for "closed".
|
||||
|
||||
**Fix.** `recvmsg` on netlink peeks without consuming and with `MSG_TRUNC` returns what is waiting.
|
||||
|
||||
**The transferable part.** A kernel that accepts a flag and does nothing with it turns a
|
||||
well-formed program into a busy loop. Do it, or refuse it.
|
||||
|
||||
|
|
|
|||
|
|
@ -271,10 +271,12 @@ copied file is a file to be understood, attributed and maintained forever.
|
|||
|
||||
[architecture/ldls.md](architecture/ldls.md) runs Linux drivers. This section
|
||||
was written when SchOS was Apache-2.0 and the boundary below was a legal
|
||||
necessity. Under GPL-2.0-or-later it is a **project policy** instead: shipping
|
||||
a GPL-2.0-only `.ko` with SchOS would now be lawful, provided the source goes
|
||||
with it. It is still not done, because a module we never ship is a module we
|
||||
never have to vouch for, track upstream, or provide source for.
|
||||
necessity. Under GPL-2.0-or-later it was a **project policy** instead, and the owner
|
||||
has since set the policy aside: Linux code and firmware blobs **may be shipped** with
|
||||
SchOS, provided the source goes with them (GPLv2 §3). The repository still carries no
|
||||
`.ko` — they are staged from the build machine's Linux kernel (`make ldls-stage`,
|
||||
`make ldls-r8169`) — so shipping one is a decision about a release, and the release has
|
||||
to carry the corresponding source.
|
||||
|
||||
### LDLS is ours
|
||||
|
||||
|
|
@ -298,10 +300,9 @@ GPLv2 §0 is explicit about where its obligations attach:
|
|||
> by this License; they are outside its scope. **The act of running the Program
|
||||
> is not restricted.**
|
||||
|
||||
So: we ship LDLS and no modules. The user obtains a module from wherever they
|
||||
get their software and runs it on their own machine. `RAMDISK_EXCLUDE` and the
|
||||
`*.ko` rule in `.gitignore` keep that boundary mechanical rather than
|
||||
intentional.
|
||||
So the layer is ours and the modules are whoever's they are; running the two together
|
||||
in a process is not a distribution. `RAMDISK_EXCLUDE` and the `*.ko` rule in `.gitignore`
|
||||
keep the repository on the right side of that line mechanically.
|
||||
|
||||
### `EXPORT_SYMBOL_GPL` does not apply
|
||||
|
||||
|
|
@ -322,7 +323,7 @@ show.
|
|||
|
||||
### BTF: the layouts are not in our source at all
|
||||
|
||||
§3 of the plan has one uncomfortable row — struct layouts. A compiled driver
|
||||
Rule 3 of [architecture/ldls.md](architecture/ldls.md) has one uncomfortable row — struct layouts. A compiled driver
|
||||
indexes fields by offset, so the shim has to reproduce them exactly, and a
|
||||
field-for-field match *looks* like transcription even when it is not.
|
||||
|
||||
|
|
|
|||
584
docs/plan/1.0.md
584
docs/plan/1.0.md
|
|
@ -7,17 +7,16 @@ A finish line, so that the list of open work stops being infinite.
|
|||
## Why this document exists
|
||||
|
||||
[schapi.md](schapi.md) §3 says the stability freeze starts at 1.0 and covers
|
||||
whatever is marked `stable` at that moment. **1.0 was never defined**, which
|
||||
means there has been no finish line — and with no finish line every gap is
|
||||
equally mandatory, the list only grows, and the project feels larger every week
|
||||
regardless of how much gets done.
|
||||
whatever is marked `stable` at that moment. 1.0 was never defined, which means
|
||||
there was no finish line — and with no finish line every gap is equally mandatory,
|
||||
the list only grows, and the project feels larger every week regardless of how much
|
||||
gets done.
|
||||
|
||||
That feeling is structural, not a judgement about progress. This document fixes
|
||||
the structure by drawing a line and putting most of the open work on the far
|
||||
side of it.
|
||||
That feeling is structural, not a judgement about progress. This document fixes the
|
||||
structure by drawing a line and putting most of the open work on the far side of it.
|
||||
|
||||
> **Anything not named in this document is not part of 1.0.** It is not
|
||||
> cancelled, not forgotten, and not up for reconsideration until 1.0 ships.
|
||||
> **Anything not named in this document is not part of 1.0.** It is not cancelled,
|
||||
> not forgotten, and not up for reconsideration until 1.0 ships.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -26,31 +25,55 @@ side of it.
|
|||
> **A machine one person can sit down at and use for an afternoon without
|
||||
> falling off an edge.**
|
||||
|
||||
Concretely — boots on real hardware, roots on SchFS, runs a shell and the real
|
||||
toybox userland, is on the network, and puts a windowed terminal on the screen.
|
||||
Concretely — boots on real hardware, roots on SchOS's own filesystem, runs a shell
|
||||
and a real userland, is on the network, and puts a windowed desktop on the screen.
|
||||
|
||||
It is deliberately a **small complete system** rather than a rich incomplete
|
||||
one. SchOS 1.0 is not Linux, does not run third-party applications, and has no
|
||||
package manager. It is a system that works, end to end, with nothing in it that
|
||||
lies about what it did.
|
||||
It started as a deliberately small system: *no third-party applications and no package
|
||||
manager*. That has changed on its own, and is worth recording rather than tidying away:
|
||||
the root is an **apk root** ([packages](../architecture/packages.md)), Alpine's
|
||||
programs run on it unmodified, and the desktop is Xorg with Window Maker
|
||||
([desktop](../architecture/desktop.md)). What did not change is the bar: nothing in the
|
||||
system lies about what it did.
|
||||
|
||||
### The acceptance test
|
||||
|
||||
One unattended run, asserted from the host, the way everything else here is
|
||||
tested. When this passes, it is 1.0:
|
||||
One unattended run, asserted from the host, the way everything else here is tested.
|
||||
When this passes, it is 1.0:
|
||||
|
||||
```
|
||||
boot on real hardware from a USB stick
|
||||
→ SchFS root mounts, generation counter advances
|
||||
→ schinit starts the compositor
|
||||
→ a terminal window appears and is asserted by pixel
|
||||
→ in it: ls, cat, a file written, a network fetch
|
||||
→ schinit starts shells on the virtual terminals
|
||||
→ X starts; a window appears and is asserted by pixel
|
||||
→ in it: ls, cat, a file written, a network fetch over HTTPS
|
||||
→ reboot
|
||||
→ the file is still there and the checksum matches
|
||||
```
|
||||
|
||||
Nothing in that sentence is a feature list. Every clause is something that is
|
||||
currently either broken, missing, or unproven.
|
||||
Where each clause stands:
|
||||
|
||||
| Clause | State |
|
||||
|---|---|
|
||||
| boot from a USB stick on real hardware | **done** on an AMD B550 desktop, an ASUS TUF A15 laptop and an Ivy Bridge HP all-in-one |
|
||||
| SchFS root, generation counter | **done** — the root is a SchFS image, in RAM when live |
|
||||
| a terminal to type in | **done** — a shell on the first six of the twelve virtual terminals (`etc/schinit/*-shell.job`); the rest are there to be started |
|
||||
| X starts, a window appears | **done** (`startx` → Window Maker + xterm); *asserted by pixel* has no automated test yet — `bootcheck` drives the serial console, and the screenshots are looked at |
|
||||
| `ls`, `cat`, a file written | **done** |
|
||||
| a network fetch over HTTPS | **done** (`wget`, `curl`, `apk`) |
|
||||
| reboot, **the file is still there** | **not met on the live image.** The root lives in RAM and everything written is gone at reboot; SchFS is crash-safe on a disk, but nothing writes a disk root. This is the one clause that fails, and it is the open decision below |
|
||||
|
||||
### The one open decision
|
||||
|
||||
The acceptance test's last clause needs a root that survives a reboot. There are two honest ways to
|
||||
close it, and which one is the owner's to choose:
|
||||
|
||||
1. **Write the installer** (`projects/schinst/`, [roadmap.md](roadmap.md) §5): partition a disk,
|
||||
write the root, install the loader. Large, and until now frozen.
|
||||
2. **Narrow the clause**: 1.0 is a *live* system, and the last line becomes "a file written to a
|
||||
SchFS disk or partition that was attached survives a reboot" — which already works
|
||||
(`make disk`, `make qemu` with `schfs.img`). Smaller, and it says what the system is.
|
||||
|
||||
Nothing else is on the path to 1.0.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -59,11 +82,10 @@ currently either broken, missing, or unproven.
|
|||
**No new capability is started until this register is empty.**
|
||||
|
||||
The failure this exists to stop is not laziness and not scope creep — it is
|
||||
*abandonment by promotion*. A thing gets built to the point where it works, the
|
||||
next thing becomes more interesting, and the first one is left in a corner:
|
||||
present in the tree, not finished, not deleted, and not visible in any list.
|
||||
Enough of those and the tree stops matching its own documentation, which is the
|
||||
state this register was written to end.
|
||||
*abandonment by promotion*. A thing gets built to the point where it works, the next
|
||||
thing becomes more interesting, and the first one is left in a corner: present in the tree,
|
||||
not finished, not deleted, and not visible in any list. Enough of those and the tree stops
|
||||
matching its own documentation.
|
||||
|
||||
Two rules, and they are the same rule seen from either end:
|
||||
|
||||
|
|
@ -71,73 +93,47 @@ Two rules, and they are the same rule seen from either end:
|
|||
>
|
||||
> **Adding a row requires closing one.** Two things in flight. Never three.
|
||||
|
||||
A row leaves this table in one of exactly two ways. Either it is **finished** —
|
||||
and finished means a description exists in [../architecture/](../architecture/),
|
||||
not that the code works — or it is **deleted**, and deleted means gone from the
|
||||
tree, not marked for deletion. A directory that has said "to be deleted" for
|
||||
months is the corner, not the cure; `projects/schwm/` and `projects/schde/`
|
||||
proved that and are now actually gone.
|
||||
A row leaves this table in one of exactly two ways. Either it is **finished** — and finished
|
||||
means a description exists in [../architecture/](../architecture/), not that the code works —
|
||||
or it is **deleted**, and deleted means gone from the tree, not marked for deletion. A
|
||||
directory that has said "to be deleted" for months is the corner, not the cure.
|
||||
|
||||
### Built, and not connected to anything
|
||||
|
||||
The galling ones. The work exists, is tested, and nothing uses it.
|
||||
|
||||
| | Where it stopped |
|
||||
|---|---|
|
||||
| ~~**The text layer**~~ | **Half closed, and the half that closed is the one that mattered.** The console draws real TrueType through `font_ops` now: cells hold a codepoint, the cell size comes from the font's measured advance, and `kernel/dev/ttf.c` has a caller that is not a self-test. What is still unused is `text.c`'s *proportional* path — the console remains a fixed grid. See Gate 0. |
|
||||
| **A font from disk** | Still true, and now deliberate rather than pending: the console typeface is linked into the kernel image, because a panic during boot has to be readable before there is a filesystem. Loading a *second* font from disk is the open part. |
|
||||
| **Socket activation** | `schinitd` parses `[Socket name]` and remembers the path. Nothing holds the socket. **This is the entire reason for copying launchd**, and it is the half that was not built. |
|
||||
| **`display_ops.wait_vblank`** | In the vtable, implemented by the VBE driver, left null by virtio-gpu. A compositor that needs it cannot rely on it. |
|
||||
| **`projects/schcom/`** | A compositor v0 (about 700 lines) over a wire format that nothing speaks any more: the desktop is Xorg. It is the one thing in the tree that has ever put a client's pixels on screen by itself, and it has no consumer. **Delete it**, or say what it is for. |
|
||||
| **The in-kernel LDLS window** | `vm.c` maps the kernel image a second time in the top 2 GB at every boot, and `include/ldls/ldls.h` describes a `/dev/ldls` loader — the start of loading Linux modules *into the kernel*, which [ldls.md](../architecture/ldls.md) rule 2 says is not how it is done. Nothing calls either. **Delete it**, or decide. |
|
||||
| **`display_ops`** | `acquire` and `wait_vblank` are in the vtable, implemented by one driver at most, with no consumer: Xorg draws through `/dev/fb0`. Kept for the GPU work ([gpu.md](gpu.md)); that is the reason it stays. |
|
||||
| **Socket activation** | `schinitd` parses `[Socket name]` and remembers the path. Nothing holds the socket. This is the entire reason for copying launchd, and it is the half that was not built. |
|
||||
| **`sys_delete`, `sys_show`, `sys_showstat`** | Orphaned entries from the old v1 filesystem's recycle bin. They die with `fs.c`. |
|
||||
|
||||
### Started, and stopped part-way
|
||||
|
||||
| | Where it stopped |
|
||||
|---|---|
|
||||
| **`foundations.md`** | All but item 4 (`setsockopt`) built: descriptor passing, `memfd`, filesystem-named `AF_UNIX`, dynamic linking, and epoll/eventfd/timerfd/signalfd. |
|
||||
| ~~**`hybrid.md`**~~ | **Closed.** All four mechanisms built; H1, H2 and H3 done. H4 — the USB eviction — is *withdrawn* rather than outstanding: USB is the bus peripherals arrive on, not a peripheral, and an eviction that has to leave an identical driver behind has moved nothing. See [hybrid.md](hybrid.md), "Decided: USB stays in the kernel". |
|
||||
| ~~**`ldls.md`**~~ | **Closed.** L0 through L4 built: the loader handles 2,440 stock modules, and both `8139too` and Linux's `e1000` run in a SchOS process through `/dev/pci` with `ping` working over each. The e1000 run is a handover — the kernel was driving that NIC and gave it up. Described in [../architecture/ldls.md](../architecture/ldls.md). Two drivers, one kernel version, one machine. |
|
||||
| **Schcom v0** | 715 lines that composite, over a wire format that is superseded. Frozen: it runs, nothing more is added, and it is kept only because it is the one thing in the tree that has ever put a client's pixels on screen. |
|
||||
| ~~**`debugger.md`**~~ | **Closed.** D0 through D5 built, including the network transport: `schdbgd` bridges TCP to `/dev/dbgro`, ships disabled, binds loopback, and cannot reach a view that changes state. Described in [../architecture/debugging.md](../architecture/debugging.md). |
|
||||
|
||||
**The row LDLS spent, and gave back.** `hybrid.md` closed — H3 built, H4
|
||||
withdrawn — and `ldls.md` took the slot it freed. It has since closed too, so
|
||||
the slot is free again. Worth being honest that this is
|
||||
the largest thing ever to enter this register, and that it was **explicitly not
|
||||
part of 1.0**: nothing in the acceptance test needs a foreign driver. It was
|
||||
taken on because the mechanisms it uses were finished and idle, and because
|
||||
the alternative to it is writing a driver for every device that exists.
|
||||
|
||||
**The row the debugger spent, and gave back.** Adding one requires closing one,
|
||||
and *A released PCI device* closed when H2 landed. It is worth saying plainly
|
||||
that the debugger is not a capability of the OS the way every other row here is:
|
||||
nothing in it ships to a user or appears in the acceptance test. The case for it
|
||||
is that **Gate 0 is a list of bugs to find in a kernel that could not be asked
|
||||
anything** — so it is the instrument the rest of this register is surveyed with,
|
||||
and it earned its slot by making every other row cheaper to close.
|
||||
|
||||
It has closed. **Every plan in `docs/plan/` that was open is now closed**, which
|
||||
means the next thing is Gate 0 itself: the survey, with the instrument built.
|
||||
|
||||
**Closed since this register was written.** *A released PCI device* — release is
|
||||
now symmetric with claim (bus mastering cleared before the DMA is freed, a
|
||||
refusal when the device is the only display), a supervised job starts the
|
||||
replacement, and it is described in
|
||||
[../architecture/devices.md](../architecture/devices.md) under "Taking it back".
|
||||
| **LDLS on real hardware** | `r8169` registers its interface on the laptop and reads its PHY; whether it carries traffic there is **not confirmed**. The native driver is out of the registry and stays as the fallback. |
|
||||
| **The laptop's embedded controller** | The EC is found and served (battery and AC read through it); **SCI queries are not handled**, so events from it are not seen. |
|
||||
| **Epiphany (WebKit)** | `apk add epiphany` installs it; its web-process child dies at start-up ("short read from the peer"). NetSurf runs. Not diagnosed. |
|
||||
|
||||
### Named in a "what it does not do" and still true
|
||||
|
||||
Not corners — these were written down honestly on the day and stayed written
|
||||
down. They are here so the register is the whole picture rather than the
|
||||
flattering part of it.
|
||||
Not corners — these were written down honestly on the day and stayed written down. They are here
|
||||
so the register is the whole picture rather than the flattering part of it. Each is described where
|
||||
it lives.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `/proc/<pid>/exe` is a regular file | so `readlink()` fails and `ps -o comm` prints `[init]` |
|
||||
| `getrusage` | does not distinguish `RUSAGE_CHILDREN` from `RUSAGE_SELF` |
|
||||
| No slab allocator | every subsystem's fixed-size table is a consequence |
|
||||
| No `getsockname`/`getpeername` | and `accept()` does not report an AF_INET peer |
|
||||
| No routing table | first interface wins, which is a rule and not routing |
|
||||
| No page cache | writable `MAP_SHARED` of a file is refused — [kernel.md](../architecture/kernel.md) |
|
||||
| `mprotect` is a no-op, nothing has NX | [kernel.md](../architecture/kernel.md) |
|
||||
| No routing table | first interface wins, which is a rule and not routing — [networking.md](../architecture/networking.md) |
|
||||
| No descriptor garbage collector | a socket passed over itself and never received leaks the pair |
|
||||
| No udev / uevent socket | Xorg is told its input devices — [desktop.md](../architecture/desktop.md) |
|
||||
| No EHCI driver | USB 2.0 ports wired only to EHCI do nothing — [usb.md](../architecture/usb.md) |
|
||||
| Touchpads are mice | no two-finger scroll — [desktop.md](../architecture/desktop.md) |
|
||||
| `ru_stime` is zero | [kernel.md](../architecture/kernel.md) |
|
||||
| No IOMMU | a driver in a process is isolated against bugs, not malice — [devices.md](../architecture/devices.md) |
|
||||
| One audio chip family | AC'97 only; laptops have HDA — [audio.md](../architecture/audio.md) |
|
||||
|
||||
### Frozen — scoped, unstarted, and not to be started
|
||||
|
||||
|
|
@ -145,433 +141,77 @@ Neither cancelled nor in progress. **Do not open these.**
|
|||
|
||||
| | |
|
||||
|---|---|
|
||||
| `projects/schinst/` | the installer, the installation environment, `.sip` |
|
||||
| `projects/schinst/` | the installer, the installation environment, `.sip` (see the open decision above) |
|
||||
| `projects/schfs-linux/` | SchFS as a filesystem for Linux |
|
||||
| [roadmap.md](roadmap.md) §4 | the interactive crash screen |
|
||||
| [roadmap.md](roadmap.md) §6 | SchFS beyond SchOS |
|
||||
| [gpu.md](gpu.md) | amdgpu and nouveau through LDLS — measured, with the work it needs written down |
|
||||
| [finch.md](finch.md) | Finch inside SchOS (the language itself is finished and lives in its own repository) |
|
||||
| [schapi.md](schapi.md) | the API layering, designed and not built |
|
||||
|
||||
Each of those is well argued and none of them is on the path to the acceptance
|
||||
test above. That is the whole test for whether something belongs in this section:
|
||||
**not "is it worth doing" but "does 1.0 fail without it".**
|
||||
Each of those is well argued and none of them is on the path to the acceptance test above. That is
|
||||
the whole test for whether something belongs in this section: **not "is it worth doing" but "does 1.0
|
||||
fail without it".**
|
||||
|
||||
---
|
||||
|
||||
## The four gates
|
||||
## The gates
|
||||
|
||||
In order. **Do not start a gate before the one before it closes**, and closing
|
||||
means a description exists in [../architecture/](../architecture/) — see
|
||||
[the rule](../README.md#the-rule-that-keeps-this-from-rotting).
|
||||
In order. **Do not start a gate before the one before it closes**, and closing means a description exists in
|
||||
[../architecture/](../architecture/).
|
||||
|
||||
### Gate 0 — Fix what is already wrong
|
||||
|
||||
**This comes first, before any new capability.** The desktop work is the
|
||||
temptation and it is the wrong thing to do next: a compositor built on a kernel
|
||||
with an unexplained hang and a socket layer that reports success for things it
|
||||
did not do will fail in ways indistinguishable from its own bugs. That is the
|
||||
same argument that put `netdev_selftest()` before the TCP/IP stack, and it was
|
||||
right then.
|
||||
|
||||
**Things that are actively wrong:**
|
||||
|
||||
- [x] ~~**The `ls -lR` hang.**~~ **Not reproducible.** Instrumented and hunted;
|
||||
see below. Left here rather than deleted because the absence of a result
|
||||
is the result.
|
||||
- [x] ~~**`setsockopt` returns 0 without setting anything.**~~ Six options now
|
||||
genuinely reach lwIP's pcb and everything else is `ENOPROTOOPT`. Round-trip
|
||||
checks in `cmd/socktest`, which a permissive stub cannot pass: it has to
|
||||
read back a value it was set *and* refuse an unknown option.
|
||||
- [x] ~~**`accept4` silently drops its flags.**~~ It is its own table entry now
|
||||
rather than an alias for `accept` — an alias would have had to read a
|
||||
fourth argument that `accept` never passes, inventing a `SOCK_NONBLOCK`
|
||||
out of whatever the caller last left in that register.
|
||||
- [x] ~~**Timestamps on everything created after boot read 1970.**~~ Both
|
||||
`know()` (SchFS) and `now_sec()` (v1) returned `uptime()` under comments
|
||||
saying the kernel had no wall clock. It has had one since `time_init()`
|
||||
started reading the RTC; the comments outlived the condition. Two errors,
|
||||
not one — `uptime()` counts 10 ms ticks and they were used as seconds, so
|
||||
the clocks also ran 100× fast.
|
||||
- [x] ~~**User processes are corrupted on SMP.**~~ **Found and fixed.**
|
||||
`syscall_entry` parked `%rax`, `%rcx` and `%rdx` in three *globals*
|
||||
across the `FS_BASE` swap, under a comment arguing that `SYSCALL`
|
||||
clearing `IF` made it safe. `IF` answers for an interrupt on *this* CPU
|
||||
and says nothing about a second CPU running the same three stores — and
|
||||
`%rcx` is the user return address `SYSRET` uses, so the CPU that lost the
|
||||
race returned its process into another process's instruction stream.
|
||||
|
||||
That is the whole of it: `rip 0x0` was a process handed a zero and
|
||||
following it, and `0xffffffffffffffea` was `-EINVAL` in a swapped `%rax`
|
||||
used as a pointer. Per-CPU words at `%gs:32..48` now. Measured at four
|
||||
CPUs against the same pristine image: **10 faults in 10 boots before, 0
|
||||
in 18 after.** See [../architecture/kernel.md](../architecture/kernel.md),
|
||||
"Per-CPU state".
|
||||
|
||||
- [x] ~~**`$(command substitution)` hangs the shell.**~~ **Not a kernel bug.**
|
||||
`ports/zsh/Makefile` answered configure's "does `kill(pid, 0)` return
|
||||
ESRCH correctly?" with `no`, which makes zsh `#define ESRCH EINVAL`.
|
||||
`waitforpid()` ends its wait on `errno != ESRCH`, so with the kernel
|
||||
correctly returning ESRCH (3) and zsh comparing against EINVAL (22), the
|
||||
shell waited for ever on a child it had already reaped. One word in a
|
||||
cross-compile cache. Autoconf's own default for that check is `yes`;
|
||||
omitting the line would have been right. See
|
||||
[../history/kernel.md](../history/kernel.md), section 7.
|
||||
- [x] ~~**`getrusage` does not distinguish `RUSAGE_CHILDREN`** from
|
||||
`RUSAGE_SELF`.~~ It did not distinguish anything: it ignored `who` and
|
||||
reported the machine's *uptime* as the caller's user time, so every
|
||||
process of every age answered with the same number. Per-process CPU time
|
||||
is sampled at the timer tick now, children's time is rolled up on reap,
|
||||
and `wait4()`'s fourth argument — a `struct rusage *` that was read off
|
||||
the stack and thrown away — is filled. That last one is why zsh's `time`
|
||||
reported `16777216.01s user` for everything: 2^24 seconds is
|
||||
uninitialised memory, and it never varied because it was never written.
|
||||
`time` now says 97% cpu for a busy loop and 0% for `sleep 2`.
|
||||
`ru_stime` is a known zero; see
|
||||
[../architecture/kernel.md](../architecture/kernel.md).
|
||||
- [x] ~~**`/proc/<pid>/exe` is a regular file, not a symlink.**~~
|
||||
`lrwxrwxrwx ... exe -> /bin/schinit`, and `readlink` returns it. It also
|
||||
needed a real `st_size`, which every other procfs file deliberately
|
||||
reports as zero: `readlink(2)` bounds its copy by it, so a symlink of
|
||||
size 0 resolves to the empty string — which looks like it worked and is
|
||||
worse than the regular file was.
|
||||
- [x] ~~**OVMF memory reporting.**~~ The region-list fix did close it. 1 GB
|
||||
reports 1017 MB usable, 2 GB reports 2041 MB, which is the right shape:
|
||||
the firmware keeps about 7 MB either way.
|
||||
- [x] ~~**`ps` printed a header and nothing else.**~~ Found while checking the
|
||||
`exe` fix. `ps` with no arguments shows only processes on *this*
|
||||
terminal, and `/proc/<pid>/stat` hardcoded `tty_nr` to 0, so nothing
|
||||
could ever match — `ps -e` always worked, which is what hid it. Fields
|
||||
14-17 (utime/stime/cutime/cstime) were hardcoded zero too, so `TIME` was
|
||||
`00:00:00` for everything however long it had run.
|
||||
|
||||
**Work already done and not connected:**
|
||||
|
||||
These are the galling ones — the replacement exists, is tested, and is unused.
|
||||
|
||||
- [x] ~~**Point the console at a real typeface.**~~ It draws JetBrains Mono
|
||||
through `font_ops` and `ttf.c`. The blocker was not the font: `vtcell.ch`
|
||||
was **one byte** holding a CP437 index, so `Ą` (U+0104) could not be
|
||||
*stored*, whatever was drawing it. The cell is a `uint32` codepoint now
|
||||
and the cell size comes from the font's measured advance.
|
||||
- [x] ~~**Delete the cell grid.**~~ Done as far as it should be done, and the
|
||||
rest of it retired — which needs explaining, because the item as written
|
||||
asked for something a terminal cannot be.
|
||||
|
||||
`include/gfx/text.h` made the case against the grid in four charges: it
|
||||
cannot do variable-width glyphs, cannot position text off a multiple of
|
||||
eight, cannot anti-alias, and cannot hold a codepoint above 255. **All
|
||||
four were about the drawing layer, and all four are now false** — glyphs
|
||||
carry metrics, PSOGM positions text at arbitrary pixels, the TrueType
|
||||
path blends 8-bit coverage, and `vtcell.ch` is a `uint32`.
|
||||
|
||||
What is left is a grid in `vt.c`, and that one is not in the way: it is
|
||||
what a terminal *is*. VT100 escapes address columns, `TIOCGWINSZ`
|
||||
reports rows and columns, and every program that draws a box assumes
|
||||
both. Deleting it would not modernise the console, it would stop it
|
||||
being a terminal.
|
||||
|
||||
So the real remainder was the duplicate: `gfx.c` walked `font_data`
|
||||
itself and blitted glyphs its own way, while `text.c` did the same job
|
||||
into a `struct surface` for nobody but a self-test. Two implementations
|
||||
that had to agree about a cell size only one of them was told had
|
||||
changed — and they did not agree, which is why every line of PSOGM once
|
||||
drew the next character's bitmap underneath itself. There is one blitter
|
||||
now: `gfx.c` hands `text_draw_glyph` the cell *as a surface of its own*,
|
||||
so clipping a tall glyph to its line is the same bounds check that clips
|
||||
to a screen. The bitmap is reached through `font_builtin()` like any
|
||||
other provider, and `gfx.c` no longer includes `font.h`.
|
||||
- [x] ~~**Load a font from disk**~~ and point the console at it. The built-in
|
||||
one is linked in on purpose — an early panic has to be readable — so this
|
||||
is about a *replaceable* font, not about having one. Drop a TrueType file
|
||||
at `\schos\console.ttf` on the ESP and the console is drawn with it.
|
||||
|
||||
**The loader reads it, not the kernel**, and that is the whole design
|
||||
decision. The kernel could open the file once the root filesystem is
|
||||
mounted — but the console has been drawing since long before that, so
|
||||
switching then means re-measuring the cell and re-laying out every
|
||||
terminal mid-boot, which wipes the kernel log off the screen at exactly
|
||||
the moment a machine being brought up for the first time needs it. The
|
||||
loader allocates the pages as `EfiReservedMemoryType`, so the E820
|
||||
handoff marks them reserved, the page allocator never offers them, and
|
||||
the kernel uses the file *in place* — no copy, and no need for a
|
||||
contiguous allocator the kernel does not have.
|
||||
- [x] ~~**Interrupt-driven receive.**~~ The e1000's handler calls
|
||||
`netdev_rx_signal()`, which ends the network thread's wait early; the frame
|
||||
is still *pulled* by the thread, because an interrupt handler can neither
|
||||
sleep on the buffer cache nor take the lwIP lock. The wait stays on the
|
||||
tick so lwIP's timers keep their floor. Proven rather than assumed:
|
||||
`/proc/netif` gained `wake_rx`/`wake_tick`, and across three pings
|
||||
`rx_frames` and `wake_rx` both went 2 -> 6.
|
||||
|
||||
**Structural, because everything after this compounds on it:**
|
||||
|
||||
- [x] ~~**A slab or pool allocator.**~~ `kernel/core/kmem.c`. Slabs are pages
|
||||
from `kalloc()` with a header at the front; free objects hold the free
|
||||
list in their own first eight bytes, so an object needs no header and
|
||||
freeing needs no search — the object's page base *is* its slab, and the
|
||||
slab names its cache. Two interfaces: named `kmem_cache` for a subsystem
|
||||
with one kind of thing, and `kmalloc`/`kmfree` over power-of-two classes
|
||||
for everything else. Visible in `/proc/slab`, asserted by
|
||||
`make kmem-test` on the host — the same arrangement as `schfs-objtest`,
|
||||
so the bookkeeping is checked in a second rather than in a boot.
|
||||
|
||||
**It does not do what this row said it would, and that is worth
|
||||
recording.** The claim was that every fixed table — `NPROC`, `NSOCK`,
|
||||
`NDEVICE`, `NFILE`, `NINODE`, `NBUF` — exists to work around `kalloc`.
|
||||
Measured: `struct proc` is 10656 bytes, `struct socket` 4928, `struct
|
||||
buf` 4272. **All three are larger than a page**, so a page-backed slab
|
||||
cannot hold them whatever it does; `NINODE` is a cache and is supposed to
|
||||
be bounded; and `NFILE`/`NDEVICE` are 48 and 120 bytes, where the table
|
||||
costs 24 KB and 3.8 KB and the ceiling has never been reached. The tables
|
||||
were not all working around `kalloc`. Freeing the big ones needs a
|
||||
contiguous multi-page allocator, which is a different thing and is not
|
||||
written.
|
||||
|
||||
What it does fix is real and was measured rather than assumed. A pipe is
|
||||
640 bytes and came from a whole 4096-byte page, so every `|` in a shell
|
||||
command burned 3.5 KB it never touched — **six pipes per page now**. And
|
||||
`procread()` built its answer in a 2048-byte array on the kernel stack,
|
||||
under a comment that had warned about exactly that since it was written;
|
||||
its frame is 64 bytes now instead of ~2100.
|
||||
- [x] ~~**SchFS on-disk format version policy.**~~ Done. `SCHFS_VERSION` for
|
||||
frame changes, plus `compat`/`ro_compat`/`incompat` feature flags in what
|
||||
was `reserved0` — so existing images keep every field offset and read
|
||||
unchanged. `uber_check()` now distinguishes *damaged* (step over it; that
|
||||
is what the ring is for) from *too new* (stop), because falling back to an
|
||||
older uberblock mounts a stale filesystem and reports success. Proven on a
|
||||
forged image: the old code mounted the stale root and exited 0.
|
||||
Still owed: an actual upgrade step, which is only writable once there are
|
||||
two formats. See [../architecture/schfs.md](../architecture/schfs.md).
|
||||
|
||||
**Found while doing the above, and fixed:**
|
||||
|
||||
- [x] ~~Every `AF_INET` socket failed with `ENFILE` until DHCP bound.~~
|
||||
`inet_new()` required an *address* before it would hand out a socket, and
|
||||
reported the refusal as "file table overflow". Creating a socket is not
|
||||
the same as having somewhere to send; a bind or connect with no route
|
||||
fails on its own with an error that says so. `net_is_ready()` (lwIP up) is
|
||||
now distinct from `net_is_up()` (DHCP bound).
|
||||
- [x] ~~`socktest` asserted "AF_INET is refused, not faked".~~ True when written,
|
||||
false since AF_INET landed — and **green the whole time**, because the
|
||||
suite runs before DHCP binds and the `ENFILE` above made the refusal look
|
||||
real. A test asserting a failure has to say *which* failure.
|
||||
|
||||
**The suite itself:**
|
||||
|
||||
Both of these were found by running `make schfs-check` rather than by reading
|
||||
it, and both predate the current work — confirmed by stashing and re-running.
|
||||
They are here rather than filed as nice-to-haves because a suite that is red for
|
||||
known reasons stops being read, and then it is red for unknown reasons too.
|
||||
|
||||
- [x] ~~**`make schfs-check` fails at baseline.**~~ Fixed, and it was not only a
|
||||
test bug. The allocation map was taken through a `palloc` callback and
|
||||
there was **no way to give it back** — no `schfs_space_release`, and a
|
||||
comment in `schfs_space_init` reading *"caller frees what was taken"* for
|
||||
an API the caller did not have.
|
||||
|
||||
So the kernel leaked the whole map on **every unmount**, up to 2 MB a
|
||||
time, and `schfs_space_init`'s own error path stranded a page it had
|
||||
already taken because `npages` was only raised after *both* allocations
|
||||
succeeded. The test's 518 leaked pages were the symptom that pointed at
|
||||
it.
|
||||
|
||||
The map now carries its own `pfree` so it can only be freed by the
|
||||
allocator that filled it, `schfs_vol_open`/`schfs_vol_format` release any
|
||||
map the volume already holds, and unmount releases. `schfs_objtest`
|
||||
asserts the balance — 520 pages taken, 520 given back — so the next leak
|
||||
fails the suite instead of being absorbed by it.
|
||||
|
||||
- [x] ~~**31 UndefinedBehaviorSanitizer errors.**~~ Fixed, plus two the test
|
||||
never reached. One root cause: a leaf's value area holds variable-length
|
||||
records packed end to end, so a record begins wherever the last one ended
|
||||
— a multiple of 8 only by luck — and the code read them by casting a
|
||||
pointer into the block, which tells the compiler they are aligned when
|
||||
they are not.
|
||||
|
||||
The seven record types that live at arbitrary offsets now carry
|
||||
`SCHFS_PACKED`. **No on-disk layout changed**, and that was measured
|
||||
before it was applied rather than assumed: every one already had explicit
|
||||
`pad` fields, so sizes and member offsets are identical with and without
|
||||
the annotation. Seven `_Static_assert`s now pin those sizes, because a
|
||||
record that silently grows misreads every existing volume from that point
|
||||
on and looks like corruption rather than like a struct that changed.
|
||||
|
||||
`schfs_node_hdr`, `schfs_item`, `schfs_kptr` and `schfs_uberblock` are
|
||||
deliberately left alone — the node header is exactly 32 bytes, so
|
||||
everything after it is already 8-aligned, and annotating them would cost
|
||||
byte-wise access on the tree's hottest paths to fix a problem they do not
|
||||
have.
|
||||
|
||||
Verified by reading an image written *before* the change: full
|
||||
`schfs_fsck --check` passes on structure, checksums, key ordering, data
|
||||
blocks and derived free space.
|
||||
|
||||
Also caught on the way: `mkschfs` passed a null pointer to `qsort` for an
|
||||
empty directory. Every implementation ignores it when the count is zero;
|
||||
the standard does not say they have to.
|
||||
|
||||
**The identity item:**
|
||||
|
||||
- [x] ~~**Boot from a USB disk.**~~ The root device is probed at boot instead of
|
||||
compiled in: ramdisk (only in a `RAMDISK_ROOT` build), then USB, then the
|
||||
IDE disks, first SchFS volume wins. Verified with **no IDE disk attached
|
||||
at all** — boots, mounts the stick, writes a file, and the file is still
|
||||
there on the next boot. `dmesg` says `root: USB disk (device 3)`, and a
|
||||
machine with no root names every candidate it looked at instead of
|
||||
panicking with "no root filesystem". See
|
||||
[../architecture/boot.md](../architecture/boot.md).
|
||||
|
||||
#### The `ls -lR` hang, and why it is closed
|
||||
|
||||
Hunted with the instrumentation it asked for, under a workload heavier than the
|
||||
one that originally triggered it — ten iterations of `ls -lR /bin`, interleaved
|
||||
with writes and `cd`, 33 execs and 13378 tree descents. Measured across it:
|
||||
|
||||
```
|
||||
inode_peak 9 / 200 unchanged
|
||||
file_peak 9 / 512 unchanged
|
||||
buf_pinned_peak 1 / 256 unchanged
|
||||
schfs0 locked 0 tx_owner_pid 0 tx_depth 0
|
||||
```
|
||||
|
||||
Nothing climbed, nothing wedged, and the repaired `holdingsleep` assertion never
|
||||
fired. **That eliminates this file's own stated hypothesis** — "a lock or
|
||||
reference-count leak somewhere in the `*at()` paths" — and it is very likely the
|
||||
bug was fixed in passing by the `*at()` dirfd rewrite, the getdents cookie
|
||||
rewrite, or SchFS becoming the root, all of which landed after it was reported.
|
||||
|
||||
It is closed rather than chased further, and that is a deliberate application of
|
||||
the two-things-in-flight rule. The instrumentation is permanent: if it ever
|
||||
happens again, `/proc/fsstat` names the table, or shows `locked 1` with
|
||||
`tx_owner_pid 0` and the volume lock is orphaned. That is a much better position
|
||||
than the one it was reported from.
|
||||
|
||||
Two real defects were found on the way and fixed, neither proven to be the
|
||||
cause: `holdingsleep()` never compared the owning pid, which disarmed all seven
|
||||
`panic()` assertions built on it; and SchFS's `tx_mine()` compared a bare
|
||||
`struct proc *`, which a recycled ptable slot makes ambiguous.
|
||||
|
||||
### Gate 1 — The foundations
|
||||
|
||||
[foundations.md](foundations.md), items 1–4. Descriptor passing, a shareable
|
||||
memory object, `AF_UNIX` in the filesystem namespace.
|
||||
|
||||
Nothing above needs these; everything below stops without them.
|
||||
|
||||
- [x] ~~**Item 1 — `sendmsg`/`recvmsg` with `SCM_RIGHTS`.**~~ Built and tested;
|
||||
fourteen checks in `cmd/socktest`, each moving data through the passed
|
||||
descriptor rather than inspecting it. Described in
|
||||
[../architecture/networking.md](../architecture/networking.md#descriptor-passing).
|
||||
This was also mechanism 1 of [hybrid.md](hybrid.md) — one primitive for
|
||||
both, which is the whole reason those are one plan.
|
||||
- [x] ~~**Item 2 — a shareable memory object.**~~ `memfd_create`, `ftruncate`,
|
||||
`mmap(MAP_SHARED)`. Built on the `shmget` segments exactly as
|
||||
[foundations.md](foundations.md) proposed — the pages, refcounts,
|
||||
`no_free` VMAs and the fork/exit accounting were all already right; what
|
||||
memfd adds is a segment reached by *descriptor* rather than by name or
|
||||
System V key, which is the whole reason it exists on Linux.
|
||||
`cmd/memfdtest`: seventeen checks, ending with a memfd sent over a
|
||||
socket to a receiver that closes every inherited copy first.
|
||||
- [x] ~~**Item 3 — `AF_UNIX` sockets in the filesystem namespace.**~~ `bind()`
|
||||
creates a `T_SOCK` node; `connect()` resolves the path and matches the
|
||||
socket bound to *that inode*, not to a string. `ls -l` shows
|
||||
`srwxr-xr-x`, `stat()` reports `S_IFSOCK`, `unlink()` removes it, and a
|
||||
second bind to a live path is `EADDRINUSE` *from the directory* — which
|
||||
is what makes a socket file left by a crashed server refuse a rebind, as
|
||||
it must. `cmd/socktest` section 15. `flock()` is still missing and is
|
||||
recorded as such.
|
||||
|
||||
### Gate 2 — Init as a role — **done**
|
||||
|
||||
Milestones I0–I3, all built. PID 1 forks a supervisor and spawns jobs on its
|
||||
behalf; the job table lives in `/proc/schinit`; a heartbeat runs both ways; and
|
||||
`proc_reincarnate()` re-execs PID 1 in place rather than panicking. Both failure
|
||||
paths are exercised by `sysrq` keys rather than reasoned about.
|
||||
|
||||
Before the desktop rather than after, because a compositor being brought up for
|
||||
the first time crashes constantly, and something that restarts it and says why is
|
||||
worth far more during that work than at the end of it.
|
||||
|
||||
The description is now [../architecture/init.md](../architecture/init.md); the
|
||||
plan document it replaced has been deleted, per
|
||||
[the rule](../README.md#the-rule-that-keeps-this-from-rotting).
|
||||
|
||||
### Gate 3 — The desktop
|
||||
|
||||
[xorg.md](xorg.md): Xorg on the framebuffer, twm and xterm, then Window
|
||||
Maker. Wayland ([wayland.md](wayland.md)) is not being worked on -- what it
|
||||
needs, DRM/KMS above all, is far larger than what X needs.
|
||||
|
||||
**W4 and W5 are not in 1.0.** Multi-window focus policy and a panel are
|
||||
post-1.0. One terminal window that works is the bar.
|
||||
| Gate | What | State |
|
||||
|---|---|---|
|
||||
| **0 — Fix what is already wrong** | the SMP register corruption, `setsockopt` that lied, timestamps that read 1970, the `ls -lR` hang (hunted, not reproducible — [history](../history/kernel.md) §24), … | **closed** |
|
||||
| **1 — The foundations** | descriptor passing, a shareable memory object (`memfd`), `AF_UNIX` in the filesystem, dynamic linking, the event descriptors | **closed** — [kernel.md](../architecture/kernel.md), [networking.md](../architecture/networking.md) |
|
||||
| **2 — Init as a role** | `schinit`, the supervisor, the heartbeat, PID 1 re-exec'd in place | **closed** — [init.md](../architecture/init.md) |
|
||||
| **3 — The desktop** | Xorg on the framebuffer, Window Maker, a terminal | **closed** — [desktop.md](../architecture/desktop.md) |
|
||||
| **4 — A root that survives a reboot** | the open decision above | **open** |
|
||||
|
||||
---
|
||||
|
||||
## Not in 1.0
|
||||
|
||||
The point of this list is that it is *written down*, so it can stop occupying
|
||||
attention. Nothing here is cancelled.
|
||||
The point of this list is that it is *written down*, so it can stop occupying attention. Nothing here is cancelled.
|
||||
|
||||
| | Why it waits |
|
||||
|---|---|
|
||||
| **The rest of the hybrid kernel** ([hybrid.md](hybrid.md)) | Still not a completeness requirement — but the mechanisms and the first eviction happened early, because the alternative to a driver compatibility layer is writing a driver for every device that exists, and that is not a plan. What is deferred is the foreign driver host, not the interface it needs. |
|
||||
| **Mesa, SDL3** | The kernel and musl do dynamic linking now (`cmd/dynhello`), but every shipped program is still static. Mesa is the single biggest post-1.0 unlock and it is a whole project. |
|
||||
| **A page cache** | `memfd` covers the desktop's needs. General writable `MAP_SHARED` of a file is a real project with no 1.0 consumer. |
|
||||
| **`.sip` and the installer** | 1.0 installs by writing a USB stick. That is how most systems start and it is honest. |
|
||||
| **SchFS snapshots, FUSE, Linux module, sparse files, online fsck** | The format version (Gate 0) is the prerequisite for all of them, and it is the only part on a clock. |
|
||||
| **The interactive crash screen** | PSOGM renders, prints a full report and emits a QR code. That is enough. |
|
||||
| **GPU acceleration, Mesa** | [gpu.md](gpu.md). A whole project; software rendering is enough to use the machine. |
|
||||
| **A page cache** | General writable `MAP_SHARED` of a file is a real project with no 1.0 consumer. |
|
||||
| **SchFS snapshots, FUSE, a Linux module, sparse files, online fsck** | The format version is the prerequisite and it exists; the rest is [roadmap.md](roadmap.md) §6. |
|
||||
| **CFF/OTF fonts, kerning, text layout** | TrueType and fixed advances are enough for a terminal. |
|
||||
| **Routing, `getsockname`/`getpeername`, IPv6** | One interface, one default route, works. |
|
||||
| **Multiple outputs, damage tracking, subsurfaces, xkbcommon** | One screen, full repaints, one keymap. |
|
||||
| **Audio** | Does not exist in any form and is not required to use a machine. |
|
||||
| **Panel, notifications, settings, file manager** | A terminal is a desktop. |
|
||||
| **SchAPI itself** | Deliberately. It should be designed *from* the compositor's experience, not before it. |
|
||||
| **Routing, IPv6, Wi-Fi** | One interface, one default route, works. |
|
||||
| **HDA audio, recording, mixing** | AC'97 plays; that is sound. |
|
||||
| **Threads' loose ends, `ptrace`, `seccomp`** | Whatever a package needs is found by running it. |
|
||||
| **SchAPI itself** | Deliberately. It should be designed from the desktop's experience, not before it. |
|
||||
|
||||
---
|
||||
|
||||
## The rule for this list
|
||||
|
||||
**Adding to Gate 0–3 requires deleting something else from it.**
|
||||
**Adding to the gates requires deleting something else from them.**
|
||||
|
||||
Not a productivity trick — a forcing function. The reason this project feels
|
||||
overwhelming is that ~20 fronts have been open simultaneously, and no amount of
|
||||
discipline makes that workable. The fix is structural.
|
||||
Not a productivity trick — a forcing function. The reason this project felt overwhelming is that ~20 fronts
|
||||
were open simultaneously, and no amount of discipline makes that workable. The fix is structural.
|
||||
|
||||
And one working limit, worth holding to literally:
|
||||
|
||||
> **Two things in flight. Never three.**
|
||||
|
||||
One in Gate 0 and one being written up. Everything else is closed — in the
|
||||
document, and out of your head.
|
||||
|
||||
---
|
||||
|
||||
## How you will know it is going well
|
||||
|
||||
Not by velocity. By this:
|
||||
|
||||
**Every closed item leaves a description behind.** When the `ls -lR` hang is
|
||||
fixed, `history/kernel.md` gains an entry saying what it actually was. When the
|
||||
console moves onto the text layer, `architecture/graphics.md` loses a line from
|
||||
its *What it does not do* section.
|
||||
|
||||
Those "does not do" sections shrinking is the real progress bar, and it is the
|
||||
only one that does not lie.
|
||||
|
||||
---
|
||||
Not by velocity. By this: **every closed item leaves a description behind.** When the `ls -lR` hang was
|
||||
closed, `history/kernel.md` gained an entry. When a gap closed, a "what it does not do" section shrank.
|
||||
Those sections shrinking is the real progress bar, and it is the only one that does not lie.
|
||||
|
||||
## A note worth keeping
|
||||
|
||||
This document was written because the work started to feel overwhelming, and
|
||||
that is worth recording rather than tidying away — because the feeling was
|
||||
about the *shape* of the work and not its quality, and the two are easy to
|
||||
confuse when you are inside it.
|
||||
This document was written because the work started to feel overwhelming, and that is worth recording rather
|
||||
than tidying away — because the feeling was about the *shape* of the work and not its quality, and the two are
|
||||
easy to confuse when you are inside it.
|
||||
|
||||
The evidence at the time: SMP working on real hardware after five bugs stacked
|
||||
behind each other; a copy-on-write filesystem whose crash test cuts the power one
|
||||
write later, 119 times, and demands a clean mount after each; a TCP/IP stack
|
||||
asserted from both ends; an xHCI driver written from the specification; and a
|
||||
documentation tree that records why things are strange so they do not get
|
||||
un-fixed by accident.
|
||||
The evidence at the time: SMP working on real hardware after five bugs stacked behind each other; a copy-on-write
|
||||
filesystem whose crash test cuts the power one write later, 119 times, and demands a clean mount after each; a
|
||||
TCP/IP stack asserted from both ends; an xHCI driver written from the specification; and a documentation tree that
|
||||
records why things are strange so they do not get un-fixed by accident. Since then: a desktop, real Linux
|
||||
programs and Linux's own drivers, running.
|
||||
|
||||
None of that is the output of a project in trouble. It was a project with too
|
||||
many tabs open.
|
||||
None of that is the output of a project in trouble. It was a project with too many tabs open.
|
||||
|
|
|
|||
|
|
@ -1,62 +1,47 @@
|
|||
# What happens next
|
||||
|
||||
Everything in this directory is **unbuilt**. When a piece of it is finished it
|
||||
leaves — deleted from here, rewritten as a description in
|
||||
[../architecture/](../architecture/). See [the rule](../README.md#the-rule-that-keeps-this-from-rotting).
|
||||
Everything in this directory is **unbuilt**. When a piece of it is finished it leaves — deleted from here,
|
||||
rewritten as a description in [../architecture/](../architecture/). See
|
||||
[the rule](../README.md#the-rule-that-keeps-this-from-rotting).
|
||||
|
||||
> ## Read [1.0.md](1.0.md) first.
|
||||
>
|
||||
> It draws a finish line and puts most of the work below on the far side of it.
|
||||
> **Anything not named there is not being worked on**, and the documents in this
|
||||
> directory describe more than 1.0 contains.
|
||||
>
|
||||
> It also holds **[the register of unfinished work](1.0.md#the-freeze-and-the-register-of-unfinished-work)**,
|
||||
> which is the one place anything half-built is listed. No new capability starts
|
||||
> until that register is empty, and a row leaves it only by being finished or by
|
||||
> being deleted from the tree.
|
||||
> It draws the finish line, says where the system stands against it, and holds
|
||||
> **[the register of unfinished work](1.0.md#the-freeze-and-the-register-of-unfinished-work)**
|
||||
> — the one place anything half-built is listed. No new capability starts until that register is
|
||||
> empty, and a row leaves it only by being finished or by being deleted from the tree.
|
||||
|
||||
---
|
||||
|
||||
## The three current decisions
|
||||
|
||||
Taken together, because they turn out to need the same foundation.
|
||||
## What is here
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **[alpine.md](alpine.md)** | **Packages.** The root is managed by Alpine's apk: Alpine's binaries run unmodified, and SchOS itself is the apk package schos-base. |
|
||||
| **[xorg.md](xorg.md)** | **The desktop.** Xorg on the framebuffer with the fbdev and evdev drivers; twm, then Window Maker. Replaces wayland.md for Gate 3. |
|
||||
| **[wayland.md](wayland.md)** | *Not being worked on; see xorg.md.* The GUI stack becomes Wayland. SchWM and SchDE are **deleted** — actually gone from the tree, not marked; Schcom survives and changes its wire format. |
|
||||
| ~~**hybrid.md**~~ | **Built.** All four mechanisms exist, the GPU driver runs outside the kernel, it survives being killed, and the display mode can be changed from a process. H4 (USB) was *withdrawn* rather than left open — USB is the bus peripherals arrive on, not a peripheral. |
|
||||
| ~~init.md~~ | **Built.** PID 1 is now a role the kernel refills; the description moved to [../architecture/init.md](../architecture/init.md). |
|
||||
| **[1.0.md](1.0.md)** | **The finish line.** What 1.0 is, the acceptance test and where each clause stands, the register, and everything explicitly deferred. |
|
||||
| **[roadmap.md](roadmap.md)** | Everything else that is scoped and agreed: storage and the installer, graphics, networking, the hardware that makes a laptop a laptop, SchFS beyond SchOS, the kernel's own gaps. |
|
||||
| **[gpu.md](gpu.md)** | Linux's own `amdgpu` through LDLS, and the `/dev/dri` a process has to serve. Measured, with the work it needs written down. |
|
||||
| **[finch.md](finch.md)** | **Later.** The author's own LLVM language for SchOS's applications and tools in place of C. The language is finished and self-hosting in its own repository; the plan here is a target, bindings generated from `include/uapi/`, and the compiler in the image. |
|
||||
| **[schapi.md](schapi.md)** | The long-term API layering and the compatibility promise. Direction agreed, nothing built. |
|
||||
|
||||
And the one they all wait on:
|
||||
## What used to be here, and is built
|
||||
|
||||
| | |
|
||||
Plans leave this directory when they are done. These did, and what replaced them is the description:
|
||||
|
||||
| Was | Now |
|
||||
|---|---|
|
||||
| **[foundations.md](foundations.md)** | Descriptor passing, shared memory, sockets in the filesystem. The critical path. **Item 1 is built** — `sendmsg`/`recvmsg` with `SCM_RIGHTS`. |
|
||||
| Descriptor passing, shared memory, sockets in the filesystem, dynamic linking, the event descriptors (`foundations.md`) | [kernel.md](../architecture/kernel.md), [networking.md](../architecture/networking.md), [syscalls.md](../architecture/syscalls.md) |
|
||||
| `init.md` | [init.md](../architecture/init.md) |
|
||||
| `hybrid.md` — drivers in processes, `/dev/pci`, a modeset API, the panic path, why USB stays | [devices.md](../architecture/devices.md) |
|
||||
| `ldls.md` — Linux drivers in a process | [ldls.md](../architecture/ldls.md) |
|
||||
| `debugger.md` | [debugging.md](../architecture/debugging.md) |
|
||||
| `alpine.md` — an apk-managed root | [packages.md](../architecture/packages.md) |
|
||||
| `xorg.md` — the desktop | [desktop.md](../architecture/desktop.md) |
|
||||
| The interactive crash screen (`roadmap.md` §4) | GMMOS, in [graphics.md](../architecture/graphics.md) |
|
||||
|
||||
And the one that says which parts of the above are actually being worked on:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **[1.0.md](1.0.md)** | The finish line. Four gates, and everything else explicitly deferred. |
|
||||
|
||||
And the one the hybrid mechanisms were built for:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| ~~ldls.md~~ | **Built.** The Linux Driver Layer for SchOS: someone else's drivers, in a process, reached through `/dev/pci`. `ping` works over `8139too` and over `e1000`. The description moved to [../architecture/ldls.md](../architecture/ldls.md). |
|
||||
| ~~debugger.md~~ | **Built**, all of it, including the network transport. A debugger that observes rather than halts and answers in this kernel's own vocabulary: symbols, a log that outlives the kernel, `/dev/dbg`, `whose`, hardware watchpoints, and `schdbgd`. The description moved to [../architecture/debugging.md](../architecture/debugging.md). |
|
||||
|
||||
And one for after 1.0, recorded so the decision is not lost:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **[finch.md](finch.md)** | **Later.** The author's own LLVM language, Finch, for SchOS's applications and tools in place of C. Already self-hosting; the plan is a target, bindings generated from `include/uapi/`, and the compiler in the image. |
|
||||
|
||||
Two longer-horizon documents sit behind these: [schapi.md](schapi.md), the API
|
||||
layering and the compatibility promise, and [roadmap.md](roadmap.md), which holds
|
||||
everything still open that is not one of the three above.
|
||||
And one that was **decided against**: `wayland.md`. Wayland needs DRM/KMS, libinput, udev and libseat, and DRM/KMS
|
||||
alone is a kernel graphics subsystem. The desktop is X. The reasoning moved to [desktop.md](../architecture/desktop.md).
|
||||
`SchWM` and `SchDE`, which were to be the window manager and the desktop on top of it, are gone from the tree: Window
|
||||
Maker is the window manager, and a panel or launcher is an ordinary client.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -64,86 +49,17 @@ everything still open that is not one of the three above.
|
|||
|
||||
Set by [1.0.md](1.0.md), which is the authority. In short:
|
||||
|
||||
### 0. Fix what is already wrong — **first**
|
||||
|
||||
Not a document of its own; it is Gate 0 of [1.0.md](1.0.md), assembled from the
|
||||
*What it does not do* sections in [../architecture/](../architecture/).
|
||||
|
||||
**Before any new capability**, and the reason is the one that put
|
||||
`netdev_selftest()` before the TCP/IP stack: a compositor built on a kernel with
|
||||
an unexplained hang and a socket layer that reports success for things it did not
|
||||
do will fail in ways indistinguishable from its own bugs.
|
||||
|
||||
### 1. `foundations.md` — the critical path
|
||||
|
||||
Both the Wayland work and the hybrid work stop here. Descriptor passing and
|
||||
shared memory are the substrate for handing a buffer to a compositor *and* for
|
||||
handing a device to a driver process. **Doing it once for both is the entire
|
||||
reason these are one plan rather than three.**
|
||||
|
||||
### 2. `init.md` — before the desktop, not after — **done**
|
||||
|
||||
A compositor being brought up for the first time crashes constantly, and
|
||||
something that restarts it and says why is worth far more during that work than
|
||||
at the end of it. Built; see [../architecture/init.md](../architecture/init.md).
|
||||
|
||||
### 3. `hybrid.md` — **moved up, and the reason is worth recording**
|
||||
|
||||
This document used to put the hybrid work *after 1.0*, deliberately last, on this
|
||||
argument:
|
||||
|
||||
> designing a userspace driver interface with no driver consuming it produces an
|
||||
> interface that is wrong in ways nobody can see
|
||||
|
||||
The argument was sound and the conclusion drawn from it was wrong. What forced
|
||||
the change was the driver situation: SchOS cannot write a driver for every device
|
||||
that exists, no project can, and the only route to the hardware support an
|
||||
established OS has is a compatibility layer with the foreign drivers kept
|
||||
*outside* the kernel. That layer is this plan's mechanisms. It is not optional and
|
||||
it is not late work.
|
||||
|
||||
The original argument still holds, and it is answered rather than ignored:
|
||||
**SchOS's own drivers are the consumers.** `vgpud` is a real driver, with real
|
||||
requirements, and building the interface against it is what keeps it a general
|
||||
mechanism rather than a hole shaped like something foreign. The compositor is
|
||||
still a demanding consumer — it is no longer the *only* one available.
|
||||
|
||||
Four mechanisms, all built. One eviction done, the second withdrawn with its
|
||||
reasons recorded, and the foreign driver host they were for is
|
||||
[../architecture/ldls.md](../architecture/ldls.md) — built, and carrying
|
||||
packets.
|
||||
|
||||
### 4. `wayland.md` — with the GPU driver either side of the seam
|
||||
|
||||
W1–W3 and a terminal. W4 and W5 are post-1.0. Item 1 of `foundations.md` —
|
||||
descriptor passing — was the thing it could not start without, and it now
|
||||
exists.
|
||||
|
||||
---
|
||||
|
||||
## In one sentence
|
||||
|
||||
> Fix what is broken, build the descriptor-passing and shared-memory substrate,
|
||||
> make init survivable, move the drivers out through an interface SchOS's own
|
||||
> drivers have to live with first — and put a terminal on the screen.
|
||||
|
||||
---
|
||||
1. **Close the one open gate**: decide what makes a root survive a reboot — the installer, or a narrower
|
||||
definition of 1.0 (see *The one open decision* there).
|
||||
2. **Empty the register.** Delete `projects/schcom/` and the unused in-kernel LDLS window, or say what they are for; build
|
||||
socket activation or remove the key.
|
||||
3. **Then**, and only then, whatever [roadmap.md](roadmap.md) says is next. GPU acceleration is the thing the laptop
|
||||
wants most ([gpu.md](gpu.md)), and Finch inside SchOS is the thing the author wants next ([finch.md](finch.md)).
|
||||
|
||||
## What this plan does not address
|
||||
|
||||
Stated here so it is not mistaken for covered:
|
||||
|
||||
- **No IOMMU**, so the hybrid work isolates against bugs and not against a
|
||||
hostile driver.
|
||||
- **No page cache**, so `foundations.md` delivers a `memfd` and not a general
|
||||
answer to shared file mappings. The general answer is still owed.
|
||||
- **No garbage collector for descriptors in flight**, so an `AF_UNIX` socket
|
||||
passed over itself and never received leaks. Bounded at thirty-two sockets, and
|
||||
nothing on this plan does it.
|
||||
- **No dynamic linking**, so no Mesa, no accelerated clients, and no third party
|
||||
shipping a `.so` for anything.
|
||||
- **A new compatibility surface** between PID 1 and PID 2, created by a plan
|
||||
whose own [schapi.md](schapi.md) §3 spends a section warning about those.
|
||||
- **The three open networking items** — `getsockname`, routing, and
|
||||
interrupt-driven receive — are untouched by any of this and still open. They
|
||||
are in [roadmap.md](roadmap.md).
|
||||
- **No IOMMU**, so a driver in a process is isolated against its own bugs and not against a hostile driver.
|
||||
- **No page cache**, so a general writable `MAP_SHARED` of a file is refused; `memfd` is the narrow answer.
|
||||
- **No installer**, so nothing a live system writes survives a reboot.
|
||||
|
|
|
|||
|
|
@ -1,80 +0,0 @@
|
|||
# apk: Alpine's packages on SchOS
|
||||
|
||||
Alpine is musl with a different init, and SchOS is musl with a different
|
||||
kernel. Alpine's binaries run on SchOS unmodified: its dynamic linker, its
|
||||
busybox, and apk itself, fetching over TLS through SchOS's network stack.
|
||||
So instead of porting programs one at a time, SchOS can install them.
|
||||
|
||||
## The model
|
||||
|
||||
The root filesystem is an apk-managed root:
|
||||
|
||||
| Part | Where it comes from |
|
||||
|---|---|
|
||||
| musl, busybox, apk-tools, alpine-keys, CA certificates | Alpine's minirootfs (`ports/alpine`) |
|
||||
| what `ports/alpine/world` lists (zsh, terminfo, ...) | Alpine's packages, pinned in `world.lock` |
|
||||
| schinit, its jobs, SchOS's tools, toybox, mksh, configuration | **schos-base**, an apk package the build makes from `sysroot/` |
|
||||
|
||||
`apk add` installs anything from Alpine's main and community repositories.
|
||||
`apk upgrade` updates Alpine's packages, and schos-base from the SchOS
|
||||
repository at `/var/lib/schos/repo`.
|
||||
|
||||
PID 1 is still schinit. Alpine's OpenRC is not installed, and packages that
|
||||
ship OpenRC services install them where nothing reads them.
|
||||
|
||||
## Who owns what
|
||||
|
||||
Where Alpine and SchOS both have a program, Alpine's wins, and SchOS keeps
|
||||
only what is its own (see `tools/posix/mkaroot.sh`):
|
||||
|
||||
- `/bin/sh`, `cat`, `sed`, `find` and the rest are busybox. Alpine's package
|
||||
scripts are written against busybox.
|
||||
- zsh is Alpine's. SchOS keeps its `zshrc` and `zshenv` (`replaces=zsh`).
|
||||
- `/etc/os-release` and `/etc/motd` are SchOS's (`replaces=`).
|
||||
- `passwd`, `group` and `profile` are Alpine's. schos-base's post-install
|
||||
sets root's shell to zsh, and SchOS's profile is `/etc/profile.d/schos.sh`.
|
||||
- mksh is `/bin/mksh`. schinit's recovery shell looks for it first because it
|
||||
is static: a broken musl cannot take recovery down with it.
|
||||
|
||||
## Building
|
||||
|
||||
make qemu boots build/schfs.img, the root as a 1 GB SchFS disk
|
||||
make iso the same root as the live system's ramdisk
|
||||
make alpine-lock re-resolve world against Alpine (needs the network)
|
||||
|
||||
It is the only root. sysroot/ is what schos-base is packaged from.
|
||||
|
||||
Only `alpine-lock` asks Alpine what is current. A normal build installs
|
||||
exactly what `world.lock` names: files in `dist/alpine/`, fetched once and
|
||||
checked against the lock's sha256, and against Alpine's signatures by apk.
|
||||
|
||||
schos-base and its repository index are signed with a key made on the first
|
||||
build (`build/apk-keys/`, never committed). The root trusts it beside
|
||||
Alpine's keys.
|
||||
|
||||
## Kernel work this needed
|
||||
|
||||
All of it found by running apk:
|
||||
|
||||
- **TCP receive window:** it reopened before the reader drained, so lwIP
|
||||
refused data and the sender waited on retransmission timers (5 KB/s). It
|
||||
now reopens on read, with a 32-segment window.
|
||||
- **mmap address reuse:** freed ranges were never reused, so musl's malloc
|
||||
used up the 256 mapping records with 780 MB of RAM free.
|
||||
- **unlinkat / renameat:** bare `-1`, read as EPERM, where apk expects ENOENT;
|
||||
renameat could not replace its target, and wrote v1 dirents into SchFS.
|
||||
- **`#!` scripts in execve**, and ENOEXEC: apk runs triggers directly.
|
||||
- **chroot**, and mounting devtmpfs and proc more than once.
|
||||
|
||||
## Next
|
||||
|
||||
- The live ISO runs the apk root from RAM. The image in the kernel is only
|
||||
what the root holds; at boot the volume grows into half of the free memory
|
||||
(4 GB of root on an 8 GB machine), backed by pages as they are written.
|
||||
Xorg installs into it in about a minute. All of it is gone at reboot. Keeping packages needs an installer that writes the root
|
||||
to a SATA or NVMe disk, and a kernel that finds its root there by the
|
||||
SchFS GPT type rather than as IDE disk 1.
|
||||
- Xorg and Window Maker are in `world`, so the image starts with them:
|
||||
`startx` runs `wmaker` from `/root/.xinitrc`. Mesa brings LLVM with it,
|
||||
about 200 MB of the 358 MB root; the root ships compressed
|
||||
([SCFF](../architecture/scff.md)), so that costs about 60 MB of ISO.
|
||||
|
|
@ -8,8 +8,8 @@
|
|||
Finch repository and does not touch this tree. It can proceed now; see
|
||||
[the gap](#the-self-hosting-gap).
|
||||
|
||||
Finch is the author's own language: <https://pandor.pl/git/nexoniarz/Finch>
|
||||
(GitHub mirror: `Nexoniarz/Finch`). The plan is to write SchOS's own
|
||||
Finch is the author's own language: <https://github.com/Nexoniarz/Finch> (mirrored to
|
||||
`pandor.pl/git/nexoniarz/Finch`, which is read-only). The plan is to write SchOS's own
|
||||
applications and tools in it, in place of C, and so leave behind what C does
|
||||
badly: silent narrowing, signed/unsigned mixing, undefined behaviour on an
|
||||
out-of-range index, a forgotten `free`.
|
||||
|
|
@ -45,7 +45,7 @@ Measured from the repository, not from its README alone:
|
|||
|
||||
1. **A `schos` target.** `finch` picks a triple, a C compiler and a linker per
|
||||
target. SchOS runs unmodified Alpine binaries and Alpine's musl
|
||||
(see [alpine.md](alpine.md)), so the first target is **the existing
|
||||
(see [packages](../architecture/packages.md)), so the first target is **the existing
|
||||
`x86_64-unknown-linux-musl`**, and a Finch program built on the host should
|
||||
run on SchOS before anything in Finch is made SchOS-specific. Test that
|
||||
before writing a line of target code.
|
||||
|
|
|
|||
|
|
@ -1,199 +0,0 @@
|
|||
# Foundations
|
||||
|
||||
The kernel work that [wayland.md](wayland.md) and [hybrid.md](hybrid.md) both
|
||||
stop at. **This is the critical path**, and it is the reason those two are one
|
||||
plan rather than two.
|
||||
|
||||
The insight is small and it decides the schedule: the substrate for passing a
|
||||
buffer to a compositor and the substrate for handing a device to a driver process
|
||||
are the same substrate. Descriptor passing and shared memory, built once.
|
||||
|
||||
---
|
||||
|
||||
## What is actually in the tree
|
||||
|
||||
Measured against the source, not against a README. Each of these was checked
|
||||
before anything below was planned around it:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| ~~`sendmsg`/`recvmsg`, and therefore **`SCM_RIGHTS`**~~ | **Built.** See item 1 below. |
|
||||
| Writable `MAP_SHARED` of a file | **Refused** — `kernel/sys/sys_mem.c:131` returns `-ENODEV`. |
|
||||
| `memfd_create` | Does not exist. |
|
||||
| `AF_UNIX` names | In a table in `kernel/net/unix.c`, not in the filesystem. |
|
||||
| `setsockopt` | `return 0;` unconditionally — `kernel/sys/sys_net.c:436`. |
|
||||
| `exec` | `ET_EXEC` only. No `PT_INTERP`, no `ET_DYN`. |
|
||||
|
||||
> **`projects/schcom/README.md` said `SCM_RIGHTS` was done while nothing in the
|
||||
> kernel implemented it** — the most load-bearing wrong line in the tree, with an
|
||||
> entire GUI plan drawn on top of it. The line is now true, which is not the same
|
||||
> as it having been right, and the README says so.
|
||||
|
||||
The `MAP_SHARED` refusal is worth defending rather than fixing carelessly. The
|
||||
code returns `-ENODEV` *honestly*: the contract of a writable shared mapping is
|
||||
that stores land in the file and other mappers see them, and there is no page
|
||||
cache that could make that true. Quietly handing back a private copy would be the
|
||||
same lie in a new place.
|
||||
|
||||
---
|
||||
|
||||
## The six items
|
||||
|
||||
In dependency order. The first three are what Wayland cannot start without.
|
||||
All but item 4 are built.
|
||||
|
||||
### 1. `sendmsg` / `recvmsg` with `SCM_RIGHTS` — **done**
|
||||
|
||||
Not optional and not substitutable. `wl_shm_create_pool` passes a **file
|
||||
descriptor**; there is no path in the Wayland protocol that passes an integer
|
||||
instead.
|
||||
|
||||
The `handle_kind` field in `projects/schcom/proto.h` was good foresight and it
|
||||
cannot help here — *our* protocol is the flexible one, and Wayland's is not.
|
||||
|
||||
Built as described: `struct msghdr`/`cmsghdr` in Linux's layout, ancillary data
|
||||
queued alongside the byte stream in `kernel/net/unix.c`, descriptor installation
|
||||
into the receiver's file table, and the three ways a message can end — received,
|
||||
received without room, never received — each releasing the kernel's reference.
|
||||
Fourteen checks in `cmd/socktest`, every one of them moving data through the
|
||||
descriptor rather than inspecting it. Documented in
|
||||
[../architecture/networking.md](../architecture/networking.md#descriptor-passing).
|
||||
|
||||
The delivery rule is the part worth knowing before building on it: **descriptors
|
||||
are pinned to the stream position of the first byte of the message that carried
|
||||
them**, and a read that has not reached that position is cut short so it stops
|
||||
before it. That is what keeps "which message did these belong to" answerable.
|
||||
|
||||
**This was also item 1 of the hybrid work**, where it is how a driver process is
|
||||
handed the device it is allowed to touch.
|
||||
|
||||
Two bugs fell out of building it, neither in the new code:
|
||||
|
||||
- `sockpair()` never set the socket's domain, so **every `socketpair()` socket
|
||||
claimed domain 0**. Invisible for as long as the data paths asked only whether
|
||||
a socket was `AF_INET` or `AF_NETLINK` — zero answers "neither" correctly.
|
||||
`SCM_RIGHTS` asks "is this `AF_UNIX`", and zero is wrong for that one.
|
||||
- `pipe_poll()` reports `POLLHUP` but not `POLLIN` for a pipe at end of file, so
|
||||
a **non-blocking read of a pipe whose writers have all gone returned `EAGAIN`
|
||||
forever** instead of 0. A program that polls and reads would spin waiting for
|
||||
an end that had already happened. `sockreadable()` always had the equivalent
|
||||
condition; the pipe path did not.
|
||||
|
||||
### 2. A shareable memory object — **done**
|
||||
|
||||
`memfd_create`, plus `mmap(MAP_SHARED)` on it genuinely working between two
|
||||
processes. Described in
|
||||
[../architecture/kernel.md](../architecture/kernel.md#shared-memory-objects);
|
||||
asserted by `cmd/memfdtest`, seventeen checks including the one that matters —
|
||||
a descriptor sent over a socket, mapped by a receiver that has closed every
|
||||
inherited copy, with stores visible in both directions.
|
||||
|
||||
Narrower than a general page cache, and sufficient. An anonymous shared object
|
||||
needs no file behind it, which is exactly why `memfd` exists on Linux at all —
|
||||
and it means this can be built on the shared-memory machinery already in
|
||||
`kernel/sys/sys_mem.c` (`shmget`/`shmat`/`shmdt` work, and `struct vma` already
|
||||
carries `shmid` and a `no_free` flag for pages that are not ours to release).
|
||||
|
||||
The general answer — a page cache, and writable `MAP_SHARED` of a real file —
|
||||
stays owed. It is a larger project and nothing on this plan needs it.
|
||||
|
||||
### 3. `AF_UNIX` sockets in the filesystem namespace — **done**
|
||||
|
||||
Clients find a Wayland compositor by `bind()`ing `$XDG_RUNTIME_DIR/wayland-0` as
|
||||
a real path, and libwayland `flock()`s the lock file beside it. A name table can
|
||||
express neither.
|
||||
|
||||
`bind()` creates a `T_SOCK` node and `connect()` resolves the path and matches
|
||||
the socket bound to **that inode** — not to a string, which is what made two
|
||||
spellings of one path miss each other. `ls -l` shows `srwxr-xr-x`, `stat()`
|
||||
reports `S_IFSOCK`, and `unlink()` removes it. Asserted by `cmd/socktest`,
|
||||
section 15. `flock()` is still not implemented; see below.
|
||||
|
||||
`kernel/net/unix.c`'s own header comment already predicted this:
|
||||
|
||||
> *"What is lost is `ls -l /tmp/schcom` showing an `s`, and `unlink()` of a socket
|
||||
> path failing with ENOENT. Both are worth fixing when the VFS next gets opened
|
||||
> up; neither blocks anything."*
|
||||
|
||||
They block something now. This is that moment, and it is a VFS change rather than
|
||||
a socket change: a socket inode type threaded through `iget`, `dirlookup` and
|
||||
`stat`, plus `flock`.
|
||||
|
||||
### 4. `setsockopt` that does not lie
|
||||
|
||||
At minimum `SO_PASSCRED` and `SO_SNDBUF`. Returning 0 for an option you ignored
|
||||
is worse than `ENOPROTOOPT`, because the caller adjusts its behaviour on the
|
||||
strength of the lie.
|
||||
|
||||
Small, and grouped here because it is in the same file as item 3 and will be
|
||||
found by the same testing.
|
||||
|
||||
### 5. Dynamic linking — **done**
|
||||
|
||||
`PT_INTERP` and `ET_DYN` in `kernel/fs/exec.c`. A program that names an
|
||||
interpreter gets it loaded at the bottom of the mmap area and started instead,
|
||||
with `AT_PHDR`, `AT_PHENT`, `AT_PHNUM`, `AT_BASE` and `AT_ENTRY` to find the
|
||||
program by; a PIE program is placed where a fixed one would be. The interpreter
|
||||
is musl's own: `ports/musl` now builds `libc.so` as well, shipped as
|
||||
`/lib/ld-musl-x86_64.so.1`, which is libc and loader in one file.
|
||||
|
||||
The kernel half that was missing besides `exec` was `mmap(MAP_FIXED)` of a
|
||||
file. It handed back zeroed pages whatever the descriptor said, and kept no
|
||||
record of them for `fork` — and a dynamic linker maps every library segment
|
||||
after the first exactly that way.
|
||||
|
||||
Every port still links `-static`, on purpose: this makes the other kind
|
||||
possible, it does not convert anything. `cmd/dynhello` is the test — a
|
||||
program against `libc.so` and a library of its own, checking TLS, the heap and
|
||||
a forked child that still has its libraries.
|
||||
|
||||
`mprotect` is accepted and not enforced, so RELRO is not read-only yet.
|
||||
|
||||
### 6. The event descriptors — **done**
|
||||
|
||||
`epoll`, `eventfd`, `timerfd` and `signalfd`, in `kernel/sys/sys_event.c`.
|
||||
libwayland's event loop is all four in one epoll set, which is why they arrive
|
||||
together; not in the original list because they were stubs that looked like
|
||||
syscalls, returning `ENOSYS` from `table.inc` entries that read as done.
|
||||
|
||||
They wait the way `poll()` does, a tick at a time, so the latency is 10 ms.
|
||||
`cmd/evtest` checks each one, then all four in one set, level- and
|
||||
edge-triggered.
|
||||
|
||||
Building it found a signal bug: a default-fatal signal killed on arrival even
|
||||
when it was blocked. A blocked signal now stays pending, which is the contract
|
||||
`signalfd` is built on.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
Each item gets a test before the thing that needs it is built, because a
|
||||
substrate that is subtly wrong fails inside the consumer and looks like the
|
||||
consumer's bug.
|
||||
|
||||
- **Item 1:** two processes, one passes an open descriptor, the other reads
|
||||
through it. Plus the leak case: a sender that exits with descriptors in flight.
|
||||
- **Item 2:** two processes map the same `memfd`, one writes, the other sees it.
|
||||
- **Item 3:** `bind()` a socket, then `stat()` and `ls -l` the path and see an
|
||||
`s`, then `unlink()` it.
|
||||
- **Items 1–3 together:** a third test that does all of it at once, because that
|
||||
combination is what Wayland actually performs and the three passing separately
|
||||
does not prove it.
|
||||
|
||||
These belong in `bootcheck` as a new switch, following the `-Sock` model.
|
||||
|
||||
---
|
||||
|
||||
## What this does not deliver
|
||||
|
||||
- **No page cache**, so writable `MAP_SHARED` of a real file stays refused.
|
||||
- **No `dlopen`** until item 5 is done, which means no Mesa and no third-party
|
||||
shared libraries.
|
||||
- **`SO_*` coverage is the two options named**, not the full set.
|
||||
- **No credential passing** (`SCM_CREDENTIALS`) — only descriptors. Nothing on
|
||||
this plan needs it, and a compositor that wants to know who connected will.
|
||||
- **No `flock()`.** libwayland locks `wayland-0.lock` beside the socket to stop
|
||||
two compositors claiming one display. The socket itself is now a real path,
|
||||
so the lock file can be created and opened; what is missing is the advisory
|
||||
lock, which is a separate mechanism and has no other caller yet.
|
||||
|
|
@ -16,7 +16,7 @@ blobs may be shipped. That supersedes LDLS's rule 1.
|
|||
|
||||
FreeBSD's drm-kmod compiles Linux's DRM source into kernel modules against a
|
||||
shim (linuxkpi). SchOS has already chosen the other route, and for good reasons
|
||||
([hybrid.md](hybrid.md)): GPU drivers live outside the kernel, a crashed driver
|
||||
([devices.md](../architecture/devices.md)): GPU drivers live outside the kernel, a crashed driver
|
||||
falls back to the framebuffer instead of taking the machine down, and LDLS
|
||||
loads unmodified `.ko` files against the running kernel's BTF instead of
|
||||
needing a source port per Linux release. vgpud already drives a GPU from a
|
||||
|
|
@ -77,8 +77,8 @@ descriptors. The kernel has to forward each of those to the driver process:
|
|||
physical ranges (VRAM through the BAR, or GTT pages from the arena), and the
|
||||
kernel maps those into the client's address space.
|
||||
- **dma-buf** becomes a kernel file object naming a buffer the server owns.
|
||||
`SCM_RIGHTS` passes it between processes (already built, foundations.md
|
||||
item 1), and X's DRI3 needs exactly that.
|
||||
`SCM_RIGHTS` passes it between processes (already built; see
|
||||
[networking.md](../architecture/networking.md#descriptor-passing)), and X's DRI3 needs exactly that.
|
||||
- **sysfs**: `/sys/class/drm/card0`, `/sys/dev/char/226:*`, and the
|
||||
vendor/device files that libdrm and Mesa read to pick a driver.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,558 +0,0 @@
|
|||
# The kernel becomes hybrid
|
||||
|
||||
"Hybrid" is used to mean everything from "a monolith with a nice device model" to
|
||||
"a microkernel that gave up". **Drawing the line precisely is most of the work**,
|
||||
so that is what this document is.
|
||||
|
||||
The device model it builds on is
|
||||
[../architecture/devices.md](../architecture/devices.md). It is last on the
|
||||
plan, and the reason is in [README.md](README.md#the-order-and-why-it-is-not-the-order-they-are-listed-in).
|
||||
|
||||
---
|
||||
|
||||
## The line
|
||||
|
||||
### Stays in the kernel, permanently
|
||||
|
||||
- scheduling, address spaces, the trap and syscall path
|
||||
- IPC and shared memory — the mechanism everything else is built out of
|
||||
- the VFS and the **root** filesystem
|
||||
- IRQ demultiplexing
|
||||
- the console and PSOGM
|
||||
|
||||
**That last one is not a size compromise, it is an invariant.** The crash screen
|
||||
must work when everything else in the system is dead, which means it cannot
|
||||
depend on anything that could *be* dead. A crash screen served by a userspace
|
||||
process is a crash screen that is absent in exactly the case it exists for.
|
||||
|
||||
### May leave, and should
|
||||
|
||||
- GPU drivers
|
||||
- USB, and every USB class driver under it
|
||||
- non-root filesystems
|
||||
- NICs after the first
|
||||
- input transformation — layouts, gestures, remapping
|
||||
|
||||
### The rule that separates the columns
|
||||
|
||||
> **Anything needed to reach a shell stays in. Anything discovered at runtime,
|
||||
> replaceable, or plausibly written by someone else goes out.**
|
||||
|
||||
That is NT's compromise and macOS's, arrived at from the same pressure, and it is
|
||||
the one that survives contact with a real machine.
|
||||
|
||||
---
|
||||
|
||||
## Built so far
|
||||
|
||||
**Mechanisms 2 and 4 exist.** `/dev/pci` (include/dev/pciuser.h, kernel/dev/pciuser.c)
|
||||
enumerates the bus, hands a device to a process exclusively, maps a BAR into it
|
||||
and allocates memory a device can be pointed at. `pcitest` proves it end to end:
|
||||
|
||||
```
|
||||
# bdf vendor:device class driver
|
||||
6 00:04.0 8086:100e network e1000
|
||||
7 00:07.0 8086:10d3 network -
|
||||
|
||||
device 7: 8086:10d3 (network)
|
||||
claimed
|
||||
BAR0 at 0x0000000081060000, 128 KB
|
||||
regs[0..3] = 0x00140261 0x00000000 0x00080283 0x00000000
|
||||
MMIO reaches the device
|
||||
DMA 8 KB at virtual 0x40001000, physical 0x3f8eb000
|
||||
```
|
||||
|
||||
Those register values are the e1000e's CTRL and STATUS, with full-duplex and
|
||||
link-up set -- which is how you tell a working mapping from one that reads
|
||||
0xFFFFFFFF off an unmapped address. Claiming a device the kernel drives is
|
||||
refused, and everything a process held is released when it exits, because a
|
||||
driver that crashed must not keep its hardware away from its replacement.
|
||||
|
||||
**Mechanism 3 exists too.** A process asks for a device's interrupts and then
|
||||
reads them, uio-style:
|
||||
|
||||
```
|
||||
interrupts routed here (line 10)
|
||||
interrupt delivered, count = 1
|
||||
ICR = 0x80000004
|
||||
```
|
||||
|
||||
`0x80000004` is the e1000e's cause register with the bit the test asserted and
|
||||
INT_ASSERTED set, read back by userspace as its acknowledgement.
|
||||
|
||||
The line is **masked by the kernel on arrival and unmasked by userspace's
|
||||
write()**, and that is the entire design rather than a detail. A PCI interrupt
|
||||
line is level-triggered: it stays asserted until the cause is cleared *at the
|
||||
device*, which is a different register on every chip and therefore something
|
||||
only the driver can do. Signalling EOI on a line that is still asserted
|
||||
re-enters the handler immediately and for ever -- 72,000 times a second, when
|
||||
this kernel last did it by accident (history/kernel.md §13).
|
||||
|
||||
So a driver that dies mid-interrupt leaves the line **masked, not unmasked**.
|
||||
The instinct is the other way round -- do not leave the device dead -- and it
|
||||
is wrong: unmasking an asserted line with no handler attached is a crashed
|
||||
driver taking the machine down with it. Verified by killing one on purpose:
|
||||
|
||||
```
|
||||
interrupt asserted; exiting WITHOUT acking (simulated crash)
|
||||
timer: 3431 ticks taken, 3436 expected
|
||||
```
|
||||
|
||||
Five ticks lost in seven seconds, which is the ordinary background. The next
|
||||
driver to claim the device re-arms the line, so masked is safe without being
|
||||
permanent -- also verified.
|
||||
|
||||
## The first eviction happened
|
||||
|
||||
`cmd/vgpud` is the virtio-gpu driver, outside the kernel. It finds the device
|
||||
by vendor and device id, asks for it, and the kernel driver hands it over:
|
||||
|
||||
```
|
||||
vgpud: virtio-gpu at 00:03.0, currently held by virtio-gpu
|
||||
virtio-gpu: released to userspace; screen is the VBE framebuffer
|
||||
vgpud: claimed
|
||||
vgpud: display is 1280x800
|
||||
vgpud: scanout is ours, framebuffer at physical 0x3eceb000
|
||||
vgpud: drew a test pattern from userspace
|
||||
```
|
||||
|
||||
Full virtio 1.0 bring-up from a process: reset, feature negotiation, virtqueue
|
||||
in DMA memory, GET_DISPLAY_INFO, RESOURCE_CREATE_2D, ATTACH_BACKING,
|
||||
SET_SCANOUT, and TRANSFER/FLUSH to present. The pattern it draws is on the
|
||||
virtio-gpu head while the console carries on in VBE on the other one -- which
|
||||
is the fallback rule working rather than being described.
|
||||
|
||||
**No syscall knows this is a GPU.** The kernel handed over a PCI device; the
|
||||
process decided what it was. That is the property that makes the interface
|
||||
general, and it is why the evictions come before anything foreign.
|
||||
|
||||
### What the first real driver asked for that a test program had not
|
||||
|
||||
- **PCI configuration space** (`PCIIOC_CFG`). A virtio device says where its
|
||||
registers are in *vendor capabilities*, which live in config space -- so a
|
||||
driver that cannot read it cannot find its own registers. Anything modern is
|
||||
discovered the same way.
|
||||
- **A way for the incumbent to let go.** `struct pci_driver` gained a
|
||||
`release` hook. A driver without one keeps its device, which is always safe
|
||||
and, for anything on the path to a shell, correct.
|
||||
- **The outgoing driver freeing its DMA.** Releasing the device while keeping
|
||||
its 4 MB framebuffer is half a handover: the incoming driver asks for one and
|
||||
is told there is no room. Found by doing it.
|
||||
- **A bigger arena.** 4 MB was exactly one framebuffer at this resolution, and
|
||||
for the moment either side of a handover there are two. 16 MB now.
|
||||
|
||||
### And one bug it reproduced immediately
|
||||
|
||||
The first working run raised this:
|
||||
|
||||
```
|
||||
irq 11: 77780 interrupts/second -- a device on this line is not acknowledging it.
|
||||
irq 11: handler 'e1000' is attached
|
||||
```
|
||||
|
||||
`vgpud` polls the used ring, so it registered no handler -- and a virtio device
|
||||
interrupts on every completion. A level-triggered line nobody acknowledges
|
||||
re-fires for ever, drowning the e1000 that shares it. The fix is one line,
|
||||
`VRING_AVAIL_F_NO_INTERRUPT` before the queue is enabled, and the value of it
|
||||
is that the storm detector built for history/kernel.md section 13 named the
|
||||
line, the rate and the neighbour without anyone having to look.
|
||||
|
||||
**All four mechanisms now exist**, and with H3 done and H4 withdrawn, nothing
|
||||
remains on this plan. What the mechanisms were built for next is the foreign
|
||||
driver host -- [ldls.md](ldls.md).
|
||||
|
||||
### One thing the interface is deliberately not
|
||||
|
||||
It is about **PCI devices**, not about any driver model, and its first users
|
||||
will be SchOS's own drivers. That ordering is the design: an interface whose
|
||||
first consumers are native drivers is a general mechanism that something
|
||||
foreign can use later, rather than a hole shaped like something foreign. The
|
||||
difference matters technically -- a Linux-shaped interface would be wrong for
|
||||
the evictions this plan actually calls for -- and it happens to be the only
|
||||
version that survives a licensing conversation.
|
||||
|
||||
---
|
||||
|
||||
## The driver came back
|
||||
|
||||
H2 is the milestone this plan calls the one that cannot be skipped, and the
|
||||
reason is that H1 proves only half of the idea. A driver that runs in a process
|
||||
is interesting; a driver that runs in a process, *dies*, and is replaced by
|
||||
another process that picks the device up is the thing the whole hybrid argument
|
||||
rests on.
|
||||
|
||||
`etc/schinit/vgpud.job` makes it a supervised job -- `KeepAlive = yes` -- so
|
||||
killing it is the whole test:
|
||||
|
||||
```
|
||||
# cat /proc/schinit
|
||||
1 4 11 os.schos.vgpud
|
||||
# kill 4
|
||||
schinitd: restarting os.schos.vgpud
|
||||
vgpud: virtio-gpu at 00:03.0
|
||||
vgpud: claimed
|
||||
vgpud: display is 1280x800
|
||||
vgpud: scanout is ours, framebuffer at physical 0x3eceb000
|
||||
vgpud: pid 8 drew a test pattern from userspace
|
||||
```
|
||||
|
||||
The test pattern ends in a twelve-block tally of the drawing process's pid,
|
||||
because a second run that redraws an identical picture is indistinguishable
|
||||
from a first run that never stopped. Read off the virtio-gpu scanout across
|
||||
four generations of the driver it says 4, 8, 10, 12. The same physical address
|
||||
comes back every time, which is the DMA arena reclaiming cleanly rather than
|
||||
drifting.
|
||||
|
||||
### What it took that H1 had not needed
|
||||
|
||||
- **A halt the kernel can apply without knowing what the device is.** A driver
|
||||
that dies leaves its device bus-mastering into memory the kernel is about to
|
||||
hand to somebody else. The kernel cannot quiesce a device it does not
|
||||
understand, but it can clear `PCI_CMD_BUS_MASTER`, and that is enough to stop
|
||||
the writes. `pciuser_stop()` does it before the DMA is freed, not after, and
|
||||
claim/release became a symmetric pair: claiming sets the bit, releasing
|
||||
clears it. `pcitest` grew a `bm` column so the state is visible rather than
|
||||
assumed.
|
||||
- **A refusal, when there is nothing to fall back to.** `vgpu_release()` now
|
||||
asks `vga_fb_restore()` whether there is a framebuffer to put the console on,
|
||||
and declines to hand the device over if there is not. The rule that the
|
||||
kernel keeps a minimal driver for everything on the path to a shell is only
|
||||
worth writing down if something enforces it.
|
||||
- **The DMA arena not being handed to two allocators at once.** This is the bug
|
||||
that stopped H2 for a day, and it is worth writing out: `freevm()` frees
|
||||
every mapped page below `PHYSTOP`, and the DMA arena is the top 16 MB of
|
||||
exactly that range. So a DMA block mapped into a driver and left for
|
||||
`freevm()` went onto the page allocator's free list when the driver died,
|
||||
while `pciuser_release_all()` handed the same pages back to the arena. Both
|
||||
allocators then believed they owned it. The restarted `vgpud` asked for its
|
||||
4 MB framebuffer, got pages `kalloc()` had already given out -- including its
|
||||
own page tables -- and `dma_alloc()`'s zeroing wiped them, so the driver died
|
||||
inside the ioctl, on the return from a syscall whose stack had stopped
|
||||
existing. `do_dma()` now records a VMA with `no_free` set, the same way
|
||||
shared memory does, so `proc_release_shared()` unmaps the range before the
|
||||
page tables are torn down.
|
||||
|
||||
### The clause that is not true yet
|
||||
|
||||
The milestone says *"the compositor reattaches"*. It does not, because there is
|
||||
no compositor running: Schcom is v0 and is not started at boot. What has been
|
||||
shown is the two parts that do not depend on it -- the screen falls back to
|
||||
VBE, and the driver restarts and takes the device back. During the gap the VBE
|
||||
head carries a live console and the virtio-gpu head still shows the dead
|
||||
driver's last frame, which is exactly what a device nobody is driving does.
|
||||
|
||||
(That last sentence is still true of a *crashed driver*, and no longer true of
|
||||
a crashed *kernel* -- see "The clause nobody wrote down" below.)
|
||||
|
||||
Reattachment is a Schcom milestone, not a hybrid one, and it is recorded that
|
||||
way rather than counted here.
|
||||
|
||||
---
|
||||
|
||||
## A modeset, from a process
|
||||
|
||||
H3 is one sentence in the milestone table -- *"a modeset API, which is the
|
||||
thing an out-of-kernel GPU driver makes affordable and an in-kernel one never
|
||||
got"* -- and the claim in it is worth testing rather than repeating.
|
||||
|
||||
```
|
||||
# schmode
|
||||
1280x800 current
|
||||
1280x800 maximum, framebuffer 4000 KB
|
||||
# schmode modes
|
||||
1280x800 *
|
||||
1280x720
|
||||
1024x768
|
||||
800x600
|
||||
640x480
|
||||
# schmode 1024 768
|
||||
1024x768 ok
|
||||
```
|
||||
|
||||
The scanout really resizes: captured from the virtio-gpu head, the frames come
|
||||
back 1280x800, then 1024x768, then 640x480, then 1280x800 again, with the
|
||||
checkerboard band showing no shear at any of them.
|
||||
|
||||
**Nothing in the kernel changed for this.** No syscall on the path has heard of
|
||||
a scanout; `schmode` writes a line to a socket and `vgpud` sends five commands
|
||||
to the device. That is the whole of the milestone, and the reason it is
|
||||
interesting is the comparison: the same five commands in the kernel would have
|
||||
been the same five commands *and a reason to be nervous*, because getting the
|
||||
sequence wrong there takes the machine instead of one process. **That is why
|
||||
the in-kernel driver never grew one** -- not difficulty, exposure.
|
||||
|
||||
### What it costs, and what it refuses
|
||||
|
||||
The framebuffer is allocated **once, at the largest mode**, and every smaller
|
||||
mode reuses it. `PCIIOC_DMA` has no free -- a driver's blocks come back when it
|
||||
exits and not before -- so a modeset that allocated would spend sixteen slots
|
||||
and then the arena, and the failure would look like a GPU bug rather than like
|
||||
counting. A mode larger than the initial allocation is refused, and says why:
|
||||
|
||||
```
|
||||
# schmode 1920 1080
|
||||
refused: larger than the framebuffer, which is fixed at startup
|
||||
```
|
||||
|
||||
That bound is 4 MB, `PCIIOC_DMA`'s cap, which at 32 bits per pixel is about
|
||||
1280x800. Raising it is a one-line change to a constant and a question about
|
||||
how much of a 16 MB arena one driver should be able to take, which is a better
|
||||
question than it looks and is not answered here.
|
||||
|
||||
### The socket, and the argument it settles
|
||||
|
||||
`vgpud` used to end in `for(;;) sleep(100);` under a comment arguing that
|
||||
building a control socket before there is a compositor to judge it is exactly
|
||||
the mistake this document spends a section warning about.
|
||||
|
||||
That argument was right about a *compositor protocol* and wrong as a reason to
|
||||
have no socket at all. A modeset API has a consumer that exists -- somebody
|
||||
changing the mode and looking at the screen -- and `schmode` is small enough
|
||||
that it can judge an interface this small. So the socket is there, it speaks
|
||||
lines of text for the same reasons
|
||||
[../architecture/debugging.md](../architecture/debugging.md) gives, and what is
|
||||
deliberately **not** on it is everything a compositor would need: no buffer
|
||||
submission, no damage, no vblank. Those still wait, for the original reason.
|
||||
|
||||
---
|
||||
|
||||
## The mechanisms
|
||||
|
||||
Four, and only the fourth is large.
|
||||
|
||||
### 1. A control channel — **built**
|
||||
|
||||
`AF_UNIX` plus `SCM_RIGHTS` — item 1 of [foundations.md](foundations.md), which
|
||||
the Wayland work also needed first. A descriptor is how a driver process is
|
||||
handed the device it is allowed to touch.
|
||||
|
||||
Built as `sendmsg`/`recvmsg` in Linux's shape, because the programs that will
|
||||
use it are not ours: libwayland and toybox both call `recvmsg` with Linux's
|
||||
structures and would not notice a friendlier door. Descriptors are pinned to the
|
||||
stream position of the first byte of their message, and the kernel holds a
|
||||
reference to each until it is received — or closes it when the socket is dropped
|
||||
with the message still queued, which is the case that leaks if nobody writes it
|
||||
down. See
|
||||
[../architecture/networking.md](../architecture/networking.md#descriptor-passing).
|
||||
|
||||
**Do not build Mach ports.** The port model is worth its cost when there is a
|
||||
naming and capability story to hang on it; there is not one yet, and inventing it
|
||||
before the first userspace driver exists is designing against an imaginary user.
|
||||
|
||||
### 2. MMIO to userspace
|
||||
|
||||
Map a PCI BAR into a process.
|
||||
|
||||
`sys_mmap` already maps the framebuffer's real physical range at its real
|
||||
address — the path exists. This is that path generalised, behind a check that the
|
||||
caller was granted that device.
|
||||
|
||||
### 3. Interrupts to userspace
|
||||
|
||||
**The handler stays in the kernel and does one thing: write to an `eventfd`.**
|
||||
The driver process reads.
|
||||
|
||||
`eventfd` exists already, and that is why it is the right primitive. A signal
|
||||
would be delivered to a process that cannot safely be interrupted mid-transaction.
|
||||
|
||||
### 4. DMA to userspace
|
||||
|
||||
Physically contiguous memory whose physical address the process is told.
|
||||
|
||||
`kernel/core/dma.c` already owns an arena reserved for exactly this, before
|
||||
`kalloc` gets anything. It needs a door, a per-device budget, and a way to hand
|
||||
the region back.
|
||||
|
||||
---
|
||||
|
||||
## The clause nobody wrote down: a panic has to reach the screen
|
||||
|
||||
Handing a device away costs something that was not on the list. The kernel
|
||||
panics, paints the report into the console framebuffer, and halts — and the
|
||||
display it gave to a process keeps showing whatever that process last put
|
||||
there. On a machine with two heads, one says the kernel is dead and the other
|
||||
shows a shell prompt.
|
||||
|
||||
It is not a race that can be won by being careful. There is no scheduling after
|
||||
a panic, so the process holding the device is never going to run again, and the
|
||||
kernel let go of the device deliberately.
|
||||
|
||||
**So the userspace driver says in advance where it is scanning out**, with
|
||||
`PCIIOC_PANICFB` (see `include/dev/pciuser.h`), and the kernel does the rest:
|
||||
|
||||
1. paints the report into that memory,
|
||||
2. asks the *kernel* driver the device was taken from to put it on screen —
|
||||
`panic_present` in `struct pci_driver`,
|
||||
3. which resets the device and rebuilds one queue from **static** memory: no
|
||||
allocation, no locks, no sleeping, because a corrupted heap is one of the
|
||||
ways a machine gets here in the first place.
|
||||
|
||||
The registration is checked like the promise it is: the range must lie inside a
|
||||
block that process got from `PCIIOC_DMA` on that device. Without that it would
|
||||
be a way to nominate any physical address in the machine to be overwritten at
|
||||
panic, which is a worse bug than the one it fixes.
|
||||
|
||||
What this buys is a property worth stating plainly: **a device can be evicted
|
||||
to userspace without the kernel losing its ability to tell you the machine
|
||||
died.** That is the difference between a driver boundary and a blind spot, and
|
||||
it is one more thing the in-kernel arrangement got for free and the hybrid one
|
||||
has to earn.
|
||||
|
||||
It costs 20 KB of static memory in the virtio-gpu driver, paid whether or not
|
||||
the machine ever panics. Allocating it on demand would be cheaper and would not
|
||||
work.
|
||||
|
||||
---
|
||||
|
||||
## The honest limitation
|
||||
|
||||
> **There is no IOMMU, so a userspace driver is isolated against its own bugs and
|
||||
> not against malice.**
|
||||
|
||||
A process that can program a device's DMA engine can write to any physical
|
||||
address, whatever its page tables say.
|
||||
|
||||
That is not a reason to stop — isolation against bugs is most of the value, and a
|
||||
crashed driver that restarts instead of panicking the machine is the entire
|
||||
point. But it must be written down, because *"userspace driver"* reads as
|
||||
*"sandboxed"* and here it does not mean that yet. VT-d/AMD-Vi is its own project
|
||||
and comes after.
|
||||
|
||||
---
|
||||
|
||||
## Order, and one thing not to do
|
||||
|
||||
### Do not build loadable kernel modules first
|
||||
|
||||
It is the obvious first step and it is a generation behind. It means designing a
|
||||
**stable in-kernel driver ABI**, which is the most expensive promise in this
|
||||
entire plan, for a mechanism Apple is actively retiring in favour of the one
|
||||
above.
|
||||
|
||||
`../architecture/devices.md` already separates the two ideas. They are separable,
|
||||
and the second one is the one to build.
|
||||
|
||||
### First eviction: the GPU driver
|
||||
|
||||
Four reasons it is the right one and not just the interesting one:
|
||||
|
||||
- **The boundary already exists and is already correct** — `struct display_ops`.
|
||||
- **It has exactly one consumer**, the compositor, so the interface has a real
|
||||
user to be judged by.
|
||||
- **There is a working fallback underneath.** The VBE framebuffer stays
|
||||
in-kernel, so a GPU driver that fails to start degrades to a working screen
|
||||
rather than to no screen.
|
||||
- It directly serves the work that will actually be happening.
|
||||
|
||||
### Second: USB — and why it turned out not to be
|
||||
|
||||
`xhci.c` is 1364 lines, self-contained, already spoken to through a bus
|
||||
abstraction, and it is where third-party drivers will show up first. This is what
|
||||
DriverKit was built for and the resemblance is not accidental.
|
||||
|
||||
#### It is not the same shape as the GPU, and that was missed
|
||||
|
||||
Written before the GPU eviction had been done, H4 says *"USB, same shape"*. It
|
||||
is not, and the difference is one this document's own rule already decides.
|
||||
|
||||
The GPU could leave because **there was a working fallback underneath it**: the
|
||||
VBE framebuffer stays in the kernel, so a GPU driver that fails to start
|
||||
degrades to a working screen. That is one of the four reasons the GPU was
|
||||
chosen to go first, listed a few lines above.
|
||||
|
||||
USB has no fallback, and it carries two things that are on the path to a shell:
|
||||
|
||||
- **The root disk.** `USBDISK_DEV` dispatches through `bio.c` to
|
||||
`usb_storage_rw`, and `usbstick.img` is a real boot medium built by `make`.
|
||||
- **The keyboard.** `usb_hid` feeds `input_report(INPUT_KBD, ...)`, which is how
|
||||
anything without PS/2 is typed on — including the Chromebox this is meant to
|
||||
boot.
|
||||
|
||||
So evicting `xhci.c` wholesale breaks the rule three sections down:
|
||||
|
||||
> **The kernel keeps a minimal driver for everything on the path to a shell, and
|
||||
> userspace owns the good one.**
|
||||
>
|
||||
> Break this and you get the circular boot failure where the compositor cannot
|
||||
> start because the display driver is a process that needs a compositor.
|
||||
|
||||
The USB version of that sentence is *the root filesystem cannot mount because
|
||||
the storage driver is a process that lives on the root filesystem*, and it is
|
||||
the same failure wearing different clothes.
|
||||
|
||||
There is a second difference, subtler and probably the harder one. `/dev/pci`
|
||||
hands over a **PCI function** — one device, one owner, nothing shared. A USB
|
||||
device is not that: it sits behind a controller that other devices also sit
|
||||
behind, so the unit of ownership and the unit of hardware are no longer the
|
||||
same thing. Handing a process "the xHCI controller" gives it the keyboard too;
|
||||
handing it "one USB device" is an interface `/dev/pci` does not have and that
|
||||
this plan has not designed.
|
||||
|
||||
#### Decided: USB stays in the kernel
|
||||
|
||||
**H4 is withdrawn.** Not deferred, not blocked on the interface above —
|
||||
withdrawn, because the premise was wrong.
|
||||
|
||||
> **USB is not a peripheral.** It is the bus the peripherals arrive on.
|
||||
|
||||
Nearly every machine has a USB port in some form, down to a bad micro-B on a
|
||||
board that has nothing else, and the ones that do not have one are not made
|
||||
worse by the stack being present. That makes USB structurally the same kind of
|
||||
thing as PCI, not the same kind of thing as a GPU — and this document has never
|
||||
proposed evicting PCI.
|
||||
|
||||
Reading the eviction rule again with that in mind, it already said so:
|
||||
|
||||
> **Anything needed to reach a shell stays in. Anything discovered at runtime,
|
||||
> replaceable, or plausibly written by someone else goes out.**
|
||||
|
||||
USB is how things are *discovered at runtime*, which is not the same as being
|
||||
one of them. The keyboard and the root disk that arrive over it are on the path
|
||||
to a shell, and the stack that finds them is part of that path.
|
||||
|
||||
The two obstacles in the section above are real and they are why the premise
|
||||
got questioned, but they are not the reason for this decision. Even with a
|
||||
minimal in-kernel fallback written and a per-device ownership interface
|
||||
designed, the answer would be the same: the good USB driver and the minimal one
|
||||
would be the same driver, and an eviction that leaves an identical copy behind
|
||||
has moved nothing.
|
||||
|
||||
What the eviction was *for* — third-party drivers, written by someone else,
|
||||
living outside the kernel — is a real goal and it survives this. It is simply
|
||||
not a USB goal; it belongs to the foreign driver host, which is
|
||||
[ldls.md](ldls.md).
|
||||
|
||||
### The rule that keeps the boot path honest
|
||||
|
||||
> **The kernel keeps a minimal driver for everything on the path to a shell, and
|
||||
> userspace owns the good one.**
|
||||
|
||||
Break this and you get the circular boot failure where the compositor cannot
|
||||
start because the display driver is a process that needs a compositor.
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
| | | |
|
||||
|---|---|---|
|
||||
| **H0** | Mechanisms 2, 3 and 4, each with a test that exercises it from a userspace program against a device nothing else is using. | done |
|
||||
| **H1** | A userspace virtio-gpu driver, publishing `display_ops` across the channel. Schcom talks to it and does not know the difference. | done, except that Schcom does not talk to it -- `cmd/vgpud` holds the device and draws its own frame |
|
||||
| **H2** | Kill the driver process on purpose. The screen falls back to VBE, the driver restarts, the compositor reattaches. **This is the milestone that proves the whole idea**, and it is the one that cannot be skipped. | done, less the compositor -- see "The driver came back" |
|
||||
| **H3** | A modeset API, which is the thing an out-of-kernel GPU driver makes affordable and an in-kernel one never got. | done -- see "A modeset, from a process" |
|
||||
| ~~**H4**~~ | ~~USB, same shape.~~ **Withdrawn** — USB is the bus peripherals arrive on, not a peripheral. See "Decided: USB stays in the kernel". | |
|
||||
|
||||
---
|
||||
|
||||
## What it will still not do
|
||||
|
||||
- **No isolation against a hostile driver**, per the IOMMU note above.
|
||||
- **No stable driver ABI**, deliberately. Out-of-tree drivers remain a
|
||||
recompile-with-the-tree affair until there is a reason to promise otherwise.
|
||||
- **The root filesystem stays in the kernel**, so SchFS cannot be restarted and a
|
||||
bug in it is still fatal. Moving it out needs an initramfs story that does not
|
||||
exist.
|
||||
- **The first NIC stays in the kernel**, because the network thread and lwIP are
|
||||
in the kernel and moving them is a separate project with no current benefit.
|
||||
- **No driver sandboxing beyond address-space separation** — no seccomp-shaped
|
||||
syscall restriction, no resource limits per driver.
|
||||
|
|
@ -1,183 +1,120 @@
|
|||
# Roadmap — everything else still open
|
||||
|
||||
The three current decisions have their own documents; see
|
||||
[README.md](README.md). This file holds what is left: work that is scoped,
|
||||
agreed and unstarted, with nothing in it that is finished.
|
||||
[1.0.md](1.0.md) draws the finish line and holds the register of what is half-built.
|
||||
This file holds what is left beyond it: work that is scoped, agreed and unstarted, with
|
||||
nothing in it that is finished.
|
||||
|
||||
**Finished work is not here.** It moved to
|
||||
[../architecture/](../architecture/) and was rewritten as a description of what
|
||||
it produced — see [the rule](../README.md#the-rule-that-keeps-this-from-rotting).
|
||||
What used to be phases 1 through 4 of this file, several hundred lines of ticked
|
||||
boxes, is now `usb`, `networking.md`, `graphics.md` and `devices.md` over there.
|
||||
**Finished work is not here.** It moved to [../architecture/](../architecture/) and was
|
||||
rewritten as a description of what it produced — see
|
||||
[the rule](../README.md#the-rule-that-keeps-this-from-rotting).
|
||||
|
||||
---
|
||||
|
||||
## 1. Storage and the boot path
|
||||
|
||||
**Done: a USB disk, a SATA disk and their partitions can be booted from.** The
|
||||
root is chosen at boot from what is present (`rootdev_mount()` in
|
||||
`kernel/proc/proc.c`), and `make disk` builds a GPT image with an ESP and a SchFS
|
||||
partition that a PC boots from a stick or a disk.
|
||||
A USB disk, a SATA disk, an NVMe SSD and their partitions boot, and the root is chosen at boot from
|
||||
what is present ([boot.md](../architecture/boot.md)).
|
||||
|
||||
NVMe (`kernel/drivers/storage/nvme/`) registers with the same block registry,
|
||||
so an NVMe SSD boots the same way.
|
||||
|
||||
**Next:** an installer that writes that layout to a disk, and interrupt-driven,
|
||||
queued I/O for AHCI and NVMe.
|
||||
- **An installer** that writes the layout `make disk` builds (a GPT disk with an ESP and a SchFS
|
||||
partition) to a machine's disk, so a root survives a reboot. See §5.
|
||||
- **Interrupt-driven, queued I/O for AHCI and NVMe.** Both poll for completion, one command at a time
|
||||
per disk: enough to boot and run, not to be fast.
|
||||
- **An EHCI driver.** USB 2.0 ports wired only to EHCI do nothing; xHCI ports work.
|
||||
|
||||
## 2. Graphics
|
||||
|
||||
- **A `/dev/dri`-style device with mode setting**, so resolution stops being
|
||||
whatever the loader asked for. This was to be done *with*
|
||||
[hybrid.md](hybrid.md)'s GPU eviction, and that eviction has now happened —
|
||||
`cmd/vgpud` drives virtio-gpu from userspace. The modeset API is the next thing
|
||||
it makes affordable, and is milestone H3 over there rather than here.
|
||||
- **Double buffering and vblank.** `display_ops` specifies both and no driver
|
||||
implements either. Schcom needs both to not tear.
|
||||
- **Port the console onto the text layer and delete the cell grid.** The layer
|
||||
exists, the rasteriser exists, and the console still indexes `font_data[256][16]`.
|
||||
- **Load a font from disk** and point the console at it.
|
||||
- **CFF/Type 2 charstrings**, so OTF fonts work. A separate `font_ops` provider
|
||||
and a bytecode interpreter, *not* an extension of the `glyf` path.
|
||||
- **Text layout** — advance widths, kerning — as a library in `lib/`, so the
|
||||
terminal and the file manager are not each solving it.
|
||||
|
||||
The console keeps its fixed grid until the first item above; proportional text
|
||||
belongs to the GUI.
|
||||
- **A `/dev/dri` device with mode setting**, so a real panel's resolution stops being whatever the
|
||||
loader's menu chose and Mesa has something to open. This is the GPU plan, [gpu.md](gpu.md).
|
||||
- **Double buffering and vblank**, which `display_ops` specifies and no driver implements; the same plan.
|
||||
- **CFF/Type 2 charstrings**, so OTF fonts work. A separate `font_ops` provider and a bytecode
|
||||
interpreter, *not* an extension of the `glyf` path.
|
||||
- **Text layout** — advance widths, kerning — as a library, so the terminal and the file manager are not
|
||||
each solving it.
|
||||
|
||||
## 3. Networking
|
||||
|
||||
Two things the stack owed on the day it landed, both still owed:
|
||||
- **A routing table** and `/proc/net/route`. With two interfaces up the first is the default route and the
|
||||
second is reachable only by an address on its own subnet; that is a rule and not routing.
|
||||
- **Wi-Fi.** The laptop has no other way to be wireless. The route is the same as for the NIC: a Linux
|
||||
driver (`iwlwifi`, `mt76`…) through LDLS, which needs `nl80211`, a firmware loader that exists, and a
|
||||
supplicant (`wpa_supplicant` is an Alpine package).
|
||||
- **IPv6.**
|
||||
- **A netlink event stream**, for anything that wants to be told a link changed.
|
||||
|
||||
- **`accept()` does not report the peer's address**, and there is no
|
||||
`getsockname`/`getpeername`. Nothing has needed them yet; a server that wants
|
||||
to log who connected will.
|
||||
- **No routing table.** With two interfaces up, the first is the default route
|
||||
and the second is reachable only by an address on its own subnet. That is a
|
||||
rule rather than an accident, and it is not routing.
|
||||
## 4. The hardware that makes a laptop a laptop
|
||||
|
||||
Two that were on this list are now done and described in
|
||||
[../architecture/networking.md](../architecture/networking.md): `setsockopt`
|
||||
honours six options and refuses the rest, and `sendmsg`/`recvmsg` carry
|
||||
`SCM_RIGHTS`. A third is closer than it was — toybox's `ping` needed `recvmsg`
|
||||
*and* `IP_RECVTTL`, and only the second is still missing.
|
||||
|
||||
## 4. The interactive crash screen
|
||||
|
||||
PSOGM renders and stops. The idea is *a small OS inside the crash*, and the
|
||||
framing decides the design:
|
||||
|
||||
> At panic time the kernel is the only thing running, which is exactly why it can
|
||||
> offer what a live system cannot: walk the stack, read another process's memory,
|
||||
> dump page tables, inspect the process table — with no scheduler and no locking
|
||||
> to respect, because nothing else can race you.
|
||||
|
||||
What makes it a *system* rather than a print statement is that it keeps the base
|
||||
machine's drivers. Concretely, the ones that work with interrupts off: the
|
||||
framebuffer (`gfx_draw_pixel` is a plain memory write), the PS/2 keyboard
|
||||
(pollable via ports 0x60/0x64), and the serial port. **Those three are a complete
|
||||
console.**
|
||||
|
||||
Everything else — IDE, USB, the timer, anything that sleeps or takes a
|
||||
sleeplock — is off limits, because the lock it needs may be precisely the one
|
||||
held by the CPU that just died.
|
||||
|
||||
So the shape is a **polled event loop over preallocated state**: no `kalloc`, no
|
||||
`sleep`/`wakeup`, no `acquire` on anything a live path might hold, one static
|
||||
command buffer, and that driver set. **That restriction is not a limitation to
|
||||
work around later; it is the invariant that makes the mode trustworthy when
|
||||
everything else is not.** Anything wanting more should be answered by rebooting
|
||||
into a recovery environment.
|
||||
|
||||
A useful precedent to read but not copy: DDB, the BSD in-kernel debugger, is
|
||||
exactly this, and its hard-won rule is the same one — *a debugger that can
|
||||
deadlock is worse than no debugger.*
|
||||
- **HDA audio.** AC'97 is what QEMU and old boards have; a laptop built this century has Intel HDA,
|
||||
which is codec enumeration, CORB/RIRB and stream descriptors. It arrives as a second `struct audio_ops`.
|
||||
- **Embedded-controller events** (the ACPI SCI), so a lid, a power button on the EC and a battery change are seen.
|
||||
- **CPU frequency control** (AMD CPPC; Intel HWP/EIST). Idle already reaches `_CST`'s C2; under load the cores run at
|
||||
the firmware's default clock.
|
||||
- **Touchpad gestures** — two-finger scroll, palm rejection — on top of I2C-HID's mouse mode.
|
||||
- **An IOMMU** (VT-d / AMD-Vi), which turns "a driver in a process is isolated against bugs" into "against malice" —
|
||||
the largest honest caveat on [ldls.md](../architecture/ldls.md).
|
||||
- **udev's job** — a uevent socket and `/sys/devices` — so X and others are told about devices instead of being
|
||||
given a list.
|
||||
|
||||
## 5. The installer and `.sip`
|
||||
|
||||
Scoped in [`projects/schinst/README.md`](../../projects/schinst/README.md),
|
||||
which holds the detail. In short:
|
||||
Scoped in [`projects/schinst/README.md`](../../projects/schinst/README.md), which holds the detail. In short:
|
||||
|
||||
- [ ] `.sip` format specification, **written down before any code**
|
||||
- [ ] `tools/mksip` — capture a directory tree into a package
|
||||
- [ ] The apply path
|
||||
- [ ] An installation environment that boots from the ISO, partitions, applies
|
||||
the `.sip`, and installs the loader
|
||||
- [ ] An installation environment that boots from the ISO, partitions, applies the `.sip`, and installs the loader
|
||||
|
||||
`.sip` is modelled on WIM, and the thing worth copying from WIM is that it is
|
||||
**file-based, not block-based**: one file holds a whole filesystem, applies to a
|
||||
partition of any size, and several images can share their common files.
|
||||
`.sip` is modelled on WIM, and the thing worth copying from WIM is that it is **file-based, not block-based**: one
|
||||
file holds a whole filesystem, applies to a partition of any size, and several images can share their common files.
|
||||
|
||||
**Most of this already exists.** `tools/mkschfs.c` and `tools/mkfs.c` both do
|
||||
content dedup with an `(st_dev, st_ino)` fast path, and SchFS's B-tree is content
|
||||
addressed. `mksip` should share that code rather than reimplement it — that is
|
||||
the reason the dedup was factored the way it was.
|
||||
**Most of this already exists.** `tools/mkschfs.c` and `tools/mkfs.c` both do content dedup with an `(st_dev, st_ino)`
|
||||
fast path, and SchFS's B-tree is content addressed. `mksip` should share that code rather than reimplement it. And
|
||||
the live image is already a compressed block image of a SchFS root ([scff.md](../architecture/scff.md)); an installer
|
||||
may need nothing more than to write that, decompressed, to a partition.
|
||||
|
||||
Open questions, all wanting a decision before code:
|
||||
|
||||
- **Compression.** Something with a small decompressor is worth more than
|
||||
something with a good ratio, because the decompressor runs in the installation
|
||||
environment.
|
||||
- **Multiple images per `.sip`** — WIM's real trick, install/recovery/boot
|
||||
sharing one blob store. Worth the complexity now, or not?
|
||||
- **Can the kernel mount a `.sip` read-only directly?** It is close to a
|
||||
filesystem already, and if it can, the installation environment's root can
|
||||
simply *be* the `.sip`.
|
||||
- **Compression.** Something with a small decompressor is worth more than something with a good ratio, because the
|
||||
decompressor runs in the installation environment. (zstd is already in the kernel for SCFF.)
|
||||
- **Multiple images per `.sip`** — WIM's real trick, install/recovery/boot sharing one blob store. Worth the complexity
|
||||
now, or not?
|
||||
- **Can the kernel mount a `.sip` read-only directly?** If it can, the installation environment's root can simply *be* the `.sip`.
|
||||
|
||||
The GUI for this is a Wayland client; see [wayland.md](wayland.md).
|
||||
The GUI for this is an X client, like everything else on the desktop.
|
||||
|
||||
## 6. SchFS beyond SchOS
|
||||
|
||||
In the order they should happen.
|
||||
|
||||
**~~An on-disk format version~~, and an upgrade path.** The version policy is
|
||||
done — see [../architecture/schfs.md](../architecture/schfs.md). What remains is
|
||||
the **upgrade step**, and it cannot be written until there are two formats to
|
||||
convert between; the first bump is when that becomes real work rather than
|
||||
speculation.
|
||||
**An upgrade step.** The format version policy is done ([schfs.md](../architecture/schfs.md)); the upgrade step cannot be
|
||||
written until there are two formats to convert between, and the first bump is when it becomes real work.
|
||||
|
||||
The format is now pinned, which is what the other two items were waiting on.
|
||||
**A Linux implementation, FUSE first.** Three reasons in order of weight: it can be tested with `fsx` and `xfstests`, which
|
||||
are far harsher than anything written alongside the code; it carries no licence entanglement, since libfuse is LGPL and used
|
||||
across a process boundary; and it can reuse the read/write logic already in `tools/mkschfs` and `tools/schfs_fsck` rather than
|
||||
being written twice and having the two disagree. A kernel module comes only after FUSE passes `xfstests`; SchOS is GPL-2.0-or-later
|
||||
([../licensing.md](../licensing.md)), so a Linux module is a technical project now and not a licensing one.
|
||||
|
||||
**A Linux implementation, FUSE first.** Three reasons in order of weight: it can
|
||||
be tested with `fsx` and `xfstests`, which are far harsher than anything written
|
||||
alongside the code; it carries no licence entanglement, since libfuse is LGPL and
|
||||
used across a process boundary; and it can reuse the read/write logic already in
|
||||
`tools/mkschfs` and `tools/schfs_fsck` rather than being written twice and having
|
||||
the two disagree.
|
||||
|
||||
A kernel module is a different project, comes only after FUSE passes `xfstests`,
|
||||
and carries the licence problem [../licensing.md](../licensing.md) describes.
|
||||
|
||||
**The missing features**, listed honestly before adding any: no journal replay
|
||||
across an unclean shutdown beyond what copy-on-write gives for free, no
|
||||
snapshots, no sparse-file support, and no online `fsck`. **Snapshots are the one
|
||||
where the existing design already pays for most of the cost** — the B-tree is
|
||||
copy-on-write, which is most of the machinery; nothing takes or names a snapshot.
|
||||
**The missing features**, listed honestly before adding any: no journal replay across an unclean shutdown beyond what
|
||||
copy-on-write gives for free, no snapshots, no sparse-file support, and no online `fsck`. **Snapshots are the one where the
|
||||
existing design already pays for most of the cost** — the B-tree is copy-on-write, which is most of the machinery; nothing takes
|
||||
or names a snapshot.
|
||||
|
||||
## 7. The kernel's own gaps
|
||||
|
||||
Not phases, and each one improves everything above it:
|
||||
|
||||
- **A slab or pool allocator.** `kalloc` hands out one page at a time, and every
|
||||
subsystem written since has invented its own fixed-size table to work around
|
||||
that. This is the single change that would most improve the rest of the kernel.
|
||||
- **A page cache**, which is what a general writable `MAP_SHARED` needs.
|
||||
[foundations.md](foundations.md) delivers a `memfd` instead, which is narrower
|
||||
and sufficient for the GUI.
|
||||
- **Symlinks in procfs**, so `/proc/<pid>/exe` can be one and `ps -o comm` stops
|
||||
printing `[init]`. Belongs with the rest of the VFS work rather than on its own.
|
||||
- **The `ls -lR` hang** — **not reproducible**, and kept here because the absence
|
||||
of a result is a result. Instrumented and hunted at length; see Gate 0 of
|
||||
[1.0.md](1.0.md). Two real defects were found on the way and fixed, neither
|
||||
proven to be the cause.
|
||||
- **No garbage collector for descriptors in flight.** An `AF_UNIX` socket passed
|
||||
over itself and never received leaks the pair. Bounded at thirty-two sockets,
|
||||
and nothing in the tree does it — but a third-party program could.
|
||||
- **A page cache**, which is what a general writable `MAP_SHARED` of a file needs. `memfd` is the narrow answer and has been enough.
|
||||
- **A working `mprotect`, and NX.** `mprotect` is accepted and ignored and every mapping is executable; RELRO, guard pages
|
||||
and a W^X JIT all want the real thing.
|
||||
- **Per-CPU run queues** and work stealing, with a load balancer that is not worse than the lock it replaces.
|
||||
- **A garbage collector for descriptors in flight.** An `AF_UNIX` socket passed over itself and never received leaks the pair;
|
||||
bounded at thirty-two, and nothing in the tree does it.
|
||||
- **Kernel-time accounting**, so `ru_stime` is not zero.
|
||||
- **A contiguous multi-page allocator**, so `struct proc` and `struct socket` can come from somewhere other than fixed tables.
|
||||
|
||||
---
|
||||
|
||||
## Kept out of scope on purpose
|
||||
|
||||
- Anything that needs pushing or publishing. This work stays local.
|
||||
- TrueType hinting.
|
||||
- XWayland.
|
||||
- DRM/KMS as a *kernel* subsystem; the GPU plan serves `/dev/dri` from a process.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,9 @@
|
|||
# SchAPI, and SchOS as a system
|
||||
|
||||
Status: **direction agreed, not yet implemented.** Nothing in this document
|
||||
Status: **direction agreed, not yet implemented**, and frozen until 1.0 ships
|
||||
([1.0.md](1.0.md)). Read it as a direction and not as a schedule: written when the plan was a
|
||||
compositor of SchOS's own, it has not been revisited since the desktop became Xorg and the
|
||||
packages became Alpine's, and §7 (what happens now) in particular predates both. Nothing in this document
|
||||
changes what is being built this week; it changes what everything is being
|
||||
built *towards*. See §7 for what actually happens now.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,177 +0,0 @@
|
|||
# The GUI becomes Wayland
|
||||
|
||||
Schcom survives and changes its wire format. SchWM and SchDE are deleted. What
|
||||
follows is why that is not a retreat, and what it costs.
|
||||
|
||||
Blocked on [foundations.md](foundations.md). Everything there is built except
|
||||
item 4, `setsockopt` that does not lie: descriptor passing, shareable memory,
|
||||
sockets in the filesystem, dynamic linking, and the epoll/eventfd/timerfd/
|
||||
signalfd set libwayland's event loop is made of.
|
||||
|
||||
---
|
||||
|
||||
## What was deleted
|
||||
|
||||
`projects/schwm/` and `projects/schde/`, both README-only; no code was lost. They
|
||||
are **gone from the tree now** rather than marked for deletion — a directory that
|
||||
has said "to be deleted" for months is a corner, not a decision. The arguments
|
||||
each made are kept below, because they are the part worth keeping.
|
||||
|
||||
### SchWM is not abandoned — Wayland dissolves it
|
||||
|
||||
In X11 the window manager is a separate privileged client because X11 puts
|
||||
compositing in the server and policy nowhere. **Wayland's central decision is
|
||||
that the compositor *is* the window manager**: there is no protocol by which a
|
||||
second process places another's windows, and every attempt to add one has been a
|
||||
private extension.
|
||||
|
||||
So SchWM's own argument —
|
||||
|
||||
> *"Compositing is mechanical and must never stall: it runs every frame and a
|
||||
> mistake there is a frozen screen. Window management is policy, full of special
|
||||
> cases and personal taste, and it changes constantly."*
|
||||
|
||||
— is a real and correct observation **about X11**, which Wayland answers a
|
||||
different way. It stays worth honouring, but as a **module boundary inside
|
||||
Schcom**, not a process boundary: policy in its own file behind its own
|
||||
interface. Do not keep it in its own process, because the protocol to talk to it
|
||||
does not exist.
|
||||
|
||||
### SchDE is re-sited rather than abandoned
|
||||
|
||||
A panel, a launcher and a wallpaper are ordinary Wayland clients using
|
||||
`wlr-layer-shell` — MIT, an XML file, no dependency. They need no directory of
|
||||
their own until one of them is written. SchDE's other observation stands on its
|
||||
own and is recorded here so it does not go with the directory: **a GUI terminal
|
||||
is not the kernel console with a window around it.** The console parses ANSI
|
||||
because it draws straight to the framebuffer; a terminal client parses ANSI
|
||||
itself and renders it with proportional-capable text. The two share no code.
|
||||
|
||||
### `projects/schinst/` stays
|
||||
|
||||
Its GUI was the part tied to SchWM, and that part becomes a Wayland client.
|
||||
Everything else in it — the `.sip` format, `mksip`, the apply path, the
|
||||
installation environment — is untouched by any of this, and remains the
|
||||
best-specified unstarted work in the tree.
|
||||
|
||||
---
|
||||
|
||||
## What survives
|
||||
|
||||
**Schcom stays, and keeps its name.** It already owns the display, allocates and
|
||||
owns client buffers, implements surface/attach/commit, routes input from
|
||||
`/dev/input/eventN`, composites multiple clients bottom-to-top, and cleans up
|
||||
after clients that die.
|
||||
|
||||
Its own README says the protocol is deliberately Wayland-shaped — surfaces rather
|
||||
than windows, buffers attached and made visible only on commit, the compositor
|
||||
telling clients when to draw. That was the right call and this is where it is
|
||||
cashed in: **what changes is the wire format and the buffer handle, not the
|
||||
architecture.**
|
||||
|
||||
---
|
||||
|
||||
## What has to exist first
|
||||
|
||||
Items 1–4 of [foundations.md](foundations.md), and the first three are absolute —
|
||||
libwayland does not start without them.
|
||||
|
||||
The sharpest is descriptor passing. `wl_shm_create_pool` passes a **file
|
||||
descriptor**, and there is no path in the protocol that passes an integer
|
||||
instead. `struct schcom_surface`'s `handle_kind` field was written precisely so
|
||||
that switching would be a new value rather than a protocol break, and it cannot
|
||||
help: *our* protocol is the flexible one.
|
||||
|
||||
---
|
||||
|
||||
## Port libwayland. Do not port Weston.
|
||||
|
||||
**libwayland is the small piece**: a wire protocol, an object map, and an event
|
||||
loop, needing `libffi` and — for the build-time scanner, on the host only —
|
||||
`libexpat`. A real port, but a bounded one.
|
||||
|
||||
**Weston is the trap.** It arrives welded to `libinput`, `udev`, `libdrm`,
|
||||
`logind`-shaped seat management and a GL renderer, and porting it means porting
|
||||
Linux's device management first and finding the compositor somewhere underneath.
|
||||
|
||||
This is precisely the lesson already recorded twice in this tree — about
|
||||
FreeBSD's `sys/dev/usb` (100k lines welded to newbus) and about FreeBSD's
|
||||
`sys/netinet` (not a library; reaches into mbufs, epoch, netisr, VNET and uma).
|
||||
|
||||
> The tree has learned this twice. It should not pay to learn it a third time.
|
||||
|
||||
So **Schcom links `libwayland-server` and remains ours.** Input comes from
|
||||
`/dev/input/eventN`, which already works, rather than from libinput. Output goes
|
||||
through `display_ops`, which already exists.
|
||||
|
||||
### Software rendering first, and probably for a long time
|
||||
|
||||
`pixman` for composition, `wl_shm` for buffers. Mesa means EGL means `dlopen`
|
||||
means a dynamic loader -- which now exists (see [foundations.md](foundations.md)
|
||||
item 5), so it is no longer what stands in the way of Mesa; a GPU driver is.
|
||||
Static binaries are still enough to reach a working desktop, and the desktop's
|
||||
own programs can stay static until something concrete demands otherwise.
|
||||
|
||||
### One sharp edge
|
||||
|
||||
`wl_keyboard` hands the client **a keymap as an XKB text blob over a file
|
||||
descriptor**. A compositor cannot avoid xkbcommon by having simpler opinions
|
||||
about keyboards — the protocol requires the blob.
|
||||
|
||||
The cheap route is one precompiled keymap string shipped in the binary, with
|
||||
xkbcommon and `xkeyboard-config` deferred until more than one layout is wanted.
|
||||
|
||||
---
|
||||
|
||||
## Licensing
|
||||
|
||||
Checked before anything is fetched — the standing rule is that the licence goes
|
||||
in `NOTICE` **before** the file is committed, not after.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| wayland, wayland-protocols | MIT |
|
||||
| `wlr-layer-shell` | MIT |
|
||||
| pixman | MIT |
|
||||
| libffi | MIT |
|
||||
| libxkbcommon | MIT |
|
||||
| xkeyboard-config | MIT |
|
||||
|
||||
All GPL-2.0-or-later compatible, all needing only a `NOTICE` entry. The
|
||||
Wayland ecosystem is permissively licensed almost throughout, so nothing here
|
||||
raises a licensing question at all.
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **W0** | [foundations.md](foundations.md) items 1–4, with the combined test that passes a descriptor *and* shares a `memfd` *and* binds a socket at a real path. |
|
||||
| **W1** | libwayland builds and installs into the sysroot. `wayland-scanner` runs on the host. |
|
||||
| **W2** | Schcom serves `wl_compositor`, `wl_shm`, `wl_surface`, `xdg_shell`. A client draws one window. |
|
||||
| **W3** | `wl_seat`: keyboard and pointer, from the evdev path that exists. |
|
||||
| **W4** | Two clients, focus, stacking, move and resize. **This is where SchWM's policy lives, as a file.** |
|
||||
| **W5** | `wlr-layer-shell`, and a panel. This is SchDE's first pixel. |
|
||||
|
||||
Every milestone from W2 on is asserted **from the host, by pixel**, the way
|
||||
`schcom-test` already does. The compositor's own report that it composited
|
||||
something proves nothing.
|
||||
|
||||
---
|
||||
|
||||
## What it will still not do
|
||||
|
||||
- **No hardware acceleration.** Software rendering only, until Mesa, which is
|
||||
not on this plan. Dynamic linking, which it also needs, is in.
|
||||
- **No XWayland**, so no X11 clients, ever, unless someone ports it.
|
||||
- **No mode setting.** Resolution stays whatever the loader asked for — see
|
||||
[hybrid.md](hybrid.md), where the GPU driver's eviction is also where a modeset
|
||||
API would arrive.
|
||||
- **No damage tracking beyond "repaint everything"**, no multiple outputs, no
|
||||
subsurfaces. Schcom's README already lists these; they are additions rather
|
||||
than rewrites, which is the reason for committing to the surface/commit model
|
||||
now.
|
||||
- **One keyboard layout**, until xkbcommon is ported.
|
||||
- **No clipboard, no drag and drop, no screen capture protocol**, none of which
|
||||
W0–W5 touches.
|
||||
|
|
@ -1,44 +0,0 @@
|
|||
# Xorg
|
||||
|
||||
The desktop is X11, on the framebuffer. This replaces [wayland.md](wayland.md)
|
||||
as the plan for Gate 3 of [1.0.md](1.0.md); Wayland stays on file and is not
|
||||
being worked on.
|
||||
|
||||
## Why X and not Wayland
|
||||
|
||||
Every Wayland compositor worth running -- labwc, and everything else built on
|
||||
wlroots -- needs DRM/KMS, libinput, udev and libseat. DRM/KMS is the largest of
|
||||
them by far: a kernel graphics subsystem, and in practice a great deal taken
|
||||
from Linux. Xorg runs on a plain framebuffer through its `fbdev` driver and
|
||||
reads input through `evdev`, both of which are interfaces SchOS already has.
|
||||
It is the shorter road to a desktop by a long way, and it brings the window
|
||||
managers with it: twm first, then Window Maker.
|
||||
|
||||
DRM/KMS is not planned. libinput, udev and libseat are worth having for their
|
||||
own sake and may come later; Xorg does not need them.
|
||||
|
||||
## What is built
|
||||
|
||||
- **The kernel calls** (`b1323e1`, checked by `cmd/xready`): evdev's queries,
|
||||
fbdev's mode-set to the current mode, `KDSETMODE`, `KDSKBMODE`, the
|
||||
`VT_PROCESS` switch handshake, and `/sys/bus/pci/devices/*/{boot_vga,
|
||||
enable,graphics/fb0}` for libpciaccess.
|
||||
- **Dynamic linking and `dlopen`** (`4330576`): Xorg loads its drivers as
|
||||
modules.
|
||||
- **`<linux/*.h>`** in the build sysroot, as `ports/linux-headers`.
|
||||
|
||||
## What is left, in order
|
||||
|
||||
1. **The X client libraries.** xorgproto, xtrans, libXau, libXdmcp,
|
||||
xcb-proto, libxcb, libX11, libXext, libXrender, libXfixes, libxkbfile,
|
||||
libfontenc, libXfont2, freetype, zlib, pixman, libpciaccess.
|
||||
2. **Keyboard and fonts.** xkbcomp and xkeyboard-config, the data X compiles
|
||||
keymaps from; a misc-fixed font.
|
||||
3. **The server.** xorg-server built with meson, GLX, glamor and DRI off;
|
||||
xf86-video-fbdev; xf86-input-evdev with libevdev and mtdev.
|
||||
4. **The first picture.** `twm` and `xterm`, started by hand from a VT.
|
||||
5. **Window Maker**, and an `startx` that works.
|
||||
|
||||
The kernel calls were written against what these programs are documented
|
||||
to do. Each step past it is expected to find a call that was not -- that is
|
||||
what running the real program is for.
|
||||
|
|
@ -26,7 +26,7 @@
|
|||
//
|
||||
// 4 MB was one framebuffer at 1280x800, which was fine while exactly one driver
|
||||
// wanted one. It stopped being fine the day a display driver could be handed to
|
||||
// userspace (docs/plan/hybrid.md): either side of that handover **both drivers
|
||||
// userspace (docs/architecture/devices.md): either side of that handover **both drivers
|
||||
// exist**, and the incoming one has to allocate its framebuffer before the
|
||||
// outgoing one lets go of its own. So the real requirement is two framebuffers
|
||||
// at once, not one.
|
||||
|
|
|
|||
|
|
@ -151,7 +151,7 @@ struct pci_driver {
|
|||
void (*attached)(struct pci_dev *, struct device *);
|
||||
// Give the device up, so a userspace driver can take it.
|
||||
//
|
||||
// docs/plan/hybrid.md is built on the kernel *handing over* a device
|
||||
// docs/architecture/devices.md is built on the kernel *handing over* a device
|
||||
// rather than a process wrestling it away: two drivers writing one set of
|
||||
// registers is not a subtle failure, it is hardware that appears to have
|
||||
// started lying. So the transfer is the incumbent's decision, and this is
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
//
|
||||
// /dev/pci -- device access from userspace.
|
||||
//
|
||||
// docs/plan/hybrid.md, mechanisms 2 and 4: MMIO and DMA to a process. This is
|
||||
// docs/architecture/devices.md, mechanisms 2 and 4: MMIO and DMA to a process. This is
|
||||
// the interface a driver that does not live in the kernel talks to, and it is
|
||||
// deliberately about *PCI devices*, not about any particular driver model.
|
||||
//
|
||||
|
|
@ -23,7 +23,7 @@
|
|||
//
|
||||
// There is no IOMMU. A process holding a DMA buffer can point the device at
|
||||
// any physical address, so this isolates a driver against its own *bugs* and
|
||||
// not against its own *intent*. docs/plan/hybrid.md says so at length and it
|
||||
// not against its own *intent*. docs/architecture/devices.md says so at length and it
|
||||
// remains true here: a crashed driver that restarts instead of taking the
|
||||
// machine down is the whole benefit, and it is not a security boundary.
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@
|
|||
//
|
||||
// It lived next to the driver while exactly one driver read it, with a note
|
||||
// saying it should move when a second reader appeared. One has: cmd/vgpud is
|
||||
// the same device driven from userspace (docs/plan/hybrid.md's first
|
||||
// the same device driven from userspace (docs/architecture/devices.md's first
|
||||
// eviction), and it needs the same register offsets and the same commands.
|
||||
//
|
||||
// Still one file rather than transport-and-protocol split in two. Both readers
|
||||
|
|
|
|||
|
|
@ -63,7 +63,7 @@
|
|||
// is not a number: it names an object the kernel already knows, and handing it
|
||||
// over is the difference between "here is a buffer" and "here is a *copy* of a
|
||||
// buffer". Wayland's wl_shm_create_pool passes one and has no integer-shaped
|
||||
// alternative; docs/plan/hybrid.md's mechanism 1 is this and nothing else.
|
||||
// alternative; docs/architecture/devices.md's mechanism 1 is this and nothing else.
|
||||
//
|
||||
// Every structure below is Linux's, byte for byte, because a musl-linked
|
||||
// program builds them with Linux's headers and we do not get a vote. The
|
||||
|
|
|
|||
|
|
@ -302,7 +302,7 @@ pciuser_mmio_ok(uint64 phys, uint64 len)
|
|||
//
|
||||
// Nothing had noticed because the only consumer was virtio-gpu, whose
|
||||
// BARs are large. The first foreign driver found it in its first minute,
|
||||
// which is the thing docs/plan/hybrid.md said a foreign driver would be
|
||||
// which is the thing docs/architecture/devices.md said a foreign driver would be
|
||||
// good for.
|
||||
//
|
||||
// The widening is real and is worth naming: a sub-page BAR hands the
|
||||
|
|
@ -477,13 +477,13 @@ do_claim(uint64 argp)
|
|||
//
|
||||
// Two drivers on one device is not a subtle failure -- it is two writers of
|
||||
// the same registers, and what it looks like from outside is hardware that
|
||||
// has started lying. The eviction path (docs/plan/hybrid.md) is for the
|
||||
// has started lying. The eviction path (docs/architecture/devices.md) is for the
|
||||
// kernel to *hand over* a device deliberately; until a driver can be told to
|
||||
// let go, the honest answer to "may I have the one you are using" is no.
|
||||
if (d->driver) {
|
||||
// Ask it to let go. A driver with no release hook keeps its device, which
|
||||
// is always safe and, for anything on the path to a shell, correct: the
|
||||
// rule in docs/plan/hybrid.md is that the kernel keeps a minimal driver
|
||||
// rule in docs/architecture/devices.md is that the kernel keeps a minimal driver
|
||||
// for everything needed to reach a prompt and userspace owns the good one.
|
||||
if (!d->drv || !d->drv->release)
|
||||
return -EBUSY;
|
||||
|
|
|
|||
|
|
@ -272,7 +272,7 @@ const struct display_ops vga_display_ops = {
|
|||
// driver that leaves has to name its replacement rather than simply detaching;
|
||||
// detaching alone leaves gfx pointing at a device nobody is driving.
|
||||
//
|
||||
// This is the fallback rule from docs/plan/hybrid.md made real: the kernel
|
||||
// This is the fallback rule from docs/architecture/devices.md made real: the kernel
|
||||
// keeps a minimal driver for everything on the path to a shell, so a fancy
|
||||
// driver leaving degrades to a working screen instead of to no screen.
|
||||
int
|
||||
|
|
|
|||
|
|
@ -557,7 +557,7 @@ static struct {
|
|||
|
||||
// Hand the device to a userspace driver.
|
||||
//
|
||||
// docs/plan/hybrid.md's first eviction, and the reason it is this device: the
|
||||
// docs/architecture/devices.md's first eviction, and the reason it is this device: the
|
||||
// boundary already exists (struct display_ops), it has exactly one consumer,
|
||||
// and there is a working fallback underneath. So letting go is safe in a way
|
||||
// it would not be for the disk the root filesystem is on.
|
||||
|
|
@ -580,7 +580,7 @@ vgpu_release(struct pci_dev *d)
|
|||
if (!g)
|
||||
return -1;
|
||||
|
||||
// The rule from docs/plan/hybrid.md, enforced rather than described: **the
|
||||
// The rule from docs/architecture/devices.md, enforced rather than described: **the
|
||||
// kernel keeps a minimal driver for everything on the path to a shell.** If
|
||||
// there is no framebuffer to fall back to, this device is the only screen the
|
||||
// machine has, and handing it to a process that might crash would make a
|
||||
|
|
@ -645,7 +645,7 @@ vgpu_release(struct pci_dev *d)
|
|||
|
||||
// --- the crash screen, on a device this driver was told to let go ----------
|
||||
//
|
||||
// docs/plan/hybrid.md's bargain has a clause nobody wrote down: a device the
|
||||
// docs/architecture/devices.md's bargain has a clause nobody wrote down: a device the
|
||||
// kernel hands away is a device the kernel cannot use to tell you the machine
|
||||
// died. The second monitor sat there showing a shell prompt from before the
|
||||
// panic, which is the most misleading thing a screen can do.
|
||||
|
|
|
|||
|
|
@ -423,7 +423,7 @@ e1000_link_up(struct device *dev)
|
|||
|
||||
// Let go of the device, so a userspace driver can have it.
|
||||
//
|
||||
// The same hook virtio-gpu grew for H1, on a NIC. docs/plan/hybrid.md's rule
|
||||
// The same hook virtio-gpu grew for H1, on a NIC. docs/architecture/devices.md's rule
|
||||
// is that the kernel keeps a minimal driver for everything on the path to a
|
||||
// shell -- and a network interface is not on that path, so this one may leave.
|
||||
//
|
||||
|
|
|
|||
|
|
@ -339,7 +339,7 @@ sockname(struct socket *s)
|
|||
// ---------------------------------------------------------------------------
|
||||
// Descriptor passing
|
||||
//
|
||||
// docs/plan/hybrid.md mechanism 1, and item 1 of foundations.md. The kernel
|
||||
// docs/architecture/devices.md mechanism 1, and item 1 of the descriptor-passing work (kernel.md). The kernel
|
||||
// holds a reference on every descriptor in flight, which is what makes the
|
||||
// sender free to close its own copy the instant sendmsg returns -- and what
|
||||
// makes dropping a socket with descriptors still queued a thing that has to be
|
||||
|
|
|
|||
|
|
@ -686,7 +686,7 @@ shm_detach(int id)
|
|||
// or race to invent a key.
|
||||
//
|
||||
// It is the same segment machinery underneath, which is exactly what
|
||||
// docs/plan/foundations.md proposed: shmget()'s pages, refcounts, no_free
|
||||
// docs/architecture/kernel.md proposed: shmget()'s pages, refcounts, no_free
|
||||
// VMAs and the fork/exit accounting are all already right. What memfd adds is
|
||||
// a segment that starts empty and is sized by ftruncate(), and a file that
|
||||
// owns one reference to it.
|
||||
|
|
|
|||
|
|
@ -812,7 +812,7 @@ SYSCALL_DEFINE(getsockopt)
|
|||
// programs that need it are not ours: libwayland and toybox's ping both call
|
||||
// recvmsg with Linux's structures and would not notice a friendlier door.
|
||||
//
|
||||
// docs/plan/foundations.md item 1, and docs/plan/hybrid.md mechanism 1.
|
||||
// docs/architecture/kernel.md item 1, and docs/architecture/devices.md mechanism 1.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// Linux caps an iovec at UIO_MAXIOV = 1024, and so does this. It was 32, and
|
||||
|
|
|
|||
|
|
@ -141,11 +141,11 @@ SYSCALL(192, 55, getsockopt)
|
|||
// reads whatever it last left there and invents a flag out of stack litter.
|
||||
SYSCALL(198, 288, accept4)
|
||||
// sendmsg/recvmsg exist for the one thing only they can do: carry a descriptor
|
||||
// from one process to another (SCM_RIGHTS). docs/plan/foundations.md item 1.
|
||||
// from one process to another (SCM_RIGHTS). docs/architecture/kernel.md item 1.
|
||||
SYSCALL(199, 46, sendmsg)
|
||||
SYSCALL(200, 47, recvmsg)
|
||||
// memfd_create: an anonymous shared memory object reached by descriptor.
|
||||
// docs/plan/foundations.md item 2 -- the thing a Wayland buffer crosses on,
|
||||
// docs/architecture/kernel.md item 2 -- the thing a Wayland buffer crosses on,
|
||||
// and the reason the descriptor passing above had to exist first.
|
||||
SYSCALL(201, 319, memfd_create)
|
||||
|
||||
|
|
|
|||
|
|
@ -27,18 +27,14 @@ Concretely, code in here:
|
|||
|
||||
| Directory | What it is |
|
||||
|------------|-----------------------------------------------------------------|
|
||||
| `schcom/` | SchOS Compositor. Owns the display, composites clients, routes input. |
|
||||
| `schcom/` | SchOS Compositor v0. Superseded by Xorg; awaiting deletion (see Status). |
|
||||
| `schinst/` | The installer, the installation environment, and `.sip` tooling. |
|
||||
| `schfs-linux/` | SchFS as a filesystem for Linux. FUSE first, a kernel module later. |
|
||||
|
||||
`schwm/` and `schde/` used to be here and have been **deleted**. In Wayland the
|
||||
compositor *is* the window manager — there is no protocol by which a second
|
||||
process places another's windows — and a panel or a launcher is an ordinary
|
||||
client that needs no home of its own until one is written. Both directories held
|
||||
a README and no code, and both had said "to be deleted" for long enough to prove
|
||||
that a directory marked for deletion is not deleted by being marked. What they
|
||||
argued for survives as a module boundary inside Schcom; see
|
||||
[`docs/plan/wayland.md`](../docs/plan/wayland.md).
|
||||
`schwm/` and `schde/` used to be here and have been **deleted**: both held a README and no code, and both
|
||||
had said "to be deleted" for long enough to prove that a directory marked for deletion is not deleted by being
|
||||
marked. The desktop is Xorg with Window Maker ([`docs/architecture/desktop.md`](../docs/architecture/desktop.md)),
|
||||
so the window manager is someone else's and a panel or launcher is an ordinary client.
|
||||
|
||||
**`schfs-linux/` is written to stand on its own.** It does not mention SchOS
|
||||
anywhere, and that is deliberate rather than an oversight: it is intended to be
|
||||
|
|
@ -55,17 +51,9 @@ by anything above it.
|
|||
|
||||
## Status
|
||||
|
||||
`schcom/` has a v0: 715 lines that run, over a wire format of its own. **That
|
||||
wire format is superseded** — the compositor moves onto Wayland, keeping the
|
||||
architecture and replacing the protocol. See
|
||||
[`docs/plan/wayland.md`](../docs/plan/wayland.md).
|
||||
`schcom/` is a compositor v0, 715 lines that run, over a wire format of its own. **It is superseded**: the desktop is
|
||||
Xorg on the framebuffer, not a compositor of ours, and nothing speaks Schcom's protocol. It is on the register in
|
||||
[`docs/plan/1.0.md`](../docs/plan/1.0.md) as *built and not connected*: delete it, or say what it is for.
|
||||
|
||||
Everything Schcom was blocked on in the kernel now exists: unix domain sockets,
|
||||
raw input events, shared memory, framebuffer `mmap`, and — as of the descriptor
|
||||
passing work — `SCM_RIGHTS`, which is the one thing a Wayland client cannot be
|
||||
written without. What is still missing is a **shareable memory object**, item 2
|
||||
of [`docs/plan/foundations.md`](../docs/plan/foundations.md): a Wayland buffer
|
||||
is a file descriptor for memory, and a System V segment id is not one.
|
||||
|
||||
`schinst/` and `schfs-linux/` are scoped and unstarted, and are **frozen** —
|
||||
see [`docs/plan/1.0.md`](../docs/plan/1.0.md).
|
||||
`schinst/` and `schfs-linux/` are scoped and unstarted, and are **frozen** -- see
|
||||
[`docs/plan/1.0.md`](../docs/plan/1.0.md).
|
||||
|
|
|
|||
BIN
projects/schcom/schcom
Normal file
BIN
projects/schcom/schcom
Normal file
Binary file not shown.
BIN
projects/schcom/schcom-demo
Normal file
BIN
projects/schcom/schcom-demo
Normal file
Binary file not shown.
|
|
@ -23,7 +23,7 @@ import sys
|
|||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
FIRST_PARTY = ["kernel", "include", "boot", "lib", "cmd", "tools"]
|
||||
FIRST_PARTY = ["kernel", "include", "boot", "lib", "cmd", "tools", "ldls", "etc"]
|
||||
LANGS = {
|
||||
".c": "C", ".h": "C", ".S": "Assembly", ".s": "Assembly", ".asm": "Assembly",
|
||||
".py": "Python", ".sh": "Shell", ".ps1": "PowerShell", ".pl": "Perl",
|
||||
|
|
|
|||
Loading…
Reference in a new issue