nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

docs: the tree moved and the documentation did not

Browse source 
The site and several documents still described the system as it was before
virtual terminals, the new schinit, the hybrid work and descriptor passing.

docs/architecture/init.md described the wrong program. It documented a single
schinit that parses manifests and supervises jobs, which has not been true since
PID 1 was split: PID 1 forks a supervisor, spawns jobs on its behalf and reaps;
the job table lives in /proc/schinit; a heartbeat runs both ways; and
proc_reincarnate() re-execs PID 1 in place rather than panicking. Rewritten
around what exists, keeping the reasoning that is still load-bearing -- the five
failures of the sibling-init scheme, the escalation ladder, and the three bugs in
the reincarnation path that each had to happen before they were obvious.

docs/plan/init.md is deleted, which is the tree's own rule for finished work and
had not been applied. Fourteen source comments referenced it by part number, so
they are repointed at the architecture doc by section name rather than left
dangling -- a deleted document nobody notices is worse than a stale one.

docs/architecture/devices.md gained the section it has been missing since
/dev/pci existed: what a driver needs from a kernel, why mmap is checked against
the caller's claims, why DMA is a call and not a mapping, and why the write back
after an interrupt is not optional. Also that a released device does not come
back by itself, which is true and was nowhere.

docs/architecture/networking.md gained a descriptor-passing section, and
docs/architecture/README.md's stack diagram gained the two layers and the third
seam it was missing. "Two seams hold the whole thing together" has not been true
since /dev/pci.

docs/plan/README.md said the hybrid work was deliberately last, on the argument
that an interface with no consumer is wrong in ways nobody can see. The argument
was right; the conclusion was not, and the document now says why it changed --
the alternative to a driver compatibility layer is writing a driver for every
device that exists, and the original objection is answered rather than ignored
because SchOS's own drivers are the consumers.

The site: a Hybrid section with the eviction transcript, cards for virtual
terminals, schinit and drivers outside the kernel, and the numbers remeasured
rather than estimated -- 46,961 lines, 207 syscalls. The boot log in app.js is
from a boot taken today, and the lines are trimmed to what the console can show
without clipping, each still a verbatim prefix of real output.

Two small rendering fixes found by looking at the pages rather than assuming:
.console pre's min-height is now scoped to the hero, so a static transcript sizes
to its content instead of reserving eight lines it does not have; and the docs
reader's contents panel strips inline marks from headings, so a heading written
`proc_reincarnate()` no longer carries its backticks into the sidebar.

Every intra-doc link checked; none broken.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz 2026-09-13 20:05:06 +02:00
commit d3c37f82b6
21 changed files with 548 additions and 336 deletions
Download patch file Download diff file Expand all files Collapse all files

24
docs/plan/1.0.md
View file

@ -237,14 +237,32 @@ memory object, `AF_UNIX` in the filesystem namespace.
Nothing above needs these; everything below stops without them.
### Gate 2 — Init as a role
- [x] ~~**Item 1 — `sendmsg`/`recvmsg` with `SCM_RIGHTS`.**~~ Built and tested;
fourteen checks in `cmd/socktest`, each moving data through the passed
descriptor rather than inspecting it. Described in
[../architecture/networking.md](../architecture/networking.md#descriptor-passing).
This was also mechanism 1 of [hybrid.md](hybrid.md) — one primitive for
both, which is the whole reason those are one plan.
- [ ] **Item 2 — a shareable memory object** (`memfd_create`, and
`mmap(MAP_SHARED)` on it genuinely working between two processes). This is
what a Wayland buffer still cannot cross on.
- [ ] **Item 3 — `AF_UNIX` sockets in the filesystem namespace.**
[init.md](init.md), milestones I0–I3.
### Gate 2 — Init as a role — **done**
Milestones I0–I3, all built. PID 1 forks a supervisor and spawns jobs on its
behalf; the job table lives in `/proc/schinit`; a heartbeat runs both ways; and
`proc_reincarnate()` re-execs PID 1 in place rather than panicking. Both failure
paths are exercised by `sysrq` keys rather than reasoned about.
Before the desktop rather than after, because a compositor being brought up for
the first time crashes constantly, and something that restarts it and says why is
worth far more during that work than at the end of it.
The description is now [../architecture/init.md](../architecture/init.md); the
plan document it replaced has been deleted, per
[the rule](../README.md#the-rule-that-keeps-this-from-rotting).
### Gate 3 — The desktop
[wayland.md](wayland.md), W1–W3, and the terminal.
@ -261,7 +279,7 @@ attention. Nothing here is cancelled.
| | Why it waits |
|---|---|
| **The hybrid kernel** ([hybrid.md](hybrid.md)) | An architecture improvement, not a completeness requirement. It is also the work that most wants a demanding consumer first, and 1.0's compositor is exactly that consumer. |
| **The rest of the hybrid kernel** ([hybrid.md](hybrid.md)) | Still not a completeness requirement — but the mechanisms and the first eviction happened early, because the alternative to a driver compatibility layer is writing a driver for every device that exists, and that is not a plan. What is deferred is the foreign driver host, not the interface it needs. |
| **Dynamic linking, Mesa, SDL3** | Everything at 1.0 is statically linked. This is the single biggest post-1.0 unlock and it is a whole project. |
| **A page cache** | `memfd` covers the desktop's needs. General writable `MAP_SHARED` of a file is a real project with no 1.0 consumer. |
| **`.sip` and the installer** | 1.0 installs by writing a USB stick. That is how most systems start and it is honest. |

50
docs/plan/README.md
View file

@ -19,14 +19,14 @@ Taken together, because they turn out to need the same foundation.
| | |
|---|---|
| **[wayland.md](wayland.md)** | The GUI stack becomes Wayland. SchWM and SchDE are deleted; Schcom survives and changes its wire format. |
| **[hybrid.md](hybrid.md)** | The kernel becomes hybrid — drivers move out into userspace processes, starting with the GPU. |
| **[init.md](init.md)** | PID 1 stops being a process and becomes a role the kernel guarantees is filled. |
| **[hybrid.md](hybrid.md)** | The kernel becomes hybrid — drivers move out into userspace processes, starting with the GPU. **Partly built**: all four mechanisms exist and the GPU driver already runs outside the kernel. |
| ~~init.md~~ | **Built.** PID 1 is now a role the kernel refills; the description moved to [../architecture/init.md](../architecture/init.md). |
And the one they all wait on:
| | |
|---|---|
| **[foundations.md](foundations.md)** | Descriptor passing, shared memory, sockets in the filesystem. The critical path. |
| **[foundations.md](foundations.md)** | Descriptor passing, shared memory, sockets in the filesystem. The critical path. **Item 1 is built** — `sendmsg`/`recvmsg` with `SCM_RIGHTS`. |
And the one that says which parts of the above are actually being worked on:
@ -61,30 +61,49 @@ shared memory are the substrate for handing a buffer to a compositor *and* for
handing a device to a driver process. **Doing it once for both is the entire
reason these are one plan rather than three.**
### 2. `init.md` — before the desktop, not after
### 2. `init.md` — before the desktop, not after — **done**
A compositor being brought up for the first time crashes constantly, and
something that restarts it and says why is worth far more during that work than
at the end of it.
at the end of it. Built; see [../architecture/init.md](../architecture/init.md).
### 3. `wayland.md` — with the GPU driver still in the kernel
### 3. `hybrid.md` — **moved up, and the reason is worth recording**
W1–W3 and a terminal. W4 and W5 are post-1.0.
This document used to put the hybrid work *after 1.0*, deliberately last, on this
argument:
### 4. `hybrid.md` — **after 1.0**
> designing a userspace driver interface with no driver consuming it produces an
> interface that is wrong in ways nobody can see
Deliberately last: **designing a userspace driver interface with no driver
consuming it produces an interface that is wrong in ways nobody can see.** The
compositor is the demanding consumer that makes the interface judgeable, so it
has to exist first.
The argument was sound and the conclusion drawn from it was wrong. What forced
the change was the driver situation: SchOS cannot write a driver for every device
that exists, no project can, and the only route to the hardware support an
established OS has is a compatibility layer with the foreign drivers kept
*outside* the kernel. That layer is this plan's mechanisms. It is not optional and
it is not late work.
The original argument still holds, and it is answered rather than ignored:
**SchOS's own drivers are the consumers.** `vgpud` is a real driver, with real
requirements, and building the interface against it is what keeps it a general
mechanism rather than a hole shaped like something foreign. The compositor is
still a demanding consumer — it is no longer the *only* one available.
Four mechanisms, all built. One eviction done, one to go, then the foreign
driver host.
### 4. `wayland.md` — with the GPU driver either side of the seam
W1–W3 and a terminal. W4 and W5 are post-1.0. Item 1 of `foundations.md` —
descriptor passing — was the thing it could not start without, and it now
exists.
---
## In one sentence
> Fix what is broken, build the descriptor-passing and shared-memory substrate,
> make init survivable, put a terminal on the screen — and only then use the
> compositor as the pressure that shapes the userspace driver interface.
> make init survivable, move the drivers out through an interface SchOS's own
> drivers have to live with first — and put a terminal on the screen.
---
@ -96,6 +115,9 @@ Stated here so it is not mistaken for covered:
hostile driver.
- **No page cache**, so `foundations.md` delivers a `memfd` and not a general
answer to shared file mappings. The general answer is still owed.
- **No garbage collector for descriptors in flight**, so an `AF_UNIX` socket
passed over itself and never received leaks. Bounded at thirty-two sockets, and
nothing on this plan does it.
- **No dynamic linking**, so no Mesa, no accelerated clients, and no third party
shipping a `.so` for anything.
- **A new compatibility surface** between PID 1 and PID 2, created by a plan

212
docs/plan/init.md
View file

@ -1,212 +0,0 @@
# PID 1 becomes a role
What `schinit` is today is [../architecture/init.md](../architecture/init.md).
This is what it becomes, and it is the one item on this plan that deliberately
breaks a Unix rule.
---
## The law being changed
Unix's rule is *PID 1 is special and must never die*. Every init since 1975 has
honoured it by being careful.
> **SchOS's rule instead: PID 1 is a role. The kernel guarantees the role is
> filled. The role's state lives outside whichever process currently holds it.**
That is a real change and it is worth making, because "must never die" is not a
mechanism — it is a hope, and the thing it hopes about is the one program on the
machine with no supervisor.
---
## The proposal this replaces
The original sketch was: init 1 starts init 2; if init 1 is killed, init 2 starts
init 3 and revives init 1.
The instinct is right — init needs something watching it — and one half of the
idea survives below. The specific arrangement has five problems, and four of them
are structural rather than fixable. They are recorded because each one is a trap
that any redundant-supervisor design falls into:
**It defends against the attack that already fails.** Userspace cannot kill PID 1
on this kernel. `init_ignores()` (`kernel/proc/proc.c:620`) drops `SIGKILL` to
`initproc`. The threat is already handled.
**Init 2 is the same binary as init 1.** If init 1 died of its own bug — a null
dereference parsing a manifest, an exec of a truncated binary — init 2 reaches
the same line and dies the same way, and *faster*, because it just watched the
input that caused it.
**The chain has no bound.** Init 2 makes init 3, which by symmetry watches init 2
and makes init 4. Without a cap it is a fork bomb whose author is the supervisor;
with a cap, the cap is the real design and the chain is decoration.
**Two supervisors is split brain.** Both hold the job table. Both see a job's pid
exit. Both restart it. Now the console shell exists twice, both reading
`/dev/console`, and every keystroke goes to whichever woke first. This is the
standard failure of redundant supervisors, and it is worse than no redundancy
because it corrupts the state it was added to protect.
**It only detects death.** The common way an init breaks is not dying — it is
**hanging**: deadlocked, spinning, or blocked forever on a disk that stopped
answering. `wait()` never returns for a process that is merely stuck, so init 2
watches a wedged init 1 forever and reports everything normal. The scheme is
armed against the rare failure and blind to the likely one.
---
## What gets built instead
Four parts. The first three are userspace; the fourth is the kernel change.
### Part 1 — split by role, not by copy
**`schinit` (PID 1)** becomes small enough to be obviously correct and then stops
changing: reap children, hold the job table in memory, supervise exactly one
child. No manifest parsing. No filesystem walking after boot. No allocation. Its
entire job is to be too boring to fail.
**`schinitd` (PID 2)** gets everything else — manifests, socket activation,
respawn policy, backoff, and the eventual `schinitctl`. It is where the bugs will
be, and it is restartable.
The detail that makes this work, and that the two-copy scheme cannot have:
> **Jobs are forked by PID 1, not by `schinitd`.** `schinitd` decides and asks;
> PID 1 acts. So every job on the machine is a child of PID 1 at all times, and
> `schinitd` crashing orphans nothing, reparents nothing and restarts nothing. It
> comes back, reads the state, and finds the world exactly as it left it.
This keeps the good half of the original idea — two processes, one watching the
other — with the roles made *different*, so the second is not a second copy of
the first one's bug.
### Part 2 — state outside the process
**`/proc/schinit`**: the serialised job table, written by PID 1 whenever it
changes, readable by anyone. A resurrected supervisor reads it back and knows
what is running without having started it.
procfs rather than a file on disk, for four reasons: it survives a read-only
root, it survives a full disk, it costs no I/O, and it is already the pattern
this kernel uses for exactly this kind of question.
The side benefit closes an item the architecture doc lists as missing — the job
table is currently invisible. `cat /proc/schinit` is the observation.
### Part 3 — a heartbeat, in both directions
The failure the original scheme cannot see.
PID 1 and `schinitd` hold a `socketpair`; each must be heard from within a
deadline. Silence from `schinitd` means PID 1 kills and restarts it. Silence from
PID 1 is the case part 4 exists for.
**A hang is now as detectable as a death**, which is the whole point.
### Part 4 — `proc_reincarnate()`
The kernel change, and the actual law change.
Today `exit()` on `initproc` calls `panic("init exiting")`
(`kernel/proc/proc.c:277`). Instead: tear down the address space, keep the
`struct proc`, keep **pid 1**, and re-exec `/sbin/schinit` in place with a
generation counter passed in.
No init 2, no init 3, no reparenting problem — **because pid 1 never stops being
pid 1**. Every orphan the kernel already reparented to `initproc` is still
correctly parented, `initproc` is still the pointer it always was, the children
survive, and the state is in `/proc/schinit`.
This is strictly better than resurrection-by-sibling for every case, and it is
the smaller change.
#### The escalation ladder
What makes it safe to have at all:
| Generation | What happens |
|---|---|
| 1 | Normal boot. |
| 2–3 | Re-exec in place. Log it, keep the jobs, carry on. |
| 4 | Re-exec, but ignore `/etc/schinit` and start only a console shell. **A manifest that kills init must not kill it forever.** |
| 5 | Stop. `panic()` with stop code `INIT_RESURRECTION_FAILED`. |
The last row is the honest one. A machine that cannot hold PID 1 up five times is
not a machine that should keep trying, and PSOGM already renders a stop code, a
call chain and a QR code of the whole report — which is a far better end than a
silent loop. The crash screen is the correct place for this to arrive.
---
## Milestones
**All four milestones are built.** `sysrq w` wedges the supervisor and PID 1
replaces it; `sysrq i` kills PID 1 and the kernel refills the role:
```
init: pid 1 exited with status 9 -- reincarnating, generation 2
schinit: reclaimed 2 running job(s) from /proc/schinit
schinit: SchOS process 1, generation 2 (reincarnated)
schinitd: adopting os.schos.console-shell
schinitd: adopting os.schos.tty2-shell
```
The shell being typed into keeps its pid across the death and rebirth of PID 1.
**I0 and I1 first.** `schinit` is 520 lines with no parser and no
allocation; `schinitd` holds the manifests. `kill 3` produces:
```
schinitd: 2 job(s) loaded, generation 1
schinitd: adopting os.schos.console-shell
schinitd: adopting os.schos.tty2-shell
schinitd: ready
```
— and `/proc/schinit` is byte-identical across the restart, same pids, same
cookies, same start times. The shell the user was typing into never noticed.
The first attempt at I1 got this wrong in the instructive way: PID 1 published
the table and `schinitd` never read it, so the restarted supervisor started
everything a *second* time. Two console shells on one terminal, which is the
split-brain this plan rejects the two-init scheme for, reached by a different
road. Publishing state is only half of it; the other half is coming back and
believing it.
| | |
|---|---|
| **I0** ✅ | Split the program. PID 1 supervises one child; `schinitd` does everything else. Jobs still forked by PID 1. |
| **I1** ✅ | `/proc/schinit`, written and read back. Kill `schinitd` by hand and watch it return without disturbing a single job. |
| **I2** ✅ | The heartbeat both ways, with a `sysrq` key that wedges `schinitd` on purpose so the timeout path is exercised rather than assumed. |
| **I3** ✅ | `proc_reincarnate()` and the ladder, with a `sysrq` key that kills PID 1 on purpose. |
---
## What it costs
**PID 1 and PID 2 must agree on a protocol**, which is a compatibility surface
where there was none — created by a plan whose own [schapi.md](schapi.md) §3
warns about exactly that. Keep it tiny, version it in its first message, and keep
it entirely internal. It is not SchAPI and nothing else may speak it.
**`proc_reincarnate()` is a genuinely dangerous function.** It runs in the
context of a process that is dying, and it must not take a lock that process may
already hold. It belongs in `proc.c` next to `exit()`, written with the care of
the panic path, and tested by a `sysrq` key rather than by reasoning.
---
## What it will still not do
- **It does not make init correct**, only survivable. A `schinitd` that starts
the wrong jobs starts them again after every restart.
- **It does not protect the jobs from each other.** A job that fills the disk or
exhausts the process table still takes the machine down, and the supervisor
surviving does not help.
- **A hang in PID 1 itself is only caught by `schinitd` noticing**, and if both
are wedged simultaneously nothing catches it. There is no hardware watchdog.
- **Five generations is a guess**, not a measured number.

28
docs/plan/roadmap.md
View file

@ -25,9 +25,10 @@ would most enlarge what SchOS can boot from.
## 2. Graphics
- **A `/dev/dri`-style device with mode setting**, so resolution stops being
whatever the loader asked for. Worth doing *with*
[hybrid.md](hybrid.md)'s GPU eviction rather than before it — moving the driver
out is what makes a modeset API affordable to design properly.
whatever the loader asked for. This was to be done *with*
[hybrid.md](hybrid.md)'s GPU eviction, and that eviction has now happened —
`cmd/vgpud` drives virtio-gpu from userspace. The modeset API is the next thing
it makes affordable, and is milestone H3 over there rather than here.
- **Double buffering and vblank.** `display_ops` specifies both and no driver
implements either. Schcom needs both to not tear.
- **Port the console onto the text layer and delete the cell grid.** The layer
@ -43,7 +44,7 @@ belongs to the GUI.
## 3. Networking
Three things the stack owed on the day it landed, all still owed:
Two things the stack owed on the day it landed, both still owed:
- **`accept()` does not report the peer's address**, and there is no
`getsockname`/`getpeername`. Nothing has needed them yet; a server that wants
@ -52,8 +53,11 @@ Three things the stack owed on the day it landed, all still owed:
and the second is reachable only by an address on its own subnet. That is a
rule rather than an accident, and it is not routing.
`setsockopt` returning 0 for everything is in
[foundations.md](foundations.md) instead, because the Wayland work needs it.
Two that were on this list are now done and described in
[../architecture/networking.md](../architecture/networking.md): `setsockopt`
honours six options and refuses the rest, and `sendmsg`/`recvmsg` carry
`SCM_RIGHTS`. A third is closer than it was — toybox's `ping` needed `recvmsg`
*and* `IP_RECVTTL`, and only the second is still missing.
## 4. The interactive crash screen
@ -159,11 +163,13 @@ Not phases, and each one improves everything above it:
and sufficient for the GUI.
- **Symlinks in procfs**, so `/proc/<pid>/exe` can be one and `ps -o comm` stops
printing `[init]`. Belongs with the rest of the VFS work rather than on its own.
- **The `ls -lR` hang.** Heavy directory listing across many commands in one long
shell session eventually makes a *later*, unrelated directory hang; the same
command succeeds on a fresh boot. Not inode or file table exhaustion — both
fail cleanly and neither fired. Next step: instrument `itable`/`ftable`
occupancy across a long session and watch what climbs.
- **The `ls -lR` hang** — **not reproducible**, and kept here because the absence
of a result is a result. Instrumented and hunted at length; see Gate 0 of
[1.0.md](1.0.md). Two real defects were found on the way and fixed, neither
proven to be the cause.
- **No garbage collector for descriptors in flight.** An `AF_UNIX` socket passed
over itself and never received leaks the pair. Bounded at thirty-two sockets,
and nothing in the tree does it — but a third-party program could.
---