d1f50c4da3
H2 of docs/plan/hybrid.md -- the milestone the plan itself calls the one
that proves the whole idea, and the one that cannot be skipped.
# cat /proc/schinit
1 4 11 os.schos.vgpud
# kill 4
schinitd: restarting os.schos.vgpud
vgpud: claimed
vgpud: pid 8 drew a test pattern from userspace
The test pattern ends in a twelve-block tally of the drawing process's
pid, because a second run that redraws an identical picture is
indistinguishable from a first run that never stopped. Read off the
virtio-gpu scanout across four generations it says 4, 8, 10, 12, and the
DMA arena hands back the same physical address every time.
What it took:
* A halt the kernel can apply to a device it does not understand.
Clearing PCI_CMD_BUS_MASTER is the only one there is, and it has to
happen before the DMA goes back to the arena -- a dead driver's device
keeps reading a framebuffer the kernel is about to give to somebody
else. Claim sets the bit, release clears it; pcitest shows it in a bm
column, with a third state so a failed read is never printed as off.
* A refusal. vgpu_release() asks vga_fb_restore() whether there is a
framebuffer to put the console back on, and declines to hand over the
device if there is not. The rule that the kernel keeps a minimal
driver for everything on the path to a shell is worth writing down
only if something enforces it.
* The DMA arena not being owned by two allocators at once. This is the
bug that stopped H2: freevm() frees every mapped page below PHYSTOP
and the arena is the top 16 MB of exactly that range, so a DMA block
mapped into a driver went onto the page allocator's free list when the
driver died, while pciuser_release_all() handed the same pages back to
the arena. The restarted vgpud asked for its framebuffer, got pages
kalloc() had already given out -- including its own page tables -- and
dma_alloc()'s zeroing wiped them, so it died inside the ioctl on the
return from a syscall whose stack had stopped existing. do_dma() now
records a VMA with no_free set, the way shared memory does.
* One exit status. KeepAlive must restart a driver that crashed and must
not restart one that has looked at the machine and found its device
absent: on hardware with no virtio-gpu that was fifteen spawns in
forty-five seconds, each printing a line to the console, for ever.
SCHINIT_EXIT_NORESTART is 78, sysexits.h's EX_CONFIG. It is not a new
manifest key -- the wait status has always travelled from PID 1 to the
supervisor, and note_exit() simply stopped discarding it.
The milestone's third clause, "the compositor reattaches", is not done
and is not claimed: there is no compositor running. That is recorded in
the plan as a Schcom milestone rather than counted here.
Closes the register row "A released PCI device" in docs/plan/1.0.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
329 lines
14 KiB
Markdown
329 lines
14 KiB
Markdown
# `schinit` — PID 1 as a role
|
||
|
||
Shaped after launchd rather than sysvinit, and it breaks one Unix rule on
|
||
purpose.
|
||
|
||
---
|
||
|
||
## The rule that was changed
|
||
|
||
Unix's rule is *PID 1 is special and must never die*. Every init since 1975 has
|
||
honoured it by being careful.
|
||
|
||
> **SchOS's rule instead: PID 1 is a role. The kernel guarantees the role is
|
||
> filled. The role's state lives outside whichever process currently holds it.**
|
||
|
||
"Must never die" is not a mechanism — it is a hope, and the thing it hopes about
|
||
is the one program on the machine with no supervisor. What replaced it is four
|
||
pieces: a split by role, state in `/proc`, a heartbeat, and a kernel that
|
||
re-execs PID 1 in place instead of panicking.
|
||
|
||
### The arrangement it is not
|
||
|
||
The obvious design — init 1 starts init 2; if init 1 dies, init 2 revives it —
|
||
was tried on paper and rejected. The failures are worth keeping, because they are
|
||
the traps any redundant-supervisor scheme falls into:
|
||
|
||
- **It defends against an attack that already fails.** Userspace cannot kill
|
||
PID 1 here: `init_ignores()` drops `SIGKILL` to `initproc`.
|
||
- **Init 2 is the same binary.** If init 1 died of its own bug, init 2 reaches
|
||
the same line and dies the same way — *faster*, having just watched the input
|
||
that caused it.
|
||
- **The chain has no bound.** Init 2 makes init 3 by symmetry. With a cap, the
|
||
cap is the real design and the chain is decoration.
|
||
- **Two supervisors is split brain.** Both hold the job table, both see a pid
|
||
exit, both restart it. Now the console shell exists twice, both reading
|
||
`/dev/console`. That is worse than no redundancy: it corrupts the state it was
|
||
added to protect.
|
||
- **It only detects death.** The common way an init breaks is *hanging*, and
|
||
`wait()` never returns for a process that is merely stuck. The scheme is armed
|
||
against the rare failure and blind to the likely one.
|
||
|
||
---
|
||
|
||
## The split
|
||
|
||
| | |
|
||
|---|---|
|
||
| **`schinit`** (PID 1) | Reaps children, holds the job table in memory, supervises exactly one child. No manifest parsing, no filesystem walking after boot, no allocation. About 520 lines. Its entire job is to be too boring to fail. |
|
||
| **`schinitd`** (PID 2) | Manifests, respawn policy, backoff, socket sections, and the eventual `schinitctl`. This is where the bugs will be, and it is restartable. |
|
||
|
||
The detail that makes this work, and that two copies of one program cannot have:
|
||
|
||
> **Jobs are forked by PID 1, not by `schinitd`.** `schinitd` decides and asks;
|
||
> PID 1 acts. Every job on the machine is a child of PID 1 at all times, so
|
||
> `schinitd` crashing orphans nothing, reparents nothing and restarts nothing. It
|
||
> comes back, reads the state, and finds the world exactly as it left it.
|
||
|
||
So there are still two processes with one watching the other — but their roles
|
||
are *different*, which is what stops the second being a second copy of the first
|
||
one's bug.
|
||
|
||
### The protocol
|
||
|
||
`include/schinit.h`. **Every message is exactly one struct**, and PID 1 reads
|
||
`sizeof(struct)` or nothing: no length prefix to get wrong, no partial-message
|
||
state machine, and no allocation in the process that must not fail. The socket is
|
||
a `socketpair` inherited as fd 3.
|
||
|
||
| To PID 1 | | To the supervisor | |
|
||
|---|---|---|---|
|
||
| `HELLO` | version handshake | `WELCOME` | PID 1's version, echoed |
|
||
| `SPAWN` | start this job | `SPAWNED` | the pid, or negative |
|
||
| `PONG` | answer to a ping | `EXITED` | a child was reaped |
|
||
| | | `PING` | answer within the deadline |
|
||
|
||
---
|
||
|
||
## State outside the process
|
||
|
||
`/proc/schinit` is the serialised job table: a generation, the supervisor's pid,
|
||
then one line per running job — cookie, pid, start tick, label.
|
||
|
||
It is **written by PID 1**, because PID 1 is the process that cannot lose it, and
|
||
written whenever the table changes rather than generated on read — a procfs
|
||
generator runs in the kernel, and the kernel has no business walking this
|
||
program's memory.
|
||
|
||
procfs rather than a file on disk for four reasons: it survives a read-only root,
|
||
it survives a full disk, it costs no I/O, and it is already the pattern this
|
||
kernel uses for exactly this kind of question.
|
||
|
||
A resurrected supervisor reads it back and adopts what is already running instead
|
||
of starting it again. The side benefit is that the job table is no longer
|
||
invisible: `cat /proc/schinit` is the observation.
|
||
|
||
---
|
||
|
||
## The heartbeat
|
||
|
||
PID 1 pings the supervisor every 2 seconds and expects an answer within 10. This
|
||
is the failure the sibling-init scheme cannot see: a supervisor that is *stuck*
|
||
rather than dead never causes a `wait()` to return.
|
||
|
||
Silence from `schinitd` means PID 1 replaces it — **without waiting for the
|
||
corpse**, because `kill()` on a wedged process never completes and PID 1 waiting
|
||
for it would be the hang it was trying to detect.
|
||
|
||
Silence from PID 1 is what the kernel change exists for.
|
||
|
||
---
|
||
|
||
## `proc_reincarnate()`
|
||
|
||
`exit()` on `initproc` used to `panic("init exiting")`. Now the kernel tears down
|
||
the address space, keeps the `struct proc`, keeps **pid 1**, and re-execs
|
||
`/bin/schinit` in place with a generation counter passed in.
|
||
|
||
No init 2, no init 3, and no reparenting problem — **because pid 1 never stops
|
||
being pid 1**. Every orphan the kernel already reparented to `initproc` is still
|
||
correctly parented, `initproc` is still the pointer it always was, the children
|
||
survive, and the state is in `/proc/schinit`.
|
||
|
||
Three details in that path are not obvious and each was a bug first:
|
||
|
||
- **`exec()` returns 0 on success.** It does not fail to return, so the code
|
||
after it is the *success* path, not an error path to panic in.
|
||
- **`killed` has to be cleared first**, or the new image exits before its first
|
||
instruction — five generations in microseconds.
|
||
- **`exec` preserves descriptors**, so the new PID 1 inherits the old
|
||
supervisor's socket and never sees the hangup that tells it the supervisor is
|
||
gone. It has to be closed deliberately.
|
||
|
||
### The escalation ladder
|
||
|
||
| Generation | What happens |
|
||
|---|---|
|
||
| 1 | Normal boot. |
|
||
| 2–3 | Re-exec in place. Log it, keep the jobs, carry on. |
|
||
| 4 | Re-exec, but ignore `/etc/schinit` entirely and start only a console shell. **A manifest that kills init must not kill it forever.** |
|
||
| 5 | Stop. `panic()`. |
|
||
|
||
The last row is the honest one. A machine that cannot hold PID 1 up five times is
|
||
not a machine that should keep trying, and PSOGM renders a stop code, a call
|
||
chain and a QR code of the whole report — a far better end than a silent loop.
|
||
|
||
### Both paths are tested by doing them, not by reasoning
|
||
|
||
Two `sysrq` keys exist for this:
|
||
|
||
| | |
|
||
|---|---|
|
||
| `sysrq i` | Kills PID 1. Signals cannot — `init_ignores()` drops them — so this sets `killed` directly, which is the kernel deciding rather than userspace asking. |
|
||
| `sysrq w` | Wedges `schinitd` on purpose, so the heartbeat timeout is exercised. There is no way back from it; reboot afterwards. |
|
||
|
||
```
|
||
init: pid 1 exited with status 9 -- reincarnating, generation 2
|
||
schinit: reclaimed 2 running job(s) from /proc/schinit
|
||
schinit: SchOS process 1, generation 2 (reincarnated)
|
||
schinitd: adopting os.schos.console-shell
|
||
schinitd: adopting os.schos.tty2-shell
|
||
```
|
||
|
||
The shell being typed into keeps its pid across the death and rebirth of PID 1.
|
||
|
||
---
|
||
|
||
## Why launchd
|
||
|
||
**Demand launching.** The supervisor holds the listening socket or watches the
|
||
path, and starts a job when something actually arrives.
|
||
|
||
That inverts the dependency graph. A job stops needing to declare what must be
|
||
running before it, because the thing it depends on answers on first use — and
|
||
runlevels stop being a concept at all.
|
||
|
||
SchOS has the two pieces that make it possible: `AF_UNIX` sockets and a real
|
||
`/proc`. Socket sections are parsed and remembered but not yet held, so this is
|
||
still the reason for the shape rather than a feature of it.
|
||
|
||
## Constraints on the programs themselves
|
||
|
||
Both link against `lib/`, which is raw syscall stubs. **There is no libc here** —
|
||
no `malloc`, no `printf`, no `strlen`. Everything is static storage and
|
||
hand-rolled, which is also why the limits are constants rather than growable:
|
||
|
||
| | |
|
||
|---|---|
|
||
| `MAXJOBS` | 32 |
|
||
| `MAXARGS` | 16 |
|
||
| manifest size | 4 KB, read into one static buffer |
|
||
|
||
## Bootstrap
|
||
|
||
Before any job runs, PID 1 creates the device nodes and mounts that a job needs
|
||
in order to have somewhere to send its output. **This cannot itself be expressed
|
||
as a job** without a bootstrapping problem, which is why it is a function and not
|
||
a manifest.
|
||
|
||
Console first, as fd 0/1/2, so every message after it has somewhere to go. Then
|
||
the `/dev` nodes — minor numbers under major 1 follow Linux's mem-major layout
|
||
where it costs nothing (`null=2`, `zero=5`, `full=7`, `random=8`, `urandom=9`) —
|
||
then `/dev/input/event0` and `event1`, the raw disks, and `/proc`.
|
||
|
||
Two details that look like carelessness and are not:
|
||
|
||
**The nodes are created unconditionally, including for absent hardware.** Opening
|
||
`/dev/hdb` with no drive present fails with `ENXIO`, which is a better answer
|
||
than a missing path. It matters more than it looks: with no node,
|
||
`echo x > /dev/hdb` silently creates an ordinary *file* of that name and appears
|
||
to succeed.
|
||
|
||
**The `mknod`'d `/proc` files still happen and are then covered by the mount.**
|
||
A failure to mount degrades to the old behaviour rather than losing `/proc`
|
||
entirely.
|
||
|
||
## Job manifests
|
||
|
||
Held by `schinitd`, not by PID 1. Every `*.job` file in `/etc/schinit` is loaded
|
||
at boot, found with `getdents64` rather than a hardcoded list — so dropping a
|
||
file in the directory is all it takes to add a job.
|
||
|
||
Sectioned `key = value`, `#` starts a comment, keys are case-insensitive
|
||
(a manifest may say `KeepAlive`, `keepalive` or `KEEPALIVE`; keys are for humans
|
||
and this is not a language).
|
||
|
||
| Key | |
|
||
|---|---|
|
||
| `Label` | A reverse-DNS name, as launchd uses. Identity, not a path. |
|
||
| `Program` | The binary to execute |
|
||
| `Args` | argv, space separated. argv[0] is conventionally the basename, so `Args = sh` for `/bin/sh` |
|
||
| `RunAtLoad` | Start it when schinit loads |
|
||
| `KeepAlive` | Restart it when it exits — unless it exits `SCHINIT_EXIT_NORESTART`, below |
|
||
| `Disabled` | Load it but do not start it |
|
||
| `ThrottleInterval` | Seconds between respawns (default 1) |
|
||
| `ControllingTerminal` | Claim the console |
|
||
| `StandardIn` / `StandardOut` / `StandardError` | Paths to open as fd 0, 1, 2 |
|
||
| `[Socket <name>]` → `Path` | Parsed, recorded, not yet honoured |
|
||
|
||
**Unknown keys and unknown sections are ignored rather than fatal.** A manifest
|
||
written for a newer schinit must not stop this one booting the machine.
|
||
|
||
## Starting a job
|
||
|
||
Each job gets **its own session** (`setsid`), so it cannot reach back and signal
|
||
the rest of the system through a shared controlling terminal.
|
||
|
||
That is exactly what makes `ControllingTerminal` necessary, and the interaction
|
||
is the least obvious thing in the program:
|
||
|
||
> A fresh session has no controlling terminal, and a terminal only delivers input
|
||
> to the foreground process group it is tracking. Without
|
||
> `ControllingTerminal = yes`, the shell comes up, echoes every keystroke through
|
||
> the line discipline, and executes none of them — which looks like a hung shell
|
||
> and is really a terminal that is not listening to it.
|
||
|
||
It is **off by default**. A daemon has no business seizing a terminal, and
|
||
launchd's model is that the terminal is something a job asks for.
|
||
|
||
`exec.c` enables the Linux ABI automatically for musl binaries, so a job must
|
||
not call `enable_linux_abi()` itself — it would renumber the syscalls the child
|
||
still has to make.
|
||
|
||
## Supervision
|
||
|
||
PID 1 loops on `wait()` and reports every reaped pid to the supervisor.
|
||
|
||
**It reaps whatever exited, including processes reparented here when their own
|
||
parent died.** Those are not jobs, and that is the whole handling they need.
|
||
|
||
A `KeepAlive` job is restarted by the supervisor asking PID 1 to spawn it again,
|
||
subject to `ThrottleInterval` — a job that exits immediately and forever would
|
||
otherwise spin the machine. launchd defaults to 10 seconds; 1 is enough here,
|
||
where the common case is a shell someone just exited on purpose and expects back.
|
||
|
||
### One exit status the supervisor reads
|
||
|
||
Throttling stops a restart loop from costing CPU. It does not stop it being a
|
||
loop, and a job that can never succeed on this machine will print its reason
|
||
every `ThrottleInterval` for the rest of the uptime — measured at fifteen spawns
|
||
in forty-five seconds for `vgpud` on a machine with no virtio-gpu.
|
||
|
||
So a `KeepAlive` job that exits **`SCHINIT_EXIT_NORESTART`** (78, sysexits.h's
|
||
`EX_CONFIG`) is not restarted, and the supervisor says which job stopped and
|
||
why:
|
||
|
||
```
|
||
vgpud: no virtio-gpu on this machine
|
||
schinitd: os.schos.vgpud has nothing to do on this machine; not restarting
|
||
```
|
||
|
||
The distinction it draws is the right one: the status is wait(2)-encoded, so a
|
||
job that was **killed or faulted** has a signal in the low bits, never matches,
|
||
and comes back exactly as `KeepAlive` promises. Only a job that exited
|
||
deliberately, saying this machine is not one it can run on, is believed.
|
||
|
||
This is not a new manifest key and there is no policy knob. The wait status has
|
||
always travelled from PID 1 to the supervisor inside `struct schinit_reply`;
|
||
`note_exit()` simply stopped discarding it.
|
||
|
||
## The fallback
|
||
|
||
With no manifests found, the supervisor says so and starts `/bin/sh` on the
|
||
console anyway.
|
||
|
||
> An unbootable-looking machine because a manifest was missing would be a poor
|
||
> trade for purity.
|
||
|
||
---
|
||
|
||
## What it does not do
|
||
|
||
- **Socket activation is not implemented.** The manifest key is parsed and the
|
||
path remembered; nothing holds the socket, so demand launching — the entire
|
||
reason for copying launchd — does not happen yet.
|
||
- **No `schinitctl`.** Jobs cannot be started, stopped or reloaded at runtime.
|
||
They can now be *inspected*, through `/proc/schinit`, which is half of it.
|
||
- **No dependency ordering of any kind**, deliberately — demand launching is
|
||
meant to make it unnecessary, and until that exists there is nothing in its
|
||
place either.
|
||
- **No log capture.** `StandardOut` is a path, so a job's output goes wherever
|
||
the manifest says and nothing aggregates it.
|
||
- **`/proc/schinit` records running jobs, not the whole table.** A job that is
|
||
loaded and stopped is not in it, so a resurrected supervisor learns what is
|
||
alive and re-reads the manifests for everything else.
|
||
- **A new compatibility surface exists between PID 1 and PID 2** that did not
|
||
before, and the tree's own [../plan/schapi.md](../plan/schapi.md) §3 spends a
|
||
section warning about exactly those. `SCHINIT_PROTO` is versioned in the
|
||
handshake, which is the least that can be done about it.
|
||
- **32 jobs, 16 arguments, 4 KB manifests**, all compile-time.
|