init: a hang is as detectable as a death, and PID 1 comes back
docs/plan/init.md milestones I2 and I3. The rule change.
**I2 -- the heartbeat.** The failure the two-init scheme could not see is that
an init usually does not die, it *hangs*: wait() never returns for a process
that is merely stuck, so a supervisor watching a wedged init reports
everything normal for ever. PID 1 now pings every two seconds and expects an
answer within ten; silence means stuck, and stuck is worth acting on.
Exercised rather than reasoned about, as the plan asks: `sysrq w` wedges
schinitd by making the scheduler pass over it for ever. It stays alive, keeps
its descriptors and appears in every listing -- which is the point.
Two things that only showed up by running it:
- kill() sets a flag the target checks when it next runs, and the premise
here is that it is not running. Waiting for a corpse that may never be
produced turns the recovery path into a second hang. PID 1 replaces the
supervisor *without waiting*; the old one's socket is closed, so the moment
it runs again its first write fails and it exits.
- the replaced supervisor must notice the hangup. poll() reports POLLHUP and
never POLLIN again, so a loop that only looks for POLLIN sits there
complaining that PID 1 has stopped pinging it.
**I3 -- proc_reincarnate().** exit() on initproc panicked. Now the kernel
tears down nothing, keeps pid 1, and re-execs /bin/schinit in place with a
generation counter. The pid stays 1, so every orphan the kernel reparented is
still correctly parented; the children survive; the open files survive, which
is why this is the first line of exit() and not the last; and the job table
survives in /proc/schinit, which is what it is there for.
The escalation ladder: generations 2-3 carry on, generation 4 ignores
/etc/schinit and starts a console shell only -- a manifest that kills init
must not kill it for ever -- and generation 5 stops, with PSOGM rendering
INIT_RESURRECTION_FAILED, a call chain and a QR code.
Three bugs found by running it, each a lesson about exec():
- exec() returns 0 on success; it does not fail to return. The new image
starts when the kernel returns to userspace through the rewritten
trapframe. Returning from exit() is the success path.
- exit() was reached because `killed` was set, and the trap path checks it
again on the way out. Leaving it set exits the new image before its first
instruction: five generations in a few microseconds, a resurrection loop
that looks exactly like the crash it was meant to survive.
- exec() keeps open files, which is what gives the new image a console --
and also keeps the previous life's socket, holding the old supervisor's
channel up so it never sees a hangup.
Verified: `sysrq i`, and the shell being typed into keeps its pid across the
death and rebirth of PID 1, with /proc/schinit byte-identical either side.
schfs-check passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
parent
c80b7bf3b2
commit
0def474272
8 changed files with 456 additions and 21 deletions
|
|
@ -52,6 +52,10 @@ extern uint64 tsc_hz;
|
|||
extern uint64 boot_tsc;
|
||||
void monotonic(uint64 *sec, uint64 *usec);
|
||||
int proc_write_schinit(int user_src, uint64 src, uint n);
|
||||
int proc_wedge(const char *name);
|
||||
int proc_reincarnate(int status);
|
||||
void proc_kill_init(void);
|
||||
extern uint init_generation;
|
||||
void cprintf(char*, ...);
|
||||
|
||||
// A log line: the serial port only, never the display.
|
||||
|
|
|
|||
|
|
@ -249,6 +249,9 @@ struct proc {
|
|||
// it; /dev/tty resolves through it. Was impossible to have before there was
|
||||
// more than one terminal to choose between.
|
||||
int ctty;
|
||||
// Never scheduled again. Set by sysrq 'w' to produce a hang on purpose --
|
||||
// see proc_wedge() and docs/plan/init.md part 3.
|
||||
int wedged;
|
||||
|
||||
// Real/effective/saved user and group ids. Nothing enforces permissions with
|
||||
// them yet (the on-disk format has no owner field until SchFS), but they
|
||||
|
|
|
|||
|
|
@ -42,12 +42,33 @@
|
|||
// supervisor -> PID 1
|
||||
#define SCHINIT_HELLO 1 // struct schinit_hello
|
||||
#define SCHINIT_SPAWN 2 // struct schinit_spawn
|
||||
#define SCHINIT_SIGNAL 3 // struct schinit_signal
|
||||
#define SCHINIT_PONG 3 // struct schinit_hello -- answer to a PING
|
||||
|
||||
// PID 1 -> supervisor
|
||||
#define SCHINIT_WELCOME 128 // struct schinit_hello, echoed with PID 1's version
|
||||
#define SCHINIT_SPAWNED 129 // struct schinit_reply -- pid, or negative on failure
|
||||
#define SCHINIT_EXITED 130 // struct schinit_reply -- a child has been reaped
|
||||
#define SCHINIT_PING 131 // struct schinit_hello -- answer within the deadline
|
||||
|
||||
// The heartbeat.
|
||||
//
|
||||
// docs/plan/init.md part 3, and the failure the two-init scheme could not see:
|
||||
// **the common way an init breaks is not dying, it is hanging.** wait() never
|
||||
// returns for a process that is merely stuck, so a supervisor watching a
|
||||
// wedged PID 1 -- or PID 1 watching a wedged supervisor -- reports everything
|
||||
// normal forever.
|
||||
//
|
||||
// So each must be heard from. PID 1 sends a PING every interval and expects a
|
||||
// PONG; silence past the deadline means the other end is stuck rather than
|
||||
// busy, and stuck is the case worth acting on.
|
||||
//
|
||||
// Measured in seconds of uptime(), which counts ticks. Ticks can be *lost*
|
||||
// when interrupts are off too long -- see kernel/core/time.c -- which makes a
|
||||
// tick-measured deadline longer in real time than it looks, never shorter.
|
||||
// That is the safe direction: a machine under load waits longer before
|
||||
// declaring a hang, rather than declaring one that has not happened.
|
||||
#define SCHINIT_PING_SECS 2
|
||||
#define SCHINIT_DEADLINE_SECS 10
|
||||
|
||||
struct schinit_hello {
|
||||
unsigned int op;
|
||||
|
|
|
|||
Loading…
Reference in a new issue