init: a hang is as detectable as a death, and PID 1 comes back

docs/plan/init.md milestones I2 and I3. The rule change.

**I2 -- the heartbeat.** The failure the two-init scheme could not see is that
an init usually does not die, it *hangs*: wait() never returns for a process
that is merely stuck, so a supervisor watching a wedged init reports
everything normal for ever. PID 1 now pings every two seconds and expects an
answer within ten; silence means stuck, and stuck is worth acting on.

Exercised rather than reasoned about, as the plan asks: `sysrq w` wedges
schinitd by making the scheduler pass over it for ever. It stays alive, keeps
its descriptors and appears in every listing -- which is the point.

Two things that only showed up by running it:

  - kill() sets a flag the target checks when it next runs, and the premise
    here is that it is not running. Waiting for a corpse that may never be
    produced turns the recovery path into a second hang. PID 1 replaces the
    supervisor *without waiting*; the old one's socket is closed, so the moment
    it runs again its first write fails and it exits.
  - the replaced supervisor must notice the hangup. poll() reports POLLHUP and
    never POLLIN again, so a loop that only looks for POLLIN sits there
    complaining that PID 1 has stopped pinging it.

**I3 -- proc_reincarnate().** exit() on initproc panicked. Now the kernel
tears down nothing, keeps pid 1, and re-execs /bin/schinit in place with a
generation counter. The pid stays 1, so every orphan the kernel reparented is
still correctly parented; the children survive; the open files survive, which
is why this is the first line of exit() and not the last; and the job table
survives in /proc/schinit, which is what it is there for.

The escalation ladder: generations 2-3 carry on, generation 4 ignores
/etc/schinit and starts a console shell only -- a manifest that kills init
must not kill it for ever -- and generation 5 stops, with PSOGM rendering
INIT_RESURRECTION_FAILED, a call chain and a QR code.

Three bugs found by running it, each a lesson about exec():

  - exec() returns 0 on success; it does not fail to return. The new image
    starts when the kernel returns to userspace through the rewritten
    trapframe. Returning from exit() is the success path.
  - exit() was reached because `killed` was set, and the trap path checks it
    again on the way out. Leaving it set exits the new image before its first
    instruction: five generations in a few microseconds, a resurrection loop
    that looks exactly like the crash it was meant to survive.
  - exec() keeps open files, which is what gives the new image a console --
    and also keeps the previous life's socket, holding the old supervisor's
    channel up so it never sees a hangup.

Verified: `sysrq i`, and the shell being typed into keeps its pid across the
death and rebirth of PID 1, with /proc/schinit byte-identical either side.

schfs-check passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2

kernel/dev/procfs.c

@@ -488,10 +488,37 @@ proc_sysrq_cmd(char cmd)
   case 't':
     text_selftest();
     break;
+  case 'i':
+    // Kill PID 1, on purpose.
+    //
+    // Signals cannot do this -- init_ignores() drops them, which is correct
+    // and is why the two-init scheme was solving a problem that did not exist.
+    // Setting `killed` directly is the kernel deciding, not userspace asking:
+    // init exits at its next trap return and proc_reincarnate() takes over.
+    //
+    // docs/plan/init.md asks for this to be "tested by a sysrq key rather than
+    // by reasoning", which is the only honest way to know the path works.
+    proc_kill_init();
+    cprintf("sysrq: pid 1 asked to exit -- it should come back\n");
+    break;
+
+  case 'w': {
+    // Wedge the supervisor: make schinitd hang, on purpose, so PID 1's
+    // heartbeat timeout is exercised instead of reasoned about. There is no
+    // way back from this; reboot afterwards.
+    int pid = proc_wedge("schinitd");
+    if (pid)
+      cprintf("sysrq: wedged schinitd (pid %d) -- pid 1 should notice "
+              "within %d seconds\n", pid, 10);
+    else
+      cprintf("sysrq: no process named schinitd to wedge\n");
+    break;
+  }
   default:
     cprintf("sysrq: unknown command '%c' "
             "(known: c = crash, u = dump USB, s = read the USB disk, "
-            "d = display self-test)\n",
+            "d = display self-test, t = text self-test, n = net self-test, "
+            "w = wedge the init supervisor, i = kill pid 1)\n",
             cmd);
     break;
   }

kernel/proc/proc.c

@@ -267,14 +267,113 @@ fork(void)
 // status is already wait(2)-encoded -- see W_EXITCODE() in proc.h. Callers
 // that die from a fault or signal pass W_EXITCODE(0, sig); a normal _exit(n)
 // passes W_EXITCODE(n, 0).
+// PID 1 is a role, and the kernel guarantees the role is filled.
+//
+// This is the rule change docs/plan/init.md exists for. Unix's rule is "PID 1
+// is special and must never die", honoured since 1975 by being careful --
+// which is not a mechanism, it is a hope, and the thing it hopes about is the
+// one program on the machine with no supervisor.
+//
+// So instead: tear nothing down, keep pid 1, and re-exec in place.
+//
+// The reason this is so short is the reason it is the right design. exec()
+// already replaces the address space, and everything else is *deliberately*
+// left alone:
+//
+//   - the pid stays 1, so every orphan the kernel reparented to initproc is
+//     still correctly parented and initproc is still the pointer it was
+//   - the children survive, because nothing killed them
+//   - the open files survive, so the new image still has a console to
+//     complain to -- this runs before exit() closes anything, which is why
+//     the call site is the first line of exit() and not the last
+//   - the state survives, in /proc/schinit, which is the whole reason the
+//     job table lives outside the process holding it
+//
+// A sibling init resurrecting a dead one would have to solve all four of those
+// and could not solve the first at all. This solves them by not creating them.
+//
+// The escalation ladder is what makes it safe to have. A machine that cannot
+// hold PID 1 up is not a machine that should keep trying for ever, and PSOGM
+// renders a stop code, a call chain and a QR code of the whole report -- which
+// is a far better end than a silent loop.
+//
+//   generation 1      normal boot
+//   generation 2-3    re-exec, keep the jobs, carry on
+//   generation 4      re-exec, but ignore /etc/schinit entirely and start only
+//                     a console shell: a manifest that kills init must not
+//                     kill it for ever
+//   generation 5      stop
+uint init_generation = 1;
+
+int
+proc_reincarnate(int status)
+{
+  static char gen[12];
+  char *argv[3];
+  int n = 0, v;
+
+  init_generation++;
+
+  if (init_generation > 5)
+    panic("init resurrection failed: five generations is enough");
+
+  // Decimal, by hand: this runs in a process that is dying and should call as
+  // little as possible.
+  v = (int)init_generation;
+  if (v == 0) {
+    gen[n++] = '0';
+  } else {
+    char tmp[12];
+    int i = 0;
+    while (v > 0) { tmp[i++] = '0' + (v % 10); v /= 10; }
+    while (i-- > 0) gen[n++] = tmp[i];
+  }
+  gen[n] = 0;
+
+  cprintf("init: pid 1 exited with status %d -- reincarnating, generation %d\n",
+          status, init_generation);
+  if (init_generation >= 4)
+    cprintf("init: generation %d starts a console shell only, "
+            "ignoring /etc/schinit\n", init_generation);
+
+  argv[0] = "schinit";
+  argv[1] = gen;
+  argv[2] = 0;
+
+  // Clear `killed` first.
+  //
+  // exit() was reached from the trap-return path *because* killed was set, and
+  // that path checks it again on the way back out. Leaving it set would exit
+  // the brand-new image before its first instruction, and the generation
+  // counter would climb to five in a few microseconds -- a resurrection loop
+  // that looks exactly like the crash it was meant to survive.
+  myproc()->killed = 0;
+
+  // No lock is held here and none may be taken: this is the first line of
+  // exit(), before the file table is walked and before ptable.lock. exec()
+  // does its own begin_op/end_op and is safe from process context.
+  if (exec("/bin/schinit", argv, 0) < 0)
+    return -1;
+  return 0;
+}
+
 void
 exit(int status)
 {
   struct proc *p;
   int fd;
 
-  if(myproc() == initproc)
-    panic("init exiting");
+  if(myproc() == initproc) {
+    // Returning from exit() is the *success* path here. exec() does not fail
+    // to return -- it returns 0 having replaced the address space, and the new
+    // image begins running when the kernel returns to userspace through the
+    // trapframe it rewrote. So there is nothing more to do but go back.
+    if (proc_reincarnate(status) == 0)
+      return;
+    // Only reached if the re-exec itself failed, which is the one case where
+    // there is genuinely nothing left to try.
+    panic("init resurrection failed: could not exec /bin/schinit");
+  }
 
   myproc()->xstate = status;
 
@@ -408,6 +507,38 @@ waitpid(int pid, int *status, int options)
   }
 }
 
+// Stop a process running, by name, and never start it again.
+//
+// This exists for one reason: docs/plan/init.md part 3 says the heartbeat's
+// timeout path should be "exercised rather than assumed", and a hang is the
+// one failure that cannot be produced by killing something. A wedged process
+// is still alive, still has its descriptors open, and still shows up in every
+// listing -- which is exactly what makes it hard to notice and exactly what
+// the heartbeat is for.
+//
+// Deliberately one-way. There is no unwedge: this is a test instrument for a
+// machine that is about to be rebooted, not a job-control mechanism. Real
+// SIGSTOP/SIGCONT is a separate thing and is not implemented.
+int
+proc_wedge(const char *name)
+{
+  struct proc *p;
+  int found = 0;
+
+  acquire(&ptable.lock);
+  for (p = ptable.proc; p < &ptable.proc[NPROC]; p++) {
+    if (p->state == UNUSED || p->pid <= 1)
+      continue;
+    if (strncmp(p->name, name, sizeof(p->name)) != 0)
+      continue;
+    p->wedged = 1;
+    found = p->pid;
+    break;
+  }
+  release(&ptable.lock);
+  return found;
+}
+
 void
 scheduler(void)
 {
@@ -417,7 +548,10 @@ scheduler(void)
     sti();
     acquire(&ptable.lock);
     for(p = ptable.proc; p < &ptable.proc[NPROC]; p++){
-      if(p->state != RUNNABLE) {
+      // A wedged process is passed over for ever. It is still RUNNABLE, still
+      // owns its descriptors and still appears in every listing -- which is
+      // the point: it looks exactly like a process that is merely busy.
+      if(p->state != RUNNABLE || p->wedged) {
         skipped++;
         continue;
       }
@@ -923,4 +1057,16 @@ int proc_exists(int pid)
     }
     release(&ptable.lock);
     return 0;
-}
+}
+// Make PID 1 exit at its next trap return, so proc_reincarnate() runs.
+//
+// Not a signal: init_ignores() drops signals to initproc and should, because
+// init is not killable from userspace on any Unix. This is the kernel
+// deciding on its own behalf, which is a different thing, and it exists so
+// the resurrection path can be exercised rather than reasoned about.
+void
+proc_kill_init(void)
+{
+  if (initproc)
+    initproc->killed = 1;
+}