H2 of docs/plan/hybrid.md -- the milestone the plan itself calls the one
that proves the whole idea, and the one that cannot be skipped.
# cat /proc/schinit
1 4 11 os.schos.vgpud
# kill 4
schinitd: restarting os.schos.vgpud
vgpud: claimed
vgpud: pid 8 drew a test pattern from userspace
The test pattern ends in a twelve-block tally of the drawing process's
pid, because a second run that redraws an identical picture is
indistinguishable from a first run that never stopped. Read off the
virtio-gpu scanout across four generations it says 4, 8, 10, 12, and the
DMA arena hands back the same physical address every time.
What it took:
* A halt the kernel can apply to a device it does not understand.
Clearing PCI_CMD_BUS_MASTER is the only one there is, and it has to
happen before the DMA goes back to the arena -- a dead driver's device
keeps reading a framebuffer the kernel is about to give to somebody
else. Claim sets the bit, release clears it; pcitest shows it in a bm
column, with a third state so a failed read is never printed as off.
* A refusal. vgpu_release() asks vga_fb_restore() whether there is a
framebuffer to put the console back on, and declines to hand over the
device if there is not. The rule that the kernel keeps a minimal
driver for everything on the path to a shell is worth writing down
only if something enforces it.
* The DMA arena not being owned by two allocators at once. This is the
bug that stopped H2: freevm() frees every mapped page below PHYSTOP
and the arena is the top 16 MB of exactly that range, so a DMA block
mapped into a driver went onto the page allocator's free list when the
driver died, while pciuser_release_all() handed the same pages back to
the arena. The restarted vgpud asked for its framebuffer, got pages
kalloc() had already given out -- including its own page tables -- and
dma_alloc()'s zeroing wiped them, so it died inside the ioctl on the
return from a syscall whose stack had stopped existing. do_dma() now
records a VMA with no_free set, the way shared memory does.
* One exit status. KeepAlive must restart a driver that crashed and must
not restart one that has looked at the machine and found its device
absent: on hardware with no virtio-gpu that was fifteen spawns in
forty-five seconds, each printing a line to the console, for ever.
SCHINIT_EXIT_NORESTART is 78, sysexits.h's EX_CONFIG. It is not a new
manifest key -- the wait status has always travelled from PID 1 to the
supervisor, and note_exit() simply stopped discarding it.
The milestone's third clause, "the compositor reattaches", is not done
and is not claimed: there is no compositor running. That is recorded in
the plan as a Schcom milestone rather than counted here.
Closes the register row "A released PCI device" in docs/plan/1.0.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
The failure this is against is not scope creep. It is abandonment by promotion:
a thing gets built to the point where it works, the next thing becomes more
interesting, and the first is left in a corner -- in the tree, not finished, not
deleted, and not in any list. Enough of those and the tree stops matching its
own documentation.
docs/plan/1.0.md now holds the register, and it is the only place anything
half-built is listed. Four sections, because the kinds are genuinely different:
built and connected to nothing (the text layer, a font from disk, socket
activation, wait_vblank); started and stopped part-way (foundations 2-5, hybrid
H2, Schcom v0, a released PCI device that nothing brings back); named in a "what
it does not do" and still true; and frozen -- scoped, unstarted, not to be
started. The rule is that nothing enters except by being finished or deleted,
and that adding a row requires closing one.
1.0.md already said "two things in flight, never three". It was right and it was
not being followed.
Then the corners themselves, because a register written over a tree full of them
is just a longer list:
- cmd/init is deleted. Nothing had execed /bin/init since initcode.S changed
to /bin/schinit; it was built and installed on every image as an unreachable
fallback, and an unreachable fallback rots rather than saves.
- projects/schwm/ and projects/schde/ are deleted. Both said "to be deleted"
in their own first paragraph, which proves only that a directory marked for
deletion is not deleted by being marked. README-only, no code lost, and the
arguments each made are moved into wayland.md where they are still true.
- projects/schcom/schcom and schcom-demo were tracked binaries. Untracked, and
.gitignore'd.
Three code comments and one document described a boot that has not happened
since 7d44eb7. cons_serial_only starts set, but kmain clears it on its first
lines now, so the splash does not survive the boot and the kernel log is on
screen from the start. That reversal was earned by a Chromebox with no serial
port freezing at boot showing nothing at all, and the comments still argued the
losing side of it.
The site:
- a download page. One ISO, what it has actually been run on -- one machine,
under test -- Ventoy and dd, what the first boot looks like step by step,
and the build if you would rather. The support table names one Chromebox
rather than listing hardware, because one Chromebox is the entire bare-metal
record and a list would be fiction.
- Assets/usb.svg, drawn to the same rule as the other marks: two tones,
geometry only, the accent spent in one place, and that place is the activity
light because it is the part that tells you something is happening.
- Download is in both navs and is the hero's primary action; a closing band
ends the front page on it.
- the uberblock-slots and drivers-outside-the-kernel figures are gone. Three
numbers left, including the fastfetch one, which stays because it is funny.
Two things caught by rendering the pages instead of assuming: .hero p was
beating .filemeta on specificity and the file line was coming out at body size,
and the dd command was one line long enough that the card clipped it at `conv=`
-- a destructive command that can scroll half of itself out of view is a good
way to take somebody's disk. It is two lines now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
The site and several documents still described the system as it was before
virtual terminals, the new schinit, the hybrid work and descriptor passing.
docs/architecture/init.md described the wrong program. It documented a single
schinit that parses manifests and supervises jobs, which has not been true since
PID 1 was split: PID 1 forks a supervisor, spawns jobs on its behalf and reaps;
the job table lives in /proc/schinit; a heartbeat runs both ways; and
proc_reincarnate() re-execs PID 1 in place rather than panicking. Rewritten
around what exists, keeping the reasoning that is still load-bearing -- the five
failures of the sibling-init scheme, the escalation ladder, and the three bugs in
the reincarnation path that each had to happen before they were obvious.
docs/plan/init.md is deleted, which is the tree's own rule for finished work and
had not been applied. Fourteen source comments referenced it by part number, so
they are repointed at the architecture doc by section name rather than left
dangling -- a deleted document nobody notices is worse than a stale one.
docs/architecture/devices.md gained the section it has been missing since
/dev/pci existed: what a driver needs from a kernel, why mmap is checked against
the caller's claims, why DMA is a call and not a mapping, and why the write back
after an interrupt is not optional. Also that a released device does not come
back by itself, which is true and was nowhere.
docs/architecture/networking.md gained a descriptor-passing section, and
docs/architecture/README.md's stack diagram gained the two layers and the third
seam it was missing. "Two seams hold the whole thing together" has not been true
since /dev/pci.
docs/plan/README.md said the hybrid work was deliberately last, on the argument
that an interface with no consumer is wrong in ways nobody can see. The argument
was right; the conclusion was not, and the document now says why it changed --
the alternative to a driver compatibility layer is writing a driver for every
device that exists, and the original objection is answered rather than ignored
because SchOS's own drivers are the consumers.
The site: a Hybrid section with the eviction transcript, cards for virtual
terminals, schinit and drivers outside the kernel, and the numbers remeasured
rather than estimated -- 46,961 lines, 207 syscalls. The boot log in app.js is
from a boot taken today, and the lines are trimmed to what the console can show
without clipping, each still a verbatim prefix of real output.
Two small rendering fixes found by looking at the pages rather than assuming:
.console pre's min-height is now scoped to the hero, so a static transcript sizes
to its content instead of reserving eight lines it does not have; and the docs
reader's contents panel strips inline marks from headings, so a heading written
`proc_reincarnate()` no longer carries its backticks into the sidebar.
Every intra-doc link checked; none broken.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
A leaf's value area holds variable-length records end to end, so a record begins
wherever the one before it finished -- a multiple of 8 only by luck. The code
reads them by casting a pointer into the block, which tells the compiler they
are aligned when they are not.
That works on x86, which is why it has never shown as a bug, and it is undefined
in C. A compiler entitled to assume 8-byte alignment may emit an instruction
that requires it, and the failure would arrive at a different optimisation level
or on a different architecture looking like filesystem corruption rather than
like codegen. UndefinedBehaviorSanitizer reported 31 of them across obj.c,
objw.c, attr.c and space.c.
Seven types now carry SCHFS_PACKED: dirent_rec, extent, extref, index_rec,
attr_rec, snap and inode. Two of those -- attr_rec and inode -- were never
reported, because schfs_objtest does not reach an unaligned instance of either.
They were latent rather than absent, and casting into a leaf is the same lie
whether or not a test has caught it yet.
NO ON-DISK LAYOUT CHANGED, and that was measured before the annotation was
applied rather than assumed: every one of these already carries explicit `pad`
fields and therefore has no implicit padding, so sizes and member offsets are
identical with and without it. Seven _Static_asserts now pin the sizes, because
a record that silently grows misreads every existing volume from that point on
and looks like corruption rather than like a struct that changed.
schfs_node_hdr, schfs_item, schfs_kptr and schfs_uberblock are deliberately left
alone. The node header is exactly 32 bytes, so the items and key pointers after
it are already 8-aligned, and annotating them would cost byte-wise access on the
tree's hottest paths to fix a problem they do not have.
Verified by reading an image written before the change: schfs_fsck --check
passes on structure, checksums, key ordering, data blocks and derived free
space, and the kernel mounts its own root unchanged.
Also here: mkschfs passed a null pointer to qsort for an empty directory. Every
implementation ignores it when the count is zero; the standard does not say they
have to.
make schfs-check now runs with no sanitizer output at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Started as "fix the ASAN leak in schfs_objtest" and the test turned out to be
the messenger. The allocation map was taken through a palloc callback and there
was no way to return it: no schfs_space_release, and a comment in
schfs_space_init reading "caller frees what was taken" for an API the caller
did not have.
Three separate leaks from that one gap:
* The kernel leaked the entire map on every unmount -- up to 2 MB, at 8 KiB
per GiB of volume -- because kmount_unmount flushed, cleared `mounted` and
walked away. This is the one that matters; the others are test-only.
* schfs_space_init stranded a page on its own error path. npages was raised
only after *both* of a pair of allocations succeeded, so a failure on the
second left the first outside the count -- unreachable to any cleanup that
could later exist.
* schfs_objtest leaked 2121728 bytes in 518 allocations, which is what made
the whole SchFS suite exit non-zero regardless of its assertions.
The map now carries its own pfree rather than taking one at release time, so it
can only ever be freed by the allocator that filled it -- the kernel's pages
come from kalloc and a host tool's from calloc, and the two are not
interchangeable. schfs_vol_open and schfs_vol_format release any map the volume
is already holding before building another, which is what stops a re-open
leaking; struct schfs_vol must therefore be zeroed before first use, and the
header now says so.
The test's own leak was the last one found and the simplest: it does
memset(&VOL, 0, sizeof(VOL)) before reopening, which wiped the map's pointers
before vol_open could release them. Eight of those, each now releasing first.
schfs_objtest asserts the balance -- 520 pages taken, 520 given back -- because
a suite that leaks cannot detect a leak: the filesystem's own allocations would
arrive as noise inside the harness's. The next one fails the run instead.
make schfs-check now exits 0. The 31 UBSAN misalignment reports are still
there and are still a Gate 0 item.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Two failures found by running `make schfs-check` rather than by reading it, both
predating the current work -- confirmed by stashing the session's changes and
re-running to byte-identical output.
* AddressSanitizer: 2121728 bytes leaked in 518 allocations, every one from
md_palloc in tools/schfs_objtest.c, the test's own page allocator. The
target exits non-zero, so the whole suite fails whatever the assertions say.
The cost is not the memory -- it is that a real leak in the filesystem would
now arrive as noise inside 2 MB of expected noise.
* UndefinedBehaviorSanitizer: 31 misaligned accesses across obj.c, objw.c,
attr.c and space.c, one root cause. Records are read by casting a pointer
into a tree leaf, and a record's offset within a leaf is not a multiple of
8. This works on x86, which is why it has never shown as a bug, and it is
undefined in C: a compiler entitled to assume 8-byte alignment may emit an
instruction that requires it, and the failure would then look like
filesystem corruption rather than codegen.
Recorded as Gate 0 items rather than as nice-to-haves for one reason: a suite
that is red for known reasons stops being read, and then it is red for unknown
reasons too. guides/testing.md now says so up front, so nobody reads the [ok]
lines and ignores the exit code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
The network thread waited on the timer and nothing else, so a frame that landed
just after a poll sat in the ring for up to 10 ms -- the largest fixed latency in
the system. That was deliberate when the stack was new (a receive path woken by
an interrupt is much harder to debug than one that is asked) and there is now a
working stack to compare against, which is exactly when it becomes safe to do.
The frame is still *pulled* by the thread through .receive. It has to be: a call
into lwIP can reach the driver, the driver can sleep on the buffer cache, and an
interrupt handler may do neither. netdev_rx_signal() delivers nothing and reads
no ring -- it only ends the wait early.
The wait stays on the timer tick rather than moving to a channel of its own, and
that is the design. lwIP's timers must still run when the link is silent, so the
clock has to remain a floor under the wait, and sleep() cannot wait on two things
at once. Joining the tick's existing wakeup costs nothing and leaves one wait
with one predicate.
rx_signal lives under tickslock for the same reason it gets no lock of its own:
sleep() releases tickslock atomically, so a driver setting the flag and waking
under that lock cannot lose the wakeup by arriving between the predicate test and
the sleep. A private lock would not be held across the sleep, and that window
would be back. Calling it from interrupt context is safe on the same terms the
timer handler already relies on -- acquire() disables interrupts, so a CPU
holding tickslock cannot be interrupted into it.
The e1000 signals on RXT0 (a frame or burst has landed) and RXDMT0 (the ring has
fallen below its threshold, which is when waiting for the clock hurts most).
Measured rather than asserted: /proc/netif reports wake_rx and wake_tick, and
across three pings rx_frames and wake_rx both went 2 -> 6 -- every received frame
woke the thread through the interrupt. wake_tick keeps advancing as the idle
floor, which is what lwIP's timers need. socktest still passes and ping is still
0% loss.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/ was ten files where a plan, a description, a guide and a bug report all
sat next to each other. It is now architecture/ (what SchOS is), plan/ (what it
will be), guides/ (how to work on it) and history/ (why it is like this), with
an index that states the one rule holding it together: when something is
finished it moves from plan/ to architecture/ and is rewritten as a description
of the thing rather than a record of building it.
A ticked checkbox says the work happened. It does not say what the work
*produced*, and six months later the checkbox is all that is left. The network
stack proved the point: finished, verified end to end, and for a month the only
description of it anywhere was a checkbox four levels of indentation deep.
So roadmap.md went from 534 lines to 181 -- nothing deleted, the done work
graduated. Seven subsystems that were finished and undescribed now have
descriptions: boot, kernel, syscalls, filesystem, networking, graphics, usb,
init. Every one ends with "What it does not do", which is the section that pays;
a document listing only strengths is marketing, and the reader who needs it most
is the one about to trip over a limitation nobody wrote down.
schfs.md still opened with "phases 1 through 3 1/2 are done, phase 4 is next"
long after all five had landed. It is a description now, with the phase record
and the hard-won bugs moved to history/.
Three corrections found on the way, all load-bearing:
* projects/schcom/README.md claimed SCM_RIGHTS was done. There is no
sendmsg/recvmsg in the kernel at all, and an entire GUI plan rested on it.
* The root README said "No TCP/IP" months after the stack landed.
* history/kernel.md sec.7 was marked "in progress" on a problem that process
groups solved.
Also adds plan/1.0.md, which draws a finish line so the open list stops being
infinite, and marks schwm/ and schde/ superseded: in Wayland the compositor is
the window manager, so that split dissolves rather than being abandoned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Nexoniarz