Gate 0's last-but-one open item. Drop a TrueType file at \schos\console.ttf on
the ESP and the console is drawn with it; with no file, the face linked into
the kernel image, which is what an early panic needs and is not a degraded
mode.
**The loader reads it, and that is the decision worth writing down.** The
kernel opening the file once the root filesystem is mounted is the obvious
design. It is also wrong: the console has been drawing since the kernel's
first line, so a font arriving later changes the cell size, which changes the
column count, which re-lays out every terminal -- and blanks them. The kernel
log would vanish off the screen at the moment the root filesystem mounted,
which is exactly the log a machine being brought up for the first time is
being read for. Since the next thing on the list is real hardware, losing that
was not a trade worth making.
So the choice happens before anything is drawn. Three details make it cheap:
- **EfiReservedMemoryType**, not EfiLoaderData. The E820 handoff turns
loader data into usable memory, on the argument that the loader's own
pages are the kernel's the moment it runs -- true of the loader, not true
of a buffer it wants the kernel to keep reading from.
- **Below 4 GB**, because the kernel reaches this through its direct map and
that is how far the direct map goes. A font above it would need an ioremap
the console cannot do before there is a console.
- **Used in place.** No copy, which matters because the kernel has no
contiguous allocator to copy 112 KB into -- the only one is the DMA arena,
and that belongs to devices.
Sized without EFI_FILE_INFO and its GUID: seek past the end and ask where that
landed, which the specification requires to clamp.
Every failure falls back and keeps going -- no file, implausible size, no
memory, short read, unparseable file. The last says so in the log, because a
font that silently did not load looks exactly like a font that did.
Verified by booting with JetBrainsMono-ExtraBold.ttf on the ESP and
screendumping the console: visibly the bold face, against the Regular of a
boot with no file. Three boots each way, no traps and no panics, socktest and
memfdtest green in all six.
Nothing ships a console.ttf: the file would be the file already in the kernel,
and two copies of a font are two things that can disagree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
552 lines
31 KiB
Markdown
552 lines
31 KiB
Markdown
# SchOS 1.0
|
||
|
||
A finish line, so that the list of open work stops being infinite.
|
||
|
||
---
|
||
|
||
## Why this document exists
|
||
|
||
[schapi.md](schapi.md) §3 says the stability freeze starts at 1.0 and covers
|
||
whatever is marked `stable` at that moment. **1.0 was never defined**, which
|
||
means there has been no finish line — and with no finish line every gap is
|
||
equally mandatory, the list only grows, and the project feels larger every week
|
||
regardless of how much gets done.
|
||
|
||
That feeling is structural, not a judgement about progress. This document fixes
|
||
the structure by drawing a line and putting most of the open work on the far
|
||
side of it.
|
||
|
||
> **Anything not named in this document is not part of 1.0.** It is not
|
||
> cancelled, not forgotten, and not up for reconsideration until 1.0 ships.
|
||
|
||
---
|
||
|
||
## What 1.0 is
|
||
|
||
> **A machine one person can sit down at and use for an afternoon without
|
||
> falling off an edge.**
|
||
|
||
Concretely — boots on real hardware, roots on SchFS, runs a shell and the real
|
||
toybox userland, is on the network, and puts a windowed terminal on the screen.
|
||
|
||
It is deliberately a **small complete system** rather than a rich incomplete
|
||
one. SchOS 1.0 is not Linux, does not run third-party applications, and has no
|
||
package manager. It is a system that works, end to end, with nothing in it that
|
||
lies about what it did.
|
||
|
||
### The acceptance test
|
||
|
||
One unattended run, asserted from the host, the way everything else here is
|
||
tested. When this passes, it is 1.0:
|
||
|
||
```
|
||
boot on real hardware from a USB stick
|
||
→ SchFS root mounts, generation counter advances
|
||
→ schinit starts the compositor
|
||
→ a terminal window appears and is asserted by pixel
|
||
→ in it: ls, cat, a file written, a network fetch
|
||
→ reboot
|
||
→ the file is still there and the checksum matches
|
||
```
|
||
|
||
Nothing in that sentence is a feature list. Every clause is something that is
|
||
currently either broken, missing, or unproven.
|
||
|
||
---
|
||
|
||
## The freeze, and the register of unfinished work
|
||
|
||
**No new capability is started until this register is empty.**
|
||
|
||
The failure this exists to stop is not laziness and not scope creep — it is
|
||
*abandonment by promotion*. A thing gets built to the point where it works, the
|
||
next thing becomes more interesting, and the first one is left in a corner:
|
||
present in the tree, not finished, not deleted, and not visible in any list.
|
||
Enough of those and the tree stops matching its own documentation, which is the
|
||
state this register was written to end.
|
||
|
||
Two rules, and they are the same rule seen from either end:
|
||
|
||
> **Nothing enters the register except by being finished or being deleted.**
|
||
>
|
||
> **Adding a row requires closing one.** Two things in flight. Never three.
|
||
|
||
A row leaves this table in one of exactly two ways. Either it is **finished** —
|
||
and finished means a description exists in [../architecture/](../architecture/),
|
||
not that the code works — or it is **deleted**, and deleted means gone from the
|
||
tree, not marked for deletion. A directory that has said "to be deleted" for
|
||
months is the corner, not the cure; `projects/schwm/` and `projects/schde/`
|
||
proved that and are now actually gone.
|
||
|
||
### Built, and not connected to anything
|
||
|
||
The galling ones. The work exists, is tested, and nothing uses it.
|
||
|
||
| | Where it stopped |
|
||
|---|---|
|
||
| ~~**The text layer**~~ | **Half closed, and the half that closed is the one that mattered.** The console draws real TrueType through `font_ops` now: cells hold a codepoint, the cell size comes from the font's measured advance, and `kernel/dev/ttf.c` has a caller that is not a self-test. What is still unused is `text.c`'s *proportional* path — the console remains a fixed grid. See Gate 0. |
|
||
| **A font from disk** | Still true, and now deliberate rather than pending: the console typeface is linked into the kernel image, because a panic during boot has to be readable before there is a filesystem. Loading a *second* font from disk is the open part. |
|
||
| **Socket activation** | `schinitd` parses `[Socket name]` and remembers the path. Nothing holds the socket. **This is the entire reason for copying launchd**, and it is the half that was not built. |
|
||
| **`display_ops.wait_vblank`** | In the vtable, implemented by the VBE driver, left null by virtio-gpu. A compositor that needs it cannot rely on it. |
|
||
|
||
### Started, and stopped part-way
|
||
|
||
| | Where it stopped |
|
||
|---|---|
|
||
| **`foundations.md`** | Item 1 built. Items 2 (a shareable memory object), 3 (`AF_UNIX` in the filesystem namespace), 4 and 5 open. **Wayland stops at item 2.** |
|
||
| ~~**`hybrid.md`**~~ | **Closed.** All four mechanisms built; H1, H2 and H3 done. H4 — the USB eviction — is *withdrawn* rather than outstanding: USB is the bus peripherals arrive on, not a peripheral, and an eviction that has to leave an identical driver behind has moved nothing. See [hybrid.md](hybrid.md), "Decided: USB stays in the kernel". |
|
||
| ~~**`ldls.md`**~~ | **Closed.** L0 through L4 built: the loader handles 2,440 stock modules, and both `8139too` and Linux's `e1000` run in a SchOS process through `/dev/pci` with `ping` working over each. The e1000 run is a handover — the kernel was driving that NIC and gave it up. Described in [../architecture/ldls.md](../architecture/ldls.md). Two drivers, one kernel version, one machine. |
|
||
| **Schcom v0** | 715 lines that composite, over a wire format that is superseded. Frozen: it runs, nothing more is added, and it is kept only because it is the one thing in the tree that has ever put a client's pixels on screen. |
|
||
| ~~**`debugger.md`**~~ | **Closed.** D0 through D5 built, including the network transport: `schdbgd` bridges TCP to `/dev/dbgro`, ships disabled, binds loopback, and cannot reach a view that changes state. Described in [../architecture/debugging.md](../architecture/debugging.md). |
|
||
|
||
**The row LDLS spent, and gave back.** `hybrid.md` closed — H3 built, H4
|
||
withdrawn — and `ldls.md` took the slot it freed. It has since closed too, so
|
||
the slot is free again. Worth being honest that this is
|
||
the largest thing ever to enter this register, and that it was **explicitly not
|
||
part of 1.0**: nothing in the acceptance test needs a foreign driver. It was
|
||
taken on because the mechanisms it uses were finished and idle, and because
|
||
the alternative to it is writing a driver for every device that exists.
|
||
|
||
**The row the debugger spent, and gave back.** Adding one requires closing one,
|
||
and *A released PCI device* closed when H2 landed. It is worth saying plainly
|
||
that the debugger is not a capability of the OS the way every other row here is:
|
||
nothing in it ships to a user or appears in the acceptance test. The case for it
|
||
is that **Gate 0 is a list of bugs to find in a kernel that could not be asked
|
||
anything** — so it is the instrument the rest of this register is surveyed with,
|
||
and it earned its slot by making every other row cheaper to close.
|
||
|
||
It has closed. **Every plan in `docs/plan/` that was open is now closed**, which
|
||
means the next thing is Gate 0 itself: the survey, with the instrument built.
|
||
|
||
**Closed since this register was written.** *A released PCI device* — release is
|
||
now symmetric with claim (bus mastering cleared before the DMA is freed, a
|
||
refusal when the device is the only display), a supervised job starts the
|
||
replacement, and it is described in
|
||
[../architecture/devices.md](../architecture/devices.md) under "Taking it back".
|
||
|
||
### Named in a "what it does not do" and still true
|
||
|
||
Not corners — these were written down honestly on the day and stayed written
|
||
down. They are here so the register is the whole picture rather than the
|
||
flattering part of it.
|
||
|
||
| | |
|
||
|---|---|
|
||
| `/proc/<pid>/exe` is a regular file | so `readlink()` fails and `ps -o comm` prints `[init]` |
|
||
| `getrusage` | does not distinguish `RUSAGE_CHILDREN` from `RUSAGE_SELF` |
|
||
| No slab allocator | every subsystem's fixed-size table is a consequence |
|
||
| No `getsockname`/`getpeername` | and `accept()` does not report an AF_INET peer |
|
||
| No routing table | first interface wins, which is a rule and not routing |
|
||
| No descriptor garbage collector | a socket passed over itself and never received leaks the pair |
|
||
|
||
### Frozen — scoped, unstarted, and not to be started
|
||
|
||
Neither cancelled nor in progress. **Do not open these.**
|
||
|
||
| | |
|
||
|---|---|
|
||
| `projects/schinst/` | the installer, the installation environment, `.sip` |
|
||
| `projects/schfs-linux/` | SchFS as a filesystem for Linux |
|
||
| [roadmap.md](roadmap.md) §4 | the interactive crash screen |
|
||
| [roadmap.md](roadmap.md) §6 | SchFS beyond SchOS |
|
||
| [schapi.md](schapi.md) | the API layering, designed and not built |
|
||
|
||
Each of those is well argued and none of them is on the path to the acceptance
|
||
test above. That is the whole test for whether something belongs in this section:
|
||
**not "is it worth doing" but "does 1.0 fail without it".**
|
||
|
||
---
|
||
|
||
## The four gates
|
||
|
||
In order. **Do not start a gate before the one before it closes**, and closing
|
||
means a description exists in [../architecture/](../architecture/) — see
|
||
[the rule](../README.md#the-rule-that-keeps-this-from-rotting).
|
||
|
||
### Gate 0 — Fix what is already wrong
|
||
|
||
**This comes first, before any new capability.** The desktop work is the
|
||
temptation and it is the wrong thing to do next: a compositor built on a kernel
|
||
with an unexplained hang and a socket layer that reports success for things it
|
||
did not do will fail in ways indistinguishable from its own bugs. That is the
|
||
same argument that put `netdev_selftest()` before the TCP/IP stack, and it was
|
||
right then.
|
||
|
||
**Things that are actively wrong:**
|
||
|
||
- [x] ~~**The `ls -lR` hang.**~~ **Not reproducible.** Instrumented and hunted;
|
||
see below. Left here rather than deleted because the absence of a result
|
||
is the result.
|
||
- [x] ~~**`setsockopt` returns 0 without setting anything.**~~ Six options now
|
||
genuinely reach lwIP's pcb and everything else is `ENOPROTOOPT`. Round-trip
|
||
checks in `cmd/socktest`, which a permissive stub cannot pass: it has to
|
||
read back a value it was set *and* refuse an unknown option.
|
||
- [x] ~~**`accept4` silently drops its flags.**~~ It is its own table entry now
|
||
rather than an alias for `accept` — an alias would have had to read a
|
||
fourth argument that `accept` never passes, inventing a `SOCK_NONBLOCK`
|
||
out of whatever the caller last left in that register.
|
||
- [x] ~~**Timestamps on everything created after boot read 1970.**~~ Both
|
||
`know()` (SchFS) and `now_sec()` (v1) returned `uptime()` under comments
|
||
saying the kernel had no wall clock. It has had one since `time_init()`
|
||
started reading the RTC; the comments outlived the condition. Two errors,
|
||
not one — `uptime()` counts 10 ms ticks and they were used as seconds, so
|
||
the clocks also ran 100× fast.
|
||
- [x] ~~**User processes are corrupted on SMP.**~~ **Found and fixed.**
|
||
`syscall_entry` parked `%rax`, `%rcx` and `%rdx` in three *globals*
|
||
across the `FS_BASE` swap, under a comment arguing that `SYSCALL`
|
||
clearing `IF` made it safe. `IF` answers for an interrupt on *this* CPU
|
||
and says nothing about a second CPU running the same three stores — and
|
||
`%rcx` is the user return address `SYSRET` uses, so the CPU that lost the
|
||
race returned its process into another process's instruction stream.
|
||
|
||
That is the whole of it: `rip 0x0` was a process handed a zero and
|
||
following it, and `0xffffffffffffffea` was `-EINVAL` in a swapped `%rax`
|
||
used as a pointer. Per-CPU words at `%gs:32..48` now. Measured at four
|
||
CPUs against the same pristine image: **10 faults in 10 boots before, 0
|
||
in 18 after.** See [../architecture/kernel.md](../architecture/kernel.md),
|
||
"Per-CPU state".
|
||
|
||
- [x] ~~**`$(command substitution)` hangs the shell.**~~ **Not a kernel bug.**
|
||
`ports/zsh/Makefile` answered configure's "does `kill(pid, 0)` return
|
||
ESRCH correctly?" with `no`, which makes zsh `#define ESRCH EINVAL`.
|
||
`waitforpid()` ends its wait on `errno != ESRCH`, so with the kernel
|
||
correctly returning ESRCH (3) and zsh comparing against EINVAL (22), the
|
||
shell waited for ever on a child it had already reaped. One word in a
|
||
cross-compile cache. Autoconf's own default for that check is `yes`;
|
||
omitting the line would have been right. See
|
||
[../history/kernel.md](../history/kernel.md), section 7.
|
||
- [x] ~~**`getrusage` does not distinguish `RUSAGE_CHILDREN`** from
|
||
`RUSAGE_SELF`.~~ It did not distinguish anything: it ignored `who` and
|
||
reported the machine's *uptime* as the caller's user time, so every
|
||
process of every age answered with the same number. Per-process CPU time
|
||
is sampled at the timer tick now, children's time is rolled up on reap,
|
||
and `wait4()`'s fourth argument — a `struct rusage *` that was read off
|
||
the stack and thrown away — is filled. That last one is why zsh's `time`
|
||
reported `16777216.01s user` for everything: 2^24 seconds is
|
||
uninitialised memory, and it never varied because it was never written.
|
||
`time` now says 97% cpu for a busy loop and 0% for `sleep 2`.
|
||
`ru_stime` is a known zero; see
|
||
[../architecture/kernel.md](../architecture/kernel.md).
|
||
- [x] ~~**`/proc/<pid>/exe` is a regular file, not a symlink.**~~
|
||
`lrwxrwxrwx ... exe -> /bin/schinit`, and `readlink` returns it. It also
|
||
needed a real `st_size`, which every other procfs file deliberately
|
||
reports as zero: `readlink(2)` bounds its copy by it, so a symlink of
|
||
size 0 resolves to the empty string — which looks like it worked and is
|
||
worse than the regular file was.
|
||
- [x] ~~**OVMF memory reporting.**~~ The region-list fix did close it. 1 GB
|
||
reports 1017 MB usable, 2 GB reports 2041 MB, which is the right shape:
|
||
the firmware keeps about 7 MB either way.
|
||
- [x] ~~**`ps` printed a header and nothing else.**~~ Found while checking the
|
||
`exe` fix. `ps` with no arguments shows only processes on *this*
|
||
terminal, and `/proc/<pid>/stat` hardcoded `tty_nr` to 0, so nothing
|
||
could ever match — `ps -e` always worked, which is what hid it. Fields
|
||
14-17 (utime/stime/cutime/cstime) were hardcoded zero too, so `TIME` was
|
||
`00:00:00` for everything however long it had run.
|
||
|
||
**Work already done and not connected:**
|
||
|
||
These are the galling ones — the replacement exists, is tested, and is unused.
|
||
|
||
- [x] ~~**Point the console at a real typeface.**~~ It draws JetBrains Mono
|
||
through `font_ops` and `ttf.c`. The blocker was not the font: `vtcell.ch`
|
||
was **one byte** holding a CP437 index, so `Ą` (U+0104) could not be
|
||
*stored*, whatever was drawing it. The cell is a `uint32` codepoint now
|
||
and the cell size comes from the font's measured advance.
|
||
- [ ] **Delete the cell grid.** The other half of the original item, and still
|
||
open. The console is a fixed grid of codepoints; `text.c`'s proportional
|
||
path has no caller. Nothing needs this before the compositor, which is
|
||
why it is separated from the row above rather than left blocking it.
|
||
- [x] ~~**Load a font from disk**~~ and point the console at it. The built-in
|
||
one is linked in on purpose — an early panic has to be readable — so this
|
||
is about a *replaceable* font, not about having one. Drop a TrueType file
|
||
at `\schos\console.ttf` on the ESP and the console is drawn with it.
|
||
|
||
**The loader reads it, not the kernel**, and that is the whole design
|
||
decision. The kernel could open the file once the root filesystem is
|
||
mounted — but the console has been drawing since long before that, so
|
||
switching then means re-measuring the cell and re-laying out every
|
||
terminal mid-boot, which wipes the kernel log off the screen at exactly
|
||
the moment a machine being brought up for the first time needs it. The
|
||
loader allocates the pages as `EfiReservedMemoryType`, so the E820
|
||
handoff marks them reserved, the page allocator never offers them, and
|
||
the kernel uses the file *in place* — no copy, and no need for a
|
||
contiguous allocator the kernel does not have.
|
||
- [x] ~~**Interrupt-driven receive.**~~ The e1000's handler calls
|
||
`netdev_rx_signal()`, which ends the network thread's wait early; the frame
|
||
is still *pulled* by the thread, because an interrupt handler can neither
|
||
sleep on the buffer cache nor take the lwIP lock. The wait stays on the
|
||
tick so lwIP's timers keep their floor. Proven rather than assumed:
|
||
`/proc/netif` gained `wake_rx`/`wake_tick`, and across three pings
|
||
`rx_frames` and `wake_rx` both went 2 -> 6.
|
||
|
||
**Structural, because everything after this compounds on it:**
|
||
|
||
- [x] ~~**A slab or pool allocator.**~~ `kernel/core/kmem.c`. Slabs are pages
|
||
from `kalloc()` with a header at the front; free objects hold the free
|
||
list in their own first eight bytes, so an object needs no header and
|
||
freeing needs no search — the object's page base *is* its slab, and the
|
||
slab names its cache. Two interfaces: named `kmem_cache` for a subsystem
|
||
with one kind of thing, and `kmalloc`/`kmfree` over power-of-two classes
|
||
for everything else. Visible in `/proc/slab`, asserted by
|
||
`make kmem-test` on the host — the same arrangement as `schfs-objtest`,
|
||
so the bookkeeping is checked in a second rather than in a boot.
|
||
|
||
**It does not do what this row said it would, and that is worth
|
||
recording.** The claim was that every fixed table — `NPROC`, `NSOCK`,
|
||
`NDEVICE`, `NFILE`, `NINODE`, `NBUF` — exists to work around `kalloc`.
|
||
Measured: `struct proc` is 10656 bytes, `struct socket` 4928, `struct
|
||
buf` 4272. **All three are larger than a page**, so a page-backed slab
|
||
cannot hold them whatever it does; `NINODE` is a cache and is supposed to
|
||
be bounded; and `NFILE`/`NDEVICE` are 48 and 120 bytes, where the table
|
||
costs 24 KB and 3.8 KB and the ceiling has never been reached. The tables
|
||
were not all working around `kalloc`. Freeing the big ones needs a
|
||
contiguous multi-page allocator, which is a different thing and is not
|
||
written.
|
||
|
||
What it does fix is real and was measured rather than assumed. A pipe is
|
||
640 bytes and came from a whole 4096-byte page, so every `|` in a shell
|
||
command burned 3.5 KB it never touched — **six pipes per page now**. And
|
||
`procread()` built its answer in a 2048-byte array on the kernel stack,
|
||
under a comment that had warned about exactly that since it was written;
|
||
its frame is 64 bytes now instead of ~2100.
|
||
- [x] ~~**SchFS on-disk format version policy.**~~ Done. `SCHFS_VERSION` for
|
||
frame changes, plus `compat`/`ro_compat`/`incompat` feature flags in what
|
||
was `reserved0` — so existing images keep every field offset and read
|
||
unchanged. `uber_check()` now distinguishes *damaged* (step over it; that
|
||
is what the ring is for) from *too new* (stop), because falling back to an
|
||
older uberblock mounts a stale filesystem and reports success. Proven on a
|
||
forged image: the old code mounted the stale root and exited 0.
|
||
Still owed: an actual upgrade step, which is only writable once there are
|
||
two formats. See [../architecture/schfs.md](../architecture/schfs.md).
|
||
|
||
**Found while doing the above, and fixed:**
|
||
|
||
- [x] ~~Every `AF_INET` socket failed with `ENFILE` until DHCP bound.~~
|
||
`inet_new()` required an *address* before it would hand out a socket, and
|
||
reported the refusal as "file table overflow". Creating a socket is not
|
||
the same as having somewhere to send; a bind or connect with no route
|
||
fails on its own with an error that says so. `net_is_ready()` (lwIP up) is
|
||
now distinct from `net_is_up()` (DHCP bound).
|
||
- [x] ~~`socktest` asserted "AF_INET is refused, not faked".~~ True when written,
|
||
false since AF_INET landed — and **green the whole time**, because the
|
||
suite runs before DHCP binds and the `ENFILE` above made the refusal look
|
||
real. A test asserting a failure has to say *which* failure.
|
||
|
||
**The suite itself:**
|
||
|
||
Both of these were found by running `make schfs-check` rather than by reading
|
||
it, and both predate the current work — confirmed by stashing and re-running.
|
||
They are here rather than filed as nice-to-haves because a suite that is red for
|
||
known reasons stops being read, and then it is red for unknown reasons too.
|
||
|
||
- [x] ~~**`make schfs-check` fails at baseline.**~~ Fixed, and it was not only a
|
||
test bug. The allocation map was taken through a `palloc` callback and
|
||
there was **no way to give it back** — no `schfs_space_release`, and a
|
||
comment in `schfs_space_init` reading *"caller frees what was taken"* for
|
||
an API the caller did not have.
|
||
|
||
So the kernel leaked the whole map on **every unmount**, up to 2 MB a
|
||
time, and `schfs_space_init`'s own error path stranded a page it had
|
||
already taken because `npages` was only raised after *both* allocations
|
||
succeeded. The test's 518 leaked pages were the symptom that pointed at
|
||
it.
|
||
|
||
The map now carries its own `pfree` so it can only be freed by the
|
||
allocator that filled it, `schfs_vol_open`/`schfs_vol_format` release any
|
||
map the volume already holds, and unmount releases. `schfs_objtest`
|
||
asserts the balance — 520 pages taken, 520 given back — so the next leak
|
||
fails the suite instead of being absorbed by it.
|
||
|
||
- [x] ~~**31 UndefinedBehaviorSanitizer errors.**~~ Fixed, plus two the test
|
||
never reached. One root cause: a leaf's value area holds variable-length
|
||
records packed end to end, so a record begins wherever the last one ended
|
||
— a multiple of 8 only by luck — and the code read them by casting a
|
||
pointer into the block, which tells the compiler they are aligned when
|
||
they are not.
|
||
|
||
The seven record types that live at arbitrary offsets now carry
|
||
`SCHFS_PACKED`. **No on-disk layout changed**, and that was measured
|
||
before it was applied rather than assumed: every one already had explicit
|
||
`pad` fields, so sizes and member offsets are identical with and without
|
||
the annotation. Seven `_Static_assert`s now pin those sizes, because a
|
||
record that silently grows misreads every existing volume from that point
|
||
on and looks like corruption rather than like a struct that changed.
|
||
|
||
`schfs_node_hdr`, `schfs_item`, `schfs_kptr` and `schfs_uberblock` are
|
||
deliberately left alone — the node header is exactly 32 bytes, so
|
||
everything after it is already 8-aligned, and annotating them would cost
|
||
byte-wise access on the tree's hottest paths to fix a problem they do not
|
||
have.
|
||
|
||
Verified by reading an image written *before* the change: full
|
||
`schfs_fsck --check` passes on structure, checksums, key ordering, data
|
||
blocks and derived free space.
|
||
|
||
Also caught on the way: `mkschfs` passed a null pointer to `qsort` for an
|
||
empty directory. Every implementation ignores it when the count is zero;
|
||
the standard does not say they have to.
|
||
|
||
**The identity item:**
|
||
|
||
- [x] ~~**Boot from a USB disk.**~~ The root device is probed at boot instead of
|
||
compiled in: ramdisk (only in a `RAMDISK_ROOT` build), then USB, then the
|
||
IDE disks, first SchFS volume wins. Verified with **no IDE disk attached
|
||
at all** — boots, mounts the stick, writes a file, and the file is still
|
||
there on the next boot. `dmesg` says `root: USB disk (device 3)`, and a
|
||
machine with no root names every candidate it looked at instead of
|
||
panicking with "no root filesystem". See
|
||
[../architecture/boot.md](../architecture/boot.md).
|
||
|
||
#### The `ls -lR` hang, and why it is closed
|
||
|
||
Hunted with the instrumentation it asked for, under a workload heavier than the
|
||
one that originally triggered it — ten iterations of `ls -lR /bin`, interleaved
|
||
with writes and `cd`, 33 execs and 13378 tree descents. Measured across it:
|
||
|
||
```
|
||
inode_peak 9 / 200 unchanged
|
||
file_peak 9 / 512 unchanged
|
||
buf_pinned_peak 1 / 256 unchanged
|
||
schfs0 locked 0 tx_owner_pid 0 tx_depth 0
|
||
```
|
||
|
||
Nothing climbed, nothing wedged, and the repaired `holdingsleep` assertion never
|
||
fired. **That eliminates this file's own stated hypothesis** — "a lock or
|
||
reference-count leak somewhere in the `*at()` paths" — and it is very likely the
|
||
bug was fixed in passing by the `*at()` dirfd rewrite, the getdents cookie
|
||
rewrite, or SchFS becoming the root, all of which landed after it was reported.
|
||
|
||
It is closed rather than chased further, and that is a deliberate application of
|
||
the two-things-in-flight rule. The instrumentation is permanent: if it ever
|
||
happens again, `/proc/fsstat` names the table, or shows `locked 1` with
|
||
`tx_owner_pid 0` and the volume lock is orphaned. That is a much better position
|
||
than the one it was reported from.
|
||
|
||
Two real defects were found on the way and fixed, neither proven to be the
|
||
cause: `holdingsleep()` never compared the owning pid, which disarmed all seven
|
||
`panic()` assertions built on it; and SchFS's `tx_mine()` compared a bare
|
||
`struct proc *`, which a recycled ptable slot makes ambiguous.
|
||
|
||
### Gate 1 — The foundations
|
||
|
||
[foundations.md](foundations.md), items 1–4. Descriptor passing, a shareable
|
||
memory object, `AF_UNIX` in the filesystem namespace.
|
||
|
||
Nothing above needs these; everything below stops without them.
|
||
|
||
- [x] ~~**Item 1 — `sendmsg`/`recvmsg` with `SCM_RIGHTS`.**~~ Built and tested;
|
||
fourteen checks in `cmd/socktest`, each moving data through the passed
|
||
descriptor rather than inspecting it. Described in
|
||
[../architecture/networking.md](../architecture/networking.md#descriptor-passing).
|
||
This was also mechanism 1 of [hybrid.md](hybrid.md) — one primitive for
|
||
both, which is the whole reason those are one plan.
|
||
- [x] ~~**Item 2 — a shareable memory object.**~~ `memfd_create`, `ftruncate`,
|
||
`mmap(MAP_SHARED)`. Built on the `shmget` segments exactly as
|
||
[foundations.md](foundations.md) proposed — the pages, refcounts,
|
||
`no_free` VMAs and the fork/exit accounting were all already right; what
|
||
memfd adds is a segment reached by *descriptor* rather than by name or
|
||
System V key, which is the whole reason it exists on Linux.
|
||
`cmd/memfdtest`: seventeen checks, ending with a memfd sent over a
|
||
socket to a receiver that closes every inherited copy first.
|
||
- [x] ~~**Item 3 — `AF_UNIX` sockets in the filesystem namespace.**~~ `bind()`
|
||
creates a `T_SOCK` node; `connect()` resolves the path and matches the
|
||
socket bound to *that inode*, not to a string. `ls -l` shows
|
||
`srwxr-xr-x`, `stat()` reports `S_IFSOCK`, `unlink()` removes it, and a
|
||
second bind to a live path is `EADDRINUSE` *from the directory* — which
|
||
is what makes a socket file left by a crashed server refuse a rebind, as
|
||
it must. `cmd/socktest` section 15. `flock()` is still missing and is
|
||
recorded as such.
|
||
|
||
### Gate 2 — Init as a role — **done**
|
||
|
||
Milestones I0–I3, all built. PID 1 forks a supervisor and spawns jobs on its
|
||
behalf; the job table lives in `/proc/schinit`; a heartbeat runs both ways; and
|
||
`proc_reincarnate()` re-execs PID 1 in place rather than panicking. Both failure
|
||
paths are exercised by `sysrq` keys rather than reasoned about.
|
||
|
||
Before the desktop rather than after, because a compositor being brought up for
|
||
the first time crashes constantly, and something that restarts it and says why is
|
||
worth far more during that work than at the end of it.
|
||
|
||
The description is now [../architecture/init.md](../architecture/init.md); the
|
||
plan document it replaced has been deleted, per
|
||
[the rule](../README.md#the-rule-that-keeps-this-from-rotting).
|
||
|
||
### Gate 3 — The desktop
|
||
|
||
[wayland.md](wayland.md), W1–W3, and the terminal.
|
||
|
||
**W4 and W5 are not in 1.0.** Multi-window focus policy and a panel are
|
||
post-1.0. One terminal window that works is the bar.
|
||
|
||
---
|
||
|
||
## Not in 1.0
|
||
|
||
The point of this list is that it is *written down*, so it can stop occupying
|
||
attention. Nothing here is cancelled.
|
||
|
||
| | Why it waits |
|
||
|---|---|
|
||
| **The rest of the hybrid kernel** ([hybrid.md](hybrid.md)) | Still not a completeness requirement — but the mechanisms and the first eviction happened early, because the alternative to a driver compatibility layer is writing a driver for every device that exists, and that is not a plan. What is deferred is the foreign driver host, not the interface it needs. |
|
||
| **Dynamic linking, Mesa, SDL3** | Everything at 1.0 is statically linked. This is the single biggest post-1.0 unlock and it is a whole project. |
|
||
| **A page cache** | `memfd` covers the desktop's needs. General writable `MAP_SHARED` of a file is a real project with no 1.0 consumer. |
|
||
| **`.sip` and the installer** | 1.0 installs by writing a USB stick. That is how most systems start and it is honest. |
|
||
| **SchFS snapshots, FUSE, Linux module, sparse files, online fsck** | The format version (Gate 0) is the prerequisite for all of them, and it is the only part on a clock. |
|
||
| **The interactive crash screen** | PSOGM renders, prints a full report and emits a QR code. That is enough. |
|
||
| **CFF/OTF fonts, kerning, text layout** | TrueType and fixed advances are enough for a terminal. |
|
||
| **Routing, `getsockname`/`getpeername`, IPv6** | One interface, one default route, works. |
|
||
| **Multiple outputs, damage tracking, subsurfaces, xkbcommon** | One screen, full repaints, one keymap. |
|
||
| **Audio** | Does not exist in any form and is not required to use a machine. |
|
||
| **Panel, notifications, settings, file manager** | A terminal is a desktop. |
|
||
| **SchAPI itself** | Deliberately. It should be designed *from* the compositor's experience, not before it. |
|
||
|
||
---
|
||
|
||
## The rule for this list
|
||
|
||
**Adding to Gate 0–3 requires deleting something else from it.**
|
||
|
||
Not a productivity trick — a forcing function. The reason this project feels
|
||
overwhelming is that ~20 fronts have been open simultaneously, and no amount of
|
||
discipline makes that workable. The fix is structural.
|
||
|
||
And one working limit, worth holding to literally:
|
||
|
||
> **Two things in flight. Never three.**
|
||
|
||
One in Gate 0 and one being written up. Everything else is closed — in the
|
||
document, and out of your head.
|
||
|
||
---
|
||
|
||
## How you will know it is going well
|
||
|
||
Not by velocity. By this:
|
||
|
||
**Every closed item leaves a description behind.** When the `ls -lR` hang is
|
||
fixed, `history/kernel.md` gains an entry saying what it actually was. When the
|
||
console moves onto the text layer, `architecture/graphics.md` loses a line from
|
||
its *What it does not do* section.
|
||
|
||
Those "does not do" sections shrinking is the real progress bar, and it is the
|
||
only one that does not lie.
|
||
|
||
---
|
||
|
||
## A note worth keeping
|
||
|
||
This document was written because the work started to feel overwhelming, and
|
||
that is worth recording rather than tidying away — because the feeling was
|
||
about the *shape* of the work and not its quality, and the two are easy to
|
||
confuse when you are inside it.
|
||
|
||
The evidence at the time: SMP working on real hardware after five bugs stacked
|
||
behind each other; a copy-on-write filesystem whose crash test cuts the power one
|
||
write later, 119 times, and demands a clean mount after each; a TCP/IP stack
|
||
asserted from both ends; an xHCI driver written from the specification; and a
|
||
documentation tree that records why things are strange so they do not get
|
||
un-fixed by accident.
|
||
|
||
None of that is the output of a project in trouble. It was a project with too
|
||
many tabs open.
|