schfs: commit every 5 s, survive a full disk, mount up to 64 TiB

Transaction groups commit from a schfsd thread every 5 seconds, on sync
and fsync, on unmount and shutdown, and early when freed-but-pending
blocks crowd the free ones -- the contract ext4 and ZFS have. A crash
still lands on the last group whole, and loses at most those seconds.
Committing before every write(2) returned cost two cache flushes and an
uberblock a call: 4 KB writes go from 2.8 to 6.6 MB/s on AHCI, and from
66 KB/s to 3.9 MB/s on QEMU's IDE disk.

Filling the disk made the root volume unmountable, on the last commit
as well: a tree update that ran out of blocks halfway was published by
the next commit. 1% of the volume (1 to 64 MiB) is now a metadata
reserve, as btrfs and ZFS keep one; data stops at it with ENOSPC as a
short write, new names at half of it, and deletes may use the rest. A
write that runs out part way records what it wrote in the inode rather
than leaving blocks past an end of file nothing moved.

Deleting a large file goes in 4 MiB steps from the end with a commit
allowed between them, as ext4 restarts its transaction through a long
truncate. Deleting 80 MB on a full disk ran out of room halfway and
leaked the rest.

The VFS's dirunlink called the object layer's whole unlink, which freed
the object with its last name, so a file removed while open lost its
data under the reader. It removes the name only now, and the object is
freed on the last iput. An object still open when the machine went down
is found by the mount's space scan and freed, like ext4's orphan list.

The allocation bitmap has a second level of pointer pages, so the mount
ceiling is 64 TiB instead of 64 GiB, and the freed-this-group bitmap
has pages only where frees happened: a 1 TiB volume takes 32 MiB.

A freed block written after the newest snapshot is reused at once
(ZFS's birth-time rule, so bfree carries the block's generation); only
older ones are kept while a snapshot exists. Deleting one snapshot no
longer stops retaining for the others, which let their blocks be
handed out again.

Checked under QEMU: crash after sync, after 8 s and within 5 s; fill to
ENOSPC, rm, remount; unlink while open; an orphan freed at the next
mount; a 1 TiB volume. schfs-check passes, with a new test for the
snapshot rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4

docs/architecture/schfs.md

@@ -156,21 +156,72 @@ Directory *depth* is unbounded on disk; path resolution is what limits it.
 
 | | limit | where it comes from |
 |---|---|---|
-| Volume, mounted | **64 GiB** | the in-RAM allocation bitmap: `SCHFS_SPACE_PAGES` (512) pages × 32768 blocks |
-| Volumes mounted at once | **4** | `SCHFS_VOL_SLOTS`; slot 0 is the root |
-| Snapshots mounted at once | **4** | `SCHFS_SNAP_SLOTS` |
-| Volume, via the buffer cache | **4 TiB** | `bio.c` addresses blocks with a `uint`, and one SchFS block spans four of them |
-| RAM per mount | **8 KiB per GiB** of volume | two bitmaps, one bit per block each |
+| Volume, mounted | **64 TiB** | the in-RAM allocation bitmap: `SCHFS_SPACE_DIRS` (1024) pointer pages × 512 bitmap pages × 32768 blocks |
+| Volumes mounted at once | **16** | `SCHFS_VOL_SLOTS`; slot 0 is the root |
+| Snapshots mounted at once | **8** | `SCHFS_SNAP_SLOTS` |
+| RAM per mount | **32 KiB per GiB** of volume | one bit per block, plus a page per 128 MiB that has seen a free since the last commit; 32 MiB for 1 TiB |
+| Metadata reserve | **1%** of the volume, 1–64 MiB | data and new names stop there with `ENOSPC`; see below |
+| Changes lost in a crash | at most **5 s** of them | `SCHFS_COMMIT_TICKS`; `sync` and `fsync` commit at once |
 | Inline value in a leaf | **1016 bytes** | `SCHFS_MAX_INLINE` = a quarter of a node |
 | Colliding names in one directory | ~**60**, sharing a hash | the packed list for one hash must fit in one inline value |
 | Directory entries at 8-byte names | ~**42** per key, unlimited keys | as above |
 | Records per leaf / children per node | **127 / 84** | 4 KiB minus a 32-byte header |
 | Mount time | one pass over the metadata | the allocation map is derived, not stored |
 
-The 64 GiB mount ceiling is the one that bites first, and it is one constant:
-raising `SCHFS_SPACE_PAGES` to 4096 makes it 512 GiB at 64 KiB of pointers per
-mount. It is set where it is because the page-pointer vector lives inside
-`struct schfs_vol`, which the kernel holds as static storage.
+The mount ceiling was 64 GiB, which refused most laptop disks: the page
+vector was one level of 512 pointers inside `struct schfs_vol`. It is two
+levels now, the way a page table is, so the struct holds 1024 pointers to
+pointer pages and the ceiling is 64 TiB for the same 8 KiB. Raising
+`SCHFS_SPACE_DIRS` raises it further.
+
+## Commits, full disks and deletes
+
+**Transaction groups commit every 5 seconds**, not after every operation:
+from the `schfsd` kernel thread, on `sync(2)` and `fsync(2)`, on unmount and
+shutdown, and early when space is short (below). This is the contract ext4
+(5 s), ZFS (5 s) and btrfs (30 s) have, and the crash argument is unchanged: a
+commit is one uberblock write, so a crash lands on the last group whole. What
+it changes is how much can be lost: at most the last few seconds, and nothing
+an `fsync` returned for.
+
+It used to commit before every `write(2)` returned, two cache flushes and an
+uberblock each time. Inside one group a block the group already wrote is
+changed in place, so a tree path rewritten a hundred times between commits
+goes to the disk once; committing per call threw that away. 1 MB in 4 KB
+writes went from 2.8 to 6.6 MB/s on AHCI under QEMU, and from 66 KB/s to
+3.9 MB/s on the emulated IDE disk.
+
+**A full disk stays mountable.** A copy-on-write tree needs new blocks to
+change anything, deleting included, and an update that runs out halfway
+leaves a half-built tree the next commit publishes. That is what filling the
+disk did: the root volume would not mount again. So 1% of the volume (at least
+1 MiB, at most 64 MiB) is kept for metadata, as btrfs's global reserve and
+ZFS's slop space are. File data stops at it with `ENOSPC`, as a short write
+of what fitted. New names stop at half of it. Tree updates in progress and
+deletes may use the rest.
+
+Blocks a group frees stay unusable until it commits, because the committed
+tree still points at them. So a group commits early once those are a quarter
+of what is free, rather than report `ENOSPC` on a disk with room on it.
+
+**Deleting a large file goes in 4 MiB steps** from the end, each one a whole
+operation the group may commit after, as ext4 restarts its transaction through
+a long truncate. One 80 MB delete was 20000 checksum-record deletions in a
+single operation, which on a full disk ran out of room halfway.
+
+**A file is freed on its last close**, not when its last name goes. The VFS
+unlinks the name, drops `nlink`, and frees the object when nothing holds it,
+so a file removed while open still reads. One that was still open when the
+machine went down has no names and no holder: the mount finds such objects
+during the scan it already makes, and frees them, as ext4 does with its orphan
+list.
+
+**Snapshots no longer stop space being reused.** A snapshot holds the tree
+committed before the group it was made in, so it can only hold blocks written
+before that group. A freed block written in or after the newest snapshot's
+group goes back to the free map at once (ZFS's birth-time rule). The older
+ones are kept until a mount works out which snapshots still reach them. It used
+to keep every freed block while any snapshot existed.
 
 ### The rest of SchOS
 
@@ -179,7 +230,7 @@ These are VFS and driver limits rather than SchFS ones:
 - A path is at most **4096 bytes** and a name **255**, Linux's `PATH_MAX` and
   `NAME_MAX` (`VFS_PATH_MAX`, `VFS_NAME_MAX`). Longer is `ENAMETOOLONG`. Both
   used to be cut short without a word, at 128 and 29.
-- `NMOUNT` allows **8** mounts.
+- `NMOUNT` allows **32** mounts, and `NBLKDEV` **32** disks and partitions.
 - The IDE driver handles the primary channel only: **2** ATA devices.
 - The IDE driver uses LBA48 past 128 GiB, so the disk is not the limit there.
 - A single `read`/`write` call is capped at **4 GiB** (`n` is a `uint`); the