schfs: the commit barrier, reflink, and attributes with an index over them

The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.

Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.

Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.

Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.

Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:

  - statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
    so every field after it landed one slot late: userland read f_frsize
    out of f_namelen, got 255, and df reported a 32 MB root filesystem
    as 1992 KB. It hid for as long as it did because the fields used to
    be invented constants, so wrong numbers looked like the usual ones.
  - struct inode's nlink was a short against SchFS's uint32, so a file
    past 32767 names came back negative -- and iput frees an object
    whose count reaches zero. It is an int now, signed on purpose, so an
    underflow lands somewhere a check can see rather than wrapping.
  - filewrite split every write to fit v1's log. That limit belongs to
    the journal, not to writing, and SchFS makes one transaction of
    whatever it is handed: a 1 MB write was becoming 170 commits and
    170 flushes for nothing. fs_ops carries max_write now, and SchFS
    sets no limit.

Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.

Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Nexoniarz 2026-08-15 16:01:22 +02:00
commit 03cc316001
39 changed files with 3942 additions and 550 deletions

View file

@ -30,16 +30,44 @@
/* CRC32c (Castagnoli), table-free so this stays usable very early. */
/* ------------------------------------------------------------------ */
// Table-driven, with the table built on first use.
//
// This was a bit-at-a-time loop -- eight shifts per byte, 32768 iterations to
// checksum one 4096-byte block -- justified by staying usable before anything
// is initialised. The table keeps that property, because it builds itself on
// the first call rather than needing an init hook: the cost is one 1 KB static
// array and a first call that does the work the old loop did for 256 bytes.
//
// It matters because every block read off the medium is checksummed in full.
// After the cursor stopped re-reading resident blocks this became the largest
// remaining cost in the read path, which is the usual shape of these things --
// removing the top item promotes whatever was underneath it.
static uint32_t crc32c_tab[256];
static int crc32c_tab_ready;
static void
crc32c_build(void)
{
for (unsigned i = 0; i < 256; i++) {
uint32_t c = i;
for (int k = 0; k < 8; k++)
c = (c >> 1) ^ (0x82F63B78u & (uint32_t)(-(int32_t)(c & 1)));
crc32c_tab[i] = c;
}
crc32c_tab_ready = 1;
}
uint32_t
schfs_crc32c(uint32_t crc, const void *buf, unsigned len)
{
const uint8_t *p = (const uint8_t *)buf;
if (!crc32c_tab_ready)
crc32c_build();
crc = ~crc;
while (len--) {
crc ^= *p++;
for (int i = 0; i < 8; i++)
crc = (crc >> 1) ^ (0x82F63B78u & (uint32_t)(-(int32_t)(crc & 1)));
}
while (len--)
crc = (crc >> 8) ^ crc32c_tab[(crc ^ *p++) & 0xFF];
return ~crc;
}
@ -260,18 +288,64 @@ schfs_btree_create(struct schfs_btree *t, struct schfs_bdev *dev,
void
schfs_cursor_init(struct schfs_cursor *c, struct schfs_btree *t, void *blk)
{
c->t = t;
c->blk = blk;
c->depth = 0;
c->valid = 0;
c->t = t;
c->blk = blk;
c->depth = 0;
c->valid = 0;
// Whatever the previous user of this buffer left in it is not ours to
// trust: a mutation between two cursors could have rewritten that block in
// place, and the copy here would be a generation behind.
c->resident = 0;
c->epoch = 0;
}
// Bumped by schfs_cache_invalidate(), which every write entry point calls
// before it touches anything. A cursor's resident block is only trusted while
// this has not moved: within one transaction group the tree rewrites its own
// nodes *in place*, so a cached copy of block N can be a generation behind
// while still carrying block N's own number in its header. Nothing else
// distinguishes the two, which is why identity checks alone are not enough.
uint64_t schfs_tree_epoch = 1;
// Load path[d]'s block into the cursor buffer, verifying it against the
// checksum recorded alongside it.
//
// The early return is the whole reason this system was slow.
//
// schfs_cursor_val calls this on *every* access to the current record, and
// iter_item calls schfs_cursor_val once per directory entry -- deliberately, so
// that a callback which disturbs the shared buffer cannot corrupt the walk. But
// node_read is not cheap: it copies 4096 bytes out of the buffer cache and then
// runs CRC32c over all 4096. Listing a 243-entry directory did that once per
// entry, and `ls /bin` moved several megabytes through both. Measured across
// one session: 398,999 block reads against 445 actual disk misses, so ~99.9% of
// them were re-reading a block already sitting in the buffer.
//
// Two conditions, and both are needed. `resident` says this cursor put that
// block here; the header says the page still holds it, which is what catches a
// second cursor sharing the same page having loaded something else in between.
// The header alone would be enough for identity, but a block cached from before
// an in-place rewrite has the same blocknr -- so the per-cursor field, cleared
// at init, is what bounds the trust to one entry point's lifetime.
static int
cur_load(struct schfs_cursor *c, int d)
{
return node_read(c->t->dev, c->path[d].blocknr, c->path[d].crc, c->blk);
uint64_t want = c->path[d].blocknr;
if (c->resident == want && c->epoch == schfs_tree_epoch) {
const struct schfs_node_hdr *h = HDR(c->blk);
if (h->magic == SCHFS_NODE_MAGIC && h->blocknr == want &&
(c->path[d].crc == 0 || c->path[d].crc == h->crc))
return 0;
}
if (node_read(c->t->dev, want, c->path[d].crc, c->blk) != 0) {
c->resident = 0;
return -2;
}
c->resident = want;
c->epoch = schfs_tree_epoch;
return 0;
}
// From the node currently in the buffer at depth `d`, walk down to a leaf,