schfs: the commit barrier, reflink, and attributes with an index over them

The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.

Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.

Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.

Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.

Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:

  - statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
    so every field after it landed one slot late: userland read f_frsize
    out of f_namelen, got 255, and df reported a 32 MB root filesystem
    as 1992 KB. It hid for as long as it did because the fields used to
    be invented constants, so wrong numbers looked like the usual ones.
  - struct inode's nlink was a short against SchFS's uint32, so a file
    past 32767 names came back negative -- and iput frees an object
    whose count reaches zero. It is an int now, signed on purpose, so an
    underflow lands somewhere a check can see rather than wrapping.
  - filewrite split every write to fit v1's log. That limit belongs to
    the journal, not to writing, and SchFS makes one transaction of
    whatever it is handed: a 1 MB write was becoming 170 commits and
    170 flushes for nothing. fs_ops carries max_write now, and SchFS
    sets no limit.

Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.

Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Nexoniarz 2026-08-15 16:01:22 +02:00
commit 03cc316001
39 changed files with 3942 additions and 550 deletions

View file

@ -25,6 +25,10 @@
#include "sleeplock.h"
#include "fs.h"
#include "buf.h"
#include "x86.h"
#include "fsstat.h"
struct fsstat fsstat;
struct {
struct spinlock lock;
@ -59,8 +63,20 @@ binit(void)
// is device-agnostic; everything below picks a backend. RAMDISK_DEV is served
// out of memory (kernel/dev/ramdisk.c), USBDISK_DEV by bulk transfers to a
// USB stick, everything else by the IDE driver.
static void bdevrw_inner(struct buf *b);
static void
bdevrw(struct buf *b)
{
uint64 t0 = rdtsc();
if(b->flags & B_DIRTY) fsstat.disk_writes++; else fsstat.disk_reads++;
bdevrw_inner(b);
fsstat.disk_cycles += rdtsc() - t0;
}
static void
bdevrw_inner(struct buf *b)
{
if(b->dev == RAMDISK_DEV){
ramdiskrw(b);
@ -73,6 +89,19 @@ bdevrw(struct buf *b)
iderw(b);
}
// Tell the device to commit whatever it has accepted but not yet made durable.
// Same dispatch as bdevrw, and the same rule: everything above this line is
// device-agnostic.
static int
bdevflush(uint dev)
{
if(dev == RAMDISK_DEV)
return 0; // memory has no volatile stage below it
if(dev == USBDISK_DEV)
return usb_storage_flush();
return ideflush(dev);
}
// Look through buffer cache for block on device dev.
// If not found, allocate a buffer.
// In either case, return locked buffer.
@ -81,6 +110,7 @@ bget(uint dev, uint64 blockno)
{
struct buf *b;
retry:
acquire(&bcache.lock);
// Is the block already cached?
@ -107,6 +137,47 @@ bget(uint dev, uint64 blockno)
return b;
}
}
// Every buffer is dirty and none is clean enough to take.
//
// This used to be a straight panic, which was right while every writer went
// through bwrite and a dirty buffer therefore meant "the log has not
// committed yet". SchFS marks tree blocks dirty and leaves them there (see
// bdirty), so an all-dirty cache is now an ordinary state: a transaction that
// touches more than NBUF blocks reaches it long before it commits.
//
// Writing one out early is always safe. The only ordering the format needs is
// that every block of the new tree precede the uberblock that names it, and
// the uberblock goes down after an explicit bflush -- so nothing here can
// land on the wrong side of that barrier.
for(b = bcache.head.prev; b != &bcache.head; b = b->prev){
if(b->refcnt == 0){
fsstat.bget_evict_dirty++;
b->refcnt++; // claim it, so nobody else recycles it either
release(&bcache.lock);
acquiresleep(&b->lock);
if(b->flags & B_DIRTY)
bdevrw(b);
// Between dropping bcache.lock and taking b's, another process may have
// found this buffer under its *old* identity and be waiting on it. It is
// clean now either way, so hand it back rather than renaming a block out
// from under them, and look for another victim.
acquire(&bcache.lock);
if(b->refcnt != 1){
b->refcnt--;
release(&bcache.lock);
releasesleep(&b->lock);
goto retry;
}
b->dev = dev;
b->blockno = blockno;
b->flags = 0;
release(&bcache.lock);
return b; // b->lock still held, refcnt 1
}
}
panic("bget: no buffers");
}
@ -116,8 +187,10 @@ bread(uint dev, uint64 blockno)
{
struct buf *b;
fsstat.bread_calls++;
b = bget(dev, blockno);
if(!(b->flags & B_VALID)) {
fsstat.bread_miss++;
bdevrw(b);
}
return b;
@ -133,6 +206,60 @@ bwrite(struct buf *b)
bdevrw(b);
}
// Mark b modified but do not write it. The buffer stays dirty in the cache
// until something evicts it or bflush() comes past.
//
// This is what makes a copy-on-write filesystem affordable. SchFS's B-tree
// seals and writes a node on every single record insert, so a hot leaf -- the
// one collecting a file's data checksums, say -- was written to the disk once
// per 4 KB of file. Measured over 1 MB: 1288 block writes for 256 blocks of
// data. Deferring collapses every one of those repeats into the last one.
//
// It is only safe because the filesystem asks for the ordering it needs
// explicitly, at commit, instead of assuming that write order is medium order.
// A filesystem that did assume that would be broken here -- and was already
// broken on any drive with a write cache, which is all of them.
void
bdirty(struct buf *b)
{
if(!holdingsleep(&b->lock))
panic("bdirty");
b->flags |= B_DIRTY;
b->flags |= B_VALID;
}
// Push every dirty buffer for `dev` out, then make the device itself commit
// what it has taken. Returns 0, or -1 if the device could not confirm it.
//
// Walks the array rather than the MRU list: brelse relinks buffers as it goes,
// and the array is the one order that cannot change underneath the walk.
int
bflush(uint dev)
{
struct buf *b;
for(b = bcache.buf; b < bcache.buf+NBUF; b++){
acquire(&bcache.lock);
if(b->dev != dev || !(b->flags & B_DIRTY)){
release(&bcache.lock);
continue;
}
b->refcnt++;
release(&bcache.lock);
acquiresleep(&b->lock);
if(b->flags & B_DIRTY) // recheck: an eviction may have beaten us to it
bdevrw(b);
releasesleep(&b->lock);
acquire(&bcache.lock);
b->refcnt--;
release(&bcache.lock);
}
return bdevflush(dev);
}
// Release a locked buffer.
// Move to the head of the MRU list.
void