schfs: the commit barrier, reflink, and attributes with an index over them

The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.

Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.

Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.

Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.

Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:

  - statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
    so every field after it landed one slot late: userland read f_frsize
    out of f_namelen, got 255, and df reported a 32 MB root filesystem
    as 1992 KB. It hid for as long as it did because the fields used to
    be invented constants, so wrong numbers looked like the usual ones.
  - struct inode's nlink was a short against SchFS's uint32, so a file
    past 32767 names came back negative -- and iput frees an object
    whose count reaches zero. It is an int now, signed on purpose, so an
    underflow lands somewhere a check can see rather than wrapping.
  - filewrite split every write to fit v1's log. That limit belongs to
    the journal, not to writing, and SchFS makes one transaction of
    whatever it is handed: a 1 MB write was becoming 170 commits and
    170 flushes for nothing. fs_ops carries max_write now, and SchFS
    sets no limit.

Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.

Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Nexoniarz 2026-08-15 16:01:22 +02:00
commit 03cc316001
39 changed files with 3942 additions and 550 deletions

View file

@ -75,6 +75,11 @@ struct schfs_vol {
int uber_slot; // ring slot the last commit used
int readonly;
// How many newer uberblocks schfs_vol_open had to reject before it found a
// root it could actually read. Zero on every healthy mount. Anything else
// means transactions were lost, which is a recovery the operator has to be
// told about rather than something to absorb quietly.
int recovered;
char label[SCHFS_LABEL_MAX];
uint8_t uuid[16];
@ -82,6 +87,14 @@ struct schfs_vol {
// ctx buffers: a split reuses ctx.b and ctx.c while the value being
// inserted is still being read from, so the value needs storage of its own.
char dirbuf[SCHFS_MAX_INLINE];
// Which attributes are indexed, as a packed list of (length, name), read
// off the disk at mount. Cached here rather than looked up because every
// attribute write asks the question and the answer is almost always no --
// a tree descent to say so would make every write pay for a feature it is
// not using. See kernel/fs/schfs/attr.c.
char idxnames[512];
unsigned idxnames_len;
};
// ---------------------------------------------------------------------------
@ -141,6 +154,12 @@ void schfs_vol_free(struct schfs_vol *v, uint64_t blk);
int schfs_space_init(struct schfs_space *s, uint64_t nblocks,
void *(*palloc)(void));
// Return an already-sized map to "empty volume", reusing its pages. Mounting
// may try several uberblocks, and each candidate root needs its scan to start
// from a clean map -- but the pages were taken once and must not be taken
// again, which is the one thing schfs_space_init cannot do twice.
void schfs_space_reset(struct schfs_space *s, uint64_t nblocks);
// Record a run as allocated. Idempotent, because an extent range may be shared
// by several files.
void schfs_space_mark(struct schfs_space *s, uint64_t blk, uint64_t n);
@ -187,6 +206,15 @@ int schfs_dir_lookup(struct schfs_vol *v, struct schfs_cursor *c,
// Iterate a directory's real entries in hash order. "." and ".." are not
// stored and are not reported; callers that need them synthesise them.
// A non-zero return from fn stops the walk and is passed back.
// A directory entry, with the tree key's hash it was found under. The hash is
// what a partial listing resumes from -- see schfs_dir_iterate_from.
typedef int (*schfs_dir_fn)(uint64_t ino, uint16_t type, const char *name,
unsigned namelen, uint64_t hash, void *arg);
int schfs_dir_iterate_from(struct schfs_vol *v, struct schfs_cursor *c,
uint64_t dir, uint64_t start_hash,
schfs_dir_fn fn, void *arg);
int schfs_dir_iterate(struct schfs_vol *v, struct schfs_cursor *c, uint64_t dir,
int (*fn)(uint64_t ino, uint16_t type, const char *name,
unsigned namelen, void *arg),
@ -244,6 +272,49 @@ int schfs_dir_empty(struct schfs_vol *v, uint64_t dir);
// schfs_file_write keeps the extent count down.
int schfs_file_reserve(struct schfs_vol *v, uint64_t ino, uint64_t nblocks);
// Share `src`'s data blocks with `dst` without copying them. Both remain
// ordinary writable files; the first write to either copies only the run it
// touches. Costs one metadata record per run, whatever the file's size.
int schfs_file_reflink(struct schfs_vol *v, uint64_t src, uint64_t dst);
// ---------------------------------------------------------------------------
// attr.c -- attributes, the index over them, and the query that uses it
// ---------------------------------------------------------------------------
// One attribute. Returns 1 and fills the buffers if present, 0 if absent, -3 if
// `out` is too small (the length is still reported), -2 on corruption.
int schfs_attr_get(struct schfs_vol *v, uint64_t ino, const char *name,
unsigned namelen, void *out, unsigned cap,
unsigned *valuelen, uint16_t *type);
// Every attribute on an object. The callback returns non-zero to stop.
int schfs_attr_iterate(struct schfs_vol *v, struct schfs_cursor *c, uint64_t ino,
int (*fn)(const char *name, unsigned namelen,
const void *val, unsigned vallen,
uint16_t type, void *arg),
void *arg);
// Set or replace an attribute, keeping any index in step.
int schfs_attr_set(struct schfs_vol *v, uint64_t ino, const char *name,
unsigned namelen, const void *val, unsigned vallen,
uint16_t type);
// Remove one. Absent is not a failure.
int schfs_attr_del(struct schfs_vol *v, uint64_t ino, const char *name,
unsigned namelen);
// Start indexing an attribute. Affects values written after this only.
int schfs_index_create(struct schfs_vol *v, const char *name, unsigned namelen);
// Load the indexed-attribute list from the tree into the volume. Called at
// mount, and after the list changes.
int schfs_index_reload(struct schfs_vol *v);
// Every object whose `name` attribute equals `val`. One tree descent, whatever
// the size of the filesystem. Returns how many ids were written, or negative.
int schfs_query_equal(struct schfs_vol *v, const char *name, unsigned namelen,
const void *val, unsigned vallen, uint64_t *out, int cap);
// Write at an arbitrary offset, allocating blocks, extent records and data
// checksums, copying any block that a committed tree can still reach, and
// splitting the extent record around it. Writing past the end extends the file;