schfs: the commit barrier, reflink, and attributes with an index over them

The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.

Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.

Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.

Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.

Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:

  - statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
    so every field after it landed one slot late: userland read f_frsize
    out of f_namelen, got 255, and df reported a 32 MB root filesystem
    as 1992 KB. It hid for as long as it did because the fields used to
    be invented constants, so wrong numbers looked like the usual ones.
  - struct inode's nlink was a short against SchFS's uint32, so a file
    past 32767 names came back negative -- and iput frees an object
    whose count reaches zero. It is an int now, signed on purpose, so an
    underflow lands somewhere a check can see rather than wrapping.
  - filewrite split every write to fit v1's log. That limit belongs to
    the journal, not to writing, and SchFS makes one transaction of
    whatever it is handed: a 1 MB write was becoming 170 commits and
    170 flushes for nothing. fs_ops carries max_write now, and SchFS
    sets no limit.

Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.

Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

include/schfs.h

@@ -40,6 +40,7 @@
 #define SCHFS_KEY_INDEX   5   // (idxid, INDEX,  valuehash) -> objectid
 #define SCHFS_KEY_CSUM    6   // (CSUM,  CSUM,   diskblk)   -> uint32 CRC32c
 #define SCHFS_KEY_SNAP    7   // (SNAP,  SNAP,    id)        -> struct schfs_snap
+#define SCHFS_KEY_EXTREF  8   // (EXTREF,EXTREF, diskblk)   -> struct schfs_extref
 
 // Data-block checksums live under one reserved object id, exactly as btrfs
 // keeps a checksum tree. Tree nodes carry their checksum in their parent
@@ -52,6 +53,16 @@
 // B-tree is copy-on-write, so the blocks a root reaches are already immutable
 // as long as nothing frees them. Keeping one costs a record.
 #define SCHFS_SNAP_OBJECTID  2ULL
+// Shared extents, keyed by the disk block a run starts at. btrfs calls this the
+// extent tree and so, in effect, is this -- one reserved object id whose records
+// say how many files point at a run.
+//
+// Only *shared* runs have a record. A run nothing has cloned has no entry at
+// all, and absence means a reference count of one. That is not a space
+// optimisation so much as a correctness one: the overwhelmingly common case
+// stays exactly as it was, so the write path pays nothing for a feature it is
+// not using, and there is no second structure to keep in step on every write.
+#define SCHFS_EXTREF_OBJECTID 3ULL
 
 // Object ids below this are reserved; the root directory is the first real one.
 #define SCHFS_FIRST_OBJECTID 16ULL
@@ -225,10 +236,25 @@ struct schfs_extent {
     // mistake this field exists to prevent. Only ever trusted together with
     // refs == 1; a shared run belongs to somebody else as well.
     uint64_t gen;
+    // Whether this run is shared, as far as *this* file knows. A hint, not the
+    // truth: the count that decides whether blocks may be released lives in the
+    // extent-reference tree (SCHFS_KEY_EXTREF), because a per-file copy cannot
+    // be decremented by the other file that shares the run.
+    //
+    // Kept because it makes the common case free: refs == 1 here means nothing
+    // has ever cloned this run, so the write path can rewrite it in place
+    // without consulting the tree at all.
     uint32_t refs;
     uint32_t pad;
 };
 
+// How many files point at a run of blocks. Present only for runs with more than
+// one reference; see SCHFS_EXTREF_OBJECTID.
+struct schfs_extref {
+    uint64_t nblocks;
+    uint64_t refs;
+};
+
 // Directory entry payload. The key's offset field holds the name hash, so a
 // lookup is one tree descent rather than a directory scan.
 //
@@ -248,6 +274,71 @@ struct schfs_dirent_rec {
 
 #define SCHFS_NAME_MAX 255
 
+// ---------------------------------------------------------------------------
+// Attributes, and the index over them
+//
+// An attribute is a named value on an object -- the same thing Linux calls an
+// extended attribute, and the thing BeFS built its query interface on. Stored
+// under (ino, ATTR, namehash), packed exactly as directory entries are, and for
+// the same reason: two names can hash alike, and a filesystem that loses one of
+// them under a collision is not one to build on.
+//
+// The point of having them is the *index*. An attribute may be indexed, and an
+// indexed attribute gets a second set of records keyed by its value rather than
+// by the object -- so "every object whose `type` is `photo`" is a range scan of
+// a few keys rather than a walk of every object in the filesystem. That is the
+// whole of what "a filesystem you can query" means here, and the reason the
+// design is one B-tree with many key types: the feature is a few hundred lines
+// on machinery that already exists, rather than a second engine.
+struct schfs_attr_rec {
+    uint32_t namelen;
+    uint32_t valuelen;
+    uint16_t type;              // SCHFS_ATTR_* below; what a query compares as
+    uint16_t flags;
+    uint32_t pad;
+    /* name bytes, then value bytes; the pair padded to an 8-byte boundary */
+};
+
+// What a value *is*, so that a query can compare two of them meaningfully. A
+// string and an integer that happen to share bytes are not the same value, and
+// ordering differs between them.
+#define SCHFS_ATTR_RAW    0     // opaque bytes; only equality is meaningful
+#define SCHFS_ATTR_STRING 1
+#define SCHFS_ATTR_INT    2     // int64, little-endian
+#define SCHFS_ATTR_TIME   3     // seconds since the epoch, int64
+
+#define SCHFS_ATTR_NAME_MAX  63
+#define SCHFS_ATTR_VALUE_MAX 255
+
+// Step from one packed attribute record to the next. 8-byte aligned so the
+// uint32s of the next one are naturally aligned.
+#define SCHFS_ATTR_LEN(namelen, valuelen) \
+    ((unsigned)((sizeof(struct schfs_attr_rec) + (namelen) + (valuelen) + 7) & ~7u))
+
+// An index entry: one object that carries a given value for a given attribute.
+//
+// Keyed (indexid, INDEX, valuehash), where indexid is the hash of the attribute
+// *name*. Several objects can share a value, so the payload is a packed list of
+// these, exactly as dirents and attributes are.
+struct schfs_index_rec {
+    uint64_t objectid;
+    uint32_t valuelen;
+    uint32_t pad;
+    /* value bytes follow, so a query can reject a hash collision */
+};
+
+#define SCHFS_INDEX_LEN(valuelen) \
+    ((unsigned)((sizeof(struct schfs_index_rec) + (valuelen) + 7) & ~7u))
+
+// Which attributes are indexed. One record, under the index object id, whose
+// value is a packed list of attribute names.
+//
+// A list rather than a flag on each attribute, because the question asked at
+// every setxattr is "is *this name* indexed", and answering it from one small
+// record is a single cached lookup rather than a search.
+#define SCHFS_INDEX_OBJECTID 4ULL
+#define SCHFS_INDEX_LIST_OFF 0ULL   /* the one record's offset in the key */
+
 // Step from one packed dirent to the next. Records are 8-byte aligned so the
 // uint64 objectid of the next one is naturally aligned.
 #define SCHFS_DIRENT_LEN(namelen) \
@@ -274,5 +365,8 @@ SCHFS_ASSERT_SIZE(schfs_uberblock,   SCHFS_BLKSIZE);
 SCHFS_ASSERT_SIZE(schfs_inode,       80);
 SCHFS_ASSERT_SIZE(schfs_extent,      32);
 SCHFS_ASSERT_SIZE(schfs_dirent_rec,  16);
+SCHFS_ASSERT_SIZE(schfs_extref,      16);
+SCHFS_ASSERT_SIZE(schfs_attr_rec,    16);
+SCHFS_ASSERT_SIZE(schfs_index_rec,   16);
 
 #endif /* SCHFS_H */