nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

schfs: the commit barrier, reflink, and attributes with an index over them

Browse source 
The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.

Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.

Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.

Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.

Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:

  - statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
    so every field after it landed one slot late: userland read f_frsize
    out of f_namelen, got 255, and df reported a 32 MB root filesystem
    as 1992 KB. It hid for as long as it did because the fields used to
    be invented constants, so wrong numbers looked like the usual ones.
  - struct inode's nlink was a short against SchFS's uint32, so a file
    past 32767 names came back negative -- and iput frees an object
    whose count reaches zero. It is an int now, signed on purpose, so an
    underflow lands somewhere a check can see rather than wrapping.
  - filewrite split every write to fit v1's log. That limit belongs to
    the journal, not to writing, and SchFS makes one transaction of
    whatever it is handed: a 1 MB write was becoming 170 commits and
    170 flushes for nothing. fs_ops carries max_write now, and SchFS
    sets no limit.

Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.

Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Nexoniarz 2026-08-15 16:01:22 +02:00
commit 03cc316001
39 changed files with 3942 additions and 550 deletions
Download patch file Download diff file Expand all files Collapse all files

335
kernel/sys/sys_fs.c
View file

@ -14,6 +14,7 @@
#include "syscall.h"
#include "uaccess.h"
#include "errno.h"
#include "schfs.h" /* SCHFS_BLKSIZE, SCHFS_NAME_MAX — statfs reports them */
// -----------------------------------------------------------------------
// Linux-ABI-compatible struct stat (x86_64 "struct stat" from
@ -965,6 +966,44 @@ addr_t sys_ioctl(void)
argaddr(1, &request); argaddr(2, &argp);
int is_dev = (f->type == FD_INODE && f->ip && f->ip->type == 3);
// FICLONE: make this file share another's data blocks. Linux's spelling,
// because `cp --reflink` and every other tool that knows about cloning
// already asks for it by this number -- inventing our own would mean
// nothing could use the feature without being patched.
//
// The argument is a *file descriptor*, not a pointer, which is why it is
// read before the argp-shaped handling below.
if (request == FICLONE) {
struct file *sf;
struct proc *pr = myproc();
int sfd = (int)argp;
if (f->type != FD_INODE || !f->ip || !f->writable)
return -EBADF;
if (sfd < 0 || sfd >= NOFILE || (sf = pr->ofile[sfd]) == 0)
return -EBADF;
if (sf->type != FD_INODE || !sf->ip || !sf->readable)
return -EBADF;
// Both sides must be on the same filesystem: a clone shares blocks, and
// blocks belong to one volume.
if (sf->ip->dev != f->ip->dev)
return -EXDEV;
if (!SCHFS_IS_VOL_DEV(f->ip->dev))
return -EOPNOTSUPP;
if (f->ip->type != T_FILE || sf->ip->type != T_FILE)
return -EINVAL;
int r = schfs_kreflink(f->ip->dev, sf->ip->inum, f->ip->inum);
if (r != 0)
return -EIO;
// The destination inode's size changed underneath the VFS's copy.
ilock(f->ip);
f->ip->size = sf->ip->size;
iunlock(f->ip);
return 0;
}
// /dev/fb0 (major 2) — geometry queries. This has to come before the
// catch-all below, which answers every non-tty ioctl with 0: a program
// asking FBIOGET_VSCREENINFO would have been told "success" and then
@ -1055,9 +1094,36 @@ addr_t sys_rmdir(void)
}
// fsync / fdatasync / sync — stubs (log-based FS commits on every op).
addr_t sys_fsync(void) { return 0; }
addr_t sys_fdatasync(void){ return 0; }
addr_t sys_sync(void) { return 0; }
// sync / fsync / fdatasync.
//
// These returned 0 without doing anything, which was defensible while every
// write reached the disk before its syscall returned. It is not defensible now:
// tree blocks are marked dirty in the buffer cache and pushed out at commit, so
// "flush what is outstanding" finally names a real action. An empty transaction
// drives the same barrier a real one does -- the dirty buffers go out and the
// device is told to commit them.
addr_t sys_sync(void)
{
schfs_ksync_all();
return 0;
}
addr_t sys_fsync(void)
{
struct file *f;
if (argfd(0, 0, &f) < 0) return -EBADF;
// Whole-volume rather than per-file: SchFS has no way to make one object
// durable on its own. A transaction group is the unit that commits, and
// this file's last write is in whichever group is open.
if (f->type == FD_INODE && f->ip && SCHFS_IS_VOL_DEV(f->ip->dev))
return schfs_ksync(f->ip->dev) == 0 ? 0 : -EIO;
return 0;
}
// No separate data-only path: the inode record and the data it describes are
// records in the same tree and commit together. Reporting success for a weaker
// guarantee than was asked for would be the wrong way round; this is stronger.
addr_t sys_fdatasync(void){ return sys_fsync(); }
// pread: read from fd at given offset without changing file position.
addr_t sys_pread(void)
@ -1106,16 +1172,43 @@ addr_t sys_pwritev2(void){ return sys_writev(); }
// fallocate: preallocate space — we don't track this; stub success.
addr_t sys_fallocate(void) { return 0; }
// statfs / fstatfs: return fake filesystem stats.
static void fill_statfs(struct statfs *sf) {
// statfs / fstatfs.
//
// These used to return the same six invented constants whatever they were
// asked about -- 150000 blocks, half of them free -- so df reported a
// filesystem that did not exist and reported it identically for every mount.
// The numbers are real now, and which ones you get depends on which filesystem
// the path landed on.
static void fill_statfs(struct statfs *sf, uint dev) {
uint64 total = 0, freeb = 0, objects = 0;
memset(sf, 0, sizeof(*sf));
sf->f_type = 0x534b4658; // 'SKFX'
sf->f_bsize = 512;
sf->f_blocks = 150000;
sf->f_bfree = 75000;
sf->f_bavail = 75000;
if (schfs_kstatfs(dev, &total, &freeb, &objects) == 0) {
sf->f_bsize = SCHFS_BLKSIZE;
sf->f_frsize = SCHFS_BLKSIZE;
sf->f_blocks = (int64)total;
sf->f_bfree = (int64)freeb;
sf->f_bavail = (int64)freeb; // no reserve, so nothing held back
// Both zero, deliberately. There is no inode table to run out of and
// the object id space is 2^64, so there is no honest pair of numbers
// here -- and df computes a "% inodes used" from them, which against a
// real f_files and a zero f_ffree would show every SchFS mount as
// completely full. Zeroes are what filesystems that allocate inodes
// dynamically report, and what df is written to expect from them.
sf->f_files = 0;
sf->f_ffree = 0;
sf->f_namelen = SCHFS_NAME_MAX;
(void)objects;
return;
}
// v1, /proc, or anything else: the old shape, but at least with a block
// size that matches the one the buffer cache actually uses.
sf->f_bsize = BSIZE;
sf->f_frsize = BSIZE;
sf->f_namelen = DIRSIZ;
sf->f_frsize = 512;
}
addr_t sys_statfs(void)
@ -1125,7 +1218,17 @@ addr_t sys_statfs(void)
if (argstr(0, &path) < 0 || argaddr(1, &bufp) < 0) return -1;
struct statfs *sf = (struct statfs*)user_ptr(bufp, sizeof(*sf));
if (!sf) return -EFAULT;
fill_statfs(sf);
// Resolve the path: which filesystem it lands on is the whole question,
// and mount crossing happens during resolution.
begin_op();
struct inode *ip = namei(path);
if (!ip) { end_op(); return -ENOENT; }
uint dev = ip->dev;
iput(ip);
end_op();
fill_statfs(sf, dev);
return 0;
}
@ -1136,7 +1239,8 @@ addr_t sys_fstatfs(void)
if (argfd(0, 0, &f) < 0 || argaddr(1, &bufp) < 0) return -1;
struct statfs *sf = (struct statfs*)user_ptr(bufp, sizeof(*sf));
if (!sf) return -EFAULT;
fill_statfs(sf);
if (f->type != FD_INODE || !f->ip) return -EBADF;
fill_statfs(sf, f->ip->dev);
return 0;
}
@ -1404,9 +1508,8 @@ addr_t sys_mount(void)
return 0;
}
// Only SchFS, and only the one volume the kernel found at boot. Checking
// the name still matters: a caller that asked for something else should be
// told no rather than quietly handed this.
// Only SchFS. Checking the name still matters: a caller that asked for
// something else should be told no rather than quietly handed this.
if (strncmp(fstype, "schfs", 6) != 0)
return -ENODEV;
@ -1429,7 +1532,11 @@ addr_t sys_mount(void)
// Distinct codes, because "resource busy" for every possible failure
// tells the caller nothing and cost an hour of guessing once already.
int r = schfs_ksnap_mount(nm, nlen);
//
// Snapshots are taken of the root volume: a snapshot is named inside
// the volume that holds it, and there is no syntax yet for saying which
// volume that is.
int r = schfs_ksnap_mount(SCHFS_DEV, nm, nlen);
if (r < 0) {
switch (r) {
case -1: return -ENODEV; /* no volume mounted to look in */
@ -1444,7 +1551,41 @@ addr_t sys_mount(void)
if (!schfs_kmounted())
return -ENODEV;
uint64 root = snapmount ? schfs_ksnap_root(snapdev) : schfs_kroot();
// The source finally names a disk.
//
// It was ignored outright before -- there was one volume and nothing to
// choose between -- so `mount -t schfs /dev/hdb /mnt` mounted whatever the
// kernel had probed at boot and said nothing about it. Now an explicit
// source is resolved, and mounting a second SchFS disk works.
//
// An empty source, or the bare word "schfs", still means the root volume:
// that is what /etc/fstab and every existing script pass.
uint fsdev_src = SCHFS_DEV;
if (!snapmount && source && source[0] && strncmp(source, "schfs", 6) != 0) {
begin_op();
struct inode *sp = namei(source);
if (!sp) { end_op(); return -ENOENT; }
ilock(sp);
// A raw disk is a character device in the devsw table (major 8, see
// kernel/dev/cdev.c) whose minor is the IDE device number.
if (sp->type != T_DEV || sp->major != 8) {
iunlockput(sp); end_op();
return -ENOTBLK;
}
uint bdev = (uint)sp->minor;
iunlockput(sp);
end_op();
int r = schfs_kmount_dev(bdev);
if (r < 0) {
// -EBUSY also covers "already mounted", which is the common typo
// and deserves to be distinguishable from "no free slot" one day.
return r;
}
fsdev_src = (uint)r;
}
uint64 root = snapmount ? schfs_ksnap_root(snapdev) : schfs_kroot(fsdev_src);
if (root == 0) {
if (snapmount)
schfs_ksnap_unmount(snapdev);
@ -1469,12 +1610,14 @@ addr_t sys_mount(void)
// the only volume there could be. A snapshot is a second device, and its
// mount point is necessarily on the live root -- there is nowhere else to
// put it -- so the old spelling refused every snapshot mount.
uint fsdev = snapmount ? snapdev : SCHFS_DEV;
uint fsdev = snapmount ? snapdev : fsdev_src;
if (dev == fsdev)
return -EBUSY;
if (mount_add(dev, inum, fsdev, root) != 0) {
if (mount_add(dev, inum, fsdev, root, target, source, "schfs") != 0) {
if (snapmount)
schfs_ksnap_unmount(snapdev);
else if (fsdev_src != SCHFS_DEV)
schfs_kumount(fsdev_src); /* undo the volume mount we just did */
return -EBUSY;
}
return 0;
@ -1499,10 +1642,158 @@ addr_t sys_umount2(void)
if (mount_del_by_root(dev, inum) != 0)
return -EINVAL;
// Release the snapshot slot with the mount. Without this the table entry
// went away and the slot stayed taken, so a volume could be unmounted and
// never mounted again until reboot.
// Release the slot with the mount. Without this the table entry went away
// and the slot stayed taken, so a volume could be unmounted and never
// mounted again until reboot.
if (SCHFS_IS_SNAP_DEV(dev))
schfs_ksnap_unmount(dev);
else if (SCHFS_IS_VOL_DEV(dev) && dev != SCHFS_DEV)
// Pushes the volume's dirty buffers out and drives a cache flush before
// letting go. The root volume (SCHFS_DEV) refuses -- there is nothing
// to fall back to -- and mount_del_by_root has already failed for it,
// since nothing mounts the root over anything.
schfs_kumount(dev);
return 0;
}
// -----------------------------------------------------------------------
// Attributes, and the index over them (SchFS phase 5)
//
// setxattr/getxattr/removexattr keep Linux's shape, so a program that already
// knows how to attach metadata to a file needs no changes. schfs_query is the
// one call with no Linux equivalent: it asks the *filesystem* a question
// instead of walking it, which is the whole point of building the index.
// -----------------------------------------------------------------------
// Resolve a path to a SchFS inode, and refuse anything that is not one.
// Attributes live in the tree, so there is nowhere to put them on v1 or /proc.
static int
xattr_inode(char *path, struct inode **ip)
{
begin_op();
struct inode *p = namei(path);
if (!p) { end_op(); return -ENOENT; }
if (!SCHFS_IS_VOL_DEV(p->dev)) { iput(p); end_op(); return -EOPNOTSUPP; }
end_op();
*ip = p;
return 0;
}
addr_t sys_setxattr(void)
{
char *path, *name;
addr_t uval;
int size, flags;
if (argstr(0, &path) < 0 || argstr(1, &name) < 0) return -EFAULT;
if (argaddr(2, &uval) < 0 || argint(3, &size) < 0) return -EFAULT;
argint(4, &flags); /* XATTR_CREATE/REPLACE: unused */
if (size < 0 || size > SCHFS_ATTR_VALUE_MAX) return -E2BIG;
int namelen = strlen(name);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
char val[SCHFS_ATTR_VALUE_MAX];
if (size > 0 && copy_from_user(val, uval, (uint64)size) < 0) return -EFAULT;
struct inode *ip;
int r = xattr_inode(path, &ip);
if (r < 0) return r;
// Stored as a string: nothing here knows the caller's intent, and a string
// is the type whose equality comparison is the one a query wants.
r = schfs_kattr_set(ip->dev, ip->inum, name, (unsigned)namelen,
val, (unsigned)size, SCHFS_ATTR_STRING);
iput(ip);
return r == 0 ? 0 : -EIO;
}
addr_t sys_getxattr(void)
{
char *path, *name;
addr_t uval;
int size;
if (argstr(0, &path) < 0 || argstr(1, &name) < 0) return -EFAULT;
if (argaddr(2, &uval) < 0 || argint(3, &size) < 0) return -EFAULT;
int namelen = strlen(name);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
struct inode *ip;
int r = xattr_inode(path, &ip);
if (r < 0) return r;
char val[SCHFS_ATTR_VALUE_MAX];
unsigned vallen = 0;
// A zero size means "how big is it?", which is how every caller sizes its
// buffer before asking again.
r = schfs_kattr_get(ip->dev, ip->inum, name, (unsigned)namelen,
size > 0 ? val : 0, (unsigned)(size > 0 ? size : 0),
&vallen);
iput(ip);
if (r == 0) return -ENODATA;
if (r == -3) return -ERANGE;
if (r != 1) return -EIO;
if (size == 0) return (addr_t)vallen;
if (copy_to_user(uval, val, vallen) < 0) return -EFAULT;
return (addr_t)vallen;
}
addr_t sys_removexattr(void)
{
char *path, *name;
if (argstr(0, &path) < 0 || argstr(1, &name) < 0) return -EFAULT;
int namelen = strlen(name);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
struct inode *ip;
int r = xattr_inode(path, &ip);
if (r < 0) return r;
r = schfs_kattr_del(ip->dev, ip->inum, name, (unsigned)namelen);
iput(ip);
return r == 0 ? 0 : -EIO;
}
// Declare an index on an attribute name. Covers values written afterwards.
addr_t sys_schfs_index(void)
{
char *name;
if (argstr(0, &name) < 0) return -EFAULT;
int namelen = strlen(name);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
return schfs_kindex(SCHFS_DEV, name, (unsigned)namelen) == 0 ? 0 : -EIO;
}
// Every object whose `name` attribute equals `value`, as object ids.
//
// Object ids rather than paths: the index maps values to objects, and a
// filesystem keyed that way has no cheap way back to a name. Turning an id into
// a path is a caller's problem, and a rarer one than the query itself.
addr_t sys_schfs_query(void)
{
char *name, *value;
addr_t uout;
int cap;
if (argstr(0, &name) < 0 || argstr(1, &value) < 0) return -EFAULT;
if (argaddr(2, &uout) < 0 || argint(3, &cap) < 0) return -EFAULT;
if (cap <= 0 || cap > 256) return -EINVAL;
int namelen = strlen(name), vallen = strlen(value);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
if (vallen > SCHFS_ATTR_VALUE_MAX) return -E2BIG;
uint64 ids[64];
if (cap > (int)(sizeof(ids) / sizeof(ids[0])))
cap = (int)(sizeof(ids) / sizeof(ids[0]));
int n = schfs_kquery(SCHFS_DEV, name, (unsigned)namelen,
value, (unsigned)vallen, ids, cap);
if (n < 0) return -ENOENT; /* no such index */
if (n > 0 && copy_to_user(uout, ids, (uint64)n * sizeof(uint64)) < 0)
return -EFAULT;
return (addr_t)n;
}