Sixty-five headers sat side by side in include/, so `#include "irq.h"`
said nothing about where irq.h was or what it belonged to, and errno.h,
stat.h and fcntl.h at the top shadowed the system's for any host tool
built with -I./include. They are in folders now -- arch/ core/ fs/
dev/ gfx/ net/ uapi/ (and ldls/ for the Linux driver layer) -- and
every include names the folder, as Linux's do: "core/proc.h".
Done mechanically and checked the hard way: the kernel built from this
tree disassembles identically to the one built from the tree before
it. The full image boots, and userland is the same set of programs.
Paths in docs and comments follow; docs/guides/layout.md has a table
of what is where, and tools/fix-includes.py rewrites old-style
includes from the folders in the tree, for code written before this.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
uber_valid() answered one question -- valid or not -- and a version mismatch
returned the same as a failed checksum. Those two states need opposite handling:
A damaged uberblock means STEP OVER IT. That is what the 128-slot ring is for,
and the older root is a consistent filesystem. This is crash recovery.
A sound uberblock from a newer format means STOP. Falling back mounts a STALE
filesystem: every commit made since the volume was upgraded is discarded, the
mount reports success, and nobody finds out.
Demonstrated rather than asserted. A forged image with two uberblocks --
(version 1, generation 1, root 129) and (version 2, generation 2, root 130) --
made the old code mount root 129 and exit 0, silently rolling the filesystem
back to before the upgrade.
uber_check() returns BAD / OK / RO / TOO_NEW, and asks its questions in order:
is it ours (magic), is it intact (crc), can we read it (version, features), does
it make sense (root in range). Compatibility is asked before sanity on purpose --
the fields a sanity check reads are only meaningfully placed if the format is one
we know.
Plus feature flags in three classes, which is what stops a version number being a
blunt instrument: compat (ignore unknown bits), ro_compat (mount read-only),
incompat (refuse). The test for which class is one question -- what happens if a
driver that does not know this feature writes to the volume? Nothing -> compat.
Something goes stale -> ro_compat. Something is unreadable -> incompat.
They occupy what was reserved0, so every later field keeps its offset and
existing images -- which have zeros there -- read unchanged. schfs_vol_commit
memsets the uberblock before filling it in, which is why v1 volumes already say
"no features" without having been written by code that knew the field existed.
That trick is available exactly once. Four _Static_asserts now pin the layout.
Verified on forged images: version bump refuses, incompat refuses, blocksize
mismatch refuses, ro_compat mounts the NEWEST root read-only rather than falling
back.
Also here, because they are the same file: SchFS's transaction owner is a pid
alongside the pointer, since a bare struct proc * does not identify a process --
ptable slots are recycled, and an exited owner's pointer compares equal to
whichever unrelated process lands in that slot next, at which point klock() skips
the lock entirely and two callers share one volume's scratch pages. And
schfs_lockinfo(), so /proc/fsstat can report a wedged volume instead of leaving
every path asleep in acquiresleep with no way to ask why.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
lwIP 2.2.0, fetched by ports/lwip and compiled straight into the kernel.
Not written from scratch, and the reason is not the packet formats: it is
congestion control, retransmission, window management and twenty years of
corner cases that only show up against a real peer. Not FreeBSD's
netinet either, though the licence would have allowed it -- that stack is
not a library, it reaches into mbufs, epoch(9), netisr, VNET and uma(9),
so adopting it means porting FreeBSD's kernel infrastructure first. lwIP
was written for exactly this situation: a small kernel with a raw
Ethernet driver and a port layer of a few hundred lines.
It is kernel code, so nothing is installed into the sysroot and there is
no second build system: the top-level Makefile compiles the files it
wants against our port headers. The file list is explicit rather than a
wildcard, because lwIP ships IPv6, 6LoWPAN, PPP, SLIP, DNS-SD and a dozen
apps in the same tree, and building what is not configured yields either
dead code or a link error naming a port function nobody meant to write.
Its sources are unmodified and its warnings are silenced for its files
only -- patching them would mean maintaining a fork.
The port layer is ours: a lock, a clock, the netif that hands frames to
e1000, lwipopts.h, and a shim/ directory of tiny headers so lwIP's
<string.h> and friends resolve to the kernel's functions instead of the
host's, whose prototypes disagree and would otherwise be found on the
default include path.
NO_SYS=1, so lwIP has no threads of its own and one kernel thread feeds
it. A thread rather than the interrupt handler, because a call into lwIP
can reach the driver, the driver can sleep, and an interrupt handler
cannot -- so the interrupt still only clears the cause register and the
thread polls. kthread_create is new and generally useful: a process with
a kernel-only page directory, no files, no cwd, no parent, dispatched
from forkret, which can therefore sleep and be preempted like anything
else. DHCP obtains an address at boot and the console says which one.
AF_INET went in exactly where phase 2 said it would. socknew dispatches
on the domain, kernel/net/inet.c implements the internet side on lwIP's
raw API, and not one syscall changed shape: bind, connect, listen,
accept, read and write reach the same descriptors they always did. TCP
and UDP both work. cmd/nettest opens a connection, writes, and reads the
answer back.
Two things are owed and named here so they are not discovered later:
recvfrom does not yet report the source address, and the socket paths
poll on the timer tick instead of sleeping on the socket. Both are
follow-ups, not redesigns.
NOTICE carries lwIP's BSD-3 text. Clause 3 is no-endorsement, not the
old advertising clause, so it sits with Apache-2.0 without difficulty.
Verified: the tree builds clean, and the headless boot check passes with
27 [ok] and no failures with the stack compiled in and the thread
running -- which is the part that would break first if netinit deadlocked
or the thread starved anything. An end-to-end connection test belongs in
that suite and is the next commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.
Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.
Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.
Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.
Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:
- statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
so every field after it landed one slot late: userland read f_frsize
out of f_namelen, got 255, and df reported a 32 MB root filesystem
as 1992 KB. It hid for as long as it did because the fields used to
be invented constants, so wrong numbers looked like the usual ones.
- struct inode's nlink was a short against SchFS's uint32, so a file
past 32767 names came back negative -- and iput frees an object
whose count reaches zero. It is an int now, signed on purpose, so an
underflow lands somewhere a check can see rather than wrapping.
- filewrite split every write to fit v1's log. That limit belongs to
the journal, not to writing, and SchFS makes one transaction of
whatever it is handed: a 1 MB write was becoming 170 commits and
170 flushes for nothing. fs_ops carries max_write now, and SchFS
sets no limit.
Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.
Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Removes the "-unix" element and with it any trademark exposure to The
Open Group.
The split follows the ecosystem layout: SchOS is the system, and the
kernel has its own name. Kernel-identity strings are the only places
that take SchIFT -- uname(2) sysname, /proc/version, and the two licence
sentences that say "kernel" outright. That is what every Unix does here:
`uname -s` on Linux reports the kernel, not the distribution. Everything
a user reads as the product -- panic screen, boot menu, error prefixes,
docs, LICENSE, NOTICE -- is SchOS.
schunix.img is schos.img, the logo assets and the musl TLS patch are
renamed, and __SCHUNIX__ is __SCHOS__ (the define and the patch that
consumes it moved together).
Two things the rename broke, both found by the bootcheck suite:
- shmtest wrote "SCHUNIX-MMAP-OK" with a hardcoded length of 15 and
compared it back by hand-counted offsets (m[3]=='U', m[14]=='K').
The shorter marker left the write running two bytes past the literal
and the compare reading past the data, failing a file-backed mmap
that was working correctly. Now written and compared through one
constant with sizeof.
- bootcheck asserted a 31-byte file after appending to
"hello-from-schunix"; the shorter string makes it 29.
Both are the same mistake: a string literal whose length is written out
somewhere else. Full suite green -- base, Tty, Ps, Sock, Fb, Usb, Evdev.
An x86-64 operating system built from xv6, now largely rewritten: SMP,
a copy-on-write root filesystem (SchFS), PCI/USB/virtio drivers, a TTF
rasteriser, and a Linux-numbered syscall ABI.
No history before this point -- the tree was never under version control.
This commit exists so the rename that follows has something to undo to.
Nexoniarz