The second of the two things the stack owed. recvfrom accepted the
address arguments and ignored them, so every caller was told nothing
about where a datagram came from -- which for UDP is most of the point:
a server that cannot see the sender cannot reply to it.
Fixing it properly meant fixing something worse that was sitting
underneath. UDP payloads went into the same byte ring TCP uses, with no
record of where one datagram ended and the next began, so two datagrams
that arrived together came back from one read as one. That is a bug
whose appearance depends on timing and whose blame lands on the sender.
Datagrams now carry a header alongside the bytes: source address, source
port, length. Sixteen of them per socket, in a ring beside the existing
one. Whole buffers would have been 16 sockets x 16 datagrams x an MTU of
static memory for something almost always empty; only the boundaries need
keeping, and those are eight bytes each. A datagram too big for the
buffer it is read into is truncated and the remainder discarded, rather
than left in the ring to be delivered as the head of the next one. One
that arrives with the queue full is dropped whole -- that is UDP's own
contract, and half a datagram in the ring would corrupt every one after.
An empty queue and a zero-length datagram are kept apart, because a
sender really can send zero bytes and the right answer to that is 0
rather than "nothing has arrived yet".
sendto grew the mirror image: a destination address on a datagram
socket, which is what a server replying to whoever just wrote to it
needs. On TCP a destination is ignored rather than refused -- the
connection decides where the bytes go, and programs pass one out of
habit. For a stream, recvfrom's source is read from the pcb: it is the
peer, and it is the same answer for every byte.
Verified: bootcheck -Net now covers UDP too, 45 [ok] and no failures.
The address check is real rather than cosmetic -- nettest deliberately
leaves its sockaddr uninitialised, so a kernel that filled in nothing
would print stack rubbish instead of 10.0.2.2 and the port the responder
is actually on. The boundaries are checked by byte count: the responder
sends 20 bytes and then 16, and the guest reports reading 20 and then
16. Treated as a stream the first read would have taken all 36.
Left for later, and deliberately: accept() still does not report the
peer's address, and getsockname/getpeername do not exist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The TCP/IP stack landed with its end-to-end test performed by hand: start
a listener, boot, type nettest, read the output. That is evidence once
and nothing afterwards, and it is exactly the kind of check that quietly
stops being run.
bootcheck -Net does the whole exchange. A listener runs on this host on a
port picked by binding one and letting go, the guest is told to connect
out to 10.0.2.2 at that port, and both ends are asserted: the guest
reports what it received, the listener reports what arrived. One side
alone proves much less than it looks -- a stack that completes a
handshake and then drops every byte satisfies any check made from inside
the guest.
The listener is a background job, a separate process, because this
harness has one hard rule: nothing may block the writer feeding QEMU's
stdin. A socket accept is precisely the wait that deadlocked two earlier
attempts at reading QEMU's output. It answers the protocol nettest
speaks and then closes -- the close is load-bearing, since nettest reads
until end of file and a listener that stayed open would hang the guest
rather than fail it.
The NIC is spelled out (-netdev user + -device e1000) rather than left to
QEMU's default. Other runs can take the default and still answer "did a
driver bind"; a run that asserts on an address should be reading its own
configuration.
/proc/netif is new, and the address is why. Until now the only report was
a line on the console at boot, which userland cannot read and a test can
only find by scraping the transcript. It gives interface, MAC, state,
address, netmask, gateway, MTU and frame counts. Named netif rather than
net on purpose: /proc/net is a *directory* on Linux, and a plain file
there would give ENOTDIR to anything reaching for /proc/net/dev, which is
worse than not having it. The counters are kept at the lwIP seam rather
than in the driver, because a frame the driver received and the netif
dropped never reached the stack.
Verified: 38 [ok], no failures. Eleven of them are new -- lwIP up, DHCP
obtaining 10.0.2.15 from SLIRP (a number nothing in the guest knows in
advance, so it is the exchange completing rather than a guess), the four
/proc/netif fields, frames counted in both directions, the handshake, the
reply read back, and the listener confirming HELLO-FROM-SCHOS arrived.
One bug found by writing it: $netPort was chosen after the command list
was built, so the guest ran `nettest 10.0.2.2` with no port and printed
its usage. An unset PowerShell variable interpolates to nothing rather
than failing, which is worth remembering next time a command in this
file looks right and behaves oddly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.
Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.
Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.
Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.
Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:
- statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
so every field after it landed one slot late: userland read f_frsize
out of f_namelen, got 255, and df reported a 32 MB root filesystem
as 1992 KB. It hid for as long as it did because the fields used to
be invented constants, so wrong numbers looked like the usual ones.
- struct inode's nlink was a short against SchFS's uint32, so a file
past 32767 names came back negative -- and iput frees an object
whose count reaches zero. It is an int now, signed on purpose, so an
underflow lands somewhere a check can see rather than wrapping.
- filewrite split every write to fit v1's log. That limit belongs to
the journal, not to writing, and SchFS makes one transaction of
whatever it is handed: a 1 MB write was becoming 170 commits and
170 flushes for nothing. fs_ops carries max_write now, and SchFS
sets no limit.
Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.
Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported from a real laptop booting through Ventoy: the F8 menu appears,
selecting Boot SchOS gives a black screen and a white dash in the corner.
Both halves of that are bugs.
The disc could not be chainloaded. Booting it directly, firmware runs the
El Torito FAT image and the loader opens \schos\kernel on that volume,
which is where mkiso.sh put it. Ventoy does not use El Torito at all: it
mounts the ISO 9660 tree and runs \EFI\BOOT\BOOTX64.EFI from there. The
loader then looks for the kernel on the volume it was itself loaded from
-- the ISO tree -- and the kernel was only ever inside the FAT image. It
is now in both. The comment in mkiso.sh had even described this failure,
as the reason the kernel is in the FAT image, without noticing the ISO
tree had the same requirement.
The error was invisible. print() redirects to the serial port once the
splash is up, which is right for a development machine and useless on a
laptop: the loader died with a perfectly good explanation and sent it to
a wire nobody was holding. What is left on screen is the firmware's own
cursor -- the white dash. die() now draws the message on the framebuffer
too, over a cleared band so it does not land on the logo.
Verified by reproducing the exact failure: the ISO tree extracted and
booted as a plain volume, as a chainloader presents it. With the kernel
removed it dies visibly with "no kernel at \schos\kernel or \kernel, EFI
status 0x000E" -- EFI_NOT_FOUND, which is what the laptop was hitting
silently. With it present, that same volume boots through to a shell.
110 [ok] on the ESP path, the ISO check passes, no failures.
ui_panel was written for this screen during the loader restyle and then
never called -- dead code sitting in boot/common/ui.c waiting for the row
that would use it. This is that row: F8 -> System Information, showing
firmware vendor and revision, UEFI revision, CPU brand, usable memory,
current mode and whether ACPI was found.
Everything is read when the screen is drawn rather than captured at
startup, because the resolution row above it can have changed the mode,
and a panel still reporting what the machine booted with is worse than no
panel at all.
Two facts are deliberately read through the same code that feeds the
kernel -- efi_type_to_e820() for what counts as usable memory, and
find_rsdp() for ACPI. A panel with its own idea of either would be a
second implementation to keep in step, and the ACPI line exists precisely
because a missing RSDP means no MADT, no IOAPIC and no SMP, which has
already been debugged once from the far end.
The screenshot caught a bug in the screen it was taken to verify: UEFI
revision read "21587.2286". EFI_TABLE_HEADER begins with a UINT64
Signature, so Revision is at offset 8 and I had read offset 4 -- the
upper half of "IBI SYST", formatted as a version number. It now reads 2.7
under OVMF, which is correct.
Verified by driving the menu over the QEMU monitor and reading the
pixels: five rows with the hint updated to 1-5, the panel centred, and
every value cross-checkable -- EDK II, firmware 1.0, UEFI 2.7, "QEMU
Virtual CPU version 2.5+" against -cpu qemu64, 1018 MB of the 1 GB given,
1280x800 matching what fbshot captures, ACPI present. The highlight sits
9 px inside both panel borders, symmetric.
F8 has a 1.5 s window, so the capture presses it repeatedly through boot
rather than guessing when OVMF finishes -- and only F8, since any other
key breaks the wait and boots.
Also: every ESP drive now says format=raw. QEMU warned about probing the
format on each of the fifteen or so places that attach it, which is new
noise on what is now the only boot path. 110 [ok], no failures, no
warnings.
./mkiso.sh from tools/posix/ failed with "missing build/kernel/kernel --
run make first" on a tree that was built perfectly well: the paths were
relative to the caller's directory rather than the repo. mkiso.sh and
schcom_test.sh now resolve the root from their own location, the way
memory-sync.sh already did. An output path given to mkiso.sh is made
absolute first, so `mkiso.sh out.iso` still writes where the caller is
standing and not where the repo is.
mutate.sh had it worse: it began with `cd /mnt/d/DEV/SchOS`, an absolute
WSL path from the Windows machine it was written on, so it could not run
against any other checkout at all.
Fixing that exposed why it mattered. Every one of its nine mutations
reported "rejected at compile time" -- and none of them had reached a
compiler. The host tools moved to build/tools/ during the bt/ -> tools/
migration and this script still asked for build/mkschfs, so make answered
"No rule to make target" and the harness read that as the mutation being
caught by the build. A mutation test whose every case is silently skipped
is precisely the thing its own header warns about: a test that cannot
fail, believed anyway.
With the paths corrected all nine are caught by the suites, which is what
this was always supposed to demonstrate:
file_read ignores the end-of-file clamp -> extracted file differs
dirent_walk advances past the next entry -> entries lost
dir_lookup returns the wrong objectid -> fsck: no such file
extents point one block past their data -> mkschfs write fails
an overwrite drops the tail of the split -> FAIL: CoW keeps the run
truncate leaves stale bytes in the tail block -> FAIL: reads as zeros
committed frees never return to the free map -> FAIL: create hog -> -3
hard links do not bump nlink -> FAIL: nlink=1, expected 2
data checksum covers only the first 16 bytes -> FAIL: tree walk
The build failure is now printed rather than swallowed, so the next time
a target is renamed it says so instead of reporting nine passes. Two
mutations also reported a bare "caught:" with no reason -- the suite had
failed but nothing matched the quote patterns -- so the fallback is now
the last thing the suite actually said.
schfs-check and schfs-test still pass, and the tree restores clean.
1449 lines of boot/bios/, tools/sign.pl and tools/padstage2.pl, plus the
build rules, the disk image and the ISO entry that existed to carry them.
Everything since roughly 2012 boots UEFI, and CSM -- the firmware-side
BIOS emulation that lets a legacy loader run on such a machine at all --
has been going away since Intel dropped support for it in 2020. So this
is not a path being retired early; it is one whose remaining hardware is
already the exception.
The cost was never the code, it was the tax on everything near it. Stage
2 had a 65-sector ceiling it overflowed silently -- 25440 bytes against
24576 -- and because the kernel is written at sector 1 + STAGE2_SECTORS,
an overflowing loader was overwritten by the very kernel it was loading.
The resolution list could not be shared with UEFI either: VBE mode
setting is INT 10h, so matching it needed unreal mode, which is the
riskiest code in the tree for a feature that already worked elsewhere.
What replaces schos.img is the staged ESP, handed to QEMU as a virtual
FAT volume. That is a development arrangement and not an installable
disk; a real GPT image with a real FAT partition is the installer's job,
and until it exists nothing here writes a disk a machine boots unaided.
Secure Boot is untouched too -- the loader is unsigned.
The ISO keeps one El Torito entry, the FAT image holding loader and
kernel. It never booted from a USB stick via BIOS anyway: the firmware
presents the stick as a hard disk while stage2main.c read the kernel back
over ATAPI.
The 32-bit multiboot entry in entry.S now has no caller and is left in
place. Deleting kernel entry code is its own change with its own way of
failing silently, and it should not ride along with this one.
Ported with it: run.ps1, the six screenshot tools, and schcom_test.sh,
each with its own OVMF varstore copy so no run inherits another's UEFI
variables.
Two things this shook out. OVMF now belongs in `all`: nothing boots
without firmware, `make clean` wipes build/ including the copy fetched
from the host, and a clean followed by a build left every run script
throwing. And it cannot be listed as a prerequisite at the top of the
file -- make expands prerequisites when it reads the line, and
$(OVMF_CODE) is defined further down, so naming it there appends nothing
and says nothing.
Verified from a clean tree: 110 [ok] and no failures on the full suite,
the ISO boots with no hard disk attached, schfs-check passes, schcom's
end-to-end pixel test passes, and fbshot captures a real 1280x800 frame.
bootcheck has only ever booted through the BIOS loader. That is the path
being removed, so the suite had to reach OVMF before the removal rather
than after it -- deleting the boot path first would leave nothing running
during the work that needs watching most.
-Uefi swaps the drives for a pflash pair and the staged ESP, keeping the
same layout: ESP on index 0, the SchFS volume on index 1. Every check
runs unaltered, so a difference between the two transcripts is a
difference in firmware and not in what was asked. The varstore is copied
per run; sharing one would let a run that changed a UEFI variable decide
how the next one behaves.
Both paths now report 110 [ok] with no failures.
Three checks did fail on UEFI, and were right to. They asserted xres 800,
yres 600 and line_length 3200 -- the mode boot/bios/stage2.S asks VBE for.
Under OVMF the firmware picks, here 1280x800 with a 5120-byte stride,
which is correct and self-consistent. The checks were pinning the BIOS
loader's fixed choice, and could do so only because that choice was fixed.
They now assert the invariants the original comment was actually after: a
plausible mode, a stride that covers a full row, and a width that is a
multiple of 8 -- the last being the mode that sheared under OVMF, where
the firmware advertised 1366 and the hardware scanned out 1360.
Also reaped a QEMU left holding schfs.img by a run started outside the
harness, which its pidfile reaper could not have known about.
--snapshots is a range scan over SCHFS_SNAP_OBJECTID, printing each
retained root's id, name, block and the generation it was taken at. The
name is printed with an explicit precision: it is bounded but a damaged
image need not have terminated it.
mkschfs grows --snapshot NAME, which takes one of the finished tree. It
runs after the commit, because a snapshot names a *committed* root --
taking it before would record a root that exists only in memory.
Verified end to end: an image built with --snapshot golden lists the
snapshot and still passes --check. An image without one prints "no
snapshots" rather than nothing at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A snapshot is a tree root that is kept. The B-tree is copy-on-write, so
the blocks a committed root reaches are already immutable -- nothing
overwrites a block a committed tree can reach -- and retaining one costs
a record: (SNAP_OBJECTID, KEY_SNAP, id) -> struct schfs_snap.
The record names the root as it was *before* the record was added.
Inserting it rewrites the path under copy-on-write and produces a new
root, so a snapshot that included its own creation would name a block
nobody had written yet.
What makes it survive is the derivation. There is no free-space map on
disk; it is rebuilt at mount by walking the tree, and that walk now
covers every snapshot root as well as the live one. Without it a block
reachable only from a snapshot looks free, gets handed out, and
overwrites the data the snapshot exists to keep -- silently, surfacing
only when the snapshot is read. A snapshot's own tree may hold older
snapshot records; those are deliberately not followed, or deleting a
snapshot could be undone by an older one's memory of it.
Runtime frees are suspended while a snapshot exists. Knowing which freed
blocks a snapshot still needs requires per-extent reference counting,
which is a design and not a flag; until that exists the safe answer is to
free nothing and let the next mount's derivation reclaim what is
genuinely unreachable. Allocation is unaffected and stays correct -- it
works from the in-use bitmap the derivation filled. The running
free-blocks *counter* does drift from the truth while a snapshot is
present, which is cosmetic and is reported by the test rather than
asserted away.
Tested in schfs_objtest: after a snapshot, the live tree shows new
contents while the snapshot still shows the old ones; a file deleted from
the live tree is still readable through the snapshot; deleting the
snapshot removes the record and the space comes back at the next mount.
schfs-check passes including the round-trip; bootcheck 108 [ok].
Not yet: snapshots are not reachable by name from userspace -- there is
no mount-a-snapshot path, so this is the format and the mechanism, not
the feature a user sees.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The compositor already walked a surface table, but nothing had ever run
two clients at once, so "it composites" rested on a single rectangle.
schcom-test now starts two overlapping clients and checks the overlap
region from the host: the later surface has to win there. A compositor
that cannot order two surfaces cannot host a window manager, and that is
the property SchWM will be built on.
Two bugs in the test itself, both of which would have read as compositor
faults:
- it waited for the first "committed" line and screenshotted then, so
the second client had not drawn yet. It waits for both now.
- one probe point sat at 400,300 -- dead centre of an 800x600 screen,
which is exactly where the pointer is drawn. Moved clear of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Clients connect over a Unix socket, are handed a shared buffer, draw into
it and commit; Schcom composites and presents. Both of the hard blockers
its README listed -- unix domain sockets and raw input events -- have been
done for a while, so nothing was in the way.
The model is Wayland's, on purpose. A surface is not a window: it is a
rectangle of pixels with no policy attached, because placement belongs to
SchWM and the moment this program starts deciding where windows go the
split has failed. Nothing is shown until COMMIT, so a client halfway
through drawing is never presented. The compositor tells clients when to
draw rather than letting them render frames nobody sees.
Wayland is MIT-licensed and supporting it later is intended, so the
divergences are the ones that are cheap to undo: a buffer handle is a
System V shm id rather than an fd over SCM_RIGHTS, and the message
carries a handle_kind field so switching is a new value rather than a
protocol break. Every message carries its own length, so an unknown
message is skipped instead of desynchronising the stream -- which is what
lets a v0 compositor survive a v1 client.
The compositor owns the shm segment, not the client: it has to survive a
client crashing mid-frame, and it must not be something a client can
resize underneath it. A dropped connection takes its surfaces and their
segments with it.
Composites into a back buffer and copies once, because compositing
straight into the framebuffer shows every intermediate state -- the
flicker would be the whole desktop being repainted under the user's eyes.
projects/ gets build glue for the first time: `make projects` builds
against the sysroot's libc and stages into sysroot/bin. It is not a
dependency of anything above it, and `make` never descends there.
Verified by `make schcom-test`, which boots SchOS, runs the compositor and
a client, screendumps the framebuffer and asserts the pixels on the host.
That last part is the point -- the compositor reporting that it
composited something proves nothing, and a screenshot nobody inspects
proves less. It checks the client's colour mid-surface, the compositor's
background, and the white border exactly at the requested origin, so
"arrived" and "arrived at the wrong offset" cannot pass for each other.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The disc carried one file -- boot/schos.bin -- with 8.3 uppercase names,
one El Torito entry, and no way to boot a UEFI machine. Now:
/boot/schos.bin BIOS entry, unchanged: no-emulation, 1+STAGE2_SECTORS
/boot/efiboot.img UEFI entry: a FAT image built with mtools
/EFI/BOOT/BOOTX64.EFI the loader, also visible as a file
/bin /etc /usr ... the staged sysroot as an ordinary tree
Rock Ridge and Joliet restore real names, permissions and symlinks;
-cache-inodes matters more than it looks, because the sysroot is a toybox
multicall binary with a couple of hundred hard links to it and without
inode caching each link becomes its own copy.
-isohybrid-gpt-basdat adds a GPT so the .iso can be dd'd to a stick and
booted by UEFI. That is UEFI only and deliberately so: the BIOS isohybrid
trick needs syslinux's isohdpfx.bin, and even with it this loader reads
the kernel back over ATAPI, which a stick presented as a hard disk cannot
serve. usbstick.img is what that is for.
The FAT image carries the kernel as well as the loader, because
BOOTX64.EFI opens \schos\kernel on the volume it was loaded from, and for
a UEFI boot that is the FAT image rather than the ISO 9660 tree. Without
it the loader halted with EFI_NOT_FOUND.
Also fixes `make iso` and `make ramroot`, which were already broken:
fs-ram.img was still built by the v1 mkfs while the kernel accepts only
SchFS as root, so the ramdisk was a format it could not mount and it
panicked with "schfs root: no SchFS volume on the root device". It is a
mkschfs image now, which needed --exclude in mkschfs -- dropping the
exclude list was not an option, because the ramdisk is linked into the
kernel binary and what goes in it is a licensing question. RAMDISK_NINODES
is gone with the v1 image: SchFS has no preallocated inode table.
Verified: BIOS ISO boots to a shell and runs commands over serial
(241 files in /bin, /etc/motd reads back). UEFI ISO boots to a prompt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three bugs, all of which surfaced as a failed check somewhere other than
the command that was actually slow.
Wait-Prompt waited a fixed 20 s. These tests run under TCG, not KVM, and
a cold `ls /bin | wc -l` takes 23 s on the Fedora laptop measured with
the guest's own /proc/uptime -- it was over the wall every time. Raised
to 60 s; a long timeout only costs time when something is genuinely
wrong, a short one costs a false failure elsewhere.
It also matched a prompt anywhere in the transcript. Between writing a
command and the guest echoing it the log still ends with the previous
prompt, so it could return before the command had started and the next
one was typed on top of it. Anchor the match to text produced after the
command was sent.
On timeout it slept two seconds and typed regardless, which is what
turned one slow command into an interleaved transcript and a failure
attributed to a later check. Wait for output to stop arriving instead.
Report any command that never produced a prompt next to the result:
after one overrun every later check is reading a transcript that may be
interleaved, and that should not be something you have to notice in the
scrollback.
With this, -Fb -Tty -Usb -Sock -Evdev -Ps is 108 [ok], 0 misses.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The `finally` stops QEMU on every path the script controls, but not when
the script itself is killed -- a CI timeout, Ctrl-C at the wrong moment,
pkill. QEMU then keeps running with no parent. Because these tests run
under TCG rather than KVM, a spare emulator pinning a core slows every
later run, and one was found still executing the guest at 86% of a core
eight minutes after its parent was gone.
That is the bad kind of failure: the suite does not break, it gets
slower, and commands drift past Wait-Prompt's timeout until the next one
is typed into a busy shell and the transcript interleaves. The check
that then fails is unrelated to whatever actually changed.
Record the pid on startup and reap the previous run's leftover before
starting a new one, the same self-healing approach already used for the
monitor port. The name is checked before killing, because pids are
recycled and killing an unrelated process would be worse than the leak.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Neither is Fedora-specific. A different host and a freshly built
userland simply drove code that Windows never reached.
tools/mkfs.c did not compile at all. The DT_UNKNOWN/DT_DIR/DT_REG
fallbacks sat above <dirent.h>, so on glibc -- which spells them as
enumerators -- the #ifndef asked before the header was read, always
answered "missing", and then the header's own enum expanded to
`enum { 0 = 0, ... }`. Move the fallbacks below the include. The
include cannot move up instead: it has to follow the #undef that ends
the `#define dirent schfs_dirent` block.
getstackpcs() page-faulted. It dereferenced rbp[1] without validating
rbp, and the guard rejected 0xffffffff -- dead code on x86-64, already
caught by the KERNBASE test -- while the value that actually turns up
is all-ones, which is above KERNBASE. rbp[1] then wrapped to 0x7 and
took trap 14. `cp` panicked the kernel every time. Bound rbp from
above and require 8-byte alignment.
That second one masks other bugs: getcallerpcs() runs on every lock
acquisition and again from the trap handler, so a walker that can
fault replaces a real diagnosis with a bogus one.
Baseline on the Fedora laptop is now green: make ports, make,
make schfs-check including the round-trip that had never run, and
bootcheck at 27/27 plus 108 [ok] across the extended suites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tools/posix/setup-fedora.sh installs the toolchain and reports what it
cannot: PowerShell has no Fedora package, and bootcheck is the test
suite, so the script says how to get pwsh rather than failing quietly.
It also checks /dev/kvm, since a Ryzen box can run the tests at native
speed.
docs/moving-to-linux.md is the actual procedure: copy the repo, restore
the memory, install, build. The memory travels inside the repo and
`memory-sync.sh import` puts it under the key this machine derives from
its own path -- /home/nexoniarz/Documents/SchOS becomes
-home-nexoniarz-Documents-SchOS, where D:\DEV\SchOS was D--DEV-SchOS.
OVMF discovery no longer assumes QEMU's Windows layout. Fedora ships
edk2-ovmf to /usr/share/edk2/ovmf/, Debian and Arch differ again, so the
Makefile searches a list and names the packages when it finds nothing.
Tested through the shell rather than $(wildcard): on Windows make is a
native binary using Win32 file calls, while `command -v` returns an MSYS
path like /c/Users/..., which $(wildcard) cannot see -- it returned
nothing on a machine that had the firmware.
Two things only that machine can test: `make ports` builds musl, toybox
and mksh from source, which Windows cannot do at all; and the SchFS
round-trip needs symlinks and hard links, so it has never actually run.
Milestone 1 of three: manifests, RunAtLoad, KeepAlive with throttling,
and reaping. Replaces the hardcoded fork-a-shell loop in cmd/init, which
stays in the tree for now.
Jobs are sectioned key=value files in /etc/schinit/*.job, kept in the
repo under etc/ and installed into the sysroot -- sysroot/ is generated
and gitignored, so anything written only there is lost on a fresh clone,
which is already true of /etc/passwd and should not become true of the
boot configuration. Unknown keys and sections are ignored rather than
fatal: a manifest written for a newer schinit must not stop this one
booting the machine.
Socket sections are parsed and recorded but not yet held. That is
milestone 2 and it is the whole reason for copying launchd rather than
sysvinit -- the supervisor holds the listener and starts the job on first
connect, which inverts the dependency graph and makes runlevels a
non-concept. The key exists now so manifests need not change later.
getdents64 is exposed to userland for this. The kernel has had it since
the Linux ABI landed, but nothing declared it in user.h or lib/usys.S, so
first-party programs had no way to list a directory at all.
Two things found by testing rather than reasoning:
- setsid() gives a job its own session, and a session has no
controlling terminal. The console then delivers input only to the
foreground process group it tracks, so the shell came up, echoed
every keystroke through the line discipline, and executed none of
them -- indistinguishable from a hung shell. ControllingTerminal=yes
claims it with TIOCSCTTY/TIOCSPGRP; off by default, since a daemon
has no business seizing the console.
- the first bootstrap() dropped /dev/input/event0-1 and most of /proc,
because it was transcribed from the part of the old init that had
been read rather than all of it. -Evdev caught it.
build/usbstick.img is built by `all` now: without it QEMU refuses the
-drive and the entire -Usb run produces no output, which reads as a
broken USB driver.
Seven suites green: base 27, Tty 51, Ps 32, Sock 64, Fb 36, Usb 31,
Evdev 29. make schfs-check passes.
boot/bios/ now holds the MBR, the El Torito stub and the real-mode stage
2; boot/uefi/ is empty and documented.
Separate directories rather than #ifdefs, because the two share almost
nothing. A BIOS loader lives in 16-bit real mode, reaches the firmware
through software interrupts, and walks itself up into long mode. A UEFI
loader is a 64-bit PE that the firmware has already placed in long mode
with paging on, calling services through a table of function pointers.
What they do share is the hand-off contract, which boot/README.md now
writes down: ELF segments placed, long mode, a 24-byte-entry memory map
at E820_MAP_ADDR, framebuffer geometry at VBE_INFO_ADDR.
That contract is the point of documenting it now. UEFI's GetMemoryMap
returns EFI_MEMORY_DESCRIPTORs, and the right move is to translate them
into the entries kernel/arch/x86_64/e820.c already parses rather than
teach the kernel a second format.
Build output paths under $(BUILD)/boot are unchanged; only sources moved.
Two bugs reported from actual use, plus a test harness that had stopped
testing.
init could be killed by Ctrl-C, taking the machine with it. SIGINT goes
to the foreground process group, init is pgid 1, and any window where
tty_fg_pgrp is 1 -- before a shell claims the terminal, after one exits,
or between the two while the key is held -- lands it on init. init has
no handler, so killed was set, exit() ran, and exit() panics outright on
initproc. Spamming Ctrl-C at a frozen program is exactly the input that
finds that window. POSIX and Linux both discard a default-action signal
for pid 1; init_ignores() now does the same, SIGKILL included, because
init is not killable from userspace on any Unix.
Everything was slow because BSIZE was 1024 while a SchFS block is 4096,
so every block cost four IDE requests, each with its own interrupt and
sleep/wakeup. exec() of the 800 KB toybox binary meant ~800 round trips.
The giveaway: `seq 1 200`, which touches no file and prints nothing,
cost the same 2.0s as `cat /bin/toybox`, and `ls | wc -l` cost exactly
double -- both dominated by loading the same binary twice.
BSIZE is 4096 now, one request per SchFS block, and NBUF is 256 blocks
(1 MB) rather than 90 KB -- smaller than the binary every command execs,
so nothing was ever still resident. ideinit() issues SET MULTIPLE MODE,
which is what makes a 4096-byte transfer legal: READ/WRITE MULTIPLE move
the drive's multiple count per interrupt and nothing had ever set it,
which is the real reason idestart() carried a sector_per_block > 7 panic.
seq 1 200 2.02s -> 0.06s
ls /bin 2.76s -> 0.68s
ls /bin | wc -l 4.68s -> 0.73s
And the harness. bootcheck called Find-Qemu from a param() default,
which is bound before any function in the body exists -- so it died on
its first line and printed an error containing neither "FAIL" nor
"Exception", the two things being grepped for. It had not run since it
was made host-portable. Worse, an earlier index-based edit had deleted
the Expect blocks for -Tty, -Usb, -Sock, -Ps and -Evdev, so five of the
seven modes asserted nothing at all. Restored, and the USB run gets its
own 16 MB build/usbstick.img: it had been pointed at build/fs.img, which
nothing builds since SchFS became root, so QEMU refused to start.
242 checks across seven modes, all passing, and make schfs-check green.
tools/windows/ .ps1 (pwsh, so usable from Linux and macOS too)
tools/posix/ .sh
tools/python/ .py
tools/ .c and .pl
The .c and .pl files stay at the top level because they are not
host-specific: the build compiles or invokes them on every platform.
Call sites updated in the Makefile, ports/mk/port.mk, docs and the
project memory. The one thing that does not survive a move on its own is
a script deriving the repo root from its own location -- all nine
PowerShell tools used Split-Path $PSScriptRoot -Parent, and memory-sync.sh
used dirname/.., both of which quietly become tools/ one level deeper.
Two levels now.
Two pre-existing Windows failures surfaced while verifying, both real and
neither caused by the move:
- schfs_fsck.c did not compile under mingw at all: --extract uses
POSIX mkdir(path,mode), symlink() and link(), none of which mingw
has. That took the whole tool with it, and `make schfs-check` with
it. mkdir maps to _mkdir; symlink and link fail loudly rather than
writing a regular file in their place, because an extracted tree
that differs from the image while reporting success defeats the
point of --extract.
- the round-trip test then failed at `ln -s`, since Git Bash cannot
create symlinks without developer mode. It skips with a message now.
Not a fudge: mkschfs's mingw build cannot see a symlink either, so
the test would have compared a tree with none against an image with
none and called it a pass -- coverage reporting itself as present
while being absent.
Verified on Windows: build clean, all seven bootcheck suites green,
make schfs-check passes.
Claude Code keeps per-project memory at <home>/.claude/projects/<key>/,
where <key> is the project's absolute path with every separator and the
drive colon turned into a dash. That makes it both rename-fragile and
machine-specific, and this project has hit each of those once:
- D:\DEV\Schunix -> D:\DEV\SchOS changed the key, and 43 memories were
invisible until moved across by hand.
- Development happens from Windows and from Linux, and each looks in a
different place for the same checkout.
Committing .claude/memory/ fixes both -- git carries it across renames
and between hosts -- and `import` puts it wherever the local machine
expects. .claude/settings.local.json stays ignored: those are per-machine
permissions.
The colon maps to a dash rather than being dropped. Deleting it gives
D-DEV-SchOS, which matches nothing and silently finds zero memories; that
was the first attempt. On Git Bash the key comes from `cygpath -w`,
because Claude Code is a Windows program and sees D:\DEV\SchOS, not
/d/DEV/SchOS.
Developed from both Windows and Linux, so nothing in the build should
care which host it runs on. It did, in four places.
Toolchain: one path instead of two. clang is a cross-compiler by
construction, so a single LLVM install targets bare-metal ELF, 16/32-bit
code below 1 MB, and Linux/musl userland from any host -- which is how
the Windows build already worked. The GNU branch beside it had drifted
into passing --no-warn-rwx-segments, a hard error on lld, which is the
default linker on both FreeBSD and macOS; and it required a brewed
x86_64-elf-gcc before anything built on a Mac at all. GNU is kept as a
fallback (USE_GNU=1) but clang+lld is chosen whenever both are present.
Host tools use the host's own cc; only mingw needs naming explicitly.
tools/mkschfs.c included <sys/sysmacros.h> unconditionally on non-mingw
hosts. That header is glibc/musl only -- macOS and the BSDs have neither
the header nor a need for it, since major()/minor() come from
<sys/types.h> there. Now guarded on __linux__.
Ports verified tarballs with `sha256sum -c -`, which does not exist on
macOS (shasum), FreeBSD/NetBSD (sha256), or OpenBSD (cksum -a sha256),
and each spells its check mode differently. tools/sha256check.sh picks
whichever of the five the host has and compares digests itself.
nproc is coreutils, so absent on macOS and BSD; falls through to sysctl
-n hw.ncpu before defaulting to 4.
The PowerShell tools now run under pwsh on any OS: QEMU is found on PATH
(the scoop location is a Windows-only fallback), and path literals use
forward slashes -- Join-Path $root 'build\x' produces a *filename
containing a backslash* on a Unix host. The backslash sweep is limited
to known directory prefixes on purpose: bootcheck sends
printf 'caf\303\251' to the guest to test UTF-8, and a blanket
substitution would have quietly changed what that asserts.
Also fixes six tools left booting build/fs.img, which stopped being
bootable when SchFS became the root filesystem: crashshot, dispshot,
fbshot, splashshot, textshot and ttyshot all take build/schfs.img now,
and the -SchFS switches that used to select it are gone.
Verified on Windows: full build, and bootcheck green. The Linux, macOS
and BSD paths are written from their documented behaviour and are NOT
executed here -- this machine has no WSL and no second host.
Removes the "-unix" element and with it any trademark exposure to The
Open Group.
The split follows the ecosystem layout: SchOS is the system, and the
kernel has its own name. Kernel-identity strings are the only places
that take SchIFT -- uname(2) sysname, /proc/version, and the two licence
sentences that say "kernel" outright. That is what every Unix does here:
`uname -s` on Linux reports the kernel, not the distribution. Everything
a user reads as the product -- panic screen, boot menu, error prefixes,
docs, LICENSE, NOTICE -- is SchOS.
schunix.img is schos.img, the logo assets and the musl TLS patch are
renamed, and __SCHUNIX__ is __SCHOS__ (the define and the patch that
consumes it moved together).
Two things the rename broke, both found by the bootcheck suite:
- shmtest wrote "SCHUNIX-MMAP-OK" with a hardcoded length of 15 and
compared it back by hand-counted offsets (m[3]=='U', m[14]=='K').
The shorter marker left the write running two bytes past the literal
and the compare reading past the data, failing a file-backed mmap
that was working correctly. Now written and compared through one
constant with sizeof.
- bootcheck asserted a 31-byte file after appending to
"hello-from-schunix"; the shorter string makes it 29.
Both are the same mistake: a string literal whose length is written out
somewhere else. Full suite green -- base, Tty, Ps, Sock, Fb, Usb, Evdev.
An x86-64 operating system built from xv6, now largely rewritten: SMP,
a copy-on-write root filesystem (SchFS), PCI/USB/virtio drivers, a TTF
rasteriser, and a Linux-numbered syscall ABI.
No history before this point -- the tree was never under version control.
This commit exists so the rename that follows has something to undo to.
