`make` had been failing since the e1000 work and I did not see it, because I
was checking build output with `grep error` -- which matches neither "Błąd" nor
mkfs's "image full". Checking the exit status instead found all of this in a
minute.
**usbstick.img was full.** `make ldls-stage` puts a module and its BTF in
sysroot/lib/modules, and a vmlinux BTF is 5 MB on its own; the 16 MB test image
could not hold it. The size cannot move -- tools/windows/bootcheck.ps1 asserts
READ CAPACITY returns exactly 32768 sectors -- so the exclusions grew instead:
lib and usr as well as bin. That image exists to prove the USB driver can read
one block, so its contents were never the point. It also stopped a GPL module
riding into a test artifact, which RAMDISK_EXCLUDE does for the ramdisk and
nothing was doing for this one.
**toybox's httpd was shadowing ours.** /bin/httpd answered `-p` with toybox's
"Unknown option 'p'", because whichever install rule ran second won the name.
TOYBOX_SKIP existed for exactly this and held one entry, `ping`, found the same
hard way. It is now generated from cmd/ -- a list you have to remember is a
list that is wrong the next time cmd/ grows, and it already was: `chvt` had
been silently toybox's, whose VT ioctls are Linux's and not ours.
**ldlsd would not link with clang.** `*out = in;` on a 96-byte struct is a call
to memcpy as far as the C standard is concerned; gcc unrolled it and clang
emitted the call, so a freestanding program linked for months and then did not
on a machine where nothing about it had changed but the compiler. lib/glue.c
now provides memcpy, memmove, memset and memcmp -- the four an implementation
may call without being asked.
**OVMF was never found on NixOS.** `$(dir $(QEMU))share` is the *bin*
directory plus "share", so it looked in /usr/bin/share. The absolute paths
after it covered that up on distributions that keep firmware in /usr; on one
that keeps nothing there it meant the firmware was undiscoverable on a machine
that had it, and the only way to build was to copy the file in by hand. It
resolves the prefix properly now and looks in <prefix>/share/qemu too.
Verified by exit status, from a clean tree and a no-op: both 0. ldls-check
passes 27. The guest boots, serves /srv to the host and answers 403 to
%2e%2e/etc/passwd.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Four complaints, and only one of them was the thing it looked like.
**"toybox is too small."** Half true, and the fix cost no licensing at all:
236 of toybox's applets are what defconfig builds, and the rest live in
toys/pending/ -- awk, bc, diff, more, expr, tr, hexdump, stty among them. Same
0BSD tarball, switched off by a config default. 26 of them are on now, 236 ->
268. Not everything: strace wants ptrace, syslogd wants BSD's facilitynames[],
git wants OpenSSL, and those three break the link.
The toybox build also works again on this machine. Its scripts start
#!/bin/bash, which does not exist on NixOS, so the binary in the tree was one
nobody could reproduce; the shebangs are rewritten to /usr/bin/env bash on the
way in. Pending applets are enabled by editing .config directly, because
`make oldconfig` is the obvious way and undoes it -- every pending symbol is
`default n`.
**BusyBox was asked for and is not here.** It is GPL-2.0-only, and `make
ramroot` links the root image *into the kernel binary*, so it would put GPLv2
inside an Apache-2.0 executable -- which is the exact case ports/Makefile's
NOTPORTS filter and docs/licensing.md §4 exist for. nano is the same answer one
version later. Both recorded in licensing.md with what was done instead, since
"no" without an alternative is how a rule gets quietly broken later.
**An editor that is not vi.** kilo, BSD-2-Clause, one 1300-line file, no
dependencies beyond termios and TIOCGWINSZ, both of which zsh already needed.
Installed as /bin/edit as well as /bin/kilo, because somebody with a file to
fix will type `edit`.
**"wget doesn't work."** Not a userland problem, and BusyBox's wget would have
failed identically: nothing on this system could turn a name into an address,
so every program taking a hostname failed in its own way and each looked like
a separate bug. There is still no DNS client in SchOS and there does not need
to be -- musl has a complete resolver and it works over ordinary UDP sockets.
The one missing fact was which nameserver, which the kernel has had all along
from DHCP and never published. /proc/resolv.conf publishes it in resolv.conf's
exact syntax; cmd/resolvconf decides whether it becomes /etc/resolv.conf, and
will not overwrite a hand-written one without -f.
# wget -O - http://example.com/ | head -1
<!doctype html><html lang="en"><head><title>Example Domain</title>
**Something to host with.** cmd/httpd: GET and HEAD, static files, index.html
for a directory and a listing otherwise, a child per connection. No CGI, no
keep-alive, no TLS, no config file. The path check is the only
security-relevant code in it and it refuses rather than normalises -- no
component may be `..`, and percent-decoding happens first, because %2e%2e is
`..` and a check before decoding does not see it. Both forms answer 403.
And the disk is 100 MB, from 32. It is sparse, so the image is 21 MB on the
host. RAMDISK_BLOCKS went 6000 -> 8000 with it: the ramdisk was at 86% and the
next thing added would have broken `make ramroot`, which nobody would have
noticed until they used it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
ports/toybox already installed its applets as hard links to one multicall
binary, with a cp fallback. The links had never happened: the install is
stamped, the stamp predated the ln, and 236 separate 784 KB copies had been
sitting in sysroot/bin ever since. Forcing the install: 194 MB -> 13 MB.
Which immediately broke everything, in a way worth writing down.
**/bin/ping exists twice.** toybox has a ping applet and cmd/ping is a
first-party program that talks to lwIP directly. The install rule for cmd/
programs was `cp`, and cp onto an existing hard link writes *through* it -- so
copying cmd/ping over the applet of the same name replaced the contents of the
shared inode and all 236 links became ping. `cat /proc/uptime` answered
"ping: not a dotted-quad address".
Invisible until the links existed: while the applets were separate copies,
overwriting one affected exactly one. Fixed from both directions, because it
can be hit either way -- the cmd/ rule removes the target before copying, and
toybox skips names SchOS provides itself.
**The USB test image stopped fitting.** mkfs writes each hard link as its own
copy, so /bin expanded to over 60 MB on the way in. That image exists to prove
the USB driver can read one block, and bootcheck asserts on its 16 MB size, so
it excludes /bin rather than growing.
The ISO is the point of all this: 196 MB before, 62 MB now, and it boots to a
shell with RAMROOT=1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
licence-check is .PHONY, and I had put it on the left of the colon as a
prerequisite of the install rule. A phony prerequisite is always out of date,
so `make` reinstalled the whole of zsh every single time -- slow, and
eventually fatal: zsh's installfns.sh trips over the installfnsdir its own
previous run left behind, so the build failed outright with
install: cannot stat 'installfnsdir/_vcs_info_hooks'
It runs inside the rule now, which is what it always should have done, and the
stale directory is removed first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Porting zsh found four defects in the kernel and one in the filesystem. None
of them were about zsh; all of them had been there the whole time.
**exec threw the environment away.** kernel/fs/exec.c wrote a single zero for
envp with the comment "(no environment)" -- not yet, but ever, for every
process. musl has been passing a perfectly good envp to execve (Linux 59, which
SchOS answers) and it was never read. So $TERM was unset, $PATH was whatever
each program invented, and a variable exported in a shell did not survive the
exec meant to carry it. exec() takes an envp, sys_exec reads argument three,
and schinit manifests grew an Environment key. The userland two-argument
exec(path, argv) stays as a wrapper that passes null explicitly -- the raw stub
is execve now, so a short caller cannot leave %rdx holding a stray pointer.
**Which overflowed the kernel stack.** Three pointer arrays, ~1.8 KB, on a
4096-byte stack, under exec -> loaduvm -> readi -> SchFS -> btree. Pid 1 went
down on the first exec. The canary caught it and named the process, which is
what it is for. They live in one kalloc'd page now.
**waitpid returned a bare -1** for both "no children" and "interrupted". musl
reads [-4095,-1] as -errno, so userspace saw EPERM; zsh printed "wait failed:
operation not permitted" and abandoned the rest of /etc/zsh/zshrc. -ECHILD and
-EINTR now. Third time a bare -1 has cost real time -- on a system with no
permission model, EPERM should be read as "a syscall returned -1".
**NVMA was 64.** musl's allocator takes a mapping per size class and does not
release them eagerly, so compinit ran out of slots and got -1, which is EPERM,
which malloc turns into NULL, which zsh reports as "fatal error: out of memory"
-- on a machine with 1000 MB free. 256 slots, and -ENOMEM.
**And a small write to SchFS costs 58 ms.** 200 writes to one open file take
11.75 s; the same to /dev/null take 0.14 s. That is the design stated at the
top of kmount.c -- one transaction group per operation, and a commit is two
flushes and an uberblock -- not a bug. But compdump writes its digest with
thousands of separate prints, so it takes minutes and looks like a hang.
compinit -D, which skips the dump, finishes in 3.6 s. This one is a decision
rather than a fix, so it is written up and left: /etc/zsh/zshrc keeps compinit
switched off and explains exactly why, with the measurements.
zsh runs as /bin/zsh and is the console shell. ZLE works -- arrow keys recall
history through terminfo, $COLUMNS reads the real 160 because TIOCGWINSZ was
fixed, colour and prompt expansion work, startup is 0.14 s. mksh keeps /bin/sh,
so schinit's no-jobs fallback is a different program from the one that could
have failed.
Also fixed while here: AT_CLKTCK was the literal 100 and is USER_HZ; the
extract rule in four ports could move a source tree inside itself when make
re-ran it, which is how zsh's configure came to write modfile=zsh-5.9/... into
config.modules and fail three steps later.
schfs-check passes. Full writeup in docs/history/kernel.md section 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
zsh's configure will happily build without a terminal library -- the probe is
AC_SEARCH_LIBS, which is soft -- and produce a shell whose cursor cannot move
and whose arrow keys do nothing. ZLE is the reason to want zsh, so this comes
first.
MIT, so it combines with Apache-2.0 without conditions.
Built static, widec (which is what zsh looks for first, and the console's
UTF-8 decoder can already take), and --disable-database --with-fallbacks so
seven terminal descriptions are compiled into the library. SchOS has no
business shipping /usr/share/terminfo, and with the fallbacks the .a is
self-contained: verified that no terminfo path strings survive in the binary.
Nothing reaches sysroot/ -- a static library is not a file the OS ships -- so
`install` means into $(PREFIX). port.mk gains NCURSES beside MUSL for the
same reason MUSL is there.
tinfotest.c is the acceptance test, and asks the question the build cannot:
not "did it link" but "does terminfo work on a SchOS console". It reports
TIOCGWINSZ, initialises from the compiled-in fallback, and names every
capability ZLE binds. Anything MISSING there is a missing feature in the
shell later.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
fastfetch builds and runs on SchOS:
OS: SchOS 0.1.0 x86_64
Kernel: SchIFT 0.1.0
Shell: sh Terminal: schinit
CPU: AMD Ryzen 7 5700G
Memory: 11.94 MiB / 1016.93 MiB (1%)
Disk (/): 4.64 MiB / 31.25 MiB (15%) - schfs
Here because it is a good acceptance test, not because SchOS needs a fetch
tool. It is ~410 C files somebody else wrote against Linux rather than against
our bugs, and running it exercises procfs, per-process directories, uname,
statfs, getpwuid, terminal detection by walking ppid, and an ANSI renderer all
at once -- which is the shape docs/plan/roadmap.md asked for when the previous
full-stack test was removed.
patches/ is empty and should stay that way. When fastfetch cannot detect
something the answer is to give the system the thing it is looking for, because
every other program looks for the same things. Its first run reported four
misses and they are being worked in that order:
OS: Could not detect OS -> /etc/os-release, this commit
Local IP: getifaddrs() failed -> AF_NETLINK, next
Host: not set by O.E.M. -> DMI in /sys, later
Users: Unable to detect any users -> utmp, a userland convention
Four things the port needs that say nothing about SchOS, recorded in the
Makefile because each cost time to find:
* Every ENABLE_* off. All 29 files calling dlopen are optional detection
modules, and ENABLE_THREADS=OFF removes the single pthread_create -- which
matters, because clone() here does not share an address space.
* -isystem the compiler's own include dir. -nostdinc removes it too, and
cpuid.h is emitted by the compiler with no musl equivalent.
* Linux UAPI headers. musl ships none by design; a musl toolchain is normally
paired with a separate linux-headers install.
* libgcc on the link line, and libc twice -- libgcc pulls symbols back out of
it (__popcountdi2), and a static link resolves left to right.
The toolchain settings are a CMake toolchain file rather than command-line
flags because CMAKE_*_INIT variables are only honoured when the cache is first
created. Reconfiguring an existing build directory ignores them silently.
Also rebuilt musl: build/ports/musl/prefix/bin/musl-gcc had the pre-move path
/home/nexoniarz/Documents/SchOS baked in by configure. The tree does not use
that wrapper -- it links with explicit -I/-L and crt objects -- so nothing was
broken by it, but it was stale and would mislead anyone who found it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
lwIP 2.2.0, fetched by ports/lwip and compiled straight into the kernel.
Not written from scratch, and the reason is not the packet formats: it is
congestion control, retransmission, window management and twenty years of
corner cases that only show up against a real peer. Not FreeBSD's
netinet either, though the licence would have allowed it -- that stack is
not a library, it reaches into mbufs, epoch(9), netisr, VNET and uma(9),
so adopting it means porting FreeBSD's kernel infrastructure first. lwIP
was written for exactly this situation: a small kernel with a raw
Ethernet driver and a port layer of a few hundred lines.
It is kernel code, so nothing is installed into the sysroot and there is
no second build system: the top-level Makefile compiles the files it
wants against our port headers. The file list is explicit rather than a
wildcard, because lwIP ships IPv6, 6LoWPAN, PPP, SLIP, DNS-SD and a dozen
apps in the same tree, and building what is not configured yields either
dead code or a link error naming a port function nobody meant to write.
Its sources are unmodified and its warnings are silenced for its files
only -- patching them would mean maintaining a fork.
The port layer is ours: a lock, a clock, the netif that hands frames to
e1000, lwipopts.h, and a shim/ directory of tiny headers so lwIP's
<string.h> and friends resolve to the kernel's functions instead of the
host's, whose prototypes disagree and would otherwise be found on the
default include path.
NO_SYS=1, so lwIP has no threads of its own and one kernel thread feeds
it. A thread rather than the interrupt handler, because a call into lwIP
can reach the driver, the driver can sleep, and an interrupt handler
cannot -- so the interrupt still only clears the cause register and the
thread polls. kthread_create is new and generally useful: a process with
a kernel-only page directory, no files, no cwd, no parent, dispatched
from forkret, which can therefore sleep and be preempted like anything
else. DHCP obtains an address at boot and the console says which one.
AF_INET went in exactly where phase 2 said it would. socknew dispatches
on the domain, kernel/net/inet.c implements the internet side on lwIP's
raw API, and not one syscall changed shape: bind, connect, listen,
accept, read and write reach the same descriptors they always did. TCP
and UDP both work. cmd/nettest opens a connection, writes, and reads the
answer back.
Two things are owed and named here so they are not discovered later:
recvfrom does not yet report the source address, and the socket paths
poll on the timer tick instead of sleeping on the socket. Both are
follow-ups, not redesigns.
NOTICE carries lwIP's BSD-3 text. Clause 3 is no-endorsement, not the
old advertising clause, so it sits with Apache-2.0 without difficulty.
Verified: the tree builds clean, and the headless boot check passes with
27 [ok] and no failures with the stack compiled in and the thread
running -- which is the part that would break first if netinit deadlocked
or the thread starved anything. An end-to-end connection test belongs in
that suite and is the next commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tools/windows/ .ps1 (pwsh, so usable from Linux and macOS too)
tools/posix/ .sh
tools/python/ .py
tools/ .c and .pl
The .c and .pl files stay at the top level because they are not
host-specific: the build compiles or invokes them on every platform.
Call sites updated in the Makefile, ports/mk/port.mk, docs and the
project memory. The one thing that does not survive a move on its own is
a script deriving the repo root from its own location -- all nine
PowerShell tools used Split-Path $PSScriptRoot -Parent, and memory-sync.sh
used dirname/.., both of which quietly become tools/ one level deeper.
Two levels now.
Two pre-existing Windows failures surfaced while verifying, both real and
neither caused by the move:
- schfs_fsck.c did not compile under mingw at all: --extract uses
POSIX mkdir(path,mode), symlink() and link(), none of which mingw
has. That took the whole tool with it, and `make schfs-check` with
it. mkdir maps to _mkdir; symlink and link fail loudly rather than
writing a regular file in their place, because an extracted tree
that differs from the image while reporting success defeats the
point of --extract.
- the round-trip test then failed at `ln -s`, since Git Bash cannot
create symlinks without developer mode. It skips with a message now.
Not a fudge: mkschfs's mingw build cannot see a symlink either, so
the test would have compared a tree with none against an image with
none and called it a pass -- coverage reporting itself as present
while being absent.
Verified on Windows: build clean, all seven bootcheck suites green,
make schfs-check passes.
Developed from both Windows and Linux, so nothing in the build should
care which host it runs on. It did, in four places.
Toolchain: one path instead of two. clang is a cross-compiler by
construction, so a single LLVM install targets bare-metal ELF, 16/32-bit
code below 1 MB, and Linux/musl userland from any host -- which is how
the Windows build already worked. The GNU branch beside it had drifted
into passing --no-warn-rwx-segments, a hard error on lld, which is the
default linker on both FreeBSD and macOS; and it required a brewed
x86_64-elf-gcc before anything built on a Mac at all. GNU is kept as a
fallback (USE_GNU=1) but clang+lld is chosen whenever both are present.
Host tools use the host's own cc; only mingw needs naming explicitly.
tools/mkschfs.c included <sys/sysmacros.h> unconditionally on non-mingw
hosts. That header is glibc/musl only -- macOS and the BSDs have neither
the header nor a need for it, since major()/minor() come from
<sys/types.h> there. Now guarded on __linux__.
Ports verified tarballs with `sha256sum -c -`, which does not exist on
macOS (shasum), FreeBSD/NetBSD (sha256), or OpenBSD (cksum -a sha256),
and each spells its check mode differently. tools/sha256check.sh picks
whichever of the five the host has and compares digests itself.
nproc is coreutils, so absent on macOS and BSD; falls through to sysctl
-n hw.ncpu before defaulting to 4.
The PowerShell tools now run under pwsh on any OS: QEMU is found on PATH
(the scoop location is a Windows-only fallback), and path literals use
forward slashes -- Join-Path $root 'build\x' produces a *filename
containing a backslash* on a Unix host. The backslash sweep is limited
to known directory prefixes on purpose: bootcheck sends
printf 'caf\303\251' to the guest to test UTF-8, and a blanket
substitution would have quietly changed what that asserts.
Also fixes six tools left booting build/fs.img, which stopped being
bootable when SchFS became the root filesystem: crashshot, dispshot,
fbshot, splashshot, textshot and ttyshot all take build/schfs.img now,
and the -SchFS switches that used to select it are gone.
Verified on Windows: full build, and bootcheck green. The Linux, macOS
and BSD paths are written from their documented behaviour and are NOT
executed here -- this machine has no WSL and no second host.
Removes the "-unix" element and with it any trademark exposure to The
Open Group.
The split follows the ecosystem layout: SchOS is the system, and the
kernel has its own name. Kernel-identity strings are the only places
that take SchIFT -- uname(2) sysname, /proc/version, and the two licence
sentences that say "kernel" outright. That is what every Unix does here:
`uname -s` on Linux reports the kernel, not the distribution. Everything
a user reads as the product -- panic screen, boot menu, error prefixes,
docs, LICENSE, NOTICE -- is SchOS.
schunix.img is schos.img, the logo assets and the musl TLS patch are
renamed, and __SCHUNIX__ is __SCHOS__ (the define and the patch that
consumes it moved together).
Two things the rename broke, both found by the bootcheck suite:
- shmtest wrote "SCHUNIX-MMAP-OK" with a hardcoded length of 15 and
compared it back by hand-counted offsets (m[3]=='U', m[14]=='K').
The shorter marker left the write running two bytes past the literal
and the compare reading past the data, failing a file-backed mmap
that was working correctly. Now written and compared through one
constant with sizeof.
- bootcheck asserted a 31-byte file after appending to
"hello-from-schunix"; the shorter string makes it 29.
Both are the same mistake: a string literal whose length is written out
somewhere else. Full suite green -- base, Tty, Ps, Sock, Fb, Usb, Evdev.
An x86-64 operating system built from xv6, now largely rewritten: SMP,
a copy-on-write root filesystem (SchFS), PCI/USB/virtio drivers, a TTF
rasteriser, and a Linux-numbered syscall ABI.
No history before this point -- the tree was never under version control.
This commit exists so the rename that follows has something to undo to.
Nexoniarz