Commit graph SchOS/kernel
Author SHA1 Message Date
Nexoniarz
393bf674f5 NetSurf: what GTK's images and libcurl's resolver needed
NetSurf (GTK 3) now starts under X and loads pages over HTTPS. Every icon
GTK draws goes gdk-pixbuf -> glycin -> a loader process -> a sealed memfd
-> GMappedFile, and each hop found something missing:

- mkaroot runs Alpine's package triggers in the root under bubblewrap.
  --no-scripts had skipped them, so there was no MIME cache, pixbuf loader
  list, compiled GSettings schemas, GIO module cache, font cache or busybox
  applet links.
- /usr/bin/bwrap is a stand-in that parses bubblewrap's options and runs
  the program with no sandbox: glycin starts its loaders only through bwrap,
  and real bwrap needs namespaces. schos-base replaces bubblewrap for it.
- ioctl: FIOCLEX, FIONCLEX, FIONBIO and FIOASYNC on any descriptor (Rust's
  std uses them; pipes and sockets answered ENOTTY).
- capget/capset: root has every capability, others none.
- socketpair accepts SOCK_DGRAM and SOCK_SEQPACKET, carried as a stream.
- waitpid and setpgid match children against the thread group leader, so a
  thread can wait for the process it spawned (ECHILD before).
- memfd: fstat reports the ftruncate length, not the page-rounded one; file
  seals (F_ADD_SEALS, F_GET_SEALS, MFD_ALLOW_SEALING) are recorded and
  enforced on later writes, truncation and writable mappings; MFD_CLOEXEC
  is honoured; read-only MAP_PRIVATE maps are allowed.
- recvfrom honours O_NONBLOCK and MSG_DONTWAIT. musl's resolver reads its
  socket until EAGAIN and blocked for ever after the first DNS answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 04:10:26 +01:00
Nexoniarz
f4d7f9100d Threads
A thread is a struct proc that shares three counted objects with others of
its group: the address space (struct mm: page tables, heap top, VMAs and a
lock for changing them), the descriptor table, and the signal dispositions.
clone() shares what its flags say and fork() copies.

- clone(): pthread_create's shape (VM, FILES, SIGHAND, THREAD, SETTLS,
  PARENT_SETTID, CHILD_CLEARTID) and posix_spawn's (VM, VFORK, a stack).
- futex: WAIT, WAIT_BITSET with timeouts, WAKE, REQUEUE, CMP_REQUEUE, keyed
  by the word's kernel address so shared pages meet. set_tid_address keeps
  the word that exit zeroes and wakes, for pthread_join.
- exit ends a thread; exit_group, a fatal signal, or the leader exiting ends
  them all. reapd frees exited threads; the leader is reaped by its parent
  once the rest are gone. getpid is the group, gettid the thread.
- Stops and resumes reach every thread; other process signals the leader.
- exec kills the other threads and unshares the dispositions.
- No TLB shootdown, so pages unmapped from a shared address space are freed
  only once every CPU has reloaded CR3 or is idle.
- procfs: /proc lists processes, task/ lists threads, comm, Threads:, Tgid.
- The console honours VMIN 0 / VTIME (btop's input drain hung on it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 03:03:50 +01:00
Nexoniarz
2372806862 Job control, function keys, and real sleeps
- SIGSTOP/SIGTSTP/SIGTTIN/SIGTTOU stop, SIGCONT resumes; the parent gets
  SIGCHLD and wait4 reports WUNTRACED/WCONTINUED; ps shows T. Ctrl-Z works,
  and mc no longer hangs waiting for its subshell to stop.
- Blocking reads (console, pty, pipe) and sleeps take a pending stop and
  carry on after SIGCONT; a caught signal without SA_RESTART ends them
  with EINTR.
- nanosleep reads tv_nsec (sleep 0.5 used to return at once), writes the
  remainder, and clock_nanosleep understands TIMER_ABSTIME.
- rt_sigaction uses the k_sigaction layout for Linux-ABI processes: the old
  action's restorer was written 112 bytes past the caller's struct. Flags
  are kept; SIGKILL and SIGSTOP cannot be caught.
- F1..F12 reach programs, as the Linux console sends them (PS/2, USB, I2C).
- TIOCLINUX subcode 6 answers "no modifiers" instead of echoing the 6 back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 02:37:45 +01:00
Nexoniarz
fd9d392416 Pseudo-terminals, and what tmux and htop needed around them
- /dev/ptmx and /dev/pts/N: Unix98 pairs with a full line discipline,
  window sizes, job control and hangup (kernel/dev/pty.c).
- The console speaks scroll regions, insert/delete line, SU/SD, RI/IND/NEL,
  DECSC/DECRC; swallows charset designations and private CSI sequences;
  and LF only returns the carriage under ONLCR.
- setsid() drops the controlling terminal, so a pane's pty becomes its ctty.
- pipe2() honours O_NONBLOCK and O_CLOEXEC (libevent's signal pipe).
- File modes are stored and enforced: chmod, fchmod(at), umask, creation
  modes; /tmp is 1777.
- /proc/stat has real user/system/idle; /proc/<pid>/task; kthread flag.
- SO_PEERCRED, IOV_MAX 1024.
- /proc/strace: name a program and its syscalls go to the kernel log.
- World gains htop, btop, tmux, mc, ncdu, fastfetch, games, X toys, NetSurf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 02:11:10 +01:00
Nexoniarz
c65bfa3fa3 boot: driver chatter goes to the log; the console clears its margins
ACPI, DMI, SMP, xHCI, USB, I2C, NVMe, r8169 and ramdisk progress lines are
kinfo now: in dmesg and on a verbose boot, off the screen of a working
machine. Failures stay cprintf.

vt_repaint also paints the strip right of the last column and below the
last row, which kept the bottom of the X desktop after Ctrl+Alt+F-key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 01:53:46 +02:00
Nexoniarz
f6c4b6377f GMMOS: USB keyboards on xHCI
The third micro driver: xhci.c with everything but finding a keyboard and
reading it taken out. It does not take over the live driver's state --
the kernel may have died inside that driver -- so each controller is
halted, reset and enumerated from nothing. Of the live driver it uses
the register mapping and the scratchpad pages the controller was lent
(DCBAA entry 0, read before the reset).

Root ports, and USB 2 hubs up to two deep (power, reset, route string,
transaction translator for a slow device behind a high-speed hub). A
boot-protocol keyboard interface gets Set Configuration, Set Protocol
boot, Set Idle, one interrupt IN endpoint and a Normal TRB, re-queued
after every report. No interrupts, no hot-plug, nothing that is not a
keyboard or a hub. A static 256 KB pool instead of the DMA allocator.

Checked under QEMU with the PS/2 controller switched off (i8042=off): a
USB keyboard on a root port, and one behind a hub, both type into the
GMMOS terminal. QEMU's hub is full speed, so the transaction-translator
path is not exercised.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 01:23:14 +02:00
Nexoniarz
c6d90e69fa GMMOS: I2C keyboards, and every keyboard at once
The second micro driver: i2c_hid.c with everything but reading a
keyboard's input reports taken out. No ACPI -- AML cannot run on a dead
kernel -- so the live driver hands over what it found at boot
(i2c_hid_keyboards): the DesignWare controller, the address, the report
size, the AMD GPIO pin if there is one, and the parsed report descriptor.
No power commands, no reset, a read-only transfer.

usb_hid.c's report-protocol rewrite is split out as hid_boot_report(),
which does nothing but rewrite, and kernel/gmmos/hidkeys.c turns boot
reports into keys for this and the USB micro drivers to come.

GMMOS now probes every micro driver and polls all that answer.

Not run: QEMU has no DesignWare I2C, and the laptops at hand have PS/2
keyboards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 01:15:50 +02:00
Nexoniarz
cb9b06d46f GMMOS: the Guru Meditation Micro OS, alive under the PSOGM
A panic no longer ends in a picture. The report is drawn as before, and
then GMMOS runs under it: the real kernel stays dead -- the other
processors are halted, and nothing of it is called that locks, sleeps,
allocates or waits for an interrupt -- and GMMOS keeps its own clock
(the TSC), its own keyboard (micro drivers, polled), and the screen.

On the report: a live status line counting down the three-minute
power-off, which any key cancels, and the keys.

  Space  the last 300 lines of this boot's log as QR codes, a page per
         symbol, numbered; + and - trade module size for capacity
  L      the log on screen, scrolled
  T      a terminal: why, log, grep, ps, cpus, pci, mem (page tables
         walked first, so a bad address cannot fault), uptime, restart,
         poweroff
  R R    restart; P P power off -- twice, so a stray key does neither

The first micro driver is PS/2, polled: kbd.c with everything but the
keys taken out. The probe asks the keyboard to echo, so a controller
with nothing behind it is not offered as one. Polling also sidesteps the
IRQ 1 polarity quirk, which is about interrupt routing.

A panic inside GMMOS (or a second one in the report) is reported and
then only blinks; it does not re-enter GMMOS.

Checked under QEMU: every screen; two QR pages read off a screenshot
with zbar decode to the log text exactly; restart reboots; left alone it
still powers off at three minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 01:11:04 +02:00
Nexoniarz
38c89b1a9a panic stops every other processor
A panic used to stop only the processor it happened on. The rest carried
on with a dead kernel: on the ASUS laptop the report flashed up and was
drawn over, and the fans stayed loud until the power button was held.

Now the panicking processor sends every other one an NMI, which arrives
even with interrupts off, and trap() halts a processor that takes one
while panic_stop is set -- first thing, before the canary check, since
that stack may be what broke. NMI gets its own per-CPU stack (IST 1):
for a few instructions at the top of syscall_entry the live stack is the
user's, at ring 0, and a fault pushing the frame there would be a double
fault on the same stack and a reset instead of a report.

A second processor that panics while one is already reporting halts
instead of drawing over it; the owner's own nested panic still gets
through.

Checked under QEMU with -smp 4: after a sysrq crash CPUs 1-3 sit in hlt
with IF clear and CPU 0 is in the report's blink loop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 00:50:50 +02:00
Nexoniarz
d8e8ae604f qr: every version, 1 to 40
PSOGM mode will put the kernel log on screen as QR codes, a page per
symbol, and version 6 held 134 bytes. Version 40 at level L holds 2,953.

What grew: blocks of two lengths and the interleave that steps around
them, the alignment-pattern grid, the version-information blocks from
version 7, the 16-bit character count from version 10. The per-version
table is down to the two columns that are genuinely transcription (EC
codewords per block, block count); total codewords come from the
geometry.

Checked against segno module for module at every version (exact-fill
payloads), and 120 symbols across all 40 versions decode with zbar.
OpenCV's detector cannot find symbols above about version 24 at all,
segno's included, so qrcheck reports those as the detector's limit.

The crash card stays a version-6 symbol (QR_CARD_BYTES), so a phone still
reads it from across the desk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 00:43:36 +02:00
Nexoniarz
5ea03c4cf5 i2c-hid: the touchpad crash was a printf format, not the driver
The PSOGM on the ASUS TUF A15 was a page fault in cprintf, from probe(),
in the touchpad driver's own success message. cprintf had no field widths:
"%04x" printed literally and consumed no argument, so the "%s" after it
was handed the product ID as a string pointer. "%lx" misprinted the same
way -- "0x4275838976x" for 0xFEDC2000 -- on the line before.

The driver's formats are fixed, and cprintf now steps over flags and
widths and takes l/ll/z as 64-bit, so a directive it does not render can
no longer shift every argument after it. Until now any %ld or %lu printed
the number followed by a stray letter.

The touchpad on/off entry in the boot menu has served its purpose and is
gone; the verbose step trace stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 00:28:23 +02:00
Nexoniarz
b15fb2fdaa i2c-hid on a stack of its own; PSOGM powers off after three minutes
The touchpad driver crashed the ASUS TUF A15 at boot. Its thread runs the
firmware's AML -- _DSM, _PS0, _CRS -- through uACPI at -O0, on an 8 KB
kernel stack, and kernel stacks are one array: running off the end lands
in the next thread's stack, and the machine dies a moment later in some
other code. The thread now runs on a 64 KB stack of its own, through
SCFF's call_on_stack().

And a crashed laptop sat with its fans up until somebody held the power
button, since the other processors keep running. On an ordinary boot the
PSOGM now says it will power off, and does, after three minutes: SLP_TYP
from \_S5, read at boot, written straight to PM1 control with no AML, which
could sleep or wait on a lock a dead processor holds. A verbose boot keeps
the screen up for ever.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-25 00:21:39 +02:00
Nexoniarz
c79b1d6d6d boot menu: turn the touchpad driver off, and trace its steps
The ASUS TUF A15 freezes at the default resolution shortly after the
loader hands over -- screen dark, Caps Lock dead -- and it is the only
machine here with an I2C-HID touchpad, which the driver added in the same
release now probes at boot. "Touchpad driver: on/off" in the F8 menu
answers whether it is the cause on the machine itself, through a new
SCHBOOT_OFF_FLAGS byte the kernel checks before starting the driver.

A verbose boot now prints each step of the probe before taking it, so if it
is the cause, the last line on the screen says which step.

Also: S in the menu wrapped at five entries whatever the count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 23:33:25 +02:00
Nexoniarz
d2c70c9129 display: confirm a resolution change, and size the console font to the screen
On the Ryzen 4600G desktop and the ASUS TUF A15, AMD's GOP "sets" the
smaller modes without leaving the panel's native timing and reports the
mode's own row length. Every check the loader made passed, and the screen
showed three or four squashed copies side by side (1920/640, 2560/640).
Nothing the firmware reports can tell such a mode from a real one.

So a mode change now asks "Keep this resolution?" in the new mode and keeps
it only on Enter within ten seconds. On a mode that does not really work the
question cannot be read, and the old mode comes back by itself.

And the reason to want a smaller mode was the text: the console font was a
fixed 16 px, unreadable on a 1440p monitor. It now scales to about 45 rows,
16 to 40 px per em, on whatever mode the firmware booted in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 23:11:22 +02:00
Nexoniarz
65617710fa ldls: the top-2 GB window Linux modules link against
A Linux module is built with -mcmodel=kernel and can only reach addresses
in the top 2 GB, where this kernel is not linked. So the kernel image is
mapped a second time at LDLS_KALIAS, with a vmemmap area and a module area
beside it, in one PDPT shared by every address space. Nothing loads a
module yet; this is the address-space groundwork, set up at every boot,
and include/ldls/ is the interface the loader will use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 22:12:40 +02:00
Nexoniarz
e43f56c619 kernel: laptop touchpads over I2C-HID
A laptop's touchpad is on the chipset's I2C bus, found only through ACPI
(PNP0C50 / ACPI0C50), and SchOS had no I2C at all, so the touchpad of
every recent laptop did nothing.

kernel/acpi/i2c_hid.c drives the DesignWare I2C controller every x86
chipset uses (AMDI0010 on Ryzen) by polling, speaks HID over I2C to the
device the _DSM points at, and feeds the reports through the USB HID
parser. On AMD the interrupt pin's level is read every tick, so a report
is read exactly when one is waiting; elsewhere the input register is polled.

The touchpad is driven in mouse mode, which it starts in after a reset:
motion, clicks and taps become /dev/input/event1 like any mouse. The HID
parser now prefers a descriptor's relative pointer over an absolute one
for this, and takes descriptors from transports other than USB.

Untested on hardware: QEMU emulates no I2C touchpad.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 21:54:33 +02:00
Nexoniarz
4a7165c951 X gets a VT: keyboard, terminal switching, and a way out
Xorg uses VTs only when stat("/dev/tty0") reports Linux's tty major 4.
SchOS's terminals sit under major 1, so X ran on no VT at all: the console
kept drawing over it, Ctrl+Alt+F-keys did nothing, and nothing ended the
session. stat now reports terminals under Linux's numbers.

startx picks the VT only when uname says Linux, so root's xserverrc passes
the current one itself. An xorg.conf.d file names X's input devices, since
Alpine's Xorg looks them up through udev and SchOS has none yet.

Alt+F1..F6 and Ctrl+Alt+F1..F6 now switch terminals as on Linux, alongside
the settable chord. USB keyboards gain arrows, Home/End/PgUp/PgDn/Ins/Del
and the switch keys, which the PS/2 path always had.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 21:45:50 +02:00
Nexoniarz
df1394eaa3 SCFF: the live root ships compressed, with Xorg and Window Maker
SCFF, the SchOS Compressed File Format (include/fs/scff.h), is a block
image cut into 128 KB chunks, each its own zstd frame, behind an index.
tools/mkscff writes it from fs-ram.img on every core (-19, 11 s) and
checks each chunk decompresses to itself. The kernel carries the .scff
as its ramdisk and unpacks a chunk the first time a block in it is
read; zero blocks share one page. The kernel and the ISO are the
compressed size, and the root costs memory only for what is touched.
See docs/architecture/scff.md.

zstd 1.5.7 comes from ports/zstd like uACPI: the decoder is compiled
into the kernel at -O2, with a static context and a 64 KB stack of its
own, and mkscff builds the whole library from the same release.

world now has xorg-server, the fbdev and evdev drivers, xinit, Window
Maker, xterm and DejaVu, and /root/.xinitrc runs wmaker. That root is
358 MB, 102 MB as SCFF.

The ISO no longer carries the root a third time as an ISO 9660 tree,
which nothing read. schos.iso: 1389 MB -> 318 MB. El Torito keeps its
own copy of the EFI image, the layout real machines have booted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 21:17:31 +02:00
Nexoniarz
1ba93dde2c kernel: syscalls get the whole kernel stack
syscall_entry put the frame at kstack + 4096 - 16, from when a kernel
stack was one page. It has been two since, so every syscall ran in the
bottom 4 KB while traps used the top, and a deep enough read -- startx
exec'ing through the new compressed ramdisk -- ran over the canary at
kstack[0]. The frame now sits at the top of KSTACKSIZE, where allocproc
and the TSS already put it, and exec's user-rsp slot moves with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 21:17:02 +02:00
Nexoniarz
8351e0323b kernel: HTTPS through wget, half-close, and /proc/<pid>/fd
busybox wget hands TLS to ssl_client over a socketpair and then does
shutdown(SHUT_WR). That cut the AF_UNIX link both ways, so ssl_client's
reply was a SIGPIPE and every https:// download hung. SHUT_WR is now a
half-close: the peer reads EOF and can still answer. TCP sockets get
shutdown() at all (tcp_shutdown for the FIN); it used to do nothing.

- exec keeps ignored signals and the signal mask, as execve(2) says.
- /proc/<pid>/fd: a link per descriptor -- the path it was opened by,
  socket:[n], pipe:[n], anon_inode:[eventpoll]. /proc/self too. The
  global inode bit moves above pid << 8; at 0x10000 it was /proc/256.
- fstat on a socket or pipe answers S_IFSOCK / S_IFIFO instead of EINVAL.
- poll, select and epoll wake as soon as a pipe, socket or input device
  changes, not on the next tick.
- Pipes are 64 KB of lazily allocated pages, copied in blocks, with
  PIPE_BUF atomicity: 300 MB through `yes | head | wc` in 7 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 20:28:03 +02:00
Nexoniarz
906318f2b6 X on the console: /dev/tty0, a palette, and startx that works
Xorg finds its VT through /dev/tty0, and without one ran on none: the
console went on drawing into the framebuffer over the X screen. tty0 is
the terminal on screen, as on Linux. FBIOPUTCMAP succeeds, as it does on
a true-colour Linux framebuffer, instead of 256 logged errors per start.

schos-base gives root a .xserverrc with -dumbSched, Xorg's switch for
input on the main loop: SchOS has no threads, and Xorg crashed in
pthread_join on its input thread when the first client left. It adds
the hostname to /etc/hosts, so xauth can resolve schos:0.

zsh loses PROMPT_SP: on a serial terminal narrower than the console its
padding wrapped and left a # above every prompt. bootcheck knows zsh's
prompt and matches without colour codes, so busybox's ls reads as text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 19:44:10 +02:00
Nexoniarz
a4ed556543 kernel: timers that fire, raw ping, and the sockets X needs
alarm() and setitimer() stored a countdown nothing counted down, so no
timer has ever fired; setitimer and getitimer were also registered with
no Linux number. alarmd now posts SIGALRM on the tick, while any process
has a timer armed. busybox's ping, /bin/ping on the Alpine root, sends
every request after the first from its SIGALRM handler.

It also opens SOCK_RAW, which was EPROTOTYPE. A raw ICMP socket is now a
ping socket whose reads start at the IP header. And a blocked socket
call gives up for a signal with a handler, not only for a kill: the
alarm woke recvfrom, and the wait put it straight back to sleep.

For Xorg: AF_UNIX abstract names, which read as the empty path and so as
EADDRINUSE ("server already running"); and getsockname and getpeername,
which did not exist, so libxcb sent no cookie and the server refused
every client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 19:44:10 +02:00
Nexoniarz
d0941780f5 ramdisk: the live root grows into half of memory
The live root was a fixed 96 MB image in the kernel: 60 MB of zeros on
the ISO, twice, and room for 60 MB of packages whatever the machine had.

Now the image is only what the root holds, plus slack, measured by
building it once large. Past its end the ramdisk goes on to half of the
memory free at mount, as pages taken the first time each block is
written; an unwritten block reads as zeros and costs nothing. SchFS
keeps no free-space map on disk, so growing a volume is a larger total
before open builds the map: grow_to, set for the ramdisk root.

8 GB machine: a 4 GB root, Xorg installed into it in 77 s with 7.6 GB
still free. The ISO is 172 MB instead of 344.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 17:20:43 +02:00
Nexoniarz
2c5a278d56 net: downloads at tens of megabits, not one
Measured against the host fetching the same 67 MB from Alpine's CDN at
11.5 MB/s, SchOS managed about 120 KB/s. Three causes:

- The receive window reopened when data entered the socket buffer,
  before the reader drained it; lwIP then refused segments and the
  sender waited on retransmission timers. It reopens on read now.
- The e1000 receive ring held 16 frames, so a full window arriving back
  to back lost everything past the sixteenth. 256 now.
- Without window scaling the window stops at 64 KB, one window per
  round trip. Scaling on, 256 KB window, 512 KB socket ring (a power of
  two, which the free-running indices need), copied in blocks.

Now 5.8 MB/s for the same file, and apk update in about two seconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 15:28:32 +02:00
Nexoniarz
2cfa3f0bf2 kernel: chroot, #! scripts, rename and mmap as apk needs them
Found by running Alpine's apk on SchOS, one failure at a time:

- chroot(2) was a stub returning EPERM. Each process now has a root:
  where an absolute path starts and where .. stops. Mounts remember
  whose root they were made under, and a walk leaves a mount by the way
  it came in, so one filesystem can be mounted twice.
- devtmpfs and proc mounted only on /dev and /proc, whatever the target;
  a chroot got "already mounted".
- mmap never reused freed address space, so musl's malloc used up the
  256 mapping records while 780 MB were free. A mergeable mapping now
  goes straight after an anonymous one whose end has room.
- unlinkat returned -1 (EPERM) where apk expects ENOENT, and renameat
  was an old copy of rename: no replacing the target, and v1 dirents
  written into SchFS. Both now share rename's implementation.
- execve runs #! scripts, and says ENOEXEC for what it cannot run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 15:28:32 +02:00
Nexoniarz
6f3175d8a5 flock, fcntl record locks
There were none: fcntl answered F_SETLK with EINVAL and flock was not a
system call. Alpine's apk, which runs unmodified on SchOS through
Alpine's own musl, locks its database before reading it and stopped
there.

kernel/fs/lock.c keeps one table of advisory locks keyed by (dev, inum):
flock() for the whole file, owned by the open file description;
F_SETLK/F_SETLKW/F_GETLK for byte ranges, owned by the process and
released when it closes any descriptor on the file or exits; and the
F_OFD_* forms, ranges owned by the description. flock and record locks
do not see each other, as on Linux. Waiters sleep until any release and
are interrupted by a signal.

cmd/locktest checks each between a parent and a child, including range
splitting, F_GETLK's report, and a blocked waiter being woken.

shell.nix gains pkg-config, meson, ninja, m4, bison, flex and xz, for
the X ports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 12:22:36 +02:00
Nexoniarz
b1323e193a input, fb, vt, sysfs: the kernel calls Xorg makes
Xorg on a framebuffer -- the fbdev video driver, the evdev input driver,
libpciaccess -- asks the kernel a fixed set of questions before it draws
anything, and gives up on the first wrong answer. These are those
answers.

evdev: EVIOCGVERSION, GID, GNAME, GPHYS, GUNIQ, GPROP, GBIT, GKEY, GABS,
GREP and the accepted setters. Xorg and libinput decide what a device
*is* from the capability bitmaps; /dev/input/eventN answered only
EVIOCGRAB, which reads as no device at all. Each device's bitmaps are
its defaults plus whatever it has reported, so a USB tablet makes the
pointer absolute.

fbdev: FBIOPUT_VSCREENINFO succeeds for the mode already set, as on
Linux -- the fbdev driver sets it every time it starts -- and still
refuses any other. FBIOPAN_DISPLAY to the origin; FBIOBLANK.

The console: KDSETMODE(KD_GRAPHICS) stops it drawing on a terminal,
KDSKBMODE(K_OFF) stops keys reaching its tty, and VT_PROCESS now does
the switch handshake: leaving the terminal sends relsig and waits for
VT_RELDISP, arriving sends acqsig. A terminal whose owner has died goes
back to the kernel whole, so a crashed X cannot leave the console dead.

/sys: boot_vga, a writable enable, and graphics/fb0 under the display
device, which is how the fbdev driver pairs a card with /dev/fb0.

ports/linux-headers: <linux/*.h> into musl's prefix, from sabotage-
linux's kernel-headers 4.19.88-2. Its checksum is recorded from the
first download; upstream publishes none.

cmd/xready makes each call and checks the answer, including the switch
handshake on tty2 and an owner that dies holding it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 12:06:04 +02:00
Nexoniarz
132a2981b1 uefi, vm: 2 MB pages where the CPU has no 1 GB ones
Every page table SchOS built -- the loader's, which the kernel starts on
and the other CPUs start through, and the kernel's own direct map and
device window -- mapped memory in 1 GB pages. Intel's desktop CPUs
before Haswell do not have them, and there the PS bit in a PDPT entry is
reserved: the first access after the switch is a page fault that cannot
be handled either. An HP all-in-one with an Ivy Bridge i3 showed "Placing
and starting the kernel" and reset.

Both now check CPUID 0x80000001 EDX bit 26. Without it the loader maps
the low 4 GB through four page directories of 2 MB pages, placed below
the kernel where nothing reuses them, and keeps the staged kernel
segments below 4 GB; the kernel gives each gigabyte of its direct map
and of ioremap() a page directory of its own.

Neither KVM nor QEMU's emulator enforces the reserved bit, so the fault
itself could not be reproduced here. With the fallback forced on, the
kernel boots at 2 and 8 GB, both CPUs start, and evtest passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 11:45:18 +02:00
Nexoniarz
8c426b2077 irq: turn a PS/2 line over when its device waits and nothing comes
An ASUS TUF A15 booted with its internal keyboard dead and a USB one
fine. These AMD laptops' firmware describes IRQ 1's polarity wrongly --
Linux keeps a quirk list -- and an edge line programmed the wrong way
up never fires: the controller holds its byte, and only the interrupt
that never comes would read it.

irq_watch() lets a driver say how to ask its device whether something
is waiting. The poller asks every tick; 50 ms of waiting with nothing
delivered turns the line's polarity over and drains the device, which
re-arms the edge, and if it still says nothing the line is polled.
Both are announced. The keyboard and the PS/2 mouse are watched.

ISA lines now also take their mode from the MADT's override entries,
which were parsed and never read.

Tested in QEMU, which ignores polarity on edge lines, with the keyboard
line masked: the first key turns it over and typing works; masked again
after the flip, it is polled and typing still works.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 11:10:29 +02:00
Nexoniarz
a4fbdb7437 epoll, eventfd, timerfd, signalfd
The set libwayland's event loop is made of, in kernel/sys/sys_event.c:
four kinds of descriptor with no inode, FD_EVENT, that read() drains
and poll() reports on. They were table entries returning ENOSYS that
read as done.

epoll registers (descriptor, file) pairs, so a closed descriptor
leaves the set; level- and edge-triggered, EPOLLONESHOT. Waiting is
poll()'s, a tick at a time. The old eventfd and signalfd numbers (284,
282) were aliased to the new calls and read flags from a garbage
register; they are their own entries now, with epoll_create and
epoll_pwait.

cmd/evtest checks each, then all four in one set. The plans are
brought up to date: everything Wayland was waiting on is built except
setsockopt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 10:59:52 +02:00
Nexoniarz
43305768cb exec, mmap, musl: dynamic linking
A program that names an interpreter in PT_INTERP gets it loaded too, at
the bottom of the mmap area, and started instead -- with AT_PHDR,
AT_PHENT, AT_PHNUM, AT_BASE and AT_ENTRY to find the program by. ET_DYN
images are placed at a base: the interpreter there, a PIE program where
a fixed one would go. Static musl programs get AT_PHDR now as well,
which is how musl finds their TLS template.

mmap(MAP_FIXED) of a file handed back zeroed pages whatever the
descriptor said, and kept no record of them, so fork() did not copy
them. A dynamic linker maps every library segment after the first that
way. Fixed mappings now read the file and are recorded wherever no
mapping already covers them.

musl is built shared as well as static, and libc.so ships as
/lib/ld-musl-x86_64.so.1 -- libc and loader in one file. Every port
still links -static on purpose.

cmd/dynhello is the test: against libc.so and a library of its own,
checking TLS, the heap, and a fork whose child still has both.
mprotect is accepted and not enforced yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 10:59:52 +02:00
Nexoniarz
6bfb367090 signals: a blocked signal stays pending instead of killing
sigsend() ended a process for a default-fatal signal (SIGUSR1, SIGTERM,
SIGINT...) on arrival, whatever its signal mask said. Blocking is
exactly the promise that it will not: the signal waits until it is
unblocked, or until the program takes it some other way -- which is
what signalfd is, and how this was found.

A blocked signal is now only made pending. If it is later unblocked
while still pending, deliver_signals() applies the default action
then. SIGKILL cannot be blocked, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 10:59:52 +02:00
Nexoniarz
5ed8f1b0d5 kernel: load at 16 MB, clear of what firmware keeps low
From 1 MB the 34 MB kernel ran across 8 MB, where OVMF keeps ACPI NVS
memory that outlives the firmware -- so it was overwritten on every
boot, after ExitBootServices. Firmware leaves its leftovers low; above
16 MB is where a machine is most reliably empty.

KERNPHYS in memlayout.h, and kernel.ld agreeing with it. Nothing else
named the old address. Under OVMF the range is now only boot-services
memory: staged, copied home after the firmware exits, no warning. Boots
as a CD and as a USB disk, 1 and 4 CPUs.

The 15 MB below the kernel is not given to the page allocator yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-24 00:13:53 +02:00
Nexoniarz
e659c6b187 bug pass: whole console lines, a quiet SCI, an honest reserve warning
- schinit, schinitd and vgpud build every message from pieces, one
  write() each, so a job starting on another CPU landed mid-line:
  "schinitd: starting vgpud: os.schos.resolvconfno virtio-gpu on this
  machine". They now collect a line and write it at the newline;
  resolvconf's verbose line is one write with its prefix.
- The ACPI SCI registers IRQ_QUIET. A desktop's SCI never fires, and the
  poller took two seconds of silence for a misrouted line ("irq 9:
  nothing has arrived on it") and called the handler every tick. A line
  is only exempt while every handler on it is quiet.
- The loader claims each segment from where the last one's claim ended,
  so two segments sharing a page no longer ask for it twice. When a
  claim is still refused, it names the memory-map entries in the way.
- Stale divesymlink() declaration gone; the kernel.md struct table
  re-measured (inode, buf and file pools grow from slabs now).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 16:03:33 +02:00
Nexoniarz
fe32833ade display: a 64-bit framebuffer address, no mirroring onto itself
Three fixes for real machines that broke at any resolution but 800x600:

- kmain() took the loader's framebuffer address as uint32, and
  vga_fb_init() re-published the scanout from that and the 16-bit VBE
  block. An aperture above 4 GB moved the console onto the wrong memory
  right after the vga: line. Both now use the loader's 64-bit display
  block, mapped with ioremap().
- The loader mirrored onto the console splitter's GOP, which forwards to
  the primary itself, and onto GOPs that share the primary's aperture.
  After a mode change that rewrote the screen over itself with another
  mode's stride. Handles without a device path and overlapping apertures
  are skipped, and a mode change re-matches the remaining mirrors.
- A mode whose reported size, stride or aperture does not add up after
  SetMode is refused, and the loader returns to the previous one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 15:46:42 +02:00
Nexoniarz
a1c965da1e include: one folder per subsystem, and includes that name it
Sixty-five headers sat side by side in include/, so `#include "irq.h"`
said nothing about where irq.h was or what it belonged to, and errno.h,
stat.h and fcntl.h at the top shadowed the system's for any host tool
built with -I./include. They are in folders now -- arch/ core/ fs/
dev/ gfx/ net/ uapi/ (and ldls/ for the Linux driver layer) -- and
every include names the folder, as Linux's do: "core/proc.h".

Done mechanically and checked the hard way: the kernel built from this
tree disassembles identically to the one built from the tree before
it. The full image boots, and userland is the same set of programs.
Paths in docs and comments follow; docs/guides/layout.md has a table
of what is where, and tools/fix-includes.py rewrites old-style
includes from the folders in the tree, for code written before this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 15:05:12 +02:00
Nexoniarz
0e39e711f1 fpu: save each process's x87/SSE registers; keep the kernel off them
Nothing ever saved the FPU and SSE registers on a context switch.
musl's memcpy, the compiler's struct copies and any arithmetic on
doubles keep live values in %xmm, so two processes sharing a CPU wrote
over each other's: two shells counting in a loop on one processor took
a NULL+8 page fault within seconds, at a different rip every time. It
reproduces on kernels from before this week too -- it had always been
there, and every crash nobody could explain was a candidate.

sched() now fxsaves the process leaving the CPU and scheduler()
fxrstors the one arriving. A new process starts from the ABI's state
(FCW 0x37F, MXCSR 0x1F80), fork copies the parent's live registers,
exec resets them, and a signal handler's are put back by rt_sigreturn
so the interrupted code finds what it had.

The kernel was free to use %xmm too -- GCC vectorised struct copies
and memset into 153 instructions of it -- so an interrupt could
clobber the registers of the program it interrupted with no switch at
all. Kernel, lwIP and uACPI objects are built -mgeneral-regs-only now
and the image has none; cmd/ keeps SSE.

Checked under QEMU: three concurrent loops three times over, on 1 CPU
and on 4, no faults; before, every run faulted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 15:02:01 +02:00
Nexoniarz
a2d987db5f sched: fair share by virtual runtime, with nice
The scheduler was xv6's round-robin: run everything RUNNABLE in slot
order and yield on every timer tick. Slot order is not fairness, a
process alone on a CPU paid a full context switch a hundred times a
second to be handed straight back to itself, and there was no way to
say that one thing mattered more than another.

Every process now carries vruntime, the CPU time it has used divided
by its weight, and the scheduler runs the runnable process with the
smallest. Fairness and interactivity both fall out of that one rule;
nice is a weight, 1.25x per level as UNIX has always had it. A slice
is 20 ms shared among the runnable processes, never below 2 ms, and
the timer tick preempts only once it is spent. A process waking from
a long sleep is brought up to min_vruntime less one period, so it is
scheduled promptly without being owed everything it slept through;
fork inherits vruntime, so forking does not reset a debt.

getpriority/setpriority are real, with Linux's offset of 20 on the
return; /proc/<pid>/schedstat and the ctxt_switches lines of status
report what the scheduler did. docs/architecture/scheduler.md has the
reasoning, including why the run queue is still a scan of ptable.

This was written in an earlier session and left uncommitted. Finished
here: a signal waking a sleeper goes through the same vruntime floor
as wakeup() does, and sched_yield() yields -- it returned 0 doing
nothing, harmless while every tick was a yield and a spin of a whole
slice once ticks stopped being one.

Checked under QEMU on 1 and 4 CPUs: a nice 10 loop beside a nice 0
one gets about a tenth of a shared CPU, and a command typed meanwhile
answers in 26 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 15:01:27 +02:00
Nexoniarz
b70e36e0a2 kalloc: never-used memory stays a range; 256 GB; free is a counter
Every page of RAM went onto the free list at boot, which means writing
each one -- a link, and kfree()'s 4 KB of junk -- before the first
process ran: 64 GB of memset on a 64 GB machine to say it was free.
Memory nobody has used yet is kept as a cursor and an end per region
now, and kalloc() takes from those once the list of given-back pages
is empty. A page is first touched by whoever asked for it.

That makes the cap on RAM cheap to raise, and PHYSTOP_MAX goes from 64
to 256 GB, which still leaves 248 GB of the kernel PDPT for device
windows. MEMREGION_MAX goes from 32 to 64 for the firmware maps that
cut RAM into many pieces.

kalloc_free_bytes() walked the whole free list under the allocator's
lock -- two million links on 8 GB, every second top or fastfetch was
running, with every CPU that wanted a page waiting behind it. It reads
a counter now. And the buffer cache, which keeps what it grows to, is
reported as Buffers in /proc/meminfo and sysinfo(2) rather than as
memory used by nothing.

Checked under QEMU at 1, 2 and 6 GB: MemFree right, zsh splitting
200000 words, fork loops, a 74 MB write.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 14:05:40 +02:00
Nexoniarz
4805dbd892 limits: 256 PCI functions, 128 sockets and devices, 64 USB interfaces
The fixed tables that a bigger machine than the one they were written
on reaches first, each raised to where it costs little:

  NPCI       64 -> 256   a desktop with cards and Thunderbolt passes 64,
                         and the devices past it -- often the disk or
                         NIC last in bus order -- were ignored; 44 KB
  NSOCK      32 -> 128   Unix sockets, which a desktop session uses by
                         the dozen; 4.9 KB each, 470 KB in all
  NDEVICE    32 -> 128   120 bytes each
  NIFDEV     16 -> 64    one dock is a hub, Ethernet, audio and a card
                         reader at two or three interfaces apiece
  NETIF_MAX   4 -> 8

Device names took one digit and clamped there, so the tenth device of
a kind and every one after it was named "...9". Two digits now.

Checked under QEMU with two bridges of test devices: 68 functions,
all in lspci, /proc/pci and /sys/bus/pci.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 13:54:37 +02:00
Nexoniarz
c1af31f75a smp: 64 processors, and say how many there are
NCPU was 16, under what a desktop has today. 64 costs 48 more TSS
I/O bitmaps and scheduler stacks, about 800 KB of BSS. Past 254 the
APIC would need x2APIC mode, which is a separate change.

With more than 16 there were two other ceilings. sched_getaffinity
wrote "CPU 0" and returned 0, and musl clears every byte past the
count returned -- so the mask came back empty and nproc, make -j and
sysconf(_SC_NPROCESSORS_ONLN) all said 1, on any machine. It returns
the mask of processors that started, and its size, now. And /proc
rendered every file into 2 KB, which /proc/cpuinfo ran out of at the
eighteenth processor without a word; /proc/pci, /proc/interrupts and
/proc/mounts were each one bigger machine from the same cut. There is
one 64 KB buffer for it now behind a sleeplock, since there is no
allocator for contiguous pages, and the generators that write raw
are given the real capacity instead of a constant.

Checked under QEMU with -smp 24: nproc 24, /proc/cpuinfo whole.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 13:50:08 +02:00
Nexoniarz
09e07423e8 fs: inode and open-file tables that grow, with a hash for iget
The inode table was NINODE (200) entries, searched end to end on every
iget(), and a panic -- "iget: no inodes" -- when all of them were held.
Held means open, a cwd, a running binary or a directory mid-walk, so a
few busy processes reached it. iget() now finds an inode through a hash
on (dev, inum), takes a free one from a list, and makes more from a
slab cache once the NINODE in the image are all in use.

The open-file table had the same shape, NFILE (512) and then ENFILE,
and gets the same treatment: a free list, and the slab past NFILE.
/proc/fsstat's peaks now read against how many exist.

getrlimit(RLIMIT_NOFILE) said unlimited, which musl turns into
sysconf(_SC_OPEN_MAX) == -1, after which zsh believed it had no
descriptors and `exec {fd}<file` failed on the first one. It reports
NOFILE now. And a comment in sys_fs.c held a "*/" that ended it early.

Checked under QEMU: six processes holding 115 files each, 713 open
descriptions and 341 inodes at once, all back to the baseline after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 13:39:14 +02:00
Nexoniarz
62d2a85ac8 bio: a buffer cache that grows to 1/32 of RAM, looked up by hash
The cache was NBUF buffers, 1 MB, on every machine: enough for one with
tens of megabytes and nothing on one with gigabytes, which re-read the
same directory and binary blocks off the disk for every command. It
still starts as those 256 in the kernel image, and a miss with every
buffer busy now makes a new one rather than evicting, up to 1/32 of RAM
or 256 MB. Growing on demand means nothing is spent that is not used,
and binit() needs nothing from the page allocator.

Lookups go through a hash on (dev, blockno) instead of walking the LRU
list, and bflush() walks a creation-order list with an unlocked filter
first, so neither costs more at 65536 buffers than it did at 256.
struct buf's data is a pointer to its own page now; the drivers copy
through their own DMA buffers, so where it sits does not matter.

On a 2 GB QEMU guest: a 40 MB file read back in 0.89 s instead of 1.83,
the big binaries in 0.17 s instead of 0.26. /proc/fsstat shows the
cache as buf_cached now / most.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 13:28:12 +02:00
Nexoniarz
51983144e3 exec: Linux's 2 MB of arguments; mmap: merge neighbouring mappings
exec took 32 arguments and 64 environment variables, in fixed arrays,
so a glob over more than 31 files failed with E2BIG. It now reads the
caller's argv and envp where they are, through the uaccess helpers and
a bounce page -- a second thread rewriting the array after a check
cannot slip a kernel address in -- and sizes the new stack to fit
them. The limits are Linux's: ARG_MAX, 2 MB of strings and pointers,
and 128 KB for any one string (USER_STR_MAX follows it).

Every mmap took one of a process's 256 VMA slots, and zsh takes one per
16 KB heap block, so `x=($(<file))` over 30000 words ran out after 4 MB
of heap and exited 1 without a word. Neighbouring anonymous mappings
now share a record, as Linux merges them -- the bump allocator puts
each new one where the last ended -- and munmap trims or splits it.
Partial unmaps of device or segment mappings are refused with EINVAL
instead of kfree()ing pages that are not the process's, and a failed
allocation half way through a mapping no longer leaks what it had.

Checked under QEMU: touch over 3000 names, /bin/echo with 50000
arguments, E2BIG past 2 MB, zsh splitting 100000 words, fork loops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 13:20:29 +02:00
Nexoniarz
eab13241a0 hwreport: describe a machine onto a USB stick, from a Linux-style /sys
hwreport writes one folder to the first read-write FAT mount, or mounts
the first USB stick that takes vfat itself and unmounts it afterwards:
summary.txt (machine, firmware, CPU, memory, every PCI and USB device
with its driver or "-- no driver"), dmesg.txt, lspci.txt in the shape of
lspci -nnk -xxx, lsusb.txt, dmi.txt, memmap.txt, the ACPI tables as raw
.dat files iasl -d reads, and copies of /proc. Serials, the UUID and the
MSDM/SLIC tables (a Windows licence key) are left out.

It reads /sys, which is new: sysfs at Linux's paths and in Linux's
formats -- /sys/bus/pci/devices with uevent, config and resource,
/sys/bus/usb/devices, /sys/firmware/acpi/tables, dmi/tables, memmap
and efi, /sys/class/dmi/id. toybox's lspci and lsusb work unmodified
against it. The USB core now keeps a table of attached devices for it.

The machine's name comes from SMBIOS (kernel/dev/dmi.c). The UEFI loader
hands over the entry point from the configuration table, the kernel
falls back to the 0xF0000 scan, and the log gets Linux's "DMI:" line.

Two bugs found on the way:

- The loader handed over at most 40 memory-map entries, in firmware
  order. A laptop's UEFI map is 60-150 descriptors with RAM above 4 GB
  listed last, so that is what was dropped. The entries now have their
  own area at 0x31000 with room for 256, are sorted before merging, and
  the kernel's 32 usable regions keep the largest when there are more.
- MemTotal and sysinfo's totalram were PHYSTOP, an address, which counts
  the PCI hole: a 6 GB machine reported 8. Now the usable RAM, summed.

usb.ids moves to /usr/share/hwdata, where lsusb looks for it, and
pci.ids joins it (`make pci-ids`, BSD-3-Clause like usb.ids).

Checked under QEMU q35: the report written to a FAT32 stick, fsck.fat
clean, DSDT.dat disassembled by iasl on the host, lspci -k and lsusb
showing names, 6 GB of RAM seen as 6137 MB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 12:54:11 +02:00
Nexoniarz
edc4e0537d schfs: commit every 5 s, survive a full disk, mount up to 64 TiB
Transaction groups commit from a schfsd thread every 5 seconds, on sync
and fsync, on unmount and shutdown, and early when freed-but-pending
blocks crowd the free ones -- the contract ext4 and ZFS have. A crash
still lands on the last group whole, and loses at most those seconds.
Committing before every write(2) returned cost two cache flushes and an
uberblock a call: 4 KB writes go from 2.8 to 6.6 MB/s on AHCI, and from
66 KB/s to 3.9 MB/s on QEMU's IDE disk.

Filling the disk made the root volume unmountable, on the last commit
as well: a tree update that ran out of blocks halfway was published by
the next commit. 1% of the volume (1 to 64 MiB) is now a metadata
reserve, as btrfs and ZFS keep one; data stops at it with ENOSPC as a
short write, new names at half of it, and deletes may use the rest. A
write that runs out part way records what it wrote in the inode rather
than leaving blocks past an end of file nothing moved.

Deleting a large file goes in 4 MiB steps from the end with a commit
allowed between them, as ext4 restarts its transaction through a long
truncate. Deleting 80 MB on a full disk ran out of room halfway and
leaked the rest.

The VFS's dirunlink called the object layer's whole unlink, which freed
the object with its last name, so a file removed while open lost its
data under the reader. It removes the name only now, and the object is
freed on the last iput. An object still open when the machine went down
is found by the mount's space scan and freed, like ext4's orphan list.

The allocation bitmap has a second level of pointer pages, so the mount
ceiling is 64 TiB instead of 64 GiB, and the freed-this-group bitmap
has pages only where frees happened: a 1 TiB volume takes 32 MiB.

A freed block written after the newest snapshot is reused at once
(ZFS's birth-time rule, so bfree carries the block's generation); only
older ones are kept while a snapshot exists. Deleting one snapshot no
longer stops retaining for the others, which let their blocks be
handed out again.

Checked under QEMU: crash after sync, after 8 s and within 5 s; fill to
ENOSPC, rm, remount; unlink while open; an orphan freed at the next
mount; a 1 TiB volume. schfs-check passes, with a new test for the
snapshot rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 12:20:05 +02:00
Nexoniarz
b336e4a566 vfs: room for 32 disks and partitions, 32 mounts, 16 SchFS volumes
NBLKDEV was 16 for every disk and partition together, which a laptop's
NVMe drive with its eight partitions and a USB stick come close to. The
mount table held 8, SchFS 4 volumes and 4 snapshots, FAT 4 volumes.
Now 32, 32, 16, 8 and 8; the device numbers still fit between the ones
already fixed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 12:19:50 +02:00
Nexoniarz
67db42b460 vfs: names up to 255 bytes and paths up to 4096, like Linux
Every path component was copied into a DIRSIZ buffer and every path
into a MAXPATH one, so names were cut at 29 bytes and paths at 127
without an error. Two names sharing their first 29 bytes opened the
same file, on SchFS and FAT alike, though both hold 255.

VFS_NAME_MAX (255) and VFS_PATH_MAX (4096) are the limits now, and
anything longer is ENAMETOOLONG. namex_from() takes its path buffers
from kalloc(), a page each, rather than putting two of them on an 8 KB
kernel stack. The v1 format still stores DIRSIZ bytes, and refuses a
longer name instead of matching on its first 30.

Checked: 255-byte names on SchFS and FAT (fsck.fat clean, and the FAT
name answers to its uppercase form), 256 refused, a 200-byte path and a
link to it, and no pages lost over 700 lookups through links.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 11:17:28 +02:00
Nexoniarz
b213ab5778 schfs: do not read a block off the disk just to overwrite it
kd_write() went through bread(), so every block a write allocated was
read from the disk first and then replaced whole. That was a quarter of
the disk operations of a SchFS write. bgetblk() is bread() without the
read, as Linux's getblk() is.

1 MB on AHCI under QEMU: 2.0 -> 2.8 MB/s in 4 KB writes, 4.5 -> 5.6
MB/s in 64 KB ones. 3 MB of random data written and copied through the
cache's evictions reads back with the same md5.

The 63 KB/s seen earlier is the legacy IDE driver on QEMU's default
machine, at about 10 ms per emulated PIO write; the same disk on AHCI
(q35) is 40 times faster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 11:02:44 +02:00
Nexoniarz
7a1e63a553 fs: follow symlinks like Linux; Linux mknod; /proc/partitions
Symlinks are followed in namex_from() now, in the middle of a path as
well as at the end, and a relative target resolves from the directory
that holds the link rather than from the caller's cwd. `ln -s f d1/s;
cat d1/s` said "Symbolic link loop" because it looked for ./f. A loop is
ELOOP after 40 links, a dangling link ENOENT, and open(O_CREAT) through
a link to a file opens the file. divesymlink() is gone.

mknod from a Linux program passes (mode, dev) where the native call
passes (major, minor), and was read as the native one: `mknod n c 1 2`
made node 432,258, and mkfifo made a character device. Both are decoded
now; a FIFO is EPERM, and an existing name is EEXIST.

/proc/mounts listed devtmpfs twice: once from a fixed line and once
from the mount table. The fixed line is gone, and mount_add() refuses
a filesystem mounted on its own root, which is where a second devtmpfs
mount lands. A read-only FAT mount says ro.

/proc/partitions exists, in Linux's format, and /proc/filesystems lists
vfat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
2026-10-23 10:50:41 +02:00