NetSurf (GTK 3) now starts under X and loads pages over HTTPS. Every icon
GTK draws goes gdk-pixbuf -> glycin -> a loader process -> a sealed memfd
-> GMappedFile, and each hop found something missing:
- mkaroot runs Alpine's package triggers in the root under bubblewrap.
--no-scripts had skipped them, so there was no MIME cache, pixbuf loader
list, compiled GSettings schemas, GIO module cache, font cache or busybox
applet links.
- /usr/bin/bwrap is a stand-in that parses bubblewrap's options and runs
the program with no sandbox: glycin starts its loaders only through bwrap,
and real bwrap needs namespaces. schos-base replaces bubblewrap for it.
- ioctl: FIOCLEX, FIONCLEX, FIONBIO and FIOASYNC on any descriptor (Rust's
std uses them; pipes and sockets answered ENOTTY).
- capget/capset: root has every capability, others none.
- socketpair accepts SOCK_DGRAM and SOCK_SEQPACKET, carried as a stream.
- waitpid and setpgid match children against the thread group leader, so a
thread can wait for the process it spawned (ECHILD before).
- memfd: fstat reports the ftruncate length, not the page-rounded one; file
seals (F_ADD_SEALS, F_GET_SEALS, MFD_ALLOW_SEALING) are recorded and
enforced on later writes, truncation and writable mappings; MFD_CLOEXEC
is honoured; read-only MAP_PRIVATE maps are allowed.
- recvfrom honours O_NONBLOCK and MSG_DONTWAIT. musl's resolver reads its
socket until EAGAIN and blocked for ever after the first DNS answer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
A thread is a struct proc that shares three counted objects with others of
its group: the address space (struct mm: page tables, heap top, VMAs and a
lock for changing them), the descriptor table, and the signal dispositions.
clone() shares what its flags say and fork() copies.
- clone(): pthread_create's shape (VM, FILES, SIGHAND, THREAD, SETTLS,
PARENT_SETTID, CHILD_CLEARTID) and posix_spawn's (VM, VFORK, a stack).
- futex: WAIT, WAIT_BITSET with timeouts, WAKE, REQUEUE, CMP_REQUEUE, keyed
by the word's kernel address so shared pages meet. set_tid_address keeps
the word that exit zeroes and wakes, for pthread_join.
- exit ends a thread; exit_group, a fatal signal, or the leader exiting ends
them all. reapd frees exited threads; the leader is reaped by its parent
once the rest are gone. getpid is the group, gettid the thread.
- Stops and resumes reach every thread; other process signals the leader.
- exec kills the other threads and unshares the dispositions.
- No TLB shootdown, so pages unmapped from a shared address space are freed
only once every CPU has reloaded CR3 or is idle.
- procfs: /proc lists processes, task/ lists threads, comm, Threads:, Tgid.
- The console honours VMIN 0 / VTIME (btop's input drain hung on it).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Found by running Alpine's apk on SchOS, one failure at a time:
- chroot(2) was a stub returning EPERM. Each process now has a root:
where an absolute path starts and where .. stops. Mounts remember
whose root they were made under, and a walk leaves a mount by the way
it came in, so one filesystem can be mounted twice.
- devtmpfs and proc mounted only on /dev and /proc, whatever the target;
a chroot got "already mounted".
- mmap never reused freed address space, so musl's malloc used up the
256 mapping records while 780 MB were free. A mergeable mapping now
goes straight after an anonymous one whose end has room.
- unlinkat returned -1 (EPERM) where apk expects ENOENT, and renameat
was an old copy of rename: no replacing the target, and v1 dirents
written into SchFS. Both now share rename's implementation.
- execve runs #! scripts, and says ENOEXEC for what it cannot run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
A program that names an interpreter in PT_INTERP gets it loaded too, at
the bottom of the mmap area, and started instead -- with AT_PHDR,
AT_PHENT, AT_PHNUM, AT_BASE and AT_ENTRY to find the program by. ET_DYN
images are placed at a base: the interpreter there, a PIE program where
a fixed one would go. Static musl programs get AT_PHDR now as well,
which is how musl finds their TLS template.
mmap(MAP_FIXED) of a file handed back zeroed pages whatever the
descriptor said, and kept no record of them, so fork() did not copy
them. A dynamic linker maps every library segment after the first that
way. Fixed mappings now read the file and are recorded wherever no
mapping already covers them.
musl is built shared as well as static, and libc.so ships as
/lib/ld-musl-x86_64.so.1 -- libc and loader in one file. Every port
still links -static on purpose.
cmd/dynhello is the test: against libc.so and a library of its own,
checking TLS, the heap, and a fork whose child still has both.
mprotect is accepted and not enforced yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Sixty-five headers sat side by side in include/, so `#include "irq.h"`
said nothing about where irq.h was or what it belonged to, and errno.h,
stat.h and fcntl.h at the top shadowed the system's for any host tool
built with -I./include. They are in folders now -- arch/ core/ fs/
dev/ gfx/ net/ uapi/ (and ldls/ for the Linux driver layer) -- and
every include names the folder, as Linux's do: "core/proc.h".
Done mechanically and checked the hard way: the kernel built from this
tree disassembles identically to the one built from the tree before
it. The full image boots, and userland is the same set of programs.
Paths in docs and comments follow; docs/guides/layout.md has a table
of what is where, and tools/fix-includes.py rewrites old-style
includes from the folders in the tree, for code written before this.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The scheduler was xv6's round-robin: run everything RUNNABLE in slot
order and yield on every timer tick. Slot order is not fairness, a
process alone on a CPU paid a full context switch a hundred times a
second to be handed straight back to itself, and there was no way to
say that one thing mattered more than another.
Every process now carries vruntime, the CPU time it has used divided
by its weight, and the scheduler runs the runnable process with the
smallest. Fairness and interactivity both fall out of that one rule;
nice is a weight, 1.25x per level as UNIX has always had it. A slice
is 20 ms shared among the runnable processes, never below 2 ms, and
the timer tick preempts only once it is spent. A process waking from
a long sleep is brought up to min_vruntime less one period, so it is
scheduled promptly without being owed everything it slept through;
fork inherits vruntime, so forking does not reset a debt.
getpriority/setpriority are real, with Linux's offset of 20 on the
return; /proc/<pid>/schedstat and the ctxt_switches lines of status
report what the scheduler did. docs/architecture/scheduler.md has the
reasoning, including why the run queue is still a scan of ptable.
This was written in an earlier session and left uncommitted. Finished
here: a signal waking a sleeper goes through the same vruntime floor
as wakeup() does, and sched_yield() yields -- it returned 0 doing
nothing, harmless while every tick was a yield and a spin of a whole
slice once ticks stopped being one.
Checked under QEMU on 1 and 4 CPUs: a nice 10 loop beside a nice 0
one gets about a tenth of a shared CPU, and a command typed meanwhile
answers in 26 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
exec took 32 arguments and 64 environment variables, in fixed arrays,
so a glob over more than 31 files failed with E2BIG. It now reads the
caller's argv and envp where they are, through the uaccess helpers and
a bounce page -- a second thread rewriting the array after a check
cannot slip a kernel address in -- and sizes the new stack to fit
them. The limits are Linux's: ARG_MAX, 2 MB of strings and pointers,
and 128 KB for any one string (USER_STR_MAX follows it).
Every mmap took one of a process's 256 VMA slots, and zsh takes one per
16 KB heap block, so `x=($(<file))` over 30000 words ran out after 4 MB
of heap and exited 1 without a word. Neighbouring anonymous mappings
now share a record, as Linux merges them -- the bump allocator puts
each new one where the last ended -- and munmap trims or splits it.
Partial unmaps of device or segment mappings are refused with EINVAL
instead of kfree()ing pages that are not the process's, and a failed
allocation half way through a mapping no longer leaks what it had.
Checked under QEMU: touch over 3000 names, /bin/echo with 50000
arguments, E2BIG past 2 MB, zsh splitting 100000 words, fork loops.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Gate 1, item 2, and the thing a Wayland buffer crosses on.
memfd_create(2) is the same segment machinery shmget() already had -- pages,
refcounts, no_free VMAs, the fork and exit accounting -- reached by
**descriptor** rather than by System V key. That difference is the entire
point and the reason it exists on Linux at all: two processes share memory by
passing the fd over a socket, so neither has to agree on a path or race to
invent a key. It is also why SCM_RIGHTS had to be item 1.
docs/plan/foundations.md proposed exactly this and it was right: what memfd
adds on top of shmget is a segment that starts empty and is sized by
ftruncate(), and a file that owns one reference to it. FD_MEMFD, and
read/write/lseek/fstat all learn about it -- a descriptor that refuses read()
is a surprise nobody needs.
Three things are refused rather than faked, each the honest version of
something this kernel cannot do:
- MAP_PRIVATE on a memfd. Every caller is sharing on purpose; a private
mapping of an anonymous object is an expensive MAP_ANONYMOUS.
- Mapping past the end. Linux faults those pages in; there is no handler
here that could.
- Shrinking while mapped. Linux truncates the pages out from under the
mapper; here a freed page would stay mapped to memory somebody else now
owns.
Writable MAP_SHARED of a *real file* is still ENODEV. That needs a page cache,
and an anonymous object has no such contract to break -- which is precisely
why memfd is the narrow answer.
**cmd/memfdtest**, seventeen checks, written so a permissive implementation
cannot pass: an mmap that quietly returned private zeroed pages satisfies every
return code and fails the one assertion that matters, because the reader is a
different process. It ends with the combination the plan singled out -- a memfd
sent over a socketpair to a receiver that closes every inherited copy first,
maps only the descriptor that arrived, reads what the sender wrote and writes
back where the sender can see it.
Two things were found by writing that test, both of which had been refusing
everything that was not FD_INODE:
- sys_fstat() never reached filestat(), so the memfd branch there was
unreachable from the syscall that needs it. It also refused pipes and
sockets, and returned -1 (which musl reads as EPERM) rather than an errno.
- sys_lseek() the same, and SEEK_END needs a length that is not in an inode.
Verified: 40 consecutive runs leave MemFree at exactly the value it started
at -- 936656 kB before and after -- so nothing leaks across creation, dual
mapping, descriptor passing and exit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docs/plan/hybrid.md mechanisms 2 and 4: MMIO and DMA to a process.
A process can enumerate the bus, claim a device exclusively, map its BARs and
get memory the device can be pointed at. Proven end to end against an e1000e
-- the exact device class we cannot drive and would want a foreign driver for:
device 7: 8086:10d3 (network)
claimed
BAR0 at 0x81060000, 128 KB
regs[0..3] = 0x00140261 0x00000000 0x00080283 0x00000000
MMIO reaches the device
DMA 8 KB at virtual 0x40001000, physical 0x3f8eb000
Those are the e1000e's CTRL and STATUS with full-duplex and link-up set, which
is how you tell a real mapping from one reading 0xFFFFFFFF off an unmapped
address. Claiming a device the kernel drives is refused -- two writers of one
register set looks like hardware that has started lying -- and everything a
process held is released when it exits, because a driver that crashed must not
keep its hardware away from its replacement.
mmap's offset is the BAR's physical address, checked against the BARs of
devices *this process has claimed*. Without that check the device is /dev/mem
with extra steps and a process could map the kernel's own memory. Mappings are
write-through: a cached write to a doorbell arrives whenever the cache feels
like it, which from a driver's side is indistinguishable from hardware that
ignores you.
The interface is about PCI devices, not about any driver model, and its first
users will be SchOS's own drivers when virtio-gpu is evicted. That ordering is
the design rather than an accident: an interface whose first consumers are
native drivers is a general mechanism something foreign can use later, instead
of a hole shaped like something foreign.
Also: PROT_* and MAP_* moved to include/posix.h. They were defined only in
sys_mem.c, so cmd/fbinfo carried a private copy and this wanted a third --
two copies of a number the kernel compares against, where a mismatch shows up
as a mapping with the wrong permissions rather than an error.
No IOMMU, so this isolates a driver against its own bugs and not against its
own DMA. The plan says so at length and it is still true.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Porting zsh found four defects in the kernel and one in the filesystem. None
of them were about zsh; all of them had been there the whole time.
**exec threw the environment away.** kernel/fs/exec.c wrote a single zero for
envp with the comment "(no environment)" -- not yet, but ever, for every
process. musl has been passing a perfectly good envp to execve (Linux 59, which
SchOS answers) and it was never read. So $TERM was unset, $PATH was whatever
each program invented, and a variable exported in a shell did not survive the
exec meant to carry it. exec() takes an envp, sys_exec reads argument three,
and schinit manifests grew an Environment key. The userland two-argument
exec(path, argv) stays as a wrapper that passes null explicitly -- the raw stub
is execve now, so a short caller cannot leave %rdx holding a stray pointer.
**Which overflowed the kernel stack.** Three pointer arrays, ~1.8 KB, on a
4096-byte stack, under exec -> loaduvm -> readi -> SchFS -> btree. Pid 1 went
down on the first exec. The canary caught it and named the process, which is
what it is for. They live in one kalloc'd page now.
**waitpid returned a bare -1** for both "no children" and "interrupted". musl
reads [-4095,-1] as -errno, so userspace saw EPERM; zsh printed "wait failed:
operation not permitted" and abandoned the rest of /etc/zsh/zshrc. -ECHILD and
-EINTR now. Third time a bare -1 has cost real time -- on a system with no
permission model, EPERM should be read as "a syscall returned -1".
**NVMA was 64.** musl's allocator takes a mapping per size class and does not
release them eagerly, so compinit ran out of slots and got -1, which is EPERM,
which malloc turns into NULL, which zsh reports as "fatal error: out of memory"
-- on a machine with 1000 MB free. 256 slots, and -ENOMEM.
**And a small write to SchFS costs 58 ms.** 200 writes to one open file take
11.75 s; the same to /dev/null take 0.14 s. That is the design stated at the
top of kmount.c -- one transaction group per operation, and a commit is two
flushes and an uberblock -- not a bug. But compdump writes its digest with
thousands of separate prints, so it takes minutes and looks like a hang.
compinit -D, which skips the dump, finishes in 3.6 s. This one is a decision
rather than a fix, so it is written up and left: /etc/zsh/zshrc keeps compinit
switched off and explains exactly why, with the measurements.
zsh runs as /bin/zsh and is the console shell. ZLE works -- arrow keys recall
history through terminfo, $COLUMNS reads the real 160 because TIOCGWINSZ was
fixed, colour and prompt expansion work, startup is 0.14 s. mksh keeps /bin/sh,
so schinit's no-jobs fallback is a different program from the one that could
have failed.
Also fixed while here: AT_CLKTCK was the literal 100 and is USER_HZ; the
extract rule in four ports could move a source tree inside itself when make
re-ran it, which is how zsh's configure came to write modfile=zsh-5.9/... into
config.modules and fail three steps later.
schfs-check passes. Full writeup in docs/history/kernel.md section 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Same disease as the network stack, one subsystem over. kernel/dev/vga.c
was three things wearing one name: the owner of the framebuffer, a
bitmap drawing library, and the text console. Only the first is a
driver, and because they shared a file they shared a prefix -- so
panic.c drew the crash screen with vga_fill_rect, console.c printed with
vga_putc, /dev/fb0 handed out vga_fb_phys, and every one of those files
named a display driver it had no business knowing about.
Split three ways:
- kernel/drivers/gpu/vga/vga_fb.c owns the VBE framebuffer -- the mode
the loader programmed and handed over, the aperture, and the display
class vtable. It is a driver and now lives with the driver that
finds the card on the bus.
- kernel/dev/gfx.c draws: pixels, shapes, glyphs, the cursor, scroll,
the ANSI erase operations. It names no driver. The framebuffer
arrives through gfx_attach_scanout(), so a driver publishes what it
owns and this file draws on it without knowing what it is.
- include/gfx.h is the interface, and says why it is deliberately not
the display class: that one is about frames -- acquire, present,
vblank -- and the crash screen has to work when nothing else does.
main() no longer calls a display driver. platform_register_devices()
takes the framebuffer address and brings up whatever owns it, which is
where driver names belong; kernel/drivers/registry.c is the file whose
entire job is naming every driver in the kernel.
virtio-gpu got the e1000 treatment as well: instances in an array, the
instance hung on dev->priv by an `attached` hook, and every op reading
it rather than a file-static. It refused the second adapter outright
before, which is not hypothetical -- integrated plus discrete is an
ordinary laptop. The VGA driver deliberately has no such hook, and says
so: there is one framebuffer the loader programmed however many
VGA-class controllers the bus turns up, and a second one publishing the
same scanout is the truth rather than a bug.
Private headers moved next to their drivers: include/virtio.h and
include/xhci.h each had exactly one includer. include/ is for headers
the rest of the kernel needs.
Deleted rather than moved: consoleinit() programmed the sixteen ANSI
colours into the VGA DAC and disabled text-mode blink, by writing ports
0x3C0-0x3C9 from the console. The comment three lines above the colour
table already said this does nothing in a 32bpp linear mode -- there is
no palette to index -- so it was both dead and in the wrong file.
Verified, and this is a refactor so the evidence is that nothing moved:
- bootcheck -Fb -Net: 59 [ok], no failures. That covers the console
through every command, /dev/fb0's geometry, writing pixels through
write() and mapping the framebuffer through mmap.
- make crashshot: the crash screen renders exactly as before --
shapes, glyphs, logo and QR code -- which is the whole of gfx.c
exercised at once, in a panic, with the scanout attached by a driver.
- make schcom-test: five pixel assertions from the host oracle still
pass, so the display class and the compositor above it are unmoved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An x86-64 operating system built from xv6, now largely rewritten: SMP,
a copy-on-write root filesystem (SchFS), PCI/USB/virtio drivers, a TTF
rasteriser, and a Linux-numbered syscall ABI.
No history before this point -- the tree was never under version control.
This commit exists so the rename that follows has something to undo to.
