Initial commit: Schunix as it stands before the SchOS rename
An x86-64 operating system built from xv6, now largely rewritten: SMP, a copy-on-write root filesystem (SchFS), PCI/USB/virtio drivers, a TTF rasteriser, and a Linux-numbered syscall ABI. No history before this point -- the tree was never under version control. This commit exists so the rename that follows has something to undo to.
This commit is contained in:
commit
d0f7acf9eb
275 changed files with 51417 additions and 0 deletions
481
docs/roadmap.md
Normal file
481
docs/roadmap.md
Normal file
|
|
@ -0,0 +1,481 @@
|
|||
# Roadmap
|
||||
|
||||
Written 2026-08-04, from the plan: USB, GPU, fonts, networking and sockets,
|
||||
then a compositor (Schcom), a window manager (SchWM) and a desktop (SchDE),
|
||||
plus an installer and a `.sip` package format.
|
||||
|
||||
This file records two things that are easy to lose: the *order* the work has
|
||||
to happen in, and the decisions that constrain it.
|
||||
|
||||
**Superseded in direction, not in order.** Schunix is now aimed at being a
|
||||
*system* rather than a Unix-like kernel with a userland attached: a NeXTSTEP/
|
||||
macOS-shaped layering, with POSIX kept complete but demoted from identity to
|
||||
personality, and a native API (**SchAPI**) above it. See
|
||||
[architecture/schapi.md](architecture/schapi.md). That decision changes what
|
||||
the phases below are building towards; it does not reorder them, and it does
|
||||
not start until the tree builds and boots natively again.
|
||||
|
||||
That assumption about §1 has since been revised. NetBSD's TCP/IP stack was the
|
||||
intended starting point; the per-file licence check found that NetBSD's
|
||||
`tcp_input.c` and `tcp_output.c` carry the WIDE Project's 4-clause advertising
|
||||
clause, which Berkeley's rescission does not reach. FreeBSD's equivalent is
|
||||
clean, and lwIP is cleaner still and a far smaller port. See Phase 2 below and
|
||||
docs/licensing.md §3.
|
||||
|
||||
---
|
||||
|
||||
## 1. Where drivers can come from
|
||||
|
||||
**BSD: yes. Linux: no.** This is not a preference, it is a licence
|
||||
incompatibility, and it decides the whole driver strategy.
|
||||
|
||||
The reasoning — Apache-2.0's patent terms counting as "further restrictions"
|
||||
under GPLv2 §6, which is why the two cannot be combined and redistributed — is
|
||||
in [licensing.md](licensing.md), along with the per-licence table and the
|
||||
NetBSD-specific rules. Read it before copying anything. The short version:
|
||||
BSD-2 and BSD-3 code can be used directly with a `NOTICE` entry; GPL-2.0-only
|
||||
code can never be, at any distance.
|
||||
|
||||
There is a third source that is better than either: **the specifications**.
|
||||
xHCI, AHCI, virtio, NVMe, the Intel e1000 and i915 datasheets are all public
|
||||
documents with no licence attached to what you write from them. That is where
|
||||
`kernel/drivers/xhci.c` came from.
|
||||
|
||||
One caveat about "port it from BSD" as a plan, learned by looking: FreeBSD's
|
||||
`sys/dev/usb` is roughly 100k lines welded to newbus, devclass, `bus_dma`,
|
||||
callouts, taskqueues and FreeBSD's locking. Porting it means porting
|
||||
FreeBSD's device framework first and finding the USB stack somewhere
|
||||
underneath. NetBSD's is more portable and the same argument applies with a
|
||||
smaller multiplier. **BSD is far more valuable as the reference you check
|
||||
yourself against than as the code you paste.** Read it when the spec is
|
||||
ambiguous — which it is, often — and write your own.
|
||||
|
||||
Reading Linux for *understanding* is fine and normal. Translating a Linux
|
||||
driver function-by-function is not; the result is a derivative work whatever
|
||||
the variable names say.
|
||||
|
||||
---
|
||||
|
||||
## 2. Ordering, and why it is not the order it was asked in
|
||||
|
||||
The requested order was USB → GPU → fonts → network → GUI → installer. That
|
||||
is a reasonable order by *interest*. By dependency it wants one change:
|
||||
|
||||
**Shared driver infrastructure comes before all of it.** USB, any GPU beyond
|
||||
the VBE framebuffer, and any NIC all need the same three things, none of
|
||||
which existed:
|
||||
|
||||
- a way to register an interrupt handler at runtime (`kernel/dev/irq.c`)
|
||||
- physically contiguous memory for rings and descriptors
|
||||
(`kernel/core/dma.c`)
|
||||
- PCI capability walking and MSI (`kernel/dev/pci.c`)
|
||||
|
||||
That work is done. It was the first step of USB, not a detour from it.
|
||||
|
||||
The other honest note: **USB is the item that does least for the GUI.** Under
|
||||
QEMU there is already a PS/2 keyboard and mouse, so USB buys real hardware
|
||||
support — which matters, but not for getting Schcom on screen. Sockets and
|
||||
raw input events are what actually block the compositor. USB is being done
|
||||
first because it was asked for first; the ordering below reflects that with
|
||||
the dependencies fixed.
|
||||
|
||||
---
|
||||
|
||||
## 3. The plan
|
||||
|
||||
### Phase 1 — USB *(done, bar the boot path)*
|
||||
|
||||
- [x] Shared infrastructure: IRQ registration, DMA arena, PCI capabilities
|
||||
- [x] xHCI: reset, DCBAA, command ring, event ring, scratchpad
|
||||
- [x] Port enumeration, Enable Slot, Address Device, control transfers
|
||||
- [x] Device descriptors read from real devices (USB2 and USB3 paths)
|
||||
- [x] Configuration and interface descriptor parsing, `SET_CONFIGURATION`
|
||||
- [x] Interrupt endpoints, driven from the IRQ handler
|
||||
- [x] HID boot protocol: keyboard and mouse into the existing input paths
|
||||
- [x] Bulk endpoints, bulk-only transport, SCSI, a block device at
|
||||
`USBDISK_DEV` — a superblock reads back through `bread()`
|
||||
- [x] Hubs — route strings, hub slot contexts, per-port reset and power. A
|
||||
mouse behind a hub enumerates a tier deep, which is what proves it
|
||||
- [ ] Making a USB disk mountable and bootable (needs the root device to be
|
||||
selectable rather than compiled in)
|
||||
|
||||
### Phase 2 — Sockets, raw input, and networking
|
||||
|
||||
Do sockets **before** the network stack. **Unix domain sockets are what Schcom
|
||||
needs** — clients and compositor have to pass messages and share buffers, and
|
||||
a pipe cannot carry an shm handle. Unix sockets are also far simpler than
|
||||
TCP/IP, so they are the right thing to build the socket layer's shape around.
|
||||
|
||||
- [x] `socket`/`bind`/`listen`/`accept`/`connect`/`send`/`recv` syscalls, at
|
||||
their real Linux numbers — they were `NO_LINUX` stubs, so a musl program
|
||||
calling `socket()` never even reached the stub
|
||||
- [x] `AF_UNIX` SOCK_STREAM, `socketpair`, `shutdown`; names live in a table
|
||||
rather than the filesystem (see the comment at the top of
|
||||
`kernel/net/unix.c` for what that costs)
|
||||
- [x] Raw input events — `/dev/input/event0` and `event1`, Linux evdev layout,
|
||||
fed by the PS/2 keyboard and mouse and by USB HID. The console keeps its
|
||||
own path; both consumers see every key
|
||||
- [ ] `SCM_RIGHTS` for passing descriptors. Possibly unnecessary: an shm id is
|
||||
an integer and travels fine over a stream socket, so this waits until
|
||||
something actually needs to pass an open file
|
||||
- [x] e1000 driver — `kernel/drivers/net/e1000.c`, from Intel's public manual
|
||||
317453. MAC from EEPROM, transmit and receive rings, link detection.
|
||||
Run with `-netdev user,id=n0 -device e1000,netdev=n0`.
|
||||
**Verified to carry real traffic**: `echo n > /proc/sysrq` sends a
|
||||
hand-built ARP request and receives the gateway's reply, which exercises
|
||||
the descriptor formats, the DD handshake, the tail bookkeeping, the DMA
|
||||
addresses and the MAC filter in one exchange. That check came before any
|
||||
protocol stack on purpose -- a stack on an unproven driver fails in ways
|
||||
indistinguishable from its own bugs
|
||||
- [ ] ARP, IPv4, ICMP, UDP, then TCP. **Not written from scratch** -- decided
|
||||
2026-08-08. Writing TCP is the one part of this where doing it yourself is
|
||||
a bad trade: it is not the packet format that is hard, it is congestion
|
||||
control, retransmission, window management and twenty years of accumulated
|
||||
corner cases that only appear against a real peer.
|
||||
**Preferred base: lwIP** (BSD-3, no advertising clause, verified). It was
|
||||
written for exactly this situation -- a small OS with a raw Ethernet
|
||||
driver -- and its port layer is a handful of files.
|
||||
**If BSD instead: FreeBSD, not NetBSD.** NetBSD's `tcp_input.c` carries
|
||||
the WIDE Project's 4-clause advertising clause; FreeBSD's has none. But
|
||||
FreeBSD's `sys/netinet` is not a library -- it reaches into mbufs,
|
||||
epoch(9), netisr, VNET and uma(9), so taking it means porting FreeBSD's
|
||||
kernel infrastructure first. See docs/licensing.md §3.
|
||||
- [ ] `AF_INET` on top of the same socket layer
|
||||
|
||||
### Phase 3 — GPU
|
||||
|
||||
"Basic GPU driver" has a right answer and a wrong one. The wrong one is
|
||||
starting with Intel or AMD hardware: modern GPU drivers are enormous and most
|
||||
of the size is mode setting, power management and a command submission model
|
||||
that only pays off once there is something to submit.
|
||||
|
||||
- [x] **virtio-gpu**, 2D. `kernel/drivers/virtio_gpu.c`, written from the OASIS
|
||||
specification. Full path: GET_DISPLAY_INFO, RESOURCE_CREATE_2D,
|
||||
RESOURCE_ATTACH_BACKING, SET_SCANOUT, TRANSFER_TO_HOST_2D,
|
||||
RESOURCE_FLUSH. Bound as the display class ahead of the VBE framebuffer.
|
||||
Run with `-vga std -device virtio-gpu-pci`; keep `-vga std`, because the
|
||||
bootloader's splash goes through VBE and there is no virtio path that
|
||||
early
|
||||
- [ ] A `/dev/dri`-style device with mode setting, so resolution stops being
|
||||
whatever the bootloader asked VBE for
|
||||
- [ ] Double buffering and vblank — Schcom needs both to not tear
|
||||
- [ ] Only then: real hardware, and only 2D/blit acceleration
|
||||
|
||||
### Phase 4 — Fonts
|
||||
|
||||
This is a bigger job than it sounds and it should be sized honestly. A TTF/OTF
|
||||
rasteriser needs: `cmap`/`glyf`/`loca`/`head`/`hmtx` table parsing, quadratic
|
||||
béziers for TrueType outlines and *cubic* ones for OpenType/CFF, a scanline
|
||||
rasteriser with anti-aliasing, and a glyph cache.
|
||||
|
||||
The bigger structural problem is that the console assumes a fixed 8×16 cell
|
||||
grid throughout — `vga_putc`, the cursor, the ANSI erase operations. Variable
|
||||
width text is not a font change, it is a text-rendering layer.
|
||||
|
||||
- [x] **The text layer** — `include/text.h` + `kernel/dev/text.c`. Glyph
|
||||
metrics, a `struct surface` target so a compositor can draw off-screen,
|
||||
8-bit coverage with real blending, UTF-8, and `font_ops` so a rasteriser
|
||||
plugs in as a second provider. This was the actual blocker: the font was
|
||||
never the problem, the character-cell grid was.
|
||||
- [ ] Port the console onto it and delete the cell grid
|
||||
- [x] A TrueType rasteriser — `kernel/dev/ttf.c`, glyf outlines, composite
|
||||
glyphs, cmap format 4, anti-aliased scanline fill. Host harness at
|
||||
`tools/ttftest.c`
|
||||
- [x] A glyph cache — direct-mapped over ASCII into a 48 KB pool
|
||||
- [ ] Load a font from disk and point the console at it, deleting the cell grid
|
||||
- [ ] CFF/Type 2 charstrings, so OTF fonts work too (a separate `font_ops`
|
||||
provider; a bytecode interpreter, not an extension of the glyf path)
|
||||
- [ ] A rasteriser. `stb_truetype.h` is public domain and therefore usable
|
||||
as-is if wanted; writing one is also reasonable and more interesting
|
||||
- [ ] **The typefaces, decided 2026-08-09.** Both verified, not assumed:
|
||||
**Roboto** for the desktop, window manager and compositor --
|
||||
**Apache-2.0**, the same licence as Schunix, so it needs nothing beyond a
|
||||
NOTICE entry.
|
||||
**JetBrains Mono** for the terminal and TTY -- **SIL OFL 1.1**, and
|
||||
notably with *no* Reserved Font Name declared, so it may be modified and
|
||||
still called JetBrains Mono. That matters more than it sounds:
|
||||
pre-rasterising a font into bitmaps at build time is a modification, and
|
||||
an RFN would have forbidden keeping the name on the result.
|
||||
OFL's one real constraint is that the font may not be sold on its own and
|
||||
must stay under OFL; bundling it in an Apache-2.0 system is exactly the
|
||||
case it is written for.
|
||||
- [ ] Text layout (advance widths, kerning) as a library in `lib/`
|
||||
- [ ] The console keeps its fixed grid; proportional text belongs to the GUI
|
||||
|
||||
**Font licensing: public domain or very permissive only.** No SIL OFL if it
|
||||
can be avoided, no "free for personal use", no CC-BY-SA. The constraint is
|
||||
the same one that governs drivers (§1): a font file is redistributed inside
|
||||
every Schunix image, so its licence attaches to the image.
|
||||
|
||||
Safe choices, in rough order of usefulness:
|
||||
|
||||
- **CC0 / public domain** — best case, nothing to comply with.
|
||||
- **MIT / BSD / Apache-2.0 fonts** — fine, needs a `NOTICE` entry.
|
||||
- **SIL OFL** — usable but has a real string attached: a modified font must
|
||||
be renamed, and OFL fonts cannot be sold on their own. Workable, but read
|
||||
it before committing one.
|
||||
|
||||
Whatever is chosen, the licence goes in `NOTICE` **before** the file is
|
||||
committed, not after.
|
||||
|
||||
### Phase 5 — The GUI, as separate projects
|
||||
|
||||
See `projects/README.md`. Schcom, SchWM and SchDE live outside the kernel
|
||||
tree on purpose.
|
||||
|
||||
- [ ] **Schcom** — the compositor. Owns the framebuffer, composites client
|
||||
buffers, routes input. Needs: unix sockets, shm (done), raw input
|
||||
events, and a GPU driver with double buffering.
|
||||
- [ ] **SchWM** — window management on top of Schcom's protocol
|
||||
- [ ] **SchDE** — panel, launcher, settings
|
||||
|
||||
Raw input is the missing kernel piece nobody thinks of until it bites:
|
||||
keyboard events currently only arrive through the tty line discipline, cooked
|
||||
and line-buffered and delivered to one foreground process. A compositor needs
|
||||
key-down/key-up with modifiers and no line discipline at all — an evdev-style
|
||||
`/dev/input`.
|
||||
|
||||
### Phase 6 — Installer and `.sip`
|
||||
|
||||
`.sip` (Schunix Installation Package), the stated model being WIM in Windows
|
||||
PE: a *file-based* image rather than a block image, so one file can hold a
|
||||
whole filesystem, be applied to a partition of any size, and hold several
|
||||
images that share their common files.
|
||||
|
||||
The pieces for this mostly exist already. `tools/mkschfs.c` does content
|
||||
dedup with an `(st_dev, st_ino)` fast path; SchFS's B-tree is content
|
||||
addressed. A sensible `.sip` is: a header, a file index (path, metadata,
|
||||
content hash), and a blob store keyed by hash, with compression per blob.
|
||||
|
||||
- [ ] `.sip` format specification, written down before any code
|
||||
- [ ] `tools/mksip` — capture a directory tree into a `.sip`
|
||||
- [ ] Kernel or userspace apply path
|
||||
- [ ] An installation environment that boots from the ISO, partitions,
|
||||
applies the `.sip`, and installs the bootloader
|
||||
|
||||
---
|
||||
|
||||
### Phase 7 — The full-stack acceptance test *(served its purpose; removed)*
|
||||
|
||||
For a while this phase was a ported copy of a well-known 1993 shooter, kept as
|
||||
an acceptance test rather than a feature. The reasoning was sound and is worth
|
||||
keeping even though the test is gone: a large program that somebody else wrote,
|
||||
which exercises `/dev/fb0`, `/dev/input/event0`, `mmap`, the ELF loader,
|
||||
`gettimeofday` and a fixed-rate tic loop *simultaneously* and notices when any
|
||||
of them lie, is a much harsher test than anything written in-house — because it
|
||||
was not written against our bugs.
|
||||
|
||||
It found real ones, all of which are fixed and covered by narrower tests now:
|
||||
|
||||
- `fcntl(F_SETFL, O_NONBLOCK)` was a documented no-op, so the first read blocked
|
||||
forever and looked exactly like a hung renderer.
|
||||
- `/dev/input/eventN` replayed its backlog to each new opener, handing a program
|
||||
the keystrokes that had launched it as input.
|
||||
- Without `EVIOCGRAB`, a full-screen program and the shell behind it both read
|
||||
the same keystrokes.
|
||||
|
||||
It was removed once it stopped finding anything: the port was ~100k lines of
|
||||
third-party source under licences that can never ship in an Apache-2.0 image,
|
||||
and carrying it had a standing cost — 33 MB of non-redistributable data staged
|
||||
into `sysroot/`, which is linked into the kernel binary for a ramdisk-root
|
||||
build. That combination is a licensing accident waiting to happen, for a test
|
||||
that was passing every time.
|
||||
|
||||
**What replaced it:** `tools/bootcheck.ps1` with its `-Sock`, `-Evdev` and `-Ps`
|
||||
modes, plus `tools/splashshot.ps1` for the boot path. These are narrower and do
|
||||
not prove the whole stack composes, which is the honest cost of the removal.
|
||||
|
||||
**The lesson worth keeping** is the shape of the test, not the program: the next
|
||||
full-stack test should be something large that we did not write, driven to a
|
||||
real pixel-level assertion rather than a log line. `tools/splashshot.ps1`'s
|
||||
colour counting exists because a log-only harness once passed on a blank screen.
|
||||
|
||||
### Phase 8 — The system layer: `schinit`, firmware, and a crash channel
|
||||
|
||||
Three ideas parked here rather than started, because each one is a project and
|
||||
the phases above are the agreed order. Recorded so they are not re-derived from
|
||||
scratch later.
|
||||
|
||||
**`schinit`, a launchd-shaped init.** The point of copying launchd rather than
|
||||
sysvinit is not the config format, it is *demand launching*: launchd holds the
|
||||
listening socket or watches the path, and starts the job when something actually
|
||||
arrives. That inverts the dependency graph — jobs stop needing to declare what
|
||||
must start before them, because the thing they depend on answers on first use.
|
||||
Schunix already has the two pieces that makes possible: `AF_UNIX` sockets
|
||||
(`kernel/net/unix.c`) and a real `/proc`. What is missing is a supervisor that
|
||||
can hold a socket on a job's behalf and hand it over on exec, plus a job
|
||||
description format. Prerequisite for this being worth anything: `ps` and `top`
|
||||
must be able to enumerate processes, or the supervisor is unobservable.
|
||||
|
||||
**A crash screen you can work in, and a QR code channel out of it.** PSOGM
|
||||
(`kernel/core/panic.c`) currently renders and stops. The idea is two staged
|
||||
things, and the staging matters:
|
||||
|
||||
- *An interactive mode — "a small OS inside the crash".* This is the right
|
||||
framing and it is worth stating precisely, because it decides the design. At
|
||||
panic time the kernel is the only thing running, which is exactly why it can
|
||||
offer what a live system cannot: walk the stack, read another process's
|
||||
memory, dump page tables, inspect the process table — with no scheduler and no
|
||||
locking to respect, because nothing else can race you.
|
||||
|
||||
What makes it a *system* rather than a print statement is that it keeps the
|
||||
base machine's drivers. Concretely, the ones that work with interrupts off:
|
||||
the VGA/framebuffer (`vga_draw_pixel` is a plain memory write), the PS/2
|
||||
keyboard (pollable via port 0x60/0x64), and the serial port. Those three are
|
||||
a complete console. Everything else — IDE, USB, the timer, anything that
|
||||
sleeps or takes a sleeplock — is off limits, because the lock it needs may be
|
||||
precisely the one held by the CPU that just died.
|
||||
|
||||
So the shape is a **polled event loop over preallocated state**: no `kalloc`,
|
||||
no `sleep`/`wakeup`, no `acquire` on anything a live path might hold, one
|
||||
static command buffer, and a driver set restricted to the three above. That
|
||||
restriction is not a limitation to work around later; it is the invariant that
|
||||
makes the mode trustworthy when everything else is not. Anything that wants
|
||||
more should be answered by rebooting into a recovery environment instead.
|
||||
|
||||
A useful precedent to read but not copy: DDB (NetBSD/FreeBSD's in-kernel
|
||||
debugger) is exactly this, and its hard-won rule is the same one — a debugger
|
||||
that can deadlock is worse than no debugger.
|
||||
- *A QR code.* **Done.** `kernel/core/qr.c` (byte mode, EC level L, versions
|
||||
1–6, no allocation) and a card on the right of the crash screen. A panic
|
||||
report is worth nothing if it cannot leave a machine whose network stack is
|
||||
dead, and a phone camera is an out-of-band channel needing no driver on our
|
||||
side. Deliberately *not* a URL: it encodes the report itself and reaches
|
||||
nothing.
|
||||
|
||||
It ended up unconditional rather than behind Enter, which is better — it
|
||||
costs nothing to always draw, and requiring a keypress would mean the one
|
||||
case where input is least trustworthy is also the case where the report is
|
||||
hardest to retrieve. Verified end-to-end by `make crashshot && make crashqr`,
|
||||
which decodes the symbol out of the captured framebuffer.
|
||||
|
||||
**UEFI.** Not a feature so much as the point at which the current bootloader
|
||||
stops being extendable: it is 16-bit real mode, INT 10h VBE, and PIO ATA, none
|
||||
of which exist on a UEFI-only machine. The realistic shape is a *second*
|
||||
loader — an EFI application that gets the framebuffer from GOP, the memory map
|
||||
from `GetMemoryMap()`, loads the kernel off a FAT ESP, and enters at the same
|
||||
place `boot/stage2main.c` does — rather than a port of the existing one. Worth
|
||||
noting that the splash work just done survives that transition: it is a mask, a
|
||||
palette and a layout, none of which care how the pixels were found. What does
|
||||
not survive is every assumption in `stage2main.c` about where things live in low
|
||||
memory (see `docs/layout.md`).
|
||||
|
||||
### SchFS beyond Schunix
|
||||
|
||||
Three ideas parked 2026-08-08, in the order they should happen.
|
||||
|
||||
**An on-disk format version, and an upgrade path.** This one is not optional and
|
||||
gets harder every day it is postponed: the moment an image exists that someone
|
||||
cares about, changing the layout means being able to read the old one. The
|
||||
superblock already carries a version field and a CRC32c; what is missing is a
|
||||
*policy* — a documented rule for what a version bump means, a refusal to mount
|
||||
something newer than the driver understands (rather than misreading it), and an
|
||||
explicit upgrade step. Cheap now, expensive later, and it blocks the other two:
|
||||
a second implementation is only meaningful against a format that is pinned.
|
||||
|
||||
**A Linux implementation.** The realistic first form is **FUSE**, not a kernel
|
||||
module — it needs no GPL entanglement (`libfuse` is LGPL and used across a
|
||||
process boundary), it can reuse the read/write logic already in `tools/mkschfs`
|
||||
and `tools/schfs_fsck` rather than being written twice, and it turns "does our
|
||||
filesystem work" into something testable with `fsx` and `xfstests`, which is a
|
||||
far harsher examination than anything in this tree. A Linux *kernel* module is a
|
||||
different project and carries the licence problem that
|
||||
[licensing.md](licensing.md) describes.
|
||||
|
||||
**The missing features.** Worth listing honestly before adding any: no journal
|
||||
replay across an unclean shutdown beyond what CoW gives for free, no snapshots
|
||||
(despite the B-tree being copy-on-write, which is most of the work), no extended
|
||||
attributes, no sparse-file support, and no online `fsck`. Snapshots are the one
|
||||
where the existing design already pays for most of the cost.
|
||||
|
||||
---
|
||||
|
||||
## 3b. Known broken
|
||||
|
||||
Two things that are wrong *now*, both reported 2026-08-08 and both diagnosed
|
||||
rather than merely observed. Neither blocks the GUI work, which is why they are
|
||||
here and not in a phase.
|
||||
|
||||
### ~~SMP is broken~~ *(fixed 2026-08-08)*
|
||||
|
||||
**SMP works.** One, two and four processors all boot; every CPU reports in with
|
||||
`cpu%d: starting`, and `/proc/cpuinfo` lists them all. `smp_start_aps` in
|
||||
`kernel/core/main.c` is 1.
|
||||
|
||||
Five distinct bugs, each hidden behind the one before it, and all invisible
|
||||
under QEMU because it publishes only one processor in the legacy MP table.
|
||||
VMware published the real count, which is why the triple fault only ever
|
||||
appeared there.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| CPU enumeration | `mpinit()` read the 1994 MP table. `kernel/arch/x86_64/acpi.c` now reads the ACPI MADT. |
|
||||
| triple fault | `entry64high` had **no AP branch** — every AP took the BSP's stack and ran `kmain()` twice. Now tests `EBX == 1`. |
|
||||
| `panic("acquire")` | `seginit()` called `kalloc()` before `%fs` existed; kalloc locks, locking reads `cpu`. BSP now preallocates each AP's page. |
|
||||
| `panic("sched locks")` | `kernel_fs_base` was **one global** read by `trapasm.S` on every entry, so all CPUs shared one per-CPU block. Now `%gs:0`. |
|
||||
| null `proc` in `sleep()` | `cpu`/`proc` were `__thread` — per *thread*, so the compiler cached the thread pointer across `sched()`, but `%fs` is per *CPU*. Now `mycpu()`/`myproc()`. |
|
||||
| `#GP` in `syscall_entry` | The new per-CPU slots landed on `%fs:(8)`, the syscall scratch word — the same memory as `%gs:8`. Moved to +16 and +24. |
|
||||
|
||||
The `__thread` one is the lesson worth keeping: **`__thread` is wrong for
|
||||
per-CPU kernel data.** It means per thread, and the compiler is entitled to
|
||||
cache the thread pointer across calls on that basis. A kernel thread changes CPU
|
||||
at every `sched()`, so the assumption is false for exactly the variables that
|
||||
most need to be right. `include/proc.h` carries the disassembly that proves it.
|
||||
|
||||
Per-CPU block layout, shared with assembly and not to be reordered:
|
||||
|
||||
+0 self-pointer x86-64 TLS ABI (%fs:0)
|
||||
+8 syscall scratch trapasm.S parks %rsp here
|
||||
+16 struct cpu *
|
||||
+24 struct proc *
|
||||
|
||||
### ~~The kernel demands 224 MB of RAM~~ *(fixed 2026-08-08)*
|
||||
|
||||
`kernel/arch/x86_64/e820.c` reads the firmware's memory map and sets `phystop`
|
||||
from it. **The floor went from 224 MB to 32 MB**, and the kernel now sees all
|
||||
the memory a machine actually has: 512 MB in gives `MemTotal: 524160 kB`, where
|
||||
before it reported 224 MB whatever was installed.
|
||||
|
||||
Three details worth keeping:
|
||||
|
||||
- It takes the end of the usable region **containing the kernel**, not the
|
||||
highest usable address anywhere. A memory map has reserved holes in it, and
|
||||
`freerange()` walks straight from the end of the kernel image to `phystop`
|
||||
handing out every page between — so the global maximum would put firmware
|
||||
tables on the free list, failing later and elsewhere and looking like anything
|
||||
but a memory map problem.
|
||||
- The ceiling is **3 GB**, because `kvmalloc()` maps 0–4 GB as four 1 GB pages
|
||||
and marks the top one uncached for PCI BARs.
|
||||
- Below `PHYSTOP_MIN` it prints why and stops, rather than triple faulting in
|
||||
silence. At 24 MB: `e820: only 23 MB usable; Schunix needs at least 24 MB`.
|
||||
|
||||
The original text is kept below because the reasoning still applies to the SMP
|
||||
half of §3b, which is the same shape of bug: a constant nobody revisited.
|
||||
|
||||
### The kernel demanded 224 MB of RAM
|
||||
|
||||
**Symptom as reported:** the OS needs ~256 MB, which is absurd next to Windows 98
|
||||
or an early Linux.
|
||||
|
||||
**Diagnosis:** it is one constant. `PHYSTOP` in `include/memlayout.h` is
|
||||
`0xE000000` — 224 MB — inherited unchanged from xv6, and `main()` hands
|
||||
`P2V(PHYSTOP)` straight to `kinit1()`/`dma_init()`. The kernel therefore
|
||||
*assumes* that much physical memory exists and walks off the end of RAM if it
|
||||
does not. Measured 2026-08-08: boots at 224M and above, triple faults at 192M
|
||||
and below. Nothing is actually being *used* — the requirement is entirely
|
||||
fictional.
|
||||
|
||||
**The fix is already half-built.** `boot/stage2.S` collects a full E820 memory
|
||||
map in real mode and leaves it at `VBE_INFO_ADDR + 0x200` (count) and `+ 0x208`
|
||||
(24-byte entries); `boot/stage2main.c` already parses it to print detected RAM.
|
||||
**The kernel never reads it.** So: have `main()` take the top of the highest
|
||||
usable E820 region instead of the constant, clamp it to what the direct map
|
||||
covers, and keep `PHYSTOP` only as a fallback for the case where the firmware
|
||||
returned nothing. That should drop the floor to something in the tens of
|
||||
megabytes and is a contained change.
|
||||
|
||||
---
|
||||
|
||||
## 4. Kept out of scope on purpose
|
||||
|
||||
- Anything that needs pushing or publishing. This work stays local.
|
||||
Loading…
Reference in a new issue