3034efd300
The network thread waited on the timer and nothing else, so a frame that landed just after a poll sat in the ring for up to 10 ms -- the largest fixed latency in the system. That was deliberate when the stack was new (a receive path woken by an interrupt is much harder to debug than one that is asked) and there is now a working stack to compare against, which is exactly when it becomes safe to do. The frame is still *pulled* by the thread through .receive. It has to be: a call into lwIP can reach the driver, the driver can sleep on the buffer cache, and an interrupt handler may do neither. netdev_rx_signal() delivers nothing and reads no ring -- it only ends the wait early. The wait stays on the timer tick rather than moving to a channel of its own, and that is the design. lwIP's timers must still run when the link is silent, so the clock has to remain a floor under the wait, and sleep() cannot wait on two things at once. Joining the tick's existing wakeup costs nothing and leaves one wait with one predicate. rx_signal lives under tickslock for the same reason it gets no lock of its own: sleep() releases tickslock atomically, so a driver setting the flag and waking under that lock cannot lose the wakeup by arriving between the predicate test and the sleep. A private lock would not be held across the sleep, and that window would be back. Calling it from interrupt context is safe on the same terms the timer handler already relies on -- acquire() disables interrupts, so a CPU holding tickslock cannot be interrupted into it. The e1000 signals on RXT0 (a frame or burst has landed) and RXDMT0 (the ring has fallen below its threshold, which is when waiting for the clock hurts most). Measured rather than asserted: /proc/netif reports wake_rx and wake_tick, and across three pings rx_frames and wake_rx both went 2 -> 6 -- every received frame woke the thread through the interrupt. wake_tick keeps advancing as the idle floor, which is what lwIP's timers need. socktest still passes and ping is still 0% loss. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
174 lines
8.2 KiB
Markdown
174 lines
8.2 KiB
Markdown
# Roadmap — everything else still open
|
|
|
|
The three current decisions have their own documents; see
|
|
[README.md](README.md). This file holds what is left: work that is scoped,
|
|
agreed and unstarted, with nothing in it that is finished.
|
|
|
|
**Finished work is not here.** It moved to
|
|
[../architecture/](../architecture/) and was rewritten as a description of what
|
|
it produced — see [the rule](../README.md#the-rule-that-keeps-this-from-rotting).
|
|
What used to be phases 1 through 4 of this file, several hundred lines of ticked
|
|
boxes, is now `usb`, `networking.md`, `graphics.md` and `devices.md` over there.
|
|
|
|
---
|
|
|
|
## 1. Storage and the boot path
|
|
|
|
**Make a USB disk mountable and bootable.** The xHCI stack, the bulk-only
|
|
transport, the SCSI layer and the block device all work — a superblock reads back
|
|
through `bread()`. What is missing is that **the root device is compiled in
|
|
rather than selectable**, so nothing can be told to mount it.
|
|
|
|
This is also the first piece of the installer, and it is the single change that
|
|
would most enlarge what SchOS can boot from.
|
|
|
|
## 2. Graphics
|
|
|
|
- **A `/dev/dri`-style device with mode setting**, so resolution stops being
|
|
whatever the loader asked for. Worth doing *with*
|
|
[hybrid.md](hybrid.md)'s GPU eviction rather than before it — moving the driver
|
|
out is what makes a modeset API affordable to design properly.
|
|
- **Double buffering and vblank.** `display_ops` specifies both and no driver
|
|
implements either. Schcom needs both to not tear.
|
|
- **Port the console onto the text layer and delete the cell grid.** The layer
|
|
exists, the rasteriser exists, and the console still indexes `font_data[256][16]`.
|
|
- **Load a font from disk** and point the console at it.
|
|
- **CFF/Type 2 charstrings**, so OTF fonts work. A separate `font_ops` provider
|
|
and a bytecode interpreter, *not* an extension of the `glyf` path.
|
|
- **Text layout** — advance widths, kerning — as a library in `lib/`, so the
|
|
terminal and the file manager are not each solving it.
|
|
|
|
The console keeps its fixed grid until the first item above; proportional text
|
|
belongs to the GUI.
|
|
|
|
## 3. Networking
|
|
|
|
Three things the stack owed on the day it landed, all still owed:
|
|
|
|
- **`accept()` does not report the peer's address**, and there is no
|
|
`getsockname`/`getpeername`. Nothing has needed them yet; a server that wants
|
|
to log who connected will.
|
|
- **No routing table.** With two interfaces up, the first is the default route
|
|
and the second is reachable only by an address on its own subnet. That is a
|
|
rule rather than an accident, and it is not routing.
|
|
|
|
`setsockopt` returning 0 for everything is in
|
|
[foundations.md](foundations.md) instead, because the Wayland work needs it.
|
|
|
|
## 4. The interactive crash screen
|
|
|
|
PSOGM renders and stops. The idea is *a small OS inside the crash*, and the
|
|
framing decides the design:
|
|
|
|
> At panic time the kernel is the only thing running, which is exactly why it can
|
|
> offer what a live system cannot: walk the stack, read another process's memory,
|
|
> dump page tables, inspect the process table — with no scheduler and no locking
|
|
> to respect, because nothing else can race you.
|
|
|
|
What makes it a *system* rather than a print statement is that it keeps the base
|
|
machine's drivers. Concretely, the ones that work with interrupts off: the
|
|
framebuffer (`gfx_draw_pixel` is a plain memory write), the PS/2 keyboard
|
|
(pollable via ports 0x60/0x64), and the serial port. **Those three are a complete
|
|
console.**
|
|
|
|
Everything else — IDE, USB, the timer, anything that sleeps or takes a
|
|
sleeplock — is off limits, because the lock it needs may be precisely the one
|
|
held by the CPU that just died.
|
|
|
|
So the shape is a **polled event loop over preallocated state**: no `kalloc`, no
|
|
`sleep`/`wakeup`, no `acquire` on anything a live path might hold, one static
|
|
command buffer, and that driver set. **That restriction is not a limitation to
|
|
work around later; it is the invariant that makes the mode trustworthy when
|
|
everything else is not.** Anything wanting more should be answered by rebooting
|
|
into a recovery environment.
|
|
|
|
A useful precedent to read but not copy: DDB, the BSD in-kernel debugger, is
|
|
exactly this, and its hard-won rule is the same one — *a debugger that can
|
|
deadlock is worse than no debugger.*
|
|
|
|
## 5. The installer and `.sip`
|
|
|
|
Scoped in [`projects/schinst/README.md`](../../projects/schinst/README.md),
|
|
which holds the detail. In short:
|
|
|
|
- [ ] `.sip` format specification, **written down before any code**
|
|
- [ ] `tools/mksip` — capture a directory tree into a package
|
|
- [ ] The apply path
|
|
- [ ] An installation environment that boots from the ISO, partitions, applies
|
|
the `.sip`, and installs the loader
|
|
|
|
`.sip` is modelled on WIM, and the thing worth copying from WIM is that it is
|
|
**file-based, not block-based**: one file holds a whole filesystem, applies to a
|
|
partition of any size, and several images can share their common files.
|
|
|
|
**Most of this already exists.** `tools/mkschfs.c` and `tools/mkfs.c` both do
|
|
content dedup with an `(st_dev, st_ino)` fast path, and SchFS's B-tree is content
|
|
addressed. `mksip` should share that code rather than reimplement it — that is
|
|
the reason the dedup was factored the way it was.
|
|
|
|
Open questions, all wanting a decision before code:
|
|
|
|
- **Compression.** Something with a small decompressor is worth more than
|
|
something with a good ratio, because the decompressor runs in the installation
|
|
environment.
|
|
- **Multiple images per `.sip`** — WIM's real trick, install/recovery/boot
|
|
sharing one blob store. Worth the complexity now, or not?
|
|
- **Can the kernel mount a `.sip` read-only directly?** It is close to a
|
|
filesystem already, and if it can, the installation environment's root can
|
|
simply *be* the `.sip`.
|
|
|
|
The GUI for this is a Wayland client; see [wayland.md](wayland.md).
|
|
|
|
## 6. SchFS beyond SchOS
|
|
|
|
In the order they should happen.
|
|
|
|
**~~An on-disk format version~~, and an upgrade path.** The version policy is
|
|
done — see [../architecture/schfs.md](../architecture/schfs.md). What remains is
|
|
the **upgrade step**, and it cannot be written until there are two formats to
|
|
convert between; the first bump is when that becomes real work rather than
|
|
speculation.
|
|
|
|
The format is now pinned, which is what the other two items were waiting on.
|
|
|
|
**A Linux implementation, FUSE first.** Three reasons in order of weight: it can
|
|
be tested with `fsx` and `xfstests`, which are far harsher than anything written
|
|
alongside the code; it carries no licence entanglement, since libfuse is LGPL and
|
|
used across a process boundary; and it can reuse the read/write logic already in
|
|
`tools/mkschfs` and `tools/schfs_fsck` rather than being written twice and having
|
|
the two disagree.
|
|
|
|
A kernel module is a different project, comes only after FUSE passes `xfstests`,
|
|
and carries the licence problem [../licensing.md](../licensing.md) describes.
|
|
|
|
**The missing features**, listed honestly before adding any: no journal replay
|
|
across an unclean shutdown beyond what copy-on-write gives for free, no
|
|
snapshots, no sparse-file support, and no online `fsck`. **Snapshots are the one
|
|
where the existing design already pays for most of the cost** — the B-tree is
|
|
copy-on-write, which is most of the machinery; nothing takes or names a snapshot.
|
|
|
|
## 7. The kernel's own gaps
|
|
|
|
Not phases, and each one improves everything above it:
|
|
|
|
- **A slab or pool allocator.** `kalloc` hands out one page at a time, and every
|
|
subsystem written since has invented its own fixed-size table to work around
|
|
that. This is the single change that would most improve the rest of the kernel.
|
|
- **A page cache**, which is what a general writable `MAP_SHARED` needs.
|
|
[foundations.md](foundations.md) delivers a `memfd` instead, which is narrower
|
|
and sufficient for the GUI.
|
|
- **Symlinks in procfs**, so `/proc/<pid>/exe` can be one and `ps -o comm` stops
|
|
printing `[init]`. Belongs with the rest of the VFS work rather than on its own.
|
|
- **The `ls -lR` hang.** Heavy directory listing across many commands in one long
|
|
shell session eventually makes a *later*, unrelated directory hang; the same
|
|
command succeeds on a fresh boot. Not inode or file table exhaustion — both
|
|
fail cleanly and neither fired. Next step: instrument `itable`/`ftable`
|
|
occupancy across a long session and watch what climbs.
|
|
|
|
---
|
|
|
|
## Kept out of scope on purpose
|
|
|
|
- Anything that needs pushing or publishing. This work stays local.
|
|
- TrueType hinting.
|
|
- XWayland.
|