SchOS/docs/architecture/schapi.md

# SchAPI, and SchOS as a system

Status: **direction agreed, not yet implemented.** Nothing in this document
changes what is being built this week; it changes what everything is being
built *towards*. See §7 for what actually happens now.

---

## 1. The decision

SchOS stops being "a Unix-like kernel with a userland bolted on" and becomes
a **system**: a kernel, a native API, and the services and applications that
sit on it, designed together.

The model is NeXTSTEP/macOS, not Linux. Specifically:

- A kernel that provides mechanism, with **more than one personality** on top.
- **POSIX/Unix is a personality, not the identity.** It stays, completely and
  seriously, because it is how you get software. It is not the thing programs
  are expected to be written against.
- **SchAPI** is the native interface: higher-level, object-shaped, designed for
  the GUI and services rather than retrofitted onto 1970s process semantics.

Two systems are being borrowed from, and they are *different* systems. It is
worth being precise about which, because they disagree:

| Borrowed | From | Why |
|---|---|---|
| Layered architecture, personalities, native API over a Unix substrate | macOS / NeXTSTEP | It is the only mainstream design that got a modern API onto a Unix base without abandoning either |
| Permanent backward compatibility | **Windows** | macOS is emphatically *not* this — Apple dropped 32-bit apps, Carbon, kexts and OpenGL, each time on a deadline |

That mix is coherent, but it is a mix. Taking the architecture from Apple and
the compatibility policy from Microsoft is a deliberate choice, and the rest of
this document is mostly about making that choice survivable — because
compatibility-forever is the single most expensive promise an OS can make, and
Windows pays for it every day.

---

## 2. The one mechanism that makes the promise affordable

**The stable interface is a library, not the syscall table.**

This is the whole trick, and it is what lets macOS keep decade-old binaries
running while freely renumbering its syscalls. On macOS, nothing may issue a
syscall directly; you link `libSystem`. Apple therefore owes compatibility only
at the library boundary, and the kernel stays editable forever.

Linux made the opposite choice — the syscall ABI *is* the contract, "we do not
break userspace" — and consequently every syscall mistake ever made is still
there, with a `2` or a `64` suffixed to the name of its replacement.

So, for SchOS:

```
    ┌──────────────────────────────────────────────┐
    │  applications, SchWM, SchDE                  │
    ├──────────────────────────────────────────────┤
    │  SchAPI          │  POSIX / Linux personality│   ← THE STABLE BOUNDARY
    ├──────────────────┴───────────────────────────┤
    │  libschos — the only thing that may syscall│
    ├──────────────────────────────────────────────┤
    │  syscall table — PRIVATE, unstable, renumberable
    ├──────────────────────────────────────────────┤
    │  kernel                                      │
    └──────────────────────────────────────────────┘
```

Rules that follow, and they are not negotiable if the promise is to hold:

1. **A program that issues `syscall` directly is unsupported.** It may break at
   any release. Only `libschos` may.
2. `libschos` exports versioned symbols. An entry point's behaviour never
   changes; a changed behaviour is a new version of the symbol.
3. The kernel may renumber, merge, split or delete syscalls at will, provided
   `libschos` still presents the same surface.

This *reconciles* with the existing decision to adopt Linux syscall numbering
(`docs/roadmap.md`, and the reason musl/toybox/mksh work at all). Linux
numbering does not go away — it becomes the **Linux personality's** entry path,
which is exactly what it already is in practice. It is a compatibility surface
that happens to be entered via `syscall`, and it is frozen because *Linux*
froze it, not because we did.

---

## 3. Stability tiers

"Nothing can ever be removed" is only affordable if you are precise about what
"anything" covers. Windows' real problem was never the documented API — it was
that undocumented behaviour became load-bearing, and then it was too late.

Every public interface carries exactly one tier:

| Tier | Promise | May change |
|---|---|---|
| `stable` | Forever, from the release that introduced it | Never. Additions only. |
| `provisional` | Until the next release | Freely |
| `internal` | None | Freely, without notice |

**Everything in SchOS today is `provisional`.** The freeze starts at 1.0 and
covers only what is marked `stable` at that moment. This is not a way of
dodging the commitment — it is the only way to make it, because a promise made
now would permanently enshrine, among other things, a `fcntl(F_SETFL)` that
was a documented no-op until last week and a `struct file` whose `O_NONBLOCK`
handling had never been exercised.

Nothing becomes `stable` by being old. It becomes `stable` by someone deciding
it is right and writing that down.

---

## 4. `compat/` — what it actually means

The instinct is right: old interfaces should not litter the current tree. But
moving *source* into a folder does not reduce the *runtime* cost, and the
runtime cost is the one that hurts.

The rule that keeps compatibility cheap:

> **A compatibility entry point must be implementable as a translation into the
> current API. If it cannot be, that is a defect in the current API — fix the
> current API before freezing it.**

So `compat/` holds shims, not parallel implementations. A shim:

- lives in `compat/<release-it-was-current-in>/`,
- is written in terms of today's API and nothing else,
- is never called by new code,
- carries its own tests, which are the only thing that proves the promise is
  still being kept.

When a shim *cannot* be expressed as a translation, that is the alarm. It means
the new API lost information the old one carried, and it is far cheaper to
notice that before 1.0 than to maintain two real implementations forever. That
is precisely the trap Win32 fell into.

---

## 5. Taking from the BSDs

`src-trunk/` is a NetBSD checkout, and a FreeBSD one is being added alongside
it. Both are reference material and parts bins; neither is built and neither
may ever end up in an image.

**Two trees is not redundancy.** They are worth having together for three
reasons that a single tree cannot give:

- Where an RFC is ambiguous — and TCP's are, constantly — two independent
  implementations agreeing is far stronger evidence than one implementation
  asserting.
- Where one is licence-encumbered, the other may not be. NetBSD's
  `tcp_input.c` carries a WIDE Project advertising clause that Berkeley's
  rescission does not cover; FreeBSD's equivalent may well be 2-clause. See
  [../licensing.md](../licensing.md) §3.
- They have opposite biases, and knowing which you are reading matters:

| | FreeBSD | NetBSD |
|---|---|---|
| Stack quality | The better stack in absolute terms — faster, more complete, more actively developed | Solid, conservative, less featureful |
| Extractability | Welded to FreeBSD: `epoch(9)`, `uma`, `netisr`, VIMAGE/vnet, `if_t`, a rich locking vocabulary | Portability *is* the project's identity; assumes far less of its host kernel |
| Use it as | The reference you check yourself against | The code you actually lift |

That table is the same lesson §1 of the roadmap already learned the hard way
with FreeBSD's `sys/dev/usb`: 100k lines welded to newbus, where porting the
USB stack means porting the device framework first. Networking is the same
shape. Read FreeBSD, lift NetBSD, write SchOS.

### Licensing — read this before copying anything

NetBSD is not uniformly licensed: 2-clause, UC 3-clause and a minority of
4-clause-with-advertising files all appear, and only the first two are
comfortable. The per-template rules, the measured counts, and the one-line
check to run before copying a file are in [../licensing.md](../licensing.md)
§3. Do not copy from `src-trunk/` without reading it.

### What is actually worth taking

Ranked by value, not by size:

1. **`sys/net`, `sys/netinet` — the TCP/IP stack.** The largest single win
   available. Phase 2 of the roadmap wants ARP/IPv4/ICMP/UDP/TCP, and this is a
   mature, clean, readable, 2-clause-heavy implementation of exactly that.
2. **`sys/kern/subr_pool.c` — the pool/slab allocator.** SchOS has `kalloc`
   and a page at a time. Every subsystem written since has invented its own
   fixed-size table to work around that.
3. **`autoconf` — the driver framework** (`config(1)`, `cfdriver`/`cfattach`,
   `device_t`). This is the grown-up version of what `kernel/drivers/registry.c`
   and `kernel/dev/irq.c` are hand-rolling. Adopt the *model*; do not try to
   port the code wholesale.
4. `sys/kern/subr_*` generally — rbtree, workqueue, condvar, kmem.

Low value: `lib/libc` (musl already fills that role), `sys/dev/usb` (we have a
working xHCI stack written from the spec).

**Take designs before code.** Every file copied is a file to be understood,
relicensed-in-notice, and maintained forever under the §3 promise.

---

## 6. What SchAPI is, concretely

Not yet designed. What is decided about it:

- It is the native interface; POSIX is available beneath it but not the idiom.
- It is reached through `libschos`, never by direct syscall.
- It is designed *with* Schcom in mind, because a compositor and its clients
  are the first real users and their needs (IPC, shared buffers, event streams,
  object lifetimes) are what will shape it.
- The IPC primitive comes first. Everything in a NeXT-shaped system rests on
  it, and unix domain sockets — already built and working — are the substrate,
  not the interface.

What is explicitly *not* decided: whether it is object-oriented in the Objective-C
sense, what language it is expressed in, and whether it takes Mach's port model
or something smaller. Those need their own document, written when Schcom has
told us what it actually needs.

---

## 7. What changes now

Almost nothing, on purpose. The ordering does not change:

- **The native Windows build must boot first.** The clang-built kernel
  currently reaches `sinit` and dies on the first trap. A direction is worth
  nothing while the tree does not run.
- Then the roadmap continues as written: network, GPU, fonts, Schcom.

What this document changes *today* is only this:

1. `src-trunk/` is reference material, excluded from every build and image.
2. New interfaces are marked `provisional` from now on, so that §3 has
   something to work with at 1.0.
3. When the network stack is written, NetBSD's is the starting point rather
   than a blank file — with the licence check in §5 applied per file.
4. `libschos` is created when there is a second personality to justify it.
   Until then, the *rule* stands even though the library does not exist yet:
   no new first-party program issues raw syscalls.