nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

ldls: the stubs, and the driver's own voice

Browse source 
docs/plan/ldls.md milestone L1. Every undefined symbol resolves to a
trampoline -- twenty generated bytes that load the symbol's index and jump to
a shared thunk, which saves the argument registers and reports the call. Then
init_module is simply called.

    $ ldls-run 8139too.ko
      module  8139too
      license GPL
      wants   94 symbols
      calling init_module at 0x416fc020

        1  _printk(0x416fb6c0, 0, 0)
          | 8139too: 8139too Fast Ethernet driver 0.9.28
        2  __pci_register_driver(0x416fb400, 0x416fff80, 0x416fbdbb)

      init_module returned 0 after 2 stub call(s)

That second line is the driver's own banner, printed through a _printk that
is ours, from a string in the module's own .rodata that a relocation put
there. There is no stronger evidence that a load worked than the loaded thing
speaking.

Stubs return zero rather than stopping, which departs from what the milestone
originally said and does so deliberately: stopping at the first call tells you
one name, carrying on tells you the sequence, and the sequence is the thing a
symbol table cannot give you. A driver that then dereferences the zero faults,
and that is a result too -- it says how far it got before it needed the kernel
to have really done something.

The first run failed on neither of those. It failed on a relocation:

    ldls-run: relocate: a signed 32-bit relocation does not fit

A module is compiled -mcmodel=kernel, so the compiler emits R_X86_64_32S for
anything it believes lives in the top 2 GB, and an ordinary mmap at 0x7f...
has nowhere to put it. Linux gives modules a region within 2 GB of the kernel
text; the harness uses MAP_32BIT to reach the same window from the other end.
On SchOS this costs nothing -- a process's mmap area starts at 0x40000000 and
is already inside it -- but it is a real constraint on where the host may load
a module, and a loader that did not check its own arithmetic would have found
it much later and much more confusingly.

Swept over 500 stock modules: 449 ran init to completion, 22 faulted on a
stub's zero, 29 have no init_module, and zero hit a loader error.

Recorded honestly in "what it will not do": running init is not working. Every
symbol is a stub, so what has been shown is that drivers ask for things in a
readable order, not that anything answers. Also recorded: every undefined
symbol in a module is NOTYPE, so the stubs cannot tell jiffies from
pci_iomap and both get a trampoline -- BTF distinguishes VAR from FUNC and
that is where the fix belongs rather than in a guess.

make ldls-check is 23 checks now, five of them L1's, all against
ldls/test/refmod.c which is ours and now has an init_module of its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz 2026-09-14 08:02:29 +02:00
commit 7a7c7611e0
8 changed files with 636 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

301
ldls/host/run.c Normal file
View file

@ -0,0 +1,301 @@
// SPDX-License-Identifier: Apache-2.0
//
// ldls-run -- load a module and call its init, on the build machine.
//
// docs/plan/ldls.md milestone L1. L0 answered "what does this driver ask
// for"; this answers "and in what order does it ask", which is the other thing
// that cannot be guessed and falls straight out of running the thing.
//
// This is a **development harness, not the deliverable.** LDLS proper runs on
// SchOS in a process, reaching hardware through /dev/pci. Running the same
// loader here first is the arrangement mkschfs already has with the kernel,
// and for the same reason: the failures at this stage are loader failures, and
// one found here costs a second instead of a boot.
//
// ldls-run 8139too.ko
//
// Nothing touches hardware. Every symbol the module wants resolves to a stub
// that logs and returns zero, so a driver gets as far as its first real
// dependency on the kernel doing something, and then says so.
//
// **No module is committed to this tree.** See docs/plan/ldls.md rule 1.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <setjmp.h>
#include <stdarg.h>
#include <sys/mman.h>
#include "../ko.h"
#include "../stub.h"
// --- what the stubs report ---------------------------------------------------
static const char **g_names;
static int g_count;
static int g_calls;
static int g_verbose;
// A handful of symbols are worth actually implementing rather than logging,
// because what they do is print -- and a driver printing its own banner
// through our code is the clearest evidence there is that the load worked.
static int
is_printk(const char *n)
{
return strcmp(n, "_printk") == 0 || strcmp(n, "printk") == 0;
}
// Linux format strings are printf's with extensions: a leading KERN_SOH level
// byte, and %p followed by a letter meaning "pretty-print this kind of
// pointer". vprintf understands neither, and handing it a %pS would read an
// argument that is not there.
//
// So the level prefix is stripped and anything carrying a pointer extension is
// printed raw. Enough for a banner, honest about the rest, and the real
// implementation belongs with the rest of the shim.
static void
printk_like(const char *fmt, unsigned long *args)
{
const char *p = fmt;
const char *s;
if (!fmt) { printf(" (printk with a null format)\n"); return; }
if ((unsigned char)p[0] == 0x01 && p[1] >= '0' && p[1] <= '7')
p += 2; /* KERN_SOH + level */
for (s = p; *s; s++)
if (s[0] == '%' && s[1] == 'p' &&
((s[2] >= 'a' && s[2] <= 'z') || (s[2] >= 'A' && s[2] <= 'Z'))) {
printf(" | %s", p);
if (p[strlen(p) - 1] != '\n') printf("\n");
return;
}
printf(" | ");
printf(p, args[1], args[2], args[3], args[4], args[5]);
if (p[0] && p[strlen(p) - 1] != '\n')
printf("\n");
}
unsigned long
ldls_stub(long idx, unsigned long *args)
{
const char *name = (idx >= 0 && idx < g_count) ? g_names[idx] : "(?)";
g_calls++;
printf(" %3d %s(", g_calls, name);
printf("%#lx, %#lx, %#lx", args[0], args[1], args[2]);
printf(")\n");
if (is_printk(name))
printk_like((const char *)args[0], args);
if (g_verbose)
printf(" args: %#lx %#lx %#lx %#lx %#lx %#lx\n",
args[0], args[1], args[2], args[3], args[4], args[5]);
// Zero, so the driver carries on and shows the next call. Stopping at the
// first tells you one name; continuing tells you the sequence, which is what
// this milestone is for.
return 0;
}
// --- surviving the module -----------------------------------------------------
static sigjmp_buf g_jump;
static volatile int g_signal;
static void
on_fault(int sig)
{
g_signal = sig;
siglongjmp(g_jump, 1);
}
// --- loading ------------------------------------------------------------------
static unsigned char *
slurp(const char *path, unsigned long long *len)
{
FILE *f = fopen(path, "rb");
unsigned char *buf;
long n;
if (!f) return 0;
if (fseek(f, 0, SEEK_END) != 0) { fclose(f); return 0; }
n = ftell(f);
if (n <= 0) { fclose(f); return 0; }
rewind(f);
buf = malloc((size_t)n);
if (!buf) { fclose(f); return 0; }
if (fread(buf, 1, (size_t)n, f) != (size_t)n) { free(buf); fclose(f); return 0; }
fclose(f);
*len = (unsigned long long)n;
return buf;
}
struct rctx { struct ko *k; struct stubs *s; };
static int
resolve(void *ctx, const char *name, unsigned long long *out)
{
struct rctx *r = (struct rctx *)ctx;
int i;
for (i = 0; i < r->k->nundef; i++)
if (strcmp(r->k->undef[i], name) == 0) {
*out = stub_addr(r->s, i);
return 0;
}
return -1; // cannot happen: the list came from the object
}
// The module's entry point. `module_init(x)` makes `init_module` an alias for
// x, so it is an ordinary defined global and needs no struct layout to find --
// which matters, because the struct that *would* hold it is one of the things
// rule 4 says not to assume.
static unsigned long long
find_symbol(struct ko *k, const char *want)
{
int i;
for (i = 0; i < k->nsym; i++) {
const char *nm = k->str + k->sym[i].st_name;
if (strcmp(nm, want) != 0)
continue;
if (k->sym[i].st_shndx == KO_SHN_UNDEF ||
k->sym[i].st_shndx >= (unsigned short)k->nsh)
continue;
if (!k->sec[k->sym[i].st_shndx].alloc)
continue;
return k->sec[k->sym[i].st_shndx].addr + k->sym[i].st_value;
}
return 0;
}
int
main(int argc, char **argv)
{
struct ko k;
struct ko_section *sec;
struct stubs stubs;
struct rctx rc;
unsigned char *img, *mem, *code;
unsigned long long len, span, entry;
int nsec, i, rv = 0;
if (argc > 1 && strcmp(argv[1], "-v") == 0) { g_verbose = 1; argv++; argc--; }
if (argc < 2) {
fprintf(stderr, "usage: ldls-run [-v] <object.ko>\n");
return 2;
}
img = slurp(argv[1], &len);
if (!img) { fprintf(stderr, "ldls-run: cannot read %s\n", argv[1]); return 1; }
nsec = ko_sections_needed(img, len);
if (nsec < 0) {
fprintf(stderr, "ldls-run: %s: not an ELF64 relocatable object\n", argv[1]);
return 1;
}
sec = calloc((size_t)(nsec ? nsec : 1), sizeof(struct ko_section));
if (!sec) { fprintf(stderr, "ldls-run: out of memory\n"); return 1; }
if (ko_open(&k, img, len, sec, nsec) < 0) {
fprintf(stderr, "ldls-run: %s\n", ko_err(&k));
return 1;
}
printf("%s\n", argv[1]);
printf(" module %s\n", ko_name(&k) ? ko_name(&k) : "(not a module)");
printf(" license %s\n", ko_modinfo(&k, "license") ?
ko_modinfo(&k, "license") : "(none declared)");
printf(" wants %d symbols\n", k.nundef);
// Executable memory for both the module and the stubs. A module's .text has
// to be executed and a trampoline has to be jumped to, so neither can live
// in ordinary malloc'd memory on a machine that enforces W^X.
//
// **And it has to be low.** A module is compiled with -mcmodel=kernel, which
// lets the compiler emit R_X86_64_32S -- a 32-bit signed absolute address --
// for anything it believes lives in the top 2 GB of the address space. Load
// the object at an ordinary mmap address around 0x7f... and those
// relocations simply do not fit, which is what the loader said the first
// time this was run.
//
// Linux solves it by giving modules their own region within 2 GB of the
// kernel text. Here MAP_32BIT does the same job from the other end: the
// first 2 GB, where a positive signed 32-bit value reaches. On SchOS this
// costs nothing, because a process's mmap area starts at 0x40000000 and is
// already inside that window -- but it is a real constraint on where the
// LDLS host may put a module, and it is not one you would guess.
//
// The stubs are mapped low for the same reason: a 32S relocation can name an
// undefined symbol just as easily as a section.
span = ko_span(&k);
mem = mmap(0, (size_t)(span ? span : 4096), PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_32BIT, -1, 0);
code = mmap(0, (size_t)(k.nundef + 1) * STUB_SIZE,
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_32BIT, -1, 0);
if (mem == MAP_FAILED || code == MAP_FAILED) {
fprintf(stderr, "ldls-run: cannot get executable memory\n");
return 1;
}
stub_build(&stubs, code, k.nundef);
g_names = k.undef;
g_count = k.nundef;
// The ones that must not be logged, because they are called at the top of
// every function and have register rules of their own. See stub_is_bare().
for (i = 0; i < k.nundef; i++)
if (stub_is_bare(k.undef[i]))
stub_make_bare(code + (unsigned long long)i * STUB_SIZE);
if (ko_layout(&k, mem, span, (unsigned long long)mem) < 0) {
fprintf(stderr, "ldls-run: layout: %s\n", ko_err(&k));
return 1;
}
rc.k = &k; rc.s = &stubs;
if (ko_relocate(&k, resolve, &rc) < 0) {
fprintf(stderr, "ldls-run: relocate: %s\n", ko_err(&k));
return 1;
}
printf(" loaded %llu bytes at %p, %d stubs at %p\n\n",
span, (void *)mem, k.nundef, (void *)code);
entry = find_symbol(&k, "init_module");
if (!entry) {
printf(" no init_module -- nothing to call\n");
return 0;
}
printf(" calling init_module at %#llx\n\n", entry);
// A module that dereferences the zero our stubs return will fault, and that
// is a *result* rather than a crash: it says exactly how far the driver got
// before it needed the kernel to have done something real.
signal(SIGSEGV, on_fault);
signal(SIGBUS, on_fault);
signal(SIGILL, on_fault);
signal(SIGFPE, on_fault);
if (sigsetjmp(g_jump, 1) == 0) {
int (*init)(void) = (int (*)(void))(void *)entry;
int r = init();
printf("\n init_module returned %d after %d stub call(s)\n", r, g_calls);
} else {
printf("\n stopped by signal %d after %d stub call(s)\n", g_signal, g_calls);
printf(" that is a result, not a crash: the driver got that far before it\n"
" needed one of those stubs to have done something real\n");
rv = 0;
}
free(sec);
free(img);
return rv;
}

136
ldls/stub.c Normal file
View file

@ -0,0 +1,136 @@
// SPDX-License-Identifier: Apache-2.0
//
// Stubs. See ldls/stub.h.
//
// Two pieces of x86-64 that have to be exactly right, and both are short
// enough to read:
//
// the trampoline 20 bytes generated per symbol, which loads the symbol's
// index and jumps to the thunk
// the thunk one shared asm block that saves the argument registers,
// calls ldls_stub(), and returns whatever it said
//
// Written from the x86-64 psABI and the instruction encodings in the Intel
// manual -- both published, both specifications, and neither one anybody's
// source code.
#include "stub.h"
// --- the thunk ---------------------------------------------------------------
//
// Reached by `jmp` from a trampoline, with %r11 holding the symbol index and
// every argument register still holding what the driver put there.
//
// The stack arithmetic is the part worth checking. A `call` from the driver
// leaves %rsp ≡ 8 (mod 16). Six pushes move it by 48, which is 0 (mod 16), so
// it is still ≡ 8 -- and the ABI wants it ≡ 0 immediately before a `call`.
// Hence the extra `sub $8`: without it every stub call would enter C with a
// misaligned stack, which is survivable right up until something uses SSE and
// then is not.
__asm__(
".text\n"
".globl ldls_thunk\n"
".hidden ldls_thunk\n"
"ldls_thunk:\n"
" pushq %r9\n"
" pushq %r8\n"
" pushq %rcx\n"
" pushq %rdx\n"
" pushq %rsi\n"
" pushq %rdi\n"
" movq %rsp, %rsi\n" /* args[0..5] = rdi, rsi, rdx, rcx, r8, r9 */
" movq %r11, %rdi\n" /* the index the trampoline put there */
" subq $8, %rsp\n"
" call ldls_stub\n"
" addq $8, %rsp\n"
" addq $48, %rsp\n"
" ret\n"
".size ldls_thunk, .-ldls_thunk\n"
);
extern void ldls_thunk(void);
// --- the trampolines ---------------------------------------------------------
static void
put64(unsigned char *p, unsigned long long v)
{
int i;
for (i = 0; i < 8; i++)
p[i] = (unsigned char)(v >> (i * 8));
}
static void
put32(unsigned char *p, unsigned int v)
{
int i;
for (i = 0; i < 4; i++)
p[i] = (unsigned char)(v >> (i * 8));
}
static int
streq(const char *a, const char *b)
{
while (*a && *a == *b) { a++; b++; }
return *a == 0 && *b == 0;
}
int
stub_is_bare(const char *name)
{
return streq(name, "__fentry__") ||
streq(name, "__x86_return_thunk") ||
streq(name, "__x86_indirect_thunk_rax");
}
void
stub_build(struct stubs *s, unsigned char *code, int n)
{
int i;
s->code = code;
s->n = n;
for (i = 0; i < n; i++) {
unsigned char *p = code + (unsigned long long)i * STUB_SIZE;
int j;
for (j = 0; j < STUB_SIZE; j++)
p[j] = 0xCC; // int3, so a mistake stops here and not later
// endbr64. Harmless without CET and required with it -- an indirect jump
// to a target that does not begin with this faults on a machine that
// enforces control-flow integrity, and a driver's calls through function
// pointers are exactly that.
p[0] = 0xF3; p[1] = 0x0F; p[2] = 0x1E; p[3] = 0xFA;
// movl $i, %r11d -- 41 bb imm32. The upper half of %r11 is zeroed by the
// 32-bit form, which is what we want for a small index.
p[4] = 0x41; p[5] = 0xBB;
put32(p + 6, (unsigned int)i);
// movabsq $ldls_thunk, %r10 -- 49 ba imm64
p[10] = 0x49; p[11] = 0xBA;
put64(p + 12, (unsigned long long)(void *)ldls_thunk);
// jmpq *%r10 -- 41 ff e2
p[20] = 0x41; p[21] = 0xFF; p[22] = 0xE2;
}
}
// A bare `ret`, for the names in stub_is_bare().
void
stub_make_bare(unsigned char *p)
{
int j;
for (j = 0; j < STUB_SIZE; j++)
p[j] = 0xCC;
p[0] = 0xF3; p[1] = 0x0F; p[2] = 0x1E; p[3] = 0xFA; // endbr64
p[4] = 0xC3; // ret
}
unsigned long long
stub_addr(const struct stubs *s, int i)
{
return (unsigned long long)(s->code + (unsigned long long)i * STUB_SIZE);
}

55
ldls/stub.h Normal file
View file

@ -0,0 +1,55 @@
// SPDX-License-Identifier: Apache-2.0
//
// Stubs: giving every undefined symbol an address that says who it was.
//
// docs/plan/ldls.md milestone L1. L0 measured what a module asks for; this is
// what answers. Every name resolves to a small block of generated code, and
// when the driver calls it the block says which name it was and what arguments
// it came with.
//
// The value is not the logging, it is the **order**. A module's undefined
// symbol list is a set and tells you nothing about sequence; what a driver
// does at init, and in what order, is the other thing that cannot be guessed
// and it falls out of running the thing.
//
// Freestanding, like ldls/ko.c, so the SchOS-side host gets the same code.
#define STUB_SIZE 32 // bytes per trampoline, generously aligned
// What a stub call reports. The embedder provides this; stub.c calls it from
// the thunk with the original argument registers intact, which is what lets a
// stub for `_printk` show the string the driver actually passed.
//
// Whatever it returns becomes the stub's return value, so returning 0 lets a
// driver carry on past a call and reveal the next one -- which is worth far
// more than stopping at the first.
unsigned long ldls_stub(long idx, unsigned long *args);
struct stubs {
unsigned char *code; // STUB_SIZE * n bytes, must be executable
int n;
};
// Build `n` trampolines into `code`. Each one, when called, ends up in
// ldls_stub() with its index.
//
// `code` has to be executable memory; this only writes the bytes.
void stub_build(struct stubs *s, unsigned char *code, int n);
// The address of stub `i`, to hand to the loader as a symbol value.
unsigned long long stub_addr(const struct stubs *s, int i);
// Some names must not be logged, because they are called so often that the log
// becomes the only thing you see, and because they have calling conventions of
// their own that a logging thunk would break.
//
// `__fentry__` is ftrace's hook, emitted at the top of every function, and it
// is required to preserve every register. `__x86_return_thunk` is the retpoline
// return. Both are correctly implemented as a bare `ret`, which is also the
// fastest thing they could possibly be.
//
// Returns 1 if the name got a bare-ret stub instead of a logging one.
int stub_is_bare(const char *name);
// Overwrite one stub with that bare `ret`.
void stub_make_bare(unsigned char *p);

20
ldls/test/refmod.c
View file

@ -77,3 +77,23 @@ MODINFO static const char mi_name[] = "name=refmod";
// Modules define this; ko_name() looks for it to tell a kernel module from an
// ordinary object file.
int __this_module;
// The entry point, under the name `module_init(x)` gives it.
//
// L1 calls this, so it is written to exercise the thing L1 is for: a sequence
// of calls out to symbols this object does not define, in an order that can be
// checked. Three calls, always the same three, always in this order -- which
// is exactly what a real driver's init produces and what no amount of reading
// the symbol table would tell you.
//
// It deliberately does *not* dereference what the stubs return. A real driver
// does, faults, and that is a useful result; a test that faults every time is
// a test that cannot tell a fault it expected from one it did not.
int
init_module(void)
{
ldls_test_print(greeting);
ldls_test_alloc(sizeof(scratch));
ldls_test_free(scratch);
return 0;
}