071499a378
e1000 -- Linux's, unmodified -- probes, opens, gets an address by DHCP and
carries ping traffic in a SchOS process. The kernel was driving that NIC and
gave it up, which makes it the first real handover as well as the second
driver.
None of what it needed was a missing feature. Every one was a call that
succeeded, returned zero and did nothing:
_find_next_bit "bit 0 is set" -- for_each_cpu never ended
netif_carrier_off carrier already up -- TCTL.EN never set
__x86_indirect_thunk_r8 the jump discarded -- 255 rx buffers, none made
dma_map_page_attrs bus address 0 -- NIC aimed at physical 0
so the rule from memcpy is now general: a stub returning zero is unsafe
wherever zero is a meaningful answer.
The loader grew ko_fixup_percpu(). The stack guard turned out to be one
per-CPU variable among several; `current` cannot be defused because the driver
dereferences it. Taking the %gs off instead leaves the same opcode reading the
same width from storage the host supplies -- a per-CPU variable in a
single-threaded host is just a variable. Eight bytes become seven and a nop,
and the displacement is not always last, so there is now just enough of an x86
decoder in ko.c to know how long the instruction is.
ldls/kpi/thunk.c is fifteen lines of assembly and the highest-value file here:
retpolines are control flow, and a stub cannot stand in for a jump.
The kernel side:
- ioperm.c, the TSS I/O permission bitmap. A driver's outl is an instruction,
not a call, and e1000's reset path picks it deliberately. IOPL=3 would hand
over the PIC and the disk with it, so it is per-port. Writing it found that
the I/O map base had been written at offset 64 rather than 102, leaving the
CPU checking ports 0-831 against the TSS's own RSP0.
- claim masks the device's IRQ line: 450,000 a second, otherwise.
- net_detach(), the missing half of net_attach_late(). An interface left in
the stack keeps its route and sends every frame to a driver that refuses
it, which looks exactly like a userspace driver that does not work.
- e1000_release() gives up the interface, not just the hardware.
8139too still passes 4/4; ldls-check still passes 27; four CPUs, no panic.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
71 lines
3.1 KiB
C
71 lines
3.1 KiB
C
// SPDX-License-Identifier: Apache-2.0
|
|
//
|
|
// Retpoline thunks. See ldls/kpi/kpi.h for the provenance rule.
|
|
//
|
|
// These are not API. They are the compiler's, emitted by `-mretpoline` /
|
|
// `-mfunction-return`, and a module built for a mitigated kernel is full of
|
|
// them: every indirect call becomes `jmp __x86_indirect_thunk_rN` and every
|
|
// return becomes `jmp __x86_return_thunk`.
|
|
//
|
|
// They have to be assembly, because what they are *is* control flow. A C
|
|
// function cannot express "jump to whatever is in %r8 without touching any
|
|
// other register", and the stub the loader supplies for an unimplemented
|
|
// symbol -- which logs its name and returns zero -- is exactly the wrong
|
|
// answer: it discards the jump.
|
|
//
|
|
// That cost a day, and the shape of the failure is the reason this file has a
|
|
// long comment. e1000 refills its receive ring through a function pointer, and
|
|
// the compiler made the call a *tail* call:
|
|
//
|
|
// mov 0x2a0(%rbx),%r8 ; adapter->alloc_rx_buf
|
|
// ... ; compute E1000_DESC_UNUSED(ring)
|
|
// jmp __x86_indirect_thunk_r8
|
|
//
|
|
// so the one thing in the log was `__x86_indirect_thunk_r8(...)` among four
|
|
// thousand other stub calls, and the visible symptom was a NIC that
|
|
// transmitted perfectly and received nothing, with RDT sitting at zero. The
|
|
// driver had asked for 255 receive buffers and the shim had answered "yes".
|
|
//
|
|
// **These are the plain versions, with no mitigation in them.** A real
|
|
// retpoline traps speculation in a loop; this is `jmp *%reg`. Spectre-v2
|
|
// hardening for foreign code in a process is not something LDLS can offer
|
|
// honestly -- the kernel it runs under is not mitigated either -- and a thunk
|
|
// that pretends otherwise would be worse than one that says so here.
|
|
|
|
// One per register, because the register is in the name -- that is the whole
|
|
// point of the encoding. `endbr64` heads each one because a module built with
|
|
// IBT reaches them by indirect jump.
|
|
#define THUNK(reg) \
|
|
".globl kpi_thunk_" #reg "\n" \
|
|
".type kpi_thunk_" #reg ", @function\n" \
|
|
"kpi_thunk_" #reg ":\n" \
|
|
" endbr64\n" \
|
|
" jmp *%" #reg "\n" \
|
|
".size kpi_thunk_" #reg ", .-kpi_thunk_" #reg "\n"
|
|
|
|
__asm__(
|
|
".text\n"
|
|
THUNK(rax) THUNK(rbx) THUNK(rcx) THUNK(rdx)
|
|
THUNK(rsi) THUNK(rdi) THUNK(rbp)
|
|
THUNK(r8) THUNK(r9) THUNK(r10) THUNK(r11)
|
|
THUNK(r12) THUNK(r13) THUNK(r14) THUNK(r15)
|
|
|
|
// __x86_return_thunk is just a return. It had been resolving to a stub,
|
|
// which happened to work -- the stub is reached by `jmp`, so its own `ret`
|
|
// returned to the right place -- while clobbering %rax with zero on the way
|
|
// out. Every function whose return value came back as 0 for no reason was
|
|
// this.
|
|
".globl kpi_return_thunk\n"
|
|
".type kpi_return_thunk, @function\n"
|
|
"kpi_return_thunk:\n"
|
|
" ret\n"
|
|
".size kpi_return_thunk, .-kpi_return_thunk\n"
|
|
|
|
// __fentry__ is the ftrace hook: on an untraced kernel it is a bare return,
|
|
// and it is called at the top of essentially every function in the module.
|
|
".globl kpi_fentry\n"
|
|
".type kpi_fentry, @function\n"
|
|
"kpi_fentry:\n"
|
|
" ret\n"
|
|
".size kpi_fentry, .-kpi_fentry\n"
|
|
);
|