nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

tools: group scripts by what runs them

Browse source 
tools/windows/   .ps1   (pwsh, so usable from Linux and macOS too)
  tools/posix/     .sh
  tools/python/    .py
  tools/           .c and .pl

The .c and .pl files stay at the top level because they are not
host-specific: the build compiles or invokes them on every platform.

Call sites updated in the Makefile, ports/mk/port.mk, docs and the
project memory. The one thing that does not survive a move on its own is
a script deriving the repo root from its own location -- all nine
PowerShell tools used Split-Path $PSScriptRoot -Parent, and memory-sync.sh
used dirname/.., both of which quietly become tools/ one level deeper.
Two levels now.

Two pre-existing Windows failures surfaced while verifying, both real and
neither caused by the move:

  - schfs_fsck.c did not compile under mingw at all: --extract uses
    POSIX mkdir(path,mode), symlink() and link(), none of which mingw
    has. That took the whole tool with it, and `make schfs-check` with
    it. mkdir maps to _mkdir; symlink and link fail loudly rather than
    writing a regular file in their place, because an extracted tree
    that differs from the image while reporting success defeats the
    point of --extract.
  - the round-trip test then failed at `ln -s`, since Git Bash cannot
    create symlinks without developer mode. It skips with a message now.
    Not a fudge: mkschfs's mingw build cannot see a symlink either, so
    the test would have compared a tree with none against an image with
    none and called it a pass -- coverage reporting itself as present
    while being absent.

Verified on Windows: build clean, all seven bootcheck suites green,
make schfs-check passes.
This commit is contained in:
Nexoniarz 2026-08-09 14:38:02 +02:00
commit 754d96ebc5
26 changed files with 127 additions and 39 deletions
Download patch file Download diff file Expand all files Collapse all files

85
tools/python/crashqr.py Normal file
View file

@ -0,0 +1,85 @@
#!/usr/bin/env python3
"""Read the QR code back off a captured crash screen.
This is the acceptance test for the crash channel, and it is deliberately
end-to-end: it does not ask the encoder what it produced, it reads the pixels
that were actually painted on the framebuffer. Everything in between -- payload
construction in panic context, the encoder, the scaling into modules, the card
and quiet zone -- is covered by whether this comes back as text.
pwsh -File tools/crashshot.ps1 # crash on purpose, screendump it
PYTHONPATH=<dir> python tools/crashqr.py
Needs opencv-python-headless and numpy; see tools/qrcheck.py for the install.
"""
import sys
try:
import cv2
import numpy as np
except ImportError:
sys.exit("crashqr: needs opencv-python-headless and numpy")
PPM = sys.argv[1] if len(sys.argv) > 1 else "build/crashscreen.ppm"
try:
data = open(PPM, "rb").read()
except OSError as e:
sys.exit(f"crashqr: {e} -- run tools/crashshot.ps1 first")
# PPM P6 header: three whitespace-separated fields after the magic. Parsed by
# scanning rather than assuming a fixed length, because QEMU's is compact but
# nothing guarantees that.
pos, fields = 2, []
while len(fields) < 3:
while data[pos:pos + 1].isspace():
pos += 1
start = pos
while not data[pos:pos + 1].isspace():
pos += 1
fields.append(int(data[start:pos]))
pos += 1
w, h = fields[0], fields[1]
img = np.frombuffer(data[pos:pos + w * h * 3], np.uint8).reshape(h, w, 3)
grey = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
# The symbol sits on the only light panel on an otherwise purple screen, so a
# brightness threshold finds it without needing to know the layout. The white
# report text passes the same threshold, which does not matter -- the detector
# locates the symbol by its finder patterns, and handing it extra screen
# costs nothing.
mask = (grey > 240).astype(np.uint8)
ys, xs = np.nonzero(mask)
if len(xs) == 0:
sys.exit("crashqr: no light region found -- was a crash screen captured?")
crop = grey[ys.min():ys.max() + 1, xs.min():xs.max() + 1]
# OpenCV's detector is sensitive to scale in ways unrelated to the encoding,
# so try a few before concluding anything. See the note in tools/qrcheck.py.
text = ""
for scale in (1, 2, 3, 4):
up = cv2.resize(crop, None, fx=scale, fy=scale, interpolation=cv2.INTER_NEAREST)
up = cv2.copyMakeBorder(up, 40, 40, 40, 40, cv2.BORDER_CONSTANT, value=255)
text, _, _ = cv2.QRCodeDetector().detectAndDecode(up)
if text:
break
if not text:
print("crashqr: FAILED -- the symbol on the crash screen did not decode")
sys.exit(1)
print("crashqr: decoded the crash screen's QR\n")
for line in text.split("\n"):
print(" ", line)
# The payload is a contract with whatever reads these later, so check its shape
# rather than only that something came back.
lines = text.split("\n")
ok = lines[0] == "SXPSOGM1"
print()
if not ok:
print(f"crashqr: FAILED -- expected magic 'SXPSOGM1', got {lines[0]!r}")
sys.exit(1)
print(f"crashqr: passed ({len(text)} bytes, {len(lines)} fields)")

184
tools/python/mklogo.py Normal file
View file

@ -0,0 +1,184 @@
#!/usr/bin/env python3
"""Turn Assets/*.png into a C header the kernel can draw.
Why this exists rather than a PNG decoder in the kernel: the boot logo is one
fixed image that changes about never, so decoding it at runtime would mean
carrying an inflate implementation in the kernel forever to answer a question
that is already known at build time. Precedent is include/font.h, which is the
same trade for the same reason.
Output is an 8-bit coverage mask, not colour. The logo is a single-colour
silhouette (a monogram in a ring), so what matters is *how much* of each pixel
is ink -- which lets the kernel composite it in whatever the theme's accent
happens to be, anti-aliased, at one byte per pixel. Storing RGB would be four
times the size and would hardcode a colour into the kernel image.
python3 tools/mklogo.py "Assets/SchOS Small Logo.png" include/logo.h 128
"""
import sys, zlib, struct
def read_png(path):
d = open(path, 'rb').read()
if d[:8] != b'\x89PNG\r\n\x1a\n':
raise SystemExit(f'{path}: not a PNG')
pos, idat, w, h, bd, ct = 8, b'', 0, 0, 0, 0
while pos < len(d):
ln, typ = struct.unpack('>I4s', d[pos:pos + 8])
body = d[pos + 8:pos + 8 + ln]
if typ == b'IHDR':
w, h, bd, ct = struct.unpack('>IIBB', body[:10])
elif typ == b'IDAT':
idat += body
elif typ == b'IEND':
break
pos += 12 + ln
if bd != 8 or ct not in (2, 6):
raise SystemExit(f'{path}: need 8-bit RGB or RGBA, got depth={bd} type={ct}')
nch = 4 if ct == 6 else 3
raw = zlib.decompress(idat)
# Undo PNG's per-scanline filters. Each row is prefixed with a filter byte
# and predicted from the row above and the pixel to the left; without this
# step the data is not an image at all.
stride = w * nch
out = bytearray(h * stride)
prev = bytearray(stride)
p = 0
for y in range(h):
f = raw[p]; p += 1
cur = bytearray(raw[p:p + stride]); p += stride
if f == 1:
for i in range(nch, stride):
cur[i] = (cur[i] + cur[i - nch]) & 0xFF
elif f == 2:
for i in range(stride):
cur[i] = (cur[i] + prev[i]) & 0xFF
elif f == 3:
for i in range(stride):
a = cur[i - nch] if i >= nch else 0
cur[i] = (cur[i] + ((a + prev[i]) >> 1)) & 0xFF
elif f == 4:
for i in range(stride):
a = cur[i - nch] if i >= nch else 0
b = prev[i]
c = prev[i - nch] if i >= nch else 0
pa, pb, pc = abs(b - c), abs(a - c), abs(a + b - 2 * c)
pr = a if (pa <= pb and pa <= pc) else (b if pb <= pc else c)
cur[i] = (cur[i] + pr) & 0xFF
elif f != 0:
raise SystemExit(f'unknown PNG filter {f}')
out[y * stride:(y + 1) * stride] = cur
prev = cur
return w, h, nch, out
def ink(px, nch, x, y, w):
"""How much ink is at (x, y), 0..255.
The source is a dark mark on white with an alpha channel. Ink is therefore
'opaque and not white': darkness weighted by alpha. Using luminance alone
would make the transparent margin count as black.
"""
i = (y * w + x) * nch
r, g, b = px[i], px[i + 1], px[i + 2]
a = px[i + 3] if nch == 4 else 255
lum = (r * 299 + g * 587 + b * 114) // 1000
return ((255 - lum) * a) // 255
def main():
if len(sys.argv) < 3:
raise SystemExit(__doc__)
src, dst = sys.argv[1], sys.argv[2]
size = int(sys.argv[3]) if len(sys.argv) > 3 else 128
w, h, nch, px = read_png(src)
# Crop to the mark. These files are 1920x1080 with the logo floating in a
# large empty field; scaling that whole canvas down would waste most of the
# output on margin and leave the mark tiny.
x0, y0, x1, y1 = w, h, -1, -1
for y in range(h):
for x in range(w):
if ink(px, nch, x, y, w) > 24:
if x < x0: x0 = x
if y < y0: y0 = y
if x > x1: x1 = x
if y > y1: y1 = y
if x1 < 0:
raise SystemExit(f'{src}: no non-white pixels found')
# Square the crop so the aspect ratio survives the scale.
cw, ch = x1 - x0 + 1, y1 - y0 + 1
side = max(cw, ch)
x0 -= (side - cw) // 2
y0 -= (side - ch) // 2
# Box filter. Averaging every source pixel that lands in an output cell is
# what keeps the ring's edge smooth; nearest-neighbour at this reduction
# ratio (roughly 8:1) would break it into dashes.
mask = bytearray(size * size)
for oy in range(size):
sy0 = y0 + oy * side // size
sy1 = max(sy0 + 1, y0 + (oy + 1) * side // size)
for ox in range(size):
sx0 = x0 + ox * side // size
sx1 = max(sx0 + 1, x0 + (ox + 1) * side // size)
tot = n = 0
for sy in range(sy0, sy1):
if sy < 0 or sy >= h: continue
for sx in range(sx0, sx1):
if sx < 0 or sx >= w: continue
tot += ink(px, nch, sx, sy, w); n += 1
mask[oy * size + ox] = tot // n if n else 0
# Normalise to full strength. The source art is a mid-grey gradient on
# white, so its darkest pixel is only about 60% "ink" -- composited as-is
# against a near-black background the mark comes out muddy. Scaling by the
# peak keeps the gradient's internal shape (the relative values are
# unchanged) while letting the brightest part of the logo actually reach the
# accent colour.
peak = max(mask)
if peak:
for i, v in enumerate(mask):
mask[i] = min(255, v * 255 // peak)
# Pack two pixels per byte. This header is included by the *bootloader* as
# well as the kernel, and stage2 has to fit between 0x7E00 and the 0x10000
# kernel-search buffer -- so the mask's size is a boot-time constraint, not
# just a matter of taste. 16 grey levels is visually indistinguishable from
# 256 for a single-colour mark, and halves the cost.
packed = bytearray(len(mask) // 2)
for i in range(0, len(mask), 2):
packed[i // 2] = ((mask[i] // 17) << 4) | (mask[i + 1] // 17)
with open(dst, 'w', newline='\n') as f:
f.write('#pragma once\n\n')
f.write('// GENERATED by tools/mklogo.py -- do not edit. Run `make logo`.\n')
f.write(f'// source: {src}\n')
f.write('//\n')
f.write('// An ink-coverage mask, not colour: 0 is background and full scale is\n')
f.write('// solid logo. Whoever draws it picks the colour, so the palette stays in\n')
f.write('// include/theme.h and can change without regenerating anything.\n')
f.write('//\n')
f.write('// Two pixels per byte, high nibble first. Included by boot/stage2main.c\n')
f.write('// as well as kernel/dev/vga.c, and stage2 must fit below the 0x10000\n')
f.write('// kernel-search buffer -- hence 4 bits rather than 8.\n\n')
f.write(f'#define LOGO_W {size}\n#define LOGO_H {size}\n\n')
f.write('// Ink at linear pixel index i, 0..255.\n')
f.write('#define LOGO_INK(i) ((unsigned char)((((i) & 1) '
'? (logo_mask[(i) >> 1] & 0xF) \\\n')
f.write(' : (logo_mask[(i) >> 1] >> 4)) * 17))\n\n')
f.write('static const unsigned char logo_mask[LOGO_W * LOGO_H / 2] = {\n')
for i in range(0, len(packed), 16):
f.write(' ' + ','.join('0x%02x' % b for b in packed[i:i + 16]) + ',\n')
f.write('};\n')
nz = sum(1 for b in mask if b)
print(f'{dst}: {size}x{size}, cropped from {cw}x{ch} at ({x0},{y0}), '
f'{nz} inked pixels, {len(packed)} bytes packed (4bpp)')
main()

185
tools/python/qrcheck.py Normal file
View file

@ -0,0 +1,185 @@
#!/usr/bin/env python3
"""Cross-check kernel/core/qr.c against independent implementations.
The self-checks inside tools/qrtest.c verify what has a local notion of
correctness -- table arithmetic, the Reed-Solomon generator polynomials, whether
a finder pattern is where it should be. They cannot verify the parts that only
mean something to a decoder: zigzag placement order, mask selection, and block
interleaving. All three have the same failure mode, which is a symbol that looks
exactly like a QR code and is noise.
Two oracles, in order of authority:
1. **A real decoder** (OpenCV). Does the symbol read back as the exact bytes
that went in? This is the only test that matches what the feature is for --
somebody pointing a phone at a crashed machine -- and it is the one that
decides pass or fail.
2. **A second encoder** (segno), matrix compared module for module. Stricter
than decoding when it agrees, because it pins down the exact mask and
placement rather than merely something readable. Applied only to payloads
that exactly fill their version, for the reason below.
segno adds a spurious zero pad codeword to every byte-mode symbol. Its
write_padding_bits() does `buff.extend([0] * (8 - (length % 8)))`, which
emits a whole byte when the stream is already on a codeword boundary -- and
in byte mode the stream is always 8L+16 bits after the terminator, so it
always is. ISO/IEC 18004 makes that conditional ("if the bit stream length is
such that it does not end at a codeword boundary"), and segno quotes exactly
that sentence above the line. It is harmless -- decoders read `count` bytes
and ignore padding -- but it means the two encoders cannot agree bit-for-bit
on any payload that needs padding, so those are compared by decoding only.
pip install --target <dir> opencv-python-headless numpy segno
PYTHONPATH=<dir> python tools/qrcheck.py
Requires build/tools/qrtest (make qrtest). Both libraries are optional; whatever
is installed gets used.
"""
import os
import subprocess
import sys
# Normalised and absolute: Windows CreateProcess rejects a relative path written
# with forward slashes, even when the file is plainly there.
QRTEST = os.path.abspath(os.path.join("build", "tools",
"qrtest.exe" if os.name == "nt" else "qrtest"))
try:
import cv2
import numpy as np
except ImportError:
cv2 = None
try:
import segno
except ImportError:
segno = None
if cv2 is None and segno is None:
sys.exit("qrcheck: needs opencv-python-headless and/or segno")
if not os.path.exists(QRTEST):
sys.exit(f"qrcheck: {QRTEST} not found -- run `make qrtest`")
def ours(text):
"""Our encoder's matrix, as a list of strings of '0'/'1'."""
out = subprocess.run([QRTEST, "-m", text], capture_output=True, text=True)
if out.returncode != 0:
raise RuntimeError(f"qrtest failed on {text!r}")
return out.stdout.split()
def our_mask(m):
"""The mask our encoder chose, read back out of the format bits.
Read from the symbol rather than reported by it, because the point is to
check what was actually written.
"""
bits = 0
for i in range(15):
if i <= 5: b = int(m[i][8])
elif i == 6: b = int(m[7][8])
elif i == 7: b = int(m[8][8])
elif i == 8: b = int(m[8][7])
else: b = int(m[8][14 - i])
bits |= b << i
return ((bits ^ 0x5412) >> 10) & 7
def render(m, scale, border):
n = len(m)
side = (n + 2 * border) * scale
img = np.full((side, side), 255, np.uint8)
for y in range(n):
for x in range(n):
if m[y][x] == "1":
img[(y + border) * scale:(y + border + 1) * scale,
(x + border) * scale:(x + border + 1) * scale] = 0
return img
# OpenCV's detector is not a reference implementation and does fail on symbols
# that are perfectly valid -- sensitive to scale and quiet-zone width in ways
# that have nothing to do with the encoding. Several renderings are tried before
# concluding anything, and a payload that still will not decode is re-tested
# against segno's symbol for the same input: if that fails too, the detector is
# what is at fault, not the encoder. Without that second step this check reports
# false failures, which is worse than not running it.
RENDERINGS = [(8, 4), (6, 6), (8, 6), (4, 4), (12, 6), (16, 4)]
def decode(m):
for scale, border in RENDERINGS:
text, _, _ = cv2.QRCodeDetector().detectAndDecode(render(m, scale, border))
if text:
return text
return ""
# Byte capacity per version: total - ecc*blocks - 2. Mirrors the table in qr.c;
# the exact-fill cases below are built from it, and a disagreement would show up
# as a version mismatch rather than passing quietly.
CAPACITY = {1: 17, 2: 32, 3: 53, 4: 78, 5: 106, 6: 134}
alpha = lambda n: "".join(chr(65 + i % 26) for i in range(n))
CASES = [
"HELLO",
"SCHOS",
"a",
"0123456789",
"SCHOS PANIC\ntrap 14 err 4 cpu 0\nrip ffffffff80104e2a",
bytes(range(1, 40)).decode("latin-1"), # non-printable payload
]
# One payload that exactly fills each version, and one just under, so both the
# padded and unpadded paths are covered at every version -- including 6, which
# is the only one with more than one Reed-Solomon block.
for v, cap in CAPACITY.items():
CASES.append(alpha(cap))
CASES.append(alpha(cap - 1))
fails = 0
print("decode (OpenCV)" if cv2 else "decode -- skipped, no opencv")
for text in CASES:
if not cv2:
break
m = ours(text)
v = (len(m) - 17) // 4
got = decode(m)
if got == text:
print(f" [ok] v{v} {len(text):3d}B")
continue
# Before blaming the encoder, check whether an independently generated
# symbol for the same payload decodes either.
if segno and not decode([
"".join(str(x) for x in row)
for row in segno.make(text.encode("latin-1"), error="l", version=v,
boost_error=False, mode="byte").matrix]):
print(f" [detector] v{v} {len(text):3d}B -- segno's symbol does not "
f"decode either; OpenCV's limitation, not ours")
continue
fails += 1
print(f" [FAIL] v{v} {len(text):3d}B decoded {got[:40]!r}")
print("\nmatrix vs segno (exact-fill payloads only)" if segno else "\nsegno -- skipped")
for v, cap in CAPACITY.items():
if not segno:
break
text = alpha(cap)
m = ours(text)
mask = our_mask(m)
q = segno.make(text.encode("latin-1"), error="l", version=v,
mask=mask, boost_error=False, mode="byte")
ref = ["".join(str(x) for x in row) for row in q.matrix]
ok = m == ref
fails += not ok
n = len(m)
diff = "" if ok else f" {sum(1 for a, b in zip(''.join(m), ''.join(ref)) if a != b)}/{n*n} differ"
print(f" [{'ok' if ok else 'FAIL'}] v{v} mask {mask} {cap:3d}B{diff}")
print()
print("qrcheck: FAILED" if fails else "qrcheck: passed")
sys.exit(1 if fails else 0)