tools: group scripts by what runs them
tools/windows/ .ps1 (pwsh, so usable from Linux and macOS too)
tools/posix/ .sh
tools/python/ .py
tools/ .c and .pl
The .c and .pl files stay at the top level because they are not
host-specific: the build compiles or invokes them on every platform.
Call sites updated in the Makefile, ports/mk/port.mk, docs and the
project memory. The one thing that does not survive a move on its own is
a script deriving the repo root from its own location -- all nine
PowerShell tools used Split-Path $PSScriptRoot -Parent, and memory-sync.sh
used dirname/.., both of which quietly become tools/ one level deeper.
Two levels now.
Two pre-existing Windows failures surfaced while verifying, both real and
neither caused by the move:
- schfs_fsck.c did not compile under mingw at all: --extract uses
POSIX mkdir(path,mode), symlink() and link(), none of which mingw
has. That took the whole tool with it, and `make schfs-check` with
it. mkdir maps to _mkdir; symlink and link fail loudly rather than
writing a regular file in their place, because an extracted tree
that differs from the image while reporting success defeats the
point of --extract.
- the round-trip test then failed at `ln -s`, since Git Bash cannot
create symlinks without developer mode. It skips with a message now.
Not a fudge: mkschfs's mingw build cannot see a symlink either, so
the test would have compared a tree with none against an image with
none and called it a pass -- coverage reporting itself as present
while being absent.
Verified on Windows: build clean, all seven bootcheck suites green,
make schfs-check passes.
This commit is contained in:
parent
531ef120ef
commit
754d96ebc5
26 changed files with 127 additions and 39 deletions
141
tools/posix/schfs_roundtrip.sh
Normal file
141
tools/posix/schfs_roundtrip.sh
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
#!/bin/sh
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
#
|
||||
# SchFS round-trip test: build an image from a directory tree, read it back
|
||||
# with a completely separate program, and diff the result against the original.
|
||||
#
|
||||
# The point is the oracle. schfs_test checks the B-tree against a model written
|
||||
# for the purpose; this checks the *filesystem* against a tree produced by
|
||||
# something with no connection to SchFS at all -- the host's own filesystem.
|
||||
# If mkschfs and schfs_fsck ever agree on something wrong, the source tree still
|
||||
# disagrees, and diff says so.
|
||||
#
|
||||
# tools/schfs_roundtrip.sh <mkschfs> <schfs_fsck> [tree]
|
||||
#
|
||||
# With no tree, a synthetic one is generated covering the cases that have
|
||||
# historically broken image builders: empty files, empty directories, files
|
||||
# spanning several blocks, files ending exactly on a block boundary, duplicate
|
||||
# content that must be hard-linked, symlinks, deep nesting and awkward names.
|
||||
|
||||
set -e
|
||||
|
||||
# Every tool here is piped into sed for indenting, and a shell only checks the
|
||||
# *last* command of a pipeline. Without this, schfs_fsck could exit non-zero --
|
||||
# or crash outright -- and the script would sail past it, which is exactly what
|
||||
# happened the first time this ran.
|
||||
if (set -o pipefail) 2>/dev/null; then set -o pipefail; fi
|
||||
|
||||
MKSCHFS=${1:?usage: schfs_roundtrip.sh <mkschfs> <schfs_fsck> [tree]}
|
||||
FSCK=${2:?usage: schfs_roundtrip.sh <mkschfs> <schfs_fsck> [tree]}
|
||||
TREE=$3
|
||||
|
||||
WORK=$(mktemp -d)
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
|
||||
# Symlinks are load-bearing here, so a host that cannot create them cannot run
|
||||
# this test at all.
|
||||
#
|
||||
# Windows is that host. Git Bash needs developer mode or MSYS=winsymlinks to
|
||||
# make one, and even where it can, mkschfs's mingw build cannot *see* one --
|
||||
# lstat degrades to stat and S_ISLNK is always false, by design and with a
|
||||
# comment saying so. The round-trip would therefore be comparing a tree with no
|
||||
# symlinks against an image with no symlinks and calling that a pass, which is
|
||||
# worse than not running: it is coverage that reports itself as present and is
|
||||
# not.
|
||||
#
|
||||
# Skipping with exit 0 so `make schfs-check` still means something on Windows
|
||||
# -- the B-tree, object and space suites are all host-independent and do run.
|
||||
if ! ln -s target "$WORK/.symlink-probe" 2>/dev/null; then
|
||||
echo "SchFS round-trip: SKIPPED -- this host cannot create symlinks."
|
||||
echo " The generated tree needs them, and mkschfs cannot see them here"
|
||||
echo " anyway. Run this on Linux, macOS or BSD for real coverage."
|
||||
exit 0
|
||||
fi
|
||||
rm -f "$WORK/.symlink-probe"
|
||||
|
||||
if [ -z "$TREE" ]; then
|
||||
# Generated by a separate script so the same tree can be rebuilt by hand
|
||||
# when debugging a failure, without running the whole round-trip.
|
||||
TREE="$WORK/tree"
|
||||
sh "$(dirname "$0")/mktree.sh" "$TREE"
|
||||
fi
|
||||
|
||||
IMG="$WORK/out.img"
|
||||
OUT="$WORK/extracted"
|
||||
|
||||
echo "SchFS round-trip"
|
||||
echo " source tree: $TREE"
|
||||
|
||||
"$MKSCHFS" "$IMG" --size 4096 --label roundtrip --sysroot "$TREE" | sed 's/^/ /'
|
||||
|
||||
echo
|
||||
"$FSCK" "$IMG" --check | sed 's/^/ /'
|
||||
|
||||
mkdir -p "$OUT"
|
||||
"$FSCK" "$IMG" --extract "$OUT" | sed 's/^/ /'
|
||||
|
||||
echo
|
||||
# --no-dereference so a symlink is compared as a symlink rather than followed;
|
||||
# without it a broken link that was stored correctly still fails the diff.
|
||||
if diff -r --no-dereference "$TREE" "$OUT"; then
|
||||
echo " [ok] extracted tree is identical to the source"
|
||||
else
|
||||
echo " FAIL: extracted tree differs from the source"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Hard links are the one thing diff cannot see: it compares contents, and two
|
||||
# separate copies with identical contents pass. Check the link count directly,
|
||||
# or dedup could silently degrade to duplication and nothing would notice.
|
||||
LINKS=$("$FSCK" "$IMG" --stat /bin/multiblock | sed -n 's/^nlink: *//p')
|
||||
if [ "$LINKS" != "2" ]; then
|
||||
echo " FAIL: /bin/multiblock has nlink=$LINKS, expected 2 (dedup did not hard-link)"
|
||||
exit 1
|
||||
fi
|
||||
echo " [ok] duplicate content stored once, with nlink=2"
|
||||
|
||||
# ...and the converse: same size, different bytes must stay separate.
|
||||
DLINKS=$("$FSCK" "$IMG" --stat /bin/decoy | sed -n 's/^nlink: *//p')
|
||||
if [ "$DLINKS" != "1" ]; then
|
||||
echo " FAIL: /bin/decoy has nlink=$DLINKS, expected 1 (dedup merged unequal files)"
|
||||
exit 1
|
||||
fi
|
||||
echo " [ok] same-size different-content files kept separate"
|
||||
|
||||
# Building the same tree twice must produce the same bytes. A difference means
|
||||
# something non-deterministic crept in, which makes every future image diff
|
||||
# useless as a signal.
|
||||
IMG2="$WORK/out2.img"
|
||||
"$MKSCHFS" "$IMG2" --size 4096 --label roundtrip --sysroot "$TREE" > /dev/null
|
||||
if cmp -s "$IMG" "$IMG2"; then
|
||||
echo " [ok] rebuilding the same tree produces an identical image"
|
||||
else
|
||||
echo " FAIL: image build is not reproducible"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Second pass with a deliberately crippled name hash.
|
||||
#
|
||||
# With the real 64-bit hash, two names in one directory never collide, so the
|
||||
# packed-dirent list -- the fiddliest code in the object layer -- never runs at
|
||||
# all. Six bits means 64 buckets for ~330 names, so every bucket holds several
|
||||
# entries and lookup, readdir and the append-on-collision path all get properly
|
||||
# exercised. Both tools must agree on the mask or nothing resolves.
|
||||
echo
|
||||
echo " forcing hash collisions (6-bit name hash)"
|
||||
IMG3="$WORK/collide.img"
|
||||
OUT3="$WORK/collide-out"
|
||||
"$MKSCHFS" "$IMG3" --size 4096 --hash-bits 6 --sysroot "$TREE" | sed 's/^/ /'
|
||||
"$FSCK" "$IMG3" --check --hash-bits 6 | sed 's/^/ /'
|
||||
mkdir -p "$OUT3"
|
||||
"$FSCK" "$IMG3" --extract "$OUT3" --hash-bits 6 | sed 's/^/ /'
|
||||
if diff -r --no-dereference "$TREE" "$OUT3" > /dev/null; then
|
||||
echo " [ok] identical tree survives a directory hash that collides constantly"
|
||||
else
|
||||
echo " FAIL: colliding-hash image does not round-trip"
|
||||
diff -r --no-dereference "$TREE" "$OUT3" | head -10
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "round-trip passed"
|
||||
Loading…
Reference in a new issue