754d96ebc5
tools/windows/ .ps1 (pwsh, so usable from Linux and macOS too)
tools/posix/ .sh
tools/python/ .py
tools/ .c and .pl
The .c and .pl files stay at the top level because they are not
host-specific: the build compiles or invokes them on every platform.
Call sites updated in the Makefile, ports/mk/port.mk, docs and the
project memory. The one thing that does not survive a move on its own is
a script deriving the repo root from its own location -- all nine
PowerShell tools used Split-Path $PSScriptRoot -Parent, and memory-sync.sh
used dirname/.., both of which quietly become tools/ one level deeper.
Two levels now.
Two pre-existing Windows failures surfaced while verifying, both real and
neither caused by the move:
- schfs_fsck.c did not compile under mingw at all: --extract uses
POSIX mkdir(path,mode), symlink() and link(), none of which mingw
has. That took the whole tool with it, and `make schfs-check` with
it. mkdir maps to _mkdir; symlink and link fail loudly rather than
writing a regular file in their place, because an extracted tree
that differs from the image while reporting success defeats the
point of --extract.
- the round-trip test then failed at `ln -s`, since Git Bash cannot
create symlinks without developer mode. It skips with a message now.
Not a fudge: mkschfs's mingw build cannot see a symlink either, so
the test would have compared a tree with none against an image with
none and called it a pass -- coverage reporting itself as present
while being absent.
Verified on Windows: build clean, all seven bootcheck suites green,
make schfs-check passes.
141 lines
5.6 KiB
Shell
141 lines
5.6 KiB
Shell
#!/bin/sh
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
#
|
|
# SchFS round-trip test: build an image from a directory tree, read it back
|
|
# with a completely separate program, and diff the result against the original.
|
|
#
|
|
# The point is the oracle. schfs_test checks the B-tree against a model written
|
|
# for the purpose; this checks the *filesystem* against a tree produced by
|
|
# something with no connection to SchFS at all -- the host's own filesystem.
|
|
# If mkschfs and schfs_fsck ever agree on something wrong, the source tree still
|
|
# disagrees, and diff says so.
|
|
#
|
|
# tools/schfs_roundtrip.sh <mkschfs> <schfs_fsck> [tree]
|
|
#
|
|
# With no tree, a synthetic one is generated covering the cases that have
|
|
# historically broken image builders: empty files, empty directories, files
|
|
# spanning several blocks, files ending exactly on a block boundary, duplicate
|
|
# content that must be hard-linked, symlinks, deep nesting and awkward names.
|
|
|
|
set -e
|
|
|
|
# Every tool here is piped into sed for indenting, and a shell only checks the
|
|
# *last* command of a pipeline. Without this, schfs_fsck could exit non-zero --
|
|
# or crash outright -- and the script would sail past it, which is exactly what
|
|
# happened the first time this ran.
|
|
if (set -o pipefail) 2>/dev/null; then set -o pipefail; fi
|
|
|
|
MKSCHFS=${1:?usage: schfs_roundtrip.sh <mkschfs> <schfs_fsck> [tree]}
|
|
FSCK=${2:?usage: schfs_roundtrip.sh <mkschfs> <schfs_fsck> [tree]}
|
|
TREE=$3
|
|
|
|
WORK=$(mktemp -d)
|
|
trap 'rm -rf "$WORK"' EXIT
|
|
|
|
# Symlinks are load-bearing here, so a host that cannot create them cannot run
|
|
# this test at all.
|
|
#
|
|
# Windows is that host. Git Bash needs developer mode or MSYS=winsymlinks to
|
|
# make one, and even where it can, mkschfs's mingw build cannot *see* one --
|
|
# lstat degrades to stat and S_ISLNK is always false, by design and with a
|
|
# comment saying so. The round-trip would therefore be comparing a tree with no
|
|
# symlinks against an image with no symlinks and calling that a pass, which is
|
|
# worse than not running: it is coverage that reports itself as present and is
|
|
# not.
|
|
#
|
|
# Skipping with exit 0 so `make schfs-check` still means something on Windows
|
|
# -- the B-tree, object and space suites are all host-independent and do run.
|
|
if ! ln -s target "$WORK/.symlink-probe" 2>/dev/null; then
|
|
echo "SchFS round-trip: SKIPPED -- this host cannot create symlinks."
|
|
echo " The generated tree needs them, and mkschfs cannot see them here"
|
|
echo " anyway. Run this on Linux, macOS or BSD for real coverage."
|
|
exit 0
|
|
fi
|
|
rm -f "$WORK/.symlink-probe"
|
|
|
|
if [ -z "$TREE" ]; then
|
|
# Generated by a separate script so the same tree can be rebuilt by hand
|
|
# when debugging a failure, without running the whole round-trip.
|
|
TREE="$WORK/tree"
|
|
sh "$(dirname "$0")/mktree.sh" "$TREE"
|
|
fi
|
|
|
|
IMG="$WORK/out.img"
|
|
OUT="$WORK/extracted"
|
|
|
|
echo "SchFS round-trip"
|
|
echo " source tree: $TREE"
|
|
|
|
"$MKSCHFS" "$IMG" --size 4096 --label roundtrip --sysroot "$TREE" | sed 's/^/ /'
|
|
|
|
echo
|
|
"$FSCK" "$IMG" --check | sed 's/^/ /'
|
|
|
|
mkdir -p "$OUT"
|
|
"$FSCK" "$IMG" --extract "$OUT" | sed 's/^/ /'
|
|
|
|
echo
|
|
# --no-dereference so a symlink is compared as a symlink rather than followed;
|
|
# without it a broken link that was stored correctly still fails the diff.
|
|
if diff -r --no-dereference "$TREE" "$OUT"; then
|
|
echo " [ok] extracted tree is identical to the source"
|
|
else
|
|
echo " FAIL: extracted tree differs from the source"
|
|
exit 1
|
|
fi
|
|
|
|
# Hard links are the one thing diff cannot see: it compares contents, and two
|
|
# separate copies with identical contents pass. Check the link count directly,
|
|
# or dedup could silently degrade to duplication and nothing would notice.
|
|
LINKS=$("$FSCK" "$IMG" --stat /bin/multiblock | sed -n 's/^nlink: *//p')
|
|
if [ "$LINKS" != "2" ]; then
|
|
echo " FAIL: /bin/multiblock has nlink=$LINKS, expected 2 (dedup did not hard-link)"
|
|
exit 1
|
|
fi
|
|
echo " [ok] duplicate content stored once, with nlink=2"
|
|
|
|
# ...and the converse: same size, different bytes must stay separate.
|
|
DLINKS=$("$FSCK" "$IMG" --stat /bin/decoy | sed -n 's/^nlink: *//p')
|
|
if [ "$DLINKS" != "1" ]; then
|
|
echo " FAIL: /bin/decoy has nlink=$DLINKS, expected 1 (dedup merged unequal files)"
|
|
exit 1
|
|
fi
|
|
echo " [ok] same-size different-content files kept separate"
|
|
|
|
# Building the same tree twice must produce the same bytes. A difference means
|
|
# something non-deterministic crept in, which makes every future image diff
|
|
# useless as a signal.
|
|
IMG2="$WORK/out2.img"
|
|
"$MKSCHFS" "$IMG2" --size 4096 --label roundtrip --sysroot "$TREE" > /dev/null
|
|
if cmp -s "$IMG" "$IMG2"; then
|
|
echo " [ok] rebuilding the same tree produces an identical image"
|
|
else
|
|
echo " FAIL: image build is not reproducible"
|
|
exit 1
|
|
fi
|
|
|
|
# Second pass with a deliberately crippled name hash.
|
|
#
|
|
# With the real 64-bit hash, two names in one directory never collide, so the
|
|
# packed-dirent list -- the fiddliest code in the object layer -- never runs at
|
|
# all. Six bits means 64 buckets for ~330 names, so every bucket holds several
|
|
# entries and lookup, readdir and the append-on-collision path all get properly
|
|
# exercised. Both tools must agree on the mask or nothing resolves.
|
|
echo
|
|
echo " forcing hash collisions (6-bit name hash)"
|
|
IMG3="$WORK/collide.img"
|
|
OUT3="$WORK/collide-out"
|
|
"$MKSCHFS" "$IMG3" --size 4096 --hash-bits 6 --sysroot "$TREE" | sed 's/^/ /'
|
|
"$FSCK" "$IMG3" --check --hash-bits 6 | sed 's/^/ /'
|
|
mkdir -p "$OUT3"
|
|
"$FSCK" "$IMG3" --extract "$OUT3" --hash-bits 6 | sed 's/^/ /'
|
|
if diff -r --no-dereference "$TREE" "$OUT3" > /dev/null; then
|
|
echo " [ok] identical tree survives a directory hash that collides constantly"
|
|
else
|
|
echo " FAIL: colliding-hash image does not round-trip"
|
|
diff -r --no-dereference "$TREE" "$OUT3" | head -10
|
|
exit 1
|
|
fi
|
|
|
|
echo
|
|
echo "round-trip passed"
|