diff --git a/README.md b/README.md index f020c69..2480e4d 100644 --- a/README.md +++ b/README.md @@ -14,17 +14,10 @@ encoder does the hard work, so the decoder only follows instructions. and natural textures — the price of having no entropy coder. - **Made for games and tools too**: mipmaps, GPU format hint, sRGB / premultiplied flags, and random access to 64×64 tiles. -- **A real texture format (YAPF 2)**: GPU-ready BC1–BC7 / BC6H / ETC2 / - ASTC blocks, 16-bit and float pixels, cubemaps, arrays and volumes, - channel meanings (normal X/Y, roughness, AO, …), coverage-preserving - mips, small-mips-first streaming, per-tile checksums, notes and - animation — see [Textures](#textures-yapf-2). - **Tiny and portable**: two C99 files, no dependencies. Plus a CLI, Python, JavaScript and plugins for the tools you already use. -Two format versions: **version 1** (an 8-bit image, the first stable -format) and **version 2** (the texture container). Plain 8-bit images are -still written as version 1, so every existing YAPF reader keeps working. +This is **YAPF version 1**, the first stable format. --- @@ -74,68 +67,7 @@ yapf compare photo.png ui.png # size and decode speed vs the original fo ``` Options when writing `.yapf`: `--mips`, `--linear` (pixels are not sRGB), -`--premult` (alpha is premultiplied), and the texture options below. - ---- - -## Textures (YAPF 2) - -Version 2 turns YAPF into a texture container for engines, designed our own -way rather than copied from KTX2 or DDS (the design and the full -specification are in [docs/FORMAT.md](docs/FORMAT.md)): - -| Need | YAPF 2 | -| :--- | :----- | -| **GPU-compressed payloads** | BC1, BC2, BC3, BC4, BC5, BC6H, BC7, ETC2 / EAC and ASTC 4×4–12×12 blocks, stored as the GPU takes them. Built-in encoders for BC1, BC3, BC4, BC5, BC7 and BC6H; or bring blocks from your own encoder. | -| **More than 8 bits** | lossless 16-bit, half and float pixels, coded as byte planes with the same fast machinery | -| **Cubes, arrays, volumes** | cubemaps and cube arrays (reflection probes, skies), texture arrays (terrain layers), 3D textures (colour-grading LUTs) | -| **Channel meaning** | per channel: normal X/Y/Z, roughness, metalness, AO, height, opacity, …; OpenGL or DirectX normal convention; linear channels inside sRGB images | -| **Streaming** | small mips are stored first: a file prefix is a complete low-resolution texture (`yapf info` prints the prefix sizes); single 64 × 64 tiles decode on their own for virtual texturing | -| **Alpha-tested mips** | `--coverage REF` keeps fences and foliage from thinning out with distance; the filter used is recorded | -| **Entropy coding (optional)** | `--entropy`: shared per-surface Huffman code books with a neighbourhood context — about 10 % smaller photos, about 3× smaller GPU blocks on disk; tiles stay independent | -| **Metadata** | notes (author, licence, source, tool, ICC profile, orientation, DPI, HDR light levels, your own tags), colour primaries and transfer (sRGB, Display P3, BT.2020, PQ, HLG, …) | -| **Integrity** | yapf-sum checksums on the header, tables and every tile; *needs* bits make an older decoder refuse cleanly what it cannot read | -| **Animation** | frames with per-frame durations and a loop count | - -```sh -yapf albedo.png albedo.yapf --format bc7 --mips # GPU-ready colour -yapf normal.png normal.yapf --format bc5 --mips --normal dx # normal map, DirectX Y -yapf orm.png orm.yapf --map ao,rough,metal --linear --mips # packed masks -yapf foliage.png foliage.yapf --format bc7 --coverage 128 # alpha-tested mips -yapf sky.hdr sky.yapf --format bc6h --mips # HDR, no RGBM tricks -yapf sky.hdr sky_lossless.yapf # lossless half float -yapf pack cube px.png nx.png py.png ny.png pz.png nz.png probe.yapf --mips -yapf pack array grass.png rock.png sand.png terrain.yapf --format bc7 --mips -yapf pack volume lut_00.png lut_01.png ... grading.yapf --linear -yapf pack anim f0.png f1.png f2.png fx.yapf --frame-ms 50 -yapf albedo.png albedo.yapf --entropy --author "Keel" --license CC-BY-4.0 -yapf info albedo.yapf # everything in the header, notes, stream prefix sizes -yapf verify albedo.yapf # check every checksum -yapf probe.yapf face2.png --face 2 --mip 1 # export one surface -``` - -In C: - -```c -int err; -yapf_texture_t *t = yapf_tex_load("albedo.yapf", 0, 0, &err); -for (uint32_t m = 0; m < t->mip_levels; m++) { - uint32_t s = yapf_tex_surface(t, 0, 0, 0, m), w, h; - yapf_tex_mip_size(t, m, &w, &h, NULL); - if (t->payload == YAPF_PAYLOAD_BLOCKS) /* e.g. YAPF_GPU_BC7 */ - glCompressedTexImage2D(GL_TEXTURE_2D, m, gl_format(t->format, t->flags), - w, h, 0, (GLsizei)yapf_tex_surface_bytes(t, m), t->data[s]); -} -yapf_tex_free(t); -``` - -Every extension opens version 2 files: GIMP (at full 16-bit / float -precision, faces and frames as layers), Blender (HDR as float images), -KDE and Linux thumbnails, VS Code (with face / layer / frame / slice -pickers), Figma and Photoshop. The Python binding (`yapf.load_texture`) -and the JavaScript port (`YAPF.decodeTexture`) read every texture; the -other bindings wrap the v1 functions, which also read 8-bit version 2 -files (2D, cube face 0, array layer 0). +`--premult` (alpha is premultiplied). --- @@ -266,47 +198,6 @@ void yapf_free(yapf_image_t *img); ``` Releases memory returned by the loaders. Passing `NULL` is safe. -### Textures (version 2) - -```c -yapf_texture_t *yapf_tex_load(const char *file, int flags, int threads, int *err); -yapf_texture_t *yapf_tex_load_memory(const void *buf, size_t size, int flags, int threads, int *err); -int yapf_tex_save(const char *file, const yapf_texture_t *t, int flags); -int yapf_tex_encode(const yapf_texture_t *t, int flags, void **out, size_t *size); -yapf_texture_t *yapf_tex_create(const yapf_texture_t *desc); /* zeroed storage */ -void yapf_tex_free(yapf_texture_t *t); -``` -A texture describes its shape (width, height, depth, layers, frames, -cube / volume, mips), storage (pixels with u8 / u16 / half / float samples, -or GPU blocks), meaning (channel map, linear mask, normal convention, -primaries, transfer, mip filter, alpha reference) and holds `data[s]` per -surface `s = yapf_tex_surface(t, frame, layer, face, mip)`. Load flag -`YAPF_LOAD_TRUST` skips checksum checks; encode flag `YAPF_ENC_ENTROPY` -allows the entropy stage. - -```c -yapf_texture_t *yapf_tex_open(const void *buf, size_t avail, int flags, int *err); -uint64_t yapf_tex_stream_bytes(const yapf_texture_t *t, uint32_t first_mip); -int yapf_tex_decode_surface(yapf_texture_t *t, const void *buf, size_t avail, uint32_t s, int threads); -int yapf_tex_decode_tile(const yapf_texture_t *t, const void *buf, size_t avail, uint32_t s, - uint32_t tx, uint32_t ty, uint32_t slice, void *out, size_t stride, - uint32_t *x, uint32_t *y, uint32_t *w, uint32_t *h); -``` -Streaming and virtual texturing: open a file from its first bytes, ask how -many bytes hold mip *m* and smaller, decode surfaces as their bytes arrive -(`YAPF_ERR_TRUNCATED` until then), or decode single tiles. - -```c -int yapf_tex_build_mips(yapf_texture_t *t, uint32_t levels, uint8_t alpha_ref); -int yapf_tex_compress(const yapf_texture_t *src, uint8_t format, yapf_texture_t **out); -int yapf_blocks_decode(uint8_t format, const void *blocks, uint32_t w, uint32_t h, void *out); -int yapf_tex_add_note(yapf_texture_t *t, const char tag[4], const void *data, uint32_t size); -const yapf_note_t *yapf_tex_find_note(const yapf_texture_t *t, const char tag[4]); -``` -Mip chains (linear-light sRGB, optional coverage preservation), the -built-in BCn encoders, BC1–BC7 decoding for previews, and notes. The -complete list, with sizes and helpers, is in [`yapf.h`](yapf.h). - ### Return codes | Constant | Value | Meaning | @@ -315,10 +206,6 @@ complete list, with sizes and helpers, is in [`yapf.h`](yapf.h). | `YAPF_ERR_INVALID` | –1 | NULL or logically invalid argument | | `YAPF_ERR_IO` | –2 | File could not be opened or written | | `YAPF_ERR_OOM` | –3 | Out of memory | -| `YAPF_ERR_CORRUPT` | –4 | Corrupt or not a YAPF file | -| `YAPF_ERR_UNSUPPORTED` | –5 | Needs a feature this decoder lacks | -| `YAPF_ERR_CHECKSUM` | –6 | A checksum does not match (v2) | -| `YAPF_ERR_TRUNCATED` | –7 | The data is not (yet) all there | ### Channel constants @@ -336,7 +223,6 @@ YAPF_CHANNELS_RGBA 4 | System | Command | Output | | :----- | :------ | :----- | | Linux, macOS, BSD | `make` · `make test` · `sudo make install` | `build/`: `libyapf.a`, `libyapf.so` / `.dylib`, `yapf` | -| NixOS / Nix | `nix-build` (or `nix-env -if .`) — see [extensions/linux](extensions/linux#nixos) | `result/`: `bin/yapf`, `lib/`, `include/`, MIME type, thumbnailer | | Linux, macOS (LLVM) | `./build.sh` | `dist//` incl. macOS universal binaries | | Windows (LLVM) | `build.bat` | `dist\windows-x64\`, `dist\windows-arm64\`: `yapf.dll`, `yapf.lib`, `libyapf.a`, `yapf.exe` | | Windows (Visual Studio) | `cl /O2 /c yapf.c` and add `yapf.obj` to your project | | @@ -370,8 +256,7 @@ domain [stb](https://github.com/nothings/stb) headers in `third_party/`. 7. **Independent tiles** — any tile can be decoded on its own, in any order, on any thread. -The version 1 specification is the comment at the top of [`yapf.h`](yapf.h); -version 2 (textures) is specified in [docs/FORMAT.md](docs/FORMAT.md). Version 1: +The complete specification is the comment at the top of [`yapf.h`](yapf.h): ``` HEADER (20 bytes) "YAPF", version 1, channels, GPU hint, mip count, @@ -388,7 +273,7 @@ TILE (64×64) header byte: stored raw, or constant-plane mask + colour tr ## Testing ```sh -make test # C suite: 2631 checks, plus JavaScript if node is installed +make test # C suite: 786 checks, plus JavaScript if node is installed node bindings/js/test.js # JavaScript: round-trips, sample file, corrupt input ``` @@ -401,16 +286,6 @@ What has been verified: C++, and under AddressSanitizer, UndefinedBehaviorSanitizer and ThreadSanitizer; - thousands of corrupted and truncated files are rejected without crashing; -- version 2: lossless round-trips of u8 / u16 / half / float pixels with 1–4 - channels, with and without entropy coding; cubes, cube arrays, arrays, - volumes and animations with notes and every description field; GPU block - payloads; checksum, needs-bit and truncation handling; streaming from file - prefixes and single-tile decoding; BCn encoder quality; linear-light and - coverage-preserving mips; fuzzing of entropy-coded and packed-block files; -- the BC1–BC7 decoders were checked against an independent reference decoder - (bcdec) on 300,000 random blocks: BC6H and BC7 bit-exact, BC1–BC5 within - the ±1 rounding the formats allow (a development check, not part of - `make test`); - every language binding runs its example; JavaScript, Python, C++, C#, Rust, Go, Java and Lua re-encode the sample to byte-identical output; - the GIMP plugin in GIMP 3.0.8 and the Blender extension in Blender 5.1; @@ -418,20 +293,14 @@ What has been verified: **Not yet verified:** running on a Mac (macOS builds are cross-compiled only), Windows on ARM, and running inside Photoshop, VS Code, Figma or a -live Dolphin / GNOME file manager. The version 2 code has been run on -Linux x64 (with and without SIMD, as C and C++, under ASan, UBSan and -TSan); its NEON path is compiled for ARM64 but has not been run yet, and -the Windows builds have not been rerun. ETC2 / EAC / ASTC blocks are -stored and loaded, but there is no built-in encoder or preview decoder -for them. +live Dolphin / GNOME file manager. --- ## Project layout ``` -yapf.h, yapf.c the library (v1 spec in yapf.h) -docs/FORMAT.md the full format specification, version 2 +yapf.h, yapf.c the library (format spec in yapf.h) tools/yapf_cli.c the `yapf` command-line tool tools/build_release.sh builds every download for all platforms tests/test_yapf.c C test suite diff --git a/bindings/js/README.md b/bindings/js/README.md index a121a1c..309c0c7 100644 --- a/bindings/js/README.md +++ b/bindings/js/README.md @@ -1,6 +1,6 @@ # yapf.js — YAPF in JavaScript -Encoder (version 1) and decoder (versions 1 and 2) in one dependency-free file. Works in +Encoder and decoder for YAPF v1 in one dependency-free file. Works in browsers, Node.js, web workers, Figma plugins and VS Code webviews, and produces exactly the same bytes as the C library. @@ -22,22 +22,3 @@ image decodes in about 150 ms in Node / Chrome (the C library: 6 ms). TypeScript types are in `yapf.d.ts`. Example: `node example.mjs ../../other/YAPF.YAPF` · tests: `node test.js`. - -## Textures (YAPF 2) - -```js -const tex = YAPF.decodeTexture(bytes); // any v1 / v2 file -// tex: { width, height, depth, layers, frames, faces, shape, mipLevels, -// payload, format, sample, channels, flags, channelMap, normal, -// primaries, transfer, frameUs, notes, surfaces: [...] } -const s = YAPF.surfaceIndex(tex, frame, layer, face, mip); -const view = YAPF.textureToRGBA(tex, s); // { width, height, rgba } for a canvas -``` - -`surfaces[s]` is a `Uint8Array` (8-bit pixels or GPU blocks, ready for -WebGL / WebGPU compressed textures), `Uint16Array` (16-bit, half) or -`Float32Array`. Checksums are verified (`{ verify: false }` skips them); -errors are `YAPF.YapfError` with a `code` (`corrupt`, `checksum`, -`truncated`, `not-yapf`, `version`, `unsupported`). `decode()` also reads -8-bit version 2 files. The decoder is checked against the C library on the -files in `test-data/`. diff --git a/bindings/js/test-data/v2-animation-notes.yapf b/bindings/js/test-data/v2-animation-notes.yapf deleted file mode 100644 index a058ba0..0000000 Binary files a/bindings/js/test-data/v2-animation-notes.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-bc3-packed.yapf b/bindings/js/test-data/v2-bc3-packed.yapf deleted file mode 100644 index 3ea3f1d..0000000 Binary files a/bindings/js/test-data/v2-bc3-packed.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-bc5-packed.yapf b/bindings/js/test-data/v2-bc5-packed.yapf deleted file mode 100644 index 258e47b..0000000 Binary files a/bindings/js/test-data/v2-bc5-packed.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-bc6h-encoded.yapf b/bindings/js/test-data/v2-bc6h-encoded.yapf deleted file mode 100644 index 24fdfbb..0000000 Binary files a/bindings/js/test-data/v2-bc6h-encoded.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-bc6h-random-blocks.yapf b/bindings/js/test-data/v2-bc6h-random-blocks.yapf deleted file mode 100644 index 4a8d304..0000000 Binary files a/bindings/js/test-data/v2-bc6h-random-blocks.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-bc7-encoded.yapf b/bindings/js/test-data/v2-bc7-encoded.yapf deleted file mode 100644 index da96f06..0000000 Binary files a/bindings/js/test-data/v2-bc7-encoded.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-bc7-random-blocks.yapf b/bindings/js/test-data/v2-bc7-random-blocks.yapf deleted file mode 100644 index e148575..0000000 Binary files a/bindings/js/test-data/v2-bc7-random-blocks.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-cube-mips.yapf b/bindings/js/test-data/v2-cube-mips.yapf deleted file mode 100644 index 3b1a801..0000000 Binary files a/bindings/js/test-data/v2-cube-mips.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-expected.json b/bindings/js/test-data/v2-expected.json deleted file mode 100644 index 148768c..0000000 --- a/bindings/js/test-data/v2-expected.json +++ /dev/null @@ -1,137 +0,0 @@ -{ - "v2-rgba8.yapf": { - "surfaces": [ - 480599072 - ], - "preview0": 480599072 - }, - "v2-float-gray-entropy.yapf": { - "surfaces": [ - 1656156314 - ], - "preview0": 2474033295 - }, - "v2-half-gray-alpha-entropy.yapf": { - "surfaces": [ - 2599040047 - ], - "preview0": 297007736 - }, - "v2-rgba8-entropy.yapf": { - "surfaces": [ - 554695116 - ], - "preview0": 554695116 - }, - "v2-u16-rgba-entropy.yapf": { - "surfaces": [ - 3416063949 - ], - "preview0": 1518500476 - }, - "v2-cube-mips.yapf": { - "surfaces": [ - 3098789294, - 2090642189, - 3178169037, - 3641419477, - 333399741, - 2333110333, - 1703812753, - 3098789294, - 2090642189, - 3178169037, - 3641419477, - 333399741, - 2333110333, - 1703812753, - 3098789294, - 2090642189, - 3178169037, - 3641419477, - 333399741, - 2333110333, - 1703812753, - 3098789294, - 2090642189, - 3178169037, - 3641419477, - 333399741, - 2333110333, - 1703812753, - 3098789294, - 2090642189, - 3178169037, - 3641419477, - 333399741, - 2333110333, - 1703812753, - 3098789294, - 2090642189, - 3178169037, - 3641419477, - 333399741, - 2333110333, - 1703812753 - ], - "preview0": 3098789294 - }, - "v2-u16-volume.yapf": { - "surfaces": [ - 1504297975, - 2857113671, - 3272367306, - 4248642449 - ], - "preview0": 3611091147 - }, - "v2-animation-notes.yapf": { - "surfaces": [ - 2496952478, - 2084186354, - 2971951804, - 4204487618 - ], - "preview0": 2496952478 - }, - "v2-bc6h-random-blocks.yapf": { - "surfaces": [ - 2051803746, - 562394057, - 2093502260 - ], - "preview0": 2386146506 - }, - "v2-bc7-random-blocks.yapf": { - "surfaces": [ - 3714498223, - 242396130, - 2835116443 - ], - "preview0": 4042788133 - }, - "v2-bc3-packed.yapf": { - "surfaces": [ - 1798067479 - ], - "preview0": 3027945807 - }, - "v2-bc5-packed.yapf": { - "surfaces": [ - 1681957115 - ], - "preview0": 1079658926 - }, - "v2-bc7-encoded.yapf": { - "surfaces": [ - 3415378546 - ], - "preview0": 851871445 - }, - "v2-bc6h-encoded.yapf": { - "surfaces": [ - 3372163902 - ], - "preview0": 89762387 - } -} \ No newline at end of file diff --git a/bindings/js/test-data/v2-float-gray-entropy.yapf b/bindings/js/test-data/v2-float-gray-entropy.yapf deleted file mode 100644 index 5dfebbb..0000000 Binary files a/bindings/js/test-data/v2-float-gray-entropy.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-half-gray-alpha-entropy.yapf b/bindings/js/test-data/v2-half-gray-alpha-entropy.yapf deleted file mode 100644 index c11c459..0000000 Binary files a/bindings/js/test-data/v2-half-gray-alpha-entropy.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-rgba8-entropy.yapf b/bindings/js/test-data/v2-rgba8-entropy.yapf deleted file mode 100644 index c709257..0000000 Binary files a/bindings/js/test-data/v2-rgba8-entropy.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-rgba8.yapf b/bindings/js/test-data/v2-rgba8.yapf deleted file mode 100644 index 1fec3ff..0000000 Binary files a/bindings/js/test-data/v2-rgba8.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-u16-rgba-entropy.yapf b/bindings/js/test-data/v2-u16-rgba-entropy.yapf deleted file mode 100644 index 334e8d8..0000000 Binary files a/bindings/js/test-data/v2-u16-rgba-entropy.yapf and /dev/null differ diff --git a/bindings/js/test-data/v2-u16-volume.yapf b/bindings/js/test-data/v2-u16-volume.yapf deleted file mode 100644 index 5dd0051..0000000 Binary files a/bindings/js/test-data/v2-u16-volume.yapf and /dev/null differ diff --git a/bindings/js/test.js b/bindings/js/test.js index 930ff58..ec37609 100644 --- a/bindings/js/test.js +++ b/bindings/js/test.js @@ -53,39 +53,5 @@ for (let i = 0; i < 2000; i++) { } check(threwOnTruncation, 'truncated files are rejected'); -/* Version 2 fixtures written by the C library: every surface and the - * preview must match what C decodes (FNV-1a hashes in v2-expected.json). */ -const fnv = (u8) => { let h = 2166136261; for (const v of u8) h = Math.imul(h ^ v, 16777619) >>> 0; return h; }; -const dataDir = path.join(__dirname, 'test-data'); -const expected = JSON.parse(fs.readFileSync(path.join(dataDir, 'v2-expected.json'), 'utf8')); -for (const [name, want] of Object.entries(expected)) { - const bytes = fs.readFileSync(path.join(dataDir, name)); - let tex = null; - try { tex = YAPF.decodeTexture(bytes); } catch (e) { check(false, `${name}: ${e.message}`); continue; } - const got = tex.surfaces.map((s) => fnv(new Uint8Array(s.buffer, s.byteOffset, s.byteLength))); - check(JSON.stringify(got) === JSON.stringify(want.surfaces), `${name}: surfaces match the C decoder`); - if (want.preview0 !== null) { - const view = YAPF.textureToRGBA(tex, 0, 0); - /* float previews may differ by 1 in rounding; compare only exact kinds */ - if (tex.payload === 1 ? tex.format !== 0x47 && tex.format !== 0x48 : tex.sample < 2) - check(fnv(view.rgba) === want.preview0, `${name}: preview matches the C library`); - } - /* damage must be caught by the checksums */ - const broken = Uint8Array.from(bytes); - broken[broken.length - 3] ^= 0x10; - let code = null; - try { YAPF.decodeTexture(broken); } catch (e) { code = e.code; } - check(code === 'checksum', `${name}: a damaged byte is reported as a checksum error`); -} -{ /* v1 decode() reads 8-bit v2 files and names what it cannot read */ - const cube = YAPF.decode(fs.readFileSync(path.join(dataDir, 'v2-cube-mips.yapf'))); - check(cube.mipLevels === 7 && cube.width === 64, 'decode() reads face 0 of a v2 cube'); - let msg = ''; - try { YAPF.decode(fs.readFileSync(path.join(dataDir, 'v2-bc7-encoded.yapf'))); } catch (e) { msg = e.message; } - check(/decodeTexture/.test(msg), 'decode() points GPU-block files to decodeTexture'); - const yapfSum = (t) => YAPF.checksum(new TextEncoder().encode(t)); - check(yapfSum('') === 0xd7550ed0 && yapfSum('YAPF') === 0xc8cc41d4, 'yapf-sum test vectors'); -} - console.log(`${tests} tests, ${fails} failures`); process.exit(fails ? 1 : 0); diff --git a/bindings/js/yapf.js b/bindings/js/yapf.js index fb487fe..194e1a7 100644 --- a/bindings/js/yapf.js +++ b/bindings/js/yapf.js @@ -1,5 +1,5 @@ /* - * yapf.js — YAPF encoder (v1) and decoder (v1 and v2) in plain JavaScript. + * yapf.js — YAPF v1 encoder / decoder in plain JavaScript. * * Copyright 2026 Nexoniarz — Apache License 2.0. * @@ -10,14 +10,9 @@ * const file = YAPF.encode(img); // image → Uint8Array * const rgba = YAPF.toRGBA(img); // any channel count → RGBA pixels * - * const tex = YAPF.decodeTexture(bytes); // any v1 / v2 file - * const view = YAPF.textureToRGBA(tex, surface); // { width, height, rgba } - * * An image is { width, height, channels (1–4), pixels: Uint8Array, * gpuFormat, flags, mipLevels, mips: [Uint8Array, …] }. - * Version 2 textures (GPU blocks, 16-bit / float, cubes, arrays, volumes, - * animation) are decoded by decodeTexture; encoding writes version 1. - * The specification: yapf.h (v1) and docs/FORMAT.md (v2). + * The specification is the comment at the top of yapf.h. */ (function (root, factory) { if (typeof module === 'object' && module.exports) module.exports = factory(); @@ -47,676 +42,7 @@ const filterValid = (f) => f <= F_UPAVG || (f >= F_UP && f < F_UP + 8); const upK = (f) => (f >= F_UP ? f - F_UP + 1 : 1); - function corrupt() { throw new YapfError('corrupt', 'corrupt file'); } - - /* ── Version 2: textures (decoding) ───────────────────────────────── */ - - const ENTROPY = 0x20, NOXFORM = 0x40, BLOCK_TILE = 16; - const HUFF_BITS = 11, HUFF_SIZE = 1 << HUFF_BITS, SLOTS = 48, CTXS = 7; - const NEED = { WIDE: 1, BLOCKS: 2, ENTROPY: 4, LAYOUT: 8, FRAMES: 16, ALL: 31 }; - const r16 = (b, p) => b[p] | (b[p + 1] << 8); - const r64 = (b, p) => r32(b, p) + r32(b, p + 4) * 4294967296; - - class YapfError extends Error { - constructor(code, msg) { super('YAPF: ' + msg); this.code = code; } - } - const fail = (code, msg) => { throw new YapfError(code, msg); }; - const bad = () => fail('corrupt', 'corrupt file'); - - /* yapf-sum: 64-bit lanes kept as [hi, lo] uint32 pairs. */ - const KH = 0x9E3779B9, KL = 0x7F4A7C15; - function mul64(ah, al, bh, bl) { /* (a · b) mod 2^64 */ - const a0 = al & 0xFFFF, a1 = al >>> 16, b0 = bl & 0xFFFF, b1 = bl >>> 16; - const p00 = a0 * b0, p01 = a0 * b1, p10 = a1 * b0, p11 = a1 * b1; - const mid = (p00 >>> 16) + (p01 & 0xFFFF) + (p10 & 0xFFFF); - const lo = ((mid & 0xFFFF) * 65536 + (p00 & 0xFFFF)) >>> 0; - const hi = (p11 + (p01 >>> 16) + (p10 >>> 16) + Math.floor(mid / 65536) + - Math.imul(ah, bl) + Math.imul(al, bh)) >>> 0; - return [hi, lo]; - } - function checksum(b, p, n) { - const lanes = []; - for (let i = 0; i < 4; i++) { - const [h, l] = mul64(0, i + 1, KH, KL); - lanes.push([(h ^ Math.floor(n / 4294967296)) >>> 0, (l ^ (n >>> 0)) >>> 0]); - } - const words = Math.ceil(n / 8); - for (let j = 0; j < words; j++) { - const q = p + j * 8; - let wl, wh; - if (q + 8 <= p + n) { wl = r32(b, q); wh = r32(b, q + 4); } - else { - const t = new Uint8Array(8); t.set(b.subarray(q, p + n)); - wl = r32(t, 0); wh = r32(t, 4); - } - const a = lanes[j & 3]; - const [h, l] = mul64((a[0] ^ wh) >>> 0, (a[1] ^ wl) >>> 0, KH, KL); - a[0] = h; a[1] = (l ^ h) >>> 0; /* a ^= a >> 32 */ - } - const rot = (x, r) => r === 32 ? [x[1], x[0]] - : [((x[0] << r) | (x[1] >>> (32 - r))) >>> 0, ((x[1] << r) | (x[0] >>> (32 - r))) >>> 0]; - const r1 = rot(lanes[1], 16), r2 = rot(lanes[2], 32), r3 = rot(lanes[3], 48 - 32); - const r3b = [r3[1], r3[0]]; /* rotl 48 = rotl 16 then swap */ - let hh = (lanes[0][0] ^ r1[0] ^ r2[0] ^ r3b[0]) >>> 0, hl = (lanes[0][1] ^ r1[1] ^ r2[1] ^ r3b[1]) >>> 0; - [hh, hl] = mul64(hh, hl, KH, KL); - return (hl ^ hh) >>> 0; - } - - /* Huffman code table → lut[off .. off + 2048) (length << 8 | symbol). */ - function huffTable(b, p, end, lut, off) { - if (p >= end) bad(); - const n = b[p++] + 1, nbytes = (n + 1) >> 1; - if (end - p < nbytes) bad(); - const len = new Uint8Array(n), count = new Uint32Array(HUFF_BITS + 2); - let kraft = 0; - for (let i = 0; i < n; i++) { - const l = (b[p + (i >> 1)] >> ((i & 1) * 4)) & 15; - if (l > HUFF_BITS) bad(); - len[i] = l; count[l]++; - if (l) kraft += HUFF_SIZE >> l; - } - if ((n & 1) && (b[p + (n >> 1)] >> 4)) bad(); - if (kraft === 0 || kraft > HUFF_SIZE) bad(); - p += nbytes; - const next = new Uint32Array(HUFF_BITS + 2); - let code = 0; - count[0] = 0; - for (let l = 1; l <= HUFF_BITS; l++) { code = (code + count[l - 1]) << 1; next[l] = code; } - for (let s = 0; s < n; s++) { - const l = len[s]; - if (!l) continue; - let c = next[l]++, r = 0; - for (let i = 0; i < l; i++) { r = (r << 1) | (c & 1); c >>= 1; } - for (let k = r; k < HUFF_SIZE; k += 1 << l) lut[off + k] = (l << 8) | s; - } - return p; - } - - /* Code book → per slot one Uint16Array of 7 × 2048 entries. */ - function readBook(b, p, len) { - const end = p + len; - if (len < 2) bad(); - const n = r16(b, p); - p += 2; - if (n === 0 || n > SLOTS * CTXS) bad(); - const book = new Array(SLOTS).fill(null), seen = new Uint8Array(SLOTS * CTXS); - for (let i = 0; i < n; i++) { - if (end - p < 3) bad(); - const slot = b[p], ctx = b[p + 1]; - if (slot >= SLOTS || ctx >= CTXS || seen[slot * CTXS + ctx]) bad(); - seen[slot * CTXS + ctx] = 1; - if (!book[slot]) book[slot] = new Uint16Array(CTXS * HUFF_SIZE); - p = huffTable(b, p + 2, end, book[slot], ctx * HUFF_SIZE); - } - if (p !== end) bad(); - return book; - } - - /* Context of a residual from the folded residuals left + above. */ - const ctxOf = (s) => (s > 0) + (s > 2) + (s > 6) + (s > 14) + (s > 30) + (s > 62); - - /* Entropy plane payload: filters, u16 length, context-coded residuals. */ - function huffPlane(b, p, end, z, tw, th, book, slot) { - const nb = (th + BAND - 1) >> 3; - if (!book || !book[slot] || end - p < nb + 2) bad(); - const filt = b.subarray(p, p + nb); - for (let i = 0; i < nb; i++) if (!filterValid(filt[i])) bad(); - p += nb; - const L = r16(b, p); - p += 2; - if (end - p < L) bad(); - const lut = book[slot], se = p + L; - let buf = 0, bits = 0, pos = p, used = 0, i = 0; - for (let y = 0; y < th; y++) - for (let x = 0; x < tw; x++, i++) { - while (bits <= 24) { buf |= (pos < se ? b[pos] : 0) << bits; pos++; bits += 8; } - const s = (x ? z[i - 1] : 0) + (y ? z[i - tw] : 0); - const e = lut[(ctxOf(s) << HUFF_BITS) | (buf & (HUFF_SIZE - 1))]; - if (!e) bad(); - const n = e >> 8; - buf >>>= n; bits -= n; used += n; - z[i] = e & 255; - } - if (Math.ceil(used / 8) !== L) bad(); - z.fill(0, tw * th, tw * th + GROUP); - return { p: se, filt }; - } - - /* Wide samples: byte planes, transforms at the sample width. */ - function decodeTileWide(b, p, end, out, imgW, tx, ty, tw, th, ch, sample, book) { - const sb = sample === 3 ? 4 : 2, bits = 8 * sb, isf = sample >= 2, np = ch * sb; - const half = bits === 32 ? 0x80000000 : 0x8000; - const mod = bits === 32 ? (x) => x >>> 0 : (x) => x & 0xFFFF; - const sar1 = (v) => mod((v >>> 1) | (v & half)); - if (end <= p) bad(); - const hdr = b[p++]; - if (hdr === STORED) { - if (end - p !== tw * th * ch * sb) bad(); - for (let y = 0; y < th; y++) { - const d = ((ty + y) * imgW + tx) * ch; - for (let i = 0; i < tw * ch; i++, p += sb) - out[d + i] = sb === 2 ? r16(b, p) : r32(b, p); - } - return; - } - if (hdr & 0x8F) bad(); - if ((hdr & NOXFORM) && (hdr & SUBGREEN)) bad(); - if (ch < 3 && (hdr & (NOXFORM | SUBGREEN))) bad(); - const xf = ch < 3 ? 2 : (hdr & NOXFORM) ? 2 : (hdr & SUBGREEN) ? 1 : 0; - const tsel = ch < 3 ? 0 : xf; - if (end - p < 2) bad(); - const cmask = r16(b, p); - p += 2; - if (cmask >> np) bad(); - let emask = 0; - if (hdr & ENTROPY) { - if (end - p < 2 || !book) bad(); - emask = r16(b, p); - p += 2; - if ((emask >> np) || (emask & cmask)) bad(); - } - const RL = TILE + 1, PL = TILE * RL; - const planes = new Uint8Array(np * PL), z = new Uint8Array(TILE_PX + GROUP); - for (let pl = 0; pl < np; pl++) { - if (!(cmask & (1 << pl))) continue; - if (p >= end) bad(); - planes.fill(b[p++], pl * PL, (pl + 1) * PL); - } - for (let pl = 0; pl < np; pl++) { - if (cmask & (1 << pl)) continue; - z.fill(0); - const r = (emask & (1 << pl)) ? huffPlane(b, p, end, z, tw, th, book, tsel * 16 + pl) - : unpackPlane(b, p, end, z, tw, th); - p = r.p; - const base = pl * PL; - for (let y = 0; y < th; y++) { - const f = r.filt[y >> 3], k = upK(f); - unfilterRow(f, planes, base + y * RL, - y ? planes : ZERO_ROW, y ? base + (y - 1) * RL : 0, - y >= k ? planes : ZERO_ROW, y >= k ? base + (y - k) * RL : 0, - z, y * tw, tw); - } - } - if (p !== end) bad(); - const v = [0, 0, 0, 0]; - for (let y = 0; y < th; y++) { - const d0 = ((ty + y) * imgW + tx) * ch, r0 = y * RL; - for (let x = 0; x < tw; x++) { - for (let c = 0; c < ch; c++) { - let a = 0; - for (let k = 0; k < sb; k++) a = a * 256 + planes[(c * sb + k) * PL + r0 + x]; - v[c] = a; - } - if (xf === 0) { - const o = mod(v[1] - half), q = mod(v[2] - half), t = mod(v[0] - sar1(q)); - const g = mod(q + t), bb = mod(t - sar1(o)); - v[0] = mod(bb + o); v[1] = g; v[2] = bb; - } else if (xf === 1) { - const g = v[0]; - v[0] = mod(v[1] + g - half); v[2] = mod(v[2] + g - half); v[1] = g; - } - for (let c = 0; c < ch; c++) { - let s = v[c]; - if (isf) s = (s & half) ? mod(s ^ half) : mod(~s); - out[d0 + x * ch + c] = s; - } - } - } - } - - /* GPU block tile: raw, or packed lane by lane with the book. */ - function decodeBlockTile(b, p, end, packed, out, o0, rowb, tbw, tbh, S, book) { - const n = tbw * tbh; - if (!packed) { - if (end - p !== n * S) bad(); - for (let r = 0; r < tbh; r++) out.set(b.subarray(p + r * tbw * S, p + (r + 1) * tbw * S), o0 + r * rowb); - return; - } - if (!book) bad(); - let buf = 0, bits = 0, pos = p, used = 0; - for (let k = 0; k < S; k++) { - const lut = book[k]; - if (!lut) bad(); - for (let i = 0; i < n; i++) { - while (bits <= 24) { buf |= (pos < end ? b[pos] : 0) << bits; pos++; bits += 8; } - const e = lut[buf & (HUFF_SIZE - 1)]; - if (!e) bad(); - const l = e >> 8; - buf >>>= l; bits -= l; used += l; - out[o0 + Math.floor(i / tbw) * rowb + (i % tbw) * S + k] = e & 255; - } - } - if (Math.ceil(used / 8) !== end - p) bad(); - } - - const BLOCK_FORMATS = { - 0x40: [4, 4, 8], 0x41: [4, 4, 16], 0x42: [4, 4, 16], 0x43: [4, 4, 8], 0x44: [4, 4, 8], - 0x45: [4, 4, 16], 0x46: [4, 4, 16], 0x47: [4, 4, 16], 0x48: [4, 4, 16], 0x49: [4, 4, 16], - 0x50: [4, 4, 8], 0x51: [4, 4, 8], 0x52: [4, 4, 16], 0x53: [4, 4, 8], 0x54: [4, 4, 8], - 0x55: [4, 4, 16], 0x56: [4, 4, 16] - }; - [[4, 4], [5, 4], [5, 5], [6, 5], [6, 6], [8, 5], [8, 6], [8, 8], [10, 5], [10, 6], [10, 8], - [10, 10], [12, 10], [12, 12]].forEach(([w, h], i) => { BLOCK_FORMATS[0x60 + i] = [w, h, 16]; }); - - function mipSize(t, m) { - return [Math.max(1, t.width >>> m), Math.max(1, t.height >>> m), - t.shape === 2 ? Math.max(1, t.depth >>> m) : 1]; - } - - function surfaceBytes(t, m) { - const [w, h, d] = mipSize(t, m); - if (t.payload === 1) { - const [bw, bh, S] = BLOCK_FORMATS[t.format]; - return Math.ceil(w / bw) * Math.ceil(h / bh) * d * S; - } - return w * h * d * t.channels * (t.sample === 0 ? 1 : t.sample === 3 ? 4 : 2); - } - - /* Decodes one surface (v2) into a typed array. */ - function decodeSurface(b, t, s, verify) { - const m = s % t.mipLevels, [w, h, d] = mipSize(t, m); - const off = t._off[s], size = t._size[s], blocks = t.payload === 1; - let tcx, tcy, nbx = 0, nby = 0, S = 0; - if (blocks) { - const f = BLOCK_FORMATS[t.format]; - nbx = Math.ceil(w / f[0]); nby = Math.ceil(h / f[1]); S = f[2]; - tcx = Math.ceil(nbx / BLOCK_TILE); tcy = Math.ceil(nby / BLOCK_TILE); - } else { - tcx = Math.ceil(w / TILE); tcy = Math.ceil(h / TILE); - } - const tiles = tcx * tcy * d; - if (size < 4) bad(); - const bl = r32(b, off); - if (bl > size - 4 || Math.floor((size - 4 - bl) / 8) < tiles) bad(); - const pre = 4 + bl; - if (verify && checksum(b, off, pre + tiles * 8) !== t._tcheck[s]) fail('checksum', 'damaged file: a checksum does not match'); - let book = null; - if (bl) { - if (!(t.needs & NEED.ENTROPY)) bad(); - book = readBook(b, off + 4, bl); - } - const table = off + pre, end = off + size; - let pos = table + tiles * 8; - const sizes = new Array(tiles), packed = new Array(tiles), starts = new Array(tiles); - for (let i = 0; i < tiles; i++) { - let sz = r32(b, table + i * 8); - packed[i] = sz >>> 31; - sz &= 0x7FFFFFFF; - if (packed[i] && (!blocks || !book)) bad(); - if (sz > end - pos) bad(); - starts[i] = pos; sizes[i] = sz; pos += sz; - } - if (pos !== end) bad(); - - const bytes = surfaceBytes(t, m); - let out, store; - if (blocks || t.sample === 0) out = store = new Uint8Array(bytes); - else if (t.sample === 3) { store = new Uint32Array(bytes / 4); out = new Float32Array(store.buffer); } - else out = store = new Uint16Array(bytes / 2); - for (let i = 0; i < tiles; i++) { - const p = starts[i], e = p + sizes[i]; - if (verify && checksum(b, p, sizes[i]) !== r32(b, table + i * 8 + 4)) - fail('checksum', 'damaged file: a checksum does not match'); - const row = Math.floor(i / tcx), z = Math.floor(row / tcy), ty = row % tcy, tx = i % tcx; - if (blocks) { - const bx0 = tx * BLOCK_TILE, by0 = ty * BLOCK_TILE, rowb = nbx * S; - decodeBlockTile(b, p, e, packed[i], store, z * nby * rowb + by0 * rowb + bx0 * S, rowb, - Math.min(BLOCK_TILE, nbx - bx0), Math.min(BLOCK_TILE, nby - by0), S, book); - } else { - const x0 = tx * TILE, y0 = ty * TILE, tw = Math.min(TILE, w - x0), th = Math.min(TILE, h - y0); - const slice = w * h * t.channels; - const view = store.subarray(z * slice, (z + 1) * slice); - if (t.sample === 0) decodeTile(b, p, e, view, w, x0, y0, tw, th, t.channels, true, book); - else decodeTileWide(b, p, e, view, w, x0, y0, tw, th, t.channels, t.sample, book); - } - } - return out; - } - - /* - * decodeTexture(bytes, { verify = true }) — any YAPF file (v1 or v2). - * Returns { version, width, height, depth, layers, frames, faces, shape, - * mipLevels, payload, format, sample, channels, flags, channelMap, - * linearMask, normal, primaries, transfer, mipFilter, alphaRef, - * loopCount, frameUs, notes: [{ tag, data }], surfaces: [...] }, where - * surfaces[s] is a Uint8Array (8-bit pixels, GPU blocks), Uint16Array - * (16-bit, half bits) or Float32Array, numbered - * s = ((frame · layers + layer) · faces + face) · mipLevels + mip. - * Errors are YapfError with .code 'not-yapf', 'version', 'unsupported', - * 'corrupt', 'checksum' or 'truncated'. - */ - function decodeTexture(input, opts) { - const b = input instanceof Uint8Array ? input : new Uint8Array(input); - const verify = !(opts && opts.verify === false); - if (b.length < 5 || b[0] !== 0x59 || b[1] !== 0x41 || b[2] !== 0x50 || b[3] !== 0x46) - fail('not-yapf', 'not a YAPF file'); - if (b[4] === 1) { - const img = decodeV1(b); - return { version: 1, width: img.width, height: img.height, depth: 1, layers: 1, frames: 1, - faces: 1, shape: 0, mipLevels: img.mipLevels, payload: 0, format: img.gpuFormat, - sample: 0, channels: img.channels, flags: img.flags, channelMap: [0, 0, 0, 0], - linearMask: 0, normal: 0, primaries: 0, transfer: 0, mipFilter: 0, alphaRef: 0, - loopCount: 0, frameUs: [], notes: [], surfaces: img.mips }; - } - if (b[4] !== 2) fail('version', 'format version ' + b[4] + ' is newer than this decoder'); - if (b.length < 64) fail('truncated', 'the file is truncated'); - if (verify && checksum(b, 0, 0x3C) !== r32(b, 0x3C)) fail('checksum', 'damaged file: a checksum does not match'); - const t = { - version: 2, channels: b[5], format: b[6], mipLevels: b[7], flags: b[8], payload: b[9], - sample: b[10], shape: b[11], width: r32(b, 0x0C), height: r32(b, 0x10), depth: r32(b, 0x14), - layers: r16(b, 0x18), frames: r16(b, 0x1A), needs: r32(b, 0x1C), - channelMap: Array.from(b.subarray(0x20, 0x24)), linearMask: b[0x24], normal: b[0x25], - primaries: b[0x26], transfer: b[0x27], mipFilter: b[0x28], alphaRef: b[0x29], - loopCount: r16(b, 0x2A), frameUs: [], notes: [] - }; - const notesSize = r32(b, 0x2C), fileSize = r64(b, 0x30); - if (t.needs & ~NEED.ALL) fail('unsupported', 'the file needs a feature this decoder does not have'); - t.faces = t.shape === 1 ? 6 : 1; - if (t.channels < 1 || t.channels > 4 || t.sample > 3 || t.payload > 1 || t.shape > 2 || - t.width < 1 || t.width > MAX_DIM || t.height < 1 || t.height > MAX_DIM || - !t.layers || !t.frames || t.mipLevels < 1 || t.mipLevels > MAX_MIPS) bad(); - if (t.shape === 2 ? (t.depth < 1 || t.depth > MAX_DIM || t.layers !== 1) : t.depth > 1) bad(); - if (t.shape !== 2) t.depth = 1; - if (t.shape === 1 && t.width !== t.height) bad(); - if (t.payload === 1 && !BLOCK_FORMATS[t.format]) bad(); - if ((t.payload === 0 && t.sample !== 0 && !(t.needs & NEED.WIDE)) || - (t.payload === 1 && !(t.needs & NEED.BLOCKS)) || - ((t.shape !== 0 || t.layers > 1) && !(t.needs & NEED.LAYOUT)) || - (t.frames > 1 && !(t.needs & NEED.FRAMES))) bad(); - const S = t.frames * t.layers * t.faces * t.mipLevels; - if (S > (1 << 24)) bad(); - const front = 64 + 16 * S + (t.frames > 1 ? 4 * t.frames : 0) + notesSize; - if (front > fileSize) bad(); - if (b.length < fileSize) fail('truncated', 'the file is truncated'); - if (verify && checksum(b, 64, front - 64) !== r32(b, 0x38)) fail('checksum', 'damaged file: a checksum does not match'); - let q = 64 + 16 * S; - for (let f = 0; t.frames > 1 && f < t.frames; f++, q += 4) t.frameUs.push(r32(b, q)); - const qe = q + notesSize; - while (q < qe) { - if (qe - q < 8) bad(); - const sz = r32(b, q + 4); - if (qe - q - 8 < sz) bad(); - t.notes.push({ tag: String.fromCharCode(b[q], b[q + 1], b[q + 2], b[q + 3]), data: b.slice(q + 8, q + 8 + sz) }); - q += 8 + sz; - } - t._off = []; t._size = []; t._tcheck = []; - for (let s = 0; s < S; s++) { - const e = 64 + 16 * s, off = r64(b, e), size = r32(b, e + 8); - if (off < front || off > fileSize || size > fileSize - off) bad(); - t._off.push(off); t._size.push(size); t._tcheck.push(r32(b, e + 12)); - } - t.surfaces = []; - for (let s = 0; s < S; s++) t.surfaces.push(decodeSurface(b, t, s, verify)); - delete t._off; delete t._size; delete t._tcheck; - return t; - } - - /* ── GPU block decoding (previews) ───────────────────────────────── */ - - const PART2 = [0xCCCC, 0x8888, 0xEEEE, 0xECC8, 0xC880, 0xFEEC, 0xFEC8, 0xEC80, 0xC800, 0xFFEC, 0xFE80, 0xE800, - 0xFFE8, 0xFF00, 0xFFF0, 0xF000, 0xF710, 0x008E, 0x7100, 0x08CE, 0x008C, 0x7310, 0x3100, 0x8CCE, 0x088C, 0x3110, - 0x6666, 0x366C, 0x17E8, 0x0FF0, 0x718E, 0x399C, 0xAAAA, 0xF0F0, 0x5A5A, 0x33CC, 0x3C3C, 0x55AA, 0x9696, 0xA55A, - 0x73CE, 0x13C8, 0x324C, 0x3BDC, 0x6996, 0xC33C, 0x9966, 0x0660, 0x0272, 0x04E4, 0x4E40, 0x2720, 0xC936, 0x936C, - 0x39C6, 0x639C, 0x9336, 0x9CC6, 0x817E, 0xE718, 0xCCF0, 0x0FCC, 0x7744, 0xEE22]; - const PART3 = [0xAA685050, 0x6A5A5040, 0x5A5A4200, 0x5450A0A8, 0xA5A50000, 0xA0A05050, 0x5555A0A0, 0x5A5A5050, - 0xAA550000, 0xAA555500, 0xAAAA5500, 0x90909090, 0x94949494, 0xA4A4A4A4, 0xA9A59450, 0x2A0A4250, 0xA5945040, - 0x0A425054, 0xA5A5A500, 0x55A0A0A0, 0xA8A85454, 0x6A6A4040, 0xA4A45000, 0x1A1A0500, 0x0050A4A4, 0xAAA59090, - 0x14696914, 0x69691400, 0xA08585A0, 0xAA821414, 0x50A4A450, 0x6A5A0200, 0xA9A58000, 0x5090A0A8, 0xA8A09050, - 0x24242424, 0x00AA5500, 0x24924924, 0x24499224, 0x50A50A50, 0x500AA550, 0xAAAA4444, 0x66660000, 0xA5A0A5A0, - 0x50A050A0, 0x69286928, 0x44AAAA44, 0x66666600, 0xAA444444, 0x54A854A8, 0x95809580, 0x96969600, 0xA85454A8, - 0x80959580, 0xAA141414, 0x96960000, 0xAAAA1414, 0xA05050A0, 0xA0A5A5A0, 0x96000000, 0x40804080, 0xA9A8A9A8, - 0xAAAAAA44, 0x2A4A5254]; - const ANCHOR2 = [15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 2, 8, 2, 2, 8, 8, 15, 2, 8, 2, 2, - 8, 8, 2, 2, 15, 15, 6, 8, 2, 8, 15, 15, 2, 8, 2, 2, 2, 15, 15, 6, 6, 2, 6, 8, 15, 15, 2, 2, 15, 15, 15, 15, 15, 2, 2, 15]; - const ANCHOR3A = [3, 3, 15, 15, 8, 3, 15, 15, 8, 8, 6, 6, 6, 5, 3, 3, 3, 3, 8, 15, 3, 3, 6, 10, 5, 8, 8, 6, 8, 5, 15, - 15, 8, 15, 3, 5, 6, 10, 8, 15, 15, 3, 15, 5, 15, 15, 15, 15, 3, 15, 5, 5, 5, 8, 5, 10, 5, 10, 8, 13, 15, 12, 3, 3]; - const ANCHOR3B = [15, 8, 8, 3, 15, 15, 3, 8, 15, 15, 15, 15, 15, 15, 15, 8, 15, 8, 15, 3, 15, 8, 15, 8, 3, 15, 6, 10, - 15, 15, 10, 8, 15, 3, 15, 10, 10, 8, 9, 10, 6, 15, 8, 15, 3, 6, 6, 8, 15, 3, 15, 15, 15, 15, 15, 15, 15, 15, 15, - 15, 3, 15, 15, 8]; - const WEIGHTS = { 2: [0, 21, 43, 64], 3: [0, 9, 18, 27, 37, 46, 55, 64], - 4: [0, 4, 9, 13, 17, 21, 26, 30, 34, 38, 43, 47, 51, 55, 60, 64] }; - const interp = (a, b, w) => (a * (64 - w) + b * w + 32) >> 6; - - function bits128(b, o) { - let pos = 0; - return (n) => { - let v = 0; - for (let i = 0; i < n; i++, pos++) v |= ((b[o + (pos >> 3)] >> (pos & 7)) & 1) << i; - return v >>> 0; - }; - } - - function rgb565(c) { - const r = (c >> 11) & 31, g = (c >> 5) & 63, bl = c & 31; - return [(r << 3) | (r >> 2), (g << 2) | (g >> 4), (bl << 3) | (bl >> 2)]; - } - - function bc1Block(b, o, out, fourOnly) { - const c0 = r16(b, o), c1 = r16(b, o + 2), idx = r32(b, o + 4), four = c0 > c1 || fourOnly; - const p0 = rgb565(c0), p1 = rgb565(c1), pal = [p0.concat(255), p1.concat(255), [0, 0, 0, 255], [0, 0, 0, 255]]; - for (let k = 0; k < 3; k++) { - if (four) { pal[2][k] = ((2 * p0[k] + p1[k] + 1) / 3) | 0; pal[3][k] = ((p0[k] + 2 * p1[k] + 1) / 3) | 0; } - else { pal[2][k] = (p0[k] + p1[k] + 1) >> 1; pal[3][k] = 0; } - } - if (!four) pal[3][3] = 0; - for (let i = 0; i < 16; i++) { const c = pal[(idx >>> (2 * i)) & 3]; for (let k = 0; k < 4; k++) out[i * 4 + k] = c[k]; } - } - - function bc4Block(b, o, out, ch, snorm) { - const pal = new Array(8); - let r0 = b[o], r1 = b[o + 1]; - if (snorm) { r0 = Math.max(-127, (r0 << 24) >> 24); r1 = Math.max(-127, (r1 << 24) >> 24); } - pal[0] = r0; pal[1] = r1; - const div = (v, d) => (v >= 0 ? Math.floor((v + (d >> 1)) / d) : -Math.floor((-v + (d >> 1)) / d)); - if (r0 > r1) for (let i = 2; i < 8; i++) pal[i] = div((8 - i) * r0 + (i - 1) * r1, 7); - else { for (let i = 2; i < 6; i++) pal[i] = div((6 - i) * r0 + (i - 1) * r1, 5); pal[6] = snorm ? -127 : 0; pal[7] = snorm ? 127 : 255; } - let lo = r32(b, o + 2), hi = r16(b, o + 6); - for (let i = 0; i < 16; i++) { - const bit = 3 * i, v = bit < 30 ? (lo >>> bit) & 7 : bit === 30 ? ((lo >>> 30) | ((hi & 1) << 2)) : (hi >> (bit - 32)) & 7; - out[i * 4 + ch] = pal[v] + (snorm ? 128 : 0); - } - } - - const BC7 = [[3, 4, 0, 0, 4, 0, 1, 0, 3, 0], [2, 6, 0, 0, 6, 0, 0, 1, 3, 0], [3, 6, 0, 0, 5, 0, 0, 0, 2, 0], - [2, 6, 0, 0, 7, 0, 1, 0, 2, 0], [1, 0, 2, 1, 5, 6, 0, 0, 2, 3], [1, 0, 2, 0, 7, 8, 0, 0, 2, 2], - [1, 0, 0, 0, 7, 7, 1, 0, 4, 0], [2, 6, 0, 0, 5, 5, 1, 0, 2, 0]]; - const subsetOf = (ns, part, i) => ns === 2 ? (PART2[part] >> i) & 1 : ns === 3 ? (PART3[part] >>> (2 * i)) & 3 : 0; - const isAnchor = (ns, part, i) => i === 0 || (ns === 2 && i === ANCHOR2[part]) || - (ns === 3 && (i === ANCHOR3A[part] || i === ANCHOR3B[part])); - - function bc7Block(b, o, out) { - const get = bits128(b, o); - let mode = 0; - while (mode < 8 && !get(1)) mode++; - if (mode === 8) { out.fill(0, 0, 64); return; } - const [ns, pb, rb, isbB, cb, ab, epb, spb, ib, ib2] = BC7[mode]; - const part = get(pb), rot = get(rb), isb = get(isbB), ne = ns * 2; - const ep = []; for (let e = 0; e < ne; e++) ep.push([0, 0, 0, 255]); - for (let c = 0; c < 3; c++) for (let e = 0; e < ne; e++) ep[e][c] = get(cb); - for (let e = 0; e < ne; e++) ep[e][3] = ab ? get(ab) : 255; - const pbit = epb | spb; - if (epb) for (let e = 0; e < ne; e++) { const p = get(1); for (let c = 0; c < 4; c++) if (c < 3 || ab) ep[e][c] = (ep[e][c] << 1) | p; } - else if (spb) for (let k = 0; k < ns; k++) { const p = get(1); for (let c = 0; c < 3; c++) { ep[2 * k][c] = (ep[2 * k][c] << 1) | p; ep[2 * k + 1][c] = (ep[2 * k + 1][c] << 1) | p; } } - for (let e = 0; e < ne; e++) for (let c = 0; c < 4; c++) { - if (c === 3 && !ab) continue; - const nb = (c < 3 ? cb : ab) + pbit, v = ep[e][c] << (8 - nb); - ep[e][c] = v | (v >> nb); - } - const idx = [], idx2 = []; - for (let i = 0; i < 16; i++) idx.push(get(ib - (isAnchor(ns, part, i) ? 1 : 0))); - for (let i = 0; i < 16; i++) idx2.push(ib2 ? get(ib2 - (i === 0 ? 1 : 0)) : 0); - const w1 = WEIGHTS[ib], w2 = WEIGHTS[ib2 || 2]; - for (let i = 0; i < 16; i++) { - const k = subsetOf(ns, part, i), A = ep[2 * k], B = ep[2 * k + 1]; - let cw, aw; - if (!ib2) cw = aw = w1[idx[i]]; - else if (!isb) { cw = w1[idx[i]]; aw = w2[idx2[i]]; } - else { cw = w2[idx2[i]]; aw = w1[idx[i]]; } - const px = [interp(A[0], B[0], cw), interp(A[1], B[1], cw), interp(A[2], B[2], cw), interp(A[3], B[3], aw)]; - if (rot) { const t = px[3]; px[3] = px[rot - 1]; px[rot - 1] = t; } - for (let c = 0; c < 4; c++) out[i * 4 + c] = px[c]; - } - } - - /* BC6H header layouts: [field, first bit, count]; fields 0–3 red w x y z, - * 4–7 green, 8–11 blue, 12 partition. */ - const RW = 0, RX = 1, RY = 2, RZ = 3, GW = 4, GX = 5, GY = 6, GZ = 7, BW = 8, BX = 9, BY = 10, BZ = 11, PD = 12; - const BC6 = [ - [0x00, 10, 5, 5, 5, [[GY,4,1],[BY,4,1],[BZ,4,1],[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,5],[GZ,4,1],[GY,0,4],[GX,0,5],[BZ,0,1],[GZ,0,4],[BX,0,5],[BZ,1,1],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], - [0x01, 7, 6, 6, 6, [[GY,5,1],[GZ,4,1],[GZ,5,1],[RW,0,7],[BZ,0,1],[BZ,1,1],[BY,4,1],[GW,0,7],[BY,5,1],[BZ,2,1],[GY,4,1],[BW,0,7],[BZ,3,1],[BZ,5,1],[BZ,4,1],[RX,0,6],[GY,0,4],[GX,0,6],[GZ,0,4],[BX,0,6],[BY,0,4],[RY,0,6],[RZ,0,6],[PD,0,5]]], - [0x02, 11, 5, 4, 4, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,5],[RW,10,1],[GY,0,4],[GX,0,4],[GW,10,1],[BZ,0,1],[GZ,0,4],[BX,0,4],[BW,10,1],[BZ,1,1],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], - [0x06, 11, 4, 5, 4, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,4],[RW,10,1],[GZ,4,1],[GY,0,4],[GX,0,5],[GW,10,1],[GZ,0,4],[BX,0,4],[BW,10,1],[BZ,1,1],[BY,0,4],[RY,0,4],[BZ,0,1],[BZ,2,1],[RZ,0,4],[GY,4,1],[BZ,3,1],[PD,0,5]]], - [0x0A, 11, 4, 4, 5, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,4],[RW,10,1],[BY,4,1],[GY,0,4],[GX,0,4],[GW,10,1],[BZ,0,1],[GZ,0,4],[BX,0,5],[BW,10,1],[BY,0,4],[RY,0,4],[BZ,1,1],[BZ,2,1],[RZ,0,4],[BZ,4,1],[BZ,3,1],[PD,0,5]]], - [0x0E, 9, 5, 5, 5, [[RW,0,9],[BY,4,1],[GW,0,9],[GY,4,1],[BW,0,9],[BZ,4,1],[RX,0,5],[GZ,4,1],[GY,0,4],[GX,0,5],[BZ,0,1],[GZ,0,4],[BX,0,5],[BZ,1,1],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], - [0x12, 8, 6, 5, 5, [[RW,0,8],[GZ,4,1],[BY,4,1],[GW,0,8],[BZ,2,1],[GY,4,1],[BW,0,8],[BZ,3,1],[BZ,4,1],[RX,0,6],[GY,0,4],[GX,0,5],[BZ,0,1],[GZ,0,4],[BX,0,5],[BZ,1,1],[BY,0,4],[RY,0,6],[RZ,0,6],[PD,0,5]]], - [0x16, 8, 5, 6, 5, [[RW,0,8],[BZ,0,1],[BY,4,1],[GW,0,8],[GY,5,1],[GY,4,1],[BW,0,8],[GZ,5,1],[BZ,4,1],[RX,0,5],[GZ,4,1],[GY,0,4],[GX,0,6],[GZ,0,4],[BX,0,5],[BZ,1,1],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], - [0x1A, 8, 5, 5, 6, [[RW,0,8],[BZ,1,1],[BY,4,1],[GW,0,8],[BY,5,1],[GY,4,1],[BW,0,8],[BZ,5,1],[BZ,4,1],[RX,0,5],[GZ,4,1],[GY,0,4],[GX,0,5],[BZ,0,1],[GZ,0,4],[BX,0,6],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], - [0x1E, 6, 6, 6, 6, [[RW,0,6],[GZ,4,1],[BZ,0,1],[BZ,1,1],[BY,4,1],[GW,0,6],[GY,5,1],[BY,5,1],[BZ,2,1],[GY,4,1],[BW,0,6],[GZ,5,1],[BZ,3,1],[BZ,5,1],[BZ,4,1],[RX,0,6],[GY,0,4],[GX,0,6],[GZ,0,4],[BX,0,6],[BY,0,4],[RY,0,6],[RZ,0,6],[PD,0,5]]], - [0x03, 10, 10, 10, 10, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,10],[GX,0,10],[BX,0,10]]], - [0x07, 11, 9, 9, 9, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,9],[RW,10,1],[GX,0,9],[GW,10,1],[BX,0,9],[BW,10,1]]], - [0x0B, 12, 8, 8, 8, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,8],[RW,11,1],[RW,10,1],[GX,0,8],[GW,11,1],[GW,10,1],[BX,0,8],[BW,11,1],[BW,10,1]]], - [0x0F, 16, 4, 4, 4, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,4],[RW,15,1],[RW,14,1],[RW,13,1],[RW,12,1],[RW,11,1],[RW,10,1],[GX,0,4],[GW,15,1],[GW,14,1],[GW,13,1],[GW,12,1],[GW,11,1],[GW,10,1],[BX,0,4],[BW,15,1],[BW,14,1],[BW,13,1],[BW,12,1],[BW,11,1],[BW,10,1]]] - ]; - const sext = (v, n) => { const m = 1 << (n - 1); v &= (1 << n) - 1; return (v ^ m) - m; }; - function bc6Unq(v, n, sgn) { - if (!sgn) { - if (n >= 15) return v; - if (v === 0) return 0; - if (v === (1 << n) - 1) return 0xFFFF; - return ((v << 16) + 0x8000) >> n; - } - if (n >= 16) return v; - const s = v < 0; if (s) v = -v; - const u = v === 0 ? 0 : v >= (1 << (n - 1)) - 1 ? 0x7FFF : ((v << 15) + 0x4000) >> (n - 1); - return s ? -u : u; - } - function bc6Finish(v, sgn) { - if (!sgn) return (v * 31) >> 6; - v = v < 0 ? -(((-v) * 31) >> 5) : (v * 31) >> 5; - return v < 0 ? 0x8000 | -v : v; - } - function bc6Block(b, o, out, sgn) { - const get = bits128(b, o); - let code = get(2); - if (code > 1) code |= get(3) << 2; - const m = BC6.findIndex((x) => x[0] === code); - if (m < 0) { out.fill(0, 0, 48); return; } - const [, W, dr, dg, db, seg] = BC6[m], f = new Array(13).fill(0); - for (const [fi, lo, n] of seg) f[fi] |= get(n) << lo; - const regions = m < 10 ? 2 : 1, ne = regions * 2, dbits = [dr, dg, db], ep = [[], [], [], []]; - for (let c = 0; c < 3; c++) { - for (let e = 0; e < ne; e++) ep[e][c] = f[c * 4 + e]; - if (sgn) ep[0][c] = sext(ep[0][c], W); - const delta = m !== 9 && m !== 10; - for (let e = 1; e < ne; e++) { - if (delta || sgn) ep[e][c] = sext(ep[e][c], dbits[c]); - if (delta) { ep[e][c] = (ep[e][c] + ep[0][c]) & ((1 << W) - 1); if (sgn) ep[e][c] = sext(ep[e][c], W); } - } - for (let e = 0; e < ne; e++) ep[e][c] = bc6Unq(ep[e][c], W, sgn); - } - const part = f[PD], ib = regions === 2 ? 3 : 4, wt = WEIGHTS[ib]; - for (let i = 0; i < 16; i++) { - const k = regions === 2 ? (PART2[part] >> i) & 1 : 0; - const anchor = i === 0 || (regions === 2 && i === ANCHOR2[part]); - const ix = get(ib - (anchor ? 1 : 0)); - for (let c = 0; c < 3; c++) out[i * 3 + c] = bc6Finish(interp(ep[2 * k][c], ep[2 * k + 1][c], wt[ix]), sgn); - } - } - - /* decodeBlocks(format, blocks, w, h) → RGBA Uint8Array (LDR) or RGB - * half-bits Uint16Array (BC6H); null for ETC2 / EAC / ASTC. */ - function decodeBlocks(format, blocks, w, h) { - if (format >= 0x50 || !BLOCK_FORMATS[format]) return null; - const S = BLOCK_FORMATS[format][2], nbx = Math.ceil(w / 4), nby = Math.ceil(h / 4); - const hdr = format === 0x47 || format === 0x48; - const out = hdr ? new Uint16Array(w * h * 3) : new Uint8Array(w * h * 4); - const px = new Int32Array(64), hx = new Int32Array(48); - let o = 0; - for (let by = 0; by < nby; by++) - for (let bx = 0; bx < nbx; bx++, o += S) { - switch (format) { - case 0x40: bc1Block(blocks, o, px, false); break; - case 0x41: - bc1Block(blocks, o + 8, px, true); - for (let i = 0; i < 16; i++) px[i * 4 + 3] = ((blocks[o + (i >> 1)] >> (4 * (i & 1))) & 15) * 17; - break; - case 0x42: bc1Block(blocks, o + 8, px, true); bc4Block(blocks, o, px, 3, false); break; - case 0x43: case 0x44: - px.fill(0); bc4Block(blocks, o, px, 0, format === 0x44); - for (let i = 0; i < 16; i++) px[i * 4 + 3] = 255; - break; - case 0x45: case 0x46: - px.fill(0); bc4Block(blocks, o, px, 0, format === 0x46); bc4Block(blocks, o + 8, px, 1, format === 0x46); - for (let i = 0; i < 16; i++) px[i * 4 + 3] = 255; - break; - case 0x49: bc7Block(blocks, o, px); break; - default: bc6Block(blocks, o, hx, format === 0x48); - } - for (let y = 0; y < 4 && by * 4 + y < h; y++) - for (let x = 0; x < 4 && bx * 4 + x < w; x++) { - const d = (by * 4 + y) * w + bx * 4 + x, i = y * 4 + x; - if (hdr) for (let c = 0; c < 3; c++) out[d * 3 + c] = hx[i * 3 + c]; - else for (let c = 0; c < 4; c++) out[d * 4 + c] = px[i * 4 + c]; - } - } - return out; - } - - function halfToFloat(h) { - const s = h & 0x8000 ? -1 : 1, e = (h >> 10) & 31, m = h & 0x3FF; - if (e === 0) return s * m * Math.pow(2, -24); - if (e === 31) return m ? NaN : s * Infinity; - return s * (1 + m / 1024) * Math.pow(2, e - 15); - } - - const linToSrgb = (v) => !(v > 0) ? 0 : v >= 1 ? 255 - : Math.round(255 * (v <= 0.0031308 ? v * 12.92 : 1.055 * Math.pow(v, 1 / 2.4) - 0.055)); - - /* - * textureToRGBA(tex, surface = 0, slice = 0) → { width, height, - * rgba: Uint8ClampedArray } for display: gray expanded, 16-bit rounded, - * half / float clamped and sRGB encoded, GPU blocks decoded. Throws for - * ETC2 / EAC / ASTC. - */ - function textureToRGBA(tex, surface, slice) { - surface = surface || 0; slice = slice || 0; - const m = surface % tex.mipLevels, [w, h, d] = mipSize(tex, m), n = w * h; - const data = tex.surfaces[surface], rgba = new Uint8ClampedArray(n * 4); - let src, ch = tex.channels, kind = tex.payload === 1 ? 'u8' : ['u8', 'u16', 'f16', 'f32'][tex.sample]; - if (tex.payload === 1) { - const per = data.length / d, blk = decodeBlocks(tex.format, data.subarray(per * slice, per * (slice + 1)), w, h); - if (!blk) fail('unsupported', 'ETC2 / EAC / ASTC blocks cannot be shown'); - if (blk instanceof Uint16Array) { src = blk; ch = 3; kind = 'f16'; } - else { - rgba.set(blk); - if (tex.channels === 3) for (let i = 0; i < n; i++) rgba[i * 4 + 3] = 255; - return { width: w, height: h, rgba }; - } - } else { - src = data.subarray(n * ch * slice, n * ch * (slice + 1)); - } - for (let i = 0; i < n; i++) { - const v = [0, 0, 0, 255]; - for (let c = 0; c < ch; c++) { - const x = src[i * ch + c], alpha = (ch === 2 && c === 1) || (ch === 4 && c === 3); - let q; - if (kind === 'u8') q = x; - else if (kind === 'u16') q = Math.round(x / 257); - else { - const f = kind === 'f16' ? halfToFloat(x) : x; - q = alpha ? Math.max(0, Math.min(255, Math.round(f * 255))) : linToSrgb(f); - } - v[ch <= 2 ? (c === 0 ? 0 : 3) : c] = q; - } - if (ch <= 2) v[1] = v[2] = v[0]; - rgba.set(v, i * 4); - } - return { width: w, height: h, rgba }; - } - - function surfaceIndex(tex, frame, layer, face, mip) { - return (((frame || 0) * tex.layers + (layer || 0)) * tex.faces + (face || 0)) * tex.mipLevels + (mip || 0); - } + function corrupt() { throw new Error('YAPF: corrupt or unsupported file'); } /* ── Decoder ──────────────────────────────────────────────────────── */ @@ -776,7 +102,7 @@ R[r0 + tw] = R[r0 + tw - 1]; } - function decodeTile(b, p, end, out, imgW, tx, ty, tw, th, ch, v2, book) { + function decodeTile(b, p, end, out, imgW, tx, ty, tw, th, ch) { if (end <= p) corrupt(); let hdr = b[p++]; if (hdr === STORED) { @@ -786,18 +112,10 @@ out.set(b.subarray(p + rowb * y, p + rowb * (y + 1)), ((ty + y) * imgW + tx) * ch); return; } - if (hdr & (v2 ? 0x80 : 0xE0)) corrupt(); /* reserved bits set */ - if ((hdr & 0x40) && (hdr & SUBGREEN)) corrupt(); - /* xf: 0 YCoCg-R, 1 subtract-green, 2 none (v2) */ - const xf = (hdr & 0x40) ? 2 : (hdr & SUBGREEN) ? 1 : 0, ent = hdr & 0x20; + if (hdr & 0xE0) corrupt(); /* reserved bits set */ + const xf = hdr & SUBGREEN; hdr &= 0x0F; if ((hdr >> ch) || (xf && ch < 3)) corrupt(); - let emask = 0; - if (ent) { /* v2: entropy planes */ - if (p >= end || !book) corrupt(); - emask = b[p++]; - if ((emask >> ch) || (emask & hdr)) corrupt(); - } const rowLen = TILE + 1, planeLen = RING * rowLen; const rows = new Uint8Array(4 * planeLen); @@ -810,8 +128,7 @@ for (let c = 0; c < ch; c++) { if (hdr & (1 << c)) continue; z[c] = new Uint8Array(TILE_PX + GROUP); - const r = (emask & (1 << c)) ? huffPlane(b, p, end, z[c], tw, th, book, xf * 16 + c) - : unpackPlane(b, p, end, z[c], tw, th); + const r = unpackPlane(b, p, end, z[c], tw, th); p = r.p; filt[c] = r.filt; } if (p !== end) corrupt(); @@ -830,11 +147,7 @@ const P0 = ri, P1 = planeLen + ri, P2 = 2 * planeLen + ri, P3 = 3 * planeLen + ri; for (let x = 0; x < tw; x++, d += ch) { if (ch >= 3) { - if (xf === 2) { - out[d] = rows[P0 + x]; - out[d + 1] = rows[P1 + x]; - out[d + 2] = rows[P2 + x]; - } else if (xf) { + if (xf) { const g = rows[P0 + x]; out[d] = rows[P1 + x] + g - 128; out[d + 1] = g; @@ -858,27 +171,11 @@ const r32 = (b, p) => (b[p] | (b[p + 1] << 8) | (b[p + 2] << 16) | (b[p + 3] << 24)) >>> 0; - /* decode(bytes) → 8-bit image. Reads version 1, and version 2 files - * that hold 8-bit pixels (2D; cube face 0 / layer 0 / frame 0). Other - * textures: see decodeTexture. */ function decode(input) { const b = input instanceof Uint8Array ? input : new Uint8Array(input); - if (b.length >= 5 && b[0] === 0x59 && b[1] === 0x41 && b[2] === 0x50 && b[3] === 0x46 && b[4] === 2) { - const t = decodeTexture(b); - if (t.payload !== 0 || t.sample !== 0 || t.shape === 2) - throw new YapfError('unsupported', 'this is a YAPF 2 texture (GPU blocks, 16-bit / float or a ' + - 'volume); use YAPF.decodeTexture and YAPF.textureToRGBA'); - const mips = t.surfaces.slice(0, t.mipLevels); - return { width: t.width, height: t.height, channels: t.channels, gpuFormat: t.format, - flags: t.flags, mipLevels: t.mipLevels, pixels: mips[0], mips }; - } - return decodeV1(b); - } - - function decodeV1(b) { if (b.length < 20 || b[0] !== 0x59 || b[1] !== 0x41 || b[2] !== 0x50 || b[3] !== 0x46) - throw new YapfError('not-yapf', 'not a YAPF file'); - if (b[4] !== 1) throw new YapfError('version', 'unsupported format version ' + b[4]); + throw new Error('YAPF: not a YAPF file'); + if (b[4] !== 1) throw new Error('YAPF: unsupported format version ' + b[4]); const ch = b[5], mipLevels = b[7], W = r32(b, 12), H = r32(b, 16); if (ch < 1 || ch > 4 || mipLevels < 1 || mipLevels > MAX_MIPS || W < 1 || W > MAX_DIM || H < 1 || H > MAX_DIM) corrupt(); @@ -1162,7 +459,5 @@ return out; } - return { decode, encode, toRGBA, decodeTexture, textureToRGBA, decodeBlocks, surfaceIndex, - checksum: (b) => checksum(b, 0, b.length), YapfError, GPU, FLAG, NEED, - VERSION: 1 /* written by encode */, READS: [1, 2] }; + return { decode, encode, toRGBA, GPU, FLAG, VERSION: 1 }; })); diff --git a/bindings/python/README.md b/bindings/python/README.md index 1a47c7a..33e52c8 100644 --- a/bindings/python/README.md +++ b/bindings/python/README.md @@ -23,17 +23,3 @@ Without the library it falls back to a pure-Python decoder that needs tells you which one is in use. Example: `YAPF_LIBRARY=../../build/libyapf.so python3 example.py ../../other/YAPF.YAPF` - -## Textures (YAPF 2) - -```python -tex = yapf.load_texture("albedo.yapf") # any v1 / v2 file, needs the C library -tex.payload, tex.format, tex.sample, tex.channels, tex.mip_levels -tex.shape, tex.layers, tex.frames, tex.channel_map, tex.normal, tex.notes -raw = tex.surface(frame=0, layer=0, face=0, mip=0) # bytes as stored (GPU blocks / samples) -rgba = tex.rgba_float() # numpy float32 (h, w, 4) for display -``` - -`yapf.load` / `yapf.decode` still return 8-bit images; on a texture they -cannot represent they raise `YapfError` saying so. Errors name the -reason (damaged, truncated, not YAPF, newer version). diff --git a/bindings/python/gradient.yapf b/bindings/python/gradient.yapf deleted file mode 100644 index d6db15b..0000000 Binary files a/bindings/python/gradient.yapf and /dev/null differ diff --git a/bindings/python/yapf.py b/bindings/python/yapf.py index 2f926be..9f8f3a1 100644 --- a/bindings/python/yapf.py +++ b/bindings/python/yapf.py @@ -1,5 +1,5 @@ """ -yapf.py — Python bindings for YAPF images (v1) and textures (v2). +yapf.py — Python bindings for YAPF v1 images. Copyright 2026 Nexoniarz — Apache License 2.0. @@ -8,11 +8,6 @@ Copyright 2026 Nexoniarz — Apache License 2.0. img.width, img.height, img.channels # pixels: bytes, row-major yapf.save("out.yapf", img) # or yapf.encode(img) -> bytes - tex = yapf.load_texture("albedo.yapf") # any v1 / v2 file (C library) - tex.payload, tex.format, tex.sample # GPU blocks, 16-bit, float, ... - raw = tex.surface(mip=0) # bytes as stored (blocks or samples) - rgba = tex.rgba_float() # numpy float32 (h, w, 4) for display - Uses the C library through ctypes when one can be found (fast, can encode); otherwise falls back to a pure-Python decoder that needs numpy (decode only). The library is looked up in this order: @@ -31,22 +26,12 @@ import platform import struct import sys -__all__ = ["Image", "Texture", "load", "save", "decode", "encode", "backend", - "load_texture", "decode_texture", +__all__ = ["Image", "load", "save", "decode", "encode", "backend", "GPU_RGBA8", "GPU_RGB8", "GPU_RG8", "GPU_R8", "GPU_SRGB8_A8", "GPU_SRGB8", "FLAG_PREMULT_ALPHA", "FLAG_SRGB", "YapfError"] GPU_RGBA8, GPU_RGB8, GPU_RG8, GPU_R8, GPU_SRGB8_A8, GPU_SRGB8 = range(6) -GPU_BC1, GPU_BC2, GPU_BC3, GPU_BC4, GPU_BC4_SNORM, GPU_BC5, GPU_BC5_SNORM = range(0x40, 0x47) -GPU_BC6H, GPU_BC6H_SF, GPU_BC7 = 0x47, 0x48, 0x49 -FLAG_PREMULT_ALPHA, FLAG_SRGB, FLAG_COVERAGE = 1, 2, 4 -PAYLOAD_PIXELS, PAYLOAD_BLOCKS = 0, 1 -SAMPLE_U8, SAMPLE_U16, SAMPLE_F16, SAMPLE_F32 = 0, 1, 2, 3 -SHAPE_2D, SHAPE_CUBE, SHAPE_VOLUME = 0, 1, 2 - -_ERRORS = {-1: "invalid argument", -2: "cannot read the file", -3: "out of memory", - -4: "corrupt or not a YAPF file", -5: "needs a feature this library does not have", - -6: "damaged: a checksum does not match", -7: "the file is truncated"} +FLAG_PREMULT_ALPHA, FLAG_SRGB = 1, 2 class YapfError(Exception): @@ -85,26 +70,6 @@ class _CImage(ctypes.Structure): ("mips", ctypes.POINTER(ctypes.POINTER(ctypes.c_uint8)))] -class _CNote(ctypes.Structure): - _fields_ = [("tag", ctypes.c_char * 4), ("size", ctypes.c_uint32), - ("data", ctypes.POINTER(ctypes.c_uint8))] - - -class _CTexture(ctypes.Structure): - _fields_ = [("width", ctypes.c_uint32), ("height", ctypes.c_uint32), ("depth", ctypes.c_uint32), - ("layers", ctypes.c_uint16), ("frames", ctypes.c_uint16), - ("shape", ctypes.c_uint8), ("mip_levels", ctypes.c_uint8), - ("payload", ctypes.c_uint8), ("format", ctypes.c_uint8), - ("sample", ctypes.c_uint8), ("channels", ctypes.c_uint8), ("flags", ctypes.c_uint8), - ("channel_map", ctypes.c_uint8 * 4), ("linear_mask", ctypes.c_uint8), - ("normal", ctypes.c_uint8), ("primaries", ctypes.c_uint8), ("transfer", ctypes.c_uint8), - ("mip_filter", ctypes.c_uint8), ("alpha_ref", ctypes.c_uint8), - ("loop_count", ctypes.c_uint16), - ("frame_us", ctypes.POINTER(ctypes.c_uint32)), - ("note_count", ctypes.c_uint32), ("notes", ctypes.POINTER(_CNote)), - ("data", ctypes.POINTER(ctypes.c_void_p)), ("internal_", ctypes.c_void_p)] - - def _platform_dir(): machine = platform.machine().lower() arch = "arm64" if machine in ("arm64", "aarch64") else "x64" @@ -132,20 +97,6 @@ def _find_library(): lib.yapf_encode.argtypes = [ctypes.POINTER(_CImage), ctypes.POINTER(ctypes.c_void_p), ctypes.POINTER(ctypes.c_size_t)] lib.yapf_free_buffer.argtypes = [ctypes.c_void_p] - try: # version 2 (texture) API - T = ctypes.POINTER(_CTexture) - lib.yapf_tex_load_memory.restype = T - lib.yapf_tex_load_memory.argtypes = [ctypes.c_char_p, ctypes.c_size_t, ctypes.c_int, - ctypes.c_int, ctypes.POINTER(ctypes.c_int)] - lib.yapf_tex_free.argtypes = [T] - lib.yapf_tex_surface_bytes.restype = ctypes.c_size_t - lib.yapf_tex_surface_bytes.argtypes = [T, ctypes.c_uint32] - lib.yapf_tex_to_rgba8.argtypes = [T, ctypes.c_uint32, ctypes.c_uint32, ctypes.c_void_p] - lib.yapf_blocks_decode.argtypes = [ctypes.c_uint8, ctypes.c_void_p, ctypes.c_uint32, - ctypes.c_uint32, ctypes.c_void_p] - lib.has_textures = True - except AttributeError: # an older library: v1 only - lib.has_textures = False return lib except (OSError, AttributeError): continue @@ -164,7 +115,7 @@ def _c_decode(data): data = bytes(data) p = _lib.yapf_load_memory_mt(data, len(data), 0) if not p: - raise YapfError(_why(data)) + raise YapfError("not a valid YAPF v1 file") try: c = p.contents mips = [] @@ -206,9 +157,6 @@ def _py_decode(data): b = np.frombuffer(bytes(data), dtype=np.uint8) if len(b) < 20 or bytes(b[:4]) != b"YAPF": raise YapfError("not a YAPF file") - if b[4] == 2: - raise YapfError("this is a YAPF 2 file; reading it needs the YAPF native library, " - "which was not found (the pure-Python decoder reads version 1 only)") if b[4] != 1: raise YapfError("unsupported YAPF version %d" % b[4]) ch, gpu, mip_levels, flags = int(b[5]), int(b[6]), int(b[7]), int(b[8]) @@ -327,127 +275,6 @@ def _py_tile(np, b, p, end, tw, th, ch, bad): return np.stack(out, axis=-1).astype(np.uint8) -# ── textures (version 2, C library) ────────────────────────────────── - -def _why(data): - """Why `decode` could not read data, in words.""" - if len(data) < 5 or data[:4] != b"YAPF": - return "not a YAPF file" - if data[4] not in (1, 2): - return "YAPF format version %d is newer than this library" % data[4] - if _lib and getattr(_lib, "has_textures", False): - err = ctypes.c_int(0) - p = _lib.yapf_tex_load_memory(data, len(data), 0, 1, ctypes.byref(err)) - if p: - _lib.yapf_tex_free(p) - return ("this is a YAPF 2 texture (GPU blocks, 16-bit / float or a volume) " - "that yapf.decode cannot return as 8-bit pixels; use yapf.load_texture") - return _ERRORS.get(err.value, "error %d" % err.value) - if data[4] == 2: - return "YAPF 2 files need the YAPF native library (not found)" - return "corrupt YAPF file" - - -class Texture: - """A decoded YAPF texture (version 1 or 2). Description fields mirror - yapf_texture_t; `surfaces[s]` holds the bytes of surface s as stored - (GPU blocks, or samples in native byte order), numbered - s = ((frame * layers + layer) * faces + face) * mip_levels + mip.""" - - def __init__(self, c, lib): - for f, _ in _CTexture._fields_: - if f not in ("frame_us", "notes", "data", "internal_", "channel_map", "note_count"): - setattr(self, f, getattr(c, f)) - self.channel_map = list(c.channel_map) - self.faces = 6 if c.shape == SHAPE_CUBE else 1 - self.frame_us = [c.frame_us[i] for i in range(c.frames)] if c.frame_us else [] - self.notes = [(c.notes[i].tag.decode("latin-1"), ctypes.string_at(c.notes[i].data, c.notes[i].size)) - for i in range(c.note_count)] - count = c.frames * c.layers * self.faces * c.mip_levels - self.surfaces = [ctypes.string_at(c.data[s], lib.yapf_tex_surface_bytes(ctypes.byref(c), s % c.mip_levels)) - for s in range(count)] - - def surface_index(self, frame=0, layer=0, face=0, mip=0): - return ((frame * self.layers + layer) * self.faces + face) * self.mip_levels + mip - - def surface(self, frame=0, layer=0, face=0, mip=0): - return self.surfaces[self.surface_index(frame, layer, face, mip)] - - def mip_size(self, mip=0): - d = max(1, self.depth >> mip) if self.shape == SHAPE_VOLUME else 1 - return max(1, self.width >> mip), max(1, self.height >> mip), d - - @property - def is_hdr(self): - """Float / half samples or BC6H blocks: values may go above 1.""" - if self.payload == PAYLOAD_BLOCKS: - return self.format in (GPU_BC6H, GPU_BC6H_SF) - return self.sample in (SAMPLE_F16, SAMPLE_F32) - - def rgba_float(self, frame=0, layer=0, face=0, mip=0, slice=0): - """One image as numpy float32 (height, width, 4), rows top to bottom: - 8 / 16-bit scaled to 0..1 (still in their stored colour space), - floats as stored, GPU blocks decoded. Needs numpy.""" - import numpy as np - w, h, d = self.mip_size(mip) - raw = self.surface(frame, layer, face, mip) - raw = raw[len(raw) // d * slice:len(raw) // d * (slice + 1)] - out = np.ones((h, w, 4), dtype=np.float32) - if self.payload == PAYLOAD_BLOCKS: - hdr = self.format in (GPU_BC6H, GPU_BC6H_SF) - buf = ctypes.create_string_buffer(w * h * (6 if hdr else 4)) - rc = _lib.yapf_blocks_decode(self.format, raw, w, h, buf) - if rc != 0: - raise YapfError("GPU block format 0x%02x cannot be decoded here" % self.format) - if hdr: - out[..., :3] = np.frombuffer(buf.raw, dtype=np.float16).reshape(h, w, 3) - else: - out[...] = np.frombuffer(buf.raw, dtype=np.uint8).reshape(h, w, 4) / 255.0 - if self.channels == 3: - out[..., 3] = 1.0 - return out - dt = {SAMPLE_U8: np.uint8, SAMPLE_U16: np.uint16, SAMPLE_F16: np.float16, SAMPLE_F32: np.float32}[self.sample] - ch = self.channels - src = np.frombuffer(raw, dtype=dt).reshape(h, w, ch).astype(np.float32) - if self.sample == SAMPLE_U8: - src /= 255.0 - elif self.sample == SAMPLE_U16: - src /= 65535.0 - out[..., :3] = src[..., :3] if ch >= 3 else src[..., :1] - if ch in (2, 4): - out[..., 3] = src[..., ch - 1] - return out - - def __repr__(self): - w, h, d = self.mip_size(0) - return "" % ( - w, h, "x%d" % d if d > 1 else "", self.mip_levels, - "blocks 0x%02x" % self.format if self.payload == PAYLOAD_BLOCKS - else "%d ch %s" % (self.channels, ["u8", "u16", "f16", "f32"][self.sample])) - - -def decode_texture(data, verify=True): - """Decode any YAPF file (v1 or v2) into a Texture. Needs the C library.""" - if not (_lib and getattr(_lib, "has_textures", False)): - raise YapfError("YAPF textures need the YAPF native library (version 2), which was not found") - data = bytes(data) - err = ctypes.c_int(0) - p = _lib.yapf_tex_load_memory(data, len(data), 0 if verify else 1, 0, ctypes.byref(err)) - if not p: - raise YapfError(_why(data) if len(data) < 5 or data[:4] != b"YAPF" - else _ERRORS.get(err.value, "error %d" % err.value)) - try: - return Texture(p.contents, _lib) - finally: - _lib.yapf_tex_free(p) - - -def load_texture(path, verify=True): - """Read and decode any .yapf file into a Texture.""" - with open(path, "rb") as f: - return decode_texture(f.read(), verify) - - # ── public API ──────────────────────────────────────────────────────── def decode(data): diff --git a/default.nix b/default.nix deleted file mode 100644 index fd79d78..0000000 --- a/default.nix +++ /dev/null @@ -1,39 +0,0 @@ -# YAPF for Nix / NixOS: the `yapf` tool, libyapf (static + shared) and -# yapf.h, plus the image/x-yapf MIME type and a freedesktop thumbnailer. -# -# nix-build build ./result -# nix-env -if . install for your user -# NixOS: environment.systemPackages = [ (pkgs.callPackage /path/to/YAPF { }) ]; -{ pkgs ? import { }, lib ? pkgs.lib, stdenv ? pkgs.stdenv }: - -stdenv.mkDerivation { - pname = "yapf"; - version = "2.0"; - - src = lib.cleanSourceWith { - src = ./.; - filter = path: type: - let base = baseNameOf path; in - !(lib.elem base [ "build" "dist" "result" ".git" "node_modules" ]); - }; - - makeFlags = [ "PREFIX=$(out)" ]; - doCheck = true; - checkTarget = "test"; - - postInstall = '' - install -Dm644 extensions/linux/yapf-mime.xml $out/share/mime/packages/yapf.xml - install -Dm644 extensions/linux/yapf.thumbnailer $out/share/thumbnailers/yapf.thumbnailer - substituteInPlace $out/share/thumbnailers/yapf.thumbnailer \ - --replace-fail "TryExec=yapf" "TryExec=$out/bin/yapf" \ - --replace-fail "Exec=yapf " "Exec=$out/bin/yapf " - ''; - - meta = { - description = "YAPF: fast lossless images and GPU-ready textures"; - homepage = "https://github.com/Nexoniarz/YAPF"; - license = lib.licenses.asl20; - platforms = lib.platforms.unix; - mainProgram = "yapf"; - }; -} diff --git a/docs/FORMAT.md b/docs/FORMAT.md deleted file mode 100644 index 1099f1e..0000000 --- a/docs/FORMAT.md +++ /dev/null @@ -1,450 +0,0 @@ -# YAPF file format - -This is the complete specification of YAPF files. `yapf.c` is the -reference implementation; where this text and the code disagree, that is a -bug in one of them. - -YAPF has two versions: - -| Version | What it holds | Who writes it | -| :------ | :------------ | :------------ | -| **1** | one 8-bit image (1–4 channels) with optional mips | `yapf_save` / `yapf_encode`, and the CLI for plain 8-bit images | -| **2** | a texture: GPU blocks or lossless pixels in 8/16-bit or float, cubes, arrays, volumes, animation, channel meanings, colour description, notes, checksums, optional entropy coding | `yapf_tex_save` / `yapf_tex_encode`, and the CLI whenever a v2 feature is used | - -Version 1 is specified at the top of [`yapf.h`](../yapf.h) and is not -repeated here, except that version 2 reuses its tiles, filters and plane -payloads (the "group form") byte for byte. - ---- - -## Design: how YAPF 2 does things its own way - -YAPF 2 is not a wrapper around someone else's container. A few ideas -carry the whole format: - -1. **Everything is tiles.** Pixels are cut into 64 × 64 tiles, GPU - blocks into tiles of 16 × 16 blocks. Each tile has its own size and - checksum in a table, so any tile can be found, checked and decoded on its - own: for multithreaded loading, for partial updates, and for virtual - texturing / "megatexture" streaming. -2. **Small mips first.** Surface data is stored from the smallest mip - to the largest. The first *N* bytes of a file are a complete, lower - resolution texture, so an engine can show something after reading a - few kilobytes and stream in the large mips later. The header carries - the file size, so a loader knows how much is still to come. -3. **Needs, not versions.** New capabilities are announced by *needs* - bits. A decoder refuses a file with a needs bit it does not know, and - can ignore everything else safely. No more version bumps that break - every reader. -4. **Wide samples are byte planes.** 16-bit and float samples are split - into byte planes (high bytes, low bytes) and coded with the same fast - 8-bit machinery as version 1: smooth high bytes compress very well, and - the decoder stays branch-light and SIMD friendly. -5. **Shared code books.** The optional entropy stage keeps its Huffman - tables once per surface, keyed by a small neighbourhood context, so - tiles stay independent and their own headers stay tiny. For GPU - blocks the tables are per byte lane, so endpoint bytes and index bytes - are coded separately. -6. **Meaning travels with the data.** Channel meanings, the normal-map - convention, linear channels inside sRGB images, colour primaries and - transfer, and how the mips were made are header fields, not file-name - suffixes. - ---- - -## Conventions - -All integers are little-endian. `u8`, `u16`, `u32` and `u64` are unsigned -integers of that many bits. Offsets are from the start of the file. - -### yapf-sum - -The checksum of a byte string of length *n*. `K = 0x9E3779B97F4A7C15`; -all arithmetic is modulo 2⁶⁴. - -``` -lane[i] = (i + 1)·K XOR n for i = 0, 1, 2, 3 -for each 8-byte word w_j of the input (little-endian; the last word is -padded with zero bytes): - a = lane[j mod 4] - a = (a XOR w_j) · K - a = a XOR (a >> 32) - lane[j mod 4] = a -h = lane[0] XOR rotl(lane[1], 16) XOR rotl(lane[2], 32) XOR rotl(lane[3], 48) -h = h · K -h = h XOR (h >> 32) -result = low 32 bits of h -``` - -Test vectors: `""` → `0xd7550ed0`, `"YAPF"` → `0xc8cc41d4`, -`"The quick brown fox jumps over the lazy dog"` → `0xbce4f410`. - ---- - -## Version 2 layout - -``` -HEADER 64 bytes -SURFACE TABLE 16 bytes per surface -FRAME TABLE 4 bytes per frame, only when frames > 1 -NOTES notes size bytes -SURFACE DATA in stream order: smallest mip first -``` - -The *front matter* is everything before the surface data: -`64 + 16·surfaces + (frames > 1 ? 4·frames : 0) + notes size` bytes. - -### Header - -| Offset | Size | Field | Notes | -| -----: | ---: | :---- | :---- | -| 0x00 | 4 | magic | `"YAPF"` | -| 0x04 | 1 | version | 2 | -| 0x05 | 1 | channels | 1–4 | -| 0x06 | 1 | format | block format (block payload) or GPU hint (pixel payload); see [GPU formats](#gpu-formats) | -| 0x07 | 1 | mip levels | 1–16 | -| 0x08 | 1 | flags | bit 0 premultiplied alpha, bit 1 sRGB, bit 2 coverage-preserving mips; other bits 0 | -| 0x09 | 1 | payload | 0 pixels (lossless tiles), 1 GPU blocks | -| 0x0A | 1 | sample | pixel payload: 0 u8, 1 u16, 2 half float, 3 float. Block payload: type of the decoded texels (0, or 2 for BC6H) | -| 0x0B | 1 | shape | 0 2D, 1 cube, 2 volume | -| 0x0C | 4 | width | 1–65535 | -| 0x10 | 4 | height | 1–65535; cubes: width = height | -| 0x14 | 4 | depth | volumes: 1–65535; otherwise 1 | -| 0x18 | 2 | layers | ≥ 1 (array layers; with a cube, a cube array); volumes: 1 | -| 0x1A | 2 | frames | ≥ 1 (animation frames) | -| 0x1C | 4 | needs | see [Needs](#needs) | -| 0x20 | 4 | channel map | meaning of channels 0–3, see [Channel meanings](#channel-meanings) | -| 0x24 | 1 | linear mask | bit *c*: channel *c* is linear even when the sRGB flag is set | -| 0x25 | 1 | normal | bits 0–1 convention (0 none, 1 OpenGL +Y, 2 DirectX −Y); bits 2–3 encoding (0 XYZ, 1 XY with Z reconstructed, 2 octahedral) | -| 0x26 | 1 | primaries | 0 unspecified, 1 BT.709/sRGB, 2 Display P3, 3 BT.2020, 4 ACES AP0, 5 ACEScg (AP1), 6 Adobe RGB | -| 0x27 | 1 | transfer | 0 unspecified (the sRGB flag decides), 1 sRGB, 2 linear, 3 gamma 2.2, 4 PQ (ST 2084), 5 HLG, 6 BT.1886 | -| 0x28 | 1 | mip filter | bits 0–6: 0 unknown, 1 box, 2 tent, 3 Kaiser, 4 Lanczos, 5 Mitchell, 6 authored; bit 7: sRGB channels were filtered in linear light | -| 0x29 | 1 | alpha ref | with the coverage flag: alpha-test reference 0–255 | -| 0x2A | 2 | loop count | animation repeats, 0 = forever | -| 0x2C | 4 | notes size | bytes of the notes area | -| 0x30 | 8 | file size | total bytes of the file | -| 0x38 | 4 | tables check | yapf-sum of bytes [0x40, front matter end) | -| 0x3C | 4 | header check | yapf-sum of bytes [0x00, 0x3C) | - -The header check covers the tables check, so the whole front matter is -protected. - -### Surfaces - -A *surface* is one mip level of one face of one layer of one frame. With -`faces = 6` for cubes (order +X, −X, +Y, −Y, +Z, −Z) and 1 otherwise, -surface number - -``` -s = ((frame · layers + layer) · faces + face) · mip_levels + mip -``` - -Mip *m* is `max(1, width >> m) × max(1, height >> m)`, and for volumes -`max(1, depth >> m)` slices deep. A volume surface holds all its slices. - -**Surface table entry** (16 bytes, one per surface, in surface order): - -| Offset | Size | Field | -| -----: | ---: | :---- | -| +0 | 8 | offset of the surface data | -| +8 | 4 | size of the surface data | -| +12 | 4 | tile-table check: yapf-sum of the surface's book size, book and tile table | - -**Stream order.** Writers store surface data for mip `mip_levels − 1` -first (all frames, layers and faces in surface order), then the next larger -mip, and so on, ending with mip 0. Readers must use the offsets and must not -rely on the order. The number of leading bytes that hold every surface of -mip *m* and smaller is the largest `offset + size` over those surfaces. - -### Frame table - -When `frames > 1`: one `u32` per frame, its display time in microseconds. - -### Notes - -A sequence of entries that exactly fills the notes area: - -``` -tag 4 bytes -length u32 -data length bytes -``` - -| Tag | Content | -| :-- | :------ | -| `AUTH` | author (UTF-8) | -| `LICN` | licence (UTF-8, e.g. an SPDX id) | -| `SRCE` | source / origin (UTF-8, e.g. a URL or path) | -| `TOOL` | the tool that wrote the file (UTF-8) | -| `TITL` | title (UTF-8) | -| `DESC` | description (UTF-8) | -| `DATE` | date, ISO 8601 (UTF-8) | -| `ICCP` | an ICC colour profile | -| `ORNT` | orientation, 1 byte: bits 0–1 clockwise quarter turns, bit 2 mirror left–right after turning | -| `DPI ` | 2 × u32: horizontal and vertical dots per inch × 1000 | -| `CLL ` | 2 × u16: maximum content light level, maximum frame-average light level (cd/m²) | - -Tags that begin with a lowercase letter belong to applications (for -example `keel`). Readers ignore tags they do not know. Tags may repeat; -order is kept. - -### Needs - -| Bit | Name | Set when | -| --: | :--- | :------- | -| 0 | WIDE | pixel payload with sample ≠ u8 | -| 1 | BLOCKS | block payload | -| 2 | ENTROPY | any surface has a code book | -| 3 | LAYOUT | shape ≠ 2D, or layers > 1 | -| 4 | FRAMES | frames > 1 | - -A decoder must refuse a file that sets a needs bit it does not implement -(this library: any bit above 4). A file that uses a feature without its -bit is corrupt. Everything that is not a needs feature — channel map, -colour fields, mip filter, notes, frame times — may be ignored. - -A decoder that only implements 8-bit 2D pixels can therefore read every -version 2 file whose needs are 0, and cleanly refuse the rest. - ---- - -## Surface data - -``` -book size u32 0 when the surface has no code book -book book size bytes -tile table 8 bytes per tile: u32 size (bit 31: packed), u32 yapf-sum of the tile bytes -tiles back to back; their sizes must add up exactly to the rest of the surface -``` - -Tiles are numbered slice by slice, rows top to bottom, tiles left to right. -For pixels the grid is `ceil(w / 64) × ceil(h / 64)` tiles per slice (edge -tiles are smaller, no padding); for blocks it is -`ceil(bx / 16) × ceil(by / 16)` tiles of 16 × 16 blocks, where -`bx = ceil(w / block width)` and `by = ceil(h / block height)`. - -Bit 31 of a tile size (*packed*) is allowed only in block tiles of a -surface with a code book. - ---- - -## Pixel tiles - -Let *B* be the bytes per sample (1, 2 or 4), *ch* the channels, and the -tile `tw × th` pixels. - -### Tile header - -``` -byte 0 0x80 stored: the raw samples follow, interleaved in - channel order, each B bytes little-endian; - exactly tw·th·ch·B bytes - otherwise coded: - bits 0–3 B = 1: bit c set = plane c is constant; B > 1: 0 - bit 4 colour transform subtract-green - bit 5 an entropy mask follows - bit 6 no colour transform - bit 7 0 - bits 4 and 6 are exclusive and need ch >= 3; with neither - set, an RGB(A) tile uses YCoCg-R -B > 1 u16 constant mask, bit p = plane p is constant -bit 5 entropy mask: u8 (B = 1) or u16 (B > 1); bit p = plane p is - entropy coded. It must not include constant planes. -then one value byte per constant plane, in plane order; - one payload per other plane, in plane order; - the tile ends exactly where its last payload ends -``` - -With B = 1 and neither bit 5 nor bit 6 set, a coded tile is exactly a -version 1 tile. - -### Planes - -The colour transform turns R, G, B (+ A) into transformed channels; it is -skipped for 1–2 channels and with bit 6. - -* **YCoCg-R** at the sample width *n* = 8·B bits, all arithmetic mod 2ⁿ, - `>>` an arithmetic shift of the *n*-bit two's complement value, `h = 2ⁿ⁻¹`: - - ``` - forward: o = R − B; t = B + (o >> 1); q = G − t; - Y = t + (q >> 1); Co = o + h; Cg = q + h - inverse: o = Co − h; q = Cg − h; t = Y − (q >> 1); - G = q + t; B = t − (o >> 1); R = B + o - ``` -* **subtract-green**: `G, R − G + h, B − G + h` (+ A), mod 2ⁿ. - -Before the transform, float samples (half or float) are mapped to -integers that keep their order: if the sign bit is set every bit is -flipped, otherwise the sign bit is set. The decoder undoes this after the -inverse transform. - -Transformed channel *c* gives B planes: plane `c·B + k` holds byte -`B − 1 − k` of the value (the most significant byte first). For B = 1 -the planes are the transformed channels. - -### Plane payloads - -A plane payload has one of two forms, chosen per plane by the entropy mask. - -**Group form** — exactly the version 1 plane payload: a filter byte per -band of 8 rows, 4-bit control codes, and groups of 8 residuals at the -group's bit width. See `yapf.h`. - -**Entropy form** - -``` -filter[ceil(th / 8)] as in the group form -u16 L bytes of the bit stream -L bytes the folded residuals of the plane in raster order -``` - -Each residual is coded with table `(slot, context)` of the surface's code -book: - -* `slot = t · 16 + p`, where *p* is the plane and *t* is 1 with - subtract-green, 2 with no transform, else 0; -* `context = min(6, floor(log2(s + 1)))`, `s = left + above`, the folded - residuals of the left and upper neighbours in this plane (0 outside the - tile) — so 0, 1–2, 3–6, 7–14, 15–30, 31–62, 63 and up. - -The stream must end in its last byte. Folded residuals then go through -the filters exactly as in the group form. - ---- - -## Block tiles - -A block tile of `tbw × tbh` blocks of *S* bytes is either - -* **raw**: the blocks, rows top to bottom, exactly `tbw·tbh·S` bytes — as - the GPU takes them; or -* **packed** (bit 31 of its size): one bit stream holding byte 0 of every - block (in raster order), then byte 1 of every block, …, byte S − 1. - Byte lane *k* is coded with table `(slot k, context 0)` of the surface's - code book. The stream must end in the last byte of the tile. - -Decoded block surfaces are row-major blocks, slice after slice, ready for -`glCompressedTexImage*` / `vkCmdCopyBufferToImage`. - ---- - -## Code book - -``` -u16 count number of tables, 1–336 -count × { - u8 slot 0–47 - u8 context 0–6 - code table -} -``` - -Each `(slot, context)` pair appears at most once. The book must fill its -size exactly. - -**Code table**: `u8 n − 1`, then *n* 4-bit code lengths (symbols -0 … n − 1, low nibble first, an odd count padded with a zero nibble). -Length 0 = no code; otherwise 1–11. At least one code; Kraft sum ≤ 1. -Codes are canonical — by length, then by symbol, the first code of each -length one more than the last code of the previous length, shifted left — -and stored least significant bit first (the first bit of a code is the -lowest unread bit of the stream), so a decoder can look up 11 bits at a -time. - ---- - -## GPU formats - -Pixel payload hints (`format` field): - -| Value | Name | Vulkan | -| ----: | :--- | :----- | -| 0x00 | RGBA8 | R8G8B8A8_UNORM | -| 0x01 | RGB8 | R8G8B8_UNORM | -| 0x02 | RG8 | R8G8_UNORM | -| 0x03 | R8 | R8_UNORM | -| 0x04 | SRGB8_A8 | R8G8B8A8_SRGB | -| 0x05 | SRGB8 | R8G8B8_SRGB | -| 0x06–0x09 | RGBA16, RGB16, RG16, R16 | R16G16B16A16_UNORM … R16_UNORM | -| 0x0A–0x0D | RGBA16F, RGB16F, RG16F, R16F | R16G16B16A16_SFLOAT … R16_SFLOAT | -| 0x0E–0x11 | RGBA32F, RGB32F, RG32F, R32F | R32G32B32A32_SFLOAT … R32_SFLOAT | - -Block formats (block payload). With the sRGB flag, use the `_SRGB` -variant where one exists. - -| Value | Name | Block | Bytes | Vulkan / DXGI | -| ----: | :--- | :---: | ----: | :------------ | -| 0x40 | BC1 | 4×4 | 8 | BC1_RGB(A)_UNORM / BC1_UNORM | -| 0x41 | BC2 | 4×4 | 16 | BC2_UNORM | -| 0x42 | BC3 | 4×4 | 16 | BC3_UNORM | -| 0x43 | BC4 | 4×4 | 8 | BC4_UNORM | -| 0x44 | BC4_SNORM | 4×4 | 8 | BC4_SNORM | -| 0x45 | BC5 | 4×4 | 16 | BC5_UNORM | -| 0x46 | BC5_SNORM | 4×4 | 16 | BC5_SNORM | -| 0x47 | BC6H | 4×4 | 16 | BC6H_UFLOAT / BC6H_UF16 | -| 0x48 | BC6H_SF | 4×4 | 16 | BC6H_SFLOAT / BC6H_SF16 | -| 0x49 | BC7 | 4×4 | 16 | BC7_UNORM | -| 0x50 | ETC2_RGB8 | 4×4 | 8 | ETC2_R8G8B8_UNORM | -| 0x51 | ETC2_RGB8A1 | 4×4 | 8 | ETC2_R8G8B8A1_UNORM | -| 0x52 | ETC2_RGBA8 | 4×4 | 16 | ETC2_R8G8B8A8_UNORM | -| 0x53 | EAC_R11 | 4×4 | 8 | EAC_R11_UNORM | -| 0x54 | EAC_R11_SNORM | 4×4 | 8 | EAC_R11_SNORM | -| 0x55 | EAC_RG11 | 4×4 | 16 | EAC_R11G11_UNORM | -| 0x56 | EAC_RG11_SNORM | 4×4 | 16 | EAC_R11G11_SNORM | -| 0x60–0x6D | ASTC 4×4, 5×4, 5×5, 6×5, 6×6, 8×5, 8×6, 8×8, 10×5, 10×6, 10×8, 10×10, 12×10, 12×12 | as named | 16 | ASTC_*_UNORM (HDR: transfer linear, sample half) | - ---- - -## Channel meanings - -| Value | Meaning | Value | Meaning | -| ----: | :------ | ----: | :------ | -| 0 | default for the channel count (Y / YA / RGB / RGBA) | 11 | metalness | -| 1 | red | 12 | ambient occlusion | -| 2 | green | 13 | height | -| 3 | blue | 14 | specular | -| 4 | alpha | 15 | emissive | -| 5 | luma | 16 | opacity (alpha-test mask) | -| 6 | normal X | 17 | mask | -| 7 | normal Y | 18 | cavity | -| 8 | normal Z | 19 | thickness | -| 9 | roughness | 20 | id | -| 10 | gloss (1 − roughness) | 255 | unused | - -Alpha (an explicit 4 or 16, or the last channel of a 2- or 4-channel image -with default meanings) is always linear. The linear mask marks further -channels that are linear inside an sRGB image, such as masks packed next to -colour. - ---- - -## Mips - -`mip filter` records how the mips were made. This library's -`yapf_tex_build_mips` uses a 2 × 2 box (2 × 2 × 2 for volumes, edges -repeat), averages sRGB 8-bit channels in linear light (bit 7), and with an -alpha reference rescales alpha on every mip so that the share of texels -with alpha *above* the reference stays that of mip 0 (coverage flag, -`alpha ref`). Alpha-tested foliage and fences then keep their density at -a distance instead of thinning out. - ---- - -## Validation summary - -A decoder rejects a file when: - -* the magic, version, a size, shape or count is out of range, or a cube is - not square; -* the header or tables check does not match (unless the reader was told to - trust the file); -* a needs bit is unknown (*unsupported*), or a feature is used without its - needs bit (*corrupt*); -* a surface lies outside the file, its book is malformed or present without - ENTROPY, or its tiles do not fill it exactly; -* a tile's yapf-sum does not match, or the tile does not decode to exactly - its size (reserved bits, invalid filters, codes, masks or lengths). - -Because every structure is validated before use, corrupt or hostile input -is rejected, never read out of bounds. diff --git a/docs/LANGUAGES.md b/docs/LANGUAGES.md index 06ab3f0..7029222 100644 --- a/docs/LANGUAGES.md +++ b/docs/LANGUAGES.md @@ -22,15 +22,6 @@ the same native code, so they are all about equally fast; the JavaScript port needs no native library at all. Every example also re-encodes the image and checks that the result is byte-identical to the original file. -**YAPF 2 textures.** Most bindings wrap the version 1 API (`yapf_load`, -`yapf_save`, …). Through it they also read 8-bit version 2 files (2D, or -face 0 / layer 0 of cubes and arrays). GPU blocks, 16-bit / float pixels, -volumes and the other version 2 features are available from C and C++ -through the `yapf_tex_*` functions in [`yapf.h`](../yapf.h) (see -[FORMAT.md](FORMAT.md)); any language with a C FFI can call them the same -way. Python (`yapf.load_texture`) and JavaScript (`YAPF.decodeTexture`) -read every version 2 texture; the JavaScript port writes version 1. - ## Getting the native library Bindings other than C, C++, Rust, Go, Zig and JavaScript load the shared diff --git a/extensions/blender/README.md b/extensions/blender/README.md index 0a990bd..046d5af 100644 --- a/extensions/blender/README.md +++ b/extensions/blender/README.md @@ -12,14 +12,6 @@ Blender 5.1). Blender cannot read `.yapf` on its own, so imported images are **packed into the .blend** and survive saving and reopening. -Every YAPF file opens, version 1 and 2: 8-bit images; GPU blocks (BC1–BC7, -BC6H) decoded; 16-bit, half and float textures as **float images** (HDR -values above 1 are kept, packed as EXR); for cubes, arrays and animations -the first image. Normal maps follow Blender's OpenGL convention: DirectX -normals get their green flipped, and X/Y-only normals (such as BC5) get Z -rebuilt. Version 2 files need the bundled native library; the pure-Python -fallback reads version 1 only. Tested in Blender 4.4.3 (nixpkgs). - ## Install 1. Get `yapf_image-blender.zip` from the diff --git a/extensions/blender/yapf_image/__init__.py b/extensions/blender/yapf_image/__init__.py index 596220b..9732222 100644 --- a/extensions/blender/yapf_image/__init__.py +++ b/extensions/blender/yapf_image/__init__.py @@ -33,61 +33,30 @@ from . import yapf # ── conversion helpers ──────────────────────────────────────────────── -def _set_colorspace(bimg, names): - for n in names: # names differ between versions - try: - bimg.colorspace_settings.name = n - return - except TypeError: - continue - - -def _read(filepath): - """(rgba float32 h×w×4, has_alpha, flags, float_data, texture or None).""" - if yapf.backend() == "c" and getattr(yapf._lib, "has_textures", False): - tex = yapf.load_texture(filepath) # every v1 / v2 file - rgba = tex.rgba_float() # frame 0, layer 0, face +X, mip 0 - if tex.normal & 3 == 2: # DirectX (-Y) → Blender's OpenGL (+Y) - rgba[..., 1] = 1.0 - rgba[..., 1] - if tex.normal & 4 or (tex.normal and tex.channels == 2): # XY: rebuild Z - x, y = rgba[..., 0] * 2.0 - 1.0, rgba[..., 1] * 2.0 - 1.0 - rgba[..., 2] = (np.sqrt(np.clip(1.0 - x * x - y * y, 0.0, 1.0)) + 1.0) * 0.5 - alpha = tex.channels in (2, 4) and not tex.normal - wide = tex.is_hdr or (tex.payload == yapf.PAYLOAD_PIXELS and tex.sample == yapf.SAMPLE_U16) - return rgba, alpha, tex.flags, wide, tex - img = yapf.load(filepath) # pure-Python decoder: v1 only +def load_image(filepath): + """Read a .yapf file into a packed bpy.types.Image.""" + img = yapf.load(filepath) w, h, ch = img.width, img.height, img.channels src = np.frombuffer(img.pixels, dtype=np.uint8).reshape(h, w, ch).astype(np.float32) / 255.0 rgba = np.ones((h, w, 4), dtype=np.float32) - rgba[..., :3] = src[..., :3] if ch >= 3 else src[..., :1] + if ch >= 3: + rgba[..., :3] = src[..., :3] + else: + rgba[..., :3] = src[..., :1] if ch in (2, 4): rgba[..., 3] = src[..., ch - 1] - return rgba, ch in (2, 4), img.flags, False, None - - -def load_image(filepath): - """Read a .yapf file into a packed bpy.types.Image. 16-bit, half, - float and BC6H textures become float images (packed as EXR so nothing - is lost); GPU blocks are decoded.""" - rgba, alpha, flags, wide, tex = _read(filepath) - h, w = rgba.shape[:2] name = os.path.basename(filepath) - bimg = bpy.data.images.new(name, w, h, alpha=alpha, float_buffer=wide) - if tex is not None and tex.is_hdr: - _set_colorspace(bimg, ("Linear Rec.709", "Linear")) - elif flags & yapf.FLAG_SRGB: - _set_colorspace(bimg, ("sRGB",)) - else: - _set_colorspace(bimg, ("Non-Color",)) - if flags & yapf.FLAG_PREMULT_ALPHA: + bimg = bpy.data.images.new(name, w, h, alpha=ch in (2, 4)) + bimg.colorspace_settings.name = "sRGB" if img.flags & yapf.FLAG_SRGB else "Non-Color" + if img.flags & yapf.FLAG_PREMULT_ALPHA: bimg.alpha_mode = "PREMUL" bimg.pixels.foreach_set(rgba[::-1].ravel()) # Blender rows go bottom-up - bimg.file_format = "OPEN_EXR" if wide else "PNG" + bimg.file_format = "PNG" bimg.pack() # keep it inside the .blend bimg.use_fake_user = True # …even before anything uses it bimg["yapf_source"] = filepath - bimg["yapf_alpha"] = alpha + bimg["yapf_alpha"] = ch in (2, 4) return bimg diff --git a/extensions/figma/README.md b/extensions/figma/README.md index ac80b84..487853f 100644 --- a/extensions/figma/README.md +++ b/extensions/figma/README.md @@ -9,12 +9,6 @@ Both directions are pixel-exact: the plugin carries its own small PNG encoder/decoder instead of going through a canvas, which would alter semi-transparent pixels. -Import reads every YAPF file, version 1 and 2. 8-bit images keep their -exact pixels; GPU blocks (BC1–BC7, BC6H), 16-bit and float textures are -converted to 8-bit for display (Figma images are 8-bit); cubes, arrays, -volumes and animations bring their first image. Damaged files are -reported as such. - ## Install (development plugin) 1. Use the **Figma desktop app**. diff --git a/extensions/figma/src/ui.src.html b/extensions/figma/src/ui.src.html index e804d73..ca131a0 100644 --- a/extensions/figma/src/ui.src.html +++ b/extensions/figma/src/ui.src.html @@ -64,26 +64,14 @@ $('tab-import').onclick = () => tab('import'); $('tab-export').onclick = () => tab('export'); /* Import: .yapf → pixels → exact PNG → Figma image */ -/* Any YAPF file → 8-bit pixels. 8-bit images keep their exact pixels and - * channels; GPU blocks, 16-bit and float textures are converted for display - * (RGBA). Cubes, arrays, volumes and animations give their first image. */ -function yapfToPixels(tex) { - const w = tex.width, h = tex.height; - if (tex.payload === 0 && tex.sample === 0) - return { width: w, height: h, channels: tex.channels, - pixels: tex.surfaces[0].subarray(0, w * h * tex.channels) }; - const v = YAPF.textureToRGBA(tex, 0, 0); - return { width: w, height: h, channels: 4, pixels: new Uint8Array(v.rgba.buffer), - note: tex.payload === 1 ? 'GPU blocks decoded' : 'converted to 8-bit' }; -} async function importFiles(files) { const images = []; for (const f of files) { try { - const img = yapfToPixels(YAPF.decodeTexture(new Uint8Array(await f.arrayBuffer()))); + const img = YAPF.decode(new Uint8Array(await f.arrayBuffer())); const png = await PNG.encode(img.pixels, img.width, img.height, img.channels); images.push({ name: f.name.replace(/\.yapf$/i, ''), width: img.width, height: img.height, png }); - status('Decoded ' + f.name + ' (' + img.width + '×' + img.height + (img.note ? ', ' + img.note : '') + ')'); + status('Decoded ' + f.name + ' (' + img.width + '×' + img.height + ')'); } catch (e) { status(f.name + ': ' + e.message); } diff --git a/extensions/figma/ui.html b/extensions/figma/ui.html index 167bd23..0b419ac 100644 --- a/extensions/figma/ui.html +++ b/extensions/figma/ui.html @@ -47,7 +47,7 @@