From a226b063685dd9f78044e4e5f0fa007f6ec8716d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 10 Oct 2026 21:57:14 +0000 Subject: [PATCH 1/7] YAPF 2: texture container with GPU blocks, wide samples and streaming Adds format version 2 alongside version 1 (still written for plain 8-bit images, so existing readers keep working). Full specification in docs/FORMAT.md. - GPU block payloads: BC1-BC7, BC6H, ETC2/EAC, ASTC 4x4-12x12, in tiles of 16x16 blocks. Built-in BC1/BC3/BC4/BC5/BC7/BC6H encoders and BC1-BC7 decoders (BC6H/BC7 bit-exact with a reference decoder). - Lossless 16-bit, half and float pixels as byte planes over the 8-bit coder; order-preserving float mapping, transforms at sample width. - Cubemaps, cube arrays, texture arrays, volumes, animation frames. - Channel meanings, normal convention, linear mask, primaries/transfer, mip filter and coverage-preserving mips (yapf_tex_build_mips). - Small mips stored first, file size in the header, surface/tile tables: decode from a file prefix and single tiles (virtual texturing). - yapf-sum checksums on header, tables and every tile; needs bits so decoders refuse cleanly what they cannot read. - Optional entropy stage: per-surface Huffman code books with a neighbourhood context (~10% smaller photos) and per-lane packing of GPU blocks (~3x smaller on disk); planes decoded interleaved. - Notes: author, licence, source, tool, ICC and application tags. - CLI: --format, --entropy, --coverage, --normal, --map, colour options, 16-bit/HDR input, pack cube/array/volume/anim, info, verify, export. - Tests for all of the above, including fuzzing; v1 API reads 8-bit v2. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DVrVthWN8GsmgHGUhhsCYm --- README.md | 138 +- docs/FORMAT.md | 450 +++++++ docs/LANGUAGES.md | 8 + tests/test_yapf.c | 501 +++++++ tools/yapf_cli.c | 862 +++++++++--- yapf.c | 3252 ++++++++++++++++++++++++++++++++++++++++++++- yapf.h | 359 +++++ 7 files changed, 5325 insertions(+), 245 deletions(-) create mode 100644 docs/FORMAT.md diff --git a/README.md b/README.md index 2480e4d..58a0f18 100644 --- a/README.md +++ b/README.md @@ -14,10 +14,17 @@ encoder does the hard work, so the decoder only follows instructions. and natural textures — the price of having no entropy coder. - **Made for games and tools too**: mipmaps, GPU format hint, sRGB / premultiplied flags, and random access to 64×64 tiles. +- **A real texture format (YAPF 2)**: GPU-ready BC1–BC7 / BC6H / ETC2 / + ASTC blocks, 16-bit and float pixels, cubemaps, arrays and volumes, + channel meanings (normal X/Y, roughness, AO, …), coverage-preserving + mips, small-mips-first streaming, per-tile checksums, notes and + animation — see [Textures](#textures-yapf-2). - **Tiny and portable**: two C99 files, no dependencies. Plus a CLI, Python, JavaScript and plugins for the tools you already use. -This is **YAPF version 1**, the first stable format. +Two format versions: **version 1** (an 8-bit image, the first stable +format) and **version 2** (the texture container). Plain 8-bit images are +still written as version 1, so every existing YAPF reader keeps working. --- @@ -67,7 +74,64 @@ yapf compare photo.png ui.png # size and decode speed vs the original fo ``` Options when writing `.yapf`: `--mips`, `--linear` (pixels are not sRGB), -`--premult` (alpha is premultiplied). +`--premult` (alpha is premultiplied), and the texture options below. + +--- + +## Textures (YAPF 2) + +Version 2 turns YAPF into a texture container for engines, designed our own +way rather than copied from KTX2 or DDS (the design and the full +specification are in [docs/FORMAT.md](docs/FORMAT.md)): + +| Need | YAPF 2 | +| :--- | :----- | +| **GPU-compressed payloads** | BC1, BC2, BC3, BC4, BC5, BC6H, BC7, ETC2 / EAC and ASTC 4×4–12×12 blocks, stored as the GPU takes them. Built-in encoders for BC1, BC3, BC4, BC5, BC7 and BC6H; or bring blocks from your own encoder. | +| **More than 8 bits** | lossless 16-bit, half and float pixels, coded as byte planes with the same fast machinery | +| **Cubes, arrays, volumes** | cubemaps and cube arrays (reflection probes, skies), texture arrays (terrain layers), 3D textures (colour-grading LUTs) | +| **Channel meaning** | per channel: normal X/Y/Z, roughness, metalness, AO, height, opacity, …; OpenGL or DirectX normal convention; linear channels inside sRGB images | +| **Streaming** | small mips are stored first: a file prefix is a complete low-resolution texture (`yapf info` prints the prefix sizes); single 64 × 64 tiles decode on their own for virtual texturing | +| **Alpha-tested mips** | `--coverage REF` keeps fences and foliage from thinning out with distance; the filter used is recorded | +| **Entropy coding (optional)** | `--entropy`: shared per-surface Huffman code books with a neighbourhood context — about 10 % smaller photos, about 3× smaller GPU blocks on disk; tiles stay independent | +| **Metadata** | notes (author, licence, source, tool, ICC profile, orientation, DPI, HDR light levels, your own tags), colour primaries and transfer (sRGB, Display P3, BT.2020, PQ, HLG, …) | +| **Integrity** | yapf-sum checksums on the header, tables and every tile; *needs* bits make an older decoder refuse cleanly what it cannot read | +| **Animation** | frames with per-frame durations and a loop count | + +```sh +yapf albedo.png albedo.yapf --format bc7 --mips # GPU-ready colour +yapf normal.png normal.yapf --format bc5 --mips --normal dx # normal map, DirectX Y +yapf orm.png orm.yapf --map ao,rough,metal --linear --mips # packed masks +yapf foliage.png foliage.yapf --format bc7 --coverage 128 # alpha-tested mips +yapf sky.hdr sky.yapf --format bc6h --mips # HDR, no RGBM tricks +yapf sky.hdr sky_lossless.yapf # lossless half float +yapf pack cube px.png nx.png py.png ny.png pz.png nz.png probe.yapf --mips +yapf pack array grass.png rock.png sand.png terrain.yapf --format bc7 --mips +yapf pack volume lut_00.png lut_01.png ... grading.yapf --linear +yapf pack anim f0.png f1.png f2.png fx.yapf --frame-ms 50 +yapf albedo.png albedo.yapf --entropy --author "Keel" --license CC-BY-4.0 +yapf info albedo.yapf # everything in the header, notes, stream prefix sizes +yapf verify albedo.yapf # check every checksum +yapf probe.yapf face2.png --face 2 --mip 1 # export one surface +``` + +In C: + +```c +int err; +yapf_texture_t *t = yapf_tex_load("albedo.yapf", 0, 0, &err); +for (uint32_t m = 0; m < t->mip_levels; m++) { + uint32_t s = yapf_tex_surface(t, 0, 0, 0, m), w, h; + yapf_tex_mip_size(t, m, &w, &h, NULL); + if (t->payload == YAPF_PAYLOAD_BLOCKS) /* e.g. YAPF_GPU_BC7 */ + glCompressedTexImage2D(GL_TEXTURE_2D, m, gl_format(t->format, t->flags), + w, h, 0, (GLsizei)yapf_tex_surface_bytes(t, m), t->data[s]); +} +yapf_tex_free(t); +``` + +The language bindings and the JavaScript port speak version 1; the C +library's v1 functions also read 8-bit version 2 files (2D, cube face 0, +array layer 0), so plugins built on them open those too. --- @@ -198,6 +262,47 @@ void yapf_free(yapf_image_t *img); ``` Releases memory returned by the loaders. Passing `NULL` is safe. +### Textures (version 2) + +```c +yapf_texture_t *yapf_tex_load(const char *file, int flags, int threads, int *err); +yapf_texture_t *yapf_tex_load_memory(const void *buf, size_t size, int flags, int threads, int *err); +int yapf_tex_save(const char *file, const yapf_texture_t *t, int flags); +int yapf_tex_encode(const yapf_texture_t *t, int flags, void **out, size_t *size); +yapf_texture_t *yapf_tex_create(const yapf_texture_t *desc); /* zeroed storage */ +void yapf_tex_free(yapf_texture_t *t); +``` +A texture describes its shape (width, height, depth, layers, frames, +cube / volume, mips), storage (pixels with u8 / u16 / half / float samples, +or GPU blocks), meaning (channel map, linear mask, normal convention, +primaries, transfer, mip filter, alpha reference) and holds `data[s]` per +surface `s = yapf_tex_surface(t, frame, layer, face, mip)`. Load flag +`YAPF_LOAD_TRUST` skips checksum checks; encode flag `YAPF_ENC_ENTROPY` +allows the entropy stage. + +```c +yapf_texture_t *yapf_tex_open(const void *buf, size_t avail, int flags, int *err); +uint64_t yapf_tex_stream_bytes(const yapf_texture_t *t, uint32_t first_mip); +int yapf_tex_decode_surface(yapf_texture_t *t, const void *buf, size_t avail, uint32_t s, int threads); +int yapf_tex_decode_tile(const yapf_texture_t *t, const void *buf, size_t avail, uint32_t s, + uint32_t tx, uint32_t ty, uint32_t slice, void *out, size_t stride, + uint32_t *x, uint32_t *y, uint32_t *w, uint32_t *h); +``` +Streaming and virtual texturing: open a file from its first bytes, ask how +many bytes hold mip *m* and smaller, decode surfaces as their bytes arrive +(`YAPF_ERR_TRUNCATED` until then), or decode single tiles. + +```c +int yapf_tex_build_mips(yapf_texture_t *t, uint32_t levels, uint8_t alpha_ref); +int yapf_tex_compress(const yapf_texture_t *src, uint8_t format, yapf_texture_t **out); +int yapf_blocks_decode(uint8_t format, const void *blocks, uint32_t w, uint32_t h, void *out); +int yapf_tex_add_note(yapf_texture_t *t, const char tag[4], const void *data, uint32_t size); +const yapf_note_t *yapf_tex_find_note(const yapf_texture_t *t, const char tag[4]); +``` +Mip chains (linear-light sRGB, optional coverage preservation), the +built-in BCn encoders, BC1–BC7 decoding for previews, and notes. The +complete list, with sizes and helpers, is in [`yapf.h`](yapf.h). + ### Return codes | Constant | Value | Meaning | @@ -206,6 +311,10 @@ Releases memory returned by the loaders. Passing `NULL` is safe. | `YAPF_ERR_INVALID` | –1 | NULL or logically invalid argument | | `YAPF_ERR_IO` | –2 | File could not be opened or written | | `YAPF_ERR_OOM` | –3 | Out of memory | +| `YAPF_ERR_CORRUPT` | –4 | Corrupt or not a YAPF file | +| `YAPF_ERR_UNSUPPORTED` | –5 | Needs a feature this decoder lacks | +| `YAPF_ERR_CHECKSUM` | –6 | A checksum does not match (v2) | +| `YAPF_ERR_TRUNCATED` | –7 | The data is not (yet) all there | ### Channel constants @@ -256,7 +365,8 @@ domain [stb](https://github.com/nothings/stb) headers in `third_party/`. 7. **Independent tiles** — any tile can be decoded on its own, in any order, on any thread. -The complete specification is the comment at the top of [`yapf.h`](yapf.h): +The version 1 specification is the comment at the top of [`yapf.h`](yapf.h); +version 2 (textures) is specified in [docs/FORMAT.md](docs/FORMAT.md). Version 1: ``` HEADER (20 bytes) "YAPF", version 1, channels, GPU hint, mip count, @@ -273,7 +383,7 @@ TILE (64×64) header byte: stored raw, or constant-plane mask + colour tr ## Testing ```sh -make test # C suite: 786 checks, plus JavaScript if node is installed +make test # C suite: 2631 checks, plus JavaScript if node is installed node bindings/js/test.js # JavaScript: round-trips, sample file, corrupt input ``` @@ -286,6 +396,16 @@ What has been verified: C++, and under AddressSanitizer, UndefinedBehaviorSanitizer and ThreadSanitizer; - thousands of corrupted and truncated files are rejected without crashing; +- version 2: lossless round-trips of u8 / u16 / half / float pixels with 1–4 + channels, with and without entropy coding; cubes, cube arrays, arrays, + volumes and animations with notes and every description field; GPU block + payloads; checksum, needs-bit and truncation handling; streaming from file + prefixes and single-tile decoding; BCn encoder quality; linear-light and + coverage-preserving mips; fuzzing of entropy-coded and packed-block files; +- the BC1–BC7 decoders were checked against an independent reference decoder + (bcdec) on 300,000 random blocks: BC6H and BC7 bit-exact, BC1–BC5 within + the ±1 rounding the formats allow (a development check, not part of + `make test`); - every language binding runs its example; JavaScript, Python, C++, C#, Rust, Go, Java and Lua re-encode the sample to byte-identical output; - the GIMP plugin in GIMP 3.0.8 and the Blender extension in Blender 5.1; @@ -293,14 +413,20 @@ What has been verified: **Not yet verified:** running on a Mac (macOS builds are cross-compiled only), Windows on ARM, and running inside Photoshop, VS Code, Figma or a -live Dolphin / GNOME file manager. +live Dolphin / GNOME file manager. The version 2 code has been run on +Linux x64 (with and without SIMD, as C and C++, under ASan, UBSan and +TSan); its NEON path is compiled for ARM64 but has not been run yet, and +the Windows builds have not been rerun. ETC2 / EAC / ASTC blocks are +stored and loaded, but there is no built-in encoder or preview decoder +for them. --- ## Project layout ``` -yapf.h, yapf.c the library (format spec in yapf.h) +yapf.h, yapf.c the library (v1 spec in yapf.h) +docs/FORMAT.md the full format specification, version 2 tools/yapf_cli.c the `yapf` command-line tool tools/build_release.sh builds every download for all platforms tests/test_yapf.c C test suite diff --git a/docs/FORMAT.md b/docs/FORMAT.md new file mode 100644 index 0000000..1099f1e --- /dev/null +++ b/docs/FORMAT.md @@ -0,0 +1,450 @@ +# YAPF file format + +This is the complete specification of YAPF files. `yapf.c` is the +reference implementation; where this text and the code disagree, that is a +bug in one of them. + +YAPF has two versions: + +| Version | What it holds | Who writes it | +| :------ | :------------ | :------------ | +| **1** | one 8-bit image (1–4 channels) with optional mips | `yapf_save` / `yapf_encode`, and the CLI for plain 8-bit images | +| **2** | a texture: GPU blocks or lossless pixels in 8/16-bit or float, cubes, arrays, volumes, animation, channel meanings, colour description, notes, checksums, optional entropy coding | `yapf_tex_save` / `yapf_tex_encode`, and the CLI whenever a v2 feature is used | + +Version 1 is specified at the top of [`yapf.h`](../yapf.h) and is not +repeated here, except that version 2 reuses its tiles, filters and plane +payloads (the "group form") byte for byte. + +--- + +## Design: how YAPF 2 does things its own way + +YAPF 2 is not a wrapper around someone else's container. A few ideas +carry the whole format: + +1. **Everything is tiles.** Pixels are cut into 64 × 64 tiles, GPU + blocks into tiles of 16 × 16 blocks. Each tile has its own size and + checksum in a table, so any tile can be found, checked and decoded on its + own: for multithreaded loading, for partial updates, and for virtual + texturing / "megatexture" streaming. +2. **Small mips first.** Surface data is stored from the smallest mip + to the largest. The first *N* bytes of a file are a complete, lower + resolution texture, so an engine can show something after reading a + few kilobytes and stream in the large mips later. The header carries + the file size, so a loader knows how much is still to come. +3. **Needs, not versions.** New capabilities are announced by *needs* + bits. A decoder refuses a file with a needs bit it does not know, and + can ignore everything else safely. No more version bumps that break + every reader. +4. **Wide samples are byte planes.** 16-bit and float samples are split + into byte planes (high bytes, low bytes) and coded with the same fast + 8-bit machinery as version 1: smooth high bytes compress very well, and + the decoder stays branch-light and SIMD friendly. +5. **Shared code books.** The optional entropy stage keeps its Huffman + tables once per surface, keyed by a small neighbourhood context, so + tiles stay independent and their own headers stay tiny. For GPU + blocks the tables are per byte lane, so endpoint bytes and index bytes + are coded separately. +6. **Meaning travels with the data.** Channel meanings, the normal-map + convention, linear channels inside sRGB images, colour primaries and + transfer, and how the mips were made are header fields, not file-name + suffixes. + +--- + +## Conventions + +All integers are little-endian. `u8`, `u16`, `u32` and `u64` are unsigned +integers of that many bits. Offsets are from the start of the file. + +### yapf-sum + +The checksum of a byte string of length *n*. `K = 0x9E3779B97F4A7C15`; +all arithmetic is modulo 2⁶⁴. + +``` +lane[i] = (i + 1)·K XOR n for i = 0, 1, 2, 3 +for each 8-byte word w_j of the input (little-endian; the last word is +padded with zero bytes): + a = lane[j mod 4] + a = (a XOR w_j) · K + a = a XOR (a >> 32) + lane[j mod 4] = a +h = lane[0] XOR rotl(lane[1], 16) XOR rotl(lane[2], 32) XOR rotl(lane[3], 48) +h = h · K +h = h XOR (h >> 32) +result = low 32 bits of h +``` + +Test vectors: `""` → `0xd7550ed0`, `"YAPF"` → `0xc8cc41d4`, +`"The quick brown fox jumps over the lazy dog"` → `0xbce4f410`. + +--- + +## Version 2 layout + +``` +HEADER 64 bytes +SURFACE TABLE 16 bytes per surface +FRAME TABLE 4 bytes per frame, only when frames > 1 +NOTES notes size bytes +SURFACE DATA in stream order: smallest mip first +``` + +The *front matter* is everything before the surface data: +`64 + 16·surfaces + (frames > 1 ? 4·frames : 0) + notes size` bytes. + +### Header + +| Offset | Size | Field | Notes | +| -----: | ---: | :---- | :---- | +| 0x00 | 4 | magic | `"YAPF"` | +| 0x04 | 1 | version | 2 | +| 0x05 | 1 | channels | 1–4 | +| 0x06 | 1 | format | block format (block payload) or GPU hint (pixel payload); see [GPU formats](#gpu-formats) | +| 0x07 | 1 | mip levels | 1–16 | +| 0x08 | 1 | flags | bit 0 premultiplied alpha, bit 1 sRGB, bit 2 coverage-preserving mips; other bits 0 | +| 0x09 | 1 | payload | 0 pixels (lossless tiles), 1 GPU blocks | +| 0x0A | 1 | sample | pixel payload: 0 u8, 1 u16, 2 half float, 3 float. Block payload: type of the decoded texels (0, or 2 for BC6H) | +| 0x0B | 1 | shape | 0 2D, 1 cube, 2 volume | +| 0x0C | 4 | width | 1–65535 | +| 0x10 | 4 | height | 1–65535; cubes: width = height | +| 0x14 | 4 | depth | volumes: 1–65535; otherwise 1 | +| 0x18 | 2 | layers | ≥ 1 (array layers; with a cube, a cube array); volumes: 1 | +| 0x1A | 2 | frames | ≥ 1 (animation frames) | +| 0x1C | 4 | needs | see [Needs](#needs) | +| 0x20 | 4 | channel map | meaning of channels 0–3, see [Channel meanings](#channel-meanings) | +| 0x24 | 1 | linear mask | bit *c*: channel *c* is linear even when the sRGB flag is set | +| 0x25 | 1 | normal | bits 0–1 convention (0 none, 1 OpenGL +Y, 2 DirectX −Y); bits 2–3 encoding (0 XYZ, 1 XY with Z reconstructed, 2 octahedral) | +| 0x26 | 1 | primaries | 0 unspecified, 1 BT.709/sRGB, 2 Display P3, 3 BT.2020, 4 ACES AP0, 5 ACEScg (AP1), 6 Adobe RGB | +| 0x27 | 1 | transfer | 0 unspecified (the sRGB flag decides), 1 sRGB, 2 linear, 3 gamma 2.2, 4 PQ (ST 2084), 5 HLG, 6 BT.1886 | +| 0x28 | 1 | mip filter | bits 0–6: 0 unknown, 1 box, 2 tent, 3 Kaiser, 4 Lanczos, 5 Mitchell, 6 authored; bit 7: sRGB channels were filtered in linear light | +| 0x29 | 1 | alpha ref | with the coverage flag: alpha-test reference 0–255 | +| 0x2A | 2 | loop count | animation repeats, 0 = forever | +| 0x2C | 4 | notes size | bytes of the notes area | +| 0x30 | 8 | file size | total bytes of the file | +| 0x38 | 4 | tables check | yapf-sum of bytes [0x40, front matter end) | +| 0x3C | 4 | header check | yapf-sum of bytes [0x00, 0x3C) | + +The header check covers the tables check, so the whole front matter is +protected. + +### Surfaces + +A *surface* is one mip level of one face of one layer of one frame. With +`faces = 6` for cubes (order +X, −X, +Y, −Y, +Z, −Z) and 1 otherwise, +surface number + +``` +s = ((frame · layers + layer) · faces + face) · mip_levels + mip +``` + +Mip *m* is `max(1, width >> m) × max(1, height >> m)`, and for volumes +`max(1, depth >> m)` slices deep. A volume surface holds all its slices. + +**Surface table entry** (16 bytes, one per surface, in surface order): + +| Offset | Size | Field | +| -----: | ---: | :---- | +| +0 | 8 | offset of the surface data | +| +8 | 4 | size of the surface data | +| +12 | 4 | tile-table check: yapf-sum of the surface's book size, book and tile table | + +**Stream order.** Writers store surface data for mip `mip_levels − 1` +first (all frames, layers and faces in surface order), then the next larger +mip, and so on, ending with mip 0. Readers must use the offsets and must not +rely on the order. The number of leading bytes that hold every surface of +mip *m* and smaller is the largest `offset + size` over those surfaces. + +### Frame table + +When `frames > 1`: one `u32` per frame, its display time in microseconds. + +### Notes + +A sequence of entries that exactly fills the notes area: + +``` +tag 4 bytes +length u32 +data length bytes +``` + +| Tag | Content | +| :-- | :------ | +| `AUTH` | author (UTF-8) | +| `LICN` | licence (UTF-8, e.g. an SPDX id) | +| `SRCE` | source / origin (UTF-8, e.g. a URL or path) | +| `TOOL` | the tool that wrote the file (UTF-8) | +| `TITL` | title (UTF-8) | +| `DESC` | description (UTF-8) | +| `DATE` | date, ISO 8601 (UTF-8) | +| `ICCP` | an ICC colour profile | +| `ORNT` | orientation, 1 byte: bits 0–1 clockwise quarter turns, bit 2 mirror left–right after turning | +| `DPI ` | 2 × u32: horizontal and vertical dots per inch × 1000 | +| `CLL ` | 2 × u16: maximum content light level, maximum frame-average light level (cd/m²) | + +Tags that begin with a lowercase letter belong to applications (for +example `keel`). Readers ignore tags they do not know. Tags may repeat; +order is kept. + +### Needs + +| Bit | Name | Set when | +| --: | :--- | :------- | +| 0 | WIDE | pixel payload with sample ≠ u8 | +| 1 | BLOCKS | block payload | +| 2 | ENTROPY | any surface has a code book | +| 3 | LAYOUT | shape ≠ 2D, or layers > 1 | +| 4 | FRAMES | frames > 1 | + +A decoder must refuse a file that sets a needs bit it does not implement +(this library: any bit above 4). A file that uses a feature without its +bit is corrupt. Everything that is not a needs feature — channel map, +colour fields, mip filter, notes, frame times — may be ignored. + +A decoder that only implements 8-bit 2D pixels can therefore read every +version 2 file whose needs are 0, and cleanly refuse the rest. + +--- + +## Surface data + +``` +book size u32 0 when the surface has no code book +book book size bytes +tile table 8 bytes per tile: u32 size (bit 31: packed), u32 yapf-sum of the tile bytes +tiles back to back; their sizes must add up exactly to the rest of the surface +``` + +Tiles are numbered slice by slice, rows top to bottom, tiles left to right. +For pixels the grid is `ceil(w / 64) × ceil(h / 64)` tiles per slice (edge +tiles are smaller, no padding); for blocks it is +`ceil(bx / 16) × ceil(by / 16)` tiles of 16 × 16 blocks, where +`bx = ceil(w / block width)` and `by = ceil(h / block height)`. + +Bit 31 of a tile size (*packed*) is allowed only in block tiles of a +surface with a code book. + +--- + +## Pixel tiles + +Let *B* be the bytes per sample (1, 2 or 4), *ch* the channels, and the +tile `tw × th` pixels. + +### Tile header + +``` +byte 0 0x80 stored: the raw samples follow, interleaved in + channel order, each B bytes little-endian; + exactly tw·th·ch·B bytes + otherwise coded: + bits 0–3 B = 1: bit c set = plane c is constant; B > 1: 0 + bit 4 colour transform subtract-green + bit 5 an entropy mask follows + bit 6 no colour transform + bit 7 0 + bits 4 and 6 are exclusive and need ch >= 3; with neither + set, an RGB(A) tile uses YCoCg-R +B > 1 u16 constant mask, bit p = plane p is constant +bit 5 entropy mask: u8 (B = 1) or u16 (B > 1); bit p = plane p is + entropy coded. It must not include constant planes. +then one value byte per constant plane, in plane order; + one payload per other plane, in plane order; + the tile ends exactly where its last payload ends +``` + +With B = 1 and neither bit 5 nor bit 6 set, a coded tile is exactly a +version 1 tile. + +### Planes + +The colour transform turns R, G, B (+ A) into transformed channels; it is +skipped for 1–2 channels and with bit 6. + +* **YCoCg-R** at the sample width *n* = 8·B bits, all arithmetic mod 2ⁿ, + `>>` an arithmetic shift of the *n*-bit two's complement value, `h = 2ⁿ⁻¹`: + + ``` + forward: o = R − B; t = B + (o >> 1); q = G − t; + Y = t + (q >> 1); Co = o + h; Cg = q + h + inverse: o = Co − h; q = Cg − h; t = Y − (q >> 1); + G = q + t; B = t − (o >> 1); R = B + o + ``` +* **subtract-green**: `G, R − G + h, B − G + h` (+ A), mod 2ⁿ. + +Before the transform, float samples (half or float) are mapped to +integers that keep their order: if the sign bit is set every bit is +flipped, otherwise the sign bit is set. The decoder undoes this after the +inverse transform. + +Transformed channel *c* gives B planes: plane `c·B + k` holds byte +`B − 1 − k` of the value (the most significant byte first). For B = 1 +the planes are the transformed channels. + +### Plane payloads + +A plane payload has one of two forms, chosen per plane by the entropy mask. + +**Group form** — exactly the version 1 plane payload: a filter byte per +band of 8 rows, 4-bit control codes, and groups of 8 residuals at the +group's bit width. See `yapf.h`. + +**Entropy form** + +``` +filter[ceil(th / 8)] as in the group form +u16 L bytes of the bit stream +L bytes the folded residuals of the plane in raster order +``` + +Each residual is coded with table `(slot, context)` of the surface's code +book: + +* `slot = t · 16 + p`, where *p* is the plane and *t* is 1 with + subtract-green, 2 with no transform, else 0; +* `context = min(6, floor(log2(s + 1)))`, `s = left + above`, the folded + residuals of the left and upper neighbours in this plane (0 outside the + tile) — so 0, 1–2, 3–6, 7–14, 15–30, 31–62, 63 and up. + +The stream must end in its last byte. Folded residuals then go through +the filters exactly as in the group form. + +--- + +## Block tiles + +A block tile of `tbw × tbh` blocks of *S* bytes is either + +* **raw**: the blocks, rows top to bottom, exactly `tbw·tbh·S` bytes — as + the GPU takes them; or +* **packed** (bit 31 of its size): one bit stream holding byte 0 of every + block (in raster order), then byte 1 of every block, …, byte S − 1. + Byte lane *k* is coded with table `(slot k, context 0)` of the surface's + code book. The stream must end in the last byte of the tile. + +Decoded block surfaces are row-major blocks, slice after slice, ready for +`glCompressedTexImage*` / `vkCmdCopyBufferToImage`. + +--- + +## Code book + +``` +u16 count number of tables, 1–336 +count × { + u8 slot 0–47 + u8 context 0–6 + code table +} +``` + +Each `(slot, context)` pair appears at most once. The book must fill its +size exactly. + +**Code table**: `u8 n − 1`, then *n* 4-bit code lengths (symbols +0 … n − 1, low nibble first, an odd count padded with a zero nibble). +Length 0 = no code; otherwise 1–11. At least one code; Kraft sum ≤ 1. +Codes are canonical — by length, then by symbol, the first code of each +length one more than the last code of the previous length, shifted left — +and stored least significant bit first (the first bit of a code is the +lowest unread bit of the stream), so a decoder can look up 11 bits at a +time. + +--- + +## GPU formats + +Pixel payload hints (`format` field): + +| Value | Name | Vulkan | +| ----: | :--- | :----- | +| 0x00 | RGBA8 | R8G8B8A8_UNORM | +| 0x01 | RGB8 | R8G8B8_UNORM | +| 0x02 | RG8 | R8G8_UNORM | +| 0x03 | R8 | R8_UNORM | +| 0x04 | SRGB8_A8 | R8G8B8A8_SRGB | +| 0x05 | SRGB8 | R8G8B8_SRGB | +| 0x06–0x09 | RGBA16, RGB16, RG16, R16 | R16G16B16A16_UNORM … R16_UNORM | +| 0x0A–0x0D | RGBA16F, RGB16F, RG16F, R16F | R16G16B16A16_SFLOAT … R16_SFLOAT | +| 0x0E–0x11 | RGBA32F, RGB32F, RG32F, R32F | R32G32B32A32_SFLOAT … R32_SFLOAT | + +Block formats (block payload). With the sRGB flag, use the `_SRGB` +variant where one exists. + +| Value | Name | Block | Bytes | Vulkan / DXGI | +| ----: | :--- | :---: | ----: | :------------ | +| 0x40 | BC1 | 4×4 | 8 | BC1_RGB(A)_UNORM / BC1_UNORM | +| 0x41 | BC2 | 4×4 | 16 | BC2_UNORM | +| 0x42 | BC3 | 4×4 | 16 | BC3_UNORM | +| 0x43 | BC4 | 4×4 | 8 | BC4_UNORM | +| 0x44 | BC4_SNORM | 4×4 | 8 | BC4_SNORM | +| 0x45 | BC5 | 4×4 | 16 | BC5_UNORM | +| 0x46 | BC5_SNORM | 4×4 | 16 | BC5_SNORM | +| 0x47 | BC6H | 4×4 | 16 | BC6H_UFLOAT / BC6H_UF16 | +| 0x48 | BC6H_SF | 4×4 | 16 | BC6H_SFLOAT / BC6H_SF16 | +| 0x49 | BC7 | 4×4 | 16 | BC7_UNORM | +| 0x50 | ETC2_RGB8 | 4×4 | 8 | ETC2_R8G8B8_UNORM | +| 0x51 | ETC2_RGB8A1 | 4×4 | 8 | ETC2_R8G8B8A1_UNORM | +| 0x52 | ETC2_RGBA8 | 4×4 | 16 | ETC2_R8G8B8A8_UNORM | +| 0x53 | EAC_R11 | 4×4 | 8 | EAC_R11_UNORM | +| 0x54 | EAC_R11_SNORM | 4×4 | 8 | EAC_R11_SNORM | +| 0x55 | EAC_RG11 | 4×4 | 16 | EAC_R11G11_UNORM | +| 0x56 | EAC_RG11_SNORM | 4×4 | 16 | EAC_R11G11_SNORM | +| 0x60–0x6D | ASTC 4×4, 5×4, 5×5, 6×5, 6×6, 8×5, 8×6, 8×8, 10×5, 10×6, 10×8, 10×10, 12×10, 12×12 | as named | 16 | ASTC_*_UNORM (HDR: transfer linear, sample half) | + +--- + +## Channel meanings + +| Value | Meaning | Value | Meaning | +| ----: | :------ | ----: | :------ | +| 0 | default for the channel count (Y / YA / RGB / RGBA) | 11 | metalness | +| 1 | red | 12 | ambient occlusion | +| 2 | green | 13 | height | +| 3 | blue | 14 | specular | +| 4 | alpha | 15 | emissive | +| 5 | luma | 16 | opacity (alpha-test mask) | +| 6 | normal X | 17 | mask | +| 7 | normal Y | 18 | cavity | +| 8 | normal Z | 19 | thickness | +| 9 | roughness | 20 | id | +| 10 | gloss (1 − roughness) | 255 | unused | + +Alpha (an explicit 4 or 16, or the last channel of a 2- or 4-channel image +with default meanings) is always linear. The linear mask marks further +channels that are linear inside an sRGB image, such as masks packed next to +colour. + +--- + +## Mips + +`mip filter` records how the mips were made. This library's +`yapf_tex_build_mips` uses a 2 × 2 box (2 × 2 × 2 for volumes, edges +repeat), averages sRGB 8-bit channels in linear light (bit 7), and with an +alpha reference rescales alpha on every mip so that the share of texels +with alpha *above* the reference stays that of mip 0 (coverage flag, +`alpha ref`). Alpha-tested foliage and fences then keep their density at +a distance instead of thinning out. + +--- + +## Validation summary + +A decoder rejects a file when: + +* the magic, version, a size, shape or count is out of range, or a cube is + not square; +* the header or tables check does not match (unless the reader was told to + trust the file); +* a needs bit is unknown (*unsupported*), or a feature is used without its + needs bit (*corrupt*); +* a surface lies outside the file, its book is malformed or present without + ENTROPY, or its tiles do not fill it exactly; +* a tile's yapf-sum does not match, or the tile does not decode to exactly + its size (reserved bits, invalid filters, codes, masks or lengths). + +Because every structure is validated before use, corrupt or hostile input +is rejected, never read out of bounds. diff --git a/docs/LANGUAGES.md b/docs/LANGUAGES.md index 7029222..17df03f 100644 --- a/docs/LANGUAGES.md +++ b/docs/LANGUAGES.md @@ -22,6 +22,14 @@ the same native code, so they are all about equally fast; the JavaScript port needs no native library at all. Every example also re-encodes the image and checks that the result is byte-identical to the original file. +**YAPF 2 textures.** The bindings wrap the version 1 API (`yapf_load`, +`yapf_save`, …). Through it they also read 8-bit version 2 files (2D, or +face 0 / layer 0 of cubes and arrays). GPU blocks, 16-bit / float pixels, +volumes and the other version 2 features are available from C and C++ +through the `yapf_tex_*` functions in [`yapf.h`](../yapf.h) (see +[FORMAT.md](FORMAT.md)); any language with a C FFI can call them the same +way. The JavaScript port reads and writes version 1 only. + ## Getting the native library Bindings other than C, C++, Rust, Go, Zig and JavaScript load the shared diff --git a/tests/test_yapf.c b/tests/test_yapf.c index 2412684..000a457 100644 --- a/tests/test_yapf.c +++ b/tests/test_yapf.c @@ -7,6 +7,8 @@ * Round-trips many sizes, channel counts and image kinds, checks mip * chains, single- vs multi-threaded decode, the bundled sample file, and * that corrupted or truncated files are rejected without crashing. + * Version 2: wide samples, shapes, GPU blocks, notes, checksums, needs, + * streaming, tile decode, block encoders and mip generation. * Exit code 0 means every test passed. */ #include "../yapf.h" @@ -14,6 +16,7 @@ #include #include #include +#include /* FNV-1a of the decoded pixels of other/YAPF.YAPF (the YAPF logo). */ #define SAMPLE_HASH 0x8891c2b4u @@ -171,6 +174,494 @@ static uint32_t fnv(const uint8_t *p, size_t n) { return h; } + +/* ====================================================================== + * Version 2 + * ====================================================================== */ + +static yapf_texture_t desc(uint32_t w, uint32_t h, int ch, int sample) { + yapf_texture_t d; + memset(&d, 0, sizeof(d)); + d.width = w; d.height = h; d.depth = 1; d.layers = 1; d.frames = 1; + d.mip_levels = 1; d.channels = (uint8_t)ch; d.sample = (uint8_t)sample; + return d; +} + +/* Fills every surface: noise, gradient or noisy curves, in the sample + * type; block payloads get bytes with some structure. */ +static void tex_fill(yapf_texture_t *t, int kind) { + uint32_t S = yapf_tex_surface_count(t); + for (uint32_t s = 0; s < S; s++) { + uint32_t m = s % t->mip_levels, w, h, d; + yapf_tex_mip_size(t, m, &w, &h, &d); + uint8_t *p = (uint8_t *)t->data[s]; + if (t->payload == YAPF_PAYLOAD_BLOCKS) { + size_t n = yapf_tex_surface_bytes(t, m); + for (size_t i = 0; i < n; i++) + p[i] = kind ? (uint8_t)(i % 16 < 8 ? (i / 64) & 0xFF : rnd()) : (uint8_t)rnd(); + continue; + } + size_t cnt = (size_t)w * h * d * t->channels; + for (size_t i = 0; i < cnt; i++) { + size_t px = i / t->channels; + uint32_t x = (uint32_t)(px % w), y = (uint32_t)((px / w) % h); + int c = (int)(i % t->channels); + double v = kind == 0 ? (rnd() % 1000) / 1000.0 + : kind == 1 ? (x * 3 + y * 5 + c * 40) % 600 / 600.0 + : ((x * x + y * y) / 50 % 700 + rnd() % 5) / 705.0; + switch (t->sample) { + case YAPF_SAMPLE_U8: p[i] = (uint8_t)(v * 255); break; + case YAPF_SAMPLE_U16: ((uint16_t *)p)[i] = (uint16_t)(v * 65535); break; + case YAPF_SAMPLE_F16: ((uint16_t *)p)[i] = yapf_float_to_half((float)(v * 20 - 2)); break; + default: ((float *)p)[i] = (float)(v * 100 - 10) + (kind == 0 ? (float)rnd() : 0.0f); break; + } + } + } +} + +static int tex_equal(const yapf_texture_t *a, const yapf_texture_t *b) { + uint32_t S = yapf_tex_surface_count(a); + if (S != yapf_tex_surface_count(b)) return 0; + for (uint32_t s = 0; s < S; s++) + if (!a->data[s] || !b->data[s] || + memcmp(a->data[s], b->data[s], yapf_tex_surface_bytes(a, s % a->mip_levels)) != 0) + return 0; + return 1; +} + +/* Encode, decode, compare; returns the encoded file (caller frees). */ +static uint8_t *tex_roundtrip(yapf_texture_t *t, int flags, size_t *len, const char *what) { + void *buf = NULL; + *len = 0; + int e = yapf_tex_encode(t, flags, &buf, len); + CHECK(e == YAPF_OK, "v2 encode %s (%d)", what, e); + if (e != YAPF_OK) return NULL; + int err = 0; + yapf_texture_t *a = yapf_tex_load_memory(buf, *len, 0, 1, &err); + yapf_texture_t *b = yapf_tex_load_memory(buf, *len, 0, 0, &err); + CHECK(a && tex_equal(a, t), "v2 roundtrip %s", what); + CHECK(b && tex_equal(b, t), "v2 roundtrip (threads) %s", what); + yapf_tex_free(a); + yapf_tex_free(b); + uint8_t *copy = (uint8_t *)malloc(*len); + memcpy(copy, buf, *len); + yapf_free_buffer(buf); + return copy; +} + +static void v2_pixels(void) { + static const int sizes[][2] = { {1, 1}, {7, 3}, {64, 64}, {65, 63}, {130, 70}, {200, 201} }; + static const char *sn[] = { "u8", "u16", "f16", "f32" }; + for (size_t si = 0; si < sizeof(sizes) / sizeof(sizes[0]); si++) + for (int ch = 1; ch <= 4; ch++) + for (int sample = 0; sample < 4; sample++) + for (int kind = 0; kind < 3; kind++) + for (int fl = 0; fl < 2; fl++) { + yapf_texture_t d = desc((uint32_t)sizes[si][0], (uint32_t)sizes[si][1], ch, sample); + yapf_texture_t *t = yapf_tex_create(&d); + char what[96]; + sprintf(what, "%dx%d ch%d %s kind%d%s", sizes[si][0], sizes[si][1], ch, + sn[sample], kind, fl ? " entropy" : ""); + tex_fill(t, kind); + size_t len; + free(tex_roundtrip(t, fl ? YAPF_ENC_ENTROPY : 0, &len, what)); + yapf_tex_free(t); + } +} + +/* Shapes, frames, notes and every description field survive. */ +static void v2_layout(void) { + struct { int w, h, ch, sample, shape, layers, frames, mips, depth; } c[] = { + { 64, 64, 4, 0, YAPF_SHAPE_CUBE, 1, 1, 7, 1 }, + { 32, 32, 3, 2, YAPF_SHAPE_CUBE, 2, 1, 6, 1 }, + { 40, 30, 4, 0, YAPF_SHAPE_2D, 5, 1, 3, 1 }, + { 33, 17, 1, 1, YAPF_SHAPE_VOLUME, 1, 1, 4, 9 }, + { 50, 50, 4, 0, YAPF_SHAPE_2D, 1, 6, 1, 1 }, + }; + for (size_t i = 0; i < sizeof(c) / sizeof(c[0]); i++) { + yapf_texture_t d = desc((uint32_t)c[i].w, (uint32_t)c[i].h, c[i].ch, c[i].sample); + d.shape = (uint8_t)c[i].shape; d.layers = (uint16_t)c[i].layers; d.frames = (uint16_t)c[i].frames; + d.mip_levels = (uint8_t)c[i].mips; d.depth = (uint32_t)c[i].depth; + d.channel_map[0] = YAPF_CH_NORMAL_X; d.channel_map[1] = YAPF_CH_ROUGHNESS; + d.linear_mask = 0x08; d.normal = YAPF_NORMAL_DX | YAPF_NORMAL_XY; + d.primaries = YAPF_PRIMARIES_DISPLAY_P3; d.transfer = YAPF_TRANSFER_PQ; + d.mip_filter = YAPF_MIP_KAISER; d.alpha_ref = 77; d.loop_count = 3; + d.flags = YAPF_FLAG_SRGB | YAPF_FLAG_COVERAGE; + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 2); + if (t->frames > 1) for (int f = 0; f < t->frames; f++) t->frame_us[f] = 16667u * (uint32_t)(f + 1); + yapf_tex_add_note(t, "AUTH", "Keel", 4); + yapf_tex_add_note(t, "ICCP", "\0\1\2", 3); + size_t len; + char what[32]; + sprintf(what, "layout case %d", (int)i); + uint8_t *file = tex_roundtrip(t, (int)(i & 1), &len, what); + yapf_texture_t *b = file ? yapf_tex_load_memory(file, len, 0, 1, NULL) : NULL; + int ok = b && b->shape == d.shape && b->layers == d.layers && b->frames == d.frames && + b->depth == d.depth && b->mip_levels == d.mip_levels && + memcmp(b->channel_map, d.channel_map, 4) == 0 && b->linear_mask == 0x08 && + b->normal == d.normal && b->primaries == d.primaries && b->transfer == d.transfer && + b->mip_filter == d.mip_filter && b->alpha_ref == 77 && b->loop_count == 3 && + b->flags == d.flags && b->note_count == 2; + const yapf_note_t *n = b ? yapf_tex_find_note(b, "ICCP") : NULL; + ok = ok && n && n->size == 3 && memcmp(n->data, "\0\1\2", 3) == 0; + if (ok && b->frames > 1) ok = b->frame_us[b->frames - 1] == 16667u * b->frames; + CHECK(ok, "v2 fields, notes and frame times survive (%s)", what); + yapf_tex_free(b); + free(file); + yapf_tex_free(t); + } +} + +static void v2_blocks(void) { + static const uint8_t fmts[] = { YAPF_GPU_BC1, YAPF_GPU_BC7, YAPF_GPU_BC4, YAPF_GPU_ASTC_6x6, + YAPF_GPU_ASTC_12x12, YAPF_GPU_ETC2_RGBA8 }; + for (size_t f = 0; f < sizeof(fmts); f++) + for (int fl = 0; fl < 2; fl++) { + yapf_texture_t d = desc(300, 200, 4, 0); + d.payload = YAPF_PAYLOAD_BLOCKS; d.format = fmts[f]; d.mip_levels = 3; + if (f == 3) { d.shape = YAPF_SHAPE_VOLUME; d.depth = 5; } + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 1); + char what[48]; + sprintf(what, "blocks 0x%02x%s", fmts[f], fl ? " packed" : ""); + size_t len; + free(tex_roundtrip(t, fl, &len, what)); + yapf_tex_free(t); + } +} + +/* Recomputes both header checks after editing a file. */ +static void reseal(uint8_t *f, size_t front) { + uint32_t a = yapf_checksum(f + 64, front - 64), b; + f[0x38] = (uint8_t)a; f[0x39] = (uint8_t)(a >> 8); f[0x3A] = (uint8_t)(a >> 16); f[0x3B] = (uint8_t)(a >> 24); + b = yapf_checksum(f, 0x3C); + f[0x3C] = (uint8_t)b; f[0x3D] = (uint8_t)(b >> 8); f[0x3E] = (uint8_t)(b >> 16); f[0x3F] = (uint8_t)(b >> 24); +} + +/* Flips, truncation, scribbles: never crash; truncation always fails. */ +static void v2_fuzz(const uint8_t *file, size_t len, const char *what) { + uint8_t *bad = (uint8_t *)malloc(len); + int truncated_ok = 1, err; + for (int it = 0; it < 2000; it++) { + memcpy(bad, file, len); + size_t n = len; + switch (it % 3) { + case 0: for (int k = 0; k < 1 + it % 7; k++) bad[rnd() % len] ^= (uint8_t)(1u << (rnd() % 8)); break; + case 1: n = rnd() % len; break; + default: bad[rnd() % len] = (uint8_t)rnd(); break; + } + int fl = (it & 4) ? YAPF_LOAD_TRUST : 0; + yapf_texture_t *x = yapf_tex_load_memory(bad, n, fl, (it & 1) ? 2 : 1, &err); + if (it % 3 == 1 && x) truncated_ok = 0; + yapf_tex_free(x); + if (it % 50 == 0) yapf_free(yapf_load_memory(bad, n)); + } + CHECK(truncated_ok, "truncated v2 files are rejected (%s)", what); + free(bad); +} + +static void v2_integrity(void) { + yapf_texture_t d = desc(150, 90, 4, YAPF_SAMPLE_F16); + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 2); + size_t len; + uint8_t *file = tex_roundtrip(t, YAPF_ENC_ENTROPY, &len, "integrity setup"); + if (!file) { yapf_tex_free(t); return; } + uint8_t *bad = (uint8_t *)malloc(len); + size_t front = 64 + 16; /* header + one surface entry */ + int err; + + /* a flipped bit in tile data is caught by its yapf-sum ... */ + memcpy(bad, file, len); + bad[len - 10] ^= 0x10; + CHECK(!yapf_tex_load_memory(bad, len, 0, 1, &err) && err == YAPF_ERR_CHECKSUM, + "corrupt tile data reports YAPF_ERR_CHECKSUM (%d)", err); + yapf_texture_t *trusted = yapf_tex_load_memory(bad, len, YAPF_LOAD_TRUST, 1, &err); + yapf_tex_free(trusted); /* may or may not decode; must not crash */ + /* ... and in the header by the header check */ + memcpy(bad, file, len); + bad[0x0C] ^= 1; + CHECK(!yapf_tex_load_memory(bad, len, 0, 1, &err) && err == YAPF_ERR_CHECKSUM, + "corrupt header reports YAPF_ERR_CHECKSUM"); + /* an unknown needs bit: refused as unsupported */ + memcpy(bad, file, len); + bad[0x1F] |= 0x80; + reseal(bad, front); + CHECK(!yapf_tex_load_memory(bad, len, 0, 1, &err) && err == YAPF_ERR_UNSUPPORTED, + "unknown needs bit is refused (%d)", err); + /* a feature in use but not announced: corrupt */ + memcpy(bad, file, len); + bad[0x1C] &= (uint8_t)~YAPF_NEED_WIDE; + reseal(bad, front); + CHECK(!yapf_tex_load_memory(bad, len, 0, 1, &err) && err == YAPF_ERR_CORRUPT, + "unannounced feature is refused (%d)", err); + /* the old API refuses what it cannot represent */ + CHECK(yapf_load_memory(file, len) == NULL, "yapf_load refuses half-float textures"); + + free(bad); + v2_fuzz(file, len, "half float + entropy"); + free(file); + yapf_tex_free(t); + + /* 8-bit RGBA with entropy planes (decoded several streams at once) */ + d = desc(140, 75, 4, 0); + t = yapf_tex_create(&d); + tex_fill(t, 2); + file = tex_roundtrip(t, YAPF_ENC_ENTROPY, &len, "fuzz setup rgba8"); + if (file) v2_fuzz(file, len, "rgba8 + entropy"); + free(file); + /* BC7 blocks with packed tiles */ + yapf_texture_t *c = NULL; + if (yapf_tex_compress(t, YAPF_GPU_BC7, &c) == YAPF_OK) { + file = tex_roundtrip(c, YAPF_ENC_ENTROPY, &len, "fuzz setup bc7"); + if (file) v2_fuzz(file, len, "packed BC7"); + free(file); + } + yapf_tex_free(c); + yapf_tex_free(t); +} + +/* Streaming: a file prefix holds the small mips; tiles decode alone. */ +static void v2_streaming(void) { + yapf_texture_t d = desc(300, 260, 4, 0); + d.shape = YAPF_SHAPE_CUBE; d.width = d.height = 260; d.mip_levels = 5; + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 2); + void *buf = NULL; + size_t len = 0; + yapf_tex_encode(t, 0, &buf, &len); + uint8_t *file = (uint8_t *)buf; + int ok = 1, err = 0; + for (uint32_t m = 0; m < 5 && ok; m++) { + yapf_texture_t *probe = yapf_tex_open(file, len, 0, &err); + uint64_t need = probe ? yapf_tex_stream_bytes(probe, m) : 0; + yapf_tex_free(probe); + yapf_texture_t *s = yapf_tex_open(file, (size_t)need, 0, &err); + ok = s != NULL && need <= len && (m > 0 || need == len); + for (uint32_t f = 0; ok && f < 6; f++) + for (uint32_t mm = 0; ok && mm < 5; mm++) { + uint32_t sf = yapf_tex_surface(s, 0, 0, f, mm); + int rc = yapf_tex_decode_surface(s, file, (size_t)need, sf, 1); + if (mm >= m) ok = rc == YAPF_OK && + memcmp(s->data[sf], t->data[sf], yapf_tex_surface_bytes(t, mm)) == 0; + else ok = rc == YAPF_ERR_TRUNCATED; + } + yapf_tex_free(s); + } + CHECK(ok, "stream prefixes decode exactly their small mips"); + + /* tile decode equals the matching part of the full surface */ + yapf_texture_t *s = yapf_tex_open(file, len, 0, &err); + uint32_t sf = yapf_tex_surface(s, 0, 0, 3, 0), tx, ty; + yapf_tex_tile_grid(s, 0, &tx, &ty); + uint8_t tile[64 * 64 * 4]; + ok = s && tx == 5 && ty == 5; + for (uint32_t y = 0; ok && y < ty; y++) + for (uint32_t x = 0; ok && x < tx; x++) { + uint32_t px, py, pw, ph; + ok = yapf_tex_decode_tile(s, file, len, sf, x, y, 0, tile, 64 * 4, &px, &py, &pw, &ph) == YAPF_OK; + for (uint32_t r = 0; ok && r < ph; r++) + ok = memcmp(tile + r * 64 * 4, (uint8_t *)t->data[sf] + ((size_t)(py + r) * 260 + px) * 4, + (size_t)pw * 4) == 0; + } + CHECK(ok, "single tiles decode to the same pixels"); + yapf_tex_free(s); + yapf_free_buffer(buf); + yapf_tex_free(t); +} + +/* v1 and v2 work through either API where the data allows it. */ +static void v2_compat(const char *sample) { + yapf_texture_t *t = yapf_tex_load(sample, 0, 0, NULL); + if (t) { + uint32_t h = fnv((const uint8_t *)t->data[0], yapf_tex_surface_bytes(t, 0)); + CHECK(h == SAMPLE_HASH, "yapf_tex_load reads the v1 sample"); + size_t len; + uint8_t *file = tex_roundtrip(t, YAPF_ENC_ENTROPY, &len, "sample as v2"); + yapf_image_t *img = file ? yapf_load_memory(file, len) : NULL; + CHECK(img && img->width == 512 && img->channels == 4 && + fnv(img->pixels, 512u * 512u * 4u) == SAMPLE_HASH, "yapf_load reads an 8-bit v2 file"); + yapf_free(img); + free(file); + yapf_tex_free(t); + } + /* cube: the v1 API sees face 0 with its mips */ + yapf_texture_t d = desc(40, 40, 3, 0); + d.shape = YAPF_SHAPE_CUBE; d.mip_levels = 3; + yapf_texture_t *c = yapf_tex_create(&d); + tex_fill(c, 1); + size_t len; + uint8_t *file = tex_roundtrip(c, 0, &len, "cube for v1 API"); + yapf_image_t *img = file ? yapf_load_memory_mt(file, len, 2) : NULL; + CHECK(img && img->mip_levels == 3 && memcmp(img->mips[2], c->data[2], 10 * 10 * 3) == 0 && + memcmp(img->pixels, c->data[0], 40 * 40 * 3) == 0, "yapf_load reads face 0 of a cube"); + yapf_free(img); + free(file); + yapf_tex_free(c); +} + +static double psnr8(const uint8_t *a, int sa, const uint8_t *b, int sb, size_t n, int ch) { + double se = 0; + for (size_t i = 0; i < n; i++) + for (int c = 0; c < ch; c++) { + double v = (double)a[i * sa + c] - b[i * sb + c]; + se += v * v; + } + if (se == 0) return 99.0; + return 10.0 * log10(255.0 * 255.0 * (double)(n * ch) / se); +} + +static void v2_gpu(void) { + const uint32_t w = 61, h = 47; /* not multiples of 4 */ + yapf_texture_t d = desc(w, h, 4, 0); + yapf_texture_t *t = yapf_tex_create(&d); + uint8_t *p = (uint8_t *)t->data[0]; + for (uint32_t y = 0; y < h; y++) + for (uint32_t x = 0; x < w; x++) { + uint8_t *q = p + ((size_t)y * w + x) * 4; + q[0] = (uint8_t)(x * 4); q[1] = (uint8_t)(y * 5); q[2] = (uint8_t)(255 - x * 2); + q[3] = (uint8_t)(((x / 8 + y / 8) & 1) ? 255 : 40 + x); + } + static const struct { uint8_t f; int ch; double min; } F[] = { + { YAPF_GPU_BC1, 3, 30 }, { YAPF_GPU_BC3, 4, 30 }, { YAPF_GPU_BC7, 4, 34 }, + { YAPF_GPU_BC4, 1, 38 }, { YAPF_GPU_BC5, 2, 38 } + }; + uint8_t *out = (uint8_t *)malloc((size_t)w * h * 4); + for (size_t i = 0; i < sizeof(F) / sizeof(F[0]); i++) { + yapf_texture_t *c = NULL; + int rc = yapf_tex_compress(t, F[i].f, &c); + int ok = rc == YAPF_OK && c->payload == YAPF_PAYLOAD_BLOCKS && + yapf_blocks_decode(F[i].f, c->data[0], w, h, out) == YAPF_OK; + double q = 0; + if (ok && F[i].f == YAPF_GPU_BC1) { + /* BC1 keeps alpha only as cut-out: compare opaque texels */ + for (size_t k = 0; k < (size_t)w * h; k++) if (p[k * 4 + 3] < 128) memcpy(out + k * 4, p + k * 4, 3); + } + if (ok) q = psnr8(p, 4, out, 4, (size_t)w * h, F[i].ch); + CHECK(ok && q >= F[i].min, "GPU format 0x%02x encodes (PSNR %.1f dB)", F[i].f, q); + if (c) { + size_t len; + char what[32]; + sprintf(what, "compressed 0x%02x", F[i].f); + free(tex_roundtrip(c, YAPF_ENC_ENTROPY, &len, what)); + } + yapf_tex_free(c); + } + CHECK(yapf_blocks_decode(YAPF_GPU_ASTC_4x4, out, 4, 4, out) == YAPF_ERR_UNSUPPORTED, + "ASTC blocks are carried, not decoded"); + free(out); + yapf_tex_free(t); + + /* BC6H: relative error of a smooth HDR sky gradient */ + yapf_texture_t dh = desc(32, 32, 3, YAPF_SAMPLE_F16); + yapf_texture_t *th = yapf_tex_create(&dh), *ch = NULL; + uint16_t *hp = (uint16_t *)th->data[0]; + for (int i = 0; i < 32 * 32; i++) + for (int k = 0; k < 3; k++) hp[i * 3 + k] = yapf_float_to_half((0.3f + 0.4f * (float)k) * + (1.0f + (float)(i % 32) * 0.02f + (float)(i / 32) * 0.15f)); + int ok = yapf_tex_compress(th, YAPF_GPU_BC6H, &ch) == YAPF_OK; + uint16_t back[32 * 32 * 3]; + double worst = 0; + if (ok) ok = yapf_blocks_decode(YAPF_GPU_BC6H, ch->data[0], 32, 32, back) == YAPF_OK; + for (int i = 0; ok && i < 32 * 32 * 3; i++) { + float a = yapf_half_to_float(hp[i]), b = yapf_half_to_float(back[i]); + double r = (a > b ? a - b : b - a) / a; + if (r > worst) worst = r; + } + CHECK(ok && worst < 0.05, "BC6H encodes HDR within 5%% (worst %.4f)", worst); + yapf_tex_free(ch); + yapf_tex_free(th); +} + +static void v2_mips(void) { + /* sRGB: a black / white checker averages to sRGB 188, not 128 */ + yapf_texture_t d = desc(64, 64, 4, 0); + d.flags = YAPF_FLAG_SRGB; + yapf_texture_t *t = yapf_tex_create(&d); + uint8_t *p = (uint8_t *)t->data[0]; + for (int i = 0; i < 64 * 64; i++) { + uint8_t v = ((i % 64 + i / 64) & 1) ? 255 : 0; + p[i * 4] = p[i * 4 + 1] = p[i * 4 + 2] = v; + p[i * 4 + 3] = v; /* alpha stays linear */ + } + int ok = yapf_tex_build_mips(t, 0, 0) == YAPF_OK && t->mip_levels == 7; + const uint8_t *m1 = ok ? (const uint8_t *)t->data[1] : NULL; + CHECK(ok && m1[0] == 188 && m1[3] == 128 && (t->mip_filter & YAPF_MIP_LINEAR_LIGHT), + "sRGB mips average in linear light (got %d, alpha %d)", m1 ? m1[0] : -1, m1 ? m1[3] : -1); + yapf_tex_free(t); + + /* alpha test: speckled foliage (30 % of texels opaque). A plain box + * filter thins it out with distance; with alpha_ref = 128 every mip + * keeps about the same share of texels above 128. */ + double plain[5] = {0}, kept[5] = {0}; + for (int pass = 0; pass < 2; pass++) { + d = desc(128, 128, 4, 0); + t = yapf_tex_create(&d); + p = (uint8_t *)t->data[0]; + rng = 99; + for (int i = 0; i < 128 * 128; i++) p[i * 4 + 3] = rnd() % 100 < 30 ? 255 : 0; + ok = yapf_tex_build_mips(t, 5, pass ? 128 : 0) == YAPF_OK && t->mip_levels == 5; + if (pass) ok = ok && (t->flags & YAPF_FLAG_COVERAGE) && t->alpha_ref == 128; + for (uint32_t m = 1; ok && m < 5; m++) { + uint32_t w, h, n = 0; + yapf_tex_mip_size(t, m, &w, &h, NULL); + const uint8_t *q = (const uint8_t *)t->data[m]; + for (uint32_t i = 0; i < w * h; i++) n += q[i * 4 + 3] > 128; + (pass ? kept : plain)[m] = (double)n / (w * h); + } + if (pass) { + size_t len; + free(tex_roundtrip(t, 0, &len, "mipped texture")); + } + yapf_tex_free(t); + } + for (uint32_t m = 1; ok && m < 5; m++) ok = kept[m] > 0.2 && kept[m] < 0.4; + CHECK(ok && plain[2] < 0.15, "coverage-preserving mips keep alpha-test coverage " + "(mip 2: %.2f, plain box %.2f)", kept[2], plain[2]); + + /* float volume: 2×2×2 box */ + d = desc(4, 4, 1, YAPF_SAMPLE_F32); + d.shape = YAPF_SHAPE_VOLUME; d.depth = 4; + t = yapf_tex_create(&d); + float *f = (float *)t->data[0]; + for (int i = 0; i < 64; i++) f[i] = (float)i; + ok = yapf_tex_build_mips(t, 0, 0) == YAPF_OK && t->mip_levels == 3; + CHECK(ok && ((float *)t->data[1])[0] == (0 + 1 + 4 + 5 + 16 + 17 + 20 + 21) / 8.0f && + ((float *)t->data[2])[0] == 31.5f, "volume mips filter in 3D"); + yapf_tex_free(t); +} + +static void v2_misc(void) { + int ok = 1; + for (uint32_t h = 0; h < 0x10000 && ok; h++) { + if ((h & 0x7C00) == 0x7C00 && (h & 0x3FF)) continue; /* NaN payloads */ + ok = yapf_float_to_half(yapf_half_to_float((uint16_t)h)) == h; + } + CHECK(ok, "every half survives half -> float -> half"); + CHECK(yapf_float_to_half(1.0f) == 0x3C00 && yapf_float_to_half(65520.0f) == 0x7C00 && + yapf_float_to_half(1e-8f) == 0 && yapf_float_to_half(-2.0f) == 0xC000, "float -> half rounding"); + /* fixed vectors pin yapf-sum to the specification */ + CHECK(yapf_checksum("", 0) == 0xd7550ed0u && yapf_checksum("YAPF", 4) == 0xc8cc41d4u && + yapf_checksum("The quick brown fox jumps over the lazy dog", 43) == 0xbce4f410u, + "yapf-sum test vectors"); + /* encoding is deterministic */ + yapf_texture_t d = desc(90, 70, 3, YAPF_SAMPLE_U16); + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 2); + void *a = NULL, *b = NULL; + size_t la = 0, lb = 0; + yapf_tex_encode(t, YAPF_ENC_ENTROPY, &a, &la); + yapf_tex_encode(t, YAPF_ENC_ENTROPY, &b, &lb); + CHECK(a && la == lb && memcmp(a, b, la) == 0, "v2 encoding is deterministic"); + yapf_free_buffer(a); + yapf_free_buffer(b); + yapf_tex_free(t); + d = desc(10, 12, 4, 0); + d.shape = YAPF_SHAPE_CUBE; + CHECK(yapf_tex_create(&d) == NULL, "a cube must be square"); +} + int main(int argc, char **argv) { static const int sizes[][2] = { {1, 1}, {1, 7}, {7, 1}, {63, 63}, {64, 64}, {65, 65}, {127, 1}, @@ -197,6 +688,16 @@ int main(int argc, char **argv) { printf("note: sample %s not found, skipped\n", sample); } + v2_pixels(); + v2_layout(); + v2_blocks(); + v2_integrity(); + v2_streaming(); + v2_compat(sample); + v2_gpu(); + v2_mips(); + v2_misc(); + printf("%d tests, %d failures\n", tests, fails); return fails != 0; } diff --git a/tools/yapf_cli.c b/tools/yapf_cli.c index effec05..af60926 100644 --- a/tools/yapf_cli.c +++ b/tools/yapf_cli.c @@ -18,20 +18,17 @@ * ------------------------------------------------------------------------- * * yapf [options] convert; formats chosen by extension - * yapf info print header information + * yapf pack [options] + * yapf info print header, tables and notes + * yapf verify check every checksum and decode * yapf bench time decoding * yapf thumbnail [size] small preview (default 256) * yapf compare [more…] size and decode speed: original vs YAPF * - * Reads PNG, JPEG, BMP, TGA, GIF (first frame), PSD (composite), PNM, HDR - * (tone-clamped) and YAPF. Writes YAPF, PNG, BMP, TGA and JPEG. - * - * Options for writing YAPF: - * --mips also store a full mip chain (2×2 box filter) - * --linear mark pixels as linear instead of sRGB - * --premult mark alpha as premultiplied - * Options for reading YAPF: - * --mip N export mip level N instead of the base image + * Reads PNG (8 / 16-bit), JPEG, BMP, TGA, GIF (first frame), PSD, PNM, + * HDR (float) and YAPF. Writes YAPF, PNG, BMP, TGA, JPEG and HDR. + * Plain 8-bit 2D images are written as YAPF version 1; anything that + * needs version 2 (see `yapf` without arguments) is written as version 2. * * Build: cc -O2 -std=c99 -Ithird_party tools/yapf_cli.c yapf.c -o yapf -lm -pthread */ @@ -84,124 +81,532 @@ static double now_s(void) { #endif } -static uint8_t gpu_hint(int ch, int srgb) { - switch (ch) { - case 1: return YAPF_GPU_R8; - case 2: return YAPF_GPU_RG8; - case 3: return srgb ? YAPF_GPU_SRGB8 : YAPF_GPU_RGB8; - default: return srgb ? YAPF_GPU_SRGB8_A8 : YAPF_GPU_RGBA8; - } + +/* ── Options ─────────────────────────────────────────────────────────── */ + +typedef struct { + int mips, srgb, premult, v2, entropy, eight_bit, float32; + int coverage; /* alpha_ref, 0 = off */ + int format; /* block format, -1 = none */ + int normal; /* YAPF_NORMAL_* */ + int linear_alpha; + int primaries, transfer; + int have_map; + uint8_t map[4]; + double frame_ms; + int loops; + /* export selection */ + int mip, layer, face, frame, slice; + const char *author, *license, *source; + const char *notes[16]; + int nnotes; +} opts_t; + +static const struct { const char *name; uint8_t v; } ch_names[] = { + { "r", YAPF_CH_RED }, { "g", YAPF_CH_GREEN }, { "b", YAPF_CH_BLUE }, { "a", YAPF_CH_ALPHA }, + { "luma", YAPF_CH_LUMA }, { "nx", YAPF_CH_NORMAL_X }, { "ny", YAPF_CH_NORMAL_Y }, + { "nz", YAPF_CH_NORMAL_Z }, { "rough", YAPF_CH_ROUGHNESS }, { "gloss", YAPF_CH_GLOSS }, + { "metal", YAPF_CH_METALNESS }, { "ao", YAPF_CH_OCCLUSION }, { "height", YAPF_CH_HEIGHT }, + { "spec", YAPF_CH_SPECULAR }, { "emissive", YAPF_CH_EMISSIVE }, { "opacity", YAPF_CH_OPACITY }, + { "mask", YAPF_CH_MASK }, { "cavity", YAPF_CH_CAVITY }, { "thickness", YAPF_CH_THICKNESS }, + { "id", YAPF_CH_ID }, { "-", YAPF_CH_UNUSED }, { "default", YAPF_CH_DEFAULT } +}; + +static const char *ch_name(uint8_t v) { + for (size_t i = 0; i < sizeof(ch_names) / sizeof(ch_names[0]); i++) + if (ch_names[i].v == v) return ch_names[i].name; + return "?"; } -/* Full mip chain with a 2×2 box filter (edges clamp for odd sizes). */ -static int build_mips(yapf_image_t *img) { - int levels = 1; - while (levels < (int)YAPF_MAX_MIPS && - ((img->width >> levels) > 0 || (img->height >> levels) > 0)) - levels++; - img->mips = (uint8_t **)calloc((size_t)levels, sizeof(uint8_t *)); - if (!img->mips) return -1; - img->mips[0] = img->pixels; - img->mip_levels = (uint8_t)levels; - - int ch = img->channels; - for (int m = 1; m < levels; m++) { - uint32_t pw = img->width >> (m - 1), ph = img->height >> (m - 1); - uint32_t w = img->width >> m, h = img->height >> m; - if (pw < 1) pw = 1; - if (ph < 1) ph = 1; - if (w < 1) w = 1; - if (h < 1) h = 1; - uint8_t *src = img->mips[m - 1]; - uint8_t *dst = (uint8_t *)malloc((size_t)w * h * ch); - if (!dst) return -1; - for (uint32_t y = 0; y < h; y++) - for (uint32_t x = 0; x < w; x++) - for (int c = 0; c < ch; c++) { - uint32_t x0 = 2 * x < pw ? 2 * x : pw - 1, x1 = 2 * x + 1 < pw ? 2 * x + 1 : pw - 1; - uint32_t y0 = 2 * y < ph ? 2 * y : ph - 1, y1 = 2 * y + 1 < ph ? 2 * y + 1 : ph - 1; - unsigned s = src[((size_t)y0 * pw + x0) * ch + c] + src[((size_t)y0 * pw + x1) * ch + c] - + src[((size_t)y1 * pw + x0) * ch + c] + src[((size_t)y1 * pw + x1) * ch + c]; - dst[((size_t)y * w + x) * ch + c] = (uint8_t)((s + 2) / 4); - } - img->mips[m] = dst; - } - return 0; -} - -static int write_image(const char *path, const uint8_t *px, int w, int h, int ch) { - if (ends_with(path, ".png")) return stbi_write_png(path, w, h, ch, px, w * ch); - if (ends_with(path, ".bmp")) return stbi_write_bmp(path, w, h, ch, px); - if (ends_with(path, ".tga")) return stbi_write_tga(path, w, h, ch, px); - if (ends_with(path, ".jpg") || ends_with(path, ".jpeg")) - return stbi_write_jpg(path, w, h, ch, px, 95); - fprintf(stderr, "yapf: unsupported output format: %s (use .yapf .png .bmp .tga .jpg)\n", path); - return 0; -} +static const struct { const char *name; uint8_t v; } fmt_names[] = { + { "bc1", YAPF_GPU_BC1 }, { "bc3", YAPF_GPU_BC3 }, { "bc4", YAPF_GPU_BC4 }, + { "bc5", YAPF_GPU_BC5 }, { "bc6h", YAPF_GPU_BC6H }, { "bc7", YAPF_GPU_BC7 } +}; static const char *gpu_name(int g) { - static const char *n[] = { "RGBA8", "RGB8", "RG8", "R8", "SRGB8_A8", "SRGB8" }; - return g >= 0 && g < 6 ? n[g] : "unknown"; + static const char *px[] = { "RGBA8", "RGB8", "RG8", "R8", "SRGB8_A8", "SRGB8", + "RGBA16", "RGB16", "RG16", "R16", "RGBA16F", "RGB16F", "RG16F", "R16F", + "RGBA32F", "RGB32F", "RG32F", "R32F" }; + static const char *bc[] = { "BC1", "BC2", "BC3", "BC4", "BC4_SNORM", "BC5", "BC5_SNORM", + "BC6H", "BC6H_SF", "BC7" }; + static const char *etc[] = { "ETC2_RGB8", "ETC2_RGB8A1", "ETC2_RGBA8", "EAC_R11", + "EAC_R11_SNORM", "EAC_RG11", "EAC_RG11_SNORM" }; + static const char *astc[] = { "ASTC_4x4", "ASTC_5x4", "ASTC_5x5", "ASTC_6x5", "ASTC_6x6", + "ASTC_8x5", "ASTC_8x6", "ASTC_8x8", "ASTC_10x5", "ASTC_10x6", "ASTC_10x8", + "ASTC_10x10", "ASTC_12x10", "ASTC_12x12" }; + if (g >= 0 && g < 18) return px[g]; + if (g >= 0x40 && g < 0x4A) return bc[g - 0x40]; + if (g >= 0x50 && g < 0x57) return etc[g - 0x50]; + if (g >= 0x60 && g < 0x6E) return astc[g - 0x60]; + return "unknown"; +} + +static uint8_t gpu_hint(int ch, int srgb, int sample) { + static const uint8_t u8[2][4] = { { YAPF_GPU_R8, YAPF_GPU_RG8, YAPF_GPU_RGB8, YAPF_GPU_RGBA8 }, + { YAPF_GPU_R8, YAPF_GPU_RG8, YAPF_GPU_SRGB8, YAPF_GPU_SRGB8_A8 } }; + static const uint8_t wide[3][4] = { + { YAPF_GPU_R16, YAPF_GPU_RG16, YAPF_GPU_RGB16, YAPF_GPU_RGBA16 }, + { YAPF_GPU_R16F, YAPF_GPU_RG16F, YAPF_GPU_RGB16F, YAPF_GPU_RGBA16F }, + { YAPF_GPU_R32F, YAPF_GPU_RG32F, YAPF_GPU_RGB32F, YAPF_GPU_RGBA32F } }; + if (sample == YAPF_SAMPLE_U8) return u8[srgb ? 1 : 0][ch - 1]; + return wide[sample - 1][ch - 1]; +} + +static int parse_map(const char *s, uint8_t *map) { + memset(map, 0, 4); + for (int c = 0; c < 4 && *s; c++) { + const char *e = strchr(s, ','); + size_t n = e ? (size_t)(e - s) : strlen(s); + size_t i; + for (i = 0; i < sizeof(ch_names) / sizeof(ch_names[0]); i++) + if (strlen(ch_names[i].name) == n && strncmp(ch_names[i].name, s, n) == 0) break; + if (i == sizeof(ch_names) / sizeof(ch_names[0])) return 0; + map[c] = ch_names[i].v; + s += n + (e ? 1 : 0); + } + return 1; +} + +/* ── Reading any input into a texture ────────────────────────────────── */ + +static yapf_texture_t *new_tex(uint32_t w, uint32_t h, int ch, int sample) { + yapf_texture_t d; + memset(&d, 0, sizeof(d)); + d.width = w; d.height = h; d.depth = 1; d.layers = 1; d.frames = 1; d.mip_levels = 1; + d.channels = (uint8_t)ch; d.sample = (uint8_t)sample; + return yapf_tex_create(&d); +} + +static yapf_texture_t *read_input(const char *path, const opts_t *o, int *from_yapf) { + *from_yapf = 0; + if (is_yapf(path)) { + int err = 0; + yapf_texture_t *t = yapf_tex_load(path, 0, 0, &err); + if (!t) fprintf(stderr, "yapf: cannot read %s (error %d)\n", path, err); + *from_yapf = 1; + return t; + } + int w, h, ch; + yapf_texture_t *t = NULL; + if (stbi_is_hdr(path)) { + float *f = stbi_loadf(path, &w, &h, &ch, 0); + if (!f) { fprintf(stderr, "yapf: cannot read %s: %s\n", path, stbi_failure_reason()); return NULL; } + int sample = o->float32 ? YAPF_SAMPLE_F32 : YAPF_SAMPLE_F16; + t = new_tex((uint32_t)w, (uint32_t)h, ch, sample); + if (t) { + size_t n = (size_t)w * h * ch; + if (sample == YAPF_SAMPLE_F32) memcpy(t->data[0], f, n * 4); + else for (size_t i = 0; i < n; i++) ((uint16_t *)t->data[0])[i] = yapf_float_to_half(f[i]); + t->transfer = YAPF_TRANSFER_LINEAR; + } + stbi_image_free(f); + } else if (stbi_is_16_bit(path) && !o->eight_bit) { + stbi_us *p = stbi_load_16(path, &w, &h, &ch, 0); + if (!p) { fprintf(stderr, "yapf: cannot read %s: %s\n", path, stbi_failure_reason()); return NULL; } + t = new_tex((uint32_t)w, (uint32_t)h, ch, YAPF_SAMPLE_U16); + if (t) memcpy(t->data[0], p, (size_t)w * h * ch * 2); + stbi_image_free(p); + } else { + stbi_uc *p = stbi_load(path, &w, &h, &ch, 0); + if (!p) { fprintf(stderr, "yapf: cannot read %s: %s\n", path, stbi_failure_reason()); return NULL; } + t = new_tex((uint32_t)w, (uint32_t)h, ch, YAPF_SAMPLE_U8); + if (t) memcpy(t->data[0], p, (size_t)w * h * ch); + stbi_image_free(p); + } + if (!t) fprintf(stderr, "yapf: %s: unsupported size or out of memory\n", path); + return t; +} + +/* Several same-sized inputs into one cube / array / volume / animation. */ +static yapf_texture_t *assemble(const char *kind, int n, char **paths, const opts_t *o) { + yapf_texture_t *first = NULL, *t = NULL; + for (int i = 0; i < n; i++) { + int y; + yapf_texture_t *in = read_input(paths[i], o, &y); + if (!in) { yapf_tex_free(t); yapf_tex_free(first); return NULL; } + if (!t) { + yapf_texture_t d = *in; + d.mip_levels = 1; + if (strcmp(kind, "cube") == 0) { + if (n % 6) { fprintf(stderr, "yapf: a cube needs 6 faces per layer (+X -X +Y -Y +Z -Z)\n"); yapf_tex_free(in); return NULL; } + d.shape = YAPF_SHAPE_CUBE; d.layers = (uint16_t)(n / 6); + } else if (strcmp(kind, "array") == 0) { + d.layers = (uint16_t)n; + } else if (strcmp(kind, "volume") == 0) { + d.shape = YAPF_SHAPE_VOLUME; d.depth = (uint32_t)n; + } else if (strcmp(kind, "anim") == 0) { + d.frames = (uint16_t)n; + } else { + fprintf(stderr, "yapf: pack kind must be cube, array, volume or anim\n"); + yapf_tex_free(in); + return NULL; + } + t = yapf_tex_create(&d); + if (!t) { fprintf(stderr, "yapf: invalid %s (cube faces must be square)\n", kind); yapf_tex_free(in); return NULL; } + first = in; + } else if (in->width != t->width || in->height != t->height || + in->channels != t->channels || in->sample != t->sample) { + fprintf(stderr, "yapf: %s does not match the size and channels of %s\n", paths[i], paths[0]); + yapf_tex_free(in); yapf_tex_free(t); yapf_tex_free(first); + return NULL; + } + size_t bytes = yapf_tex_surface_bytes(in, 0); + if (t->shape == YAPF_SHAPE_VOLUME) memcpy((uint8_t *)t->data[0] + bytes * i, in->data[0], bytes); + else if (t->frames > 1) memcpy(t->data[yapf_tex_surface(t, (uint32_t)i, 0, 0, 0)], in->data[0], bytes); + else if (t->shape == YAPF_SHAPE_CUBE) memcpy(t->data[yapf_tex_surface(t, 0, (uint32_t)i / 6, (uint32_t)i % 6, 0)], in->data[0], bytes); + else memcpy(t->data[yapf_tex_surface(t, 0, (uint32_t)i, 0, 0)], in->data[0], bytes); + if (in != first) yapf_tex_free(in); + } + if (t && t->frames > 1) + for (int f = 0; f < t->frames; f++) t->frame_us[f] = (uint32_t)(o->frame_ms * 1000.0 + 0.5); + yapf_tex_free(first); + return t; +} + +/* ── Writing .yapf ───────────────────────────────────────────────────── */ + +static int needs_v2(const yapf_texture_t *t, const opts_t *o) { + return o->v2 || o->entropy || o->format >= 0 || o->coverage || o->normal || o->have_map || + o->linear_alpha || o->primaries || o->transfer || o->author || o->license || + o->source || o->nnotes || t->sample != YAPF_SAMPLE_U8 || t->shape != YAPF_SHAPE_2D || + t->layers > 1 || t->frames > 1 || t->payload != YAPF_PAYLOAD_PIXELS || t->note_count; +} + +static int write_yapf(yapf_texture_t *t, const char *out, const opts_t *o, int from_yapf) { + if (!from_yapf) { + t->flags = (uint8_t)((o->srgb && t->sample == YAPF_SAMPLE_U8 && !o->normal ? YAPF_FLAG_SRGB : 0) | + (o->premult ? YAPF_FLAG_PREMULT_ALPHA : 0)); + t->format = gpu_hint(t->channels, t->flags & YAPF_FLAG_SRGB, t->sample); + } + if (o->normal) { + t->normal = (uint8_t)o->normal; + t->channel_map[0] = YAPF_CH_NORMAL_X; + t->channel_map[1] = YAPF_CH_NORMAL_Y; + if (t->channels >= 3) t->channel_map[2] = YAPF_CH_NORMAL_Z; + else t->normal |= YAPF_NORMAL_XY; + t->flags &= (uint8_t)~YAPF_FLAG_SRGB; + } + if (o->have_map) memcpy(t->channel_map, o->map, 4); + if (o->linear_alpha && (t->channels == 2 || t->channels == 4)) + t->linear_mask |= (uint8_t)(1u << (t->channels - 1)); + if (o->primaries) t->primaries = (uint8_t)o->primaries; + if (o->transfer) t->transfer = (uint8_t)o->transfer; + if (o->frame_ms > 0 && t->frames > 1) + for (int f = 0; f < t->frames; f++) t->frame_us[f] = (uint32_t)(o->frame_ms * 1000.0 + 0.5); + if (o->loops >= 0) t->loop_count = (uint16_t)o->loops; + + if (o->mips && t->payload == YAPF_PAYLOAD_PIXELS) { + int rc = yapf_tex_build_mips(t, 0, (uint8_t)o->coverage); + if (rc != YAPF_OK) { + fprintf(stderr, "yapf: cannot build mips%s\n", o->coverage ? " (--coverage needs an alpha channel)" : ""); + return 1; + } + } + int v2 = needs_v2(t, o); + if (o->format >= 0) { + yapf_texture_t *c = NULL; + int rc = yapf_tex_compress(t, (uint8_t)o->format, &c); + if (rc != YAPF_OK) { + fprintf(stderr, "yapf: cannot compress to %s: %s\n", gpu_name(o->format), + o->format == YAPF_GPU_BC6H ? "BC6H needs a 16-bit or HDR (float) image" + : "LDR block formats need an 8-bit image"); + return 1; + } + opts_t rest = *o; /* the blocks are made: write them */ + rest.format = -1; + rest.mips = 0; + rest.v2 = 1; + int r2 = write_yapf(c, out, &rest, 1); + yapf_tex_free(c); + return r2; + } + if (!v2) { + /* 8-bit 2D: version 1, readable by every YAPF decoder */ + yapf_image_t img; + memset(&img, 0, sizeof(img)); + img.width = t->width; img.height = t->height; img.channels = t->channels; + img.gpu_format = t->format; img.flags = t->flags; img.mip_levels = t->mip_levels; + img.mips = (uint8_t **)t->data; + img.pixels = (uint8_t *)t->data[0]; + if (yapf_save(out, &img) != YAPF_OK) { fprintf(stderr, "yapf: cannot write %s\n", out); return 1; } + return 0; + } + if (!yapf_tex_find_note(t, "TOOL")) yapf_tex_add_note(t, "TOOL", "yapf command-line tool", 22); + if (o->author) yapf_tex_add_note(t, "AUTH", o->author, (uint32_t)strlen(o->author)); + if (o->license) yapf_tex_add_note(t, "LICN", o->license, (uint32_t)strlen(o->license)); + if (o->source) yapf_tex_add_note(t, "SRCE", o->source, (uint32_t)strlen(o->source)); + for (int i = 0; i < o->nnotes; i++) { + const char *eq = strchr(o->notes[i], '='); + if (!eq || eq - o->notes[i] != 4) { fprintf(stderr, "yapf: --note wants TAG=text with a 4-letter tag\n"); return 1; } + yapf_tex_add_note(t, o->notes[i], eq + 1, (uint32_t)strlen(eq + 1)); + } + if (yapf_tex_save(out, t, o->entropy ? YAPF_ENC_ENTROPY : 0) != YAPF_OK) { + fprintf(stderr, "yapf: cannot write %s\n", out); + return 1; + } + return 0; +} + +/* ── Exporting a surface to a common format ──────────────────────────── */ + +/* One surface slice as 8-bit pixels (*hdr = 0) or float RGB(A) (*hdr = 1). */ +static void *surface_view(const yapf_texture_t *t, uint32_t s, uint32_t slice, + uint32_t *w, uint32_t *h, int *ch, int *hdr) +{ + uint32_t m = s % t->mip_levels, d; + yapf_tex_mip_size(t, m, w, h, &d); + if (slice >= d) return NULL; + size_t n = (size_t)*w * *h, slice_bytes = yapf_tex_surface_bytes(t, m) / d; + const uint8_t *src = (const uint8_t *)t->data[s] + slice_bytes * slice; + *hdr = 0; + if (t->payload == YAPF_PAYLOAD_BLOCKS) { + int is6 = t->format == YAPF_GPU_BC6H || t->format == YAPF_GPU_BC6H_SF; + void *out = malloc(n * (is6 ? 6 : 4)); + if (!out) return NULL; + if (yapf_blocks_decode(t->format, src, *w, *h, out) != YAPF_OK) { free(out); return NULL; } + if (!is6) { *ch = 4; return out; } + float *f = (float *)malloc(n * 3 * sizeof(float)); + for (size_t i = 0; f && i < n * 3; i++) f[i] = yapf_half_to_float(((uint16_t *)out)[i]); + free(out); + *ch = 3; *hdr = 1; + return f; + } + *ch = t->channels; + if (t->sample == YAPF_SAMPLE_U8) { + uint8_t *o = (uint8_t *)malloc(n * *ch); + if (o) memcpy(o, src, n * *ch); + return o; + } + if (t->sample == YAPF_SAMPLE_U16) { + uint8_t *o = (uint8_t *)malloc(n * *ch); + for (size_t i = 0; o && i < n * *ch; i++) o[i] = (uint8_t)((((const uint16_t *)src)[i] + 128) / 257); + return o; + } + float *f = (float *)malloc(n * *ch * sizeof(float)); + for (size_t i = 0; f && i < n * *ch; i++) + f[i] = t->sample == YAPF_SAMPLE_F16 ? yapf_half_to_float(((const uint16_t *)src)[i]) + : ((const float *)src)[i]; + *hdr = 1; + return f; +} + +static uint8_t *to8(const float *f, size_t n) { + uint8_t *o = (uint8_t *)malloc(n); + for (size_t i = 0; o && i < n; i++) { + float v = f[i] < 0 ? 0 : f[i] > 1 ? 1 : f[i]; + o[i] = (uint8_t)(v * 255.0f + 0.5f); + } + return o; +} + +static int write_image(const char *path, const void *px, int w, int h, int ch, int hdr) { + if (ends_with(path, ".hdr")) { + if (!hdr) { + float *f = (float *)malloc((size_t)w * h * ch * sizeof(float)); + for (size_t i = 0; f && i < (size_t)w * h * ch; i++) f[i] = ((const uint8_t *)px)[i] / 255.0f; + int ok = f && stbi_write_hdr(path, w, h, ch, f); + free(f); + return ok; + } + return stbi_write_hdr(path, w, h, ch, (const float *)px); + } + uint8_t *tmp = NULL; + if (hdr) { + tmp = to8((const float *)px, (size_t)w * h * ch); + if (!tmp) return 0; + fprintf(stderr, "yapf: note: float pixels clamped to 0..1 for %s (use .hdr to keep them)\n", path); + px = tmp; + } + int ok = 0; + if (ends_with(path, ".png")) ok = stbi_write_png(path, w, h, ch, px, w * ch); + else if (ends_with(path, ".bmp")) ok = stbi_write_bmp(path, w, h, ch, px); + else if (ends_with(path, ".tga")) ok = stbi_write_tga(path, w, h, ch, px); + else if (ends_with(path, ".jpg") || ends_with(path, ".jpeg")) ok = stbi_write_jpg(path, w, h, ch, px, 95); + else fprintf(stderr, "yapf: unsupported output format: %s (use .yapf .png .bmp .tga .jpg .hdr)\n", path); + free(tmp); + return ok; +} + +static int export_image(const yapf_texture_t *t, const char *in, const char *out, const opts_t *o) { + uint32_t faces = yapf_tex_faces(t); + if ((uint32_t)o->mip >= t->mip_levels || (uint32_t)o->layer >= t->layers || + (uint32_t)o->face >= faces || (uint32_t)o->frame >= t->frames) { + fprintf(stderr, "yapf: %s has %u mip(s), %u layer(s), %u face(s), %u frame(s)\n", + in, t->mip_levels, t->layers, faces, t->frames); + return 1; + } + uint32_t s = yapf_tex_surface(t, (uint32_t)o->frame, (uint32_t)o->layer, (uint32_t)o->face, (uint32_t)o->mip); + uint32_t w, h; + int ch, hdr; + void *px = surface_view(t, s, (uint32_t)o->slice, &w, &h, &ch, &hdr); + if (!px) { + fprintf(stderr, "yapf: cannot export %s: %s\n", in, + t->payload == YAPF_PAYLOAD_BLOCKS ? "no built-in decoder for this block format (ETC2 / ASTC)" + : "no such slice, or out of memory"); + return 1; + } + if (t->sample == YAPF_SAMPLE_U16 && !ends_with(out, ".hdr")) + fprintf(stderr, "yapf: note: 16-bit samples written as 8-bit\n"); + int ok = write_image(out, px, (int)w, (int)h, ch, hdr); + free(px); + if (!ok) { fprintf(stderr, "yapf: cannot write %s\n", out); return 1; } + return 0; +} + +/* ── info / verify / bench / thumbnail ───────────────────────────────── */ + +static uint8_t *read_all(const char *path, size_t *n) { + FILE *f = fopen(path, "rb"); + if (!f) return NULL; + fseek(f, 0, SEEK_END); + long sz = ftell(f); + rewind(f); + uint8_t *buf = sz >= 0 ? (uint8_t *)malloc((size_t)sz + 1) : NULL; + if (buf && fread(buf, 1, (size_t)sz, f) != (size_t)sz) { free(buf); buf = NULL; } + fclose(f); + *n = buf ? (size_t)sz : 0; + return buf; +} + +static const char *err_name(int e) { + switch (e) { + case YAPF_ERR_INVALID: return "invalid argument"; + case YAPF_ERR_IO: return "cannot read the file"; + case YAPF_ERR_OOM: return "out of memory"; + case YAPF_ERR_CORRUPT: return "corrupt or not a YAPF file"; + case YAPF_ERR_UNSUPPORTED: return "needs a feature this decoder does not have"; + case YAPF_ERR_CHECKSUM: return "checksum mismatch (damaged file)"; + case YAPF_ERR_TRUNCATED: return "file is truncated"; + default: return "error"; + } } static int cmd_info(const char *path) { - FILE *f = fopen(path, "rb"); - uint8_t hdr[20]; - if (!f || fread(hdr, 1, 20, f) != 20 || memcmp(hdr, "YAPF", 4) != 0) { - fprintf(stderr, "yapf: %s is not a YAPF file\n", path); - if (f) fclose(f); + size_t n; + uint8_t *buf = read_all(path, &n); + int err = 0; + yapf_texture_t *t = buf ? yapf_tex_open(buf, n, 0, &err) : NULL; + if (!t) { + fprintf(stderr, "yapf: %s: %s\n", path, buf ? err_name(err) : "cannot open"); + free(buf); return 1; } - fseek(f, 0, SEEK_END); - long size = ftell(f); - fclose(f); - unsigned w = hdr[12] | hdr[13] << 8 | (unsigned)hdr[14] << 16 | (unsigned)hdr[15] << 24; - unsigned h = hdr[16] | hdr[17] << 8 | (unsigned)hdr[18] << 16 | (unsigned)hdr[19] << 24; + static const char *shape[] = { "2D", "cube", "volume" }; + static const char *sample[] = { "8-bit", "16-bit", "half float", "float" }; + static const char *prim[] = { "unspecified", "BT.709 / sRGB", "Display P3", "BT.2020", + "ACES AP0", "ACEScg", "Adobe RGB" }; + static const char *tf[] = { "unspecified", "sRGB", "linear", "gamma 2.2", "PQ", "HLG", "BT.1886" }; + static const char *mf[] = { "unknown", "box", "tent", "Kaiser", "Lanczos", "Mitchell", "authored" }; + uint32_t w, h, d; + yapf_tex_mip_size(t, 0, &w, &h, &d); printf("%s\n", path); - printf(" format version : %u\n", hdr[4]); - printf(" size : %u x %u, %u channel(s)\n", w, h, hdr[5]); - printf(" mip levels : %u\n", hdr[7]); - printf(" gpu format : %s\n", gpu_name(hdr[6])); - printf(" flags : %s%s\n", hdr[8] & YAPF_FLAG_SRGB ? "sRGB " : "linear ", - hdr[8] & YAPF_FLAG_PREMULT_ALPHA ? "premultiplied-alpha" : ""); - printf(" file size : %ld bytes (%.1f%% of raw)\n", size, - 100.0 * (double)size / ((double)w * h * (hdr[5] ? hdr[5] : 1))); - yapf_image_t *img = yapf_load(path); - printf(" decodes : %s\n", img ? "ok" : "FAILED (corrupt or unsupported)"); - yapf_free(img); - return img ? 0 : 1; + printf(" format version : %u\n", buf[4]); + printf(" size : %u x %u%s", w, h, t->shape == YAPF_SHAPE_VOLUME ? "" : "\n"); + if (t->shape == YAPF_SHAPE_VOLUME) printf(" x %u\n", d); + printf(" shape : %s, %u layer(s), %u frame(s)\n", shape[t->shape], t->layers, t->frames); + printf(" mip levels : %u\n", t->mip_levels); + if (t->payload == YAPF_PAYLOAD_BLOCKS) + printf(" payload : GPU blocks, %s (%u channel(s))\n", gpu_name(t->format), t->channels); + else + printf(" payload : lossless pixels, %u channel(s), %s, GPU hint %s\n", + t->channels, sample[t->sample], gpu_name(t->format)); + printf(" flags : %s%s%s\n", t->flags & YAPF_FLAG_SRGB ? "sRGB" : "linear", + t->flags & YAPF_FLAG_PREMULT_ALPHA ? ", premultiplied alpha" : "", + t->flags & YAPF_FLAG_COVERAGE ? ", coverage-preserving mips" : ""); + if (buf[4] == 2) { + uint32_t needs = (uint32_t)buf[0x1C] | (uint32_t)buf[0x1D] << 8; + printf(" needs : %s%s%s%s%s%s\n", needs ? "" : "nothing beyond 8-bit 2D", + needs & YAPF_NEED_WIDE ? "wide " : "", needs & YAPF_NEED_BLOCKS ? "blocks " : "", + needs & YAPF_NEED_ENTROPY ? "entropy " : "", needs & YAPF_NEED_LAYOUT ? "layout " : "", + needs & YAPF_NEED_FRAMES ? "frames" : ""); + printf(" channels :"); + for (int c = 0; c < t->channels; c++) + printf(" %s%s", ch_name(t->channel_map[c]), t->linear_mask & (1u << c) ? "(linear)" : ""); + printf("\n"); + if (t->normal) + printf(" normal map : %s, %s\n", + (t->normal & 3) == YAPF_NORMAL_GL ? "OpenGL (+Y)" : (t->normal & 3) == YAPF_NORMAL_DX ? "DirectX (-Y)" : "no convention", + t->normal & YAPF_NORMAL_XY ? "XY (Z reconstructed)" : t->normal & YAPF_NORMAL_OCT ? "octahedral" : "XYZ"); + printf(" colour : primaries %s, transfer %s\n", + t->primaries < 7 ? prim[t->primaries] : "?", t->transfer < 7 ? tf[t->transfer] : "?"); + if (t->mip_levels > 1) + printf(" mip filter : %s%s%s\n", (t->mip_filter & 0x7F) < 7 ? mf[t->mip_filter & 0x7F] : "?", + t->mip_filter & YAPF_MIP_LINEAR_LIGHT ? ", linear light" : "", + t->flags & YAPF_FLAG_COVERAGE ? "" : ""); + if (t->flags & YAPF_FLAG_COVERAGE) printf(" alpha ref : %u\n", t->alpha_ref); + if (t->frames > 1) + printf(" animation : %u frames, first %.1f ms, loops %s\n", t->frames, + t->frame_us ? t->frame_us[0] / 1000.0 : 0.0, t->loop_count ? "" : "forever"); + for (uint32_t i = 0; i < t->note_count; i++) { + const yapf_note_t *nt = &t->notes[i]; + int text = 1; + for (uint32_t k = 0; k < nt->size; k++) if (nt->data[k] < 9) text = 0; + if (text && nt->size < 200) printf(" note %.4s : %.*s\n", nt->tag, (int)nt->size, nt->data); + else printf(" note %.4s : %u bytes\n", nt->tag, nt->size); + } + printf(" stream prefix :"); + for (uint32_t m = t->mip_levels; m-- > 0; ) { + uint32_t mw; + yapf_tex_mip_size(t, m, &mw, NULL, NULL); + printf(" %u px: %llu B%s", mw, (unsigned long long)yapf_tex_stream_bytes(t, m), m ? "," : "\n"); + } + } + size_t raw = 0; + for (uint32_t s = 0; s < yapf_tex_surface_count(t); s++) { + uint32_t mw, mh, md; + yapf_tex_mip_size(t, s % t->mip_levels, &mw, &mh, &md); + raw += (size_t)mw * mh * md * t->channels * (t->payload ? 1 : yapf_sample_bytes(t->sample)); + } + printf(" file size : %zu bytes (%.1f%% of raw)\n", n, 100.0 * (double)n / (double)raw); + yapf_tex_free(t); + t = yapf_tex_load_memory(buf, n, 0, 0, &err); + printf(" decodes : %s\n", t ? "ok (checksums verified)" : err_name(err)); + yapf_tex_free(t); + free(buf); + return t ? 0 : 1; +} + +static int cmd_verify(const char *path) { + size_t n; + uint8_t *buf = read_all(path, &n); + int err = YAPF_ERR_IO; + yapf_texture_t *t = buf ? yapf_tex_load_memory(buf, n, 0, 0, &err) : NULL; + if (t) printf("%s: ok%s\n", path, buf[4] == 2 ? " (header, tables and every tile checksum match)" + : " (version 1: decodes; no checksums in v1)"); + else printf("%s: FAILED — %s\n", path, err_name(err)); + yapf_tex_free(t); + free(buf); + return t ? 0 : 1; } static int cmd_bench(const char *path) { - FILE *f = fopen(path, "rb"); - if (!f) { fprintf(stderr, "yapf: cannot open %s\n", path); return 1; } - fseek(f, 0, SEEK_END); - long n = ftell(f); - rewind(f); - uint8_t *buf = (uint8_t *)malloc((size_t)n); - if (!buf || fread(buf, 1, (size_t)n, f) != (size_t)n) { fclose(f); free(buf); return 1; } - fclose(f); - yapf_image_t *img = yapf_load_memory(buf, (size_t)n); - if (!img) { fprintf(stderr, "yapf: %s does not decode\n", path); free(buf); return 1; } - double raw = (double)img->width * img->height * img->channels; - yapf_free(img); - for (int mt = 0; mt < 2; mt++) { + size_t n; + uint8_t *buf = read_all(path, &n); + if (!buf) { fprintf(stderr, "yapf: cannot open %s\n", path); return 1; } + int err; + yapf_texture_t *t = yapf_tex_load_memory(buf, n, 0, 1, &err); + if (!t) { fprintf(stderr, "yapf: %s: %s\n", path, err_name(err)); free(buf); return 1; } + double raw = 0; + for (uint32_t s = 0; s < yapf_tex_surface_count(t); s++) raw += (double)yapf_tex_surface_bytes(t, s % t->mip_levels); + yapf_tex_free(t); + for (int mode = 0; mode < 3; mode++) { + int threads = mode == 1 ? 0 : 1, flags = mode == 2 ? YAPF_LOAD_TRUST : 0; + if (mode == 2 && buf[4] != 2) break; /* Best of 15 rounds; each round repeats enough decodes to last * ~20 ms, so short images are measured warm and not timer-bound. */ double one = now_s(); - yapf_free(mt ? yapf_load_memory_mt(buf, (size_t)n, 0) : yapf_load_memory(buf, (size_t)n)); + yapf_tex_free(yapf_tex_load_memory(buf, n, flags, threads, NULL)); one = now_s() - one; int reps = (int)(0.02 / (one > 1e-6 ? one : 1e-6)) + 1; double best = 1e9; for (int r = 0; r < 15; r++) { - double t = now_s(); - for (int k = 0; k < reps; k++) - yapf_free(mt ? yapf_load_memory_mt(buf, (size_t)n, 0) - : yapf_load_memory(buf, (size_t)n)); - t = (now_s() - t) / reps; - if (t < best) best = t; + double tt = now_s(); + for (int k = 0; k < reps; k++) yapf_tex_free(yapf_tex_load_memory(buf, n, flags, threads, NULL)); + tt = (now_s() - tt) / reps; + if (tt < best) best = tt; } - printf(" %-12s %8.3f ms %6.2f GB/s\n", mt ? "all cores" : "1 thread", best * 1e3, raw / best / 1e9); + static const char *nm[] = { "1 thread", "all cores", "no checks" }; + printf(" %-12s %8.3f ms %6.2f GB/s\n", nm[mode], best * 1e3, raw / best / 1e9); } free(buf); return 0; @@ -210,23 +615,31 @@ static int cmd_bench(const char *path) { /* Preview no larger than size×size: start from the smallest mip that is * still big enough, then average boxes of source pixels. */ static int cmd_thumbnail(const char *in, const char *out, int size) { - yapf_image_t *img = yapf_load_mt(in, 0); - if (!img) { fprintf(stderr, "yapf: cannot read %s\n", in); return 1; } + int err; + yapf_texture_t *t = yapf_tex_load(in, 0, 0, &err); + if (!t) { fprintf(stderr, "yapf: cannot read %s: %s\n", in, err_name(err)); return 1; } if (size < 1) size = 256; int m = 0; - while (m + 1 < img->mip_levels && - (int)(img->width >> (m + 1)) >= size && (int)(img->height >> (m + 1)) >= size) + while (m + 1 < t->mip_levels && + (int)(t->width >> (m + 1)) >= size && (int)(t->height >> (m + 1)) >= size) m++; - uint32_t sw = img->width >> m ? img->width >> m : 1, sh = img->height >> m ? img->height >> m : 1; - const uint8_t *src = img->mips[m]; + uint32_t sw, sh; + int ch, hdr; + uint8_t *src = (uint8_t *)surface_view(t, (uint32_t)m, 0, &sw, &sh, &ch, &hdr); + yapf_tex_free(t); + if (!src) { fprintf(stderr, "yapf: cannot preview %s\n", in); return 1; } + if (hdr) { + uint8_t *b = to8((const float *)src, (size_t)sw * sh * ch); + free(src); + if (!(src = b)) return 1; + } double k = (double)size / (double)(sw > sh ? sw : sh); if (k > 1.0) k = 1.0; uint32_t dw = (uint32_t)(sw * k + 0.5), dh = (uint32_t)(sh * k + 0.5); if (dw < 1) dw = 1; if (dh < 1) dh = 1; - int ch = img->channels; uint8_t *dst = (uint8_t *)malloc((size_t)dw * dh * ch); - if (!dst) { yapf_free(img); return 1; } + if (!dst) { free(src); return 1; } for (uint32_t y = 0; y < dh; y++) { uint32_t y0 = (uint32_t)((uint64_t)y * sh / dh), y1 = (uint32_t)((uint64_t)(y + 1) * sh / dh); if (y1 <= y0) y1 = y0 + 1; @@ -238,14 +651,14 @@ static int cmd_thumbnail(const char *in, const char *out, int size) { for (uint32_t yy = y0; yy < y1; yy++) for (uint32_t xx = x0; xx < x1; xx++) sum += src[((size_t)yy * sw + xx) * ch + c]; - uint64_t n = (uint64_t)(y1 - y0) * (x1 - x0); - dst[((size_t)y * dw + x) * ch + c] = (uint8_t)((sum + n / 2) / n); + uint64_t nn = (uint64_t)(y1 - y0) * (x1 - x0); + dst[((size_t)y * dw + x) * ch + c] = (uint8_t)((sum + nn / 2) / nn); } } } int ok = stbi_write_png(out, (int)dw, (int)dh, ch, dst, (int)dw * ch); free(dst); - yapf_free(img); + free(src); if (!ok) fprintf(stderr, "yapf: cannot write %s\n", out); return ok ? 0 : 1; } @@ -285,7 +698,7 @@ static const char *ext_name(const char *path) { * and a lossless check. */ static int cmd_compare(int n, char **paths) { int rc = 0; - double tot_src = 0, tot_yapf = 0, tot_src_t = 0, tot_yapf_t = 0; + double tot_src = 0, tot_yapf = 0, tot_src_t = 0, tot_yapf_t = 0, tot_ent = 0; for (int i = 0; i < n; i++) { const char *path = paths[i]; if (is_yapf(path)) { @@ -310,7 +723,7 @@ static int cmd_compare(int n, char **paths) { memset(&img, 0, sizeof(img)); img.width = (uint32_t)w; img.height = (uint32_t)h; img.channels = (uint8_t)ch; img.mip_levels = 1; img.pixels = px; img.flags = YAPF_FLAG_SRGB; - img.gpu_format = gpu_hint(ch, 1); + img.gpu_format = gpu_hint(ch, 1, YAPF_SAMPLE_U8); void *enc = NULL; size_t esz = 0; @@ -339,6 +752,28 @@ static int cmd_compare(int n, char **paths) { printf(" %-5s %10s %5.1f%% of raw decode %8.3f ms %.1fx faster, %.0f%% of the %s size\n", "YAPF", fmt_kb((double)esz, b), 100.0 * esz / raw, t_one * 1e3, t_src / t_one, 100.0 * esz / fsz, ext_name(path)); + /* version 2 with the optional Huffman planes */ + { + yapf_texture_t v; + void *data[1] = { px }; + size_t esz2 = 0; + void *enc2 = NULL; + double t_e; + memset(&v, 0, sizeof(v)); + v.width = (uint32_t)w; v.height = (uint32_t)h; v.depth = 1; v.layers = 1; v.frames = 1; + v.mip_levels = 1; v.channels = (uint8_t)ch; v.flags = YAPF_FLAG_SRGB; v.data = data; + if (yapf_tex_encode(&v, YAPF_ENC_ENTROPY, &enc2, &esz2) == YAPF_OK) { + yapf_texture_t *c2 = yapf_tex_load_memory(enc2, esz2, 0, 1, NULL); + if (!c2 || memcmp(c2->data[0], px, (size_t)w * h * ch) != 0) lossless = 0; + yapf_tex_free(c2); + YAPF_TIME_BEST(t_e, yapf_tex_free(yapf_tex_load_memory(enc2, esz2, 0, 1, NULL))); + printf(" %-5s %10s %5.1f%% of raw decode %8.3f ms %.1fx faster, %.0f%% of the %s size\n", + "YAPF+E", fmt_kb((double)esz2, c), 100.0 * esz2 / raw, t_e * 1e3, + t_src / t_e, 100.0 * esz2 / fsz, ext_name(path)); + tot_ent += (double)esz2; + yapf_free_buffer(enc2); + } + } printf(" YAPF on all cores: decode %.3f ms (%.1f GB/s), encode %.0f ms, %s\n\n", t_all * 1e3, raw / t_all / 1e9, t_enc * 1e3, lossless ? "lossless: yes" : "LOSSLESS CHECK FAILED"); @@ -350,8 +785,10 @@ static int cmd_compare(int n, char **paths) { } if (n > 1 && tot_yapf_t > 0) { char a[32], b[32]; - printf("Total: originals %s, YAPF %s (%.0f%%); decoding %.1fx faster on one core\n", - fmt_kb(tot_src, a), fmt_kb(tot_yapf, b), 100.0 * tot_yapf / tot_src, tot_src_t / tot_yapf_t); + char e[32]; + printf("Total: originals %s, YAPF %s (%.0f%%), YAPF+E %s (%.0f%%); decoding %.1fx faster on one core\n", + fmt_kb(tot_src, a), fmt_kb(tot_yapf, b), 100.0 * tot_yapf / tot_src, + fmt_kb(tot_ent, e), 100.0 * tot_ent / tot_src, tot_src_t / tot_yapf_t); } return rc; } @@ -359,96 +796,131 @@ static int cmd_compare(int n, char **paths) { static void usage(void) { fprintf(stderr, "usage: yapf [options] convert (formats by extension)\n" - " yapf info show header information\n" + " yapf pack [options]\n" + " yapf info header, tables and notes\n" + " yapf verify check every checksum\n" " yapf bench time decoding\n" " yapf thumbnail [size] preview, default 256 px\n" " yapf compare [more...] size and speed vs the original format\n" "\n" - "reads: .yapf .png .jpg .bmp .tga .gif .psd .pnm .hdr\n" - "writes: .yapf .png .jpg .bmp .tga\n" + "reads: .yapf .png (8/16-bit) .jpg .bmp .tga .gif .psd .pnm .hdr\n" + "writes: .yapf .png .jpg .bmp .tga .hdr\n" "\n" "options when writing .yapf:\n" - " --mips store a full mip chain\n" - " --linear pixels are linear, not sRGB\n" - " --premult alpha is premultiplied\n" + " --mips store a full mip chain (sRGB averaged in linear light)\n" + " --coverage REF mips keep alpha-test coverage at alpha REF (0-255)\n" + " --format F GPU blocks: bc1 bc3 bc4 bc5 bc6h bc7\n" + " --entropy Huffman-coded planes: smaller, slower to decode\n" + " --linear pixels are linear, not sRGB\n" + " --premult alpha is premultiplied\n" + " --linear-alpha alpha is linear even in an sRGB image\n" + " --normal gl|dx normal map (OpenGL +Y / DirectX -Y); implies linear\n" + " --map a,b,c,d channel meanings: r g b a luma nx ny nz rough gloss metal\n" + " ao height spec emissive opacity mask cavity thickness id -\n" + " --primaries P bt709 p3 bt2020 aces acescg adobe\n" + " --transfer T srgb linear gamma22 pq hlg bt1886\n" + " --8bit read 16-bit images as 8-bit\n" + " --float32 keep .hdr input as 32-bit float (default half)\n" + " --frame-ms N animation frame time (pack anim), --loops N\n" + " --author S --license S --source S --note TAG=text\n" + " --v2 write version 2 even for a plain 8-bit image\n" "options when reading .yapf:\n" - " --mip N export mip level N\n"); + " --mip N --layer N --face N --frame N --slice N which image to export\n"); +} + +static int parse_opts(int argc, char **argv, int start, opts_t *o) { + static const char *prim[] = { "", "bt709", "p3", "bt2020", "aces", "acescg", "adobe" }; + static const char *tf[] = { "", "srgb", "linear", "gamma22", "pq", "hlg", "bt1886" }; + memset(o, 0, sizeof(*o)); + o->srgb = 1; + o->format = -1; + o->loops = -1; + for (int i = start; i < argc; i++) { + const char *a = argv[i], *v = i + 1 < argc ? argv[i + 1] : NULL; + if (strcmp(a, "--mips") == 0) o->mips = 1; + else if (strcmp(a, "--linear") == 0) o->srgb = 0; + else if (strcmp(a, "--premult") == 0) o->premult = 1; + else if (strcmp(a, "--entropy") == 0) o->entropy = 1; + else if (strcmp(a, "--v2") == 0) o->v2 = 1; + else if (strcmp(a, "--8bit") == 0) o->eight_bit = 1; + else if (strcmp(a, "--float32") == 0) o->float32 = 1; + else if (strcmp(a, "--linear-alpha") == 0) o->linear_alpha = 1; + else if (!v) return 0; + else if (strcmp(a, "--coverage") == 0) { o->coverage = atoi(v); o->mips = 1; i++; + if (o->coverage < 1 || o->coverage > 255) return 0; } + else if (strcmp(a, "--format") == 0) { + size_t k; + for (k = 0; k < sizeof(fmt_names) / sizeof(fmt_names[0]); k++) + if (strcmp(v, fmt_names[k].name) == 0) o->format = fmt_names[k].v; + if (o->format < 0) return 0; + i++; + } + else if (strcmp(a, "--normal") == 0) { + o->normal = strcmp(v, "gl") == 0 ? YAPF_NORMAL_GL : strcmp(v, "dx") == 0 ? YAPF_NORMAL_DX : 0; + if (!o->normal) return 0; + i++; + } + else if (strcmp(a, "--map") == 0) { if (!parse_map(v, o->map)) return 0; o->have_map = 1; i++; } + else if (strcmp(a, "--primaries") == 0 || strcmp(a, "--transfer") == 0) { + const char **names = a[2] == 'p' ? prim : tf; + int found = 0; + for (int k = 1; k < 7; k++) if (strcmp(v, names[k]) == 0) found = k; + if (!found) return 0; + if (a[2] == 'p') o->primaries = found; else o->transfer = found; + i++; + } + else if (strcmp(a, "--frame-ms") == 0) { o->frame_ms = atof(v); i++; } + else if (strcmp(a, "--loops") == 0) { o->loops = atoi(v); i++; } + else if (strcmp(a, "--author") == 0) { o->author = v; i++; } + else if (strcmp(a, "--license") == 0) { o->license = v; i++; } + else if (strcmp(a, "--source") == 0) { o->source = v; i++; } + else if (strcmp(a, "--note") == 0) { if (o->nnotes == 16) return 0; o->notes[o->nnotes++] = v; i++; } + else if (strcmp(a, "--mip") == 0) { o->mip = atoi(v); i++; } + else if (strcmp(a, "--layer") == 0) { o->layer = atoi(v); i++; } + else if (strcmp(a, "--face") == 0) { o->face = atoi(v); i++; } + else if (strcmp(a, "--frame") == 0) { o->frame = atoi(v); i++; } + else if (strcmp(a, "--slice") == 0) { o->slice = atoi(v); i++; } + else return 0; + } + if (o->mip < 0 || o->layer < 0 || o->face < 0 || o->frame < 0 || o->slice < 0) return 0; + return 1; } int main(int argc, char **argv) { - if (argc == 3 && strcmp(argv[1], "info") == 0) return cmd_info(argv[2]); - if (argc == 3 && strcmp(argv[1], "bench") == 0) return cmd_bench(argv[2]); + if (argc == 3 && strcmp(argv[1], "info") == 0) return cmd_info(argv[2]); + if (argc == 3 && strcmp(argv[1], "verify") == 0) return cmd_verify(argv[2]); + if (argc == 3 && strcmp(argv[1], "bench") == 0) return cmd_bench(argv[2]); if (argc >= 3 && strcmp(argv[1], "compare") == 0) return cmd_compare(argc - 2, argv + 2); if ((argc == 4 || argc == 5) && strcmp(argv[1], "thumbnail") == 0) return cmd_thumbnail(argv[2], argv[3], argc == 5 ? atoi(argv[4]) : 256); - if (argc < 3) { usage(); return 2; } + + opts_t o; + if (argc >= 5 && strcmp(argv[1], "pack") == 0) { + int first_opt = 3; + while (first_opt < argc && strncmp(argv[first_opt], "--", 2) != 0) first_opt++; + int n = first_opt - 3 - 1; /* inputs, then the output */ + if (n < 1 || !parse_opts(argc, argv, first_opt, &o)) { usage(); return 2; } + const char *out = argv[first_opt - 1]; + if (!is_yapf(out)) { fprintf(stderr, "yapf: pack writes a .yapf file\n"); return 2; } + if (o.frame_ms <= 0) o.frame_ms = 100; + yapf_texture_t *t = assemble(argv[2], n, argv + 3, &o); + if (!t) return 1; + int rc = write_yapf(t, out, &o, 0); + yapf_tex_free(t); + return rc; + } + if (argc < 3 || !parse_opts(argc, argv, 3, &o)) { usage(); return 2; } const char *in = argv[1], *out = argv[2]; - int mips = 0, srgb = 1, premult = 0, mip_out = 0; - for (int i = 3; i < argc; i++) { - if (strcmp(argv[i], "--mips") == 0) mips = 1; - else if (strcmp(argv[i], "--linear") == 0) srgb = 0; - else if (strcmp(argv[i], "--premult") == 0) premult = 1; - else if (strcmp(argv[i], "--mip") == 0 && i + 1 < argc) mip_out = atoi(argv[++i]); - else { usage(); return 2; } - } - - /* Read */ - yapf_image_t src; - yapf_image_t *loaded = NULL; - uint8_t *stb_px = NULL; - memset(&src, 0, sizeof(src)); - if (is_yapf(in)) { - loaded = yapf_load_mt(in, 0); - if (!loaded) { fprintf(stderr, "yapf: cannot read %s (missing, corrupt or not YAPF v1)\n", in); return 1; } - src = *loaded; - } else { - int w, h, ch; - if (stbi_is_16_bit(in)) - fprintf(stderr, "yapf: note: %s has 16-bit channels; YAPF stores 8 bits per channel\n", in); - stb_px = stbi_load(in, &w, &h, &ch, 0); - if (!stb_px) { fprintf(stderr, "yapf: cannot read %s: %s\n", in, stbi_failure_reason()); return 1; } - src.width = (uint32_t)w; src.height = (uint32_t)h; src.channels = (uint8_t)ch; - src.pixels = stb_px; src.mip_levels = 1; - src.flags = srgb ? YAPF_FLAG_SRGB : 0; - src.gpu_format = gpu_hint(ch, srgb); - } - - /* Write */ - int rc = 0; + int from_yapf; + yapf_texture_t *t = read_input(in, &o, &from_yapf); + if (!t) return 1; + int rc; if (is_yapf(out)) { - yapf_image_t dst = src; - if (!loaded) { - if (premult) dst.flags |= YAPF_FLAG_PREMULT_ALPHA; - } - dst.mips = NULL; - dst.mip_levels = 1; - if (mips && build_mips(&dst) != 0) { fprintf(stderr, "yapf: out of memory\n"); rc = 1; } - if (!rc && yapf_save(out, &dst) != YAPF_OK) { fprintf(stderr, "yapf: cannot write %s\n", out); rc = 1; } - if (dst.mips) { - for (int m = 1; m < dst.mip_levels; m++) free(dst.mips[m]); - free(dst.mips); - } + rc = write_yapf(t, out, &o, from_yapf); } else { - const uint8_t *px = src.pixels; - uint32_t w = src.width, h = src.height; - if (mip_out > 0) { - if (!loaded || mip_out >= src.mip_levels) { - fprintf(stderr, "yapf: %s has no mip level %d\n", in, mip_out); - rc = 1; - } else { - px = src.mips[mip_out]; - w = src.width >> mip_out ? src.width >> mip_out : 1; - h = src.height >> mip_out ? src.height >> mip_out : 1; - } - } - if (!rc && !write_image(out, px, (int)w, (int)h, src.channels)) { - fprintf(stderr, "yapf: cannot write %s\n", out); - rc = 1; - } + rc = export_image(t, in, out, &o); } - - yapf_free(loaded); - stbi_image_free(stb_px); + yapf_tex_free(t); return rc; } diff --git a/yapf.c b/yapf.c index d926109..5f49a19 100644 --- a/yapf.c +++ b/yapf.c @@ -396,7 +396,9 @@ YAPF__SIMD_FN void yapf__simd_inverse_row(uint8_t *d, const uint8_t *P0, } __m128i p2 = _mm_loadu_si128((const __m128i *)(const void *)(P2 + k)); __m128i r, g, b; - if (xf) { /* subtract-green */ + if (xf == 2) { /* none (v2) */ + r = p0; g = p1; b = p2; + } else if (xf) { /* subtract-green */ g = p0; r = _mm_add_epi8(_mm_sub_epi8(p1, bias), g); b = _mm_add_epi8(_mm_sub_epi8(p2, bias), g); @@ -520,7 +522,9 @@ YAPF__SIMD_FN void yapf__simd_inverse_row(uint8_t *d, const uint8_t *P0, continue; } uint8x16_t p2 = vld1q_u8(P2 + k), r, g, b; - if (xf) { + if (xf == 2) { + r = p0; g = p1; b = p2; + } else if (xf) { g = p0; r = vaddq_u8(vsubq_u8(p1, bias), g); b = vaddq_u8(vsubq_u8(p2, bias), g); @@ -854,6 +858,943 @@ static void yapf__unfilter_row(uint8_t f, uint8_t *R, const uint8_t *U, * stores the tile raw whenever coding would not make it smaller. */ #define YAPF__SUBGREEN 0x10 +#define YAPF__ENTROPY 0x20 /* v2: entropy plane mask follows */ +#define YAPF__NOXFORM 0x40 /* v2: planes are R, G, B (+ A) as stored */ + +/* ── yapf-sum ────────────────────────────────────────────────────────── */ +/* + * The version 2 checksum. Four independent 64-bit lanes take the 8-byte + * little-endian words of the input in turn (word j goes to lane j mod 4; + * the last word is zero padded): + * a = (a ^ w) * K; a ^= a >> 32; K = 0x9E3779B97F4A7C15 + * Lane i starts at (i + 1) * K ^ n, n = byte length. The result is + * h = a0 ^ rotl(a1, 16) ^ rotl(a2, 32) ^ rotl(a3, 48); + * h *= K; h ^= h >> 32; low 32 bits of h. + * The lanes keep four multiplies in flight, so it runs well above the + * decoder's own speed. + */ +#define YAPF__K 0x9E3779B97F4A7C15ull + +static uint64_t yapf__rotl(uint64_t v, int r) { return (v << r) | (v >> (64 - r)); } + +uint32_t yapf_checksum(const void *data, size_t size) { + const uint8_t *p = (const uint8_t *)data; + uint64_t n = (uint64_t)size; + uint64_t a[4]; + for (int i = 0; i < 4; i++) a[i] = ((uint64_t)(i + 1) * YAPF__K) ^ n; + size_t j = 0, words = size / 8; + for (; j + 4 <= words; j += 4) { + for (int i = 0; i < 4; i++) { + uint64_t v = (a[i] ^ yapf__r64le(p + (j + i) * 8)) * YAPF__K; + a[i] = v ^ (v >> 32); + } + } + for (; j < words; j++) { + uint64_t v = (a[j & 3] ^ yapf__r64le(p + j * 8)) * YAPF__K; + a[j & 3] = v ^ (v >> 32); + } + if (size & 7) { + uint8_t tail[8] = {0}; + memcpy(tail, p + words * 8, size & 7); + uint64_t v = (a[j & 3] ^ yapf__r64le(tail)) * YAPF__K; + a[j & 3] = v ^ (v >> 32); + } + uint64_t h = a[0] ^ yapf__rotl(a[1], 16) ^ yapf__rotl(a[2], 32) ^ yapf__rotl(a[3], 48); + h *= YAPF__K; + h ^= h >> 32; + return (uint32_t)h; +} + +static uint16_t yapf__r16le(const uint8_t *p) { + return (uint16_t)(p[0] | (p[1] << 8)); +} + +static void yapf__w16le(uint8_t *p, uint32_t v) { + p[0] = (uint8_t)v; + p[1] = (uint8_t)(v >> 8); +} + +static void yapf__w64le(uint8_t *p, uint64_t v) { + yapf__w32le(p, (uint32_t)v); + yapf__w32le(p + 4, (uint32_t)(v >> 32)); +} + +/* ── Entropy stage (version 2, optional) ─────────────────────────────── */ +/* + * A surface that uses it starts with a code book: Huffman tables keyed by + * (slot, context). Pixel planes: slot = transform · 16 + plane (transform + * 0 YCoCg-R, 1 subtract-green, 2 none), context = how busy the neighbours + * are (folded residuals left + above, in 7 classes). Block tiles: slot = + * byte lane of the block, context 0. Sharing the tables per surface + * keeps tiles independent and their headers tiny. + * + * Code table: 1 byte n − 1 (n = symbols 0..n−1 described), then n 4-bit + * code lengths (0 = unused, 1..11), low nibble first; an odd + * count is padded with a zero nibble. Kraft sum <= 1. + * Codes are canonical (shorter first, then by symbol, as in deflate) and + * written least significant bit first, so a decoder reads them with one + * 2048-entry table lookup. + */ +#define YAPF__HUFF_BITS 11 +#define YAPF__HUFF_SIZE (1u << YAPF__HUFF_BITS) + +static uint32_t yapf__bitrev(uint32_t c, int n) { + uint32_t r = 0; + for (int i = 0; i < n; i++) { r = (r << 1) | (c & 1); c >>= 1; } + return r; +} + +/* Parses a code table into lut (entry = length << 8 | symbol, 0 = no + * code). Returns the byte after the table, or NULL. */ +static const uint8_t *yapf__huff_table(const uint8_t *p, const uint8_t *end, + uint16_t *lut) +{ + if (p >= end) return NULL; + uint32_t n = (uint32_t)*p++ + 1, nbytes = (n + 1) / 2; + if ((size_t)(end - p) < nbytes) return NULL; + uint8_t len[256]; + uint32_t count[YAPF__HUFF_BITS + 1] = {0}, kraft = 0; + for (uint32_t i = 0; i < n; i++) { + uint8_t l = (uint8_t)((p[i >> 1] >> ((i & 1) * 4)) & 15); + if (l > YAPF__HUFF_BITS) return NULL; + len[i] = l; + count[l]++; + if (l) kraft += YAPF__HUFF_SIZE >> l; + } + if ((n & 1) && (p[n >> 1] >> 4)) return NULL; + if (kraft == 0 || kraft > YAPF__HUFF_SIZE) return NULL; + p += nbytes; + + uint32_t next[YAPF__HUFF_BITS + 2], code = 0; + count[0] = 0; + for (int l = 1; l <= YAPF__HUFF_BITS; l++) { + code = (code + count[l - 1]) << 1; + next[l] = code; + } + memset(lut, 0, YAPF__HUFF_SIZE * sizeof(uint16_t)); + for (uint32_t s = 0; s < n; s++) { + int l = len[s]; + if (!l) continue; + uint32_t r = yapf__bitrev(next[l]++, l); + for (uint32_t k = r; k < YAPF__HUFF_SIZE; k += 1u << l) + lut[k] = (uint16_t)((l << 8) | s); + } + return p; +} + +/* Bit reader for code streams; reading past the end yields zeros and is + * caught by yapf__br_done. */ +typedef struct { + const uint8_t *p; + size_t L, pos; + uint64_t buf, used; + int bits; +} yapf__br; + +static void yapf__br_init(yapf__br *b, const uint8_t *p, size_t L) { + b->p = p; b->L = L; b->pos = 0; b->buf = 0; b->used = 0; b->bits = 0; +} + +static int yapf__br_sym(yapf__br *b, const uint16_t *lut) { + if (b->bits < (int)YAPF__HUFF_BITS) { + if (b->pos + 8 <= b->L) { + b->buf |= yapf__r64le(b->p + b->pos) << b->bits; + b->pos += (size_t)((63 - b->bits) >> 3); + b->bits |= 56; + } else { + while (b->bits <= 56) { + b->buf |= (uint64_t)(b->pos < b->L ? b->p[b->pos] : 0) << b->bits; + b->pos++; + b->bits += 8; + } + } + } + uint16_t e = lut[b->buf & (YAPF__HUFF_SIZE - 1)]; + if (!e) return -1; + int l = e >> 8; + b->buf >>= l; + b->bits -= l; + b->used += (uint64_t)l; + return e & 0xFF; +} + +/* The stream must end in its last byte. */ +static int yapf__br_done(const yapf__br *b) { + return (b->used + 7) / 8 == b->L; +} + +#define YAPF__SLOTS 48 +#define YAPF__CTX 7 + +/* Context of a residual from its left and upper neighbours' folded + * residuals s = l + u: 0, 1–2, 3–6, 7–14, 15–30, 31–62, 63 and up + * (= min(6, floor(log2(s + 1)))). Branch free: the decoder computes it + * on the critical path of every sample. */ +static int yapf__ctx(uint32_t s) { + return (s > 0) + (s > 2) + (s > 6) + (s > 14) + (s > 30) + (s > 62); +} + +/* The same as table offsets for the decoder: ctx(min(s, 63)) << 11. */ +#define YAPF__C(c) (uint16_t)((c) << 11) +static const uint16_t yapf__ctx_off[64] = { /* index min(s, 63) */ + YAPF__C(0), YAPF__C(1), YAPF__C(1), YAPF__C(2), YAPF__C(2), YAPF__C(2), YAPF__C(2), + YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), + YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), + YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), + YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), + YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), + YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), + YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), + YAPF__C(6) +}; +#undef YAPF__C + +/* Decoder side of a code book: per slot one array of 7 × 2048 lookup + * entries (contexts without a table stay 0 = invalid), so context and + * bits form a single index. */ +typedef struct { + uint16_t *slot[YAPF__SLOTS]; + uint16_t *mem; +} yapf__dbook; + +/* Book: u16 table count, then per table: slot, context, code table. */ +static int yapf__book_read(const uint8_t *p, size_t len, yapf__dbook *bk) { + const uint8_t *end = p + len, *q; + uint8_t seen[YAPF__SLOTS][YAPF__CTX]; + uint32_t slots = 0; + memset(bk, 0, sizeof(*bk)); + memset(seen, 0, sizeof(seen)); + if (len < 2) return -1; + uint32_t n = yapf__r16le(p); + p += 2; + if (n == 0 || n > YAPF__SLOTS * YAPF__CTX) return -1; + /* first pass: validate the layout and count the slots in use */ + for (uint32_t i = 0, k; i < n; i++) { + if (end - p < 3) return -1; + uint8_t slot = p[0], ctx = p[1]; + if (slot >= YAPF__SLOTS || ctx >= YAPF__CTX || seen[slot][ctx]) return -1; + seen[slot][ctx] = 1; + for (k = 0; k < YAPF__CTX && (k == ctx || !seen[slot][k]); k++) {} + if (k == YAPF__CTX) slots++; /* first table of this slot */ + uint32_t ns = (uint32_t)p[2] + 1; + p += 3 + (ns + 1) / 2; + if (p > end) return -1; + } + if (p != end) return -1; + bk->mem = (uint16_t *)calloc((size_t)slots * YAPF__CTX * YAPF__HUFF_SIZE, sizeof(uint16_t)); + if (!bk->mem) return -1; + uint32_t used = 0; + for (q = end - len + 2; q < end; ) { + uint8_t slot = q[0], ctx = q[1]; + if (!bk->slot[slot]) bk->slot[slot] = bk->mem + (size_t)used++ * YAPF__CTX * YAPF__HUFF_SIZE; + q = yapf__huff_table(q + 2, end, bk->slot[slot] + (size_t)ctx * YAPF__HUFF_SIZE); + if (!q) return -1; + } + return 0; +} + +/* Entropy plane payload: filters, u16 stream length, the folded + * residuals in raster order, each with the table of its context. + * + * Every residual's context needs the one decoded just before it, so one + * stream is a chain of dependent steps. The planes of a tile are + * independent streams, though: the decoder runs up to four of them in + * the same loop and the CPU overlaps their chains. */ +typedef struct { + const uint8_t *p; + size_t L; + const uint16_t *lut; + uint8_t *z; +} yapf__hstream; + +/* Parses the header of an entropy plane into *s; returns the end of the + * payload or NULL. */ +static const uint8_t *yapf__huff_head(const uint8_t *p, const uint8_t *end, + const uint8_t **filt_out, uint32_t th, + const yapf__dbook *bk, int slot, + uint8_t *z, yapf__hstream *s) +{ + uint32_t nb = (th + YAPF__BAND - 1) / YAPF__BAND; + if (!bk || !bk->slot[slot] || (size_t)(end - p) < nb + 2) return NULL; + *filt_out = p; + for (uint32_t b = 0; b < nb; b++) + if (!yapf__filter_valid(p[b])) return NULL; + p += nb; + s->L = yapf__r16le(p); + p += 2; + if ((size_t)(end - p) < s->L) return NULL; + s->p = p; + s->lut = bk->slot[slot]; + s->z = z; + return p + s->L; +} + +#define YAPF__HSTATE(k) \ + uint64_t buf##k = 0; \ + int bits##k = 0; \ + size_t pos##k = 0; \ + uint32_t l##k = 0; \ + const uint8_t *P##k = s[k].p, *U##k = yapf__zero_row; \ + const size_t L##k = s[k].L; \ + const uint16_t *T##k = s[k].lut; \ + uint8_t *Z##k = s[k].z; + +#define YAPF__HROW(k) \ + U##k = y ? Z##k + i - tw : yapf__zero_row; \ + l##k = 0; + +#define YAPF__HSTEP(k) \ + { \ + if (pos##k + 8 <= L##k) { /* branch free while not near the end */ \ + buf##k |= yapf__r64le(P##k + pos##k) << bits##k; \ + pos##k += (size_t)((63 - bits##k) >> 3); \ + bits##k |= 56; \ + } else if (bits##k < (int)YAPF__HUFF_BITS) { \ + while (bits##k <= 56) { \ + buf##k |= (uint64_t)(pos##k < L##k ? P##k[pos##k] : 0) << bits##k; \ + pos##k++; \ + bits##k += 8; \ + } \ + } \ + uint32_t s_ = l##k + U##k[x]; \ + uint32_t e_ = T##k[yapf__ctx_off[s_ < 63 ? s_ : 63] | \ + (uint32_t)(buf##k & (YAPF__HUFF_SIZE - 1))]; \ + int n_ = (int)(e_ >> 8); \ + if (!e_) return -1; \ + buf##k >>= n_; \ + bits##k -= n_; \ + l##k = e_ & 0xFF; \ + Z##k[i] = (uint8_t)l##k; \ + } + +/* bits consumed = 8·pos − bits still buffered; the stream must end in + * its last byte */ +#define YAPF__HEND(k) if ((8 * (uint64_t)pos##k - (uint64_t)bits##k + 7) / 8 != L##k) return -1; + +static int yapf__hdec1(const yapf__hstream *s, uint32_t tw, uint32_t th) { + YAPF__HSTATE(0) + for (uint32_t y = 0, i = 0; y < th; y++) { + YAPF__HROW(0) + for (uint32_t x = 0; x < tw; x++, i++) YAPF__HSTEP(0) + } + YAPF__HEND(0) + return 0; +} + +static int yapf__hdec2(const yapf__hstream *s, uint32_t tw, uint32_t th) { + YAPF__HSTATE(0) YAPF__HSTATE(1) + for (uint32_t y = 0, i = 0; y < th; y++) { + YAPF__HROW(0) YAPF__HROW(1) + for (uint32_t x = 0; x < tw; x++, i++) { YAPF__HSTEP(0) YAPF__HSTEP(1) } + } + YAPF__HEND(0) YAPF__HEND(1) + return 0; +} + +static int yapf__hdec3(const yapf__hstream *s, uint32_t tw, uint32_t th) { + YAPF__HSTATE(0) YAPF__HSTATE(1) YAPF__HSTATE(2) + for (uint32_t y = 0, i = 0; y < th; y++) { + YAPF__HROW(0) YAPF__HROW(1) YAPF__HROW(2) + for (uint32_t x = 0; x < tw; x++, i++) { YAPF__HSTEP(0) YAPF__HSTEP(1) YAPF__HSTEP(2) } + } + YAPF__HEND(0) YAPF__HEND(1) YAPF__HEND(2) + return 0; +} + +static int yapf__hdec4(const yapf__hstream *s, uint32_t tw, uint32_t th) { + YAPF__HSTATE(0) YAPF__HSTATE(1) YAPF__HSTATE(2) YAPF__HSTATE(3) + for (uint32_t y = 0, i = 0; y < th; y++) { + YAPF__HROW(0) YAPF__HROW(1) YAPF__HROW(2) YAPF__HROW(3) + for (uint32_t x = 0; x < tw; x++, i++) { + YAPF__HSTEP(0) YAPF__HSTEP(1) YAPF__HSTEP(2) YAPF__HSTEP(3) + } + } + YAPF__HEND(0) YAPF__HEND(1) YAPF__HEND(2) YAPF__HEND(3) + return 0; +} + +/* Decodes n (1..) streams, four at a time; zeroes each tail group. */ +static int yapf__huff_run(const yapf__hstream *s, int n, uint32_t tw, uint32_t th) { + for (int k = 0; k < n; k += 4) { + int m = n - k < 4 ? n - k : 4, rc; + rc = m == 4 ? yapf__hdec4(s + k, tw, th) : m == 3 ? yapf__hdec3(s + k, tw, th) + : m == 2 ? yapf__hdec2(s + k, tw, th) : yapf__hdec1(s + k, tw, th); + if (rc != 0) return -1; + } + for (int k = 0; k < n; k++) memset(s[k].z + (size_t)tw * th, 0, YAPF__GROUP); + return 0; +} + +/* One entropy plane on its own. */ +static const uint8_t *yapf__huff_plane(const uint8_t *p, const uint8_t *end, + uint8_t *z, const uint8_t **filt_out, + uint32_t tw, uint32_t th, + const yapf__dbook *bk, int slot) +{ + yapf__hstream s; + p = yapf__huff_head(p, end, filt_out, th, bk, slot, z, &s); + if (!p || yapf__huff_run(&s, 1, tw, th) != 0) return NULL; + return p; +} + +/* Code lengths (<= 11 bits) for freq[0..n). Plain Huffman by the + * two-queue method; if a code comes out too long the counts are halved + * and it is built again. */ +static void yapf__huff_lengths(const uint32_t *freq_in, uint32_t n, uint8_t *len) { + uint32_t freq[256], sym[256], m = 0; + memcpy(freq, freq_in, n * sizeof(uint32_t)); + for (;;) { + m = 0; + for (uint32_t s = 0; s < n; s++) { len[s] = 0; if (freq[s]) sym[m++] = s; } + if (m == 0) return; + if (m == 1) { len[sym[0]] = 1; return; } + /* insertion sort by frequency (m <= 256) */ + for (uint32_t i = 1; i < m; i++) { + uint32_t v = sym[i], j = i; + while (j > 0 && freq[sym[j - 1]] > freq[v]) { sym[j] = sym[j - 1]; j--; } + sym[j] = v; + } + /* nodes 0..m-1 leaves (sorted), m..2m-2 internal in creation order */ + uint64_t w[512]; + uint32_t parent[512], li = 0, ii = m, nn = m; + for (uint32_t i = 0; i < m; i++) w[i] = freq[sym[i]]; + for (uint32_t k = 0; k + 1 < m; k++) { + uint32_t pick[2]; + for (int t = 0; t < 2; t++) { + if (li < m && (ii >= nn || w[li] <= w[ii])) pick[t] = li++; + else pick[t] = ii++; + } + w[nn] = w[pick[0]] + w[pick[1]]; + parent[pick[0]] = parent[pick[1]] = nn; + nn++; + } + uint8_t depth[512]; + depth[nn - 1] = 0; + int maxl = 0; + for (uint32_t k = nn - 1; k-- > 0; ) { + depth[k] = (uint8_t)(depth[parent[k]] + 1); + if (k < m && depth[k] > maxl) maxl = depth[k]; + } + if (maxl <= (int)YAPF__HUFF_BITS) { + for (uint32_t i = 0; i < m; i++) len[sym[i]] = depth[i]; + return; + } + for (uint32_t s = 0; s < n; s++) if (freq[s]) freq[s] = (freq[s] >> 1) | 1; + } +} + +typedef struct { + uint64_t acc; + int bits; + yapf__out *o; +} yapf__bitw; + +static void yapf__bitw_put(yapf__bitw *b, uint32_t code, int len) { + b->acc |= (uint64_t)code << b->bits; + b->bits += len; + while (b->bits >= 8) { + b->o->buf[b->o->len++] = (uint8_t)b->acc; + b->acc >>= 8; + b->bits -= 8; + } +} + +static void yapf__bitw_flush(yapf__bitw *b) { + if (b->bits > 0) b->o->buf[b->o->len++] = (uint8_t)b->acc; + b->acc = 0; + b->bits = 0; +} + +/* Canonical codes for lengths len[0..ns). */ +static void yapf__huff_codes(const uint8_t *len, uint32_t ns, uint16_t *code) { + uint32_t count[YAPF__HUFF_BITS + 1] = {0}, next[YAPF__HUFF_BITS + 2], c = 0; + for (uint32_t s = 0; s < ns; s++) if (len[s]) count[len[s]]++; + for (int l = 1; l <= (int)YAPF__HUFF_BITS; l++) { + c = (c + count[l - 1]) << 1; + next[l] = c; + } + for (uint32_t s = 0; s < ns; s++) + code[s] = len[s] ? (uint16_t)yapf__bitrev(next[len[s]]++, len[s]) : 0; +} + +typedef struct { /* encoder side of a code book */ + uint32_t freq[YAPF__SLOTS][YAPF__CTX][256]; + uint8_t len[YAPF__SLOTS][YAPF__CTX][256]; + uint16_t code[YAPF__SLOTS][YAPF__CTX][256]; +} yapf__ebook; + +/* Encoder modes for the tile writers. */ +#define YAPF__ENC_PLAIN 0 /* groups only */ +#define YAPF__ENC_STATS 1 /* groups, and count symbols for a book */ +#define YAPF__ENC_BOOK 2 /* entropy planes / packed blocks */ + +/* Builds every table that has symbols and writes the book. */ +static int yapf__book_write(yapf__ebook *eb, yapf__out *o) { + uint32_t n = 0; + if (yapf__reserve(o, 2) != 0) return -1; + size_t at = o->len; + o->len += 2; + memset(eb->len, 0, sizeof(eb->len)); + for (int s = 0; s < YAPF__SLOTS; s++) + for (int c = 0; c < YAPF__CTX; c++) { + const uint32_t *f = eb->freq[s][c]; + uint32_t ns = 256; + while (ns > 0 && !f[ns - 1]) ns--; + if (ns == 0) continue; + yapf__huff_lengths(f, ns, eb->len[s][c]); + yapf__huff_codes(eb->len[s][c], ns, eb->code[s][c]); + if (yapf__reserve(o, 3 + 128) != 0) return -1; + o->buf[o->len++] = (uint8_t)s; + o->buf[o->len++] = (uint8_t)c; + o->buf[o->len++] = (uint8_t)(ns - 1); + for (uint32_t k = 0; k < ns; k += 2) + o->buf[o->len++] = (uint8_t)(eb->len[s][c][k] | (k + 1 < ns ? eb->len[s][c][k + 1] << 4 : 0)); + n++; + } + yapf__w16le(o->buf + at, n); + return n ? 0 : 1; /* 1: nothing to code */ +} + +static void yapf__plane_stats(yapf__ebook *eb, int slot, const uint8_t *z, uint32_t tw, uint32_t th) { + for (uint32_t y = 0, i = 0; y < th; y++) + for (uint32_t x = 0; x < tw; x++, i++) { + uint32_t s = (x ? z[i - 1] : 0u) + (y ? z[i - tw] : 0u); + eb->freq[slot][yapf__ctx(s)][z[i]]++; + } +} + +/* Entropy plane payload (see yapf__huff_plane). Returns 1 when some + * symbol has no code in the book (the caller keeps the group form). */ +static int yapf__huff_emit_plane(yapf__out *o, const uint8_t *z, const uint8_t *filt, + uint32_t tw, uint32_t th, const yapf__ebook *eb, int slot) +{ + uint32_t nb = (th + YAPF__BAND - 1) / YAPF__BAND; + uint64_t bits = 0; + for (uint32_t y = 0, i = 0; y < th; y++) + for (uint32_t x = 0; x < tw; x++, i++) { + uint32_t s = (x ? z[i - 1] : 0u) + (y ? z[i - tw] : 0u); + uint8_t l = eb->len[slot][yapf__ctx(s)][z[i]]; + if (!l) return 1; + bits += l; + } + size_t L = (size_t)((bits + 7) / 8); + if (L > 0xFFFF) return 1; + if (yapf__reserve(o, nb + 2 + L + 8) != 0) return -1; + memcpy(o->buf + o->len, filt, nb); + o->len += nb; + yapf__w16le(o->buf + o->len, (uint32_t)L); + o->len += 2; + yapf__bitw bw = { 0, 0, o }; + for (uint32_t y = 0, i = 0; y < th; y++) + for (uint32_t x = 0; x < tw; x++, i++) { + uint32_t s = (x ? z[i - 1] : 0u) + (y ? z[i - tw] : 0u); + int c = yapf__ctx(s); + yapf__bitw_put(&bw, eb->code[slot][c][z[i]], eb->len[slot][c][z[i]]); + } + yapf__bitw_flush(&bw); + return 0; +} + +/* ── Plane set emission (version 2) ──────────────────────────────────── */ +/* + * Writes a coded v2 tile: header byte, (wide) u16 constant mask, entropy + * mask if any plane is entropy coded, constant values, payloads. With a + * book (mode BOOK) each plane is coded both ways and the entropy form is + * kept only if it is at least 1/32 smaller; mode STATS feeds the book. + */ +typedef struct { + int np; /* planes */ + int wide; /* masks are u16 (wide samples) */ + uint32_t cmask; /* constant planes */ + const uint8_t *cval; /* constant value per plane */ + const uint8_t *z[16]; /* folded residuals per plane */ + const uint8_t *filt[16]; +} yapf__planeset; + +static int yapf__emit_planeset(yapf__out *o, yapf__out *tmp, uint8_t hdr, + const yapf__planeset *ps, uint32_t tw, uint32_t th, + int mode, yapf__ebook *eb) +{ + size_t off[17]; + uint32_t emask = 0; + int tsel = (hdr & YAPF__SUBGREEN) ? 1 : (hdr & YAPF__NOXFORM) ? 2 : 0; + tmp->len = 0; + for (int c = 0; c < ps->np; c++) { + off[c] = tmp->len; + if (ps->cmask & (1u << c)) continue; + if (yapf__emit_plane(tmp, ps->z[c], ps->filt[c], tw, th) != 0) return -1; + if (mode == YAPF__ENC_STATS) yapf__plane_stats(eb, tsel * 16 + c, ps->z[c], tw, th); + if (mode != YAPF__ENC_BOOK) continue; + size_t gsz = tmp->len - off[c]; + int r = yapf__huff_emit_plane(tmp, ps->z[c], ps->filt[c], tw, th, eb, tsel * 16 + c); + if (r < 0) return -1; + size_t hsz = r ? (size_t)-1 : tmp->len - off[c] - gsz; + if (!r && hsz + gsz / 32 < gsz) { + memmove(tmp->buf + off[c], tmp->buf + off[c] + gsz, hsz); + tmp->len = off[c] + hsz; + emask |= 1u << c; + } else { + tmp->len = off[c] + gsz; + } + } + if (yapf__reserve(o, 5 + (size_t)ps->np + tmp->len) != 0) return -1; + o->buf[o->len++] = (uint8_t)(hdr | (emask ? YAPF__ENTROPY : 0)); + if (ps->wide) { yapf__w16le(o->buf + o->len, ps->cmask); o->len += 2; } + if (emask) { + if (ps->wide) { yapf__w16le(o->buf + o->len, emask); o->len += 2; } + else o->buf[o->len++] = (uint8_t)emask; + } + for (int c = 0; c < ps->np; c++) + if (ps->cmask & (1u << c)) o->buf[o->len++] = ps->cval[c]; + if (tmp->len) memcpy(o->buf + o->len, tmp->buf, tmp->len); + o->len += tmp->len; + return 0; +} + +/* ── Wide samples (version 2) ────────────────────────────────────────── */ +/* + * 16- and 32-bit samples are coded as byte planes with the 8-bit + * machinery. Floats first become order-preserving integers (sign set: + * all bits flipped; else the sign bit set), so nearby values stay nearby. + * The colour transforms are the 8-bit ones at the sample width (mod 2^16 + * or 2^32); a third choice keeps R, G, B as they are. Channel c gives + * planes c·B .. c·B + B − 1, most significant byte first, so the smooth + * high bytes and the noisy low bytes each get their own filters. + */ +#define YAPF__MAX_PLANES 16 +#define YAPF__ROW (YAPF_TILE_SIZE + 1) + +static size_t yapf__sbytes(int sample) { + return sample == YAPF_SAMPLE_U8 ? 1 : sample == YAPF_SAMPLE_F32 ? 4 : 2; +} + +static uint32_t yapf__vmask(int bits) { + return bits == 32 ? 0xFFFFFFFFu : (1u << bits) - 1; +} + +static uint32_t yapf__fmap(uint32_t v, int bits) { + uint32_t s = 1u << (bits - 1); + return (v & s) ? (~v & yapf__vmask(bits)) : (v | s); +} + +static uint32_t yapf__funmap(uint32_t v, int bits) { + uint32_t s = 1u << (bits - 1); + return (v & s) ? (v ^ s) : (~v & yapf__vmask(bits)); +} + +/* Arithmetic shift right by one of a bits-wide two's complement value. */ +static uint32_t yapf__sar1(uint32_t v, int bits) { + return ((v >> 1) | (v & (1u << (bits - 1)))) & yapf__vmask(bits); +} + +static uint32_t yapf__get_sample(const uint8_t *p, int sample) { + if (sample == YAPF_SAMPLE_U8) return *p; + if (sample == YAPF_SAMPLE_F32) { uint32_t v; memcpy(&v, p, 4); return v; } + uint16_t v; memcpy(&v, p, 2); return v; +} + +static void yapf__put_sample(uint8_t *p, int sample, uint32_t v) { + if (sample == YAPF_SAMPLE_U8) { *p = (uint8_t)v; return; } + if (sample == YAPF_SAMPLE_F32) { memcpy(p, &v, 4); return; } + uint16_t h = (uint16_t)v; + memcpy(p, &h, 2); +} + +typedef struct { + uint8_t plane[2][YAPF__MAX_PLANES][YAPF__TILE_PX]; + uint8_t z[2][YAPF__MAX_PLANES][YAPF__TILE_PX + YAPF__GROUP]; + uint8_t filt[2][YAPF__MAX_PLANES][YAPF_TILE_SIZE / YAPF__BAND]; + uint32_t val[4][YAPF__TILE_PX]; +} yapf__wscratch; + +static int yapf__encode_tile_wide( + yapf__out *o, yapf__out *tmp, yapf__wscratch *sc, + const uint8_t *pixels, uint32_t img_w, + uint32_t tx, uint32_t ty, uint32_t tw, uint32_t th, + int ch, int sample, int mode, yapf__ebook *eb) +{ + int sb = (int)yapf__sbytes(sample), bits = 8 * sb; + int isf = sample == YAPF_SAMPLE_F16 || sample == YAPF_SAMPLE_F32; + int np = ch * sb; + uint32_t total = tw * th, half = 1u << (bits - 1), m = yapf__vmask(bits); + size_t start = o->len, raw = (size_t)total * ch * sb; + int tries = ch >= 3 ? 3 : 1, cur = 0; + uint32_t best_est = UINT32_MAX, cmask[2] = { 0, 0 }; + uint8_t cval[2][YAPF__MAX_PLANES]; + uint8_t xfs[2] = { 0, 0 }; + + /* t = 0: YCoCg-R, 1: subtract-green, 2: none (only with 3+ channels; + * with 1–2 channels the planes are the channels). */ + for (int t = 0; t < tries; t++) { + for (uint32_t y = 0; y < th; y++) { + const uint8_t *row = pixels + ((size_t)(ty + y) * img_w + tx) * (size_t)ch * sb; + for (uint32_t x = 0; x < tw; x++) { + uint32_t v[4], i = y * tw + x; + for (int c = 0; c < ch; c++) { + v[c] = yapf__get_sample(row + ((size_t)x * ch + c) * sb, sample); + if (isf) v[c] = yapf__fmap(v[c], bits); + } + if (ch >= 3 && t == 0) { + uint32_t r = v[0], g = v[1], b = v[2]; + uint32_t o_ = (r - b) & m, t_ = (b + yapf__sar1(o_, bits)) & m; + uint32_t q = (g - t_) & m; + v[0] = (t_ + yapf__sar1(q, bits)) & m; + v[1] = (o_ + half) & m; + v[2] = (q + half) & m; + } else if (ch >= 3 && t == 1) { + uint32_t r = v[0], g = v[1], b = v[2]; + v[0] = g; + v[1] = (r - g + half) & m; + v[2] = (b - g + half) & m; + } + for (int c = 0; c < ch; c++) sc->val[c][i] = v[c]; + } + } + /* planes, constant mask, filters, size estimate */ + uint8_t (*plane)[YAPF__TILE_PX] = sc->plane[cur]; + uint32_t est = 0, cm = 0; + for (int c = 0; c < ch; c++) + for (int k = 0; k < sb; k++) { + int pl = c * sb + k, sh = 8 * (sb - 1 - k); + uint8_t *P = plane[pl]; + for (uint32_t i = 0; i < total; i++) P[i] = (uint8_t)(sc->val[c][i] >> sh); + uint32_t i = 1; + while (i < total && P[i] == P[0]) i++; + if (i == total) { cm |= 1u << pl; cval[cur][pl] = P[0]; est += 16; continue; } + est += yapf__choose_filters(P, tw, th, sc->z[cur][pl], sc->filt[cur][pl]); + } + if (est < best_est) { + best_est = est; + cmask[cur] = cm; + xfs[cur] = (uint8_t)t; + cur ^= 1; /* keep this set, work in the other one */ + } + } + int keep = cur ^ 1; + uint8_t hdr = ch < 3 ? 0 : xfs[keep] == 1 ? YAPF__SUBGREEN : xfs[keep] == 2 ? YAPF__NOXFORM : 0; + + yapf__planeset ps; + memset(&ps, 0, sizeof(ps)); + ps.np = np; + ps.wide = 1; + ps.cmask = cmask[keep]; + ps.cval = cval[keep]; + for (int c = 0; c < np; c++) { ps.z[c] = sc->z[keep][c]; ps.filt[c] = sc->filt[keep][c]; } + if (yapf__emit_planeset(o, tmp, hdr, &ps, tw, th, mode, eb) != 0) return -1; + + if (o->len - start > raw + 1) { /* stored: raw LE samples */ + o->len = start; + if (yapf__reserve(o, raw + 1) != 0) return -1; + o->buf[o->len++] = YAPF__STORED; + for (uint32_t y = 0; y < th; y++) { + const uint8_t *row = pixels + ((size_t)(ty + y) * img_w + tx) * (size_t)ch * sb; + for (uint32_t i = 0; i < tw * (uint32_t)ch; i++) { + uint32_t v = yapf__get_sample(row + (size_t)i * sb, sample); + for (int k = 0; k < sb; k++) o->buf[o->len++] = (uint8_t)(v >> (8 * k)); + } + } + } + return 0; +} + +typedef struct { + uint8_t plane[YAPF__MAX_PLANES][YAPF_TILE_SIZE * YAPF__ROW]; + uint8_t z[YAPF__TILE_PX + YAPF_TILE_SIZE]; +} yapf__wdec; + +static int yapf__decode_tile_wide( + const uint8_t *data, size_t len, + uint8_t *pixels, uint32_t img_w, + uint32_t tx, uint32_t ty, uint32_t tw, uint32_t th, + int ch, int sample, int simd, const yapf__dbook *bk, yapf__wdec *sc) +{ + int sb = (int)yapf__sbytes(sample), bits = 8 * sb, np = ch * sb; + int isf = sample == YAPF_SAMPLE_F16 || sample == YAPF_SAMPLE_F32; + uint32_t half = 1u << (bits - 1), m = yapf__vmask(bits); + const uint8_t *p = data, *end = data + len; + size_t rowb = (size_t)tw * ch * sb; + + if (len == 0) return -1; + uint8_t hdr = *p++; + if (hdr == YAPF__STORED) { + if (len != 1 + rowb * th) return -1; + for (uint32_t y = 0; y < th; y++) { + uint8_t *d = pixels + ((size_t)(ty + y) * img_w + tx) * (size_t)ch * sb; + for (uint32_t i = 0; i < tw * (uint32_t)ch; i++, p += sb) { + uint32_t v = 0; + for (int k = sb; k-- > 0; ) v = (v << 8) | p[k]; + yapf__put_sample(d + (size_t)i * sb, sample, v); + } + } + return 0; + } + if (hdr & 0x8F) return -1; + if ((hdr & YAPF__NOXFORM) && (hdr & YAPF__SUBGREEN)) return -1; + if (ch < 3 && (hdr & (YAPF__NOXFORM | YAPF__SUBGREEN))) return -1; + int xf = (hdr & YAPF__NOXFORM) ? 2 : (hdr & YAPF__SUBGREEN) ? 1 : 0; + if (ch < 3) xf = 2; + + if (end - p < 2) return -1; + uint32_t cmask = yapf__r16le(p), emask = 0; + p += 2; + if (cmask >> np) return -1; + if (hdr & YAPF__ENTROPY) { + if (end - p < 2 || !bk) return -1; + emask = yapf__r16le(p); + p += 2; + if ((emask >> np) || (emask & cmask)) return -1; + } + for (int pl = 0; pl < np; pl++) { + if (!(cmask & (1u << pl))) continue; + if (p >= end) return -1; + memset(sc->plane[pl], *p++, sizeof(sc->plane[pl])); + } + for (int pl = 0; pl < np; pl++) { + if (cmask & (1u << pl)) continue; + const uint8_t *filt = NULL; + memset(sc->z + (size_t)tw * th, 0, sizeof(sc->z) - (size_t)tw * th); + p = (emask & (1u << pl)) + ? yapf__huff_plane(p, end, sc->z, &filt, tw, th, bk, (ch < 3 ? 0 : xf) * 16 + pl) + : yapf__unpack_plane(p, end, sc->z, &filt, tw, th, simd); + if (!p) return -1; + if (tw != YAPF_TILE_SIZE) { + size_t used = ((size_t)tw * th + YAPF__GROUP - 1) / YAPF__GROUP * YAPF__GROUP; + memset(sc->z + used, 0, sizeof(sc->z) - used); + } + uint8_t *P = sc->plane[pl]; + for (uint32_t y = 0; y < th; y++) { + uint8_t f = filt[y / YAPF__BAND]; + uint32_t k = f >= YAPF__F_UP ? (uint32_t)(f - YAPF__F_UP + 1) : 1; + uint8_t *R = P + (size_t)y * YAPF__ROW; + const uint8_t *U = y ? R - YAPF__ROW : yapf__zero_row; + const uint8_t *K = y >= k ? R - (size_t)k * YAPF__ROW : yapf__zero_row; + const uint8_t *Z = sc->z + (size_t)y * tw; +#if YAPF__SIMD + if (simd) { + if (f == YAPF__F_LEFT || f == YAPF__F_GRAD) + yapf__simd_serial_row(f, R, U, Z, tw); + else if (f >= YAPF__F_UP) + yapf__simd_vertical_row(R, K, K, Z, tw); + else if (f == YAPF__F_UPAVG) + yapf__simd_vertical_row(R, U, U + 1, Z, tw); + else + yapf__simd_vertical_row(R, U + 1, U + 1, Z, tw); + R[tw] = R[tw - 1]; + continue; + } +#endif + yapf__unfilter_row(f, R, U, K, Z, tw); + } + } + if (p != end) return -1; + + for (uint32_t y = 0; y < th; y++) { + uint8_t *d = pixels + ((size_t)(ty + y) * img_w + tx) * (size_t)ch * sb; + size_t r = (size_t)y * YAPF__ROW; + for (uint32_t x = 0; x < tw; x++) { + uint32_t v[4]; + for (int c = 0; c < ch; c++) { + uint32_t a = 0; + for (int k = 0; k < sb; k++) a = (a << 8) | sc->plane[c * sb + k][r + x]; + v[c] = a; + } + if (xf == 0) { + uint32_t o_ = (v[1] - half) & m, q = (v[2] - half) & m; + uint32_t t_ = (v[0] - yapf__sar1(q, bits)) & m; + uint32_t g = (q + t_) & m; + uint32_t b = (t_ - yapf__sar1(o_, bits)) & m; + v[0] = (b + o_) & m; + v[1] = g; + v[2] = b; + } else if (xf == 1) { + uint32_t g = v[0]; + v[0] = (v[1] + g - half) & m; + v[2] = (v[2] + g - half) & m; + v[1] = g; + } + for (int c = 0; c < ch; c++) + yapf__put_sample(d + ((size_t)x * ch + c) * sb, sample, + isf ? yapf__funmap(v[c], bits) : v[c]); + } + } + return 0; +} + +/* ── GPU block tiles (version 2) ─────────────────────────────────────── */ +/* + * A block tile holds up to 16 × 16 blocks, rows of blocks top to bottom, + * exactly as the GPU takes them. Packed tiles (bit 31 of the tile size) + * code the bytes lane by lane — byte 0 of every block, then byte 1, … — + * each lane with its own table of the surface's code book (slot = lane): + * endpoint bytes and index bytes have very different statistics. The + * bit stream fills the tile exactly. + */ +static int yapf__decode_block_tile(const uint8_t *src, size_t len, int packed, + uint8_t *dst, size_t dst_row, + uint32_t tbw, uint32_t tbh, uint32_t S, + const yapf__dbook *bk) +{ + size_t n = (size_t)tbw * tbh, raw = n * S; + if (!packed) { + if (len != raw) return -1; + for (uint32_t r = 0; r < tbh; r++) + memcpy(dst + r * dst_row, src + (size_t)r * tbw * S, (size_t)tbw * S); + return 0; + } + if (!bk) return -1; + yapf__br br; + yapf__br_init(&br, src, len); + for (uint32_t k = 0; k < S; k++) { + const uint16_t *lut = bk->slot[k]; + if (!lut) return -1; + for (size_t i = 0; i < n; i++) { + int v = yapf__br_sym(&br, lut); + if (v < 0) return -1; + dst[(i / tbw) * dst_row + (i % tbw) * S + k] = (uint8_t)v; + } + } + return yapf__br_done(&br) ? 0 : -1; +} + +/* Writes one block tile; *packed is set when the packed form is used. */ +static int yapf__encode_block_tile(yapf__out *o, const uint8_t *src, size_t src_row, + uint32_t tbw, uint32_t tbh, uint32_t S, + int mode, yapf__ebook *eb, int *packed) +{ + size_t n = (size_t)tbw * tbh, raw = n * S; + *packed = 0; + if (mode == YAPF__ENC_STATS) + for (uint32_t r = 0; r < tbh; r++) + for (size_t i = 0; i < (size_t)tbw * S; i++) + eb->freq[i % S][0][src[r * src_row + i]]++; + if (mode == YAPF__ENC_BOOK) { + uint64_t bits = 0; + for (uint32_t r = 0; r < tbh && bits != UINT64_MAX; r++) + for (size_t i = 0; i < (size_t)tbw * S; i++) { + uint8_t l = eb->len[i % S][0][src[r * src_row + i]]; + if (!l) { bits = UINT64_MAX; break; } + bits += l; + } + if (bits != UINT64_MAX && (bits + 7) / 8 < raw) { + if (yapf__reserve(o, (size_t)(bits + 7) / 8 + 8) != 0) return -1; + yapf__bitw bw = { 0, 0, o }; + for (uint32_t k = 0; k < S; k++) /* lane by lane */ + for (size_t i = 0; i < n; i++) { + uint8_t v = src[(i / tbw) * src_row + (i % tbw) * S + k]; + yapf__bitw_put(&bw, eb->code[k][0][v], eb->len[k][0][v]); + } + yapf__bitw_flush(&bw); + *packed = 1; + return 0; + } + } + if (yapf__reserve(o, raw) != 0) return -1; + for (uint32_t r = 0; r < tbh; r++) { + memcpy(o->buf + o->len, src + r * src_row, (size_t)tbw * S); + o->len += (size_t)tbw * S; + } + return 0; +} /* Encoder scratch, heap allocated once per level (too big for small * thread stacks). */ @@ -867,7 +1808,7 @@ static int yapf__encode_tile( yapf__out *o, yapf__scratch *sc, const uint8_t *pixels, uint32_t img_w, uint32_t tx, uint32_t ty, uint32_t tw, uint32_t th, - int ch) + int ch, int mode, yapf__ebook *eb, yapf__out *tmp) { uint32_t total = tw * th; size_t start = o->len; @@ -909,6 +1850,21 @@ static int yapf__encode_tile( } int t = (tries == 2 && est[1] < est[0]) ? 1 : 0; + if (mode != YAPF__ENC_PLAIN) { /* v2: statistics / entropy planes */ + yapf__planeset ps; + uint8_t cval[4]; + memset(&ps, 0, sizeof(ps)); + ps.np = ch; + ps.cmask = mask[t]; + ps.cval = cval; + for (int c = 0; c < ch; c++) { + cval[c] = sc->plane[t][c][0]; + ps.z[c] = sc->z[t][c]; + ps.filt[c] = sc->filt[t][c]; + } + if (yapf__emit_planeset(o, tmp, (uint8_t)(mask[t] | (t ? YAPF__SUBGREEN : 0)), + &ps, tw, th, mode, eb) != 0) return -1; + } else { if (yapf__reserve(o, 1 + 4) != 0) return -1; o->buf[o->len++] = (uint8_t)(mask[t] | (t ? YAPF__SUBGREEN : 0)); for (int c = 0; c < ch; c++) @@ -916,6 +1872,7 @@ static int yapf__encode_tile( for (int c = 0; c < ch; c++) if (!(mask[t] & (1u << c)) && yapf__emit_plane(o, sc->z[t][c], sc->filt[t][c], tw, th) != 0) return -1; + } /* Fall back to a stored tile if coding did not pay off. */ if (o->len - start > raw + 1) { @@ -931,11 +1888,14 @@ static int yapf__encode_tile( return 0; } +/* v2 (version 2 files) additionally allows bit 5 (a mask byte follows: + * bit c = plane c is entropy coded with book bk) and bit 6 (no colour + * transform). */ static int yapf__decode_tile( const uint8_t *data, size_t len, uint8_t *pixels, uint32_t img_w, uint32_t tx, uint32_t ty, uint32_t tw, uint32_t th, - int ch, int simd) + int ch, int simd, int v2, const yapf__dbook *bk) { uint8_t rows[4][YAPF__RING][YAPF_TILE_SIZE + 1]; /* last rows, ring */ uint8_t z[4][YAPF__TILE_PX + YAPF__GROUP]; @@ -953,19 +1913,31 @@ static int yapf__decode_tile( p + rowb * y, rowb); return 0; } - if (hdr & 0xE0) return -1; /* reserved bits set */ - uint8_t xf = hdr & YAPF__SUBGREEN; + if (hdr & (v2 ? 0x80 : 0xE0)) return -1; /* reserved bits set */ + if ((hdr & YAPF__NOXFORM) && (hdr & YAPF__SUBGREEN)) return -1; + uint8_t xf = (hdr & YAPF__NOXFORM) ? 2 : (hdr & YAPF__SUBGREEN) ? 1 : 0; + uint8_t emask = 0; + int ent = hdr & YAPF__ENTROPY; hdr &= 0x0F; if ((hdr >> ch) || (xf && ch < 3)) return -1; /* unknown bits set */ + if (ent) { + if (p >= end || !bk) return -1; /* entropy needs the surface book */ + emask = *p++; + if ((emask >> ch) || (emask & hdr)) return -1; + } for (int c = 0; c < ch; c++) { if (!(hdr & (1u << c))) continue; if (p >= end) return -1; memset(rows[c], *p++, sizeof(rows[c])); } + yapf__hstream hs[4]; + int nh = 0; for (int c = 0; c < ch; c++) { if (hdr & (1u << c)) continue; - p = yapf__unpack_plane(p, end, z[c], &filt[c], tw, th, simd); + p = (emask & (1u << c)) + ? yapf__huff_head(p, end, &filt[c], th, bk, xf * 16 + c, z[c], &hs[nh++]) + : yapf__unpack_plane(p, end, z[c], &filt[c], tw, th, simd); if (!p) return -1; /* SIMD row kernels read whole 16-sample chunks; keep the unused * tail of narrow tiles defined. */ @@ -975,6 +1947,7 @@ static int yapf__decode_tile( } } if (p != end) return -1; + if (nh && yapf__huff_run(hs, nh, tw, th) != 0) return -1; /* Row by row: undo the filters of every plane (independent dependency * chains the CPU can overlap), then the colour transform while the @@ -1042,7 +2015,9 @@ static int yapf__decode_tile( if (ch == 3) { /* Planes → separate R, G, B rows (vectorises), then interleave. */ uint8_t rgb[3][YAPF_TILE_SIZE]; - if (xf) { + if (xf == 2) { + memcpy(rgb[0], P0, tw); memcpy(rgb[1], P1, tw); memcpy(rgb[2], P2, tw); + } else if (xf) { for (uint32_t x = 0; x < tw; x++) { uint8_t g = P0[x]; rgb[0][x] = (uint8_t)(P1[x] + g - 128); @@ -1063,6 +2038,10 @@ static int yapf__decode_tile( for (uint32_t x = 0; x < tw; x++, d += 3) { d[0] = rgb[0][x]; d[1] = rgb[1][x]; d[2] = rgb[2][x]; } + } else if (xf == 2) { + for (uint32_t x = 0; x < tw; x++, d += 4) { + d[0] = P0[x]; d[1] = P1[x]; d[2] = P2[x]; d[3] = P3[x]; + } } else if (xf) { YAPF__INVERSE_SG(4, d[3] = P3[x]) } else if (ch == 4) { @@ -1114,7 +2093,7 @@ static int yapf__encode_level( uint32_t tw = mw - x0 < YAPF_TILE_SIZE ? mw - x0 : YAPF_TILE_SIZE; uint32_t th = mh - y0 < YAPF_TILE_SIZE ? mh - y0 : YAPF_TILE_SIZE; size_t start = o->len; - rc = yapf__encode_tile(o, sc, pixels, mw, x0, y0, tw, th, ch); + rc = yapf__encode_tile(o, sc, pixels, mw, x0, y0, tw, th, ch, YAPF__ENC_PLAIN, NULL, NULL); if (rc == 0) yapf__w32le(o->buf + t * 4, (uint32_t)(o->len - start)); } } @@ -1122,17 +2101,56 @@ static int yapf__encode_level( return rc; } -/* A level whose tile table has been validated; tiles can then be decoded - * in any order, from any thread. */ +/* A level (surface) whose tile table has been validated; tiles can then + * be decoded in any order, from any thread. Version 1 levels are one + * 8-bit slice with 4-byte table entries; version 2 adds slices, wide + * samples, block tiles and 8-byte entries (size, yapf-sum). */ typedef struct { const uint8_t *data; size_t *offs; /* tcount + 1 byte offsets into data */ uint8_t *pixels; - uint32_t mw, mh, tcx, tcy; + uint32_t mw, mh, md, tcx, tcy; /* tcy: tile rows per slice */ int ch; + int sample; /* YAPF_SAMPLE_* (pixel payload) */ int simd; /* SIMD kernels usable on this CPU */ + int v2; + int verify; /* check tile yapf-sums */ + uint32_t blk; /* bytes per GPU block, 0 = pixel payload */ + uint32_t nbx, nby; /* blocks across / down */ + yapf__dbook *book; /* the surface's code book, or NULL */ } yapf__level; +static void yapf__level_free(yapf__level *lv) { + free(lv->offs); + lv->offs = NULL; + if (lv->book) { free(lv->book->mem); free(lv->book); lv->book = NULL; } +} + +/* Parses the tile table; lv must have data, the grid and v2 set. */ +static int yapf__level_table(yapf__level *lv, size_t len) { + uint64_t tc64 = (uint64_t)lv->tcx * lv->tcy * lv->md; + size_t ent = lv->v2 ? 8 : 4; + if ((uint64_t)(len / ent) < tc64) return -1; + size_t tcount = (size_t)tc64; + lv->offs = (size_t *)malloc((tcount + 1) * sizeof(size_t)); + if (!lv->offs) return -1; + + size_t pos = tcount * ent; + for (size_t t = 0; t < tcount; t++) { + uint32_t sz = yapf__r32le(lv->data + t * ent); + if (lv->v2) { + if ((sz >> 31) && (!lv->blk || !lv->book)) return -1; /* packed: blocks + book */ + sz &= 0x7FFFFFFFu; + } + if (sz > len - pos) return -1; + lv->offs[t] = pos; + pos += sz; + } + lv->offs[tcount] = pos; + if (lv->v2 && pos != len) return -1; + return 0; +} + static int yapf__level_init(yapf__level *lv, const uint8_t *data, size_t len, uint8_t *pixels, uint32_t mw, uint32_t mh, int ch) { @@ -1140,44 +2158,80 @@ static int yapf__level_init(yapf__level *lv, const uint8_t *data, size_t len, lv->pixels = pixels; lv->mw = mw; lv->mh = mh; + lv->md = 1; lv->tcx = yapf__tiles(mw); lv->tcy = yapf__tiles(mh); lv->ch = ch; - - size_t tcount = (size_t)lv->tcx * lv->tcy; - if (len / 4 < tcount) return -1; - lv->offs = (size_t *)malloc((tcount + 1) * sizeof(size_t)); - if (!lv->offs) return -1; - - size_t pos = tcount * 4; - for (size_t t = 0; t < tcount; t++) { - uint32_t sz = yapf__r32le(data + t * 4); - if (sz > len - pos) return -1; - lv->offs[t] = pos; - pos += sz; - } - lv->offs[tcount] = pos; - return 0; + lv->sample = YAPF_SAMPLE_U8; + return yapf__level_table(lv, len); } -/* Decode tile rows first_row, first_row + step, ... */ +/* Decodes tile (tx, row) of a level, row counting across all slices. + * dst == NULL writes into the level's pixels; otherwise into dst with + * dst_w texels (or blocks) per row, the tile at its top-left corner. + * Returns 0, -1 (corrupt) or YAPF_ERR_CHECKSUM. */ +static int yapf__level_tile(const yapf__level *lv, uint32_t tx, uint32_t row, + yapf__wdec *wsc, uint8_t *dst, size_t dst_w) +{ + size_t t = (size_t)row * lv->tcx + tx; + uint32_t z = row / lv->tcy, ty = row % lv->tcy; + const uint8_t *src = lv->data + lv->offs[t]; + size_t len = lv->offs[t + 1] - lv->offs[t]; + + if (lv->v2 && lv->verify && yapf_checksum(src, len) != yapf__r32le(lv->data + t * 8 + 4)) + return YAPF_ERR_CHECKSUM; + + if (lv->blk) { + uint32_t bx0 = tx * YAPF_BLOCK_TILE, by0 = ty * YAPF_BLOCK_TILE; + uint32_t tbw = lv->nbx - bx0 < YAPF_BLOCK_TILE ? lv->nbx - bx0 : YAPF_BLOCK_TILE; + uint32_t tbh = lv->nby - by0 < YAPF_BLOCK_TILE ? lv->nby - by0 : YAPF_BLOCK_TILE; + int packed = (int)(yapf__r32le(lv->data + t * 8) >> 31); + size_t rowb; + if (!dst) { + rowb = (size_t)lv->nbx * lv->blk; + dst = lv->pixels + (size_t)z * lv->nby * rowb + (size_t)by0 * rowb + (size_t)bx0 * lv->blk; + } else { + rowb = dst_w * lv->blk; + } + return yapf__decode_block_tile(src, len, packed, dst, rowb, tbw, tbh, lv->blk, lv->book); + } + + uint32_t x0 = tx * YAPF_TILE_SIZE, y0 = ty * YAPF_TILE_SIZE; + uint32_t tw = lv->mw - x0 < YAPF_TILE_SIZE ? lv->mw - x0 : YAPF_TILE_SIZE; + uint32_t th = lv->mh - y0 < YAPF_TILE_SIZE ? lv->mh - y0 : YAPF_TILE_SIZE; + size_t px = yapf__sbytes(lv->sample) * (size_t)lv->ch; + uint8_t *base; + uint32_t img_w; + if (!dst) { + base = lv->pixels + (size_t)z * lv->mw * lv->mh * px; + img_w = lv->mw; + } else { + base = dst; + img_w = (uint32_t)dst_w; + x0 = y0 = 0; + } + if (lv->sample == YAPF_SAMPLE_U8) + return yapf__decode_tile(src, len, base, img_w, x0, y0, tw, th, lv->ch, lv->simd, lv->v2, lv->book); + return yapf__decode_tile_wide(src, len, base, img_w, x0, y0, tw, th, lv->ch, + lv->sample, lv->simd, lv->book, wsc); +} + +/* Decode tile rows first_row, first_row + step, ... (over all slices) */ static int yapf__level_decode_rows(const yapf__level *lv, uint32_t first_row, uint32_t step) { - for (uint32_t ty = first_row; ty < lv->tcy; ty += step) { - for (uint32_t tx = 0; tx < lv->tcx; tx++) { - size_t t = (size_t)ty * lv->tcx + tx; - uint32_t x0 = tx * YAPF_TILE_SIZE, y0 = ty * YAPF_TILE_SIZE; - uint32_t tw = lv->mw - x0 < YAPF_TILE_SIZE ? lv->mw - x0 : YAPF_TILE_SIZE; - uint32_t th = lv->mh - y0 < YAPF_TILE_SIZE ? lv->mh - y0 : YAPF_TILE_SIZE; - if (yapf__decode_tile(lv->data + lv->offs[t], - lv->offs[t + 1] - lv->offs[t], - lv->pixels, lv->mw, x0, y0, tw, th, lv->ch, - lv->simd) != 0) - return -1; - } + uint32_t rows = lv->tcy * lv->md; + yapf__wdec *wsc = NULL; + int rc = 0; + if (!lv->blk && lv->sample != YAPF_SAMPLE_U8 && first_row < rows) { + wsc = (yapf__wdec *)malloc(sizeof(yapf__wdec)); + if (!wsc) return YAPF_ERR_OOM; } - return 0; + for (uint32_t ty = first_row; ty < rows && rc == 0; ty += step) + for (uint32_t tx = 0; tx < lv->tcx && rc == 0; tx++) + rc = yapf__level_tile(lv, tx, ty, wsc, NULL, 0); + free(wsc); + return rc; } /* ── Threads ─────────────────────────────────────────────────────────── */ @@ -1257,7 +2311,12 @@ static int yapf__decode_levels(const yapf__level *levels, int nlevels, yapf__thread th[YAPF__MAX_THREADS]; n = threads > 0 ? threads : yapf__cpu_count(); if (n > YAPF__MAX_THREADS) n = YAPF__MAX_THREADS; - if ((uint32_t)n > levels[0].tcy) n = (int)levels[0].tcy; + { + uint32_t rows = 0; + for (int m = 0; m < nlevels; m++) + if (levels[m].tcy * levels[m].md > rows) rows = levels[m].tcy * levels[m].md; + if ((uint32_t)n > rows) n = (int)rows; + } /* Starting a thread costs about as much as decoding a few tiles, so * give each one at least YAPF__TILES_PER_THREAD tiles of work. */ { @@ -1291,7 +2350,7 @@ static int yapf__decode_levels(const yapf__level *levels, int nlevels, #endif for (int i = 0; i < n; i++) - if (jobs[i].rc != 0) return -1; + if (jobs[i].rc != 0) return jobs[i].rc; return 0; } @@ -1299,9 +2358,12 @@ static int yapf__decode_levels(const yapf__level *levels, int nlevels, * yapf_load_memory / yapf_load * ======================================================================== */ +static yapf_image_t *yapf__load_v2_image(const uint8_t *buf, size_t size, int threads); + static yapf_image_t *yapf__load(const uint8_t *buf, size_t size, int threads) { if (!buf || size < YAPF__HDR_LEN) return NULL; if (memcmp(buf, YAPF__MAGIC, 4) != 0) return NULL; + if (buf[4] == 2) return yapf__load_v2_image(buf, size, threads); uint8_t version = buf[4]; uint8_t channels = buf[5]; @@ -1373,7 +2435,7 @@ yapf_image_t *yapf_load_memory_mt(const void *buffer, size_t size, int threads) return yapf__load((const uint8_t *)buffer, size, threads); } -static yapf_image_t *yapf__load_file(const char *filename, int threads) { +static uint8_t *yapf__read_file(const char *filename, size_t *out_len) { if (!filename) return NULL; FILE *f = fopen(filename, "rb"); if (!f) return NULL; @@ -1394,7 +2456,14 @@ static yapf_image_t *yapf__load_file(const char *filename, int threads) { fclose(f); if (!buf) return NULL; if (err) { free(buf); return NULL; } + *out_len = len; + return buf; +} +static yapf_image_t *yapf__load_file(const char *filename, int threads) { + size_t len = 0; + uint8_t *buf = yapf__read_file(filename, &len); + if (!buf) return NULL; yapf_image_t *img = yapf__load(buf, len, threads); free(buf); return img; @@ -1513,3 +2582,2098 @@ void yapf_free(yapf_image_t *img) { } free(img); } + +/* ======================================================================== + * Version 2: textures + * ======================================================================== */ + +#define YAPF__V2_HDR 64 +#define YAPF__V2_SURFACE 16 +#define YAPF__MAX_SURFACES (1u << 24) + +static const struct { uint8_t fmt, bw, bh, bytes; } yapf__block_formats[] = { + { YAPF_GPU_BC1, 4, 4, 8 }, { YAPF_GPU_BC2, 4, 4, 16 }, + { YAPF_GPU_BC3, 4, 4, 16 }, { YAPF_GPU_BC4, 4, 4, 8 }, + { YAPF_GPU_BC4_SNORM, 4, 4, 8 }, { YAPF_GPU_BC5, 4, 4, 16 }, + { YAPF_GPU_BC5_SNORM, 4, 4, 16 }, { YAPF_GPU_BC6H, 4, 4, 16 }, + { YAPF_GPU_BC6H_SF, 4, 4, 16 }, { YAPF_GPU_BC7, 4, 4, 16 }, + { YAPF_GPU_ETC2_RGB8, 4, 4, 8 }, { YAPF_GPU_ETC2_RGB8A1, 4, 4, 8 }, + { YAPF_GPU_ETC2_RGBA8, 4, 4, 16 }, { YAPF_GPU_EAC_R11, 4, 4, 8 }, + { YAPF_GPU_EAC_R11_SNORM, 4, 4, 8 }, { YAPF_GPU_EAC_RG11, 4, 4, 16 }, + { YAPF_GPU_EAC_RG11_SNORM, 4, 4, 16 }, + { YAPF_GPU_ASTC_4x4, 4, 4, 16 }, { YAPF_GPU_ASTC_5x4, 5, 4, 16 }, + { YAPF_GPU_ASTC_5x5, 5, 5, 16 }, { YAPF_GPU_ASTC_6x5, 6, 5, 16 }, + { YAPF_GPU_ASTC_6x6, 6, 6, 16 }, { YAPF_GPU_ASTC_8x5, 8, 5, 16 }, + { YAPF_GPU_ASTC_8x6, 8, 6, 16 }, { YAPF_GPU_ASTC_8x8, 8, 8, 16 }, + { YAPF_GPU_ASTC_10x5, 10, 5, 16 }, { YAPF_GPU_ASTC_10x6, 10, 6, 16 }, + { YAPF_GPU_ASTC_10x8, 10, 8, 16 }, { YAPF_GPU_ASTC_10x10, 10, 10, 16 }, + { YAPF_GPU_ASTC_12x10, 12, 10, 16 }, { YAPF_GPU_ASTC_12x12, 12, 12, 16 } +}; + +int yapf_block_info(uint8_t format, uint32_t *bw, uint32_t *bh, uint32_t *bytes) { + for (size_t i = 0; i < sizeof(yapf__block_formats) / sizeof(yapf__block_formats[0]); i++) + if (yapf__block_formats[i].fmt == format) { + if (bw) *bw = yapf__block_formats[i].bw; + if (bh) *bh = yapf__block_formats[i].bh; + if (bytes) *bytes = yapf__block_formats[i].bytes; + return 1; + } + return 0; +} + +size_t yapf_sample_bytes(uint8_t sample) { + return sample <= YAPF_SAMPLE_F32 ? yapf__sbytes(sample) : 0; +} + +/* Library-side state of a texture: the file tables of an opened file, + * and the marker that the library owns data, notes and frame times. */ +typedef struct { + int version; /* 1 or 2 (0: made by yapf_tex_create) */ + int flags; /* YAPF_LOAD_* */ + uint32_t needs; + uint64_t file_size, front; + uint64_t *off; + uint32_t *size, *tcheck; +} yapf__texi; + +static uint32_t yapf__faces(const yapf_texture_t *t) { + return t->shape == YAPF_SHAPE_CUBE ? 6u : 1u; +} + +uint32_t yapf_tex_faces(const yapf_texture_t *tex) { + return tex ? yapf__faces(tex) : 0; +} + +/* Checks a description; *count receives the number of surfaces. */ +static int yapf__tex_check(const yapf_texture_t *t, uint32_t *count) { + if (!t) return -1; + if (t->channels < 1 || t->channels > 4) return -1; + if (t->sample > YAPF_SAMPLE_F32 || t->payload > YAPF_PAYLOAD_BLOCKS) return -1; + if (t->shape > YAPF_SHAPE_VOLUME) return -1; + if (t->width == 0 || t->width > YAPF_MAX_DIM) return -1; + if (t->height == 0 || t->height > YAPF_MAX_DIM) return -1; + if (t->layers == 0 || t->frames == 0) return -1; + if (t->mip_levels < 1 || t->mip_levels > YAPF_MAX_MIPS) return -1; + if (t->shape == YAPF_SHAPE_VOLUME) { + if (t->depth == 0 || t->depth > YAPF_MAX_DIM || t->layers != 1) return -1; + } else if (t->depth > 1) { + return -1; + } + if (t->shape == YAPF_SHAPE_CUBE && t->width != t->height) return -1; + if (t->payload == YAPF_PAYLOAD_BLOCKS && !yapf_block_info(t->format, NULL, NULL, NULL)) + return -1; + uint64_t n = (uint64_t)t->frames * t->layers * yapf__faces(t) * t->mip_levels; + if (n > YAPF__MAX_SURFACES) return -1; + /* the largest surface must be addressable (32-bit builds) */ + uint64_t texels = (uint64_t)t->width * t->height * (t->shape == YAPF_SHAPE_VOLUME ? t->depth : 1); + if (texels * 4 * 4 > (uint64_t)(SIZE_MAX / 2)) return -1; + if (count) *count = (uint32_t)n; + return 0; +} + +uint32_t yapf_tex_surface_count(const yapf_texture_t *tex) { + uint32_t n = 0; + return yapf__tex_check(tex, &n) == 0 ? n : 0; +} + +uint32_t yapf_tex_surface(const yapf_texture_t *tex, uint32_t frame, + uint32_t layer, uint32_t face, uint32_t mip) +{ + return ((frame * tex->layers + layer) * yapf__faces(tex) + face) * tex->mip_levels + mip; +} + +void yapf_tex_mip_size(const yapf_texture_t *tex, uint32_t mip, + uint32_t *w, uint32_t *h, uint32_t *d) +{ + uint32_t a = tex->width >> mip, b = tex->height >> mip; + uint32_t c = tex->shape == YAPF_SHAPE_VOLUME ? tex->depth >> mip : 1; + if (w) *w = a ? a : 1; + if (h) *h = b ? b : 1; + if (d) *d = c ? c : 1; +} + +/* Blocks across and down for a mip of a block texture. */ +static void yapf__mip_blocks(const yapf_texture_t *t, uint32_t mip, + uint32_t *nbx, uint32_t *nby, uint32_t *S) +{ + uint32_t w, h, bw = 4, bh = 4, bytes = 16; + yapf_tex_mip_size(t, mip, &w, &h, NULL); + yapf_block_info(t->format, &bw, &bh, &bytes); + *nbx = (w + bw - 1) / bw; + *nby = (h + bh - 1) / bh; + *S = bytes; +} + +size_t yapf_tex_surface_bytes(const yapf_texture_t *tex, uint32_t mip) { + uint32_t w, h, d; + yapf_tex_mip_size(tex, mip, &w, &h, &d); + if (tex->payload == YAPF_PAYLOAD_BLOCKS) { + uint32_t nbx, nby, S; + yapf__mip_blocks(tex, mip, &nbx, &nby, &S); + return (size_t)nbx * nby * d * S; + } + return (size_t)w * h * d * tex->channels * yapf__sbytes(tex->sample); +} + +void yapf_tex_tile_grid(const yapf_texture_t *tex, uint32_t mip, + uint32_t *tiles_x, uint32_t *tiles_y) +{ + uint32_t tx, ty; + if (tex->payload == YAPF_PAYLOAD_BLOCKS) { + uint32_t nbx, nby, S; + yapf__mip_blocks(tex, mip, &nbx, &nby, &S); + tx = (nbx + YAPF_BLOCK_TILE - 1) / YAPF_BLOCK_TILE; + ty = (nby + YAPF_BLOCK_TILE - 1) / YAPF_BLOCK_TILE; + } else { + uint32_t w, h; + yapf_tex_mip_size(tex, mip, &w, &h, NULL); + tx = yapf__tiles(w); + ty = yapf__tiles(h); + } + if (tiles_x) *tiles_x = tx; + if (tiles_y) *tiles_y = ty; +} + +static yapf_texture_t *yapf__tex_new(void) { + yapf_texture_t *t = (yapf_texture_t *)calloc(1, sizeof(yapf_texture_t)); + if (!t) return NULL; + t->internal_ = calloc(1, sizeof(yapf__texi)); + if (!t->internal_) { free(t); return NULL; } + return t; +} + +void yapf_tex_free(yapf_texture_t *tex) { + if (!tex) return; + yapf__texi *ti = (yapf__texi *)tex->internal_; + uint32_t n = 0; + if (tex->data) { + n = (uint32_t)((uint64_t)tex->frames * tex->layers * yapf__faces(tex) * tex->mip_levels); + for (uint32_t s = 0; s < n; s++) free(tex->data[s]); + free(tex->data); + } + for (uint32_t i = 0; i < tex->note_count; i++) free(tex->notes[i].data); + free(tex->notes); + free(tex->frame_us); + if (ti) { free(ti->off); free(ti->size); free(ti->tcheck); free(ti); } + free(tex); +} + +/* Copies the description fields (everything up to loop_count). */ +static void yapf__tex_copy_desc(yapf_texture_t *d, const yapf_texture_t *s) { + d->width = s->width; d->height = s->height; + d->depth = s->shape == YAPF_SHAPE_VOLUME ? s->depth : 1; + d->layers = s->layers; d->frames = s->frames; d->shape = s->shape; + d->mip_levels = s->mip_levels; d->payload = s->payload; d->format = s->format; + d->sample = s->sample; d->channels = s->channels; d->flags = s->flags; + memcpy(d->channel_map, s->channel_map, 4); + d->linear_mask = s->linear_mask; d->normal = s->normal; + d->primaries = s->primaries; d->transfer = s->transfer; + d->mip_filter = s->mip_filter; d->alpha_ref = s->alpha_ref; + d->loop_count = s->loop_count; +} + +yapf_texture_t *yapf_tex_create(const yapf_texture_t *desc) { + yapf_texture_t probe; + uint32_t n; + if (!desc) return NULL; + probe = *desc; + if (probe.shape != YAPF_SHAPE_VOLUME && probe.depth == 0) probe.depth = 1; + if (yapf__tex_check(&probe, &n) != 0) return NULL; + yapf_texture_t *t = yapf__tex_new(); + if (!t) return NULL; + yapf__tex_copy_desc(t, &probe); + t->data = (void **)calloc(n, sizeof(void *)); + if (!t->data) { yapf_tex_free(t); return NULL; } + if (t->frames > 1) { + t->frame_us = (uint32_t *)calloc(t->frames, sizeof(uint32_t)); + if (!t->frame_us) { yapf_tex_free(t); return NULL; } + } + for (uint32_t s = 0; s < n; s++) { + t->data[s] = calloc(1, yapf_tex_surface_bytes(t, s % t->mip_levels)); + if (!t->data[s]) { yapf_tex_free(t); return NULL; } + } + return t; +} + +int yapf_tex_add_note(yapf_texture_t *tex, const char tag[4], + const void *data, uint32_t size) +{ + if (!tex || !tex->internal_ || !tag || (size && !data)) return YAPF_ERR_INVALID; + yapf_note_t *n = (yapf_note_t *)realloc(tex->notes, (tex->note_count + 1) * sizeof(yapf_note_t)); + if (!n) return YAPF_ERR_OOM; + tex->notes = n; + n += tex->note_count; + memcpy(n->tag, tag, 4); + n->size = size; + n->data = (uint8_t *)malloc(size ? size : 1); + if (!n->data) return YAPF_ERR_OOM; + if (size) memcpy(n->data, data, size); + tex->note_count++; + return YAPF_OK; +} + +const yapf_note_t *yapf_tex_find_note(const yapf_texture_t *tex, const char tag[4]) { + if (!tex || !tag) return NULL; + for (uint32_t i = 0; i < tex->note_count; i++) + if (memcmp(tex->notes[i].tag, tag, 4) == 0) return &tex->notes[i]; + return NULL; +} + +/* ── Writing ─────────────────────────────────────────────────────────── */ + +typedef struct { + yapf__scratch *sc; + yapf__wscratch *wsc; + yapf__ebook *eb; + yapf__out tmp; +} yapf__enc; + +/* u32 book size, the book, the tile table and the tiles of one surface. + * *checked receives the length the tile-table check covers. */ +static int yapf__encode_tiles(yapf__out *o, yapf__enc *e, const yapf_texture_t *t, + uint32_t mip, const uint8_t *src, int mode, + const yapf__out *book, size_t *checked) +{ + uint32_t w, h, d, tcx, tcy, nbx = 0, nby = 0, S = 0; + int blocks = t->payload == YAPF_PAYLOAD_BLOCKS; + int ch = t->channels, sb = (int)yapf__sbytes(t->sample); + yapf_tex_mip_size(t, mip, &w, &h, &d); + yapf_tex_tile_grid(t, mip, &tcx, &tcy); + if (blocks) yapf__mip_blocks(t, mip, &nbx, &nby, &S); + size_t n = (size_t)tcx * tcy * d, bl = book ? book->len : 0; + size_t slice = blocks ? (size_t)nbx * nby * S : (size_t)w * h * ch * sb; + + o->len = 0; + if (yapf__reserve(o, 4 + bl + n * 8) != 0) return -1; + yapf__w32le(o->buf, (uint32_t)bl); + if (bl) memcpy(o->buf + 4, book->buf, bl); + size_t table = 4 + bl; + memset(o->buf + table, 0, n * 8); + o->len = table + n * 8; + *checked = o->len; + + size_t t_i = 0; + for (uint32_t z = 0; z < d; z++) + for (uint32_t ty = 0; ty < tcy; ty++) + for (uint32_t tx = 0; tx < tcx; tx++, t_i++) { + size_t start = o->len; + int packed = 0, rc; + const uint8_t *sl = src + (size_t)z * slice; + if (blocks) { + uint32_t bx0 = tx * YAPF_BLOCK_TILE, by0 = ty * YAPF_BLOCK_TILE; + uint32_t tbw = nbx - bx0 < YAPF_BLOCK_TILE ? nbx - bx0 : YAPF_BLOCK_TILE; + uint32_t tbh = nby - by0 < YAPF_BLOCK_TILE ? nby - by0 : YAPF_BLOCK_TILE; + rc = yapf__encode_block_tile(o, sl + ((size_t)by0 * nbx + bx0) * S, + (size_t)nbx * S, tbw, tbh, S, mode, e->eb, &packed); + } else { + uint32_t x0 = tx * YAPF_TILE_SIZE, y0 = ty * YAPF_TILE_SIZE; + uint32_t tw = w - x0 < YAPF_TILE_SIZE ? w - x0 : YAPF_TILE_SIZE; + uint32_t th = h - y0 < YAPF_TILE_SIZE ? h - y0 : YAPF_TILE_SIZE; + rc = sb == 1 + ? yapf__encode_tile(o, e->sc, sl, w, x0, y0, tw, th, ch, mode, e->eb, &e->tmp) + : yapf__encode_tile_wide(o, &e->tmp, e->wsc, sl, w, x0, y0, tw, th, ch, + t->sample, mode, e->eb); + } + if (rc != 0) return -1; + size_t sz = o->len - start; + yapf__w32le(o->buf + table + t_i * 8, (uint32_t)sz | ((uint32_t)packed << 31)); + yapf__w32le(o->buf + table + t_i * 8 + 4, yapf_checksum(o->buf + start, sz)); + } + return 0; +} + +/* One surface; with entropy: a counting pass, the book, a coding pass, + * and the book version is kept only if it comes out smaller. */ +static int yapf__encode_surface(yapf__out *o, yapf__enc *e, const yapf_texture_t *t, + uint32_t mip, const uint8_t *src, int entropy, + size_t *checked, int *used_entropy) +{ + if (!entropy || !e->eb) + return yapf__encode_tiles(o, e, t, mip, src, YAPF__ENC_PLAIN, NULL, checked); + memset(e->eb->freq, 0, sizeof(e->eb->freq)); + if (yapf__encode_tiles(o, e, t, mip, src, YAPF__ENC_STATS, NULL, checked) != 0) return -1; + yapf__out book = { NULL, 0, 0 }, alt = { NULL, 0, 0 }; + size_t alt_checked = 0; + int rc = yapf__book_write(e->eb, &book); + if (rc == 0) rc = yapf__encode_tiles(&alt, e, t, mip, src, YAPF__ENC_BOOK, &book, &alt_checked); + if (rc == 0 && alt.len < o->len) { + yapf__out tmp = *o; + *o = alt; + alt = tmp; + *checked = alt_checked; + *used_entropy = 1; + } + free(book.buf); + free(alt.buf); + return rc < 0 ? -1 : 0; +} + +int yapf_tex_encode(const yapf_texture_t *tex, int flags, void **out_data, size_t *out_size) { + if (!out_data || !out_size) return YAPF_ERR_INVALID; + *out_data = NULL; + *out_size = 0; + uint32_t S; + yapf_texture_t t; + if (!tex) return YAPF_ERR_INVALID; + t = *tex; + if (t.shape != YAPF_SHAPE_VOLUME && t.depth == 0) t.depth = 1; + if (yapf__tex_check(&t, &S) != 0 || !t.data) return YAPF_ERR_INVALID; + for (uint32_t s = 0; s < S; s++) if (!t.data[s]) return YAPF_ERR_INVALID; + if (t.frames > 1 && !t.frame_us) return YAPF_ERR_INVALID; + uint64_t notes = 0; + for (uint32_t i = 0; i < t.note_count; i++) { + if (t.notes[i].size && !t.notes[i].data) return YAPF_ERR_INVALID; + notes += 8 + (uint64_t)t.notes[i].size; + } + if (notes > 0xFFFFFFFFull) return YAPF_ERR_INVALID; + + int err = YAPF_OK, used_ent = 0; + int entropy = (flags & YAPF_ENC_ENTROPY) != 0; + yapf__out *bufs = (yapf__out *)calloc(S, sizeof(yapf__out)); + uint32_t *tchk = (uint32_t *)calloc(S, sizeof(uint32_t)); + yapf__enc e; + memset(&e, 0, sizeof(e)); + if (!bufs || !tchk) err = YAPF_ERR_OOM; + if (err == YAPF_OK && t.payload == YAPF_PAYLOAD_PIXELS) { + if (t.sample == YAPF_SAMPLE_U8) e.sc = (yapf__scratch *)malloc(sizeof(yapf__scratch)); + else e.wsc = (yapf__wscratch *)malloc(sizeof(yapf__wscratch)); + if (!e.sc && !e.wsc) err = YAPF_ERR_OOM; + } + if (err == YAPF_OK && entropy && !(e.eb = (yapf__ebook *)malloc(sizeof(yapf__ebook)))) + err = YAPF_ERR_OOM; + for (uint32_t s = 0; s < S && err == YAPF_OK; s++) { + size_t n = 0; + if (yapf__encode_surface(&bufs[s], &e, &t, s % t.mip_levels, + (const uint8_t *)t.data[s], entropy, &n, &used_ent) != 0) + err = YAPF_ERR_OOM; + else + tchk[s] = yapf_checksum(bufs[s].buf, n); + } + + uint64_t front = YAPF__V2_HDR + (uint64_t)S * YAPF__V2_SURFACE + + (t.frames > 1 ? 4ull * t.frames : 0) + notes; + uint64_t total = front; + for (uint32_t s = 0; s < S && err == YAPF_OK; s++) total += bufs[s].len; + if (err == YAPF_OK && total > (uint64_t)SIZE_MAX) err = YAPF_ERR_INVALID; + + uint8_t *file = NULL; + if (err == YAPF_OK && !(file = (uint8_t *)malloc((size_t)total))) err = YAPF_ERR_OOM; + if (err == YAPF_OK) { + uint32_t needs = 0; + if (t.payload == YAPF_PAYLOAD_PIXELS && t.sample != YAPF_SAMPLE_U8) needs |= YAPF_NEED_WIDE; + if (t.payload == YAPF_PAYLOAD_BLOCKS) needs |= YAPF_NEED_BLOCKS; + if (used_ent) needs |= YAPF_NEED_ENTROPY; + if (t.shape != YAPF_SHAPE_2D || t.layers > 1) needs |= YAPF_NEED_LAYOUT; + if (t.frames > 1) needs |= YAPF_NEED_FRAMES; + + uint8_t *h = file; + memset(h, 0, YAPF__V2_HDR); + memcpy(h, YAPF__MAGIC, 4); + h[4] = 2; + h[5] = t.channels; h[6] = t.format; h[7] = t.mip_levels; + h[8] = t.flags; h[9] = t.payload; h[10] = t.sample; h[11] = t.shape; + yapf__w32le(h + 0x0C, t.width); + yapf__w32le(h + 0x10, t.height); + yapf__w32le(h + 0x14, t.depth); + yapf__w16le(h + 0x18, t.layers); + yapf__w16le(h + 0x1A, t.frames); + yapf__w32le(h + 0x1C, needs); + memcpy(h + 0x20, t.channel_map, 4); + h[0x24] = t.linear_mask; h[0x25] = t.normal; + h[0x26] = t.primaries; h[0x27] = t.transfer; + h[0x28] = t.mip_filter; h[0x29] = t.alpha_ref; + yapf__w16le(h + 0x2A, t.loop_count); + yapf__w32le(h + 0x2C, (uint32_t)notes); + yapf__w64le(h + 0x30, total); + + /* surface data, smallest mips first */ + uint64_t pos = front; + for (uint32_t m = t.mip_levels; m-- > 0; ) + for (uint32_t s = m; s < S; s += t.mip_levels) { + uint8_t *ent = file + YAPF__V2_HDR + (size_t)s * YAPF__V2_SURFACE; + yapf__w64le(ent, pos); + yapf__w32le(ent + 8, (uint32_t)bufs[s].len); + yapf__w32le(ent + 12, tchk[s]); + memcpy(file + pos, bufs[s].buf, bufs[s].len); + pos += bufs[s].len; + } + uint8_t *q = file + YAPF__V2_HDR + (size_t)S * YAPF__V2_SURFACE; + if (t.frames > 1) + for (uint32_t f = 0; f < t.frames; f++, q += 4) yapf__w32le(q, t.frame_us[f]); + for (uint32_t i = 0; i < t.note_count; i++) { + memcpy(q, t.notes[i].tag, 4); + yapf__w32le(q + 4, t.notes[i].size); + if (t.notes[i].size) memcpy(q + 8, t.notes[i].data, t.notes[i].size); + q += 8 + t.notes[i].size; + } + yapf__w32le(h + 0x38, yapf_checksum(file + YAPF__V2_HDR, (size_t)(front - YAPF__V2_HDR))); + yapf__w32le(h + 0x3C, yapf_checksum(file, 0x3C)); + *out_data = file; + *out_size = (size_t)total; + } + + if (bufs) for (uint32_t s = 0; s < S; s++) free(bufs[s].buf); + free(bufs); + free(tchk); + free(e.sc); + free(e.wsc); + free(e.eb); + free(e.tmp.buf); + return err; +} + +int yapf_tex_save(const char *filename, const yapf_texture_t *tex, int flags) { + if (!filename) return YAPF_ERR_INVALID; + void *data = NULL; + size_t size = 0; + int err = yapf_tex_encode(tex, flags, &data, &size); + if (err != YAPF_OK) return err; + FILE *f = fopen(filename, "wb"); + if (!f) err = YAPF_ERR_IO; + else { + if (fwrite(data, 1, size, f) != size) err = YAPF_ERR_IO; + if (fclose(f) != 0 && err == YAPF_OK) err = YAPF_ERR_IO; + } + yapf_free_buffer(data); + return err; +} + +/* ── Reading ─────────────────────────────────────────────────────────── */ + +static yapf_texture_t *yapf__open_fail(yapf_texture_t *t, int *err, int code) { + if (err) *err = code; + yapf_tex_free(t); + return NULL; +} + +/* Tiles in surface s (all slices). */ +static uint64_t yapf__surface_tiles(const yapf_texture_t *t, uint32_t mip) { + uint32_t tcx, tcy, d; + yapf_tex_tile_grid(t, mip, &tcx, &tcy); + yapf_tex_mip_size(t, mip, NULL, NULL, &d); + return (uint64_t)tcx * tcy * d; +} + +yapf_texture_t *yapf_tex_open(const void *buffer, size_t avail, int flags, int *err) { + const uint8_t *b = (const uint8_t *)buffer; + if (err) *err = YAPF_OK; + if (!b) return yapf__open_fail(NULL, err, YAPF_ERR_INVALID); + if (avail < 5) return yapf__open_fail(NULL, err, YAPF_ERR_TRUNCATED); + if (memcmp(b, YAPF__MAGIC, 4) != 0) return yapf__open_fail(NULL, err, YAPF_ERR_CORRUPT); + if (b[4] != 1 && b[4] != 2) return yapf__open_fail(NULL, err, YAPF_ERR_UNSUPPORTED); + + yapf_texture_t *t = yapf__tex_new(); + if (!t) return yapf__open_fail(NULL, err, YAPF_ERR_OOM); + yapf__texi *ti = (yapf__texi *)t->internal_; + ti->flags = flags; + uint32_t S; + + if (b[4] == 1) { + if (avail < YAPF__HDR_LEN) return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + t->channels = b[5]; t->format = b[6]; t->mip_levels = b[7]; t->flags = b[8]; + t->width = yapf__r32le(b + 12); t->height = yapf__r32le(b + 16); + t->depth = 1; t->layers = 1; t->frames = 1; + ti->version = 1; + if (yapf__tex_check(t, &S) != 0) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + ti->front = YAPF__HDR_LEN + (uint64_t)S * YAPF__MIP_ENTRY; + ti->file_size = avail; + if (avail < ti->front) return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + } else { + if (avail < YAPF__V2_HDR) return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + if (!(flags & YAPF_LOAD_TRUST) && yapf_checksum(b, 0x3C) != yapf__r32le(b + 0x3C)) + return yapf__open_fail(t, err, YAPF_ERR_CHECKSUM); + t->channels = b[5]; t->format = b[6]; t->mip_levels = b[7]; t->flags = b[8]; + t->payload = b[9]; t->sample = b[10]; t->shape = b[11]; + t->width = yapf__r32le(b + 0x0C); + t->height = yapf__r32le(b + 0x10); + t->depth = yapf__r32le(b + 0x14); + t->layers = yapf__r16le(b + 0x18); + t->frames = yapf__r16le(b + 0x1A); + ti->needs = yapf__r32le(b + 0x1C); + memcpy(t->channel_map, b + 0x20, 4); + t->linear_mask = b[0x24]; t->normal = b[0x25]; + t->primaries = b[0x26]; t->transfer = b[0x27]; + t->mip_filter = b[0x28]; t->alpha_ref = b[0x29]; + t->loop_count = yapf__r16le(b + 0x2A); + uint32_t notes = yapf__r32le(b + 0x2C); + ti->file_size = yapf__r64le(b + 0x30); + ti->version = 2; + + if (ti->needs & ~(uint32_t)YAPF_NEED_ALL) return yapf__open_fail(t, err, YAPF_ERR_UNSUPPORTED); + if (yapf__tex_check(t, &S) != 0) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + /* every feature in use must be announced */ + if ((t->payload == YAPF_PAYLOAD_PIXELS && t->sample != YAPF_SAMPLE_U8 && !(ti->needs & YAPF_NEED_WIDE)) || + (t->payload == YAPF_PAYLOAD_BLOCKS && !(ti->needs & YAPF_NEED_BLOCKS)) || + ((t->shape != YAPF_SHAPE_2D || t->layers > 1) && !(ti->needs & YAPF_NEED_LAYOUT)) || + (t->frames > 1 && !(ti->needs & YAPF_NEED_FRAMES))) + return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + + ti->front = YAPF__V2_HDR + (uint64_t)S * YAPF__V2_SURFACE + + (t->frames > 1 ? 4ull * t->frames : 0) + notes; + if (ti->front > ti->file_size) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + if (avail < ti->front) return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + if (!(flags & YAPF_LOAD_TRUST) && + yapf_checksum(b + YAPF__V2_HDR, (size_t)(ti->front - YAPF__V2_HDR)) != yapf__r32le(b + 0x38)) + return yapf__open_fail(t, err, YAPF_ERR_CHECKSUM); + + const uint8_t *q = b + YAPF__V2_HDR + (size_t)S * YAPF__V2_SURFACE; + if (t->frames > 1) { + t->frame_us = (uint32_t *)malloc(t->frames * sizeof(uint32_t)); + if (!t->frame_us) return yapf__open_fail(t, err, YAPF_ERR_OOM); + for (uint32_t f = 0; f < t->frames; f++, q += 4) t->frame_us[f] = yapf__r32le(q); + } + const uint8_t *qe = q + notes; + while (q < qe) { + if (qe - q < 8) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + uint32_t sz = yapf__r32le(q + 4); + if ((size_t)(qe - q - 8) < sz) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + if (yapf_tex_add_note(t, (const char *)q, q + 8, sz) != YAPF_OK) + return yapf__open_fail(t, err, YAPF_ERR_OOM); + q += 8 + sz; + } + } + + t->data = (void **)calloc(S, sizeof(void *)); + ti->off = (uint64_t *)calloc(S, sizeof(uint64_t)); + ti->size = (uint32_t *)calloc(S, sizeof(uint32_t)); + ti->tcheck = (uint32_t *)calloc(S, sizeof(uint32_t)); + if (!t->data || !ti->off || !ti->size || !ti->tcheck) + return yapf__open_fail(t, err, YAPF_ERR_OOM); + for (uint32_t s = 0; s < S; s++) { + uint32_t mip = s % t->mip_levels; + if (ti->version == 1) { + const uint8_t *e = b + YAPF__HDR_LEN + (size_t)s * YAPF__MIP_ENTRY; + ti->off[s] = yapf__r32le(e); + ti->size[s] = yapf__r32le(e + 4); + } else { + const uint8_t *e = b + YAPF__V2_HDR + (size_t)s * YAPF__V2_SURFACE; + ti->off[s] = yapf__r64le(e); + ti->size[s] = yapf__r32le(e + 8); + ti->tcheck[s] = yapf__r32le(e + 12); + if (ti->off[s] > ti->file_size || ti->size[s] > ti->file_size - ti->off[s]) + return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + if (ti->size[s] < 4 || (ti->size[s] - 4) / 8 < yapf__surface_tiles(t, mip)) + return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + } + if (ti->off[s] < ti->front) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + } + return t; +} + +uint64_t yapf_tex_stream_bytes(const yapf_texture_t *tex, uint32_t first_mip) { + if (!tex || !tex->internal_) return 0; + const yapf__texi *ti = (const yapf__texi *)tex->internal_; + if (!ti->off) return 0; + uint32_t S = yapf_tex_surface_count(tex); + uint64_t end = ti->front; + for (uint32_t s = 0; s < S; s++) + if (s % tex->mip_levels >= first_mip && ti->off[s] + ti->size[s] > end) + end = ti->off[s] + ti->size[s]; + return end; +} + +/* Prepares the level for surface s; alloc: allocate tex->data[s]. */ +static int yapf__surface_level(yapf_texture_t *t, const uint8_t *buf, size_t avail, + uint32_t s, yapf__level *lv, int alloc) +{ + const yapf__texi *ti = (const yapf__texi *)t->internal_; + uint32_t mip = s % t->mip_levels, w, h, d; + memset(lv, 0, sizeof(*lv)); + if (ti->off[s] + ti->size[s] > avail) return YAPF_ERR_TRUNCATED; + uint64_t tiles = yapf__surface_tiles(t, mip); + const uint8_t *sd = buf + ti->off[s]; + size_t ssz = ti->size[s], pre = 0; + if (ti->version == 2) { + /* u32 book size, book, tile table: covered by the tile-table check */ + uint32_t bl = yapf__r32le(sd); + if (ssz < 4 || bl > ssz - 4 || (ssz - 4 - bl) / 8 < tiles) return YAPF_ERR_CORRUPT; + pre = 4 + (size_t)bl; + if (!(ti->flags & YAPF_LOAD_TRUST) && yapf_checksum(sd, pre + (size_t)tiles * 8) != ti->tcheck[s]) + return YAPF_ERR_CHECKSUM; + if (bl) { + if (!(ti->needs & YAPF_NEED_ENTROPY)) return YAPF_ERR_CORRUPT; + lv->book = (yapf__dbook *)malloc(sizeof(yapf__dbook)); + if (!lv->book) return YAPF_ERR_OOM; + if (yapf__book_read(sd + 4, bl, lv->book) != 0) return YAPF_ERR_CORRUPT; + } + } + if (alloc && !t->data[s]) { + t->data[s] = malloc(yapf_tex_surface_bytes(t, mip)); + if (!t->data[s]) return YAPF_ERR_OOM; + } + yapf_tex_mip_size(t, mip, &w, &h, &d); + lv->data = sd + pre; + lv->pixels = (uint8_t *)t->data[s]; + lv->mw = w; lv->mh = h; lv->md = d; + lv->ch = t->channels; + lv->sample = t->sample; + lv->v2 = ti->version == 2; + lv->verify = !(ti->flags & YAPF_LOAD_TRUST); +#if YAPF__SIMD + lv->simd = yapf__simd_available(); +#endif + yapf_tex_tile_grid(t, mip, &lv->tcx, &lv->tcy); + if (t->payload == YAPF_PAYLOAD_BLOCKS) { + uint32_t S; + yapf__mip_blocks(t, mip, &lv->nbx, &lv->nby, &S); + lv->blk = S; + } + if (yapf__level_table(lv, ssz - pre) != 0) return YAPF_ERR_CORRUPT; + return YAPF_OK; +} + +static int yapf__map_rc(int rc) { + return rc == 0 ? YAPF_OK : rc == YAPF_ERR_CHECKSUM || rc == YAPF_ERR_OOM ? rc : YAPF_ERR_CORRUPT; +} + +int yapf_tex_decode_surface(yapf_texture_t *tex, const void *buffer, size_t avail, + uint32_t surface, int threads) +{ + if (!tex || !tex->internal_ || !buffer) return YAPF_ERR_INVALID; + if (!((yapf__texi *)tex->internal_)->off) return YAPF_ERR_INVALID; + if (surface >= yapf_tex_surface_count(tex)) return YAPF_ERR_INVALID; + yapf__level lv; + int rc = yapf__surface_level(tex, (const uint8_t *)buffer, avail, surface, &lv, 1); + if (rc == YAPF_OK) rc = yapf__map_rc(yapf__decode_levels(&lv, 1, threads)); + yapf__level_free(&lv); + return rc; +} + +int yapf_tex_decode_tile(const yapf_texture_t *tex, const void *buffer, size_t avail, + uint32_t surface, uint32_t tx, uint32_t ty, uint32_t slice, + void *out, size_t stride, + uint32_t *x, uint32_t *y, uint32_t *w, uint32_t *h) +{ + if (!tex || !tex->internal_ || !buffer || !out) return YAPF_ERR_INVALID; + if (!((const yapf__texi *)tex->internal_)->off) return YAPF_ERR_INVALID; + if (surface >= yapf_tex_surface_count(tex)) return YAPF_ERR_INVALID; + uint32_t mip = surface % tex->mip_levels, mw, mh, md, tcx, tcy; + yapf_tex_mip_size(tex, mip, &mw, &mh, &md); + yapf_tex_tile_grid(tex, mip, &tcx, &tcy); + if (tx >= tcx || ty >= tcy || slice >= md) return YAPF_ERR_INVALID; + + /* unit: bytes per texel (or block); the row must hold the tile */ + size_t unit; + uint32_t span, bw = 1, bh = 1, across; + if (tex->payload == YAPF_PAYLOAD_BLOCKS) { + uint32_t bytes = 16, nbx, nby, S; + yapf_block_info(tex->format, &bw, &bh, &bytes); + yapf__mip_blocks(tex, mip, &nbx, &nby, &S); + unit = bytes; + span = YAPF_BLOCK_TILE; + across = nbx; + } else { + unit = (size_t)tex->channels * yapf__sbytes(tex->sample); + span = YAPF_TILE_SIZE; + across = mw; + } + { + uint32_t need = across - tx * span < span ? across - tx * span : span; + if (stride % unit != 0 || stride / unit < need) return YAPF_ERR_INVALID; + } + yapf__level lv; + int rc = yapf__surface_level((yapf_texture_t *)tex, (const uint8_t *)buffer, avail, + surface, &lv, 0); + if (rc == YAPF_OK) { + yapf__wdec *wsc = NULL; + if (!lv.blk && lv.sample != YAPF_SAMPLE_U8) { + wsc = (yapf__wdec *)malloc(sizeof(yapf__wdec)); + if (!wsc) rc = YAPF_ERR_OOM; + } + if (rc == YAPF_OK) + rc = yapf__map_rc(yapf__level_tile(&lv, tx, slice * lv.tcy + ty, wsc, + (uint8_t *)out, stride / unit)); + free(wsc); + } + yapf__level_free(&lv); + if (rc == YAPF_OK) { + uint32_t px = tx * span * bw, py = ty * span * bh; + uint32_t pw = span * bw, ph = span * bh; + if (x) *x = px; + if (y) *y = py; + if (w) *w = mw - px < pw ? mw - px : pw; + if (h) *h = mh - py < ph ? mh - py : ph; + } + return rc; +} + +yapf_texture_t *yapf_tex_load_memory(const void *buffer, size_t size, + int flags, int threads, int *err) +{ + yapf_texture_t *t = yapf_tex_open(buffer, size, flags, err); + if (!t) return NULL; + yapf__texi *ti = (yapf__texi *)t->internal_; + if (ti->version == 2 && size < ti->file_size) + return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + uint32_t S = yapf_tex_surface_count(t); + yapf__level *lv = (yapf__level *)calloc(S, sizeof(yapf__level)); + int rc = lv ? YAPF_OK : YAPF_ERR_OOM; + for (uint32_t s = 0; s < S && rc == YAPF_OK; s++) + rc = yapf__surface_level(t, (const uint8_t *)buffer, size, s, &lv[s], 1); + if (rc == YAPF_ERR_TRUNCATED) rc = YAPF_ERR_CORRUPT; + if (rc == YAPF_OK) rc = yapf__map_rc(yapf__decode_levels(lv, (int)S, threads)); + if (lv) for (uint32_t s = 0; s < S; s++) yapf__level_free(&lv[s]); + free(lv); + if (rc != YAPF_OK) return yapf__open_fail(t, err, rc); + return t; +} + +yapf_texture_t *yapf_tex_load(const char *filename, int flags, int threads, int *err) { + size_t len = 0; + uint8_t *buf = yapf__read_file(filename, &len); + if (!buf) { if (err) *err = YAPF_ERR_IO; return NULL; } + yapf_texture_t *t = yapf_tex_load_memory(buf, len, flags, threads, err); + free(buf); + return t; +} + +/* v1 API on a version 2 file: frame 0, layer 0, face 0 of an 8-bit + * pixel texture (not volumes), with its mips. */ +static yapf_image_t *yapf__load_v2_image(const uint8_t *buf, size_t size, int threads) { + yapf_texture_t *t = yapf_tex_open(buf, size, 0, NULL); + if (!t) return NULL; + yapf__texi *ti = (yapf__texi *)t->internal_; + if (t->payload != YAPF_PAYLOAD_PIXELS || t->sample != YAPF_SAMPLE_U8 || + t->shape == YAPF_SHAPE_VOLUME || size < ti->file_size) { + yapf_tex_free(t); + return NULL; + } + yapf__level lv[YAPF_MAX_MIPS]; + int rc = YAPF_OK; + memset(lv, 0, sizeof(lv)); + for (uint32_t m = 0; m < t->mip_levels && rc == YAPF_OK; m++) + rc = yapf__surface_level(t, buf, size, m, &lv[m], 1); + if (rc == YAPF_OK) rc = yapf__decode_levels(lv, t->mip_levels, threads); + for (uint32_t m = 0; m < YAPF_MAX_MIPS; m++) yapf__level_free(&lv[m]); + + yapf_image_t *img = NULL; + if (rc == 0) img = (yapf_image_t *)calloc(1, sizeof(yapf_image_t)); + if (img) img->mips = (uint8_t **)calloc(t->mip_levels, sizeof(uint8_t *)); + if (img && img->mips) { + img->width = t->width; img->height = t->height; img->channels = t->channels; + img->gpu_format = t->format; img->flags = t->flags; img->mip_levels = t->mip_levels; + for (uint32_t m = 0; m < t->mip_levels; m++) { + img->mips[m] = (uint8_t *)t->data[m]; /* surface m: frame 0, layer 0, face 0 */ + t->data[m] = NULL; + } + img->pixels = img->mips[0]; + } else if (img) { + free(img); + img = NULL; + } + yapf_tex_free(t); + return img; +} + +/* ======================================================================== + * Half floats + * ======================================================================== */ + +uint16_t yapf_float_to_half(float f) { + uint32_t x; + memcpy(&x, &f, 4); + uint32_t sign = (x >> 16) & 0x8000u, e = (x >> 23) & 0xFFu, m = x & 0x7FFFFFu; + if (e == 255) return (uint16_t)(sign | 0x7C00u | (m ? 0x200u | (m >> 13) : 0)); + int32_t ee = (int32_t)e - 127 + 15; + if (ee >= 31) return (uint16_t)(sign | 0x7C00u); + if (ee <= 0) { /* subnormal or zero */ + if (ee < -10) return (uint16_t)sign; + m |= 0x800000u; + int shift = 14 - ee; + uint32_t hm = m >> shift, rem = m & ((1u << shift) - 1), halfway = 1u << (shift - 1); + if (rem > halfway || (rem == halfway && (hm & 1))) hm++; + return (uint16_t)(sign | hm); + } + uint32_t h = sign | ((uint32_t)ee << 10) | (m >> 13), rem = m & 0x1FFFu; + if (rem > 0x1000u || (rem == 0x1000u && (h & 1))) h++; /* carries into the exponent */ + return (uint16_t)h; +} + +float yapf_half_to_float(uint16_t h) { + uint32_t s = (uint32_t)(h & 0x8000u) << 16, e = (h >> 10) & 31u, m = h & 0x3FFu, x; + if (e == 0) { + if (!m) x = s; + else { + e = 113; + while (!(m & 0x400u)) { m <<= 1; e--; } + x = s | (e << 23) | ((m & 0x3FFu) << 13); + } + } else if (e == 31) { + x = s | 0x7F800000u | (m << 13); + } else { + x = s | ((e + 112) << 23) | (m << 13); + } + float f; + memcpy(&f, &x, 4); + return f; +} + +/* ======================================================================== + * GPU block codecs (BC1–BC7) + * ======================================================================== */ + +/* BC6H / BC7 partitions. 2 subsets: bit i set = pixel i in subset 1. + * 3 subsets: 2 bits per pixel. Anchors: the pixel of each extra subset + * whose index drops its top bit. */ +static const uint16_t yapf__part2[64] = { + 0xCCCC, 0x8888, 0xEEEE, 0xECC8, 0xC880, 0xFEEC, 0xFEC8, 0xEC80, + 0xC800, 0xFFEC, 0xFE80, 0xE800, 0xFFE8, 0xFF00, 0xFFF0, 0xF000, + 0xF710, 0x008E, 0x7100, 0x08CE, 0x008C, 0x7310, 0x3100, 0x8CCE, + 0x088C, 0x3110, 0x6666, 0x366C, 0x17E8, 0x0FF0, 0x718E, 0x399C, + 0xAAAA, 0xF0F0, 0x5A5A, 0x33CC, 0x3C3C, 0x55AA, 0x9696, 0xA55A, + 0x73CE, 0x13C8, 0x324C, 0x3BDC, 0x6996, 0xC33C, 0x9966, 0x0660, + 0x0272, 0x04E4, 0x4E40, 0x2720, 0xC936, 0x936C, 0x39C6, 0x639C, + 0x9336, 0x9CC6, 0x817E, 0xE718, 0xCCF0, 0x0FCC, 0x7744, 0xEE22 +}; + +static const uint32_t yapf__part3[64] = { + 0xAA685050, 0x6A5A5040, 0x5A5A4200, 0x5450A0A8, 0xA5A50000, 0xA0A05050, 0x5555A0A0, 0x5A5A5050, + 0xAA550000, 0xAA555500, 0xAAAA5500, 0x90909090, 0x94949494, 0xA4A4A4A4, 0xA9A59450, 0x2A0A4250, + 0xA5945040, 0x0A425054, 0xA5A5A500, 0x55A0A0A0, 0xA8A85454, 0x6A6A4040, 0xA4A45000, 0x1A1A0500, + 0x0050A4A4, 0xAAA59090, 0x14696914, 0x69691400, 0xA08585A0, 0xAA821414, 0x50A4A450, 0x6A5A0200, + 0xA9A58000, 0x5090A0A8, 0xA8A09050, 0x24242424, 0x00AA5500, 0x24924924, 0x24499224, 0x50A50A50, + 0x500AA550, 0xAAAA4444, 0x66660000, 0xA5A0A5A0, 0x50A050A0, 0x69286928, 0x44AAAA44, 0x66666600, + 0xAA444444, 0x54A854A8, 0x95809580, 0x96969600, 0xA85454A8, 0x80959580, 0xAA141414, 0x96960000, + 0xAAAA1414, 0xA05050A0, 0xA0A5A5A0, 0x96000000, 0x40804080, 0xA9A8A9A8, 0xAAAAAA44, 0x2A4A5254 +}; + +static const uint8_t yapf__anchor2[64] = { + 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, + 15, 2, 8, 2, 2, 8, 8, 15, 2, 8, 2, 2, 8, 8, 2, 2, + 15, 15, 6, 8, 2, 8, 15, 15, 2, 8, 2, 2, 2, 15, 15, 6, + 6, 2, 6, 8, 15, 15, 2, 2, 15, 15, 15, 15, 15, 2, 2, 15 +}; +static const uint8_t yapf__anchor3a[64] = { + 3, 3, 15, 15, 8, 3, 15, 15, 8, 8, 6, 6, 6, 5, 3, 3, + 3, 3, 8, 15, 3, 3, 6, 10, 5, 8, 8, 6, 8, 5, 15, 15, + 8, 15, 3, 5, 6, 10, 8, 15, 15, 3, 15, 5, 15, 15, 15, 15, + 3, 15, 5, 5, 5, 8, 5, 10, 5, 10, 8, 13, 15, 12, 3, 3 +}; +static const uint8_t yapf__anchor3b[64] = { + 15, 8, 8, 3, 15, 15, 3, 8, 15, 15, 15, 15, 15, 15, 15, 8, + 15, 8, 15, 3, 15, 8, 15, 8, 3, 15, 6, 10, 15, 15, 10, 8, + 15, 3, 15, 10, 10, 8, 9, 10, 6, 15, 8, 15, 3, 6, 6, 8, + 15, 3, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 3, 15, 15, 8 +}; + +static const uint8_t yapf__w2[4] = { 0, 21, 43, 64 }; +static const uint8_t yapf__w3[8] = { 0, 9, 18, 27, 37, 46, 55, 64 }; +static const uint8_t yapf__w4[16] = { 0, 4, 9, 13, 17, 21, 26, 30, 34, 38, 43, 47, 51, 55, 60, 64 }; + +static const uint8_t *yapf__weights(int bits) { + return bits == 2 ? yapf__w2 : bits == 3 ? yapf__w3 : yapf__w4; +} + +static int yapf__bc_interp(int a, int b, int w) { + return (a * (64 - w) + b * w + 32) >> 6; +} + +/* 128-bit little-endian bit reader / writer */ +static uint32_t yapf__getbits(const uint8_t *b, unsigned *pos, unsigned n) { + uint32_t v = 0; + for (unsigned i = 0; i < n; i++, (*pos)++) + v |= (uint32_t)((b[*pos >> 3] >> (*pos & 7)) & 1u) << i; + return v; +} + +static void yapf__putbits(uint8_t *b, unsigned *pos, uint32_t v, unsigned n) { + for (unsigned i = 0; i < n; i++, (*pos)++) + if ((v >> i) & 1u) b[*pos >> 3] |= (uint8_t)(1u << (*pos & 7)); +} + +/* ── BC1–BC5 ─────────────────────────────────────────────────────────── */ + +static void yapf__565(uint16_t c, int *rgb) { + int r = (c >> 11) & 31, g = (c >> 5) & 63, b = c & 31; + rgb[0] = (r << 3) | (r >> 2); + rgb[1] = (g << 2) | (g >> 4); + rgb[2] = (b << 3) | (b >> 2); +} + +/* BC1 colour block; four_only: BC2 / BC3 always use four colours. */ +static void yapf__bc1_block(const uint8_t *s, uint8_t out[16][4], int four_only) { + uint16_t c0 = yapf__r16le(s), c1 = yapf__r16le(s + 2); + uint32_t idx = yapf__r32le(s + 4); + int pal[4][4]; + yapf__565(c0, pal[0]); + yapf__565(c1, pal[1]); + pal[0][3] = pal[1][3] = pal[2][3] = pal[3][3] = 255; + for (int k = 0; k < 3; k++) { + if (c0 > c1 || four_only) { + pal[2][k] = (2 * pal[0][k] + pal[1][k] + 1) / 3; + pal[3][k] = (pal[0][k] + 2 * pal[1][k] + 1) / 3; + } else { + pal[2][k] = (pal[0][k] + pal[1][k] + 1) / 2; + pal[3][k] = 0; + } + } + if (!(c0 > c1 || four_only)) pal[3][3] = 0; + for (int i = 0; i < 16; i++) + for (int k = 0; k < 4; k++) out[i][k] = (uint8_t)pal[(idx >> (2 * i)) & 3][k]; +} + +/* BC4 channel block into out[i][ch]; snorm values are shifted by +128. */ +static void yapf__bc4_block(const uint8_t *s, uint8_t out[16][4], int ch, int snorm) { + int pal[8]; + uint64_t idx = 0; + for (int k = 0; k < 6; k++) idx |= (uint64_t)s[2 + k] << (8 * k); + if (!snorm) { + int r0 = s[0], r1 = s[1]; + pal[0] = r0; pal[1] = r1; + if (r0 > r1) for (int i = 2; i < 8; i++) pal[i] = ((8 - i) * r0 + (i - 1) * r1 + 3) / 7; + else { + for (int i = 2; i < 6; i++) pal[i] = ((6 - i) * r0 + (i - 1) * r1 + 2) / 5; + pal[6] = 0; pal[7] = 255; + } + } else { + int r0 = (int8_t)s[0], r1 = (int8_t)s[1]; + if (r0 < -127) r0 = -127; + if (r1 < -127) r1 = -127; + pal[0] = r0; pal[1] = r1; + if (r0 > r1) { + for (int i = 2; i < 8; i++) { + int v = (8 - i) * r0 + (i - 1) * r1; + pal[i] = v >= 0 ? (v + 3) / 7 : -((-v + 3) / 7); + } + } else { + for (int i = 2; i < 6; i++) { + int v = (6 - i) * r0 + (i - 1) * r1; + pal[i] = v >= 0 ? (v + 2) / 5 : -((-v + 2) / 5); + } + pal[6] = -127; pal[7] = 127; + } + for (int i = 0; i < 8; i++) pal[i] += 128; + } + for (int i = 0; i < 16; i++) out[i][ch] = (uint8_t)pal[(idx >> (3 * i)) & 7]; +} + +/* ── BC7 ─────────────────────────────────────────────────────────────── */ + +static const struct { + uint8_t ns, pb, rb, isb, cb, ab, epb, spb, ib, ib2; +} yapf__bc7_modes[8] = { + { 3, 4, 0, 0, 4, 0, 1, 0, 3, 0 }, { 2, 6, 0, 0, 6, 0, 0, 1, 3, 0 }, + { 3, 6, 0, 0, 5, 0, 0, 0, 2, 0 }, { 2, 6, 0, 0, 7, 0, 1, 0, 2, 0 }, + { 1, 0, 2, 1, 5, 6, 0, 0, 2, 3 }, { 1, 0, 2, 0, 7, 8, 0, 0, 2, 2 }, + { 1, 0, 0, 0, 7, 7, 1, 0, 4, 0 }, { 2, 6, 0, 0, 5, 5, 1, 0, 2, 0 } +}; + +static int yapf__subset(int ns, int part, int i) { + if (ns == 2) return (yapf__part2[part] >> i) & 1; + if (ns == 3) return (int)((yapf__part3[part] >> (2 * i)) & 3); + return 0; +} + +static int yapf__is_anchor(int ns, int part, int i) { + if (i == 0) return 1; + if (ns == 2) return i == yapf__anchor2[part]; + if (ns == 3) return i == yapf__anchor3a[part] || i == yapf__anchor3b[part]; + return 0; +} + +static void yapf__bc7_block(const uint8_t *s, uint8_t out[16][4]) { + unsigned pos = 0; + int mode = 0; + while (mode < 8 && !yapf__getbits(s, &pos, 1)) mode++; + if (mode == 8) { memset(out, 0, 64); return; } + int ns = yapf__bc7_modes[mode].ns, cb = yapf__bc7_modes[mode].cb, ab = yapf__bc7_modes[mode].ab; + int part = (int)yapf__getbits(s, &pos, yapf__bc7_modes[mode].pb); + int rot = (int)yapf__getbits(s, &pos, yapf__bc7_modes[mode].rb); + int isb = (int)yapf__getbits(s, &pos, yapf__bc7_modes[mode].isb); + int ne = ns * 2, ep[6][4]; + for (int c = 0; c < 3; c++) + for (int e = 0; e < ne; e++) ep[e][c] = (int)yapf__getbits(s, &pos, cb); + for (int e = 0; e < ne; e++) ep[e][3] = ab ? (int)yapf__getbits(s, &pos, ab) : 255; + int pbit = yapf__bc7_modes[mode].epb | yapf__bc7_modes[mode].spb; + if (yapf__bc7_modes[mode].epb) { + for (int e = 0; e < ne; e++) { + int p = (int)yapf__getbits(s, &pos, 1); + for (int c = 0; c < 4; c++) if (c < 3 || ab) ep[e][c] = (ep[e][c] << 1) | p; + } + } else if (yapf__bc7_modes[mode].spb) { + for (int k = 0; k < ns; k++) { + int p = (int)yapf__getbits(s, &pos, 1); + for (int c = 0; c < 3; c++) { + ep[2 * k][c] = (ep[2 * k][c] << 1) | p; + ep[2 * k + 1][c] = (ep[2 * k + 1][c] << 1) | p; + } + } + } + for (int e = 0; e < ne; e++) + for (int c = 0; c < 4; c++) { + int bits = (c < 3 ? cb : ab) + pbit; + if (c == 3 && !ab) continue; + int v = ep[e][c] << (8 - bits); + ep[e][c] = v | (v >> bits); + } + int ib = yapf__bc7_modes[mode].ib, ib2 = yapf__bc7_modes[mode].ib2; + int idx[16], idx2[16]; + for (int i = 0; i < 16; i++) + idx[i] = (int)yapf__getbits(s, &pos, ib - yapf__is_anchor(ns, part, i)); + for (int i = 0; i < 16; i++) + idx2[i] = ib2 ? (int)yapf__getbits(s, &pos, ib2 - (i == 0)) : 0; + const uint8_t *w1 = yapf__weights(ib), *w2 = yapf__weights(ib2 ? ib2 : 2); + for (int i = 0; i < 16; i++) { + int k = yapf__subset(ns, part, i), px[4]; + const int *a = ep[2 * k], *b = ep[2 * k + 1]; + int cw, aw; + if (!ib2) { cw = w1[idx[i]]; aw = w1[idx[i]]; } + else if (!isb) { cw = w1[idx[i]]; aw = w2[idx2[i]]; } + else { cw = w2[idx2[i]]; aw = w1[idx[i]]; } + for (int c = 0; c < 3; c++) px[c] = yapf__bc_interp(a[c], b[c], cw); + px[3] = yapf__bc_interp(a[3], b[3], aw); + if (rot) { int t = px[3]; px[3] = px[rot - 1]; px[rot - 1] = t; } + for (int c = 0; c < 4; c++) out[i][c] = (uint8_t)px[c]; + } +} + +/* ── BC6H ────────────────────────────────────────────────────────────── */ +/* + * Header layouts as sequences of (field, first bit, bit count), in the + * order they appear after the mode bits. Fields: 0–3 red w, x, y, z; + * 4–7 green; 8–11 blue; 12 partition. The same table drives the reader + * and the encoder's writer. + */ +#define YAPF__F(f, lo, n) { f, lo, n } +typedef struct { uint8_t f, lo, n; } yapf__seg; +enum { RW, RX, RY, RZ, GW, GX, GY, GZ, BW, BX, BY, BZ, PD }; + +static const yapf__seg yapf__bc6_m0[] = { + {GY,4,1},{BY,4,1},{BZ,4,1},{RW,0,10},{GW,0,10},{BW,0,10},{RX,0,5},{GZ,4,1},{GY,0,4}, + {GX,0,5},{BZ,0,1},{GZ,0,4},{BX,0,5},{BZ,1,1},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m1[] = { + {GY,5,1},{GZ,4,1},{GZ,5,1},{RW,0,7},{BZ,0,1},{BZ,1,1},{BY,4,1},{GW,0,7},{BY,5,1},{BZ,2,1}, + {GY,4,1},{BW,0,7},{BZ,3,1},{BZ,5,1},{BZ,4,1},{RX,0,6},{GY,0,4},{GX,0,6},{GZ,0,4},{BX,0,6}, + {BY,0,4},{RY,0,6},{RZ,0,6},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m2[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,5},{RW,10,1},{GY,0,4},{GX,0,4},{GW,10,1},{BZ,0,1},{GZ,0,4}, + {BX,0,4},{BW,10,1},{BZ,1,1},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m3[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,4},{RW,10,1},{GZ,4,1},{GY,0,4},{GX,0,5},{GW,10,1},{GZ,0,4}, + {BX,0,4},{BW,10,1},{BZ,1,1},{BY,0,4},{RY,0,4},{BZ,0,1},{BZ,2,1},{RZ,0,4},{GY,4,1},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m4[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,4},{RW,10,1},{BY,4,1},{GY,0,4},{GX,0,4},{GW,10,1},{BZ,0,1}, + {GZ,0,4},{BX,0,5},{BW,10,1},{BY,0,4},{RY,0,4},{BZ,1,1},{BZ,2,1},{RZ,0,4},{BZ,4,1},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m5[] = { + {RW,0,9},{BY,4,1},{GW,0,9},{GY,4,1},{BW,0,9},{BZ,4,1},{RX,0,5},{GZ,4,1},{GY,0,4},{GX,0,5}, + {BZ,0,1},{GZ,0,4},{BX,0,5},{BZ,1,1},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m6[] = { + {RW,0,8},{GZ,4,1},{BY,4,1},{GW,0,8},{BZ,2,1},{GY,4,1},{BW,0,8},{BZ,3,1},{BZ,4,1},{RX,0,6}, + {GY,0,4},{GX,0,5},{BZ,0,1},{GZ,0,4},{BX,0,5},{BZ,1,1},{BY,0,4},{RY,0,6},{RZ,0,6},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m7[] = { + {RW,0,8},{BZ,0,1},{BY,4,1},{GW,0,8},{GY,5,1},{GY,4,1},{BW,0,8},{GZ,5,1},{BZ,4,1},{RX,0,5}, + {GZ,4,1},{GY,0,4},{GX,0,6},{GZ,0,4},{BX,0,5},{BZ,1,1},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m8[] = { + {RW,0,8},{BZ,1,1},{BY,4,1},{GW,0,8},{BY,5,1},{GY,4,1},{BW,0,8},{BZ,5,1},{BZ,4,1},{RX,0,5}, + {GZ,4,1},{GY,0,4},{GX,0,5},{BZ,0,1},{GZ,0,4},{BX,0,6},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m9[] = { + {RW,0,6},{GZ,4,1},{BZ,0,1},{BZ,1,1},{BY,4,1},{GW,0,6},{GY,5,1},{BY,5,1},{BZ,2,1},{GY,4,1}, + {BW,0,6},{GZ,5,1},{BZ,3,1},{BZ,5,1},{BZ,4,1},{RX,0,6},{GY,0,4},{GX,0,6},{GZ,0,4},{BX,0,6}, + {BY,0,4},{RY,0,6},{RZ,0,6},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m10[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,10},{GX,0,10},{BX,0,10},{0,0,0} }; +static const yapf__seg yapf__bc6_m11[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,9},{RW,10,1},{GX,0,9},{GW,10,1},{BX,0,9},{BW,10,1},{0,0,0} }; +static const yapf__seg yapf__bc6_m12[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,8},{RW,11,1},{RW,10,1},{GX,0,8},{GW,11,1},{GW,10,1}, + {BX,0,8},{BW,11,1},{BW,10,1},{0,0,0} }; +static const yapf__seg yapf__bc6_m13[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,4},{RW,15,1},{RW,14,1},{RW,13,1},{RW,12,1},{RW,11,1},{RW,10,1}, + {GX,0,4},{GW,15,1},{GW,14,1},{GW,13,1},{GW,12,1},{GW,11,1},{GW,10,1}, + {BX,0,4},{BW,15,1},{BW,14,1},{BW,13,1},{BW,12,1},{BW,11,1},{BW,10,1},{0,0,0} }; + +static const struct { + uint8_t code, codebits, w, dr, dg, db; + const yapf__seg *seg; +} yapf__bc6_modes[14] = { + { 0x00, 2, 10, 5, 5, 5, yapf__bc6_m0 }, { 0x01, 2, 7, 6, 6, 6, yapf__bc6_m1 }, + { 0x02, 5, 11, 5, 4, 4, yapf__bc6_m2 }, { 0x06, 5, 11, 4, 5, 4, yapf__bc6_m3 }, + { 0x0A, 5, 11, 4, 4, 5, yapf__bc6_m4 }, { 0x0E, 5, 9, 5, 5, 5, yapf__bc6_m5 }, + { 0x12, 5, 8, 6, 5, 5, yapf__bc6_m6 }, { 0x16, 5, 8, 5, 6, 5, yapf__bc6_m7 }, + { 0x1A, 5, 8, 5, 5, 6, yapf__bc6_m8 }, { 0x1E, 5, 6, 6, 6, 6, yapf__bc6_m9 }, + { 0x03, 5, 10, 10, 10, 10, yapf__bc6_m10 }, { 0x07, 5, 11, 9, 9, 9, yapf__bc6_m11 }, + { 0x0B, 5, 12, 8, 8, 8, yapf__bc6_m12 }, { 0x0F, 5, 16, 4, 4, 4, yapf__bc6_m13 } +}; + +static int yapf__sext(int v, int bits) { + int m = 1 << (bits - 1); + v &= (1 << bits) - 1; + return (v ^ m) - m; +} + +static int yapf__bc6_unq(int v, int bits, int sgn) { + if (!sgn) { + if (bits >= 15) return v; + if (v == 0) return 0; + if (v == (1 << bits) - 1) return 0xFFFF; + return ((v << 16) + 0x8000) >> bits; + } + if (bits >= 16) return v; + int s = v < 0; + if (s) v = -v; + int u; + if (v == 0) u = 0; + else if (v >= (1 << (bits - 1)) - 1) u = 0x7FFF; + else u = ((v << 15) + 0x4000) >> (bits - 1); + return s ? -u : u; +} + +static uint16_t yapf__bc6_finish(int v, int sgn) { + if (!sgn) return (uint16_t)((v * 31) >> 6); + v = v < 0 ? -(((-v) * 31) >> 5) : (v * 31) >> 5; + return (uint16_t)(v < 0 ? 0x8000 | -v : v); +} + +static void yapf__bc6_block(const uint8_t *s, uint16_t out[16][3], int sgn) { + unsigned pos = 0; + int code = (int)yapf__getbits(s, &pos, 2), m = -1; + if (code > 1) code |= (int)yapf__getbits(s, &pos, 3) << 2; + for (int i = 0; i < 14; i++) + if (yapf__bc6_modes[i].code == code) { m = i; break; } + if (m < 0) { memset(out, 0, 96); return; } /* reserved modes */ + + int f[13] = {0}; + for (const yapf__seg *g = yapf__bc6_modes[m].seg; g->n; g++) + f[g->f] |= (int)yapf__getbits(s, &pos, g->n) << g->lo; + int W = yapf__bc6_modes[m].w, regions = m < 10 ? 2 : 1, ne = regions * 2; + int db[3] = { yapf__bc6_modes[m].dr, yapf__bc6_modes[m].dg, yapf__bc6_modes[m].db }; + int ep[4][3]; + for (int c = 0; c < 3; c++) { + for (int e = 0; e < ne; e++) ep[e][c] = f[c * 4 + e]; + if (sgn) ep[0][c] = yapf__sext(ep[0][c], W); + int delta = m != 9 && m != 10; + for (int e = 1; e < ne; e++) { + if (delta || sgn) ep[e][c] = yapf__sext(ep[e][c], db[c]); + if (delta) { + ep[e][c] = (ep[e][c] + ep[0][c]) & ((1 << W) - 1); + if (sgn) ep[e][c] = yapf__sext(ep[e][c], W); + } + } + for (int e = 0; e < ne; e++) ep[e][c] = yapf__bc6_unq(ep[e][c], W, sgn); + } + int part = f[PD], ib = regions == 2 ? 3 : 4; + const uint8_t *wt = yapf__weights(ib); + for (int i = 0; i < 16; i++) { + int k = regions == 2 ? (yapf__part2[part] >> i) & 1 : 0; + int anchor = i == 0 || (regions == 2 && i == yapf__anchor2[part]); + int idx = (int)yapf__getbits(s, &pos, ib - anchor); + for (int c = 0; c < 3; c++) + out[i][c] = yapf__bc6_finish(yapf__bc_interp(ep[2 * k][c], ep[2 * k + 1][c], wt[idx]), sgn); + } +} + +int yapf_blocks_decode(uint8_t format, const void *blocks, uint32_t w, uint32_t h, void *out) { + uint32_t bw, bh, S; + if (!blocks || !out || !w || !h) return YAPF_ERR_INVALID; + if (!yapf_block_info(format, &bw, &bh, &S)) return YAPF_ERR_INVALID; + if (format >= YAPF_GPU_ETC2_RGB8) return YAPF_ERR_UNSUPPORTED; + const uint8_t *b = (const uint8_t *)blocks; + uint32_t nbx = (w + 3) / 4, nby = (h + 3) / 4; + int hdr = format == YAPF_GPU_BC6H || format == YAPF_GPU_BC6H_SF; + for (uint32_t by = 0; by < nby; by++) + for (uint32_t bx = 0; bx < nbx; bx++, b += S) { + uint8_t px[16][4]; + uint16_t hx[16][3]; + switch (format) { + case YAPF_GPU_BC1: yapf__bc1_block(b, px, 0); break; + case YAPF_GPU_BC2: + yapf__bc1_block(b + 8, px, 1); + for (int i = 0; i < 16; i++) px[i][3] = (uint8_t)(((b[i / 2] >> (4 * (i & 1))) & 15) * 17); + break; + case YAPF_GPU_BC3: + yapf__bc1_block(b + 8, px, 1); + yapf__bc4_block(b, px, 3, 0); + break; + case YAPF_GPU_BC4: case YAPF_GPU_BC4_SNORM: + memset(px, 0, sizeof(px)); + yapf__bc4_block(b, px, 0, format == YAPF_GPU_BC4_SNORM); + for (int i = 0; i < 16; i++) px[i][3] = 255; + break; + case YAPF_GPU_BC5: case YAPF_GPU_BC5_SNORM: + memset(px, 0, sizeof(px)); + yapf__bc4_block(b, px, 0, format == YAPF_GPU_BC5_SNORM); + yapf__bc4_block(b + 8, px, 1, format == YAPF_GPU_BC5_SNORM); + for (int i = 0; i < 16; i++) px[i][3] = 255; + break; + case YAPF_GPU_BC7: yapf__bc7_block(b, px); break; + default: yapf__bc6_block(b, hx, format == YAPF_GPU_BC6H_SF); break; + } + for (uint32_t y = 0; y < 4 && by * 4 + y < h; y++) + for (uint32_t x = 0; x < 4 && bx * 4 + x < w; x++) { + size_t o = (size_t)(by * 4 + y) * w + bx * 4 + x; + if (hdr) memcpy((uint16_t *)out + o * 3, hx[y * 4 + x], 6); + else memcpy((uint8_t *)out + o * 4, px[y * 4 + x], 4); + } + } + return YAPF_OK; +} + +/* ── Encoders ────────────────────────────────────────────────────────── */ +/* + * Straightforward, decent-quality encoders so YAPF can make GPU textures + * on its own: principal axis endpoints with least-squares refinement. + * For the best BC7 / ASTC quality use a dedicated encoder and store its + * blocks (payload = BLOCKS). + */ + +/* Principal axis of n points of dim k (power iteration). */ +static void yapf__pca(const float (*p)[4], int n, int k, float *mean, float *axis) { + for (int c = 0; c < k; c++) { + mean[c] = 0; + for (int i = 0; i < n; i++) mean[c] += p[i][c]; + mean[c] /= (float)n; + } + float cov[4][4] = {{0}}; + for (int i = 0; i < n; i++) + for (int a = 0; a < k; a++) + for (int b = 0; b < k; b++) + cov[a][b] += (p[i][a] - mean[a]) * (p[i][b] - mean[b]); + float v[4] = { 1.0f, 0.7f, 0.4f, 0.2f }; + for (int it = 0; it < 12; it++) { + float nv[4] = {0}, len = 0; + for (int a = 0; a < k; a++) { + for (int b = 0; b < k; b++) nv[a] += cov[a][b] * v[b]; + len += nv[a] * nv[a]; + } + if (len < 1e-12f) break; + float mx = 0; /* scale by the largest component */ + for (int a = 0; a < k; a++) mx = nv[a] > mx ? nv[a] : -nv[a] > mx ? -nv[a] : mx; + for (int a = 0; a < k; a++) v[a] = nv[a] / mx; + } + for (int a = 0; a < k; a++) axis[a] = v[a]; +} + +/* Endpoints at the extremes of the projections onto the principal axis. */ +static void yapf__extents(const float (*p)[4], int n, int k, float *e0, float *e1) { + float mean[4], axis[4], lo = 1e30f, hi = -1e30f, len2 = 0; + yapf__pca(p, n, k, mean, axis); + for (int c = 0; c < k; c++) len2 += axis[c] * axis[c]; + if (len2 < 1e-20f) len2 = 1.0f; + for (int i = 0; i < n; i++) { + float t = 0; + for (int c = 0; c < k; c++) t += (p[i][c] - mean[c]) * axis[c]; + if (t < lo) lo = t; + if (t > hi) hi = t; + } + lo /= len2; /* the axis need not be unit length */ + hi /= len2; + for (int c = 0; c < k; c++) { e0[c] = mean[c] + axis[c] * lo; e1[c] = mean[c] + axis[c] * hi; } +} + +/* Least squares endpoints for given interpolation weights (0..1). */ +static int yapf__lsq(const float (*p)[4], const float *wt, int n, int k, float *e0, float *e1) { + float aa = 0, ab = 0, bb = 0, ax[4] = {0}, bx[4] = {0}; + for (int i = 0; i < n; i++) { + float b = wt[i], a = 1.0f - b; + aa += a * a; ab += a * b; bb += b * b; + for (int c = 0; c < k; c++) { ax[c] += a * p[i][c]; bx[c] += b * p[i][c]; } + } + float det = aa * bb - ab * ab; + if (det < 1e-8f && det > -1e-8f) return -1; + for (int c = 0; c < k; c++) { + e0[c] = (ax[c] * bb - bx[c] * ab) / det; + e1[c] = (bx[c] * aa - ax[c] * ab) / det; + } + return 0; +} + +static int yapf__clampi(float v, int lo, int hi) { + int i = (int)(v + 0.5f); + if (v < 0) i = (int)(v - 0.5f); + return i < lo ? lo : i > hi ? hi : i; +} + +static uint16_t yapf__to565(const float *c) { + return (uint16_t)((yapf__clampi(c[0] * 31.0f / 255.0f, 0, 31) << 11) | + (yapf__clampi(c[1] * 63.0f / 255.0f, 0, 63) << 5) | + yapf__clampi(c[2] * 31.0f / 255.0f, 0, 31)); +} + +/* BC1 colour block from 16 RGBA pixels. four: force 4-colour mode + * (BC2 / BC3); otherwise pixels with alpha < 128 become transparent. */ +static void yapf__bc1_encode(const uint8_t px[16][4], uint8_t *out, int four) { + float p[16][4]; + int transparent = 0, n = 0, map[16]; + for (int i = 0; i < 16; i++) { + if (!four && px[i][3] < 128) { transparent = 1; map[i] = -1; continue; } + map[i] = n; + for (int c = 0; c < 3; c++) p[n][c] = px[i][c]; + n++; + } + if (n == 0) { /* all transparent */ + yapf__w16le(out, 0); yapf__w16le(out + 2, 0); yapf__w32le(out + 4, 0xFFFFFFFFu); + return; + } + float e0[4], e1[4]; + uint32_t best_idx = 0, best_err = UINT32_MAX; + uint16_t best_c0 = 0, best_c1 = 0; + yapf__extents((const float (*)[4])p, n, 3, e0, e1); + for (int it = 0; it < 3; it++) { + uint16_t c0 = yapf__to565(e1), c1 = yapf__to565(e0); + int three = transparent; + if (three ? c0 > c1 : c0 < c1) { uint16_t t = c0; c0 = c1; c1 = t; } + if (!three && c0 == c1) { /* single colour */ + if (c1 > 0) c1--; else c0++; + if (c0 < c1) { uint16_t t = c0; c0 = c1; c1 = t; } + } + uint8_t blk[8], full[16][4]; + /* decode a block with indices 0, 1, 2, 3 to get the palette */ + yapf__w16le(blk, c0); yapf__w16le(blk + 2, c1); yapf__w32le(blk + 4, 0xE4u); + yapf__bc1_block(blk, full, four); + int nc = three ? 3 : 4; + uint32_t idx = 0, err = 0; + float wt[16]; + static const float wv4[4] = { 0.0f, 1.0f, 1.0f / 3, 2.0f / 3 }; + static const float wv3[3] = { 0.0f, 1.0f, 0.5f }; + for (int i = 0; i < 16; i++) { + if (map[i] < 0) { idx |= 3u << (2 * i); continue; } + uint32_t be = UINT32_MAX; + int bi = 0; + for (int j = 0; j < nc; j++) { + uint32_t e = 0; + for (int c = 0; c < 3; c++) { int d = (int)full[j][c] - px[i][c]; e += (uint32_t)(d * d); } + if (e < be) { be = e; bi = j; } + } + idx |= (uint32_t)bi << (2 * i); + err += be; + wt[map[i]] = three ? wv3[bi] : wv4[bi]; + } + if (err < best_err) { best_err = err; best_idx = idx; best_c0 = c0; best_c1 = c1; } + if (err == 0) break; + /* refine: endpoints in palette order (index 0 = c0) */ + float n0[4], n1[4]; + if (yapf__lsq((const float (*)[4])p, wt, n, 3, n0, n1) != 0) break; + for (int c = 0; c < 3; c++) { e1[c] = n0[c]; e0[c] = n1[c]; } + } + yapf__w16le(out, best_c0); + yapf__w16le(out + 2, best_c1); + yapf__w32le(out + 4, best_idx); +} + +/* BC4 block from 16 values. */ +static void yapf__bc4_encode(const uint8_t *v, uint8_t *out) { + int lo = 255, hi = 0; + for (int i = 0; i < 16; i++) { if (v[i] < lo) lo = v[i]; if (v[i] > hi) hi = v[i]; } + uint64_t best_bits = 0; + uint32_t best_err = UINT32_MAX; + int best0 = hi, best1 = lo; + for (int d0 = 0; d0 < 3; d0++) + for (int d1 = 0; d1 < 3; d1++) { + int r0 = hi - d0, r1 = lo + d1; + if (r0 < r1) continue; + if (r0 == r1) { if (d0 || d1) continue; } + uint8_t blk[8] = { (uint8_t)r0, (uint8_t)r1, 0, 0, 0, 0, 0, 0 }; + uint8_t pal[16][4]; + int palv[8]; + for (int j = 0; j < 8; j++) { + uint64_t ix = 0; + for (int i = 0; i < 16; i++) ix |= (uint64_t)j << (3 * i); + for (int k = 0; k < 6; k++) blk[2 + k] = (uint8_t)(ix >> (8 * k)); + yapf__bc4_block(blk, pal, 0, 0); + palv[j] = pal[0][0]; + } + uint64_t bits = 0; + uint32_t err = 0; + for (int i = 0; i < 16; i++) { + int bj = 0, be = 1 << 30; + for (int j = 0; j < 8; j++) { + int d = palv[j] - v[i]; + if (d * d < be) { be = d * d; bj = j; } + } + bits |= (uint64_t)bj << (3 * i); + err += (uint32_t)be; + } + if (err < best_err) { best_err = err; best_bits = bits; best0 = r0; best1 = r1; } + } + out[0] = (uint8_t)best0; + out[1] = (uint8_t)best1; + for (int k = 0; k < 6; k++) out[2 + k] = (uint8_t)(best_bits >> (8 * k)); +} + +/* BC7 mode 6 (one subset, RGBA 7.7.7.7 + p-bit per endpoint, 4-bit + * indices). Opaque blocks keep the p-bits at 1 so alpha stays 255. + * Returns the squared error. */ +static uint32_t yapf__bc7_mode6(const uint8_t px[16][4], uint8_t *out) { + float p[16][4], e0[4], e1[4]; + int opaque = 1; + for (int i = 0; i < 16; i++) { + for (int c = 0; c < 4; c++) p[i][c] = px[i][c]; + if (px[i][3] != 255) opaque = 0; + } + yapf__extents((const float (*)[4])p, 16, opaque ? 3 : 4, e0, e1); + if (opaque) e0[3] = e1[3] = 255; + + uint8_t best[16]; + uint32_t best_err = UINT32_MAX; + for (int it = 0; it < 3; it++) { + int q[2][4], pb[2]; + const float *e[2] = { e0, e1 }; + for (int k = 0; k < 2; k++) { + uint32_t bestp = UINT32_MAX; + for (int pbit = opaque ? 1 : 0; pbit < 2; pbit++) { + uint32_t er = 0; + int qq[4]; + for (int c = 0; c < 4; c++) { + qq[c] = yapf__clampi((e[k][c] - (float)pbit) / 2.0f, 0, 127); + int d = (2 * qq[c] + pbit) - (int)(e[k][c] + 0.5f); + er += (uint32_t)(d * d); + } + if (er < bestp) { bestp = er; pb[k] = pbit; memcpy(q[k], qq, sizeof(qq)); } + } + } + int v[2][4]; + for (int k = 0; k < 2; k++) for (int c = 0; c < 4; c++) v[k][c] = 2 * q[k][c] + pb[k]; + int idx[16]; + uint32_t err = 0; + float wt[16]; + for (int i = 0; i < 16; i++) { + uint32_t be = UINT32_MAX; + int bi = 0; + for (int j = 0; j < 16; j++) { + uint32_t er = 0; + for (int c = 0; c < 4; c++) { + int d = yapf__bc_interp(v[0][c], v[1][c], yapf__w4[j]) - px[i][c]; + er += (uint32_t)(d * d); + } + if (er < be) { be = er; bi = j; } + } + idx[i] = bi; + err += be; + wt[i] = yapf__w4[bi] / 64.0f; + } + if (err < best_err) { + best_err = err; + int swap = idx[0] >= 8; + int a = swap ? 1 : 0, b = swap ? 0 : 1; + memset(best, 0, 16); + unsigned pos = 0; + yapf__putbits(best, &pos, 1u << 6, 7); + for (int c = 0; c < 4; c++) { + yapf__putbits(best, &pos, (uint32_t)q[a][c], 7); + yapf__putbits(best, &pos, (uint32_t)q[b][c], 7); + } + yapf__putbits(best, &pos, (uint32_t)pb[a], 1); + yapf__putbits(best, &pos, (uint32_t)pb[b], 1); + for (int i = 0; i < 16; i++) { + int ix = swap ? 15 - idx[i] : idx[i]; + yapf__putbits(best, &pos, (uint32_t)ix, i == 0 ? 3 : 4); + } + } + if (err == 0) break; + float n0[4], n1[4]; + if (yapf__lsq((const float (*)[4])p, wt, 16, 4, n0, n1) != 0) break; + memcpy(e0, n0, sizeof(n0)); + memcpy(e1, n1, sizeof(n1)); + if (opaque) e0[3] = e1[3] = 255; + } + memcpy(out, best, 16); + return best_err; +} + +/* Fits indices of `bits` bits for the points of one subset against + * endpoints v0, v1 (k channels); returns the error, fills idx / wt. */ +static uint32_t yapf__bc7_fit(const uint8_t px[16][4], const int *sel, int n, + const int *v0, const int *v1, int c0, int k, int bits, + int *idx, float *wt) +{ + const uint8_t *w = yapf__weights(bits); + uint32_t err = 0; + for (int i = 0; i < n; i++) { + uint32_t be = UINT32_MAX; + int bi = 0; + for (int j = 0; j < (1 << bits); j++) { + uint32_t e = 0; + for (int c = c0; c < c0 + k; c++) { + int d = yapf__bc_interp(v0[c], v1[c], w[j]) - px[sel[i]][c]; + e += (uint32_t)(d * d); + } + if (e < be) { be = e; bi = j; } + } + idx[i] = bi; + wt[i] = w[bi] / 64.0f; + err += be; + } + return err; +} + +/* BC7 mode 5: RGB 7 bits and alpha 8 bits on separate 2-bit indices, so + * colour and alpha need not move together (cut-outs, decals). */ +static uint32_t yapf__bc7_mode5(const uint8_t px[16][4], uint8_t *out) { + int sel[16]; + float p[16][4], e0[4], e1[4]; + for (int i = 0; i < 16; i++) { sel[i] = i; for (int c = 0; c < 4; c++) p[i][c] = px[i][c]; } + yapf__extents((const float (*)[4])p, 16, 3, e0, e1); + + int bq[2][3] = {{0}}, bidx[16] = {0}; + uint32_t cerr = UINT32_MAX; + for (int it = 0; it < 3; it++) { + int q[2][3], v[2][4] = {{0}}, idx[16]; + float wt[16]; + for (int c = 0; c < 3; c++) { + q[0][c] = yapf__clampi(e0[c] * 127.0f / 255.0f, 0, 127); + q[1][c] = yapf__clampi(e1[c] * 127.0f / 255.0f, 0, 127); + v[0][c] = (q[0][c] << 1) | (q[0][c] >> 6); + v[1][c] = (q[1][c] << 1) | (q[1][c] >> 6); + } + uint32_t e = yapf__bc7_fit(px, sel, 16, v[0], v[1], 0, 3, 2, idx, wt); + if (e < cerr) { cerr = e; memcpy(bq, q, sizeof(q)); memcpy(bidx, idx, sizeof(idx)); } + float n0[4], n1[4]; + if (e == 0 || yapf__lsq((const float (*)[4])p, wt, 16, 3, n0, n1) != 0) break; + memcpy(e0, n0, sizeof(n0)); + memcpy(e1, n1, sizeof(n1)); + } + int lo = 255, hi = 0; + for (int i = 0; i < 16; i++) { if (px[i][3] < lo) lo = px[i][3]; if (px[i][3] > hi) hi = px[i][3]; } + int ba[2] = { lo, hi }, aidx[16] = {0}; + uint32_t aerr = UINT32_MAX; + for (int d0 = 0; d0 < 4; d0++) + for (int d1 = 0; d1 < 4; d1++) { + int a0 = lo + d0, a1 = hi - d1, v0[4], v1[4], idx[16]; + float wt[16]; + if (a0 > a1) continue; + v0[3] = a0; v1[3] = a1; + uint32_t e = yapf__bc7_fit(px, sel, 16, v0, v1, 3, 1, 2, idx, wt); + if (e < aerr) { aerr = e; ba[0] = a0; ba[1] = a1; memcpy(aidx, idx, sizeof(idx)); } + } + /* anchors: the first index of each set keeps its top bit clear */ + if (bidx[0] >= 2) { + for (int c = 0; c < 3; c++) { int t = bq[0][c]; bq[0][c] = bq[1][c]; bq[1][c] = t; } + for (int i = 0; i < 16; i++) bidx[i] = 3 - bidx[i]; + } + if (aidx[0] >= 2) { + int t = ba[0]; ba[0] = ba[1]; ba[1] = t; + for (int i = 0; i < 16; i++) aidx[i] = 3 - aidx[i]; + } + unsigned pos = 0; + memset(out, 0, 16); + yapf__putbits(out, &pos, 1u << 5, 6); + yapf__putbits(out, &pos, 0, 2); /* no rotation */ + for (int c = 0; c < 3; c++) { + yapf__putbits(out, &pos, (uint32_t)bq[0][c], 7); + yapf__putbits(out, &pos, (uint32_t)bq[1][c], 7); + } + yapf__putbits(out, &pos, (uint32_t)ba[0], 8); + yapf__putbits(out, &pos, (uint32_t)ba[1], 8); + for (int i = 0; i < 16; i++) yapf__putbits(out, &pos, (uint32_t)bidx[i], i ? 2 : 1); + for (int i = 0; i < 16; i++) yapf__putbits(out, &pos, (uint32_t)aidx[i], i ? 2 : 1); + return cerr + aerr; +} + +/* BC7 mode 1: two subsets, RGB 6 bits + a shared p-bit per subset, 3-bit + * indices; for opaque blocks with two colour regions. */ +static uint32_t yapf__bc7_mode1(const uint8_t px[16][4], int part, uint8_t *out) { + int q[2][2][3], pb[2], idx16[16]; + uint32_t total = 0; + for (int k = 0; k < 2; k++) { + int sel[16], n = 0; + float p[16][4], e0[4], e1[4]; + for (int i = 0; i < 16; i++) + if (yapf__subset(2, part, i) == k) { + sel[n] = i; + for (int c = 0; c < 3; c++) p[n][c] = px[i][c]; + n++; + } + yapf__extents((const float (*)[4])p, n, 3, e0, e1); + uint32_t best = UINT32_MAX; + for (int it = 0; it < 2; it++) { + for (int pbit = 0; pbit < 2; pbit++) { + int qq[2][3], v[2][4] = {{0}}, idx[16]; + float wt[16]; + for (int c = 0; c < 3; c++) { + const float *e[2] = { e0, e1 }; + for (int j = 0; j < 2; j++) { + qq[j][c] = yapf__clampi((e[j][c] * 127.0f / 255.0f - (float)pbit) / 2.0f, 0, 63); + int x = (qq[j][c] << 1) | pbit; + v[j][c] = (x << 1) | (x >> 6); + } + } + uint32_t er = yapf__bc7_fit(px, sel, n, v[0], v[1], 0, 3, 3, idx, wt); + if (er < best) { + best = er; + pb[k] = pbit; + memcpy(q[k], qq, sizeof(qq)); + for (int i = 0; i < n; i++) idx16[sel[i]] = idx[i]; + } + if (it == 0 && pbit == 1) { + float n0[4], n1[4]; + if (yapf__lsq((const float (*)[4])p, wt, n, 3, n0, n1) == 0) { + memcpy(e0, n0, sizeof(n0)); + memcpy(e1, n1, sizeof(n1)); + } + } + } + } + total += best; + int anchor = k == 0 ? 0 : yapf__anchor2[part]; + if (idx16[anchor] >= 4) { + for (int c = 0; c < 3; c++) { int t = q[k][0][c]; q[k][0][c] = q[k][1][c]; q[k][1][c] = t; } + for (int i = 0; i < 16; i++) if (yapf__subset(2, part, i) == k) idx16[i] = 7 - idx16[i]; + } + } + unsigned pos = 0; + memset(out, 0, 16); + yapf__putbits(out, &pos, 2, 2); + yapf__putbits(out, &pos, (uint32_t)part, 6); + for (int c = 0; c < 3; c++) + for (int k = 0; k < 2; k++) { + yapf__putbits(out, &pos, (uint32_t)q[k][0][c], 6); + yapf__putbits(out, &pos, (uint32_t)q[k][1][c], 6); + } + yapf__putbits(out, &pos, (uint32_t)pb[0], 1); + yapf__putbits(out, &pos, (uint32_t)pb[1], 1); + for (int i = 0; i < 16; i++) + yapf__putbits(out, &pos, (uint32_t)idx16[i], 3 - yapf__is_anchor(2, part, i)); + return total; +} + +/* BC7: mode 6, plus mode 5 when there is alpha, plus mode 1 on the two + * most promising partitions (least summed per-subset variance) for + * opaque blocks. The block with the smallest error wins. */ +static void yapf__bc7_encode(const uint8_t px[16][4], uint8_t *out) { + uint8_t cand[16]; + uint32_t best = yapf__bc7_mode6(px, out), e; + int opaque = 1; + for (int i = 0; i < 16; i++) if (px[i][3] != 255) opaque = 0; + if (best == 0) return; + if (!opaque) { + if ((e = yapf__bc7_mode5(px, cand)) < best) { best = e; memcpy(out, cand, 16); } + return; + } + float score[64]; + for (int part = 0; part < 64; part++) { + float s = 0; + for (int k = 0; k < 2; k++) { + float sum[3] = {0}, sq[3] = {0}; + int n = 0; + for (int i = 0; i < 16; i++) + if (yapf__subset(2, part, i) == k) { + n++; + for (int c = 0; c < 3; c++) { sum[c] += px[i][c]; sq[c] += (float)px[i][c] * px[i][c]; } + } + for (int c = 0; c < 3; c++) s += sq[c] - sum[c] * sum[c] / (float)n; + } + score[part] = s; + } + for (int t = 0; t < 2; t++) { + int bp = 0; + for (int part = 1; part < 64; part++) if (score[part] < score[bp]) bp = part; + score[bp] = 1e30f; + if ((e = yapf__bc7_mode1(px, bp, cand)) < best) { best = e; memcpy(out, cand, 16); } + } +} + +/* BC6H unsigned, one region: tries modes 11–14 (10.10, 11.9, 12.8, 16.4) + * and keeps the one with the smallest error in half-float bits. */ +static void yapf__bc6_encode(const uint16_t px[16][3], uint8_t *out) { + float p[16][4], e0[4], e1[4]; + int hv[16][3]; + for (int i = 0; i < 16; i++) + for (int c = 0; c < 3; c++) { + int h = px[i][c]; + if (h & 0x8000) h = 0; /* negative: unsigned format */ + if (h > 0x7BFF) h = 0x7BFF; /* inf / nan: largest finite */ + hv[i][c] = h; + p[i][c] = (float)h * 64.0f / 31.0f; /* interpolation domain */ + } + yapf__extents((const float (*)[4])p, 16, 3, e0, e1); + + uint8_t best[16]; + uint64_t best_err = UINT64_MAX; + memset(best, 0, 16); + for (int m = 10; m < 14; m++) { + int W = yapf__bc6_modes[m].w, d = yapf__bc6_modes[m].dr; + float f0[4], f1[4]; + memcpy(f0, e0, sizeof(f0)); + memcpy(f1, e1, sizeof(f1)); + for (int it = 0; it < 2; it++) { + int q[2][3], ok = 1; + for (int c = 0; c < 3; c++) { + const float *e[2] = { f0, f1 }; + for (int k = 0; k < 2; k++) { + float u = e[k][c]; + int v = W >= 15 ? yapf__clampi(u, 0, (1 << W) - 1) + : yapf__clampi((u - (float)(1 << (15 - W))) / (float)(1 << (16 - W)), + 0, (1 << W) - 1); + q[k][c] = v; + } + if (m != 10) { + int delta = q[1][c] - q[0][c]; + if (delta < -(1 << (d - 1)) || delta > (1 << (d - 1)) - 1) ok = 0; + } + } + if (!ok) break; + int u[2][3]; + for (int k = 0; k < 2; k++) for (int c = 0; c < 3; c++) u[k][c] = yapf__bc6_unq(q[k][c], W, 0); + int idx[16]; + uint64_t err = 0; + float wt[16]; + for (int i = 0; i < 16; i++) { + uint64_t be = UINT64_MAX; + int bi = 0; + for (int j = 0; j < 16; j++) { + uint64_t er = 0; + for (int c = 0; c < 3; c++) { + int64_t dd = (int64_t)yapf__bc6_finish(yapf__bc_interp(u[0][c], u[1][c], yapf__w4[j]), 0) - hv[i][c]; + er += (uint64_t)(dd * dd); + } + if (er < be) { be = er; bi = j; } + } + idx[i] = bi; + err += be; + wt[i] = yapf__w4[bi] / 64.0f; + } + if (err < best_err) { + best_err = err; + int swap = idx[0] >= 8, a = swap ? 1 : 0, b = swap ? 0 : 1; + int f[13] = {0}; + for (int c = 0; c < 3; c++) { + f[c * 4] = q[a][c]; + f[c * 4 + 1] = m == 10 ? q[b][c] : (q[b][c] - q[a][c]) & ((1 << d) - 1); + } + /* with swapped endpoints the delta must still fit */ + int fits = 1; + if (m != 10) + for (int c = 0; c < 3; c++) { + int delta = q[b][c] - q[a][c]; + if (delta < -(1 << (d - 1)) || delta > (1 << (d - 1)) - 1) fits = 0; + } + if (fits) { + memset(best, 0, 16); + unsigned pos = 0; + yapf__putbits(best, &pos, yapf__bc6_modes[m].code, 5); + for (const yapf__seg *g = yapf__bc6_modes[m].seg; g->n; g++) + yapf__putbits(best, &pos, (uint32_t)(f[g->f] >> g->lo), g->n); + for (int i = 0; i < 16; i++) { + int ix = swap ? 15 - idx[i] : idx[i]; + yapf__putbits(best, &pos, (uint32_t)ix, i == 0 ? 3 : 4); + } + } else { + best_err = UINT64_MAX; + } + } + float n0[4], n1[4]; + if (yapf__lsq((const float (*)[4])p, wt, 16, 3, n0, n1) != 0) break; + memcpy(f0, n0, sizeof(n0)); + memcpy(f1, n1, sizeof(n1)); + } + } + memcpy(out, best, 16); +} + +/* Reads texel (x, y) of a pixel surface slice as RGBA8 / RGB half. */ +static void yapf__texel_rgba8(const yapf_texture_t *t, const uint8_t *src, uint32_t w, + uint32_t h, uint32_t x, uint32_t y, uint8_t *o) +{ + if (x >= w) x = w - 1; /* edge blocks repeat the last texel */ + if (y >= h) y = h - 1; + const uint8_t *s = src + ((size_t)y * w + x) * t->channels; + switch (t->channels) { + case 1: o[0] = o[1] = o[2] = s[0]; o[3] = 255; break; + case 2: o[0] = o[1] = o[2] = s[0]; o[3] = s[1]; break; + case 3: o[0] = s[0]; o[1] = s[1]; o[2] = s[2]; o[3] = 255; break; + default: memcpy(o, s, 4); break; + } +} + +static void yapf__texel_half(const yapf_texture_t *t, const uint8_t *src, uint32_t w, + uint32_t h, uint32_t x, uint32_t y, uint16_t *o) +{ + if (x >= w) x = w - 1; + if (y >= h) y = h - 1; + size_t sb = yapf__sbytes(t->sample); + const uint8_t *s = src + ((size_t)y * w + x) * t->channels * sb; + for (int c = 0; c < 3; c++) { + int k = t->channels >= 3 ? c : 0; + uint32_t v = yapf__get_sample(s + (size_t)k * sb, t->sample); + if (t->sample == YAPF_SAMPLE_F16) { o[c] = (uint16_t)v; continue; } + float f; + if (t->sample == YAPF_SAMPLE_F32) memcpy(&f, &v, 4); + else f = t->sample == YAPF_SAMPLE_U16 ? (float)v / 65535.0f : (float)v / 255.0f; + o[c] = yapf_float_to_half(f); + } +} + +int yapf_tex_compress(const yapf_texture_t *src, uint8_t format, yapf_texture_t **out) { + uint32_t S; + if (!out) return YAPF_ERR_INVALID; + *out = NULL; + if (yapf__tex_check(src, &S) != 0 || src->payload != YAPF_PAYLOAD_PIXELS || !src->data) + return YAPF_ERR_INVALID; + int hdr = format == YAPF_GPU_BC6H; + if (format != YAPF_GPU_BC1 && format != YAPF_GPU_BC3 && format != YAPF_GPU_BC4 && + format != YAPF_GPU_BC5 && format != YAPF_GPU_BC7 && !hdr) + return YAPF_ERR_UNSUPPORTED; + if (hdr == (src->sample == YAPF_SAMPLE_U8)) return YAPF_ERR_INVALID; + + yapf_texture_t d = *src; + d.payload = YAPF_PAYLOAD_BLOCKS; + d.format = format; + d.sample = hdr ? YAPF_SAMPLE_F16 : YAPF_SAMPLE_U8; + d.channels = format == YAPF_GPU_BC4 ? 1 : format == YAPF_GPU_BC5 ? 2 : hdr ? 3 + : format == YAPF_GPU_BC3 ? 4 : (src->channels == 4 || src->channels == 2) ? 4 : 3; + for (int c = d.channels; c < 4; c++) d.channel_map[c] = YAPF_CH_DEFAULT; + d.linear_mask &= (uint8_t)((1u << d.channels) - 1); + if (d.normal && d.channels == 2) d.normal = (uint8_t)((d.normal & 3) | YAPF_NORMAL_XY); + yapf_texture_t *t = yapf_tex_create(&d); + if (!t) return YAPF_ERR_OOM; + for (uint32_t i = 0; i < src->note_count; i++) + if (yapf_tex_add_note(t, src->notes[i].tag, src->notes[i].data, src->notes[i].size) != YAPF_OK) { + yapf_tex_free(t); + return YAPF_ERR_OOM; + } + if (t->frames > 1 && src->frame_us) memcpy(t->frame_us, src->frame_us, t->frames * sizeof(uint32_t)); + + for (uint32_t s = 0; s < S; s++) { + uint32_t m = s % src->mip_levels, w, h, dd, nbx, nby, B; + yapf_tex_mip_size(src, m, &w, &h, &dd); + yapf__mip_blocks(t, m, &nbx, &nby, &B); + size_t slice = (size_t)w * h * src->channels * yapf__sbytes(src->sample); + uint8_t *o = (uint8_t *)t->data[s]; + for (uint32_t z = 0; z < dd; z++) { + const uint8_t *sl = (const uint8_t *)src->data[s] + z * slice; + for (uint32_t by = 0; by < nby; by++) + for (uint32_t bx = 0; bx < nbx; bx++, o += B) { + uint8_t px[16][4]; + uint16_t hx[16][3]; + for (int i = 0; i < 16; i++) { + uint32_t x = bx * 4 + (uint32_t)(i & 3), y = by * 4 + (uint32_t)(i >> 2); + if (hdr) yapf__texel_half(src, sl, w, h, x, y, hx[i]); + else yapf__texel_rgba8(src, sl, w, h, x, y, px[i]); + } + if (format == YAPF_GPU_BC4 || format == YAPF_GPU_BC5) { + /* raw channels, not the gray expansion */ + for (int k = 0; k < (format == YAPF_GPU_BC5 ? 2 : 1); k++) { + uint8_t v[16]; + for (int i = 0; i < 16; i++) { + uint32_t x = bx * 4 + (uint32_t)(i & 3), y = by * 4 + (uint32_t)(i >> 2); + if (x >= w) x = w - 1; + if (y >= h) y = h - 1; + int c = k < src->channels ? k : 0; + v[i] = sl[((size_t)y * w + x) * src->channels + c]; + } + yapf__bc4_encode(v, o + 8 * k); + } + } else if (format == YAPF_GPU_BC1) { + yapf__bc1_encode((const uint8_t (*)[4])px, o, 0); + } else if (format == YAPF_GPU_BC3) { + uint8_t a[16]; + for (int i = 0; i < 16; i++) a[i] = px[i][3]; + yapf__bc4_encode(a, o); + yapf__bc1_encode((const uint8_t (*)[4])px, o + 8, 1); + } else if (format == YAPF_GPU_BC7) { + yapf__bc7_encode((const uint8_t (*)[4])px, o); + } else { + yapf__bc6_encode((const uint16_t (*)[3])hx, o); + } + } + } + } + *out = t; + return YAPF_OK; +} + +/* ======================================================================== + * Mip generation + * ======================================================================== */ + +/* x^2.4 for x in [0, 1] without libm: x² · x^0.4, x^0.4 by Newton on + * y^5 = x². */ +static double yapf__pow24(double x) { + if (x <= 0) return 0; + double a = x * x, y = 1.0; + for (int i = 0; i < 60; i++) { + double y4 = y * y * y * y, ny = y - (y4 * y - a) / (5.0 * y4); + if (ny == y) break; + y = ny; + } + return x * x * y; +} + +static double yapf__srgb_to_lin(double v) { + return v <= 0.04045 ? v / 12.92 : yapf__pow24((v + 0.055) / 1.055); +} + +/* 8-bit sRGB: code → linear, and the linear thresholds halfway between + * codes, so linear → code rounds exactly as in sRGB space. */ +typedef struct { float lin[256], mid[255]; } yapf__srgb_tab; + +static void yapf__srgb_init(yapf__srgb_tab *t) { + for (int i = 0; i < 256; i++) t->lin[i] = (float)yapf__srgb_to_lin(i / 255.0); + for (int i = 0; i < 255; i++) t->mid[i] = (float)yapf__srgb_to_lin((i + 0.5) / 255.0); +} + +static float yapf__lin_to_srgb(const yapf__srgb_tab *t, float v) { + int lo = 0, hi = 255; /* first code whose mid >= v */ + while (lo < hi) { + int m = (lo + hi) / 2; + if (t->mid[m] < v) lo = m + 1; else hi = m; + } + return (float)lo / 255.0f; +} + +/* Index of the alpha channel, or -1. */ +static int yapf__alpha_channel(const yapf_texture_t *t) { + for (int c = 0; c < t->channels; c++) + if (t->channel_map[c] == YAPF_CH_ALPHA || t->channel_map[c] == YAPF_CH_OPACITY) return c; + if (t->channel_map[t->channels - 1] == YAPF_CH_DEFAULT && (t->channels == 2 || t->channels == 4)) + return t->channels - 1; + return -1; +} + +static float yapf__load_f(const yapf_texture_t *t, const uint8_t *p) { + uint32_t v = yapf__get_sample(p, t->sample); + switch (t->sample) { + case YAPF_SAMPLE_U8: return (float)v / 255.0f; + case YAPF_SAMPLE_U16: return (float)v / 65535.0f; + case YAPF_SAMPLE_F16: return yapf_half_to_float((uint16_t)v); + default: { float f; memcpy(&f, &v, 4); return f; } + } +} + +static void yapf__store_f(const yapf_texture_t *t, uint8_t *p, float f) { + uint32_t v; + switch (t->sample) { + case YAPF_SAMPLE_U8: v = (uint32_t)yapf__clampi(f * 255.0f, 0, 255); break; + case YAPF_SAMPLE_U16: v = (uint32_t)yapf__clampi(f * 65535.0f, 0, 65535); break; + case YAPF_SAMPLE_F16: v = yapf_float_to_half(f); break; + default: memcpy(&v, &f, 4); break; + } + yapf__put_sample(p, t->sample, v); +} + +/* Share of alpha values above ref after scaling by k, measured on the + * values as they will be stored (q = 255 or 65535; 0 for floats). */ +static double yapf__coverage(const float *a, size_t n, float ref, float k, float q) { + size_t c = 0; + for (size_t i = 0; i < n; i++) { + float v = a[i] * k; + if (q > 0) { + v = v > 1.0f ? q : (float)(int)(v * q + 0.5f); + if (v > ref * q) c++; + } else if (v > ref) { + c++; + } + } + return n ? (double)c / (double)n : 0.0; +} + +int yapf_tex_build_mips(yapf_texture_t *tex, uint32_t levels, uint8_t alpha_ref) { + uint32_t S; + if (!tex || !tex->internal_ || yapf__tex_check(tex, &S) != 0) return YAPF_ERR_INVALID; + if (tex->payload != YAPF_PAYLOAD_PIXELS) return YAPF_ERR_INVALID; + uint32_t full = 1; + while (full < YAPF_MAX_MIPS && + ((tex->width >> full) || (tex->height >> full) || + (tex->shape == YAPF_SHAPE_VOLUME && (tex->depth >> full)))) + full++; + if (levels == 0 || levels > full) levels = full; + + int ach = yapf__alpha_channel(tex); + if (alpha_ref && ach < 0) return YAPF_ERR_INVALID; + int srgb[4]; + for (int c = 0; c < tex->channels; c++) + srgb[c] = (tex->flags & YAPF_FLAG_SRGB) && tex->sample == YAPF_SAMPLE_U8 && c != ach && + !(tex->linear_mask & (1u << c)) && + (tex->transfer == YAPF_TRANSFER_UNSPECIFIED || tex->transfer == YAPF_TRANSFER_SRGB); + + yapf__srgb_tab *st = (yapf__srgb_tab *)malloc(sizeof(yapf__srgb_tab)); + if (!st) return YAPF_ERR_OOM; + yapf__srgb_init(st); + + yapf_texture_t d = *tex; + d.mip_levels = (uint8_t)levels; + uint32_t NS = yapf_tex_surface_count(&d); + void **data = (void **)calloc(NS, sizeof(void *)); + if (!data) { free(st); return YAPF_ERR_OOM; } + int rc = YAPF_OK; + size_t sb = yapf__sbytes(tex->sample), ch = tex->channels; + float qs = tex->sample == YAPF_SAMPLE_U8 ? 255.0f : tex->sample == YAPF_SAMPLE_U16 ? 65535.0f : 0.0f; + + for (uint32_t base = 0; base < NS && rc == YAPF_OK; base += levels) { + uint32_t old = base / levels * tex->mip_levels; /* mip 0 of this face in tex */ + data[base] = tex->data[old]; + tex->data[old] = NULL; + float *alpha0 = NULL; + double target = 0; + for (uint32_t m = 1; m < levels && rc == YAPF_OK; m++) { + uint32_t pw, ph, pd, w, h, dd; + yapf_tex_mip_size(&d, m - 1, &pw, &ph, &pd); + yapf_tex_mip_size(&d, m, &w, &h, &dd); + const uint8_t *src = (const uint8_t *)data[base + m - 1]; + uint8_t *dst = (uint8_t *)malloc(yapf_tex_surface_bytes(&d, m)); + float *alpha = alpha_ref ? (float *)malloc((size_t)w * h * dd * sizeof(float)) : NULL; + if (!dst || (alpha_ref && !alpha)) { free(dst); free(alpha); rc = YAPF_ERR_OOM; break; } + data[base + m] = dst; + if (alpha_ref && m == 1) { + size_t n0 = (size_t)pw * ph * pd; + alpha0 = (float *)malloc(n0 * sizeof(float)); + if (!alpha0) { free(alpha); rc = YAPF_ERR_OOM; break; } + for (size_t i = 0; i < n0; i++) alpha0[i] = yapf__load_f(tex, src + (i * ch + ach) * sb); + target = yapf__coverage(alpha0, n0, alpha_ref / 255.0f, 1.0f, qs); + } + for (uint32_t z = 0; z < dd; z++) + for (uint32_t y = 0; y < h; y++) + for (uint32_t x = 0; x < w; x++) { + uint32_t xs[2] = { 2 * x < pw ? 2 * x : pw - 1, 2 * x + 1 < pw ? 2 * x + 1 : pw - 1 }; + uint32_t ys[2] = { 2 * y < ph ? 2 * y : ph - 1, 2 * y + 1 < ph ? 2 * y + 1 : ph - 1 }; + uint32_t zs[2] = { 2 * z < pd ? 2 * z : pd - 1, 2 * z + 1 < pd ? 2 * z + 1 : pd - 1 }; + int nz = d.shape == YAPF_SHAPE_VOLUME ? 2 : 1; + size_t o = (((size_t)z * h + y) * w + x) * ch; + for (size_t c = 0; c < ch; c++) { + float sum = 0; + for (int k = 0; k < nz; k++) + for (int j = 0; j < 2; j++) + for (int i = 0; i < 2; i++) { + size_t q = (((size_t)zs[k] * ph + ys[j]) * pw + xs[i]) * ch + c; + sum += srgb[c] ? st->lin[src[q]] : yapf__load_f(tex, src + q * sb); + } + float v = sum / (float)(4 * nz); + if (srgb[c]) v = yapf__lin_to_srgb(st, v); + if (alpha && (int)c == ach) alpha[o / ch] = v; + yapf__store_f(tex, dst + (o + c) * sb, v); + } + } + if (alpha) { + /* scale alpha so coverage at alpha_ref matches mip 0 */ + size_t n = (size_t)w * h * dd; + float lo = 0.0f, hi = 8.0f, ref = alpha_ref / 255.0f; + for (int it = 0; it < 20; it++) { + float mid = 0.5f * (lo + hi); + if (yapf__coverage(alpha, n, ref, mid, qs) < target) lo = mid; else hi = mid; + } + double clo = yapf__coverage(alpha, n, ref, lo, qs), chi = yapf__coverage(alpha, n, ref, hi, qs); + float k = (target - clo <= chi - target) ? lo : hi; /* nearer bound */ + for (size_t i = 0; i < n; i++) { + float v = alpha[i] * k; + if (tex->sample != YAPF_SAMPLE_F16 && tex->sample != YAPF_SAMPLE_F32 && v > 1.0f) v = 1.0f; + yapf__store_f(tex, dst + (i * ch + ach) * sb, v); + } + free(alpha); + } + } + free(alpha0); + } + if (rc != YAPF_OK) { + for (uint32_t s = 0; s < NS; s++) { + if (s % levels == 0) { /* give mip 0 back */ + if (data[s]) tex->data[s / levels * tex->mip_levels] = data[s]; + } else { + free(data[s]); + } + } + free(data); + free(st); + return rc; + } + free(st); + for (uint32_t s = 0; s < S; s++) free(tex->data[s]); /* old mips (mip 0 moved) */ + free(tex->data); + tex->data = data; + tex->mip_levels = (uint8_t)levels; + int any_srgb = 0; + for (int c = 0; c < tex->channels; c++) any_srgb |= srgb[c]; + tex->mip_filter = (uint8_t)(YAPF_MIP_BOX | (any_srgb ? YAPF_MIP_LINEAR_LIGHT : 0)); + if (alpha_ref) { tex->flags |= YAPF_FLAG_COVERAGE; tex->alpha_ref = alpha_ref; } + else tex->flags &= (uint8_t)~YAPF_FLAG_COVERAGE; + return YAPF_OK; +} diff --git a/yapf.h b/yapf.h index 4e60939..5f57933 100644 --- a/yapf.h +++ b/yapf.h @@ -38,6 +38,13 @@ * loader can upload the decoded buffer straight to a texture without any * additional transcoding step. * + * Two versions exist. Version 1 (below) is a single 8-bit 2D image with + * optional mips. Version 2 is the texture container: GPU block payloads + * (BCn / ETC2 / ASTC), 16-bit and float samples, cubemaps, arrays and + * volumes, channel meanings, per-tile checksums, optional entropy-coded + * planes, notes (metadata) and animation frames. Its complete + * specification is docs/FORMAT.md; a summary follows the v1 spec. + * * ========================================================================= * FILE FORMAT (version 1) * ========================================================================= @@ -140,6 +147,37 @@ * ───────────────────────────────────────────────── * bit 0 PREMULT_ALPHA alpha is premultiplied into RGB * bit 1 SRGB pixels are in sRGB gamma (informational only) + * bit 2 COVERAGE (v2) mips keep alpha-test coverage at alpha_ref + * + * ========================================================================= + * FILE FORMAT (version 2) — summary; the full text is docs/FORMAT.md + * ========================================================================= + * + * HEADER (64 bytes) + * 0x00 "YAPF" 0x04 version 2 0x05 channels 0x06 format + * 0x07 mips 0x08 flags 0x09 payload 0x0A sample 0x0B shape + * 0x0C width u32 0x10 height u32 0x14 depth u32 + * 0x18 layers u16 0x1A frames u16 0x1C needs u32 + * 0x20 channel map[4] 0x24 linear mask 0x25 normal 0x26 primaries + * 0x27 transfer 0x28 mip filter 0x29 alpha ref 0x2A loop count u16 + * 0x2C notes size u32 0x30 file size u64 + * 0x38 tables check u32 (yapf-sum of bytes [0x40, data start)) + * 0x3C header check u32 (yapf-sum of bytes [0, 0x3C)) + * SURFACE TABLE 16 bytes per surface: offset u64, size u32, + * tile-table check u32. Surface s of (frame, layer, + * face, mip) is ((frame·layers + layer)·faces + face)·mips + mip + * FRAME TABLE u32 duration in µs per frame (only when frames > 1) + * NOTES tag[4], length u32, bytes — repeated + * SURFACE DATA smallest mips first, so a file prefix is a complete + * low-resolution texture. Per surface: u32 book size, + * code book (Huffman tables by slot and context, only + * with entropy coding), tile table (size u32 with bit 31 = + * packed, yapf-sum u32 per tile), then the tiles. Pixel + * tiles are the v1 tiles plus byte planes for wide samples + * and entropy-coded planes; block tiles hold 16 × 16 GPU + * blocks, raw or packed lane by lane. + * NEEDS a decoder must refuse a file with a needs bit it does + * not implement; everything else may be ignored safely. */ #ifndef YAPF_H @@ -156,11 +194,58 @@ #define YAPF_GPU_R8 0x03 #define YAPF_GPU_SRGB8_A8 0x04 #define YAPF_GPU_SRGB8 0x05 +/* v2 pixel hints for wide samples */ +#define YAPF_GPU_RGBA16 0x06 +#define YAPF_GPU_RGB16 0x07 +#define YAPF_GPU_RG16 0x08 +#define YAPF_GPU_R16 0x09 +#define YAPF_GPU_RGBA16F 0x0A +#define YAPF_GPU_RGB16F 0x0B +#define YAPF_GPU_RG16F 0x0C +#define YAPF_GPU_R16F 0x0D +#define YAPF_GPU_RGBA32F 0x0E +#define YAPF_GPU_RGB32F 0x0F +#define YAPF_GPU_RG32F 0x10 +#define YAPF_GPU_R32F 0x11 +/* v2 GPU block formats (payload = YAPF_PAYLOAD_BLOCKS). The sRGB flag + * selects the _SRGB variant where the API has one. */ +#define YAPF_GPU_BC1 0x40 /* RGB(A1), 8 bytes per 4×4 */ +#define YAPF_GPU_BC2 0x41 /* RGBA, explicit 4-bit alpha */ +#define YAPF_GPU_BC3 0x42 /* RGBA */ +#define YAPF_GPU_BC4 0x43 /* R */ +#define YAPF_GPU_BC4_SNORM 0x44 +#define YAPF_GPU_BC5 0x45 /* RG (normal maps) */ +#define YAPF_GPU_BC5_SNORM 0x46 +#define YAPF_GPU_BC6H 0x47 /* RGB half float, unsigned */ +#define YAPF_GPU_BC6H_SF 0x48 /* RGB half float, signed */ +#define YAPF_GPU_BC7 0x49 /* RGB(A) */ +#define YAPF_GPU_ETC2_RGB8 0x50 +#define YAPF_GPU_ETC2_RGB8A1 0x51 +#define YAPF_GPU_ETC2_RGBA8 0x52 +#define YAPF_GPU_EAC_R11 0x53 +#define YAPF_GPU_EAC_R11_SNORM 0x54 +#define YAPF_GPU_EAC_RG11 0x55 +#define YAPF_GPU_EAC_RG11_SNORM 0x56 +#define YAPF_GPU_ASTC_4x4 0x60 /* ASTC: 0x60 + footprint index */ +#define YAPF_GPU_ASTC_5x4 0x61 +#define YAPF_GPU_ASTC_5x5 0x62 +#define YAPF_GPU_ASTC_6x5 0x63 +#define YAPF_GPU_ASTC_6x6 0x64 +#define YAPF_GPU_ASTC_8x5 0x65 +#define YAPF_GPU_ASTC_8x6 0x66 +#define YAPF_GPU_ASTC_8x8 0x67 +#define YAPF_GPU_ASTC_10x5 0x68 +#define YAPF_GPU_ASTC_10x6 0x69 +#define YAPF_GPU_ASTC_10x8 0x6A +#define YAPF_GPU_ASTC_10x10 0x6B +#define YAPF_GPU_ASTC_12x10 0x6C +#define YAPF_GPU_ASTC_12x12 0x6D /* ── Flags ───────────────────────────────────────────────────────────── */ #define YAPF_FLAG_PREMULT_ALPHA 0x01 #define YAPF_FLAG_SRGB 0x02 +#define YAPF_FLAG_COVERAGE 0x04 /* v2: mips preserve alpha-test coverage */ /* ── Channel constants ───────────────────────────────────────────────── */ @@ -176,12 +261,16 @@ #define YAPF_ERR_IO -2 #define YAPF_ERR_OOM -3 #define YAPF_ERR_CORRUPT -4 +#define YAPF_ERR_UNSUPPORTED -5 /* valid file, feature this decoder lacks */ +#define YAPF_ERR_CHECKSUM -6 /* a yapf-sum does not match */ +#define YAPF_ERR_TRUNCATED -7 /* data lies beyond the bytes available yet */ /* ── Limits ──────────────────────────────────────────────────────────── */ #define YAPF_MAX_DIM 65535u #define YAPF_MAX_MIPS 16u #define YAPF_TILE_SIZE 64u +#define YAPF_BLOCK_TILE 16u /* v2 block tiles: 16 × 16 GPU blocks */ /* ── Image descriptor ────────────────────────────────────────────────── */ @@ -196,6 +285,150 @@ typedef struct { uint8_t **mips; /* mips[0]=base alias, mips[1..mip_levels-1] owned */ } yapf_image_t; +/* ── Version 2: texture description ──────────────────────────────────── */ + +/* payload */ +#define YAPF_PAYLOAD_PIXELS 0 /* lossless YAPF tiles */ +#define YAPF_PAYLOAD_BLOCKS 1 /* GPU blocks, uploaded as they are */ + +/* sample: how one channel value is stored (pixel payload); for block + * payloads, the type of the decoded texels (U8, or F16 for BC6H) */ +#define YAPF_SAMPLE_U8 0 +#define YAPF_SAMPLE_U16 1 +#define YAPF_SAMPLE_F16 2 /* IEEE half, as uint16_t bits */ +#define YAPF_SAMPLE_F32 3 + +/* shape */ +#define YAPF_SHAPE_2D 0 +#define YAPF_SHAPE_CUBE 1 /* 6 faces: +X −X +Y −Y +Z −Z; with layers + > 1 a cube array */ +#define YAPF_SHAPE_VOLUME 2 /* depth slices; mips halve depth too */ + +/* needs: features a decoder must implement to read the file */ +#define YAPF_NEED_WIDE 0x01 /* sample other than U8 (pixel payload) */ +#define YAPF_NEED_BLOCKS 0x02 /* GPU block payload */ +#define YAPF_NEED_ENTROPY 0x04 /* code books: entropy planes, packed blocks */ +#define YAPF_NEED_LAYOUT 0x08 /* cube, volume or layers > 1 */ +#define YAPF_NEED_FRAMES 0x10 /* frames > 1 */ +#define YAPF_NEED_ALL 0x1F /* everything this library implements */ + +/* channel meanings (channel_map[c]); 0 = default for the channel count */ +#define YAPF_CH_DEFAULT 0 +#define YAPF_CH_RED 1 +#define YAPF_CH_GREEN 2 +#define YAPF_CH_BLUE 3 +#define YAPF_CH_ALPHA 4 +#define YAPF_CH_LUMA 5 +#define YAPF_CH_NORMAL_X 6 +#define YAPF_CH_NORMAL_Y 7 +#define YAPF_CH_NORMAL_Z 8 +#define YAPF_CH_ROUGHNESS 9 +#define YAPF_CH_GLOSS 10 /* smoothness = 1 − roughness */ +#define YAPF_CH_METALNESS 11 +#define YAPF_CH_OCCLUSION 12 /* ambient occlusion */ +#define YAPF_CH_HEIGHT 13 +#define YAPF_CH_SPECULAR 14 +#define YAPF_CH_EMISSIVE 15 +#define YAPF_CH_OPACITY 16 /* alpha-test / cut-out mask */ +#define YAPF_CH_MASK 17 /* generic mask */ +#define YAPF_CH_CAVITY 18 +#define YAPF_CH_THICKNESS 19 +#define YAPF_CH_ID 20 /* material or object id */ +#define YAPF_CH_UNUSED 255 + +/* normal: bits 0–1 convention, bits 2–3 encoding */ +#define YAPF_NORMAL_NONE 0x00 +#define YAPF_NORMAL_GL 0x01 /* +Y up (OpenGL, Blender, Unity) */ +#define YAPF_NORMAL_DX 0x02 /* −Y (DirectX, Unreal) */ +#define YAPF_NORMAL_XYZ 0x00 +#define YAPF_NORMAL_XY 0x04 /* Z = sqrt(1 − x² − y²) is reconstructed */ +#define YAPF_NORMAL_OCT 0x08 /* octahedral */ + +/* primaries / transfer (0 = unspecified: the sRGB flag decides) */ +#define YAPF_PRIMARIES_UNSPECIFIED 0 +#define YAPF_PRIMARIES_BT709 1 /* = sRGB */ +#define YAPF_PRIMARIES_DISPLAY_P3 2 +#define YAPF_PRIMARIES_BT2020 3 +#define YAPF_PRIMARIES_ACES_AP0 4 +#define YAPF_PRIMARIES_ACES_AP1 5 /* ACEScg */ +#define YAPF_PRIMARIES_ADOBE_RGB 6 + +#define YAPF_TRANSFER_UNSPECIFIED 0 +#define YAPF_TRANSFER_SRGB 1 +#define YAPF_TRANSFER_LINEAR 2 +#define YAPF_TRANSFER_GAMMA22 3 +#define YAPF_TRANSFER_PQ 4 /* SMPTE ST 2084 */ +#define YAPF_TRANSFER_HLG 5 +#define YAPF_TRANSFER_BT1886 6 /* gamma 2.4 */ + +/* mip filter (bits 0–6) | YAPF_MIP_LINEAR_LIGHT */ +#define YAPF_MIP_UNKNOWN 0 +#define YAPF_MIP_BOX 1 +#define YAPF_MIP_TENT 2 +#define YAPF_MIP_KAISER 3 +#define YAPF_MIP_LANCZOS 4 +#define YAPF_MIP_MITCHELL 5 +#define YAPF_MIP_AUTHORED 6 /* made by hand / by an external tool */ +#define YAPF_MIP_LINEAR_LIGHT 0x80 /* sRGB channels were filtered in linear light */ + +/* load flags */ +#define YAPF_LOAD_TRUST 0x01 /* skip checksum verification */ + +/* encode flags */ +#define YAPF_ENC_ENTROPY 0x01 /* allow entropy-coded planes and packed + blocks where they are smaller: ~10 % + smaller photos, ~3× smaller GPU blocks + on disk; slower to decode */ + +/* A note (metadata entry). Well-known tags: "AUTH" author, "LICN" + * licence, "SRCE" source, "TOOL" creating tool, "TITL" title, "DESC" + * description, "DATE" ISO 8601 date (all UTF-8 text); "ICCP" ICC profile; + * "ORNT" orientation (1 byte); "DPI " 2 × u32 milli-dots per inch; + * "CLL " 2 × u16 max content / max frame-average light level (cd/m²). + * Tags that start with a lowercase letter belong to applications. */ +typedef struct { + char tag[4]; + uint32_t size; + uint8_t *data; +} yapf_note_t; + +/* + * A version 2 texture. Surfaces are numbered + * s = ((frame × layers + layer) × faces + face) × mip_levels + mip + * (faces = 6 for cubes, else 1) and data[s] holds the whole surface: + * pixels: width_m × height_m × depth_m × channels samples, rows top to + * bottom, slices in order; samples in the native byte order + * (uint8_t, uint16_t, half bits as uint16_t, float); + * blocks: ceil(width_m / bw) × ceil(height_m / bh) blocks per slice, + * row-major — exactly what glCompressedTexImage / Vulkan take. + * width_m = max(1, width >> m), likewise height and (volumes) depth. + */ +typedef struct { + uint32_t width, height, depth; /* depth: volumes only, else 1 */ + uint16_t layers; /* array layers, >= 1 */ + uint16_t frames; /* animation frames, >= 1 */ + uint8_t shape; /* YAPF_SHAPE_* */ + uint8_t mip_levels; /* 1–16 */ + uint8_t payload; /* YAPF_PAYLOAD_* */ + uint8_t format; /* YAPF_GPU_*: block format, or hint */ + uint8_t sample; /* YAPF_SAMPLE_* */ + uint8_t channels; /* 1–4 */ + uint8_t flags; /* YAPF_FLAG_* */ + uint8_t channel_map[4]; /* YAPF_CH_* per channel */ + uint8_t linear_mask; /* bit c: channel c is linear even in + an sRGB image (e.g. alpha, masks) */ + uint8_t normal; /* YAPF_NORMAL_* */ + uint8_t primaries, transfer; /* YAPF_PRIMARIES_*, YAPF_TRANSFER_* */ + uint8_t mip_filter; /* YAPF_MIP_* */ + uint8_t alpha_ref; /* coverage reference (with COVERAGE) */ + uint16_t loop_count; /* animation loops, 0 = forever */ + uint32_t *frame_us; /* frames durations in µs, or NULL */ + uint32_t note_count; + yapf_note_t *notes; + void **data; /* one pointer per surface */ + void *internal_; /* library use: file tables, ownership */ +} yapf_texture_t; + /* ── Public API ──────────────────────────────────────────────────────── */ #ifdef __cplusplus @@ -249,6 +482,132 @@ void yapf_free_buffer(void *data); */ void yapf_free(yapf_image_t *img); +/* + * The v1 functions above also read version 2 files that hold 8-bit pixels + * of a 2D, cube or array texture: they return frame 0, layer 0, face 0 + * with all its mips. yapf_save / yapf_encode always write version 1. + */ + +/* ── Version 2: textures ─────────────────────────────────────────────── */ + +/* + * yapf_tex_create — a library-owned texture with the description of + * `desc` (every field up to loop_count; notes, frame times and data are + * not copied) and zeroed storage for every surface. frame_us is + * allocated (zeroed) when frames > 1. NULL on invalid description / OOM. + */ +yapf_texture_t *yapf_tex_create(const yapf_texture_t *desc); + +/* yapf_tex_free — release a texture made by the library. NULL-safe. */ +void yapf_tex_free(yapf_texture_t *tex); + +/* Surface helpers. */ +uint32_t yapf_tex_faces(const yapf_texture_t *tex); /* 6 or 1 */ +uint32_t yapf_tex_surface_count(const yapf_texture_t *tex); +uint32_t yapf_tex_surface(const yapf_texture_t *tex, uint32_t frame, + uint32_t layer, uint32_t face, uint32_t mip); +void yapf_tex_mip_size(const yapf_texture_t *tex, uint32_t mip, + uint32_t *w, uint32_t *h, uint32_t *d); +size_t yapf_tex_surface_bytes(const yapf_texture_t *tex, uint32_t mip); +size_t yapf_sample_bytes(uint8_t sample); /* 1, 2 or 4 */ + +/* Notes: add (library-owned textures only; data is copied) and find. */ +int yapf_tex_add_note(yapf_texture_t *tex, const char tag[4], + const void *data, uint32_t size); +const yapf_note_t *yapf_tex_find_note(const yapf_texture_t *tex, const char tag[4]); + +/* + * yapf_tex_encode / yapf_tex_save — write a version 2 file. `flags` is + * YAPF_ENC_*. Works on library-owned and caller-built textures alike + * (nothing is taken over). Returns YAPF_OK or a YAPF_ERR_* code. + */ +int yapf_tex_encode(const yapf_texture_t *tex, int flags, + void **out_data, size_t *out_size); +int yapf_tex_save(const char *filename, const yapf_texture_t *tex, int flags); + +/* + * yapf_tex_load / yapf_tex_load_memory — read a version 1 or 2 file and + * decode every surface on up to `threads` threads (0 = one per core). + * `flags` is YAPF_LOAD_*. On failure NULL is returned and *err (if not + * NULL) receives the YAPF_ERR_* reason. + */ +yapf_texture_t *yapf_tex_load(const char *filename, int flags, int threads, int *err); +yapf_texture_t *yapf_tex_load_memory(const void *buffer, size_t size, + int flags, int threads, int *err); + +/* + * Streaming and partial decoding. + * + * yapf_tex_open — read only the header, tables and notes. `avail` may be + * less than the whole file (a download in progress): it must cover the + * front matter, else YAPF_ERR_TRUNCATED. All data[] pointers are NULL. + * + * yapf_tex_stream_bytes — how many leading bytes of the file hold every + * surface of mip `first_mip` and smaller (the file stores small mips + * first, so these are a usable texture on their own). + * + * yapf_tex_decode_surface — decode surface s into tex->data[s] + * (allocated). `buffer`/`avail` is the file or a prefix of it; returns + * YAPF_ERR_TRUNCATED when the surface is not in it yet. + * + * yapf_tex_tile_grid / yapf_tex_decode_tile — virtual texturing: the + * surfaces of mip m are cut into tiles (64 × 64 pixels, or 16 × 16 + * blocks), tx across, ty down, per slice. decode_tile writes one tile + * into `out`, rows `stride` bytes apart, and reports its pixel rectangle. + */ +yapf_texture_t *yapf_tex_open(const void *buffer, size_t avail, int flags, int *err); +uint64_t yapf_tex_stream_bytes(const yapf_texture_t *tex, uint32_t first_mip); +int yapf_tex_decode_surface(yapf_texture_t *tex, const void *buffer, size_t avail, + uint32_t surface, int threads); +void yapf_tex_tile_grid(const yapf_texture_t *tex, uint32_t mip, + uint32_t *tiles_x, uint32_t *tiles_y); +int yapf_tex_decode_tile(const yapf_texture_t *tex, const void *buffer, size_t avail, + uint32_t surface, uint32_t tx, uint32_t ty, uint32_t slice, + void *out, size_t stride, + uint32_t *x, uint32_t *y, uint32_t *w, uint32_t *h); + +/* + * yapf_tex_build_mips — replace the mips of a library-owned pixel texture + * with a chain made from mip 0: `levels` levels (0 = down to 1 × 1). + * 2 × 2 (× 2 for volumes) box filter; sRGB channels are averaged in + * linear light. alpha_ref > 0 rescales alpha on every mip so the share + * of texels above alpha_ref stays that of mip 0 (fences and foliage do + * not thin out); it sets YAPF_FLAG_COVERAGE. Records the filter used. + */ +int yapf_tex_build_mips(yapf_texture_t *tex, uint32_t levels, uint8_t alpha_ref); + +/* ── GPU blocks ──────────────────────────────────────────────────────── */ + +/* Block footprint and size of a YAPF_GPU_* block format; 0 if not one. */ +int yapf_block_info(uint8_t format, uint32_t *block_w, uint32_t *block_h, + uint32_t *block_bytes); + +/* + * yapf_tex_compress — a new library-owned texture holding `src` (pixel + * payload) as GPU blocks. Built-in encoders: BC1, BC3, BC4, BC5, BC7 and + * BC6H (unsigned). LDR formats take 8-bit pixels; BC6H takes 16-bit, + * half or float pixels. For other formats bring blocks from your own + * encoder: a payload = BLOCKS texture with data[] filled. + */ +int yapf_tex_compress(const yapf_texture_t *src, uint8_t format, + yapf_texture_t **out); + +/* + * yapf_blocks_decode — decode a w × h image of BC1–BC7 blocks (previews, + * tools). LDR formats write RGBA8 (BC4 / BC5: R / RG, then 0, 255; + * snorm values shifted by +128); BC6H writes RGB half floats. Returns + * YAPF_ERR_UNSUPPORTED for ETC2 / EAC / ASTC. + */ +int yapf_blocks_decode(uint8_t format, const void *blocks, uint32_t w, uint32_t h, + void *out); + +/* yapf-sum, the checksum used by version 2 files. */ +uint32_t yapf_checksum(const void *data, size_t size); + +/* Half float conversion helpers. */ +uint16_t yapf_float_to_half(float f); +float yapf_half_to_float(uint16_t h); + #ifdef __cplusplus } #endif From fb7212c62b418c22fdf4fe900f23b88adfc11e6c Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 11 Oct 2026 09:09:51 +0000 Subject: [PATCH 2/7] GIMP plugin: NixOS development shell Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DVrVthWN8GsmgHGUhhsCYm --- extensions/gimp/README.md | 6 ++++++ extensions/gimp/gimp-shell.nix | 7 +++++++ 2 files changed, 13 insertions(+) create mode 100644 extensions/gimp/gimp-shell.nix diff --git a/extensions/gimp/README.md b/extensions/gimp/README.md index d3dbae0..2aa18f0 100644 --- a/extensions/gimp/README.md +++ b/extensions/gimp/README.md @@ -16,6 +16,7 @@ You need GIMP 3 with its development files (`gimptool`). | Arch / Manjaro | `sudo pacman -S gimp` (includes them) | | Debian / Ubuntu | `sudo apt install libgimp-3.0-dev` (GIMP 3 packages) | | Fedora | `sudo dnf install gimp-devel` | +| NixOS | `nix-shell gimp-shell.nix` (in this folder), then build inside that shell | | Windows | MSYS2: `pacman -S mingw-w64-ucrt-x86_64-gimp` | | macOS | build GIMP 3 from source or use MacPorts/Homebrew | @@ -28,6 +29,11 @@ gimptool --install yapf-gimp.c That compiles the plugin (it pulls in `../../yapf.c` itself) and installs it into your personal GIMP plug-ins folder. Restart GIMP. +On NixOS, build with the same nixpkgs channel your GIMP comes from (the +plugin links against that GIMP's libraries), and build it again after a +GIMP update. Your GIMP must be GIMP 3: `environment.systemPackages = [ +pkgs.gimp3 ];`, or `nix-shell -p gimp3 --run gimp` to try it. + ### Manual build ```sh diff --git a/extensions/gimp/gimp-shell.nix b/extensions/gimp/gimp-shell.nix new file mode 100644 index 0000000..7a8fadf --- /dev/null +++ b/extensions/gimp/gimp-shell.nix @@ -0,0 +1,7 @@ +# Development shell for building the plugin on NixOS: nix-shell gimp-shell.nix +# `gimp3` is GIMP 3 on every channel (on unstable it is an alias of `gimp`). +with import {}; +mkShell { + nativeBuildInputs = [ pkg-config ]; + buildInputs = [ gimp3 gimp3.dev gtk3 glib gdk-pixbuf gegl babl cairo pango ]; +} From 83588d8e2af7f7585f96b7d21c69278d5d41ee12 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 11 Oct 2026 09:30:22 +0000 Subject: [PATCH 3/7] GIMP plugin: open every YAPF 2 file and say why a file fails Uses the texture loader: GPU blocks are decoded for viewing, 16-bit and float files open at full precision, and cube faces, array layers, slices and animation frames become layers. Load errors name the reason instead of a generic failure. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DVrVthWN8GsmgHGUhhsCYm --- extensions/gimp/README.md | 10 ++- extensions/gimp/yapf-gimp.c | 160 ++++++++++++++++++++++++++---------- 2 files changed, 124 insertions(+), 46 deletions(-) diff --git a/extensions/gimp/README.md b/extensions/gimp/README.md index 2aa18f0..f350ff1 100644 --- a/extensions/gimp/README.md +++ b/extensions/gimp/README.md @@ -2,7 +2,11 @@ Open and export `.yapf` images in **GIMP 3.0 or later**. -- **File → Open** reads `.yapf` (Gray, Gray + Alpha, RGB, RGBA). +- **File → Open** reads every `.yapf` file, version 1 and 2: Gray, Gray + + Alpha, RGB, RGBA in 8-bit, 16-bit, half and float precision; GPU blocks + (BC1–BC7, BC6H HDR) decoded for viewing; cube faces, array layers, volume + slices and animation frames as separate layers. When a file cannot be + opened, the message says why (damaged, truncated, not YAPF, newer format). - **File → Export As… → `name.yapf`** writes the visible image (layers are flattened into a copy; your document is untouched), with an optional **Store mipmaps** checkbox for game engines. @@ -50,5 +54,9 @@ GIMP config folder is `~/.config/GIMP/3.0` on Linux, - Pixels are saved with the sRGB flag, because GIMP's 8-bit buffers are gamma encoded. Files marked linear load as linear. - When a file has mipmaps, GIMP opens the full-size image. +- ETC2 / EAC / ASTC blocks cannot be shown yet. Export writes 8-bit + version 1 files, as before. - Tested with GIMP 3.0.8 on Linux: opening matches the reference decoder pixel for pixel, and export equals GIMP's own flattened PNG of the image. + The version 2 loader was tested in GIMP 3.0.6 (nixpkgs 25.05, headless): + BC7, BC5, BC6H, half float, 16-bit, cube, animation and a damaged file. diff --git a/extensions/gimp/yapf-gimp.c b/extensions/gimp/yapf-gimp.c index 7153f10..948ee13 100644 --- a/extensions/gimp/yapf-gimp.c +++ b/extensions/gimp/yapf-gimp.c @@ -138,8 +138,44 @@ static GimpProcedure *yapf_plugin_create_procedure(GimpPlugIn *plug_in, return procedure; } +static GimpValueArray *yapf_error(GimpProcedure *procedure, const gchar *msg); + /* ── Load handler ────────────────────────────────────────────────────── */ +/* Why a file could not be opened, in words. */ +static gchar *yapf_load_reason(const gchar *path, int err) { + FILE *f = fopen(path, "rb"); + uint8_t h[8] = {0}; + size_t n = f ? fread(h, 1, sizeof(h), f) : 0; + if (f) fclose(f); + if (!f || err == YAPF_ERR_IO) + return g_strdup("The file cannot be read."); + if (n < 5 || memcmp(h, "YAPF", 4) != 0) + return g_strdup("This is not a YAPF file (it does not start with \"YAPF\")."); + if (h[4] != 1 && h[4] != 2) + return g_strdup_printf("This YAPF file is format version %u; this plug-in reads " + "versions 1 and 2. Update the plug-in.", h[4]); + switch (err) { + case YAPF_ERR_UNSUPPORTED: + return g_strdup("This YAPF file needs a feature this plug-in does not have yet. " + "Update the plug-in."); + case YAPF_ERR_CHECKSUM: + return g_strdup("This YAPF file is damaged: a checksum does not match."); + case YAPF_ERR_TRUNCATED: + return g_strdup("This YAPF file is incomplete (truncated)."); + case YAPF_ERR_OOM: + return g_strdup("Out of memory."); + default: + return g_strdup_printf("This YAPF file (version %u) is corrupt, or was written by a " + "tool that does not follow the YAPF specification.", h[4]); + } +} + +static const gchar *yapf_face_name(uint32_t f) { + static const gchar *n[6] = { "+X", "-X", "+Y", "-Y", "+Z", "-Z" }; + return f < 6 ? n[f] : "?"; +} + static GimpValueArray *yapf_load_run( GimpProcedure *procedure, GimpRunMode run_mode, @@ -151,60 +187,94 @@ static GimpValueArray *yapf_load_run( { (void)run_mode; (void)metadata; (void)flags; (void)config; (void)run_data; - gchar *path = g_file_get_path(file); - yapf_image_t *img = yapf_load_mt(path, 0); /* all cores */ + gchar *path = g_file_get_path(file); + int err = 0; + yapf_texture_t *t = yapf_tex_load(path, 0, 0, &err); /* v1 and v2, all cores */ + if (!t) { + gchar *why = yapf_load_reason(path, err); + g_free(path); + GError *e = g_error_new(GIMP_PLUG_IN_ERROR, 0, "Failed to load YAPF file: %s", why); + g_free(why); + return gimp_procedure_new_return_values(procedure, GIMP_PDB_EXECUTION_ERROR, e); + } g_free(path); - if (!img) { - GError *err = g_error_new(GIMP_PLUG_IN_ERROR, 0, - "Failed to load YAPF file."); - return gimp_procedure_new_return_values(procedure, - GIMP_PDB_EXECUTION_ERROR, err); + /* What the pixels look like after decoding: GPU blocks decode to + * RGBA 8-bit, BC6H to RGB half float. */ + gboolean blocks = t->payload == YAPF_PAYLOAD_BLOCKS; + gboolean bc6 = blocks && (t->format == YAPF_GPU_BC6H || t->format == YAPF_GPU_BC6H_SF); + if (blocks && t->format >= YAPF_GPU_ETC2_RGB8) { + yapf_tex_free(t); + return yapf_error(procedure, "Failed to load YAPF file: it holds ETC2 / EAC / ASTC " + "GPU blocks, which this plug-in cannot show yet."); } + int ch = blocks ? (bc6 ? 3 : 4) : t->channels; + int sample = blocks ? (bc6 ? YAPF_SAMPLE_F16 : YAPF_SAMPLE_U8) : t->sample; + /* sRGB data is gamma encoded; floats are always linear */ + gboolean gamma = (t->flags & YAPF_FLAG_SRGB) && + (sample == YAPF_SAMPLE_U8 || sample == YAPF_SAMPLE_U16); - /* Map YAPF channel count to GIMP image / layer types. - * Use gamma-corrected babl formats (R'G'B') when the file carries the - * sRGB flag; fall back to linear (RGB) otherwise. */ - gboolean is_srgb = (img->flags & YAPF_FLAG_SRGB) != 0; - GimpImageBaseType base_type = GIMP_RGB; - GimpImageType layer_type = GIMP_RGBA_IMAGE; - const gchar *babl_fmt = is_srgb ? "R'G'B'A u8" : "RGBA u8"; + static const gchar *chn[2][4] = { { "Y", "YA", "RGB", "RGBA" }, { "Y'", "Y'A", "R'G'B'", "R'G'B'A" } }; + static const gchar *typ[4] = { "u8", "u16", "half", "float" }; + gchar *fmt_name = g_strdup_printf("%s %s", chn[gamma ? 1 : 0][ch - 1], typ[sample]); + const Babl *fmt = babl_format(fmt_name); + g_free(fmt_name); - switch (img->channels) { - case YAPF_CHANNELS_GRAY: - base_type = GIMP_GRAY; - layer_type = GIMP_GRAY_IMAGE; - babl_fmt = is_srgb ? "Y' u8" : "Y u8"; - break; - case YAPF_CHANNELS_GRAY_ALPHA: - base_type = GIMP_GRAY; - layer_type = GIMP_GRAYA_IMAGE; - babl_fmt = is_srgb ? "Y'A u8" : "YA u8"; - break; - case YAPF_CHANNELS_RGB: - base_type = GIMP_RGB; - layer_type = GIMP_RGB_IMAGE; - babl_fmt = is_srgb ? "R'G'B' u8" : "RGB u8"; - break; - default: /* RGBA */ - break; + GimpPrecision prec; + switch (sample) { + case YAPF_SAMPLE_U16: prec = gamma ? GIMP_PRECISION_U16_NON_LINEAR : GIMP_PRECISION_U16_LINEAR; break; + case YAPF_SAMPLE_F16: prec = GIMP_PRECISION_HALF_LINEAR; break; + case YAPF_SAMPLE_F32: prec = GIMP_PRECISION_FLOAT_LINEAR; break; + default: prec = gamma ? GIMP_PRECISION_U8_NON_LINEAR : GIMP_PRECISION_U8_LINEAR; break; } + GimpImageBaseType base = ch <= 2 ? GIMP_GRAY : GIMP_RGB; + GimpImageType ltype = ch == 1 ? GIMP_GRAY_IMAGE : ch == 2 ? GIMP_GRAYA_IMAGE + : ch == 3 ? GIMP_RGB_IMAGE : GIMP_RGBA_IMAGE; - GimpImage *image = gimp_image_new(img->width, img->height, base_type); - GimpLayer *layer = gimp_layer_new(image, "Background", - img->width, img->height, layer_type, - 100.0, - gimp_image_get_default_new_layer_mode(image)); - gimp_image_insert_layer(image, layer, NULL, 0); + /* Mip 0 of every frame, layer, face and slice becomes one GIMP layer + * (the first at the bottom, as GIMP animations expect). */ + uint32_t w, h, d, faces = yapf_tex_faces(t); + yapf_tex_mip_size(t, 0, &w, &h, &d); + GimpImage *image = gimp_image_new_with_precision(w, h, base, prec); + size_t texel = (size_t)ch * yapf_sample_bytes((uint8_t)sample); + void *tmp = blocks ? g_malloc((size_t)w * h * texel) : NULL; + gint count = (gint)(t->frames * t->layers * faces * d); - GeglBuffer *buffer = gimp_drawable_get_buffer(GIMP_DRAWABLE(layer)); - gegl_buffer_set(buffer, - GEGL_RECTANGLE(0, 0, (gint)img->width, (gint)img->height), - 0, babl_format(babl_fmt), - img->pixels, GEGL_AUTO_ROWSTRIDE); - g_object_unref(buffer); + for (uint32_t fr = 0; fr < t->frames; fr++) + for (uint32_t ly = 0; ly < t->layers; ly++) + for (uint32_t fc = 0; fc < faces; fc++) + for (uint32_t z = 0; z < d; z++) { + uint32_t s = yapf_tex_surface(t, fr, ly, fc, 0); + size_t slice = yapf_tex_surface_bytes(t, 0) / d; + const uint8_t *src = (const uint8_t *)t->data[s] + slice * z; + if (blocks) { + yapf_blocks_decode(t->format, src, w, h, tmp); + src = (const uint8_t *)tmp; + } + GString *name = g_string_new(NULL); + if (t->frames > 1) { + g_string_append_printf(name, "Frame %u", fr + 1); + if (t->frame_us) + g_string_append_printf(name, " (%ums)", (t->frame_us[fr] + 500) / 1000); + } + if (t->layers > 1) g_string_append_printf(name, "%sLayer %u", name->len ? ", " : "", ly); + if (faces > 1) g_string_append_printf(name, "%sFace %s", name->len ? ", " : "", yapf_face_name(fc)); + if (d > 1) g_string_append_printf(name, "%sSlice %u", name->len ? ", " : "", z); + if (!name->len) g_string_append(name, "Background"); - yapf_free(img); + GimpLayer *layer = gimp_layer_new(image, name->str, (gint)w, (gint)h, ltype, + 100.0, gimp_image_get_default_new_layer_mode(image)); + g_string_free(name, TRUE); + gimp_image_insert_layer(image, layer, NULL, 0); + if (count > 1 && !(fr == 0 && ly == 0 && fc == 0 && z == 0)) + gimp_item_set_visible(GIMP_ITEM(layer), FALSE); + GeglBuffer *buffer = gimp_drawable_get_buffer(GIMP_DRAWABLE(layer)); + gegl_buffer_set(buffer, GEGL_RECTANGLE(0, 0, (gint)w, (gint)h), 0, fmt, + src, GEGL_AUTO_ROWSTRIDE); + g_object_unref(buffer); + } + g_free(tmp); + yapf_tex_free(t); GimpValueArray *ret = gimp_procedure_new_return_values(procedure, GIMP_PDB_SUCCESS, NULL); From a6c032d824daef9ba734bce8dff1eb96b719b777 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 11 Oct 2026 09:40:30 +0000 Subject: [PATCH 4/7] Extensions read YAPF 2; Nix package - KDE thumbnailer: texture loader, decodes only the mip it needs; GPU blocks, 16-bit and float via the new yapf_tex_to_rgba8. Tested through KDE's plugin loader with KF6 from nixpkgs. - Photoshop: shows the tool's error message; HDR textures open as 32-bit (.hdr), 16-bit stays 16-bit; 16-bit documents save as 16-bit. - CLI: 16-bit PNG export, readable conversion errors. - default.nix: yapf tool, library, header, MIME type and thumbnailer (absolute store path); runs the test suite. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DVrVthWN8GsmgHGUhhsCYm --- README.md | 1 + default.nix | 39 +++++++++++++ extensions/kde/README.md | 6 ++ extensions/kde/yapfthumbnail.cpp | 73 +++++++++++++------------ extensions/linux/README.md | 19 +++++++ extensions/photoshop/Open YAPF.jsx | 6 +- extensions/photoshop/README.md | 10 ++++ extensions/photoshop/Save as YAPF.jsx | 8 ++- extensions/photoshop/yapf_common.jsxinc | 42 +++++++++++++- tests/test_yapf.c | 30 ++++++++++ tools/yapf_cli.c | 72 ++++++++++++++++++++++-- yapf.c | 68 +++++++++++++++++++++++ yapf.h | 10 ++++ 13 files changed, 336 insertions(+), 48 deletions(-) create mode 100644 default.nix diff --git a/README.md b/README.md index 58a0f18..3b3315f 100644 --- a/README.md +++ b/README.md @@ -332,6 +332,7 @@ YAPF_CHANNELS_RGBA 4 | System | Command | Output | | :----- | :------ | :----- | | Linux, macOS, BSD | `make` · `make test` · `sudo make install` | `build/`: `libyapf.a`, `libyapf.so` / `.dylib`, `yapf` | +| NixOS / Nix | `nix-build` (or `nix-env -if .`) — see [extensions/linux](extensions/linux#nixos) | `result/`: `bin/yapf`, `lib/`, `include/`, MIME type, thumbnailer | | Linux, macOS (LLVM) | `./build.sh` | `dist//` incl. macOS universal binaries | | Windows (LLVM) | `build.bat` | `dist\windows-x64\`, `dist\windows-arm64\`: `yapf.dll`, `yapf.lib`, `libyapf.a`, `yapf.exe` | | Windows (Visual Studio) | `cl /O2 /c yapf.c` and add `yapf.obj` to your project | | diff --git a/default.nix b/default.nix new file mode 100644 index 0000000..fd79d78 --- /dev/null +++ b/default.nix @@ -0,0 +1,39 @@ +# YAPF for Nix / NixOS: the `yapf` tool, libyapf (static + shared) and +# yapf.h, plus the image/x-yapf MIME type and a freedesktop thumbnailer. +# +# nix-build build ./result +# nix-env -if . install for your user +# NixOS: environment.systemPackages = [ (pkgs.callPackage /path/to/YAPF { }) ]; +{ pkgs ? import { }, lib ? pkgs.lib, stdenv ? pkgs.stdenv }: + +stdenv.mkDerivation { + pname = "yapf"; + version = "2.0"; + + src = lib.cleanSourceWith { + src = ./.; + filter = path: type: + let base = baseNameOf path; in + !(lib.elem base [ "build" "dist" "result" ".git" "node_modules" ]); + }; + + makeFlags = [ "PREFIX=$(out)" ]; + doCheck = true; + checkTarget = "test"; + + postInstall = '' + install -Dm644 extensions/linux/yapf-mime.xml $out/share/mime/packages/yapf.xml + install -Dm644 extensions/linux/yapf.thumbnailer $out/share/thumbnailers/yapf.thumbnailer + substituteInPlace $out/share/thumbnailers/yapf.thumbnailer \ + --replace-fail "TryExec=yapf" "TryExec=$out/bin/yapf" \ + --replace-fail "Exec=yapf " "Exec=$out/bin/yapf " + ''; + + meta = { + description = "YAPF: fast lossless images and GPU-ready textures"; + homepage = "https://github.com/Nexoniarz/YAPF"; + license = lib.licenses.asl20; + platforms = lib.platforms.unix; + mainProgram = "yapf"; + }; +} diff --git a/extensions/kde/README.md b/extensions/kde/README.md index 6283557..f5b49bc 100644 --- a/extensions/kde/README.md +++ b/extensions/kde/README.md @@ -48,3 +48,9 @@ Log out and back in, then enable **YAPF Images** in Dolphin's preview settings. Built against KF6 / Qt 6 and tested by loading the plugin through KDE's plugin factory and creating thumbnails (RGBA, RGB, mipmapped, wide images; corrupt files are rejected). Not yet tried inside a running Dolphin. + +Reads YAPF version 1 and 2: GPU blocks (BC1–BC7, BC6H), 16-bit and float +textures, cubes (face +X), arrays, volumes and animations (the first +image). Only the mip level the thumbnail needs is decoded. Tested with +KF6 from nixpkgs 25.05 on BC7, BC5, BC6H, half float, 16-bit, cube and +animation files. ETC2 / ASTC files get no thumbnail. diff --git a/extensions/kde/yapfthumbnail.cpp b/extensions/kde/yapfthumbnail.cpp index 6842a65..fe46adc 100644 --- a/extensions/kde/yapfthumbnail.cpp +++ b/extensions/kde/yapfthumbnail.cpp @@ -3,8 +3,11 @@ #include #include +#include #include +#include + extern "C" { #include "yapf.h" } @@ -20,52 +23,50 @@ public: KIO::ThumbnailResult create(const KIO::ThumbnailRequest &request) override { - const QByteArray path = request.url().toLocalFile().toLocal8Bit(); - yapf_image_t *img = yapf_load(path.constData()); - if (!img) { + QFile file(request.url().toLocalFile()); + if (!file.open(QIODevice::ReadOnly)) { + return KIO::ThumbnailResult::fail(); + } + const QByteArray data = file.readAll(); + + // Version 1 and 2: read the tables, then decode only the mip we + // need (frame 0, layer 0, face 0, slice 0). + int err = 0; + yapf_texture_t *tex = yapf_tex_open(data.constData(), size_t(data.size()), 0, &err); + if (!tex) { return KIO::ThumbnailResult::fail(); } - // Start from the smallest stored mip level that still covers the - // requested size; Qt scales the rest. + // The smallest stored mip that still covers the requested size; + // Qt scales the rest. const QSize want = request.targetSize(); - int m = 0; - while (m + 1 < img->mip_levels && - int(img->width >> (m + 1)) >= want.width() && - int(img->height >> (m + 1)) >= want.height()) { + uint32_t m = 0, w = 0, h = 0; + while (m + 1 < tex->mip_levels) { + yapf_tex_mip_size(tex, m + 1, &w, &h, nullptr); + if (int(w) < want.width() || int(h) < want.height()) { + break; + } ++m; } - const int w = qMax(1, int(img->width >> m)); - const int h = qMax(1, int(img->height >> m)); - const uchar *px = img->mips[m]; + yapf_tex_mip_size(tex, m, &w, &h, nullptr); + const uint32_t s = yapf_tex_surface(tex, 0, 0, 0, m); - QImage image; - switch (img->channels) { - case 1: - image = QImage(px, w, h, w, QImage::Format_Grayscale8).copy(); - break; - case 2: { // gray + alpha → RGBA - image = QImage(w, h, QImage::Format_RGBA8888); - for (int y = 0; y < h; ++y) { - const uchar *s = px + size_t(y) * w * 2; - uchar *d = image.scanLine(y); - for (int x = 0; x < w; ++x, s += 2, d += 4) { - d[0] = d[1] = d[2] = s[0]; - d[3] = s[1]; - } + QImage image(int(w), int(h), (tex->flags & YAPF_FLAG_PREMULT_ALPHA) ? QImage::Format_RGBA8888_Premultiplied + : QImage::Format_RGBA8888); + bool ok = !image.isNull() && + yapf_tex_decode_surface(tex, data.constData(), size_t(data.size()), s, 0) == YAPF_OK; + if (ok) { + // RGBA 8-bit rows (blocks decoded, 16-bit / float converted) + QByteArray rgba(int(size_t(w) * h * 4), Qt::Uninitialized); + ok = yapf_tex_to_rgba8(tex, s, 0, reinterpret_cast(rgba.data())) == YAPF_OK; + for (int y = 0; ok && y < int(h); ++y) { + memcpy(image.scanLine(y), rgba.constData() + size_t(y) * w * 4, size_t(w) * 4); } - break; } - case 3: - image = QImage(px, w, h, w * 3, QImage::Format_RGB888).copy(); - break; - default: - image = QImage(px, w, h, w * 4, - (img->flags & YAPF_FLAG_PREMULT_ALPHA) ? QImage::Format_RGBA8888_Premultiplied - : QImage::Format_RGBA8888).copy(); - break; + yapf_tex_free(tex); + if (!ok) { + return KIO::ThumbnailResult::fail(); } - yapf_free(img); if (image.width() > want.width() || image.height() > want.height()) { image = image.scaled(want, Qt::KeepAspectRatio, Qt::SmoothTransformation); diff --git a/extensions/linux/README.md b/extensions/linux/README.md index 04c08f2..b3a4d86 100644 --- a/extensions/linux/README.md +++ b/extensions/linux/README.md @@ -13,3 +13,22 @@ on your `PATH` — `sudo make install` puts it in `/usr/local/bin`). They work in file managers that use freedesktop thumbnailers: GNOME Files (Nautilus), Nemo, Caja, and Thunar (with tumbler). KDE Dolphin uses its own plugin system and is not covered yet. + +Thumbnails cover every YAPF file the tool reads: version 1 and 2, GPU +blocks (BC1–BC7, BC6H), 16-bit and float textures, and the first image of +cubes, arrays and animations. KDE Dolphin: see [../kde](../kde). + +### NixOS + +`/usr/share` is not used on NixOS. Install the package from the +repository's `default.nix` instead — it carries the MIME type and a +thumbnailer that calls `yapf` by its full store path: + +```nix +# configuration.nix +environment.systemPackages = [ (pkgs.callPackage /path/to/YAPF { }) ]; +environment.pathsToLink = [ "/share/thumbnailers" ]; # if your desktop does not link it already +``` + +For your user only: `nix-env -if /path/to/YAPF`, then +`sh extensions/linux/install.sh` (the per-user install). diff --git a/extensions/photoshop/Open YAPF.jsx b/extensions/photoshop/Open YAPF.jsx index 52860bf..addfb56 100644 --- a/extensions/photoshop/Open YAPF.jsx +++ b/extensions/photoshop/Open YAPF.jsx @@ -14,9 +14,11 @@ if (!(files instanceof Array)) files = [files]; for (var i = 0; i < files.length; i++) { - var tmp = yapfTempFile(".png"); + // 8 / 16-bit images come through PNG (16 bits are kept); half, + // float and BC6H HDR textures through Radiance .hdr (32-bit). + var tmp = yapfTempFile(yapfIsHdr(files[i]) ? ".hdr" : ".png"); if (!yapfRun([files[i].fsName, tmp.fsName], tmp)) { - alert("Could not open " + files[i].name + ".\n\n" + YAPF_TOOL_HELP); + alert("Could not open " + files[i].name + ".\n\n" + yapfFailText()); return; } var doc = app.open(tmp); diff --git a/extensions/photoshop/README.md b/extensions/photoshop/README.md index 7cea292..7ce8fbf 100644 --- a/extensions/photoshop/README.md +++ b/extensions/photoshop/README.md @@ -25,6 +25,16 @@ native plugin. Instead of copying `yapf` next to the scripts you can put it on your `PATH`. +## What opens how + +Every YAPF file the `yapf` tool reads (version 1 and 2): 8-bit images open +as 8-bit documents, 16-bit textures as 16-bit documents, half / float and +BC6H HDR textures as 32-bit documents (through Radiance `.hdr`), other GPU +blocks (BC1–BC7) decoded to 8-bit. Cubes, arrays and animations open their +first image. When a file cannot be opened, the alert shows the tool's +reason (for example a damaged file). Saving keeps 8 or 16 bits per +channel. + ## Status The scripts are written for Photoshop CC's ExtendScript and are not yet diff --git a/extensions/photoshop/Save as YAPF.jsx b/extensions/photoshop/Save as YAPF.jsx index 1ca326c..05e7cf2 100644 --- a/extensions/photoshop/Save as YAPF.jsx +++ b/extensions/photoshop/Save as YAPF.jsx @@ -22,8 +22,10 @@ try { if (copy.mode !== DocumentMode.RGB && copy.mode !== DocumentMode.GRAYSCALE) copy.changeMode(ChangeMode.RGB); - if (copy.bitsPerChannel !== BitsPerChannelType.EIGHT) - copy.bitsPerChannel = BitsPerChannelType.EIGHT; + // 8 and 16 bits per channel are kept (16-bit becomes a 16-bit + // YAPF texture); 32-bit documents are saved as 16-bit. + if (copy.bitsPerChannel === BitsPerChannelType.THIRTYTWO) + copy.bitsPerChannel = BitsPerChannelType.SIXTEEN; var opts = new PNGSaveOptions(); opts.interlaced = false; copy.saveAs(tmp, opts, true, Extension.LOWERCASE); @@ -35,6 +37,6 @@ if (mips) args.push("--mips"); var ok = yapfRun(args, out); tmp.remove(); - if (!ok) { alert("Could not write " + out.name + ".\n\n" + YAPF_TOOL_HELP); return; } + if (!ok) { alert("Could not write " + out.name + ".\n\n" + yapfFailText()); return; } alert("Saved " + out.fsName + " (" + Math.round(out.length / 1024) + " KB)"); })(); diff --git a/extensions/photoshop/yapf_common.jsxinc b/extensions/photoshop/yapf_common.jsxinc index 2a77128..10e312e 100644 --- a/extensions/photoshop/yapf_common.jsxinc +++ b/extensions/photoshop/yapf_common.jsxinc @@ -20,16 +20,52 @@ function yapfTool() { return names[0]; // rely on the PATH } -// Runs the tool; returns true when `expectOut` was produced. -function yapfRun(args, expectOut) { +// Runs the tool with its output (stdout) going to `stdoutFile` when given. +// Returns the tool's error text ("" when it printed none). +function yapfExec(args, stdoutFile) { + var errFile = yapfTempFile(".txt"); var cmd = yapfQuote(yapfTool()); for (var i = 0; i < args.length; i++) cmd += " " + (args[i].charAt(0) === "-" ? args[i] : yapfQuote(args[i])); + if (stdoutFile) cmd += " > " + yapfQuote(stdoutFile.fsName); + cmd += " 2> " + yapfQuote(errFile.fsName); if (YAPF_IS_WINDOWS) cmd = '"' + cmd + '"'; // cmd.exe strips one pair of quotes - if (expectOut.exists) expectOut.remove(); app.system(cmd); + var text = yapfReadText(errFile); + errFile.remove(); + return text; +} + +function yapfReadText(f) { + if (!f.exists || !f.open("r")) return ""; + var t = f.read(); + f.close(); + return t.replace(/\s+$/, ""); +} + +// Runs the tool; returns true when `expectOut` was produced. The tool's +// own message (why it failed) is left in yapfLastError. +var yapfLastError = ""; +function yapfRun(args, expectOut) { + if (expectOut.exists) expectOut.remove(); + yapfLastError = yapfExec(args, null); return expectOut.exists; } +// The tool's message, or the help text when the tool could not be run. +function yapfFailText() { + return /yapf: /.test(yapfLastError) ? yapfLastError : YAPF_TOOL_HELP; +} + +// True when a .yapf file holds float / HDR data (half, float, BC6H), so it +// should open as a 32-bit document instead of a clamped 8-bit one. +function yapfIsHdr(file) { + var info = yapfTempFile(".txt"); + yapfExec(["info", file.fsName], info); + var text = yapfReadText(info); + info.remove(); + return /payload\s*:[^\n]*(half float|, float|BC6H)/.test(text); +} + function yapfTempFile(ext) { return new File(Folder.temp.fsName + "/yapf_" + new Date().getTime() + "_" + Math.floor(Math.random() * 1e6) + ext); diff --git a/tests/test_yapf.c b/tests/test_yapf.c index 000a457..c2952b2 100644 --- a/tests/test_yapf.c +++ b/tests/test_yapf.c @@ -632,6 +632,35 @@ static void v2_mips(void) { yapf_tex_free(t); } +static void v2_preview(void) { + uint8_t out[4 * 4 * 4]; + /* gray + alpha u16 */ + yapf_texture_t d = desc(4, 4, 2, YAPF_SAMPLE_U16); + yapf_texture_t *t = yapf_tex_create(&d); + uint16_t *p = (uint16_t *)t->data[0]; + for (int i = 0; i < 16; i++) { p[2 * i] = 65535; p[2 * i + 1] = 32896; } + int ok = yapf_tex_to_rgba8(t, 0, 0, out) == YAPF_OK && + out[0] == 255 && out[1] == 255 && out[2] == 255 && out[3] == 128; + yapf_tex_free(t); + /* linear half 0.2140 → sRGB 128, 2.0 clamps, alpha stays linear */ + d = desc(4, 4, 4, YAPF_SAMPLE_F16); + t = yapf_tex_create(&d); + p = (uint16_t *)t->data[0]; + for (int i = 0; i < 16; i++) { + p[4 * i] = yapf_float_to_half(0.2158605f); p[4 * i + 1] = yapf_float_to_half(2.0f); + p[4 * i + 2] = yapf_float_to_half(-1.0f); p[4 * i + 3] = yapf_float_to_half(0.5f); + } + ok = ok && yapf_tex_to_rgba8(t, 0, 0, out) == YAPF_OK && + out[0] == 128 && out[1] == 255 && out[2] == 0 && out[3] == 128; + /* BC7 and BC6H go through the block decoders */ + yapf_texture_t *c = NULL; + ok = ok && yapf_tex_compress(t, YAPF_GPU_BC6H, &c) == YAPF_OK && + yapf_tex_to_rgba8(c, 0, 0, out) == YAPF_OK && out[3] == 255 && out[1] == 255; + yapf_tex_free(c); + yapf_tex_free(t); + CHECK(ok, "yapf_tex_to_rgba8 converts every payload for display"); +} + static void v2_misc(void) { int ok = 1; for (uint32_t h = 0; h < 0x10000 && ok; h++) { @@ -697,6 +726,7 @@ int main(int argc, char **argv) { v2_gpu(); v2_mips(); v2_misc(); + v2_preview(); printf("%d tests, %d failures\n", tests, fails); return fails != 0; diff --git a/tools/yapf_cli.c b/tools/yapf_cli.c index af60926..b18330b 100644 --- a/tools/yapf_cli.c +++ b/tools/yapf_cli.c @@ -167,6 +167,8 @@ static int parse_map(const char *s, uint8_t *map) { return 1; } +static const char *err_name(int e); + /* ── Reading any input into a texture ────────────────────────────────── */ static yapf_texture_t *new_tex(uint32_t w, uint32_t h, int ch, int sample) { @@ -182,7 +184,7 @@ static yapf_texture_t *read_input(const char *path, const opts_t *o, int *from_y if (is_yapf(path)) { int err = 0; yapf_texture_t *t = yapf_tex_load(path, 0, 0, &err); - if (!t) fprintf(stderr, "yapf: cannot read %s (error %d)\n", path, err); + if (!t) fprintf(stderr, "yapf: cannot read %s: %s\n", path, err_name(err)); *from_yapf = 1; return t; } @@ -399,6 +401,58 @@ static uint8_t *to8(const float *f, size_t n) { return o; } +/* 16-bit PNG (stb_image_write only writes 8 bits): rows of big-endian + * samples, filter 0, compressed with stb's zlib. */ +static uint32_t png_crc(const uint8_t *p, size_t n, uint32_t c) { + c = ~c; + for (size_t i = 0; i < n; i++) { + c ^= p[i]; + for (int k = 0; k < 8; k++) c = (c >> 1) ^ (0xEDB88320u & (0u - (c & 1))); + } + return ~c; +} + +static void png_chunk(FILE *f, const char *type, const uint8_t *d, uint32_t n) { + uint8_t b[4] = { (uint8_t)(n >> 24), (uint8_t)(n >> 16), (uint8_t)(n >> 8), (uint8_t)n }; + fwrite(b, 1, 4, f); + fwrite(type, 1, 4, f); + if (n) fwrite(d, 1, n, f); + uint32_t c = png_crc(d, n, png_crc((const uint8_t *)type, 4, 0)); + uint8_t cb[4] = { (uint8_t)(c >> 24), (uint8_t)(c >> 16), (uint8_t)(c >> 8), (uint8_t)c }; + fwrite(cb, 1, 4, f); +} + +static int write_png16(const char *path, const uint16_t *px, int w, int h, int ch) { + size_t row = (size_t)w * ch * 2 + 1; + uint8_t *raw = (uint8_t *)malloc(row * h); + if (!raw) return 0; + for (int y = 0; y < h; y++) { + uint8_t *r = raw + row * y; + r[0] = 0; + for (int i = 0; i < w * ch; i++) { + uint16_t v = px[(size_t)y * w * ch + i]; + r[1 + 2 * i] = (uint8_t)(v >> 8); + r[2 + 2 * i] = (uint8_t)v; + } + } + int zlen = 0; + uint8_t *z = stbi_zlib_compress(raw, (int)(row * h), &zlen, 8); + free(raw); + FILE *f = z ? fopen(path, "wb") : NULL; + if (!f) { STBIW_FREE(z); return 0; } + static const uint8_t sig[8] = { 0x89, 'P', 'N', 'G', 13, 10, 26, 10 }; + static const uint8_t ctype[5] = { 0, 0, 4, 2, 6 }; + uint8_t ihdr[13] = { (uint8_t)(w >> 24), (uint8_t)(w >> 16), (uint8_t)(w >> 8), (uint8_t)w, + (uint8_t)(h >> 24), (uint8_t)(h >> 16), (uint8_t)(h >> 8), (uint8_t)h, + 16, ctype[ch], 0, 0, 0 }; + fwrite(sig, 1, 8, f); + png_chunk(f, "IHDR", ihdr, 13); + png_chunk(f, "IDAT", z, (uint32_t)zlen); + png_chunk(f, "IEND", NULL, 0); + STBIW_FREE(z); + return fclose(f) == 0; +} + static int write_image(const char *path, const void *px, int w, int h, int ch, int hdr) { if (ends_with(path, ".hdr")) { if (!hdr) { @@ -445,9 +499,19 @@ static int export_image(const yapf_texture_t *t, const char *in, const char *out : "no such slice, or out of memory"); return 1; } - if (t->sample == YAPF_SAMPLE_U16 && !ends_with(out, ".hdr")) - fprintf(stderr, "yapf: note: 16-bit samples written as 8-bit\n"); - int ok = write_image(out, px, (int)w, (int)h, ch, hdr); + int ok; + if (t->payload == YAPF_PAYLOAD_PIXELS && t->sample == YAPF_SAMPLE_U16 && ends_with(out, ".png")) { + /* keep all 16 bits */ + uint32_t m = s % t->mip_levels, d; + yapf_tex_mip_size(t, m, NULL, NULL, &d); + const uint16_t *src = (const uint16_t *)((const uint8_t *)t->data[s] + + yapf_tex_surface_bytes(t, m) / d * (uint32_t)o->slice); + ok = write_png16(out, src, (int)w, (int)h, t->channels); + } else { + if (t->sample == YAPF_SAMPLE_U16 && !ends_with(out, ".hdr")) + fprintf(stderr, "yapf: note: 16-bit samples written as 8-bit (use .png to keep them)\n"); + ok = write_image(out, px, (int)w, (int)h, ch, hdr); + } free(px); if (!ok) { fprintf(stderr, "yapf: cannot write %s\n", out); return 1; } return 0; diff --git a/yapf.c b/yapf.c index 5f49a19..281977e 100644 --- a/yapf.c +++ b/yapf.c @@ -4677,3 +4677,71 @@ int yapf_tex_build_mips(yapf_texture_t *tex, uint32_t levels, uint8_t alpha_ref) else tex->flags &= (uint8_t)~YAPF_FLAG_COVERAGE; return YAPF_OK; } + +/* ======================================================================== + * Previews + * ======================================================================== */ + +/* Linear 0..1 → sRGB code, exact to the 8-bit rounding (thresholds). */ +static uint8_t yapf__lin_to_code(const yapf__srgb_tab *t, float v) { + if (!(v > 0.0f)) return 0; /* also NaN */ + if (v >= 1.0f) return 255; + return (uint8_t)(yapf__lin_to_srgb(t, v) * 255.0f + 0.5f); +} + +int yapf_tex_to_rgba8(const yapf_texture_t *tex, uint32_t surface, uint32_t slice, uint8_t *out) { + uint32_t S; + if (!out || yapf__tex_check(tex, &S) != 0 || surface >= S || !tex->data || !tex->data[surface]) + return YAPF_ERR_INVALID; + uint32_t m = surface % tex->mip_levels, w, h, d; + yapf_tex_mip_size(tex, m, &w, &h, &d); + if (slice >= d) return YAPF_ERR_INVALID; + size_t n = (size_t)w * h; + const uint8_t *src = (const uint8_t *)tex->data[surface] + yapf_tex_surface_bytes(tex, m) / d * slice; + + int ch = tex->channels, sample = tex->sample; + uint16_t *half = NULL; + if (tex->payload == YAPF_PAYLOAD_BLOCKS) { + if (tex->format != YAPF_GPU_BC6H && tex->format != YAPF_GPU_BC6H_SF) { + int rc = yapf_blocks_decode(tex->format, src, w, h, out); + if (rc == YAPF_OK && tex->channels == 3) /* BC1 / BC7 without alpha */ + for (size_t i = 0; i < n; i++) out[i * 4 + 3] = 255; + return rc; + } + half = (uint16_t *)malloc(n * 3 * sizeof(uint16_t)); + if (!half) return YAPF_ERR_OOM; + yapf_blocks_decode(tex->format, src, w, h, half); + src = (const uint8_t *)half; + ch = 3; + sample = YAPF_SAMPLE_F16; + } + yapf__srgb_tab *st = NULL; + if (sample == YAPF_SAMPLE_F16 || sample == YAPF_SAMPLE_F32) { + st = (yapf__srgb_tab *)malloc(sizeof(yapf__srgb_tab)); + if (!st) { free(half); return YAPF_ERR_OOM; } + yapf__srgb_init(st); + } + size_t sb = yapf__sbytes(sample); + for (size_t i = 0; i < n; i++) { + uint8_t v[4] = { 0, 0, 0, 255 }; + for (int c = 0; c < ch; c++) { + uint32_t x = yapf__get_sample(src + (i * ch + c) * sb, sample); + int alpha = (ch == 2 && c == 1) || (ch == 4 && c == 3); + uint8_t b; + if (sample == YAPF_SAMPLE_U8) b = (uint8_t)x; + else if (sample == YAPF_SAMPLE_U16) b = (uint8_t)((x + 128) / 257); + else { + float f; + if (sample == YAPF_SAMPLE_F16) f = yapf_half_to_float((uint16_t)x); + else memcpy(&f, &x, 4); + b = alpha ? (uint8_t)yapf__clampi(f * 255.0f, 0, 255) : yapf__lin_to_code(st, f); + } + v[ch <= 2 ? (c == 0 ? 0 : 3) : c] = b; + } + if (ch <= 2) v[1] = v[2] = v[0]; + memcpy(out + i * 4, v, 4); + } + free(st); + free(half); + return YAPF_OK; +} diff --git a/yapf.h b/yapf.h index 5f57933..fdca43b 100644 --- a/yapf.h +++ b/yapf.h @@ -601,6 +601,16 @@ int yapf_tex_compress(const yapf_texture_t *src, uint8_t format, int yapf_blocks_decode(uint8_t format, const void *blocks, uint32_t w, uint32_t h, void *out); +/* + * yapf_tex_to_rgba8 — one slice of a decoded surface as 8-bit RGBA for + * display (thumbnails, previews): gray is expanded, 16-bit is rounded to + * 8 bits, float and half are clamped to 0..1 and sRGB encoded (they are + * linear), GPU blocks are decoded. out holds w × h × 4 bytes of that + * mip. Returns YAPF_ERR_UNSUPPORTED for ETC2 / EAC / ASTC blocks. + */ +int yapf_tex_to_rgba8(const yapf_texture_t *tex, uint32_t surface, uint32_t slice, + uint8_t *out); + /* yapf-sum, the checksum used by version 2 files. */ uint32_t yapf_checksum(const void *data, size_t size); From fdbd0a6b8f4f47cb0ea6b728c2f5ea41aaa54450 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 11 Oct 2026 09:43:38 +0000 Subject: [PATCH 5/7] Python binding and Blender read YAPF 2 yapf.load_texture / decode_texture expose every surface and the texture description; rgba_float() for display (blocks decoded, 16-bit / float kept). Blender imports 16-bit / HDR as float images packed as EXR, decodes GPU blocks, follows the normal-map convention (DirectX flip, Z rebuilt for XY normals). Errors say why. Tested in Blender 4.4.3. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DVrVthWN8GsmgHGUhhsCYm --- bindings/python/README.md | 14 ++ bindings/python/gradient.yapf | Bin 0 -> 3140 bytes bindings/python/yapf.py | 181 +++++++++++++++++++++- extensions/blender/README.md | 8 + extensions/blender/yapf_image/__init__.py | 55 +++++-- 5 files changed, 242 insertions(+), 16 deletions(-) create mode 100644 bindings/python/gradient.yapf diff --git a/bindings/python/README.md b/bindings/python/README.md index 33e52c8..1a47c7a 100644 --- a/bindings/python/README.md +++ b/bindings/python/README.md @@ -23,3 +23,17 @@ Without the library it falls back to a pure-Python decoder that needs tells you which one is in use. Example: `YAPF_LIBRARY=../../build/libyapf.so python3 example.py ../../other/YAPF.YAPF` + +## Textures (YAPF 2) + +```python +tex = yapf.load_texture("albedo.yapf") # any v1 / v2 file, needs the C library +tex.payload, tex.format, tex.sample, tex.channels, tex.mip_levels +tex.shape, tex.layers, tex.frames, tex.channel_map, tex.normal, tex.notes +raw = tex.surface(frame=0, layer=0, face=0, mip=0) # bytes as stored (GPU blocks / samples) +rgba = tex.rgba_float() # numpy float32 (h, w, 4) for display +``` + +`yapf.load` / `yapf.decode` still return 8-bit images; on a texture they +cannot represent they raise `YapfError` saying so. Errors name the +reason (damaged, truncated, not YAPF, newer version). diff --git a/bindings/python/gradient.yapf b/bindings/python/gradient.yapf new file mode 100644 index 0000000000000000000000000000000000000000..d6db15b0ad6b6d0b9f2e10c24685c3b32de793ae GIT binary patch literal 3140 zcma!a3~*y)W@ThzU|?u~Vi_QVbddloP;Dky z4I&HxsBz`z;EAYf*I=QEB0ExMR@{s-1Y zATXNYD9@~eJi~G9WExNiOKJm~^Wdek#sIgx{zDsCC@n9gfoym&F~FK$7CRT*{s(T% Puz bytes + tex = yapf.load_texture("albedo.yapf") # any v1 / v2 file (C library) + tex.payload, tex.format, tex.sample # GPU blocks, 16-bit, float, ... + raw = tex.surface(mip=0) # bytes as stored (blocks or samples) + rgba = tex.rgba_float() # numpy float32 (h, w, 4) for display + Uses the C library through ctypes when one can be found (fast, can encode); otherwise falls back to a pure-Python decoder that needs numpy (decode only). The library is looked up in this order: @@ -26,12 +31,22 @@ import platform import struct import sys -__all__ = ["Image", "load", "save", "decode", "encode", "backend", +__all__ = ["Image", "Texture", "load", "save", "decode", "encode", "backend", + "load_texture", "decode_texture", "GPU_RGBA8", "GPU_RGB8", "GPU_RG8", "GPU_R8", "GPU_SRGB8_A8", "GPU_SRGB8", "FLAG_PREMULT_ALPHA", "FLAG_SRGB", "YapfError"] GPU_RGBA8, GPU_RGB8, GPU_RG8, GPU_R8, GPU_SRGB8_A8, GPU_SRGB8 = range(6) -FLAG_PREMULT_ALPHA, FLAG_SRGB = 1, 2 +GPU_BC1, GPU_BC2, GPU_BC3, GPU_BC4, GPU_BC4_SNORM, GPU_BC5, GPU_BC5_SNORM = range(0x40, 0x47) +GPU_BC6H, GPU_BC6H_SF, GPU_BC7 = 0x47, 0x48, 0x49 +FLAG_PREMULT_ALPHA, FLAG_SRGB, FLAG_COVERAGE = 1, 2, 4 +PAYLOAD_PIXELS, PAYLOAD_BLOCKS = 0, 1 +SAMPLE_U8, SAMPLE_U16, SAMPLE_F16, SAMPLE_F32 = 0, 1, 2, 3 +SHAPE_2D, SHAPE_CUBE, SHAPE_VOLUME = 0, 1, 2 + +_ERRORS = {-1: "invalid argument", -2: "cannot read the file", -3: "out of memory", + -4: "corrupt or not a YAPF file", -5: "needs a feature this library does not have", + -6: "damaged: a checksum does not match", -7: "the file is truncated"} class YapfError(Exception): @@ -70,6 +85,26 @@ class _CImage(ctypes.Structure): ("mips", ctypes.POINTER(ctypes.POINTER(ctypes.c_uint8)))] +class _CNote(ctypes.Structure): + _fields_ = [("tag", ctypes.c_char * 4), ("size", ctypes.c_uint32), + ("data", ctypes.POINTER(ctypes.c_uint8))] + + +class _CTexture(ctypes.Structure): + _fields_ = [("width", ctypes.c_uint32), ("height", ctypes.c_uint32), ("depth", ctypes.c_uint32), + ("layers", ctypes.c_uint16), ("frames", ctypes.c_uint16), + ("shape", ctypes.c_uint8), ("mip_levels", ctypes.c_uint8), + ("payload", ctypes.c_uint8), ("format", ctypes.c_uint8), + ("sample", ctypes.c_uint8), ("channels", ctypes.c_uint8), ("flags", ctypes.c_uint8), + ("channel_map", ctypes.c_uint8 * 4), ("linear_mask", ctypes.c_uint8), + ("normal", ctypes.c_uint8), ("primaries", ctypes.c_uint8), ("transfer", ctypes.c_uint8), + ("mip_filter", ctypes.c_uint8), ("alpha_ref", ctypes.c_uint8), + ("loop_count", ctypes.c_uint16), + ("frame_us", ctypes.POINTER(ctypes.c_uint32)), + ("note_count", ctypes.c_uint32), ("notes", ctypes.POINTER(_CNote)), + ("data", ctypes.POINTER(ctypes.c_void_p)), ("internal_", ctypes.c_void_p)] + + def _platform_dir(): machine = platform.machine().lower() arch = "arm64" if machine in ("arm64", "aarch64") else "x64" @@ -97,6 +132,20 @@ def _find_library(): lib.yapf_encode.argtypes = [ctypes.POINTER(_CImage), ctypes.POINTER(ctypes.c_void_p), ctypes.POINTER(ctypes.c_size_t)] lib.yapf_free_buffer.argtypes = [ctypes.c_void_p] + try: # version 2 (texture) API + T = ctypes.POINTER(_CTexture) + lib.yapf_tex_load_memory.restype = T + lib.yapf_tex_load_memory.argtypes = [ctypes.c_char_p, ctypes.c_size_t, ctypes.c_int, + ctypes.c_int, ctypes.POINTER(ctypes.c_int)] + lib.yapf_tex_free.argtypes = [T] + lib.yapf_tex_surface_bytes.restype = ctypes.c_size_t + lib.yapf_tex_surface_bytes.argtypes = [T, ctypes.c_uint32] + lib.yapf_tex_to_rgba8.argtypes = [T, ctypes.c_uint32, ctypes.c_uint32, ctypes.c_void_p] + lib.yapf_blocks_decode.argtypes = [ctypes.c_uint8, ctypes.c_void_p, ctypes.c_uint32, + ctypes.c_uint32, ctypes.c_void_p] + lib.has_textures = True + except AttributeError: # an older library: v1 only + lib.has_textures = False return lib except (OSError, AttributeError): continue @@ -115,7 +164,7 @@ def _c_decode(data): data = bytes(data) p = _lib.yapf_load_memory_mt(data, len(data), 0) if not p: - raise YapfError("not a valid YAPF v1 file") + raise YapfError(_why(data)) try: c = p.contents mips = [] @@ -157,6 +206,9 @@ def _py_decode(data): b = np.frombuffer(bytes(data), dtype=np.uint8) if len(b) < 20 or bytes(b[:4]) != b"YAPF": raise YapfError("not a YAPF file") + if b[4] == 2: + raise YapfError("this is a YAPF 2 file; reading it needs the YAPF native library, " + "which was not found (the pure-Python decoder reads version 1 only)") if b[4] != 1: raise YapfError("unsupported YAPF version %d" % b[4]) ch, gpu, mip_levels, flags = int(b[5]), int(b[6]), int(b[7]), int(b[8]) @@ -275,6 +327,127 @@ def _py_tile(np, b, p, end, tw, th, ch, bad): return np.stack(out, axis=-1).astype(np.uint8) +# ── textures (version 2, C library) ────────────────────────────────── + +def _why(data): + """Why `decode` could not read data, in words.""" + if len(data) < 5 or data[:4] != b"YAPF": + return "not a YAPF file" + if data[4] not in (1, 2): + return "YAPF format version %d is newer than this library" % data[4] + if _lib and getattr(_lib, "has_textures", False): + err = ctypes.c_int(0) + p = _lib.yapf_tex_load_memory(data, len(data), 0, 1, ctypes.byref(err)) + if p: + _lib.yapf_tex_free(p) + return ("this is a YAPF 2 texture (GPU blocks, 16-bit / float or a volume) " + "that yapf.decode cannot return as 8-bit pixels; use yapf.load_texture") + return _ERRORS.get(err.value, "error %d" % err.value) + if data[4] == 2: + return "YAPF 2 files need the YAPF native library (not found)" + return "corrupt YAPF file" + + +class Texture: + """A decoded YAPF texture (version 1 or 2). Description fields mirror + yapf_texture_t; `surfaces[s]` holds the bytes of surface s as stored + (GPU blocks, or samples in native byte order), numbered + s = ((frame * layers + layer) * faces + face) * mip_levels + mip.""" + + def __init__(self, c, lib): + for f, _ in _CTexture._fields_: + if f not in ("frame_us", "notes", "data", "internal_", "channel_map", "note_count"): + setattr(self, f, getattr(c, f)) + self.channel_map = list(c.channel_map) + self.faces = 6 if c.shape == SHAPE_CUBE else 1 + self.frame_us = [c.frame_us[i] for i in range(c.frames)] if c.frame_us else [] + self.notes = [(c.notes[i].tag.decode("latin-1"), ctypes.string_at(c.notes[i].data, c.notes[i].size)) + for i in range(c.note_count)] + count = c.frames * c.layers * self.faces * c.mip_levels + self.surfaces = [ctypes.string_at(c.data[s], lib.yapf_tex_surface_bytes(ctypes.byref(c), s % c.mip_levels)) + for s in range(count)] + + def surface_index(self, frame=0, layer=0, face=0, mip=0): + return ((frame * self.layers + layer) * self.faces + face) * self.mip_levels + mip + + def surface(self, frame=0, layer=0, face=0, mip=0): + return self.surfaces[self.surface_index(frame, layer, face, mip)] + + def mip_size(self, mip=0): + d = max(1, self.depth >> mip) if self.shape == SHAPE_VOLUME else 1 + return max(1, self.width >> mip), max(1, self.height >> mip), d + + @property + def is_hdr(self): + """Float / half samples or BC6H blocks: values may go above 1.""" + if self.payload == PAYLOAD_BLOCKS: + return self.format in (GPU_BC6H, GPU_BC6H_SF) + return self.sample in (SAMPLE_F16, SAMPLE_F32) + + def rgba_float(self, frame=0, layer=0, face=0, mip=0, slice=0): + """One image as numpy float32 (height, width, 4), rows top to bottom: + 8 / 16-bit scaled to 0..1 (still in their stored colour space), + floats as stored, GPU blocks decoded. Needs numpy.""" + import numpy as np + w, h, d = self.mip_size(mip) + raw = self.surface(frame, layer, face, mip) + raw = raw[len(raw) // d * slice:len(raw) // d * (slice + 1)] + out = np.ones((h, w, 4), dtype=np.float32) + if self.payload == PAYLOAD_BLOCKS: + hdr = self.format in (GPU_BC6H, GPU_BC6H_SF) + buf = ctypes.create_string_buffer(w * h * (6 if hdr else 4)) + rc = _lib.yapf_blocks_decode(self.format, raw, w, h, buf) + if rc != 0: + raise YapfError("GPU block format 0x%02x cannot be decoded here" % self.format) + if hdr: + out[..., :3] = np.frombuffer(buf.raw, dtype=np.float16).reshape(h, w, 3) + else: + out[...] = np.frombuffer(buf.raw, dtype=np.uint8).reshape(h, w, 4) / 255.0 + if self.channels == 3: + out[..., 3] = 1.0 + return out + dt = {SAMPLE_U8: np.uint8, SAMPLE_U16: np.uint16, SAMPLE_F16: np.float16, SAMPLE_F32: np.float32}[self.sample] + ch = self.channels + src = np.frombuffer(raw, dtype=dt).reshape(h, w, ch).astype(np.float32) + if self.sample == SAMPLE_U8: + src /= 255.0 + elif self.sample == SAMPLE_U16: + src /= 65535.0 + out[..., :3] = src[..., :3] if ch >= 3 else src[..., :1] + if ch in (2, 4): + out[..., 3] = src[..., ch - 1] + return out + + def __repr__(self): + w, h, d = self.mip_size(0) + return "" % ( + w, h, "x%d" % d if d > 1 else "", self.mip_levels, + "blocks 0x%02x" % self.format if self.payload == PAYLOAD_BLOCKS + else "%d ch %s" % (self.channels, ["u8", "u16", "f16", "f32"][self.sample])) + + +def decode_texture(data, verify=True): + """Decode any YAPF file (v1 or v2) into a Texture. Needs the C library.""" + if not (_lib and getattr(_lib, "has_textures", False)): + raise YapfError("YAPF textures need the YAPF native library (version 2), which was not found") + data = bytes(data) + err = ctypes.c_int(0) + p = _lib.yapf_tex_load_memory(data, len(data), 0 if verify else 1, 0, ctypes.byref(err)) + if not p: + raise YapfError(_why(data) if len(data) < 5 or data[:4] != b"YAPF" + else _ERRORS.get(err.value, "error %d" % err.value)) + try: + return Texture(p.contents, _lib) + finally: + _lib.yapf_tex_free(p) + + +def load_texture(path, verify=True): + """Read and decode any .yapf file into a Texture.""" + with open(path, "rb") as f: + return decode_texture(f.read(), verify) + + # ── public API ──────────────────────────────────────────────────────── def decode(data): diff --git a/extensions/blender/README.md b/extensions/blender/README.md index 046d5af..0a990bd 100644 --- a/extensions/blender/README.md +++ b/extensions/blender/README.md @@ -12,6 +12,14 @@ Blender 5.1). Blender cannot read `.yapf` on its own, so imported images are **packed into the .blend** and survive saving and reopening. +Every YAPF file opens, version 1 and 2: 8-bit images; GPU blocks (BC1–BC7, +BC6H) decoded; 16-bit, half and float textures as **float images** (HDR +values above 1 are kept, packed as EXR); for cubes, arrays and animations +the first image. Normal maps follow Blender's OpenGL convention: DirectX +normals get their green flipped, and X/Y-only normals (such as BC5) get Z +rebuilt. Version 2 files need the bundled native library; the pure-Python +fallback reads version 1 only. Tested in Blender 4.4.3 (nixpkgs). + ## Install 1. Get `yapf_image-blender.zip` from the diff --git a/extensions/blender/yapf_image/__init__.py b/extensions/blender/yapf_image/__init__.py index 9732222..596220b 100644 --- a/extensions/blender/yapf_image/__init__.py +++ b/extensions/blender/yapf_image/__init__.py @@ -33,30 +33,61 @@ from . import yapf # ── conversion helpers ──────────────────────────────────────────────── -def load_image(filepath): - """Read a .yapf file into a packed bpy.types.Image.""" - img = yapf.load(filepath) +def _set_colorspace(bimg, names): + for n in names: # names differ between versions + try: + bimg.colorspace_settings.name = n + return + except TypeError: + continue + + +def _read(filepath): + """(rgba float32 h×w×4, has_alpha, flags, float_data, texture or None).""" + if yapf.backend() == "c" and getattr(yapf._lib, "has_textures", False): + tex = yapf.load_texture(filepath) # every v1 / v2 file + rgba = tex.rgba_float() # frame 0, layer 0, face +X, mip 0 + if tex.normal & 3 == 2: # DirectX (-Y) → Blender's OpenGL (+Y) + rgba[..., 1] = 1.0 - rgba[..., 1] + if tex.normal & 4 or (tex.normal and tex.channels == 2): # XY: rebuild Z + x, y = rgba[..., 0] * 2.0 - 1.0, rgba[..., 1] * 2.0 - 1.0 + rgba[..., 2] = (np.sqrt(np.clip(1.0 - x * x - y * y, 0.0, 1.0)) + 1.0) * 0.5 + alpha = tex.channels in (2, 4) and not tex.normal + wide = tex.is_hdr or (tex.payload == yapf.PAYLOAD_PIXELS and tex.sample == yapf.SAMPLE_U16) + return rgba, alpha, tex.flags, wide, tex + img = yapf.load(filepath) # pure-Python decoder: v1 only w, h, ch = img.width, img.height, img.channels src = np.frombuffer(img.pixels, dtype=np.uint8).reshape(h, w, ch).astype(np.float32) / 255.0 rgba = np.ones((h, w, 4), dtype=np.float32) - if ch >= 3: - rgba[..., :3] = src[..., :3] - else: - rgba[..., :3] = src[..., :1] + rgba[..., :3] = src[..., :3] if ch >= 3 else src[..., :1] if ch in (2, 4): rgba[..., 3] = src[..., ch - 1] + return rgba, ch in (2, 4), img.flags, False, None + + +def load_image(filepath): + """Read a .yapf file into a packed bpy.types.Image. 16-bit, half, + float and BC6H textures become float images (packed as EXR so nothing + is lost); GPU blocks are decoded.""" + rgba, alpha, flags, wide, tex = _read(filepath) + h, w = rgba.shape[:2] name = os.path.basename(filepath) - bimg = bpy.data.images.new(name, w, h, alpha=ch in (2, 4)) - bimg.colorspace_settings.name = "sRGB" if img.flags & yapf.FLAG_SRGB else "Non-Color" - if img.flags & yapf.FLAG_PREMULT_ALPHA: + bimg = bpy.data.images.new(name, w, h, alpha=alpha, float_buffer=wide) + if tex is not None and tex.is_hdr: + _set_colorspace(bimg, ("Linear Rec.709", "Linear")) + elif flags & yapf.FLAG_SRGB: + _set_colorspace(bimg, ("sRGB",)) + else: + _set_colorspace(bimg, ("Non-Color",)) + if flags & yapf.FLAG_PREMULT_ALPHA: bimg.alpha_mode = "PREMUL" bimg.pixels.foreach_set(rgba[::-1].ravel()) # Blender rows go bottom-up - bimg.file_format = "PNG" + bimg.file_format = "OPEN_EXR" if wide else "PNG" bimg.pack() # keep it inside the .blend bimg.use_fake_user = True # …even before anything uses it bimg["yapf_source"] = filepath - bimg["yapf_alpha"] = ch in (2, 4) + bimg["yapf_alpha"] = alpha return bimg From 0ba1f0af176e1485484ab2db69b3928622328139 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 11 Oct 2026 09:51:48 +0000 Subject: [PATCH 6/7] JavaScript port decodes YAPF 2; VS Code and Figma open every file yapf.js: decodeTexture (container, yapf-sum checks, code books with context decoding, 16-bit / float planes, GPU block tiles), BC1-BC7 / BC6H block decoding and textureToRGBA for display. Matches the C library on 417 generated files (surfaces bit-exact) and survives fuzzing with clean YapfErrors; fixtures in bindings/js/test-data keep it checked. VS Code preview: pickers for mip, face, layer, frame and slice; storage, normal-map convention and version in the info bar. Figma import: textures converted to 8-bit, 8-bit images stay exact. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01DVrVthWN8GsmgHGUhhsCYm --- bindings/js/README.md | 21 +- bindings/js/test-data/v2-animation-notes.yapf | Bin 0 -> 13362 bytes bindings/js/test-data/v2-bc3-packed.yapf | Bin 0 -> 5102 bytes bindings/js/test-data/v2-bc5-packed.yapf | Bin 0 -> 5563 bytes bindings/js/test-data/v2-bc6h-encoded.yapf | Bin 0 -> 1692 bytes .../js/test-data/v2-bc6h-random-blocks.yapf | Bin 0 -> 4716 bytes bindings/js/test-data/v2-bc7-encoded.yapf | Bin 0 -> 5620 bytes .../js/test-data/v2-bc7-random-blocks.yapf | Bin 0 -> 4716 bytes bindings/js/test-data/v2-cube-mips.yapf | Bin 0 -> 14818 bytes bindings/js/test-data/v2-expected.json | 137 ++++ .../js/test-data/v2-float-gray-entropy.yapf | Bin 0 -> 11260 bytes .../test-data/v2-half-gray-alpha-entropy.yapf | Bin 0 -> 5339 bytes bindings/js/test-data/v2-rgba8-entropy.yapf | Bin 0 -> 1765 bytes bindings/js/test-data/v2-rgba8.yapf | Bin 0 -> 97 bytes .../js/test-data/v2-u16-rgba-entropy.yapf | Bin 0 -> 3672 bytes bindings/js/test-data/v2-u16-volume.yapf | Bin 0 -> 5736 bytes bindings/js/test.js | 34 + bindings/js/yapf.js | 727 ++++++++++++++++- extensions/figma/README.md | 6 + extensions/figma/src/ui.src.html | 16 +- extensions/figma/ui.html | 743 +++++++++++++++++- extensions/vscode/README.md | 15 +- extensions/vscode/media/preview.js | 118 ++- 23 files changed, 1751 insertions(+), 66 deletions(-) create mode 100644 bindings/js/test-data/v2-animation-notes.yapf create mode 100644 bindings/js/test-data/v2-bc3-packed.yapf create mode 100644 bindings/js/test-data/v2-bc5-packed.yapf create mode 100644 bindings/js/test-data/v2-bc6h-encoded.yapf create mode 100644 bindings/js/test-data/v2-bc6h-random-blocks.yapf create mode 100644 bindings/js/test-data/v2-bc7-encoded.yapf create mode 100644 bindings/js/test-data/v2-bc7-random-blocks.yapf create mode 100644 bindings/js/test-data/v2-cube-mips.yapf create mode 100644 bindings/js/test-data/v2-expected.json create mode 100644 bindings/js/test-data/v2-float-gray-entropy.yapf create mode 100644 bindings/js/test-data/v2-half-gray-alpha-entropy.yapf create mode 100644 bindings/js/test-data/v2-rgba8-entropy.yapf create mode 100644 bindings/js/test-data/v2-rgba8.yapf create mode 100644 bindings/js/test-data/v2-u16-rgba-entropy.yapf create mode 100644 bindings/js/test-data/v2-u16-volume.yapf diff --git a/bindings/js/README.md b/bindings/js/README.md index 309c0c7..a121a1c 100644 --- a/bindings/js/README.md +++ b/bindings/js/README.md @@ -1,6 +1,6 @@ # yapf.js — YAPF in JavaScript -Encoder and decoder for YAPF v1 in one dependency-free file. Works in +Encoder (version 1) and decoder (versions 1 and 2) in one dependency-free file. Works in browsers, Node.js, web workers, Figma plugins and VS Code webviews, and produces exactly the same bytes as the C library. @@ -22,3 +22,22 @@ image decodes in about 150 ms in Node / Chrome (the C library: 6 ms). TypeScript types are in `yapf.d.ts`. Example: `node example.mjs ../../other/YAPF.YAPF` · tests: `node test.js`. + +## Textures (YAPF 2) + +```js +const tex = YAPF.decodeTexture(bytes); // any v1 / v2 file +// tex: { width, height, depth, layers, frames, faces, shape, mipLevels, +// payload, format, sample, channels, flags, channelMap, normal, +// primaries, transfer, frameUs, notes, surfaces: [...] } +const s = YAPF.surfaceIndex(tex, frame, layer, face, mip); +const view = YAPF.textureToRGBA(tex, s); // { width, height, rgba } for a canvas +``` + +`surfaces[s]` is a `Uint8Array` (8-bit pixels or GPU blocks, ready for +WebGL / WebGPU compressed textures), `Uint16Array` (16-bit, half) or +`Float32Array`. Checksums are verified (`{ verify: false }` skips them); +errors are `YAPF.YapfError` with a `code` (`corrupt`, `checksum`, +`truncated`, `not-yapf`, `version`, `unsupported`). `decode()` also reads +8-bit version 2 files. The decoder is checked against the C library on the +files in `test-data/`. diff --git a/bindings/js/test-data/v2-animation-notes.yapf b/bindings/js/test-data/v2-animation-notes.yapf new file mode 100644 index 0000000000000000000000000000000000000000..a058ba0b98bae59261660ed45e9d200acd0b4137 GIT binary patch literal 13362 zcmY*=4RjmVmEN7l0W>p$ls7|=0x8LPGZ;_lmao{9GjL?*_)n{H0?-^lS;Ck#>t*kvc0=b zn(j~9?b+@*o9$zq_OwHg!o2za`R@I`?_M3rl$5Hd zN>Wv2jcmDDA+n;b5xKe}snmCNsjp&fr@AJ`mFj`zxDwx)*7h&&j7!!1+V{JLTvKHj=ys2!p89gO!yx05Z-Ff0O2}8`LCY|!eht&=Vy9=E`vbv z-~Lq*Tw3@V<{~YGcUQf(=QC=-ZjT(v$WII%au7tP%|rPZ=0gG>qref`62c?fN-c_| zj`Uy7=Q)B(lYH(r)?3;TfwIpva(*w!4|DvDrj}PZ3MP_?+~2qH2Q%oFtY2ehS9 z2`D#CJ$}!)1l$4^bJ(;@UP8uFRKhKVXv5R0(Hd^`gQ0R1P;CJaU4%TTsGMFBz%bPs z)m{W$E)~gw3988zbior0#A#ww0Fxs%At+@f+J;s}#C@Wa2!OuWE+Yr4 zo;L@CJqsQnsMk;pQK?RH?0St_s#Xh=W3&vAT>=DDAyQr-fmWL%#6=pVRbqKd4HF53 z2ZzHKK`7NzIYhCvq|thC3xs|%3xkSYQ8=^V1$n{mfF+?|U|%wj2bNGuNwuX?JMqBh znq|4UsdZ2MLo1y<-;;bbo=)!FuARMeuhy-_&!)BbzGOVUH=T^{y)&JzZVbfts_}R? z`zmSYm*esC>12Di_A(n!r`g8w^quLu(($&m)?GiFuE)FYQo8F+zXraE!4%zyK>&Vb zq}-?PZZTcuZm(W#o))q(Cw_0{`kW(1JCwR0cFV?x+uNFAAw5xSQcq17emHsTyFpu2 zjMpRHLvin~<#+hIzp-W(;ZnKok97Xs9XMRk3YK7t!|{%cP0K%$dTvfCK|6evYo(4wnj|(dzoC5~fWN7+*;zGAqO&d>L%8(#UeS7SEIoV{?;` zm4X5Nk0|yFFJ0#9wbPwN9;f0ig=^Q%p!CK^7K&3pc>D>G8O@VlWt z^26qn=rrf8kTz>%^>E>~&T~(b%z}{hgsVn-b7pQ0eq4@LTr9DhY}NOxbd{P8Eo}a1 z$5Y$#7-9AJuYE!F5-xTkglZacVo8w{*a@#-0 zZ6Y)OUVT@ZNNMBA96aC+XFLPG)#_U~6vBNEG`Eh7B%UdT!z`4Xt9PMkGb%htrwA7{ zQ7xzm8_%7i5%+R032|$#CoJkAtOT`7W+G|)#M!NwP;v@du9Z@=|Hr;Kn?+JIrz3X7 zSiSkLF0F=(wt*-8VXpn@gQ=F%gf8M|$`lJ2u6o>EC7%UdQ6g`qzQ?wkN(68)p@?XdYUiRcqt}!-8?FY{tyDLWgAvnk+Clw-NyyHUF%L; zaR$NkZ>D;DZ+vex&H%ZA)%gBpHeT)i%_4@&1IcuAdO6;GXIy=at(Z=-#TyuXS?gBV zHV2e$CB2($AUnP{nNFt|dgEu=ZrbYI+CKy0*atnmwd2}cnBKPh;OkePTXR=;5U;X^ ziuBU1$W3$VbWMuXz-*>1Q&4lV?dV+71B?B4q%I2aN1M|fTe#B#xV!MyA%Xk&gL-Ev z^{ns^ZeH29Msa{19O=n8=9W}UEXu?4B7Zzm-)FrhOmTQE)}?{DLm$8RXFtc_A5vMk z0>2*G@ruY#z#12J>k`fJt*0Ypep5C#-FdYC!)U1D$6ReeS%O$#%pG3|Pd%fE$mJgE zViBjK&S$K#(~amRL_1;AL6FsFBofRVX}N4|dYjLkocUs-$*`eX#HX~z4n<6rs6pTJ zA31SsSr|V&X1?fmuFkY{?Y{J*N_`reg~N-3Q|IB$+=8b5{XNdmTQ@ntVjv_hMSxUJ z46Y>GPiBS4^FLbAJ4V5qrseF328u3AnUj$jTkC7_)e*rhl-_+%Z-?{#w7PV!vQUQ! zAnOgEo8`L>S(+>%ZC7|Uv(C**vFuA937wH57(mUk$%F63>yF(p=T>&hGs%^wtwn!} zyK6c|&)~wJd?nlwD4~B$GWrpai&8AJL)ikpLZSZLk%KyWvx0$Y7DSYNaku#(z7g`S zo5qFl)`<_wW)*kV(EXI^eEYdr#W)7CiHCBD^3V+a&eu_!Q~eCOPq7#7c!z)M`E`b4!h}0-RE&y;%Xxb zwR9n|u4V9EQusG>xv$o){4<&{s{F`0z%G2d&L3(IK#sd{cs%mL!_OXi(!~|UG?$te`$h`WMCRkihxJ*oQkBog~tEpbF z#=<-bMvn|Eo^Fh2OM-pZo}kGl0&$8D{vWcIwvTnF=`L+Y{j=5XE`@>afV#|(cA(nL z{z<0m%`Ca4+3d^gTXK14x1#M&v-u>0D+5tFrS4_;WMd3Ee{>-G7$Mw0ogBT*|woH0Xt_qqSH?VdSvy$teu`Hr9G9&2vX=V7~n%p9NnaaonXf* zWi26N{G@}!4^DT79w~q-yS16kTvCnZ>EG2TZFX*9TfeWj*+H(;X^w9G!c}2J7;tz) z2~>IlPW9h|6DApMiLYn!@bKvtUhFX-R5)je-1zU4xrTBK8~n&JSvywyh6gngL{>GV zo4LfMDGZ&tD)v_N-xXWFjP-nO<>LEhxNR8o#pf1LNpY>A3ho^Gd~{0TQy>P6#FZyN zR|9jtrHwbsmHTc7M!5{CfM;M%9+niD#zM+@0u7tux2ief9yB1^{& zbM!`dd^WV3d>$>Rd}`Dz6~>MBu+?e z7(or|k-8&D*R;yHMTH}Eb+>;gq|VM+1%icw5I5hC!U7|3q9t%tu~IoBa^72)UpeG6vZjHnvE0BIhxxji@`PHyH60+Nm12s*DNUB{xPGQD14Swc z^Cwz!Tsdr27fjKhG6WiLl1Xl|vSRBX&S|IwcMD`sJuvIzFkQyjoDNDvaZRB#m=r{v zH<6TsY@J2UtK1QNtX7)Gw5D*btz{Z2q{@bKeBAUGXPpd~`6w^fi4j%tLXb#7tu83z z5ziRK8QJC09*LIiBvNHT=ZPkWsts&uB4x8MF@ ze=_k(?Ba&PVY!+elfYknIl315GY&db@cHQ2f81JjY{DYu$iIU!GBYFZJtlz4e>rr> zy4(W}!5@(w3KtHQdC={IEWa0$xWV=J8P3`%-{AYOkPH1X|7jzUYvH9F`j>}J5<;Uq zkwY!=Hr^5t4OtbFX_*CzA)0)E@@6kr5k;Um6+WMFa-o*0R^tTVhGj5$uV5O`OIU=i z^p*f}cG)q}4C@}v1;f5&asr(K0X2nclM?g_1l~}A@|g2t0^*#^dl{+bGkGnRG&GIY3~*w-xhMeEkskOGMjl@q z*8&SCTL>}(5FnJ%Jof$;BB)g^r5fy<+EAf}Fr`fUO(s3H4|7socj`n5XTn7B7@a3{ zoIp6^FO+Sdq%3g2Qc-S!_|QQVc^vTZg#zNUh!Zb1vf7-F4JkF>N*IYUpb-UOjxLO| z_diD{O|^@{Ni9GyDqQmFr6>=Y5QT6Q*GWSQ2-E>lmh}vYUm`39@w~}^Y&$8GB3w(1 zE)hY4B_$-LJj6xi=4flvp4WR`wt9YE-LEihv!78Q7U{d|-Bo=faKBvbW!);rbiJ2wLU1~cnof8zT$zN_)H(%o}S>t;i7HU4X~L8O4YxzZ&p z?t@Lha#TAU3r~0E`kpM8@c-J-yy0zF9zaor?=Izc*P=5+58N;P= z=lY`nD?54t-Cg*_Q!tz{7OqHZ#CI?)X`szR54ONEx5aNlqga8)nCIYTz%iVXITB}?anYc&DZ=e=2}CzwK->L zV|4dpmZ(uaQ5I$V?4L>UuH@vT*;M+^LFI3xM9z7F3qSdbCl3Ci+{eX+aCQ1}8yDxf zrr>ZHZBeSVgSJ^&6h^2}KBcTj7NXbPdKWMH);W9)^ik zuw|2sqOs)4& zno?RjZh1TEgXcyI=9pqQ50!dJ>$qTZ3uI-W{y107aHt!#8e&wkv?KvwRtByWb5x98 z0pZ|@+lzjuQ4zkj4?6*AzN;J~K}QDAs7%5a+mC*22s_SKdtH?MZ+F(7+cHwkCWx>= zj#ym3+tn!NAg*K1RBMDF1_>Ge@+doVbn@pCU#1uBm!9WW7*v&@!OH(ZL>7&_DGE$MXqEF)`- zhG}t?(XYKM$)uZ?yEnkQ?>G7=zHgb0?Po0RtoHqQH~MGzMDBp{bLqcY9=9lN?uhdp z=frz7;XFNfMh33wfeNNA4e#scl9i#mqv@mEjwVtSg)ZmZD~8cF7@nBji`S$R$BTDF zun}0vYf~bR*Uhl9oZSKxZ>9Cs?cbi6orD_^yJgYldQhxf()?i(nW~A>wyoc$hWF+^ zAsnG*^69<{;YpiO+{I4cN^57 zk<4G636T<~id(}d#0_Sk&FG(q2t;b(uLE-xjQ8Q0&7^CL8pd$M6(B2HM)N&^(Y5HD zxEBc_`%FX$RnC@Zjd(EKSR-Rsx2%4E3uHVYnMx8zxl-*)^W z7>Qf0PNhy`b)xg9?jAon+b=FZK{LnUIjxmt(PrLN9+R9%`tBt@lPxcVCzctNzxW{@#8rQ_~`ER?sQ4Xq+8{e0ONJc`gy`ph@M zg3iA2%|WAAs71w9gp0{XhP6j0(+W`ffWs6cIxI|!_Nl?Y2%H^PtZ{poQ#a_$y#LUs z{KBm#y=MlIm4|1et>=!2haR;#rWnM6(7XHT%!&1Kr~FZ`P)JV}`eVt^y%H3!?7kNG zY9jNW!9OwFrVh;!S28VC;JDG>k~VHb-MGpMCMy;@xZUoALeC4R7JK z`1TFYfFW!@`y^S0tZw)X40+4(y-NM;@>%(1=JZt$u;GpT$BYEFk7^T3^G4Ago184Y z_4Y4uHhJNd`3onW+f-UKfaSwst`Z%yqa#UFI73}j&J2aZhZ+yx5pYt$iqvH6y@sCWL#&!{Af|v3>k3s1yfLtS>ojlFR z=wCPjI{>UmKJFprUBOQx&hNUWXNNhGK#SGYXnqlse8W75-oMTQ7v47U`%&=&YwqfkdGhZ3B7QmsMCVD3-H3PhR9VPzL` zM39AxrCQ%o0}*-NVkUE;MY#ZcmZUTzszj0|IQ@`PRR>Jd3#c)~+wAd&Clcz&c>7D8 zvE!4VieXc3f^SvI<$lXXurd7)rf;^i{rTI!g$mMG^@f#fsxncxt1{C#N^+;#mPE=; zCT_M?<#bj3+)dKPYFAZ`$FIfr*V*SleEC3qzrs8=k@3NXe&VOW(p!K1!%UKu zZ+;_gJ}KYm?HvX4sfQ0W86d zh}~HfGA^

>&dJpK%x=RRiL9QI1;dnSqWKN(C_eaBWFKi-0-WzsPQ`RSP&$L$uAf zP_Kh6)@lL~+_@3B#F)NT6L3_J4}wrx0-bPPqXYy4c;kfG!Wp4JC#!m~EQqN_O_*iX z9CwZ@n#`6|0kj}!)EXYa3j~loO-m^)NHt;*<~5fJ{Tf9ANG2-|VU8;knMo&X3UwcJ z3Y;9>CO0+j*$C9M`nGbsr1VFEa(RL8{m)%`DC{)l+9fj3BWO z(>;tTu^*;s*t%6#;OR;&Gb!|Xm94We7AGv&zgpi2{9PMkO8uWv6T2J04So=EgcGI} z#5?X99YpBaF{cn#Ey?}NrFSD_sO-!z(ayQ?=AKV+mt*k?E1vjYjG4QJp87_2{f_U+ z@u7b1z6e^;(tFb8o)b5tpMq~L@4NK z>-Mas-81T<$1>~v5LjqHk8nN{o=ta`z2M!8f(fgi3UyrHp&|MMSVL8i?yuX=m4(Mo z%$7~rriCFIBO;k7CHQ)XgeGDq7y_Prh)28Ww`nn}YeBP*58d^FB-tjqhn&21d*m#$ zv&wrfU%ek^o%^zNOAl7w*vw-0o=7MiN*=mcsrq+I{X&X+kk0%fLAA={anO&5&5~9v zTwM=-lA9wPojp~7Qra%b<-JhvDuqpoKfj&eF_ zmj&faXj^M;fUo0G7*e+#NzA0R&Lb=aTrw3{|2uLGbhye>k1=f4B&g2EZWa}==d_*gzYoA zx8Aiu;Mk6|?Hgd;%St*MKnJFAn8-m~dEr@(rTa`RI{;>-{`RB1Q3*qx%IWaLJ0eQZ z)Ooljv8u~oDEAKS!SNo>Y{aEiguYNTXBGZRe{XLBx zyCb{VVvGtBUDHnuKqy3;8sPXr!|c5!wr=0jAGKoY$o#tVv?#x5`&>FgLjCb?S_^BbuZQ=I>Jc(VUlS9QN*38Nh;ntb8Q zoizm(FG)e=ql}6YPHWJfunQr(-#_lgY71|g{+02hS>K&WUl2z=bL&T3WQN|KU}uzD z|4eq_u6Ep@ld;N@@ix|>He8w=YP7=cK5|DdcXj;g)|<$`F)qr5C$aD@DI^F-bx+b# zBje*R{M4Dm9a_ED-@#N&{j0ZPrFRZ#ax^uYu4Q#w(FnSI@Q@rmbHWhc3zdnmScezXeD6j*6DJgg!Fm6mmV(|Nl^ z8zBFr6j$N-*zxb-SvfTH zP;}CgK{>9VIj5oRWK7VfWM?7zd%V=JIOBh=_pc7_`ogIv`^H!94~(n4wL0~UNo*yg z+YZhDfQpI!{2FK`awto8t}U;FJ-@b_9O^r)BfkQaxNAdSdnDwm)RiW_eS}YiwgsGO zMip|OnAgaGM%FdK7jW>S2W}%yM>Q`~cx(=<1Z|ZVEb&(0xWB2FYjm2Wom!!pRrgJq zpZ@wD$S5gp`M)6KdObzLT!1|u55xA)PV15^fmWpIK$Q4ZV;Vt&7uF`YyPDRXv9d<1 zFog}!&~{U0jF8(xey-gilES3G>K$HmI6`6mp4^!0IsDcWL4=gKGX+V-ysy|1<64xW zRi&h`MjT8HTZA6IV&{|^R)}oGlpK8oQ|$1a$SgSYVwJBsq>EjO1?!qRO37gtw5ZvS zN{*dDckqkw#4PW%17n6XCx(E-VHC~RYk?p{$FHeHTGCT4p1?%|=B3&(4imbWe#Lf(XKOa|+cd$e@?Dy)9yF;vegp@sn`it8U&MI}_2DGX3$kSCOx&oUWoVZ~ zSto)b@-ict2*zOCK{iP0P=lAG2(Js4$kzns6)39VOlFYQ&3Ie2QpXHHn%ah{pHu28GtXa*zeXhWEbCrl*{a^fAXV>18_?OC zL>nop%Ibl--8UJqhPEGOjN~h`%qRFOmxuy8i5L{@8IxE3*Y8x(L(JH+cTV?;ze4Du zZ2gJ(1PGC+(1Ne;h-PrX{_~E^3_JJeryu+^uaXWBZL4z1DR?z3{`s^V>wwtA=l`CQ z*h5l~N#KdhjZA}*MLle>CXitqgG_-Ra3_u&;{<`)8rPcqi$-MxG1AKCc{0??Tzidp z6joa9_1FU+%d;N4&>sXM#5`e?v(=m;P@#aC)=B1ice*Gu3YH1OJZs%r97QHfA{6Z# z`hAs!;Eow(^9l$|_0<;6b=fo=V>@&?Qxx((<>~l5C!!cufy#=&AlVY~E>du7y+Wh ztj!9JWmCdJ3{a%!!36>inXFL?0zur7=t2-w1+ykNx`T~qzyuFr_r)Qk;Wey=FwEXk zj@7%^PC0^8o;kE2U8KkpDuUhz&;SEc4c4q7aUIhX%HWom`r-&*V;i|ODlj??1`*&; zP=Elgk}O3Wv(LRH6wOu*9bDNeGp8c})oauM5wC#^ksY#b5>7)*`zah?*CEg$*gGQFp1H@&8)n3by@>l? z*Y>ma7v*L4UNQrVxm?}p-K@lO=iYd8_wIE3ovz*c89&>wl^GA)AZbhuF;7`}hiRuO zt84xfqjwx|{MW~x>k0pNqApM3!mHlgoe!#9=kN>n4V;@i_A$}OT=WB0k&B;}54NA~ zqJH|;qAsaZC(LjAFDSi(lR^6jUM&5B%s(;x#2tH7n``=JoWqX7e@QGT!9>3bxzXM8 z-ClFYGe13PUKC$Hfrf7e$=wYiaHrbWGw%e#ZQX2M`@u?VW-hihk*RuHLQb(A7qaCh zCn$y&_^!v@a!MqB3?E;fvXYs>=( z2qo_>J)Rb&^g!BlTc%Y$neB|-P@K5Q&+EksTKh{!U7OZ(A5O097`X>-#!|=~u#DY# zZGLpBqfq-DElOAgQ0DK;e$yAMo1yPKP2GoW&R^q>U&~o~U+YS7G3<~;_(jfFtX6n1 z*Vi@7s^Q+y>8tS8J8PuPd-bih~X`xbG%C2NGIwNO6#W?{xKp49JOJlXeXxmQ+`{5{!G-v}r@We5UJ$?)u> zd~@aYmtPddX5Op@R4s%S6taMy zSR3XC-)Sl9oW(UpXCJTE)m`wj2TGNn^i5u>F&RbXeIpjDsC(5b{Fu9$%C4Rre8Pg1u z)Mk!Tz%e)uW}{UW;LKV;`Nr5+9zijKLu0u7$D?KEQP{li(u;a+>}9($?>O_DjE_qU zjjY#nn__I35BBRl$q%(=HQvQE3`-On(CYrgKTb1pm1Hh*H_I8!e9%;eIac6d$lTCG z>8|*O5zVs6Z?ug@euLU7E9ht&6(R;uw%tBfvt?hqSak^f6Ngwl04UH{?1Ad$_V2Cl z>KlkoxQh1dN58keSEl!=Pq3zp+b9vinLHdH3^ED2SI%3Pw*A2XQH_?6#dWzCP+~Nk z4t@8jBLA21p^OmzZ`=7mC<@1#QJ*fpNjr<*T`9-d zCB?!o&r@nQMx$no!xe z<}ucb;sC$7r4o7&a!?D9wf#l@P85ewWn!phQ7FU$++wy{rfSZ% z?SZEMtAskf&tUV@_y_EN8FW`ax8XeOWpGKS)z505WvJU%XBI+teS3FxBSY?s@7ujM z&FVd@3ajqhSS`8sa=NzU=_T9lySVgQGy9dfH%_!~@|e`gU|3l68X-evO+^ASNXT5` zii6zYRT$0ZMuKO&nx91)g+U`_6?Enx`$C{;`3q%}iOf(?noZsBE|7A2h&h}T8zj;1 zUYeLNy^x1*D35T%2L*m>($s^#-afnVVE$NhkOEG>qc7a|%1WkX#%#c1&i-^EVm)7T z5mzYtbjx~EqDk&GHy@7*mrj2rbZSm8`9RB4{@#&qhCdyEF7?W;e^5v)L~BObNE069 zp`7l>zmqlE`5r6MfD!W+`voA+qt-H7;q?5aq8ZKc z8kObTdZjJK^>j=;c^I7&B|yL1?D2kT#@oTOKB6D9UU)ta8YY$a^np{EOAgOVKbkXl zt8=hNsMiP8DV`4wA1N~h&YhgD8SP4_UA|zt+nuRno|fZ>S832_*F?C7vXTvAo-<>+ zC~Jm$$0X>27-EH*!+zh%zdTf7UDq0`h89S)4+-`|I3MF3lrw2j2wm5-z@Vmy6XD>c zD?DEc3*%6jgoScpoYj&oLbC;@fI>WSh=_`gTwNxt(q_8BXpKuaGf4!~sTy<7L^%$1 zwBYsGA1&|++Xu!^g%QOJOde?jyS~Js1>dX!nZ`8~ts0)J$-piXOy+9!~X_9RbYJp literal 0 HcmV?d00001 diff --git a/bindings/js/test-data/v2-bc3-packed.yapf b/bindings/js/test-data/v2-bc3-packed.yapf new file mode 100644 index 0000000000000000000000000000000000000000..3ea3f1d1db813bbe0c37e3535d628b986aec0445 GIT binary patch literal 5102 zcmZu!2|UzW`#<(UgjOM9vP80#ov~(%v5YNhXch`hV(3zSStmqEF(pM&wz98Di*mDv zLbgzp&`yQxJ-^|;xA%YlPd)o{&iS6_JoEc}xEbNi**T5b02=`A002G6(1gqexS{^P zG`nskB4hDas(|g`a{OvRCK~6~__2eKAIjbu$>}j)Sm5ps~75ER9B?5m4U;5NQY}7$9Tf;IYoJfD#&u9)Lmj zVl7!c7H3=1-3(t+->c7MZH(=pfjgVjiekT=lntvla29Mzd4xBK=0hxVO zt3Zu1Zk1^$DD>%g_`oDQ13~RoLYyGvxW7H9&EPWRHy_7^|GmS2hJyT^NUUc46EO!a z|2WXX^AP!g6^ie7jC-Ju`;A$R<6Z-nIUzVqG|a3L>^KrihQGXj5ggn9K?+^eN1upA z2$U$p`PCq5qQ7=P!&$(e@-r~i3~ z)gUh|7IB-7jhxji-&r)V2FXbjO78M(f(Z#nH>3RMnDd2PNl&S~jz zK0xPG>X&;`jM75sq4Iha1ylPJ62YX_E6gY(tu#A-i~FJOV^Q)0PMEBUq6!`EK_Qs>lre-!xvr!!D*Mz55Pt9lJ(f3bvRwf%b3Xk`1OSDeGI=e zHDL}j``vAbObfjp*BGCF_;P@8^s~Nkuk6E@Dd{v)f3%vxv!Edrn{{q|$6!Slw33vj zt5ZL#Tog(YDlHy&*d=6tNrvRsC#(FOU6|Z0`m|5BF{gEty>vqJwnhoZr_~G*X4>wY zRJD;wuhc)i@4Ko_@MI)dW;ADeXzZLcj=rGPmir{!?I_)6Y@A|w;<$%2{nia5rOS!3 z%Dt@yM`O5^XuYKWZVzQ`$J+h+pOy&M&Px+u3{(y4*1=bCrME#mS*_R4hzS=3D>m;Q2`;@!Df@G&mw@aFfgQbsZIS2sJD=|+ljPMl#r zKsY9%Ds)@AbGC?c-1t>=e9z+TsiYTeGL6msDc-akxmwwq_qJ_V)Ailm+V727RCL1{ zZ9=F_-XmwJtOlFjG;ChpZL1@#EoL|uc{yE^S0be;=Tjcn4-3xk-Xg}x?nx=<_7(Lo zc~vsFM@O@0=%O&&wcg`PB^RqG?iJ(*nYPpS^(@FT%^PT=3d@W2an4c2*v7PuU!8p~ z%-wi5DhWN9e475}pJ`OVdp-|!w^ClhN`|Rv=BE3@5){Wewe$LN_REG_h3&iuK04RG ziI3gfEB`(wx>F^{J%h@XLi}RkD-yw7U|@akd3vHPUx#((4$pbfdhCgFPVb&OzTi9+ zD*welb3RBU_C>}+^)XI458q+NSv7p#)h@;S54GKcIHJCjm9GguNtP@uf3jjkG)F_- zz~y;u=k)anb&1m1eYhRR7AHN<%2w%!;le)-w$B>gucI8&BrJM=i+C%fMB{c|s+_wu zaHsCZP*u~eD{WS)U%OsM$rXKM*yf*;obO^@^E_BH_~YKT(U~i^&L`8cl{2S*p14a* z+AmSgNcIfGpZ%;Kaim(+SIp^5c|V=n^>V+d`3JnfSAXI;LklfRq}y$gOlFKx&9S~q zU-|{k8)43F_@gRE=L%?_-Z#;xtSi}A;99V7Z8qE5x|k%u-XdV;Y?@K^=(*2fx#Wxc zH$N<^vrg>wk~2y?HmUL2@wDB6?;*mxiKahQD!zJ^>}o)yNfkv8wtZ~&x7+xx?Ql=f zS>MEgZ6l`(lo75Yhp36Q%8xb`ot3+OgvuvF@`-db7Iw$(Lq4~#93mj~O0*!6Y#@9*qAR=-Ee;Ed9#2FI;DwwUr*Q|3$0Ij{GhtUeQq z&Hu2kGovZ_Jg+|UQF)S7)J7VwzMEyck$?X<$wVaWDPZR4KMFPMvPceBceqL1ezdXN zf=?T$b#&DoT5{`a_qH`Q+urK4Tljsy5%;}oqV?xm9o`Ll4tHKq>+xh3-gI)zyI523 zOTa4cri0>f$tjTIZk5+sU!a4_OLbt@tR*yMKgv;4427Mek94_SU0pqevyaIaD{=Jp{x|Rc;1N4-eGqdnd7<=E=!@|4(I9VMcBq*{0gip1= z-Gdk4@QjLv=~m7k+vR)G>zHq4rKP={VMPNDRyyxaXcaKcQ9Z7ipr~2-GTkjqGojU# z;ym+9O;^3al~56~yG;GKj5FaR)6I0kywIDnaia^=(q6(;5~^kHw6+eOaW#vzPQecB z5wc7n4dA3LQ~bp&uuz(aK!FA-{W+e=^+Rb%>aS!=>4Da{o0a}+&V?8G11Zb>V!)z{ z4iK!bE{9M`xIdK}_F;h8pHW5GtB1+r#|9c0&g1$#FlR~_L*i>g_Tsb zQc^Fnp57%JcH7aV_!_&r=pLc5Ize$8X=B9;jx+5kp>kEcBI#yt-U^jaO#=nLj|?wN zRz0HzV$s7stO@9*^IFUOzQ z8F>1}Vot=G<3UoNWNi{}7mPgjN@;(eQXLboG;+|=MK)3=*Z0>L@wAn~Qw{x*1!C6v zJ$m~R9FBZ*pq)Nw*X7;ETbiKjtb?1F&}u0S?bG0!iL}>!#-8&ozm8+1I@0dTT}&D{ zx87{4)pJW_r|X-)YQ2bg6Zm-f?B*Ng+@KKqoJVu=d~1PgCo$weJT~G}m_wVb=z}A9 zB|;T9Hyr5wPF_xviaT3!NAm8twXwQiV#BppnuJslg`TQ?AHb&3?xJtVb}5OWV_yRl z$9rwd8?_8f_Z91%j2~if0L%WLg15jsdH-h4m|BnRqSuKRF5MK!Tu<=j02v~B4inwG zM~vCKrUi4$M-gi*p~(g`=V5O~32=MW=7o%}&$&`7c1wO>&yZHriT>%eP1s!nr*v}7 zzQ63ScHUF>Idn&JrJ>TKf4$g5cZJ{l_?Hs}BeCzp>#CNjJ!X>w(mM_&Pc8)2o-&NM zmE5v4`O&bgrFN#n_rMGPh-dME6*G5BZ#xf-?x3~=hluCC4XoEaxt=^UH#y+<+;44K znA_30Tch^72mV_8S*SVWTitN^PQ!5KM|y<#*1Ol1j#WpTT$WI-np~KV(7D>v;+gZo z!x4Lsw@pR5Y{N#fqsx=@0;`<9+2vOXHbPe`2z9q{B{1(fMG_j-w;_ zh3wP&euiv265v|P-y7WN_Bu)i)2P~Z$>Gzt4llmJRIpQdRPMcXV>R)_I#Br?5hcoTLQlgxB1(P_OJP zS_+r9L>8~jvW{7u$D^o^U9PEW#uu8oAE@x^O*E^e7eg2^Rn&7 zS~IswS0fWb7E`Y$@!Q2$D)RSYV>|HzO1DWjEUkdg!IJawag|B^5%ytUwW#^ZIfX}R zrjBx7`?Qd@i?`9QAu-x%m(1gB-V47hRW`Hf8}n6IoOq$0eR1=5*%+rz-Mqr%Ov>OJ zX}RF@x!F%5741E8BRrJfXs6d#?O?8Xr~T7_Ur9bG^UDco#}lEH)UhWA_SwBx9kAn; zY~e_=4)2=k?zxj`eU+9GFIm4Y!o8m8z2#oED%4VsSX$7$^31H~tD(B`x)9&_+1}aVdiwnE zk@?wrXjO&H4@16x^yBct{kr-4A0r?ykGPV5P!C-ky>^xmeyTYBhli_KxN@P|;L$?q z<3ohlj$jwjzwSqvhTmB@H+ny*iFR=Q^X9(}G>wl;)+RShdVC07H0*Al&biuety|aI zK7By9ENrHAwn9CumpN3|K0S1$zdlU8Zhm-rxLy+iXb7ACD*^&o)+ksj{>|Mb$47qB zGJeyY2dx_{`bJL$`QK43Jf5uC?)bRcdTSdWf3I&I?fa{D-8&UyUA_13uLCb##Ol@7Tcuaj()O+Pb^Y9-* z{gQjN&3}|=_SX1E=hX~dl^L4TdnqH{Usm>jslP^NcIe&GeZ7v^7VQgG?bD;e8-l+C g&MCx225k%|s$i<|&r)REPMhyi*cAw-r^y;79yl%-8b zQ8JcFDG`Z~lI-7|skit0zB9i&XSwH|d(Qu!^LUs$?cwLQ=7abk$P0@z&16apUn2v`CIOT|;E1R4P>B7sR{ zvIqn^jmbi@*laQzLinKl7%Tzn23(0)Ch#W!UjmUxqfv<@B8?zKWYg(vHUeDvp=B}% zjwAoh4Tm5QNC3^XnL!W}aD*%Ck)2?`ARGb=fSVyWAv53yL>_@NLdYK(2SG6J3W2)L$mP!2aqF+4l0160ICS6nFbL7 zkX42#7~1P^9mfu!b8iEL{;h!utbe zR~&)J9T2=57!U%O4JwvMB+#fdDv8OR78;lj9D-&;Xhawc2mwqIa03hcQwT&VfkdT} zh*T0)h>E5nG&Y(I&_q^1-?)w9j{F9mE8`#-b^_3w{b_n`mVXE2Pgm)I#{zKk&;{F| z;o;xV0T~B?2_niPoz;gQYSZ09pbkJ7FN@yNz@ra5h(e{$LZ}6Kyy*voeAT7{7 z;4u)Gdo>Z-!`{nnGtiH`NPqk&6kLE4;lIfeqJqc$Plc#71VhGy03fqJlL43r)Js8w z#~=_<2$mGbZFLqn_kWP*Y6K9<8X~YjKoS5Lg7(BgR4UIODBMF38V^y?1fIfuPtZOz z3OC+g0_C8|qRRbx1}6;okbx^~8BwpaSqRZYUt!3sm412@%3mXn zvc-xS3)E$V3czT=3Plh^R(Ml4O<)RmIjqH zomg0z5r8q&KD5iX#7@Qovt?~|y2;Ffp4}FQ9#M|U&z#x*nDR#XY}r!H$i?+`9XS#f zgsKK_Q+IU_UuV0u&C{~xclz>bryg7wjXr}fiBHki9hCz`OS$O1ENC<4gOC)4%)Y3| z=XW5;Lbzp?rw`0+ zz{+46tqgB`(p$%{PxzKV5~>UtZaMf^Q`w56Ss5}EA*q^qWp!FsB!xrnrsTiti zFkZ*7&C2-smbzvr<{q4i@jaaR%=^W_AxYJnSKoGDYQC$)*7MD`_E@WUtK-FVaiEb- zsB~}1ds^4RWsmu6U;n^iA;XH1uru18@s?e9?^N0I!by1VYnGL3YOd}7=ptzMrTvQ5 zwc>dA+*FLCo3)R;&xRw#Z`7mYaF2+iNKdrT(aBV~&W8g4(DFDS8M%^(HGe|0pS+M4FZxz_Kw&*Adjk$5vJdryXwYkSa> z4X%;{{5FDF1Np@*ZFIFOA71gf7h)Gx&sffsa0+72*DZy>J*{ZLRndinik8EgV?Hz1 z4ad#if43ApwaaQ6wb93VRUD$h-t<*v_Ku>+;PM#9CRg6&NmH)o9)-82%7hC(J1@e1 ziP|oenpNgh38k~by7p2ROAZ#OtgRwH6t^lV9Q%1z`Qvj-({BD3ZTzX4A1^vTblotn zn68KoldDc~-`nM3Wf60{Um`~>y*p)fyi{*zjEnGf8!gHCV+2Q9^V5&&zP96gzBe&- z?oD=_472Y%c%}(%YL46>^YNXhktxCQ(;)p-@ZFl&l+i|$-osHH!xJmCTHNacs++a! zkAGScugfnvA6~l&%+j+Z0*H^xB=h^9;K8`bIiW28dDLEReQ6*f77AWNf~*JJCT};cczFKj+zHb=6;G zO6RG;71ow#_n3PX7%W7D?`LK26FIp_w#VjlHjX@K^4;o|Na6_^|6TEjjF`2k2|Q_| zrKjH2!Nh8eM#Ew1Mp*6Rqtu16RrlkIPONsdc;L4CoLl6EH(HP;=J-x?ZNqDaRP2*2 zRy-eYzY>zC0;SmnoK>;(3_b!rX_@;PH(U7oTqss$^uu|2t&iO`MWt8OWZ4&-OOLsB z`M{0!%B#<|xmSv6+)=xvbvj%9nc%=RA4IC4AuFl5N-DMM_CPjX8+tC9-ozI~wl zxO(7pQCM1^!Xe41oXhJ>UsopGkaI4cSKk?iH{Dj@CpqRaehE8)POZQ9Ei&!IOwISm zk9QV#PS2SHef5%E%{!L)}R890!-Ql`>Nk#9YX2$9ReQtje-4@+) z$<*#;g(Ln!M|Dx{{Dyy?7mRrZi$BMjseYm=%T+{dI&$vqk>y>w(Ww`67V>`N>XK9X zTes|6Io{s;wtd`x=W9k^nflK0(xk2%VG%7dk4xY;(Hq#p>u+ZEmKQT@*6l${-4BUZ zHx#&I@N|3PFiP@?ll7~zcgX{EUx||c>MDb3NmxKT)6sG*;-RO6%Xgjh5r;--`JLto)HD57e&QR@$7t!-A>5r6}|3oL?(8IH5F9GJEHeG2v#N3L{y8Qor82 z#7I}mPLooPTi2+q^|X*`VhTUfa7_1b9v_v3peq_Jn;zv`o{|URdLYVviv=Re8EX{a ze!KY)q%33Q@iMAGZJqt)_3W#+NN{%ID9c;I+S%oR!?sY1Q*8Sb6|GGcuPS^@6I`vq zN0U<;D6Pwm&M(YjMum7XgtToV8Oe#F9JNlO{`+x}1W`@|JnjCt{A!u@v#Qghvwn}u z>w?x;uI2N9^qfk z+!tVv7a4B!PdYasE&XMir`xdIO7&;S3=vt!HTsE|eb;PWPGzP4vcWJzuS-2-?67{c z!k||3O2o#&nIiL9^G~8osTITgfx_8RE9SJFJz_EnY|5i_rmp77#_1MrQBOiOr}w!2 z)XJ=r(ss|w36_=2&(Y~&hWb&gs*lZ)FGy<|dYN^~g@)44Wm_43Z~goMnWd!LwF#Qx zTKmp#9m+gyf3th6?0sKeT}g>Hdd_0Tcuj~@s{BfOp)Y2|r#xPJWS&$x!q`~zt0eO^ zyXulr`eZq=q;|1`pcl`|dm=b#phzjdy?8P@_PhPgec0Z9SK`v>md{Chb|DWeI!f7D z(r$M+fhTbF5;<326bx>&OS5eCR*|K;^nA5_u6HLR|CoK?{<$l14=;WaTWNG*wvw?q zO59xerA&o&!>;vpJ=zX4UfQM^j+PjaQw@$0uDDY3+=9=ZcsIL~VV8bcKT`4A<5${H z&d^BE+*Wq` zgL~W1P(i|FMAD*Ift~ zbx|i~R%%qlCT~3HwPo2cY_CMJN)%?Qjq3LyxNG{2uX+0`p{~GEXtTZ)_O?q1sfw@t zSo(RC*ts40-OvrG$Gf|160C(wG$TEOGHqXf81#EFKUlGFDqknFzq=znt^3UlHyI37fwwpI|6sWbS+$?6cvO21W zV|M?wceaJcOaUGp(1cXQHW}qmtq%$K#Q4|G+)mcn+_4_hy)LBSm&*m_gl>rG%=78< zZizCtE_JodwabeIy3y+!HhJCe?w8XbI`*fqQnN}Ym9vu#!Y>?nLhv9g7*o@d6|DL5 zh84z4-Zcr-S+d)o8MTeXd_KE(kja1i-1Tu$;pi-9&vX^=y|*;+O#a@tS%%2OtF5}d)%Q&+pZ~mD(VF<9ozprtlF`?5ZbY7{ z>`9ZI`C6ljUHTMjaJzNKu3(WO|N2>}_ zwZxj2BIfBqHJs@sOm*?V*ZFT92m4Hw?k~>9EYHp-VXZN_#m=4wD80=iF-Jn?xBE;u z4lB1jT`X9g5nUNpqpwp~Q{Pt>LE9{Ho;)k0h^=s-gY$EFqeIb!ff3SqciEYc z-pH+QG8lCn?0BN_@zi{QQ-Tp_zvP62A4AqO>f>6TZdrj*G&;*%TXp6I zcX8On!#SOeI`~NO%WTZPK>Puv!u{mIL#62_VmCjyd0?nvdG)aFj{4o@rP)~RHWSj; zm-cCM`^tumLRWrhBe~{$dvCLKGCxky+Z!w9T@kD_; zC5*HZ1A;{tf*!KJ9%OknjkGT2pv0S!F-uu={pXaUaksMuq7nh98b~TU^i#L<*)Fokv_OHZ{^2&rAf@UbAI$4tXiftLRiwdelc^4$@4# zLJRX(yJNAri_sowEb1M?sLwIu_fus-jy*?87q^R5H|Gb{JnM<1kyyQv#zxmi{f~CN zBQGz_hO*`(7mmziH2;cO4x0*kpY%-aL~znTiY%r#$=21t2G-|@x=S;%w5K<*Y>4r8q_Q1lPOQ% zB|ULC`c5C$>{EH`5ygvhZ*r}UUVwJm#8}u*cfP1#^E`|7hc&OqYAF9`XezgI5uMID z`rcKVnsnGuaMfGYhw|PefrS03JLyRK0sOBMmz)k9+tvMoK2>vWu|!IKm&po!{z`@N z(^Y;lJFMQ4bnC234?&19T1P|8c1%Yh=FlQqIXoqNaEFSl%;n;T3OW((ZQ<#;+v690 zmHK|Bi9c=jmd;ewonlNS|G0$7pX{V;5{}Sxc%ZX}^>h;zCHq_>YH~wnfqr<|&1JiW z9GAVw1(&&dw^D|}gS!0px{UskJ#)QBsm4jbL}f%bONTW)#J5AUrLX{d_L>yGuU5CD z$m2sxWrLvwH%*RgyVF>lc*49fMPWJAt;@fBd#vQ`-75rVciSe0pN(7Hr5z^p9O<8U9uFt(G2EbBBzJ{60m? Zl4Q)Ol*Swhtco(qLC{`PjXgww{s#{lod^H` literal 0 HcmV?d00001 diff --git a/bindings/js/test-data/v2-bc6h-encoded.yapf b/bindings/js/test-data/v2-bc6h-encoded.yapf new file mode 100644 index 0000000000000000000000000000000000000000..24fdfbb22ee2bcbd9bef2300587789ea8cdfab95 GIT binary patch literal 1692 zcmb_cZAep57(Uy|Dck%g2>Vi(pJ9mFkHA8lnkh4rzDRdqf){hO;l1y*w_C=``@*L&`DAM|?{F84m?p8Gz}d7k&2DoJGt zmv@ANI9!y05OZ_j2?K}=&;PIC2zKQ0u9}sJti~;s8xzW5{M5>9KZXVHervW;%R|V? zOUm(WOrZe(!oR5`fq-FbBacF_0 z-PdxJRi6x+B-sfmBXoBj3p$eM_hqBy9>@}t$P zeo+DDN62ociOI0NkKcdEavM3J?F+b~{T#<1`_a|+@UtivKpqq2T z(2eyF;E%MUaBMGWzeC+Bwy!XE5Vm@5NS$94zr4R;TRr0soA09FV0^1x!WYX<5_fR_J#4FRl*0cf*?tUn9@4FsV!QV0 z1NqtpucYnhOx8fYOn*tYUoUpn2K{quZ=MUqN1}`K>n0_easLfM|AjjbTYT#wX8lkf z_mf0?=a^fP3Y|heF1*=$iOK0x8aY4 zQtO7FJHBIn@DKKLre6QX`Cb>8Z%Y|}?=<=`NrgKPfmr*@k>S?|=f{}tl`|6SVfU{z b#V{A-hyC!>%H0Vu_cHvKZ>h_{Z{v(l8+$1TvadkFS zXi-GLFSVXq@Lf?qn8Ybb((3`zDSGQfm5f_GT)~82YL_yHYe?}> z%FzrMweFDl6&gv(|sXL27@Ku_&x|_Z8T+O9iJaXr1@_8u7BPjrY*V^xxWwd)3K?Spgt3v zi({K#!Q4yzE+p?yes~r19MlI!HT2SBwNyeRNH60kT7m${76_&!X+NPA{$YGLLu-ZT zVj0FlNahS8pb+9D*pZc^8YMBjQ%ZtMFFF+L8S*mvJ}&8!ifILqso)Wi29I0q&NA)9 z?M0OF4+Gl}pJ@+#%QHMuecUc?VWDws<$h47GcVXYI55=*pS;tg*3cbp2PnXTA3un^ z%2EKhb6s`OM;!DdG73hG;AQ%!yeC@F^9*w$agq^w;`6|Bu`J7L4EaBN+|}rV(Ed>y zQ*Owjkl|M{bk)kJs)VnfpzBa|fX2E+H;}pMafaFdnWSap+@@->CJ3}fBL&VoKAHyZ zq=ROBK|8v>!T*p`kIOQZ*juflh%3!gOLiVdjXCSsc`rdD{MPIQ!Dx3EHsUb1g#aM| zn>`oNBiEh7i^u>n{me`|W?@aba>0*FjOkrT;g!VET|kh(tS3ybIYvhOF#i5aWqipqtubu^;7W2;G#OWLz2Wj2jTM|Bc0Qd0)Tvrp$@j@zHU8hbn>l zR8oa41cnQ;0>WM7d%}p}TLM^uPQ6)X!0^`R+*aHok8L;egl}^8DOuZwsONmZ?vx^c zrk+@0KL2aM;B#lXd3YEpwMYu8|KNz2g#4cnFc%x{uWpCVZ2d6{lz=0`6LTuhl@Za- zg;!-75drehBZ@@KXD{cfOYA1a`F##LUS4bECCbTSyuNb`P4;#G#qFLN(BO9V*~?4s z4QeVPSUB8kW^?dmq7|@iIoc_p7eFW2=z%t|7n-4o(nocA0t0?wRt!hkIb0q<)~^k) z8iU{luo5NR23aPPMilTuUA|Ap_mt1Qf0Da`B6J`y%(lv;DU(EbqzK#$YSEl}XK7HJ zM-7G%b-9Q$9w}oWF4^O;Q3RBQ^QXb~KYe#0A_RisM|N%19MFbU%g)z-LF+9qDP-v$qimgF7CVa^;#Knj84NG9frmbm3{6ipm4?psau=>-THbmD99{(HUmI)a-{%% z32J$;=g@cPxDkfI=PL1;qSwWW(v!|{CY+b)3G8lAraxW4>Ju;)fTDHn*6DVm7NkH@ zEl!`;$0Q%kPa+GF@&s@|oJJb7`6HBxbeJq0I$`XJH~FBf7oFB2MM>C0?zyut{%JG8 zA(Z5m!#5EI4-LmL@|4o@7Y4oM>|oYqn_ef65qo_<`SoKF!Nw+Fld955Tpu&2$Y0eN zLT3Z__w&kM(|-0Xf<+Qp&a>fUth9N_K?R0nIN_Ah|3&U=!0n+=XjFMwqqhMi+!BTk zKp_<|yI*%|`3gH!WgM$bEK$^NIwyjcJJ>;Bnqgchg?``(<*J$BX(JLyeB+s)Nv1ge zz3RI~gXypg37}!JdCS{Tz~$=3iYpqdLYcN<73<6Y@0}%MeCkgqC(oCw9S8>rN)T%s zS|m|`xIhDvksyw5(LlcNB-#rA_JJk!w zCtvtpnaHX-Jx#4d56N*Iqe!qYPWHlCyb(4tSd&&9PdG0wjedkikZDhA`_ZJXcJ?-12r%Q+zeGO)?dWH4tGiIMOD*Li8{a z|FH_k+6DqT0Iwp5T_-k*ovX~>D>p9TG>2)fq!&paCn8$YDW&Hm)m!~97{X}KofNa9 z(oQS`;Vj(^w<(jI|AX02Cp7$iS@;M?f=$VWPHoHAC{>su1z!YegG$FTwassc%XEPi^zVC{UmN2#l_ds1QR&S3V@zWf=ZO`+G}SGJf9=Rm1LClk@QA3n@oh{ z;3!)hOaqs?iyFgP2x%=>@T}j?FJGpcalH*%A;V#2Z7r_cKX$;FL)#-Z=U5^URjJBG zdP?W90d@uMSSoK=sUk8A;n^DItcexTeu=%Pk&1wViVKy=tx#{F1^gH_CEBe}wz z5ha9wdQ~4RO2uD&EwIoLj}yH@o2fG`KJfmoBO<9`xIBWPC;F3%UDv7m1rbk;jiS<= zA4Wb!sC!>z)O*2|4nwaqgO4^_AKD`6j$!_JJ0KWlQ`36Q!uIisG{guw6gmtwkQ51< zlFiFU#-iTUnvad3I8f!qjbXANME!tG%ORSlOoZNWVqf)k38E2}b7uj|H5h z^Q%MWsiboSmTZ^TW-U|xW;)S zyS8vDc&UFn+#PZ-l1$g92s8NoXpZ7cpV(n>Sxx~50qhNI#BP$hj!<0A3(fXV^?{RM zVfSe(WFC}^sW6`(o+{WsZOBOp1BJg?3!lZy^AM=I<&<dM7v`-e{=4?+EHT8fO@gBu z%&Xpl*Jy$RS2SCC`%UzMF0Gq+8xUG~uk`(2m6H<2JL=;M%e8$=GH`pVfY6=BxV~Vip!%6@OHidy|rbiP=$UBe1 z;lRMLoe(mzXDBcrY{bVHm|w$omo~4~)J55od`L^md%MzCecAce?Vk;z`NxTv1~K@;owm3EePZ!gfm`Z86r1rE zVcrn;RiYEUn<0ng33a$G5RQ21L4saR%bY4{dsJQMe0*4NbcokomWoqq%&B(c^PzwIG z+N_9*7wO`^a=u!!80Y$c_1}&Hwd17M!|#S!Ous>hfY6gUR`DC>PPvPqo7rU`XzwA`@aWLRN8U-Q#W~HOgzzwktRry%e5#(e8pbKC# zZEBjprM(=-G=Q@t#_sBK#sm%zVty+Xgx3vW*i1lM+BmeN9;YKGc|~Qwmrw}`w?x`%ozxp|Acz2gZ*`S5fIl ze#_V#C~@dwz&~VnbMaYqEd!G|ny{T-CH2KY`;Ha=2=-ZV_-Y?EHs-OY?>(e-L_! zZQ-Im5&}Od8Lr*t@Z!hN!CDFgd69Bn#KejRqx_E>}5n>|?>v-cqv$hXLme{m3fLfmD9d}s$*eo#6-Ze0O8x17j zJMo&8somW7l8yHy;(fREMd+vrC(9MOv*8OR)~Hq{J6eHRQ1sdvKQe2B^^XB_9Z4sJ zC0Z&JH+~j9Kr^@8?6aATruzr0o5v1Qw~~5crkU1p)7Ws{YYS4vTCPa~O#I+6tny^7 zs7&FLDupAz-rBa!GDqbYP>8*~B^e5sTs2rlW7+Z_WWnV_OlbX$#|y=oK&Fbqu4 zpL6@39bW&)$*`{h&?OjS9y25(C*;f)PQe;6?m zO9}z&avzHA;MJJ*L1wo-vj&~jE^{%49`E`sGPr|f)yNu*GTX_#Hg8M3z$4YF z@N;$BGd@OcC^&v^#2y4H_qE@9Y&V+zk)eYQkguf3R@}-6j%I)fS$=-ltKzCb^w*0Z z;Fqlt+8%3 literal 0 HcmV?d00001 diff --git a/bindings/js/test-data/v2-bc7-encoded.yapf b/bindings/js/test-data/v2-bc7-encoded.yapf new file mode 100644 index 0000000000000000000000000000000000000000..da96f06f05cd1c1c72ba2486375ca96188f26596 GIT binary patch literal 5620 zcmZ8l4NzOvot?#_n?npO6?_{)?i2UvycpB?G44$5 z%d@_>-7zFhd#Sb-1reWpb4f|Kx$U@q)uqtUTa5@Mvr{&(FV4z5P~hbQaTW4fh=u*% zdvoKP4b9M#d%5TQ{=bj&KY6s;vcI74pbDuF`e%et6};iU@K&J$`2GK13tQz+$iK3; zDxB|LvgB^)hR+}RM_exrHzV{b%?~Hfu0iO}siXgZ<$DUnUmmqrEsfQvkj1C4n&TMk zK^Uo#noY6TmNBnZ##hqcy>sL7g&SB^ZP%y^i##4LdxU;u48}yGQJ2i};ZMG9ZGQuM zo9(z%Q_6;U?w4law|Ld30)ze!D;#EH5|H!&Bq61}EHWkwMcMG+yQd#|>< z_~i%U6>%y?#f6$DN17_PZPd@;kbTAx!lw^>GxFloZtlMO@B5$8v2MU4z6u$SC%}3G z42IfD;{X@%1a@Nu@5Lql0zGE_9D4}VqfkBj@lw{ndUb#oYt-tZZZ*rccvwylFjlJz z3ZAPcQCGwnk@1&^_>+iLb42w7Ra&%W747%?t-aP>q526+X~`PDhoO8G@d_r7pACn< zo@xuq_>JW(%dx+{bL#c&oT%{-vsua1xig13TyXGj;qAqOwvH*Py!8Ke9Z zgBS(B;r_aC`)Gp8<6Fv>4g39hrcC4~0uH$8t*} zqYYy@{tXwqE`ph+2h=K1SkK#EU%4FEwLF1p;&z7O3C`wc4X*057F3D)6Z= zLV(UJ^IcTRuxz*|!-dIJLW%gRBZv^QIU=Z@Af=i$$5k*w5i^nTW5aAZosDHOHw15& z-B7xI%^qI`s2A|Tt#p)r1o%qCqlHY4TNFQuS#*=t0;3q~x_RPl` zBG+x0DbzLBl3FetUi_0~AtM}@{Jx=ffVk|l&N;kc!26!P$vH6@PF7ExZ6i{Cly1Ty zUCNQN^W=-qzCT=5ws{W&)n$C+IeaDD1%8EYdh}la57@cD3n@SZGcm(;JJuAHT)cSy zw~y2}xD|dq0WL*9%ij=Wepiu5BntncIX@R79X1u1&E2^0Q&!A3a!;(0jviYfVjh;L z{VE+YBL{KGdN))iF7#!;8O!vM&#oS(%8E{^2r`R~R`^D%s0zxrioxq@Os$}I4-~MW zH4#?)t_?XtI^K~|@=o-=I3@$W@9kBkuVO|AX+!stwUI5)o6F8pev7{%?u*NKAfLVb z$2R)0$CWYWvmY1O+~e}Nq>STJD-9*|DE5JW4JBm=v#goU(2Ac?fKQpX9;Ow3JsyUy zVdEd-<0i;=eL2`YtFj0g5icL(lYSfRck0-zX#L}>mSGKPk>`=_WI$*Sd2|x2EIq^#?nnl)Wd)y zDh!4=*4slxp-K=UEGgEe_OiRQ!*adpAh@b$wQs} z3O>e=N>?|G{aenzh2s2Rzj3GsOBu=xA6}y$f1;C5<@{XEU07OqJ=w%g^NT7}VbW?ClMi8!|q9`9(gvtg>XrAg@IKt^vPbT}7i+$uAW}`HZLI zahcz8qldrw-Rpd2S>l&eRv{0^kB((-;gb>V^R-pAhQ`KNYs+LGk@^A6DFkZwiS>~8 z^9}Z|9egMi%KO`sOlO;I6C&S`7vOJ)F5s96q!hkYI-SUGO5q!aFzo(;6mC7}Ebe65 zSi^mF!S>9e>_0sP{Lc}#^UM6$!}G<#@D0c(DUUJ1g?i^ew|w1DSG# z-*70nD37VC*;IC=G5pq&!Y{B3dS&9Go<+Qlp9qlyS>5fx9OZl;Z|E@7K^NBKMdsEBd#q^4V zM-O^M$a>G;Rrsa-82T~T`-sVG=5zX8SLo0=LZQ$*ihnVe&Q5=uT=_}QDZU|?T*fUm zj@G97ruw?^Tg_7o%VhO~BTZCE5q3Fxe$V$Pd3JgY>}9?6pV7_S)5 z>06`q>uA8!o8!Ax8^FMJC`E?!dt2&tx8ZJwh(9G9JY?;(?b-_Z2R%4aphwVOB|$v@ zKy|opMb@X$jJVWFqxPAPlyP@ocICjMeSL~Q5rrx$nD}WKAE{^gz7^Wmena7ts(_b| zQS`f>IGrx1YhadqZhFsBUa~=j;0{sWHBVj@`%|$yaO`(R@hfZ2UwzvCoQF+j+u81) z@|(6YR>m*px9DdL=smP61bE~;YgJ(!(1BveVBy~8di=V@Z}ro?Od1!Tg2a5YyEZnT zX}F%4#w8nI1>Nr&hUYWQ6TQns>7OP4*GHB0!}7!h#{XtqVPrtRQXbXXFuSy*vILhT ze^R1e^)A=Zr+B6R@^Ig+sLY)y;1l}!l8ttk>#3t5J_Y&}`|&eSKWZ=IV@S@w$f0w& zL_Bg{Zx;0e5cstKzi_`v%yS7Z z*gKZFRdao6iAepRVH+;McMV^+yiq<&}R_TWd$z|U2YABZ!& z2J#TM$o*s_y1YWS&is20&npP%2Y#7czhMXdKT+9iE4hAqW1-IJa6Se475!=iyjE)y z!@T^lgb(zK^u*+JI`jJzT!Lc z;4QD4AMaulKjGCOhaL2t$7?pZ0dZ2On|bZB^JjNxt=Lcd>9nvK^ha$gS}S&Y%=qct zdQT&CPeCK?XJkD-_B2BL($;7Cmc{FK;164A8UQ8!mcJ0s&jCKwGi194VQA)#xO(z= zj3kENQTSvHb#!$@cqNB#3)W#*=vjU)pI4$Dc)%<23;xy8Fd^el5J`Xf60P6M`1^a) zd46?}|GZ;PxE4GTer)&ca6dyQ{Ap4*@Y;LM^3H0v>8Km<6UkrHTiOs#iFv5hn@diH z7Z-0%35;IXQmZRgF|cDo&ybt1Z6|Mszp#hk`3~{%W$R<*2?_{nIUio8Ngc(nu}YRw!~yI zIXTfiDy%zX>a^@C*&}u9RGj4Skf9?(rs&~odH#BFuVos}k45)>HgtIJ+c3IhnZ~{H zd=oGqcayf=x+T@OcE=X{hAu>Ua7e-T_c8Hw?1__uHP%*}zTMJm?YD~iL3`b_^)h#5 zaQNWRNcY54ZoW?K4CWm(x)i?S%!%`p*S{JZ9;(YEMicUUQ?oa-*{sU;BdzeMg>Dm` zm{9nW_Ft&D&~nJp*|e(zc`;26Sw70^v%~a-3Enw=w&MVT9UFeOi}Q_6)DI7RKQb}3 zKz?yDHt9B=tA^fOh(@Bq$tmscYInSb{a89r17CYvE&3AV!cboXdGEJ&*$i9hOV-5u zbJhgj_6jKe-7ALh(D3Bc&5SbNPQRreI^+cK2mMp!+oC>1g$QSQzDXY2w}Y7NF1riP zvvU9R8WK-e;(02UH??)c^TCODLe5wA!{~5qKG;1KCenGN{7rZs58pqeNOa8ySD&N) z0DSd&`>p-L!%xM&bU$zxAcfgrKRWm7Ut){13zo*l>@6Dj8`PUb{i7Ff$P_XS+kx5Ox~z|L_ykl4=7jpe`OmmRMN_?&G0Pa)JUIXTv8ucG zRETaL8_)L#e%&QtFrDM;4Y9sredPLGyn^o=s%xK`UR3G_Y>5a{x%!o9VK!^Rn;`#r zaz1&?csJ=GUmO{7&*HnUuU$|rqVUbd>@C}mD>?mb#lXG0vm;4P1>yW|*ha^#y^U#q zT6kH`E969O+zp&`KJ0q^%fV4Y-B4`g!|BC*{a|)o%9Ro}aWLP$yS0ADptawh#_aWb z7uKmZHe~5rDqDCtSHE`I|8W1cptv8r3Ez&4S>ui0!hC_tPyg&KKnwW2Y4Wh7hca{o zzkquHpLBjU?$CPiw58vghWbIj^>eHp)4>Sx=B!#@&_V(pCN>@$A0H>z{71STGjw3T zg`vgkxje$O)$i@6`$=@>eNhkgUgtG`Dil@lKs{5yp@?Ty>i9m4-FuFT6%t-e+_S{| zJOIz@)%AK4!gWLC`qQTKPq+enfS;Mm>8%v>&pV(V3O_0GAC6aPNt@`#qqF3?y0YLjIj`*y6UaR)Y>dB zQ;WiXg`K<4J+ups<1pi=(H#F@VeRmI_1X4m;3qcXt~@8I8}chHNP63McXh~o!r2G= zy<9IH?3aMZJtS!EGp}zo>9L!1n1?!sgw|)=+dKD%&W#~|<%3m@NmuKwN{^Kk~+ zo$=iB@4~P?ceG@}c!oLSdv-ADIB7rnoZT|;QVf@8$n&_pC{CgbUzi~=84kh6!K}?Jgf3EgP7oZ9 z%Z;`JW}z_&cFxjPMlGz`Y}WzFymdb597qipw_3anG;?QWt}X z8XkKybs-}F0001B0{{SCc2x0~=oJ}N{eeK8Iqb|!NRvqWm2v_IO+^HcT!VTrSk;I#(Niy&caW3X$3bU0ze$> z#xD!+p-s&QKiKio3(CQ(GT%zApp{s)0Va1|z>o3<3yjvs%b4$eukpynB9X8CPc*JQ zBUq7RNjK&SFV6Rn|Jl(Grqe)V(D~3R=ts=I8+k!JRSWxpLO6vcP5mv2qZa+gF#f^0 zpz;U)UhWz2m#ze@H?EWp>M2#S92}DK_(E9jYZ-HYAlq(2g@u=r#MAhU%0ONWLJCx_ znky&G{9ogSRU}lkQM3m3TMR^`py(Ui48vD?OWK1P$SLg?uxy`>?;mDcF@4P%;e#P2 zE$wcc|2BYF1DoUarx@bV6={kFItr(8!8XON80s$=Y2xzCp_$sIY)o=OHTMVmDORTL zG~^if0^jVh{y}VdRmP=k^Xy|}gzYl@bH-hj?lbX5-mkjt@_0ifm5AC*@|%f#_AeZg zOnxCLVuj+LVgPIoIckejlK{&J+rD>>K46^usA2{%#s5IX05q5whP@a@LlulD<9UKz z(d9%xIu+mS3WXm%?QoFMPt$z@TWRy{{;F?P=eko%9hYgl#nBH+CjH1H(b^2f@y3N*HUGYZ2!92el>%hD$p)vvPsLnF zl|0R^#U}?1%mLOyRtLuD&NU)kVMq77`?(R{NXGkQvugvVX=F3SRxDSSCIJ1sMBHzV z-F&($-CU|{@VCqGf{JJnzL1XWkw*z#rs-q=Yjf=|3ZbqFaPD+!NSMm(0Ga*k!mSyE zds6C?*TVd?LHe3P=C;vVvT<$LQE2&CE|KNEV6LkvttdRKB8$9-wTVn;ixdHZs0O9Gjk0In#0$ITy$)hhysP zP;=#afWw8W!;V%Th0&uh@ke?4^lqeFIe@S4E_&z>{TNxxH- zUik$Q>eIS*`r3RoF-uBtL|QG2Z@Esd2!YcyyOIN+J%&igaNQ}MX__Qo~#%+B3sA%y*WwL8-y)h zr#8L!jtd64yn)n#-O{t_T>I`)XSkBn=Y(rYi=6~gzWu97t6*wLKST{LCF3HN%R2Mi zTLq09!oR#QtfdT!`dSq{^V-0Y-447M*3k%{W z4JiqxeeWG|i5V)W8zTH9-$x&ULxkQPag|0AR`w%3o#FvHv{ox*#SX66y@ghCNcgVK z(PINgg@R)GP|%da-is(bnyqNYvGQSf57^%@V3}@-4>DQdN?)x~`kOYn&Ln@aG4$4$ zPRB)v5*wIaC@NjG{_)=HzAPvjRaOTdZ>$HEcPew*nv7To#7=_zp{smRoS>9s;_1{< zUjYd8za_q{6lgxJL*@@xIfLbgp?yo)>CTR*J2JWvgeRNuaOb_@YHw>lpL?<=`-Wb1 zD$V-EmrT@hZ@>1Lo}w)l&@cpraAwFstN?{)Zg~gOVh-9N_Pa@}R6JpkX$|hI;AwoB zCrwtfXmyDfkX+%ROF!xD1RCyE=&Akr<7eNpp^_sUjU*G5)Y-f@VlV37<6tQ$!fC{C zEe6QZ4YC0!5v%$Tr0W6a@h%|8qk2 z7p0A>Byu%wq?99`4{fPKG25H__Q;OVF73cK#}%WSJiOfSbN# z_rMqSwTAVMW+&vwb)YZ*5Ut}-Aj!;^R5EpM7vd~7l$HAixXG@(F1XY4WM8xignC{W zdxV|+^}792T^PxD;93rbYW$d5r#bSE0uE||s5c5wg)g4cMKiRgQ$Lm|+z(N9%pb95 z&#FS3QLRsGedFc#K+4bXk2{2lW_NN|4Lht+u0ubSHVXIuQO#c~YKX*i zuJ<$d36*t_C!iVHa}sGY8mDr#2;BfZ>F%FqDA$^f^Hl!NGvJp7m$~rHirH6%H~iK` zi=a8{G5-z{mlmHG=b`x*4jbs~xWV``P>*y+13_s;y;GwJDgdo6wMEqti{2&Am_}(q zh`KKZ#x}ute|9XjpY}YK((!$|gk(B<;0l2D(_tG22&0xt?aeZcUBW{DI6J&w(ETQm#oY_(bi+YSXC*83Sw zM)zWM+CXb9{m_Ec+GDYVUU$SAG)74=%FT)=O30KJp|UgEd&u(w;LsjLoplmX830I%c_wn z;6&=_&w_hXh2IfFt(iAfwW>Pg$Id1V(1XXUd;_UnHcJ>G z`&(j_>i${M+)VHRJVJDRIGpigy45p}ZYsu|!k6{v;MmQLwP_k??1gBUSpdJ$V09=n zu8S6%?|c=cjzw^+xuP`9Sq?4a9mnR3aL279eaei(F-}iOH;sf){e*Qkt?w@tG)aES zKF{v(Hhs$FNe>^xV*$BmFpeW^8?&n~Q7l`{zI6W|4#*|MtRxo}#YEyWMWj;n*nji* z>e3LIrdYQ0`H!*XjBizltQhsX{`^ z&VKn~8o3o!W*{v-mw3=puaB6FTA~n$1FeOj*K7pz!dg~caJ35S34c-Zj3)flQMiZa#xg*M6;LtJD~%_ivnQ>S8yFA* zjT~GH_OV@x0ui4Z964%r zGrR>>KkgN8rbFJbR5>j9^$LsP^s`3FfADI{e>}s&qt)E59BUwhkKNTl39rw^HXNC% zfgyoS7kZJ)r!Y^-VJFsP(bFlY^Ln;AH7Gz6u*MTxvA?` zI5&)te&jMxHtuLL&QE)haKm?s3CJk$v)WXg_kZ_#+dMXviOn7g?yh^awHnznOb&zG-A0DcRXwii zU@(TpX&YO=ow_>m=!on)_QL=*GG)L%$tpQ;r;sqmmZ>&l)?@$@?G9g;hveVk1qqk~u6c9tzA5v8;Z&qmgZI(K z&&`MNvQ(0?*T#LNtRR*pTOQ|6(K|i=xwRwFfrjQFQ)-e49N+V3h#!^N0w=?_x3h{L zlRmcXLIu*jF9a;=hxg8rHd=*_=xE%12p%B0#5i4F1~FetdPcGtj&q?TY;-w^0mVin z_@PvhzXh%=BeN`sYGd`Kk_>$&+0<`ebzj4Zkn#RKVz;EJ69q0B+U2BY=>-X-V^hYUf zWY-Db=i(ZZ>1C(nl^eb9*n%jKUDR{)O9p`Y2>Fi+Y;A%7BAn-Un1cw-_}=myFDP+A z>7Eg`gtN#CccuMFarbvk9^}EY>1ip||3bObOzJ&_e4az;ZBamrmO-2NCUKhy$sEjG z>pyu;x0SE?8k{;hzmBDh2jA&@X3c6AjN0%vBt!;yoCWze9@*89mp*1+vu-$_P+h^@ zUFk5f~{kp1thT44`7N}{V}o* zqEb)!ss8O1^C<%F^bz$F+_(*0S6kI-(Ds^=bVX_PFT)FNt*#GuOm}LagIt610pcM_ zazoHVL44agoJmhiY$fb3J7fT6Ywv#NFhL z58}&Gvcxa|Tn5^7x#LtxG+6B1`XQ`J_qkwf*IkxCcmTsvQQ%H5kpO5GsaoTcHmMQBzArODR5Sg|t2>`ibXu<{)}eaAd00)bMLZq z_spF+-#O>rIs2VEk5P&rKMdXa7k_HMY)QEtx=&{>9onqqn^4|4 z=uR^^UVv^*)APr_Tq9+Tx4TKo8gCDJ0!`zU!7gkXZ?(eLcsnzAmoj)`C3(NbtFDl; z#^bJ(HQt^K9?#%K89b(B?cYE!7w>KyfTr>C4N}&4pJed*Gk9-h@LDo>e7)4C{R>*8 ztnsEZc&?gH9nV)Xcpqo*${D;u=Ulwz*134&o|LtJ)ePPT8N9DEcwHGh)P^D2m+XpH z6m~=%SM3B&)TR&Va6}5F4%*z-$y&-?mcO)qB|PAY!gmxpY-+FBp8A$zRD)f zDpG}8bC5G~(ei{u^q(%GbVEjef6RH_wf`>wbjA2%=0`0`v)=yY1A4tZKTm^0NFhs9 z5l?hbG)Aa>t7Q?Et60F$9h*>Uk^aA3r>~#CXO_@pNlg}{LrQl8KY!xvfphH*m{ci! zk4&AKnL4q5>cqfrAKf`_ja!@c)_8A1Y>(665#srdH^Q7`l7+;pM(lxj@E*^Pm>Uph zq}hR?A`I}?SeOK1=m%tYWdy&OA9(&1P~&q@6OINr6oZZtxEFNP;Oqo}->nINO}`2V zgn2x!-5( zgQJ$|=^t(<|AC~Lgw9PuSBcOhbiDy+68ggW2Q~?vk-2T&TFfMLlhAb~V-mVa=(;X4 z3H>Tn_lBXHEf=%pqPM2ZmP-;W#XT~^&<#U3d+7RIgV{q@kuZDci}j_4N$4h_n}lu> SI=&J&FGa6|x7bTHq5lD#lpiDj literal 0 HcmV?d00001 diff --git a/bindings/js/test-data/v2-expected.json b/bindings/js/test-data/v2-expected.json new file mode 100644 index 0000000..148768c --- /dev/null +++ b/bindings/js/test-data/v2-expected.json @@ -0,0 +1,137 @@ +{ + "v2-rgba8.yapf": { + "surfaces": [ + 480599072 + ], + "preview0": 480599072 + }, + "v2-float-gray-entropy.yapf": { + "surfaces": [ + 1656156314 + ], + "preview0": 2474033295 + }, + "v2-half-gray-alpha-entropy.yapf": { + "surfaces": [ + 2599040047 + ], + "preview0": 297007736 + }, + "v2-rgba8-entropy.yapf": { + "surfaces": [ + 554695116 + ], + "preview0": 554695116 + }, + "v2-u16-rgba-entropy.yapf": { + "surfaces": [ + 3416063949 + ], + "preview0": 1518500476 + }, + "v2-cube-mips.yapf": { + "surfaces": [ + 3098789294, + 2090642189, + 3178169037, + 3641419477, + 333399741, + 2333110333, + 1703812753, + 3098789294, + 2090642189, + 3178169037, + 3641419477, + 333399741, + 2333110333, + 1703812753, + 3098789294, + 2090642189, + 3178169037, + 3641419477, + 333399741, + 2333110333, + 1703812753, + 3098789294, + 2090642189, + 3178169037, + 3641419477, + 333399741, + 2333110333, + 1703812753, + 3098789294, + 2090642189, + 3178169037, + 3641419477, + 333399741, + 2333110333, + 1703812753, + 3098789294, + 2090642189, + 3178169037, + 3641419477, + 333399741, + 2333110333, + 1703812753 + ], + "preview0": 3098789294 + }, + "v2-u16-volume.yapf": { + "surfaces": [ + 1504297975, + 2857113671, + 3272367306, + 4248642449 + ], + "preview0": 3611091147 + }, + "v2-animation-notes.yapf": { + "surfaces": [ + 2496952478, + 2084186354, + 2971951804, + 4204487618 + ], + "preview0": 2496952478 + }, + "v2-bc6h-random-blocks.yapf": { + "surfaces": [ + 2051803746, + 562394057, + 2093502260 + ], + "preview0": 2386146506 + }, + "v2-bc7-random-blocks.yapf": { + "surfaces": [ + 3714498223, + 242396130, + 2835116443 + ], + "preview0": 4042788133 + }, + "v2-bc3-packed.yapf": { + "surfaces": [ + 1798067479 + ], + "preview0": 3027945807 + }, + "v2-bc5-packed.yapf": { + "surfaces": [ + 1681957115 + ], + "preview0": 1079658926 + }, + "v2-bc7-encoded.yapf": { + "surfaces": [ + 3415378546 + ], + "preview0": 851871445 + }, + "v2-bc6h-encoded.yapf": { + "surfaces": [ + 3372163902 + ], + "preview0": 89762387 + } +} \ No newline at end of file diff --git a/bindings/js/test-data/v2-float-gray-entropy.yapf b/bindings/js/test-data/v2-float-gray-entropy.yapf new file mode 100644 index 0000000000000000000000000000000000000000..5dfebbbb2e603ba8ef49fd6ae8befd80a04c676c GIT binary patch literal 11260 zcmbVS1yCGYv);ws-QC?KA-KCc1cJM}1((3$F2M=zlAyueHE8hQZh^PCQt#G%|Es$9 z?>aNx{q@(~vomMT)J(08h_WOE5C8-KAORu(0N@RH13Tai2!ICbzsnQj?+9H8oC`&2 zuaxp1gbFY&yg>j>4+j8Xg0UBJagWl!C;wv~1ZIK|{xSV;M36^FY8n9e!Y*c(_qWhL zcfpx?|9v*d+$`B00P(`E7LxqWd;XmX0s(%ViT_fCL<2%h{J*^a6kz*5;9roxn7`8?e^;Ep6d4e57XV-b2>*_OZ^b_ium)%T!Tk&P zoB5yM|I_F1Nq_MnfKXSzxyHr-V`TsU#uzNW0Am=K-*B+?hyHCbfmQ~9S-*{6GgA4F z6-F8QPXinRfW6=3H!j%v&HWVsTmNhSN(Mqddw76n1}vH>ZR!6ujiYXJbWe_ zj}Pm2q9dEwI~Yjdxaq&`CCtP5Z+jWDi9uB%=N%aONTdQ~((Ofc!@hEI=7_XG;SQ%?A}zRlhzIX*sJIzb~fN#?xA4 zT%p8;Q&*eZDi)ye63u&(;B?H5Y4Z*KFs=`ouqrRM_D(0`DD-fG%0OJroBPYBqb|h` z6T=Rm?spzg|DAoJ0p3BQ8sO|?**}><*iLo_hrZ0~@_L3wkdZ-Ezckf!+rxOhbUOrZ zKj{=NqdY49c<>0z_EZ>wfkY`k^g;e!9K~9rlW^+{MmnCO>*n-SOWn+DItjZnkG`$(i(uKmD@u`Uc%D>!k$1Z@KtTHIgwwFV>m#yG|A1h;(h>l68AEvFd6sHW zma})b(c8RY7BW7XqToCE<$hq7sMaO1-EppDO@1ThzE2*G;i@+)Eg5}X<_?lgR!i+m3d-o;7#1{c zJOa#jJTV_FhB`;t^x04$p1@bxCMSQCRm1Mpj-7%dWW1YEivO{bfhKAP(XV5{ZYz-r;9 z^1SBy0swk>P>_svfq3Oi$+yd9 zOU3e;KI?q}%o-P)?ViUF4eqdHa`R=~3}c7Qj>CiRxM{LZL_;%*Pa@58OFfvLQMR)t zr0}{tB4|dccd2Y<|kn({zi< z;ix&V?KD5cq>EPDo;_tF$u0$%WS?XV(~oqLxYa@6HkE(#N5&2MAZ2w-YztrLv}=v7uIMt*DWvPyUcdY)oqq0Q%VP zok)C*llrM6zgao*s2aInvThoy^r#}Dwd80uQM2OpQulcUK$Rg&HNKy0DUX<-Tt2ke z#Vl`{|FweOiK5W0K@+Be>0pK(%l?-VwaOqaZ;CWJWgsKNiY#yt4f-)fCl|wIBI8pO z62PRZq15U*1l|-{I0Is%ZeRgs(D9hd8^Yc3c%`!3U^hpfU*bALT{ZoV9)y{{E)$vo z{Kn3*epEBFn?4rYS0AUwUBhURYouP|6@9vz3;#)wN0IO(AV`)QLC}}A#&oB9P3lX1p&qMw(e;R@-~|8P2qw*pbP7J^;C>k zSM2P$KgiE?G@ldnLfWaDa*BIdlK{qupE5J`bF`$Gw*WvcyTo zhHma+-=Wm%l!MwY`a(+UjvR^7s2y7ex)+}OXr=d0-o#CYk}szPV)R`z_1|X*yiM8* zt98?G4bhxdf*i=ctk&a@zA?z0Y46;l#5y}QEoX7;8wCj0$H(8O!>c_+VIRimrzFfA zv_ui$i5l@WBhp9YpHL+4e)){xYp1WNjHAD;ZNbB6QT3!zLQkUB#Tb|&ikG7np>Zo7 z%LLjf#XrDQ3qPzIp5{tEDNRZiQaijo2}~m31tojmHn@=)FjrYHX0*}L>nH>fFN9@? zLTZ%>M{$=a{d~#QN?!&nH!&AEHMjG;Q+ z=RMP*n)vIpdZKrUb(ncY>u(8Hr5)bmzJJsaXZaR>*P4N9lU2hLGM)IrWy~woUW$Fb zW(djfOcOwfRW=xpOCxDs^@7Ps8gY9>D3LMV@Z~6aF_fUk?9=WQdsO zB-xdbc#@oIHe%8t&IIdtX@y3MU*cBW6NaG2*EX+Bk0s*j>#H92B_@JsJf11(-r_`4 zR>rjjAT@=;2&u;2{+);#B=$Kc{PH|**(A!siA^;b*&@4eoC8mY2q_b`t*tbU;q4@{ zay8}2u}6iv0UwcUzn^h1 zsG7G3&An%mla!{CM{*?gEw4t9~NGC1%G(Z89XaD1`@-)ead^{wxUhE$a)}{yk6TPya<+6;@S7 z`3fs;4r0m)?(puM=Q8n_wJX=HG(2>ykc?ioCcLlBP-q35{0pLU>7bCtq{mmw22T9# zd{^)15dgsidnZo?EDhXA%Fjesac%8wq6RNZS2x{C!8O!vy-=y<)*e5z{SB=P&U zoS9iVIuoKU93qHc+gZ@%0~f~HpXDlRI+&fS zul1u|%oAh`&>iHs^1lW3wm!eB?pzvDmG>FJW|D2Vs#=rX^1_Pn5)qJB!+>Ra?{LEt z2vyt$K~C)EIpyO*`F0Ju|7OYBG0)F-{ch)>F=%e}!%mQ}z(%8>Yr&b~Xh$mfu)4-& z76a@Pb4am3t*|>v={~d+rh|muI~nT6EWC;2YPu>#R!bxr+?B=S48=?zfY^^^E2m9b zrVdeO)AR3VHrSRqWXOcpGOYnJo={+i?cl+uUq(>u*_2YAnIzt^(4)ReLjb`(O3NB;?Vr#QeyF4O1OLc$2hlDkn zDP0^cVFMVDmD^%EGW?*?h}J#J^MU}*548E$LJBnm9&8H8bl9fDO=ER#p^3lBg3=46;v$aCQgvX;e;v0ShKwo#)hCdTX>Dqc&6|&QBUs(C&H#q> z?tjl1_y*Sl4!3(KBzPs@)CE*?N86&h#c(VY;4=ORvvGQ@+s#VGXzfW@YA}C<%i2#Y3NGd!% zzeyv}zy*Xo!r(xA4uKrV$*&c^*XeA;JhPXZPIIu`L|Q)Np@Ar(X-zzvu5(R~mCv|; zklxR07}etlS=c^FuAH!8G=KO;{+4KKzq{Suw;*)8q&Piq8_nQ*vIb_R!LiV{zA>ls zGSm0SrVmRSsGWE#%8`d(gcami3%YAPxgW;EID!5O#bOvEhhO9oH$jjhYf4e;2Bi)W zg|kg>RCW!_XQ70$OxSU)p{qGXg`Xb08hB*K-=U`l#vSDF@aT_1(Op zm?-cBu{({6ca#Y3$q6`pB?Y`$!>XZLhz`K_oWElC^7yFHs;h?yN=_v?*L6e6&D`K? zh-_bF2tGypOacU{V6>+^%!lv^7d5(eR{CXnsGdsw2%V`F4p|9_@XUpe&EN1jxXF#% zQ3A>Q%(WB-Z8%DB5K}W!JyR%UN;totNV`KLyg>l=ZBiM*JlCav)0L@aEF}wR-3f(n z4c|4~FLo-pX-TAG=UrrX7b>m`#K^Q!#J(Zufpm;J{mNQS#)P9>m*y(IWK$dPLTo<0 zk-7BFL_Pi#K0jS$)B$yI-%i1gW7*5NLh;P=*!8_|j|Bw~ZFasqsnp*Q=gwJucZH1Ex|QF2 zTcPIg7Q7()$dD>$bgoba*t&NYM3i76C(cP9!Zzw7-%0Q2&3PnaC@iL4kgTD=c#U71^mc%##4YLHZ_tCN7_1t2(VcITZU-CRD$qH9vk|y7^ zRT&Di_*<1Ox?qTX^k?kQp0AdW&EoML+O^DvT{1Wu@Z(C!maH|ysC3$6i~0OTS!b^2 zWICd{X6n5!LOa85-f~VI@R_kAwpH*@nXygI`_$Ei>~sNM^Ri~dGz+REG1ef6RJa)T z2tf7l4l*Jx()oD#plG0abClU<5*TCxZ7=<(Qnc~79JcMolSyGmkq%KYx{e+Jt;*p5 z)@f)6q-H}JE0(5t1v!*b;dvQ(`N%-l)Z#2hh#mk ztnd`l#tvdgu$bz3vif_q7!B@{K*FPt%IeBmvJh)y1Ra3_WNlgr4*brEEK=XqE^1Yl z?*_j;-8i}~uSl1kdUU_!Mg|g`j*donCPQ~qN!?RcS1?R} zbs&U9Bj30Q2FQEyU*~O1V+X^+b=^t^NK=NckNlLC53IF8Uai%4CZcMbWG||9w8Zql z{aVZOnO7>6T#;=W_KG_7&L_J5QTECXyb%#gOGChNw}6F6By*3UUjOq? zU;t1%wP;LgVUZT+GGmimi?m04nba-(dYv7o1g%=pZs_z`Kf3A^cWg~M zuEY z{(G0^(CoQ}-fU%IoI(VcI|zxeT+^PR^T1uK&)?j3dx+@vGNx%^)P3bJ3(OSa8KEYZ zcZWH>g@KeC&e(?1zR4A6J0wTxhbN?X9*w)3>lohvO;Q;uGYnatAT`s-RFgxFB?v<6 zRr@m^Y4c~#Z&fwA`2yn!X0480%jP^AFk?`}a9x?GVjy=O9tok{*(q9J*$}^in-9w! z4J{KACxo3DgUtf$Mr)@DG=3I$!9+irFZf+Mb=$UmKV>HE2$nPEQV6f7(AKy#FQJ}? zMk-sDZ3*TE9~JxBk8wm#qn_U{i_>4sVd>B5v|dBRcGS-rJ1&On#cjg&ieJa* zNUn`W5<{55(n;88$22~u&Ig~5m=8zbn;SZMhXuP78M}kJl+3i-m?$s*TTx*n1jL9F ziE-bWbs}*KiA9oEJk_QuXxZKuU+6>`=?ph*N41GBnW-#c0+(B zIh*Iq0=0Us>A6eHouKL?(2=+kbUM`yCwz9UC1B0}B&orfqdKJH4r)IQwU@U<^K? zfTXPl3{Gtm@`<~)l1I=)LnUlhY^wbN_gdf0?AJ7Ji>B;oM`Y@X;1ZgoXPC=&Vims0e?IiTJ*wqx-3zon2%0wZT;*&sv4RH^+;k+i~64Tt+kXQ zqA4(MuE=i6bTlTXbc3o)A{9v+Qs#~?$4b{>TE~{J0o=&DnR-&G*+QCW=PviWLP%YjlDv|NeHW_9q z4-wcC?mAlJCxn7_uAbTT1;pW5nJ;`ZJw!X`??qDN&EOCnTqUJub&@NoSA&|#^mNDM z%qp6^x;^m?wli2kg*Lfv!$Yr5(^R=M@ORtjHY!!OWmhX~?ObGrUp2B&EV!NGWaX5u z8p~0W^tS(WVwF2g)N-%J1~Nmh(CGVOS1=es<+Y*W-+gLCUtZ(BLlugeRdw%tNtcfA z$spNC_c46sw~xO zr`fu>U&Z&yY4NVRVu9>0q_p$n;O=b~d$&_;L%F_69eJfN(l~GXk-^Pst5VgO=Hk-C zY5+%K&fu22f7zJCW`UZC*63XusIfvfvUTV1VE(v{eD2m{2UWPqYhaw-o++B5kSVK* zTrZC-OHj(DP3377$F2x>%1u_?`%++*`qtoSW1OjJhv%4S9Q;8gOCoi57sY^0Rj#^e zqiRrEDA-FBqEf>*aWJS+TbysbV=!NoP(5yYKHuLNwQTxeeJP%regbe$95*U zMp(g*LPO0wbSH;cFY8jOFuPo5?I>|UOm0|EL_(+R+-)IZK&TuiID^ZzqGb{7$mV;S z@Yj#SdAr6teh}=d!d>y6{=9sx+SVL4C92-4I4w+!nJNZP;wiD%y8HDy{oLvZQRrVV z{Rx|)rU@{oaaPH+ix0fMe;!kexB<o5ngjUEHE=Pv>!BF#KTIkNaAV6 zWhM?h%g?%ud{-Y&+7Z~M;XT~w3FcL8gqyb%sRS)Hb%|GJ`JvqVmWdz5395Z~!VR<% zYSSOkpj{{o8&$A2qil_Dnm$XTX0|gTL`d(Z()Vk`W0cAI>AGzKxeC)<3*ew_1XcLL zHFVOstaNBUR9{Hy5PF^Gy&k!shlrvX!Ipy1waX}M5>~>g>wq}C;wVq46`abUQh7Z_ zUu&-*Uc;Ari%*o@dLleX`+4QT$|l>Efi%*2jm|$2!7=#Le!_yd!4W~U_$zCfx-x94 z=HbaqOo%)_Dj}O7IfHnRi_zgGov^GoHWK(}tE&#Vu{#mMCFa}iD zej07#J)*Vzkuvo%KFQbyM7_*~EU$OM9|m!}iD#r)GHaM`nD5oF;JSs|a)RG?G9qYm zvcWz?r)^7RZ3G8jTIKiz1 z9#lNPh8#iW{@v{0Wd*IyI}rf^gH(%24H7dhC5Wj}*_MGD0gNbb`WBQa=!KZb2u-KB zMsDXUteoTz(!MxhFput5E4gkT`gNUmEf2Dg{m3LQ0h?X?d{dOBl}C|vhR3Z!cbgvF z995RR*<2&em32xFr`yPeKVo%DqK4X@Yp~!Q?5?Do?d>fne6UMD`K5-tYCFXW&6>NE zQWVKQg2tQJj%}O?V-1`_B;8ef94X7MG)RIa*D&Q$dMcs+d^jM%j!H$^CCa}Sm)?(N z7YV|^LIAxn7lC;v^Zv>SF1i+|FPab$n3jM(5OwC6Clv!H6d|wJa|grAotD?`h-hi7 zmKF^s!(%_$DzLkHQT9`L&I~1l)Ya z4&Iz0Tq(-QQm8Ax8c$zzgcU+R6XBjl$&Jn3ywMsnIo#UAiY%=Kp2{ffv)W)N=MWuw-S2*+^c6gdzTP9PZ z1q^=Yb&(PM#A0FP;6gGYe2SSGO*LNe^w1qEVw%i2=C#lLO|O5SQg$2#vg1OQMJ-tT z9G=1!026f9BL%Chaz{t>HE@s%FYRNiCuM?;v>^b?O0}LEs~#UYgkum1TSPpu@5p_9 z1fGyN6^&_-jVz7(5PG|vxF%IUuKdw`ErvGFx*0dbqf!CMvhGbq4zY4y7Xuar>D38d zz3kAnYXY^euqm*Sz{MmBR_{}lz@ZumrP^~`&(%v<9S zs0scSDiwSCWiAEdq~rG&#n@aT8CELS0Lh0k8c(As-_&@O*ulU3Rlt)mn2mpgtXuWkl%zX?r z%a9^_32r4N)wy*^Pvn}NW&>iqKdZ$j#n2=9`e}?H#NUcemn+}F*SH50np(GO7F@}2 zTxM_eK`ljO<=vuaM*r-%aK@=`3sWUDNmqT_?TE=}@9LFDsolQJ$tG&{9Ky|p)2DII z*gF_ST5`#0>*+PLMFbi6v(~x&Y1UH8HV0azA&AW;22vjpkWpCjEh2o~r4LA}kwH*{4`2Cvc2_PR9V(L2uy&3OvJ;qxpQS<1CAgLqQrGt z2%k1k_HFi0KabuHpdFzz^#ehByQ)DZjLAD6 zsX@W)d#fT(mMUN(*@9587%(MZF@{GH3ldL(?(ONAevNZVN1=&s7<;%Qpmg7{4luR6 zRfTv*=yG-Nv7?|kblK4}Rk%a@YbZ%fAle!^(v}FMfv!0QZ){Dj)8MobL94t0+Il;S z!Pm_BZSz}?r9=X_$?LhQH>&xhL2`6qlNnyPR2MZEKx27ZI3ETDb#=M0a`=eywfA!$ z1#Wlb^$1_Xm0HRmV6`9wLWiKGv|cj#U0}y)Q2=#b+@6 z4U0wNReKl|fZcp5?QY~=kSpWaiBB7OH1s9JVRyC5dv8yX?(tc-=kl}wf>@>Kn-Rp_ zN51~&Ni|+q3~vrjMszwA=&01-wVbiZCnmbk(_{7cg%oYl7>(Kufv^ZV=i64U(clm8 zcrNE&^Pl7PvBh5B)dVkOHH6P>g#a2@Yq+sLbABbh(v@<16>_G6E%zZpvi4Rg4%Z)z zOS>0Q7z@M9Iy~+>EN8#7PfC;6MWf+KKEP)|lS>NKZRFXsyUY62Qb@|Dn|gF=Xb~!s z!N+Axl+hOE;gThOtN!i^mNUAQBtFnS=L6??Ys;g^l{{p3Hw_57uWE9|E-mv`j;C%3 zewD!ZJ6r&zA6GUFrf}ZaBON`tVK=~U6|uc8%ExlVJ9c?>Ydyml_E8v|h;@XONYUan8B>*^6f`y2?>Vg6?H12 z2{Ab4BZ1;UN18t}bi^d<3t-WdtDSn+D2B&_6Y&TG{^JTN01FolfCj)oBL;N#dm{Iu za4%G4UweG1VqXhZOrNmP|Ngok5UK)1`(4oa0Og5`7D;*KT&awGaY^Y^;zL8Y7^#XI z$7B;PlucW3S0D75hAbCpk1q%l6W{1FkU0Es*sEfVhxnrc8Xn*)DF%;*)1k#cQ-cF_ w4XOI;7Uv`dyQr);Dn4*3aghXJT+`h}TJRAN z@d2qQl|ia41RoJWe4(x^;@GXa3O?2}j8f2!wc8pnGrE8-weEk;y*Dp#O1r<|o^#Lt z{!jkzk4sf-3{4{nAsVR=LNnnHUf_>H5?H;dMb4cJMMcxhXZ5RDULYUNucvj0e0>ob z4rd!E+FP~X@1Y9c~pZ9vJ~iN&PT>EbgoGIS_|{6p~kC1jw`^}$?j+FzpgbF$iADM=(0!cJI?6uoscyfFL$UWGk>1j{1V_7yuHN;DLI>3Nb- z;%SqG7rZLmUhqDYlIX*sx{S9wT7(!MA2(@_Elc@4!D>KS{3yg5+&}{mGU5j&wuD=e z57&N%MJv4wYV}@~V&lCe*krHP)o#i2rBT7J?@8Q}0*PqSnlx!m)bG|5(?V;KFNMyH zk!V?oh`Ppg80+NXuMH+GdBOvS;YqztdL4Mm;}mE$?x#?m!NAJ$d)LB>JiA5obKN46 zboA;LoHi{+?5m-@w_{fbuh@AfXb?FYj#x8-o|Xj|fsau@6Q>_Ols+0R#mgU?>~(AG zX0HQ-Z=NsZkIh2rr`r%o*@L}D<6$yhf3&O?8tG)Ez& zWuLUq2%NJ!n(T+1O#i(M_a$@S#I+HuOstZL%h(g$+PC5d4*&^9sk0Xs^l8ZwTYoLtCPGBI& z3%fy#mj*yL@q&zg+zxT+VnxmZgE&p(L@UlVe0Kx^rx$D-5S$oAE(JKkcLgl$1h}Bf z6R#k{v2iS(!7Cszi9>Qa$L6_r3MsHnC-Lt_KZv|<7RA$C@lTod>Q39Tf;k19?H~F( zs>0q5JKdIRUAm`Qr#rIePHLB!Vvp^&*Cw=N559f= z@!cbtdFg8wo=Um(?ufwX)JYd_I4WA7x3=h_zNypLo_x5fbk)H-9j8(>KL6cuBXjw~ z%BOmt%*Dq}ESFy@7imEMeTJpc9P7hM&eT4Fxn)*A?;VKQT-cS~y!@;1s;mmL zI^TBZ0(|wKb$HfgVgk;^GXpaualTT&q&o(Jt3H_xeW&{NoO|}*Fh5|Nd)A<1-tN4K zO}NPhFLPEOdjXvRcJ+$JlCZjugF^^ae($UfZe?A<-y@nLZnv-Hq`~+5dJp+W>|YDX zO%wYme~5|UE|)i-)*M|Nmb5l#!9qgTa%p;@5Wg(q%?d0|3QpwjMyghFH-=+3zou4N zRq!PeOq+%e^AwgXb+^2ER?zejff&71f0>+Wwd`!z3z3m5)0*Kpn@{l(n>(wVEh3;( zTBNCnF%75q^lV1~fVN)PPUu~EIjfYX4#pOn{%LU+B=*FwFVSGyGDhyVc~<#&r(-OM zgoGd(S@sa1bR~FI%Kak21=PtjJQ8BmjAim6$hz?u0du5P7K`P**ReHLX$1iNZY7A| z03%bya9g!$SV|!sbBq*+xMeM|q>#M5iWno@(QilF4(ix2l#_fcvU=!Xf)O!jUaRD)8Gs1jwf-0^o0JF@9 zhZxxipb9gL8yqAY0`EKvIK&#_0-&6Tjsyn$H<}wQf-!DhKoTLnCI_(vRp3l|j0ZRZ zG#3P%ws4qaEa3QD3UI_v4D19f;z_`hFk*NGN;nrr44CwoP1rm`Y?GnJS08H$?;)Hbv* z_vqGGG0fKLKff1#vu4xo&iqhaV@c~5a_H~P=W6P@e%Q96J}MZ{WQN2?CHsdX>YD}t z>9~JI+@5JplbzZP5ci%?9%5M*UBnfYgaO!9W!3aCa>(vU5iG z_oVP6LabVsElda)c5|H~cq7A=Bp;S{hv!m>+Ym{$DtLEdRhbYI+*g?U1hJ@iE7BZ> zj5cl{L9Y)&GM@a7l UP6JWmlxw<{9 literal 0 HcmV?d00001 diff --git a/bindings/js/test-data/v2-rgba8-entropy.yapf b/bindings/js/test-data/v2-rgba8-entropy.yapf new file mode 100644 index 0000000000000000000000000000000000000000..c7092573a4577c3d790610e3cb0eb2beb5f38ebd GIT binary patch literal 1765 zcmeHHO=}cE5UuW>UU$ZTn|1W4NpNNSK=K2`%r345y^P{Pg@}JZa*^mooRtK^8Z>GI zkK@NdZ;D_Jfv`x>gLv_z1dJZMsRRTS4C3og7eOK5O`C0ItE>CftE$(1te)-T0WnTQ zb<815rYEBS-(SbOl3CgDqK7Va&U7-n;X2l*o*!RXkHaC>7Bph_Y0+)`jsJpJ&@Tk> zQbVwSB(h*s0%<1b3qw+?9IHO?8~j1A0WV>A2{2-8k0gq&mF5O<=H8P}d$)!sa%RU> z{_x!5?C7^gqf4JpEzN&eoOR~pNaKBV4!`l2wS`?*c(cVbielL=D9*K&$|60at%)p7 z$XIJrmT0Sw$u^g8C((KDopIhG1yp3L3kDe01_O*R4FimzJ1uaW2Y^E8tVmY8H$wE|pAsl_6?akMla4=aak6f{y==l_Z&gW4Tf literal 0 HcmV?d00001 diff --git a/bindings/js/test-data/v2-rgba8.yapf b/bindings/js/test-data/v2-rgba8.yapf new file mode 100644 index 0000000000000000000000000000000000000000..1fec3ffe0030d7915753af9e47e6bf7745c7c4c2 GIT binary patch literal 97 zcma!a3~*y&VPIrpU|?WG!5|(cNQ4TVVsHPn?6eR=0F*BXq|FK#et@{FK-}jMYQry4 Hd)6EPQ|bzU literal 0 HcmV?d00001 diff --git a/bindings/js/test-data/v2-u16-rgba-entropy.yapf b/bindings/js/test-data/v2-u16-rgba-entropy.yapf new file mode 100644 index 0000000000000000000000000000000000000000..334e8d8332b1dc657ea236037435323a3054c5fa GIT binary patch literal 3672 zcmeHJUu;uV7(e&i-rJ?iq=izjiLj9@>9CLyicgx|E!~h2lXmWoiCF|^W@9K|B6}f9 zOVGgBgolYlVsbM!LoPhbpq(-anBh!}CIeZNRPceIXq<^AOpW6FednIrZXG5(@M7eq z_nhy1-|u{Xe&6Y7pQ=wZaSu}%KJ|>TI=oomRhWjk^l5*<(R|W2@%@L-cdbuIKpFJI z4_$xe7-wt|bQcr|a~^fh>K-gqrg*AUm2tg6Q9VpU6$;aJc*eW}@oVPrVgbL=V<|{N+XSe3DIv-c6xvM-&H_czy>WXI# zS9OPNT_?7^tV_Y+`S<6JHr1=}D(>Yt{%S5YT1iqR&0C)j2eXVp51WqD(Eb0OnlCkGVL zUsuRIJ4D3@uTfc}=6laAJd$WHC@*_N4ybi+BZ?;9oXV61`JxUla8-Lco$tzhvF+HE zTzajL^0}9rj;{&h(|2)c-^J8l;qg@Ocxox>w)W<>#+o`~uaRYGQ|DTO9T$tf@zh#a zn#Rad(HZNK5Wu0?1`oYt?u=o}v{inFosq*SuJwR3VmKt|rqtKDg<4yZvNcBbSFc+Y z-&i2>ztVbfp6N_)f%R)QT9yY?GQ+Dq<5XHesj}TS!e7 zRuS7n`(QDSXZ0h6jn=zRyB^mGV>kuSSVLjsL>B@`R8$z%KJ2&tC?d{Za^fLWX3%c7 z2b*nr2kjpZ4D4Ug6X}Tz>>v2u7ibJ7_d{%mKOBc#1$9|6mLwULcryM3iNFu|lh2Ww z_BI5QA(ClxFnM9wOZ7{PCC-qYG@MkZ%yLnsDEwHlq~VlRpys;;#Dhc4Miu-x)Reod zptiQ6dX?l2_|K_)iOLm8hi^B~TXk?JSV6Rn4&fB2ox0!z5_}Y>`N(AjCAPV&1R5{L zB`YX6~~p-2VRZ3ti%J)&SuwpbMNY#+!-$QQiEjS<;64)7W*2@WXx-3KZMIZzfN+)?gjA6Gi38N2c!1g*m5rvZ9gM`Gt|E1Nj~}hu zhrX}qsqUfflIZ(|ZdQ8U=CkfnDbBiTan0;JwELUu%YMVHQfAZ)1u+lxw_NSVC2IAz z_!zE~=9(dBBO3IW`&*i`5`}A&C7NsC#HhJZCk42`1q2kv`yT)2JVxD&*Y>HLm-?%Ujy5T^4a{F%;AtNxJll(Q!%XR-|c>&yI=#eL@l dDxbTZBjFVxoP$uuIg$uOVimLjDI$vHmP6#G9DzneP_YssMD zpaDQ2H2?sz@Q{E90`Af<{@)g?_&4XnupPR-JEUP6^*5a;2LRTTi{60vn|_D{pl#bV z(~F-KM8c?T_SS{_!81P7BJRU0oZ9FnhAXs1A0*{^f`tIw9 zSX3>70bficvha>92_(LvjRy+@QJWwQkXiu*?;#uVK$-v|weq+;SQr*RivaKj4q)Re z0I#i$*G2+uX%xT*0&jtcPZ6~8AwC&^v^D^(%h$kTf+-`ahB2Y$lm1m0Pxo9bU8jd- z^@Cu*n`j8MaK#m8>3bnoJQj_xAp(Hh2&{FAI7QkC4wfL;AYRS|Uol7mBoYw8&0ECt z@b-qChn*rD0j6`iQ{8}RFm{uMSXZuH-5^g0`XzWnbi|8`KV zf9HRF{?(WRB0_8p#r(<+tTp=hKYZ3d0$5Cpz0Viwf7#*~dcg57Z1}eRGG)R`0xSnbS*F+1LI zQz48u+9jjb<(w*>&s4ye^xA#%B^bq$XO=2q91J;hms9`w-$O;o0uIL%%yL8AbV!uuh@CKi8D5}uy;LAZ738}D5=KAzmQf{gnv zZf!+46Y>mm z(hH{qx66N+J|T$Pu~T~P(>(3qfI|87&JOeWW1$wNklyz%;Cbtv4X1% z_0jgLt9xE_3$K-XT)Q;3EHq<8)Y6uPgR9$isK;I3?iDnza=7EtXuF+g^|e}O#EFW(j~j`)&$4wx(ppPYI>wR+>}TEWYv_OrgS)N?YJ z^Z9OJQKH}R!FCTAZjLeR%;2grrtU!rA-19NX21-@8CG)09aVgRvD6)y>whTO$cc1} zz}Pk^qjX;(4OZ)v^Yuua_CSdRuSjoOZ z!r&cUhz8Z{^^W0N_u)uZeecLcd#U;gK_w{d%P*s% zZJZt4q^iPXG&bI}?Wr0=axL>?eQ4G0 zcihfZ4VAXIAa6Jm#Xz<)Za7LuX=mvypEos|B=ekV?Iqo3`xv{N=Cn)&O_Yf-EEAwTPC)f zaqz+;uKNe`Tq{G$Vyt6@`3O8-0nS&?++fpUv{PZ(UrX~RI&RITcU}-JKaE>^9l3`8 zVeG6=cjeM8rD#KfE`9FT#M7^yS*o^OP3e|1huj;Or>5Oq4ruG_P8(}BSAfo3d36nS zq7V2Ia?iQRoJvQrZ+j$mr`qD~P44dr*cs5%Twb7N>;H(L2#zn*Rwqhs7iMY-qAtmaQ0R(kL~R3DVqd{aJpL7;B?(syO{U- zhBeLyx{cz{IXY(&8TEO$?=*yX){J8IGgOE{8JB5Yj_AQEm4oBwEXiX{5E?EFK*S4cfvBvr0R{SdCG*a zN&0xZoU0Tge*%*ALF4iAVh9zL*dF)j!H8``gA3EIXC)7BR9`&pzUyQ3NABQso)Qj8 zB@%v$0$HlJI7J=`gI#hgpSF00T4V>65T>26pqadK%fkuy{E&{S#7QUj167$MOIeDY z`M8AE@fp@&!$c82=Z!}!{`~DO;JArgdWyBQ>n<*5A7}i?Xr>jL|2mRS=!Om`*JpMW z-(0y6Y1Qi!tk-Q6bP#J(y}J}0#rBYoGH4pJXv3Z3(LMH#5K#rppy#@~-4v4(`wdE2 zk2oPJxyWLrhA=p<`1YX?S zXm-5dp(mKl%5G)22jw~v`%YZgkwms&v2lSLqz$~@o%l)HZXmjcSiv@Flut-5dV*@? z#!H)0o%`JChcV7yUZYHqZ#5Dd`*ise{OO#gjyws}INa>$v=+wO;2ZS;g)xU!X@FGnZY*;&yF2EcKgl3O4O;N z$5yKTkoazRUg6Ishletz=TPlwd6)h-D_SDDv86sGUsjXZf; zxBYDQ%iof&&A%DhVD8nSEF60CF6s~F!ScAc0TFL;blX5GWx&NNdibLEwbAx%g^@F; zmM+8TrFq%%Yd20^Qoqz%7*SX2|MT&m+sdD){!&<=zpTyCHF$WnRMudrNBzB`xb?KZ?PBCuNeybDIpT zSYg`KAm`@f>$+-gFPP?C_!-l^M-EMAO0%k!0uP>H8K_CRm$kIIkI_<+#bndteGHD@ zNtc)WoO?V3hOLBCu|hceKX;mqEC_)%Pa&J-0hIP#s4^o5NJ=*(XL%Xt+@*i()9az@ zbfq*tx+StMmp|v1_XuNM7U-6$_os70K$yUG^nlRO)3~}>T5^a%NbR`b>HFxG8zZD3 zUrU}EmfCmZXi%)sI*e=+WnvX>Bea?5%J+E5!v*_?-D|MhLv0~aFTt;yqBJ6QGCQQO zn%{-L=rtZ!qxDa1q(-%4MI+qa#!B! zsFH&XhX2YF3W-bs!TImsy`Ng{SX_?lG@}Z`x&8TLE zeTs-_fBIzM@Znu?e;xa@`29*@CutFWG!G5ROXN99W+Zw^!$|J5hAW(Nl;eDGu`rh zlXw1@DvjNUmvAk{iysnSZt{{PA$ip_xw}#-(&9FBcIH)Em$LXgwFF~G(Rw>cCH!8g zNRNH-rECgeTbGhvg}AwGE|qhU!?p(_YJP1=!LdXIl`l7F z2Cy4xX$^vT)JC&nVyLqu*5ynt{<;XKr=f|mxt-mii|y+Wx)&0`s6N80=pvE5DmL<`(lKt`@lX~cfLtA>dR zZtk@b&uLEjQn{9ntl1=!m(`J3^mgkWDz212YhjQEbft}!g E05%-b^8f$< literal 0 HcmV?d00001 diff --git a/bindings/js/test.js b/bindings/js/test.js index ec37609..930ff58 100644 --- a/bindings/js/test.js +++ b/bindings/js/test.js @@ -53,5 +53,39 @@ for (let i = 0; i < 2000; i++) { } check(threwOnTruncation, 'truncated files are rejected'); +/* Version 2 fixtures written by the C library: every surface and the + * preview must match what C decodes (FNV-1a hashes in v2-expected.json). */ +const fnv = (u8) => { let h = 2166136261; for (const v of u8) h = Math.imul(h ^ v, 16777619) >>> 0; return h; }; +const dataDir = path.join(__dirname, 'test-data'); +const expected = JSON.parse(fs.readFileSync(path.join(dataDir, 'v2-expected.json'), 'utf8')); +for (const [name, want] of Object.entries(expected)) { + const bytes = fs.readFileSync(path.join(dataDir, name)); + let tex = null; + try { tex = YAPF.decodeTexture(bytes); } catch (e) { check(false, `${name}: ${e.message}`); continue; } + const got = tex.surfaces.map((s) => fnv(new Uint8Array(s.buffer, s.byteOffset, s.byteLength))); + check(JSON.stringify(got) === JSON.stringify(want.surfaces), `${name}: surfaces match the C decoder`); + if (want.preview0 !== null) { + const view = YAPF.textureToRGBA(tex, 0, 0); + /* float previews may differ by 1 in rounding; compare only exact kinds */ + if (tex.payload === 1 ? tex.format !== 0x47 && tex.format !== 0x48 : tex.sample < 2) + check(fnv(view.rgba) === want.preview0, `${name}: preview matches the C library`); + } + /* damage must be caught by the checksums */ + const broken = Uint8Array.from(bytes); + broken[broken.length - 3] ^= 0x10; + let code = null; + try { YAPF.decodeTexture(broken); } catch (e) { code = e.code; } + check(code === 'checksum', `${name}: a damaged byte is reported as a checksum error`); +} +{ /* v1 decode() reads 8-bit v2 files and names what it cannot read */ + const cube = YAPF.decode(fs.readFileSync(path.join(dataDir, 'v2-cube-mips.yapf'))); + check(cube.mipLevels === 7 && cube.width === 64, 'decode() reads face 0 of a v2 cube'); + let msg = ''; + try { YAPF.decode(fs.readFileSync(path.join(dataDir, 'v2-bc7-encoded.yapf'))); } catch (e) { msg = e.message; } + check(/decodeTexture/.test(msg), 'decode() points GPU-block files to decodeTexture'); + const yapfSum = (t) => YAPF.checksum(new TextEncoder().encode(t)); + check(yapfSum('') === 0xd7550ed0 && yapfSum('YAPF') === 0xc8cc41d4, 'yapf-sum test vectors'); +} + console.log(`${tests} tests, ${fails} failures`); process.exit(fails ? 1 : 0); diff --git a/bindings/js/yapf.js b/bindings/js/yapf.js index 194e1a7..fb487fe 100644 --- a/bindings/js/yapf.js +++ b/bindings/js/yapf.js @@ -1,5 +1,5 @@ /* - * yapf.js — YAPF v1 encoder / decoder in plain JavaScript. + * yapf.js — YAPF encoder (v1) and decoder (v1 and v2) in plain JavaScript. * * Copyright 2026 Nexoniarz — Apache License 2.0. * @@ -10,9 +10,14 @@ * const file = YAPF.encode(img); // image → Uint8Array * const rgba = YAPF.toRGBA(img); // any channel count → RGBA pixels * + * const tex = YAPF.decodeTexture(bytes); // any v1 / v2 file + * const view = YAPF.textureToRGBA(tex, surface); // { width, height, rgba } + * * An image is { width, height, channels (1–4), pixels: Uint8Array, * gpuFormat, flags, mipLevels, mips: [Uint8Array, …] }. - * The specification is the comment at the top of yapf.h. + * Version 2 textures (GPU blocks, 16-bit / float, cubes, arrays, volumes, + * animation) are decoded by decodeTexture; encoding writes version 1. + * The specification: yapf.h (v1) and docs/FORMAT.md (v2). */ (function (root, factory) { if (typeof module === 'object' && module.exports) module.exports = factory(); @@ -42,7 +47,676 @@ const filterValid = (f) => f <= F_UPAVG || (f >= F_UP && f < F_UP + 8); const upK = (f) => (f >= F_UP ? f - F_UP + 1 : 1); - function corrupt() { throw new Error('YAPF: corrupt or unsupported file'); } + function corrupt() { throw new YapfError('corrupt', 'corrupt file'); } + + /* ── Version 2: textures (decoding) ───────────────────────────────── */ + + const ENTROPY = 0x20, NOXFORM = 0x40, BLOCK_TILE = 16; + const HUFF_BITS = 11, HUFF_SIZE = 1 << HUFF_BITS, SLOTS = 48, CTXS = 7; + const NEED = { WIDE: 1, BLOCKS: 2, ENTROPY: 4, LAYOUT: 8, FRAMES: 16, ALL: 31 }; + const r16 = (b, p) => b[p] | (b[p + 1] << 8); + const r64 = (b, p) => r32(b, p) + r32(b, p + 4) * 4294967296; + + class YapfError extends Error { + constructor(code, msg) { super('YAPF: ' + msg); this.code = code; } + } + const fail = (code, msg) => { throw new YapfError(code, msg); }; + const bad = () => fail('corrupt', 'corrupt file'); + + /* yapf-sum: 64-bit lanes kept as [hi, lo] uint32 pairs. */ + const KH = 0x9E3779B9, KL = 0x7F4A7C15; + function mul64(ah, al, bh, bl) { /* (a · b) mod 2^64 */ + const a0 = al & 0xFFFF, a1 = al >>> 16, b0 = bl & 0xFFFF, b1 = bl >>> 16; + const p00 = a0 * b0, p01 = a0 * b1, p10 = a1 * b0, p11 = a1 * b1; + const mid = (p00 >>> 16) + (p01 & 0xFFFF) + (p10 & 0xFFFF); + const lo = ((mid & 0xFFFF) * 65536 + (p00 & 0xFFFF)) >>> 0; + const hi = (p11 + (p01 >>> 16) + (p10 >>> 16) + Math.floor(mid / 65536) + + Math.imul(ah, bl) + Math.imul(al, bh)) >>> 0; + return [hi, lo]; + } + function checksum(b, p, n) { + const lanes = []; + for (let i = 0; i < 4; i++) { + const [h, l] = mul64(0, i + 1, KH, KL); + lanes.push([(h ^ Math.floor(n / 4294967296)) >>> 0, (l ^ (n >>> 0)) >>> 0]); + } + const words = Math.ceil(n / 8); + for (let j = 0; j < words; j++) { + const q = p + j * 8; + let wl, wh; + if (q + 8 <= p + n) { wl = r32(b, q); wh = r32(b, q + 4); } + else { + const t = new Uint8Array(8); t.set(b.subarray(q, p + n)); + wl = r32(t, 0); wh = r32(t, 4); + } + const a = lanes[j & 3]; + const [h, l] = mul64((a[0] ^ wh) >>> 0, (a[1] ^ wl) >>> 0, KH, KL); + a[0] = h; a[1] = (l ^ h) >>> 0; /* a ^= a >> 32 */ + } + const rot = (x, r) => r === 32 ? [x[1], x[0]] + : [((x[0] << r) | (x[1] >>> (32 - r))) >>> 0, ((x[1] << r) | (x[0] >>> (32 - r))) >>> 0]; + const r1 = rot(lanes[1], 16), r2 = rot(lanes[2], 32), r3 = rot(lanes[3], 48 - 32); + const r3b = [r3[1], r3[0]]; /* rotl 48 = rotl 16 then swap */ + let hh = (lanes[0][0] ^ r1[0] ^ r2[0] ^ r3b[0]) >>> 0, hl = (lanes[0][1] ^ r1[1] ^ r2[1] ^ r3b[1]) >>> 0; + [hh, hl] = mul64(hh, hl, KH, KL); + return (hl ^ hh) >>> 0; + } + + /* Huffman code table → lut[off .. off + 2048) (length << 8 | symbol). */ + function huffTable(b, p, end, lut, off) { + if (p >= end) bad(); + const n = b[p++] + 1, nbytes = (n + 1) >> 1; + if (end - p < nbytes) bad(); + const len = new Uint8Array(n), count = new Uint32Array(HUFF_BITS + 2); + let kraft = 0; + for (let i = 0; i < n; i++) { + const l = (b[p + (i >> 1)] >> ((i & 1) * 4)) & 15; + if (l > HUFF_BITS) bad(); + len[i] = l; count[l]++; + if (l) kraft += HUFF_SIZE >> l; + } + if ((n & 1) && (b[p + (n >> 1)] >> 4)) bad(); + if (kraft === 0 || kraft > HUFF_SIZE) bad(); + p += nbytes; + const next = new Uint32Array(HUFF_BITS + 2); + let code = 0; + count[0] = 0; + for (let l = 1; l <= HUFF_BITS; l++) { code = (code + count[l - 1]) << 1; next[l] = code; } + for (let s = 0; s < n; s++) { + const l = len[s]; + if (!l) continue; + let c = next[l]++, r = 0; + for (let i = 0; i < l; i++) { r = (r << 1) | (c & 1); c >>= 1; } + for (let k = r; k < HUFF_SIZE; k += 1 << l) lut[off + k] = (l << 8) | s; + } + return p; + } + + /* Code book → per slot one Uint16Array of 7 × 2048 entries. */ + function readBook(b, p, len) { + const end = p + len; + if (len < 2) bad(); + const n = r16(b, p); + p += 2; + if (n === 0 || n > SLOTS * CTXS) bad(); + const book = new Array(SLOTS).fill(null), seen = new Uint8Array(SLOTS * CTXS); + for (let i = 0; i < n; i++) { + if (end - p < 3) bad(); + const slot = b[p], ctx = b[p + 1]; + if (slot >= SLOTS || ctx >= CTXS || seen[slot * CTXS + ctx]) bad(); + seen[slot * CTXS + ctx] = 1; + if (!book[slot]) book[slot] = new Uint16Array(CTXS * HUFF_SIZE); + p = huffTable(b, p + 2, end, book[slot], ctx * HUFF_SIZE); + } + if (p !== end) bad(); + return book; + } + + /* Context of a residual from the folded residuals left + above. */ + const ctxOf = (s) => (s > 0) + (s > 2) + (s > 6) + (s > 14) + (s > 30) + (s > 62); + + /* Entropy plane payload: filters, u16 length, context-coded residuals. */ + function huffPlane(b, p, end, z, tw, th, book, slot) { + const nb = (th + BAND - 1) >> 3; + if (!book || !book[slot] || end - p < nb + 2) bad(); + const filt = b.subarray(p, p + nb); + for (let i = 0; i < nb; i++) if (!filterValid(filt[i])) bad(); + p += nb; + const L = r16(b, p); + p += 2; + if (end - p < L) bad(); + const lut = book[slot], se = p + L; + let buf = 0, bits = 0, pos = p, used = 0, i = 0; + for (let y = 0; y < th; y++) + for (let x = 0; x < tw; x++, i++) { + while (bits <= 24) { buf |= (pos < se ? b[pos] : 0) << bits; pos++; bits += 8; } + const s = (x ? z[i - 1] : 0) + (y ? z[i - tw] : 0); + const e = lut[(ctxOf(s) << HUFF_BITS) | (buf & (HUFF_SIZE - 1))]; + if (!e) bad(); + const n = e >> 8; + buf >>>= n; bits -= n; used += n; + z[i] = e & 255; + } + if (Math.ceil(used / 8) !== L) bad(); + z.fill(0, tw * th, tw * th + GROUP); + return { p: se, filt }; + } + + /* Wide samples: byte planes, transforms at the sample width. */ + function decodeTileWide(b, p, end, out, imgW, tx, ty, tw, th, ch, sample, book) { + const sb = sample === 3 ? 4 : 2, bits = 8 * sb, isf = sample >= 2, np = ch * sb; + const half = bits === 32 ? 0x80000000 : 0x8000; + const mod = bits === 32 ? (x) => x >>> 0 : (x) => x & 0xFFFF; + const sar1 = (v) => mod((v >>> 1) | (v & half)); + if (end <= p) bad(); + const hdr = b[p++]; + if (hdr === STORED) { + if (end - p !== tw * th * ch * sb) bad(); + for (let y = 0; y < th; y++) { + const d = ((ty + y) * imgW + tx) * ch; + for (let i = 0; i < tw * ch; i++, p += sb) + out[d + i] = sb === 2 ? r16(b, p) : r32(b, p); + } + return; + } + if (hdr & 0x8F) bad(); + if ((hdr & NOXFORM) && (hdr & SUBGREEN)) bad(); + if (ch < 3 && (hdr & (NOXFORM | SUBGREEN))) bad(); + const xf = ch < 3 ? 2 : (hdr & NOXFORM) ? 2 : (hdr & SUBGREEN) ? 1 : 0; + const tsel = ch < 3 ? 0 : xf; + if (end - p < 2) bad(); + const cmask = r16(b, p); + p += 2; + if (cmask >> np) bad(); + let emask = 0; + if (hdr & ENTROPY) { + if (end - p < 2 || !book) bad(); + emask = r16(b, p); + p += 2; + if ((emask >> np) || (emask & cmask)) bad(); + } + const RL = TILE + 1, PL = TILE * RL; + const planes = new Uint8Array(np * PL), z = new Uint8Array(TILE_PX + GROUP); + for (let pl = 0; pl < np; pl++) { + if (!(cmask & (1 << pl))) continue; + if (p >= end) bad(); + planes.fill(b[p++], pl * PL, (pl + 1) * PL); + } + for (let pl = 0; pl < np; pl++) { + if (cmask & (1 << pl)) continue; + z.fill(0); + const r = (emask & (1 << pl)) ? huffPlane(b, p, end, z, tw, th, book, tsel * 16 + pl) + : unpackPlane(b, p, end, z, tw, th); + p = r.p; + const base = pl * PL; + for (let y = 0; y < th; y++) { + const f = r.filt[y >> 3], k = upK(f); + unfilterRow(f, planes, base + y * RL, + y ? planes : ZERO_ROW, y ? base + (y - 1) * RL : 0, + y >= k ? planes : ZERO_ROW, y >= k ? base + (y - k) * RL : 0, + z, y * tw, tw); + } + } + if (p !== end) bad(); + const v = [0, 0, 0, 0]; + for (let y = 0; y < th; y++) { + const d0 = ((ty + y) * imgW + tx) * ch, r0 = y * RL; + for (let x = 0; x < tw; x++) { + for (let c = 0; c < ch; c++) { + let a = 0; + for (let k = 0; k < sb; k++) a = a * 256 + planes[(c * sb + k) * PL + r0 + x]; + v[c] = a; + } + if (xf === 0) { + const o = mod(v[1] - half), q = mod(v[2] - half), t = mod(v[0] - sar1(q)); + const g = mod(q + t), bb = mod(t - sar1(o)); + v[0] = mod(bb + o); v[1] = g; v[2] = bb; + } else if (xf === 1) { + const g = v[0]; + v[0] = mod(v[1] + g - half); v[2] = mod(v[2] + g - half); v[1] = g; + } + for (let c = 0; c < ch; c++) { + let s = v[c]; + if (isf) s = (s & half) ? mod(s ^ half) : mod(~s); + out[d0 + x * ch + c] = s; + } + } + } + } + + /* GPU block tile: raw, or packed lane by lane with the book. */ + function decodeBlockTile(b, p, end, packed, out, o0, rowb, tbw, tbh, S, book) { + const n = tbw * tbh; + if (!packed) { + if (end - p !== n * S) bad(); + for (let r = 0; r < tbh; r++) out.set(b.subarray(p + r * tbw * S, p + (r + 1) * tbw * S), o0 + r * rowb); + return; + } + if (!book) bad(); + let buf = 0, bits = 0, pos = p, used = 0; + for (let k = 0; k < S; k++) { + const lut = book[k]; + if (!lut) bad(); + for (let i = 0; i < n; i++) { + while (bits <= 24) { buf |= (pos < end ? b[pos] : 0) << bits; pos++; bits += 8; } + const e = lut[buf & (HUFF_SIZE - 1)]; + if (!e) bad(); + const l = e >> 8; + buf >>>= l; bits -= l; used += l; + out[o0 + Math.floor(i / tbw) * rowb + (i % tbw) * S + k] = e & 255; + } + } + if (Math.ceil(used / 8) !== end - p) bad(); + } + + const BLOCK_FORMATS = { + 0x40: [4, 4, 8], 0x41: [4, 4, 16], 0x42: [4, 4, 16], 0x43: [4, 4, 8], 0x44: [4, 4, 8], + 0x45: [4, 4, 16], 0x46: [4, 4, 16], 0x47: [4, 4, 16], 0x48: [4, 4, 16], 0x49: [4, 4, 16], + 0x50: [4, 4, 8], 0x51: [4, 4, 8], 0x52: [4, 4, 16], 0x53: [4, 4, 8], 0x54: [4, 4, 8], + 0x55: [4, 4, 16], 0x56: [4, 4, 16] + }; + [[4, 4], [5, 4], [5, 5], [6, 5], [6, 6], [8, 5], [8, 6], [8, 8], [10, 5], [10, 6], [10, 8], + [10, 10], [12, 10], [12, 12]].forEach(([w, h], i) => { BLOCK_FORMATS[0x60 + i] = [w, h, 16]; }); + + function mipSize(t, m) { + return [Math.max(1, t.width >>> m), Math.max(1, t.height >>> m), + t.shape === 2 ? Math.max(1, t.depth >>> m) : 1]; + } + + function surfaceBytes(t, m) { + const [w, h, d] = mipSize(t, m); + if (t.payload === 1) { + const [bw, bh, S] = BLOCK_FORMATS[t.format]; + return Math.ceil(w / bw) * Math.ceil(h / bh) * d * S; + } + return w * h * d * t.channels * (t.sample === 0 ? 1 : t.sample === 3 ? 4 : 2); + } + + /* Decodes one surface (v2) into a typed array. */ + function decodeSurface(b, t, s, verify) { + const m = s % t.mipLevels, [w, h, d] = mipSize(t, m); + const off = t._off[s], size = t._size[s], blocks = t.payload === 1; + let tcx, tcy, nbx = 0, nby = 0, S = 0; + if (blocks) { + const f = BLOCK_FORMATS[t.format]; + nbx = Math.ceil(w / f[0]); nby = Math.ceil(h / f[1]); S = f[2]; + tcx = Math.ceil(nbx / BLOCK_TILE); tcy = Math.ceil(nby / BLOCK_TILE); + } else { + tcx = Math.ceil(w / TILE); tcy = Math.ceil(h / TILE); + } + const tiles = tcx * tcy * d; + if (size < 4) bad(); + const bl = r32(b, off); + if (bl > size - 4 || Math.floor((size - 4 - bl) / 8) < tiles) bad(); + const pre = 4 + bl; + if (verify && checksum(b, off, pre + tiles * 8) !== t._tcheck[s]) fail('checksum', 'damaged file: a checksum does not match'); + let book = null; + if (bl) { + if (!(t.needs & NEED.ENTROPY)) bad(); + book = readBook(b, off + 4, bl); + } + const table = off + pre, end = off + size; + let pos = table + tiles * 8; + const sizes = new Array(tiles), packed = new Array(tiles), starts = new Array(tiles); + for (let i = 0; i < tiles; i++) { + let sz = r32(b, table + i * 8); + packed[i] = sz >>> 31; + sz &= 0x7FFFFFFF; + if (packed[i] && (!blocks || !book)) bad(); + if (sz > end - pos) bad(); + starts[i] = pos; sizes[i] = sz; pos += sz; + } + if (pos !== end) bad(); + + const bytes = surfaceBytes(t, m); + let out, store; + if (blocks || t.sample === 0) out = store = new Uint8Array(bytes); + else if (t.sample === 3) { store = new Uint32Array(bytes / 4); out = new Float32Array(store.buffer); } + else out = store = new Uint16Array(bytes / 2); + for (let i = 0; i < tiles; i++) { + const p = starts[i], e = p + sizes[i]; + if (verify && checksum(b, p, sizes[i]) !== r32(b, table + i * 8 + 4)) + fail('checksum', 'damaged file: a checksum does not match'); + const row = Math.floor(i / tcx), z = Math.floor(row / tcy), ty = row % tcy, tx = i % tcx; + if (blocks) { + const bx0 = tx * BLOCK_TILE, by0 = ty * BLOCK_TILE, rowb = nbx * S; + decodeBlockTile(b, p, e, packed[i], store, z * nby * rowb + by0 * rowb + bx0 * S, rowb, + Math.min(BLOCK_TILE, nbx - bx0), Math.min(BLOCK_TILE, nby - by0), S, book); + } else { + const x0 = tx * TILE, y0 = ty * TILE, tw = Math.min(TILE, w - x0), th = Math.min(TILE, h - y0); + const slice = w * h * t.channels; + const view = store.subarray(z * slice, (z + 1) * slice); + if (t.sample === 0) decodeTile(b, p, e, view, w, x0, y0, tw, th, t.channels, true, book); + else decodeTileWide(b, p, e, view, w, x0, y0, tw, th, t.channels, t.sample, book); + } + } + return out; + } + + /* + * decodeTexture(bytes, { verify = true }) — any YAPF file (v1 or v2). + * Returns { version, width, height, depth, layers, frames, faces, shape, + * mipLevels, payload, format, sample, channels, flags, channelMap, + * linearMask, normal, primaries, transfer, mipFilter, alphaRef, + * loopCount, frameUs, notes: [{ tag, data }], surfaces: [...] }, where + * surfaces[s] is a Uint8Array (8-bit pixels, GPU blocks), Uint16Array + * (16-bit, half bits) or Float32Array, numbered + * s = ((frame · layers + layer) · faces + face) · mipLevels + mip. + * Errors are YapfError with .code 'not-yapf', 'version', 'unsupported', + * 'corrupt', 'checksum' or 'truncated'. + */ + function decodeTexture(input, opts) { + const b = input instanceof Uint8Array ? input : new Uint8Array(input); + const verify = !(opts && opts.verify === false); + if (b.length < 5 || b[0] !== 0x59 || b[1] !== 0x41 || b[2] !== 0x50 || b[3] !== 0x46) + fail('not-yapf', 'not a YAPF file'); + if (b[4] === 1) { + const img = decodeV1(b); + return { version: 1, width: img.width, height: img.height, depth: 1, layers: 1, frames: 1, + faces: 1, shape: 0, mipLevels: img.mipLevels, payload: 0, format: img.gpuFormat, + sample: 0, channels: img.channels, flags: img.flags, channelMap: [0, 0, 0, 0], + linearMask: 0, normal: 0, primaries: 0, transfer: 0, mipFilter: 0, alphaRef: 0, + loopCount: 0, frameUs: [], notes: [], surfaces: img.mips }; + } + if (b[4] !== 2) fail('version', 'format version ' + b[4] + ' is newer than this decoder'); + if (b.length < 64) fail('truncated', 'the file is truncated'); + if (verify && checksum(b, 0, 0x3C) !== r32(b, 0x3C)) fail('checksum', 'damaged file: a checksum does not match'); + const t = { + version: 2, channels: b[5], format: b[6], mipLevels: b[7], flags: b[8], payload: b[9], + sample: b[10], shape: b[11], width: r32(b, 0x0C), height: r32(b, 0x10), depth: r32(b, 0x14), + layers: r16(b, 0x18), frames: r16(b, 0x1A), needs: r32(b, 0x1C), + channelMap: Array.from(b.subarray(0x20, 0x24)), linearMask: b[0x24], normal: b[0x25], + primaries: b[0x26], transfer: b[0x27], mipFilter: b[0x28], alphaRef: b[0x29], + loopCount: r16(b, 0x2A), frameUs: [], notes: [] + }; + const notesSize = r32(b, 0x2C), fileSize = r64(b, 0x30); + if (t.needs & ~NEED.ALL) fail('unsupported', 'the file needs a feature this decoder does not have'); + t.faces = t.shape === 1 ? 6 : 1; + if (t.channels < 1 || t.channels > 4 || t.sample > 3 || t.payload > 1 || t.shape > 2 || + t.width < 1 || t.width > MAX_DIM || t.height < 1 || t.height > MAX_DIM || + !t.layers || !t.frames || t.mipLevels < 1 || t.mipLevels > MAX_MIPS) bad(); + if (t.shape === 2 ? (t.depth < 1 || t.depth > MAX_DIM || t.layers !== 1) : t.depth > 1) bad(); + if (t.shape !== 2) t.depth = 1; + if (t.shape === 1 && t.width !== t.height) bad(); + if (t.payload === 1 && !BLOCK_FORMATS[t.format]) bad(); + if ((t.payload === 0 && t.sample !== 0 && !(t.needs & NEED.WIDE)) || + (t.payload === 1 && !(t.needs & NEED.BLOCKS)) || + ((t.shape !== 0 || t.layers > 1) && !(t.needs & NEED.LAYOUT)) || + (t.frames > 1 && !(t.needs & NEED.FRAMES))) bad(); + const S = t.frames * t.layers * t.faces * t.mipLevels; + if (S > (1 << 24)) bad(); + const front = 64 + 16 * S + (t.frames > 1 ? 4 * t.frames : 0) + notesSize; + if (front > fileSize) bad(); + if (b.length < fileSize) fail('truncated', 'the file is truncated'); + if (verify && checksum(b, 64, front - 64) !== r32(b, 0x38)) fail('checksum', 'damaged file: a checksum does not match'); + let q = 64 + 16 * S; + for (let f = 0; t.frames > 1 && f < t.frames; f++, q += 4) t.frameUs.push(r32(b, q)); + const qe = q + notesSize; + while (q < qe) { + if (qe - q < 8) bad(); + const sz = r32(b, q + 4); + if (qe - q - 8 < sz) bad(); + t.notes.push({ tag: String.fromCharCode(b[q], b[q + 1], b[q + 2], b[q + 3]), data: b.slice(q + 8, q + 8 + sz) }); + q += 8 + sz; + } + t._off = []; t._size = []; t._tcheck = []; + for (let s = 0; s < S; s++) { + const e = 64 + 16 * s, off = r64(b, e), size = r32(b, e + 8); + if (off < front || off > fileSize || size > fileSize - off) bad(); + t._off.push(off); t._size.push(size); t._tcheck.push(r32(b, e + 12)); + } + t.surfaces = []; + for (let s = 0; s < S; s++) t.surfaces.push(decodeSurface(b, t, s, verify)); + delete t._off; delete t._size; delete t._tcheck; + return t; + } + + /* ── GPU block decoding (previews) ───────────────────────────────── */ + + const PART2 = [0xCCCC, 0x8888, 0xEEEE, 0xECC8, 0xC880, 0xFEEC, 0xFEC8, 0xEC80, 0xC800, 0xFFEC, 0xFE80, 0xE800, + 0xFFE8, 0xFF00, 0xFFF0, 0xF000, 0xF710, 0x008E, 0x7100, 0x08CE, 0x008C, 0x7310, 0x3100, 0x8CCE, 0x088C, 0x3110, + 0x6666, 0x366C, 0x17E8, 0x0FF0, 0x718E, 0x399C, 0xAAAA, 0xF0F0, 0x5A5A, 0x33CC, 0x3C3C, 0x55AA, 0x9696, 0xA55A, + 0x73CE, 0x13C8, 0x324C, 0x3BDC, 0x6996, 0xC33C, 0x9966, 0x0660, 0x0272, 0x04E4, 0x4E40, 0x2720, 0xC936, 0x936C, + 0x39C6, 0x639C, 0x9336, 0x9CC6, 0x817E, 0xE718, 0xCCF0, 0x0FCC, 0x7744, 0xEE22]; + const PART3 = [0xAA685050, 0x6A5A5040, 0x5A5A4200, 0x5450A0A8, 0xA5A50000, 0xA0A05050, 0x5555A0A0, 0x5A5A5050, + 0xAA550000, 0xAA555500, 0xAAAA5500, 0x90909090, 0x94949494, 0xA4A4A4A4, 0xA9A59450, 0x2A0A4250, 0xA5945040, + 0x0A425054, 0xA5A5A500, 0x55A0A0A0, 0xA8A85454, 0x6A6A4040, 0xA4A45000, 0x1A1A0500, 0x0050A4A4, 0xAAA59090, + 0x14696914, 0x69691400, 0xA08585A0, 0xAA821414, 0x50A4A450, 0x6A5A0200, 0xA9A58000, 0x5090A0A8, 0xA8A09050, + 0x24242424, 0x00AA5500, 0x24924924, 0x24499224, 0x50A50A50, 0x500AA550, 0xAAAA4444, 0x66660000, 0xA5A0A5A0, + 0x50A050A0, 0x69286928, 0x44AAAA44, 0x66666600, 0xAA444444, 0x54A854A8, 0x95809580, 0x96969600, 0xA85454A8, + 0x80959580, 0xAA141414, 0x96960000, 0xAAAA1414, 0xA05050A0, 0xA0A5A5A0, 0x96000000, 0x40804080, 0xA9A8A9A8, + 0xAAAAAA44, 0x2A4A5254]; + const ANCHOR2 = [15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 2, 8, 2, 2, 8, 8, 15, 2, 8, 2, 2, + 8, 8, 2, 2, 15, 15, 6, 8, 2, 8, 15, 15, 2, 8, 2, 2, 2, 15, 15, 6, 6, 2, 6, 8, 15, 15, 2, 2, 15, 15, 15, 15, 15, 2, 2, 15]; + const ANCHOR3A = [3, 3, 15, 15, 8, 3, 15, 15, 8, 8, 6, 6, 6, 5, 3, 3, 3, 3, 8, 15, 3, 3, 6, 10, 5, 8, 8, 6, 8, 5, 15, + 15, 8, 15, 3, 5, 6, 10, 8, 15, 15, 3, 15, 5, 15, 15, 15, 15, 3, 15, 5, 5, 5, 8, 5, 10, 5, 10, 8, 13, 15, 12, 3, 3]; + const ANCHOR3B = [15, 8, 8, 3, 15, 15, 3, 8, 15, 15, 15, 15, 15, 15, 15, 8, 15, 8, 15, 3, 15, 8, 15, 8, 3, 15, 6, 10, + 15, 15, 10, 8, 15, 3, 15, 10, 10, 8, 9, 10, 6, 15, 8, 15, 3, 6, 6, 8, 15, 3, 15, 15, 15, 15, 15, 15, 15, 15, 15, + 15, 3, 15, 15, 8]; + const WEIGHTS = { 2: [0, 21, 43, 64], 3: [0, 9, 18, 27, 37, 46, 55, 64], + 4: [0, 4, 9, 13, 17, 21, 26, 30, 34, 38, 43, 47, 51, 55, 60, 64] }; + const interp = (a, b, w) => (a * (64 - w) + b * w + 32) >> 6; + + function bits128(b, o) { + let pos = 0; + return (n) => { + let v = 0; + for (let i = 0; i < n; i++, pos++) v |= ((b[o + (pos >> 3)] >> (pos & 7)) & 1) << i; + return v >>> 0; + }; + } + + function rgb565(c) { + const r = (c >> 11) & 31, g = (c >> 5) & 63, bl = c & 31; + return [(r << 3) | (r >> 2), (g << 2) | (g >> 4), (bl << 3) | (bl >> 2)]; + } + + function bc1Block(b, o, out, fourOnly) { + const c0 = r16(b, o), c1 = r16(b, o + 2), idx = r32(b, o + 4), four = c0 > c1 || fourOnly; + const p0 = rgb565(c0), p1 = rgb565(c1), pal = [p0.concat(255), p1.concat(255), [0, 0, 0, 255], [0, 0, 0, 255]]; + for (let k = 0; k < 3; k++) { + if (four) { pal[2][k] = ((2 * p0[k] + p1[k] + 1) / 3) | 0; pal[3][k] = ((p0[k] + 2 * p1[k] + 1) / 3) | 0; } + else { pal[2][k] = (p0[k] + p1[k] + 1) >> 1; pal[3][k] = 0; } + } + if (!four) pal[3][3] = 0; + for (let i = 0; i < 16; i++) { const c = pal[(idx >>> (2 * i)) & 3]; for (let k = 0; k < 4; k++) out[i * 4 + k] = c[k]; } + } + + function bc4Block(b, o, out, ch, snorm) { + const pal = new Array(8); + let r0 = b[o], r1 = b[o + 1]; + if (snorm) { r0 = Math.max(-127, (r0 << 24) >> 24); r1 = Math.max(-127, (r1 << 24) >> 24); } + pal[0] = r0; pal[1] = r1; + const div = (v, d) => (v >= 0 ? Math.floor((v + (d >> 1)) / d) : -Math.floor((-v + (d >> 1)) / d)); + if (r0 > r1) for (let i = 2; i < 8; i++) pal[i] = div((8 - i) * r0 + (i - 1) * r1, 7); + else { for (let i = 2; i < 6; i++) pal[i] = div((6 - i) * r0 + (i - 1) * r1, 5); pal[6] = snorm ? -127 : 0; pal[7] = snorm ? 127 : 255; } + let lo = r32(b, o + 2), hi = r16(b, o + 6); + for (let i = 0; i < 16; i++) { + const bit = 3 * i, v = bit < 30 ? (lo >>> bit) & 7 : bit === 30 ? ((lo >>> 30) | ((hi & 1) << 2)) : (hi >> (bit - 32)) & 7; + out[i * 4 + ch] = pal[v] + (snorm ? 128 : 0); + } + } + + const BC7 = [[3, 4, 0, 0, 4, 0, 1, 0, 3, 0], [2, 6, 0, 0, 6, 0, 0, 1, 3, 0], [3, 6, 0, 0, 5, 0, 0, 0, 2, 0], + [2, 6, 0, 0, 7, 0, 1, 0, 2, 0], [1, 0, 2, 1, 5, 6, 0, 0, 2, 3], [1, 0, 2, 0, 7, 8, 0, 0, 2, 2], + [1, 0, 0, 0, 7, 7, 1, 0, 4, 0], [2, 6, 0, 0, 5, 5, 1, 0, 2, 0]]; + const subsetOf = (ns, part, i) => ns === 2 ? (PART2[part] >> i) & 1 : ns === 3 ? (PART3[part] >>> (2 * i)) & 3 : 0; + const isAnchor = (ns, part, i) => i === 0 || (ns === 2 && i === ANCHOR2[part]) || + (ns === 3 && (i === ANCHOR3A[part] || i === ANCHOR3B[part])); + + function bc7Block(b, o, out) { + const get = bits128(b, o); + let mode = 0; + while (mode < 8 && !get(1)) mode++; + if (mode === 8) { out.fill(0, 0, 64); return; } + const [ns, pb, rb, isbB, cb, ab, epb, spb, ib, ib2] = BC7[mode]; + const part = get(pb), rot = get(rb), isb = get(isbB), ne = ns * 2; + const ep = []; for (let e = 0; e < ne; e++) ep.push([0, 0, 0, 255]); + for (let c = 0; c < 3; c++) for (let e = 0; e < ne; e++) ep[e][c] = get(cb); + for (let e = 0; e < ne; e++) ep[e][3] = ab ? get(ab) : 255; + const pbit = epb | spb; + if (epb) for (let e = 0; e < ne; e++) { const p = get(1); for (let c = 0; c < 4; c++) if (c < 3 || ab) ep[e][c] = (ep[e][c] << 1) | p; } + else if (spb) for (let k = 0; k < ns; k++) { const p = get(1); for (let c = 0; c < 3; c++) { ep[2 * k][c] = (ep[2 * k][c] << 1) | p; ep[2 * k + 1][c] = (ep[2 * k + 1][c] << 1) | p; } } + for (let e = 0; e < ne; e++) for (let c = 0; c < 4; c++) { + if (c === 3 && !ab) continue; + const nb = (c < 3 ? cb : ab) + pbit, v = ep[e][c] << (8 - nb); + ep[e][c] = v | (v >> nb); + } + const idx = [], idx2 = []; + for (let i = 0; i < 16; i++) idx.push(get(ib - (isAnchor(ns, part, i) ? 1 : 0))); + for (let i = 0; i < 16; i++) idx2.push(ib2 ? get(ib2 - (i === 0 ? 1 : 0)) : 0); + const w1 = WEIGHTS[ib], w2 = WEIGHTS[ib2 || 2]; + for (let i = 0; i < 16; i++) { + const k = subsetOf(ns, part, i), A = ep[2 * k], B = ep[2 * k + 1]; + let cw, aw; + if (!ib2) cw = aw = w1[idx[i]]; + else if (!isb) { cw = w1[idx[i]]; aw = w2[idx2[i]]; } + else { cw = w2[idx2[i]]; aw = w1[idx[i]]; } + const px = [interp(A[0], B[0], cw), interp(A[1], B[1], cw), interp(A[2], B[2], cw), interp(A[3], B[3], aw)]; + if (rot) { const t = px[3]; px[3] = px[rot - 1]; px[rot - 1] = t; } + for (let c = 0; c < 4; c++) out[i * 4 + c] = px[c]; + } + } + + /* BC6H header layouts: [field, first bit, count]; fields 0–3 red w x y z, + * 4–7 green, 8–11 blue, 12 partition. */ + const RW = 0, RX = 1, RY = 2, RZ = 3, GW = 4, GX = 5, GY = 6, GZ = 7, BW = 8, BX = 9, BY = 10, BZ = 11, PD = 12; + const BC6 = [ + [0x00, 10, 5, 5, 5, [[GY,4,1],[BY,4,1],[BZ,4,1],[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,5],[GZ,4,1],[GY,0,4],[GX,0,5],[BZ,0,1],[GZ,0,4],[BX,0,5],[BZ,1,1],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], + [0x01, 7, 6, 6, 6, [[GY,5,1],[GZ,4,1],[GZ,5,1],[RW,0,7],[BZ,0,1],[BZ,1,1],[BY,4,1],[GW,0,7],[BY,5,1],[BZ,2,1],[GY,4,1],[BW,0,7],[BZ,3,1],[BZ,5,1],[BZ,4,1],[RX,0,6],[GY,0,4],[GX,0,6],[GZ,0,4],[BX,0,6],[BY,0,4],[RY,0,6],[RZ,0,6],[PD,0,5]]], + [0x02, 11, 5, 4, 4, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,5],[RW,10,1],[GY,0,4],[GX,0,4],[GW,10,1],[BZ,0,1],[GZ,0,4],[BX,0,4],[BW,10,1],[BZ,1,1],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], + [0x06, 11, 4, 5, 4, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,4],[RW,10,1],[GZ,4,1],[GY,0,4],[GX,0,5],[GW,10,1],[GZ,0,4],[BX,0,4],[BW,10,1],[BZ,1,1],[BY,0,4],[RY,0,4],[BZ,0,1],[BZ,2,1],[RZ,0,4],[GY,4,1],[BZ,3,1],[PD,0,5]]], + [0x0A, 11, 4, 4, 5, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,4],[RW,10,1],[BY,4,1],[GY,0,4],[GX,0,4],[GW,10,1],[BZ,0,1],[GZ,0,4],[BX,0,5],[BW,10,1],[BY,0,4],[RY,0,4],[BZ,1,1],[BZ,2,1],[RZ,0,4],[BZ,4,1],[BZ,3,1],[PD,0,5]]], + [0x0E, 9, 5, 5, 5, [[RW,0,9],[BY,4,1],[GW,0,9],[GY,4,1],[BW,0,9],[BZ,4,1],[RX,0,5],[GZ,4,1],[GY,0,4],[GX,0,5],[BZ,0,1],[GZ,0,4],[BX,0,5],[BZ,1,1],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], + [0x12, 8, 6, 5, 5, [[RW,0,8],[GZ,4,1],[BY,4,1],[GW,0,8],[BZ,2,1],[GY,4,1],[BW,0,8],[BZ,3,1],[BZ,4,1],[RX,0,6],[GY,0,4],[GX,0,5],[BZ,0,1],[GZ,0,4],[BX,0,5],[BZ,1,1],[BY,0,4],[RY,0,6],[RZ,0,6],[PD,0,5]]], + [0x16, 8, 5, 6, 5, [[RW,0,8],[BZ,0,1],[BY,4,1],[GW,0,8],[GY,5,1],[GY,4,1],[BW,0,8],[GZ,5,1],[BZ,4,1],[RX,0,5],[GZ,4,1],[GY,0,4],[GX,0,6],[GZ,0,4],[BX,0,5],[BZ,1,1],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], + [0x1A, 8, 5, 5, 6, [[RW,0,8],[BZ,1,1],[BY,4,1],[GW,0,8],[BY,5,1],[GY,4,1],[BW,0,8],[BZ,5,1],[BZ,4,1],[RX,0,5],[GZ,4,1],[GY,0,4],[GX,0,5],[BZ,0,1],[GZ,0,4],[BX,0,6],[BY,0,4],[RY,0,5],[BZ,2,1],[RZ,0,5],[BZ,3,1],[PD,0,5]]], + [0x1E, 6, 6, 6, 6, [[RW,0,6],[GZ,4,1],[BZ,0,1],[BZ,1,1],[BY,4,1],[GW,0,6],[GY,5,1],[BY,5,1],[BZ,2,1],[GY,4,1],[BW,0,6],[GZ,5,1],[BZ,3,1],[BZ,5,1],[BZ,4,1],[RX,0,6],[GY,0,4],[GX,0,6],[GZ,0,4],[BX,0,6],[BY,0,4],[RY,0,6],[RZ,0,6],[PD,0,5]]], + [0x03, 10, 10, 10, 10, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,10],[GX,0,10],[BX,0,10]]], + [0x07, 11, 9, 9, 9, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,9],[RW,10,1],[GX,0,9],[GW,10,1],[BX,0,9],[BW,10,1]]], + [0x0B, 12, 8, 8, 8, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,8],[RW,11,1],[RW,10,1],[GX,0,8],[GW,11,1],[GW,10,1],[BX,0,8],[BW,11,1],[BW,10,1]]], + [0x0F, 16, 4, 4, 4, [[RW,0,10],[GW,0,10],[BW,0,10],[RX,0,4],[RW,15,1],[RW,14,1],[RW,13,1],[RW,12,1],[RW,11,1],[RW,10,1],[GX,0,4],[GW,15,1],[GW,14,1],[GW,13,1],[GW,12,1],[GW,11,1],[GW,10,1],[BX,0,4],[BW,15,1],[BW,14,1],[BW,13,1],[BW,12,1],[BW,11,1],[BW,10,1]]] + ]; + const sext = (v, n) => { const m = 1 << (n - 1); v &= (1 << n) - 1; return (v ^ m) - m; }; + function bc6Unq(v, n, sgn) { + if (!sgn) { + if (n >= 15) return v; + if (v === 0) return 0; + if (v === (1 << n) - 1) return 0xFFFF; + return ((v << 16) + 0x8000) >> n; + } + if (n >= 16) return v; + const s = v < 0; if (s) v = -v; + const u = v === 0 ? 0 : v >= (1 << (n - 1)) - 1 ? 0x7FFF : ((v << 15) + 0x4000) >> (n - 1); + return s ? -u : u; + } + function bc6Finish(v, sgn) { + if (!sgn) return (v * 31) >> 6; + v = v < 0 ? -(((-v) * 31) >> 5) : (v * 31) >> 5; + return v < 0 ? 0x8000 | -v : v; + } + function bc6Block(b, o, out, sgn) { + const get = bits128(b, o); + let code = get(2); + if (code > 1) code |= get(3) << 2; + const m = BC6.findIndex((x) => x[0] === code); + if (m < 0) { out.fill(0, 0, 48); return; } + const [, W, dr, dg, db, seg] = BC6[m], f = new Array(13).fill(0); + for (const [fi, lo, n] of seg) f[fi] |= get(n) << lo; + const regions = m < 10 ? 2 : 1, ne = regions * 2, dbits = [dr, dg, db], ep = [[], [], [], []]; + for (let c = 0; c < 3; c++) { + for (let e = 0; e < ne; e++) ep[e][c] = f[c * 4 + e]; + if (sgn) ep[0][c] = sext(ep[0][c], W); + const delta = m !== 9 && m !== 10; + for (let e = 1; e < ne; e++) { + if (delta || sgn) ep[e][c] = sext(ep[e][c], dbits[c]); + if (delta) { ep[e][c] = (ep[e][c] + ep[0][c]) & ((1 << W) - 1); if (sgn) ep[e][c] = sext(ep[e][c], W); } + } + for (let e = 0; e < ne; e++) ep[e][c] = bc6Unq(ep[e][c], W, sgn); + } + const part = f[PD], ib = regions === 2 ? 3 : 4, wt = WEIGHTS[ib]; + for (let i = 0; i < 16; i++) { + const k = regions === 2 ? (PART2[part] >> i) & 1 : 0; + const anchor = i === 0 || (regions === 2 && i === ANCHOR2[part]); + const ix = get(ib - (anchor ? 1 : 0)); + for (let c = 0; c < 3; c++) out[i * 3 + c] = bc6Finish(interp(ep[2 * k][c], ep[2 * k + 1][c], wt[ix]), sgn); + } + } + + /* decodeBlocks(format, blocks, w, h) → RGBA Uint8Array (LDR) or RGB + * half-bits Uint16Array (BC6H); null for ETC2 / EAC / ASTC. */ + function decodeBlocks(format, blocks, w, h) { + if (format >= 0x50 || !BLOCK_FORMATS[format]) return null; + const S = BLOCK_FORMATS[format][2], nbx = Math.ceil(w / 4), nby = Math.ceil(h / 4); + const hdr = format === 0x47 || format === 0x48; + const out = hdr ? new Uint16Array(w * h * 3) : new Uint8Array(w * h * 4); + const px = new Int32Array(64), hx = new Int32Array(48); + let o = 0; + for (let by = 0; by < nby; by++) + for (let bx = 0; bx < nbx; bx++, o += S) { + switch (format) { + case 0x40: bc1Block(blocks, o, px, false); break; + case 0x41: + bc1Block(blocks, o + 8, px, true); + for (let i = 0; i < 16; i++) px[i * 4 + 3] = ((blocks[o + (i >> 1)] >> (4 * (i & 1))) & 15) * 17; + break; + case 0x42: bc1Block(blocks, o + 8, px, true); bc4Block(blocks, o, px, 3, false); break; + case 0x43: case 0x44: + px.fill(0); bc4Block(blocks, o, px, 0, format === 0x44); + for (let i = 0; i < 16; i++) px[i * 4 + 3] = 255; + break; + case 0x45: case 0x46: + px.fill(0); bc4Block(blocks, o, px, 0, format === 0x46); bc4Block(blocks, o + 8, px, 1, format === 0x46); + for (let i = 0; i < 16; i++) px[i * 4 + 3] = 255; + break; + case 0x49: bc7Block(blocks, o, px); break; + default: bc6Block(blocks, o, hx, format === 0x48); + } + for (let y = 0; y < 4 && by * 4 + y < h; y++) + for (let x = 0; x < 4 && bx * 4 + x < w; x++) { + const d = (by * 4 + y) * w + bx * 4 + x, i = y * 4 + x; + if (hdr) for (let c = 0; c < 3; c++) out[d * 3 + c] = hx[i * 3 + c]; + else for (let c = 0; c < 4; c++) out[d * 4 + c] = px[i * 4 + c]; + } + } + return out; + } + + function halfToFloat(h) { + const s = h & 0x8000 ? -1 : 1, e = (h >> 10) & 31, m = h & 0x3FF; + if (e === 0) return s * m * Math.pow(2, -24); + if (e === 31) return m ? NaN : s * Infinity; + return s * (1 + m / 1024) * Math.pow(2, e - 15); + } + + const linToSrgb = (v) => !(v > 0) ? 0 : v >= 1 ? 255 + : Math.round(255 * (v <= 0.0031308 ? v * 12.92 : 1.055 * Math.pow(v, 1 / 2.4) - 0.055)); + + /* + * textureToRGBA(tex, surface = 0, slice = 0) → { width, height, + * rgba: Uint8ClampedArray } for display: gray expanded, 16-bit rounded, + * half / float clamped and sRGB encoded, GPU blocks decoded. Throws for + * ETC2 / EAC / ASTC. + */ + function textureToRGBA(tex, surface, slice) { + surface = surface || 0; slice = slice || 0; + const m = surface % tex.mipLevels, [w, h, d] = mipSize(tex, m), n = w * h; + const data = tex.surfaces[surface], rgba = new Uint8ClampedArray(n * 4); + let src, ch = tex.channels, kind = tex.payload === 1 ? 'u8' : ['u8', 'u16', 'f16', 'f32'][tex.sample]; + if (tex.payload === 1) { + const per = data.length / d, blk = decodeBlocks(tex.format, data.subarray(per * slice, per * (slice + 1)), w, h); + if (!blk) fail('unsupported', 'ETC2 / EAC / ASTC blocks cannot be shown'); + if (blk instanceof Uint16Array) { src = blk; ch = 3; kind = 'f16'; } + else { + rgba.set(blk); + if (tex.channels === 3) for (let i = 0; i < n; i++) rgba[i * 4 + 3] = 255; + return { width: w, height: h, rgba }; + } + } else { + src = data.subarray(n * ch * slice, n * ch * (slice + 1)); + } + for (let i = 0; i < n; i++) { + const v = [0, 0, 0, 255]; + for (let c = 0; c < ch; c++) { + const x = src[i * ch + c], alpha = (ch === 2 && c === 1) || (ch === 4 && c === 3); + let q; + if (kind === 'u8') q = x; + else if (kind === 'u16') q = Math.round(x / 257); + else { + const f = kind === 'f16' ? halfToFloat(x) : x; + q = alpha ? Math.max(0, Math.min(255, Math.round(f * 255))) : linToSrgb(f); + } + v[ch <= 2 ? (c === 0 ? 0 : 3) : c] = q; + } + if (ch <= 2) v[1] = v[2] = v[0]; + rgba.set(v, i * 4); + } + return { width: w, height: h, rgba }; + } + + function surfaceIndex(tex, frame, layer, face, mip) { + return (((frame || 0) * tex.layers + (layer || 0)) * tex.faces + (face || 0)) * tex.mipLevels + (mip || 0); + } /* ── Decoder ──────────────────────────────────────────────────────── */ @@ -102,7 +776,7 @@ R[r0 + tw] = R[r0 + tw - 1]; } - function decodeTile(b, p, end, out, imgW, tx, ty, tw, th, ch) { + function decodeTile(b, p, end, out, imgW, tx, ty, tw, th, ch, v2, book) { if (end <= p) corrupt(); let hdr = b[p++]; if (hdr === STORED) { @@ -112,10 +786,18 @@ out.set(b.subarray(p + rowb * y, p + rowb * (y + 1)), ((ty + y) * imgW + tx) * ch); return; } - if (hdr & 0xE0) corrupt(); /* reserved bits set */ - const xf = hdr & SUBGREEN; + if (hdr & (v2 ? 0x80 : 0xE0)) corrupt(); /* reserved bits set */ + if ((hdr & 0x40) && (hdr & SUBGREEN)) corrupt(); + /* xf: 0 YCoCg-R, 1 subtract-green, 2 none (v2) */ + const xf = (hdr & 0x40) ? 2 : (hdr & SUBGREEN) ? 1 : 0, ent = hdr & 0x20; hdr &= 0x0F; if ((hdr >> ch) || (xf && ch < 3)) corrupt(); + let emask = 0; + if (ent) { /* v2: entropy planes */ + if (p >= end || !book) corrupt(); + emask = b[p++]; + if ((emask >> ch) || (emask & hdr)) corrupt(); + } const rowLen = TILE + 1, planeLen = RING * rowLen; const rows = new Uint8Array(4 * planeLen); @@ -128,7 +810,8 @@ for (let c = 0; c < ch; c++) { if (hdr & (1 << c)) continue; z[c] = new Uint8Array(TILE_PX + GROUP); - const r = unpackPlane(b, p, end, z[c], tw, th); + const r = (emask & (1 << c)) ? huffPlane(b, p, end, z[c], tw, th, book, xf * 16 + c) + : unpackPlane(b, p, end, z[c], tw, th); p = r.p; filt[c] = r.filt; } if (p !== end) corrupt(); @@ -147,7 +830,11 @@ const P0 = ri, P1 = planeLen + ri, P2 = 2 * planeLen + ri, P3 = 3 * planeLen + ri; for (let x = 0; x < tw; x++, d += ch) { if (ch >= 3) { - if (xf) { + if (xf === 2) { + out[d] = rows[P0 + x]; + out[d + 1] = rows[P1 + x]; + out[d + 2] = rows[P2 + x]; + } else if (xf) { const g = rows[P0 + x]; out[d] = rows[P1 + x] + g - 128; out[d + 1] = g; @@ -171,11 +858,27 @@ const r32 = (b, p) => (b[p] | (b[p + 1] << 8) | (b[p + 2] << 16) | (b[p + 3] << 24)) >>> 0; + /* decode(bytes) → 8-bit image. Reads version 1, and version 2 files + * that hold 8-bit pixels (2D; cube face 0 / layer 0 / frame 0). Other + * textures: see decodeTexture. */ function decode(input) { const b = input instanceof Uint8Array ? input : new Uint8Array(input); + if (b.length >= 5 && b[0] === 0x59 && b[1] === 0x41 && b[2] === 0x50 && b[3] === 0x46 && b[4] === 2) { + const t = decodeTexture(b); + if (t.payload !== 0 || t.sample !== 0 || t.shape === 2) + throw new YapfError('unsupported', 'this is a YAPF 2 texture (GPU blocks, 16-bit / float or a ' + + 'volume); use YAPF.decodeTexture and YAPF.textureToRGBA'); + const mips = t.surfaces.slice(0, t.mipLevels); + return { width: t.width, height: t.height, channels: t.channels, gpuFormat: t.format, + flags: t.flags, mipLevels: t.mipLevels, pixels: mips[0], mips }; + } + return decodeV1(b); + } + + function decodeV1(b) { if (b.length < 20 || b[0] !== 0x59 || b[1] !== 0x41 || b[2] !== 0x50 || b[3] !== 0x46) - throw new Error('YAPF: not a YAPF file'); - if (b[4] !== 1) throw new Error('YAPF: unsupported format version ' + b[4]); + throw new YapfError('not-yapf', 'not a YAPF file'); + if (b[4] !== 1) throw new YapfError('version', 'unsupported format version ' + b[4]); const ch = b[5], mipLevels = b[7], W = r32(b, 12), H = r32(b, 16); if (ch < 1 || ch > 4 || mipLevels < 1 || mipLevels > MAX_MIPS || W < 1 || W > MAX_DIM || H < 1 || H > MAX_DIM) corrupt(); @@ -459,5 +1162,7 @@ return out; } - return { decode, encode, toRGBA, GPU, FLAG, VERSION: 1 }; + return { decode, encode, toRGBA, decodeTexture, textureToRGBA, decodeBlocks, surfaceIndex, + checksum: (b) => checksum(b, 0, b.length), YapfError, GPU, FLAG, NEED, + VERSION: 1 /* written by encode */, READS: [1, 2] }; })); diff --git a/extensions/figma/README.md b/extensions/figma/README.md index 487853f..ac80b84 100644 --- a/extensions/figma/README.md +++ b/extensions/figma/README.md @@ -9,6 +9,12 @@ Both directions are pixel-exact: the plugin carries its own small PNG encoder/decoder instead of going through a canvas, which would alter semi-transparent pixels. +Import reads every YAPF file, version 1 and 2. 8-bit images keep their +exact pixels; GPU blocks (BC1–BC7, BC6H), 16-bit and float textures are +converted to 8-bit for display (Figma images are 8-bit); cubes, arrays, +volumes and animations bring their first image. Damaged files are +reported as such. + ## Install (development plugin) 1. Use the **Figma desktop app**. diff --git a/extensions/figma/src/ui.src.html b/extensions/figma/src/ui.src.html index ca131a0..e804d73 100644 --- a/extensions/figma/src/ui.src.html +++ b/extensions/figma/src/ui.src.html @@ -64,14 +64,26 @@ $('tab-import').onclick = () => tab('import'); $('tab-export').onclick = () => tab('export'); /* Import: .yapf → pixels → exact PNG → Figma image */ +/* Any YAPF file → 8-bit pixels. 8-bit images keep their exact pixels and + * channels; GPU blocks, 16-bit and float textures are converted for display + * (RGBA). Cubes, arrays, volumes and animations give their first image. */ +function yapfToPixels(tex) { + const w = tex.width, h = tex.height; + if (tex.payload === 0 && tex.sample === 0) + return { width: w, height: h, channels: tex.channels, + pixels: tex.surfaces[0].subarray(0, w * h * tex.channels) }; + const v = YAPF.textureToRGBA(tex, 0, 0); + return { width: w, height: h, channels: 4, pixels: new Uint8Array(v.rgba.buffer), + note: tex.payload === 1 ? 'GPU blocks decoded' : 'converted to 8-bit' }; +} async function importFiles(files) { const images = []; for (const f of files) { try { - const img = YAPF.decode(new Uint8Array(await f.arrayBuffer())); + const img = yapfToPixels(YAPF.decodeTexture(new Uint8Array(await f.arrayBuffer()))); const png = await PNG.encode(img.pixels, img.width, img.height, img.channels); images.push({ name: f.name.replace(/\.yapf$/i, ''), width: img.width, height: img.height, png }); - status('Decoded ' + f.name + ' (' + img.width + '×' + img.height + ')'); + status('Decoded ' + f.name + ' (' + img.width + '×' + img.height + (img.note ? ', ' + img.note : '') + ')'); } catch (e) { status(f.name + ': ' + e.message); } diff --git a/extensions/figma/ui.html b/extensions/figma/ui.html index 0b419ac..167bd23 100644 --- a/extensions/figma/ui.html +++ b/extensions/figma/ui.html @@ -47,7 +47,7 @@