mirror of
https://github.com/Nexoniarz/YAPF.git
synced 2026-11-03 04:54:26 +00:00
YAPF v1: fast byte-aligned format, SIMD, tools and plugins
Format (version 1, replaces the earlier experimental versions): - per-tile YCoCg-R or subtract-green colour transform, 8-bit modular - 12 row filters per 8-row band (Left, Gradient, Up-right, Up-average, Up 1-8 rows), none with more than a one-add serial dependency - residuals in groups of 8 at the group's width (w bits x 8 = w bytes), 4-bit control codes with repeat-previous-width runs chosen by DP - constant and stored tiles, independent 64x64 tiles, mips, GPU hints - all sizes validated before decoding; corrupt input is rejected Library: - SIMD on by default: SSSE3 (runtime CPUID check; GCC, Clang, MSVC, MinGW) and NEON on ARM64; -DYAPF_NO_SIMD for plain C - yapf_load_memory, yapf_load_mt, yapf_load_memory_mt - 6-10x faster decoding than PNG (stb_image) on one core; -O2 and -O3 builds perform the same - fixes for macOS (sysconf) and the Windows import library exports Tools and integrations: - `yapf` CLI: convert PNG/JPG/TGA/BMP/PSD/GIF <-> YAPF, info, bench, thumbnail, mips - JavaScript and Python bindings (byte-identical to the C encoder) - GIMP 3 plugin: exports the flattened image, optional mipmaps - Blender extension: import (also as plane), export, drag and drop - VS Code preview, Figma plugin, Photoshop scripts, Linux MIME type and thumbnailer - Makefile, updated build.sh / build.bat, tools/build_release.sh - tests/test_yapf.c (784 checks) and bindings/js/test.js Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Nexoniarz
parent
b9b8c93a22
commit
2ca726e3af
48 changed files with 14905 additions and 710 deletions
131
yapf.h
131
yapf.h
|
|
@ -20,21 +20,30 @@
|
|||
*
|
||||
* YAPF — lossless, GPU-ready image format.
|
||||
*
|
||||
* Space savings come from two cheap mathematical steps — no entropy coder,
|
||||
* no lookup tables, no opcode dispatch. Decode is straight arithmetic:
|
||||
* Designed for single-core decode speed. Everything is byte aligned:
|
||||
* no bit reader, no entropy coder, no context model. Per row the decoder
|
||||
* dispatches one filter; per 8 samples it reads a 4-bit control code and
|
||||
* exactly w bytes of payload, unpacked with one 64-bit load and shifts.
|
||||
* The encoder does the hard work (filter search, colour transform choice,
|
||||
* optimal control codes); the decoder only follows instructions.
|
||||
*
|
||||
* 1. YCoCg-R color transform (RGB↔Y/Co/Cg, fully reversible integers)
|
||||
* 2. 2-D neighbor prediction + per-tile adaptive bit-packing
|
||||
* 1. Per tile: YCoCg-R or subtract-green colour transform, 8-bit modular
|
||||
* 2. Per band of 8 rows: one of 12 row filters
|
||||
* 3. Residuals in groups of 8, each group at its bit width (0–8);
|
||||
* 4-bit control codes, with codes that repeat the previous width
|
||||
* 4. Independent 64×64 tiles with a size table → random access and
|
||||
* optional parallel decode
|
||||
*
|
||||
* The file carries GPU metadata (format hint, mip levels, flags) so a
|
||||
* loader can upload the decoded buffer straight to a texture without any
|
||||
* additional transcoding step.
|
||||
*
|
||||
* =========================================================================
|
||||
* FILE FORMAT (version 2)
|
||||
* FILE FORMAT (version 1)
|
||||
* =========================================================================
|
||||
*
|
||||
* All multi-byte integers are little-endian.
|
||||
* All multi-byte integers are little-endian. All arithmetic on samples
|
||||
* is mod 256.
|
||||
*
|
||||
* ┌─────────────────────────────────────────────────────────────────────┐
|
||||
* │ HEADER — 20 bytes │
|
||||
|
|
@ -42,7 +51,7 @@
|
|||
* │ Offset │ Size │ Field │
|
||||
* ├────────┼──────┼────────────────────────────────────────────────────┤
|
||||
* │ 0x00 │ 4 │ Magic: "YAPF" │
|
||||
* │ 0x04 │ 1 │ Version: 2 │
|
||||
* │ 0x04 │ 1 │ Version: 1 │
|
||||
* │ 0x05 │ 1 │ Channels: 1 gray / 2 gray+A / 3 RGB / 4 RGBA │
|
||||
* │ 0x06 │ 1 │ GPU format hint (YAPF_GPU_* constant) │
|
||||
* │ 0x07 │ 1 │ Mip levels (1 = base only, max 16) │
|
||||
|
|
@ -58,53 +67,64 @@
|
|||
* │ +0 │ 4 │ Byte offset of mip data from start of file (uint32) │
|
||||
* │ +4 │ 4 │ Byte size of mip data (uint32) │
|
||||
* └────────┴──────┴────────────────────────────────────────────────────┘
|
||||
* Mip m has size max(1, width >> m) × max(1, height >> m).
|
||||
*
|
||||
* ┌─────────────────────────────────────────────────────────────────────┐
|
||||
* │ PIXEL DATA — per mip level, referenced by the mip index │
|
||||
* │ MIP LEVEL DATA │
|
||||
* ├─────────────────────────────────────────────────────────────────────┤
|
||||
* │ Pixels are divided into 32×32 tiles, stored in row-major order. │
|
||||
* │ The last column / row of tiles may be smaller (no padding). │
|
||||
* │ Pixels are divided into 64×64 tiles in row-major order; the last │
|
||||
* │ column / row of tiles may be smaller (no padding). │
|
||||
* │ │
|
||||
* │ Encoding pipeline (encoder): │
|
||||
* │ a) RGB→YCoCg-R (skipped for gray/gray-alpha channels) │
|
||||
* │ b) 2-D prediction residuals (see below) │
|
||||
* │ c) Determine per-channel bit depth from max |residual| in tile │
|
||||
* │ d) Write tile header + packed residuals │
|
||||
* │ uint32 tile_size[tile_count] byte size of each tile │
|
||||
* │ tile data, back to back │
|
||||
* │ │
|
||||
* │ Tile layout: │
|
||||
* │ [bit_depth per channel] 1 byte each, channels bytes total │
|
||||
* │ [first pixel, raw] 1 byte per channel │
|
||||
* │ [packed residuals] bit_depth[c] bits per sample, │
|
||||
* │ stored MSB-first, zero-padded to the │
|
||||
* │ next byte boundary at the end of tile │
|
||||
* │ TILE │
|
||||
* │ 1 byte header │
|
||||
* │ 0x80 stored: tile_w × tile_h × channels raw bytes │
|
||||
* │ follow, interleaved, in the original channel order │
|
||||
* │ otherwise coded: │
|
||||
* │ bits 0–3 bit c set = plane c is constant (only bits below │
|
||||
* │ the channel count may be set) │
|
||||
* │ bit 4 colour transform: 0 = YCoCg-R, 1 = subtract-green │
|
||||
* │ (RGB / RGBA only; must be 0 for gray) │
|
||||
* │ bits 5–7 must be 0 │
|
||||
* │ coded tiles then hold one value byte per constant plane, and a │
|
||||
* │ plane payload per other plane, both in plane order. The tile │
|
||||
* │ must end exactly where its last payload ends. │
|
||||
* │ │
|
||||
* │ Prediction (lossless 2-D): │
|
||||
* │ pixel (0,0) in tile → stored raw, no residual │
|
||||
* │ pixel (x,0) → predict = left │
|
||||
* │ pixel (0,y) → predict = above │
|
||||
* │ pixel (x,y) → predict = left + above − diag │
|
||||
* │ residual = (actual − predict) wrapped mod 256, stored signed │
|
||||
* │ PLANES │
|
||||
* │ gray / gray+A: the channels as stored. │
|
||||
* │ RGB(A), YCoCg-R (>> is an arithmetic shift of the value as int8):│
|
||||
* │ planes Y, Co, Cg (+ A) │
|
||||
* │ forward: o = R − B; t = B + (o >> 1); q = G − t; │
|
||||
* │ Y = t + (q >> 1); Co = o + 128; Cg = q + 128 │
|
||||
* │ inverse: o = Co − 128; q = Cg − 128; t = Y − (q >> 1); │
|
||||
* │ G = q + t; B = t − (o >> 1); R = B + o │
|
||||
* │ RGB(A), subtract-green: │
|
||||
* │ planes G, R − G + 128, B − G + 128 (+ A) │
|
||||
* │ │
|
||||
* │ Bit packing: │
|
||||
* │ bits_needed = bit_depth[c] (0 means all residuals are 0) │
|
||||
* │ Each residual is stored as a (bits_needed)-bit two's-complement │
|
||||
* │ signed integer, MSB first, packed across byte boundaries. │
|
||||
* │ bit_depth 1 → only the sign bit (value is −1 or 0) │
|
||||
* │ bit_depth 0 → no bits written; all samples in channel = 0 │
|
||||
* │ │
|
||||
* │ YCoCg-R forward transform (applied before prediction): │
|
||||
* │ Co = R − B │
|
||||
* │ t = B + (Co >> 1) │
|
||||
* │ Cg = G − t │
|
||||
* │ Y = t + (Cg >> 1) │
|
||||
* │ Stored channel order: Y, Co, Cg (replaces R, G, B) │
|
||||
* │ Alpha is left unchanged. │
|
||||
* │ │
|
||||
* │ YCoCg-R inverse transform (applied after prediction): │
|
||||
* │ t = Y − (Cg >> 1) │
|
||||
* │ G = Cg + t │
|
||||
* │ B = t − (Co >> 1) │
|
||||
* │ R = B + Co │
|
||||
* │ PLANE PAYLOAD │
|
||||
* │ filter[ceil(tile_h / 8)] one byte per band of 8 rows. │
|
||||
* │ L / A / D / AR = left, above, above-left, above-right sample; │
|
||||
* │ samples outside the tile read as 0, except that AR in the │
|
||||
* │ last column reads as A. │
|
||||
* │ 0 Left pred = L │
|
||||
* │ 1 Gradient pred = L + A − D │
|
||||
* │ 2 Up-right pred = AR │
|
||||
* │ 3 Up-average pred = (A + AR) >> 1 (not mod 256) │
|
||||
* │ 16 + k−1 Up k pred = sample k rows above, k = 1..8 │
|
||||
* │ control nibbles, low nibble of each byte first, covering │
|
||||
* │ exactly ceil(tile_w × tile_h / 8) groups of 8 consecutive │
|
||||
* │ residuals in raster order: │
|
||||
* │ 0–8 one group of width w │
|
||||
* │ 9–15 2, 3, 4, 8, 16, 32 or 64 groups at the width of │
|
||||
* │ the last 0–8 code in this plane (0 if none yet) │
|
||||
* │ an odd nibble count is padded with a zero high nibble │
|
||||
* │ group data: w bytes per group, the 8 values packed little- │
|
||||
* │ endian, value i in bits [i·w, i·w + w) │
|
||||
* │ Residual r = (sample − pred) mod 256 is stored folded: │
|
||||
* │ v = (r << 1) ^ (r >> 7) (0, −1, 1, −2, … → 0, 1, 2, 3, …) │
|
||||
* │ The last group is padded with zero values. │
|
||||
* └─────────────────────────────────────────────────────────────────────┘
|
||||
*
|
||||
* GPU FORMAT HINTS (YAPF_GPU_* values for the gpu_format field)
|
||||
|
|
@ -161,7 +181,7 @@
|
|||
|
||||
#define YAPF_MAX_DIM 65535u
|
||||
#define YAPF_MAX_MIPS 16u
|
||||
#define YAPF_TILE_SIZE 32u
|
||||
#define YAPF_TILE_SIZE 64u
|
||||
|
||||
/* ── Image descriptor ────────────────────────────────────────────────── */
|
||||
|
||||
|
|
@ -189,6 +209,21 @@ extern "C" {
|
|||
*/
|
||||
yapf_image_t *yapf_load(const char *filename);
|
||||
|
||||
/*
|
||||
* yapf_load_memory — decode a .yapf file already in memory (e.g. from a
|
||||
* game archive). Same ownership and failure rules as yapf_load().
|
||||
*/
|
||||
yapf_image_t *yapf_load_memory(const void *buffer, size_t size);
|
||||
|
||||
/*
|
||||
* yapf_load_mt / yapf_load_memory_mt — same as above, but decode tiles on
|
||||
* up to `threads` threads (0 = one per CPU core). Output is identical.
|
||||
* Without thread support (YAPF_NO_THREADS, or an unknown platform) these
|
||||
* decode on the calling thread.
|
||||
*/
|
||||
yapf_image_t *yapf_load_mt(const char *filename, int threads);
|
||||
yapf_image_t *yapf_load_memory_mt(const void *buffer, size_t size, int threads);
|
||||
|
||||
/*
|
||||
* yapf_save — encode an image to disk.
|
||||
* img->mip_levels controls how many levels are written; set to 1 to write
|
||||
|
|
|
|||
Loading…
Reference in a new issue