knot

Tie a file into a compact .knot archive, and untie it back. A from-scratch DEFLATE-style compressor (LZ77 + Huffman), written in Rust.

Loose shoelaces are long and messy. Tie them in a knot and they're contained, smaller, and your shoes don't fall out. Same idea for files: knot cares about size and integrity, so every archive carries a CRC32 checksum.

On plain text and source code it lands within a few percent of gzip -9 and zip:

input	original	knot	gzip -9	zip -9
Rust source	46,877	14,293	13,753	13,864
/etc/services	299,903	72,331	70,391	70,518

Install

Prebuilt binaries are on the releases page. Pick your platform:

Prebuilt binary

tar xzf knot-linux-x86_64.tar.gz
sudo install -m755 knot /usr/local/bin/knot

Build from source

# Arch: sudo pacman -S rustup && rustup default stable
git clone https://github.com/realtripleg/knot
cd knot
cargo build --release
install -Dm755 target/release/knot ~/.local/bin/knot

Prebuilt binary

Apple Silicon → knot-macos-aarch64.tar.gz; Intel → knot-macos-x86_64.tar.gz.

tar xzf knot-macos-*.tar.gz
sudo install -m755 knot /usr/local/bin/knot
# clear the "unidentified developer" quarantine flag
xattr -d com.apple.quarantine /usr/local/bin/knot

Build from source

# install rustup from https://rustup.rs, then:
git clone https://github.com/realtripleg/knot
cd knot
cargo build --release
sudo install -m755 target/release/knot /usr/local/bin/knot

Prebuilt binary

Download knot-windows-x86_64.zip and extract knot.exe. Run it from that folder, or add it to your Path (PowerShell):

mkdir $env:USERPROFILE\bin -Force
move knot.exe $env:USERPROFILE\bin
[Environment]::SetEnvironmentVariable("Path", $env:Path + ";$env:USERPROFILE\bin", "User")

Build from source

# install rustup from https://rustup.rs, then in PowerShell:
git clone https://github.com/realtripleg/knot
cd knot
cargo build --release   # target\release\knot.exe

Usage

Three verbs: tie (compress), untie (decompress), inspect (metadata).

tie — compress

$ knot tie report.txt
tied report.txt -> report.txt.knot (46877 -> 14291 bytes, 30.5%)

Writes a .knot next to the original. Inputs that can't shrink are stored verbatim, so the archive never grows.

untie — decompress

$ knot untie report.txt.knot
untied report.txt.knot -> report.txt (46877 bytes, checksum OK)

Restores the original bytes and verifies the CRC32 checksum. Use -o/--output to restore elsewhere.

inspect — metadata

$ knot inspect report.txt.knot
report.txt.knot
  format:          KNOT v1
  mode:            compressed
  original name:   report.txt
  original size:   46877 bytes
  on-disk size:    14291 bytes (header + 14260 payload)
  ratio:           30.5% of original
  crc32:           0xc03cba1e

File format

A .knot file is a fixed header followed by the payload. Integers are little-endian.

offset	size	field
0	4	magic bytes `KNOT`
4	1	format version (currently 1)
5	1	flags (bit 0 = stored / uncompressed)
6	2	filename length `N`
8	N	original filename (UTF-8, basename only)
8+N	8	original size in bytes
16+N	4	CRC32 of the original bytes
20+N	…	payload (compressed, or raw if stored)

The compressed payload is a single bitstream: two Huffman code-length tables (286 literal/length codes and 30 distance codes, 4 bits each), then the token stream, ending with an end-of-block marker.

How it works

Two classic stages, the same pair zip and gzip use:

LZ77 slides a 32 KB window over the input and replaces repeated byte sequences with (length, distance) back-references ("copy 12 bytes from 400 bytes back"). Anything not part of a repeat is emitted as a literal byte.
Huffman coding gives common symbols short bit patterns and rare ones longer patterns. knot builds two canonical Huffman tables (one for literals and match lengths, one for distances) and stores only the code lengths, since both sides rebuild identical codes from those alone.

Untying reverses both, then confirms the size and checksum match what was stored.