01 / The idea
Most of the line is dots.
A test runner prints one character per test: . for a pass, F for a failure, s for a skip, E for an error, and x for a test that was expected to fail. Here is one run, 57 characters: .....F...s....F....s...F...E...s......F..x..s..F..E..F.x..
Stored as text, every character costs 8 bits, 456 in all. But 43 of the 57 are the same dot. If a dot cost 1 bit and the rare letters a few more, the line would shrink to a fraction of that.
Huffman coding gives each byte a code whose length depends on how often it appears: common bytes get short codes, rare bytes long ones, and no code is the start of another.
Watch it count the line, merge its way to a tree, and read the codes off that tree.
Short codes for what you say most.
- .43
- F6
- s4
- E2
- x2
Heap: E 2x 2s 4F 6. 43
Most of the line is dots.
57 bytes, 5 different ones: “.” 43, “F” 6, “s” 4, “E” 2, “x” 2. At 8 bits each that is 456 bits, and 43 of those bytes say the same thing.
Reduced motion: choose a scene to see its completed state.
Read this scene
57 bytes, 5 different ones: “.” 43, “F” 6, “s” 4, “E” 2, “x” 2. At 8 bits each that is 456 bits, and 43 of those bytes say the same thing.
0 of 4 merges done. Waiting in the heap, lightest first: E 2, x 2, s 4, F 6, . 43.
Watch and Step through replay the code for the test-run line. Try it runs the same TypeScript on your own text, one merge at a time, and starts fresh each time you open it.
The dot ends up one step from the root with a 1-bit code, and the two rarest letters four steps down. The whole line takes 83 bits.
02 / Name the rule
Merge the two lightest. Put the pair back.
Start with one tree per byte, each weighing its count. Take the two lightest trees out, join them under a new node that weighs their sum, and put that tree back. Repeat until one tree is left. The tree taken out first goes on the 0 side and the other on the 1 side.
43 · 6 · 4 · 2 · 2
How often each byte appears. A byte that never appears gets no code.
Two lightest → one tree
A min-heap hands over the two lightest trees every time.
Depth = code length
The dot sits 1 step from the root; E and x sit 4.
Ties need a rule, or two programs could build different trees. Ours: equal weights go to the
smaller id. A leaf’s id is its byte value and every merged tree gets an id from 256 up, so a
byte comes before a merged tree of the same weight. In the story, s weighs 4 and so does the tree of E and x; s is taken first.
One number falls out of the merges. Each merge adds one bit to every byte beneath it, so the coded line’s length is the sum of the merged weights: 4 + 8 + 14 + 57 = 83 bits.
Why merging the two lightest can’t be beatenAn exchange argument
Take any code for these counts, with at least two different bytes, that uses the fewest bits. Its deepest level holds at least two leaves: a node with only one child could be removed, shortening codes. Put the two rarest bytes there. Swapping a rarer byte into a deeper spot and a more common one into a shallower spot never adds bits, so the code is still as short.
Now treat those two siblings as one pretend byte whose count is their sum. Splitting it back apart turns any code for that smaller problem into a code for the original, costing exactly the sum of the two counts in extra bits. So the best code for the original is the best code for the smaller problem plus a fixed amount, and merging the two lightest first is always safe. The same argument covers every merge after it.
RFC 1951 puts the result plainly: the Huffman algorithm constructs “an optimal prefix code (one which represents strings with those symbol frequencies using the fewest bits of any possible prefix codes for that alphabet).”
03 / Read the shape
The tree gives lengths. The lengths give the codes.
Reading the tree gives each byte a path: 0 for left, 1 for right. But a decoder shouldn’t need your tree. DEFLATE sends only the lengths, and RFC 1951 §3.2.2 adds two rules so the lengths decide every code: “All codes of a given bit length have lexicographically consecutive values, in the same order as the symbols they represent” and “Shorter codes lexicographically precede longer codes.”
| Byte | Count | Length | Path in the tree | Canonical code |
|---|---|---|---|---|
. | 43 | 1 | 1 | 0 |
F | 6 | 2 | 00 | 10 |
s | 4 | 3 | 010 | 110 |
E | 2 | 4 | 0110 | 1110 |
x | 2 | 4 | 0111 | 1111 |
Same lengths, different bits, the same 83-bit total. Our encoder writes the canonical codes,
and a decoder needs only the five lengths. RFC 1951’s own example is in the shared cases:
lengths 2, 1, 3, 3 for A, B, C, D give 10, 0, 110, 111.
Basic form is buildCode and canonicalCodes. In the wild encodes, decodes from the lengths alone, and decides whether a
message is worth coding. At the call site runs three messages. Both languages
print the same four lines.
buildCode counts the bytes and merges the two lightest trees with a min-heap, lighter first and the smaller id on a tie. canonicalCodes turns the lengths into codes with RFC 1951’s two rules.
// Huffman coding over bytes: count each byte, then keep merging the two lightest trees
// until one is left. A byte's code length is how deep it ends up in that tree.
export const MAX_INPUT = 1024; // a teaching bound; it keeps every code well within 15 bits
export const MAX_LENGTH = 15; // DEFLATE's longest code (RFC 1951)
export type HuffmanErrorCode = 'too-long' | 'bad-lengths' | 'missing-code' | 'bad-bits';
export class HuffmanError extends Error {
readonly code: HuffmanErrorCode;
constructor(code: HuffmanErrorCode, message: string) {
super(message);
this.name = 'HuffmanError';
this.code = code;
}
}
// A leaf's id is its byte value. Each merge makes a node with the next id from 256.
export type TreeNode = { id: number; weight: number; left: number | null; right: number | null };
export type Code = {
counts: number[]; // how often each byte 0–255 appears
nodes: TreeNode[]; // the leaves in byte order, then one node per merge, in merge order
lengths: number[]; // each byte's code length, 0 if it never appears
};
// Lighter first. Equal weights go to the smaller id, so leaves come before merged nodes.
const lighter = (a: TreeNode, b: TreeNode) =>
a.weight < b.weight || (a.weight === b.weight && a.id < b.id);
export function buildCode(input: Uint8Array): Code {
if (input.length > MAX_INPUT)
throw new HuffmanError('too-long', `input is limited to ${MAX_INPUT} bytes`);
const counts = Array<number>(256).fill(0);
for (const byte of input) counts[byte]++;
const nodes: TreeNode[] = [];
for (let byte = 0; byte < 256; byte++)
if (counts[byte] > 0) nodes.push({ id: byte, weight: counts[byte], left: null, right: null });
const lengths = Array<number>(256).fill(0);
if (nodes.length === 1) {
lengths[nodes[0].id] = 1; // a lone byte still needs one bit each time it appears
return { counts, nodes, lengths };
}
const leaves = nodes.length;
const heap: TreeNode[] = [];
for (const node of nodes) push(heap, node);
while (heap.length > 1) {
const left = pop(heap); // the lightest tree gets bit 0
const right = pop(heap); // the next lightest gets bit 1
const merged = {
id: 256 + nodes.length - leaves,
weight: left.weight + right.weight,
left: left.id,
right: right.id
};
nodes.push(merged);
push(heap, merged);
}
// Walk down from the last node made, the root. A leaf's depth is its code length.
const byId = new Map(nodes.map((node) => [node.id, node]));
const stack: [TreeNode, number][] = nodes.length > 0 ? [[nodes[nodes.length - 1], 0]] : [];
while (stack.length > 0) {
const [node, depth] = stack.pop()!;
if (node.left === null || node.right === null) lengths[node.id] = depth;
else stack.push([byId.get(node.left)!, depth + 1], [byId.get(node.right)!, depth + 1]);
}
return { counts, nodes, lengths };
}
// RFC 1951 §3.2.2: shorter codes come first, and codes of one length follow byte order.
// The lengths alone decide every code, so a decoder needs only the lengths.
export function canonicalCodes(lengths: readonly number[]): string[] {
checkLengths(lengths);
const count = Array<number>(MAX_LENGTH + 1).fill(0);
for (const length of lengths) if (length > 0) count[length]++;
const next = Array<number>(MAX_LENGTH + 1).fill(0);
let code = 0;
for (let bits = 1; bits <= MAX_LENGTH; bits++) {
code = (code + count[bits - 1]) << 1;
next[bits] = code;
}
return lengths.map((length) =>
length === 0 ? '' : (next[length]++).toString(2).padStart(length, '0')
);
} // Huffman coding over bytes: count each byte, then keep merging the two lightest trees
// until one is left. A byte's code length is how deep it ends up in that tree.
const (
MaxInput = 1024 // a teaching bound; it keeps every code well within 15 bits
MaxLength = 15 // DEFLATE's longest code (RFC 1951)
)
type HuffmanError struct {
Code string // "too-long", "bad-lengths", "missing-code", or "bad-bits"
Message string
}
func (e *HuffmanError) Error() string { return e.Code + ": " + e.Message }
// A leaf's ID is its byte value. Each merge makes a node with the next ID from 256.
// Left and Right are -1 for a leaf.
type Node struct {
ID, Weight, Left, Right int
}
type Code struct {
Counts [256]int // how often each byte appears
Nodes []Node // the leaves in byte order, then one node per merge, in merge order
Lengths [256]int // each byte's code length, 0 if it never appears
}
func BuildCode(input []byte) (Code, error) {
var code Code
if len(input) > MaxInput {
return code, &HuffmanError{"too-long", fmt.Sprintf("input is limited to %d bytes", MaxInput)}
}
for _, b := range input {
code.Counts[b]++
}
for b, count := range code.Counts {
if count > 0 {
code.Nodes = append(code.Nodes, Node{ID: b, Weight: count, Left: -1, Right: -1})
}
}
if len(code.Nodes) == 1 {
code.Lengths[code.Nodes[0].ID] = 1 // a lone byte still needs one bit each time it appears
return code, nil
}
leaves := len(code.Nodes)
trees := &forest{}
for _, node := range code.Nodes {
heap.Push(trees, node)
}
for trees.Len() > 1 {
left := heap.Pop(trees).(Node) // the lightest tree gets bit 0
right := heap.Pop(trees).(Node) // the next lightest gets bit 1
merged := Node{ID: 256 + len(code.Nodes) - leaves, Weight: left.Weight + right.Weight, Left: left.ID, Right: right.ID}
code.Nodes = append(code.Nodes, merged)
heap.Push(trees, merged)
}
// Walk down from the last node made, the root. A leaf's depth is its code length.
byID := map[int]Node{}
for _, node := range code.Nodes {
byID[node.ID] = node
}
type visit struct{ node, depth int }
stack := []visit{}
if len(code.Nodes) > 0 {
stack = append(stack, visit{code.Nodes[len(code.Nodes)-1].ID, 0})
}
for len(stack) > 0 {
top := stack[len(stack)-1]
stack = stack[:len(stack)-1]
node := byID[top.node]
if node.Left < 0 {
code.Lengths[node.ID] = top.depth
} else {
stack = append(stack, visit{node.Left, top.depth + 1}, visit{node.Right, top.depth + 1})
}
}
return code, nil
}
// CanonicalCodes follows RFC 1951 §3.2.2: shorter codes come first, and codes of one length
// follow byte order. The lengths alone decide every code, so a decoder needs only the lengths.
func CanonicalCodes(lengths [256]int) ([256]string, error) {
var codes [256]string
if err := checkLengths(lengths); err != nil {
return codes, err
}
var count, next [MaxLength + 1]int
for _, length := range lengths {
if length > 0 {
count[length]++
}
}
code := 0
for bits := 1; bits <= MaxLength; bits++ {
code = (code + count[bits-1]) << 1
next[bits] = code
}
for b, length := range lengths {
if length > 0 {
codes[b] = fmt.Sprintf("%0*b", length, next[length])
next[length]++
}
}
return codes, nil
} Reading the TypeScriptBytes, a small heap, and bit strings
Input is a Uint8Array, so text goes through TextEncoder and é counts as two bytes. The heap is a few lines of push and pop, ordered
by weight and then id, the structure from the Binary heap lesson.
Codes and the encoded message are strings of 0 and 1 so they
can be shown; a real encoder packs bits into bytes. Errors are a HuffmanError whose code matches the Go version.
Reading the Gocontainer/heap and fixed arrays
forest implements heap.Interface, so container/heap keeps the trees in order; its documentation says “A heap is a common way to implement a priority queue.” Counts, lengths, and codes are [256] arrays, one slot per byte, and errors
come back as *HuffmanError.
What is refusedInput size, lengths, and bits
Input over 1,024 bytes is too-long. Lengths are bad-lengths when one is outside 0 to 15 or when together they ask for more
codes than exist, like three 1-bit codes. Encoding a byte that has no code is missing-code. Decoding reports bad-bits for a character other than
0 or 1, bits that stop in the middle of a code, or bits that are no code at all.
A message made of one byte value gets a 1-bit code, 0, and an empty message
needs no bits and no table. Both languages check, on the shared cases and 300 generated
inputs, that every code is prefix-free, canonical, and decodes back to the input, and
that its total matches a separate two-queue calculation.
04 / Try a decision
Where does a code end?
Codes of different lengths only work if the decoder can tell where each one stops. Before you check, read a few bits with each set.
That is why Huffman codes are read off the leaves of a tree. A leaf has no children, so no code can continue into another. RFC 1951 describes the guarantee: a parser “can always parse an encoded string unambiguously symbol-by-symbol.”
05 / Follow the cost
The table isn’t free.
| Operation | Time | Extra space | What it assumes |
|---|---|---|---|
| Count the bytes | O(n) | O(1) | One pass over n bytes into 256 counters. |
| Build the tree | O(k log k) | O(k) | k distinct bytes, at most 256: k pushes, then k − 1 merges of two pops and one push. |
| Turn lengths into codes | O(1) | O(1) | One pass over the 256 lengths and the 15 possible code lengths, whatever the message. |
| Encode | O(b) | O(b) | Writes b bits, one code per byte and at most 15 bits each. |
| Decode | O(b) | O(n) | One step per bit. Knowing where each length’s codes start means no table is searched. |
Building the tree only touches the distinct bytes, and there are at most 256 of them. For a message of any size, the work is the pass over its bytes and bits, not the tree.
The catch is the table. A decoder needs the lengths, so they travel with the message. This lesson’s own table format spends 8 bits saying how many bytes have codes (less one, so all 256 fit), then 12 bits per byte: 8 naming it and 4 for its length. The test run’s five bytes cost 68 bits of table, and the line still goes from 57 bytes to 19.
Short or even messages lose. ok is 2 bits of message and 32 bits of table: 5
bytes to send 2. Sixteen letters that each appear once get 4 bits apiece, no better than any
fixed 4-bit code for sixteen symbols, and with the table they take 33 bytes instead of 16.
And a single repeated byte can’t drop below a bit apiece: eight zs still take 4 bytes instead of 8.
How does DEFLATE keep the table small and the codes short?Compressed lengths and a length limit
DEFLATE sends lengths too, and it squeezes them: “the code length sequences themselves are compressed using a Huffman code,” with lengths running from 0 to 15 (RFC 1951 §3.2.7). The RFC also notes that its codes “must not exceed certain maximum code lengths,” which “complicates the algorithm for computing code lengths from symbol frequencies.”
We cap input at 1,024 bytes instead. Counts that grow like 1, 1, 2, 3, 5, 8… push the rarest byte deepest; 986 bytes of them give a 13-bit code, and the tests check that every length stays within 15.
06 / Give it a real job
Code the line when it pays.
A CI dashboard keeps the progress line of every test run. planMessage in In the wild builds a code for one line, encodes it, adds the table, and
sends the coded version only if it packs into fewer bytes than the text. The test run goes
from 57 bytes to 19. The two-byte ok and the sixteen letters go out as they are.
DEFLATE makes that decision block by block. The PNG specification describes compressed data stored as blocks, “each of which can represent raw (uncompressed) data, LZ77-compressed data encoded with fixed Huffman codes, or LZ77-compressed data encoded with custom Huffman codes.” HPACK takes the other road: its code is fixed in the standard, so no table travels with each header, at the price of fitting typical headers rather than yours.
Huffman coding runs where bytes are packed and unpacked: in the server or build step that
writes gzip, PNG, or HTTP/2 headers, and in the browser’s own network and image code that
reads them; nothing in a component computes it. When a response arrives gzip- or
Brotli-encoded, fetch decodes it before your code reads a byte, as the LZ77 lesson shows. If you ever hold
compressed bytes yourself, hand them to a real decoder such as DecompressionStream rather than writing one.
07 / Make the call
Huffman counts. It doesn’t look for repeats.
Huffman coding sees how often each byte appears, never in what order. abababab and aaaabbbb get the same code and the same size. When a
message repeats whole runs, find those first: that is LZ77’s job, and DEFLATE does it before
Huffman coding.
For real data, ship a real format. In Go, compress/flate implements DEFLATE and compress/gzip the gzip format; in the browser, CompressionStream. Build your own code only
when both sides share an alphabet you control, and decide up front whether a fixed table
like HPACK’s or a table per message fits better.
SourcesStandards, specifications, and packages, checked 13 September 2026
- RFC 7541, HPACK: “a compression
format for efficiently representing HTTP header fields, to be used in HTTP/2”; §5.2’s
Huffman flag on string literals; Appendix B’s code, “generated from statistics obtained
on a large sample of HTTP headers,” with
/at 6 bits; Appendix C.4.1, wherewww.example.comis a 12-byte Huffman-encoded literal. - RFC 9113, HTTP/2: “Each endpoint has an HPACK encoder context and an HPACK decoder context.”
- RFC 1951, DEFLATE: §3.2.1 on prefix codes and optimality, §3.2.2’s two rules, algorithm, and examples, §3.2.7 on code lengths 0 to 15 compressed with a Huffman code.
- PNG (Third Edition): “PNG compression method 0 is deflate compression with a sliding window,” stored as blocks with fixed or custom Huffman codes.
- D. Huffman, “A Method for the Construction of Minimum-Redundancy Codes,” Proceedings of the Institute of Radio Engineers 40(9), 1952, as cited by RFC 7541.
- Go 1.24.0 source:
container/heap,compress/flate(“implements the DEFLATE compressed data format, described in RFC 1951”), andcompress/gzip(“as specified in RFC 1952”).
08 / Take the idea with you
Explain a coded line without saying “Huffman.”
“Count how often each character appears. Keep joining the two rarest groups until everything is in one tree. A character’s code is its path from the top, so common ones get short codes, and no code is the start of another.”
Before moving on, pick a log file or a JSON response you know and guess which bytes would get the shortest codes. Then count how many different bytes it uses: that is the table you would have to send.
Connections to follow nextRelated lessons
- Binary heap is the priority queue behind every merge.
- LZ77 compression is DEFLATE’s first half: it removes repeats before Huffman codes what is left.
- Aho–Corasick matching also walks a tree of prefixes, its trie, one character at a time.