← Applied algorithms
Text, names, and changes From a glossary to a scanning machine

Aho–Corasick matching

Find every term in one pass.

You have probably written this one: new RegExp(terms.join('|'), 'g'). Give it auth, author, queue, and retry, run it over “The author retries the queue after auth.”, and it finds auth, queue, and auth. It never reports author. Put author first and you lose the auth inside it instead.

A regex takes the first alternative that matches and moves past it. Aho–Corasick reads the document once and reports every term it finds, overlaps included. We’ll build it for a docs glossary.

TypeScriptGoOne glossary scan in each language

01 / The idea

Your regex already lost a match.

Aho–Corasick finds every occurrence of a fixed set of terms in one left-to-right pass over a text. It builds a small machine from the glossary once, then feeds it the document one symbol at a time. Each symbol moves the machine to a new state, and a state knows which terms end there.

Searching for each term on its own works too, and for one or two terms it is the right call. With a glossary, every term tries every position again: four terms over our forty-character sentence is 160 starting points before a single hit is kept. The machine takes one step per character, plus one report per hit.

Watch it build the machine for our glossary, fall back when a letter doesn’t fit, and report all four hits. Then open Try it and give it your own terms.

Aho–Corasick

Find every term in one pass.

GLOSSARY authauthorqueueretry 0 / 16 states
  1. rootauth
  2. rootauthor
  3. rootqueue
  4. rootretry

a state an earlier term already built

01/ 03
Share beginnings

auth and author share four states.

Insert each term one letter at a time. 20 letters need only 16 states: auth and author share a, au, aut, and auth.

Reduced motion: choose a scene to see its completed state.

Read this scene

Insert each term one letter at a time. 20 letters need only 16 states: auth and author share a, au, aut, and auth.

0 of 16 states built. Insert each term one letter at a time. 20 letters need only 16 states: auth and author share a, au, aut, and auth.

Watch and Step through use the support glossary. Try it runs the same TypeScript on your own terms and starts fresh each time you open it.

The four hits are auth and author, both starting at position 4, queue at 23, and auth again at 35. retry finds nothing: the document says retries, and matching is exact.

Follow the example in your languages.

These choices apply to every comparison below. The lab runs the TypeScript; the other languages are checked against the same cases.

02 / Name the rule

Shared beginnings become shared states.

Start with a trie: insert each term one symbol at a time, reusing a branch when it already exists. auth and author share a, au, aut, and auth, so our four terms, twenty letters in all, need sixteen states plus the root.

Each state remembers the prefix that leads to it, and the state where a term ends remembers which term. Empty and repeated terms are rejected up front. With no repeats, every hit belongs to exactly one term, so two hits can never tie on position.

TypeScriptInsert terms into a trie
glossary.ts
export function buildGlossary(terms: readonly string[]): Glossary {
	if (terms.length < 1 || terms.length > MAX_TERMS)
		throw new GlossaryError('term-count', `Use 1 to ${MAX_TERMS} terms.`);
	const glossary: Glossary = { terms: [...terms], lengths: [], states: [state('')] };
	const seen = new Set<string>();
	terms.forEach((term, index) => {
		if (term === '') throw new GlossaryError('empty-term', 'Terms cannot be empty.');
		const symbols = scalars(term, MAX_TERM_SCALARS, 'long-term', 'A term');
		if (seen.has(term)) throw new GlossaryError('duplicate-term', `“${term}” appears twice.`);
		seen.add(term);
		let current = 0;
		for (const symbol of symbols) {
			let next = glossary.states[current].next.get(symbol);
			if (next === undefined) {
				// A new branch. Terms that share a beginning share these states.
				next = glossary.states.length;
				glossary.states[current].next.set(symbol, next);
				glossary.states.push(state(glossary.states[current].prefix + symbol));
			}
			current = next;
		}
		glossary.states[current].term = index;
		glossary.lengths.push(symbols.length);
	});
	linkFailures(glossary);
	return glossary;
}
GoInsert terms into a trie
glossary.go
func BuildGlossary(terms []string) (*Glossary, error) {
	if len(terms) < 1 || len(terms) > MaxTerms {
		return nil, &GlossaryError{"term-count", fmt.Sprintf("use 1 to %d terms", MaxTerms)}
	}
	g := &Glossary{Terms: append([]string(nil), terms...), States: []State{newState("")}}
	seen := map[string]bool{}
	for index, term := range terms {
		if term == "" {
			return nil, &GlossaryError{"empty-term", "terms cannot be empty"}
		}
		if !utf8.ValidString(term) {
			return nil, &GlossaryError{"malformed-text", "a term must be valid UTF-8"}
		}
		length := utf8.RuneCountInString(term)
		if length > MaxTermScalars {
			return nil, &GlossaryError{"long-term", fmt.Sprintf("a term is limited to %d Unicode scalar values", MaxTermScalars)}
		}
		if seen[term] {
			return nil, &GlossaryError{"duplicate-term", fmt.Sprintf("%q appears twice", term)}
		}
		seen[term] = true
		current := 0
		for _, symbol := range term {
			next, ok := g.States[current].Next[symbol]
			if !ok {
				// A new branch. Terms that share a beginning share these states.
				next = len(g.States)
				g.States[current].Next[symbol] = next
				g.States = append(g.States, newState(g.States[current].Prefix+string(symbol)))
			}
			current = next
		}
		g.States[current].Term = index
		g.Lengths = append(g.Lengths, length)
	}
	g.linkFailures()
	return g, nil
}

Don’t restart after a mismatch.

A trie on its own answers one question: does a term start here? Reading retries, the machine walks r, re, ret, retr, and then meets i where retry needed y. Going back to the start of the word to try again would read those letters twice.

Instead, every state gets a failure link: the longest ending of its prefix that is also the beginning of some term. From retr, that ending is r. The machine follows the link, looks for i from r, finds no branch, follows r’s link to the root, and reads on. The document cursor never moves backward.

TypeScriptLink each state to its longest useful ending
glossary.ts
function linkFailures({ states }: Glossary): void {
	// Breadth-first: every failure target is shallower, so it is already linked.
	const queue = [...states[0].next.values()]; // one-symbol states fail to the root
	for (let head = 0; head < queue.length; head++) {
		const parent = queue[head];
		for (const [symbol, child] of states[parent].next) {
			queue.push(child);
			let fallback = states[parent].fail;
			while (fallback !== 0 && !states[fallback].next.has(symbol)) fallback = states[fallback].fail;
			const fail = states[fallback].next.get(symbol) ?? 0;
			states[child].fail = fail;
			// Point at the nearest term along the failure chain instead of copying its outputs.
			states[child].output = states[fail].term !== null ? fail : states[fail].output;
		}
	}
}
GoLink each state to its longest useful ending
glossary.go
func (g *Glossary) linkFailures() {
	states := g.States
	// Breadth-first: every failure target is shallower, so it is already linked.
	queue := make([]int, 0, len(states))
	for _, child := range states[0].Next { // one-symbol states fail to the root
		queue = append(queue, child)
	}
	for head := 0; head < len(queue); head++ {
		parent := queue[head]
		for symbol, child := range states[parent].Next {
			queue = append(queue, child)
			fallback := states[parent].Fail
			for fallback != 0 {
				if _, ok := states[fallback].Next[symbol]; ok {
					break
				}
				fallback = states[fallback].Fail
			}
			fail := states[fallback].Next[symbol] // 0, the root, when absent
			states[child].Fail = fail
			// Point at the nearest term along the failure chain instead of copying its outputs.
			if states[fail].Term != -1 {
				states[child].Output = fail
			} else {
				states[child].Output = states[fail].Output
			}
		}
	}
}

The links are computed breadth-first. A failure target is always shorter than the state that points at it, so by the time we reach a state, everything it could point to already has its own link.

The same pass sets an output link: the nearest state along the failure chain where some term ends. With he and she in the glossary, the state she links to he. When the scan arrives at she, following output links reports he as well.

03 / Read the shape

Every overlap is evidence.

The scan is the loop from the story. Fall back until the next symbol fits or you reach the root, take the transition, then walk the output chain and report every term that ends at this position. Hits come back sorted by start, then end.

TypeScriptScan the document once
glossary.ts
export function scan(glossary: Glossary, text: string): Hit[] {
	const symbols = scalars(text, MAX_TEXT_SCALARS, 'long-text', 'The document');
	const { states, terms, lengths } = glossary;
	const hits: Hit[] = [];
	let current = 0;
	symbols.forEach((symbol, index) => {
		// Fall back along failure links. The document cursor never moves backward.
		while (current !== 0 && !states[current].next.has(symbol)) current = states[current].fail;
		current = states[current].next.get(symbol) ?? 0;
		// The term that ends here, then every shorter term that ends here too.
		let at = states[current].term !== null ? current : states[current].output;
		for (; at !== -1; at = states[at].output) {
			const term = states[at].term!;
			hits.push({ term: terms[term], start: index + 1 - lengths[term], end: index + 1 });
		}
	});
	// Terms are unique, so start then end decides every tie.
	return hits.sort((a, b) => a.start - b.start || a.end - b.end);
}
GoScan the document once
glossary.go
func (g *Glossary) Scan(text string) ([]Hit, error) {
	if !utf8.ValidString(text) {
		return nil, &GlossaryError{"malformed-text", "the document must be valid UTF-8"}
	}
	if utf8.RuneCountInString(text) > MaxTextScalars {
		return nil, &GlossaryError{"long-text", fmt.Sprintf("the document is limited to %d Unicode scalar values", MaxTextScalars)}
	}
	hits := []Hit{}
	current, index := 0, 0
	for _, symbol := range text {
		// Fall back along failure links. The document cursor never moves backward.
		for current != 0 {
			if _, ok := g.States[current].Next[symbol]; ok {
				break
			}
			current = g.States[current].Fail
		}
		current = g.States[current].Next[symbol]
		// The term that ends here, then every shorter term that ends here too.
		at := g.States[current].Output
		if g.States[current].Term != -1 {
			at = current
		}
		for ; at != -1; at = g.States[at].Output {
			term := g.States[at].Term
			hits = append(hits, Hit{g.Terms[term], index + 1 - g.Lengths[term], index + 1})
		}
		index++
	}
	// Terms are unique, so start then end decides every tie.
	sort.SliceStable(hits, func(a, b int) bool {
		if hits[a].Start != hits[b].Start {
			return hits[a].Start < hits[b].Start
		}
		return hits[a].End < hits[b].End
	})
	return hits, nil
}

The matcher’s job ends at evidence. auth inside author is a real occurrence. Whether a page shows both, only the longer one, or neither is a presentation decision, and it belongs to the caller. Keeping every hit leaves that decision open.

Reading the TypeScriptA Map per state and a null term

A Glossary is plain data: an array of State records, each with a Map from symbol to the next state’s index. Array.from(text) splits the document into Unicode scalars, so 🌊 is one symbol at one position, although it takes two UTF-16 code units. End positions are exclusive.

next.get(symbol) ?? 0 sends a symbol with no branch back to the root. term is null where no term ends, and output is -1 at the end of the chain. Errors are a GlossaryError whose code matches the Go version.

Reading the GoA rune map, a zero value, and -1

Next is a map[rune]int. Reading a missing key gives the zero value, 0, which is the root, so the transition after the fallback loop needs no check. range over a string yields runes but reports byte offsets, so Scan keeps its own index to count scalars.

Term and Output use -1 for none. Errors are *GlossaryError values with the same codes as TypeScript, and sort.SliceStable puts the hits in order.

What is refusedTerms, sizes, and text

Zero terms or more than 12 is term-count. An empty term is empty-term, a term over 24 symbols is long-term, and a repeated term is duplicate-term. A document over 240 symbols is long-text. TypeScript rejects unpaired surrogates and Go rejects invalid UTF-8, both as malformed-text. Nothing is truncated.

Matching is exact and case-sensitive. There is no case folding, Unicode normalization, or word boundary, so auth also matches inside author and authentic. Both languages run the same shared cases.

04 / Try a decision

Should “he” count inside “she”?

Decide what the matcher returns before any page decides what to show.

The glossary has “he” and “she”. The document says “she”. What should the matcher return?

05 / Follow the cost

Build once, scan many.

Aho–Corasick: time and extra space, for P term symbols, T document symbols, and z hits
OperationTimeExtra spaceWhat it assumes
Build the trieO(P)O(P)One lookup or insert per term symbol, with average constant-time map lookups. At most P states plus the root.
Link failures and outputsO(P)O(P)Breadth-first, one queue entry per state. Every fallback shortens a depth that grew by at most one per symbol.
Walk one documentO(T + z)O(T + z)One transition per symbol, fallbacks paid for by earlier steps, one output link per hit. The space is the document’s symbols and the hits.
Sort the hitsO(z log z)O(z)Counted in comparisons. The shown scan sorts by start, then end. Drop the sort to keep O(T + z).
Hold the machine—O(P)Each state keeps its transitions, one failure link, and one output link, never a copied list of terms.

Let P be the total length of the terms and T the length of the document. Building the trie takes one lookup or insert per term symbol: O(P), with average constant-time map lookups. Linking failures is O(P) as well, because every fallback shortens a depth that grew by at most one per symbol.

The walk itself is O(T + z), where z is the number of hits. Each symbol takes one transition, fallbacks are paid for by earlier steps, and each output link visited produces a hit. z can be large: a, aa, and aaa over aaaa already give nine hits for four letters, and no algorithm reports them in less.

The scan shown above then sorts its hits by start, then end, which adds O(z log z): O(T + z log z) in all. The walk finds hits in the order they end, so a caller happy with that order can drop the sort and keep O(T + z).

Our glossary machine has sixteen states and a root, built once and reused for every document. That reuse is where the design pays off.

Why output links instead of copied lists?Where the linear build comes from

An easier construction copies each failure target’s list of terms into the state that points at it. Take short terms a, aa, aaa and one long term made only of a: every state along the long term then holds a copy of all three short terms. Add more short terms and lengthen the long one, and the copies grow faster than the total length of the glossary.

An output link is one number per state. The build stays O(P), and the scan pays for each hit when it reports it.

06 / Give it a real job

Mark the glossary on every docs page.

A documentation site keeps a glossary: product terms, each with a short definition. Every article should mark those terms so a reader can look one up without leaving the page. The glossary changes when someone edits it. The articles are read all day.

That split is what the machine is good at. Build it once when the glossary loads, then scan each article’s text as it renders. The call site builds once and scans two documents with the same machine.

TypeScriptBuild once, scan every document
glossary.ts
export const supportGlossary = ['auth', 'author', 'queue', 'retry'];
export const supportDocument = 'The author retries the queue after auth.';

export function runExample(): void {
	const glossary = buildGlossary(supportGlossary); // build once
	for (const document of [supportDocument, 'Retry the queue, then check auth.']) {
		const hits = scan(glossary, document); // scan every document
		console.log(hits.map((hit) => `${hit.term}@${hit.start}-${hit.end}`).join(' ') || 'no terms');
	}
}
GoBuild once, scan every document
glossary.go
var supportGlossary = []string{"auth", "author", "queue", "retry"}

const supportDocument = "The author retries the queue after auth."

func main() {
	glossary, err := BuildGlossary(supportGlossary) // build once
	if err != nil {
		fmt.Println(err)
		return
	}
	for _, document := range []string{supportDocument, "Retry the queue, then check auth."} {
		hits, err := glossary.Scan(document) // scan every document
		if err != nil {
			fmt.Println(err)
			return
		}
		fmt.Println(format(hits))
	}
}

Both versions print auth@4-8 author@4-10 queue@23-28 auth@35-39, then queue@10-15 auth@28-32. Retry with a capital R is not retry.

Copy and run the complete exampleNo packages or services required

Save the selected file and run node --experimental-strip-types glossary.ts with Node 22.18 or newer, or go run glossary.go with Go 1.23 or newer. The TypeScript complete view ends with a call to runExample().

TypeScriptComplete glossary example
glossary.ts
// A bounded, inspectable teaching implementation. No external dependencies.
export const MAX_TERMS = 12;
export const MAX_TERM_SCALARS = 24;
export const MAX_TEXT_SCALARS = 240;

export type Hit = { term: string; start: number; end: number };

export type GlossaryErrorCode =
	'term-count' | 'empty-term' | 'long-term' | 'duplicate-term' | 'long-text' | 'malformed-text';

export class GlossaryError extends Error {
	code: GlossaryErrorCode;
	constructor(code: GlossaryErrorCode, message: string) {
		super(message);
		this.name = 'GlossaryError';
		this.code = code;
	}
}

export type State = {
	prefix: string; // the symbols that lead here from the root, kept for inspection
	next: Map<string, number>;
	fail: number; // the longest proper suffix of prefix that is also a prefix in the trie
	term: number | null; // the term that ends exactly here
	output: number; // the nearest state along failure links that ends a term, or -1
};
export type Glossary = { terms: string[]; lengths: number[]; states: State[] };

// Unicode scalar values. Checks the whole text first, then the length.
function scalars(text: string, limit: number, code: GlossaryErrorCode, label: string): string[] {
	const result = Array.from(text);
	for (const symbol of result) {
		const point = symbol.codePointAt(0)!;
		if (point >= 0xd800 && point <= 0xdfff)
			throw new GlossaryError('malformed-text', `${label} must be well-formed Unicode.`);
	}
	if (result.length > limit)
		throw new GlossaryError(code, `${label} is limited to ${limit} Unicode scalar values.`);
	return result;
}

const state = (prefix: string): State => ({
	prefix,
	next: new Map(),
	fail: 0,
	term: null,
	output: -1
});

export function buildGlossary(terms: readonly string[]): Glossary {
	if (terms.length < 1 || terms.length > MAX_TERMS)
		throw new GlossaryError('term-count', `Use 1 to ${MAX_TERMS} terms.`);
	const glossary: Glossary = { terms: [...terms], lengths: [], states: [state('')] };
	const seen = new Set<string>();
	terms.forEach((term, index) => {
		if (term === '') throw new GlossaryError('empty-term', 'Terms cannot be empty.');
		const symbols = scalars(term, MAX_TERM_SCALARS, 'long-term', 'A term');
		if (seen.has(term)) throw new GlossaryError('duplicate-term', `“${term}” appears twice.`);
		seen.add(term);
		let current = 0;
		for (const symbol of symbols) {
			let next = glossary.states[current].next.get(symbol);
			if (next === undefined) {
				// A new branch. Terms that share a beginning share these states.
				next = glossary.states.length;
				glossary.states[current].next.set(symbol, next);
				glossary.states.push(state(glossary.states[current].prefix + symbol));
			}
			current = next;
		}
		glossary.states[current].term = index;
		glossary.lengths.push(symbols.length);
	});
	linkFailures(glossary);
	return glossary;
}

function linkFailures({ states }: Glossary): void {
	// Breadth-first: every failure target is shallower, so it is already linked.
	const queue = [...states[0].next.values()]; // one-symbol states fail to the root
	for (let head = 0; head < queue.length; head++) {
		const parent = queue[head];
		for (const [symbol, child] of states[parent].next) {
			queue.push(child);
			let fallback = states[parent].fail;
			while (fallback !== 0 && !states[fallback].next.has(symbol)) fallback = states[fallback].fail;
			const fail = states[fallback].next.get(symbol) ?? 0;
			states[child].fail = fail;
			// Point at the nearest term along the failure chain instead of copying its outputs.
			states[child].output = states[fail].term !== null ? fail : states[fail].output;
		}
	}
}

export function scan(glossary: Glossary, text: string): Hit[] {
	const symbols = scalars(text, MAX_TEXT_SCALARS, 'long-text', 'The document');
	const { states, terms, lengths } = glossary;
	const hits: Hit[] = [];
	let current = 0;
	symbols.forEach((symbol, index) => {
		// Fall back along failure links. The document cursor never moves backward.
		while (current !== 0 && !states[current].next.has(symbol)) current = states[current].fail;
		current = states[current].next.get(symbol) ?? 0;
		// The term that ends here, then every shorter term that ends here too.
		let at = states[current].term !== null ? current : states[current].output;
		for (; at !== -1; at = states[at].output) {
			const term = states[at].term!;
			hits.push({ term: terms[term], start: index + 1 - lengths[term], end: index + 1 });
		}
	});
	// Terms are unique, so start then end decides every tie.
	return hits.sort((a, b) => a.start - b.start || a.end - b.end);
}

export function findAll(terms: readonly string[], text: string): Hit[] {
	return scan(buildGlossary(terms), text);
}

export type Step = {
	index: number;
	symbol: string;
	from: number;
	fallbacks: number[]; // states visited through failure links before the next transition
	to: number;
	hits: Hit[];
};

// The same walk as scan, recorded one symbol at a time for the lesson's story and lab.
export function trace(glossary: Glossary, text: string): Step[] {
	const symbols = scalars(text, MAX_TEXT_SCALARS, 'long-text', 'The document');
	const { states, terms, lengths } = glossary;
	const steps: Step[] = [];
	let current = 0;
	symbols.forEach((symbol, index) => {
		const from = current;
		const fallbacks: number[] = [];
		while (current !== 0 && !states[current].next.has(symbol)) {
			current = states[current].fail;
			fallbacks.push(current);
		}
		current = states[current].next.get(symbol) ?? 0;
		const hits: Hit[] = [];
		let at = states[current].term !== null ? current : states[current].output;
		for (; at !== -1; at = states[at].output) {
			const term = states[at].term!;
			hits.push({ term: terms[term], start: index + 1 - lengths[term], end: index + 1 });
		}
		steps.push({ index, symbol, from, fallbacks, to: current, hits });
	});
	return steps;
}

export const supportGlossary = ['auth', 'author', 'queue', 'retry'];
export const supportDocument = 'The author retries the queue after auth.';

export function runExample(): void {
	const glossary = buildGlossary(supportGlossary); // build once
	for (const document of [supportDocument, 'Retry the queue, then check auth.']) {
		const hits = scan(glossary, document); // scan every document
		console.log(hits.map((hit) => `${hit.term}@${hit.start}-${hit.end}`).join(' ') || 'no terms');
	}
}

runExample();
GoComplete glossary example
glossary.go
// A bounded, inspectable teaching implementation. No external dependencies.
package main

import (
	"fmt"
	"sort"
	"strings"
	"unicode/utf8"
)

const (
	MaxTerms       = 12
	MaxTermScalars = 24
	MaxTextScalars = 240
)

type Hit struct {
	Term       string
	Start, End int
}

// GlossaryError carries the same codes as the TypeScript version.
type GlossaryError struct {
	Code    string
	Message string
}

func (e *GlossaryError) Error() string { return e.Message }

type State struct {
	Prefix string // the symbols that lead here from the root, kept for inspection
	Next   map[rune]int
	Fail   int // the longest proper suffix of Prefix that is also a prefix in the trie
	Term   int // the term that ends exactly here, or -1
	Output int // the nearest state along failure links that ends a term, or -1
}

type Glossary struct {
	Terms   []string
	Lengths []int
	States  []State
}

func newState(prefix string) State {
	return State{Prefix: prefix, Next: map[rune]int{}, Term: -1, Output: -1}
}

func BuildGlossary(terms []string) (*Glossary, error) {
	if len(terms) < 1 || len(terms) > MaxTerms {
		return nil, &GlossaryError{"term-count", fmt.Sprintf("use 1 to %d terms", MaxTerms)}
	}
	g := &Glossary{Terms: append([]string(nil), terms...), States: []State{newState("")}}
	seen := map[string]bool{}
	for index, term := range terms {
		if term == "" {
			return nil, &GlossaryError{"empty-term", "terms cannot be empty"}
		}
		if !utf8.ValidString(term) {
			return nil, &GlossaryError{"malformed-text", "a term must be valid UTF-8"}
		}
		length := utf8.RuneCountInString(term)
		if length > MaxTermScalars {
			return nil, &GlossaryError{"long-term", fmt.Sprintf("a term is limited to %d Unicode scalar values", MaxTermScalars)}
		}
		if seen[term] {
			return nil, &GlossaryError{"duplicate-term", fmt.Sprintf("%q appears twice", term)}
		}
		seen[term] = true
		current := 0
		for _, symbol := range term {
			next, ok := g.States[current].Next[symbol]
			if !ok {
				// A new branch. Terms that share a beginning share these states.
				next = len(g.States)
				g.States[current].Next[symbol] = next
				g.States = append(g.States, newState(g.States[current].Prefix+string(symbol)))
			}
			current = next
		}
		g.States[current].Term = index
		g.Lengths = append(g.Lengths, length)
	}
	g.linkFailures()
	return g, nil
}


func (g *Glossary) linkFailures() {
	states := g.States
	// Breadth-first: every failure target is shallower, so it is already linked.
	queue := make([]int, 0, len(states))
	for _, child := range states[0].Next { // one-symbol states fail to the root
		queue = append(queue, child)
	}
	for head := 0; head < len(queue); head++ {
		parent := queue[head]
		for symbol, child := range states[parent].Next {
			queue = append(queue, child)
			fallback := states[parent].Fail
			for fallback != 0 {
				if _, ok := states[fallback].Next[symbol]; ok {
					break
				}
				fallback = states[fallback].Fail
			}
			fail := states[fallback].Next[symbol] // 0, the root, when absent
			states[child].Fail = fail
			// Point at the nearest term along the failure chain instead of copying its outputs.
			if states[fail].Term != -1 {
				states[child].Output = fail
			} else {
				states[child].Output = states[fail].Output
			}
		}
	}
}


func (g *Glossary) Scan(text string) ([]Hit, error) {
	if !utf8.ValidString(text) {
		return nil, &GlossaryError{"malformed-text", "the document must be valid UTF-8"}
	}
	if utf8.RuneCountInString(text) > MaxTextScalars {
		return nil, &GlossaryError{"long-text", fmt.Sprintf("the document is limited to %d Unicode scalar values", MaxTextScalars)}
	}
	hits := []Hit{}
	current, index := 0, 0
	for _, symbol := range text {
		// Fall back along failure links. The document cursor never moves backward.
		for current != 0 {
			if _, ok := g.States[current].Next[symbol]; ok {
				break
			}
			current = g.States[current].Fail
		}
		current = g.States[current].Next[symbol]
		// The term that ends here, then every shorter term that ends here too.
		at := g.States[current].Output
		if g.States[current].Term != -1 {
			at = current
		}
		for ; at != -1; at = g.States[at].Output {
			term := g.States[at].Term
			hits = append(hits, Hit{g.Terms[term], index + 1 - g.Lengths[term], index + 1})
		}
		index++
	}
	// Terms are unique, so start then end decides every tie.
	sort.SliceStable(hits, func(a, b int) bool {
		if hits[a].Start != hits[b].Start {
			return hits[a].Start < hits[b].Start
		}
		return hits[a].End < hits[b].End
	})
	return hits, nil
}


func FindAll(terms []string, text string) ([]Hit, error) {
	g, err := BuildGlossary(terms)
	if err != nil {
		return nil, err
	}
	return g.Scan(text)
}

func format(hits []Hit) string {
	if len(hits) == 0 {
		return "no terms"
	}
	parts := make([]string, len(hits))
	for i, hit := range hits {
		parts[i] = fmt.Sprintf("%s@%d-%d", hit.Term, hit.Start, hit.End)
	}
	return strings.Join(parts, " ")
}

var supportGlossary = []string{"auth", "author", "queue", "retry"}

const supportDocument = "The author retries the queue after auth."

func main() {
	glossary, err := BuildGlossary(supportGlossary) // build once
	if err != nil {
		fmt.Println(err)
		return
	}
	for _, document := range []string{supportDocument, "Retry the queue, then check auth."} {
		hits, err := glossary.Scan(document) // scan every document
		if err != nil {
			fmt.Println(err)
			return
		}
		fmt.Println(format(hits))
	}
}

Build UIs?Your regex drops overlapping terms, and the browser can now paint every one of them.

Where it already is in your components

It is the alternation regex from the top of this page. In Node 22, /auth|author/g over our document returns auth@4 and auth@35. /author|auth/g returns author@4 and auth@35. /he|she|his|hers/g over ushers returns only she@1. A JavaScript regex tries the alternatives in order, keeps the first that matches, and resumes after it, so it reports at most one term per stretch of text. If you have ever sorted terms longest-first before joining them, this is why: it trades one lost match for another.

The usual next step has a warning of its own. Wrapping each match in <mark> and handing the string to dangerouslySetInnerHTML or {@html} sends the page’s text back through the HTML parser. React’s docs say that unless the markup comes from a trusted source, “it is trivial to introduce an XSS vulnerability this way.” Svelte’s say: “Never render unsanitized content.” The glossary is yours. The article text often isn’t.

When you have to own it

Back to the docs page. The hits are ready; the question is how to show them without rewriting the article’s markup. The CSS Custom Highlight API paints ranges of text you choose, “without affecting the DOM structure in the page.” MDN lists the API as Baseline since June 2025, and the ::highlight() pseudo-element that styles it as Baseline since March 2026, so check it against the browsers you support.

Overlaps are the point. Register one highlight per category, such as API terms and product names, and give each a priority. Where two highlights cover the same letters, the higher priority styles them; with equal priorities, the one registered last wins. Only a few properties apply inside ::highlight(), among them color, background color, and text decoration.

Two details bite. Hits count Unicode scalars, but a DOM range counts UTF-16 code units, so one emoji early in a paragraph shifts every offset after it; codeUnitOffsets maps one to the other. And a highlight is paint, not an element: nobody can click it. For links, choose hits that don’t overlap and render them as text and <a> elements. linkSegments keeps the earliest, longest hit, so author gets the link and the auth inside it doesn’t.

highlight.ts
import { buildGlossary, scan, type Glossary, type Hit } from '../glossary';

// Hits count Unicode scalar values; DOM ranges count UTF-16 code units. Map one to the other.
export function codeUnitOffsets(text: string): number[] {
	const offsets = [0];
	for (const symbol of text) offsets.push(offsets[offsets.length - 1] + symbol.length);
	return offsets;
}

export type Category = { name: string; terms: readonly string[]; priority: number };

// Build once, when the glossary loads: every category's terms go into one machine.
export function buildCategories(categories: readonly Category[]): Glossary {
	return buildGlossary(categories.flatMap((category) => category.terms));
}

// The article's text nodes, in document order.
export function textNodes(root: Node): Text[] {
	const walker = document.createTreeWalker(root, NodeFilter.SHOW_TEXT);
	const nodes: Text[] = [];
	for (let node = walker.nextNode(); node; node = walker.nextNode()) nodes.push(node as Text);
	return nodes;
}

// Scan every text node, then paint every hit, overlaps included, without adding an element.
// Each category collects ranges from all the nodes before its highlight is registered once.
// scan() keeps the lesson's 240-symbol teaching bound: lift MAX_TEXT_SCALARS for real articles.
export function paintGlossary(
	nodes: Iterable<Text>,
	glossary: Glossary,
	categories: readonly Category[],
	registry: HighlightRegistry = CSS.highlights
): void {
	const ranges = new Map<string, Range[]>(categories.map((category) => [category.name, []]));
	for (const node of nodes) {
		const offsets = codeUnitOffsets(node.data);
		for (const hit of scan(glossary, node.data)) {
			for (const category of categories) {
				if (!category.terms.includes(hit.term)) continue;
				const range = new Range();
				range.setStart(node, offsets[hit.start]);
				range.setEnd(node, offsets[hit.end]);
				ranges.get(category.name)!.push(range);
			}
		}
	}
	for (const category of categories) {
		const highlight = new Highlight(...ranges.get(category.name)!);
		highlight.priority = category.priority; // where categories overlap, the higher one styles it
		registry.set(`glossary-${category.name}`, highlight);
	}
}
// Usage: const glossary = buildCategories(categories); then, after each article renders,
// paintGlossary(textNodes(article), glossary, categories).
// In CSS: ::highlight(glossary-api) { background-color: …; } Only a few properties apply.

export type Segment = { text: string; hit: Hit | null };

// Links can't overlap, so choose: the earliest hit wins, then the longest. Hits that
// overlap a chosen one are left out. Render the segments as text and <a>, never as HTML.
export function linkSegments(text: string, hits: readonly Hit[]): Segment[] {
	const symbols = Array.from(text);
	const chosen = [...hits].sort((a, b) => a.start - b.start || b.end - a.end);
	const segments: Segment[] = [];
	let cursor = 0;
	for (const hit of chosen) {
		if (hit.start < cursor) continue;
		if (hit.start > cursor)
			segments.push({ text: symbols.slice(cursor, hit.start).join(''), hit: null });
		segments.push({ text: symbols.slice(hit.start, hit.end).join(''), hit });
		cursor = hit.end;
	}
	if (cursor < symbols.length) segments.push({ text: symbols.slice(cursor).join(''), hit: null });
	return segments;
}

The sketch builds one machine from every category’s terms when the glossary loads. After an article renders, paintGlossary scans each of its text nodes with that machine and collects every category’s ranges across all of them before registering one highlight per category, so no paragraph’s marks replace another’s. React and Svelte don’t change any of it: call it in an effect after the article renders, and render linkSegments with an ordinary keyed loop.

One thing to change before it ships: scan keeps this lesson’s teaching bound of 240 symbols per document, and a long paragraph is one text node. Raise or remove MAX_TEXT_SCALARS for real articles.

07 / Make the call

When a regex or indexOf is enough.

Reach for the machine when the terms stay fixed for a while, texts keep arriving, and every occurrence matters: glossary marks, tagging support tickets by keyword, or flagging terms in pasted text before someone reviews it.

Keep it simple otherwise. For one or two terms, indexOf or a regex is clearer. If you only want the longest term at each position and never overlaps, a regex of escaped terms sorted longest-first does exactly that. If the terms change on every keystroke, building a new machine each time may cost more than it saves, so measure before choosing.

Case folding, Unicode normalization, and word boundaries all change what counts as a match. Decide them in the input contract, with tests, before adding them to the machine.

SourcesThe original paper, match policies, and platform references

08 / Take the idea with you

Explain a glossary scan without saying “Aho–Corasick.”

“Put every term in a trie. Give each state a link to its longest ending that starts another term, and a link to the nearest term along that chain. Read the text once. When a letter doesn’t fit, follow links instead of rewinding, and report every term that ends where you are.”

Then explain the page: “The matcher reports every hit. The page decides what to paint, what to link, and which term wins an overlap.”

Before moving on, open a component that highlights or links terms and look for join('|'). Now you know which matches it can’t show you.

Connections to follow nextRelated lessons
  • Levenshtein distance finds a close name when the text isn’t an exact match.
  • Myers diff compares two whole versions of a text instead of searching one.
  • Queue and deque is the structure behind the breadth-first pass that links failures, and the Trie lesson builds the structure this machine starts from.