The machine, whole

The Elasticsearch essays · August 2026

Across this series, Elasticsearch has resolved into three machines wearing one coat: a document store that searches words, a columnar store that sweeps numbers, and a vector store that navigates meaning. This short capstone puts them side by side, and shows the single query that plays all three at once.

One machine, three stores

Here they are, then, side by side. A document store: whole JSON in _source, analyzed text in inverted indexes, match and score, the home of search and logs. A columnar store: every keyword, number and date field also kept as doc values, swept by aggregations and compressed to a few bytes per metric point, the home of dashboards and time series. A vector store: embeddings quantized by BBQ and navigated by HNSW, the home of semantic retrieval and RAG. Not three products glued together, one write path, one cluster, one query planning all three. Click a question:

"Which errors mention timeouts?"

Analyze the query → walk posting lists in the inverted index → BM25-rank the matches. Pure document store.

"Average checkout latency, per hour, this week?"

No documents are read at all: a date_histogram sweeps the duration column and buckets it. Pure columnar store.

"Incidents similar to this description?"

Embed the description → hop the HNSW graph → rescore finalists exactly. Pure vector store.

"Errors like this one, in checkout, under 2s, last hour, ranked by relevance"

All three at once: keyword and range filters prune, BM25 and kNN both rank, RRF fuses. One query, three engines.

Document store

inverted index · match and score

Columnar store

doc values · sweep and aggregate

Vector store

HNSW + BBQ · navigate meaning

Click each question: the stores that light up say what they do for it. The first three questions each need one store; the last uses all three in a single request.

One query, every engine

Here is the whole machine at work: a support engineer types "customers report payment failures during checkout", scoped to the last 24 hours and the checkout service, hybrid-ranked, reranked, top ten. Scrub the query through its life:

the query's life

Bar heights are on a log scale. Ten million documents in, ten out, and each stage is one of this series' essays doing its job: structures decide what's eligible, two rankers decide what's relevant, fusion decides what's agreed, and a heavyweight model decides what's first. Every count on the way down is the machine spending as little as possible at each level of care.

Start anywhere: search, the index itself, ranking, segments, distribution, the cluster, numbers, metrics, TS and PromQL, ingest, lifecycle, columnar logs, vectors, HNSW, semantic search, ES|QL, geo.

The machine, whole

One distributed machine, then, with several specialized memories: an inverted index mapping tokens, prefixes and even misspellings to documents; doc values laying every keyword, number and date field out as sweepable, mergeable, approximately-summarizable columns; BKD trees for ranges; an HNSW graph over quantized vectors arranging meaning geometrically. All fed by one write path of immutable segments, refined by pipelines, aged across tiers by lifecycle policies, replicated and healed by one cluster, ranked by two judges and a referee, and addressed in whichever language fits the question.

And every single part makes the same wager: do generous, careful work when data arrives, so that questions cost almost nothing. The next time an answer appears before your finger leaves the key, you'll know the trick: the machine answered fast because, in a very real sense, it did the work before you asked.