ES|QL: one language for the whole machine

The Elasticsearch essays · August 2026

Elasticsearch speaks several dialects, but its newest and most expressive voice is ES|QL, a piped language in which each stage transforms a table. The readable syntax is the visible tip of an entire query engine, and this essay descends through all of it: what the planner does to your pipes, and how pages of columnar blocks flow through operators at execution time.

How do you talk to all this? The classic interface is the Query DSL, JSON in, JSON out, and specialized dialects exist for security event sequences (EQL), BI tools (SQL), and Kibana's search bar (KQL). But the newest and most expressive voice is ES|QL, a piped language in which each stage transforms a table, and which can now even speak PromQL for metrics. Build the query stage by stage:

FROM logs-payments | WHERE status_code >= 500 | STATS errors = COUNT(*), p95 = PERCENTILE(duration_ms, 95) BY service | SORT errors DESC | LIMIT 2

Click the grayed-out stages to switch them on, in order, and watch the table transform: filter, then aggregate, then sort, then trim. Each pipe hands a table to the next.

What the machine does with your pipes

That readable pipe syntax is the visible tip of an entire query engine. Between your keystrokes and the answer lies an assembly line: the query is parsed into a tree, analyzed against the mappings so every field resolves to a real, typed column, optimized by dozens of rewrite rules, and finally split into the piece that ships to every data node and the piece that stays behind to merge. Drag through the four phases and watch what happens to one real query; green marks what each phase changed:

FROM retro_arcade | WHERE country == "gb" | STATS sales = SUM(price * 0.75) BY year = BUCKET(date, 1 year)

Step through the four phases: the parser only knows syntax (every ? is an unresolved name); the analyzer binds names to typed columns and quietly adds a LIMIT 1000 safety net; the optimizer folds constants and hoists the arithmetic out of the aggregation; the physical planner cuts the tree in two at the exchange. Green marks what each phase changed. Errors are caught here, before execution; at runtime a bad value becomes null and a warning, never a crash.

Notice the deepest cut, in the last phase: the aggregation splits into a partial half that runs on every data node and a final half at the coordinator, the exact same partial-then-merge pattern from distributed search. And each data node re-optimizes its own fragment locally: a node whose indices don't even have the field can answer with a shrug of nulls without reading anything.

The compute engine: pages, blocks, drivers

Execution belongs to a purpose-built engine that never processes rows one at a time. Data moves in pages, batches of rows, where each column of the page is an immutable block pulled straight from the doc-values columns. Chains of operators, one small machine each for reading, filtering, computing, aggregating, transform the pages, and a driver runs each chain, one driver per segment, so the parallelism of segments becomes the parallelism of your query. Run it:

timeline

parallel drivers3

rows per page4

Each driver pulls pages from its segment through the operator chain: read columns, filter (rows visibly drop), compute the new column (it appears), then fold into a partial sum. Partials stream to the coordinator, which merges them into the final answer.
The experiments: drop to 1 driver and the same 24 rows take three times as long, exactly the segments story. Change the page size and watch the trade: big pages amortize overhead, small pages start streaming results sooner.

The drivers here parallelize over segments; the partial-then-final aggregation is distributed search's oldest trick; the blocks are doc values in flight.