A metric data point is the smallest document imaginable, arriving millions of times per second, forever. Stored as an ordinary document it drags along machinery built for documents, and the wrapping weighs more than the gift. This essay walks the rebuild, and every figure is an experiment: you apply the engineering yourself, one dial at a time. (The columnar foundations live in Numbers at scale.)
A metric data point is the smallest document imaginable: a metric name, a few identifying labels (which host, which service), a timestamp, a number. Perhaps twenty bytes of actual information, arriving millions of times per second, forever. Stored as an ordinary document, each point drags along machinery built for documents: an _id, sequence numbers for concurrency control, index entries per field. The wrapping weighs more than the gift; historically, about 25 bytes per point.
Elasticsearch's time-series mode (TSDS) attacks that overhead with three ideas, and the figure below lets you apply them one at a time, in any order. Each colored square is one data point from one of three series; the bar underneath is where its bytes actually go:
Walk the toggles and you've walked the engineering. Sort by series, then time, and route each series to a single shard, so every series is one contiguous, sequential run on disk; nothing else works without this. Then the big one: for timestamps and dimensions, lightweight per-block skip summaries replace the inverted index and BKD tree entirely, deleting the duplicated per-field index machinery, about ten of the twenty-five bytes, while the skip-driven queries later in this essay stay fast. The _id becomes synthetic, computed as _tsid plus timestamp instead of stored and indexed (a bloom filter keeps get-by-id working), five more bytes. Sequence numbers are trimmed because metrics never see concurrent updates: four bytes of replication metadata gone. And the numeric codec blocks widened from 128 to 512 values, so repeating dimension patterns, IPs and MAC addresses cycling host by host, finally fit inside the compressor's window. Together: 25 bytes become 3.75, and a day of a billion points shrinks from 25 GB to under 4.
Queries got the mirror treatment. Because the data is sorted by series and time, each series is a run of compressed blocks, and each block carries a tiny label: which series, which time span. A metrics query almost always begins with where and when, "payments, last four hours", and both conditions are answered by reading labels, not data. Blocks that can't match are skipped before they are ever decompressed. Steer the query yourself:
What happens to the surviving blocks is just as deliberate. They decode zero-copy, straight into the primitive-typed blocks the compute engine works on, no intermediate row objects, no allocator churn; a dimension that repeats for a thousand consecutive points isn't even expanded, it becomes a constant block, one value and a count. The work parallelizes by slicing the _tsid range by prefix, so every thread still reads its slice of disk in order. Even the humble act of rounding timestamps into buckets was rebuilt: when the bucketing is timezone-agnostic, it collapses to a modulo, worth a third of the response time on some queries. None of these is a headline; together they are the headline.
The most important question you can ask of a counter, how fast is it growing?, hides a trap. Counters only go up, until the process restarts and the counter falls back to zero. Subtract naively across a restart and you compute a huge negative rate; a dashboard built on that lies to you at exactly the moments that matter, the restarts. Elasticsearch's rate() walks each series in time order (the sort again) and treats every drop as a reset, adding the lost baseline back:
This is what the TS command in ES|QL is for: it computes functions like rate() inside each series first, in one ordered pass, and only then aggregates across series, with sliding windows built from reusable per-bucket partial results. Time-series math expressed in the same piped language as everything else.
None of this matters if the data can't get in fast enough, so the write path was rebuilt around the same shape. Toggle the three optimizations and watch the cost of accepting one million points per second:
One last elegance: PromQL, the language most of the world's dashboards already speak, runs against these very same column files. Same storage, two dialects; nobody rewrites their dashboards to switch engines. And when this data ages, the lifecycle machinery of the life of a log can downsample it, replacing raw points with pre-computed rollups, another order of magnitude cheaper.
When metric data ages, lifecycle management downsamples it: raw points become rollups, another order of magnitude cheaper.