OpenMetrics 1.0 vs 2.0 scrape demo

The number of series each scrape stores. If OpenMetrics 1.0 and 2.0 carried the same data, go-om1 would match go-om2 and java-om1 would match java-om2.

count by (job) ({job=~"(go|java)-om[12]"})

Both panels are empty when the two scrapes store exactly the same series. The Go example does not enable _created series for OM1 (EnableOpenMetricsTextCreatedSamples), while OM2 always carries start timestamps inline as st@, which Prometheus does not store as series. The UTF-8 gauge http.server.active_requests is stored as is with OM1, but as http_server_active_requests with OM2, as Prometheus does not ask for escaping=allow-utf-8 when scraping OM2.

group without (job) (label_replace({job="go-om1"}, "name", "$1", "__name__", "(.+)"))
unless
group without (job) (label_replace({job="go-om2"}, "name", "$1", "__name__", "(.+)"))

group without (job) (label_replace({job="go-om2"}, "name", "$1", "__name__", "(.+)"))
unless
group without (job) (label_replace({job="go-om1"}, "name", "$1", "__name__", "(.+)"))

For client_java with OpenMetrics 2.0 enabled, OM2 exposes counters without the _total suffix, e.g. http_requests instead of http_requests_total, so they are different series. The same UTF-8 escaping difference as for Go applies.

group without (job) (label_replace({job="java-om1"}, "name", "$1", "__name__", "(.+)"))
unless
group without (job) (label_replace({job="java-om2"}, "name", "$1", "__name__", "(.+)"))

group without (job) (label_replace({job="java-om2"}, "name", "$1", "__name__", "(.+)"))
unless
group without (job) (label_replace({job="java-om1"}, "name", "$1", "__name__", "(.+)"))

The rates of all 4 jobs line up, with a regex on the name to also match the Java OM2 counters without _total. They only differ by when each job scrapes.

sum by (job) (rate({__name__=~"http_requests(_total)?", job=~"(go|java)-om[12]"}[1m]))

sum by (job) (rate({__name__=~"http_request_size(_bytes_total)?", job=~"(go|java)-om[12]"}[1m]))

The same gauge, with the same value 3, is found under its UTF-8 name for the OM1 jobs and under its escaped name for the OM2 jobs.

{"http.server.active_requests"}

http_server_active_requests

OM2 exposes each histogram as one composite sample, {count:...,sum:...,bucket:[...]}, which Prometheus stores as the same classic _bucket, _count and _sum series as the OM1 ones. The native buckets of the composite sample are ignored, as scrape_native_histograms is off.

histogram_quantile(0.9, sum by (job, le) (rate(http_request_duration_seconds_bucket{job=~"(go|java)-om[12]"}[1m])))

sum by (job) (rate(http_request_duration_seconds_count{job=~"(go|java)-om[12]"}[1m]))

Summaries are composite samples in OM2 too, {count:...,sum:...,quantile:[...]}, stored as the same quantile, _count and _sum series.

rpc_latency_seconds{quantile="0.99", job=~"(go|java)-om[12]"}

sum by (job) (rate(rpc_latency_seconds_sum{job=~"(go|java)-om[12]"}[1m])) / sum by (job) (rate(rpc_latency_seconds_count{job=~"(go|java)-om[12]"}[1m]))

Both opened with exemplars shown. OM1 attaches histogram exemplars to the _bucket series of their bucket, while the exemplars of an OM2 composite sample are stored on the _count series, so the OM1 jobs show them in the first panel and the OM2 jobs in the second.

sum by (job, le) (rate(http_request_duration_seconds_bucket{le=~"0.05|0.1|0.25", job=~"(go|java)-om[12]"}[1m]))

sum by (job) (rate(http_request_duration_seconds_count{job=~"(go|java)-om[12]"}[1m]))