1 Definition and characteristics

High-cardinality metrics are metrics whose identifying dimensions—typically labels or tags—can assume a very large number of distinct combinations. Each distinct combination corresponds to a separate time series. When these dimensions include unstable identifiers (such as request IDs, session tokens, or full URLs with variable components), the number of time series can grow quickly, even if the underlying measurement (e.g., latency) is straightforward.

1.1 Cardinality in metrics systems

Cardinality refers to the number of unique values for a label and, more importantly, the number of distinct label combinations that define series identity. In systems where metrics are stored and queried as time series, cardinality determines how many series must be ingested, indexed, retained, and scanned to answer queries.

A key property is that cardinality is often multiplicative across multiple labels: if one label has 1,000 distinct values and another has 100, the theoretical upper bound on series count can reach 100,000. Real-world distributions are frequently skewed, but even partial skew can still create large series inventories.

1.2 Time-series growth and resource implications

When cardinality is high, the system must manage more series objects and associated metadata. This increases:

  • Ingestion workload (more series to create, update, or route)
  • Storage footprint (more persistent series data or compressed chunks)
  • Index size (more entries for label values and combinations)
  • Query cost (more series to filter, aggregate, and transmit)

The effect is not limited to steady-state. High-cardinality sources may create bursts (for example, a temporary spike in request IDs or newly generated paths), which can lead to ingestion lag, query timeouts, or memory pressure in query engines.

1.3 Typical sources of high cardinality (labels, IDs, paths)

Common sources include:

  • Unique identifiers: user IDs, request IDs, transaction IDs, correlation IDs
  • Full URLs and paths: including variable segments or IDs inside the path
  • Query parameters: using raw key/value parameter combinations as label values
  • Per-session dimensions: labels tied to short-lived sessions or browsers
  • Dynamic resource identifiers: labels derived from ephemeral resource names in autoscaling environments

In many designs, these values are appealing because they appear directly “diagnostic.” However, metrics are typically intended for aggregation and trend analysis, so embedding uniqueness tends to conflict with their purpose.

1.4 Measuring cardinality (concepts and estimations)

Cardinality is measured at several levels:

  • Per-label distinct counts: number of unique values observed for a specific label
  • Series count: number of unique combinations of labels that constitute distinct time series
  • Growth rate: how quickly new series appear over time

Because exact counts may be expensive, systems often use approximations (e.g., sketch-based estimators) or sampling from historical data. Operational teams may compute cardinality over time windows (daily or hourly) to understand not only the current size but also whether growth is trending upward.

2 Impact on observability pipelines

High-cardinality metrics affect the entire pipeline from instrumentation through storage and querying. The failure mode is frequently gradual: systems become “slower” until they become unreliable, with resource bottlenecks appearing in unexpected components.

2.1 Ingestion and write amplification

When each incoming observation maps to a distinct label combination, ingestion must perform additional work: create series, update indexes, and route data for many targets. This can create write amplification—more operational effort per measured event. Even if each individual metric update is small, the cumulative overhead of managing series identity and label metadata can dominate.

Backpressure can occur when ingesters cannot keep up, causing dropped samples, increased latency, or queue growth. In some architectures, routing or fan-out to multiple storage shards may intensify these effects.

2.2 Storage and indexing costs

Storage cost arises both from the raw or compressed time-series samples and from the metadata required to locate and interpret series. Label indexes can become a major contributor to memory and disk usage, particularly when label values vary widely or when many combinations exist.

Indexing costs also extend the lifecycle of the data: compaction and retention processes must consider more series, and the system may spend more time maintaining structures needed for efficient query execution.

2.3 Query latency and memory pressure

Query engines commonly filter by label matchers and then aggregate across selected series. With high-cardinality dimensions, the number of candidate series can be enormous, leading to:

  • Longer scan time across series
  • Larger intermediate results (e.g., per-series reductions)
  • Higher memory usage for holding partial aggregates or label sets

Memory pressure can manifest during peak dashboard viewing, ad hoc investigations, or alert evaluations that rely on broad label filters.

2.4 Alerting and dashboard scalability issues

Dashboards often query many panels concurrently. If panels include label dimensions with high churn or wide value sets, the number of series returned can multiply across panels. This can lead to slow page loads, repeated expensive queries, and user-visible timeouts.

Alerting systems, though typically configured with smaller query windows, can still suffer when alerts include non-selective label matchers. An alert rule that effectively scans a huge series set can become unstable as cardinality grows, increasing both cost and the chance of delayed firing.

3 Causes and instrumentation patterns

High cardinality is usually introduced intentionally by instrumentation choices, often driven by a desire for traceability or “ground truth.” Understanding the common patterns helps teams redesign metrics without losing diagnostic value.

3.1 High-entropy label values

Entropy describes how unpredictable a label value is across observations. Labels derived from random or unique elements (e.g., request IDs) have high entropy and therefore high cardinality. Labels derived from stable attributes (e.g., service name, environment, region) have lower entropy and are generally safer.

The difference matters because metrics systems are designed to aggregate across repeated measurements. When the label space resembles identifiers rather than categories, the aggregation surface shrinks while the number of series grows.

3.2 “Tagging everything” anti-patterns

A frequent anti-pattern is attaching every known field—especially those from logs or traces—directly as metric labels. While this seems to preserve flexibility, it effectively converts metrics into a fragmented index of events. The result is difficult querying and expensive storage with limited incremental analytical benefit.

A more sustainable approach is to treat metric labels as dimensions of interest: values that define meaningful groupings suitable for aggregation and comparison.

3.3 URL/path and query-parameter expansion

Including the full path template expansion is a classic source of cardinality. If an application includes resource identifiers in the URL (e.g., /users/12345/orders/6789), the number of distinct paths approaches the number of entities and requests. Similarly, query parameters used verbatim can explode the label space because parameters may vary by user, time, or experimentation.

Common mitigations involve using normalized route templates (e.g., /users/{id}/orders/{id}) or extracting only low-cardinality attributes such as HTTP method and coarse route category.

3.4 Per-user/per-session labels

Per-user and per-session dimensions are tempting because they appear to directly answer “who is affected.” In practice, metrics are better suited for cohort-level aggregation. Per-user labels typically create one series per user or per active session, causing both storage growth and operational instability.

If per-user analysis is required, it is often better handled by logs, tracing, or specialized analytics systems designed for entity-level investigation rather than time-series aggregation.

3.5 Multi-tenant and dynamic resource identifiers

Multi-tenant systems sometimes label metrics with tenant identifiers. This can be acceptable when the tenant set is bounded and stable, but it becomes risky if tenants are created dynamically or if identifiers include unique resource instance names (for example, ephemeral cluster nodes or job IDs).

Dynamic naming tied to autoscaling or job orchestration can create continuous churn. Even if each identifier is “just another label value,” the churn translates to new series creation and eventually to resource strain.

4 Mitigation strategies

Mitigation aims to retain the usefulness of metrics while controlling series explosion. Strategies are typically combined: redesign labels, shift some data to other telemetry types, and enforce governance so the problem does not reappear.

4.1 Label design and cardinality budgeting

Cardinality budgeting assigns expected upper bounds to the number of distinct series a metric might generate. This encourages deliberate choices about which labels are necessary and which can be removed or transformed.

Budgeting can be implemented early in the design process by estimating distinct counts per label (based on known value ranges) and then assessing the combined series count. Even rough estimates help teams avoid designs that are likely to exceed operational limits.

4.2 Normalization and bucketing

Normalization transforms variable forms into stable categories. For URLs, this often means mapping raw paths to route templates or coarse categories. For identifiers, this can mean mapping to known buckets (e.g., “small,” “medium,” “large” based on size thresholds) rather than using exact values.

Bucketing reduces label entropy while preserving analytical intent. The trade-off is reduced granularity, which teams must judge against their observability goals.

4.3 Aggregation and rollups (precompute vs query-time)

Aggregation reduces the number of series by computing higher-level summaries before storage or during query execution. Two common styles are:

  • Precompute/rollups: compute and store aggregated metrics (e.g., by region and endpoint group) so queries read fewer series.
  • Query-time aggregation: rely on the query engine to aggregate many series on demand.

Precompute generally reduces query cost and can improve dashboard responsiveness. Query-time aggregation avoids additional storage but can shift the burden to the query engine, which may still struggle if series counts are very large.

4.4 Sampling and downsampling approaches

Sampling reduces the volume of observations that lead to new series creation or label combinations. Downsampling can also reduce retention granularity. However, sampling does not inherently fix cardinality if the labels remain unique—sampling may reduce total points but can still create many series identities.

Sampling is most effective when it is paired with label redesign, such that the remaining series correspond to stable dimensions.

4.5 Redaction/limiting unique identifiers

Redaction removes or replaces unique fields that create one-off label values. For example, request IDs can be omitted from metric labels entirely. If linkage to debugging workflows is needed, some teams use alternative correlation mechanisms (such as exemplars) rather than full identifier labeling.

Limiting refers to placing strict caps on which identifier values can enter labels, often by allowing only a bounded subset of values (e.g., only the latest N) or by truncating values with careful understanding of collision behavior.

4.6 Limiting or filtering label sets at ingestion

Ingestion-time filtering ensures that metrics violating governance rules do not overwhelm the system. This can include dropping metrics with disallowed label keys, restricting label values to normalized forms, or truncating label sets to approved dimensions.

Such controls are most effective when combined with feedback to instrumentors, so teams learn which labels are risky and can correct future releases.

5 Detection and governance

Because high cardinality can be introduced gradually (through new features or instrumentation updates), detection and governance provide early warning and durable prevention.

5.1 Automated cardinality monitoring

Automated monitoring tracks cardinality metrics such as series count per metric name, distinct values per label, and rate of new series creation. These signals are often exported as separate operational metrics and visualized similarly to other SLO/SLA indicators.

Effective monitoring distinguishes between normal seasonal changes and pathological growth. The goal is to detect runaway series creation before it impacts ingest or query performance.

5.2 Thresholds and anomaly detection

Thresholds can be static (a fixed upper bound) or dynamic (relative to a baseline). Static thresholds are simpler but may be brittle across environments. Dynamic thresholds use historical data to define what “normal” looks like.

Anomaly detection may trigger when cardinality increases rapidly over a short window, which can reflect instrumentation changes or unintended label expansions.

5.3 Ownership and instrumentation review processes

Governance frequently includes identifying an owner for each metric and requiring review for label schema changes. Ownership clarifies accountability: who updates dashboards, who understands the metric’s intent, and who responds when cardinality spikes.

Instrumentation reviews can focus on whether label keys are necessary, whether label values are stable, and whether any label could inadvertently become a unique identifier.

5.4 Schema and naming conventions for labels

Schema conventions standardize label meaning and reduce accidental proliferation. Examples include:

  • Reserved label keys for approved dimensions (service, environment, region)
  • Prohibiting high-entropy label keys except under controlled exceptions
  • Consistent formatting for normalized dimensions (e.g., route categories)

Naming conventions help both human operators and automated tools reason about which dimensions are safe to query.

6 Alternatives and complementary techniques

High-cardinality problems are sometimes best addressed by using other telemetry types or complementary techniques that fit the analysis goal.

6.1 Using exemplars for trace linkage

Exemplars are references attached to metric points to enable linking to representative traces. This allows operators to jump from aggregated latency or error rate trends to individual requests without turning request IDs into permanent metric labels for every observation.

Exemplars typically provide a small sample of trace linkage rather than exhaustive identifiers, preserving metric aggregation performance while improving debuggability.

6.2 When to prefer logs or traces over metrics

Logs and traces are better suited for event-level investigation where uniqueness matters. If a question requires examining specific user journeys, request details, or exact payload attributes, logs or traces can capture those without inflating metric cardinality.

A common division of labor is: metrics answer “how is the system behaving over time,” while logs and traces answer “what happened for this specific instance.”

6.3 Deriving metrics from events with controlled dimensions

Events can be transformed into metrics as long as the derived dimensions are controlled. For instance, from request events, teams can derive metrics grouped by method and route template, while excluding raw user identifiers and variable path segments.

This approach preserves the advantage of metrics (aggregation and alerting) while ensuring the label space remains bounded.

6.4 Metric-to-dimension redesign workflows

Redesign workflows treat label schemas as evolving artifacts. A typical process includes:

  1. Identify the metric causing cardinality strain.
  2. Determine which labels are high entropy.
  3. Replace raw identifiers with normalized dimensions or remove them.
  4. Update dashboards and alert rules accordingly.
  5. Monitor cardinality and functional correctness after rollout.

Over time, this can reduce recurrence of similar issues as teams learn from past instrumentation mistakes.

7 Querying and operational best practices

Operational practices help maintain performance when high-cardinality metrics cannot be eliminated quickly, and they reduce the cost of normal day-to-day work.

7.1 Writing queries that avoid unbounded label filters

Queries should avoid matchers that select broadly across high-cardinality label values. For example, filtering on “any value of request-id” or using regular expressions over variable path segments can cause the query to touch most series.

Instead, queries should prefer stable dimensions and narrow time windows. When exploring, operators can also use stepwise narrowing: start with coarse aggregation and progressively refine.

7.2 Choosing aggregation strategies (sum, rate, histogram approaches)

Selecting the correct aggregation pattern can reduce intermediate results. For example:

  • Rates: aggregating error rates often uses per-second normalization (e.g., based on counters).
  • Histograms: distributions can be computed from bounded bucket labels rather than embedding unbounded identifiers.

The best choice depends on the metric type and the intended question. Correct aggregation also reduces the tendency to “group by everything” merely to find a pattern.

7.3 Dashboard patterns for high-cardinality environments

Dashboards should be designed to keep query result sizes manageable. Common patterns include:

  • Limiting the number of dimensions used in “group by” clauses
  • Providing top-N breakdowns by a stable metric, rather than enumerating all label values
  • Using precomputed rollups where available
  • Caching or reusing shared query fragments across panels

These patterns prevent user-facing lag and reduce the risk of accidental expensive queries.

7.4 Alert rules that remain stable under label churn

Alerting rules should be resilient to label-value churn. This typically means avoiding alerts that require tracking every unique label value or matching on unstable dimensions. Instead, alerts should focus on aggregations across stable dimensions and on metric thresholds derived from bounded series sets.

Operationally, alert rules can also incorporate guardrails such as minimum sample counts and broader aggregations to reduce sensitivity to label churn artifacts.

8 Case studies and worked examples

The following worked examples illustrate practical redesigns that reduce cardinality while keeping observability outcomes intact.

8.1 Reducing cardinality for HTTP request metrics

Initial design: An application exports http_request_duration_seconds with labels including method, path, and status. The path label is populated with the raw request path (e.g., /users/123/orders/456), creating a near-unique value for many requests.

Observed impact: Series count grows rapidly, ingestion slows, and dashboards become sluggish because queries grouped by path return too many series.

Redesign:

  • Replace path with a normalized route template such as /users/{id}/orders/{id}
  • Keep method and status as stable dimensions
  • Optionally add an additional low-cardinality label like service or environment

Result: Series identity becomes bounded by the number of known route templates rather than the number of unique URLs. Alerts and dashboards can now aggregate across routes reliably.

8.2 Migrating from per-user labels to cohort/group labels

Initial design: Metrics include user_id as a label to help identify “problem users.” Each active user produces a distinct time series.

Observed impact: Cardinality is proportional to the active user base and grows with new users. Query engines require scanning large series sets, and storage retention becomes expensive.

Migration:

  • Remove user_id from metric labels
  • Replace with a cohort label derived from stable attributes, such as plan_tier or account_age_bucket
  • For individual user debugging, use logs or exemplars to trace from aggregated symptoms to specific instances

Result: The metric remains useful for identifying systematic issues affecting groups, while individual investigation shifts to telemetry designed for entity-level analysis.

8.3 Handling ephemeral resources in autoscaling systems

Initial design: In an autoscaling environment, metrics include instance_id or pod_name as labels. Each new instance yields new series.

Observed impact: Cardinality churn is continuous; dashboards show shifting label values and alert evaluations become inconsistent in cost. The number of series can spike during scaling events.

Mitigation:

  • Remove per-instance identifiers from metric labels
  • Use stable labels such as service, cluster, and deployment (where deployment identity is bounded)
  • If instance-level visibility is needed temporarily, use logs/traces or a separate, controlled metric with strict limits

Result: Metric aggregation tracks system-level behavior across stable groupings, reducing both time-series churn and operational instability during scaling.