Trace sampling
Sampling decides which traces your workspace keeps for search and the trace waterfall. Tracix samples at the APM gateway, after each trace completes, so it can keep every trace with an error and every slow one, and only a share of the rest.
How sampling works
Tail sampling is off by default: the gateway stores every span it receives.
When you turn it on, the gateway waits for each trace to complete and keeps it when:
- Any span has an error. Always.
- Its duration is at least the slow threshold, 500 ms by default.
- Otherwise, a random sample of the remaining traces, 10% by default.
Service metrics, service health and the service map are computed from every span before sampling, so they stay complete. Sampling applies only to traces, and only to the ones that arrive through the APM gateway: metrics and logs are never sampled.
Keep whole traces in your applications
Tracix has no head sampling control in the console. The commands on the Setup page don't set a sampler, so SDKs keep every trace (parentbased_always_on), and the gateway decides with the whole trace in view. If you set a sampler in an SDK, the gateway only sees what it keeps.
Set the sampling policy
You need permission to edit policies, which the Editor and Admin roles have.
Open Ingestion
In your workspace, open Applications › APM & Traces, then the Ingestion tab. The Sampling policy card shows the gateway's policy and its revision.
Turn on tail sampling
Turn on Tail sampling, then set Duration is at least (the slow threshold, in milliseconds) and Otherwise keep a sample of (the share of other traces, in whole percent).
Check the estimate
Before you save, Estimated effect of unsaved changes shows how much each service would keep, from the last hour of span data. Drop rules aren't included in the estimate.
Save
Select Save and apply. The gateway picks the change up with its next configuration push. If someone else changed the policy since you opened the page, the save is refused: reload and apply your change again.
Override the rules for a service
Under Service overrides, select Add override and enter the exact service.name. For that service you can set:
- Sample percentage: its own share of other traces. Fractions such as
0.5are allowed. - Slow threshold: its own slow threshold, in milliseconds.
- Drop when attribute … equals any of: drop the service's traces when an attribute matches one of the values, such as
http.routeequal to/healthzor/readyz.
An override applies to whole traces that pass through the service. When a trace passes through several services with overrides, the most permissive one wins. Drop rules win over everything else, errors included, so use them only for traffic you never need, such as health checks.
Overrides apply only while tail sampling is on.
See what's kept
The Ingestion tab shows what reaches the gateway and what it stores:
| Figure | Meaning |
|---|---|
| Received | Spans that reached the gateway |
| Stored | Spans written to your workspace |
| Keep rate | Stored as a share of received |
| Kept for errors, Kept as slow, Baseline sample | Why traces were kept |
The By service table breaks these down per service and shows which rule applies to it: Override, Gateway default, All stored, or OBI, not sampled for services traced with eBPF, which don't pass through the gateway. When an error points to a trace that wasn't kept, Tracix says the trace was not stored by tail sampling.