Logs and pipelines
Logs reach Tracix from files, journald, Windows Event Log, syslog, Kubernetes pods, cloud services and OpenTelemetry SDKs. This page shows how to search them in the Log Explorer, parse and clean them with pipelines that run in your collectors, and decide how long each source is kept.
Before you begin
Turn on a log source first: a logs integration in a collector policy, Kubernetes container logs, or the logs your OpenTelemetry SDKs export. Every line is stored with OpenTelemetry fields such as the timestamp, severity, body, trace ID and resource attributes, under the dataset it came from.
Search logs
Open Insights › Log Explorer. It starts on the last 15 minutes. Type a query in the filter bar, narrow it with the facets on the left, and open a line to see all of its fields.
| To find | Type | Notes |
|---|---|---|
| Words in the message | error timeout | Every word must appear. Words match whole: err does not match "error", but *err* does. |
| A phrase | "connection reset" | |
| A field value | service:checkout-api | Exact and case-sensitive. |
| A value pattern | host:ip-172* | * at the start or end. Case-insensitive. |
| Any of several values | status:(error OR crit) | Works within one field. |
| Everything except | -service:healthcheck | |
| Lines that have a field | http.route:* | -http.route:* finds lines without it. |
| A value with spaces | user.name:"Dana Ito" |
service, status (crit, error, warn, info, debug), host, source, env, dataset and trace_id are built in. Any other key is looked up in the log attributes, then the resource attributes, then fields in a JSON body.
The facet rail filters by status, dataset, service, host and source, plus facets you add. Values you pick in one facet combine with OR, and facets combine with AND.
Patterns, transactions and charts
- Group into turns the list into Patterns, which collapse similar lines with IDs, addresses and numbers masked so a new kind of error stands out, or Transactions, which stitch lines together by a key such as
host. - In Fields mode, Visualize as charts any query as a Timeseries, Top List, Bar Chart, Table, Tree Map, Pie Chart or Scatter Plot, counting lines or measuring a field with sum, average, minimum, maximum, distinct count or percentiles. Save to dashboard puts the chart on a dashboard.
- The Views menu saves the query, columns and chart as a view, and Copy share link shares it. CSV downloads the results.
Follow new lines
There is no streaming live tail. To watch new lines arrive, keep a relative time range such as the last 15 minutes and set the time picker's Auto refresh to 10s, 30s, 1m or 5m. Each refresh runs your query again.
From a log line to its trace
A log line's trace_id shows as a value, not a link: the Log Explorer cannot open the trace from a line yet. Search trace_id:<id> to see every line from the same request. Going the other way, a trace's Logs tab lists the lines that carry its ID. See APM and tracing.
Pipelines
A pipeline is an ordered list of steps for one dataset. Tracix compiles it to the OpenTelemetry Transformation Language (OTTL) and runs it in your collectors, after Kubernetes and host metadata are added and before anything is stored. Pipelines parse, enrich and filter: they turn raw lines into fields, add and rename fields, redact secrets and drop noise. They do not route logs anywhere.
Step types
| Step | What it does |
|---|---|
parse_json | Parses a JSON body or field into fields. |
regex | Extracts fields with an RE2 regular expression and named groups, such as (?P<status_code>\d{3}). There is no grok or dissect syntax. |
kv | Parses key=value pairs, with your own delimiters. |
csv | Splits a delimited line into the columns you name. |
timestamp | Sets the record's time from a field, with a strptime layout. A layout without a time zone is read as UTC. |
severity | Sets the severity from a level field, or from keywords in the message. |
rename, add, drop_field | Rename, add or remove a field. |
compute | Sets a field from an expression. |
redact | Replaces matches of a pattern, with [REDACTED] by default. |
filter | Drops the lines an OTTL condition matches. |
ottl | Raw OTTL statements, for anything else. Flagged for review. |
Most steps take an optional if (OTTL) condition, so they apply only to the lines that match it. A pipeline has at most 128 steps.
Build and publish a pipeline
Open the pipeline editor
Open Settings › Data Management › Pipelines, or select Fix in pipeline in the Log Explorer while it shows a single dataset. The left rail lists your pipelines, the datasets, Tracix's managed pipelines and a Needs pipeline queue of datasets that arrive unparsed.
Add steps
Pick a dataset, or select new dataset…, then choose steps from ＋ Add step…. Generate drafts steps for you when your workspace has an AI model connected. Managed pipelines are read-only: select Duplicate to customize one.
Simulate on real lines
Select Load real samples to pull recent lines from the dataset, then Run simulate. The simulator runs the same engine your collectors run, on up to 25 lines, and shows each step's output. Select Save draft when it looks right.
Try it on one collector
Choose a canary agent and select Canary to run the draft on a single collector first. Stop canary ends the trial.
Publish
Select Publish to fleet, or Promote to fleet from a canary. Every active policy is rendered again and your collectors reload.
Pipelines apply to new data only
Lines already stored keep the fields they arrived with. Publishing never parses old data again.
Every version is kept. History lists them, and Restore as draft brings an old version back to edit. Archive retires a pipeline. Changing pipelines needs permission to manage log pipelines, which admins have by default.
How pipelines are applied
- One pipeline per dataset. A pipeline you publish takes precedence over a built-in or integration pipeline for the same dataset. A custom log source with no pipeline attached is stored raw until you choose a parsing pipeline in its settings.
- The final pipeline runs last on every log stream, parsed or raw. Use it for rules that apply to everything, such as redaction.
- Failures are tagged, not dropped. When an extraction step's condition held but it extracted nothing, the line is kept unchanged and tagged with
tracix.pipeline.failure.step. A parse failures facet then appears in the Log Explorer. - Every output gets every log stream. A collector can export to other OTLP endpoints besides your workspace, but there is no per-source routing: each output that accepts logs receives all of them.
Data lifecycle
Lifecycle rules decide how long data stays on fast local storage, when it moves to object storage and when it is deleted. Open Settings › Data Management › Lifecycle.
| Data | Default |
|---|---|
| Most tables | Cold after 1 day, deleted after 30 days |
| Security log datasets | Cold after 30 days, deleted after 396 days |
| Software inventory | Deleted after 90 days |
Cold data stays searchable through the same tables, so queries do not change. Moves and deletes happen as the database merges its data, within minutes.
Change how long a source is kept
Pick the table or dataset
Under Lifecycle policies, find the table. Tables that hold logs and metrics expand into a row per dataset, so you can keep authentication logs longer than container output. A dataset row says whether it inherits table default or overrides table default.
Set the days
Set Cold after, the Cold store and Delete after, then select Apply.
Check what shortening removes
If the new rule deletes data sooner, Tracix counts the rows and bytes it will remove and shows This shortens retention. Select Apply anyway to go ahead.
To keep data for an investigation or an audit, turn on legal hold for the table or dataset. Deletion stops while the hold is on; moves to cold storage carry on.
Use your own object storage
Your workspace comes with a managed bucket for cold data. To use your own, select Add store under Object stores, choose Amazon S3, an S3-compatible store, Google Cloud Storage (HMAC) or Azure Blob, enter the bucket or container, region, endpoint and credential, and select Register store. Then pick it as the Cold store for a table.
Changing lifecycle rules and stores needs the data lifecycle permissions, which admins have by default.