# Troubleshoot the Tracix Collector

Fix a Tracix Collector that won't install, enroll, start, apply its policy, write data or upgrade, with the cause, the fix and how to check it worked.

Start with the service status and the collector's log: most problems name their cause there, and the installer prints the likely cause when enrollment fails. Each problem below lists its cause, the fix and how to check that it worked.

In the commands on this page, replace `<workspace>` with your workspace ID: the eight characters that end your workspace's address, before `.app.tracix.io`.

## Check the basics

| Platform   | Service status                                     | Log                                                                 |
| ---------- | -------------------------------------------------- | ------------------------------------------------------------------- |
| Linux      | `systemctl status tracix-collector`                | `sudo journalctl -u tracix-collector -f`                            |
| macOS      | `sudo launchctl print system/com.tracix.collector` | `tail -f /usr/local/var/log/tracix-collector/stderr.log`            |
| Windows    | `Get-Service -Name TracixCollector`                | `Get-WinEvent -ProviderName TracixCollector -MaxEvents 50`          |
| Kubernetes | `kubectl get pods -n tracix`                       | `kubectl logs -n tracix -l app.kubernetes.io/name=tracix-collector` |
| Docker     | `docker ps --filter name=tracix-agent`             | `docker logs -f tracix-agent`                                       |

On hosts, the collector's health check answers on `http://127.0.0.1:13133`. See [Local health check](https://tracix.io/docs/collect/collector/configuration#local-health-check).

## Installing and enrolling

### The installer can't reach the release host or the control plane

**Cause.** Before it changes anything, the installer connects to the release host and to your workspace's control plane over HTTPS. It stops when DNS, a firewall, TLS or a proxy gets in the way, and names which one:

| The installer says                                                    | What to do                                                                                                                |
| --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `DNS: this host cannot resolve …`                                     | Check the host's DNS servers.                                                                                             |
| `TCP: cannot connect to … on port 443` or `timeout: no answer from …` | Allow outbound TCP 443 to the host it names, or pass `--proxy` if this network reaches the internet only through a proxy. |
| `TLS: the TLS handshake with … failed`                                | A TLS-inspecting proxy or firewall is in the way. Exempt the Tracix hosts from inspection.                                |
| `proxy: the proxy requires authentication (HTTP 407)`                 | Put the user and password in the `--proxy` URL.                                                                           |

**Fix.** Open the path, or pass `--proxy URL` (`-Proxy URL` on Windows), and run the installer again. See [Ports, proxies and firewalls](https://tracix.io/docs/collect/collector/ports-and-proxies) for every host the collector needs.

**Verify.** Test the path from the host. Ping is expected to fail, because Tracix's load balancers don't answer it.

#### Linux

```bash
curl -sv https://agents-<workspace>.app.tracix.io
```

#### macOS

```bash
curl -sv https://agents-<workspace>.app.tracix.io
```

#### Windows

```powershell
Test-NetConnection agents-<workspace>.app.tracix.io -Port 443
```

### The installer stops because the host clock is off

**Cause.** The collector stamps everything it sends with the host's clock. A clock more than two minutes away from the control plane's would put the host's data outside current views, so the installer turns on time sync and stops if the clock is still off.

**Fix.** Sync the clock, for example with `sudo timedatectl set-ntp true` or chrony, with NTP reachable, then run the installer again.

**Verify.** The installer reports the clock as in sync and carries on.

### Install not confirmed: the control plane rejected the enrollment token

**Cause.** The installer waits up to 90 seconds for the collector to enroll. An `HTTP 401` means the token was revoked, has expired, or belongs to another workspace.

**Fix.** In your workspace, open **Collectors › Policies**, select the policy and open **Settings**. Under **Enrollment tokens**, use **Show** on an active token, or **Create token**, and run the installer again with it. The service stays installed and keeps retrying, so running the installer again is safe.

**Verify.** The installer prints `Installation complete!` with the collector's ID, and the collector appears under **Collectors**.

### An enrollment token is required

**Cause.** The installer had no token: `--token-file` was missing, and there was no terminal to prompt on, as in an automation tool. On a running collector, the log line `auth_token is required` means the same thing.

**Fix.** Save the token in a file only root can read and pass it with `--token-file` (`-TokenFile` on Windows).

**Verify.** The installer gets past the token step and waits for enrollment.

### macOS: "operation not permitted"

**Cause.** Some terminals, such as those built into IDEs, are sandboxed, and `sudo launchctl` fails inside them.

**Fix.** Run the installer from the Terminal app.

**Verify.** `sudo launchctl print system/com.tracix.collector` shows `state = running`.

## Running

### The service doesn't start

**Cause.** The log names it:

* `endpoint is required`: the environment file is missing or empty.
* `auth_token is required`: the collector has no enrollment token and hasn't enrolled before.
* `config validation failed`: the configuration file is damaged.

**Fix.** Run the installer again with `--endpoint` and the token. It rewrites the environment file and a fresh bootstrap configuration, keeps the old configuration beside it as `config.yaml.bak.<timestamp>`, and enrolls again.

**Verify.** The service status shows it running, and the health check returns `200`.

### The collector shows a clock warning

**Cause.** The host clock is ahead of or behind the control plane. Everything the collector sends lands in the past or the future, so its data falls outside current views and its devices can read as stale.

**Fix.** Sync the host's time with NTP, or with Windows Time on Windows.

**Verify.** The warning next to the collector's name on **Collectors** goes away after its next heartbeat.

### Config rejected

**Cause.** The collector couldn't load a revision of its policy. It keeps running the last configuration it applied and picks up the next revision it can load.

**Fix.** Open the collector or its policy. The **Config rejected** notice shows the revision and the reason, usually one integration's setting. Fix that integration on the policy and save, which creates a new revision.

**Verify.** The notice clears once the collectors apply the new revision.

### Integration failed

**Cause.** An integration loaded but failed to start, for example because another process holds the port it listens on, or a service it reads refused the connection. The collector switched that integration off and keeps running the rest of the policy.

**Fix.** The error shows under the integration in the collector's details and on the policy, where the integration is marked **Failed to start**. Free the port or change the integration's settings, then save the policy.

**Verify.** The marker clears when the collector applies the new revision.

### Ingest failing

**Cause.** The collector can't write some of its data to your workspace. It retries batches that may still succeed; batches no retry can fix are dropped, so that data is missing until you fix the cause.

**Fix.** Open the collector: the notice lists each table that failed, the reason in plain words and, under **Error details**, the database's own message. If the reason is **can't reach ClickHouse**, check that the host still reaches `ch-<workspace>.app.tracix.io` on TCP 443, through the proxy if it uses one, and that your workspace's source IP allowlist, if it has one, includes the host's address.

**Verify.** The notice clears once new writes succeed.

### The collector is offline

**Cause.** The collector's heartbeats stopped: the service stopped, the host is down, or the network path to the control plane closed.

**Fix.** Check the service and its log on the host, then the path to `agents-<workspace>.app.tracix.io` on TCP 443. When the WebSocket can't connect, the collector falls back to HTTPS polling, so a proxy that blocks WebSockets doesn't stop it from reporting.

**Verify.** The collector's status returns to running on **Collectors**.

### SNMP traps don't arrive on a Linux collector

**Cause.** The trap listener uses UDP 162, a port below 1024. The collector binds it with the `CAP_NET_BIND_SERVICE` capability, which hosts installed before the installer granted it don't have. Upgrades replace only the binary, not the service unit, so they don't add it.

**Fix.** Run the current installer once on the host.

**Verify.** Open the collector on **Collectors**: **Listening on** lists `UDP` port 162.

## Upgrading

### An upgrade failed

**Cause.** The download, the checksum, the new binary's check of its configuration, or the restart failed. The **Release** column shows **Retry**.

**Fix.** Select **Retry** to make a new attempt. If the policy updates its collectors automatically, Tracix also retries after 15 minutes, 1 hour and 4 hours. If a new Linux or macOS binary can't start at all, [restore the previous binary](https://tracix.io/docs/collect/collector/upgrade#restore-the-previous-binary).

**Verify.** The **Release** column shows **current**.

### A collector shows "manual"

**Cause.** The installation can't upgrade itself: an older install, or a Docker container started without the Docker socket.

**Fix.** Install it again once with the console's current command. See [Upgrade by hand](https://tracix.io/docs/collect/collector/upgrade#upgrade-by-hand).

**Verify.** The **Release** column shows **current** or offers **Upgrade**.

### Zero-code tracing doesn't start: OBI runtime setup required

**Cause.** The policy turns on zero-code eBPF tracing, but the collector wasn't installed with the eBPF permissions it needs, so Tracix withholds that integration and runs the rest.

**Fix.** On the policy page, the notice lists the collectors and either the Helm upgrade to run or **View OBI install steps**, which opens an install command with the eBPF permissions (`--enable-obi` on Linux). See [Zero-code eBPF tracing](https://tracix.io/docs/collect/instrument/ebpf).

**Verify.** The notice clears once the collectors report the eBPF capability.

## Kubernetes and Docker

### Collector pods crash and restart

**Cause.** The collector exits on start, usually because of a missing or wrong credential.

**Fix.** Read the previous container's log, and check that the secret named in `existingSecret` has the keys `control-plane-url`, `agent-token`, `otlp-endpoint` and `otlp-token`.

```bash
kubectl logs -n tracix -l app.kubernetes.io/name=tracix-collector --previous
```

**Verify.** `kubectl get pods -n tracix` shows every collector pod running and ready.

### A Docker collector comes back as a new collector

**Cause.** The container was recreated without the `tracix-agent-state` volume, which holds the collector's identity.

**Fix.** Use the console's Docker command, which creates the volume and mounts it at `/var/lib/tracix-collector`. Unenroll the duplicate collectors from **Collectors**.

**Verify.** Recreate the container with the same command; the same collector stays on **Collectors**.

## Still stuck?

Collect the service status, the last 100 lines of the log and the collector's ID from **Collectors**, and contact support. Don't include enrollment tokens or the environment file's contents.

- [Install the collector](https://tracix.io/docs/collect/collector/install): Install steps for every platform

- [Ports, proxies and firewalls](https://tracix.io/docs/collect/collector/ports-and-proxies): Every host and port the collector uses

- [Configuration reference](https://tracix.io/docs/collect/collector/configuration): Files, environment and health check