# OTLP endpoint and ingest keys

Send OpenTelemetry traces, metrics and logs to your workspace's OTLP endpoint, and create, pause and use the ingest keys that authenticate them.

Applications instrumented with OpenTelemetry send their data to your workspace's APM gateway over OTLP. Each request carries an ingest key, which authenticates it and sets the default environment for its data.

In the commands on this page, replace `<workspace>` with your workspace ID: the eight characters that end your workspace's address, before `.app.tracix.io`.

## Endpoints

Your workspace has two OTLP endpoints. Both take traces, metrics and logs, and both listen on port 443 with TLS, so the URLs have no port.

In these addresses, `<workspace>` is your workspace ID: the 8-character code at the end of your workspace's address, such as `a7k3p2q9` in `production-environment-a7k3p2q9.app.tracix.io`.

| Protocol                    | Endpoint                                      | Use it for                                        |
| --------------------------- | --------------------------------------------- | ------------------------------------------------- |
| OTLP/HTTP (`http/protobuf`) | `https://otlp-http-<workspace>.app.tracix.io` | Most SDKs, and the commands on the **Setup** page |
| OTLP/gRPC                   | `https://otlp-<workspace>.app.tracix.io`      | SDKs and collectors set to gRPC                   |

You can copy both from your workspace: open **Applications › APM & Traces**, then the **Setup** tab, and find **Gateway endpoints**.

With OTLP/HTTP, set the endpoint as the base URL: SDKs add `/v1/traces`, `/v1/metrics` and `/v1/logs` themselves.

> **Match the protocol to the endpoint**
>
> The two endpoints are different services. Sending gRPC to the HTTP endpoint fails with HTTP 415. Some SDKs default to gRPC, including the Python distro and .NET's OTLP exporter, so always set `OTEL_EXPORTER_OTLP_PROTOCOL` to match the endpoint you use.

## Create an ingest key

Creating, listing or changing ingest keys needs permission to manage APM, which the workspace Admin role has. People without it see a note on the **Setup** page instead of the keys, and replace `YOUR_INGEST_KEY` in the commands with a key someone gives them.

**1. Open the ingest keys**

In your workspace, open **Applications › APM & Traces**, then the **Setup** tab. **Ingest keys** is at the bottom of the page.

**2. Create the key**

Select **New key**. Enter a **Name** that says which services use it, such as `Production backend`, the **Environment** it is for (default `production`), and, optionally, a **Service group**. Select **Create key**.

**3. Save it now**

The new key appears once, above the list, with **Copy key** beside it. It starts with `apm_`. Save it where your services read their secrets: Tracix shows the key only this once. Until you leave or reload the page, the commands on the **Setup** page use it.

The key's environment and service group are defaults. The gateway adds them to data that doesn't name its own, as `deployment.environment.name` and `tracix.service_group.name`; whatever your service sets wins. Use one key per environment, so data without its own environment still lands in the right one.

After that, the list shows each key masked, and the **Setup** page's commands show `YOUR_INGEST_KEY` for you to replace with the key you saved. A key you didn't save can't be shown again: create a new one and pause the old one.

### Pause a key

**Pause key** stops the gateway accepting a key, and **Resume key** turns it back on. Either change takes effect within about 30 seconds.

> **Keys can't be deleted or set to expire yet**
>
> Ingest keys have no delete, expiry or last-used date yet. Today you take a key out of service by pausing it. Keep keys out of source control, and pass them to services as secrets or environment variables.

## Send data

Send the key in the `x-tracix-ingest-key` header. The gateway doesn't accept it as `Authorization: Bearer`, and [platform API keys](https://tracix.io/docs/admin/api-keys) don't work here.

### From an OpenTelemetry SDK

Every OpenTelemetry SDK reads the standard environment variables:

```bash
export OTEL_SERVICE_NAME="checkout-api"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment.name=production"
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp-http-<workspace>.app.tracix.io"
export OTEL_EXPORTER_OTLP_HEADERS="x-tracix-ingest-key=<ingest key>"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
```

For the full setup of each language, including the agent to install and how to run it, see [Languages and SDKs](https://tracix.io/docs/collect/instrument/languages). The **Setup** page writes the same commands with your endpoint and key filled in.

### From an OpenTelemetry Collector you run

Point an `otlphttp` exporter at the HTTP endpoint and add the header:

```yaml
exporters:
otlphttp:
  endpoint: https://otlp-http-<workspace>.app.tracix.io
  headers:
    x-tracix-ingest-key: ${env:TRACIX_INGEST_KEY}

service:
pipelines:
  traces:
    exporters: [otlphttp]
  metrics:
    exporters: [otlphttp]
  logs:
    exporters: [otlphttp]
```

Set `TRACIX_INGEST_KEY` in the collector's environment rather than writing the key into the file.

### Send a test record

This sends one log record over OTLP/HTTP with JSON, the same way the console's own examples do:

```bash
curl -sS -o /dev/null -w "%{http_code}\n" \
  -X POST "https://otlp-http-<workspace>.app.tracix.io/v1/logs" \
  -H "content-type: application/json" \
  -H "x-tracix-ingest-key: <ingest key>" \
  -d '{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"curl-test"}}]},"scopeLogs":[{"logRecords":[{"timeUnixNano":"'"$(date +%s)"'000000000","body":{"stringValue":"hello from curl"}}]}]}]}'
```

Expected output:

```text
200
```

A `401` means the key is missing, unknown or paused. A `415` means the request used the wrong protocol for the endpoint.

## Check that data arrives

The **Setup** page has a **Verify** card that looks at the last 15 minutes every 15 seconds. For the service and key you entered, it shows whether the key is active, then **Traces from** your service once the first span arrives, and whether runtime metrics and logs are coming in.

Services appear under **Applications › APM & Traces** on the **Services** tab, and logs in **Insights › Log Explorer**. See [APM and tracing](https://tracix.io/docs/observe/apm).

## What the gateway does with your data

* **Authenticates** every request with the ingest key. A request without an active key is refused.
* **Fills in defaults**: the key's environment and service group, where your data has none.
* **Removes credentials from URLs** in span names and URL attributes, such as `user:password@`, webhook paths and token query parameters, before anything is stored.
* **Computes service metrics and the service map** from every span, then applies your [sampling policy](https://tracix.io/docs/collect/instrument/sampling) to decide which traces it stores.

## Send through a Tracix Collector instead

A Tracix Collector can also receive OTLP: add the [OTLP integration](https://tracix.io/integrations/otlp) to its policy, and it listens on TCP 4317 (gRPC) and 4318 (HTTP) on that host. Applications then send to the collector without an ingest key, and the collector writes their data to your workspace with its policy's output. Keep in mind:

* the listener doesn't authenticate senders, so expose it only to the applications that should use it;
* data that arrives this way skips the APM gateway, so the gateway's sampling policy doesn't apply to it.

## Source IP allowlist

If your workspace restricts which networks can reach it, add the addresses your applications send from. See [Source IP allowlist](https://tracix.io/docs/collect/collector/ports-and-proxies#source-ip-allowlist).

## Next steps

- [Languages and SDKs](https://tracix.io/docs/collect/instrument/languages): Instrument a service in your language

- [Sampling](https://tracix.io/docs/collect/instrument/sampling): Decide which traces Tracix stores

- [Browser RUM](https://tracix.io/docs/collect/instrument/browser-rum): Send data from browsers with an app key