Monitors and alerting
Monitor every signal, and alert where your team works
Watch services, logs, hosts, Kubernetes, network devices and real users with ready-made monitors or your own SQL. Route alerts to Slack, Microsoft Teams, PagerDuty, Opsgenie, webhooks or your email gateway, with one notification per problem.
- 222 ready-made monitors
- Threshold or baseline
- Monitors as code
How it works
From a signal to the right person, once
One alerting engine serves every product area, so a slow service, a failing port and a security match follow the same rules.
- 01PickStart from a template, switch on a ready-made monitor, or write a read-only SQL query.
- 02TuneSet alert and warning thresholds with recovery values, a window, and what missing data means.
- 03EvaluateEach monitor checks every service, host or device it covers, every minute by default.
- 04GroupRelated alerts join one problem, so a team gets one notification instead of many.
- 05RouteNotification policies match severity, level and tags, and send to the channels you choose.
- 06PauseDowntimes hold notifications during planned work while evaluation carries on.
Monitors
Start from a template, or write the query yourself
Templates cover services, logs, hosts, Kubernetes, network devices, real users and security matches, and a read-only SQL monitor covers anything else in your workspace. A monitor tracks each service, host or device it finds on its own.
- Fixed thresholds with their own recovery values, or a baseline that learns daily or weekly rhythms
- Choose what missing data means: keep the last state, count it as zero, mark it as no data, or resolve
- Backtest a fixed threshold against the last 7 days before you save it
Routing
One notification per problem, sent where your teams work
Notification policies decide where an alert goes by its severity, its level and its monitor's tags. Related alerts send one message per destination, and flapping alerts wait until they settle.
Policies that match
Severity, level and monitor tags, with an optional delay before a route sends.
One message per problem
Non-critical problems wait 3 minutes to settle before they notify.
Calm by design
An alert that flips back and forth holds its recovery until it is stable.
Downtimes
One-off, daily or weekly windows mute notifications while evaluation continues.
As code
Keep monitors in Git and review them like code
Describe monitors and dashboards as JSON bundles in your own repository. A command-line tool validates them against your workspace, plans the change, and applies only the plan you reviewed.
- Apply refuses with a conflict if anything changed since the plan, so nothing lands unreviewed
- Test fixtures run each threshold against sample values, recovery included
- Monitors managed from Git are locked in the UI, while enabling, snoozing and destinations stay editable
{
"apiVersion": "tracix.io/v1",
"package": "team-checkout",
"resources": [{
"kind": "monitor",
"uid": "checkout-p95-latency",
"attributes": {
"templateId": "apm.latency_p95",
"condition": "above",
"thresholds": { "alert": 500, "warn": 350 },
"noData": "last_known"
},
"tests": [{ "name": "slow", "value": 611, "expected": "alert" }]
}]
}Better together
Alerts are where the rest of operations starts
Every alert lands in the same workspace as the problems, SLOs and anomaly jobs built on it.
- Problems and casesRelated alerts fold into one problem with a suspected cause and the changes that came before it.Learn more
- SLOs and on-callEvery SLO gets a burn-rate monitor, and escalation policies page the on-call team.Learn more
- Anomaly detectionJobs that learn each series' normal range alert through the same notification policies.Learn more
Channels
Where alerts can go, and how they get there
Channels keep any secret in the credential store, never in their settings, and each one has a test send. Notification policies choose which alerts reach which channels.
| Channel | How Tracix delivers | Set up with |
|---|---|---|
| Slack | A message through an incoming webhook | Webhook URL |
| Microsoft Teams | A message through an incoming webhook | Webhook URL |
| PagerDuty | Events API v2 with a deduplication key, so a recovery resolves the incident | PagerDuty credential |
| Opsgenie | Creates an alert, and closes it by alias when the alert recovers | Opsgenie API key |
| Webhook | A JSON POST with a delivery ID, optionally signed with HMAC | URL, with header, basic, bearer or signing auth |
| An HTTP POST to your own email gateway, which sends the mail | Gateway URL |
There is no built-in mail server, SMS or voice channel. Route alerts to PagerDuty or Opsgenie when you need phone and mobile paging.
What happens when a source stops sending data?
Can Tracix send email without a gateway of our own?
Can we bring alerts from Prometheus?
Can we acknowledge an alert from Slack?
Switch on your first ready-made monitor
Pick a monitor from the library, add a Slack, Teams or PagerDuty channel, and route it with one notification policy.