Monitoring
Voxeltron collects host and deployment metrics into its local store for alerts. The TUI and AI preview can also read logs, metrics and traces from your observability services.
Configure the sources
Add a [monitoring] section to /etc/voxeltron/voxeltron.toml:
[monitoring]
prometheus_url = "http://127.0.0.1:9090"
loki_url = "http://127.0.0.1:3100"
jaeger_url = "http://127.0.0.1:16686"
poll_interval_secs = 15prometheus_url is optional when configuring alert channels alone.loki_url and jaeger_url are optional. Without them, log and trace queries use the daemon's own stores or return no data.
Validate and restart:
$ sudo voxeltrond config validate
$ sudo systemctl restart voxeltrondSee logs and metrics
Select monitoring in the TUI left rail. The section shows the current log view and lets you edit the query with / and run it with Enter. Thequery_logs, query_metrics and query_traces AI tools use the same sources.
For a single app, vxl app status reports the current endpoint and health metadata:
$ vxl app status --name api --server prodAlerts
Community and Solo can read alerts in the TUI dashboard and monitoring view, and through the agent's list_alerts tool in the AI preview. Alert records are stored by the daemon. Creating and listing custom alert rules require Team or Enterprise. Delivery channels are named under [monitoring.channels]:
[monitoring.channels.ops]
type = "webhook"
url = "https://hooks.example.com/voxeltron"
[monitoring.channels.oncall]
type = "slack"
url = "https://hooks.slack.com/services/..."A new daemon seeds four enabled rules into its persistent rule store. Each rule sends to every channel named in [monitoring.channels]; adding a channel is the only setup required. Rules are evaluated every 10 seconds, even with AI disabled, and a firing alert is delivered once until it resolves. The built-in rules watch the host filesystem, host memory, deployment health and certificates for domains routed through Caddy.
Alert payload
Webhooks include alert_id, rule_id, message,state (Firing or Resolved), and an RFC 3339 timestamp. Context fields are project (the app name), project_id,deployment_id, component, and container_id. Unknown values and project fields for host alerts are empty strings. Container IDs identify actual runtime containers, never deployments. When known, exit_code is an integer and reason is a separate diagnostic string. Unknown exit codes and reasons are omitted. Lifecycle alerts also include restart_count when available. Human messages are short sentences that name the affected project, for exampleshop: container exited (exit code 1). See vxl logs shop.An unknown exit code is written as exit code unknown in the sentence. Slack headlines and email subjects use the same sentence. Structured details stay in their own fields rather than appearing as JSON in the message. A deployment that never becomes healthy sends deploy_failure; a running container that dies sends crash_detected, once per incident until sustained recovery. Repeated failures can also send crash_quarantine.
Default rules
- Disk usage: host disk above 85% for 5 minutes.
- Memory usage: host memory above 90% for 5 minutes.
- Health check: a deployment's health check failing for 3 minutes.
- Certificate expiry: a routed Caddy domain has under 14 days left.
The certificate check makes a daily TLS handshake to each domain in the Caddy route store. It does not contact any other host. The remaining lifetime is recorded as thecert_expiry_days metric.
voxeltrond config render shows the following switches:
[alerts]
disk_usage = true
memory_usage = true
health_check = true
certificate_expiry = trueDisable an individual default by setting its switch to false and restarting the daemon. A deleted default rule is remembered by the rule store and is not added again on restart.
Health checks
$ vxl doctor local
$ vxl doctor remote --server prod
$ vxl doctor container --server prod --deployment-id <deployment-id>doctor remote checks the daemon, SSH, Docker, proxy, DNS and Cloudflare reachability. doctor container inspects Docker isolation and hardening.