Prometheus 모니터링 재전송: Grafana의 이메일 반송에 대한 경고

작성자

카테고리:

← 피드로
DEV Community · Robbe Verhelst · 2026-07-22 개발(SW)

If you are looking for Resend Prometheus monitoring, Resend Grafana alerts, or a way to alert on email bounces, that is exactly what resend-exporter is for.

It is a small Prometheus exporter and webhook receiver for Resend email events. It receives Resend webhooks, verifies their Svix signatures, exposes clean Prometheus metrics, and gives Grafana something useful to alert on when transactional email delivery breaks.

The reason I built it was simple.

An appointment confirmation email bounced.

The appointment itself existed. The application did its job. Resend knew the email had failed. But unless someone checked the right dashboard or the customer asked about it, that failure could quietly sit there as a support problem instead of an operational signal.

That felt wrong.

Transactional email is part of the product. Password resets, appointment confirmations, invoices, login links, onboarding flows, receipts: when they fail, the user does not care that your API returned 200 OK.

It turns this:

Resend knows an email bounced somewhere in a vendor dashboard.

Into this:

Prometheus sees resend_email_events_total{event_type="email.bounced"} increase and Grafana can alert me.

The problem with silent Resend email failure

Most web apps treat email as a side effect.

The request succeeds, the database is updated, and some email provider gets asked to deliver a message. At that point the application often considers the work done.

But delivery is where the user value actually happens.

If an appointment confirmation bounces, the appointment is technically created but operationally degraded. If a password reset fails, authentication is broken for that user. If an invoice email never arrives, someone eventually has to untangle the mess.

Email providers like Resend already emit useful delivery events:

  • email.sent
  • email.delivered
  • email.delivery_delayed
  • email.bounced
  • email.failed
  • email.complained

The missing piece, for me, was getting those events into the monitoring stack I already use.

I did not want another dashboard to remember.

I wanted Prometheus metrics, Grafana dashboards, and alert rules.

What the Resend Prometheus exporter does

resend-exporter is deliberately small:

Resend
  -> Svix-signed webhook
  -> resend-exporter
       - verify
       - count
       - log
  -> /metrics for Prometheus
  -> Grafana dashboards and alerts
  -> JSON logs for Loki or stdout

Enter fullscreen mode Exit fullscreen mode

It exposes four HTTP endpoints:

Method Path Purpose POST /webhooks/resend Receive Resend webhook events GET /metrics Prometheus metrics GET /healthz Liveness probe GET /readyz Readiness probe

The webhook handler verifies Resend’s Svix signature, validates the payload, updates Prometheus counters and gauges, and emits one structured JSON log line per accepted event.

No database. No queue. No Resend API key required for the basic webhook mode.

Just receive, verify, count, log.

Metrics are not logs

The most important design choice was separating metrics from details.

Prometheus should answer questions like:

  • how many emails bounced?
  • did failures increase in the last five minutes?
  • which event types are happening?
  • are failures concentrated by sending domain or broad recipient domain?
  • has the webhook gone stale?

It should not store every recipient address, subject line, or Resend email ID as labels.

That way lies cardinality pain.

So the exporter exposes metrics like:

resend_webhook_events_total{event_type="email.bounced",domain="acme.dev"}
resend_email_events_total{event_type="email.failed",from_domain="acme.dev",to_domain="outlook.com"}
resend_webhook_signature_failures_total
resend_webhook_handler_errors_total{reason="invalid_json"}
resend_webhook_last_event_timestamp_seconds{event_type="email.delivered"}

Enter fullscreen mode Exit fullscreen mode

Those labels are intentionally boring.

The more sensitive forensic detail goes into structured logs:

{
  "level": "warn",
  "event_type": "email.bounced",
  "resend_email_id": "3ebe19b6-1dcc-4534-8442-9dc689ee439b",
  "from_domain": "example.com",
  "to_domain": "outlook.com",
  "recipient_count": 1,
  "reason": "Recipient mail server not found",
  "event_created_at": "2026-07-21T00:24:00Z"
}

Enter fullscreen mode Exit fullscreen mode

Metrics tell you that something is wrong.

Logs tell you what broke.

Keeping those jobs separate makes the exporter much safer to operate.

Keeping Prometheus labels under control

Email data is cardinality bait.

It is very tempting to add labels like:

to="[email protected]"
subject="Your appointment confirmation"
email_id="..."

Enter fullscreen mode Exit fullscreen mode

That would make the first dashboard look convenient and the second month of Prometheus look stupid.

resend-exporter avoids high-cardinality labels by design:

  • full recipient emails are never metric labels
  • subjects are never metric labels
  • Resend email IDs are never metric labels
  • recipient domains are bucketed
  • the long tail can collapse into other
  • extra recipient domains can be allowlisted when needed

For private deployments, the logs can include more detail. For shared or stricter setups, redaction can stay strict or hash sensitive fields.

The current redaction modes are:

Mode Behavior strict Do not log recipient email or subject hash Log stable SHA-256 hashes for correlation none Log raw values for private deployments

The default is strict.

Because “just put it in a label” is how dashboards become landmines.

Resend alerting examples for Prometheus and Grafana

Once the events are in Prometheus, the alert rules become ordinary PromQL.

Any failed email:

increase(resend_email_events_total{event_type="email.failed"}[5m]) > 0

Enter fullscreen mode Exit fullscreen mode

Any bounced transactional email:

increase(resend_email_events_total{event_type="email.bounced"}[5m]) > 0

Enter fullscreen mode Exit fullscreen mode

Repeated delivery delays:

increase(resend_email_events_total{event_type="email.delivery_delayed"}[30m]) >= 3

Enter fullscreen mode Exit fullscreen mode

Signature failures:

increase(resend_webhook_signature_failures_total[5m]) > 0

Enter fullscreen mode Exit fullscreen mode

That last one is useful because webhook endpoints are public by necessity. Bad signatures should be rejected, counted, and visible.

Running resend-exporter

With Docker:

docker run -p 8080:8080 \
  -e RESEND_WEBHOOK_SECRET=whsec_... \
  ghcr.io/robbeverhelst/resend-exporter:latest

Enter fullscreen mode Exit fullscreen mode

With Helm:

helm install resend-exporter oci://ghcr.io/robbeverhelst/charts/resend-exporter \
  --set resend.webhookSecret=whsec_... \
  --set serviceMonitor.enabled=true

Enter fullscreen mode Exit fullscreen mode

For local testing, the repo includes a compose playground with the exporter, Prometheus alert rules, and a provisioned Grafana dashboard:

docker compose up --build

Enter fullscreen mode Exit fullscreen mode

Then configure a Resend webhook to point at:

https://your-host/webhooks/resend

Enter fullscreen mode Exit fullscreen mode

The webhook path must be internet-reachable.

The metrics endpoint should not be.

What ships today

The repository currently includes:

  • Bun/TypeScript implementation
  • Svix webhook signature verification
  • Prometheus metrics via prom-client
  • structured JSON logging
  • strict/hash/none redaction modes
  • recipient-domain bucketing
  • Docker image
  • Helm chart
  • docker-compose playground
  • Grafana dashboard
  • Prometheus alert examples
  • CI and automated releases

Repo: github.com/robbeverhelst/resend-exporter

The current release is v0.2.1.

What I still want to add

The next useful feature is optional Resend API reconciliation.

Webhook-only mode is intentionally simple, but an API key could enable:

  • backfilling recent events after exporter downtime
  • checking whether required Resend webhook events are configured
  • enriching sparse webhook events
  • exposing configuration health metrics

I also want to add delivery-delay histograms, bounce-type breakdowns, and first-class PrometheusRule support in the Helm chart.

The larger point

This project is small, but the pattern matters.

If a user-facing side effect can fail after your application says “success”, it deserves observability.

Transactional email is not background noise. It is production behavior.

And if production behavior breaks, I want the same boring machinery that watches everything else to notice.

That is all resend-exporter really does.

It makes email delivery failures page-shaped.

원문에서 계속 ↗

코멘트

답글 남기기

이메일 주소는 공개되지 않습니다. 필수 필드는 *로 표시됩니다