Observability

Vercel's built-in logs capture console.* output and unhandled exceptions; on top of that we emit structured JSON through a small logger wrapper.

What's available now

  • Structured logger at src/lib/log.tslog.info, log.warn, log.error, logError(event, err, context?). All wrap console.* and write {level, event, ts, ...context} JSON so Vercel log search can filter by event.
  • Log level via LOG_LEVELdebug | info | warn | error, default info. An unrecognised value falls back to info rather than muting the log, which is the failure direction that keeps errors visible. debug is loud enough to be expensive on the send paths, so it is a deliberate temporary setting rather than a default.
  • Sentry mirror via SENTRY_DSN — optional and inert by default. captureException / captureMessage in src/lib/observability.ts no-op unless both the DSN is set and @sentry/nextjs is installed; it is an optional dependency loaded through a runtime require so the build never demands it. When live, logError(...) mirrors to captureException and warn / error lines mirror to captureMessage (info is deliberately dropped to keep volume sane). Unset, errors still reach the JSON log lines — only the mirror is missing.
  • Per-request correlation via X-Request-Id — generated in withApiAuth (src/lib/api/request-id.ts) and echoed back on every /api/v1/* response.
  • /api/health — four checks: Postgres, Upstash, schema drift and crons. Used by external uptime monitors. database, redis and crons decide the verdict — 503 when any fails, because this is the endpoint that exists to wake someone. Schema drift does not: it reports as checks.schema.state: "drifted" plus a schema_drift warning and leaves status alone, the same split probe-health makes, because next build does not apply migrations and on this repo's shared dev/prod Neon instance a deploy ahead of the applied ledger is routine — a monitor that pages for that gets muted, and takes the real outages with it. Alert on warnings deliberately if you want to be woken by drift. logError("health.check_failed", ...) fires only on failure so polling doesn't spam the log, and detail stays in that log — the public payload carries a constant error: "check_failed" so a probe response cannot fingerprint versions or leak hostnames. The body is written for an anonymous reader: no error text, no env var names, no cron ages. warnings[] is a list of bare codes (billing_config, billing_config_unreadable, schema_drift); the sentence behind each is logged as health.warning once per process, since these are static misconfigurations and the route is polled at up to 60/min/IP. Rate-limited to 60/min/IP; each hit runs 3 latency probes plus one drift and one ledger read. Both ledger-backed checks stay quiet about what they cannot see: crons read unknown (green, labelled) when the ledger has no rows or does not exist yet, and drift reads unknown when the migration ledger is unreadable.
  • cron_runs — one row per authorised cron invocation, opened by withCronRun before the handler runs and closed after it answers. A job killed at maxDuration is a row with no finished_at; a job dying every tick is a column of ok = false with the error. This is what /api/health reads as the platform heartbeat: the newest ok probe-health run, stale past 15 minutes — three of its five-minute ticks. The heartbeat job must be one withCronRun wraps, which send-scheduled (the single unwrapped route) is not; src/lib/api/cron-heartbeat.wiring.test.ts fails the build if the constant ever names an unwrapped job. Writes are best-effort — a failure here never changes a cron's own response — and only runs that presented CRON_SECRET are recorded, so a local curl cannot write a row production reads as a tick. Retention prunes it at 30 days but always keeps each job's newest row, so a cron dead for longer than the window cannot quietly turn the check green again.
  • Vercel Function Logs — stdout / stderr.
  • Upstash Analytics — rate-limit call counts (enabled on @upstash/ratelimit).
  • Stripe Dashboard — subscription and webhook events.
  • AWS CloudWatch — SES + SNS call-level metrics.
  • Webhook delivery logwebhook_deliveries table rows; inspection UI on /overview/webhooks.
  • Audit logaudit_logs table; dashboard at /overview/settings/audit.

What the logger covers today

Surface Event prefix Notes
Cron: scheduled sends cron.scheduled.* usage_increment and fanout failures, plus cron.scheduled.run summary
Cron: webhook retry cron.retry_webhooks.run processed + due counts
Cron: idempotency cleanup cron.cleanup_idempotency.run delete count
Cron: overage report cron.overage.* run summary; per-org Stripe create failures
Webhook: SES webhook.ses.* signature failures, fan-out failures, processed summary
Webhook: SNS SMS webhook.sns_sms.* signature failures, fan-out failures, processed summary
Webhook: inbound email webhook.inbound_email.* signature failures, fan-out failures, processed summary
Template test send template.test_send* success + failure paths
Stripe webhook stripe.webhook.* handler-level failures

Webhook fan-out now runs inside after(() => ...) from next/server, so the handler returns a 200 before provider callbacks happen and errors surface via the logger only.

Still missing

  • No OTEL / APM.
  • Sentry is wired but not switched on — the shim exists and reads SENTRY_DSN, but this deployment neither sets the DSN nor installs @sentry/nextjs, so every capture is a no-op. Both steps are needed; either alone changes nothing.
  • No latency histograms beyond Vercel defaults.
  • No webhook_deliveries failure surface beyond the dashboard view.

Quick wins next

  1. Sentrynpm install @sentry/nextjs and set SENTRY_DSN. No code change: src/lib/observability.ts already routes logError and warn / error lines through it. Optionally add sentry.server.config.ts for advanced setup.
  2. Vercel OTEL — export traces to Honeycomb / Datadog / Jaeger.
  3. Weekly audit log email to owners summarizing notable actions.