Cost Observability

Per-transaction connector cost

Cost Observability is the surface that answers “what did it actually cost us to move this money?” — interchange, scheme, processor, and acquirer fees broken down per connector and currency, plus interchange downgrades, cost anomalies, and a forward-looking fee forecast.

It is a thin read-only proxy over Hyperswitch’s analytics OLAP surface (Kafka + ClickHouse + a scheduler), in keeping with ADR-007 · Maximum native Hyperswitch bias. MVS owns no cost data of its own here: every metric is computed in ClickHouse by Hyperswitch and surfaced through /analytics/v1/cost/*. The route layer adds exactly one thing on top — graceful degradation.

The single most important fact about this surface: it does almost nothing useful unless the OLAP profile is provisioned. Hyperswitch’s analytics pipeline (Kafka + ClickHouse + scheduler) is gated behind olap.enabled in the Helm values and is off by default in any freshly built environment. When it is off, every endpoint here returns a deep-link to the Control Center, and the operator dashboard renders an empty state. See OLAP dependency & degraded mode and the Hyperswitch OLAP runbook.

Where this lives

LayerPath
Route handlerapps/mvs-pay-extensions/src/routes/cost-observability.ts
Testsapps/mvs-pay-extensions/src/routes/cost-observability.test.ts
Hyperswitch client (analytics namespace)apps/mvs-pay-extensions/src/services/hyperswitch-client.ts
OpenAPI fragmentfern/openapi/fragments/cost-observability.json
Operator dashboardapps/mvs-pay/app/(dashboard)/cost-observability/
Bring-up runbookdocs/runbooks/hyperswitch-olap.md (rendered: Hyperswitch OLAP)

The routes are mounted under the authenticated API surface in apps/mvs-pay-extensions/src/app.ts:

1// apps/mvs-pay-extensions/src/app.ts
2api.route("/cost-observability", costObservabilityRoutes);
3app.route("/v1", api);

so the public base path is /v1/cost-observability. Like every route on the api sub-app it sits behind authMiddleware() (internal secret + tenant resolution) and auditLog(). It is not rate-limited and not wrapped in idempotency — both are correct for read-only GETs. The spec reference for the surface is docs/superpowers/specs/2026-05-28-mvs-pay-perfection/14-phase-4-power-features.md § M22.2.

This is an internal, service-to-service surface. Every request requires the x-internal-secret header plus a tenant (x-org-id or x-org-slug); see Authentication & Tenancy and ADR-010 · Internal secret / deferred auth. The browser never calls these endpoints directly — the operator dashboard’s Next.js route handlers proxy through the SDK, which injects the headers. See the Merchant & Operator UI surface page.

The four endpoints

Four GET endpoints, each a best-effort proxy to one Hyperswitch analytics OLAP query. See the API Reference (tag: Cost Observability) for the full request/response schemas.

EndpointUpstream OLAP queryRequired query paramsWhat it answers
GET /v1/cost-observability/fee-breakdown/analytics/v1/cost/fee_breakdownfrom, toInterchange / scheme / processor / acquirer fees, per connector + currency
GET /v1/cost-observability/downgrades/analytics/v1/cost/downgradesfrom, toPayments that landed in a higher-fee tier than optimal interchange
GET /v1/cost-observability/anomalies/analytics/v1/cost/anomaliesfrom, toSudden cost spikes per connector / scheme / region
GET /v1/cost-observability/forecast/analytics/v1/cost/forecasthorizon (30d | 90d)Expected fee outlay over the next 30 / 90 days

Query parameters

The first three endpoints share a date-range contract, assembled by rangeQuery() in the route handler:

ParamRequiredNotes
fromyesRange start (ISO-8601). Missing from or to400 MISSING_RANGE.
toyesRange end (ISO-8601).
profileIdnoNarrow to one business profile. Forwarded as profile_id upstream.
connectornoNarrow to one connector.
currencynoNarrow to one currency.

forecast does not take a range — it takes horizon, which is validated against the literal set {"30d", "90d"}; anything else is 400 INVALID_HORIZON.

The optional narrowing params are accepted by every endpoint at the API edge (the OpenAPI fragment declares profileId / connector / currency on fee-breakdown, downgrades, and anomalies) but the client does not forward all of them to every upstream query. Reading hyperswitch-client.ts analytics:

  • feeBreakdown forwards profile_id, connector, currency.
  • downgrades forwards only profile_id (drops connector / currency).
  • anomalies forwards only from / to (drops all three narrowing params).
  • forecast forwards only horizon.

So passing connector=stripe to /downgrades or /anomalies is silently ignored, not an error. If you need server-side filtering there, it has to be added to the client method, not just the route.

Response shape: a discriminated union

Every endpoint returns one of two shapes. This is the crux of the surface:

1// Happy path — OLAP data
2{ "data": [ /* rows for this metric */ ] }
3
4// Degraded path — OLAP unavailable
5{ "link_to_dashboard": "https://.../analytics/cost-observability/fee-breakdown" }

The caller (SDK / operator dashboard) must branch on which key is present. The OpenAPI fragment models this explicitly as a oneOf of CostObservability<Metric>Data and CostObservabilityDashboardLink.

The OLAP data rows are typed in the OpenAPI fragment (the client itself only types them as { data: unknown[] }):

  • fee-breakdownconnector, currency, interchange_amount, scheme_amount, processor_amount, acquirer_amount, total_fees, transaction_count, volume.
  • downgradespayment_id, connector, expected_tier, actual_tier, fee_delta, currency, captured_at.
  • anomaliesconnector, scheme, region, metric (fee_rate | volume | decline_rate), delta_pct, detected_at.
  • forecastdate, expected_fees, currency, confidence_low, confidence_high.

OLAP dependency & degraded mode

This surface is only as good as the analytics pipeline behind it. That pipeline is the OLAP profile documented in the Hyperswitch OLAP runbook: three moving parts that must all be healthy before a single cost row exists.

If any link in that chain is missing — most commonly because olap.enabled is false and the StatefulSets were never created — the analytics service returns 404/501 (or 503), and the route degrades.

How degradation works

Two small helpers in cost-observability.ts do all the work:

1// 404/501 from Hyperswitch's analytics surface = OLAP disabled or metric
2// not exposed. We swallow those and degrade to a deep-link.
3function isFallbackEligible(err: HyperswitchApiError): boolean {
4 return err.statusCode === 404 || err.statusCode === 501;
5}
6
7function dashboardLink(path: string): { link_to_dashboard: string } {
8 return { link_to_dashboard: `${DASHBOARD_BASE_URL}/${path}` };
9}

The per-endpoint control flow is identical:

1

Validate input

fee-breakdown / downgrades / anomalies require from + to (400 MISSING_RANGE otherwise). forecast requires horizon ∈ {30d, 90d} (400 INVALID_HORIZON otherwise). Validation runs before any upstream call.

2

Best-effort call upstream

Call the matching hyperswitch.analytics.* method. On success, return its { data } payload verbatim.

4

Propagate everything else

Any other status (e.g. 403, 500, 503) is passed through mapErr(), which re-emits the upstream responseBody at the upstream status code. Real errors are not hidden behind a deep-link.

The fallback is only triggered by 404 and 501. A 503 from the analytics service — which the OLAP runbook calls out as “ClickHouse isn’t reachable from the analytics service” — is not fallback-eligible and will surface to the caller as a 503. That is intentional (a 503 is a transient fault worth retrying / paging on, not a “feature is off” signal), but it means “OLAP is degraded” and “OLAP is disabled” produce different caller-visible behaviour. Know which one you’re looking at.

The fallback URL is built from DASHBOARD_BASE_URL:

1const DASHBOARD_BASE_URL =
2 process.env.HYPERSWITCH_DASHBOARD_URL ??
3 "https://hyperswitch.payments.internal/analytics/cost-observability";

So the link is ${HYPERSWITCH_DASHBOARD_URL || default}/<metric>, where <metric> is fee-breakdown, downgrades, anomalies, or forecast. The path segment deliberately mirrors the endpoint name so an operator lands on the right Control Center view. The same data is rendered there via the embedded Control Center analytics page — which itself only has data if OLAP is on, so a deep-link from a disabled environment leads to an empty Control Center chart, not a populated one. The link is an escape hatch, not a fix.

What the operator sees

When the route returns { link_to_dashboard }, the operator dashboard at apps/mvs-pay/app/(dashboard)/cost-observability/ renders an empty state rather than charts. (Note this page ships on disk but is not on the navigation rail returned by getMvsPayNavSections() — it is reached contextually; see the Merchant & Operator UI page.)

Forecast is the honest stub

Be explicit about this one, because it is the most likely to mislead:

Hyperswitch v1.123.1 does not surface its forecast model via the OLAP API. The OpenAPI fragment says so plainly: the forecast endpoint “typically degrades to a link_to_dashboard deep-link.” In practice /v1/cost-observability/forecast returns a deep-link in essentially every environment today, OLAP-on or OLAP-off. fee-breakdown / downgrades / anomalies will return real { data } once OLAP is provisioned and the upstream metric exists; forecast is a placeholder that returns a link until Hyperswitch ships the metric (or MVS computes it from ClickHouse itself, which is not done).

The endpoint still validates horizon strictly so the contract is honest about what it accepts even while the data isn’t there.

Worked examples

Happy path (OLAP provisioned, data exists):

$curl -fsS \
> -H "x-internal-secret: $MVS_INTERNAL_SECRET" \
> -H "x-org-slug: acme" \
> "https://extensions.mvspay.internal/v1/cost-observability/fee-breakdown?from=2026-04-01&to=2026-05-01&connector=stripe&currency=USD"
$# { "data": [ { "connector": "stripe", "currency": "USD",
># "interchange_amount": 812.40, "scheme_amount": 91.10,
># "processor_amount": 220.00, "acquirer_amount": 64.00,
># "total_fees": 1187.50, "transaction_count": 4123,
># "volume": 318204.55 } ] }

Degraded path (OLAP not provisioned → 404 upstream → deep-link):

$curl -fsS \
> -H "x-internal-secret: $MVS_INTERNAL_SECRET" \
> -H "x-org-slug: acme" \
> "https://extensions.mvspay.internal/v1/cost-observability/fee-breakdown?from=2026-04-01&to=2026-05-01"
$# { "link_to_dashboard":
># "https://hyperswitch.payments.internal/analytics/cost-observability/fee-breakdown" }

Validation error (missing range):

$curl -s \
> -H "x-internal-secret: $MVS_INTERNAL_SECRET" \
> -H "x-org-slug: acme" \
> "https://extensions.mvspay.internal/v1/cost-observability/fee-breakdown?from=2026-04-01"
$# 400
$# { "error": { "code": "MISSING_RANGE", "message": "from and to are required" } }

These three shapes — data, link_to_dashboard, and the error envelope — are exactly what the tests in cost-observability.test.ts assert.

Turning the data on

If the dashboard is showing empty states and you need real numbers, the fix is not in this route — it’s provisioning OLAP. The full procedure is the Hyperswitch OLAP runbook; the short version:

1

Confirm the toggle

OLAP is gated by a single key, olap.enabled, in deploy/hyperswitch/values-{prod,staging}.yaml. The values blocks already exist in the repo; in an environment where it was off you usually don’t edit anything, you just verify and sync.

2

Sync via Argo CD

argocd app sync hyperswitch-prod (optionally targeting the Kafka / ClickHouse StatefulSets and scheduler Deployment to limit blast radius) reconciles the new resources.

3

Bring up Kafka → ClickHouse → scheduler in order

Kafka brokers first (ClickHouse and the scheduler depend on them), then ClickHouse (its Kafka-engine source tables need brokers reachable for DDL replay), then confirm the scheduler is consuming topics with zero/declining consumer-group lag.

4

Verify end-to-end

Hit /analytics/v1/health (expect {"status":"healthy"}), confirm the Control Center Analytics tab renders, and run the smoke test: a fresh sandbox payment should appear in the dashboard within 60 seconds.

The OLAP profile has real hardware cost — prod allocates roughly 250 Gi of new EBS plus a few Gi of pod memory across Kafka (3 brokers), ClickHouse (single replica, do not run it under a 4 Gi memory limit), and the scheduler. There is no PITR on Kafka (replication factor 3 covers it) and ClickHouse PITR is EBS-snapshot based with a target RPO of ~1 hour. Total ClickHouse data loss is only recoverable up to Kafka’s retention horizon (default 7 days). Read the runbook’s sizing and disaster-recovery sections before turning it on in a new environment.

Gaps & gotchas for the next owner

Three of the four endpoints degrade silently — only fee-breakdown logs an info line when it falls back to a deep-link. If you’re debugging “why is the cost dashboard empty,” don’t expect a log trail from downgrades / anomalies / forecast. The signal that OLAP is off is the { link_to_dashboard } shape in the response, not the logs.

isFallbackEligible treats 404 and 501 identically. Upstream, 404 tends to mean “OLAP disabled / namespace returns nothing” and 501 means “metric not implemented for this chart version.” Both collapse into the same deep-link, so the route cannot tell you which it was. If you need to distinguish “the pipeline is off” from “Hyperswitch never built this metric,” you have to look at the upstream status directly (e.g. via the analytics service logs), not the route’s response.

As noted above, connector / currency are documented on downgrades and anomalies in the OpenAPI fragment but dropped by the client. This is a real mismatch between the published contract and behaviour — a consumer filtering client-side will get correct (if unfiltered) data; a consumer expecting server-side filtering will be surprised. Closing this means extending the analytics.downgrades / analytics.anomalies query mappings in hyperswitch-client.ts.

forecast is gated on Hyperswitch shipping a forecast metric over OLAP, which v1.123.1 does not. Treat any plan that depends on forecast returning { data } as blocked until either (a) a Hyperswitch upgrade exposes it (see the Hyperswitch Upgrade runbook) or (b) MVS computes the forecast from ClickHouse directly — neither of which exists today.