Data sources and performance roadmap
Internal engineering roadmap — updated September 15, 2026. The Enterprise Systems Team plans to reduce query cost and response time by using existing AlloyDB data where bounded, indexed reads are safe, and by separating recent operational reads from large historical queries where appropriate.
These are planned optimizations, not delivered source changes or performance guarantees. Current sources below describe the inspected implementation and configuration; deployment, coverage and freshness must be verified during rollout. Internal caller activation remains a separate gate.
What hot and cold mean
- Hot: AlloyDB serves a bounded merchant/account lookup or an efficient maintained aggregate. Hot does not mean ingestion is real-time.
- Hot/cold: recent operational data is served from AlloyDB; older history or broad analytics stays in the warehouse behind the same API contract.
- Pure hot: preferred when AlloyDB covers the complete required history and can serve it safely; we will not add a second path unnecessarily.
We will choose retention windows and cutover dates after coverage and query-plan validation. Existing tables alone do not establish that a join is safe under load.
Current source and planned direction
All endpoint paths below are relative to /api/partner. “Not yet on API” means the source/service is available elsewhere in our platform or under evaluation, not that an endpoint can be called today.
| Capability | Current source | Planned direction |
|---|---|---|
Merchant identity and status — /merchants* |
RiseCRM, with AlloyDB authorization | Retain the merchant master; avoid substituting incomplete mirrored identity data. |
TSYS and Fiserv Omaha profiles — MID /processor-profile/* |
AlloyDB enabled in internal configuration; warehouse fallback retained | Retain hot reads; validate historical coverage before removing fallback. |
TSYS pricing — MID /pricing/tsys/*, /pricing/tsys |
Warehouse | Planned hot current-price reads using existing AlloyDB pricing records; evaluate hot/cold history or pure hot where safe. Broad portfolio work needs its own cost proof. |
Settled transactions and daily totals — /transactions, MID /transactions* |
Legacy warehouse feed; portal uses the newer Atlas feed | Planned hot/cold recent TSYS and supported Fiserv reads using existing AlloyDB transaction/batch-detail data, with Atlas-compatible semantics and stable paging. Fiserv parity remains a separate coverage check. |
| Processor batches — not yet on API | Portal warehouse aggregates; TSYS and Fiserv per-batch records have AlloyDB schema support | Planned hot merchant batch reads and hot/cold history, subject to ingestion, batch/terminal/date identity and join safety; daily Fiserv totals are a different grain. |
ACH deposits and deposit-linked batches — MID /deposits, /deposits/{deposit_id}, /deposits/{deposit_id}/batches/{header_id} |
AlloyDB (TSYS ACH Detail mirror; same source as the merchant portal). Month-end fee debits come from the pipeline's month_end_fee_debits view: the monthly ACH file where TSYS sends one, otherwise derived from the monthly fee report (see Month-end fees). Endpoints implemented; deployment and serving-tier validation pending |
Planned hot, pending query-plan and capacity validation. Bounded merchant, bank and processing-date reads; funding adjustments keep their signs; coverage is reported as unknown and effective dates do not confirm bank receipt. Fiserv-processed merchants receive 404 RESOURCE_NOT_FOUND with an explicit Fiserv funding-unavailable message before any funding query. Missing or ambiguous TSYS funding identity receives a distinct 404 RESOURCE_NOT_FOUND: "TSYS funding identity is unavailable for this merchant". Neither response means "no deposits". Unknown or null processor metadata falls through to TSYS identity resolution. |
TSYS chargebacks — /chargebacks*, MID /chargebacks |
Warehouse case rollups | Planned hot merchant case reads from existing AlloyDB actions; evaluate hot/cold history. Raw actions must preserve the current case/status/amount semantics. |
| Fiserv disputes — not yet on API | No normalized API adapter; AlloyDB chargeback and sync tables exist | Planned hot feasibility and endpoint work after ingestion, coverage, MID mapping and case semantics are verified. |
Statement list/detail/PDF — /statements*, MID /statements* |
AlloyDB metadata + GCS PDF storage | Retain hot metadata and object streaming, including custom-over-standard PDF precedence. |
| Dispute case-action PDFs — not yet on API | Portal AlloyDB metadata + GCS documents | Planned reuse of the existing hot metadata and authorized document stream. |
PCI status/history — MID /compliance* |
AlloyDB mirror | Retain hot. Detailed scans/reports/certificates use the compliance vendor, not the warehouse. |
Authorizations — /authorizations, MID /authorizations* |
AlloyDB: the TSYS authorization file mirror (daily) and, for merchants processed on Fiserv Omaha FSP, the Fiserv authorization feed mirror (refreshed roughly hourly; history from 2026-01-01 with a rolling 15-month floor). and, for merchants on Fiserv North or Fiserv Omaha Retail, the Fiserv AccessOne authorization feed (refreshed daily; times are the processor's wall-clock time in authorized_at_local). Fiserv rows are purchases, pre-authorization requests, pre-authorization completions and returns (negative amounts) |
Retain hot for TSYS; Fiserv Omaha FSP merchant reads are planned hot pending capacity validation. meta.data_coverage reports tsys_only, fiserv_fsp or fiserv_accessone, and new values may be added. The portfolio route covers TSYS only. |
| Dashboard, sales and card mix — not yet on API | Portal warehouse aggregates | Planned hot/cold merchant summaries; precompute bounded aggregates where raw-row joins are expensive. |
| Cost and hours insights — not yet on API | Portal AlloyDB TSYS reconstruction/authorization data; Fiserv fee metadata exists but mapping is unfinished | Planned hot endpoint reuse and Fiserv cost mapping/coverage validation. Fiserv hours coverage is not established. |
Gateway transaction reads — MID /gateway-transactions* |
ARISE warehouse; NMI vendor API; local account routing | Planned hot/cold feasibility against the gateway store once ingestion/account parity is proven. Existing vendor reads remain where data is not mirrored. No gateway writes are added. |
Residual reporting and exports — /residuals/* |
Warehouse financial marts + object exports | Retain warehouse for broad portfolio analytics; evaluate bounded summary caching only after profiling. |
Capital rollups — /capital/* |
Warehouse | Retain warehouse for partner financial rollups. A merchant session is a separate Jaris integration, not a rollup or storage migration. |
| Support and user/access management — not yet on API | Zendesk/RiseCRM; AlloyDB user assignments and identity-provider provisioning | Retain appropriate vendor and identity sources; preserve trusted human authorization. |
Protecting the shared database
We will move a read only when it has verified coverage, selective indexes, bounded date/page limits and acceptable query plans under representative concurrent load. A warehouse scan will not simply become an unbounded relational join. Large portfolio queries may remain in the warehouse even when merchant detail moves to AlloyDB.
For tiered reads we will define a coverage boundary, preserve deduplication, corrections, totals and cursor order, and report unavailable data explicitly. We will not silently mask source failures with empty results or use unrestricted warehouse fallback during database overload.
We will compare correctness, query cost and latency, validate authorization on both tiers, and roll out with a tested rollback. This work does not increase rate limits or enable new callers.
What this means for your integration
Continue to call the documented API from your authorized server-side BFF. You do not need database credentials, a reporting database of your own, or source-specific requests. Shared paths, response contracts and authorization remain the integration boundary; use the source/freshness metadata each operation exposes rather than assuming hot means current.
The first planned investigations are pricing, recent transactions/batches and merchant disputes where AlloyDB records already exist. Existing hot funding, statement, compliance and authorization services provide a starting point for the missing merchant reads. No migration date or latency target is committed on this page.
Internal documentation hosting
Internal Modulate documentation is published with Scalar. External partner documentation remains on Fern. We reserve our single free Fern domain for external partners; this roadmap belongs on the internal Scalar site, with verified @flute.com users as its intended audience. Site access is configured separately in Scalar.
