PesaGuard — technical case study
A walk through the problem, the architecture, and what testing actually showed, in the order an engineer would reason through it.
Problem
M-Pesa transaction volumes for a small merchant were reconciled by hand at end of day, which meant discrepancies — failed callbacks, duplicate webhooks, timing mismatches — were only caught after the fact, sometimes days later.
Requirements
Reconcile transactions against the ledger continuously rather than in a daily batch. Flag anomalies for review within minutes, not days. Never lose a transaction record even if a downstream service is briefly unavailable.
Architecture
Webhook ingestion into an event processor, fanned out to a Kafka-backed fraud path and a direct-write validation path into PostgreSQL. See the full interactive diagram on the Architecture page.
Data model
Two core tables: transactions (system of record, keyed on transaction ID) and ledger_entries (external ledger snapshot). Reconciliation is a set difference between the two, not a mutation of either.
Implementation
Built incrementally: webhook ingestion and storage first, then the reconciliation job, then the Kafka fraud-scoring path once the core pipeline was stable. See selected snippets on the Code page.
Engineering decisions
Chose PostgreSQL over a NoSQL store for the ledger because reconciliation is fundamentally a relational join problem. Chose Kafka specifically for the fraud path (not the whole pipeline) because only that path needed replay and multiple independent consumers.
Bottlenecks
The reconciliation query slowed as the ledger grew, since it scanned all unsettled rows on every run. Resolved with a partial index — see the indexing snippet on the Code page — rather than denormalizing the schema.
Failure handling
Idempotency keys on transaction IDs make webhook retries safe. Validation failures route to a dead-letter topic instead of blocking the stream. The fraud scorer falls back to rule-based scoring if the statistical model is unavailable.
Security
Webhook signatures are verified with HMAC before any processing. Database roles are scoped per service with least privilege. No raw M-Pesa identifiers are logged outside the primary transaction table.
Performance
Load test conditions: 100 simulated tenants writing concurrently at concurrency level 8, sustained until 1,000,000 rows were written. Figures are from a load test run, not production traffic — see the Experiments page for method.
Results
Reconciliation moved from a manual, end-of-day process to a continuous automated check, with the pilot customer now seeing discrepancies flagged within the same processing cycle rather than the next business day.
Lessons learned
Introducing Kafka only where replay genuinely mattered kept the system easier to operate than putting every path through a message bus by default. A partial index solved the bottleneck more cheaply than a schema rewrite would have.