Imagine you're paying ₹50,000 for your college tuition.
You click Pay.
Processing...Nothing happens.
Five seconds later...
Still spinning.
You panic.
You click Pay again.
Nothing.
So you refresh the page.
Meanwhile...
Your bank retries the request.
The payment gateway retries it too.
By the time your backend receives everything...
It has received five identical payment requests.
Request #1
Request #2
Request #3
Request #4
Request #5Now imagine your backend simply does this.
processPayment();
deductMoney();
saveTransaction();Congratulations.
You've just charged the customer five times.
The Naive Solution
Many engineers assume:
"I'll just check if the payment already exists."
SELECT *
FROM payments
WHERE transaction_id = ?If not found...
Process payment.
Looks correct.
Until two requests arrive simultaneously.
Server A
↓
Payment Not Found
--------------------
Server B
↓
Payment Not FoundBoth continue.
Both deduct money.
You've recreated the exact same race condition that causes double booking.
The Real Problem
Payment systems don't fail because payments are difficult.
They fail because networks are unreliable.
Requests can be:
- Retried
- Delayed
- Duplicated
- Delivered out of order
Your backend must assume every request might arrive multiple times.
The Solution: Idempotency
One of the most important concepts in distributed systems is Idempotency.
It simply means:
Performing the same operation multiple times should produce the same result.
Every payment request includes a unique key.
POST /payments
Idempotency-Key:
8d7fa2-91bc-32The server first checks:
Have I already processed this key?If yes...
It doesn't charge the card again.
It simply returns the original response.
200 OK
Payment SuccessfulThe customer clicked Pay five times.
Money was deducted only once.
Where Is The Key Stored?
Usually inside Redis or a database.
Idempotency Key
↓
Payment Status
↓
ResponseThe next request with the same key immediately returns the stored result.
No business logic executes again.
But What If The Server Crashes?
Here's a tougher interview question.
Suppose the payment succeeds.
Money leaves the customer's bank.
Before the server responds...
The server crashes.
The client never receives:
200 OKNaturally...
The client retries.
Without idempotency...
The customer gets charged again.
With idempotency...
The backend sees the same key.
Already Processed
↓
Return Previous ResultNo duplicate payment.
Payment States
Real payment systems don't simply have:
Success
FailedThey usually move through several states.
Created
↓
Pending
↓
Processing
↓
CompletedOr
Created
↓
Pending
↓
FailedIf another request arrives while the payment is still processing...
The backend simply returns:
Payment Already In ProgressInstead of starting another transaction.
Why Queues Matter
Suppose Razorpay receives:
100,000 payments
per secondCalling the bank synchronously would overwhelm downstream systems.
Instead...
Payments are often placed into a queue.
Client
↓
API
↓
Kafka
↓
Payment Worker
↓
BankWorkers process requests one at a time.
Retries become easier.
Failures become isolated.
The API remains responsive.
Exactly-Once Processing
Interviewers love asking:
Can Kafka guarantee exactly once?
The honest answer is:
Not by itself.
Exactly-once behavior is achieved by combining:
- Idempotency
- Transactions
- Offset management
- Deduplication
No single technology solves the problem alone.
What Happens If The Bank Times Out?
Another classic scenario.
Your backend sends:
Debit ₹50,000No response.
Did the bank receive it?
Nobody knows.
Should you retry?
Maybe.
Should you assume failure?
Also dangerous.
Production systems usually move the payment into a temporary state.
Pending ConfirmationA background worker later checks with the bank.
Payment Confirmed
↓
Completedor
Payment Failed
↓
RefundGuessing is never acceptable when real money is involved.
Follow-Up Questions Interviewers Love
Why not trust the frontend?
Because clients can retry, refresh, disconnect, or even send malicious duplicate requests.
The backend must guarantee correctness.
Why not use a database transaction?
Database transactions protect your database.
They cannot roll back a successful bank transfer.
Distributed systems require different techniques like idempotency and compensation.
Why use queues?
Queues smooth traffic spikes, isolate failures, and allow asynchronous processing without blocking users.
What happens if Redis loses the idempotency key?
Most payment systems persist critical information in a durable database and often replicate Redis. The key itself is valuable, but the payment record remains the source of truth.
Lessons Beyond Payments
The same idea appears everywhere.
- Booking movie tickets.
- Reserving hotel rooms.
- Creating online orders.
- Sending emails.
- Processing refunds.
- Scheduling deliveries.
Any operation that must happen exactly once eventually relies on idempotency.
Final Thoughts
Building a payment system isn't about calling a bank API.
It's about assuming that everything will eventually fail.
Networks disconnect.
Servers crash.
Users refresh pages.
Gateways retry requests.
Banks respond slowly.
The systems that survive production aren't the ones that process payments the fastest.
They're the ones that guarantee the customer is charged exactly once, no matter how chaotic the network becomes.
That's the kind of engineering users never notice—but always depend on.