Anatomy of a bet — a guided trace tour¶
This document walks you through a single dice bet, span by span. By the end you'll understand how a POST /casino/games/house/dice/bet travels from the controller through the Prisma transaction, out to Redis, and into the BullMQ side-effect queue — and you'll be able to read any house-game trace in Jaeger without guidance.
Prerequisites: you've completed Day 1 of the onboarding curriculum. You have the stack running, you've placed your first bet, and you know where Jaeger lives. You don't yet need to understand the bet pipeline in detail — that's what this tour teaches.
Time: ~20 minutes to read, ~30 minutes with the exercises.
1. Setup — viewing the trace¶
Open Jaeger at http://localhost:16686.
If the reference trace 8aaa902b3964af1d33dec7000bb36e02 is still in retention, open it directly:
If it has expired, generate a fresh one:
After the test passes, find the new trace in Jaeger:
- Service →
ebit-api - Operation →
POST /casino/games/house/dice/bet - Sort by Most Recent
- Click the first result
You should see roughly 102 spans in a single trace, all under ebit-api. The total duration is ~100–120 ms. Pin this tab — you'll reference it throughout the tour.
2. Trace overview — the shape of a bet¶
Before we dive into individual spans, notice the trace's structure. The 102 spans fall into six tiers, each representing a different layer of the stack:
Tier Span count Example span name
────────────────────── ────────── ──────────────────────────────
HTTP entry + middleware 12 POST /casino/games/house/dice/bet
Controller + handler 2 DiceController.bet, bet
Prisma transaction 1 prisma:client:transaction
├─ Prisma operations 9 prisma:client:operation (findUnique, create, update…)
│ └─ Engine queries 9 prisma:engine:db_query (SELECT, INSERT, UPDATE…)
└─ Engine commit 1 prisma:engine:commit_transaction
Post-commit side effects 5 ioredis publish, evalsha, unlink
The middleware spans (CORS, cookie parser, JSON body parser, JWT authentication) are identical across every authenticated endpoint. We'll skip them and focus on the 25 spans that are specific to the bet pipeline.
Read the trace top-to-bottom in Jaeger — each span is a child of the one above it. The indentation shows you the call tree. The horizontal bar shows you the wall-clock time.
3. Span-by-span walkthrough¶
Each entry below gives the span name, what's happening, the source code location, and why it matters. The spans are in execution order.
Span 1 — POST /casino/games/house/dice/bet (root)¶
What: Express receives the HTTP request. This root span wraps the entire request lifecycle.
Where: Generated by @opentelemetry/instrumentation-http. The route is registered by NestJS from dice.controller.ts:31.
Why it matters: The root span's duration is the user-perceived latency. Everything else is a child of this span. If this span is slow, dig into its children to find the bottleneck.
Span 2 — DiceController.bet (nestjs.type: request_context)¶
What: NestJS's request context wraps the controller method. Tags: nestjs.callback = bet, http.route = /casino/games/house/dice/bet.
Where: apps/api/src/casino/house/dice/dice.controller.ts:31-37
Why it matters: This span includes guard execution (JWT validation, throttle check) plus the full handler. If you see a long gap between this span starting and the handler span starting, that gap is spent in guards.
The controller is decorated with @Throttle({ bets: { limit: 25, ttl: 5000 } }) at line 33 — 25 bets per 5 seconds per user, shared across all house-game endpoints.
Span 3 — bet (nestjs.type: handler)¶
What: The actual handler method. Calls this.diceService.play(dto, request.user).
Where: apps/api/src/casino/house/dice/dice.controller.ts:35
Why it matters: This is where application code begins. Everything from here down is within DiceService.play, which is wrapped by @PlaceBetLock.
Span 4 — ioredis-SETNX (PlaceBetLock acquire)¶
What: Redis SETNX sets the key bet-lock:<userId> with a 5-second TTL. This is the per-user mutex that serialises all bet operations.
Where: Decorator at libs/shared/src/security/place-bet-lock.decorator.ts:19-30. Applied to DiceService.play at apps/api/src/casino/house/dice/dice.service.ts:34.
Why it matters: Without this lock, two concurrent bets from the same user could race on the fairness seed nonce, producing duplicate random values. The lock also prevents double-settlement. If this span shows retries (multiple SETNX attempts), the user has another bet in flight.
Span 5 — prisma:client:transaction (outer transaction boundary)¶
What: Opens a Prisma interactive transaction. All database operations from here until the commit are atomic — either everything writes or nothing does.
Where: @PrismaTransactional() decorator on BetService.createAndSettleBet at apps/api/src/bet/bet.service.ts:560.
Why it matters: This is the bet's atomicity guarantee. The nine Prisma operations below are all children of this span. If any throws, the entire transaction rolls back — balance deduction, transaction records, and the bet row all revert together.
Span 6 — prisma:client:operation — UserFairnessSeeds upsert¶
What: popUserSeed(userId) fetches the user's current server seed, client seed, and nonce, then atomically increments the nonce.
Where: apps/api/src/provably-fair/provably-fair.service.ts:44-52, called from dice.service.ts:45.
Why it matters: This is the provably-fair RNG input. The returned serverSeed + clientSeed + nonce are fed into HMAC-SHA256 to produce the dice roll. Because this runs inside the transaction, a failed bet rolls back the nonce increment — so the same nonce can be reused on retry without breaking fairness guarantees.
The upsert (not just update) auto-creates a seed row on a user's first-ever bet, so seeding scripts don't need to pre-populate UserFairnessSeeds.
Span 7 — prisma:client:operation — GameIdentity findUnique¶
What: Looks up the game record for dice (resolves to Game.id = 7).
Where: BetHelper.getHouseGameIdentity('dice'), called from dice.service.ts before createAndSettleBet.
Why it matters: Provides the gameId foreign key for the Bet record. Also checks game.enabled — if an admin has disabled the game, this is where the request rejects with CASINO_GAME_NOT_AVAILABLE.
Span 8 — prisma:client:operation — UserSelfExclusion findFirst¶
What: checkCanUserBet(identity) queries for active self-exclusion records.
Where: apps/api/src/bet/bet.service.ts:342-354, delegating to apps/api/src/users-limits/user-limits.service.ts:19.
Why it matters: Regulatory/responsible-gambling check. If the user has set a self-exclusion period, this throws before any balance mutation. The check is cached for 60 seconds via @Cacheable — so repeated bets within a minute skip the DB query (you'll see this span disappear on rapid consecutive bets).
Span 9 — prisma:client:operation — UserBalance update (WITHDRAW)¶
What: Deducts the bet amount from the user's spendable balance. 1000 DBC → 999.9 DBC.
Where: apps/api/src/accounting/services/accounting.service.ts:193 — the createOrFindTransaction method issues a conditional UPDATE UserBalance SET amount = amount - :betAmount WHERE amount >= :betAmount.
Why it matters: The WHERE amount >= betAmount clause is the insufficient-funds guard. If the user's balance is too low, the update affects zero rows, and the service throws USER_INSUFFICIENT_FUNDS. There is no separate balance check — the guard and the deduction are the same statement, preventing a TOCTOU race.
Span 10 — prisma:client:operation — Transaction create (WITHDRAW)¶
What: Creates the ledger entry for the withdrawal: tx-withdraw-DBC-7-bet-<uuid>, type = WITHDRAW, status = COMPLETED, tag = BET.
Where: apps/api/src/accounting/services/accounting.service.ts — within the same createOrFindTransaction call.
Why it matters: Every balance change has a corresponding Transaction row. The deterministic id format (tx-<direction>-<currency>-<gameId>-<betId>) means that replaying the same bet with the same roundId hits a unique constraint before creating a duplicate ledger entry.
Span 11 — prisma:client:operation — UserBalance upsert (DEPOSIT)¶
What: Credits the payout to the user's balance. 999.9 DBC → 1000.098 DBC (the user won).
Where: Same createOrFindTransaction flow, but for the deposit leg.
Why it matters: This uses upsert rather than update because a user might win in a currency they've never held — the upsert creates the UserBalance row on demand. On a loss (payout = 0), this span and span 12 are absent entirely — the deposit leg is gated by payout > 0.
Span 12 — prisma:client:operation — Transaction create (DEPOSIT)¶
What: Creates the deposit ledger entry: tx-deposit-DBC-7-bet-<uuid>, type = DEPOSIT, status = COMPLETED.
Where: Same accounting flow as span 10, deposit leg.
Why it matters: On a losing bet, you won't see this span. That asymmetry is worth knowing — if you're debugging a trace and notice only one Transaction create, the user lost.
Span 13 — prisma:client:operation — UserPromoCode findFirst¶
What: Looks up any active promo code for commission calculation in beforeSettleBet.
Where: apps/api/src/bet/bet.service.ts — beforeSettleBet method, called at line 565.
Why it matters: This is a no-op for the seeded test user (no promo code). In production, promo codes affect the commissionGgrUsdAmount field on the bet. You'll see this span on every bet trace, but its duration is negligible.
Span 14 — prisma:client:operation — Bet create¶
What: Writes the Bet row with status = SETTLED, amount = 0.1, payout = 0.198, gameId = 7, roundId = <uuid>.
Where: apps/api/src/bet/bet.service.ts:569 — this.betRepository.createBet(...).
Why it matters: This is the bet record. The @@unique([roundId, userId]) constraint at libs/_prisma/src/schema/api.prisma:668 is the only backstop against double-settlement. If the same roundId + userId pair is submitted twice, this create throws a unique violation and the entire transaction rolls back.
Span 15 — prisma:engine:commit_transaction¶
What: The Prisma engine commits the transaction to Postgres.
Where: Internal to @prisma/client. Corresponds to the COMMIT statement in the underlying SQL.
Why it matters: Everything before this span is tentative — a crash before commit means nothing was written. Everything after this span is a side effect triggered by PrismaTransactional.onClosed, which fires only after a successful commit. This is the durability boundary.
Span 16 — ioredis-PUBLISH (BalanceUpdated — WITHDRAW)¶
What: Publishes the withdrawal balance change to Redis pub/sub channel server_channel_event.BalanceUpdated.
Where: apps/api/src/accounting/services/accounting.service.ts:365-386 — notifyBalanceUpdated, triggered by the onClosed callback.
Why it matters: The ebit-rt service (port 4001) subscribes to this channel and relays balance updates to the user's websocket room. This is how the UI updates the balance counter in real time. Note: this publish is fire-and-forget — if Redis is down, the user's UI shows a stale balance until the next page refresh.
Span 17 — ioredis-PUBLISH (BalanceUpdated — DEPOSIT)¶
What: Same as span 16, but for the deposit leg.
Where: Same notifyBalanceUpdated path.
Why it matters: Two PUBLISH events per winning bet (one per ledger entry). The UI receives both in rapid succession — you'll see the balance dip then rise. On a losing bet, only one PUBLISH fires.
Span 18 — ioredis-EVALSHA (bet_settled_queue enqueue)¶
What: BullMQ enqueues a job on bet_settled_queue via a Redis Lua script (EVALSHA).
Where: apps/api/src/bet/queue/bet.queue-producer.ts:31-46 — pushBet method. Queue name defined at apps/api/src/bet/queue/const.ts:1.
Why it matters: This is where the bet's side effects begin their separate lifecycle. The job carries the serialised bet data and a notificationDelayMs hint. The queue processor (same process, different consumer) fans out into:
- Live-bets notification (latest/high-roller/lucky/big wins)
- Rakeback calculation
- Leaderboard UPSERT
- Affiliate notification
- User stats update
All of these happen asynchronously. If Redis is unavailable at this moment, the HTTP response still returns 201 — but the side effects are silently lost (SF-007).
Span 19 — ioredis-EVALSHA (updateSessionQueue)¶
What: BullMQ enqueues a session-touch job. Every authenticated request refreshes the user's session timestamp.
Where: apps/api/src/auth/session/session.queue-producer.ts.
Why it matters: This span appears on every authenticated request, not just bets. It's the same session update you saw when you traced sign-in on Day 1. If you're comparing two traces and wondering "why does this extra EVALSHA appear?" — this is why.
Span 20 — ioredis-UNLINK (PlaceBetLock release)¶
What: Deletes the bet-lock:<userId> key, releasing the per-user mutex.
Where: libs/shared/src/security/place-bet-lock.decorator.ts — the WaitMutex finally block.
Why it matters: This is the last span in the trace. After this, the user can place another bet. The gap between span 15 (commit) and span 20 (unlock) is the "post-commit side-effect window" — about 5–10 ms of Redis publishes and queue enqueues. The HTTP response returns somewhere between the commit and the unlock.
4. Cross-references¶
Each major section of this trace maps to a section in the flow documentation:
| Trace section | Flow doc | Section |
|---|---|---|
| Controller + guards | dropbet-bet-place.md |
§1 User-visible contract |
| PlaceBetLock | dropbet-bet-place.md |
§4.2 (handler) |
| popUserSeed + RNG | dropbet-bet-place.md |
§4.2 (UserFairnessSeeds upsert) |
| createAndSettleBet | dropbet-bet-place.md |
§4.2 (full per-method walkthrough) |
| Balance + Transaction writes | dropbet-bet-place.md |
§5 Data model |
| Side-effect queue | dropbet-bet-place.md |
§4.3 (onClosed side effects) |
| Failure modes | dropbet-bet-place.md |
§6 Failure modes |
| Provably-fair scheme | glossary.md | provably fair, popUserSeed, nonce |
| PlaceBetLock semantics | glossary.md | @PlaceBetLock |
| bet_settled_queue | glossary.md | bet_settled_queue, BetQueueProducer |
5. Reader exercises¶
Try to answer these before checking the answers. Each question tests whether you understood the why, not just the what.
Q1. Why does popUserSeed run inside the Prisma transaction?¶
Q2. What happens if Redis is down when span 18 (bet_settled_queue enqueue) fires?¶
Q3. Why are there two BalanceUpdated publishes on a winning bet but only one on a losing bet?¶
Q4. If the PlaceBetLock key expires mid-handler (its TTL elapses before the handler returns), what could go wrong?¶
Q5. You're looking at a trace and you see only one Transaction create span instead of two. Is this a bug?¶
Answers¶
A1. If popUserSeed ran outside the transaction, a failed bet (e.g., insufficient funds at span 9) would still increment the nonce. The user's next bet would skip a nonce value. While not a security issue, it breaks the provably-fair audit trail — the user expects nonce N to correspond to a bet, but nonce N was consumed by a failed attempt. Running inside the transaction means a rollback reverts the nonce increment too.
A2. The HTTP response still returns 201 with the correct DiceBetResponseDto. The bet is settled and persisted in Postgres. But the side effects — live-bets broadcast, rakeback calculation, leaderboard update, affiliate notification — are silently dropped. The pushBet call wraps the enqueue in a .catch() that logs via EvoLogger.fatal but does not throw. This is documented as SF-007 in the security register. The user's balance is correct; the downstream consumers just never learn about the bet.
A3. The accounting service creates one Transaction per ledger movement. A winning bet has two movements: WITHDRAW (deduct the bet amount) and DEPOSIT (credit the payout). Each createOrFindTransaction call triggers a notifyBalanceUpdated in its onClosed callback. A losing bet has payout = 0, so the deposit leg is skipped entirely — no UserBalance upsert, no Transaction create, no BalanceUpdated publish. The code gate is in buildHouseTxArgs: the deposit transaction arguments are only constructed when payout > 0.
A4. Two concurrent bets from the same user could both acquire the lock (the first user's SETNX key expired, the second user's SETNX succeeds). Both handlers call popUserSeed — and if they read the same nonce before either increments it, they produce identical HMAC-SHA256 outputs and therefore identical dice rolls. The @@unique([roundId, userId]) constraint on the Bet table prevents double-settlement of the same round, but the fairness-seed race (SF-005) means two different rounds could share the same RNG input. The defence is the lock TTL being much longer than the handler duration (~100 ms handler vs multi-second TTL).
A5. Not a bug — the user lost. The deposit leg (payout > 0 gate) was skipped because the payout was zero. You'll see: one UserBalance update (the withdrawal), one Transaction create (the withdrawal ledger entry), and the Bet create with payout = 0. The trace is shorter on a loss than on a win. Compare a few traces side-by-side to build intuition for the shape of winning vs. losing bets.