Test entitlement races with controlled concurrent requests

Understanding Entitlement Race Conditions

Entitlement systems enforce business rules governing what a user can access, consume, or execute. When these checks involve database state, they are vulnerable to Time-of-Check to Time-of-Use flaws if not properly isolated. In concurrent environments, multiple requests may read the same initial state before any request commits its mutation.

Without adequate concurrency control, two parallel requests might both validate an entitlement limit, such as a maximum number of API calls or available promotional credits, and both proceed to consume the resource. Testing for these conditions requires intentionally delaying execution or flooding the endpoint with parallel requests to force overlapping transactions.

Concrete Worked Example: Coupon Redemption

Consider a hypothetical synthetic fixture managing coupon redemptions. The endpoint accepts a POST request to apply a single-use discount to a cart. The server implementation opens a database transaction, queries the coupon status to ensure it is marked as unused, applies the discount to the cart, updates the coupon status to used, and commits.

If an engineer sends two identical redemption requests for the same coupon concurrently, both requests may evaluate the SELECT query simultaneously. Because neither transaction has committed the UPDATE yet, both requests see the coupon as unused. Both applications apply the discount and update the coupon status to used, resulting in double redemption. Testing this requires a test runner capable of dispatching multiple HTTP requests within the same millisecond window to a controlled staging environment.

Safe Test Matrix and Expected Outcomes

To reliably verify concurrency controls, engineers must define a safe test matrix using synthetic, isolated data fixtures. A standard test matrix for a single-use entitlement under concurrent load involves dispatching N requests simultaneously. The outcomes determine the effectiveness of the concurrency mitigation.

In this matrix, we define the baseline and expected outcomes for a successful defense.

• Scenario: 5 concurrent requests to redeem a single-use token.
• Expected Allowed Outcome: Exactly 1 request receives an HTTP 200 Success response and the token is consumed.
• Expected Denied Outcome: Exactly 4 requests receive an HTTP 409 Conflict or HTTP 400 Bad Request response, indicating the token is already consumed or the transaction failed.
• Failure Outcome: 2 or more requests receive an HTTP 200 Success, indicating a race condition was successfully exploited.

Limitations of PostgreSQL Read Committed Isolation

According to the PostgreSQL 18 documentation on Transaction Isolation, Read Committed is the default isolation level. When a transaction uses this level, a SELECT query sees only data committed before the query began. However, two successive SELECT commands within the same transaction can see different data if concurrent transactions commit changes in between.

For updating commands like UPDATE, DELETE, and SELECT FOR UPDATE, PostgreSQL will find target rows committed as of the command start time. If a concurrent transaction has already updated the target row, the Read Committed transaction will wait. If the first updater commits, the second updater re-evaluates its WHERE clause against the new row state. If the condition still matches, it updates the new row. This behavior makes Read Committed generally unsuitable for complex entitlement checks where the initial SELECT dictates the application logic, as it cannot prevent the Time-of-Check to Time-of-Use anomaly described in our coupon example.

Mitigating Races with Repeatable Read and Serializable Levels

To address these anomalies, PostgreSQL offers stronger isolation levels. The Repeatable Read level guarantees that a transaction sees a snapshot of the database as of the start of its first non-transaction-control statement. It prevents dirty reads and nonrepeatable reads. If a concurrent transaction modifies a row that the Repeatable Read transaction subsequently attempts to modify, PostgreSQL will abort the transaction with the message ERROR: could not serialize access due to concurrent update. Applications must catch this error and retry the transaction.

The Serializable isolation level provides the strictest isolation, emulating serial transaction execution. It operates similarly to Repeatable Read but also monitors for serialization anomalies using predicate locks. These locks, visible in the pg_locks system view as SIReadLock, track read/write dependencies without blocking. If a dependency cycle is detected that would lead to an anomaly, one of the transactions is rolled back with an ERROR: could not serialize access due to read/write dependencies among transactions. When using Serializable transactions, any data read from a permanent table cannot be considered valid until the reading transaction successfully commits.

Implementation Tradeoffs and Failure Modes

Moving from Read Committed to Repeatable Read or Serializable requires architectural tradeoffs. The primary tradeoff is the necessity of implementing robust application-level retry logic. Because PostgreSQL resolves conflicts at these higher isolation levels by aborting transactions, the application must transparently catch SQLSTATE 40001 errors and restart the operation from the beginning.

Failure modes in this architecture occur when developers forget to implement retry loops, leading to intermittent 500 Internal Server Errors under load. Furthermore, overusing Serializable isolation without optimizing queries can lead to memory exhaustion from tracking too many SIReadLocks, which PostgreSQL mitigates by promoting fine-grained tuple locks to coarser page locks. Alternatively, engineers can remain on Read Committed and use explicit row locking via SELECT FOR UPDATE, but this introduces blocking and potential deadlocks if application lock acquisition ordering is inconsistent.

Sources and Acknowledgments

Factual claims regarding PostgreSQL concurrency models, Read Committed behaviors, Repeatable Read snapshot isolation, Serializable predicate locks, and SQLSTATE 40001 errors are sourced directly from the official PostgreSQL documentation.

Drafted with Gemini and checked for source links, advisory identifiers, structure and unsupported product claims. Examples are synthetic and are not scan results.

Drafted with Gemini and checked for source links, advisory identifiers, structure and unsupported product claims. Examples are synthetic and are not scan results.

• PostgreSQL 18: Transaction Isolation