How to solve concurrency at scale?

System Design
1

how concurrency will be handled in applications like ticketmaster. sceanrio: thousands of users simulateneously trying to book ticket to same movie show. how can we reduce the contention.

2

Designing for high concurrency implies we try to stay away from transactions and use ā€œoptimistic lockingā€ - rather than pessimistic locking. This usually has an impact on how you design your APIā€™s and even within an API how you break things into transaction boundaries. At a high level the less amount of code you have in transaction the better.

3

@ashish3312 @Dushyant what are your thoughts ?

4

Optimistic Locking really isnā€™t a great approach for high transaction requests that need correctness. Thatā€™s why places like Ticketmaster tend to use holding queues to process requests. Amazon learned this a long time ago when they started moving into waitlists.

Iā€™d refine your question to look at how to handle high writes in a NoSQL situation where locking a document will hurt performance.

5

@makers welcome to the community and ā€‹thanks for the insight.

What are holding queues? is it the same as message queues?

6

Itā€™s really moving a synchronous request to an asynchronous one (take each request and move it to an ordered queue to process).

Airbnb had write up around how to handle payments that was similar I think :slight_smile:

1 Like
7

Three other techniques to increase the number of concurrent requests that can be handled:

  1. Publish-subscribe notifications, which reduces the load of polling.
  2. Using a workflow orchestration system like Airflow or Durable Task Framework for long-running operations which fan out to multiple other operations.
  3. Add more compute hardware (eg virtual machines).
1 Like
8

For higher scale data operations, NoSql is also a better option over Sql for the following reasons:

  1. No joins so data can be read from one place
  2. No data transformation so data is read and written as is without any extra processing overhead.
  3. Itā€™s easier to scale out NoSql databases compared to Sql databases. (https://www.mongodb.com/basics/scaling)
1 Like
9

@makers you need to isolate the transactions atleast for the ticket booking part, because we need to ensure that if the user has chooses seat no x, we need to ensure that seat X is not assign to any other user, So putting into a Queue system does not make sense.

Concurrency
How to handle concurrency; such that no two users are able to book the same seat?

We can use transactions in SQL databases to avoid any clashes. For example, if we are using SQL server we can utilize Transaction Isolation Levels to lock the rows before we update them. Note: within a transaction, if we read rows we get a write-lock on them so that they canā€™t be updated by anyone else. Here is the sample code:

SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
 
BEGIN TRANSACTION;
 
    -- Suppose we intend to reserve three seats (IDs: 54, 55, 56) for ShowID=99 
    Select * From ShowSeat where ShowID=99 && ShowSeatID in (54, 55, 56) && isReserved=0 
 
    -- if the number of rows returned by the above statement is NOT three, we can return failure to the user.
    update ShowSeat table...
    update Booking table ...
 
COMMIT TRANSACTION;
ā€˜Serializableā€™ is the highest isolation level and guarantees safety from Dirty, Nonrepeatable, and Phantoms reads.

Once the above database transaction is successful, we can safely assume that the reservation has been marked successfully and no two customers will be able to reserve the same seat.

Read JDBC Transaction Isolation Levels for details.

Also, some part like sending the email, notification can be done in more async fashion.

  1. Once ticket is booked, it can put into a queue.
  2. Worker can pick this up and send email/notification etc.
1 Like
10

1st method which is Dirty read, but in highly concurrent system like book my show/ticket master dirty read is not good idea because every time you have high chance of failure.

Lets say when a transaction1 begin it mark seat1,seat2 unavailable since its selected though its not completed yet.
When transaction2 starts it does not see seat1, seat2ā€¦

if because of some reason transaction1 abort/timeout seat1,seat2 will be available

more read below how transaction isolation works.

11

@Soumya_Acharya thanks for the details . I think optimistic locking is more scalable

12

in the context of movie ticket booking at scale, you want to use serialization isolation level.
Which provide highest level of concurrency preventing the dirty read.

https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ConceptsGuide/Other/SERIALIZABLEIsolation.htm