Skip to main content

Twitter System Design

Functional Requirements of the System.​

  • Users should be able to create tweets.
  • Users should be able to search for tweets by query.
  • Tweet content will be restricted to 280 characters of text.

Non-Functional Requirements of the System.​

  • The system needs to be highly available.
  • The system should be highly reliable. Tweets created by users should never be lost.
  • The accepatble latency for search results is 200ms.

Capacity Estimation​

We have 500M new tweets each day. Given that the average tweet is 33 characters long, we can estimate a conservative average tweet size of 134 bytes. We calculated that by multiplying the character length by the character encoding's byte size per encoded character. We need to use the character encoding utf8mb4 to support the emoji's used in tweets. utf8mb4 uses up to 4 bytes to encode each character. Lastly we add two bytes to account of the overhead of storing text in mysql.

Character LengthCharacter EncodingAverage Tweet Size
33utf8mb4134
Average Tweet SizeTotal TweetsTotal BytesPer Year Storage
134500M134_33_4 ~ 67 GBs67GB * 365 days = 24.445 TB

Characteristics of the System.​

Statistics of the System.​

DescriptionValue
Write600 per second.
Read600,000 per second.
Total number of Users330 million Monthly Active Users.

Twitter is a Read Heavy System. So the system that you make should be Read Heavy, Eventual Consistency and Storage Efficient.
πŸ“–Read Heavy: Something
πŸ“–Eventual Consistency: Something
πŸ“–Storage Efficient: Something

Sub-Systems of the System.​

How to generate User Timelines.​

How to generate Home Timelines.​

Database Design.​

Choosing a Database.​

Best choice for the database is Redis and along with it have to use some DB to store the data.

How Redis supports the Twitter System​

πŸ“– Redis drives Timeline, Twitter's most important service. Timeline is an index of tweets indexed by an id. Chaining tweets together in a list produces the Home Timeline. The User Timeline, which consists of tweets the user has tweeted, is just another list.

Database Structure.​

βœ… User Table

DescriptionValue
User IDUnique ID of the User.
FollowersNumber of Followers of the User.
TweetsNumber of Tweets of the User.

βœ… Tweet Table

DescriptionValue
Tweet IDUnique ID of the Tweet.
User IDUser ID of the User.
TextText of the Tweet.
Retweet CountNumber of Retweets of the Tweet.

βœ… Follower Table

DescriptionValue
User IDUser ID of the User.
Follower IDUser ID of the Follower.

Scaling Database.​

API Design and Implementation.​

GET Requests.​

APIDescription
/Get the Home Timeline of the User.
/user/:user_idGet the User Timeline of the User.
/tweet/:tweet_idGet the Tweet.
/user/:user_id/followersGet the Followers of the User.

POST Requests.​

APIDescription
/user/:user_id/followFollow the User.
/user/:user_id/unfollowUnfollow the User.
/tweetPost a Tweet.

DELETE Requests.​

APIDescription
/user/:user_id/unfollowUnfollow the User.
/tweet/:tweet_idDelete the Tweet.

Authentication.​

APIDescription
/loginLogin the User.
/logoutLogout the User.

Error Handling.​

APIDescription
/errorGet the Error.

Caching.​

APIDescription
/cacheGet the Cache.
/cache/clearClear the Cache.
/cache/clear/:keyClear the Cache.
/cache/putPut the Cache.
/cache/put/:keyPut the Cache.

Monitoring.​

APIDescription
/metricsGet the Metrics.
/metrics/clearClear the Metrics.
/metrics/clear/:keyClear the Metrics.
/metrics/putPut the Metrics.
/metrics/put/:keyPut the Metrics.

References​