Senior
Design a scalable messenger supporting 150 million users, 75 million DAU, 225 million MAU, 1.2M read / 300k write peak QPS, 5 million concurrent users, 60 PB new data per year, 30% growth annually, SLA 99.95%, p99 <200 ms for read, <300 ms for write. CONTEXT Design a distributed messaging system similar to WhatsApp supporting 1:1 and group chats, message delivery, online status display, and multimedia transfer (photos, videos, audio). The system must ensure high availability, low latency, high concurrency, and global scalability. FUNCTIONAL REQUIREMENTS - Support for personal (1:1) and group chats with add/remove participants - Sending and receiving text and multimedia files No explicit implementation of end-to-end encryption at service or client level, apart from general annotation. - No clear description of sharding and replication of databases by chat_id or user_id for scalability and fault tolerance. - No explicit component or mechanism for offline message synchronization and delivery receipts. - How load balancing between databases and services is implemented, especially during peak loads. **Bottlenecks to consider:**
sobes.tech AI
Answer from AI
When designing a scalable messaging app with such requirements, it is important to consider several key aspects:
-
Data sharding and replication
- Partitioning data by
user_idorchat_idto distribute load across databases. - Replication to ensure fault tolerance and fast read access.
- Partitioning data by
-
Offline synchronization and delivery receipts handling
- Storing messages on the server until delivery is confirmed.
- Push notifications and synchronization mechanisms upon reconnection.
-
Load balancing
- Using load balancers at the API and service levels.
- Horizontal scaling of microservices.
-
Ensuring SLA and low latency
- Caching frequently requested data (e.g., online statuses).
- Using fast storage solutions (in-memory, NoSQL) for messages and statuses.
-
Media files
- Dedicated storage (CDN, Object Storage) for media.
- Asynchronous loading and delivery.
-
Security
- End-to-end encryption ideally implemented on the client side.
- Server responsible for routing and storing encrypted data.
-
Service architecture
- Microservices approach: separate services for messages, users, media, notifications.
- Message queues (Kafka, RabbitMQ) for handling peak loads.
-
Monitoring and alerting
- Performance metrics, latency, error tracking.
- Automatic scaling as load increases.
Ultimately, the architecture should be distributed, with clear responsibility separation, support for horizontal scaling, and high availability. Special attention should be paid to message delivery mechanisms, offline user synchronization, and efficient storage of large data volumes.