Sobes.tech
Junior — Middle

How is data distributed across shards in a database?

sobes.tech AI

Answer from AI

Data distribution across shards (sharding) in databases is usually carried out using the following methods:

  • Horizontal sharding by key (Hash Sharding): data is distributed across shards based on a hash function of the key (e.g., user_id). This ensures an even distribution of load.
  • Range sharding (Range Sharding): data is divided into ranges of key values (e.g., by date or ID), which is convenient for range queries.
  • Geographical sharding: data is distributed by regions or locations.
  • Sharding by data type or functionality: different data types are stored on different shards.

Example of hash sharding in Python:

import hashlib

def get_shard(key, num_shards):
    hash_digest = hashlib.md5(str(key).encode()).hexdigest()
    hash_int = int(hash_digest, 16)
    return hash_int % num_shards

# Usage
shard_id = get_shard(user_id, 4)  # 4 shards

This approach allows determining which shard will contain data with a given key.