Sobes.tech
Junior — Middle

Is it possible to implement separate read and write operations using different shards?

sobes.tech AI

Answer from AI

Yes, in Go, it is possible to implement separate read and write operations using different shards (parts) to improve performance and reduce locking.

The idea is to split the overall data structure into multiple shards — independent substructures, each with its own lock (e.g., mutex). Read and write operations are directed to the appropriate shard based on the hash of the key.

Advantages:

  • Parallel operations on different keys can be performed without mutual blocking.
  • Reduces contention for a single global lock.

Example of a simplified implementation of a sharded map:

import (
    "hash/fnv"
    "sync"
)

type shard struct {
    sync.RWMutex
    m map[string]interface{}
}

type ShardedMap struct {
    shards []shard
}

func NewShardedMap(numShards int) *ShardedMap {
    shards := make([]shard, numShards)
    for i := range shards {
        shards[i].m = make(map[string]interface{})
    }
    return &ShardedMap{shards: shards}
}

func (sm *ShardedMap) getShard(key string) *shard {
    h := fnv.New32a()
    h.Write([]byte(key))
    return &sm.shards[uint(h.Sum32())%uint(len(sm.shards))]
}

func (sm *ShardedMap) Set(key string, value interface{}) {
    s := sm.getShard(key)
    s.Lock()
    defer s.Unlock()
    s.m[key] = value
}

func (sm *ShardedMap) Get(key string) (interface{}, bool) {
    s := sm.getShard(key)
    s.RLock()
    defer s.RUnlock()
    val, ok := s.m[key]
    return val, ok
}

Thus, write and read operations are distributed across different shard mutexes, increasing scalability.