Sobes.tech
Junior — Senior

Limiting the number of concurrently running goroutines during mass request sending

livecode

Task condition

It is required to write a mechanism that will control the number of concurrently executing goroutines when sending a large batch of requests (for example, 10,000). All requests must be completed, but at any given time, there should be no more than N active goroutines. Error handling in this case is not required.

package main

import (
  "context"
  "fmt"
  "strconv"
  "sync"

  "github.com/sirupsen/logrus"
)

type Request struct {
  Payload string
}

type Client interface {
  SendRequest(ctx context.Context, request Request) error
}

type client struct{}

func (c client) SendRequest(ctx context.Context, request Request) error {
  fmt.Println("send request", request.Payload)
  return nil
}

func main() {
  ctx := context.Background()
  c := client{}
  requests := make([]Request, 100)
  for i := 0; i < 100; i++ {
    requests[i] = Request{Payload: strconv.Itoa(i)}
  }

  log := logrus.New()
  makeBatchApiCalls(ctx, c, log, requests)
}

func makeBatchApiCalls(ctx context.Context, c Client, log *logrus.Logger, requests []Request) {
  wg := sync.WaitGroup{}
  for _, r := range requests {
    r := r
    wg.Add(1)
    go func() {
      defer wg.Done()
      err := c.SendRequest(ctx, r)
      if err != nil {
        log.WithError(err).Error("send request")
      }
    }()
  }
  wg.Wait()
}

Your task is to add logic to the makeBatchApiCalls function (or create a helper function) that will limit the number of concurrently launched goroutines to a given value N, while ensuring that each request from the input array is processed.