Middle
How to limit parallelism when making 10 API requests, where the maximum is 3 simultaneous requests?
sobes.tech AI
Answer from AI
To limit parallelism to 3 concurrent requests when making 10 API requests, you can use a semaphore from the asyncio module in Python. A semaphore allows you to restrict the number of coroutines running simultaneously.
Example using asyncio and a semaphore:
import asyncio
import aiohttp
async def fetch(session, url, semaphore):
async with semaphore: # limit parallelism
async with session.get(url) as response:
return await response.text()
async def main():
urls = ["http://example.com/api/data" for _ in range(10)]
semaphore = asyncio.Semaphore(3) # maximum 3 concurrent requests
async with aiohttp.ClientSession() as session:
tasks = [fetch(session, url, semaphore) for url in urls]
results = await asyncio.gather(*tasks)
print(results)
asyncio.run(main())
This way, no more than 3 requests will be executed simultaneously, and the others will wait for the semaphore to be released.