Concurrency Throttling with asyncio.Semaphore

1Concept

An `asyncio.Semaphore` manages an internal counter to restrict the maximum number of concurrent operations, preventing microservices from overwhelming downstream databases or exceeding API rate limits.

2Architecture Diagram

Semaphore(limit=2):
[ Request 1: Granted ] [ Request 2: Granted ] [ Request 3: Waiting until 1 completes ]

3Code Example

import asyncio

semaphore = asyncio.Semaphore(2) # Max 2 concurrent tasks

async def limited_access_resource(task_id: int):
    async with semaphore:
        print(f"Task {task_id} acquired access lock.")
        await asyncio.sleep(0.02)
        print(f"Task {task_id} released access lock.")

async def main():
    await asyncio.gather(*(limited_access_resource(i) for i in range(1, 4)))

asyncio.run(main())

4Expected Output

Task 1 acquired access lock.
Task 2 acquired access lock.
Task 1 released access lock.
Task 3 acquired access lock.
Task 2 released access lock.
Task 3 released access lock.

5Key Takeaways

• ✓Use Semaphores to throttle external API rate limits (e.g. OpenAI / Stripe rate caps).
• ✓Always acquire semaphores with `async with semaphore:` to ensure release.
• ✓`asyncio.BoundedSemaphore` prevents releasing more times than acquired.