Leverage concurrent HTTP Get requests with asyncio

[tpiekarski]

Overview

Sending HTTP Request should happen concurrently with multiple threads using asyncio to save time when dispatching requests for hundreds or thousands of URLs. The distribution and amount of threads should be decided by the runtime and not calculated manually ahead.

Use asyncio when you can, threading when you must.

Links

• Speed Up Your Python Program With Concurrency
• Hands-on Python 3 Concurrency With the asyncio Module
• Python Threading Tutorial: Run Code Concurrently Using the Threading Module

[tpiekarski]

Migration from requests to aiohttp finished. But using asyncio.gather(...) is not working like expected.
Requests do not get sent concurrently, instead they're sent sequentially.
Something is not right... 😞

[tpiekarski]

Additional resources

• Understanding Python's asyncio