Python Concurrency: GIL, Threading, Multiprocessing and Asyncio

Introduction

When I started learning concurrency in Python, the first thing that confused me was why there are so many different ways to execute multiple tasks.

We have threading, multiprocessing, and asyncio.

At first, they all seemed to solve the same problem: doing multiple things at once.

But then I realized that they don’t solve exactly the same problem.

The real question is:

What is preventing our program from finishing faster?

Sometimes the CPU is busy doing calculations.

Sometimes the CPU is barely doing anything because the program is waiting for an API, database, file, or network response.

Once I understood this difference, threading, multiprocessing, the GIL, and asyncio started connecting naturally.

What You Will Learn

• Why concurrency is needed
• Concurrency vs parallelism
• I/O-bound vs CPU-bound tasks
• Why threading exists
• What the GIL changes
• Why multiprocessing is needed
• Why asyncio exists even though we already have threading
• Event loop
• Coroutines
• async and await
• Tasks and Futures
• asyncio.gather()
• asyncio.wait()
• asyncio.wait_for()
• Cancellation
• Cooperative concurrency

Where Does the Problem Start?

Imagine I have three API requests.

Each request takes around 5 seconds to return a response.

If I execute them normally:

request_a()
request_b()
request_c()

the first request starts and waits for its response.

Only after it finishes does the second request start.

Then the third.

So roughly:

A → wait 5 sec → finish

              B → wait 5 sec → finish

                                C → wait 5 sec → finish

Enter fullscreen mode Exit fullscreen mode

Total time:

5 + 5 + 5 = 15 seconds

But something feels wrong here.

While A is waiting for the server, our CPU doesn’t need to spend those entire 5 seconds calculating something for A.

A lot of that time is simply waiting.

So why can’t we use that waiting time to make progress on B?

This is where concurrency starts becoming useful.

Concurrency

Concurrency means multiple tasks can make progress during overlapping periods of time.

Instead of:

A → WAIT → finish

         B → WAIT → finish

                      C → WAIT → finish

Enter fullscreen mode Exit fullscreen mode

we want something closer to:

A → start → WAIT ─────────────→ finish
↓
B → start → WAIT ────→ finish
↓
C → start → WAIT → finish

When A cannot make progress because it is waiting, something else can make progress.

This does not necessarily mean A and B are executing CPU instructions at exactly the same moment.

That leads to another concept I initially mixed up with concurrency.

Concurrency vs Parallelism

Concurrency is about handling multiple tasks over overlapping time.

Parallelism means multiple tasks are literally executing at the same time.

Imagine one CPU core.

It can do something like:

A → execute
A → wait

B → execute
B → wait

A → resume

That’s concurrency.

Now imagine multiple CPU cores:

Core 1 → Task A

Core 2 → Task B
Now A and B can actually execute simultaneously.

That’s parallelism.

So I started thinking about it like this:

Concurrency
→ multiple tasks make progress over overlapping time

Parallelism
→ multiple tasks actually execute at the same time

But now another question comes.

How do I know whether my program needs concurrency or parallelism?

For that, I first need to know what kind of work my program is doing.

I/O-Bound vs CPU-Bound

This distinction connects almost everything else.

Suppose I make an API request.

The CPU might spend a tiny amount of time sending the request and processing the response.

But most of the time looks like:

Send request
↓
WAIT
WAIT
WAIT
WAIT
↓
Response

This is an I/O-bound task.

Examples include:

• API requests
• Database queries
• Network operations
• File operations

The important part is:

Most of the time is spent waiting.

Now consider:

total = 0

for i in range(100_000_000):
total += i * i

Here the CPU is constantly calculating.

CALCULATE
CALCULATE
CALCULATE
CALCULATE

There isn’t much waiting time.

This is CPU-bound work.

Examples include heavy calculations, some data processing, image processing, and computation-heavy algorithms.

Now the problem becomes much clearer.

For I/O-bound work, I want to make use of the time where one operation is waiting.

For CPU-bound work, I want more actual CPU execution power.

This is where threading enters.

Why Threading?

Imagine:

API Request A → waiting

API Request B → ready to start

There is no reason for B to wait just because A is waiting for the network.

With threading, we can have multiple threads inside the same process.

Python Process
│
├── Thread A
├── Thread B
└── Thread C

Suppose Thread A makes an API request.

Thread A
↓
send request
↓
WAIT

While A is blocked waiting for I/O, another thread can make progress.

Thread A → WAITING

Thread B → RUNNING

This makes threading useful for many I/O-bound problems.

The threads also share the memory of their process, which makes communication between them relatively straightforward, although shared mutable data can create synchronization problems.

At this point I had another thought.

If threads can execute my work, and my laptop has multiple CPU cores, why don’t I just use threads for CPU-heavy calculations too?

That’s where the GIL becomes important.

The GIL

In standard GIL-enabled CPython, there is something called the Global Interpreter Lock.

Consider:

ONE Python Process
↓
CPython Interpreter
↓
GIL
↓
┌──────┼──────┐
↓ ↓ ↓
T1 T2 T3

Multiple threads can exist.

But for execution of Python bytecode in that interpreter, the threads contend for the GIL.

This means that creating four threads for four heavy pure-Python calculations doesn’t normally mean those four threads will execute Python bytecode simultaneously across four CPU cores.

This was an important correction to my original thinking.

I initially thought:

If the threads are working with different variables, why should Python stop them?

But the GIL isn’t locking only the variables I created.

It is part of CPython’s interpreter/runtime design and historically simplifies and protects important internal object and memory-management operations.

So even if:

Thread A → variable x

Thread B → variable y

both threads still execute within the same interpreter and contend for its GIL.

This explains why threading is often excellent for I/O-bound work but isn’t normally the solution for getting multi-core parallelism from CPU-heavy pure-Python code.

So now there is another problem.

How do we actually use multiple CPU cores for Python calculations?

Why Multiprocessing?

Instead of creating multiple threads inside one process, we can create multiple processes.

Process A

Process B

Process C

Each process normally has its own address space and Python interpreter state.

Conceptually:

Process A
→ Interpreter A
→ GIL A

Process B
→ Interpreter B
→ GIL B

Now the operating system can potentially schedule them on different CPU cores.

Core 1 → Process A

Core 2 → Process B

This gives us actual CPU parallelism when hardware resources are available.

That’s why multiprocessing makes sense for CPU-bound pure-Python work.

But multiprocessing introduces its own trade-off.

Threads Share Memory, Processes Don’t Normally Share It

Suppose:

count = 0

With threads:

ONE PROCESS

Shared count
↑
┌───┴───┐
↓ ↓
T1 T2

Both threads are inside the same process and can access its memory.

With multiprocessing:

Parent Process
count = 0

Process A
count = 0

Process B
count = 0


If A changes its `count` to 1 and B changes its `count` to 1, that doesn't automatically mean:

Parent count = 2


The processes normally have separate address spaces.

So multiprocessing gives us:


CPU parallelism
        ↓
Separate processes
        ↓
More isolation
        ↓
But sharing data becomes more complicated


Now I understood the basic split:


I/O-bound
→ Threading

CPU-bound
→ Multiprocessing


But then I reached another question.

**If threading already handles I/O-bound work, why does Python have asyncio?**

---

# Why Asyncio If Threading Already Exists?

Imagine a server handling a very large number of network connections.

With a thread-per-operation design, we might imagine:

Connection 1 → Thread 1
Connection 2 → Thread 2
Connection 3 → Thread 3
...

This can work, but threads aren't free.

The operating system has to manage them, schedule them, switch between them, and allocate resources for them.

And the interesting part is that many network operations spend most of their time doing this:

WAITING


So I started thinking:

**Do I really need a large number of OS threads just to manage a large number of waiting operations?**

Asyncio gives us another model.

Instead of relying on one thread per async operation, an event loop, commonly running on one thread, can coordinate many asynchronous tasks.

Enter fullscreen mode Exit fullscreen mode

text id=”al7b5p”
ONE THREAD
↓
EVENT LOOP
↓
┌────┼────┐
↓ ↓ ↓
A B C

This is where the **event loop** becomes important.

---

# What Does the Event Loop Actually Do?

Suppose we have:

Task A
Task B
Task C


One event-loop thread can only execute one piece of Python code at an instant.

So something needs to keep track of:

Who can run?

Who is waiting?

Whose I/O has become ready?

Who should resume?


That's the job of the event loop.

Imagine A starts:

Event Loop
    ↓
Task A


A sends an API request and cannot continue until the response arrives.

Instead of blocking the event-loop thread, A can suspend.

Now:

A → WAITING
B → READY
C → READY


The event loop can run B.

Later, when A's awaited operation becomes ready:
A → READY


the event loop can eventually resume A.

So I started thinking of the event loop as a **coordinator**.

It keeps async work moving based on what is ready and what is waiting.

But this creates another problem.

**How can a Python function stop in the middle and continue later?**

That's where coroutines enter.

---

# Coroutines

A normal function roughly behaves like:

START
 ↓
execute
 ↓
execute
 ↓
RETURN


For asyncio, we need something capable of:

START
 ↓
execute
 ↓
SUSPEND
 ↓
something else runs
 ↓
RESUME
 ↓
execute
 ↓
RETURN


That's the important idea behind a coroutine.

In Python:

async def download():
    print("Starting")

    await asyncio.sleep(2)

    print("Finished")

`async def` defines a coroutine function.

When we call it:


download()

we get a coroutine object representing that asynchronous operation.

The coroutine can later be awaited or scheduled.

Now we need something that allows the coroutine to suspend.

That's `await`.

---

# What Does `await` Really Mean?

Consider:


data = await get_data()
At first, I interpreted `await` as:

> Wait here until the result comes.

That is correct from the **coroutine's point of view**, but it misses the most important part.

The coroutine cannot continue past that line until `get_data()` is ready.

But if the awaited operation needs to wait and supports asynchronous suspension, the coroutine can give control back to the event loop.

Task A
  ↓
await get_data()
  ↓
result isn't ready
  ↓
A SUSPENDS
  ↓
Event Loop gets control
  ↓
runs another ready task


Later:

get_data becomes ready
        ↓
Task A becomes ready
        ↓
Event Loop resumes A
        ↓
data = result


This was the point where asyncio started making sense to me.

`await` doesn't mean:

Enter fullscreen mode Exit fullscreen mode

text id=”84k0f5″
Stop the entire program.

It means more like:

> I cannot continue until this awaited operation is ready. If I need to suspend, let the event loop make progress elsewhere.

But there is another important trap.

---

# `async` Doesn't Automatically Mean Concurrent

Suppose:

Enter fullscreen mode Exit fullscreen mode

python id=”8yr9go”
async def main():
await task_a()
await task_b()

If A takes 3 seconds and B takes 2 seconds, B isn't even reached until the first `await` completes.

So:

Enter fullscreen mode Exit fullscreen mode

text id=”s8z1op”
A → 3 seconds → finish
↓
B → 2 seconds → finish

Total is around 5 seconds.

So just writing:

Enter fullscreen mode Exit fullscreen mode

python id=”5s46wq”
async def

doesn't automatically make everything concurrent.

Sometimes we want to tell the event loop:

> Schedule this coroutine so it can make progress independently while I do other async work.

This leads to **Tasks**.

---

# Tasks

Suppose:

Enter fullscreen mode Exit fullscreen mode

python id=”pivjwp”
task_a()

creates a coroutine object.

Now:

Enter fullscreen mode Exit fullscreen mode

python id=”e3cjbs”
asyncio.create_task(task_a())

schedules that coroutine with the running event loop as a Task.

Think:

Enter fullscreen mode Exit fullscreen mode

text id=”mt0pf2″
Coroutine
↓
create_task()
↓
Task
↓
scheduled with Event Loop

Now we can do:

Enter fullscreen mode Exit fullscreen mode

python id=”52ks84″
a = asyncio.create_task(task_a())
b = asyncio.create_task(task_b())

await a
await b

Both A and B are scheduled before we wait for their completion.

So if A suspends:

Enter fullscreen mode Exit fullscreen mode

text id=”w55qv8″
A → await → suspend

B can make progress.

This gives us concurrent async execution.

---

# Then What Is a Future?

After understanding Tasks, Futures sounded complicated, but the underlying idea is simple.

A Future represents:

**A result that may not exist yet but should become available later.**

Imagine:

Enter fullscreen mode Exit fullscreen mode

text id=”v1u6m7″
API request
↓
Future
↓
PENDING
↓
response arrives
↓
DONE
↓
result available

If I write:

Enter fullscreen mode Exit fullscreen mode

python id=”q53g1v”
result = await future

and the Future isn't complete, my coroutine can suspend.

When the Future becomes complete, the coroutine can resume with its result or receive its exception.

A useful relationship is:

Enter fullscreen mode Exit fullscreen mode

text id=”rz0pvz”
Future
→ represents an eventual result

Task
→ schedules/drives a coroutine
and also represents its eventual result

In asyncio, a Task is a specialized kind of Future.

---

# What If I Have Many Async Operations?

Suppose I have:

Enter fullscreen mode Exit fullscreen mode

text id=”rg2yio”
A
B
C

and I want all of them to run concurrently and then collect their results.

I could manually create and await Tasks.

But asyncio gives us:

Enter fullscreen mode Exit fullscreen mode

python id=”fdvev2″
results = await asyncio.gather(
task_a(),
task_b(),
task_c()
)

Conceptually:

Enter fullscreen mode Exit fullscreen mode

text id=”shludv”
A ──────────────┐
B ──────────┐ │
C ──────┐ │ │
↓ ↓ ↓
all complete
↓
results

So I think of `gather()` as:

**Run/await these async operations concurrently and collect their results together.**

But sometimes I don't want to simply wait for everything.

Maybe I want more control.

---

# `asyncio.wait()`

Imagine:

Enter fullscreen mode Exit fullscreen mode

text id=”1rbsr4″
A → 10 sec
B → 2 sec
C → 6 sec

Maybe my requirement is:

> As soon as one finishes, tell me what is finished and what is still pending.

That's where `wait()` is useful.

Enter fullscreen mode Exit fullscreen mode

python id=”l82zyr”
done, pending = await asyncio.wait(
tasks,
return_when=asyncio.FIRST_COMPLETED
)

After around 2 seconds:

Enter fullscreen mode Exit fullscreen mode

text id=”f0qq5y”
DONE
→ B

PENDING
→ A
→ C

So the difference becomes clearer:

Enter fullscreen mode Exit fullscreen mode

text id=”nv8vgo”
gather()
→ collect results from async operations

wait()
→ give me control over done/pending tasks

But what if my problem isn't multiple tasks?

What if one operation is simply taking too long?

---

# `asyncio.wait_for()`

Suppose:

Enter fullscreen mode Exit fullscreen mode

python id=”wuy3st”
await get_data()

normally takes a few seconds.

But if the server has a problem, I don't want my program waiting indefinitely.

I can say:

Enter fullscreen mode Exit fullscreen mode

python id=”3uswb8″
await asyncio.wait_for(
get_data(),
timeout=5
)

Now I'm giving the operation a maximum waiting time.

If it doesn't finish within that timeout, `wait_for()` normally cancels the awaited operation and raises a timeout exception.

So:

Enter fullscreen mode Exit fullscreen mode

text id=”4hq10d”
wait()
→ completion control over tasks

wait_for()
→ timeout around an awaitable

This naturally brings us to cancellation.

---

# Cancellation

Suppose a download is running:

Enter fullscreen mode Exit fullscreen mode

text id=”wm16rk”
Downloading…
Downloading…
Downloading…

and the user clicks Cancel.

We can request cancellation of a Task:

Enter fullscreen mode Exit fullscreen mode

python id=”0vqtaj”
task.cancel()

But asyncio cancellation is **cooperative**.

It isn't best understood as Python violently killing arbitrary code at a random CPU instruction.

Cancellation is delivered through asyncio's task machinery, typically by raising `CancelledError` in the task at an appropriate point.

That allows the coroutine to clean up.

For example:

Enter fullscreen mode Exit fullscreen mode

python id=”5mr1rk”
async def download():

try:
    await get_file()

except asyncio.CancelledError:
    print("Cleaning up...")
    raise

Enter fullscreen mode Exit fullscreen mode

This might allow us to close a connection, remove a temporary file, or release some resource before stopping.

And this finally connects to one of the most important ideas behind asyncio.

---

# Cooperative Concurrency

Why is this called **cooperative concurrency**?

Because async tasks need to cooperate with the scheduler.

Imagine:

Enter fullscreen mode Exit fullscreen mode

text id=”78r1ks”
Task A
↓
runs
↓
await
↓
gives control back
↓
Event Loop
↓
Task B

A reaches a point where it cannot make progress and allows the event loop to run other work.

Then B does the same.

Enter fullscreen mode Exit fullscreen mode

text id=”nn3etb”
A → run → await ───────────→ resume
↓
B → run → await ─────→ resume

But consider:

Enter fullscreen mode Exit fullscreen mode

python id=”3ib4ap”
async def calculate():

while True:
    do_heavy_calculation()

Enter fullscreen mode Exit fullscreen mode

There is no useful asynchronous suspension point.

Even though the function says:

Enter fullscreen mode Exit fullscreen mode

python id=”dl81dq”
async def

the CPU-heavy code can occupy the event-loop thread and prevent other async tasks from getting a chance to run.

This is why `asyncio` isn't a replacement for multiprocessing.

They solve different problems.

---

# Connecting Everything

This is the flow that finally made the entire topic clear to me.

Start with the program:

Enter fullscreen mode Exit fullscreen mode

text id=”gwhicf”
My program is slow
↓
Why?

There are two major possibilities:

Enter fullscreen mode Exit fullscreen mode

text id=”qx4c7o”
PROGRAM
↓
Where is time spent?
↓
┌────────┴────────┐
↓ ↓
WAITING CALCULATING
↓ ↓
I/O-BOUND CPU-BOUND

If the program is mostly waiting:

Enter fullscreen mode Exit fullscreen mode

text id=”0e3rd0″
I/O-bound
↓
Need concurrency
↓
Threading OR Asyncio

Threading:

Enter fullscreen mode Exit fullscreen mode

text id=”r1hj7k”
Multiple OS threads
inside a process

Asyncio:

Enter fullscreen mode Exit fullscreen mode

text id=”28ktn5″
Event loop
↓
coordinates many
async operations
↓
coroutines suspend
using await

If the program is mostly calculating:

Enter fullscreen mode Exit fullscreen mode

text id=”74sdq7″
CPU-bound
↓
Threads in standard
GIL-enabled CPython
don’t normally give us
multi-core Python-bytecode
parallelism
↓
Multiprocessing
↓
Separate processes
↓
Multiple CPU cores

And inside asyncio itself:

async def
   ↓
Coroutine
   ↓
scheduled as Task
   ↓
Event Loop
   ↓
Run
   ↓
await
   ↓
Suspend
   ↓
Run other ready work
   ↓
Resume

Enter fullscreen mode Exit fullscreen mode

Once I stopped learning threading, multiprocessing, GIL, asyncio, event loop, coroutines, and await as isolated definitions and instead followed the problem each concept solves, the entire topic became much easier to understand.

The main lesson I took from Python concurrency is:

Don’t first ask, “Should I use threading, multiprocessing, or asyncio?”

First ask:

“What is my program doing most of the time — waiting or calculating?”

Once that is clear, choosing the right concurrency model becomes much easier.