How to Detect and Fix a Python Memory Leak

Introduction

Python's automatic garbage collection can give you a false sense of safety. Most of the time, it works so well you forget it exists, but then you ship a long-running scraper, a background worker, or an API service that stays up for days, and memory usage grows a little every hour until the process gets slow, gets OOM killed, or simply falls over.

That creeping growth is usually a Python memory leak: objects that are no longer useful but are still being held in memory. Something still has a reference to it, so Python's garbage collector can't free memory, and allocated memory keeps climbing.

In this guide, you'll learn:

• Why leaks still happen in a garbage-collected language like Python.
• What tends to cause Python memory leaks.
• A practical workflow on how to detect a memory leak in Python.
• The Python memory leak detection tools that are worth reaching for.
• How to fix memory leaks without guesswork.
• Why web scraping workloads are susceptible to memory leaks.

Why Python isn't immune to memory leaks

Before you can fix memory leaks, it helps to understand how Python's memory management actually works. The leak is usually a reference problem rather than a pure allocation problem.

CPython primarily uses reference counting, so every object tracks how many strong references point to it. When that reference count drops to zero, the object can be freed immediately. On top of that, Python's garbage collector runs a cyclic GC to clean up reference cycles, where objects reference each other in a loop and can never reach a reference count of zero on their own.

That sounds airtight, but there are a few reasons Python memory leaks still happen:

• Reference cycles can become uncollectable in some edge cases, especially when finalizers are involved. If objects in a cycle have custom cleanup behavior, Python may refuse to collect them automatically because the order of finalization is ambiguous.
• C extension code can allocate memory outside Python's direct control. Python might drop references correctly, yet native memory allocation sticks around, or returns to an allocator pool rather than to the OS, making memory growth harder to interpret.
• Long-running processes amplify small mistakes. A single leaked object per iteration can be invisible in a short script and catastrophic in a daemon that runs 24/7.
• Caches and registries are often intentional… until they are not. A dict that was meant to store a few keys can quietly store millions if you never evict.

With that foundation in place, how do you identify these patterns creating leaks in real projects?

Common causes of Python memory leaks

Now that you know why garbage collection is not a force field, you can spot the typical ways code ends up holding references. The fastest route to a fix is matching your symptom to a known pattern, then verifying it with tooling in the next section.

Circular references

Circular references are two or more objects referencing each other, so their reference counts never reach zero. Python's cyclic garbage collector usually handles this, but cycles can become a problem when finalizers are involved, or when you accidentally keep one object reachable from a long-lived root.

Here's a minimal example that creates a reference cycle:

class A:
    def __init__(self):
        self.b = None


class B:
    def __init__(self):
        self.a = None


a = A()
b = B()
a.b = b
b.a = a  # reference cycle

On its own, that cycle is typically garbage-collected once nothing else references a or b. The trouble starts when you add custom cleanup, or when a global collection still points at one side of the cycle. At that point, reference cycles can accumulate and memory usage grows over time.

Unbounded caches and global collections

Unbounded caches are one of the most common causes of memory leaking in production. You store data in a module-level dict, a list, or a set because it's convenient, but there's no eviction strategy. Every request, job, or scrape iteration appends another entry, and nothing ever frees memory.

# cache.py
CACHE = {}


def process(key, value):
    # Looks harmless until key cardinality explodes
    CACHE[key] = value

The problem is not caching. The problem is caching without a bound, without a TTL, and without a realistic sense of how many unique keys you will see in a long-running process.

Unclosed resources

If there's one leak category that hits scrapers and backend workers hardest, it's unclosed resources. File handles, sockets, database connections, HTTP sessions, browser sessions, and page objects all come with buffers and native allocations that do not magically disappear at the end of a function.

def process(path):
    f = open(path, "rb")
    data = f.read()
    # forgot f.close()
    return data

When scraping code, the same issue shows up with headless browser lifecycles. If you create a browser, context, or page per iteration and don't close it, you are effectively asking the heap to keep everything alive.

Closures holding references

Closures are subtle because the code looks clean. An inner function captures variables from an enclosing scope, which can unintentionally keep large objects alive long after you thought they were gone.

def make_handler(large_objects):
    def handler(event):
        # large_objects is captured and kept alive
        return len(large_objects) + event["value"]

    return handler

If large_objects is a big list, a parsed DOM, a response body, or any other large object, that closure can keep it reachable for the lifetime of the handler.

Event listeners and callbacks

Anything that registers callbacks on long-lived objects can leak if you never unregister. The classic examples are event buses, signal handlers, pub/sub registries, and frameworks that keep global callback lists.

class EventBus:
    def __init__(self):
        self._listeners = []

    def on(self, fn):
        self._listeners.append(fn)

    def emit(self, event):
        for fn in self._listeners:
            fn(event)

If a listener is a bound method, you're also keeping self alive via that bound method reference. Over time, you end up with a listener list full of objects you thought were dead.

These causes map nicely to detection strategies. Each one leaves a different fingerprint in memory allocation, object counts, and reference graphs.

How to detect a memory leak in Python

Once you suspect your process is leaking memory, the first step is to confirm that it's real growth rather than a temporary spike, then narrow it down to a specific object type, file, or line of code. Memory usage is noisy, and modern allocators can make free memory look like it never returns.

A memory spike is normal. You load a big response, build an in-memory structure, then drop it, and memory settles back down. A genuine leak looks different: memory usage grows steadily across iterations or requests, even after work is complete.

A practical way to think about it:

Pattern	What you'll see	What it usually means
Spike	Memory jumps up, but then drops after the work finishes and garbage collection runs	A normal burst of memory allocation that gets cleaned up
Leak	Memory climbs in a sawtooth pattern, where each peak is higher than the last, until the process gets OOM killed or you restart it	Objects are still being held in memory, so memory usage grows over time

Start with a few low-friction checks before you pull out heavier profiling tools.

• Watch memory over time, not at a single point – Use whatever monitoring you already have: container metrics, systemd, Kubernetes dashboards, or a basic RSS graph. What matters is whether memory usage grows across many iterations.
• Use sys.getsizeof() for quick spot checks – This tactic won't show the full size of container contents, but it can be useful for sanity checks when you suspect a specific object is getting bigger.
• Force a collection to rule out collectible garbage – If you call gc.collect() after an iteration and memory still climbs, you are more likely dealing with objects that remain reachable, or native allocations outside Python's GC.

import gc


def process():
    # your workload
    ...


for i in range(10_000):
    process()
    if i % 100 == 0:
        gc.collect()

• Confirm growth with a light tracemalloc loop – Before you reach for a full profiler, confirm that allocated memory is trending upward inside your program, not just in external charts.

import tracemalloc
import time


tracemalloc.start()


def process():
    # Replace with your real work
    data = [b"x" * 1024 for _ in range(1000)]
    return data  # simulate a bug where caller holds references


held = []


for i in range(2000):
    held.append(process())  # leaking memory on purpose
    if i % 200 == 0:
        current, peak = tracemalloc.get_traced_memory()
        print(f"iter={i} current={current/1024/1024:.2f}MB peak={peak/1024/1024:.2f}MB")
        time.sleep(0.01)

If current keeps rising, you have confirmed the direction of travel. From here, the next step is choosing the right tool to identify what is being held and where it was allocated.

Python memory leak detection tools

Now that you've confirmed memory growth is the issue, you can move from suspicion to proof. The trick is matching the tool to the shape of the leak, whether that's line-level growth, object-count growth, reference cycles, or native allocations.

Below are the Python memory leak detection tools that tend to cover the vast majority of real-world cases, with minimal working examples you can drop into a project.

tracemalloc

tracemalloc is the best starting point for most developers because it is built into the standard library and gives you file-and-line attribution for Python memory allocation. It won't catch every native allocation, but it is excellent at finding which parts of your code are allocating growing memory.

Take two snapshots and compare them to highlight the delta between two points in time – an approach that's especially useful when memory usage grows slowly.

import tracemalloc


tracemalloc.start()


def process():
    # your workload
    data = [str(i) for i in range(50_000)]
    return data


snapshot1 = tracemalloc.take_snapshot()


# Simulate repeated work
leak = []
for _ in range(20):
    leak.append(process())


snapshot2 = tracemalloc.take_snapshot()


top_stats = snapshot2.compare_to(snapshot1, "lineno")


print("Top differences:")
for stat in top_stats[:10]:
    print(stat)

What to look for:

• A small number of lines being responsible for most allocated memory growth.
• Allocations that grow with each iteration rather than stabilizing.
• Code paths that create large objects, then store data somewhere long-lived.

If tracemalloc points to a broad function, the next tool helps you pinpoint the exact line.

memory_profiler

memory_profiler provides line-by-line memory usage inside a function. It's great when you know roughly where the leak is, but want to see which operation causes a jump.

Install:

pip install memory_profiler

Add the decorator and run the script through the profiler:

from memory_profiler import profile


@profile
def process():
    data = []
    for i in range(100_000):
        data.append("x" * 100)  # example allocation
    return data


if __name__ == "__main__":
    process()

Run:

python -m memory_profiler your_script.py

What it tells you:

• Which line triggers a big memory allocation.
• Whether memory drops after leaving a scope or stays elevated.
• Whether a loop allocates a little each iteration and never releases.

Once you know which line causes the growth, you can usually find the reference pattern. If you suspect a cycle or a listener registry, you need visibility into object graphs.

objgraph

When the problem is not a single heavy line but a reference chain that never breaks, objgraph is a solid choice. It helps you inspect object counts and visualize what is holding references.

Install:

pip install objgraph

Start by checking which types are growing:

import objgraph
import gc


gc.collect()
objgraph.show_most_common_types(limit=10)

If you see a type that keeps increasing, the next question is, why is it still reachable? show_backrefs() can generate a graph of references leading back to a root.

import objgraph


# Suppose you suspect "MyThing" instances are leaking
leaking = objgraph.by_type("MyThing")
objgraph.show_backrefs(leaking[:1], max_depth=5)

This is especially useful for the identification of:

• Circular references that keep an object alive.
• Global registries holding references.
• Event listeners keeping bound methods alive.

If your leak involves C extensions or native memory, tracemalloc and objgraph can miss the real culprit. That's where a lower-level tracer helps.

memray

Memray is a modern memory profiler that can trace Python allocations and native allocations, making it useful when C libraries are involved, or when memory allocation happens outside the pure Python heap.

Install:

pip install memray

A common workflow is to run your script under Memray and produce a report:

memray run -o memray.bin your_script.py
memray flamegraph memray.bin

If you're chasing a Python memory leak that only appears under production-like load, memray's overhead and reporting can be a better fit than always-on line profiling.

gc module

Sometimes, the most direct path is to ask Python's gc to tell you what it cannot collect. This is particularly relevant when reference cycles involve finalizers.

import gc


gc.set_debug(gc.DEBUG_UNCOLLECTABLE)


# Run your workload, then:
unreachable = gc.collect()
print("unreachable:", unreachable)
print("garbage:", len(gc.garbage))

If gc.garbage grows, you have objects that are not being garbage collected. That is a strong signal that your leak involves uncollectable cycles, often related to cleanup code or finalization behavior.

At this point, you should know where memory's being allocated and what's being held. The next section is about turning that information into fixes that actually free memory.

How to fix the most common Python memory leaks

Once you've identified what's holding references, fixing a Python memory leak becomes less mysterious. The pattern is usually the same: break the strong reference chain, bound the data structure, or guarantee cleanup. After each change, re-run the same profiling tools to confirm memory growth is gone.

Break circular references with weak references

If you truly need objects to reference each other, use weak references so the relationship does not keep both objects alive forever.

import weakref


class Parent:
    def __init__(self):
        self.child = None


class Child:
    def __init__(self, parent):
        self.parent = weakref.ref(parent)  # weak reference


p = Parent()
c = Child(p)
p.child = c

This is especially helpful when you have graphs of objects, caches of instances, or callback-heavy designs where reference cycles are easy to create.

If you suspect __del__ is part of the issue, prefer context managers and explicit cleanup methods over finalizers. Finalizers are easy to get wrong and can interact badly with reference cycles.

Put limits on caches and global collections

If you are caching, make it bounded. If you are storing data globally, decide how it leaves.

A simple but effective option is functools.lru_cache with a maxsize:

from functools import lru_cache


@lru_cache(maxsize=1024)
def expensive_lookup(key):
    return key * 2

For TTL-based caches, a common approach is to use a dedicated cache library or a background job that prunes keys. The point is not the implementation, it's the policy – a cache without eviction is a memory leak disguised as a feature.

Also, watch out for Python debug accumulators:

DEBUG_ROWS = []


def process(row):
    DEBUG_ROWS.append(row)  # grows forever unless you cap it

If you need logging, log. If you need sampling, sample. Do not accidentally store data forever because it was convenient during development.

Always close resources with context managers

If a resource has a close() method, treat it as mandatory. In long-running processes, unclosed resources often show up as memory issues first, then performance issues, then hard failures.

For files:

def process(path):
    with open(path, "rb") as f:
        return f.read()

For HTTP sessions:

import requests


def process(urls):
    with requests.Session() as session:
        return [session.get(u).text for u in urls]

Database connections should follow the same pattern. The exact API differs, but the idea is constant: guarantee cleanup even when an exception happens.

Don't let closures keep large objects alive

If a closure has to exist, avoid capturing large objects. Pass the minimal value you need, or restructure so the closure does not outlive the data.

Here's a bad pattern:

def build_processor(large_objects):
    def process(event):
        return large_objects[event["id"]]

    return process

Here's a better pattern:

def build_processor(index_lookup):
    def process(event):
        return index_lookup(event["id"])

    return process

If you have to break a reference after use, setting a variable to None can help in long loops where the variable would otherwise keep pointing at a large object until the next iteration:

for item in items:
    big = compute_big_object(item)
    handle(big)
    big = None  # drop reference early

This is not a magic fix, but it can reduce peak memory and help the garbage collector free memory sooner.

Remove event listeners and callbacks explicitly

If you register a handler, make sure you can unregister it.

class EventBus:
    def __init__(self):
        self._listeners = []

    def on(self, fn):
        self._listeners.append(fn)
        return fn

    def off(self, fn):
        self._listeners = [x for x in self._listeners if x is not fn]

If you're dealing with bound methods, consider weak references for listeners so the listener does not keep self alive. Python provides tools like weakref.WeakMethod for this exact reason.

Keep data local, not global

This sounds basic, but it's a root cause of many leaks. Local variables go out of scope; global variables do not. If you store data in a module-level structure, you're making it live for the lifetime of the process.

Prefer returning values, yielding results, or writing to external storage. If you truly need a global registry, make it bounded and actively pruned.

And whatever fix you apply, treat verification as part of the change:

• Take two snapshots again.
• Run the memory profiler again.
• Confirm that allocated memory stabilizes and that memory usage no longer grows.

With those mechanics nailed down, you can zoom in on where leaks are especially painful: scraping.

Python memory leaks in web scraping

If the previous section was about general fixes, web scraping is where those fixes get stress-tested.

Scrapers are long-lived, repetitive, and stateful. They open network connections, hold large responses, create page objects, and run callback-heavy automation code. In other words, they're the perfect machines for turning small reference mistakes into expensive memory growth.

The most common scraping-related leak patterns look like this:

• Not closing browser sessions, contexts, or pages – If you launch a browser in a loop, or create pages and never close them, you will leak native memory, GPU buffers, and internal browser state.

Here's a bad pattern:

from playwright.sync_api import sync_playwright


def process(urls):
    with sync_playwright() as p:
        for url in urls:
            browser = p.chromium.launch(headless=True)
            page = browser.new_page()
            page.goto(url)
            page.content()
            # forgot page.close() and browser.close()

A better pattern is to reuse the browser, scope pages tightly, and close them deterministically:

from playwright.sync_api import sync_playwright


def process(urls):
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        try:
            for url in urls:
                page = browser.new_page()
                try:
                    page.goto(url, wait_until="domcontentloaded")
                    _ = page.content()
                finally:
                    page.close()
        finally:
            browser.close()

• Creating new browser instances instead of reusing contexts – A single browser with multiple isolated contexts is often the right balance for memory usage and isolation. Launching a fresh browser process for every iteration is expensive and easy to mismanage.
• Holding references to page objects after navigation – It's easy to accidentally store a page, a response, or a parsed DOM in a global list for debugging, metrics, or retries. Those objects can keep entire trees of data alive, including large objects you thought were temporary.
• Callback-heavy automation that captures state – Scraping frameworks and browser automation libraries encourage event-driven patterns. If a callback closes over a page, a browser, or a large buffer, it can keep that object alive for much longer than expected.

Operational reality matters here.

Managing your own headless Chrome means lifecycle management is your problem: how many instances exist, when they close, what happens on exceptions, and how you isolate sessions.

Even if your Python code is clean, a single missing close() in a rare code path can cause memory issues that only appear under load.

Offloading browser sessions to a managed service like Browserless changes the failure mode. Instead of your Python process owning the browser lifecycle, you hand sessions off to infrastructure designed to create, isolate, and clean up those sessions automatically. It removes one of the most common sources of leaking memory in scraping systems: orphaned browsers, contexts, and pages that never get torn down.

There's also a security benefit. Leaks do not just waste memory – if an object containing sensitive data stays reachable, it can hang around longer than intended. Tight session lifecycles and enforced cleanup reduce that risk, as well as the performance improvements.

With scraping, the most important habit is to treat every iteration like it might be the one that throws an exception. If cleanup only happens on the happy path, memory growth is inevitable.

Conclusion

Python memory leaks are rarely dramatic at first. They look like a little extra memory usage, a small increase in allocated memory after each iteration, or a service that gradually gets slower. But in long-running processes, that small leak becomes a real outage. The heap grows, performance drops, and eventually the process gets OOM killed.

Thankfully, there's a straightforward, reliable workflow to prevent this:

• Confirm memory growth rather than guessing – Use a simple loop and tracemalloc.get_traced_memory() to prove memory usage grows.
• Pinpoint the source – Compare two snapshots with tracemalloc, then use memory_profiler for line-level clarity, or objgraph when you suspect reference cycles and holding references.
• Fix the reference pattern – Break cycles with weak references, bound caches, close resources with context managers, unregister callbacks, and keep large objects in local variables rather than globals.
• Verify the fix – Re-run the same profiling tools and make sure the growth curve flattens.

If your main pain point is web scraping, Browserless can remove one of the biggest leak sources entirely: browser lifecycle management. Instead of debugging why a stray page object survived an exception path, you can focus on scraping logic while sessions are created and cleaned up for you.

If you want to stop browser sessions from causing Python memory leaks, start a free Browserless trial.

Python memory leak FAQs

Can Python have memory leaks if it has a garbage collector?

Yes. CPython uses reference counting plus a cyclic garbage collector, but memory still leaks whenever an object stays reachable. Unbounded caches, module-level collections, lingering closures, and unclosed resources all keep references alive. Reference cycles can also become uncollectable when finalizers are involved, and C extensions can allocate memory outside Python's control.

What is the difference between a Python memory leak and a memory spike?

A spike is normal: memory jumps while you build a large in-memory structure, then drops once the work finishes and garbage collection runs. A leak climbs in a sawtooth where each peak is higher than the last, until the process is OOM killed or you restart it. The signal is memory that keeps growing across iterations even after the work is done.

How do you detect a memory leak in Python?

Watch memory over time rather than at a single point, then confirm the trend inside your program with a short tracemalloc loop that prints tracemalloc.get_traced_memory() every few hundred iterations. If the current value keeps rising after you call gc.collect(), you have a real leak rather than collectible garbage.

What are the best tools to find a Python memory leak?

tracemalloc is the best starting point because it ships with the standard library and gives file-and-line attribution. Reach for memory_profiler when you want line-by-line usage inside a function, objgraph when you suspect reference cycles or listener registries, memray when C extensions or native allocations are involved, and the gc module when you need to surface uncollectable cycles.

How do you fix a Python memory leak?

Break the reference chain, bound the data structure, or guarantee cleanup. Use weak references for object graphs, add a maxsize or TTL to caches, close resources with context managers, unregister event listeners, and keep large objects in local variables instead of globals. After each change, re-run the same profiling tools to confirm the growth curve flattens.

Why do web scrapers leak memory in Python?

Scrapers are long-lived and stateful. They hold network connections and large responses, and they create browser, context, and page objects across thousands of iterations. If you launch a browser in a loop or never close pages, you leak native memory and internal browser state, and a single missing close() on an error path can cause growth that only shows up under load. Reusing one browser, scoping pages tightly, and closing them in a finally block all help. Offloading session lifecycle to a managed service like Browserless caps the damage too, since dead sessions get reaped at their timeout even if a close() is missed, though you should still close sessions explicitly.