Query Optimization on Johnny Metz

Django Time-Based Lookups: A Performance Trap

Mon, 19 Jan 2026 00:00:00 -0700

Django’s field lookups are one of the ORM’s best features, but time-based lookups can quietly bypass database indexes, turning fast queries into expensive full table scans.

A Slow Production Query

I ran into this while debugging a 30 second query on a large table (~25 million rows):

class Event(models.Model):
    timestamp = models.DateTimeField(db_index=True)

Event.objects.filter(timestamp__date=datetime.date(2026, 1, 5)).count()

It generates SQL like this:

SELECT COUNT(*) FROM event
WHERE timestamp::date='2026-01-05';

At first glance, it looked totally reasonable — a simple filter on an indexed field. But after checking the query plan, I discovered the issue: the query can’t use the index and falls back to a full table scan because it casts the field to a date.

Avoiding Duplicate Objects in Django Querysets

Sat, 03 Jan 2026 00:00:00 -0700

When filtering Django querysets across relationships, you can easily end up with duplicate objects in your results. This is a common gotcha that happens with both one-to-many (1:N) and many-to-many (N:N) relationships. Let’s explore why this happens and the best way to avoid it.

The Problem

When you filter a queryset by traversing a relationship, Django performs a SQL JOIN. If a parent object has multiple related objects that match your filter, the parent object appears multiple times in the result set.

Speed Up Django Queries with values() over only()

Sun, 29 Jun 2025 13:11:26 -0700

If your Django queries feel slow, the problem might not be your database — it might be your ORM. Recently, I was working with a query that took 25 seconds to run through the Django ORM, but the underlying SQL completed in just 2 seconds. With a single change, I got the ORM query down to 2 seconds as well — and reduced the memory footprint by 70%.

Let’s walk through what happened, and why using .values() instead of .only() can dramatically improve performance.

Avoiding PostgreSQL Pitfalls: The Hidden Cost of Failing Inserts

Sun, 15 Jun 2025 18:00:00 -0700

A simple insert query turned into a silent performance killer.

Our frontend pings our server every few minutes to track device activity. Each ping attempts to insert a row into a DevicePingDaily table, which has a unique constraint on (device_id, date) to ensure only one record per device per day.

In Django, the logic looked like this:

try:
    DevicePingDaily.objects.create(device=device, date=today)
except IntegrityError:
    pass

It seemed harmless. But as traffic grew, latency spiked and API timeouts increased. Observability tools quickly pointed to the culprit:

5 Ways to Get the Latest Book Per Author in Django

Thu, 14 Nov 2024 12:00:00 -0700

In a Django application, fetching the latest record for each group is a common yet challenging task, especially when working with large datasets. Whether you’re building an analytics dashboard or managing grouped data, finding an efficient solution is key. In this blog post, we’ll explore five different approaches to tackle this problem, ranked from least to most effective based on performance and readability, using the following Book model:

class Book(models.Model):
    title = models.CharField(max_length=255)
    author = models.ForeignKey(
        Author, on_delete=models.CASCADE, related_name="books"
    )
    genre = models.CharField(max_length=255)
    published_at = models.DateTimeField()
    is_featured = models.BooleanField(default=False)

Solution 1: Python Max with Prefetch

latest_books = [
    max(author.books.all(), key=lambda x: x.published_at, default=None)
    for author in Author.objects.prefetch_related("books")
]

Performs heavy computation in Python rather than leveraging the database, which is bad for performance. Also makes two database queries (better solutions do it in one).

Supercharge Your Django App: 7 Sneaky Tricks to Crush Slow Database Queries

Sun, 13 Aug 2023 12:15:13 -0400

Optimizing Django query performance is critical for building performant web applications. Django provides many tools and methods for optimizing database queries in its Database access optimization documentation. In this blog post, we will explore a collection of additional and essential tips I’ve compiled over the years to help you pinpoint and resolve your inefficient Django queries.

Kill Long-Running Queries with a Statement Timeout

PostgreSQL supports a statement_timeout parameter that allows you to set a maximum time limit per query. This is useful for preventing long-running queries from tying up precious resources and slowing down your application. My team at PixieBrix experienced an incident where a few long-running queries resulted in a full database outage. Setting a statement timeout in your Django settings can help prevent this from happening.

Optimize Django Query Performance by combining Select Related and Prefetch Related

Sat, 13 May 2023 15:48:39 -0400

Introduction

When building a Django application, one of the key challenges developers face is optimizing database query performance. Django provides two tools,select_related and prefetch_related, that reduce the number of database queries, and increase the performance of your application.

This blog post explores the power of these two methods and how to combine them to maximize your application’s query performance.

My team at PixieBrix implemented these techniques and have seen a significant improvement in query performance and customer satisfaction.

Sort a Django Queryset by a Custom Order

Mon, 02 Jan 2023 17:00:00 -0800

Sorting a Django queryset by a custom order is trickier than it looks, and choosing the wrong solution for your use case can cripple your application’s performance.

For example, let’s take a simple Todo model:

class Todo(models.Model):
    class Priority(models.TextChoices):
        HIGH = "HIGH", _("High")
        MEDIUM = "MEDIUM", _("Medium")
        LOW = "LOW", _("Low")

    title = models.CharField(max_length=255)
    priority = models.CharField(max_length=10, choices=Priority.choices, db_index=True)
    done = models.BooleanField(default=False)

And a list of todos:

Id	Title	Priority	Done
1	Add linters	Medium	Yes
2	Fix bug	High	No
3	Increase test coverage	Medium	No
4	Refactor views	Low	No
5	Run tests in CI	High	Yes

Our query set goal is to order these items from high to low priority. For priority levels with multiple items, we’ll also order by title to keep the results consistent. The expected result should be the id’s in the following order: [2, 5, 1, 3, 4]

5 ways to get all Django objects with a related object

Fri, 01 Oct 2021 13:08:48 -0700

In Django, a related object is a model instance used in the one-to-many or many-to-many context.

For example, let’s look at the built-in User model, which has a many-to-many relationship with the Group model.

class User(models.Model):
    groups = models.ManyToManyField(Group, related_name="groups")

For any given User object, all linked Group objects are called “related objects”. Here are 5 ways to fetch all User objects with at least one related Group object.

Iterate over each object in Python

users = []
for user in User.objects.prefetch_related("groups"):
    if user.groups.exists():
        users.append(user)

Probably the most popular approach but there are two problems:

Why you need to use Subqueries in Django

Wed, 21 Jul 2021 11:56:09 -0700

The Django ORM is a powerful tool but certain aspects of it are counterintuitive, such as the SQL order of execution.

Let’s look at an example of this trap and how we can fix it using subqueries:

class Book(models.Model):
    class Meta:
        constraints = [
            models.UniqueConstraint(
                fields=["name", "edition"],
                name="%(app_label)s_%(class)s_unique_name_edition",
            )
        ]

    name = models.CharField(max_length=255)
    edition = models.CharField(max_length=255)
    release_year = models.PositiveIntegerField(null=True)

I want to write a query that reads:

Find all N+1 violations in your Django app

Tue, 13 Apr 2021 23:10:50 -0800

The N+1 problem is a common database performance issue. It plagues ORM’s, such as Django and SQLAlchemy, because it leads to your application making more database queries than necessary.

Let’s look at a basic example in Django.

class Artist(models.Model):
    name = models.CharField(max_length=255)

class Song(models.Model):
    artist = models.ForeignKey(Artist, on_delete=models.CASCADE)
    name = models.CharField(max_length=255)

def print_songs():
    for song in Song.objects.all():
        print(f"{song.artist.name} - {song.name}")

Now let’s create a unit test to ensure print_songs runs successfully. We’re using pytest and pytest-django so we can create the test data using a pytest fixture. When we run the unit test, you’ll notice it passes and our songs are printed to the terminal. Everything seems fine on the surface.