Query Optimization (N+1)¶

django-graphex inspects the incoming GraphQL selection and builds an optimized Django queryset for the list resolvers, so nested relations do not trigger the classic N+1 query problem. This happens automatically in DjangoListObjectField, DjangoFilterPaginateListField, DjangoFilterListField and DjangoModelType.QueryFields(), and for single-object lookups (DjangoObjectField and DjangoModelType retrieve / RetrieveField) — all routed through django_graphex.utils.queryset_factory.

Optimization surface at a glance¶

This page is the hub for every query-optimization feature — each is demonstrated below with models + types + a GraphQL query, and links to its dedicated reference page for the deep detail:

• select_related / prefetch_related derived from the selection (+ prefetch-crossing select_related drop) — below, this page.
• .only() column projection (root span + inside each Prefetch child) — below, this page.
• DB-side nested pagination (ROW_NUMBER() window slicing + filter-aware totalCount) — below; full decline list in Nested Lists → Performance (N+1).
• AnnotatedField (selection-driven .alias() / .annotate()) — below; full reference in Fields → AnnotatedField.
• Per-field optimize_<field> hook — below; full reference in Nested Lists → Per-field optimize hook.
• Typed GenericForeignKey unions (per-content-type GenericPrefetch narrowing) — below; full type-side declaration in Types → DjangoUnionType.

All of it is governed by the OPTIMIZE_* settings and the global OPTIMIZER_SAFE_MODE fail-safe.

What it does¶

For the fields requested in the query, the optimizer:

• adds select_related for forward ForeignKey / OneToOneField (and reverse one-to-one) — including nested dotted paths (a__b__c);
• adds prefetch_related for ManyToManyField and reverse relations (also nested, e.g. author__posts);
• adds prefetch_related for a GenericForeignKey (resolved in a second query on field.name; the parent's content-type-id and object-id columns are kept), and for a GenericRelation reverse side — which is .only()-narrowed, keeping its content-type / object-id attnames;
• applies a conservative .only() column projection both across the select_related span and inside each Prefetch child queryset (see caveats).

An automatic FK or forward/reverse OneToOne field whose destination type overrides get_queryset is a deliberate exception to the join plan. The optimizer replaces that unscoped join with a batched, request-scoped prefetch and resolves the nullable field from its visible result. This also runs for ordinary list reads when OPTIMIZE_QUERYSET=False; a custom-returned parent without a planned prefetch is checked at the field resolver instead. A cached unscoped relation is never used as visibility evidence. Automatic declared and reverse M2M output lists also use a scoped prefetch when the destination type overrides get_queryset. Their totalCount is taken from visible members before nested pagination; M2M window slicing is not introduced. The optimized M2M destination queryset is bound to the originating list's database alias before its visibility hook runs, matching ordinary traversal. Explicit resolver fields keep their separate contract.

It is transparent to the DjangoListObjectType wrapper (results / totalCount / pageInfo), to fragments and to inline fragments, and matches both camelCase and snake_case field names.

When GraphQL merges repeated selections of the same root response field, the optimizer plans from all of their selection sets. For example, a direct allPosts { results { title } } selection and a fragment selecting allPosts { results { author { name } } } share one result; the optimizer still joins author and keeps its requested columns. A relation selected only in a later root field node does not require one extra query per returned post.

Repeated spreads of an acyclic fragment are planned once per distinct optimizer context (model, GraphQL type, and relation path), rather than expanding the same subtree once for every spread route. For example, two spreads of a fragment selecting author { name } keep the same select_related("author") and column projection as before. Reusing that fragment under different relation paths still plans each path separately. This changes optimizer planning work, not GraphQL execution, response fields, query cost, or the number of SQL queries. Repeated fragment plans are reapplied in selection order, so a later spread retains its existing precedence over an interleaved selection of the same relation or annotation alias.

Example¶

# models.py
class Author(models.Model):
    name = models.CharField(max_length=100)

class Tag(models.Model):
    label = models.CharField(max_length=50)

class Post(models.Model):
    title = models.CharField(max_length=200)
    body = models.TextField()
    author = models.ForeignKey(Author, related_name="posts", on_delete=models.CASCADE)
    tags = models.ManyToManyField(Tag, related_name="posts")

# schema.py
from django_graphex.fields import DjangoListObjectField
from django_graphex.core import ObjectType
from django_graphex.types import DjangoListObjectType, DjangoObjectType


class AuthorType(DjangoObjectType):
    class Meta:
        model = Author

class TagType(DjangoObjectType):
    class Meta:
        model = Tag

class PostType(DjangoObjectType):
    class Meta:
        model = Post

class PostListType(DjangoListObjectType):
    class Meta:
        model = Post

class Query(ObjectType):
    all_posts = DjangoListObjectField(PostListType)

A nested query:

{
  allPosts {
    results {
      title
      author { name }   # ForeignKey  -> select_related("author")
      tags { label }    # ManyToMany   -> prefetch_related("tags")
    }
    totalCount
  }
}

runs a constant number of database queries no matter how many posts are returned:

1. SELECT ... FROM post JOIN author ... (rows + author joined, select_related)
2. SELECT ... FROM tag ... (one extra query for the prefetch_related)
3. SELECT COUNT(*) ... (from the list field's totalCount)

Without optimization the same query would run 1 + N (one author query per post) + N (one tag query per post). This is verified in the test-suite with assertNumQueries.

Single objects¶

The same optimization applies to single-object queries. For:

{
  post(id: 1) {
    title
    author { name }   # select_related("author")
    tags { label }    # prefetch_related("tags")
  }
}

the lookup runs 1 query (the row with its forward relations joined in) + 1 per prefetched relation, instead of one query per nested relation.

.only() column projection¶

When OPTIMIZE_ONLY_FIELDS is enabled (default), the optimizer also restricts the selected columns with .only(), loading just the fields the query asks for. To stay correct it is conservative:

• it always keeps the primary key (for every model in the select_related span), forward ForeignKey columns and the model's Meta.ordering columns;
• a model that selects a computed / property / custom-named field is loaded in full (not narrowed), so that property keeps working;
• narrowing also applies inside each Prefetch child queryset: the child gets its own .only() keeping the reverse-FK back column, the child's Meta.ordering columns, its pk and any GenericRelation ct/fk attnames; a small set of forward-FK heads is added back to the child select_related so the narrowing does not re-introduce an N+1. A child whose sub-selection hits a computed / property leaf is full-loaded (bare-string prefetch, no .only()).

DJANGO_GRAPHEX = {
    "OPTIMIZE_ONLY_FIELDS": False,
}

Settings¶

Configure in settings.py under DJANGO_GRAPHEX:

DJANGO_GRAPHEX = {
    "OPTIMIZE_QUERYSET": True,
    "OPTIMIZE_ONLY_FIELDS": True,
    "OPTIMIZE_NESTED_PAGINATION": True,
    "OPTIMIZER_SAFE_MODE": False,
    "OPTIMIZER_WORK_BUDGET": 2000,
    "OPTIMIZE_ANNOTATED_FIELDS": True,
}

OPTIMIZER_SAFE_MODE (fail-safe degrade)¶

By default (OPTIMIZER_SAFE_MODE = False) the optimizer fails loud: if building the optimized queryset raises, the exception propagates and you get a 500 — so you find the bug.

Setting it to True wraps the whole optimization in a try/except: on any exception the entire resolve degrades to the un-optimized base queryset and a WARNING is logged (django_graphex.utils) instead of surfacing the error. The boundary is coarse — it degrades the whole resolve, not a single field — so a raising per-field optimize_<field> hook is caught here too. It does not cover a malformed AnnotatedField expression that raises FieldError at SQL-eval time (that happens outside the build boundary; annotation injection has its own narrower guard that skips just the annotation).

DJANGO_GRAPHEX = {
    "OPTIMIZER_SAFE_MODE": True,
}

OPTIMIZER_WORK_BUDGET (work cap)¶

Fragments make a short document describe a huge selection tree. When a fragment reaches the next one through different relations (author, coAuthors, posts, coauthoredPosts, ...), every path is a different ORM lookup, so the lookups the optimizer would emit grow as 4**n for n fragments. Memoizing by fragment cannot plan an exponential set in linear time, and emitting it would also mean one query per lookup once there is data.

OPTIMIZER_WORK_BUDGET (default 2000) therefore caps, per operation execution and per walker, how many selection contexts the optimizer expands. A realistic document stays orders of magnitude below it. The budget is shared by all root fields (aliases) of one execution; each item of a batch is its own execution. Once any walker spends it, the root field being planned is resolved without the optimizer plan, exactly as with OPTIMIZE_QUERYSET = False, and so are the request's later root fields: the data is identical and relations behind a get_queryset scope stay scoped, but relations load through the normal resolvers (possibly one query per parent). A partial plan is never kept, because it could disagree with itself and drop the ordering, limit or offset of a nested list. A DEBUG record on the django_graphex.utils logger names the walker that hit the cap. Use MAX_QUERY_DEPTH / MAX_QUERY_COST to reject such documents outright; set the budget to None only to restore the unbounded planning.

DB-side nested pagination (window slicing)¶

When a reverse-FK nested list is paginated (with a LimitOffset or Page paginator), the optimizer slices each parent's page in the database instead of loading every child and slicing in memory. It does this inside the single Prefetch it already builds for that level — so the query count stays constant as the number of parents grows.

Using the same Author (1) ─→ (N) Post models from above, with a paginator and a nested filter:

# models.py
class Post(models.Model):
    title = models.CharField(max_length=200)
    author = models.ForeignKey(Author, related_name="posts", on_delete=models.CASCADE)

    class Meta:
        ordering = ("-id",)

# schema.py
from django_graphex.fields import DjangoListObjectField
from django_graphex.core import ObjectType
from django_graphex.paginations import LimitOffsetGraphqlPagination
from django_graphex.types import DjangoListObjectType, DjangoObjectType

class PostType(DjangoObjectType):
    class Meta:
        model = Post
        filter_fields = {"title": ["icontains"]}     # enables the nested filter

class PostListType(DjangoListObjectType):
    class Meta:
        model = Post
        pagination = LimitOffsetGraphqlPagination(default_limit=10, ordering="-id")

class AuthorType(DjangoObjectType):
    class Meta:
        model = Author

class AuthorListType(DjangoListObjectType):
    class Meta:
        model = Author

class Query(ObjectType):
    authors = DjangoListObjectField(AuthorListType)

Filter on the field, paginate/order on results:

{
  authors {
    results {
      name
      posts(filter: { title: { icontains: "x" } }) {   # filter on the field
        results(limit: 2, ordering: "-id") { title }    # paginate/order on results
        totalCount
      }
    }
    totalCount
  }
}

What it optimizes: the one Prefetch covering every author's posts adds two window functions and filters to the page window, so it fetches only each author's requested page rows DB-side (not all-then-slice-in-memory):

_gqx_rn    = Window(RowNumber(), partition_by=[F("author_id")], order_by=...)
_gqx_total = Window(Count("*"),  partition_by=[F("author_id")])
# ...then .filter(_gqx_rn__gt=offset, _gqx_rn__lte=offset + limit)

totalCount is the per-partition filtered COUNT(*) read off _gqx_total, so it reflects the nested filter without an extra query. Adding authors never adds queries — the level stays at a constant query count.

Controlled by OPTIMIZE_NESTED_PAGINATION (default True). When a relation is not windowable, the same single Prefetch is reused and the page is ordered + sliced in memory instead — see Nested Lists → Performance (N+1) for the full decline list (cursor paginator, M2M, relation-aggregate child, non-concrete ordering, full-load sub-selection, .distinct(), OPTIMIZE_QUERYSET=False).

Per-field optimize hook¶

To customize the child queryset for a specific nested list field — add a select_related, a custom annotation, a default ordering — declare an optimize_<snake_field> static method on the parent type:

from django_graphex.fields import DjangoNestedListObjectField

class AuthorType(DjangoObjectType):
    posts = DjangoNestedListObjectField(PostListType, accessor="posts")

    class Meta:
        model = Author

    @staticmethod
    def optimize_posts(queryset, info, **kwargs):
        # kwargs: filter_value (filter input or None), is_window (True on the
        # DB-side windowed path). Compose on the optimizer-built child queryset.
        return queryset.select_related("category")

• The method name is optimize_ + the snake_case GraphQL field name (blogPosts → optimize_blog_posts), declared on the parent type.
• It is called once per query (not once per parent) and must return a QuerySet — anything else emits a WARNING and the built queryset is used unchanged.
• kwargs are filter_value (the filter input or None) and is_window (True only on the DB-side windowed path).
• A raising hook is caught by OPTIMIZER_SAFE_MODE when it is True (whole resolve degrades); with the default False it propagates. Only DjangoNestedListObjectField supports the hook.

See Nested Lists → Per-field optimize hook for the full rules, the safe-mode interaction and a complete example.

Typed GenericForeignKey unions (per-content-type narrowing)¶

A GenericForeignKey exposed as a DjangoUnionType (member types in Meta.types, owner opting in via Meta.unions) lets clients select per-member fields with inline fragments:

{
  attachments {
    results {
      caption
      target {
        __typename
        ... on AccountType { balance }
        ... on InvoiceType { amount }
      }
    }
  }
}

What it optimizes: on Django 5.0+ with OPTIMIZE_ONLY_FIELDS on, the optimizer routes the union GFK through a GenericPrefetch that builds one .only()-narrowed queryset per content type — the Account queryset fetches balance, the Invoice queryset fetches amount — batched across all parents (no N+1). On Django < 5.0 it degrades gracefully to a single bare full-load Prefetch (it never imports GenericPrefetch and never narrows columns).

Named fragment spreads are equivalent to the inline form here: a document that declares fragment Money on AccountType { balance } and spreads it under target narrows the Account queryset exactly as ... on AccountType { balance } would, and a selection that mixes both forms merges them into a single per-member queryset instead of deferring the spread's columns.

You may also group member selections under a valid fragment on the union itself. Both inline and named union wrappers retain the same per-member batching and nested relation joins as direct member fragments:

query {
  allComments {
    results {
      target {
        ... on CommentTargetUnion {
          ... on OrderType { total customer { label } }
        }
        ...AllAccountTargets
      }
    }
  }
}

fragment AllAccountTargets on CommentTargetUnion {
  ... on AccountType { balance }
}

The wrapper has no Django model of its own: the optimizer descends into its valid member selections, then builds the same per-content-type querysets it would build without the wrapper. This does not infer GraphQL type relationships from Django multi-table inheritance or make invalid fragments valid.

The type-side declaration order is load-bearing (members → union → owner LAST). See Types → DjangoUnionType and Types → per-content-type narrowing for the full declaration and the proxy-model de-dup behavior.

Selection-driven annotations (AnnotatedField)¶

An AnnotatedField is a declarative GraphQL field backed by a Django ORM annotation that the optimizer injects only when the field is selected in the incoming query (and only when OPTIMIZE_ANNOTATED_FIELDS is True, the default). Unselected annotated fields add no SQL. A built-in resolver reads the annotation off the row, so no resolve_<field> is needed.

A forward-FK relation that the optimizer placed in select_related is auto-promoted to prefetch_related when its child sub-selection contains an AnnotatedField — DB annotations cannot be pushed through a SQL JOIN, so the child annotation rides on the promoted Prefetch's queryset instead.

This promotion also follows applicable named and inline GraphQL fragments on the relation or its list results wrapper. It respects fragment type conditions and bound @include/@skip directives: an excluded AnnotatedField does not trigger the extra prefetch. Previously, selecting a related annotation only through a valid fragment could resolve it as null without a GraphQL error.

The promotion follows the whole forward-FK chain, not just its first hop: in { comments { post { author { postCount } } } } it is post__author that gets promoted, so an AnnotatedField several hops down resolves like any other. (It used to be detected only one hop from the root, which left the deeper field resolving to null with no error.)

Declare it on the type and let the selection drive it (Author (1) ─→ (N) Post):

# schema.py
from graphql import GraphQLInt
from django.db.models import Count
from django_graphex.fields import AnnotatedField, DjangoListObjectField
from django_graphex.core import ObjectType
from django_graphex.types import DjangoListObjectType, DjangoObjectType

class AuthorType(DjangoObjectType):
    # Injected ONLY when `postCount` is selected; the built-in resolver reads it
    # off the row, so no resolve_post_count is needed.
    post_count = AnnotatedField(GraphQLInt, Count("posts"))

    class Meta:
        model = Author

class AuthorListType(DjangoListObjectType):
    class Meta:
        model = Author

class Query(ObjectType):
    authors = DjangoListObjectField(AuthorListType)

{
  authors {
    results {
      name
      postCount        # selected -> optimizer adds .annotate(_gqx_ann_post_count=Count("posts"))
    }
  }
}

What it optimizes: when postCount is in the selection the optimizer adds the Count("posts") annotation (one aggregate, no extra round-trip per author); when it is not selected, no annotation and no extra SQL are emitted. The expression may also be a zero-arg callable (lambda: Count("posts")) for fresh per-request construction, and aliases= applies .alias() before .annotate().

See Fields → AnnotatedField for the full signature, arguments table and the forward-FK promotion note.

@skip and @include directives¶

The optimizer honors @skip and @include on every selection node — fields, inline fragments, and fragment spreads. A selection that is excluded is not added to select_related / prefetch_related and its columns are not included in .only(). This prevents over-fetching related rows that the client will never use.

query GetPosts($loadAuthor: Boolean!, $skipTags: Boolean!) {
  posts {
    results {
      title
      # When $loadAuthor is false the optimizer skips the author join entirely
      author @include(if: $loadAuthor) {
        name
      }
      # When $skipTags is true the tags prefetch is skipped entirely
      tags @skip(if: $skipTags) {
        label
      }
    }
  }
}

{ "loadAuthor": false, "skipTags": true }

With these variables, the author FK is not added to select_related (and author_id is not included in the .only() projection), and the tags M2M is not added to prefetch_related. Passing the flag as a variable is how @skip / @include are used in practice: the client toggles sections of one static query per request (a detail panel open or closed, a mobile vs. desktop view) instead of maintaining and re-parsing different query strings.

Custom resolvers¶

If your query type defines resolve_<field> that returns a QuerySet, the optimizer adopts it as the base queryset and still applies select_related / prefetch_related on top of it. (.only() is skipped for custom-resolved querysets, since they may already shape their own columns.) Resolvers that return anything other than a QuerySet are left untouched.

Example — scope a list to the current user while keeping optimizer benefits:

from django_graphex.fields import DjangoListObjectField
from django_graphex.core import ObjectType
from django_graphex.types import DjangoListObjectType

class PostListType(DjangoListObjectType):
    class Meta:
        model = Post
        pagination = LimitOffsetGraphqlPagination(default_limit=10, ordering="-id")

class Query(ObjectType):
    my_posts = DjangoListObjectField(PostListType, description="Posts by the current user")

    def resolve_my_posts(self, info, **kwargs):
        # Return a queryset — the optimizer still adds select_related / prefetch_related
        # on top of this base. For per-request scoping that composes cleanly with
        # pagination and filtering, use get_queryset on the NODE type (PostType) or
        # Meta.queryset on the list type — a list type has no scoping hook of its own.
        return Post.objects.filter(author=info.context.user)

The resolver returns a QuerySet, so the optimizer picks it up and adds select_related("author") / prefetch_related("tags") etc. based on the GraphQL selection. If the resolver returned a plain list, the optimizer would leave it untouched (no queryset to decorate).

For an automatic forward FK whose destination type overrides get_queryset, the optimizer uses a scoped prefetch in place of that FK's select_related join. This lets one destination query serve all selected parents while excluding hidden targets. Reverse-FK list prefetches and window-pagination queries apply the child type's scope before counting or slicing. A default, unchanged hook keeps the ordinary join/prefetch plan. With OPTIMIZE_QUERYSET=False, output still honors these scopes, but each uncached related value may require its own database check. This is a correctness guarantee, not a promise that disabling optimization preserves batching.

Optimized mutation re-read¶

DjangoModelType.perform_mutate re-reads the saved object from the database so the mutation response reflects the freshest DB state (annotations, default values, etc.). Starting in v1.2.1, this re-read is selection-aware: the optimizer inspects the mutation's selection set, locates the sub-field node for Meta.output_field_name (e.g. post inside { ok post { … } }), and applies select_related for every to-one relation (ForeignKey, OneToOneField) that appears in that sub-selection.

For destination types with the default no-op row hook, this eliminates N+1 queries on mutation responses that nest related objects:

mutation CreatePost($input: NewPostInput!) {
  postCreate(newPost: $input) {
    ok
    post {
      title
      author { name }     # joined via select_related — no extra query
      category { title }  # joined via select_related — no extra query
    }
  }
}

If a destination has a row-scope hook, its automatic to-one field still checks that hook. A mutation response without a batched scoped prefetch can issue an additional membership query for the destination.

No code changes are required; the optimization is applied automatically whenever DjangoModelType.perform_mutate is called. If the selection set cannot be parsed (e.g. a custom info stub without field nodes), the method falls back to the plain unoptimized re-read so existing behaviour is preserved.