Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES/plugin_api/+rv-content-batch-qs.deprecation
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Deprecated `RepositoryVersion.content_batch_qs`.
60 changes: 13 additions & 47 deletions pulpcore/app/models/repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from django_lifecycle import AFTER_UPDATE, BEFORE_CREATE, BEFORE_DELETE, hook
from rest_framework.exceptions import APIException

from pulpcore.app.loggers import deprecation_logger
from pulpcore.app.util import (
batch_qs,
cache_key,
Expand Down Expand Up @@ -1060,55 +1061,20 @@ def content(self):

def content_batch_qs(self, content_qs=None, order_by_params=("pk",), batch_size=1000):
"""
Generate content batches to efficiently iterate over all content.

Generates query sets that span the `content_qs` content of the repository
version. Each yielded query set evaluates to at most `batch_size` content records.
This is useful to limit the memory footprint when iterating over all content of
a repository version.

.. note::

* This generator is not safe against changes (i.e. add/remove content) during
the iteration!

* As the method uses slices internally, the queryset must be ordered to yield
stable results. By default, it is ordered by primary key.

Args:
content_qs (django.db.models.QuerySet) The queryset for Content that will be
restricted further to the content present in this repository version. If not given,
`Content.objects.all()` is used (to iterate over all content present in the
repository version). A plugin may want to use a specific subclass of
[pulpcore.plugin.models.Content][] or use e.g. `filter()` to select
a subset of the repository version's content.
order_by_params (tuple of str): The parameters for the `order_by` clause
for the content. The Default is `("pk",)`. This needs to
specify a stable order. For example, if you want to iterate by
decreasing creation time stamps use `("-pulp_created", "pk")` to
ensure that content records are still sorted by primary key even
if their creation timestamp happens to be equal.
batch_size (int): The maximum batch size.

Yields:
[django.db.models.QuerySet][]: A QuerySet representing a slice of the content.

Example:
The following code could be used to loop over all `FileContent` in
`repository_version`. It prefetches the related
[pulpcore.plugin.models.ContentArtifact][] instances for every batch::

repository_version = ...

batch_generator = repository_version.content_batch_qs(
content_class=FileContent.objects.all()
)
for content_batch_qs in batch_generator:
content_batch_qs.prefetch_related("contentartifact_set")
for content in content_batch_qs:
...
DEPRECATED: Don't use this method

Instead use this:
>>> content = version.get_content(
>>> FileContent.objects.prefetch_related("contentartifact_set")
>>> ).order_by("pk")
>>>
>>> for item in content.iterator(chunk_size=1000):
>>> ...
"""
deprecation_logger.warning(
"content_batch_qs is deprecated and will be removed in pulpcore 3.130. "
"Please use get_content with .iterator() instead."
)
version_content_qs = self.get_content(content_qs).order_by(*order_by_params)
yield from batch_qs(version_content_qs, batch_size=batch_size)

Expand Down
Loading