Refactor async_delete mapping and filter logic

[Maffooch]

Improve clarity and accuracy of the async_delete mapping and filter logic by updating the query parameters and ensuring proper ID usage throughout the deletion process.

Fixes stack trace

[15/Dec/2025 12:14:40] ERROR [dojo.api_v2.exception_handler:49] Model instances passed to related filters must be saved.
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/rest_framework/views.py", line 512, in dispatch
    response = handler(request, *args, **kwargs)
  File "/app/dojo/api_v2/views.py", line 1891, in destroy
    async_del.delete(instance)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/app/dojo/decorators.py", line 104, in __wrapper__
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.13/site-packages/celery/local.py", line 182, in __call__
    return self._get_current_object()(*a, **kw)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/celery/app/task.py", line 411, in __call__
    return self.run(*args, **kwargs)
           ~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/app/dojo/utils.py", line 2062, in delete
    self.crawl(obj, model_list)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/app/dojo/decorators.py", line 104, in __wrapper__
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.13/site-packages/celery/local.py", line 182, in __call__
    return self._get_current_object()(*a, **kw)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/celery/app/task.py", line 411, in __call__
    return self.run(*args, **kwargs)
           ~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/app/dojo/utils.py", line 2078, in crawl
    objects_to_delete = model.objects.only("id").filter(**filter_dict).distinct().order_by("id")
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/django/db/models/query.py", line 1478, in filter
    return self._filter_or_exclude(False, args, kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/django/db/models/query.py", line 1496, in _filter_or_exclude
    clone._filter_or_exclude_inplace(negate, args, kwargs)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/django/db/models/query.py", line 1506, in _filter_or_exclude_inplace
    self._query.add_q(Q(*args, **kwargs))
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/django/db/models/sql/query.py", line 1609, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
                ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/django/db/models/sql/query.py", line 1641, in _add_q
    child_clause, needed_inner = self.build_filter(
                                 ~~~~~~~~~~~~~~~~~^
        child,
        ^^^^^^
    ...<7 lines>...
        update_join_types=update_join_types,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.13/site-packages/django/db/models/sql/query.py", line 1555, in build_filter
    condition = self.build_lookup(lookups, col, value)
  File "/usr/local/lib/python3.13/site-packages/django/db/models/sql/query.py", line 1385, in build_lookup
    lookup = lookup_class(lhs, rhs)
  File "/usr/local/lib/python3.13/site-packages/django/db/models/lookups.py", line 30, in __init__
    self.rhs = self.get_prep_lookup()
               ~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/django/db/models/fields/related_lookups.py", line 147, in get_prep_lookup
    self.rhs = get_normalized_value(self.rhs, self.lhs)[0]
               ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/django/db/models/fields/related_lookups.py", line 45, in get_normalized_value
    raise ValueError("Model instances passed to related filters must be saved.")
ValueError: Model instances passed to related filters must be saved.

[dryrunsecurity]

This pull request contains a race condition: the code re-initializes the task_results list inside the loop over model types, so only the last batch of asynchronous Celery delete tasks is waited on, allowing the parent object to be deleted before earlier child deletions complete and potentially leaving orphaned records.

Race Condition in Asynchronous Deletion in dojo/utils.py

Vulnerability

Race Condition in Asynchronous Deletion

Description

The task_results list is re-initialized within the for model_info in model_list: loop. This means that for each type of child object being processed (e.g., Endpoints, Findings, Tests, Engagements related to a Product_Type), the task_results list is cleared. Consequently, the subsequent loop for task_result in task_results: will only wait for the asynchronous delete_chunk tasks related to the last model_info processed. The delete_chunk calls are indeed asynchronous Celery tasks, as indicated by the @app.task decorator on delete_chunk and the hasattr(result, "get") check, which is characteristic of Celery's AsyncResult objects. This leads to a race condition where the main parent object (obj) is deleted by self.delete_chunk([obj]) before all its dependent child objects (from previous model_info iterations) have been fully processed and deleted, potentially resulting in orphaned records in the database.

django-DefectDojo/dojo/utils.py

Lines 2073 to 2076 in 4821353

	task_results = []
	model = model_info[0]
	model_query = model_info[1]
	filter_dict = {model_query: obj.id}

---

All finding details can be found in the DryRun Security Dashboard.

[valentijnscholten]

iirc my intention was to move this to the left. looks like instead i moved the lines below to the right.

[valentijnscholten]

Looks OK, but we may have to consider not doing it in parallel at considering the locking issues it's causing.

[mtesauro]

Approved