Skip to content

FileWatcher only looks for .gitignore in watched directory, not project root #591

@NicolasFerec

Description

@NicolasFerec

Taskiq version

0.12.1

Python version

Python 3.14

OS

Linux

What happened?

When using --reload-dir with a subdirectory in a monorepo, the FileWatcher triggers an infinite reload loop because it only looks for .gitignore in the watched directory itself, not in the project root.

Root cause: In taskiq/cli/watcher.py, the FileWatcher constructor does:

gpath = path / ".gitignore"
if use_gitignore and gpath.exists():
    self.gitignore = parse_gitignore(gpath)

This means when you run:

taskiq worker --reload-dir back/domains -tp 'back/domains/**/*tasks.py' -fsd back.broker:broker

It looks for .gitignore at back/domains/.gitignore (which doesn't exist) instead of the project root .gitignore. Since no gitignore is loaded, Python's __pycache__ directories and .pyc files trigger reload events, creating an infinite loop.

Expected behavior: FileWatcher should walk up the directory tree to find .gitignore files, similar to how git itself works. Many tools (git, ripgrep, etc.) search parent directories for config files.

Current workaround: Use watchmedo from the watchdog package instead:

watchmedo auto-restart \
    --directory=back \
    --pattern=*.py \
    --recursive \
    --ignore-patterns='*/__pycache__/*;*.pyc;*.pyo' \
    -- taskiq worker -tp 'back/domains/**/*tasks.py' -fsd back.broker:broker

This works because watchmedo respects the root .gitignore.

Steps to reproduce

  1. Create a monorepo structure with frontend and backend:
project/
├── .gitignore  (contains __pycache__/)
├── front/
└── back/
    └── domains/
        └── tasks.py
  1. Run taskiq with a subdirectory reload:
taskiq worker --reload-dir back/domains -r -tp 'back/domains/**/*tasks.py' -fsd back.broker:broker
  1. Observer infinite reload loop as Python creates __pycache__/ directories

Suggested fix

Modify FileWatcher.__init__ to search for .gitignore in parent directories:

def __init__(
    self,
    callback: Callable[..., None],
    path: Path,
    use_gitignore: bool = True,
    **callback_kwargs: Any,
) -> None:
    self.callback = callback
    self.gitignore = None
    
    if use_gitignore:
        # Walk up the tree to find .gitignore (like git does)
        current = path.resolve()
        while current != current.parent:
            gpath = current / ".gitignore"
            if gpath.exists():
                self.gitignore = parse_gitignore(gpath)
                break
            current = current.parent
    
    self.callback_kwargs = callback_kwargs

Related issues

Relevant log output

[Task] Sending task=...
[FileWatcher] Reloading due to change in back/domains/__pycache__/tasks.cpython-314.pyc
[Task] Sending task=...
[FileWatcher] Reloading due to change in back/domains/__pycache__/tasks.cpython-314.pyc
... (infinite loop)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions