Skip to content

Feat/jobs taskiq - #73

Merged
Iamsdt merged 4 commits into
mainfrom
feat/jobs-taskiq
Nov 7, 2026
Merged

Iamsdt merged 4 commits into
mainfrom
feat/jobs-taskiq

Conversation

@Iamsdt

@Iamsdt Iamsdt commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator

Background tasks on TaskIQ

Adds durable background tasks to 10xgraph-api as a thin layer over TaskIQ: we own one broker, wire monitoring, auth and the run config, and add worker / scheduler commands and HTTP routes. TaskIQ does the queueing, retries, results and scheduling.

Developer API

from typing import Annotated
from taskiq import TaskiqDepends
from tenxgraph_api.jobs import broker, TaskContext

@broker.task(task_name="make_report", http=True)
async def make_report(topic: str, config: dict, ctx: Annotated[TaskContext, TaskiqDepends()]) -> dict:
    if await ctx.cancelled():
        return {"cancelled": True}
    return {"topic": topic, "user_id": config["user_id"]}

…egration

- Added `worker.py` to handle worker startup and shutdown, including task importing and broker configuration.
- Enhanced `GraphConfig` to parse worker configuration from `10xgraph.json`.
- Created unit tests for worker configuration, context management, and middleware functionality.
- Developed integration tests for Redis broker functionality, including idle queue reclaim and lease renewal.
- Established a testing environment with fixtures for task execution and state management.
- Implemented error handling and logging for task execution and middleware processes.
- Removed unused imports and fixtures from unit tests in `conftest.py`.
- Added new unit tests for worker commands in `test_commands.py` to validate argument parsing and environment setup.
- Enhanced configuration validation in `test_config.py` to support new broker types and ensure proper error handling for unsupported configurations.
- Updated task scheduling tests in `test_context.py` to streamline the enqueueing process and remove deprecated functionality.
- Introduced new route tests in `test_routes.py` to cover task management endpoints with authorization checks.
- Cleaned up worker tests in `test_worker.py` by removing obsolete tests related to task importing and checkpointer setup.
- Added comprehensive validation for worker configuration fields in `test_config_editor.py` to ensure correct schema definitions and error messages.
@Iamsdt
Iamsdt merged commit 9796c32 into main Nov 7, 2026
8 of 9 checks passed
@Iamsdt
Iamsdt deleted the feat/jobs-taskiq branch November 7, 2026 06:27
@Iamsdt
Iamsdt restored the feat/jobs-taskiq branch November 7, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant