Repository navigation
feat(authoring): testcase generator DSL, variant families, Polygon legacy import, statement attachments [wave F items 122/126/127/128] - #355
Prekzursil wants to merge 2 commits into
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
| if not template: | ||
| raise ValueError("the script needs a print template line") | ||
| # every placeholder must be declared | ||
| for field in re.findall(r"\{(\w+)\}", template): | ||
| if field not in variables: | ||
| raise ValueError(f"template references undeclared field '{field}'") | ||
| return {"vars": variables, "template": template} |
There was a problem hiding this comment.
Bug: Malformed print templates pass parse_dsl validation but trigger unhandled exceptions during str.format in generate(), resulting in HTTP 500 errors.
Severity: MEDIUM
Suggested Fix
Validate the format string during parse_dsl by rendering it with dummy data or using string.Formatter().parse(template). Alternatively, wrap calls to generate() in GenerateTestsView in a try/except (ValueError, IndexError, AttributeError) block and return a 400 response.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/generator_dsl.py#L49-L55
Potential issue: `parse_dsl` validates placeholders with `re.findall(r"\{(\w+)\}",
template)`, which only matches bare `{name}` placeholders and ignores other format
specifications. When `generate()` calls `str.format(**values)`, malformed or unexpected
format syntax—such as unclosed braces, positional braces `{}`, or invalid format
specifiers—causes `IndexError`, `ValueError`, or `AttributeError`.
`GenerateTestsView.post` only catches `ValueError` during `parse_dsl`, leaving
`generate(...)` calls unguarded against these exceptions and returning an unhandled 500
error instead of a validation error.
Also affects:
backend/problems/authoring_ops.py:77~84
Did we get this right? 👍 / 👎 to inform future reviews.
| def post(self, request, problem_id): | ||
| problem = Problem.objects.filter(pk=problem_id).first() | ||
| if problem is None: | ||
| return Response({"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND) | ||
| if not _authoring_gate(request.user): | ||
| return Response( | ||
| {"detail": "Authors and verifiers only."}, | ||
| status=status.HTTP_403_FORBIDDEN, | ||
| ) |
There was a problem hiding this comment.
Bug: Authoring endpoints lack problem ownership and status validation, allowing unauthorized cross-author test generation, variant spawning, and attachment access.
Severity: HIGH
Suggested Fix
Enforce ProblemObjectPermissions and _guard_contest_pin on write endpoints like GenerateTestsView, VariantSpawnView, and AttachmentView. Scope queryset lookups to authorized or approved problems in VariantListView.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/authoring_ops.py#L65-L73
Potential issue: Authoring endpoints rely solely on role checks in `_authoring_gate`
rather than verifying problem ownership, status, or contest pinning. Consequently, users
with authoring roles can append tests to any problem, including approved or
contest-pinned problems, spawn variants from other authors' private or draft problems,
and list draft variant details or attachments across authors without proper
authorization checks.
Also affects:
backend/problems/authoring_ops.py:114~146backend/problems/authoring_ops.py:173~177backend/problems/authoring_ops.py:216~265
Did we get this right? 👍 / 👎 to inform future reviews.
| _LINE_RE = re.compile( | ||
| r"^\s*(\w+)\s*=\s*(int|str)\s*\(\s*(-?\d+)\s*,\s*(-?\d+)\s*\)\s*$" | ||
| ) |
There was a problem hiding this comment.
Bug: DSL str() generator does not enforce an upper bound on length, allowing excessive CPU and memory consumption during test generation.
Severity: MEDIUM
Suggested Fix
Enforce a reasonable maximum limit on hi (e.g., 100_000) for str type generators within parse_dsl in backend/problems/generator_dsl.py.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/generator_dsl.py#L20-L22
Potential issue: The generator DSL accepts unbounded integer bounds for string
generators `str(lo, hi)`. When building a string of length `rng.randint(lo, hi)`,
`generate()` executes `"".join(rng.choice(alphabet) for _ in range(length))` without
capping `hi`. Large values such as `10**8` or `10**9` lead to extreme CPU consumption
and multi-gigabyte memory allocations across up to 50 iterations, potentially exhausting
worker resources.
Also affects:
backend/problems/authoring_ops.py:83~93
Did we get this right? 👍 / 👎 to inform future reviews.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4a96695d30
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| problem = Problem.objects.filter(pk=problem_id).first() | ||
| if problem is None: | ||
| return Response({"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND) | ||
| if not _authoring_gate(request.user): |
There was a problem hiding this comment.
Enforce problem-level authorization in authoring endpoints
Any problem creator or teacher can supply another author's problem ID and pass _authoring_gate. Consequently, GenerateTestsView can append scored tests with empty expected outputs to someone else's approved problem, and VariantSpawnView can copy another author's private statement into a draft owned by the caller. Unlike ProblemViewSet.get_queryset() and ProblemObjectPermissions, these endpoints enforce neither visibility nor ownership/status restrictions. Apply the corresponding object-level access checks before reading or modifying the selected problem, including attachment operations.
Useful? React with 👍 / 👎.
| base_order = problem.test_cases.count() | ||
| created = 0 | ||
| for offset in range(data["count"]): | ||
| rendered = generate(data["dsl"], data["seed"] + offset) | ||
| TestCase.objects.create( |
There was a problem hiding this comment.
Preserve the contest pin before generating test cases
For a problem in an upcoming or running contest, this loop creates immediately judge-visible test cases without calling _guard_contest_pin. The existing TestCaseViewSet.perform_create, update, and delete paths explicitly invoke that guard to freeze the test suite. Even a verifier who is otherwise authorized to edit the problem can now bypass that freeze, changing grading during the contest; the generated empty expected outputs make this especially disruptive. Invoke the same contest guard before creating any rows.
Useful? React with 👍 / 👎.
| length = rng.randint(lo, hi) | ||
| alphabet = "abcdefghijklmnopqrstuvwxyz" | ||
| values[name] = "".join(rng.choice(alphabet) for _ in range(length)) |
There was a problem hiding this comment.
Bound generated string sizes before executing the DSL
A valid script such as s = str(1000000000, 1000000000)\nprint {s} passes parsing and the endpoint's 20,000-character script limit, then performs a billion random choices and builds a huge string inside the web worker. With up to 50 generations per request, an author can exhaust worker memory or monopolize request processing using a tiny payload. Enforce per-variable and aggregate generated-output limits during validation, before generation begins.
Useful? React with 👍 / 👎.
| statement_i18n=dict(parent.statement_i18n), | ||
| status=Problem.ProblemStatus.DRAFT, | ||
| author=request.user, | ||
| difficulty=parent.difficulty, | ||
| default_time_limit_ms=parent.default_time_limit_ms, | ||
| default_memory_limit_kb=parent.default_memory_limit_kb, | ||
| variant_parent=root, |
There was a problem hiding this comment.
Carry the import permission ledger into spawned variants
When the parent is imported content, the child copies its statement but leaves import_source unset. Both ProblemViewSet.submit_for_approval and approve_problem enforce publication rights only when problem.import_source_id is present, so spawning a variant removes the ledger check and allows the copied material to proceed through approval even if the source permission is revoked or expires. Preserve the parent's import-source association on the child so variants remain subject to the same publication restrictions.
Useful? React with 👍 / 👎.
| try { | ||
| const response = await ProblemService.uploadAttachment(Number(problemId), fileName); | ||
| setMessage(t('attachment_uploaded', 'Uploaded {{name}}', { name: response.name })); |
There was a problem hiding this comment.
Upload actual file contents instead of the default placeholder
The author-tools Upload action passes only a filename, so ProblemService.uploadAttachment always uses its default contentBase64 = 'eA==', which decodes to the single byte x. There is no file selector or content-reading path in this page. Thus uploading figure.png reports success but stores a one-byte non-image rather than the author's attachment. Add file selection, read the selected bytes, and pass their base64 encoding explicitly instead of silently substituting placeholder content.
Useful? React with 👍 / 👎.
| # every placeholder must be declared | ||
| for field in re.findall(r"\{(\w+)\}", template): | ||
| if field not in variables: | ||
| raise ValueError(f"template references undeclared field '{field}'") | ||
| return {"vars": variables, "template": template} |
There was a problem hiding this comment.
Validate the complete print template before generation
The placeholder regex accepts malformed or unsupported format strings: print { and print {missing!s} both pass parse_dsl, but generate subsequently raises ValueError or KeyError in str.format. GenerateTestsView catches errors only around its initial parse, so these ordinary DSL mistakes return HTTP 500 instead of the promised validation response. Parse and validate the complete format syntax, rejecting unsupported fields and unmatched braces before creating test cases.
Useful? React with 👍 / 👎.
| class GenerateTestsView(APIView): | ||
| """Item 122 — render the generator DSL into real TestCase rows.""" | ||
|
|
||
| permission_classes = [permissions.IsAuthenticated] |
There was a problem hiding this comment.
Apply the privileged two-factor gate to authoring writes
When TWO_FACTOR_ENFORCE is enabled, these new write endpoints still require only authentication and the role check. A privileged account without enrolled TOTP can therefore create problem variants and mutate test suites through these routes, although ProblemViewSet.get_permissions() applies PrivilegedRoleRequires2FA to content creation and editing in that configuration. Compose the same permission into the new authoring write views so they do not bypass the enforced content-supply-chain restriction.
Useful? React with 👍 / 👎.
| difficulty=parent.difficulty, | ||
| default_time_limit_ms=parent.default_time_limit_ms, | ||
| default_memory_limit_kb=parent.default_memory_limit_kb, | ||
| variant_parent=root, | ||
| variant_note=note, |
There was a problem hiding this comment.
Preserve the parent's judging configuration in variants
Spawning a variant of an interactive, output-only, floating-point, or custom-checker problem copies its statement and samples but silently resets its judging configuration to model defaults: problem_type=NORMAL and exact line comparison, with no checker or interactor. In submissions/tasks.py, those fields select the execution and comparison branches, so the child no longer implements the task described by its copied statement; for example, a valid alternative answer to a custom-checker task is rejected against the single stored sample answer. Copy the parent's problem type and associated judge/comparison configuration alongside its time and memory limits.
Useful? React with 👍 / 👎.
| try: | ||
| parse_dsl(data["dsl"]) | ||
| except ValueError as exc: | ||
| return Response({"dsl": str(exc)}, status=status.HTTP_400_BAD_REQUEST) |
| class VariantListView(APIView): | ||
| """Item 126 — the family tree for one problem (root + siblings).""" | ||
|
|
||
| permission_classes = [permissions.AllowAny] | ||
|
|
||
| def get(self, request, problem_id): | ||
| problem = Problem.objects.filter(pk=problem_id).first() | ||
| if problem is None: | ||
| return Response({"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND) | ||
| root = problem.variant_root |
There was a problem hiding this comment.
Bug: VariantListView allows unauthenticated access and omits status/visibility checks, exposing unpublished draft problem metadata to anonymous users.
Severity: MEDIUM
Suggested Fix
Enforce authentication and author permissions (e.g. permissions.IsAuthenticated and _authoring_gate), or filter the variant family queryset using the same visibility rules applied in ProblemViewSet.get_queryset.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/authoring_ops.py#L168-L177
Potential issue: `VariantListView` uses `permission_classes = [permissions.AllowAny]`
and queries problems across the family (`Q(pk=root.pk) | Q(variant_parent=root)`)
without filtering by `status=APPROVED` or reusing `ProblemViewSet.get_queryset`
visibility rules. As a result, unauthenticated callers can list draft and pending
problems, leaking titles, variant notes, statuses, and IDs of unpublished problems.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: affeaef5de
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| family = Problem.objects.filter( | ||
| dj_models.Q(pk=root.pk) | dj_models.Q(variant_parent=root) | ||
| ).order_by("id") |
There was a problem hiding this comment.
Filter variant families by the caller's problem visibility
When a family contains draft or private problems, this AllowAny endpoint exposes their IDs, titles, parameter notes, and statuses to anonymous callers. Querying an approved root also reveals its unpublished children, so checking only the requested problem would not prevent the disclosure. Unlike ProblemViewSet.get_queryset() in backend/problems/views.py, neither query applies visibility restrictions. Apply the same visibility policy to both the requested problem and the returned family members.
Useful? React with 👍 / 👎.
…gacy import, statement attachments [wave F items 122/126/127/128]
- item 122: problems/generator_dsl.py — declarative int/str variable
declarations + print template; parse errors pinpoint the line;
generate() is deterministic per (script, seed);
POST /problems/{id}/generate-tests/ renders real TestCase rows
(inputs first; expected outputs flow through the existing
expected-output tool / reference solution)
- item 126: Problem.variant_parent/variant_note — children point at the
family ROOT (grandchildren flatten to the root); spawn copies statement
+ samples into a DRAFT child; GET /problems/{id}/variants/list/ tree;
/author/tools UI
- item 127: the package importer now also accepts the legacy Polygon
extensionless tests/<n> + tests/<n>.a layout (Kattis .in/.ans stays
primary; seen-set dedup when both forms exist)
- item 128: StatementAttachment — base64 upload onto media storage with a
2MB/10-file cap and sanitized names, inline/download kinds;
GET/POST /problems/{id}/attachments/
- classroom effort-board test made wall-clock independent on this branch
- BACKLOG statuses for 122/126/127/128
affeaef to
1bbc235
Compare
| if not ATTACHMENT_NAME_RE.match(data["name"]): | ||
| return Response( | ||
| {"name": "Letters, digits, dash, underscore, dot and space only."}, | ||
| status=status.HTTP_400_BAD_REQUEST, | ||
| ) |
There was a problem hiding this comment.
Bug: Attachment uploads do not validate file extensions or MIME types, allowing potentially dangerous files like HTML or SVG to be uploaded and served from the application media path.
Severity: MEDIUM
Suggested Fix
Validate uploaded attachment names and content against an explicit allowlist of safe extensions (such as standard image formats and PDFs). Alternatively, verify MIME types and ensure uploaded media files are served with a Content-Disposition: attachment header or hosted on an isolated, separate origin.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/authoring_ops.py#L269-L273
Potential issue: The attachment upload endpoint accepts any file extension matching
`ATTACHMENT_NAME_RE` without verifying the file extension or MIME content type against
an allowlist. Uploaded files like `.html` or `.svg` are saved using their original
extensions directly into media storage under `attachments/p{id}/{timestamp}-{name}`. If
media assets are served directly from the same origin without an overriding
`Content-Disposition: attachment` or strict Content Security Policy, executing scripts
contained in these uploads can lead to stored cross-site scripting (XSS).
Also affects:
backend/problems/authoring_ops.py:286~289
Wave F PR 8/9 — BACKLOG section I.
Items
problems/generator_dsl.py: a tiny declarative language (n = int(1, 100),s = str(3, 8),print {n} {q} {s}); parse errors pinpoint the offending line;generate(script, seed)is fully deterministic per (script, seed) across platforms.POST /problems/{id}/generate-tests/renders real TestCase rows (up to 50 per call, optional sample flag)./author/toolsUI runs it against a draft.Problem.variant_parent/variant_note: children point at the family ROOT (grandchildren flatten;variant_rootwalks with a cycle guard). Spawn copies the statement + samples into a DRAFT child with a per-variant note;GET /problems/{id}/variants/list/returns the whole family; UI on/author/tools.tests/<n>+tests/<n>.alayout is now accepted alongside the Kattis.in/.ansform, with a seen-set so packages carrying both forms import each test once. Both layouts tested, mixed-package case included.StatementAttachmentmodel: base64 upload onto media storage (sanitized names, 2 MB per file, 10 per statement, inline/download kinds, uploader attribution);GET/POST /problems/{id}/attachments/; UI list + upload on/author/tools.Gates (CI-identical commands, run in the worktree)
SECRET_KEY=x DB_PASSWORD=x python -m pytest -q --cov-fail-under=100→ 2104 passed, TOTAL 100% (new:problems/test_authoring_ops.py, 17 tests; generator_dsl.py + authoring_ops.py at 100%)npm run test773 passed ·npx tsc --noEmitclean ·npm run lint0/0 ·npm run buildgreen · new e2ee2e/wave-f-tools.spec.ts3 tests green