Skip to content

feat(authoring): testcase generator DSL, variant families, Polygon legacy import, statement attachments [wave F items 122/126/127/128] - #355

Open
Prekzursil wants to merge 2 commits into
mainfrom
wf-f/generators
Open

Prekzursil wants to merge 2 commits into
mainfrom
wf-f/generators

Conversation

@Prekzursil

Copy link
Copy Markdown
Owner

Wave F PR 8/9 — BACKLOG section I.

Items

  • 122 Testcase generator DSL — problems/generator_dsl.py: a tiny declarative language (n = int(1, 100), s = str(3, 8), print {n} {q} {s}); parse errors pinpoint the offending line; generate(script, seed) is fully deterministic per (script, seed) across platforms. POST /problems/{id}/generate-tests/ renders real TestCase rows (up to 50 per call, optional sample flag). /author/tools UI runs it against a draft.
  • 126 Variant families — Problem.variant_parent/variant_note: children point at the family ROOT (grandchildren flatten; variant_root walks with a cycle guard). Spawn copies the statement + samples into a DRAFT child with a per-variant note; GET /problems/{id}/variants/list/ returns the whole family; UI on /author/tools.
  • 127 Polygon-format import — completes the Wave C importer: the legacy extensionless Polygon tests/<n> + tests/<n>.a layout is now accepted alongside the Kattis .in/.ans form, with a seen-set so packages carrying both forms import each test once. Both layouts tested, mixed-package case included.
  • 128 Statement attachments — StatementAttachment model: base64 upload onto media storage (sanitized names, 2 MB per file, 10 per statement, inline/download kinds, uploader attribution); GET/POST /problems/{id}/attachments/; UI list + upload on /author/tools.

Gates (CI-identical commands, run in the worktree)

  • backend: SECRET_KEY=x DB_PASSWORD=x python -m pytest -q --cov-fail-under=100 → 2104 passed, TOTAL 100% (new: problems/test_authoring_ops.py, 17 tests; generator_dsl.py + authoring_ops.py at 100%)
  • frontend: npm run test 773 passed · npx tsc --noEmit clean · npm run lint 0/0 · npm run build green · new e2e e2e/wave-f-tools.spec.ts 3 tests green
  • style: ruff 0.15.17 + biome 2.5.0 + oxlint 1.69.0 clean; 16 new i18n keys × 30 catalogs (parity green)
  • ports the wall-clock-independent effort-board test to this branch

@Prekzursil
Prekzursil enabled auto-merge (squash) November 18, 2026 13:50
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-18T15:04:30.028587Z 1bbc235 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Comment on lines +49 to +55
if not template:
raise ValueError("the script needs a print template line")
# every placeholder must be declared
for field in re.findall(r"\{(\w+)\}", template):
if field not in variables:
raise ValueError(f"template references undeclared field '{field}'")
return {"vars": variables, "template": template}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Malformed print templates pass parse_dsl validation but trigger unhandled exceptions during str.format in generate(), resulting in HTTP 500 errors.
Severity: MEDIUM

Suggested Fix

Validate the format string during parse_dsl by rendering it with dummy data or using string.Formatter().parse(template). Alternatively, wrap calls to generate() in GenerateTestsView in a try/except (ValueError, IndexError, AttributeError) block and return a 400 response.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/generator_dsl.py#L49-L55

Potential issue: `parse_dsl` validates placeholders with `re.findall(r"\{(\w+)\}",
template)`, which only matches bare `{name}` placeholders and ignores other format
specifications. When `generate()` calls `str.format(**values)`, malformed or unexpected
format syntax—such as unclosed braces, positional braces `{}`, or invalid format
specifiers—causes `IndexError`, `ValueError`, or `AttributeError`.
`GenerateTestsView.post` only catches `ValueError` during `parse_dsl`, leaving
`generate(...)` calls unguarded against these exceptions and returning an unhandled 500
error instead of a validation error.

Also affects:

  • backend/problems/authoring_ops.py:77~84

Did we get this right? 👍 / 👎 to inform future reviews.

Comment on lines +65 to +73
def post(self, request, problem_id):
problem = Problem.objects.filter(pk=problem_id).first()
if problem is None:
return Response({"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND)
if not _authoring_gate(request.user):
return Response(
{"detail": "Authors and verifiers only."},
status=status.HTTP_403_FORBIDDEN,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Authoring endpoints lack problem ownership and status validation, allowing unauthorized cross-author test generation, variant spawning, and attachment access.
Severity: HIGH

Suggested Fix

Enforce ProblemObjectPermissions and _guard_contest_pin on write endpoints like GenerateTestsView, VariantSpawnView, and AttachmentView. Scope queryset lookups to authorized or approved problems in VariantListView.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/authoring_ops.py#L65-L73

Potential issue: Authoring endpoints rely solely on role checks in `_authoring_gate`
rather than verifying problem ownership, status, or contest pinning. Consequently, users
with authoring roles can append tests to any problem, including approved or
contest-pinned problems, spawn variants from other authors' private or draft problems,
and list draft variant details or attachments across authors without proper
authorization checks.

Also affects:

  • backend/problems/authoring_ops.py:114~146
  • backend/problems/authoring_ops.py:173~177
  • backend/problems/authoring_ops.py:216~265

Did we get this right? 👍 / 👎 to inform future reviews.

Comment on lines +20 to +22
_LINE_RE = re.compile(
r"^\s*(\w+)\s*=\s*(int|str)\s*\(\s*(-?\d+)\s*,\s*(-?\d+)\s*\)\s*$"
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: DSL str() generator does not enforce an upper bound on length, allowing excessive CPU and memory consumption during test generation.
Severity: MEDIUM

Suggested Fix

Enforce a reasonable maximum limit on hi (e.g., 100_000) for str type generators within parse_dsl in backend/problems/generator_dsl.py.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/generator_dsl.py#L20-L22

Potential issue: The generator DSL accepts unbounded integer bounds for string
generators `str(lo, hi)`. When building a string of length `rng.randint(lo, hi)`,
`generate()` executes `"".join(rng.choice(alphabet) for _ in range(length))` without
capping `hi`. Large values such as `10**8` or `10**9` lead to extreme CPU consumption
and multi-gigabyte memory allocations across up to 50 iterations, potentially exhausting
worker resources.

Also affects:

  • backend/problems/authoring_ops.py:83~93

Did we get this right? 👍 / 👎 to inform future reviews.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a96695d30

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +66 to +69
problem = Problem.objects.filter(pk=problem_id).first()
if problem is None:
return Response({"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND)
if not _authoring_gate(request.user):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Enforce problem-level authorization in authoring endpoints

Any problem creator or teacher can supply another author's problem ID and pass _authoring_gate. Consequently, GenerateTestsView can append scored tests with empty expected outputs to someone else's approved problem, and VariantSpawnView can copy another author's private statement into a draft owned by the caller. Unlike ProblemViewSet.get_queryset() and ProblemObjectPermissions, these endpoints enforce neither visibility nor ownership/status restrictions. Apply the corresponding object-level access checks before reading or modifying the selected problem, including attachment operations.

Useful? React with 👍 / 👎.

Comment on lines +81 to +85
base_order = problem.test_cases.count()
created = 0
for offset in range(data["count"]):
rendered = generate(data["dsl"], data["seed"] + offset)
TestCase.objects.create(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the contest pin before generating test cases

For a problem in an upcoming or running contest, this loop creates immediately judge-visible test cases without calling _guard_contest_pin. The existing TestCaseViewSet.perform_create, update, and delete paths explicitly invoke that guard to freeze the test suite. Even a verifier who is otherwise authorized to edit the problem can now bypass that freeze, changing grading during the contest; the generated empty expected outputs make this especially disruptive. Invoke the same contest guard before creating any rows.

Useful? React with 👍 / 👎.

Comment on lines +67 to +69
length = rng.randint(lo, hi)
alphabet = "abcdefghijklmnopqrstuvwxyz"
values[name] = "".join(rng.choice(alphabet) for _ in range(length))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bound generated string sizes before executing the DSL

A valid script such as s = str(1000000000, 1000000000)\nprint {s} passes parsing and the endpoint's 20,000-character script limit, then performs a billion random choices and builds a huge string inside the web worker. With up to 50 generations per request, an author can exhaust worker memory or monopolize request processing using a tiny payload. Enforce per-variable and aggregate generated-output limits during validation, before generation begins.

Useful? React with 👍 / 👎.

Comment on lines +138 to +144
statement_i18n=dict(parent.statement_i18n),
status=Problem.ProblemStatus.DRAFT,
author=request.user,
difficulty=parent.difficulty,
default_time_limit_ms=parent.default_time_limit_ms,
default_memory_limit_kb=parent.default_memory_limit_kb,
variant_parent=root,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Carry the import permission ledger into spawned variants

When the parent is imported content, the child copies its statement but leaves import_source unset. Both ProblemViewSet.submit_for_approval and approve_problem enforce publication rights only when problem.import_source_id is present, so spawning a variant removes the ledger check and allows the copied material to proceed through approval even if the source permission is revoked or expires. Preserve the parent's import-source association on the child so variants remain subject to the same publication restrictions.

Useful? React with 👍 / 👎.

Comment on lines +110 to +112
try {
const response = await ProblemService.uploadAttachment(Number(problemId), fileName);
setMessage(t('attachment_uploaded', 'Uploaded {{name}}', { name: response.name }));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Upload actual file contents instead of the default placeholder

The author-tools Upload action passes only a filename, so ProblemService.uploadAttachment always uses its default contentBase64 = 'eA==', which decodes to the single byte x. There is no file selector or content-reading path in this page. Thus uploading figure.png reports success but stores a one-byte non-image rather than the author's attachment. Add file selection, read the selected bytes, and pass their base64 encoding explicitly instead of silently substituting placeholder content.

Useful? React with 👍 / 👎.

Comment on lines +51 to +55
# every placeholder must be declared
for field in re.findall(r"\{(\w+)\}", template):
if field not in variables:
raise ValueError(f"template references undeclared field '{field}'")
return {"vars": variables, "template": template}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate the complete print template before generation

The placeholder regex accepts malformed or unsupported format strings: print { and print {missing!s} both pass parse_dsl, but generate subsequently raises ValueError or KeyError in str.format. GenerateTestsView catches errors only around its initial parse, so these ordinary DSL mistakes return HTTP 500 instead of the promised validation response. Parse and validate the complete format syntax, rejecting unsupported fields and unmatched braces before creating test cases.

Useful? React with 👍 / 👎.

class GenerateTestsView(APIView):
"""Item 122 — render the generator DSL into real TestCase rows."""

permission_classes = [permissions.IsAuthenticated]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Apply the privileged two-factor gate to authoring writes

When TWO_FACTOR_ENFORCE is enabled, these new write endpoints still require only authentication and the role check. A privileged account without enrolled TOTP can therefore create problem variants and mutate test suites through these routes, although ProblemViewSet.get_permissions() applies PrivilegedRoleRequires2FA to content creation and editing in that configuration. Compose the same permission into the new authoring write views so they do not bypass the enforced content-supply-chain restriction.

Useful? React with 👍 / 👎.

Comment on lines +141 to +145
difficulty=parent.difficulty,
default_time_limit_ms=parent.default_time_limit_ms,
default_memory_limit_kb=parent.default_memory_limit_kb,
variant_parent=root,
variant_note=note,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the parent's judging configuration in variants

Spawning a variant of an interactive, output-only, floating-point, or custom-checker problem copies its statement and samples but silently resets its judging configuration to model defaults: problem_type=NORMAL and exact line comparison, with no checker or interactor. In submissions/tasks.py, those fields select the execution and comparison branches, so the child no longer implements the task described by its copied statement; for example, a valid alternative answer to a custom-checker task is rejected against the single stored sample answer. Copy the parent's problem type and associated judge/comparison configuration alongside its time and memory limits.

Useful? React with 👍 / 👎.

try:
parse_dsl(data["dsl"])
except ValueError as exc:
return Response({"dsl": str(exc)}, status=status.HTTP_400_BAD_REQUEST)
Comment on lines +168 to +177
class VariantListView(APIView):
"""Item 126 — the family tree for one problem (root + siblings)."""

permission_classes = [permissions.AllowAny]

def get(self, request, problem_id):
problem = Problem.objects.filter(pk=problem_id).first()
if problem is None:
return Response({"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND)
root = problem.variant_root

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: VariantListView allows unauthenticated access and omits status/visibility checks, exposing unpublished draft problem metadata to anonymous users.
Severity: MEDIUM

Suggested Fix

Enforce authentication and author permissions (e.g. permissions.IsAuthenticated and _authoring_gate), or filter the variant family queryset using the same visibility rules applied in ProblemViewSet.get_queryset.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/authoring_ops.py#L168-L177

Potential issue: `VariantListView` uses `permission_classes = [permissions.AllowAny]`
and queries problems across the family (`Q(pk=root.pk) | Q(variant_parent=root)`)
without filtering by `status=APPROVED` or reusing `ProblemViewSet.get_queryset`
visibility rules. As a result, unauthenticated callers can list draft and pending
problems, leaking titles, variant notes, statuses, and IDs of unpublished problems.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: affeaef5de

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +178 to +180
family = Problem.objects.filter(
dj_models.Q(pk=root.pk) | dj_models.Q(variant_parent=root)
).order_by("id")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Filter variant families by the caller's problem visibility

When a family contains draft or private problems, this AllowAny endpoint exposes their IDs, titles, parameter notes, and statuses to anonymous callers. Querying an approved root also reveals its unpublished children, so checking only the requested problem would not prevent the disclosure. Unlike ProblemViewSet.get_queryset() in backend/problems/views.py, neither query applies visibility restrictions. Apply the same visibility policy to both the requested problem and the returned family members.

Useful? React with 👍 / 👎.

…gacy import, statement attachments [wave F items 122/126/127/128]

- item 122: problems/generator_dsl.py — declarative int/str variable
  declarations + print template; parse errors pinpoint the line;
  generate() is deterministic per (script, seed);
  POST /problems/{id}/generate-tests/ renders real TestCase rows
  (inputs first; expected outputs flow through the existing
  expected-output tool / reference solution)
- item 126: Problem.variant_parent/variant_note — children point at the
  family ROOT (grandchildren flatten to the root); spawn copies statement
  + samples into a DRAFT child; GET /problems/{id}/variants/list/ tree;
  /author/tools UI
- item 127: the package importer now also accepts the legacy Polygon
  extensionless tests/<n> + tests/<n>.a layout (Kattis .in/.ans stays
  primary; seen-set dedup when both forms exist)
- item 128: StatementAttachment — base64 upload onto media storage with a
  2MB/10-file cap and sanitized names, inline/download kinds;
  GET/POST /problems/{id}/attachments/
- classroom effort-board test made wall-clock independent on this branch
- BACKLOG statuses for 122/126/127/128
Comment on lines +269 to +273
if not ATTACHMENT_NAME_RE.match(data["name"]):
return Response(
{"name": "Letters, digits, dash, underscore, dot and space only."},
status=status.HTTP_400_BAD_REQUEST,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Attachment uploads do not validate file extensions or MIME types, allowing potentially dangerous files like HTML or SVG to be uploaded and served from the application media path.
Severity: MEDIUM

Suggested Fix

Validate uploaded attachment names and content against an explicit allowlist of safe extensions (such as standard image formats and PDFs). Alternatively, verify MIME types and ensure uploaded media files are served with a Content-Disposition: attachment header or hosted on an isolated, separate origin.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/authoring_ops.py#L269-L273

Potential issue: The attachment upload endpoint accepts any file extension matching
`ATTACHMENT_NAME_RE` without verifying the file extension or MIME content type against
an allowlist. Uploaded files like `.html` or `.svg` are saved using their original
extensions directly into media storage under `attachments/p{id}/{timestamp}-{name}`. If
media assets are served directly from the same origin without an overriding
`Content-Disposition: attachment` or strict Content Security Policy, executing scripts
contained in these uploads can lead to stored cross-site scripting (XSS).

Also affects:

  • backend/problems/authoring_ops.py:286~289

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants