Skip to content

feat(authoring): visual wizard, statement linter, readability, sidecar report [wave F items 121/123/124/134] - #354

Open
Prekzursil wants to merge 4 commits into
mainfrom
wf-f/authoring-quality
Open

Prekzursil wants to merge 4 commits into
mainfrom
wf-f/authoring-quality

Conversation

@Prekzursil

Copy link
Copy Markdown
Owner

Wave F PR 7/9 — BACKLOG section I.

Items

  • 121 Visual authoring wizard — GET /problems/wizard/steps/ returns the 7-step checklist (basics → limits → samples → tests → reference solution → translation → review); with ?problem_id= each step probes live completion (statement set, limits set, ≥2 samples, ≥3 tests, reference solution present, RO mirror present, review submitted). /author/wizard page with per-step ticks, lint, readability and sidecar panels; nav link in the creator group.
  • 123 Statement linter — statement_quality.statement_lint: missing Input/Output/Constraints/Example sections, unbalanced $ math delimiters, >60-word runaway sentences, title-without-statement parity drift — per locale. POST /problems/{id}/statement-lint/ (author/verifier gate).
  • 134 Readability scoring — Flesch reading-ease adapted per locale (Romanian vowel-group syllable counting; English classic heuristic), 5 bands (very_easy→very_difficult), plus a bilingual consistency verdict (locales within 15 points). GET /problems/{id}/readability/.
  • 124 EN+translation sidecar authoring — translation_gaps: per-locale presence of every localizable field (title/statement/hints) powering the wizard's dual-pane sidecar panel. GET /problems/{id}/translation-status/.

Gates (CI-identical commands, run in the worktree)

  • backend: SECRET_KEY=x DB_PASSWORD=x python -m pytest -q --cov-fail-under=100 → 2109 passed, TOTAL 100% (new: problems/test_statement_quality.py, 22 tests)
  • frontend: npm run test 771 passed · npx tsc --noEmit clean · npm run lint 0/0 · npm run build green · new e2e e2e/wave-f-authoring.spec.ts 2 tests green
  • style: ruff 0.15.17 + biome 2.5.0 + oxlint 1.69.0 clean; 14 new i18n keys × 30 catalogs (parity green)
  • also fixes the classroom effort-board test to be fully wall-clock independent (pinned instants + widened window)

@Prekzursil
Prekzursil enabled auto-merge (squash) November 18, 2026 10:39
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-18T15:04:46.583766Z 0bb8076 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Comment on lines +186 to +193
text = _statement_text(problem, locale)
if not text:
continue
entry = readability_score(text, locale)
report["locales"][locale] = entry
scores.append(entry["score"])
if len(scores) == 2:
report["consistent"] = abs(scores[0] - scores[1]) <= 15

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Comparing readability scores across two locales raises a TypeError if one locale contains non-letter placeholders resulting in a None score.
Severity: MEDIUM

Suggested Fix

Filter out None values from scores or verify both scores are not None before calculating their difference in problem_readability.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/statement_quality.py#L186-L193

Potential issue: In `problem_readability`, non-empty statement strings containing no
alphabetical characters (such as placeholders, punctuation, or bare formulas) pass the
`if not text:` guard. `readability_score` subsequently returns `None` for the score and
appends `None` to `scores`. When both locales are present, `len(scores) == 2` evaluates
`abs(scores[0] - scores[1])`, raising a `TypeError: unsupported operand type(s) for -:
'NoneType' and 'float'` and resulting in an unhandled 500 error when fetching
readability metrics.

Did we get this right? 👍 / 👎 to inform future reviews.

Comment on lines +155 to +161
raw_id = request.query_params.get("problem_id")
if raw_id:
problem = Problem.objects.filter(pk=raw_id).first()
if problem is None:
return Response(
{"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Passing a non-numeric problem_id query parameter causes an unhandled ValueError in Problem.objects.filter.
Severity: LOW

Suggested Fix

Validate that raw_id consists of digits before querying Problem.objects.filter, returning an HTTP 400 or 404 response if invalid.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/statement_quality_views.py#L155-L161

Potential issue: In `WizardStepsView.get`, `request.query_params.get("problem_id")` is
retrieved without type validation and passed directly to
`Problem.objects.filter(pk=raw_id)`. If a non-numeric string is supplied, Django's ORM
raises a `ValueError` during queryset construction on integer primary keys, causing an
unhandled 500 server error for direct API requests.

Did we get this right? 👍 / 👎 to inform future reviews.

Comment on lines +48 to +58
for key in SECTION_KEYS:
if key == "examples":
if text and not any(word.lower() in lowered for word in EXAMPLE_WORDS):
warnings.append(
{
"code": f"missing_examples_{locale}",
"locale": locale,
"message": (
"The statement never mentions examples — add a "
"worked Example section."
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Statement quality checks lack Romanian equivalents for section headings, causing false-positive lint warnings on valid Romanian statements.
Severity: LOW

Suggested Fix

Extend the heading checks for input, output, and constraints to accept their localized Romanian equivalents (e.g., 'date de intrare', 'date de ieșire', 'restricții').

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/statement_quality.py#L48-L58

Potential issue: The statement quality checker evaluates `input`, `output`, and
`constraints` headings only against English terms across all locales. Romanian
statements following standard Romanian headings (such as 'Date de intrare', 'Date de
ieșire', and 'Restricții') fail substring matching, generating false-positive lint
warnings (`missing_input_ro`, `missing_output_ro`, `missing_constraints_ro`).

Did we get this right? 👍 / 👎 to inform future reviews.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 334a7c69ad

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +55 to +56
auth.user?.role === 'PROBLEM_CREATOR' ||
auth.user?.role === 'PROBLEM_VERIFIER' ||

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Accept the backend's creator and verifier role values

For real creator and verifier accounts, the backend serializes roles as CREATOR and VERIFIER (backend/users/models.py), and AuthContext preserves those values without normalization. This check accepts only the legacy long spellings, so both intended roles receive the creators-only message and never load the wizard. Accept the canonical short values as well; the current tests conceal the problem by mocking PROBLEM_CREATOR.

Useful? React with 👍 / 👎.

Comment on lines +157 to +158
problem = Problem.objects.filter(pk=raw_id).first()
if problem is None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Enforce problem visibility before returning authoring reports

A creator or teacher can supply another author's unpublished problem ID and retrieve its completion checklist, including reference-solution availability and review status. The other three new endpoints similarly expose lint, readability, and translation metadata using only the role gate. This bypasses the existing visibility boundary in ProblemViewSet.get_queryset, which restricts unpublished problems to their author and staff/admin/verifiers. Apply equivalent problem-scoped visibility checks to all four endpoints rather than querying unrestricted Problem.objects.

Useful? React with 👍 / 👎.

Comment on lines +191 to +193
scores.append(entry["score"])
if len(scores) == 2:
report["consistent"] = abs(scores[0] - scores[1]) <= 15

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Exclude unscorable locales from the consistency calculation

When both locale statements are nonempty but either contains no characters matched by the word regex, readability_score returns score: None, which is appended here and then subtracted. For example, statements containing 123 + 456 in both locales reproduce a TypeError, turning the readability endpoint into a 500. Such nonempty strings pass the existing localized-field validator. Calculate consistency only when both scores are numeric, leaving it null otherwise.

Useful? React with 👍 / 👎.

Comment on lines +62 to +65
if text and key not in lowered:
warnings.append(
{
"code": f"missing_{key}_{locale}",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Recognize Romanian section headings in Romanian statements

For a Romanian statement with normal headings such as Date de intrare, Date de ieșire, and Restricții, this check emits all three missing-section warnings because it searches exclusively for the English keys. Consequently, a correctly translated statement cannot get a clean lint report without inserting English headings. Use locale-specific heading aliases, as the examples check already attempts to do, and cover an actual Romanian statement in the clean-statement test.

Useful? React with 👍 / 👎.

Comment on lines +85 to +87
useEffect(() => {
if (isAuthor) load();
}, [isAuthor, load]);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Ignore responses belonging to a previously selected draft

When an authorized user changes the draft ID while a previous load is pending, each input change starts another load, but neither the effect nor the API wrapper cancels or invalidates earlier requests. A slower response for the previous draft can therefore overwrite the current draft's checklist and reports, or repopulate the report panels after the input is cleared. This can also mix one problem's checklist with another problem's lint results. Use an effect cleanup or request-generation guard so only the currently selected draft can update these states.

Useful? React with 👍 / 👎.

Comment on lines +77 to +87
if (readData) setReadability(readData);
if (transData) setTranslation(transData);
}
} catch (err) {
setError(err instanceof Error ? err.message : 'error');
}
}, [problemId]);

useEffect(() => {
if (isAuthor) load();
}, [isAuthor, load]);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Lack of request cancellation or race condition guards in load can cause stale data or outdated step checklists to overwrite current state.
Severity: MEDIUM

Suggested Fix

Add a cancellation flag (let cancelled = false) or an AbortController signal inside the useEffect running load, ensuring setState is only called if the request is still active. Clear previous steps state upon error and validate that problemId is a valid positive integer before triggering fetch requests.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/app/author/wizard/page.tsx#L62-L87

Potential issue: In `src/app/author/wizard/page.tsx`, the `load` effect triggers on
every keystroke as `problemId` changes without cancellation or request sequencing.
Out-of-order network responses can allow stale data from an earlier request to overwrite
state from the most recent request. Additionally, when `getWizardSteps` fails, the error
catch handler sets `error` but does not clear `steps`, leaving the previous problem's
checklist visible alongside the error. Non-numeric input also results in `NaN` requests
while rendering an empty generic checklist.

Comment on lines +95 to +99
def post(self, request, problem_id):
problem = Problem.objects.filter(pk=problem_id).first()
if problem is None:
return Response({"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND)
if not _authoring_gate(request.user):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: _authoring_gate performs a role-only check and lacks an ownership check, allowing problem creators and teachers to view authoring metadata for other authors' draft problems.
Severity: MEDIUM

Suggested Fix

Update _authoring_gate or the corresponding views to ensure non-staff/non-verifier users can only access problems where problem.author_id == request.user.pk (or approved problems), aligning with ProblemViewSet.get_queryset and TestStrengthView.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/statement_quality_views.py#L95-L99

Potential issue: The new statement quality and authoring endpoints rely on
`_authoring_gate` which only verifies whether a user holds a role like
`PROBLEM_CREATOR`, `TEACHER`, or `PROBLEM_VERIFIER`. The views fetch `Problem` objects
directly by primary key without checking `problem.author_id == request.user.pk` or
problem approval status. Consequently, any creator or teacher can access lint reports,
readability scores, translation statuses, and wizard completion steps for other authors'
draft or pending problems.

Also affects:

  • backend/problems/statement_quality_views.py:113~117
  • backend/problems/statement_quality_views.py:130~134
  • backend/problems/statement_quality_views.py:154~158

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4febc60acb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

if key == "samples":
return problem.test_cases.filter(is_sample=True).count() >= 2
if key == "tests":
return problem.test_cases.count() >= 3

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require hidden tests before marking their step complete

For a draft containing three sample cases and no hidden cases, this condition marks the “Hidden tests” step complete even though the advertised “full suite beyond the samples” is absent. Both the Samples and Hidden tests steps therefore show green for a sample-only problem, misleading authors about whether their judging suite is ready. Filter by is_sample=False when checking hidden-test completion so samples cannot satisfy this step.

Useful? React with 👍 / 👎.

problem = None
raw_id = request.query_params.get("problem_id")
if raw_id:
problem = Problem.objects.filter(pk=raw_id).first()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate the wizard problem ID before querying

For an authorized request with ?problem_id=1.5 or another non-integer value, Django's integer primary-key conversion raises an uncaught ValueError, producing a 500 instead of a validation response. This is also reachable through the wizard's unrestricted text input: entering 1.5 preserves the fractional value through Number(problemId) and getWizardSteps. Validate the query parameter as an integer before passing it to the ORM and return a controlled 400 for malformed values.

Useful? React with 👍 / 👎.

Comment on lines +175 to +185
statement = (
problem.statement_i18n.get("en", "")
if isinstance(problem.statement_i18n, dict)
else ""
)
return bool(statement.strip())
if key == "limits":
return problem.default_time_limit_ms > 0 and problem.default_memory_limit_kb > 0
if key == "samples":
return problem.test_cases.filter(is_sample=True).count() >= 2
if key == "tests":

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: _step_complete calls .strip() on .get() results without handling explicit None values in statement_i18n, causing an AttributeError when a locale key is mapped to null.
Severity: LOW

Suggested Fix

Safely fall back to an empty string when the retrieved value is falsey or None, for example using (problem.statement_i18n.get("en") or "").strip() and (problem.statement_i18n.get("ro") or "").strip(), consistent with _statement_text.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: backend/problems/statement_quality_views.py#L175-L185

Potential issue: In `_step_complete`, `problem.statement_i18n.get('en', '')` and
`problem.statement_i18n.get('ro', '')` are called before invoking `.strip()`. If
`statement_i18n` contains an explicit `null` value for a locale (which can occur through
direct ORM manipulations or Django admin edits where raw JSON is saved), `.get(...)`
returns `None` instead of the default empty string. Calling `.strip()` on `None` raises
an `AttributeError` during wizard step checks.

…r report [wave F items 121/123/124/134]

- item 121: /problems/wizard/steps/ — the 7-step authoring checklist with
  live per-step completion probes (?problem_id=); /author/wizard page
- item 123: statement_quality.statement_lint — missing sections, unbalanced
  $-math, runaway sentences, title/statement parity drift (bilingual);
  POST /problems/{id}/statement-lint/
- item 134: Flesch reading-ease per locale (RO vowel-group syllables) +
  bands + bilingual consistency; GET /problems/{id}/readability/
- item 124: translation_gaps sidecar report (per-locale field presence);
  GET /problems/{id}/translation-status/
- classroom effort-board test made fully wall-clock independent
- BACKLOG statuses for 121/123/124/134
@Prekzursil
Prekzursil force-pushed the wf-f/authoring-quality branch from 6bcc7c2 to 0bb8076 Compare November 18, 2026 15:02
Comment on lines +53 to +60
const isAuthor = useMemo(
() =>
auth.user?.role === 'PROBLEM_CREATOR' ||
auth.user?.role === 'PROBLEM_VERIFIER' ||
auth.user?.role === 'TEACHER' ||
auth.user?.role === 'ADMIN',
[auth.user?.role],
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: isAuthor in the authoring wizard checks only legacy role strings ('PROBLEM_CREATOR', 'PROBLEM_VERIFIER'), locking out users with backend roles 'CREATOR' and 'VERIFIER'.
Severity: HIGH

Suggested Fix

Update the role check in src/app/author/wizard/page.tsx to accept both short and legacy role values (i.e., include 'CREATOR' and 'VERIFIER' alongside 'PROBLEM_CREATOR' and 'PROBLEM_VERIFIER'), matching patterns used in other pages like contests/create.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/app/author/wizard/page.tsx#L53-L60

Potential issue: In `src/app/author/wizard/page.tsx`, the role check for `isAuthor`
expects legacy role names like `'PROBLEM_CREATOR'` and `'PROBLEM_VERIFIER'`. However,
the backend stores and serializes roles as `'CREATOR'` and `'VERIFIER'`. Because the
wizard page does not accept the short role names, valid creators and verifiers fail the
role check and are locked out with the message 'The wizard is for problem creators and
verifiers.'

Also affects:

  • src/components/layout/Navbar.tsx:117~126

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant