Repository navigation
feat(authoring): visual wizard, statement linter, readability, sidecar report [wave F items 121/123/124/134] - #354
Prekzursil wants to merge 4 commits into
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
| text = _statement_text(problem, locale) | ||
| if not text: | ||
| continue | ||
| entry = readability_score(text, locale) | ||
| report["locales"][locale] = entry | ||
| scores.append(entry["score"]) | ||
| if len(scores) == 2: | ||
| report["consistent"] = abs(scores[0] - scores[1]) <= 15 |
There was a problem hiding this comment.
Bug: Comparing readability scores across two locales raises a TypeError if one locale contains non-letter placeholders resulting in a None score.
Severity: MEDIUM
Suggested Fix
Filter out None values from scores or verify both scores are not None before calculating their difference in problem_readability.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/statement_quality.py#L186-L193
Potential issue: In `problem_readability`, non-empty statement strings containing no
alphabetical characters (such as placeholders, punctuation, or bare formulas) pass the
`if not text:` guard. `readability_score` subsequently returns `None` for the score and
appends `None` to `scores`. When both locales are present, `len(scores) == 2` evaluates
`abs(scores[0] - scores[1])`, raising a `TypeError: unsupported operand type(s) for -:
'NoneType' and 'float'` and resulting in an unhandled 500 error when fetching
readability metrics.
Did we get this right? 👍 / 👎 to inform future reviews.
| raw_id = request.query_params.get("problem_id") | ||
| if raw_id: | ||
| problem = Problem.objects.filter(pk=raw_id).first() | ||
| if problem is None: | ||
| return Response( | ||
| {"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND | ||
| ) |
There was a problem hiding this comment.
Bug: Passing a non-numeric problem_id query parameter causes an unhandled ValueError in Problem.objects.filter.
Severity: LOW
Suggested Fix
Validate that raw_id consists of digits before querying Problem.objects.filter, returning an HTTP 400 or 404 response if invalid.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/statement_quality_views.py#L155-L161
Potential issue: In `WizardStepsView.get`, `request.query_params.get("problem_id")` is
retrieved without type validation and passed directly to
`Problem.objects.filter(pk=raw_id)`. If a non-numeric string is supplied, Django's ORM
raises a `ValueError` during queryset construction on integer primary keys, causing an
unhandled 500 server error for direct API requests.
Did we get this right? 👍 / 👎 to inform future reviews.
| for key in SECTION_KEYS: | ||
| if key == "examples": | ||
| if text and not any(word.lower() in lowered for word in EXAMPLE_WORDS): | ||
| warnings.append( | ||
| { | ||
| "code": f"missing_examples_{locale}", | ||
| "locale": locale, | ||
| "message": ( | ||
| "The statement never mentions examples — add a " | ||
| "worked Example section." | ||
| ), |
There was a problem hiding this comment.
Bug: Statement quality checks lack Romanian equivalents for section headings, causing false-positive lint warnings on valid Romanian statements.
Severity: LOW
Suggested Fix
Extend the heading checks for input, output, and constraints to accept their localized Romanian equivalents (e.g., 'date de intrare', 'date de ieșire', 'restricții').
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/statement_quality.py#L48-L58
Potential issue: The statement quality checker evaluates `input`, `output`, and
`constraints` headings only against English terms across all locales. Romanian
statements following standard Romanian headings (such as 'Date de intrare', 'Date de
ieșire', and 'Restricții') fail substring matching, generating false-positive lint
warnings (`missing_input_ro`, `missing_output_ro`, `missing_constraints_ro`).
Did we get this right? 👍 / 👎 to inform future reviews.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 334a7c69ad
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| auth.user?.role === 'PROBLEM_CREATOR' || | ||
| auth.user?.role === 'PROBLEM_VERIFIER' || |
There was a problem hiding this comment.
Accept the backend's creator and verifier role values
For real creator and verifier accounts, the backend serializes roles as CREATOR and VERIFIER (backend/users/models.py), and AuthContext preserves those values without normalization. This check accepts only the legacy long spellings, so both intended roles receive the creators-only message and never load the wizard. Accept the canonical short values as well; the current tests conceal the problem by mocking PROBLEM_CREATOR.
Useful? React with 👍 / 👎.
| problem = Problem.objects.filter(pk=raw_id).first() | ||
| if problem is None: |
There was a problem hiding this comment.
Enforce problem visibility before returning authoring reports
A creator or teacher can supply another author's unpublished problem ID and retrieve its completion checklist, including reference-solution availability and review status. The other three new endpoints similarly expose lint, readability, and translation metadata using only the role gate. This bypasses the existing visibility boundary in ProblemViewSet.get_queryset, which restricts unpublished problems to their author and staff/admin/verifiers. Apply equivalent problem-scoped visibility checks to all four endpoints rather than querying unrestricted Problem.objects.
Useful? React with 👍 / 👎.
| scores.append(entry["score"]) | ||
| if len(scores) == 2: | ||
| report["consistent"] = abs(scores[0] - scores[1]) <= 15 |
There was a problem hiding this comment.
Exclude unscorable locales from the consistency calculation
When both locale statements are nonempty but either contains no characters matched by the word regex, readability_score returns score: None, which is appended here and then subtracted. For example, statements containing 123 + 456 in both locales reproduce a TypeError, turning the readability endpoint into a 500. Such nonempty strings pass the existing localized-field validator. Calculate consistency only when both scores are numeric, leaving it null otherwise.
Useful? React with 👍 / 👎.
| if text and key not in lowered: | ||
| warnings.append( | ||
| { | ||
| "code": f"missing_{key}_{locale}", |
There was a problem hiding this comment.
Recognize Romanian section headings in Romanian statements
For a Romanian statement with normal headings such as Date de intrare, Date de ieșire, and Restricții, this check emits all three missing-section warnings because it searches exclusively for the English keys. Consequently, a correctly translated statement cannot get a clean lint report without inserting English headings. Use locale-specific heading aliases, as the examples check already attempts to do, and cover an actual Romanian statement in the clean-statement test.
Useful? React with 👍 / 👎.
| useEffect(() => { | ||
| if (isAuthor) load(); | ||
| }, [isAuthor, load]); |
There was a problem hiding this comment.
Ignore responses belonging to a previously selected draft
When an authorized user changes the draft ID while a previous load is pending, each input change starts another load, but neither the effect nor the API wrapper cancels or invalidates earlier requests. A slower response for the previous draft can therefore overwrite the current draft's checklist and reports, or repopulate the report panels after the input is cleared. This can also mix one problem's checklist with another problem's lint results. Use an effect cleanup or request-generation guard so only the currently selected draft can update these states.
Useful? React with 👍 / 👎.
| if (readData) setReadability(readData); | ||
| if (transData) setTranslation(transData); | ||
| } | ||
| } catch (err) { | ||
| setError(err instanceof Error ? err.message : 'error'); | ||
| } | ||
| }, [problemId]); | ||
|
|
||
| useEffect(() => { | ||
| if (isAuthor) load(); | ||
| }, [isAuthor, load]); |
There was a problem hiding this comment.
Bug: Lack of request cancellation or race condition guards in load can cause stale data or outdated step checklists to overwrite current state.
Severity: MEDIUM
Suggested Fix
Add a cancellation flag (let cancelled = false) or an AbortController signal inside the useEffect running load, ensuring setState is only called if the request is still active. Clear previous steps state upon error and validate that problemId is a valid positive integer before triggering fetch requests.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: src/app/author/wizard/page.tsx#L62-L87
Potential issue: In `src/app/author/wizard/page.tsx`, the `load` effect triggers on
every keystroke as `problemId` changes without cancellation or request sequencing.
Out-of-order network responses can allow stale data from an earlier request to overwrite
state from the most recent request. Additionally, when `getWizardSteps` fails, the error
catch handler sets `error` but does not clear `steps`, leaving the previous problem's
checklist visible alongside the error. Non-numeric input also results in `NaN` requests
while rendering an empty generic checklist.
| def post(self, request, problem_id): | ||
| problem = Problem.objects.filter(pk=problem_id).first() | ||
| if problem is None: | ||
| return Response({"detail": "Not found."}, status=status.HTTP_404_NOT_FOUND) | ||
| if not _authoring_gate(request.user): |
There was a problem hiding this comment.
Bug: _authoring_gate performs a role-only check and lacks an ownership check, allowing problem creators and teachers to view authoring metadata for other authors' draft problems.
Severity: MEDIUM
Suggested Fix
Update _authoring_gate or the corresponding views to ensure non-staff/non-verifier users can only access problems where problem.author_id == request.user.pk (or approved problems), aligning with ProblemViewSet.get_queryset and TestStrengthView.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/statement_quality_views.py#L95-L99
Potential issue: The new statement quality and authoring endpoints rely on
`_authoring_gate` which only verifies whether a user holds a role like
`PROBLEM_CREATOR`, `TEACHER`, or `PROBLEM_VERIFIER`. The views fetch `Problem` objects
directly by primary key without checking `problem.author_id == request.user.pk` or
problem approval status. Consequently, any creator or teacher can access lint reports,
readability scores, translation statuses, and wizard completion steps for other authors'
draft or pending problems.
Also affects:
backend/problems/statement_quality_views.py:113~117backend/problems/statement_quality_views.py:130~134backend/problems/statement_quality_views.py:154~158
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4febc60acb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if key == "samples": | ||
| return problem.test_cases.filter(is_sample=True).count() >= 2 | ||
| if key == "tests": | ||
| return problem.test_cases.count() >= 3 |
There was a problem hiding this comment.
Require hidden tests before marking their step complete
For a draft containing three sample cases and no hidden cases, this condition marks the “Hidden tests” step complete even though the advertised “full suite beyond the samples” is absent. Both the Samples and Hidden tests steps therefore show green for a sample-only problem, misleading authors about whether their judging suite is ready. Filter by is_sample=False when checking hidden-test completion so samples cannot satisfy this step.
Useful? React with 👍 / 👎.
| problem = None | ||
| raw_id = request.query_params.get("problem_id") | ||
| if raw_id: | ||
| problem = Problem.objects.filter(pk=raw_id).first() |
There was a problem hiding this comment.
Validate the wizard problem ID before querying
For an authorized request with ?problem_id=1.5 or another non-integer value, Django's integer primary-key conversion raises an uncaught ValueError, producing a 500 instead of a validation response. This is also reachable through the wizard's unrestricted text input: entering 1.5 preserves the fractional value through Number(problemId) and getWizardSteps. Validate the query parameter as an integer before passing it to the ORM and return a controlled 400 for malformed values.
Useful? React with 👍 / 👎.
| statement = ( | ||
| problem.statement_i18n.get("en", "") | ||
| if isinstance(problem.statement_i18n, dict) | ||
| else "" | ||
| ) | ||
| return bool(statement.strip()) | ||
| if key == "limits": | ||
| return problem.default_time_limit_ms > 0 and problem.default_memory_limit_kb > 0 | ||
| if key == "samples": | ||
| return problem.test_cases.filter(is_sample=True).count() >= 2 | ||
| if key == "tests": |
There was a problem hiding this comment.
Bug: _step_complete calls .strip() on .get() results without handling explicit None values in statement_i18n, causing an AttributeError when a locale key is mapped to null.
Severity: LOW
Suggested Fix
Safely fall back to an empty string when the retrieved value is falsey or None, for example using (problem.statement_i18n.get("en") or "").strip() and (problem.statement_i18n.get("ro") or "").strip(), consistent with _statement_text.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: backend/problems/statement_quality_views.py#L175-L185
Potential issue: In `_step_complete`, `problem.statement_i18n.get('en', '')` and
`problem.statement_i18n.get('ro', '')` are called before invoking `.strip()`. If
`statement_i18n` contains an explicit `null` value for a locale (which can occur through
direct ORM manipulations or Django admin edits where raw JSON is saved), `.get(...)`
returns `None` instead of the default empty string. Calling `.strip()` on `None` raises
an `AttributeError` during wizard step checks.
…r report [wave F items 121/123/124/134]
- item 121: /problems/wizard/steps/ — the 7-step authoring checklist with
live per-step completion probes (?problem_id=); /author/wizard page
- item 123: statement_quality.statement_lint — missing sections, unbalanced
$-math, runaway sentences, title/statement parity drift (bilingual);
POST /problems/{id}/statement-lint/
- item 134: Flesch reading-ease per locale (RO vowel-group syllables) +
bands + bilingual consistency; GET /problems/{id}/readability/
- item 124: translation_gaps sidecar report (per-locale field presence);
GET /problems/{id}/translation-status/
- classroom effort-board test made fully wall-clock independent
- BACKLOG statuses for 121/123/124/134
6bcc7c2 to
0bb8076
Compare
| const isAuthor = useMemo( | ||
| () => | ||
| auth.user?.role === 'PROBLEM_CREATOR' || | ||
| auth.user?.role === 'PROBLEM_VERIFIER' || | ||
| auth.user?.role === 'TEACHER' || | ||
| auth.user?.role === 'ADMIN', | ||
| [auth.user?.role], | ||
| ); |
There was a problem hiding this comment.
Bug: isAuthor in the authoring wizard checks only legacy role strings ('PROBLEM_CREATOR', 'PROBLEM_VERIFIER'), locking out users with backend roles 'CREATOR' and 'VERIFIER'.
Severity: HIGH
Suggested Fix
Update the role check in src/app/author/wizard/page.tsx to accept both short and legacy role values (i.e., include 'CREATOR' and 'VERIFIER' alongside 'PROBLEM_CREATOR' and 'PROBLEM_VERIFIER'), matching patterns used in other pages like contests/create.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: src/app/author/wizard/page.tsx#L53-L60
Potential issue: In `src/app/author/wizard/page.tsx`, the role check for `isAuthor`
expects legacy role names like `'PROBLEM_CREATOR'` and `'PROBLEM_VERIFIER'`. However,
the backend stores and serializes roles as `'CREATOR'` and `'VERIFIER'`. Because the
wizard page does not accept the short role names, valid creators and verifiers fail the
role check and are locked out with the message 'The wizard is for problem creators and
verifiers.'
Also affects:
src/components/layout/Navbar.tsx:117~126
Wave F PR 7/9 — BACKLOG section I.
Items
GET /problems/wizard/steps/returns the 7-step checklist (basics → limits → samples → tests → reference solution → translation → review); with?problem_id=each step probes live completion (statement set, limits set, ≥2 samples, ≥3 tests, reference solution present, RO mirror present, review submitted)./author/wizardpage with per-step ticks, lint, readability and sidecar panels; nav link in the creator group.statement_quality.statement_lint: missing Input/Output/Constraints/Example sections, unbalanced$math delimiters, >60-word runaway sentences, title-without-statement parity drift — per locale.POST /problems/{id}/statement-lint/(author/verifier gate).GET /problems/{id}/readability/.translation_gaps: per-locale presence of every localizable field (title/statement/hints) powering the wizard's dual-pane sidecar panel.GET /problems/{id}/translation-status/.Gates (CI-identical commands, run in the worktree)
SECRET_KEY=x DB_PASSWORD=x python -m pytest -q --cov-fail-under=100→ 2109 passed, TOTAL 100% (new:problems/test_statement_quality.py, 22 tests)npm run test771 passed ·npx tsc --noEmitclean ·npm run lint0/0 ·npm run buildgreen · new e2ee2e/wave-f-authoring.spec.ts2 tests green