Skip to content

GH Issue 1720: Test saveBatch and saveRuns transform callbacks during folder delete - #3265

Merged
labkey-jeckels merged 8 commits into
developfrom
fb_batch_save_transforms_1720
Oct 29, 2026
Merged

labkey-jeckels merged 8 commits into
developfrom
fb_batch_save_transforms_1720

Conversation

@labkey-jeckels

Copy link
Copy Markdown
Contributor

Rationale

Adds test coverage for GH Issue 1720: saving assay batches or runs through the API must not hold a database transaction while transform scripts run.

Related Pull Requests

Changes

  • saveBatch and saveRuns variants of the folder-delete callback test
  • A batch whose second run's script fails saves nothing and cleans up the scripts' working directories
  • Updating an existing run keeps its results when another run in the request fails, and replaces them when the save succeeds

Also makes the existing-run update test prove its own transform ran.
@labkey-jeckels
labkey-jeckels marked this pull request as ready for review October 23, 2026 15:00
@labkey-jeckels labkey-jeckels self-assigned this Oct 23, 2026
Run run = new Run();
run.setProperties(Map.of(SLEEP_SECONDS, sleepSeconds, CALLBACK_STATUS, callbackStatus));
run.setResultData(List.of(Map.of("ParticipantID", participantId, "VisitID", 1)));
return run;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider supporting material and data inputs/outputs for completeness.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Rather than widening callbackRun, the tests set inputs and outputs on the Run it returns. Added a save with material and data inputs and outputs that checks each is attached exactly once, and one that posts a material as both input and output to confirm the cycle is refused before any script runs. That second test showed the save API reported a cycle as a server error, so it's now a validation error.

labkey-jeckels added a commit to LabKey/platform that referenced this pull request Oct 29, 2026
…he save transaction (#8220)

## Rationale
The assay batch and run save APIs now run Standard assay transform
scripts before opening their database transaction, so a script that
calls back into the server can no longer deadlock with other work such
as a folder delete ([GH Issue
1720](LabKey/internal-issues#1720)). This
extends the GH Issue 1707 fix for single-run imports to requests that
save several runs together: every run's scripts finish before the shared
transaction opens, and if any script fails nothing is written.

It also fixes a regression from #8183 that broke
`BiologicsAssayFilesTest.testImportAssayWithFileFieldData` ([TeamCity
failure](https://teamcity.labkey.org/buildConfiguration/LabKey_Trunk_Premium_ProductSuites_Biologics_BiologicsBPostgres/4231858)):
when an assay has a transform script, the uploaded data file's audit
event was recorded outside the import transaction and lost its
transaction ID.

## Related Pull Requests
- LabKey/testAutomation#3265

## Changes
- Save handlers can look up a request's data and materials without
creating them
- saveBatch and saveRuns run each run's transform scripts in a read-only
first pass, then save the runs inside the transaction and record the
scripts' output
- A run whose data changes while its scripts run fails the save and asks
the client to retry
- saveBatch and saveRuns reject a request that lists the same run more
than once
- Standard assays configured for background upload get the same behavior
through these APIs
- Custom file-based assay save handlers keep running scripts inside the
transaction
- Audit events for files uploaded before a transform script runs are
recorded in the import's transaction again; a failed import records them
only for files it keeps

Tested locally: AssayTransformCallbackTest passes, including the new
saveBatch and saveRuns cases and the new file audit cases.

---------

Co-authored-by: Lum <klum@labkey.com>
@labkey-jeckels
labkey-jeckels merged commit c2b2053 into develop Oct 29, 2026
9 checks passed
@labkey-jeckels
labkey-jeckels deleted the fb_batch_save_transforms_1720 branch October 29, 2026 13:22
XingY pushed a commit to LabKey/platform that referenced this pull request Oct 29, 2026
…he save transaction (#8220)

## Rationale
The assay batch and run save APIs now run Standard assay transform
scripts before opening their database transaction, so a script that
calls back into the server can no longer deadlock with other work such
as a folder delete ([GH Issue
1720](LabKey/internal-issues#1720)). This
extends the GH Issue 1707 fix for single-run imports to requests that
save several runs together: every run's scripts finish before the shared
transaction opens, and if any script fails nothing is written.

It also fixes a regression from #8183 that broke
`BiologicsAssayFilesTest.testImportAssayWithFileFieldData` ([TeamCity
failure](https://teamcity.labkey.org/buildConfiguration/LabKey_Trunk_Premium_ProductSuites_Biologics_BiologicsBPostgres/4231858)):
when an assay has a transform script, the uploaded data file's audit
event was recorded outside the import transaction and lost its
transaction ID.

## Related Pull Requests
- LabKey/testAutomation#3265

## Changes
- Save handlers can look up a request's data and materials without
creating them
- saveBatch and saveRuns run each run's transform scripts in a read-only
first pass, then save the runs inside the transaction and record the
scripts' output
- A run whose data changes while its scripts run fails the save and asks
the client to retry
- saveBatch and saveRuns reject a request that lists the same run more
than once
- Standard assays configured for background upload get the same behavior
through these APIs
- Custom file-based assay save handlers keep running scripts inside the
transaction
- Audit events for files uploaded before a transform script runs are
recorded in the import's transaction again; a failed import records them
only for files it keeps

Tested locally: AssayTransformCallbackTest passes, including the new
saveBatch and saveRuns cases and the new file audit cases.

---------

Co-authored-by: Lum <klum@labkey.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants