fix: stop patches from resetting dataset size fields to 0

[minottic]

Description

Having explicit defaults makes plainToInstance attach fields with 0s even when missing in the patch payload. This then updates the document's values in mongo to 0 even when missing in the payload

Raw and derived datasets have dedicated DTOs so this fixes the bug for other types

Tests included

• Included for each change/fix?
• Passing?

Documentation

• swagger documentation updated (required for API changes)
• official documentation updated

official documentation info

Summary by Sourcery

Prevent dataset size-related fields from being unintentionally reset when applying partial updates to v3 custom datasets.

Bug Fixes:

• Ensure size, packedSize, numberOfFiles, and numberOfFilesArchived are not defaulted to 0 when omitted from update payloads.

Tests:

• Add regression tests verifying dataset size-related fields are preserved across unrelated PATCH operations on v3 custom datasets.

[sourcery-ai]

Hey - I've found 1 issue

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="test/DatasetCustom.js" line_range="625-634" />
<code_context>
+        });
+    });
+
+    it("0910: explicitly sets packedSize and numberOfFilesArchived", async () => {
+      return request(appUrl)
+        .patch(`/api/v3/Datasets/${encodeURIComponent(sizeFieldsPid)}`)
+        .send({ packedSize: 6789, numberOfFilesArchived: 3 })
+        .set("Accept", "application/json")
+        .set({ Authorization: `Bearer ${accessTokenAdminIngestor}` })
+        .expect(TestData.SuccessfulPatchStatusCode)
+        .expect("Content-Type", /json/)
+        .then((res) => {
+          res.body.should.have.property("packedSize").and.equal(6789);
+          res.body.should.have.property("numberOfFilesArchived").and.equal(3);
+        });
+    });
</code_context>
<issue_to_address>
**suggestion (testing):** Consider adding a test where only a subset of size-related fields is patched to ensure the others are preserved.

Current tests cover patching an unrelated field when all four size-related fields are initially set, but not patching only one or two of those fields. To guard against regressions from the previous defaulting-to-0 behavior, please add a test that patches only one field (e.g., `size`) and asserts that `packedSize`, `numberOfFiles`, and `numberOfFilesArchived` remain unchanged.

Suggested implementation:

```javascript
    it("0900: adds a new custom dataset with explicit size and numberOfFiles", async () => {
      const customDatasetWithSize = {
        ...TestData.CustomDatasetCorrect,
        size: 12345,

```

```javascript
        .expect(TestData.EntryCreatedStatusCode)
        .expect("Content-Type", /json/)
        .then((res) => {
          res.body.should.have.property("pid").and.be.a("string");
          res.body.should.have.property("size").and.equal(12345);
          res.body.should.have.property("numberOfFiles").and.equal(6);
          sizeFieldsPid = res.body["pid"];
        });
    });

    it("0910: explicitly sets packedSize and numberOfFilesArchived", async () => {
      return request(appUrl)
        .patch(`/api/v3/Datasets/${encodeURIComponent(sizeFieldsPid)}`)
        .send({ packedSize: 6789, numberOfFilesArchived: 3 })
        .set("Accept", "application/json")
        .set({ Authorization: `Bearer ${accessTokenAdminIngestor}` })
        .expect(TestData.SuccessfulPatchStatusCode)
        .expect("Content-Type", /json/)
        .then((res) => {
          res.body.should.have.property("packedSize").and.equal(6789);
          res.body.should.have.property("numberOfFilesArchived").and.equal(3);
        });
    });

    it("0920: patching only size preserves other size-related fields", async () => {
      // first, ensure all four size-related fields are explicitly set
      await request(appUrl)
        .patch(`/api/v3/Datasets/${encodeURIComponent(sizeFieldsPid)}`)
        .send({
          size: 12345,
          packedSize: 6789,
          numberOfFiles: 6,
          numberOfFilesArchived: 3,
        })
        .set("Accept", "application/json")
        .set({ Authorization: `Bearer ${accessTokenAdminIngestor}` })
        .expect(TestData.SuccessfulPatchStatusCode)
        .expect("Content-Type", /json/)
        .then((res) => {
          res.body.should.have.property("size").and.equal(12345);
          res.body.should.have.property("packedSize").and.equal(6789);
          res.body.should.have.property("numberOfFiles").and.equal(6);
          res.body.should.have.property("numberOfFilesArchived").and.equal(3);
        });

      // then, patch only `size` and ensure the other fields remain unchanged
      return request(appUrl)
        .patch(`/api/v3/Datasets/${encodeURIComponent(sizeFieldsPid)}`)
        .send({ size: 54321 })
        .set("Accept", "application/json")
        .set({ Authorization: `Bearer ${accessTokenAdminIngestor}` })
        .expect(TestData.SuccessfulPatchStatusCode)
        .expect("Content-Type", /json/)
        .then((res) => {
          res.body.should.have.property("size").and.equal(54321);
          res.body.should.have.property("packedSize").and.equal(6789);
          res.body.should.have.property("numberOfFiles").and.equal(6);
          res.body.should.have.property("numberOfFilesArchived").and.equal(3);
        });
    });

    });

```

If the surrounding describe block already has setup/teardown that manipulates `sizeFieldsPid` or size-related fields, you may want to adjust the initial patch in test `0920` to align with the values established there (e.g., using different starting values or reusing ones from previous tests). Also ensure that `accessTokenAdminIngestor`, `appUrl`, and `TestData.SuccessfulPatchStatusCode` are defined in the test file or shared test helpers as they are used consistently across existing tests.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[fpotier]

Does the size update mechanism handle null?
For instance if I create a dataset without specifying a size and then add an origdatablock (I believe this triggers a recompute of the size)
Would be a breaking change but since the backend does the book keeping for size, packedSize etc. would it make sense to remove/ignore them from create/update dtos?

[minottic]

good question, let me add a test for that and we see that

[minottic]

test added, codewise they are defaulted in mongo here before reaching the DB and in size improved in this pr #2860 and addressed with the current behaviour here and here and datablock tests covered here