BI-2973: Prevent Duplicate Variants Error from Blocking Subsequent Genotype Data Imports

[humsika]

Description

Story: BI-2973

This PR updates BI-API VCF validation for Genotype Data Import so invalid files are rejected before submission to Gigwa.

Summary of changes:

• Added REF/ALT validation with HTSJDK-based VCF parsing/validation.
• Added duplicate Gigwa positional key validation using variant type + chromosome + position.
• Updated user-facing validation messages for:
  • invalid / unsupported REF or ALT allele values
  • duplicate positional keys
• Added logging around BI-API-side VCF validation to help identify whether failures occur before the file is sent to Gigwa.
• Updated integration tests and VCF test fixtures to cover the new validation flow.

Dependencies

bi-web: develop
bi-api: feature/BI-2973

Testing

Validated / covered scenarios:

• VCF containing unsupported REF or ALT allele values returns:
  • The file is not a valid VCF or contains unsupported REF/ALT allele values.
• VCF containing duplicate Gigwa positional keys returns:
  • Duplicate chromosomal position(s) detected. CHROM:POS key must be unique for variant type.
• Valid VCF files continue through the existing Genotype Data Import workflow successfully.

Suggested reviewer testing:

1. Upload a VCF with unsupported allele values such as X or - and confirm the invalid VCF / unsupported REF-ALT error is shown.
2. Upload a VCF with duplicate records for the same variant type, chromosome, and position and confirm the duplicate positional key error is shown.
3. Upload a valid VCF and confirm the import proceeds successfully.
4. If in case we want to look at the specific reason(s) why given VCF file is getting rejected, the newly added logs in BI-API will provide those details.

Checklist:

• I have performed a self-review of my own code
• I have tested my code and ensured it meets the acceptance criteria of the story
• I have create/modified unit and/or integration tests to cover this change or tests are not applicable
• I have commented my code, particularly in hard-to-understand areas
• I have either updated the source of truth or arranged for update with product owner if needed https://breedinginsight.atlassian.net/wiki/spaces/BI/pages/1559953409/Source+of+Truth

[HMS17]

The error messages all look great both on the UI and logs for all the error cases I tried. However when I tried the happy path with multiple vcfs that had worked before, it successfully uploaded to localstack but then when gigwa went to process it it got the error "Gigwa had an error saving the genotype data: Found no data to import!". Changing back to develop the happy path worked, so it seems something is going wrong with the happy path in BI-2973.

[humsika]

I checked the BI-2973 against develop, and the only flow change on our side is the added BI-API validation step before processSubmission(...). Once validation passes, the LocalStack upload and Gigwa submission path is the same as before, and the same fileContents byte array is passed through, so I’m not seeing an obvious happy-path logic change in the code that would explain Found no data to import!.

Since your test is getting past BI-API validation and successfully uploading to LocalStack, it looks like the issue is happening later when Gigwa processes the uploaded file. I’ll keep digging, but from the code diff alone I’m not seeing a direct happy-path change after validation passes.

[HMS17]

@humsika Re-ran BI-2973 branch on local and submission to gigwa is working now for happy path. Looks like something must have been off regarding containers in that build for some strange reason.

[HMS17]

All passing locally! Great work!