feat: introduce v4 policy with type

[minottic]

Description

This is a reimplementation of #2925 with a better data model at a cost of more code to keep v3.

The idea is that the v3 policy object that encoded job type harcoding (archiveEmailNotification ... ) is now exposed via v4 as a generic policy object with a "type", an extra free form obj "policyParams" and common "emailNotification" and "emailTo". This will later allow to narrow what every job could do. The DB mirrors the v4 shape, thus some dynamic reshaping is required by v3 which, on CRUDs, merges the last 2 policies in the DB of type archive and retrieve for a given ownerGroup. Even though the v3 code is a bit convoluted, an easier data model will pay out as soon as we shift to v4 and since policies are not yet very used in the current code base, this might be the time.

The v3 version of policies had no uniqueness constraint though policies where considered unique per ownergroup, given the default policy mapping. The migration here introduces a new index making (ownerGroup, type) unique unless they are marked with a superseededBy key that is introduced to avoid removing duplicates generate in v3 implementation

Tests included

• Included for each change/fix?
• Passing?

Documentation

• swagger documentation updated (required for API changes)
• official documentation updated

official documentation info

Summary by Sourcery

Introduce generic v4 policy resources while preserving the existing v3 policy API through a compatibility layer and migrated split storage.

New Features:

• Introduce version 4 policy CRUD APIs supporting generic job types, shared notification fields, and job-specific policy parameters.
• Add filtering, projection, pagination, sorting, and access-control handling for v4 policy resources.

Bug Fixes:

• Preserve legacy v3 policy behavior while preventing superseded historical documents from affecting reads, updates, counts, and deletes.
• Ensure partial policy updates retain unrelated settings and recreate missing archive or retrieve counterparts when needed.
• Default unspecified sort directions to ascending.

Enhancements:

• Reshape policy persistence around one document per owner group and job type with uniqueness enforced for live policies.
• Maintain a v3 compatibility layer that maps split archive and retrieve documents into the existing flat policy representation.
• Add migration support for splitting existing policies and reversing the transformation when rolling back.

Tests:

• Add unit and integration coverage for v3 compatibility, split-storage edge cases, filtering, ordering, pagination, and v4 policy CRUD and authorization behavior.

[sourcery-ai]

Sorry @minottic, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 22 hours by commenting @sourcery-ai review. Upgrade to get a review now.