feat: Add domain config SQL driver - #1139
Conversation
4c0d363 to
f7666b5
Compare
There was a problem hiding this comment.
this migration is not necessary - it is part of the initial schema installed by the python keystone - no native compatible provider defines migration
There was a problem hiding this comment.
Agreed — dropped. The migration module, its migrations() override and the sea-orm-migration dependency are all gone; the two tables are now created only from their entities on keystone-manage db sync, like the other natively-compatible providers. Verified end to end against a fresh SQLite database:
$ keystone-manage --config keystone.conf db sync
schema sync completed successfully
$ sqlite3 db.sqlite ".schema whitelisted_config"
CREATE TABLE "whitelisted_config" ( "domain_id" varchar(64) NOT NULL, "group" varchar(255) NOT NULL, "option" varchar(255) NOT NULL, "value" text NOT NULL, CONSTRAINT "pk-whitelisted_config" PRIMARY KEY ("domain_id", "group", "option") );
The branch is also rebased onto main now that #1138 has merged, so it is a single commit and no longer conflicting.
0545044 to
e30db97
Compare
| let options = config.to_options(); | ||
|
|
||
| let txn = db.begin().await.context("starting the transaction")?; | ||
| delete::delete_rows(&txn, domain_id, None, None).await?; |
There was a problem hiding this comment.
in the next changes please add a comment here why the delete is done this way and at all referring to the python keystone behavior
There was a problem hiding this comment.
Added. The comment now covers why the delete is there at all and why it looks the way it does:
// Delete-then-insert, not an upsert: this is a replace, so an option the
// domain has stored but `config` does not carry has to be gone afterwards,
// and an upsert would leave it behind. The delete is unfiltered for the
// same reason — it has to reach the groups the request does not mention —
// and it covers both tables, so a sensitive option is not left orphaned
// behind a replaced `ldap` group. This is what python-keystone's
// `create_config_options` does: it deletes every row of the domain from
// `WhiteListedConfig` and `SensitiveConfig` before inserting the new
// option list, in one transaction so a failed insert cannot leave the
// domain unconfigured.The last clause was only an assertion, so it now has a test behind it too (test_create_rolls_the_delete_back_when_the_insert_fails): the insert fails after both deletes have run, and the transaction log ends in ROLLBACK with no COMMIT.
There was a problem hiding this comment.
one more table is missing: config_register. This one is actually used by 3 more provider methods: obtain_registration, read_registration, release_registration. They have only domain_id and driver_type (string)
There was a problem hiding this comment.
Added, table and the three methods.
entity/config_register.rs transcribes python-keystone's schema: type varchar(64) as the primary key and domain_id varchar(64) NOT NULL. type is a reserved word, so the field is r#type with an explicit column_name = "type" (same as policy-store-driver-sql); the tests assert the emitted SQL to pin that down. It is created from its entity on keystone-manage db sync alongside the other two.
src/registration.rs holds obtain/read/release, wired to obtain_registration, read_registration and release_registration on DomainConfigBackend. Behaviour follows the python driver:
- obtain inserts and lets the primary key decide, rather than reading the current holder and then writing — two nodes claiming a type concurrently cannot both be told they hold it. Losing the race is the answer, not a failure: the unique-constraint violation becomes
Ok(false), the way python swallowsDBDuplicateEntry, including the case of the domain already holding it itself. - read answers
Option<String>rather than raising, soNoneis python'sConfigRegistrationNotFoundfor the caller to interpret. - release is always filtered by domain, so a type registered to somebody else is left alone rather than stolen, and releasing what is not held is silent.
One deliberate divergence, documented on the function: driver_type is Option<&str>, so None is the only spelling of "every type this domain holds". python tests its type argument for truth, so type="" releases everything there; here it selects the registration of the empty type, of which there is none. There is a test pinning that.
Covered by 12 new tests, the exclusivity ones against a real SQLite database since the behaviour under test is the database enforcing the key. Whole crate: 62 tests green, clippy clean.
Nothing calls these yet — the provider that consumes them comes with Phase 3/4, same as the rest of the backend.
4da9eb8 to
cae4af5
Compare
The SQL persistence behind the /v3/domains/{domain_id}/config API
family: one row per option keyed by (domain_id, group, option), the
value stored as the JSON it was written as, matching python-keystone's
JsonBlob.
The tables are python-keystone's whitelisted_config and
sensitive_config, created from their entities on `keystone-manage db
sync`. They are part of python-keystone's initial schema, so this
driver ships no migration of its own.
Keeping the secrets in their own table is what makes the read paths
safe by construction rather than by filtering: the group and option
scoped reads back the endpoints that hand options to a client, and
they only ever select from whitelisted_config. Only the whole-config
read touches sensitive_config, because the identity backend has to be
handed the bind password, and DomainConfig skips it on serialization.
A third table, config_register, holds no option at all: it records
which domain holds the registration of a configuration type, the lock
python-keystone uses so that at most one domain drives a given
mechanism. It backs obtain_registration, read_registration and
release_registration, added to the backend trait alongside it.
Exclusivity is the primary key doing the work rather than a read
followed by a write, so two nodes claiming a type concurrently cannot
both be told they hold it; the one that loses answers false, the way
python-keystone swallows DBDuplicateEntry.
Rows that drifted out of the configurable set are dropped when a row
is decoded, which is the one place every read passes through, so a
group that is no longer configurable and an option that is no longer
whitelisted read alike. Dropping the latter deeper down would let it
reach the emptiness check the reads make their None decision on, and
be served outright by the option scoped read, which has no later
DomainConfig::from_options to drop it.
%(option)s references are stored verbatim and resolved by
DomainConfig::substitute, never on a driver read: resolving them in
get_domain_config would inline the bind password into ldap.url, which
that same read serves to the API.
Nothing resolves a DomainConfigBackend yet; the driver is reachable
only through the registry it submits itself to (name: "sql").
Part of openstack-experimental#954. Closes openstack-experimental#956.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Yousef Hussein <ymh1874@gmail.com>
cae4af5 to
1baf3bb
Compare
ac96cb6
Phase 1 of #954: the SQL persistence for the
/v3/domains/{domain_id}/configAPI family.Rebased onto
mainnow that #1138 (Phase 0, the core types and thebackend trait) has merged, so this is a single commit against
main.There is still no service or handler, so nothing resolves a backend of
the
DomainConfigBackendkind yet — the driver is reachable only throughthe registry it submits itself to (
name: "sql"), and its tables throughkeystone-manage db sync.Storage
One row per option keyed by
(domain_id, group, option), value in a textcolumn as the JSON it was written as, matching python-keystone's
JsonBlob— whatever type a client writes is the type a later readreturns.
The tables are named
whitelisted_configandsensitive_config, i.e.python-keystone's names, rather than the
domain_config/domain_config_sensitivesketched in the issue. Every other*-driver-sqlcrate in this workspace transcribes python-keystone's schema down to the
table and column names (
user,group,local_user, …), and Phase 0'score types already document these two by name.
They are part of the initial schema python-keystone installs, so this
driver ships no migration of its own (per review feedback): the tables
are created from their entities on
keystone-manage db sync, like everyother natively-compatible provider. A test against a real SQLite database
covers that path, which also pins down that the reserved words
groupandoptionare usable as column names.Registrations
A third table,
config_register, holds no option at all (added per reviewfeedback): it records which domain holds the registration of a
configuration type — the lock python-keystone uses so that at most one
domain drives a given mechanism. python-keystone's schema again,
type varchar(64)as the primary key anddomain_id varchar(64) NOT NULL;typeis a reserved word, so the field isr#typewith an explicitcolumn_name, as inpolicy-store-driver-sql.It backs the three methods the reviewer named, added to
DomainConfigBackendalongside it:obtain_registrationinserts and lets the primary key decide, ratherthan reading the current holder and then writing, so two nodes claiming
a type concurrently cannot both be told they hold it. Losing that race
is the answer rather than a failure — the unique-constraint violation
becomes
Ok(false), the way python-keystone swallowsDBDuplicateEntry,including when the domain already holds the registration itself.
read_registrationanswersOption<String>rather than raising, soNoneis python-keystone'sConfigRegistrationNotFoundand the callerdecides what it means.
release_registrationis always filtered by domain, so a typeregistered to another domain is left alone rather than stolen, and
releasing what is not held is silent. Its
driver_typeis anOption,so
Noneis the only spelling of "every type this domain holds":python-keystone tests that argument for truth, so
type=""releaseseverything there, whereas here it selects the registration of the empty
type, of which there is none. Documented and tested.
Sensitive options
Keeping secrets in their own table is what makes the read paths safe by
construction rather than by filtering:
to a client, and they only ever select from
whitelisted_config;Nonewithoutissuing a statement at all;
sensitive_config, because theidentity backend has to be handed the bind password — and
DomainConfigskips it on serialization, so it still cannot reach a response.
Substitution
%(option)sreferences are stored verbatim and resolved byDomainConfig::substitute, not on a driver read. Resolving them insideget_domain_configwould inline the bind password intoldap.url, whichthat same read serves to the API; the resolved form is for the identity
backend alone, so it belongs to the resolution layer, where the issue also
lists it (Phase 3).
Behaviour notes
longer configurable is skipped with a warning rather than failing the
whole read, which would otherwise leave the domain unreadable — and its
identity backend uninitializable — until an operator deleted the row.
Both spellings of that drift, a group that is no longer configurable and
an option that is no longer whitelisted, are dropped as the row is
decoded, which is the one place every read passes through. Dropping the
latter deeper down, where
DomainConfig::from_optionsdoes it, would letit reach the emptiness check the reads make their
Nonedecision on — soa domain holding nothing but a drifted row would answer
200with anempty configuration where a drifted group answers
404— and theoption scoped read, which has no later
from_optionsto drop it, wouldserve it outright.
delete_configdoes; the row count answers that without a precedingread, and the transaction rolls back the half that did match. Deleting a
whole configuration that is not there stays silent, also matching
python-keystone's driver.
nor sensitive: storing one would create a row every later read skips.
are what the caller just supplied and they are skipped on serialization,
so echoing them back cannot put a secret on the wire, while a caller
that patches only
ldap.passwordstill sees a group that is not empty.All of them answer with what was stored rather than what was asked for,
the way python-keystone answers from the rows it wrote: the two differ
over a group the whitelist emptied, which the request still carries and
no later read would ever report.
Outside the driver crate
core-types,GroupMismatch, for a groupscoped write whose payload addresses another group (python-keystone's
"trying to update group X, so that, and only that, …").
DomainConfigBackendadded toPluginManager's backend registry so thedriver's
inventory::submit!has a registry to submit to. Resolving andholding the backend comes with the provider that consumes it — Phase 3/4.
DomainConfigBackendtrait,obtain_registration,read_registrationandrelease_registration.They keep python-keystone's names rather than the CRUD prefixes the rest
of the trait uses, since "obtain" and "release" are what the operations
are;
get_default_*already sits outside that pattern too.Testing
cargo test -p openstack-keystone-domain-config-driver-sql— 62 tests,including nine against a real SQLite database, for the schema and for the
registration paths whose behaviour is the database enforcing the primary
key rather than a statement.
cargo test -p openstack-keystone-core-types -p openstack-keystone-corekeystone-manage db syncagainst a fresh SQLite database creates allthree tables, with the composite primary key on the two option tables;
db upstill runs clean.cargo clippy --all-targetson the touched crates: no new warnings.Part of #954. Closes #956.