Initial Foundational Semantics Spec

[willpugh]

Summary

Related Issues

This is MD version of the foundational semantics spec. It started in Google Docs: Proposed Foundational Semantics.

This follows from the discussions in the expression language slack group, and mail

NOTE: There were two comments in the original doc around join direction behaviour (to not lose fact table rows) and null safe joins. These have been addressed in this version, by being explicit about when null safe joins are required and being explicit about joining from the fact tables out.

Checklist

Specification

• Spec changes are included in core-spec/ and follow the existing structure
• Spec changes have been discussed on the mailing list or in a linked issue
• Breaking changes to the spec are clearly called out in the summary

Ontology

• Ontology changes in ontology/ are consistent with spec changes
• New or modified terms are defined and documented

Converters

• Converter logic in converters/ is updated to reflect spec or ontology changes
• New converters include tests under the converter's test directory

Validation

• Validation rules in validation/ are updated if the spec changed
• New validation cases are covered by tests

Documentation

• docs/ is updated to reflect any user-facing changes
• New features or behaviors are documented with examples where appropriate
• CONTRIBUTING.md is updated if the contribution process changed

Examples

• examples/ are added or updated for any new spec constructs or converter support

Tests

• All existing tests pass (pytest / CI green)
• New functionality is covered by tests

Compliance

• ASF license headers are present on all new source files
• No third-party dependencies are added without PMC/IPMC approval

[marioapiphani]

Will — read the full document. Strong support, both for the content and for the foundational-first approach: As discussed in #46, full support from an SAP perspective.

The five user-visible semantics — especially Semantic 5 (typed error, never a plausible-but-wrong number) and Semantic 1's NULL-bucketing of unmatched fact rows — address the query-time family of what we saw in the field. The NULL determinism work (§5.1 ordering defaults, §6.11 empty/NULL aggregate behaviour, NULL-safe stitch merges) pins down the places where observed engines quietly disagree. Adding composite keys as first-class citizens in relationships (from_columns/to_columns, unique_keys) covers the compound-identity shape that SAP-class models live on — our four-part keys map onto §6.4's inference very cleanly.

The worst losses we measured were not query-time — they were transport-time: an observed ontology platform (Fabric) accepted parts of our semantic model with a success response and silently discarded them. We only caught it by re-reading what was stored. The Foundation guarantees no silently wrong answer; nothing yet guarantees no silently dropped model construct. A conformance rule in the same spirit — an implementation MUST either represent a construct, reject it with a typed error, or explicitly report it as unsupported (never accept-and-drop) — would extend Semantic 5's philosophy to interchange itself, and would have caught every silent loss in our audit. The working draft's reference to a conformance suite (§11.1) looks like this rule's natural home. If useful, I'm happy to share the anonymized loss table as test material.

In SAP, there are 2 validations

1. Sentinel values vs. NULL. In SAP-sourced data, "missing" is usually not NULL — it's an initial value ('', '0000000000', '00000000'). Semantic 1's NULL bucket won't catch those; a broken reference surfaces as a legitimate-looking key instead. Not a Foundation blocker, but worth a future hook (declared sentinel values on a dataset, folding them into the NULL bucket) — otherwise real-world ERP data quietly defeats the guarantee.
2. Declared cardinality, when it lands. §6.4's inference is the right conservative default, but closed-world sources like SAP always know their cardinality — associations declare it. When the deferred declared-cardinality/RI proposal comes, two requests from the emitter side: keep declaration cheap (we declare, we never infer), and make declared-vs-inferred disagreement a typed error and not a silenced accept on any side.

Two editorial catches: (1) the body references §9, §10, §11, §12 (SQL-interface proposal, deferred-features registry, extension flags + conformance suite, vendor comparison incl. "§12.A divergence note") — none of which exist in the committed file or, as far as I can see, in the working draft either. As committed, readers hit dangling anchors; either trimming the references or landing stub sections would tighten it. (2) The working draft's §4.6.1 explicitly pins quoted-identifier case-sensitivity ("id" ≠ id, "ID" = id) and notes the conformance suite asserts it — the committed §4.6.1 dropped that pin. If the position now lives in the expression-language proposal, a pointer would help; if it was dropped accidentally, it's worth restoring — it's exactly the kind of cross-engine divergence this document exists to kill.

One clarifying question: the ontology layer's requires/derived_by fields also carry ANSI-SQL expressions. Is the intent that Ossie_SQL_2026 plus these evaluation semantics govern those expressions too, or does the ontology layer get its own semantics pass? Asking because we're building a CSN (SAP model language) → Ossie converter that emits closed-world constraints into requires, and we'd like to target whichever contract will govern them.

Happy to turn any of the above into concrete test fixtures — that's the form feedback is most useful in.