Skip to content

fix: deterministic LF line endings for generated files - #1477

Draft
joaodinissf wants to merge 4 commits into
dsldevkit:masterfrom
joaodinissf:fix/lf-normalize-runtime-generators
Draft

fix: deterministic LF line endings for generated files#1477
joaodinissf wants to merge 4 commits into
dsldevkit:masterfrom
joaodinissf:fix/lf-normalize-runtime-generators

Conversation

@joaodinissf

@joaodinissf joaodinissf commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Problem

Generated-file line endings are platform-dependent: Xtext's default ILineSeparatorInformation returns System.getProperty("line.separator"), so any headless generation on Windows (MWE2 workflows, standalone builders, tests, analysis reports) emits CRLF — while the repository's policy is LF (.gitattributes: * text=auto eol=lf). The index stays clean thanks to git's normalization, but the on-disk bytes differ per platform: generated artifacts aren't reproducible, incremental builders see spurious content changes, and non-git consumers of src-gen get platform-dependent files.

Fix

Make LF the deterministic separator for all machine-owned output:

  • LfLineSeparatorInformation (new, in the exported com.avaloq.tools.ddk.xtext.formatting package, next to its interface's home): returns "\n". Bound via bindILineSeparatorInformation() in the runtime module of all seven DDK languages (Check, CheckCfg, Export, Expression, Format, Scope, Valid). Xtext's IWhitespaceInformationProvider.Default injects exactly this type and feeds it to the IFilePostProcessor (LineSeparatorHarmonizer, or the trace-preserving TraceAwarePostProcessor for the Xbase-based languages) that post-processes IFileSystemAccess text writes — so headless generation becomes LF. In the IDE, the UI module's preference-based whitespace provider takes precedence for file writes, so workspace behavior is unchanged. Note for downstream products: headless generation of DDK DSLs on Windows now emits LF instead of CRLF — this is the intended behavior change.
  • Generation workflows: ModelInference.mwe2/TypeModel.mwe2 gain lineDelimiter = "\n" on their EcoreGenerator components, and GenerateTestLanguage.mwe2/GenerateHelloWorld.mwe2 drop their unconditional lineDelimiter = "\r\n" pins — the last workflows whose output contradicted the checked-out (LF) form of their own committed src-gen.
  • KeywordAnalysisHelper: the diagnostic/report writers use an LfPrintWriter (a PrintWriter whose println() terminates with \n) instead of platform-terminated println.
  • LfNormalizingFileSystemAccess is removed: it normalized before the FSA, but the post-processor runs afterwards and owned the final bytes all along — so it never achieved its goal, and stringifying the JvmModelGenerator's TreeAppendable destroyed the trace-region identity the trace-aware post-processor relies on. The runtime-module binding replaces it at the actual enforcement point.

Tests

  • LineEndingDeterminismTest drives the real headless write path (JavaIoFileSystemAccess + harmonizer) with the LF binding and a "\r\n" control binding, proving the post-processor follows whatever is bound.
  • CheckLineSeparatorBindingTest and FormatLineSeparatorBindingTest pin the Guice convention wiring from two of the seven runtime injectors; the remaining five languages declare the identical binding method but have no runtime test harness to assert it in.
  • AbstractCheckGenerationTestCase now asserts every generated file is CR-free and injects members into its in-memory FSA so the post-processor chain — and with it the LF binding — is on the exercised path.
  • LfPrintWriterTest covers the report writers' terminator.

Verification

  • Mechanism verified against the Xtext 2.43 sources: IWhitespaceInformationProvider.Default injects ILineSeparatorInformation, the bound IFilePostProcessor applies it to IFileSystemAccess text writes, and the default being replaced is System.getProperty("line.separator"); on the IDE path PreferenceStoreWhitespaceInformationProvider consults the injected ILineSeparatorInformation only when no resource URI is available.

🤖 Generated with Claude Code

joaodinissf and others added 4 commits August 11, 2026 00:40
The keyword diagnostic/report writers used PrintWriter.println, whose
terminator is the platform separator — the one raw-IO emission path
the FSA-level LF normalization never covers (dsldevkit#1345; revives the
KeywordAnalysisHelper piece of the closed dsldevkit#1354). All println(...)
overloads terminate via println(), so a minimal LfPrintWriter override
of that single method makes every call site emit LF; the report
builders themselves already append '\n' literals.

Assisted by Claude

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ModelInference.mwe2 and TypeModel.mwe2 were generating committed
src-gen without an explicit lineDelimiter — their three EcoreGenerator
components fell back to the platform default, so regenerating on
Windows would produce CRLF output against the LF policy.
CustomClassAwareEcoreGenerator already forwards getLineDelimiter()
into the EMF generator adapter.

GenerateTestLanguage.mwe2 and GenerateHelloWorld.mwe2 went further in
the wrong direction: they pinned lineDelimiter = "\r\n", emitting CRLF
unconditionally on every host while their committed src-gen checks out
as LF (.gitattributes: * text=auto eol=lf) — so any regeneration
rewrote both trees wholesale. Pin them to "\n" like every other DDK
workflow. Completes dsldevkit#1413's coverage.

Assisted by Claude

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…atorInformation

Xtext's IFilePostProcessor (LineSeparatorHarmonizer, or the
trace-preserving TraceAwarePostProcessor for Xbase languages)
post-processes text IFileSystemAccess writes, but its default
ILineSeparatorInformation is System.lineSeparator() — so headless
builds (xtext-maven-plugin, MWE2) emit platform-dependent endings that
fight the repository's LF policy (.gitattributes '* text=auto eol=lf',
dsldevkit#1314) and rewrite generated files on other platforms. Bind
LfLineSeparatorInformation ('\n') in every DDK language runtime module
(Check, CheckCfg, Scope, Export, Format, Valid, Expression) so
generation is deterministic for all current and future generators of
these languages at the pipeline's actual enforcement point. In the IDE
the UI modules' preference/sensing IWhitespaceInformationProvider
still takes precedence for file writes, so workspaces keep converging
to the checked-out form.

The class lives in the exported com.avaloq.tools.ddk.xtext.formatting
package (alongside its interface's home in Xtext) — putting it in a
package named "generator" would split the package already exported by
the com.avaloq.tools.ddk.xtext.generator bundle.

With the binding at the enforcement point, Check/CheckCfg's
LfNormalizingFileSystemAccess wrapper is redundant and is removed: it
never controlled the final bytes (the post-processor runs after it),
and stringifying the JvmModelGenerator's TreeAppendable destroyed the
ITraceRegionProvider identity that trace-aware post-processing relies
on, silently dropping trace regions for CRLF content.

Completes the direction of dsldevkit#1331/dsldevkit#1413 at the root; tracked by dsldevkit#1345.

Assisted by Claude

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four layers, none of which existed before:
- LineEndingDeterminismTest: drives the real headless write path
  (JavaIoFileSystemAccess + LineSeparatorHarmonizer) with the LF
  binding and, as a control, with a '\r\n' binding — proving the
  post-processor follows the bound separator (the mechanism that made
  output platform-dependent) and that the LF binding normalizes CRLF,
  CR and LF alike.
- CheckLineSeparatorBindingTest and FormatLineSeparatorBindingTest:
  pin the Guice module-convention wiring by resolving
  ILineSeparatorInformation from the Check and Format runtime
  injectors. Scope, Export, Valid, Expression and CheckCfg declare the
  identical binding method but have no runtime test harness to assert
  it in.
- AbstractCheckGenerationTestCase now asserts no generated file
  contains CR — and injects members into its InMemoryFileSystemAccess
  so the IFilePostProcessor chain (and with it the LF binding) is
  actually on the exercised path, upgrading every existing Check
  generation test into an emission regression guard.
- LfPrintWriterTest covers the raw-IO report writer's terminator.

Assisted by Claude

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@joaodinissf
joaodinissf force-pushed the fix/lf-normalize-runtime-generators branch from de2642c to 84b3f31 Compare August 10, 2026 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant