From f05c838df873da5022ef4629b8a5987046d5b89c Mon Sep 17 00:00:00 2001 From: Vaclav Date: Wed, 23 Sep 2026 14:30:02 +0200 Subject: [PATCH] Document what sets the package and path of generated Java Classifier.packageName drives both the package statement and the output directory through two independent code paths; when empty, both fall back to the model long name. Covers virtualPackage (not read by textgen), package recovery by JavaParser and JavaToMpsConverter, and the generate into model folder case. Verified from source on 252.23892, 261.25134 and 262.9437. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/generated-java-package-and-path.md | 321 ++++++++++++++++++++++++ 1 file changed, 321 insertions(+) create mode 100644 docs/generated-java-package-and-path.md diff --git a/docs/generated-java-package-and-path.md b/docs/generated-java-package-and-path.md new file mode 100644 index 0000000..8282c1e --- /dev/null +++ b/docs/generated-java-package-and-path.md @@ -0,0 +1,321 @@ +# What determines the package and file path of MPS-generated Java + +**Question this answers:** a BaseLanguage `ClassConcept` (or `Interface`, `EnumClass`, `Annotation`) is generated to a +`.java` file. What decides the `package` statement inside that file, and what decides the directory it is written to? +Which property must a programmatic caller set to place a classifier in a chosen Java package — and is `virtualPackage` +that property? + +**Answer:** a single string property, `Classifier.packageName`, drives both, through two code paths that never consult +each other. When it is empty — its default, and the state of every freshly created or freshly parsed classifier — +neither path errors and neither falls back to the Java default package: both independently fall back to the **long name +of the containing model** (its qualified name without the stereotype). The failure mode of forgetting the property is +therefore a silently _different_ package, not an absent one. `virtualPackage` is a different property with a different +job and no textgen code reads it. + +Verified against three MPS development builds: + +| Build | Line | `JetBrains/MPS` commit | +| ------------- | ------ | -------------------------------- | +| **252.23892** | 2025.2 | `0f12866f47e7` (branch `2025.2`) | +| **261.25134** | 2026.1 | `499c4a0fe6bf` (branch `2026.1`) | +| **262.9437** | 2026.2 | `060a56244647` (branch `master`) | + +All statements below were read from source; the two decisive methods (`BaseLanguageTextGen.getPackageName` and +`TextGenAspectDescriptor.getPath_ClassConcept`) are **character-identical** across all three builds. Line numbers are +from 262.9437; other builds can be shifted by a few lines. Source paths are relative to the +[JetBrains/MPS](https://github.com/JetBrains/MPS) repository root. For MPS's own languages the generated Java under +`source_gen/` is cited, since that is the readable form of the aspect models. Code excerpts are **condensed**: generated +`SPropertyOperations`/`SNodeOperations` qualifiers and property constants are shortened, and elisions are marked `...`. + +## Where the API lives + +Artifact locations were checked against an MPS 2025.3 distribution. + +| Element | Distribution artifact | +| ---------------------------------------------------------------------------------- | ------------------------------------------------------- | +| `Classifier.packageName` (structure, textgen and behavior descriptors) | `languages/baseLanguage/jetbrains.mps.baseLanguage.jar` | +| `jetbrains.mps.text.*` (`TextUnit`, `impl.ModelOutline`, `rt.TextGenModelOutline`) | `lib/mps-textgen.jar` | +| `jetbrains.mps.project.facets.*`, `jetbrains.mps.smodel.*` | `lib/mps-core.jar` | +| `org.jetbrains.mps.openapi.*` | `lib/mps-openapi.jar` | +| `jetbrains.mps.lang.core.plugin.TextGen_Facet` (the make-time writer) | `languages/make/jetbrains.mps.make.facets.jar` | +| `jetbrains.mps.java.core.newparser.JavaParser`, `JavaToMpsConverter` | `plugins/mps-java/lib/java-core.jar` | +| `org.eclipse.jdt.internal.compiler.ast.ImportReference` (ECJ) | `lib/eclipse.jar` | + +## The property + +`packageName` is declared on `Classifier`, so `ClassConcept`, `Interface`, `EnumClass` and `Annotation` all inherit it +(**Verified**): + +```java +// languages/baseLanguage/baseLanguage/source_gen/jetbrains/mps/baseLanguage/structure/StructureAspectDescriptor.java +// inside createDescriptorForClassifier(), origin node 1107461130800 +b.property("packageName", 0x26be0cf68be19d69L).type(PrimitiveTypeId.STRING).origin("2791683072064593257").done(); +``` + +It is a plain string property with no property constraint — no validator, getter or setter override for it anywhere in +`jetbrains.mps.baseLanguage`'s constraints aspect (**Verified**, by absence: `Classifier_Constraints.java` exists but +does not mention it, and no other `*_Constraints.java` in that language references the property). + +In the IDE it is edited from the **inspector**, not the main editor +(`.../source_gen/jetbrains/mps/baseLanguage/editor/ClassConcept_InspectorBuilder_a.java:98`, and the equivalent builders +for `Interface`, `EnumClass`, `Annotation`). A reader who has only ever used the main editor will not have seen it, +which is part of why the fallback below is usually encountered rather than chosen. + +## Path A — the `package` statement and cross-file imports + +`ClassConcept_TextGen.generateText` calls `BaseLanguageTextGen.fileHeader`, which emits the package declaration into the +`HEADER` text area, and only for a top-level classifier (**Verified**): + +```java +// languages/baseLanguage/baseLanguage/source_gen/jetbrains/mps/baseLanguage/textGen/BaseLanguageTextGen.java:131-147 +public static void fileHeader(SNode cls, final TextGenContext ctx) { + boolean topClassifier = !((boolean) Classifier__BehaviorDescriptor.isInner_idsWroEc0xXl.invoke(cls)); + if (topClassifier) { + tgs.pushTextArea("HEADER"); + tgs.append("package " + BaseLanguageTextGen.getPackageName(cls, ctx) + ";"); + ... +``` + +```java +// same file, :270-276 +protected static String getPackageName(SNode cls, final TextGenContext ctx) { + if (isNotEmptyString(getString(as(getContainingRoot(cls), Classifier), packageName))) { + return getString(as(getContainingRoot(cls), Classifier), packageName); + } + return SModelOperations.getModelName(SNodeOperations.getModel(cls)); +} +``` + +Two things follow, both **Verified** from this method: + +- The fallback is `SModelOperations.getModelName(model)`, which is `model.getName().getLongName()` — the model's + qualified name without its stereotype + (`core/kernel/smodelRuntime/source_gen/jetbrains/mps/lang/smodel/generator/smodelAdapter/SModelOperations.java:146`) — + not the empty string. +- It reads `getContainingRoot(cls)`, so a **nested** classifier never contributes its own `packageName`; the root + classifier's value wins for the whole file. + +The same method is also consulted when _other_ files reference this classifier: `getPackageAndShortName` (`:209`, calls +at `:247` and `:252`) and `internalClassifierName` (`:104`, call at `:111`) take the referenced classifier's package +from it. The result is passed through `appendClassName` (`:295`) to `getClassName` (`:277`), which asks +`ClassifierUnitContext.getClassifierRefText` and, through it, `ImportsContext` whether the reference needs an import or +can be written short. A classifier's `packageName` therefore affects the text of files that merely mention it, not only +its own. + +`ImportsContext` computes the _current_ file's package by a third route — `JavaNameUtil.packageName(getFqName(root))` +(`.../textGen/ImportsContext.java:34`) — and `Classifier.getFqName()` has the same precedence, preferring `packageName` +and otherwise deferring to `INamedConcept.getFqName()` (**Verified**): + +```java +// .../behavior/Classifier__BehaviorDescriptor.java:243-252 +if (parentClassifier != null) { return getFqName(parentClassifier) + '.' + name; } +if (isNotEmptyString(getString(__thisNode__, packageName))) { return getString(__thisNode__, packageName) + '.' + name; } +return INamedConcept__BehaviorDescriptor.getFqName_idhEwIO9y.invokeSuper(__thisNode__, Classifier); +``` + +```java +// languages/core/core/source_gen/jetbrains/mps/lang/core/behavior/INamedConcept__BehaviorDescriptor.java:31-42 +String longName = SModelOperations.getModelName(model); +if (longName == null || longName.equals("")) { return name; } +return longName + '.' + name; +``` + +## Path B — the output directory + +The output path is decided separately, in the textgen aspect descriptor, and does **not** call `getPackageName` +(**Verified**): + +```java +// .../textGen/TextGenAspectDescriptor.java:402-407 (and the identical _Interface / _Annotation / _EnumClass twins) +private static String getPath_ClassConcept(SNode node) { + if (isNotEmptyString(SPropertyOperations.getString(node, PROPS.packageName$n3Xr))) { + return SPropertyOperations.getString(node, PROPS.packageName$n3Xr).replace('.', '/'); + } + return null; // <- not "", and not the model name +} +``` + +That value is handed to `UnitBuilder.path(...)` (same file, `:436`) and ends up as the unit's `getFilePath()`. The +contract for `null` is stated on the interface (**Verified**): + +```java +// core/textgen/source/jetbrains/mps/text/TextUnit.java:48-58 +/** + * Tell desired location of the text outcome + * @return {@code null} to use default value derived from qualified model name. + */ +@Nullable default String getFilePath() { return null; } +``` + +`ModelOutline` passes the value through unchanged in the ordinary case; its `myDefaultModelPath` is applied only on the +branch where the _unit name itself_ contains a separator +(`core/textgen/source/jetbrains/mps/text/impl/ModelOutline.java:115-142`). The real resolution of `null` happens at +write time, in the make facet (**Verified**): + +```java +// languages/languageDesign/make/solutions/jetbrains.mps.make.facets/ +// source_gen/jetbrains/mps/lang/core/plugin/TextGen_Facet.java:334-343 +if (tu.getFilePath() == null) { + if (!(seenFileNames.add(tu.getFileName()))) { /* "Duplicate unit name ..." warning */ } + javaSourcesLoc.saveStream(tu.getFileName(), tu.getBytes()); +} else { + FileDeltaCollector fdc = msfm.newPrimaryStreamHandler(tu.getFilePath()); + fdc.saveStream(tu.getFileName(), tu.getBytes()); + rdm.addDelta(fdc.getDelta()); +} +``` + +The two destinations resolve differently: + +```text +packageName empty -> javaSourcesLoc = msfm.getPrimaryStreamHandler() + = GenerationTargetFacet.getOutputLocation(model) + = overriddenOutputDir(model), else // + +packageName set -> msfm.newPrimaryStreamHandler(path) + = GenerationTargetFacet.getOutputRoot(model).getDescendant() + = ( overriddenOutputDir(model), else ) + // +``` + +Supporting declarations (**Verified**): `ModuleStaleFileManager.ModelStaleFileManager.getPrimaryStreamHandler` / +`newPrimaryStreamHandler` in the same solution (`:144-178`); `JavaModuleFacet.getOutputLocation(SModel)` and +`getOutputRoot(SModel)` in `core/project/source/jetbrains/mps/project/facets/JavaModuleFacet.java:123-155`; +`FileGenerationUtil.getDefaultOutputDir(SModelReference, IFile)`, which takes `reference.getName().getLongName()`, +replaces `.` with the file-system separator and appends the result to the output root +(`core/kernel/source/jetbrains/mps/generator/fileGenerator/FileGenerationUtil.java:50-54`). Models of a test module +resolve the same way through `TestsFacet` (`core/project/source/jetbrains/mps/project/facets/TestsFacet.java:50-92`), +with the test output root in place of ``. Both branches honour the per-model _generate into model folder_ +override, which `JavaModuleOperations.getOverriddenOutputDir` derives from +`GeneratableSModel.isGenerateIntoModelFolder()` (`.../facets/JavaModuleOperations.java:71-81`); it returns the directory +that holds the model file, and only when the model's data source is a single file (`FileDataSource`). For file-per-root +persistence (`FilePerRootDataSource` extends `FolderDataSource`, +`core/kernel/source/jetbrains/mps/persistence/FilePerRootDataSource.java:31`) the option has no effect. + +Consequences worth knowing: + +- **Duplicate-name detection only guards the fallback branch** (**Verified** from the snippet above). `seenFileNames` is + consulted only when `getFilePath() == null`. Two root classifiers in one model that share a name _and_ a non-empty + `packageName` are both saved to the same file with no duplicate warning; that the second write silently replaces the + first is **Inferred** — the stream handler's overwrite behavior was not traced. +- **Auxiliary artifacts do not follow the class** (**Verified**). Trace/debug info and the cross-model cache are + registered against `javaSourcesLoc` — the model's standard location — regardless of any unit's own path + (`TextGen_Facet.java:351-353`). +- **Directory and package diverge under _generate into model folder_** (**Verified** from the resolution above). This + applies only to single-file models; see above. Without the override, both paths yield the same dotted name — + `packageName` if set, else the model long name — so the file lands in the directory that matches its `package` + statement. With the override, an empty `packageName` writes directly into the model file's directory, and a set + `packageName` writes into `//`. The directory is then derived from where + the model file is stored, not from the package, and matches the package only if the model file happens to sit in a + matching directory. + +## `virtualPackage` is a different property with a different job + +`virtualPackage` is declared on `BaseConcept`, not on `Classifier` +(`core/kernel/kernelSolution/source_gen/jetbrains/mps/smodel/SNodeUtil.java:69`, property id `0x115eca8579f` in +`jetbrains.mps.lang.core`). It is a **logical-view organisation** property: it groups roots into folders in the project +pane and in editor tabs, and drag-and-drop in that pane is implemented by rewriting it (**Verified**): + +- `workbench/mps-ui/source/jetbrains/mps/ide/ui/tree/smodel/SModelTreeNode.java:117` and + `.../tree/smodel/PackageNode.java:59` — tree grouping. +- `workbench/mps-workbench/source/jetbrains/mps/ide/projectPane/ProjectPaneDnDListener.java:116-137` — drag-and-drop + assigns it. +- `workbench/mps-workbench/source/jetbrains/mps/ide/editorTabs/tabfactory/tabs/CreateGroupsBuilder.java:178-180` — + propagated to aspect roots so concept tabs stay grouped together. + +The only generation-side use is `TemplateGenerator` copying it from an input root to the corresponding output root, with +a comment confirming the root-only scope +(`core/generator/source/jetbrains/mps/generator/impl/TemplateGenerator.java:447-452`). **No textgen code reads it** +(**Verified**, by absence: no occurrence in `core/textgen/`, in the generated textgen code of +`jetbrains.mps.baseLanguage`, or in the `TextGen_Facet` write path — the only hit under those trees is the property +being used to organise the aspect model's own roots, which is the logical-view role described above). Setting it has no +effect on generated Java. + +## Recovering the package from parsed Java + +`JavaParser` recovers the package declaration of parsed source but does not write it into any node; neither `JavaParser` +nor the `newparser` converters set `Classifier.packageName` (**Verified**, by absence: nothing under `.../newparser/` +references the property; the string `packageName` occurs there only as local variable names): + +```java +// plugins/mps-java/core/modules/jetbrains.mps.java.core/source_gen/jetbrains/mps/java/core/newparser/JavaParser.java +// :104-110, inside case CLASS / CLASS_STUB +if (compRes.currentPackage != null) { + StringBuffer sb = new StringBuffer(); + compRes.currentPackage.print(0, sb, false); + resultPackageName = sb.toString(); +} +// exposed as JavaParseResult.getPackage() (:565-567) +``` + +The string is the **bare dotted name** — `com.acme`, with no `package` keyword and no semicolon — which is exactly the +form `packageName` expects. **Verified** by disassembling ECJ's +`org.eclipse.jdt.internal.compiler.ast.ImportReference.print(int, StringBuffer, boolean)` as bundled in +`lib/eclipse.jar` of the MPS 2025.3 distribution (not in the three source builds above): it joins `tokens` with `'.'` +and appends `".*"` only when the third argument is `true` _and_ the on-demand bit is set; MPS passes `false`. + +Limits, all **Verified** from the same switch: + +- `resultPackageName` is populated only for `FeatureKind.CLASS` and `CLASS_STUB`. The `CLASS_CONTENT` and `STATEMENTS` + branches never assign it, so `getPackage()` is `null` there. +- It is also `null` when the source declares no package, since `compRes.currentPackage` is then `null`. + +MPS's own importer, `JavaToMpsConverter` (same directory), carries the package across by **model name** instead of by +property, and only in one of its two modes (**Verified**). In both modes `convertToMps` first runs `parseFile` on every +file (`:165`): it reads `getPackage()` (`:359`), rejects files in the default package (`:364`) or whose package does not +match their directory (`:372`), and groups roots by package. It then branches on the target (`:188`): + +- **Module target** (constructors taking an `SModule`, used for example by `MigrateSourcesToMPS_Action`, + `NewModelFromSource_Action` and the IntelliJ plugin's `ConvertPackageToModel`). `getModel` (`:950-958`) places each + group in the module's model whose full name equals the package, creating it if needed. The comparison uses + `SModel.getModelName()`, which includes any stereotype, so an existing `com.acme@tests` model is not reused for + package `com.acme`. The empty `packageName` falls back to that model's name, so the generated package matches the + source. +- **Model target** (constructor taking an `SModel`, `:137`, used by `GetModelContentsFromSource_Action`). Every root + goes into the given model regardless of its declared package (`:203-212`). + +`JavaPaster` (paste Java as MPS nodes) uses neither mode: it calls `JavaParser.parse` directly +(`plugins/mps-java/platform/modules/jetbrains.mps.java.platform/source_gen/jetbrains/mps/java/platform/util/JavaPaster.java:128`) +and builds a model-target converter only to resolve references (`:237-238`). + +So any flow that inserts parsed classifiers into a model whose long name is not their declared package — the converter's +model-target mode, `JavaPaster`, or any other direct caller of `JavaParser` — produces classes whose generated package +is the model's long name, silently disagreeing with the source they were parsed from, unless the caller copies +`getPackage()` into `Classifier.packageName`. Putting the classifier in a model named after the package, or setting +`packageName` on it, are the two ways to keep them in agreement. + +## Writing the property + +`packageName` is an ordinary string property, so the usual model-write rules apply: the write needs MPS write access (a +command or write action), and the choice of setter follows [`property-value-setters.md`](property-value-setters.md) — +for a string property both `setProperty` and `setPropertyValue` happen to round-trip, but +`setProperty(node, property, String)` is the one that mirrors the read path for every datatype. + +Set it on the **root** classifier: a value on a nested classifier is read by neither path. Path A takes +`getContainingRoot` (**Verified**); Path B runs only for text units, and `TextGenAspectDescriptor.breakdownToUnits` +(`:340-352`) creates those only for the model's root nodes whose concept is exactly `ClassConcept`, `Interface`, +`EnumClass` or `Annotation` (**Verified**). Because the match uses `equals`, a root of a _subconcept_ gets no unit from +this descriptor; it depends on its own language's textgen (see Unresolved). + +Clearing it (setting `null` or `""`) returns both paths to the model-name fallback — `isNotEmptyString` guards both +(**Verified**). + +## Version differences + +None found across the three builds checked. `BaseLanguageTextGen.getPackageName` is at `:270-276` in all three, and +`TextGenAspectDescriptor.getPath_ClassConcept` at `:398-403` (252) / `:402-407` (261, 262), with identical bodies; the +line shift is unrelated surrounding generated code. The `packageName` property id `0x26be0cf68be19d69` is stable. + +Not checked: releases before the 2025.2 line. `FileGenerationUtil` is `@Deprecated(since = "3.4", forRemoval = true)` +and `TextUnit.getFilePath` is marked `PROVISIONAL API`, so the _path_ half of this note is the half more likely to move +in a future release; the `packageName` property itself is load-bearing for the emitted `package` statement and unlikely +to change shape. + +## Unresolved + +- Only `jetbrains.mps.baseLanguage` was checked, where the four `getPath_*` methods are identical. Other languages that + generate `.java` were not surveyed. That includes languages whose root concepts extend `ClassConcept` and so get no + text unit from BaseLanguage's `breakdownToUnits`. Whether their own textgen derives the path and package with the same + precedence is unknown. +- The behavior described is read from source only. No runtime probe was run to observe a generated file landing in a + `packageName`-derived directory, nor to confirm the silent-overwrite case for two same-named roots sharing a + `packageName`. Both are straightforward to demonstrate with a two-root model and one make.