Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ To undo it, remove the entries in Windows Security → App & browser control →
| **CAD**: DWG, DXF<br>Gerber PCB layers (GBR, GTL, GBL, GTO, …) read;<br>G-code of 3D printers and CNC machines (GCODE, GCO, NGC) read | PDF, SVG, PNG, DWG ↔ DXF;<br>G-code → toolpath (from above or isometric) or the slicer's preview |
| **Fonts**: TTF, OTF, WOFF, WOFF2, EOT;<br>FON (Windows bitmap fonts) read | each other;<br>FON → one font per size |
| **Data**: JSON, JSON Lines, YAML, TOML, XML<br>vCard, iCalendar | each other, CSV and XLSX (tables);<br>vCard, iCalendar ↔ CSV |
| **3D models**: GLB, glTF, STL, OBJ (+ MTL), 3MF, PLY;<br>FBX read | each other, PNG preview |
| **3D models**: GLB, glTF, STL, OBJ (+ MTL), 3MF, PLY, studiomdl data (SMD);<br>FBX, Source engine model (MDL, with its VTF textures) read | each other, PNG preview |
| **Geospatial vectors**: Shapefile, GeoJSON, KML, KMZ, GPX<br>CSV with coordinates; Garmin FIT, OpenStreetMap (.osm) read | each other and CSV / XLSX / JSON;<br>projected data (UTM, Web Mercator, Lambert, Albers) comes out as WGS 84 longitude / latitude |

Unmarked formats work out of the box, without Office, Hancom Office or LibreOffice.
Expand Down
35 changes: 33 additions & 2 deletions docs/ENGINES.md
Original file line number Diff line number Diff line change
Expand Up @@ -1434,8 +1434,8 @@ to draw). G-code is a read-only CAD format (fields: 3D models and CAD).

## 3D models

`Filee.Engines/Models3D` converts GLB, glTF, STL, OBJ, 3MF and PLY into each other, and FBX into any of them, in
managed code: every reader
`Filee.Engines/Models3D` converts GLB, glTF, STL, OBJ, 3MF and PLY into each other, FBX and Source engine models (MDL)
into any of them, and studiomdl data (SMD) to and from them, in managed code: every reader
fills one in-memory model (`Model3D`: parts with positions, normals, UVs, vertex colours and triangles; materials
with a base colour and a base colour texture) and every writer takes it, so each pair is one step. glTF and GLB are
read and written with [SharpGLTF](https://github.com/vpenades/SharpGLTF) (MIT); STL, OBJ/MTL, PLY, 3MF and the FBX
Expand Down Expand Up @@ -1469,6 +1469,8 @@ What each format keeps:
| OBJ + MTL | `o` / `g` / `usemtl` as parts, `v` (+ r g b), `vt`, `vn`, faces of any size; `mtllib` with `Kd`, `d` / `Tr`, `map_Kd` (options skipped) next to the file | **the `.mtl` and the textures are written next to the `.obj`** (`name.obj`, `name.mtl`, `name_texture.png`, named like the output); vertex colours as `v x y z r g b` |
| 3MF | objects, components (also in other model parts, as Bambu Studio writes them), build transforms, base materials, colour groups (materials extension); `support` / `other` objects skipped | one object per part, base materials from material colours, a colour group from vertex colours |
| FBX (read only) | binary 6.x–7.x (64-bit records from 7.5, zlib-compressed arrays) and ASCII; mesh geometry with polygons of any size, the first normal, UV, colour and material layer (by polygon vertex, control point or polygon, or all the same; direct or indexed); the model hierarchy from the connections with translation, rotation (every Euler order, pre- / post-rotation, pivots and offsets), scaling and geometric transforms; property templates from Definitions; material `DiffuseColor` and `Opacity`; the diffuse texture embedded (Video `Content`) or named next to the file; animation, skins, blend shapes, NURBS, cameras and lights skipped (the rest pose is read) | — (not written) |
| SMD (studiomdl data) | the triangles of a reference SMD, one part per material (bones and weights not kept) | from an MDL: its bones, bind pose and weights (see below); from the others: a static prop with one root bone |
| MDL (Source engine, read only) | versions 44–49 with the `.vvd` and `.dx90.vtx` (else `.vtx`, `.dx80.vtx`, `.sw.vtx`) beside it (see below) | — (needs Valve's studiomdl) |
| PLY | ASCII, binary little / big endian; vertex position, normal, UV (`s`/`t`, `u`/`v`, `texture_u`/`texture_v`), colours of any type; face lists with per-corner `texcoord` and face colours; `comment TextureFile`; other elements skipped | binary little endian, one mesh; material colours become vertex colours |

Lost on the way (no place in the target): **STL** keeps only the triangles (no colours, UVs, normals or materials);
Expand All @@ -1483,6 +1485,35 @@ FBX's texture names) are read only from the model's folder or a folder below it.
points anywhere else is looked up by its name in the model's folder instead, so a downloaded model cannot pull a
picture from elsewhere on the disk into its output.

**Source engine models** (`.mdl`, #1526, the Games field; `Models3D/SourceModel.cs`): Garry's Mod, Counter-Strike:
Source, Team Fortress 2 and Half-Life 2 models are three files studiomdl writes side by side: the `.mdl` (bones,
textures, texture folders, the skin table, body parts → models → meshes), the `.vvd` (vertices with up to three bone
weights; fixups say which runs make LOD 0) and the `.dx90.vtx` (per mesh, strip groups of triangle lists or strips
indexing the mesh's vertices). All three must have the `.mdl`'s checksum, else the files belong to different builds and
the model is refused; a missing `.vvd` or `.vtx` is named. The model is read in its bind pose with its default body
groups (the first model of each body part), the first skin and LOD 0; animations, flexes, attachments and the physics
mesh (`.phy`) are not read. Source is Z up and measured in inches (25.4 mm), its triangles are clockwise and its UVs
start at the top left, so the model is turned to Filee's Y up, millimetres, counter-clockwise and bottom-left UVs; each
mesh becomes a part named after its model, with one material per texture. Later branches' 8-byte-longer strip groups
(CS:GO) are read when the indices of the usual layout don't fit.
The textures are found as Garry's Mod finds them: in the addon's or game's own layout, the `materials` folder beside the
`models` folder the model is in, `materials/<texture folder>/<texture>.vmt` for each of the model's texture folders
(`$cdmaterials`), letter case aside, then the material's `$basetexture` (also through a `patch` material's `include`) as
a `.vtf`, decoded by the VTF reader into a PNG texture; `$nocull` makes the material double-sided. Only `.vmt` and
`.vtf` files below that `materials` folder are read, never through `..`. A model outside such a layout, or one whose
materials come from the game itself (the HL2 textures), keeps plain materials. OBJ output writes the textures beside the
`.obj`, GLB embeds them.

**Studiomdl data** (`.smd`, #1526, the Games field; `Models3D/SmdFormat.cs`): the text Valve's studiomdl compiles models
from (`nodes`, a `skeleton` at `time 0`, `triangles`: a material line and three vertex lines of parent bone, position,
normal, UV and bone weights), Z up, in inches, counter-clockwise, UVs from the bottom left. An MDL becomes a reference
SMD that studiomdl can compile again: its bones with their bind pose (position and rotation relative to the parent), its
triangles with up to three bone weights per vertex and the textures as materials, read straight from the model so the
skeleton is kept (Filee's own 3D model has no place for it). Any other model becomes a static prop: one `root` bone,
every vertex weighted to it, the material's (or texture's) name as the material line, normals computed where the model
has none. Reading takes the triangles of a reference SMD as a static model; an animation SMD (no triangles) is refused
with "no triangles", a broken line with its number. The "SMD" preset is in the palette, after glTF.

**3D → pictures**: a small software rasteriser (`MeshRenderer`) draws an isometric view from the front right, above,
with one light and flat shading per triangle, vertex colours or the texture interpolated across it, and a z-buffer,
at 2048 px and scaled down to 1024 px for smooth edges, on a transparent background. The preset's image options
Expand Down
4 changes: 2 additions & 2 deletions src/Filee.Core/Formats/FormatFields.cs
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ .. Enum.GetValues<FormatCategory>().Select(category => category.ToString().ToLow
"dts", "tar", "tgz", "tbz2", "txz", "tar7z", "tlz", "gz", "bz2", "xz", "lz", "lzma", "jar", "zxp", "nupkg", "apks",
"aab", "msix", "mcpack", "mcaddon", "mcworld", "mctemplate", "mrpack", "cab", "gerber",
"cpio", "deb", "rpm", "dwg", "ttf", "otf", "woff", "woff2", "ass", "ssa", "ttml", "scc", "glb", "gltf",
"stl", "obj", "3mf", "ply", "fbx", "gcode", "geojson", "kml", "kmz", "gpx", "shp", "fit", "osm", "xm", "it", "ani", "vtf", "gma", "vpk", "bsp"
"stl", "obj", "3mf", "ply", "fbx", "gcode", "geojson", "kml", "kmz", "gpx", "shp", "fit", "osm", "xm", "it", "ani", "vtf", "gma", "vpk", "bsp", "mdl", "smd"
]),
(FormatGrade.Legacy,
[
Expand All @@ -100,7 +100,7 @@ .. Enum.GetValues<FormatCategory>().Select(category => category.ToString().ToLow
// Layout, print, photo post-production, signs and embroidery.
(["psd", "psb", "exr", "hdr", "xcf", "kra", "pdn", "raw", "eps", "ps", "ai", "cdr", "plt", "dst"], ["design"]),
// Game textures and cutscenes, Minecraft packs and modpacks.
(["dds", "vtf", "usm", "bik", "mcpack", "mcaddon", "mcworld", "mctemplate", "mrpack", "gma", "vpk", "bsp"], ["game"]),
(["dds", "vtf", "usm", "bik", "mcpack", "mcaddon", "mcworld", "mctemplate", "mrpack", "gma", "vpk", "bsp", "mdl", "smd"], ["game"]),
// Medical imaging and astronomy.
(["dcm", "fits"], ["science"]),
// Software work: raw image formats of graphics tools, app icons and cursors, config and data files, flat XML
Expand Down
5 changes: 5 additions & 0 deletions src/Filee.Core/Formats/FormatRegistry.cs
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,11 @@ public static class FormatRegistry
new("3mf", "3MF", FormatCategory.Model3D, ["3mf"]),
new("ply", "PLY", FormatCategory.Model3D, ["ply"]),
new("fbx", "FBX", FormatCategory.Model3D, ["fbx"], Writable: false),
// Source engine models (Garry's Mod and other Source games), read with the .vvd and .vtx beside them and their
// textures from the "materials" folder above them.
new("mdl", "Source model (MDL)", FormatCategory.Model3D, ["mdl"], Writable: false),
// Studiomdl data, the text the Source engine's model compiler reads (from an MDL with its bones and weights).
new("smd", "Studiomdl data (SMD)", FormatCategory.Model3D, ["smd"]),

// Geospatial vector data (CSV with coordinates is a Data format and reaches these through the geo engine)
new("geojson", "GeoJSON", FormatCategory.Geo, ["geojson"]),
Expand Down
10 changes: 10 additions & 0 deletions src/Filee.Core/Presets/BuiltInData.cs
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,7 @@ .. CatalogPresets(),
.. DataPresets(),
.. CodecAndSubtitlePresets(),
.. ModelPresets(),
.. SourceModelPresets(),
.. GeoPresets(),
.. TracePresets(),
];
Expand Down Expand Up @@ -603,6 +604,15 @@ public static List<Preset> ModelPresets() =>
new() { Id = "to-gltf", Name = "glTF", TargetFormat = "gltf" },
];

/// <summary>
/// Studiomdl data for Source engine models (#1526, added to the library in settings schema 100): in the palette only,
/// after glTF, in the Games field (off in a new install).
/// </summary>
public static List<Preset> SourceModelPresets() =>
[
new() { Id = "to-smd", Name = "SMD", TargetFormat = "smd" },
];

/// <summary>
/// Geospatial vector targets (added in settings schema 25, see SettingsMigrations). CSV, the sixth slice of the
/// donut, is the spreadsheet preset. KMZ is a KML in a ZIP and is on no donut.
Expand Down
2 changes: 1 addition & 1 deletion src/Filee.Core/Settings/AppSettings.cs
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ public static bool IsFlipKeyAllowed(string key, IEnumerable<TriggerGesture> gest
public sealed class AppSettings
{
/// <summary>Current schema version. Increase when the format changes and add a migration.</summary>
public const int CurrentSchemaVersion = 99;
public const int CurrentSchemaVersion = 100;

public int SchemaVersion { get; set; } = CurrentSchemaVersion;

Expand Down
21 changes: 21 additions & 0 deletions src/Filee.Core/Settings/UserDataStore.cs
Original file line number Diff line number Diff line change
Expand Up @@ -1193,6 +1193,13 @@ public static bool ApplyToLibrary(int from, List<Preset> presets, List<ToolbarPr
// v99: CrossOver logs (.cxlog, #1383) learn the text donut; custom claims stay put.
if (from < 99)
changed |= LearnExtensions(profiles, out _, ["cxlog"]);
// v100: Source engine models (.mdl) and studiomdl data (.smd) learn the 3D models donut and "SMD" joins the 3D
// models in the palette after glTF; custom claims stay put.
if (from < 100)
{
changed |= AddSourcePresets(presets);
changed |= LearnExtensions(profiles, out _, ["mdl", "smd"]);
}
return changed;
}

Expand Down Expand Up @@ -1266,6 +1273,20 @@ private static bool AddGma(List<Preset> presets)
return true;
}

/// <summary>"SMD" after the last 3D model preset of the library (glTF), or at the end.</summary>
private static bool AddSourcePresets(List<Preset> presets)
{
var changed = false;
foreach (var preset in BuiltInData.SourceModelPresets().Where(p => presets.All(existing => existing.Id != p.Id)))
{
var models = BuiltInData.ModelPresets().Select(p => p.Id).ToHashSet();
var last = presets.FindLastIndex(p => models.Contains(p.Id));
presets.Insert(last < 0 ? presets.Count : last + 1, preset);
changed = true;
}
return changed;
}

private static bool AddCropVideo(List<Preset> presets)
{
var at = presets.FindIndex(p => p.Id == "tool-clip");
Expand Down
3 changes: 1 addition & 2 deletions src/Filee.Engines/Archives/Vpk.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
// Valve pak files (.vpk), the archives the Source engine keeps its games' files in (Half-Life 2, Counter-Strike: Source,
// Team Fortress 2, the content Garry's Mod mounts). Read by Filee; layout (little endian) as in Valve's vpklib and
// srctools:
// Team Fortress 2, the content Garry's Mod mounts). Read by Filee; layout (little endian):
//
// signature u32 0x55AA1234, version u32 (1 or 2), tree size u32
// version 2: file data size, archive MD5 size, other MD5 size, signature size (u32 each; the sections are not read)
Expand Down
42 changes: 36 additions & 6 deletions src/Filee.Engines/Models3D/ModelConverter.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// 3D models in managed code only (SharpGLTF for glTF, Filee's own readers and writers for the rest): GLB, glTF, STL,
// OBJ, 3MF and PLY into each other through one in-memory model (Model3D), FBX (read only) into any of them, and an
// isometric preview picture.
// OBJ, 3MF and PLY into each other through one in-memory model (Model3D), FBX and Source engine models (MDL, with their
// textures; read only) into any of them, studiomdl data (SMD) to and from them (from an MDL with its bones and weights),
// and an isometric preview picture.
// See docs/ENGINES.md, "3D models", for units, axes and what each format keeps.

using Filee.Core.Conversion;
Expand All @@ -19,8 +20,8 @@ public sealed class ModelConverter(ILogger? logger = null) : IConverter, IImageT
/// <summary>The 3D formats, each readable and writable.</summary>
public static readonly string[] Formats = ["glb", "gltf", "stl", "obj", "3mf", "ply"];

/// <summary>Every 3D format that can be read: <see cref="Formats"/> and FBX.</summary>
public static readonly string[] InputFormats = [.. Formats, "fbx"];
/// <summary>Every 3D format that can be read: <see cref="Formats"/>, FBX, Source engine models (MDL) and SMD.</summary>
public static readonly string[] InputFormats = [.. Formats, "fbx", "mdl", "smd"];

/// <summary>Long edge of the preview picture before the preset's resize.</summary>
internal const int PreviewSize = 1024;
Expand All @@ -39,15 +40,20 @@ public sealed class ModelConverter(ILogger? logger = null) : IConverter, IImageT
public IReadOnlyList<ConversionEdge> Edges { get; } =
[
.. from source in InputFormats from target in Formats where source != target select new ConversionEdge(source, target),
// SMD, the source studiomdl compiles: from an MDL with its skeleton, from the others as a static prop.
.. from source in InputFormats where source != "smd" select new ConversionEdge(source, "smd"),
.. from source in InputFormats from target in ImageEncoder.Writable select ImageEncoder.Edge(source, target),
];

public EngineStatus GetStatus() => EngineStatus.Available("GLB, glTF, STL, OBJ, 3MF, PLY ↔ each other; FBX → each of them",
public EngineStatus GetStatus() => EngineStatus.Available("GLB, glTF, STL, OBJ, 3MF, PLY, SMD ↔ each other; FBX, MDL → each of them",
$"{EngineVersions.BuiltIn} · {EngineVersions.Library("SharpGLTF", typeof(SharpGLTF.Schema2.ModelRoot))}");

public Task<IReadOnlyList<string>> ConvertAsync(ConversionStep step, IProgress<double>? progress, CancellationToken cancellationToken) =>
Task.Run<IReadOnlyList<string>>(() =>
{
// A Source model keeps its skeleton and weights in an SMD, which Filee's static model has no place for.
if (step.From == "mdl" && step.To == "smd")
return WriteSkeletalSmd(step, cancellationToken);
var reading = progress is null ? null : new SubProgress(progress, 0, 0.6);
var model = Read(step.InputPath, step.From, reading, cancellationToken);
_logger.LogDebug("{File}: {Parts} parts, {Vertices} vertices, {Triangles} triangles, {Materials} materials",
Expand Down Expand Up @@ -80,6 +86,8 @@ internal static Model3D Read(string path, string format, IProgress<double>? prog
"3mf" => ThreeMfFormat.Read(path, progress, cancellationToken),
"glb" or "gltf" => GltfFormat.Read(path, progress, cancellationToken),
"fbx" => FbxFormat.Read(path, progress, cancellationToken),
"mdl" => SourceModel.Load(path, cancellationToken).ToModel3D(path, cancellationToken),
"smd" => SmdFormat.Read(path, progress, cancellationToken),
_ => throw new NotSupportedException($"Unknown 3D format '{format}'."),
};
model.Validate();
Expand All @@ -97,7 +105,7 @@ internal static void Write(Model3D model, string format, IOutputAllocator output
if (output.Allocate(format) is not { } path)
return;
outputs.Add(path);
if (!Formats.Contains(format))
if (!Formats.Contains(format) && format != "smd")
{
WriteImage(model, format, imageOptions, path, cancellationToken);
return;
Expand All @@ -114,13 +122,35 @@ internal static void Write(Model3D model, string format, IOutputAllocator output
case "3mf":
ThreeMfFormat.Write(model, stream, cancellationToken);
break;
case "smd":
SmdFormat.Write(model, stream, cancellationToken);
break;
case "glb":
case "gltf":
GltfFormat.Write(model, stream, binary: format == "glb", cancellationToken);
break;
}
}

/// <summary>A Source model as a reference SMD with its bones, bind pose and weights.</summary>
private static IReadOnlyList<string> WriteSkeletalSmd(ConversionStep step, CancellationToken cancellationToken)
{
var model = SourceModel.Load(step.InputPath, cancellationToken);
if (step.Output.Allocate("smd") is not { } path)
return [];
try
{
using var stream = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.None, 1 << 16);
SmdFormat.WriteSkeletal(model, stream, cancellationToken);
}
catch
{
TryDelete(path);
throw;
}
return [path];
}

/// <summary>The .obj, the .mtl next to it (when the model has materials) and the textures next to that.</summary>
private static void WriteObj(Model3D model, IOutputAllocator output, List<string> outputs, CancellationToken cancellationToken)
{
Expand Down
Loading
Loading