Skip to content

Latest commit

 

History

History
217 lines (174 loc) · 12.6 KB

script-development.md

File metadata and controls

217 lines (174 loc) · 12.6 KB

Script Development

Scripts are separate processes that talk to SharpTagManager over stdin/stdout using a small JSON-RPC protocol, so any language capable of reading/writing JSON on stdin/stdout works. The examples on this page happen to use Python, but nothing about the protocol is Python-specific. This page covers writing your own script and its descriptor. For installing and running scripts as an end user, see Script Usage.

A script can add tags, replace tags, edit the image, or any combination — all through the same single request/response. See Script Examples for complete, working scripts to use as starting points.

The JSON-RPC protocol

The app launches your script as a subprocess and sends one JSON request per line to its stdin; your script replies with one JSON response per line on stdout. Each message arrives as a single line — the beautified versions below are just for readability.

Request:

{"id": 0, "method": "process", "params": {"image_path": "C:\\...\\photo.jpg"}}

Example:

{
  "id": 3,
  "method": "process",
  "params": {
    "image_path": "C:\\Datasets\\cats\\photo_042.jpg",
    "item": "photo_042",
    "path": "C:\\Datasets\\cats\\photo_042.json",
    "tags": [
      { "name": "cat", "layer": "manual", "weight": 1.0 }
    ],
    "layers": [
      {
        "name": "manual", "type": "TagLayer_Tags", "description": null,
        "tags_are_editable": true, "tint_color": "#FFFFFF",
        "visible": true, "locked": false, "enabled": true, "exportable": true,
        "autogenerated": false, "grouping": "None", "sorting": "None",
        "sorting_order_reverse": false, "placeholder_spots": 0, "assimilates": false,
        "wildcard_list": null, "prune_unused": false, "autocomplete_mode": "Full",
        "trigger_tag": null
      }
    ]
  }
}

The app always sends everything a script might need — it doesn't guess what a script wants based on which folder it's in. params.tags is the image's current tag list (same shape as the result's tags, see below), and params.layers is the project's layers with their type and options (skipping any that are disabled or locked — those aren't meaningfully in play, so there's nothing useful for a script to do with them; see Tag Layers for what each of those fields controls). params.tags/params.layers/params.item/params.path are only present when the call was made against a known image item belonging to an open project (e.g. from the app's UI) rather than a bare path, and params.tags is omitted entirely if the image has no tags yet. Useful for prompting a captioning model with context about tags/layers that already exist — see Scripts/lmstudio_caption/prompt_processor.py for a substitution syntax built on top of this data.

Response:

{"id": 0, "result": {...}, "error": null}

Example — a script that both adds tags and edited the image:

{
  "id": 3,
  "result": {
    "tags": [
      { "name": "cat", "layer": "img2tag", "weight": 0.94 },
      { "name": "outdoors", "layer": "img2tag", "weight": 0.81 }
    ],
    "tags_mode": "replace_layer",
    "default_layer": "img2tag",
    "image_modified": true
  },
  "error": null
}
  • id must be echoed back unchanged.
  • error should be null on success, or a string describing the failure.
  • result is a dict; every field in it is optional — only include what your script actually did.
  • Anything your script writes to stderr is shown in the app's script output window — use it for logging/progress, not stdout (stdout is reserved for RPC responses).

result.tags — adding or replacing tags

A list where each entry is either a plain tag name string, or a dict with more detail:

{"name": "cat", "layer": "img2tag", "weight": 0.87}

Supported fields: name (required), layer, weight, bound_area (a bounding box / polygon, see ScriptRpc.cs for the exact shape if you need it).

result.tags_mode controls what happens to tags not returned:

tags_mode Behavior
"add" (default) Just merges tags in — nothing existing is removed.
"replace_layer" Also removes any existing tag in default_layer (or, if omitted, a layer named after the script itself) that wasn't returned. Use this for a tagger you re-run over the same image — it replaces its own previous suggestions instead of piling up.
"replace_all" Removes every existing tag not returned, project-wide. Use this for a script that edits/reorders the whole tag set.

result.image_modified — editing the image

If your script rewrote the image file at image_path in place, set "image_modified": true so the app knows to flush its thumbnail/preview cache and show the new version. Nothing else is required — just load the file, transform it, and save it back to the same path.

One-shot vs. persistent scripts

The simplest scripts read one request from stdin, write one response, and exit — see the one-shot examples in Script Examples. This is fine for fast scripts, but wasteful if your script has a slow startup (e.g. loading a model).

For that case, write a persistent script instead: loop reading requests and writing responses for as long as the process stays alive (see the persistent variants). The app keeps the process running between calls and reuses it, so the slow part (model load) only happens once. Persistent scripts should exit on their own after a period of inactivity rather than running forever — the examples use a 60 second idle timeout.

The -startup flag

The app may launch your script with a bare -startup flag (e.g. from the "Scripting" menu's "Run script startup") just to confirm it starts, with no request to process yet. A one-shot script should print a message to stderr and exit cleanly rather than blocking on a stdin read that will never come. A persistent script generally doesn't need to treat this specially, as long as it's fine starting up with no immediate work.

Your script also needs to tolerate one extra bare mode flag the app appends after every other argument: -process or -startup. Whatever argument-parsing your language/framework uses, it needs to accept this flag rather than error out on an unrecognized argument — either by explicitly declaring it (and ignoring it), or by tolerating unknown arguments in general. For example, with Python's argparse:

parser.add_argument("-process", action="store_true", help=argparse.SUPPRESS)
parser.add_argument("-startup", action="store_true", help=argparse.SUPPRESS)

Script descriptors (.json)

By default the app guesses how to launch a script from its file extension. For anything more specific — custom arguments, a dedicated Python environment, user-configurable options — drop a .json descriptor next to the script. The app scans the Scripts folder for .json files and treats each one as a descriptor.

Minimal example:

{
  "name": "My Tagger",
  "run":  "python",
  "args": ["my_script.py"]
}

Full example:

{
  "name": "Florence-2 Tagger",
  "run":  "python",
  "args": ["florence2/florence2.py"],
  "venv": {
    "pythonVersion": "3.10",
    "requirements":  "florence2/requirements.txt"
  },
  "workingDirectory": "florence2",
  "env": { "HF_HOME": "D:/models/huggingface" },
  "options": [
    {
      "arg":         "--device",
      "label":       "Device",
      "description": "Which device to run inference on.",
      "type":        "enum",
      "values":      ["auto", "cuda", "cpu"],
      "default":     "auto"
    },
    {
      "arg":     "--timeout",
      "label":   "Idle timeout (seconds)",
      "type":    "integer",
      "default": 600,
      "min":     0,
      "max":     3600
    }
  ]
}

Fields:

  • name — display name in the app's script lists/menus. Falls back to the filename if omitted.
  • run — the executable to launch (e.g. python, node, perl, or a custom .exe). Resolved from PATH unless it matches a file next to the descriptor. Ignored (replaced by the venv's own interpreter) if venv is set.
  • args — arguments passed before any user-configured options and the mode flag. Entries that resolve to a file/folder next to the descriptor are expanded to absolute paths automatically — this is how you reference your own script file.
  • options — user-configurable values shown in the app's "Script Options" dialog before running, appended as CLI arguments. See below.
  • venv — optional dedicated Python virtual environment for this script (see next section). Omit entirely to just use whatever python is on PATH.
  • workingDirectory — working directory for the launched process, relative to the descriptor's own folder (or absolute). Defaults to the app's own working directory if omitted.
  • env — extra environment variables merged on top of the inherited environment.

Options

Each entry in options becomes a CLI flag. Supported type values:

type Behavior
bool Checkbox. Emits the bare flag (no value) when checked; emits nothing when unchecked.
enum Dropdown from values, but still freely editable — not a strict allow-list.
path-enum Dropdown listing the files found in directory (a live scan — new files show up without restarting the app), also freely editable. The value stored is just the file name; the app expands it to a full path (joined with directory) when building the command line, unless it's already an absolute path.
integer / float Numeric field, clamped to min/max if set.
string Plain text field.
multiline Text field with a "..." button opening a multi-line text editor. Embedded newlines are preserved and passed to the script as one argument.
file / folder Text field with a "..." picker button. directory sets the file picker's initial folder (file only).

Example of one option per type:

"options": [
  { "arg": "--verbose", "label": "Verbose logging", "type": "bool", "default": false },
  { "arg": "--device", "label": "Device", "type": "enum", "values": ["auto", "cuda", "cpu"], "default": "auto" },
  { "arg": "--prompt-file", "label": "Prompt file", "type": "path-enum", "directory": "prompts", "default": "default.txt" },
  { "arg": "--batch-size", "label": "Batch size", "type": "integer", "default": 8, "min": 1, "max": 64 },
  { "arg": "--confidence", "label": "Confidence threshold", "type": "float", "default": 0.5, "min": 0.0, "max": 1.0 },
  { "arg": "--prefix", "label": "Tag prefix", "type": "string", "default": "" },
  { "arg": "--model-path", "label": "Model file", "type": "file", "directory": "models" },
  { "arg": "--output-dir", "label": "Output folder", "type": "folder" }
]

For path-enum, directory is required and — like file/folder args elsewhere — a relative path resolves against the descriptor's own folder, so "prompts" means "the prompts folder next to this .json file".

With --verbose checked, --device left at auto, --batch-size set to 16, --confidence left at its default, and --model-path/--output-dir left blank, the app appends this to the launched command: --verbose --device auto --batch-size 16 --confidence 0.5. Unchecked bool options and empty string/file/folder values are omitted entirely rather than emitted as an empty value.

Per-script Python environments (venv)

If your script needs specific dependencies (or a specific Python version) rather than whatever happens to be on the user's PATH, add a venv block to its descriptor:

"venv": {
  "pythonVersion": "3.11",
  "requirements":  "requirements.txt"
}

The app manages this automatically via uv (downloaded on first use — no manual install needed):

  • uv venv --python <pythonVersion> creates a .venv next to the descriptor, downloading that exact Python interpreter if it isn't already available. Users don't need Python pre-installed.
  • uv pip install -r <requirements> installs dependencies into it, only re-running when requirements.txt (or pythonVersion) actually changes since the last successful install.
  • Your script then runs under that venv's interpreter instead of whatever python resolves to on PATH.

Leave any hardware-specific dependency (e.g. a CUDA-specific PyTorch build) out of requirements.txt — there's no way to express "pick the right one for this machine" there. Instead, have your script detect the hardware and install that dependency itself on first run.

Scripts with no venv block are unaffected — they run against python on PATH exactly as before.