Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ The choices that the code does not explain, with the options that were rejected.
- **Marks on lines of output come from line states**: `[!code highlight]` and the state directives mark the line in the output panel with `decorateLine()`, the code that marks lines of code, so the classes, the screen reader name and the labels are the same. The build puts each marked line in a `hidden` element in the figure, and the client clones it when the line prints. A highlight uses Expressive Code's mark colours through `scb-state-mark`. The output colours meet 4.5:1 only on the code background, so a marked line takes the text colour of its label, which meets 4.5:1 on the tint. With line states off, the state directives are unknown, as on code lines, and `[!code highlight]` on an output line warns. *Rejected:* the output colours corrected for each tint (a new setting for each state and stream); `.ec-line` elements in the panel for Expressive Code's per-theme colours (client modules and components find code lines by that class); HTML strings in a data attribute (needs `innerHTML`); a `<template>` (Starlight's `<Tabs>` parses its HTML again with Sätteri, which drops the template content).
- **Scripted output takes a language, so terminal colours come from Shiki's `ansi`**: `runnable.outputLang` on the fence line, and `lang=` on an `[!output]` line for the lines after it. The user chose this over Rich markup (`[red]…[/]`): EC already renders `ansi`, and a language works for `json` or `yaml` output too. The output lines leave the code before EC highlights, so `output-highlight.ts` renders them with a second EC engine on the site's themes, then corrects each token colour to 4.5:1 on its own background, with its alpha blended in, because EC corrects syntax colours only and ANSI colours and `dim` come out unreadable on many themes. EC switches token colours with the theme only inside `.ec-line`, so the runnable plugin writes the same rules for the panel with `themedCss()`. `txt` keeps a part in its stream colour. A marked line keeps only its text, as the line above says. `\e[`, `\x1b[`, `\033[` and `\u001b[` stand for the escape character, which a fence line cannot hold, and a second backslash before one keeps it as text (`\\e[` shows `\e[`), for output such as a regular expression. Other backslashes stay as written. Real runs with `ansi` are parsed in the client (`client/shared/ansi.ts`), because their output is known only in the browser: the 16 standard colours are `codeblocksRunnable.ansi*` style settings from the theme at 4.5:1, `dim` takes the muted colour, and 256-colour and RGB colours stay as printed. Other languages do not colour a run. *Rejected:* a Rich markup parser (a port of Rich's markup, style and colour code for one use); `.ec-line` in the panel (as above); Shiki in the client for runs (far over the client budget).
- **`runnable.static` shows scripted output from the build, with no button, no live region and no `data-scb-runnable`, so no client module loads.** It is the page's own text, so it is not a decoration: search indexes it and HTML readers such as starlight-llms-txt keep it. It needs scripted output; without it the block warns and keeps its button. The user rejected `runnable.ran` as too close to `run`, and wanted no **Run again**.
- **Python runs act as `python <title> <args>`**, so that a command line tool in a block is the code of a real file. `sys.argv` is the title (else `main.py`) and `runnable.args`, split as a shell splits them; `SystemExit` prints only a message that is not a number, as CPython does; `COLUMNS` is the panel width, measured before each run. A session stops at `sys.exit()`, as the REPL does. *Rejected:* asking authors for `standalone_mode=False` and `prog_name` in Click calls (code that differs from what readers run); running on after `sys.exit()` (unlike Python); a separate attribute for the program name (the title already names the file).
- **No `runnable.autorun`.** Running code on page load was built and removed before release: a Python block then downloads Pyodide with every page view, and blocks on one page need a queue that races with clicked runs. `runnable.static` covers output that should show from the start.
- **`[!run]` holds hidden code that runs in place of the visible code**, so a terminal block can show `python hello.py --help` and run Python. The lines leave the code in the notation plugin, as output lines do, up to `[!run end]`, the next `[!output]` or the end. `lang=` picks the runtime. The figure carries the code in `data-scb-runnable-code`, and the run is never a REPL session. Scripted output wins over `[!run]`, with a warning. *Rejected:* `runnable.lang` on the fence line with the code in hidden lines (hidden lines are part of the copied text).
- **The output panel is a shade darker than the code, in the code font size**, by the user's choice, so the output reads as a separate part of the block. `outputBackground` darkens the code background by an amount for each theme type, which gives about the same step in light and dark themes, and every output colour (streams, labels, ANSI colours, highlighted tokens, line state tints) is corrected or tested against it. Inline code and highlighted output share one second Expressive Code engine (`renderTokens()`). *Rejected:* the terminal title bar colour (lighter than the code in some themes).
Expand Down
30 changes: 20 additions & 10 deletions docs/e2e/runnable.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,17 @@ const JS = 1;
const TS = 2;
const PYPI = 3;
const PACKAGES = 4;
const ERROR = 5;
const LOOP = 6;
const SCRIPTED = 7;
const FAILED = 8;
const MARKS = 9;
const OUTPUT_ONLY = 10;
const ANSI = 11;
const ANSI_RUN = 12;
const RUN_OTHER = 13;
const TITLE = 14;
const CLI = 5;
const ERROR = 6;
const LOOP = 7;
const SCRIPTED = 8;
const FAILED = 9;
const MARKS = 10;
const OUTPUT_ONLY = 11;
const ANSI = 12;
const ANSI_RUN = 13;
const RUN_OTHER = 14;
const TITLE = 15;

test.beforeEach(async ({ page }) => {
await page.goto('./features/run-code/');
Expand Down Expand Up @@ -139,6 +140,15 @@ test('installs Python packages from PyPI, by import name or from runnable.packag
await expect(named.locator('.scb-run-stdout')).toHaveText('hello-world-run-code', { timeout: 120_000 });
});

test('a command line tool gets its arguments and program name, and sys.exit() shows no traceback', async ({ page }) => {
once();
test.setTimeout(240_000);
const cli = example(page, CLI);
await cli.locator('.scb-run').click();
await expect(cli.locator('.scb-run-stdout')).toContainText('Usage: hello.py [OPTIONS] NAME', { timeout: 180_000 });
await expect(cli.locator('.scb-run-stderr')).toHaveCount(0);
});

test('the button sits under the block, or in the title bar with runnable.button', async ({ page }) => {
await expect(example(page, JS).locator('.scb-run-controls .scb-run')).toBeVisible();
await expect(example(page, JS).locator('.header .scb-run')).toHaveCount(0);
Expand Down
7 changes: 6 additions & 1 deletion docs/src/content/docs/extend/add-a-runtime.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ A runtime is a module whose default export has two functions:
```ts
interface Runtime {
load(code: string, options?: { packages?: string[] }): Promise<void>;
run(code: string, options: { signal: AbortSignal; session?: boolean }): Promise<{ stdout: string; stderr: string }>;
run(
code: string,
options: { signal: AbortSignal; session?: boolean; args?: string[]; file?: string; columns?: number },
): Promise<{ stdout: string; stderr: string }>;
}
```

Expand All @@ -28,6 +31,8 @@ The browser imports the module when a reader clicks **Run code** for the first t

When the block is a session with prompts, `code` is its commands and `session` is `true`. Run the commands as the language's REPL does, so that the output shows the value of each expression, as the block does.

`args` holds the words of the block's `runnable.args` attribute, and `file` the title of the block, for a runtime with command line arguments. `columns` is the width of the output panel in characters, for code that fits its output to a terminal. The Python runtime gives them to the code as `sys.argv` and the `COLUMNS` environment variable.

The `signal` of `run()` aborts when the run takes longer than `runnable.timeout`. The runtime must then stop the code and reject the promise. The plugin shows the timeout message, and calls `load()` again before the next run.

## An example: the JavaScript runtime
Expand Down
37 changes: 37 additions & 0 deletions docs/src/content/docs/features/run-code.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Click **Run code**. The first time, the browser downloads Python (Pyodide, a bui
| `runnable.label="<text>"`, `runnable.againLabel="<text>"` | Code block fence line |
| `runnable.timeout=<ms>` | Code block fence line |
| `runnable.packages="<name> <name>"` | Code block fence line |
| `runnable.args="<arguments>"` | Code block fence line |
| `runnable.button="below"`, `runnable.button="title"`, `runnable.button="both"` | Code block fence line |
| `[!run]`, `[!run lang=<language>]`, `[!run end]` | Comment in the code, on its own line |
| `[!output]`, `[!output stdout]`, `[!output stderr]`, `[!output end]`, with `lang=<language>` after the stream | Comment in the code, on its own line |
Expand Down Expand Up @@ -79,6 +80,17 @@ The block needs no `runnable` attribute and no runtime, so it works in every lan

`runnable.output` on the fence line does the same, with `\n` for each new line.

### Command line arguments

`runnable.args` gives the code the arguments of a command line. Separate the arguments with spaces, and put quotes around an argument that has spaces, such as `runnable.args="--name 'Ada Lovelace'"`. The title of the block is the name of the program. A block without a title runs as `main.py`.

In Python, `sys.argv` holds the name and the arguments, as it does for `python hello.py --count 2`. So a command line tool made with [argparse](https://docs.python.org/3/library/argparse.html), [Click](https://click.palletsprojects.com/) or [Typer](https://typer.tiangolo.com/) runs with no change to its code:

- `sys.exit()` ends the run as it ends a program, with no traceback. An exit message that is not a number shows as standard error.
- The `COLUMNS` environment variable is the width of the output panel, in characters. Help text that reads it, such as the help of argparse and Rich, fits the panel. Click makes its help at least 50 characters wide, so on a narrow screen its help can scroll.

The JavaScript and TypeScript runtimes do not read the arguments.

### Output from a file

`runnable.outputFile` reads the output from a file when the site builds, such as a file that a tool writes from a real run. The path is relative to the page, as the path of an image is:
Expand Down Expand Up @@ -188,6 +200,31 @@ print(slugify("Hello, World: Run Code!"))

<Example code={packages} layout="tabs" hiddenAttributes="apiLinks=false" />

### A command line tool

The title gives the program name in the usage line, and `runnable.args` gives the arguments. The code is the same as in a file that runs in a terminal:

export const cli = `
\`\`\`py runnable title="hello.py" runnable.args="--help"
import click

@click.command()
@click.option("--count", default=1, help="Number of greetings.")
@click.argument("name")
def hello(count, name):
"""Greet NAME, COUNT times."""
for _ in range(count):
click.echo(f"Hello, {name}!")

if __name__ == "__main__":
hello()
\`\`\`
`;

<Example code={cli} layout="tabs" hiddenAttributes="apiLinks=false" />

The help of a [Typer](https://typer.tiangolo.com/) app needs two packages that Pyodide does not install with Rich, so add `runnable.packages="markdown-it-py pygments"` to the fence line. For the colours of [rich-click](https://ewels.github.io/rich-click/) or Typer, add [`runnable.outputLang=ansi`](#output-in-a-language), and set `FORCE_COLOR` in a [hidden line](/starlight-codeblocks/features/hidden-lines/).

### An error

An error in Python shows the traceback, from the line of the example that failed:
Expand Down
3 changes: 3 additions & 0 deletions packages/starlight-codeblocks/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@
- Split diff: a **Unified** and **Split** switch in the title bar of each block that removes and adds lines. **Split** shows the old code on the left and the new code on the right, as a GitHub pull request does. `splitDiff.view`, `splitDiff.switch` and `splitDiff.sync` set the start view, remove the switch and stop it syncing across the page (#37)
- Untitled blocks with title bar controls use the tab bar colours of titled blocks (#37)
- Skipped code: `[!code skip]` on its own line shows a marker in place of code that a block leaves out, at the indentation of its line. The text after the directive goes in the marker, else it reads "Code not shown". The line keeps its line number, and copies as a comment with the text, such as `// Not shown: Imports`. A comment that holds only `...`, such as `// ...`, and `...` alone on a line in JSON, also show as markers and copy as written. `skippedCode.commentAliases` and `skippedCode.bareAliases` change these aliases for each language. `skippedCode: false` turns the feature off (#28)
- Run code: `runnable.args="<arguments>"` gives the code command line arguments. In Python, `sys.argv` holds the title of the block (else `main.py`) and the arguments, so a Click, Typer or argparse tool runs with no change to its code (#43)
- Run code: in Python, `sys.exit()` ends a run as it ends a program: with no `SystemExit` traceback, and with an exit message that is not a number on standard error. Before, `--help` in a Click command showed a traceback (#43)
- Run code: the `COLUMNS` environment variable of a Python run is the width of the output panel in characters, so help text from argparse and Rich fits the panel. Custom runtimes get `args`, `file` and `columns` in the options of `run()` (#43)
- API links: a starlight-pydocs registry left over from an `astro dev` restart that removed starlight-pydocs from the config is dropped, so Python names no longer link to its old pages

## 1.4.1
Expand Down
23 changes: 22 additions & 1 deletion packages/starlight-codeblocks/src/client/runnable.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,20 @@ const status = (text: string) => line('p', 'scb-run-status', text);
const stdout = (text: string, ansi = false) => line('pre', 'scb-run-stdout', text, { ansi });
const error = (prefix: string, text: string, ansi = false) => line('pre', 'scb-run-stderr', text, { prefix, ansi });

/** The number of characters that fit on a line of the panel. */
function columns(panel: HTMLElement) {
const probe = stdout('0'.repeat(10));
// The panel is a live region.
probe.setAttribute('aria-hidden', 'true');
panel.append(probe);
const range = document.createRange();
range.selectNodeContents(probe);
// Both widths include any transform on the page, so that it cancels out.
const n = Math.floor((probe.getBoundingClientRect().width * 10) / range.getBoundingClientRect().width);
probe.remove();
return Number.isFinite(n) && n > 0 ? n : undefined;
}

/** Prints the output that the author wrote, a line at a time, as if the code ran. */
async function replay(figure: HTMLElement, panel: HTMLElement, output: string) {
const delay = Number(figure.dataset.scbRunnableDelay) || 0;
Expand Down Expand Up @@ -87,7 +101,14 @@ async function run(figure: HTMLElement, panel: HTMLElement) {
});
const timer = setTimeout(() => controller.abort(new DOMException('Timed out', 'TimeoutError')), timeout);
try {
const output = await Promise.race([runtime.run(code, { signal: controller.signal, session }), stopped]);
const options = {
signal: controller.signal,
session,
args: JSON.parse(figure.dataset.scbRunnableArgs ?? '[]'),
file: figure.dataset.scbRunnableFile,
columns: columns(panel),
};
const output = await Promise.race([runtime.run(code, options), stopped]);
const parts = [];
const ansi = figure.hasAttribute('data-scb-runnable-ansi');
if (output.stdout) parts.push(stdout(output.stdout, ansi));
Expand Down
3 changes: 2 additions & 1 deletion packages/starlight-codeblocks/src/expressive-code/core.ts
Original file line number Diff line number Diff line change
Expand Up @@ -242,12 +242,13 @@ export function blockSetting<T>(
parse: (raw: string) => T | undefined,
fallback: T,
expected: string,
otherwise = 'The block uses the site setting.',
): T {
const option = context.codeBlock.metaOptions.list(key).at(-1);
if (!option) return fallback;
const value = parse(String(option.value));
if (value !== undefined) return value;
warn(context, `\`${option.raw.trim()}\` must be ${expected}. The block uses the site setting.`);
warn(context, `\`${option.raw.trim()}\` must be ${expected}. ${otherwise}`);
return fallback;
}

Expand Down
36 changes: 35 additions & 1 deletion packages/starlight-codeblocks/src/expressive-code/runnable.ts
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,21 @@ async function readOutputFile(context: Parameters<typeof warn>[0], path: string)
}
}

const SHELL_WORD = /(?:"(?:[^"\\]|\\.)*"|'[^']*'|\\.|[^\s"'\\])+/g;

/**
* Splits text into words as a shell does: quotes keep spaces in a word, and a backslash escapes the next character.
* `undefined` for a quote that does not close.
*/
function shellWords(raw: string): string[] | undefined {
if (raw.replace(SHELL_WORD, '').trim()) return undefined;
return (raw.match(SHELL_WORD) ?? []).map((word) =>
word.replace(/"((?:[^"\\]|\\.)*)"|'([^']*)'|\\(.)/g, (_, double, single, escaped) =>
double !== undefined ? double.replace(/\\([\\"$`])/g, '$1') : (single ?? escaped),
),
);
}

const NEEDS_OUTPUT = '`runnable.static` needs output: `[!output]` lines, `runnable.output` or `runnable.outputFile`.';

/**
Expand Down Expand Up @@ -481,6 +496,8 @@ export function pluginRunnable({
metaOptions.getString('runnable.outputLang')?.trim() || (/\.ansi$/i.test(path ?? '') ? 'ansi' : undefined);
if (scripted !== undefined && run.length > 0)
warn(context, '`[!run]` has no effect in a block with scripted output: the button prints the output.');
if (scripted !== undefined && metaOptions.list('runnable.args').length > 0)
warn(context, '`runnable.args` has no effect in a block with scripted output: the button prints the output.');
const say = uiTexts(context.locale);
const buttonText = blockSetting(context, 'runnable.label', text, label ?? say('runnable.run'), 'some text');
const again = blockSetting(
Expand Down Expand Up @@ -644,8 +661,26 @@ export function pluginRunnable({
(raw) => raw.trim().split(/\s+/).join(' ') || undefined,
'',
'one or more package names',
'The block installs only the packages of its imports.',
);
if (packages) figure.properties.dataScbRunnablePackages = packages;
const name = bundledLanguage(language)?.name ?? language;
const args = blockSetting(
context,
'runnable.args',
shellWords,
[],
'words, with quotes that close',
'The block runs with no arguments.',
);
if (
args.length > 0 &&
language !== 'python' &&
modules[language] === BUILT_IN[language as keyof typeof BUILT_IN]
)
warn(context, `The ${name} runtime does not read \`runnable.args\`. The block runs without them.`);
if (args.length > 0) figure.properties.dataScbRunnableArgs = JSON.stringify(args);
if (codeBlock.props.title) figure.properties.dataScbRunnableFile = codeBlock.props.title;
keepCopiedText(renderData.blockAst, codeBlock.code);
if (run.length > 0) {
figure.properties.dataScbRunnableCode = encodeCode(run.map((line) => line.text).join('\n'));
Expand All @@ -657,7 +692,6 @@ export function pluginRunnable({
[...session].map(([line, prompt]) => line.text.slice(prompt.length)).join('\n'),
);
}
const name = bundledLanguage(language)?.name ?? language;
addButtons(
livePanel({
loading: say('runnable.loading', { language: name }),
Expand Down
Loading
Loading