Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .dockerignore

This file was deleted.

1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ node_modules/
data/
/logs/
.direnv/
.fake-backend-config.json
*.local

# Python caches
Expand Down
11 changes: 7 additions & 4 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,10 @@ honour. The reasons behind major choices are in the
[decision log](docs/agents/architecture/decisions.md).

> **Status:** target design, revised 27 October 2026 after architecture review. This design replaces the
> platform described in `docs/content/` (container-image jobs, dataset and
> result pools, team namespaces). It starts fresh: no users or data are migrated.
> previous container-image platform (dataset and result pools, team
> namespaces), which was never deployed and has been removed from the tree; it
> remains in Git history before this change. It starts fresh: no users or data
> are migrated.

## Goals

Expand Down Expand Up @@ -380,8 +382,9 @@ deploy/ k3s manifests, systemd units, host setup
web-interface/ portal
```

The existing `src/` and `cli/` modules belong to the platform being replaced.
They are removed once the new path serves jobs.
The previous platform's `src/`, `cli/`, `deploy/` and `web-interface/` were
removed on 27 October 2026; the portal is rebuilt in `web-interface/` against
the new API.

## Milestones

Expand Down
106 changes: 22 additions & 84 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,105 +1,43 @@
# Mist

UTMIST's research compute platform. Submit Python scripts or tagged container
images, choose CPU, NVIDIA GPUs or Tenstorrent boards, upload datasets, and
collect persistent results through the same website or CLI.
UTMIST's research compute platform. Mist works like the batch systems on
national HPC clusters: each researcher gets a folder inside their team folders,
logs in over SSH, moves files with `scp`/`rsync`, and runs batch or interactive
jobs that a scheduler places on NVIDIA GPU, CPU and Tenstorrent hardware.

**Private website:** [http://100.73.139.66:8088](http://100.73.139.66:8088), accessible through authorized
Tailscale connectivity. A Mist account and team membership are also required.
## Status

## Architecture redesign — October 26
Under redesign (October 2026). The previous container-image platform was never
deployed and has been removed from the tree; see Git history before 27 October
2026 if you need it. The new implementation starts from the design docs.

Mist is moving to an HPC-style design: per-team folders, restricted SSH login,
and jobs that run as the researcher's own Linux user under Mist's own scheduler.
Jobs no longer need images or mounts. Start with [ARCHITECTURE.md](ARCHITECTURE.md);
component contracts and the decision log are in
[docs/agents/architecture/](docs/agents/architecture/decisions.md). The
sections below and `docs/content/` describe the platform being replaced.
## Design

## Storage update — October 21

New submissions use ordinary team dataset/result folders on QuietBox, accessed
through the direct Ethernet link. Storage allowances are advisory; team access
is enforced. See [current storage operations](deploy/k3s/storage/README.md) for
layout, setup, API semantics, verification and rollback. The earlier operating
snapshots describe the retired image-backed layout.

## Current installation
- [ARCHITECTURE.md](ARCHITECTURE.md): system overview, components, owners,
milestones and code layout.
- [docs/agents/architecture/](docs/agents/architecture/decisions.md): per-component
contracts and the decision log.
- [docs/agents/plans/](docs/agents/plans/jobs-and-scheduling.md): implementation
plans per owner.

## Cluster

| Machine | Role | LAN address | Tailscale address |
| ---------------- | --------------------------------------------------------- | ------------ | ----------------- |
| `utmist-z1opa08` | k3s server, API/auth, website, CPU and two RTX A4000 GPUs | `10.0.0.175` | `100.73.139.66` |
| `utmist-tt` | k3s worker, four n300 boards/eight chips, shared storage | `10.0.0.112` | `100.95.175.37` |


The cluster and NFS use the existing LAN. Tailscale provides private portal and
administrator SSH access. Kubernetes/containerd run workload containers.

## Documentation

Start with the [documentation index](docs/README.md).
| `utmist-z1opa08` | Two RTX A4000 GPUs, CPU | `10.0.0.175` | `100.73.139.66` |
| `utmist-tt` | Four n300 Tenstorrent boards (eight chips), shared storage | `10.0.0.112` | `100.95.175.37` |

- [Using the portal, teams and jobs](docs/content/department-rollout.md)
- [k3s setup, accelerator allocation and diagnostics](docs/content/k3s-setup.md)
- [Tailscale and SSH setup](docs/content/tailscale-ssh.md)
- [Architecture and code organization](docs/content/architecture.md)
- [Storage and upload limits](docs/content/storage.md)
- [Deployment and operations](deploy/private/README.md)
- [Developer setup](docs/content/development.md), [CLI](cli/docs/setup.md), [testing](docs/content/testing.md)
## Development

Main contains the department release merged through
[PR #110](https://github.com/UTMIST/Mist/pull/110). The earlier main and operating
snapshots are preserved in the [historical archive](docs/content/archive/_index.md).

## Development and checks

Use Go **1.25.1** and Node.js **22.16.0 or newer**. The Go workspace contains the
API (`src`) and CLI (`cli`). From the repository root:
A Nix dev shell (`flake.nix`, `shell.nix`) provides Go, Node.js, Python, k3s and
kubectl. Check documentation links with:

```bash
go test ./src/... ./cli/...
npm --prefix web-interface ci
npm --prefix web-interface test
npm --prefix web-interface run lint
npm --prefix web-interface run build
python3 scripts/check-docs.py
```

For a local UI against the authenticated deployed API, follow
[developer setup](docs/content/development.md). For the existing installation, deploy with
`bash deploy/private/build-and-deploy.sh`; read its [prerequisites](deploy/private/README.md)
before running it. Fresh-host provisioning is covered separately by the k3s guide.

The current API uses Kubernetes only. The unfinished Redis/Docker supervisor,
fake frontend account/chart and dummy CLI configuration have been retired.
Docker remains build tooling for images; it does not schedule Mist jobs.

Custom accelerator images need compatible training libraries. Credits, Jupyter,
chaining, browser image archives/builds, private pull-credential management and
distributed multi-machine training are outside this release. Automatic backups,
HTTPS and monitoring/cleanup products were explicitly excluded. This is a private
HTTP installation with one control plane, one auth instance and one storage host.

### Documentation

Humans are responsible for editing the human facing docs. This is because the human facing docs will be read by humans, and it is rude to give them untested instructions to follow.

Documentation should include explanations of design decisions for later platform maintainers.

## Frontend Testing

how to use fake backend for testing frontend.

(temporary, functionality, to be removed in future commit.)

```bash
# Terminal 1
npm --prefix web-interface run dev -- --host 127.0.0.1 --port 3001 --strictPort

# Terminal 2
npm --prefix web-interface run fake-backend
```



 
83 changes: 0 additions & 83 deletions cli/cmd/api_client.go

This file was deleted.

13 changes: 0 additions & 13 deletions cli/cmd/auth.go

This file was deleted.

Loading
Loading