Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .dockerignore

This file was deleted.

1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ node_modules/
data/
/logs/
.direnv/
.fake-backend-config.json
*.local

# Python caches
Expand Down
411 changes: 411 additions & 0 deletions ARCHITECTURE.md

Large diffs are not rendered by default.

101 changes: 29 additions & 72 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,96 +1,53 @@
# Mist

UTMIST's research compute platform. Submit Python scripts or tagged container
images, choose CPU, NVIDIA GPUs or Tenstorrent boards, upload datasets, and
collect persistent results through the same website or CLI.
UTMIST's research compute platform. Mist works like the batch systems on
national HPC clusters: each researcher gets a folder inside their team folders,
logs in over SSH, moves files with `scp`/`rsync`, and runs batch or interactive
jobs that a scheduler places on NVIDIA GPU, CPU and Tenstorrent hardware.

**Private website:** [http://100.73.139.66:8088](http://100.73.139.66:8088), accessible through authorized
Tailscale connectivity. A Mist account and team membership are also required.
## Status

## Storage update — October 21
Under redesign (October 2026). The previous container-image platform was never
deployed and has been removed from the tree; see Git history before 27 October
2026 if you need it. The new implementation starts from the design docs.

New submissions use ordinary team dataset/result folders on QuietBox, accessed
through the direct Ethernet link. Storage allowances are advisory; team access
is enforced. See [current storage operations](deploy/k3s/storage/README.md) for
layout, setup, API semantics, verification and rollback. The earlier operating
snapshots describe the retired image-backed layout.
## Design

## Current installation
- [ARCHITECTURE.md](ARCHITECTURE.md): system overview, components, owners,
milestones and code layout.
- [docs/agents/architecture/](docs/agents/architecture/decisions.md): per-component
contracts and the decision log.
- [docs/agents/plans/](docs/agents/plans/jobs-and-scheduling.md): implementation
plans per owner.

## Cluster

| Machine | Role | LAN address | Tailscale address |
| ---------------- | --------------------------------------------------------- | ------------ | ----------------- |
| `utmist-z1opa08` | k3s server, API/auth, website, CPU and two RTX A4000 GPUs | `10.0.0.175` | `100.73.139.66` |
| `utmist-tt` | k3s worker, four n300 boards/eight chips, shared storage | `10.0.0.112` | `100.95.175.37` |
| `utmist-z1opa08` | Two RTX A4000 GPUs, CPU | `10.0.0.175` | `100.73.139.66` |
| `utmist-tt` | Four n300 Tenstorrent boards (eight chips), shared storage | `10.0.0.112` | `100.95.175.37` |

## Development

The cluster and NFS use the existing LAN. Tailscale provides private portal and
administrator SSH access. Kubernetes/containerd run workload containers.

## Documentation

Start with the [documentation index](docs/README.md).

- [Using the portal, teams and jobs](docs/content/department-rollout.md)
- [k3s setup, accelerator allocation and diagnostics](docs/content/k3s-setup.md)
- [Tailscale and SSH setup](docs/content/tailscale-ssh.md)
- [Architecture and code organization](docs/content/architecture.md)
- [Storage and upload limits](docs/content/storage.md)
- [Deployment and operations](deploy/private/README.md)
- [Developer setup](docs/content/development.md), [CLI](cli/docs/setup.md), [testing](docs/content/testing.md)

Main contains the department release merged through
[PR #110](https://github.com/UTMIST/Mist/pull/110). The earlier main and operating
snapshots are preserved in the [historical archive](docs/content/archive/_index.md).

## Development and checks

Use Go **1.25.1** and Node.js **22.16.0 or newer**. The Go workspace contains the
API (`src`) and CLI (`cli`). From the repository root:
A Nix dev shell (`flake.nix`, `shell.nix`) provides Go, Node.js, Python, k3s and
kubectl. Run the Go tests with:

```bash
go test ./src/... ./cli/...
npm --prefix web-interface ci
npm --prefix web-interface test
npm --prefix web-interface run lint
npm --prefix web-interface run build
go test ./...
```

For a local UI against the authenticated deployed API, follow
[developer setup](docs/content/development.md). For the existing installation, deploy with
`bash deploy/private/build-and-deploy.sh`; read its [prerequisites](deploy/private/README.md)
before running it. Fresh-host provisioning is covered separately by the k3s guide.
Database tests start an embedded Postgres as the current user (downloaded to
`~/.embedded-postgres-go` on first run). To use an existing server instead, set
`MIST_TEST_DATABASE_URL` to a superuser URL; each test creates its own database.

The current API uses Kubernetes only. The unfinished Redis/Docker supervisor,
fake frontend account/chart and dummy CLI configuration have been retired.
Docker remains build tooling for images; it does not schedule Mist jobs.
Check documentation links with:

Custom accelerator images need compatible training libraries. Credits, Jupyter,
chaining, browser image archives/builds, private pull-credential management and
distributed multi-machine training are outside this release. Automatic backups,
HTTPS and monitoring/cleanup products were explicitly excluded. This is a private
HTTP installation with one control plane, one auth instance and one storage host.
```bash
python3 scripts/check-docs.py
```

### Documentation

Humans are responsible for editing the human facing docs. This is because the human facing docs will be read by humans, and it is rude to give them untested instructions to follow.

Documentation should include explanations of design decisions for later platform maintainers.

## Frontend Testing

how to use fake backend for testing frontend.

(temporary, functionality, to be removed in future commit.)

```bash
# Terminal 1
npm --prefix web-interface run dev -- --host 127.0.0.1 --port 3001 --strictPort

# Terminal 2
npm --prefix web-interface run fake-backend
```



 
83 changes: 0 additions & 83 deletions cli/cmd/api_client.go

This file was deleted.

13 changes: 0 additions & 13 deletions cli/cmd/auth.go

This file was deleted.

148 changes: 0 additions & 148 deletions cli/cmd/auth_login.go

This file was deleted.

Loading