Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 44 additions & 97 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,115 +1,62 @@
# Mist

UTMIST's compute platform. The Kubernetes pilot runs submitted CPU, NVIDIA,
and Tenstorrent workloads as real Jobs. The Jobs page and CLI use the same
API for submission, status, logs, and cancellation. The local foundation adds
live hardware inventory, self-service container images, team workspaces,
scoped shared storage and a durable fair job queue.
UTMIST's research compute platform. Submit Python scripts or tagged container
images, choose CPU, NVIDIA GPUs or Tenstorrent boards, upload datasets, and
collect persistent results through the same website or CLI.

## Current pilot
**Private website:** http://100.73.139.66:8088, accessible through authorized
Tailscale connectivity. A Mist account and team membership are also required.

- `utmist-z1opa08`: k3s server, CPU workloads, two NVIDIA RTX A4000 GPUs.
- `utmist-tt`: QuietBox worker, four n300 boards with two Wormhole chips each.
- Request one or two NVIDIA GPUs, or one to four Tenstorrent boards. Other
jobs can use the remaining devices; requests wait when capacity is occupied.
- Website: http://100.73.139.66:8088. API on that origin: `/api`; auth: `/auth`.
## Current installation

The department rollout includes admin/member login, team common/member folders,
explicit cross-team read/use grants, enforced storage/resource limits, durable
fair admission and revocation. The portal uses interactive compute choices,
clear feature panels and searchable, paginated job rows that expand for details.
Use **http://100.73.139.66:8088** privately through Tailscale.
| Machine | Role | LAN address | Tailscale address |
|---|---|---|---|
| `utmist-z1opa08` | k3s server, API/auth, website, CPU and two RTX A4000 GPUs | `10.0.0.175` | `100.73.139.66` |
| `utmist-tt` | k3s worker, four n300 boards/eight chips, shared storage | `10.0.0.112` | `100.95.175.37` |

See [the current department operating guide](docs/department-rollout.md),
[private deployment operations](deploy/private/README.md),
[completed execution plan and evidence](docs/department-rollout-execution.md),
[portal design](docs/design-system.md), and
[release testing](docs/testing.md). The earlier foundation guides record
historical phases.
The cluster and NFS use the existing LAN. Tailscale provides private portal and
administrator SSH access. Kubernetes/containerd run workload containers.

Credits, Jupyter, chaining, browser image archives/builds and arbitrary distributed
training remain outside this release. Custom TT code/images must use compatible
TT libraries. Automatic backups, HTTPS and monitoring/cleanup products were
explicitly excluded. Hosting remains private HTTP over Tailscale.
## Documentation

## Developer preview against this cluster
Start with the [documentation index](docs/README.md).

**Keep production authentication/team enforcement enabled for research users.**
The standalone trusted-owner mode below is only for development.
- [Using the portal, teams and jobs](docs/department-rollout.md)
- [k3s setup, accelerator allocation and diagnostics](docs/k3s-setup.md)
- [Tailscale and SSH setup](docs/tailscale-ssh.md)
- [Architecture and code organization](docs/architecture.md)
- [Storage and upload limits](docs/storage.md)
- [Deployment and operations](deploy/private/README.md)
- [Developer setup](docs/development.md), [CLI](cli/docs/setup.md), [testing](docs/testing.md)

Use Go 1.25.1 and Node.js 22.16.0 or newer. Apply the workload prerequisites
and claim templates described in the deployment guide first. Kubernetes is
now the default executor; Redis and Docker supervisors are unnecessary for
this path. This server has both runtimes under `~/.local/share/mist-runtimes`:
Main contains the department release merged through
[PR #110](https://github.com/UTMIST/Mist/pull/110). The earlier main and operating
snapshots are preserved in the [historical archive](docs/archive/README.md).

```bash
export PATH=/home/utmist/.local/share/mist-runtimes/go-1.25.1/bin:/home/utmist/.local/share/mist-runtimes/node-22.16.0/bin:$PATH
```

```bash
export KUBECONFIG=/home/utmist/.kube/config
# Standalone development defaults to a trusted local owner with auth disabled.
# For the deployed authenticated/shared API, use its existing port-forward.
export MIST_PILOT_OWNER=local-dev
cd src
go run .
```

If the deployed API's port-forward already occupies port 3000, use that API
or stop only `mist-api-forward.service` before starting a local backend.
Standalone `go run` uses the default base-image allowlist. Set
`MIST_ALLOWED_IMAGES` to the deployment's approved references when developing
with the packaged example images or other custom images.

In another terminal:

```bash
cd web-interface
npm ci
npm run dev -- --host 127.0.0.1 --port 3001 --strictPort
```
## Development and checks

The Vite proxy forwards `/api` and `/auth` to the API on port 3000. The installed user
services already run both endpoints on this machine:
Use Go **1.25.1** and Node.js **22.16.0 or newer**. The Go workspace contains the
API (`src`) and CLI (`cli`). From the repository root:

```bash
systemctl --user status mist-api-forward mist-web
go test ./src/... ./cli/...
npm --prefix web-interface ci
npm --prefix web-interface test
npm --prefix web-interface run lint
npm --prefix web-interface run build
```

## CLI

From the repository root:

```bash
go -C cli build -o ../bin/mist .
printf "print('Hello from a real Mist job')\n" > /tmp/hello.py
export MIST_API_URL=http://100.73.139.66:8088/api
bin/mist auth login --email your-member-email@example.org
bin/mist job submit /tmp/hello.py --compute CPU
bin/mist job list --all
bin/mist job status <job-id>
bin/mist job logs <job-id>
bin/mist job cancel <active-job-id>
```
For a local UI against the authenticated deployed API, follow
[developer setup](docs/development.md). For the existing installation, deploy with
`bash deploy/private/build-and-deploy.sh`; read its [prerequisites](deploy/private/README.md)
before running it. Fresh-host provisioning is covered separately by the k3s guide.

Use `--compute NVIDIA --devices 1` for one GPU, or
`--compute TT --devices 1` for one two-chip board. The submitted script must
actually use the chosen accelerator. `--api-url` or `MIST_API_URL` selects an
API endpoint. See [CLI setup](cli/docs/setup.md) for resource options.

## Tests and legacy development

```bash
go -C src test -run 'Test(Kubernetes|Hardware|GPURequest|CustomImage|TenstorrentCustom|Shared|Member)' ./...
go -C cli test ./...
cd web-interface
npm test -- --run
npm run build
npx tsc --noEmit
```
The current API uses Kubernetes only. The unfinished Redis/Docker supervisor,
fake frontend account/chart and dummy CLI configuration have been retired.
Docker remains build tooling for images; it does not schedule Mist jobs.

The older Docker/Redis executor is available with `MIST_EXECUTOR=docker`.
Its supervisor execution is incomplete. For its existing tests, start a
local Redis instance and run `go -C src test ./...`; Docker integration tests
also need Docker permissions and their test image. This path is retained for
compatibility and is separate from the validated Kubernetes executor.
Custom accelerator images need compatible training libraries. Credits, Jupyter,
chaining, browser image archives/builds, private pull-credential management and
distributed multi-machine training are outside this release. Automatic backups,
HTTPS and monitoring/cleanup products were explicitly excluded. This is a private
HTTP installation with one control plane, one auth instance and one storage host.
5 changes: 3 additions & 2 deletions auth-service/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,6 @@ Database-backed rate limiting survives restarts. Upload/storage metadata live
in Go/NFS, not in this auth database. Member passwords use the library's password
implementation; Go does not hash passwords or mint authentication tokens.

See [operations and recovery](../deploy/private/README.md) for online backups,
restoration, private HTTP/HTTPS configuration and deployment.
See [deployment operations](../deploy/private/README.md) for persistent-state
handling and private HTTP deployment. Automatic backups/recovery and HTTPS were
excluded from this release; the existing backup script is a manual utility.
49 changes: 12 additions & 37 deletions cli/cmd/config.go
Original file line number Diff line number Diff line change
@@ -1,40 +1,15 @@
package cmd

import "fmt"

// Config flags
type ConfigCmd struct {
DefaultCluster string `help:"Set the default compute cluster." optional: ""`
Show bool `help:"Show current configuration."`
}

func (h *ConfigCmd) Run(ctx *AppContext) error {
// Some dummy config; Call API or something
defaultConfig := map[string]string{
"defaultCluster": "AMD-cluster-1",
"region": "us-east",
}

if h.Show && h.DefaultCluster != "" {
fmt.Printf("Cannot use --show and --default-cluster together")
return nil
}

if h.DefaultCluster != "" {
// This is not actually set.
fmt.Printf("Setting default cluster to: %s\n", h.DefaultCluster)
return nil
}

if h.Show {
fmt.Println("Current configuration: ")
for key, value := range defaultConfig {
fmt.Printf(" %s: %s \n", key, value)
}
return nil
}

fmt.Println("No config action specified. Use --help for options.")

return nil
import (
"os"
"path/filepath"
)

func defaultConfigPath() string {
dir, err := os.UserConfigDir()
if err != nil {
home, _ := os.UserHomeDir()
return filepath.Join(home, ".config", "mist", "config.json")
}
return filepath.Join(dir, "mist", "config.json")
}
58 changes: 0 additions & 58 deletions cli/cmd/config_test.go

This file was deleted.

12 changes: 2 additions & 10 deletions cli/cmd/job.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,7 @@ package cmd
type JobCmd struct {
Submit JobSubmitCmd `cmd:"" help:"Submit a new job"`
Cancel JobCancelCmd `cmd:"" help:"Cancel an existing job"`
// Delete JobDeleteCmd `cmd: "" help: "Delete an existing job"`
Status JobStatusCmd `cmd:"" help:"Check the status of a job"`
// Cancel CancelCmd `cmd:"" help:"Cancel a running job"`
Logs JobLogsCmd `cmd:"" help:"Read workload logs"`
List ListCmd `cmd:"" help:"List all jobs" default:1`
}

func (j *JobCmd) Run() error {
// Possible fallback if no subcommand is providFAre yed
// fmt.Println("(job root) – try 'mist job submit|status|list|cancel' or mist help")
return nil
Logs JobLogsCmd `cmd:"" help:"Read workload logs"`
List ListCmd `cmd:"" help:"List jobs"`
}
4 changes: 0 additions & 4 deletions cli/cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ type Config struct {
SessionCookie string `json:"session_cookie,omitempty"`
APIBaseURL string `json:"api_base_url,omitempty"`
TeamID string `json:"team_id,omitempty"`
// maybe APIBaseURL, etc.
}

type AppContext struct {
Expand All @@ -34,13 +33,10 @@ type Globals struct {
type CLI struct {
Globals

// Define your CLI structure here: Top Level Commands
Team TeamCmd `cmd:"" help:"List and select team workspaces"`
Auth AuthCmd `cmd:"" help:"Authentication commands"`
Job JobCmd `cmd:"" help:"Job management commands"`
// Config ConfigCmd `cmd:"" help:"Configuration commands"`
Help HelpCmd `cmd:"" help:"Show help information"`
// Config ConfigCmd `cmd:"" help: "Display Cluster Configuration"`
}

func loadConfig(path string) (*Config, error) {
Expand Down
72 changes: 0 additions & 72 deletions cli/cmd/testutil.go

This file was deleted.

Loading
Loading