Skip to content

Commit 0a513c3

Browse files
Merge pull request #44 from Datasance/release/3.9.0
Release/3.9.0
2 parents 81a352b + b7b3824 commit 0a513c3

167 files changed

Lines changed: 22615 additions & 1952 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.env.example‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ NODE_ENV=development
66

77
# EdgeOps Console static embed (npm run build:console → dev/console/build)
88
EDGEOPS_CONSOLE_PATH=dev/console/build # must be absolute path
9-
EDGEOPS_CONSOLE_VERSION=v1.0.13
9+
EDGEOPS_CONSOLE_VERSION=v1.1.0-rc.6
1010
# EDGEOPS_CONSOLE_REPO=https://github.com/Datasance/edgeops-console
1111
# EDGEOPS_CONSOLE_FLAVOR=datasance
1212

‎.github/actions/set-build-env/action.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ runs:
88
shell: bash
99
run: |
1010
VERSION="${{ env.EDGEOPS_CONSOLE_VERSION }}"
11-
if [ -z "$VERSION" ]; then VERSION="1.0.13"; fi
11+
if [ -z "$VERSION" ]; then VERSION="1.1.0-rc.6"; fi
1212
echo "EDGEOPS_CONSOLE_VERSION=$VERSION" >> "${GITHUB_ENV}"
1313
1414
REPO="${{ env.EDGEOPS_CONSOLE_REPO }}"

‎CHANGELOG.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,129 @@
11
# Changelog
22

3+
## [v3.9.0-rc.9]
4+
5+
### Added
6+
7+
- **Fog status (host capacity + OS)** — agent `PUT /api/v3/agent/status` accepts and persists `systemCpus`, `systemTotalMemory`, `systemAvailableMemory`, `systemTotalDisk`, `systemAvailableDisk`, `systemOs`, `systemOsVersion`, and `systemKernelVersion`. User `GET /api/v3/iofog` and list return the stored values. `systemTotalCpu` remains host CPU busy **0–100%**; `diskUsage` remains Edgelet data-directory usage in **GiB**, not host filesystem totals. Columns on 3.9.0 baseline and 3.8→3.9 upgrade (sqlite, mysql, postgres). Omitted keys leave prior values unchanged.
8+
9+
### Changed
10+
- Embedded **EdgeOps Console** default version **v1.1.0-rc.5** → **v1.1.0-rc.6**
11+
12+
## [v3.9.0-rc.8]
13+
14+
### Changed
15+
- Embedded **EdgeOps Console** default version **v1.1.0-rc.4** → **v1.1.0-rc.5**
16+
17+
## [v3.9.0-rc.7]
18+
19+
### Changed
20+
21+
- **Fog status** — user GET and list return `knowledgeStatus` as the stored JSON string, matching `modelStatus`. Callers parse it.
22+
- Dockerfile runtime base image digest pin refreshed for **`ubi9/nodejs-24-minimal`**.
23+
24+
## [v3.9.0-rc.6]
25+
26+
Fleet Knowledge artifacts: curated documents, datasets, and vector indexes distributed like Models, with a separate name namespace and Hub dataset pulls.
27+
28+
### Added
29+
30+
- **Fleet Knowledge API** — `GET/POST/PATCH/DELETE /api/v3/knowledge`, YAML create/upsert (`kind: Knowledge`), and volumeMount-style `GET/POST/DELETE …/knowledge/:name/link`; server-generated uuid; agent `GET /api/v3/agent/knowledge` returns linked rows only (`{ "knowledge": [ … ] }`).
31+
- **`PATCH /api/v3/microservices/:uuid/knowledge`** — user microservices only; `microserviceKnowledge` change flag and rebuild rules for bind path / permissions / empty↔non-empty catalog transitions.
32+
- **Fog status** — `knowledgeStatus` (JSON string), `activeKnowledge` (managed count), `knowledgeLastUpdate` (Unix ms).
33+
- **v3.9.0 schema** — `Knowledge`, `FogKnowledge`, catalog tables, change-tracking flags, fog status columns on baseline and 3.8→3.9 upgrade.
34+
35+
### Changed
36+
37+
- **`modelLastUpdate`** — documented as Unix milliseconds (same clock as `knowledgeLastUpdate`).
38+
- **MS catalog validation** — `knowledge.bindPath` must not collide with volumes, tmpfs, or the models catalog path. A non-empty `bindPath` on a models or knowledge catalog requires at least one item.
39+
- **RBAC + swagger** — resource `knowledge`; viewer get/list; SRE/developer/admin full + link.
40+
- Dockerfile base image digest pins refreshed for **`node:24-bookworm`** (console-builder and builder stages)
41+
- Embedded **EdgeOps Console** default version **v1.1.0-rc.3** → **v1.1.0-rc.4**
42+
43+
### Notes
44+
45+
- Knowledge names are a separate namespace from Models. HF Knowledge uses the Hub dataset API; there is no `repoType` field.
46+
- `format` is a soft hint. Known values persist; anything else stores `unknown`. Unknown format is not a 400.
47+
- `getChanges.prune` still does not delete volumes. Unused **local** Knowledge prune is agent-side on the same flag. `system prune` does not drop Knowledge trees.
48+
49+
## [v3.9.0-rc.5]
50+
51+
Additive microservice `VOLUME` mapping `scope` for Edgelet private vs node-global shared claims. Reclaim stays on-node (`edgelet volume`); no new Controller prune flag.
52+
53+
### Added
54+
55+
- **`volumeMappings[].scope`** — `private` (default) or `shared`. Meaningful only when `type` is `volume`. Omit, null, or empty → `private`. Unknown values on `type: volume` are **400**. Other mapping types always store `private`.
56+
- **`VolumeMappings.scope`** column on 3.9.0 baseline and 3.8→3.9 upgrade (sqlite, mysql, postgres).
57+
58+
### Changed
59+
60+
- **Agent microservice list** includes `scope` on each volume mapping (lowercase).
61+
- **Volume-mapping create/delete** sets microservice `rebuild` and `microserviceCommon` so the agent applies the change.
62+
- **`type: volume` `hostDestination`** charset matches Edgelet: `[a-zA-Z0-9][a-zA-Z0-9_.-]*`.
63+
- Embedded **EdgeOps Console** default version **v1.1.0-rc.2** → **v1.1.0-rc.3**
64+
65+
### Notes
66+
67+
- Shared names are node-global. Controller does not unique-check names or `runAsUser` across consumers (local workloads exist). Use distinct names (`nodered-config`).
68+
- Controller and system microservices always persist `private`. Explicit `scope: shared` on those workloads is **400**.
69+
- `deleteWithCleanup` remains unused; operators reclaim with `edgelet volume` / `deprovision --purge-volumes`.
70+
- Switching private → shared does not copy existing UUID directories. Older agents ignore `scope` (every `VOLUME` stays per-UUID) with no merge after upgrade.
71+
72+
## [v3.9.0-rc.3]
73+
74+
Observed microservice status and fog liveness: live CPU/memory only while a microservice is RUNNING, and node quietness judged by when Controller last received a status PUT.
75+
76+
### Fixed
77+
78+
- **Stopped/inactive microservices** no longer keep the last CPU/memory sample after stop, deactivate, or a non-RUNNING agent report.
79+
- **Fog liveness** uses Controller receipt time (`lastStatusTime`), not the agent's clock, so clock skew no longer flaps nodes UNKNOWN.
80+
- **Status jobs** walk the fleet in small transactions and no longer delete microservices from the liveness cleaner.
81+
82+
### Added
83+
84+
- **Microservice last-crash columns** — `last_error`, `last_error_at`, `restart_count` on `MicroserviceStatuses` (baseline 3.9.0 and 3.8→3.9 upgrade).
85+
86+
### Changed
87+
88+
- **GET microservice meters** — `status.cpuUsage` and `status.memoryUsage` are 0 unless observed `status` is `RUNNING`.
89+
- **Operator stop / application deactivate** — GET shows `STOPPING` and zero meters immediately, before the next agent PUT.
90+
- **Microservice last crash** — GET `status` always includes `lastError`, `lastErrorAt`, and `restartCount`. `errorMessage` is the current failure (cleared by the agent after 30s RUNNING); last crash is kept after recovery.
91+
- Embedded **EdgeOps Console** default version **v1.1.0-rc.1** → **v1.1.0-rc.2**
92+
93+
## [v3.9.0-rc.2]
94+
95+
### Fixed
96+
97+
- **`PATCH /api/v3/iofog/{uuid}` after Controller upgrade** — stale `bluetoothEnabled` / `abstractedHardwareEnabled` keys in `"FogPlatformSpecs".spec_json` (written before HAL removal) caused validation failure with misleading `Invalid fog platform spec field 'instance'`. Platform spec merge and upsert now whitelist allowed fields only; validation errors name the rejected field.
98+
- **`POST /api/v3/runtimeClasses/{name}/link`** — refused link when the RuntimeClass name is not in the target fog `availableRuntimes` (same rule as microservice deploy runtime validation).
99+
100+
101+
## [v3.9.0-rc.1]
102+
103+
Fleet models, RuntimeClass, registry extensions, microservice catalog PATCH, MicroserviceTemplate, and HAL/BLE removal for Edgelet v1.1+ handoff. Targets **v3.9.0** schema (user tags).
104+
105+
### Added
106+
107+
- **Fleet models API** — `GET/POST/PATCH/DELETE /api/v3/models`, YAML create/upsert, and volumeMount-style `GET/POST/DELETE …/models/:name/link`; server-generated uuid; agent `GET /api/v3/agent/models` returns linked rows only.
108+
- **RuntimeClass API** — `GET/POST/PATCH/DELETE /api/v3/runtimeClasses`, YAML, link endpoints (edgelet fogs only), and agent `GET /api/v3/agent/runtimeClasses`.
109+
- **Registry `type` / `ca` / `insecure`** — Hub system registry by Hugging Face url+`hf`; MS and catalog image pulls remain **oci-only**.
110+
- **`PATCH /api/v3/microservices/:uuid/models`** — `microserviceModels` change flag and rebuild rules for bind path / permissions / empty↔non-empty catalog transitions.
111+
- **Microservice container fields** — commands/cmd alias, CDI devices, capabilities, health check, and related agent wire fields.
112+
- **MicroserviceTemplate** — CRUD + YAML; deploy overlay via `POST /microservices` + `template`.
113+
- **Agent status ingest** — `modelStatus`, `activeModels`, `modelLastUpdate`, `runtimeClasses`, `availableCdiDevices`, `podId`; getChanges flags `models`, `runtimeClasses`, `microserviceModels`.
114+
- **v3.9.0 schema** — `Models`, `RuntimeClasses`, link tables, registry columns, template tables. Fresh install uses baseline 3.9.0 migrate+seed; upgrade from 3.8.0 applies incremental 3.9.0 only.
115+
116+
### Changed
117+
118+
- **MS deploy** — auto-links referenced fleet models and RuntimeClass on edgelet agents; validates runtime against `availableRuntimes`.
119+
- **RBAC + swagger** — models, runtimeClasses, microserviceTemplates, and agent model/runtime routes documented and catalogued.
120+
- Embedded **EdgeOps Console** default version **v1.0.13** → **v1.1.0-rc.1**
121+
122+
### Removed
123+
124+
- **HAL/BLE provisioning** — user HAL/USB routes, RESTBlue/HAL system catalog items, fog `bluetooth`/`hal` flags, and `deviceScanFrequency` from agent config wire (Edge Guard retained).
125+
- **Agent HAL routes** — hardware inventory PUT/GET paths under `/api/v3/agent/hal/*`.
126+
3127

4128
## [v3.8.3-rc.3] - September 2026
5129

‎Dockerfile‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,10 @@
22
# ioFog overrides: EDGEOPS_CONSOLE_REPO=https://github.com/eclipse-iofog/edgeops-console
33
# EDGEOPS_CONSOLE_FLAVOR=iofog
44
# node:24-bookworm — pin manifest list digest for reproducible multi-arch builds
5-
FROM node:24-bookworm@sha256:be23f54a88d34e8824c741b19b91064094f92c1c97b194144bfc8b50d67258e2 AS console-builder
5+
FROM node:24-bookworm@sha256:64af3819f9275802414d7cdc38c27e9d82bd564dec4d4da87d008255d36c63b4 AS console-builder
66

77
ARG EDGEOPS_CONSOLE_REPO=https://github.com/Datasance/edgeops-console
8-
ARG EDGEOPS_CONSOLE_VERSION=v1.0.13
8+
ARG EDGEOPS_CONSOLE_VERSION=v1.1.0-rc.6
99
ARG EDGEOPS_CONSOLE_FLAVOR=datasance
1010

1111
RUN apt-get update \
@@ -28,7 +28,7 @@ RUN test -f build/index.html \
2828
&& cp -a build /tmp/console/build
2929

3030

31-
FROM node:24-bookworm@sha256:be23f54a88d34e8824c741b19b91064094f92c1c97b194144bfc8b50d67258e2 AS builder
31+
FROM node:24-bookworm@sha256:64af3819f9275802414d7cdc38c27e9d82bd564dec4d4da87d008255d36c63b4 AS builder
3232

3333
ARG PKG_VERSION
3434

@@ -48,9 +48,9 @@ RUN npm pack
4848

4949

5050
# ubi9/nodejs-24-minimal:latest — pin manifest list digest for reproducible multi-arch builds
51-
FROM registry.access.redhat.com/ubi9/nodejs-24-minimal@sha256:c7b8cb436feace96ca0b43eab7326394893e0ac9c2c6b46895727e3b9202fb9d
51+
FROM registry.access.redhat.com/ubi9/nodejs-24-minimal@sha256:e3392c129520d2ebce748c3d0048343d0ec60be560cf220b60a7ba5d2bd2484a
5252

53-
ARG EDGEOPS_CONSOLE_VERSION=v1.0.13
53+
ARG EDGEOPS_CONSOLE_VERSION=v1.1.0-rc.6
5454
ARG IMAGE_REGISTRY
5555
ARG OCI_SOURCE_REPO
5656
ARG CONTROLLER_DISTRIBUTION=iofog

‎Makefile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# Local Docker build — mirrors CI/release build-args (see .github/actions/set-build-env).
2-
# Override any variable: make build FLAVOR=iofog EDGEOPS_CONSOLE_VERSION=v1.0.13
2+
# Override any variable: make build FLAVOR=iofog EDGEOPS_CONSOLE_VERSION=v1.1.0-rc.6
33

44
FLAVOR ?= datasance
55
IMAGE_NAME ?= controller
@@ -25,7 +25,7 @@ else
2525
$(error FLAVOR must be "datasance" or "iofog", got "$(FLAVOR)")
2626
endif
2727

28-
EDGEOPS_CONSOLE_VERSION ?= v1.0.13
28+
EDGEOPS_CONSOLE_VERSION ?= v1.1.0-rc.6
2929

3030
IMAGE_REF = $(IMAGE_REGISTRY)/$(IMAGE_NAME):$(DOCKER_TAG)
3131

‎README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,9 @@ API: `http://localhost:51121` · Console (embedded or split): set `CONSOLE_URL`
125125
|-------|-----|
126126
| Architecture | [docs/architecture.md](docs/architecture.md) |
127127
| RBAC | [docs/rbac-reference.md](docs/rbac-reference.md) |
128-
| External OIDC | [docs/external-oidc-client-setup.md](docs/external-oidc-client-setup.md) |
128+
| OIDC (embedded and external) | [docs/oidc-configuration.md](docs/oidc-configuration.md) |
129+
| External OIDC client | [docs/external-oidc-client-setup.md](docs/external-oidc-client-setup.md) |
130+
| External OIDC providers | [docs/external-oidc-providers.md](docs/external-oidc-providers.md) |
129131

130132
## License
131133

‎docs/architecture.md‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ Used by EdgeOps Console, `potctl`, `iofogctl`, and automation.
109109
| Certificates | `/api/v3/certificate` | PKI operations |
110110
| WebSocket | `ws` routes in `src/routes/` | Exec and logs (Bearer token) |
111111

112-
Browser login uses the OAuth BFF (`GET /api/v3/user/oauth/authorize`); CLI uses `POST /api/v3/user/login`. See [oidc-configuration.md](oidc-configuration.md), [external-oidc-client-setup.md](external-oidc-client-setup.md), and [rbac-reference.md](rbac-reference.md).
112+
Browser login uses the OAuth BFF (`GET /api/v3/user/oauth/authorize`); CLI uses `POST /api/v3/user/login`. See [oidc-configuration.md](oidc-configuration.md), [external-oidc-client-setup.md](external-oidc-client-setup.md), [external-oidc-providers.md](external-oidc-providers.md), and [rbac-reference.md](rbac-reference.md).
113113

114114
### Agent API (fog token — no OIDC)
115115

@@ -131,15 +131,15 @@ Full request/response shapes: **`docs/swagger.yaml`** (agent paths).
131131

132132
| Job | File | Role |
133133
|-----|------|------|
134-
| Fog status | `fog-status-job.js` | Mark agents offline when status pings lapse |
134+
| Fog status | `fog-status-job.js` | Liveness cleaner: chunked transactions mark quiet fogs UNKNOWN using Controller receipt time (`lastStatusTime`). Desired-active microservices become UNKNOWN with CPU/memory 0. Desired-inactive STOPPED workloads are left alone. Does not delete microservices. Overlapping runs on the same process are skipped. |
135135
| Controller heartbeat | `controller-heartbeat-job.js` | Control-plane liveness |
136136
| Fog token cleanup | `fog-token-cleanup-job.js` | Expire stale agent tokens |
137137
| Controller cleanup | `controller-cleanup-job.js` | Orphaned controller MS housekeeping |
138138
| Event cleanup | `event-cleanup-job.js` | Audit event retention |
139139
| NATS reconcile | `nats-reconcile-worker-job.js` | NATS operator sync |
140140
| Platform reconcile | `platform-reconcile-worker-job.js` | Fog + service platform claim/reconcile (one job, two queues) |
141141
| Fog platform sweep | `fog-platform-sweep-job.js` | Drift detection; re-enqueue stale fog/service tasks |
142-
| Stopped app status | `stopped-app-status-job.js` | Application state maintenance |
142+
| Stopped app status | `stopped-app-status-job.js` | Desired-inactive safety net: leftover RUNNING, STOPPING, DELETED, or DELETING observed status is rewritten to STOPPED with CPU/memory 0. Overlapping runs on the same process are skipped. |
143143

144144
### Platform reconcile (fog + service + resolver)
145145

@@ -356,6 +356,12 @@ Controller v3.8 and Edgelet share a **frozen field-agent REST contract** on `/ap
356356

357357
**Volume mounts** — `GET volumeMounts` returns bind/volume shapes plus system-injected immutable `serviceAccount` entries.
358358

359+
**Volume mapping scope** — microservice `volumeMappings[]` may include `scope` (`private` default / `shared`). Meaningful only when `type` is `volume`. Shared names are node-global on that fog. Reclaim is on-node (`edgelet volume` / `deprovision --purge-volumes`); Controller does not prune volumes. `deleteWithCleanup` remains unused.
360+
361+
**Fleet models** — weight artifacts at `/api/v3/models`, linked per fog. Agent `GET /api/v3/agent/models` returns linked rows only. User microservices bind them with a `models` catalog (`bindPath`, `permissions`, `items[].name`).
362+
363+
**Fleet Knowledge** — documents, datasets, and vector indexes, same distribution shape as models and a separate name namespace. Operator CRUD, YAML (`kind: Knowledge`), and per-fog link live at `/api/v3/knowledge`. User microservices bind a `knowledge` catalog and can change it with `PATCH /api/v3/microservices/:uuid/knowledge`. Agent `GET /api/v3/agent/knowledge` returns linked rows only. `format` is a soft hint. Hugging Face pulls use the dataset API. Fog `modelLastUpdate` and `knowledgeLastUpdate` are Unix milliseconds.
364+
359365
**Service account RBAC** — microservice roles use apiGroup **`edgelet.iofog.org/v1`**; SA/role changes trigger `microserviceList` change tracking.
360366

361367
For the full bilateral contract (including ControlPlane env vars and verification references), see Edgelet documentation:
@@ -466,6 +472,7 @@ Agent routes and WebSocket exec/logs for agents are **outside** OIDC — see [rb
466472
| [pki.md](pki.md) | Central CAs, cert renewal, NATS operator rotation |
467473
| [oidc-configuration.md](oidc-configuration.md) | Embedded/external auth modes and env vars |
468474
| [external-oidc-client-setup.md](external-oidc-client-setup.md) | External IdP client configuration |
475+
| [external-oidc-providers.md](external-oidc-providers.md) | Default groups/roles and IdP recipes |
469476
| [operations/database-transactions.md](operations/database-transactions.md) | Transaction runner, OTEL metrics, SQLITE_BUSY runbook |
470477
| [operations/sizing.md](operations/sizing.md) | Hardware sizing by fog count (K8s and Remote CP) |
471478
| [CONTRIBUTING](../CONTRIBUTING) | Dual-mirror CI and development |

0 commit comments

Comments
 (0)