optimizer: preserve plan semantics through dispatch and prove realized value

Problem

The optimizer's core equations and Core-side replay are sound, but the full path from config and solver output to device dispatch does not preserve several decisions and hard limits. Further solver tuning cannot fix plans that runtime changes or metrics that cannot attribute optimizer value.

This issue records a read-only audit of checkout 8ca77904 and fetched origin/master c9c7919e. The relevant logic was the same on both.

Confirmed failures

1. Per-storage plans are validated, then discarded

The worker returns storage_power_w and storage_energy_wh, and Core validates each storage. mpc.SlotDirective then carries only aggregate fleet energy. Control creates a new split by mode, capacity, limits and PV locality.

A deterministic reproduction planned a feasible 100 W + 1900 W = 2000 W split. Runtime produced 100 W + 1000 W = 1100 W.

Impact:

planned and delivered SoC paths differ;
cost and terminal value use energy runtime may not deliver;
a heterogeneous fleet may hit driver clamps although the optimizer plan passed validation.

2. Explicit zero power limits change meaning

MPC treats batteries.<id>.max_charge_w: 0 as "forbid charge". driverLimitsFrom only copies positive battery limits. Control treats zero as unset and falls back to MaxCommandW = 5000 W.

A config-to-control reproduction produced a +5000 W charge target from an explicit zero charge limit. Discharge has the same fault.

Related: #816 covers a separate slew/non-following-device failure that also exposes inconsistent zero/default handling.

3. PV-only export can exceed the declared hard ceiling

The fuse/export guard can scale battery discharge. If PV alone causes the overage, it returns unchanged targets and depends on a planner curtail directive. ComputePVCurtail does not react directly to fresh meter export above min(fuse, MaxExportW).

4. Full PV curtailment cannot be represented

pv_limit_w = 0 means both:

no curtailment; and
an active cap of zero watts.

Python, Go fallback, Core validation and driver dispatch all collide on this sentinel.

Reproduction: PV -5000 W, load 0 W, full battery, export price -100 ore/kWh. Python chooses zero grid flow and serializes a zero PV limit. Core interprets that as no cap, replays -5000 W grid flow and rejects the plan. Go fallback also cannot emit the full-curtail action.

5. An older replan can replace newer operator intent

Service.replan takes a params snapshot and always publishes its result. It has no request generation or latest-wins guard.

A deterministic test started a blocked self_consumption solve, completed a later SetMode(arbitrage) solve, then released the old solve. The old plan became active last.

6. Solver deadlines and worker queue ownership disagree

Core sends 0.8 * optimizer timeout. Python gives that full limit separately to service and economic solves. Total solver time can approach 1.6 * Core timeout, before model build and queue time.

The Unix worker starts one thread per connection but serializes solves with one global lock. A request that Core has already abandoned can finish and block newer requests.

7. `USER_LIMIT` is accepted as an optimal status

CVXPY USER_LIMIT can return ok: true without a proven service optimum. Core catches many physical inconsistencies, but a physically valid incumbent may still miss a feasible EV target or be much more expensive.

8. Mode-specific economic terms leak across modes

The Go contract applies min_arbitrage_spread_ore_kwh only to arbitrage modes. Python adds it to discharge cost in every mode.

Reproduction in self_consumption, 2 kW load and 80% SoC:

spread 0: about 2050 W discharge;
spread 100: no discharge.

Explicit relaxed can also allow simultaneous charge/discharge under unsafe tariffs and then fail replay.

9. Planner config can give Python and Go fallback different physics

Config validation does not reject:

soc_min_pct >= soc_max_pct;
efficiency above 1;
invalid per-battery SoC windows;
invalid weights.

Python can reject the request while Go DP accepts and uses the same bad values. batteries.<id>.weight is currently not wired to control.

10. Current savings and backtest output cannot prove optimizer value

/api/savings/daily compares actual grid cost with a no-PV/no-battery baseline. It combines PV value, battery value, export revenue and EV timing. It is not incremental optimizer value.

The production cost integrator also applies the next sample across arbitrarily long telemetry gaps. A separate, currently unused savings integrator already rejects gaps over 20 minutes and reports coverage.

The optimizer backtest:

sums overlapping plan horizons;
may select a plan produced after a measured bucket;
assumes planned battery power was delivered;
ignores PV curtailment in realized grid replay;
does not link plan revision, clamps, command results and measured per-device power.

Input and model gaps to measure after correctness fixes

forecast.solar uses the maximum sample in an hour as the full-hour power;
every forecast price gets fixed confidence 0.6 although the model stores sample counts and MAE;
PV scenarios use fixed probabilities 0.60/0.25/0.15, one absolute spread, identical load and no price paths;
local household and Nordic market time features use UTC buckets across CET/CEST;
per-device efficiency, degradation cost and capability are not fed through the live planner contract.

Order of work

PR 1: preserve explicit zero battery limits

Keep unset and explicit zero distinct through config, PowerLimits and every clamp.
Add config-to-dispatch tests for charge and discharge.
Keep Slew anchored on actual output + MaxCommandW default pins the controller into commanding more charge than it wants, indefinitely, when a device stops following #816's separate slew fault out of scope.

PR 2: make export protection and curtail state explicit

React to fresh PV-only export above the effective ceiling.
Separate curtail active from limit_w.
Validate and dispatch an active zero cap without converting it to release.
Define driver support for a true zero cap or a documented minimum safe cap.
Test partial cap, full cap, release, unsupported PV and residual uncontrollable export.

PR 3: make replans and solver work latest-wins

Add a monotonic replan generation and commit only the newest request.
Snapshot reason and mode with the request.
Use one absolute deadline across queue, model build, service solve, economic solve and response.
Drop expired/superseded work before it enters the solve lock.
Do not accept USER_LIMIT without a validated feasible incumbent and service proof.

Follow-up: execute the per-storage plan

Carry per-device Wh budgets through SlotDirective.
Use runtime redistribution only for a device failure or hard clamp.
Feed delivered per-device energy back into the next replan.

Follow-up: prove realized value

Persist and link:

plan and model revision;
forecast issue/source/revision;
Core-validated target;
clamp reason;
driver command result;
measured grid, PV, load, EV, battery power and SoC;
data coverage.

Run leak-free walk-forward scoring one executed first action at a time. Compare no battery, simple self-consumption, Go DP, active Python/challenger and perfect-foresight lower bound with the same exogenous data and terminal energy.

Acceptance gates

No configured hard limit changes meaning across layers.
The plan Core validates is the plan control attempts to deliver.
Older work cannot replace newer operator intent.
Timed-out work cannot block a newer solve.
Every accepted plan passes independent physical replay and required service checks.
Optimizer gain is reported separately from PV and hardware value.
Missing data lowers coverage instead of creating energy.
A deterministic full-stack test runs optimizer -> validator -> dispatch -> driver simulator -> measured SoC/cost.

Current evidence

66 optimizer tests passed.
go test -race ./internal/mpc ./internal/control passed outside the sandbox.
611 golden dispatch records passed.
Temporary targeted tests reproduced the fleet split, explicit-zero and stale-replan faults.
No existing test runs the complete optimizer-to-realized-cost path.

Existing work with right of way

feat(web): turn a driver's write path on from Settings #769 touches go/cmd/ftw/main.go and go/internal/control/dispatch.go.
perf(optimizer): use direct HiGHS for shared storage #883 owns direct HiGHS and optimizer/ftw_optimizer/model.py; its active review thread covers shared post-curtailment baseline parity.
feat(roofmodel): optional shadow-aware irradiance via vostok #736/feat(roofmodel): derive PV array geometry from Lantmäteriet open geodata #735/feat(weather): add STRÅNG as an irradiance source and declare source coverage #734 touch go/internal/mpc/external_optimizer.go; they are draft/dirty.
fix(config): reject >3 fuse phases and duplicate site meters #844 and older PRs own current config.go work.
fix(drivers): preempt blocked commands for autonomous defaults #798 owns the stale-driver autonomous-default path and still needs its separate human safety review.

Implementation PRs must stack on or wait for those owners rather than replace their changes.

Implementation has started in three review-sized draft PRs:

fix(control): preserve explicit zero battery limits #887 preserves explicit one-sided 0 W battery limits through config and every runtime clamp.
fix(control): mitigate PV-only export over limits #888 adds partial reactive PV-only export mitigation and exposes any reduction that safe positive caps cannot cover.
fix(mpc): keep the newest replan #889 makes replan publication latest-wins and keeps plan mode/reason/diagnostics paired.

#887 and #888 are stacked on the exact reviewed head of #769 because they overlap its Core files. Retarget them to master after #769 lands. #889 targets current master.

Still open here: an explicit active-vs-release PV curtail contract with driver zero/minimum capability, solver cancellation and one shared deadline, per-storage dispatch, and the closed-loop measurement work.

Implementation update: four more review-sized draft PRs are now open.

fix(optimizer): enforce result and mode semantics #890 fixes optimizer mode semantics and rejects unproven CVXPY USER_LIMIT results. It targets master, is mergeable, and CI is green.
fix(optimizer): bound queued solves to one deadline #893 bounds queued and active solver work to one monotonic worker deadline. It is stacked on fix(optimizer): enforce result and mode semantics #890 because it needs the same solver files. Review its incremental diff, then retarget it to master after fix(optimizer): enforce result and mode semantics #890 lands. The custom stack base does not trigger GitHub checks; local make verify, make verify-all, MPC race tests, Core↔Python tests and 112 Python tests pass.
fix(savings): reject long telemetry gaps #892 stops telemetry gaps over 20 minutes from creating energy, cost or savings; adds weighted history/price coverage; and labels saved_ore as total site value rather than optimizer gain. It targets master, is mergeable, and CI including Core/full-stack is green.
fix(optimizer): make backtest first actions causal #891 makes backtest first-action scoring causal and non-overlapping. It excludes unmodeled PV curtailment and labels the result as an independent grid-cost counterfactual, not delivered savings. It targets master, is mergeable, and CI is green.

#883 has landed, so its former file-ownership block is resolved through #890/#893.

Suggested review order:

fix(optimizer): enforce result and mode semantics #890, then fix(optimizer): bound queued solves to one deadline #893.
fix(savings): reject long telemetry gaps #892 and fix(optimizer): make backtest first actions causal #891 independently.
Keep fix(control): preserve explicit zero battery limits #887/fix(control): mitigate PV-only export over limits #888 stacked behind feat(web): turn a driver's write path on from Settings #769 as already documented.
After fix(mpc): keep the newest replan #889 and the dispatch stack settle, carry per-storage plans into runtime and add the plan → validation → clamp → command-result → measured-power linkage.

Still not solved: a transport-independent absolute deadline field across every Core transport, explicit active-zero PV curtail capability in the canonical drivers, per-storage execution, and true closed-loop optimizer value attribution. No PR has been merged by this implementation pass.

Post-merge and beta status

#890–#894 are merged. The final master commit is f9297c06073836cca2a629cfa775ded9d6984731:

fix(optimizer): enforce result and mode semantics #890: mode and USER_LIMIT semantics
fix(optimizer): make backtest first actions causal #891: causal, non-overlapping first-action backtest
fix(savings): reject long telemetry gaps #892: telemetry-gap rejection and coverage
fix(optimizer): bound queued solves to one deadline #893: one bounded queue/solve deadline
perf(app): reduce uplink, history, and background work #894: app/uplink performance, including bounded dashboard poll reads

Two prereleases were published from that exact commit:

Core v1.16.1-beta.19, run 31468877785
- Core digest: sha256:513be2b6b4b11cb7ce2cef98c827bd15826681ce98ce3ce9d41248edb039dc53
- Updater digest: sha256:993114f6c56bd01f2bcd687160b506dd9ec84d0938862ae124b3ebb89ef2f85f
Optimizer optimizer-v1.4.0-beta.2, run 31469463012
- Optimizer digest: sha256:fe92b7cf965c9380dba3de00980f4f05d2c272876709606526e7cb3d23aee890

The fixed tags and :beta aliases match. The Core and updater canonical/compatibility mirrors also match. All images contain linux/amd64 and linux/arm64 manifests whose revision labels resolve to f9297c0.

Verification passed on the exact release commit:

local make verify-all: 117 Python tests, all Go packages, Core↔Optimizer contract tests, compose migration, e2e, vet/build, and amd64/arm64/Windows cross-builds;
fresh PR checks before each merge;
optimizer release: Python suite, self-update version contract, Core↔Optimizer multi-device contract, and multi-arch image build.

Master push run 31468510104 is red only because the registry job runs gh pr view "" during a push event. Core Go test/vet, web, full stack and the Device Support contract are green; the final umbrella job only propagates that context error. Registry blobs also match the merged webapp. This is a workflow context fault, not a code regression.

#824 was not merged or used. It remains open and blocked. No stable promotion happened. Publication is verified here; real-site beta validation is still outstanding.

update draft fix(mpc): keep the newest replan #889 onto current master, rerun full checks and review latest-wins publication;
keep fix(control): preserve explicit zero battery limits #887/fix(control): mitigate PV-only export over limits #888 behind feat(web): turn a driver's write path on from Settings #769 until its hardware/driver pairing is approved;
then add superseded-replan cancellation on top of fix(mpc): keep the newest replan #889;
retain the remaining open gates: active-zero PV curtail capability, transport-independent deadline propagation, config physics validation, per-storage execution, and plan → validation → clamp → command result → measured power/SoC linkage.

Issue #885 stays open until those acceptance gates and real-site beta checks pass.

Latest-wins Core work is merged

fix(mpc): keep the newest replan #889 merged as 6e23450f47df23e5a023477d034089c4ab8ed741. It pairs mode, reason, plan and diagnostics under one generation guard, so an older solve cannot replace newer operator intent.
fix(mpc): cancel superseded replans #895 merged as 5213dd5916a5a1513114cb2676e96a1793562375. A newer generation or Stop now cancels Core's active replan request, skips stale Go fallback/shadow/publication/diagnostic work, rejects new work during shutdown, and waits for all accepted replans before Optimizer.Close.

#895 keeps caller cancellation separate from service cancellation. Caller deadlines still reach the external optimizer and retain the prior safe Go-DP fallback. Supersession and Stop set the service-cancel flag before canceling the context, so stale work cannot take that fallback path.

Verification on the exact #895 head:

five controlled concurrency tests passed 20 consecutive runs with the race detector; independent audit repeated them 50 times;
full MPC, API and process-wiring race suites passed;
make verify-all passed: 117 Python tests, all Go tests, Core to Optimizer contracts, compose migration, end-to-end tests, vet/build, and Linux arm64/amd64 plus Windows amd64 cross-builds;
all PR checks passed, with no review comments or inline threads; three independent final reviews found no P0-P3 issue.

Post-merge run 31474732641 has green Core, Optimizer contract, full-stack and Device Support jobs. Its only primary failure is the known push-context bug: the registry job runs gh pr view "" on master; the umbrella go test + vet job only propagates that failure. This is not a code failure.

Remaining deadline/cancellation boundary: Core now stops waiting and doing stale follow-on work, but an already active native solve in the Unix optimizer sidecar continues while Python holds SOLVE_LOCK, until its existing solver deadline. A newer request can still wait behind it. The next solver-lifecycle slice needs worker-side active-solve supersession or an isolated solve process that can be stopped safely, plus an end-to-end test proving request 2 starts before request 1's old deadline.

Issue #885 remains open. #824 remains unmerged and was not used.

Active cancellation follow-up merged

#896 is merged as 16ddcf88940020767c9274a826d37a74323ffd5e.

This closes the next bounded part of the stale-work problem:

Core sends a request-scoped cancel_request frame when a sent Unix optimizer request is superseded, canceled or reaches its deadline.
The worker can wake and remove queued work before it starts.
The current primary direct-HiGHS path uses HiGHS user interrupts for active solves.
Canceled work cannot fall through to CVXPY or write a stale response.

Verification before merge:

all PR checks passed;
make verify-all passed;
134 optimizer tests passed;
all Go packages, Core↔Optimizer contract, Compose migration, full-stack e2e, vet, build and cross-builds passed;
the new Unix transport cases passed 20 times under race testing;
the cancel wire test passed 100 times;
worker and direct-HiGHS cancel tests each passed 50 times.

Post-merge run 31478951607 used the exact merge SHA. Core, optimizer, full stack and Device Support contract jobs passed. The run is red only because the existing registry job executes gh pr view "" on a master push; the umbrella job then reports that failed dependency. This is a workflow-context bug, not a code or test regression.

Scope remains narrow: active native interruption is implemented for direct HiGHS. CVXPY/CLARABEL still stop only at existing checkpoints/deadlines, while stale result suppression remains in force. Full CVXPY cancellation needs process isolation and stays open as a separate item. No release or real-site validation was performed for this merge.

Current-slot weather fix merged

#897 is merged as 694721075262e13bd9d60622ec407e212e082c9a.

The provenance audit found a correctness bug before the metadata loss point: Core loaded weather with the same 15-minute margin as prices. An hourly weather row could therefore be dropped even while it still covered the current price slot. The PV input could use a later row, and the twin-drift check could resample against a different row from the one used to build the plan.

The merged fix:

keeps the preceding weather hour for both plan construction and drift resampling;
stops direct PV estimates from carrying backward before forecast coverage;
keeps cloud fallback on the nearest row instead of skipping to a later row.

Verification:

make verify-all passed;
all Go and 134 optimizer tests passed;
Core↔Optimizer, Compose migration and full-stack e2e passed;
four focused regressions passed 100 times and 20 times under the race detector;
the full MPC race suite and vet passed;
PR CI was fully green before merge.

Post-merge run 31481353074 used the exact merge SHA. Core, optimizer, full stack and Device Support contract jobs passed. The run is red only because the existing master-push registry step executes gh pr view ""; the umbrella job then reports that failed dependency. This is the known workflow-context bug, not a code regression.

The next slice is in progress: retain each plan slot's price/weather source and local availability time in the persisted diagnostic. This still will not claim provider issue time, provider revision or model revision until those values exist. No release or real-site validation was performed for #897.

Input provenance merged

PR #898 merged as c0957028a801d81db6cf1a9f0e9090eb00c0064b.

Core now stores a versioned, backward-compatible record of each MPC slot's price input and consulted weather row, including local availability time. The fields survive the external optimizer path, Go fallback, diagnostic JSON, SQLite persistence, and restore. Legacy schema 0 remains unchanged; negative or future schemas are rejected. The weather fields identify the row used, not full PV-model provenance.

Verification:

focused provenance tests: 100 repetitions
focused race tests: 20 repetitions
full MPC package with and without race detection
make verify and make verify-all
PR CI: Core, Optimizer contract, full stack, driver contract, changeset, and required go test + vet all passed

Post-merge run 31483991243 ran on the exact merge SHA. Core, Optimizer contract, full stack, and Device Support passed. Its only primary failure is the existing push-context bug: the registry has not drifted from the app runs gh pr view "" on a master push. The umbrella go test + vet job then fails only because it aggregates that result.

No beta or stable release was published for this slice, and no real-site validation is claimed. #824 was not merged.

Next high-ROI measurement work remains the plan → validation → clamp → command result → measured power/SoC link needed for true optimizer-value attribution. File ownership with the open control stack must be refreshed before that work starts.

#899 merged: stable Core decision identity

Merged #899 as 68c95ff3facf0780f0cb9b6c2b5106ad992144a3.

What changed:

Core assigns a host-owned UUID only after the latest-wins acceptance gate.
(decision_id, slot_start_ms) now identifies a planned action through the live plan, diagnostic JSON/SQLite, restart restore, and SlotDirective.
Legacy diagnostics remain valid with an unknown, empty decision ID.
Invalid or overlapping input slots stop before either solver. Solver actions must match the input slot count, start, and length before Core publishes them.

Verification:

focused regression tests passed 200 times;
focused race tests passed 25 times;
full make verify-all passed, including Go, 134 Python tests, the Core-to-Optimizer contract, full-stack tests, migrations, vet, builds, and cross-builds;
every PR check passed before merge; the PR had no unresolved reviews or threads.

Post-merge classification:

master run 31487723283 has green Core, Optimizer contract, full-stack, and driver-contract jobs;
the only failure is the known push-context workflow bug: the registry job calls gh pr view "" on a master push, so its compare steps never run; the umbrella job fails only because it receives that failed job state;
brand cleanup and the Changesets updater both completed successfully.

No beta, site validation, or stable promotion was done for this merge. #824 remains blocked and was not merged.

Next slice: fail closed on invalid MPC physics at the service boundary before both the external optimizer and Go fallback. The control-to-command-to-measurement join remains next after the current control-file owners are resolved.

fix(mpc): reject invalid planning physics (#900)

Physics-validation slice merged

#900 merged to master as 60057bb798697a77fab21deea667da404f54e068.

Core now validates effective slot, fleet, storage and loadpoint physics before either planner runs. It rejects non-finite or out-of-range values, keeps the primary and Go-fallback loadpoint views equal, samples PV uncertainty once, and keeps the prior plan when Go DP would have to invent an in-band starting SoC during battery recovery.

Validation:

make verify-all, including 134 Python tests, Go tests, optimizer contracts, compose migration checks and linux/arm64, linux/amd64 and windows/amd64 builds
focused MPC tests 100 times
focused MPC race tests 25 times
full go test -race ./internal/mpc
all PR checks green, including Core, optimizer contract, full stack and driver contract

Post-merge run 31492351952 ran the real Core, optimizer, full-stack and driver jobs successfully. Its only primary failure is the known push-context bug in the registry has not drifted from the app: it executes gh pr view "" on a push event. The go test + vet umbrella fails only because it aggregates that contract-job result.

No release was published. #824 remains untouched. The next #885 work is the control-side join from decision_id through clamp, command result and measured power/SoC, after the open control owners are sequenced.

Progress: SoC observation provenance

Merged #901 as 16d33fdf535e78bc182e149e54a86e9fdbe55022.

This Core slice now keeps a separate SoC source time, carries old SoC without inventing a new observation, suppresses duplicate long-format SoC samples for cached updates, and makes vehicle selection use the SoC source time. The Lua-to-Core contract accepts soc_fresh=false; missing soc_fresh stays backward compatible.

Validation completed before merge:

make verify-all passed.
The full telemetry and driver race suites passed.
Focused freshness tests passed 100 times; the race subset passed 25 times.
The regression tests fail on the exact old base for the expected reasons.
PR CI, including Core, full stack, driver contract and required go test + vet, passed.

The post-merge test run has green Core, full-stack and driver-contract jobs. Its only primary failure is the known push-context bug in the registry has not drifted from the app: the job tries to resolve an empty PR number on a push. The aggregate go test + vet job then fails only because it includes that result.

This is a Core foundation, not a complete Tesla production fix. The current canonical Tesla driver does not yet emit soc_fresh=false from its cached emit_last() path. That change, a new driver version and tests, and an FTW bundle-pin update remain release gates. The shared device-driver catalogue files are still owned by older open PRs, so no competing driver PR was opened. No beta or stable release was made.

Progress: ambiguous EV deadlines fixed

PR #902 is squash-merged as e45cc62757f4840a5ced4fcaca5145ffa7ef010c.

Before the fix, the same three-slot EV input with a 40% target and target_slot_idx: -1 produced different first actions: Go DP chose 0 W because it treated the value as no deadline, while Python/HiGHS chose 2500 W because it clamped the value to the first slot. Core now rejects any positive target with a negative deadline index before either solver runs. A zero target remains the sole no-target sentinel; index 0 remains the first real deadline; later indices still clamp to the last horizon slot.

Verification:

regression tests failed on the old base for both the pure validator and Service publish path;
focused tests passed 100 times and focused race tests passed 25 times;
the full MPC suite and race suite passed;
go vet ./internal/mpc and make verify-all passed;
all PR checks passed, with no review comments or open threads.

Post-merge run 31498857166 is still marked in progress because the Device Support job has not finalized even though every one of its steps completed successfully. Core, optimizer and full-stack jobs passed. The registry job hit the known push-event gh pr view "" workflow bug; the aggregate job failed only because of that contract job. No beta or stable release was made.

Next active slice: fail closed on invalid raw MPC fuse/export limits before clamp logic can mask them. The broader config and dispatch fix remains separate because open PRs own those files.

Post-merge run 31498857166 has now finalized. Device Support completed successfully, so Core, optimizer, Device Support and full stack are all green on e45cc627. The run is red only because the known push-event registry step calls gh pr view ""; the aggregate job mirrors that one workflow failure. This corrects the pending status in the prior comment.

Progress: raw MPC grid limits now fail closed

PR #903 is squash-merged as 5b9ec1771d35b5712128de0275c0a90c7e07d782.

Before the fix, a negative raw fuse became slot limits 0/0, which both planners treat as unlimited. With an 11 kW fuse, negative, NaN and infinite max_export_w values were silently replaced by the fuse cap. The old-base service tests then called the solver and published a new plan; the Go-DP repro also replaced the prior plan for a NaN export limit.

Core now requires both raw Service limits to be finite and non-negative before any state read, clamp or solver call. Exact zero keeps its documented unset/disabled meaning. Invalid values retain the latest accepted plan and create no new decision ID or diagnostic.

Verification:

old-base-red for negative fuse and negative/NaN/infinite export on the external path, plus NaN export on Go-DP;
focused tests 100 times and focused race tests 25 times;
full MPC and full MPC race suites;
go vet ./internal/mpc;
make verify-all, including 134 Python tests, all Go tests, optimizer contracts, full stack, migrations, builds and three cross-builds;
two independent clean reviews and all PR checks green.

Post-merge run 31501780846 has green Core, optimizer, Device Support and full stack jobs. Its only primary failure is the known push-event workflow bug that calls gh pr view ""; the aggregate job fails secondarily. This is not a code-test failure. No beta or stable release was made.

Scope remains honest: this closes the MPC Service boundary only. Config and dispatch still need their own finite/range guards once the current owners of those files land or coordinate their branches.

#904 merged: relaxed optimizer flow guards

#904 merged to master as f36cd79dc571084729628ab683ba71430807472b.

What changed:

unsafe relaxed storage and meter flows now use binary direction guards;
safe relaxed cases remain continuous;
shared and recourse retry one cost-neutral replay failure with the same deadline;
progressive hedging cannot bypass a required discrete guard.

Evidence:

166 Python tests passed;
make test and make verify-all passed before merge;
all PR checks passed, including optimizer and Core to optimizer contracts;
the post-merge optimizer and contract job passed on the exact merge SHA.

The post-merge test workflow is red only because the existing push-context registry step failed at Which copy of the app to compare against; it did not run the comparison. The code-specific optimizer job passed. Brand cleanup and release automation both passed. No beta or stable release was dispatched.

Next optimizer correctness item: skip the redundant progressive-hedging iteration when the initial consensus already has zero residual. The current extra iteration can hit USER_LIMIT and turn a valid one-scenario plan into internal_error.

Issue stays open for the remaining acceptance gates.

PH zero-residual fix merged

#905 merged to master as d8f534d9b7e566f5c19055648cfbf948391ebfc0.

Before the fix, a one-scenario plan had an initial PH residual of 0 W but still ran a second solver call. The real repro returned USER_LIMIT and internal_error in 5 of 5 runs.

The optimizer now checks the initial residual before the PH iteration loop. The same case makes one solver call, reports ph_iterations: 0, keeps residual 0 W and returns a replay-valid plan.

Evidence:

167 optimizer tests passed;
the focused regression passed 20 of 20 repeated runs;
make verify-all passed;
all PR checks passed;
the post-merge optimizer and Core to optimizer contract job passed on the exact merge SHA.

The post-merge test workflow is red only because the existing push-context registry step failed at Which copy of the app to compare against. Brand cleanup and release automation passed. No beta or stable release was dispatched.

Issue remains open for the remaining control and measurement gates.

Core beta.20 and Optimizer beta.3 published

The paired prereleases are published from the exact same master commit, d8f534d9b7e566f5c19055648cfbf948391ebfc0:

Core v1.16.1-beta.20, run 31571035184
- Core index: sha256:4571037b26537ab0d3f68c69ffedc222721460d6fedd63d7f7e89f057856594f
- Updater index: sha256:2f17dca28a15d4a52b418101803478ef9499781402db222e1b47c6a39e5ff159
Optimizer optimizer-v1.4.0-beta.3, run 31571035919
- Optimizer index: sha256:c3abf0633fbab2d68dc2a6c53daa7ad8947125021dcf32eaf49bf30a1abf892f

Both workflows completed successfully. The optimizer gate passed the Python suite, self-update version contract and Core to Optimizer contract before image publication.

Independent registry checks confirm:

each image contains linux/amd64 and linux/arm64 manifests;
both platforms carry revision d8f534d9b7e566f5c19055648cfbf948391ebfc0 and the expected version;
each fixed tag matches its :beta alias;
Core and updater canonical images match their compatibility mirrors.

No stable release was made. #824 remains open, blocked and untouched. This verifies publication, not real-site beta behavior. The issue remains open for site validation and the remaining control, command-result and measured-outcome linkage.

optimizer: preserve plan semantics through dispatch and prove realized value #885

Description

Problem

Confirmed failures

1. Per-storage plans are validated, then discarded

2. Explicit zero power limits change meaning

3. PV-only export can exceed the declared hard ceiling

4. Full PV curtailment cannot be represented

5. An older replan can replace newer operator intent

6. Solver deadlines and worker queue ownership disagree

7. USER_LIMIT is accepted as an optimal status

8. Mode-specific economic terms leak across modes

9. Planner config can give Python and Go fallback different physics

10. Current savings and backtest output cannot prove optimizer value

Input and model gaps to measure after correctness fixes

Order of work

PR 1: preserve explicit zero battery limits

PR 2: make export protection and curtail state explicit

PR 3: make replans and solver work latest-wins

Follow-up: execute the per-storage plan

Follow-up: prove realized value

Acceptance gates

Current evidence

Existing work with right of way

Activity

frahlg commented on Aug 11, 2026

frahlg commented on Aug 11, 2026

frahlg commented on Aug 11, 2026

Post-merge and beta status

frahlg commented on Aug 11, 2026

Latest-wins Core work is merged

frahlg commented on Aug 11, 2026

Active cancellation follow-up merged

frahlg commented on Aug 11, 2026

Current-slot weather fix merged

frahlg commented on Aug 11, 2026

Input provenance merged

frahlg commented on Aug 11, 2026

#899 merged: stable Core decision identity

frahlg commented on Aug 11, 2026

Physics-validation slice merged

frahlg commented on Aug 11, 2026

Progress: SoC observation provenance

frahlg commented on Aug 11, 2026

Progress: ambiguous EV deadlines fixed

frahlg commented on Aug 11, 2026

frahlg commented on Aug 11, 2026

Progress: raw MPC grid limits now fail closed

frahlg commented on Aug 11, 2026

#904 merged: relaxed optimizer flow guards

frahlg commented on Aug 11, 2026

PH zero-residual fix merged

frahlg commented on Aug 12, 2026

Core beta.20 and Optimizer beta.3 published

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

7. `USER_LIMIT` is accepted as an optimal status