Skip to content

Add eager start support for standalone activities - #12349

Open
wenlong-gu wants to merge 8 commits into
temporalio:mainfrom
wenlong-gu:codex/saa-eager-start-saa
Open

wenlong-gu wants to merge 8 commits into
temporalio:mainfrom
wenlong-gu:codex/saa-eager-start-saa

Conversation

@wenlong-gu

Copy link
Copy Markdown

Task overview

Part 3 of 3 for ACT-1118, adding eager start for Standalone Activities (SAA).

Today, an SAA start must create the activity, enqueue a dispatch task, send it
through Matching, and wait for a worker poll. That queue round trip adds latency
and unnecessary Matching work when the caller already has a ready local worker.

This change lets an eligible StartActivityExecution request return the first
Activity task directly in its response. The server persists the Activity before
returning the task and skips only the initial Matching dispatch.

Behavior and safety

Normal SAA: new -> SCHEDULED -> Matching -> STARTED
Eager SAA:  new -> STARTED -> task returned in start response

Eager start retains attempt initialization and Schedule-to-Close,
Start-to-Close, and heartbeat protection. It omits only the initial dispatch
and Schedule-to-Start timeout. Later attempts always use the normal Matching
path.

An eager task is returned only when result.Created == true:

  • Same-request-ID retries return the existing execution without an eager task.
  • Conflict outcomes, including USE_EXISTING, return no eager task.

This intentionally differs from eager Workflow Task retry behavior. Activity
code can have external side effects, so re-delivering the same task could run
user code twice.

Scope of this PR

  • Adds namespace-scoped eager-start configuration and eligibility fallback.
  • Adds the eager initial state transition and skips the initial Matching
    dispatch task.
  • Builds a standard worker PollActivityTaskQueueResponse and task token from
    the persisted Activity component.
  • Returns that response only for newly created executions.
  • Adds unit, functional, and design-document coverage.

Dependencies

This is a draft until API-Go is merged and a go.temporal.io/api release is
available. At that point this PR will pin the released version in go.mod.
No local replace directive is committed here.

Validation

With the local generated API-Go dependency used only for verification:

  • CGO_ENABLED=0 go test -tags test_dep ./chasm/lib/activity -count=1
  • CGO_ENABLED=0 go test -tags test_dep ./tests -run '^TestStandaloneActivityTestSuite$' -count=1
  • CGO_ENABLED=0 make lint-code-fast
  • git diff --check

The functional coverage verifies a newly created eager SAA returns a task, a
same-request-ID retry returns no task, USE_EXISTING returns no task, and the
original eager task can complete normally.

Local benchmark

Measured 200 paired runs using an in-process Temporal server test cluster:

Path p50 p95 p99
Normal SAA 2.372 ms 58.654 ms 88.855 ms
Eager SAA 1.588 ms 5.028 ms 7.362 ms

This is approximately 33% lower p50 and 91–92% lower p95/p99 latency in the
local environment. It measures server-side task delivery only; it excludes SDK
worker scheduling, Activity business-code execution, network latency, and
production load.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Wenlong Gu seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@wenlong-gu
wenlong-gu marked this pull request as ready for review October 2, 2026 15:25
@wenlong-gu
wenlong-gu requested review from a team as code owners October 2, 2026 15:25
@wenlong-gu
wenlong-gu requested review from fretz12 and a balanced review from Copilot October 2, 2026 19:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The eager response omits standard execution metadata, loses SDK metadata, and lacks the documented rollout metrics.

Review effort: Balanced
Findings: 4 Medium severity

Open (4)
What changed in this PR

Adds eager delivery of the first standalone Activity task, bypassing initial Matching dispatch while preserving persisted state and timeout protection.

Changes:

  • Adds namespace-gated eager-start handling and task construction.
  • Extends state-machine, parity, and functional coverage.
  • Documents lifecycle, safety, and performance behavior.
File Description
tests/​activity_workflow_driver.go Adds eager workflow Activity driver.
tests/​activity_standalone_test.go Tests eager response deduplication.
tests/​activity_standalone_driver.go Supports eager standalone tasks.
tests/​activity_parity_test.go Adds eager retry parity coverage.
tests/​activity_driver.go Models eagerly started attempts.
docs/​architecture/​saa-eager-start-plan.md Documents design and validation.
chasm/​lib/​activity/​statemachine.go Adds atomic eager-start transition.
chasm/​lib/​activity/​statemachine_test.go Tests eager transition tasks.
chasm/​lib/​activity/​responses.go Builds eager worker responses and tokens.
chasm/​lib/​activity/​model/​vocabulary.go Adds eager model configuration.
chasm/​lib/​activity/​model/​model.go Models initially started state.
chasm/​lib/​activity/​model/​model_test.go Tests eager model behavior.
chasm/​lib/​activity/​handler.go Creates and returns eager tasks.
chasm/​lib/​activity/​frontend.go Applies eligibility fallback.
chasm/​lib/​activity/​frontend_test.go Tests fallback conditions.
chasm/​lib/​activity/​config.go Adds namespace eager-start configuration.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread chasm/lib/activity/responses.go
Comment thread chasm/lib/activity/statemachine.go
Comment thread docs/architecture/saa-eager-start-plan.md Outdated
Comment thread tests/activity_parity_test.go
Comment thread docs/architecture/saa-eager-start-plan.md Outdated
return &workflowservice.PollActivityTaskQueueResponse{
TaskToken: token,
WorkflowNamespace: request.namespace,
WorkflowExecution: &commonpb.WorkflowExecution{RunId: key.RunID},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to set this. SAA has no parent wf


ActivityEagerExecutionCounter = NewCounterDef("activity_eager_execution")
ActivityEagerExecutionCounter = NewCounterDef("activity_eager_execution")
StandaloneActivityEagerStartAcceptedCounter = NewCounterDef("standalone_activity_eager_start_accepted")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we have this for wf start. Also, since we're tracking denied, ActivityEagerExecutionCounter effectively tracks it. I suggest we remove this for consistency

ActivityEagerExecutionCounter = NewCounterDef("activity_eager_execution")
ActivityEagerExecutionCounter = NewCounterDef("activity_eager_execution")
StandaloneActivityEagerStartAcceptedCounter = NewCounterDef("standalone_activity_eager_start_accepted")
StandaloneActivityEagerStartDeniedCounter = NewCounterDef("standalone_activity_eager_start_denied")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer to call this just activity_eager_execution_denied, though I know this applies only to SAA. more consistent with wf metric

SpeculativeWorkflowTaskRollbacks = NewCounterDef("speculative_workflow_task_rollbacks")

ActivityEagerExecutionCounter = NewCounterDef("activity_eager_execution")
ActivityEagerExecutionCounter = NewCounterDef("activity_eager_execution")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't forget parity test with WFA

}
// TODO(saa): when eager start is supported, deny it if start delay > 0 (same as workflow behavior).
if req.GetRequestEagerExecution() {
metricsHandler := h.metricsHandler.WithTags(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

activity_eager_execution currently uses WFA’s namespace/task-queue metric scope, while this new SAA scope uses namespace/operation. Reusing the same metric with different label keys can cause Prometheus to reject the series.

We addressed this WFA/SAA parity problem previously in #11806 by sharing standard Activity metric-scope construction,by adding empty SAA labels when WFA had labels SAA could not yet populate.

Could we follow that pattern here: extract the current WFA activity_eager_execution scope into a shared common/metrics helper, migrate WFA to use it, then use it from SAA after the eager task is successfully constructed? This would also let us reuse ActivityEagerExecutionCounter instead of introducing standalone_activity_eager_start_accepted. Please add a WFA/SAA label-parity test.

return nil
}

func (a *Activity) applyEagerStarted(ctx chasm.MutableContext, event eagerStartEvent) error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you just a TODO to populate LastDeploymentVersion once it's supoorted

attempt.DispatchTime = timestamppb.New(dispatchTime)

if timeout := a.GetScheduleToStartTimeout().AsDuration(); timeout > 0 {
if dispatch && a.GetScheduleToStartTimeout().AsDuration() > 0 {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if dispatch && a.GetScheduleToStartTimeout().AsDuration() > 0 {
if timeout := a.GetScheduleToStartTimeout().AsDuration(); dispatch && timeout > 0 { {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't forget to remove all these changes. FYI, I don't think you need the go.mod replace for dev purposes. Just make a branch off api-go and don't use a fork

Comment thread Makefile

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment as above, don't forget to remove. And no need for this in future if you don't use replace

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add test cases that simulate the case where the eager response is "lost", then through Start-to-Close, Heartbeat, or Schedule-to-Close timeouts it falls back through the regular task polls and is able to complete successfuly?

@fretz12

fretz12 commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

can you make the PR desc follow the standard template? And personally I'd prefer to just keep it concise and to the point rather than too much AI noise

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants