Skip to content

implement time skipping for schedule v2 - #12197

Open
feiyang3cat wants to merge 8 commits into
mainfrom
fx/vts-chasm-schedule
Open

implement time skipping for schedule v2#12197
feiyang3cat wants to merge 8 commits into
mainfrom
fx/vts-chasm-schedule

Conversation

@feiyang3cat

@feiyang3cat feiyang3cat commented Sep 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

What changed?

  • implement time skipping for schedule v2
  • ban time skipping for schedule v1

How did you test it?

  • built
  • run locally and tested manually
  • covered by existing tests
  • added new unit test(s)
  • added new functional test(s)

need to merge the API first and change dependance to main temporalio/api#856

@feiyang3cat
feiyang3cat added this pull request to stack #12198 September 20, 2026 18:28
@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch 3 times, most recently from 502c84d to 2ce1494 Compare September 20, 2026 19:00
feiyang3cat
feiyang3cat commented Sep 20, 2026
View reviewed changes
Comment thread service/frontend/workflow_handler.go Outdated
feiyang3cat
feiyang3cat commented Sep 20, 2026
View reviewed changes
Comment thread chasm/lib/scheduler/scheduler.go Outdated
@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch 8 times, most recently from 4ff7f75 to 4af3bb1 Compare September 26, 2026 17:37
@feiyang3cat feiyang3cat mentioned this pull request Sep 26, 2026
Open
@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch 4 times, most recently from 35c5a98 to 5dd3dce Compare September 27, 2026 05:08
feiyang3cat
feiyang3cat commented Sep 27, 2026
View reviewed changes
Comment thread chasm/lib/scheduler/scheduler_migrate_task.go
@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch 4 times, most recently from 13e0234 to 484ad7d Compare September 27, 2026 06:22
@feiyang3cat
feiyang3cat marked this pull request as ready for review September 27, 2026 06:22
@feiyang3cat
feiyang3cat requested review from a team as code owners September 27, 2026 06:22
@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch from 4a85881 to 960c2f6 Compare October 9, 2026 04:02
@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch from 960c2f6 to 4ba3f46 Compare October 9, 2026 04:23
@feiyang3cat
feiyang3cat added this pull request to stack #12481 October 9, 2026 04:25
feiyang3cat
feiyang3cat commented Oct 9, 2026
View reviewed changes
Comment thread chasm/lib/scheduler/scheduler.go
@@ -349,6 +351,34 @@ func (s *Scheduler) LifecycleState(ctx chasm.Context) chasm.LifecycleState {
return chasm.LifecycleStateRunning
}

@feiyang3cat feiyang3cat Oct 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for reviewers: to decide whether the current schedule is at state that allow time skipping, and this function is called at close transaction time

@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch from 4ba3f46 to e968ef7 Compare October 9, 2026 05:11
@feiyang3cat
feiyang3cat removed this pull request from stack #12481 October 9, 2026 05:14
feiyang3cat
feiyang3cat commented Oct 9, 2026
View reviewed changes
Comment thread service/frontend/workflow_handler.go
if !config.GetEnabled() {
return nil
}
if schedule.GetPolicies().GetOverlapPolicy() == enumspb.SCHEDULE_OVERLAP_POLICY_ALLOW_ALL {

@feiyang3cat feiyang3cat Oct 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for reviewers: newly added

@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch 2 times, most recently from a617e9f to beac176 Compare October 9, 2026 05:55
feiyang3cat
feiyang3cat commented Oct 9, 2026
View reviewed changes
Comment thread chasm/lib/scheduler/scheduler.go
ErrSentinel = serviceerror.NewNotFound("schedule is a sentinel")
ErrSentinelBlocked = serviceerror.NewUnavailable("schedule is a sentinel; please retry after sentinel expires")
ErrMigrationPending = serviceerror.NewUnavailable("schedule has a pending migration to workflow; please retry later")
ErrTimeSkippingMigration = serviceerror.NewFailedPrecondition("schedule with time skipping cannot migrate to workflow-backed scheduler")

@feiyang3cat feiyang3cat Oct 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for reviewers: could you help confirm if returning this error type is safe to block migration for this chedule

@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch 2 times, most recently from 09ee31f to a740006 Compare October 9, 2026 06:26
feiyang3cat
feiyang3cat commented Oct 9, 2026
View reviewed changes
Comment thread service/frontend/validators.go
// validateTimeSkippingStatePropagation keeps propagation state internal-only while CHASM Schedules
// start Workflows through Frontend.
// TODO: Delete this function once they start Workflows directly from History.
func validateTimeSkippingStatePropagation(

@feiyang3cat feiyang3cat Oct 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @simvlad

feiyang3cat
feiyang3cat commented Oct 9, 2026
View reviewed changes
Comment thread service/frontend/workflow_handler.go
return nil
}

func (wh *WorkflowHandler) validateAndPopulateWorkflowTimeSkippingConfig(

@feiyang3cat feiyang3cat Oct 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

split previous validator to one for wf and one for schedule

@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch from a740006 to 8b7ec2b Compare October 9, 2026 06:44
@feiyang3cat
feiyang3cat force-pushed the fx/vts-chasm-schedule branch from 8b7ec2b to e6e8b8a Compare October 9, 2026 07:10
feiyang3cat
feiyang3cat commented Oct 9, 2026
View reviewed changes
Comment thread chasm/lib/scheduler/invoker_tasks.go

// Record time taken from action eligible to workflow started.
if !start.Manual {
desiredTime := cmp.Or(start.DesiredTime, start.ActualTime)

@feiyang3cat feiyang3cat Oct 9, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check the todo comment in chasm/lib/scheduler/invoker.go

feiyang3cat
feiyang3cat commented Oct 9, 2026
View reviewed changes
Comment thread chasm/lib/scheduler/invoker.go
idx := slices.IndexFunc(i.BufferedStarts, func(start *schedulespb.BufferedStart) bool {
return start.Attempt == 0
})
// TODO(time-skipping): The completed workflow and its schedule use independent virtual clocks, so

@feiyang3cat feiyang3cat Oct 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

related to chasm/lib/scheduler/invoker_tasks.go L759
it seems we can choose to change this time to ctx.Now() but even if we don't change it will be a rare issue and only impacts metrics

and this is the only time schedule reads directly from workflows

@feiyang3cat
feiyang3cat added this pull request to stack #12483 October 9, 2026 07:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@davidporter-id-au davidporter-id-au Awaiting requested review from davidporter-id-au
@simvlad simvlad Awaiting requested review from simvlad
@chaptersix chaptersix Awaiting requested review from chaptersix

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@feiyang3cat