Repository navigation
Conversation
4b7d81f to
723d153
Compare
| PERIODIC_STATUS_PERIOD_S = 30.0 | ||
|
|
||
| OPERATION_ORDER_TOLERANCE = timedelta(seconds=60) | ||
| """Operations are recorded as they complete, so they are ordered by completion time to within this tolerance.""" |
There was a problem hiding this comment.
I don't understand why the operations are ordered within a 60 seconds tolerance because they are recorded as they complete?
Also: I think making this assumption here of logic defined elsewhere is not good. If that assumption changes (and I don't think it is guaranteed by anything?), this will break this logic here.
Maybe replace operations with a data structure that has some guarantees on the sorting?
| ) | ||
|
|
||
|
|
||
| def first_index_completed_since( |
There was a problem hiding this comment.
Have you considered instead doing something like summarize_and_report_step?
step_ops = [
op
for op in operations
if op.completed_at.datetime >= step_start_time
and op.completed_at.datetime <= step_end_time
]|
Is the purpose of this PR to reduce the processing time required by check_stability_criteria by reducing the number of operations to iterate over when checking various criteria? If so, I would probably expect a search for the cutoff index to use a more performant algorithm (like binary search) and then just pass the cutoff/start index as an additional parameter to check_stability_criteria rather than creating an entirely new list to pass to check_stability_criteria. Though I am surprised that the check_stability_criteria processing time would be appreciable, even with a pretty large volume of operations. |
|
Thank you @BenjaminPelletier and @mickmis, let me re-open this issue with more evidence so we can better address root cause. |
This PR improves the criteria evaluation process for large number of operations by only considering operations of interest for the current evaluated step.