Skip to content

Report per operation timings in the multi editor and perspective tests - #4232

Open
vogella wants to merge 1 commit into
eclipse-platform:masterfrom
vogella:perf-tests-multi-editor-perspective
Open

Report per operation timings in the multi editor and perspective tests#4232
vogella wants to merge 1 commit into
eclipse-platform:masterfrom
vogella:perf-tests-multi-editor-perspective

Conversation

@vogella

@vogella vogella commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Continues #4230 for the two tests that cover the operations users feel most: opening many editors and switching perspectives. Both drop org.eclipse.test.performance, warm up before measuring, time the individual operations and print the distribution, so a change in one operation can be attributed instead of disappearing into a single number that nothing recorded.

OpenMultipleEditorTest also reports the first and the last ten editors separately, which turns it into the only test we have that shows how cost scales with the number of open editors. Opening the 90th editor currently costs 20 to 67 percent more than the 10th, and closing an editor while 90 are open costs about three times as much as when 10 remain. Because both ends come from the same run, that comparison is unaffected by machine load, which the absolute numbers very much are.

While adding assertions I found that the test never measured what it claimed: the workbench recycles the oldest editor once REUSE_EDITORS are open, defaulting to 99, so the hundredth open silently reused the first editor. The count is now 90 with an assertion explaining the threshold. PerspectiveSwitchTest had a related problem, silently returning and printing to stdout when a perspective is missing, which reported a pass for something never measured; it now skips through an assumption, so the three JDT cases show up as skipped rather than green.

Sample output of a local run:

openMultiple[perf_outline, closeEach] open, first 10  n=10  min= 35.52  p50= 43.01  p90= 67.45 (ms)
openMultiple[perf_outline, closeEach] open, last 10   n=10  min= 54.62  p50= 67.08  p90= 76.15 (ms)
openMultiple[perf_outline, closeEach] close, first 10 n=10  min= 25.48  p50= 30.01  p90= 39.69 (ms)
openMultiple[perf_outline, closeEach] close, last 10  n=10  min=  6.22  p50=  9.46  p90= 17.50 (ms)
openMultiple[perf_text, closeAll] closeAllEditors     n=1   1011.07 (ms, 90 editors)
PerspectiveSwitch to [org.eclipse.ui.tests.performancePerspective1] n=50 min= 16.74 p50= 18.60 (ms)

Migrates OpenMultipleEditorTest and PerspectiveSwitchTest the same way as
OpenCloseEditorTest and EditorSwitchTest: drop org.eclipse.test.performance,
warm up before measuring, time the individual operations and print the
distribution through UIPerformanceTestUtil.reportTimings. The performance
database these reported to has not been configured for years, so
commitMeasurements and assertPerformance were no-ops and the tests could
neither fail nor report anything.

OpenMultipleEditorTest additionally reports the first and the last ten
editors separately. Cost per editor grows with the number already open, and
that is only visible from the two ends of the same run, which also makes the
comparison immune to machine load. It also opens 90 instead of 100 editors:
the workbench recycles the oldest editor once REUSE_EDITORS are open, which
defaults to 99, so the hundredth open silently reused the first editor and
the test measured reuse rather than opening. An assertion now pins that down.

PerspectiveSwitchTest reports each switch direction separately and skips
through an assumption when a perspective is missing, instead of printing to
stdout and returning, which reported a pass for something never measured.
Three of its four cases reference JDT perspectives and are skipped in this
target platform, which is now visible in the test results.
@github-actions

Copy link
Copy Markdown
Contributor

Test Results

   858 files  ±0     858 suites  ±0   52m 16s ⏱️ + 5m 14s
 8 130 tests ±0   7 887 ✅ +3  243 💤 ±0  0 ❌  - 1 
20 292 runs  ±0  19 638 ✅ +3  654 💤 ±0  0 ❌  - 1 

Results for commit 2e1e66d. ± Comparison against base commit aa34a9d.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant