Report per operation timings in OpenMultipleEditorTest - #4239
Merged
vogella merged 1 commit intoAug 13, 2026
Merged
Conversation
OpenMultipleEditorTest measured opening 100 editors and closing them all in one block, so neither an individual open nor the teardown could be attributed. Each open is now timed on its own, and the close phase is timed per variant, one sample for closeAll and one per editor for closeEach. Since the subject of this test is the accumulation of editors, a single distribution over all 100 opens would average away what is being measured. The first and the last 25 opens are therefore reported separately, which makes visible whether an open gets more expensive as the editor area fills up. Drop org.eclipse.test.performance, following the same move already made for OpenCloseEditorTest and EditorSwitchTest. The performance database it reports to has not been configured for years, which made commitMeasurements and assertPerformance no-ops and left the test unable to fail or to report anything. The test warms up before measuring so that class loading and JIT no longer end up in the reported times, and it is bounded by time instead of a single fixed round. It now asserts that every editor actually opened and that none are left behind, so it can fail. The parameterized display name was showing the extension under a closeAll label and the closeAll flag under a closeEach label. Widen the label column in reportTimings so the longer labels stay aligned.
Contributor
Contributor
Author
|
Change in test code, performance tests are not part of the CI verification (only compiled) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Applies the same treatment to
OpenMultipleEditorTestthatOpenCloseEditorTestandEditorSwitchTestalready received: it dropsorg.eclipse.test.performance, whose measurements went nowhere because the performance database has not been configured for years, and times the individual operations instead.Each of the 100 opens is now timed on its own and the close phase is timed per variant. Since this test is about the accumulation of editors, the first and last 25 opens are reported as separate distributions, so it is visible whether an open gets more expensive as the editor area fills up. The test warms up before measuring and is bounded by time rather than a single fixed round.
The practical gain is that the test can now fail and can now report something. It asserts that every editor really opened and that none are left behind, and a manual run prints min, median, 90th percentile and maximum per operation. The first numbers already show that closing 100 editors one by one costs noticeably more than
closeAllEditors, and that the per open cost does not grow at the floor as editors accumulate.Also fixes the parameterized display name, which labelled the extension as
closeAlland thecloseAllflag ascloseEach.