Unpin selenium to fix CI flakiness against current Chrome - #3955
Open
T4rk1n wants to merge 2 commits into
Open
Conversation
The testing requirements capped selenium at <=4.2.0 (2022), which predates Selenium Manager. CI installs the current stable Chrome (now 151) via an unpinned browser-actions/setup-chrome, and selenium 4.2 cannot reliably provision or drive it, producing scattered StaleElementReferenceException / TimeoutException failures across unrelated browser integration tests on every push and PR. Require selenium>=4.11.0 (mature Selenium Manager auto-provisions a matching chromedriver) up to the current latest 4.46.0.
Unpinning selenium exposed two deterministic breaks the 4.2.0 cap had hidden: - browser.py set the 'marionette' Firefox capability, which modern selenium/geckodriver reject with InvalidArgumentException (marionette is the implicit, only protocol now). Removed it. - Three test modules used the find_element(s)_by_* helper methods that selenium removed in 4.3. Migrated them to find_element(s)(By.*, ...).
Contributor
Author
|
Follow-up commit
Remaining |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Recent pushes and PRs have been going red intermittently across the browser-based integration tests. The failures are scattered across unrelated Selenium tests (
test_persistence,test_csp,test_multi_output,test_derived_props, async callbacks, table server tests…), a different subset each run, all surfacing asStaleElementReferenceException/TimeoutException. That pattern is environmental flakiness, not a bad merge (which would fail the same test deterministically).Root cause
requirements/testing.txtpinnedselenium>=3.141.0,<=4.2.0(selenium 4.2.0 is from 2022).browser-actions/setup-chrome@v1withchrome-version: stable— unpinned — so CI now installs Chrome 151.Two amplifiers: the dependabot pip bump that would have raised selenium never landed on
dev, and the recent React 18/19 test matrix roughly doubled the browser shards, so a single flake reddens the whole run more often.Fix
Bump the pin to
selenium>=4.11.0,<=4.46.0. The>=4.11.0floor guarantees a mature Selenium Manager that auto-provisions a chromedriver matching whatever stable Chrome CI installs (this is whyinstall-chromedriver: falsein the setup step remains correct).Compatibility checks
find_element_by_*APIs anywhere indash/(those were dropped in selenium 4.3).webdriver.Chrome(options=...)/webdriver.Remote(command_executor=..., options=...).dash/testing/browser.pyuses, plus Selenium Manager availability, against selenium 4.46.0.Follow-ups (not in this PR)
test_async_cbsc001_simple_callback) and may be genuinely broken rather than flaky — worth a targeted look once this settles the noise.stablefine either way.