Add CI integration for @elastic/eui-test-helpers (M3)

[steliosmavro]

Summary

What: CI integration (Milestone 3) for @elastic/eui-test-helpers, building on the EuiComboBoxObject helper added in #9644.

Why: The helper package needs its validation tests running in EUI CI, with extra flakiness coverage when an underlying component changes, so breakage is caught before it reaches downstream Scout tests.

How:

• Validation on every PR — a test-helpers task in pipeline_test.sh builds the EUI workspaces + Storybook, installs Chromium, and runs the Playwright validation suite. Storybook is served by Playwright's webServer (static build + http-server devDependency); reuseExistingServer leaves the local dev-server workflow untouched.
• Flake detection on component change — pipeline_test_helpers_flaky.sh diffs the PR against its merge base and, for any changed component with a correlated helper, dynamically uploads a step re-running that helper's specs --repeat-each=25. Correlation is by directory name (eui/src/components/<name> ⇄ test-helpers/src/playwright/components/<name>) — no map to maintain. A flaky failure fails the build; no-op when nothing correlated changed. Pure bash, no node on the agent host.
• Scheduled run — pipeline_test_helpers_scheduled.yml runs all specs --repeat-each=5 and notifies Slack on failure.

This is Milestone 3 of the eui-test-helpers epic (M2 was #9644).

Note

One-time Buildkite setup (not in-repo, needs org-admin/Vault): schedule a build of pipeline_test_helpers_scheduled.yml every 8h (cron 0 */8 * * *); enable the Buildkite Slack integration and confirm the channel (currently #appex-qa).

API Changes

component / parent	prop / child	change	description
None			CI + private-package tooling only

Screenshots

N/A — CI configuration only.

Impact Assessment

• 🔴 Breaking changes
• 💅 Visual changes
• 🧪 Test impact — additive only; adds CI steps, changes no existing behavior.
• 🔧 Hard to integrate

Impact level: 🟢 None — package is private; changes are additive CI plus a private-package devDependency.

Release Readiness

• Documentation: wiki/contributing-to-eui/testing/eui-test-helpers.md + package CONTRIBUTING
• Figma
• Migration guide
• Adoption plan: Milestone 4 — consumed by Kibana Scout.

QA instructions for reviewer

• CI: the EUI test-helpers validation tests step runs the suite on this PR. The flake-detection step no-ops here (no correlated component changed); it triggers only when a component with a helper is modified.
• Locally:

  nvm use && yarn
  yarn workspace @elastic/eui build:workspaces && yarn workspace @elastic/eui start  # Storybook on :6006
  # in another terminal:
  yarn workspace @elastic/eui-test-helpers test

Checklist before marking Ready for Review

• Filled out all sections above
• QA: light/dark, high contrast, mobile, browsers, keyboard, screen reader
• QA: Tested in CodeSandbox and Kibana — Kibana consumption is M4.
• QA: Tested docs changes
• Tests: validation specs (from Add EuiComboBoxObject Playwright helper to @elastic/eui-test-helpers #9644) now run in CI, plus flake-detection and scheduled coverage.
• Changelog — package is private, not released.
• Breaking changes label

Reviewer checklist

• Approved Impact Assessment
• Approved Release Readiness

[elastic-vault-github-plugin-prod]

3 visual difference(s) found - expand to review, then click Approve visual changes to update baselines

euicodeblock (1 difference)

Story	Before	After	Diff
virtualized code block scrolling desktop

euipopover (1 difference)

Story	Before	After	Diff
playground desktop

euidatagrid (1 difference)

Story	Before	After	Diff
additional controls options desktop

[Copilot]

Pull request overview

Integrates @elastic/eui-test-helpers Playwright validation tests into EUI’s Buildkite CI, adding PR validation, component-correlated flake detection reruns, and a scheduled flaky-run pipeline, with supporting documentation updates.

Changes:

• Add a test-helpers CI task that builds workspaces + Storybook and runs the Playwright suite (including a static Storybook web server in CI).
• Add a flake-detection script that detects correlated component changes and re-runs affected helper specs with --repeat-each.
• Add a scheduled Buildkite pipeline and document the new CI behavior in the wiki + package CONTRIBUTING.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
yarn.lock	Locks new dependency graph for http-server used by CI Storybook serving.
wiki/contributing-to-eui/testing/README.md	Adds EUI test-helpers to the testing documentation index.
wiki/contributing-to-eui/testing/eui-test-helpers.md	New wiki page documenting CI validation, flake detection, and scheduled runs.
packages/test-helpers/playwright.config.ts	Adds Playwright webServer to serve static Storybook in CI (and reuse dev server locally).
packages/test-helpers/package.json	Adds http-server devDependency for serving static Storybook.
packages/test-helpers/CONTRIBUTING.md	Documents CI integration and the local vs CI Storybook serving behavior.
.buildkite/scripts/pipelines/pipeline_test.sh	Adds test-helpers task and plumbs PLAYWRIGHT_ARGS into the container.
.buildkite/scripts/pipelines/pipeline_test_helpers_flaky.sh	New script that detects correlated component changes and uploads rerun steps with repeats.
.buildkite/pipelines/pipeline_test_helpers_scheduled.yml	New scheduled pipeline to run all helper specs with repeats and Slack notify on failure.
.buildkite/pipelines/pipeline_pull_request_test.yml	Adds PR-time validation step and flake-detection step to the CI pipeline.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[tkajtoch]

Hi @steliosmavro!

Scheduled run — pipeline_test_helpers_scheduled.yml runs all specs --repeat-each=5 and notifies Slack on failure.

My personal take on this is that we don't need it. We can make all CI of eui-test-helpers with the flaky repeat configuration for very high certainty of their correctness. I'm not a huge fan of cron jobs for these kinds of things. We can run the same test suite when changes get merged to main, so that it all gets validated once more, but periodic runs of playwright/jest test suites are unlikely to find anything, and the setup is quite cumbersome and expensive.

Here's what I would recommend:

• All PRs run eui-test-helpers test suites as any other unit and integration tests
• PRs that touch eui-test-helpers run all test suites in flaky mode, rerunning them many times + testing against different kinds of machines / throttling configurations (Emulation.setCPUThrottlingRate)
• Optionally, there's a periodic job (1 a day?) that runs tests against different browser versions: current stable release, beta release, and an experimental release

LMK if you need any assistance with this! We have some other Buildkite changes in open PRs and I'm happy to help with this :D

[steliosmavro]

Thanks @tkajtoch I agree 👍 Dropped the scheduled job + Slack from the PR. For the on-merge run, this PR or a follow up? Since nothing runs on main​ today I'd guess that's more your side to enable so happy to do a separate change on top. I'd leave it as is for now.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 7d0fa1b
• Documentation website
• Storybook
• 🚫 Visual regression tests skipped (remove the skip-vrt label to re-enable)

History

• 💚 Build #3965 succeeded 7f9a700
• 💚 Build #3928 succeeded db0118e
• 💔 Build #3920 failed 810ecb6

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 7d0fa1b

History

• 💚 Build #6856 succeeded 7f9a700
• 💚 Build #6825 succeeded db0118e
• 💚 Build #6823 succeeded 810ecb6
• 💚 Build #6819 succeeded 810ecb6