Add seed argument to #[gpui::test] attribute macro (#26764)

This PR introduces the arguments `seed` and `seeds` to `gpui::test`,
e.g.:
- `#[gpui::test(seed = 10)]`
- `#[gpui::test(seeds(10, 20, 30, 40))]`

Which allows us to run a test against a specific seed value without
slowing
down our tests like `iterations` does with high values.

This was motivated by a diff hunk test that only fails in a 400+ seed,
but is
slow to run 400+ times for every `cargo test`.

If your test failed with a specific seed, you can now add the `seed` arg
to
increase the chances of detecting a regression.

There are now three ways of setting seeds, the `SEED` env var,
`iterations`,
and the args this PR adds. See docs in `gpui::test`.

---

I also relaxed the limitation on `retries` not working with
`iterations`, as
that seemed unnecessary.

Release Notes:

- N/A
This commit is contained in:
João Marcos 2025-03-14 13:40:02 -03:00 committed by GitHub
parent 83dfdb0cfe
commit f587e95a7e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 165 additions and 90 deletions

View file

@ -144,7 +144,8 @@ pub fn box_shadow_style_methods(input: TokenStream) -> TokenStream {
}
/// `#[gpui::test]` can be used to annotate test functions that run with GPUI support.
/// it supports both synchronous and asynchronous tests, and can provide you with
///
/// It supports both synchronous and asynchronous tests, and can provide you with
/// as many `TestAppContext` instances as you need.
/// The output contains a `#[test]` annotation so this can be used with any existing
/// test harness (`cargo test` or `cargo-nextest`).
@ -160,11 +161,25 @@ pub fn box_shadow_style_methods(input: TokenStream) -> TokenStream {
/// Using the same `StdRng` for behavior in your test will allow you to exercise a wide
/// variety of scenarios and interleavings just by changing the seed.
///
/// `#[gpui::test]` also takes three different arguments:
/// - `#[gpui::test(iterations=10)]` will run the test ten times with a different initial SEED.
/// - `#[gpui::test(retries=3)]` will run the test up to four times if it fails to try and make it pass.
/// - `#[gpui::test(on_failure="crate::test::report_failure")]` will call the specified function after the
/// # Arguments
///
/// - `#[gpui::test]` with no arguments runs once with the seed `0` or `SEED` env var if set.
/// - `#[gpui::test(seed = 10)]` runs once with the seed `10`.
/// - `#[gpui::test(seeds(10, 20, 30))]` runs three times with seeds `10`, `20`, and `30`.
/// - `#[gpui::test(iterations = 5)]` runs five times, providing as seed the values in the range `0..5`.
/// - `#[gpui::test(retries = 3)]` runs up to four times if it fails to try and make it pass.
/// - `#[gpui::test(on_failure = "crate::test::report_failure")]` will call the specified function after the
/// tests fail so that you can write out more detail about the failure.
///
/// You can combine `iterations = ...` with `seeds(...)`:
/// - `#[gpui::test(iterations = 5, seed = 10)]` is equivalent to `#[gpui::test(seeds(0, 1, 2, 3, 4, 10))]`.
/// - `#[gpui::test(iterations = 5, seeds(10, 20, 30)]` is equivalent to `#[gpui::test(seeds(0, 1, 2, 3, 4, 10, 20, 30))]`.
/// - `#[gpui::test(seeds(10, 20, 30), iterations = 5]` is equivalent to `#[gpui::test(seeds(0, 1, 2, 3, 4, 10, 20, 30))]`.
///
/// # Environment Variables
///
/// - `SEED`: sets a seed for the first run
/// - `ITERATIONS`: forces the value of the `iterations` argument
#[proc_macro_attribute]
pub fn test(args: TokenStream, function: TokenStream) -> TokenStream {
test::test(args, function)