Commit graph

8 commits

Author SHA1 Message Date
Oleksiy Syvokon
8382afb2ba
evals: Run unit evals CI weekly (#35950)
Release Notes:

- N/A
2025-08-10 14:43:48 +00:00
Peter Tripp
952e3713d7
ci: Switch to Namespace (#35835)
Follow-up to:
- https://github.com/zed-industries/zed/pull/35826

Release Notes:

- N/A
2025-08-07 23:16:25 +00:00
Peter Tripp
7679db99ac
ci: Switch from BuildJet to GitHub runners (#35826)
In response to an ongoing BuildJet outage, consider migrating CI to
GitHub hosted runners.

Also includes revert of (causing flaky tests):
- https://github.com/zed-industries/zed/pull/35741

Downsides:
- Cost (2x)
- Force migration to Ubuntu 22.04 from 20.04 will bump our glibc minimum
from 2.31 to 2.35. Which would break RHEL 9.x (glibc 2.34), Ubuntu 20.04
(EOL) and derivatives.

Release Notes:

- N/A
2025-08-07 16:59:11 -04:00
Peter Tripp
70bde54a2c
ci: Lint GitHub Actions workflows with actionlint (#34729)
Added [rhysd/actionlint](https://github.com/rhysd/actionlint/) a static
checker for GitHub Actions workflow files.
Install locally with `brew install actionlint` the run with
`actionlint`.

Inspired by: https://github.com/zed-industries/zed/pull/34704 which
yielded this observation:

> In github actions: 
> 1. strings are truthy
> 2. `${{ }}` will become a string if it doesn't wrap the whole value.
>
> So `if: false && true` becomes `false`
> and `if: ${{ false && true }}` becomes `false`
> but `if: false && ${{ true }}` becomes `"false && true"` which
evaluates true
> The reason you sometimes need `${{ }}` is because YAML doesn't like
`!`
> so `if: !false` is invalid yaml
> and `if: ${{ !false }}` works just fine.

Changes:
- Add `actionlint` job
- Refactor `job_spec` job to be more readable
- Fix all `actionlint` and `shellcheck` errors in Actions workflows (62
in all)
- Add `self-mini-macos` and `self-32vcpu-windows-2022` labels to
self-hosted runners. Not strictly related, but useful if you need to
take a runner out of the rotation (since `macOS`, `self-hosted`, and
`ARM64` are auto-set and cannot be added/removed).
- Change ci.yml macos_relase to target `self-mini-macos` instead of
`bundle` which was previously deprecated.

This would've caught the error fixed in
https://github.com/zed-industries/zed/pull/34704. Here's what that [job
failure](https://github.com/zed-industries/zed/actions/runs/16376993944/job/46279281842?pr=34729)
would've looked like.

Release Notes:

- N/A
2025-07-18 15:15:07 -04:00
Peter Tripp
02dfaf7799
ci: Suppress evals on forks (#32479)
Be kind to those with Zed forks.

Example [action run on
fork](https://github.com/G36maid/freebsd-ports-zed/actions/runs/15525942275)
where [this
job](https://github.com/G36maid/freebsd-ports-zed/actions/runs/15549650437/job/43777665341)
will wait forever. Sorry @G36maid

Release Notes:

- N/A
2025-06-10 18:20:03 +00:00
Ben Brandt
e4bd115a63
More resilient eval (#32257)
Bubbles up rate limit information so that we can retry after a certain
duration if needed higher up in the stack.

Also caps the number of concurrent evals running at once to also help.

Release Notes:

- N/A
2025-06-09 18:07:22 +00:00
Ben Brandt
ddf70b3bb8
Add mismatched tag threshold parameter to eval function (#32190)
Replace hardcoded 0.10 threshold with configurable parameter and set
0.05 default for most tests, with 0.2 for from_pixels_constructor
eval that produces more mismatched tags.

Release Notes:

- N/A
2025-06-05 21:30:05 +00:00
Ben Brandt
dda614091a
eval: Add eval unit tests as a CI job (#32152)
We run the unit evals once a day in the middle of the night, and trigger
a Slack post if it fails.


Release Notes:

- N/A

---------

Co-authored-by: Oleksiy Syvokon <oleksiy.syvokon@gmail.com>
2025-06-05 13:16:27 +00:00