This commit weaves through new APIs for language::BufferChunks, multi_buffer::MultiBufferChunks and inlay_map::InlayChunks that allow seeking with an upper-bound. This allows us to omit doing syntax highligting and looking up diagnostics for folded ranges. This in turn directly improves performance of assistant panel with large contexts.
Release Notes:
- Fixed poor performance when editing in the assistant panel after
inserting large files using slash commands
---------
Co-authored-by: Max <max@zed.dev>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
This PR removes the primary/secondary distinction for
`CachedLspAdapter`s.
After #15624 we weren't relying on the `is_primary` field anywhere, so
we can remove it.
Release Notes:
- N/A
Note that this shouldn't have any visible user-facing behavior yet. The
feature is incomplete but we wanna merge early to avoid a long-running
branch.
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
In the past, Zed used a single switch called `autoclose` to control both
`autoclose` and `auto_surround` functionalities:
+ `autoclose`: when input '(', append ')' automatically.
+ `auto_surround`: when select text and input '(', surround text with
'(' and ')' automatically.
This PR separates `auto_surround` from `autoclose` to support `<`.
Previously, if `autoclose` of `<` was set to `false`, `auto_surround`
couldn't be used. However, setting `autoclose` to `true` would affect
the default behavior of simple expression. For example, `a < b` would
become `a <> b`.
For more information, see #13187.
Fix#12898.
Release Notes:
- Added support for `auto_surround`
([#12898](https://github.com/zed-industries/zed/issues/12898)).
Follow-up of
https://github.com/zed-industries/zed/pull/12095#issuecomment-2123230762
reverting back part of https://github.com/zed-industries/zed/pull/11558
that was related to `language.toml` parsing.
Now all extensions that define `prettier_parser_name` in their language
configs, will enable formatting untitled buffers without any extra
language settings like
```json
{
"languages": {
"JSON": {
"prettier": {
"allowed": true,
"parser": "json"
}
}
}
}
```
Release Notes:
- Improved ergonomics of untitled buffer formatting with prettier, no
extra language settings are needed by default.
In #12003 we found ourselves in need for precise region tracking in
which a given runnable has an effect in order to grab variables from it.
This PR makes it so that in task modal all task variables from queries
overlapping current cursor position.
However, in the process of working on that I've found that we cannot
always use a top-level capture to represent the full match range of
runnable (which has been my assumption up to this point). Tree-sitter
captures cannot capture sibling groups; we did just that in Rust
queries.
Thankfully, none of the extensions are affected as in them, a capture is
always attached to single node. This PR adds annotations to them
nonetheless; we'll be able to get rid of top-level captures in extension
runnables.scm once this PR is in stable version of Zed.
Release Notes:
- N/A
This PR changes the interface of ContextProvider, allowing it to inspect
*all* variables set so far during the process of building
`TaskVariables`. This makes it possible to capture e.g. an identifier in
tree-sitter query, process it and then export it as a task variable.
Notably, the list of variables includes captures prefixed with leading
underscore; they are removed after all calls to `build_context`, but it
makes it possible to capture something and then conditionally preserve
it (and perhaps modify it).
Release Notes:
- N/A
Closes https://github.com/zed-industries/zed/issues/11517
* Removes forced prettier parser name for languages, making `auto`
command to run prettier on every file by default.
* Moves prettier configs away from plugin language declarations into
language settings
Release Notes:
- N/A
From now on, only top-level captures are treated as runnable tags and
the rest is appended to task context as custom environmental variables
(unless the name is prefixed with _, in which case the capture is
ignored). This is most likely gonna help with Pest-like test runners.
Release Notes:
- N/A
---------
Co-authored-by: Remco <djsmits12@gmail.com>
Notable things I've had to fix due to 1.78:
- Better detection of unused items
- New clippy lint (`assigning_clones`) that points out places where assignment operations with clone rhs could be replaced with more performant `clone_into`
Release Notes:
- N/A
This fixes a tricky intermittent issue I was seeing, where failed to
chunk certain files correctly because of the way we reuse Tree-sitter
`Parser` instances across parses.
I've also accounted for leading comments in chunk boundaries, so that
items are grouped with their leading comments whenever possible when
chunking.
Finally, we've changed the `debug project index` action so that it opens
a simple debug view in a pane, instead of printing paths to the console.
This lets you click into a path and see how it was chunked.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
This chunking strategy uses the existing `outline` query to chunk files.
We try to find chunk boundaries that are:
* at starts or ends of lines
* nested within as few outline items as possible
Release Notes:
- N/A
Before, `tasks_ui` set most of the context with `SymbolContextProvider`
providing the symbol data part of the context. Now, there's a
`BasicContextProvider` that forms all standard Zed context and it
automatically serves as a base, with no need for other providers like
`RustContextProvider` to call it as before.
Also, stop adding `SelectedText` task variable into the context for
blank text selection.
Release Notes:
- N/A
This introduces semantic indexing in Zed based on chunking text from
files in the developer's workspace and creating vector embeddings using
an embedding model. As part of this, we've created an embeddings
provider trait that allows us to work with OpenAI, a local Ollama model,
or a Zed hosted embedding.
The semantic index is built by breaking down text for known
(programming) languages into manageable chunks that are smaller than the
max token size. Each chunk is then fed to a language model to create a
high dimensional vector which is then normalized to a unit vector to
allow fast comparison with other vectors with a simple dot product.
Alongside the vector, we store the path of the file and the range within
the document where the vector was sourced from.
Zed will soon grok contextual similarity across different text snippets,
allowing for natural language search beyond keyword matching. This is
being put together both for human-based search as well as providing
results to Large Language Models to allow them to refine how they help
developers.
Remaining todo:
* [x] Change `provider` to `model` within the zed hosted embeddings
database (as its currently a combo of the provider and the model in one
name)
Release Notes:
- N/A
---------
Co-authored-by: Nathan Sobo <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Antonio <antonio@zed.dev>
This PR adds the ability for extensions to provide certain language
settings via the language `config.toml`.
These settings are then merged in with the rest of the settings when the
language is loaded from the extension.
The language settings that are available are:
- `tab_size`
- `hard_tabs`
- `soft_wrap`
Additionally, for bundled languages we moved these settings out of the
`settings/default.json` and into their respective `config.toml`s .
For languages currently provided by extensions, we are leaving the
values in the `settings/default.json` temporarily until all released
versions of Zed are able to load these settings from the extension.
---
Along the way we ended up refactoring the `Settings::load` method
slightly, introducing a new `SettingsSources` struct to better convey
where the settings are being loaded from.
This makes it easier to load settings from specific locations/sets of
locations in an explicit way.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
This PR adds the ability for extensions to implement
`language_server_workspace_configuration` to provide workspace
configuration to the language server.
We've used the Dart extension as a motivating example for this, pulling
it out into an extension in the process.
Release Notes:
- Removed built-in support for Dart, in favor of making it available as
an extension. The Dart extension will be suggested for download when you
open a `.dart` file.
---------
Co-authored-by: Max <max@zed.dev>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
This PR adds `label_for_symbol` to the extension API.
As a motivating example, we implemented `label_for_symbol` for the
Haskell extension.
Release Notes:
- N/A
Co-authored-by: Max <max@zed.dev>
This PR adds the ability for extensions to implement
`label_for_completion` to customize completions coming back from the
language server.
We've used the Gleam extension as a motivating example, adding
`label_for_completion` support to it.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
Once we enable extensions to customize the labels of completions and
symbols, this new structure will allow this to be done with a single
WASM call, instead of one WASM call per completion / symbol.
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Marshall <marshall@zed.dev>
Introducing the Active File Context portion of #9705. When someone is in
the assistant panel it now includes the active file as a system message
on send while showing them a nice little display in the lower right:
For this iteration, I'd love to see the following before we land this:
* [x] Toggle-able context - user should be able to disable sending this
context
* [x] Show nothing if there is no context coming in
* [x] Update token count as we change items
* [x] Listen for a more finely scoped event for when the active item
changes
* [x] Create a global for pulling a file icon based on a path. Zed's
main way to do this is nested within project panel's `FileAssociation`s.
* [x] Get the code fence name for a Language for the system prompt
* [x] Update the token count when the buffer content changes
I'm seeing this PR as the foundation for providing other kinds of
context -- diagnostic summaries, failing tests, additional files, etc.
Release Notes:
- Added file context to assistant chat panel
([#9705](https://github.com/zed-industries/zed/issues/9705)).
<img width="1558" alt="image"
src="https://github.com/zed-industries/zed/assets/836375/86eb7e50-3e28-4754-9c3f-895be588616d">
---------
Co-authored-by: Conrad Irwin <conrad@zed.dev>
Co-authored-by: Nathan <nathan@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Mikayla Maki <mikayla@zed.dev>
We're doing it. Svelte support is moving into an extension. This PR
fixes some issues that came up along the way.
Notes
* extensions need to be able to retrieve the path the `node` binary
installed by Zed
* previously we were silently swallowing any errors that occurred while
loading a grammar
* npm commands ran by extensions weren't run in the right directory
* Tree-sitter's WASM stdlib didn't support a C function (`strncmp`)
needed by the Svelte parser's external scanner
* the way that LSP installation status was reported was unnecessarily
complex
Release Notes:
- Removed built-in support for the Svelte and Gleam languages, because
full support for those languages is now available via extensions. These
extensions will be suggested for download when you open a `.svelte` or
`.gleam` file.
---------
Co-authored-by: Marshall <marshall@zed.dev>
This PR also introduces built-in tasks for Rust and Elixir. Note that
this is not a precedent for future PRs to include tasks for more
languages; we simply want to find the rough edges with tasks & language
integrations before proceeding to task contexts provided by extensions.
As is, we'll load tasks for all loaded languages, so in order to get
Elixir tasks, you have to open an Elixir buffer first. I think it sort
of makes sense (though it's not ideal), as in the future where
extensions do provide their own tasks.json, we'd like to limit the # of
tasks surfaced to the user to make them as relevant to the project at
hand as possible.
Release Notes:
- Added built-in tasks for Rust and Elixir files.
Our goal is to extract Svelte support into an extension, since we've
seen problems with the Tree-sitter Svelte parser crashing due to bugs in
the external scanner. In order to do this, we need a couple more
capabilities in LSP extensions:
* [x] `initialization_options` - programmatically controlling the JSON
initialization params sent to the language server
* [x] `prettier_plugins` - statically specifying a list of prettier
plugins that apply for a given language.
* [x] `npm_install_package`
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Closes https://github.com/zed-industries/zed/issues/9458
When flying in a plane being totally offline, I've discovered that my
Rust projects do not have any LSP support and rust-analyzer disappeared
out of `~/Library/Application Support/Zed/languages/rust-analyzer/`
directory.
Looking at the
[bad.log](https://github.com/zed-industries/zed/files/14627508/bad.log),
it appears that `get_language_server_command` tries to find a newer LSP
server version and fails on
80bc6c8cc8/crates/language/src/language.rs (L339)
bailing out of all installation related-methods up to here:
80bc6c8cc8/crates/project/src/project.rs (L2916)
where the code thinks that the binary installation process had failed,
cleans the existing directory and tries to install the language server
again:
```log
[2024-03-17T15:14:13+02:00 WARN isahc::handler] request completed with error: failed to resolve host name
[2024-03-17T15:14:13+02:00 ERROR project] failed to start language server "rust-analyzer": error fetching latest release
[2024-03-17T15:14:13+02:00 ERROR project] server stderr: Some("")
[2024-03-17T15:14:13+02:00 INFO project] retrying installation of language server "rust-analyzer" in 1s
[2024-03-17T15:14:13+02:00 ERROR util] crates/lsp/src/lsp.rs:720: oneshot canceled
[2024-03-17T15:14:14+02:00 INFO project] About to spawn test binary
[2024-03-17T15:14:14+02:00 WARN project] test binary failed to launch
[2024-03-17T15:14:14+02:00 WARN project] test binary check failed
[2024-03-17T15:14:14+02:00 INFO project] beginning to reinstall server
[2024-03-17T15:14:14+02:00 INFO language::language_registry] deleting server container
[2024-03-17T15:14:14+02:00 INFO language::language_registry] starting language server "rust-analyzer", path: "/Users/someonetoignore/work/other/local_test", id: 2
[2024-03-17T15:14:14+02:00 INFO language] fetching latest version of language server "rust-analyzer"
[2024-03-17T15:14:14+02:00 WARN isahc::handler] request completed with error: failed to resolve host name
[2024-03-17T15:14:14+02:00 ERROR project] failed to start language server "rust-analyzer": error fetching latest release
[2024-03-17T15:14:14+02:00 ERROR project] server stderr: Some("")
[2024-03-17T15:14:14+02:00 INFO project] retrying installation of language server "rust-analyzer" in 1s
[2024-03-17T15:14:15+02:00 ERROR util] crates/languages/src/rust.rs:335: no cached binary
[2024-03-17T15:14:15+02:00 INFO project] About to spawn test binary
............
```
The PR extracts away all binary fetching-related code into a single
method that does not fail the entire `get_language_server_command` and
allows it to recover and reuse the existing binary:
[good.log](https://github.com/zed-industries/zed/files/14627507/good.log)
```log
[2024-03-17T15:12:24+02:00 INFO language::language_registry] starting language server "rust-analyzer", path: "/Users/someonetoignore/work/other/local_test", id: 1
[2024-03-17T15:12:24+02:00 INFO language] fetching latest version of language server "rust-analyzer"
[2024-03-17T15:12:24+02:00 WARN isahc::handler] request completed with error: failed to resolve host name
[2024-03-17T15:12:24+02:00 INFO language] failed to fetch newest version of language server LanguageServerName("rust-analyzer"). falling back to using "/Users/someonetoignore/Library/Application Support/Zed/languages/rust-analyzer/rust-analyzer-2024-03-11"
[2024-03-17T15:12:24+02:00 INFO lsp] starting language server. binary path: "/Users/someonetoignore/Library/Application Support/Zed/languages/rust-analyzer/rust-analyzer-2024-03-11", working directory: "/Users/someonetoignore/work/other/local_test", args: []
```
Release Notes:
- Fixed language servers erased from the disk when project is opened
offline
Fixes: https://github.com/zed-industries/zed/issues/9234
This doesn't address `vue` as it has a slightly different install code,
but it should be fairly simple to add - I'll add it in in a follow-up.
This PR will allow all (except `vue`) node-based language servers to
update. It is mostly just throwing in a method into the `NodeRuntime`
trait that is used for checking if a package doesn't exist locally, or
is out of date, by checking the version against what's newest, and
installing. If any parsing of the `package.json` data fails along the
way, it assumes something has gone awry on the users system, logs the
error, and then proceeds with trying to install the package, so that
users don't get stuck on version if their package has some bad data.
Outside of adding this method, it just adds that check in all of the
language server's individual `fetch_server_binary` methods.
Release Notes:
- Added updating for node-based language servers
([#9234](https://github.com/zed-industries/zed/issues/9234)).
This fixes#9292 by adding a section to the language server settings
that allows users to specify the binary path and arguments with which to
start up a language server.
Example user settings for `rust-analyzer`:
```json
{
"lsp": {
"rust-analyzer": {
"binary": {
"path": "/Users/thorstenball/tmp/rust-analyzer-aarch64-apple-darwin",
"arguments": ["--no-log-buffering"]
}
}
}
}
```
Constraints:
* Right now this only allows ABSOLUTE paths.
* This is only used by `rust-analyzer` integration right now, but the
setting can be used for other language servers. We just need to update
the adapters to also respect that setting.
Release Notes:
- Added ability to specify `rust-analyzer` binary `path` (must be
absolute) and `arguments` in user settings. Example: `{"lsp":
{"rust-analyzer": {"binary": {"path": "/my/abs/path/rust-analyzer",
"arguments": ["--no-log-buffering"] }}}}`
([#9292](https://github.com/zed-industries/zed/issues/9292)).
Co-authored-by: Ricard Mallafre <rikitzzz@gmail.com>
Closes#5178
Release Notes:
- Added a `file_types` setting that can be used to associate languages
with file names and file extensions. For example, to interpret all `.c`
files as C++, and files called `MyLockFile` as TOML, add the following
to `settings.json`:
```json
{
"file_types": {
"C++": ["c"],
"TOML": ["MyLockFile"]
}
}
```
As with most zed settings, this can be configured on a per-directory
basis by including a local `.zed/settings.json` file in that directory.
---------
Co-authored-by: Marshall <marshall@zed.dev>
In this PR, we've added two new methods that LSP extensions can call:
* `shell_env()`, for retrieving the environment variables set in the
user's default shell in the worktree
* `which(command)`, for looking up paths to an executable (accounting
for the user's shell env in the worktree)
To test this out, we moved the `uiua` language support into an
extension. We went ahead and removed the built-in support, since this
language is extremely obscure. Sorry @mikayla-maki. To continue coding
in Uiua in Zed, for now you can `Add Dev Extension` from the extensions
pane, and select the `extensions/uiua` directory in the Zed repo. Very
soon, we'll support publishing these extensions so that you'll be able
to just install it normally.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
This PR supplements tasks with additional environment variables; ideally
we'll be able to write a task like:
`cargo test -p $ZED_CURRENT_PACKAGE -- $ZED_CURRENT_FUNCTION`
- [x] Flesh out multibuffer interactions
- [x] Add ZED_SYMBOL detection based on tree-sitter queries
- [ ] Add release note and demo
- [x] Figure out a solution for rerun dilemma - should `task: rerun`
reevaluate contexts for tasks?
This PR introduced the following variables:
- ZED_COLUMN - current line column
- ZED_ROW - current line row
and the following, which are available for buffers with associated
files:
- ZED_WORKTREE_ROOT - absolute path to the root of the current worktree.
- ZED_FILE - absolute path to the file
- ZED_SYMBOL - currently selected symbol; should match the last symbol
shown in a symbol breadcrumb (e.g. `mod tests > fn test_task_contexts`
should be equal to ZED_SYMBOL of `test_task_contexts`). Note that this
isn't necessarily a test function or a function at all.
Also, you can use them in `cwd` field of definitions (note though that
we're using https://docs.rs/subst/latest/subst/#features for that, so
don't expect a full shell functionality to work); the syntax should
match up with your typical Unix shell.
Release Notes:
- Added task contexts, which are additional environment variables set by
Zed for task execution; task content is dependent on the state of the
editor at the time the task is spawned.
---------
Co-authored-by: Anthony <anthonyeid7@protonmail.com>
This PR adds **internal** ability to run arbitrary language servers via
WebAssembly extensions. The functionality isn't exposed yet - we're just
landing this in this early state because there have been a lot of
changes to the `LspAdapter` trait, and other language server logic.
## Next steps
* Currently, wasm extensions can only define how to *install* and run a
language server, they can't yet implement the other LSP adapter methods,
such as formatting completion labels and workspace symbols.
* We don't have an automatic way to install or develop these types of
extensions
* We don't have a way to package these types of extensions in our
extensions repo, to make them available via our extensions API.
* The Rust extension API crate, `zed-extension-api` has not yet been
published to crates.io, because we still consider the API a work in
progress.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Nathan <nathan@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
We couldn't reproduce the panic, but I believe it was possible when
uninstalling an extension while one if its grammars was still loading.
Release Notes:
- Fixed a crash that could happen when uninstalling a language extension
while its grammar was loading.
---------
Co-authored-by: Conrad <conrad@zed.dev>
After a lot of back-and-forth, this is a small attempt to implement
solutions (1) and (3) in
https://github.com/zed-industries/zed/issues/7902. The goal is to have a
minimal change that helps users get started with Zed, until we have
extensions ready.
Release Notes:
- Added detection of user-installed `gopls` to Go language server
adapter. If a user has `gopls` in `$PATH` when opening a worktree, it
will be used.
- Added detection of user-installed `zls` to Zig language server
adapter. If a user has `zls` in `$PATH` when opening a worktree, it will
be used.
Example:
I don't have `go` installed globally, but I do have `gopls`:
```
~ $ which go
go not found
~ $ which gopls
/Users/thorstenball/code/go/bin/gopls
```
But I do have `go` in a project's directory:
```
~/tmp/go-testing φ which go
/Users/thorstenball/.local/share/mise/installs/go/1.21.5/go/bin/go
~/tmp/go-testing φ which gopls
/Users/thorstenball/code/go/bin/gopls
```
With current Zed when I run `zed ~/tmp/go-testing`, I'd get the dreaded
error:
But with the changes in this PR, it works:
```
[2024-02-23T11:14:42+01:00 INFO language::language_registry] starting language server "gopls", path: "/Users/thorstenball/tmp/go-testing", id: 1
[2024-02-23T11:14:42+01:00 INFO language::language_registry] found user-installed language server for Go. path: "/Users/thorstenball/code/go/bin/gopls", arguments: ["-mode=stdio"]
[2024-02-23T11:14:42+01:00 INFO lsp] starting language server. binary path: "/Users/thorstenball/code/go/bin/gopls", working directory: "/Users/thorstenball/tmp/go-testing", args: ["-mode=stdio"]
```
---------
Co-authored-by: Antonio <antonio@zed.dev>
This is a refactor, follow-up to the work we've been doing on loading
WASM language extensions.
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
